When reverting 1.20 to 1.19.4 the blocks get messed up

Downgrading is not supported. Additionally invalid as it's modded.

Shaders is modded?
So if I use optifine on that server with shaders it's automatically invalid because it's "modded"

(Also another note, that server for the player who was asking for help downgraded to 1.19.4 and then back to 1.20 because of the visible bamboo stairs)

Optifine and shaders are mods, yes; still, if the issue is caused by downgrading, it's not supported. Any form of downgrading, even down and back up, are not supported, only upgrading.