Smooth lighting / block shading does not work properly in certain cases

Let's dig into this a little bit because I think this issue's title is a poor descriptor of the problem and I don't want future bug reports to be wrongly labeled as duplicates of this issue.

This is really an issue with ambient occlusion in smooth lighting, as it only occurs with smooth lighting on (note that minimum and maximum smooth lighting makes no difference, as per https://bugs.mojang.com/browse/MC-58668) and it is unrelated to actual in-game torch or sun light levels. We can see it in full effect if we use an all white texture and night vision:

Some parts are so jarringly un-smooth that it appears that there are some gray blocks, even though there is only 100% white blocks used in the scene.

Contrast with this scene in which wide (2x2) tunnels are used and the ambient occlusion is correct:

(one might say that you can see some subtle lines here that don't make it "truly" smooth, but this is just a side effect of the linear interpolation, and is unnoticeable once a proper texture is applied.)

I think a better title for this issue would be "Tight spaces break smoothness of smooth lighting's ambient occlusion effect"

Still an issue in 1.14.4 and 19w42a

Still present in 1.15 Pre-release 3

The fabric API alone, seems to fix this somehow. Just thought I would share that here, it might be useful,,,?

Confirmed for all 20w snapshots, including up to 20w09a.

Can confirm in 23w17a.

Still can confirm in 23w18a

Affects 1.20.1 to 1.20.2-pre2

Can confirm in 23w51b.

Can confirm in 1.21.5