I am experiencing this bug on a realm in 1.20.10, and my platform is Windows 10. Please reopen this report since it is not fixed as far as I can tell.
A good work around that I have discovered is to place the jukebox on top of a single fence. Also make sure that the jukebox isn't touching any blocks. Since the fence is not a full block, the disc can still be retrieved if it happens to eject from the bottom of the jukebox. It looks kind of strange but I have run into no issues since I first started doing this.
Thank you! Sneaking and right clicking the allay with a name tag worked