Same issue in xbox series x. Version 1.19.80. Cursor jumps to another slot in inventory menu, chest menu, any screen with a item slot. Noticed it the most when the cursor is to the edge of a item screen. It jumps randomly to the top to another item slot. Single player world with 2nd player. Issue occurs on both players 1 and 2.
This issue has been going on since version 1.19. The size of the structure does not matter as to the villager will sit on the opposite side of the wall next to a bed instead of moving a few blocks to the open door. Also the villagers will sometimes sleep next to an open bed on the floor