Ability to Break Blocks in Adventure & Spectator Mode

The Bug

In Minecraft 1.20.4, and many other versions, you are able to "break" blocks in adventure and spectator mode. I say "break" with quotation marks because it only affects a single block and after finishing breaking it, the mining animation resets and the block is not actually broken.

How to Reproduce

1. In survival mode, start breaking a block

2. While holding down the break block button, use F3+F4 to switch gamemodes to either adventure mode or spectator mode

3. While holding down the break block button on the previously broken block, the animation will play.

Likely Explanation

I know nothing about Minecraft's code, but I would assume that this bug stems from the ability of breaking blocks until letting go of the mouse button. When breaking the block, the player opens a GUI for gamemode switching, and, similarly to opening the menu screen, the game still thinks that the player is breaking the block, so when they start breaking it again in a new gamemode, the game lets them.

Notes

• The block does not break when the block's breaking progress is completed (animation resets)

• The animation, unlike in survival, stays at the same progress after letting go of the break block button/clicking on another block (does not reset unless in survival)

• This glitch works after switching back to another gamemode then back to spectator/adventure, even without clicking on the block again

• On switch to survival mode, block breaking progress resets as normal

• I have seen several reports on this, but none have been marked as fixed, so reporting with the addition of adventure mode in hopes of it being noticed by someone who is able to fix