How to reproduce it
Give yourself 2 items with an attribute modifier. Make sure that they share the same id, and are applied when the item is in any slot.
Place one of them in the main hand, and the other in the off hand.
Remove only one of them from one of the hands.
Expected result:
Since the player still have an item with attribute modifier, the modifier should still exist.
Actual result:
The game notices that one attribute modifier with that id was removed, so it removes the modifier, ignoring the presence of the item in the other hand.
This is already being tracked in MC-272769