Recipe categories disappear when server is sending new recipes

Recipe categories disappear when server is sending new recipes.  

The client is running vanilla 1.19.4 and the server is running a modified server version.  

The server is manually sending a packet that replaces all recipes, and if the inventory screen is open when that packet arrives the categories disappear on the left. When closing and opening the inventory again the categories are back.

EDIT: Turns out you can run the reload command on a vanilla client in singleplayer to achieve a similar effect.

I have tested this on 1.19.3, 1.19.4 and 1.20.4.

How to reproduce:

1. Place a commandblock with the reload command

2. Place a few repeaters and a button to trigger it on a delay

3. Run "/recipe give @s *" to get all recipes

4. Enter survival

5. Press the button and open the inventory

Expected results: The categories stay and does not disappear.

Observed results: When the client recieves new recipes the categories disappear in the recipe book. The categories reappear once the inventory is closed and opened again.