The session directory the game reaches out to returns the session and you can join from the xbox app on windows without issue.
Doing some more testing, this seems to be fixable if you delay sending the response packet from the server by a second.
This issue has become worse in 1.16.100, the menu now doesn't show at all.
I have edited the issue to follow the format and contain a bit more information, thanks for the response.
It looks like this was resolved in some version around 1.21.100