Naming item in anvils with special characters

When naming items in anvils with symbols not supported by Minecraft which shows as a generic EMSP symbol, when clicking the named item which is named '?', the item disappears until closing the GUI which throws the item out, named.

What I expected was normal anvil behaviour.