Hi Bruce! I may not have been clear enough with the original bug report. I run Minefold and helped write most of the networking stack to run over 60,000 Minecraft servers. The server software is vanilla Minecraft. The bug is in the protocol (the language that the servers speak to clients with). Well, it's not really a bug, it's an inconsistency.
The "???" is because we don't know what host the ping packet is trying to connect to and therefore can't offer back accurate player counts.
Great to hear from you Nathan. However, I'm really disappointed that this is intentional and I strongly disagree with your reasoning. Firstly, let me outline some misconceptions you have. I'll then go onto explain some of the history, alternatives & benefits to virtual-hosting.
[media]"Strings ain't small". I scraped the first 10 pages of mcserverlist.net (script) and found that the average size of a hostname (including IP addresses and ports) is 17 characters. Assuming you want to encode them the typical Java way (1 byte length with UTF16 encoded chars) that would take on average 35 bytes. If you wanted to do the same but limit hostnames to ASCII characters (none of the servers in those top 10 pages had non-ASCII chars), it would only be 18 bytes. If you wanted to include the port that would be another 2 bytes.
"Packets aren't usually nice and whole". For broadband connections on Windows, the default MTU is 1500 bytes. No (practical) network interface has a MTU of less than 1,200 bytes. Even with the existing magic byte, the ping response will practically never hit this limit and therefore be nice and whole.
Pt 3, "Act normally". I'd propose that the hostname is always sent and always ignored by the official MC Server. It only means something if you're running a proxy (like us).
"Respond with the data ASAP". Our proxy keeps an in memory cache of player counts, it can respond very quickly.
"Quite some time has passed now". If our proxy isn't able to respond in about 30ms then we've failed.
Your worry that a server can't respond to a TCP request with a practical maximum of 300 bytes in 3s seem to be based on intuition rather than reason.
You mention that SRV based hosting is a viable option. We bring up servers dynamically: when nobody is playing the process isn't running. Even if the TTL was set very low, and every ISP honoured low TTLs, we just can't do this with DNS. We would have to start a server on every DNS lookup which would be very inefficient. We also dynamically adjust servers (either to move the processes around or at least hold the connection) and bill customers based on their connection to their connection to them. SRV records dictate what sort of infrastructure we should be running. It's good for traditional hosts (like Gameservers). Virtual-hosts lets us be free to experiment with more cutting-edge architectures.
EDIT: Removed misunderstanding about SRV records and slight troll.
The entire http internet is based on virtual hosting. It's a proven concept. Websites all around the world have proven reliability and performance by allowing an intermediary to act as an endpoint to the user. In a cloud environment where we bring machines are being brought up and down dynamically, buffer connections as resources are allocated etc. a virtual-hosts based proxy is a must.
In addition, the hostname and port are already included in the handshake request. It feels unusual to me that you should be so anti virtual-hosts based routing but still enable it elsewhere in the protocol.
The only work needed to make this change is to add the hostname and port in the ping-request, and then ignore it in the server. I understand that it might not seem like a massive win for Minecraft but it's a simple change that would make the 65,000 players of Minefold very happy.
Scienced!