Remotely controlling the Minecraft client results in erratic mouse movement. (Spinning erratically while looking directly up.) This can easily be duplicated by testing with RDC.
I use Kavoom to control a second account running on a second machine, giving me seamless use of a single mouse and keyboard between two computers. This has worked flawlessly through version 1.12.2. However, now I am now unable to control movement in the snapshot versions on the second machine. The mouse works normally in menus, but not when controlling the character.
Linked issues
is duplicated by 3
Attachments
Comments 38
Massive comment inbound:
I decided to look over this more thoroughly.
1.12.2
In 1.12.2, the majority of this logic is in MouseHelper, which uses Mouse.setGrabbed. This method is implemented by LWJGL here, but the main logic is in implementation.grabMouse, for windows implemented here. That calls WindowsMouse.grab, which doesn't do anything interesting; however, the relative movement does happen in WindowsMouse.handleMouseMoved.
The logic that actually moves the cursor to the center of the screen is in WindowsDisplay.updateCursor. This handles displaying the cursor and also setting it to a transparent/hidden cursor. It also calls centerCursor, which in turn calls two methods. The second is current_display.setMousePosition (where current_display is this; I'm not sure why it's set up like that), which calls WindowsMouse.setPosition. That just handles some internal stuff. The more important one is nSetCursorPosition, which is a native method implemented in org_lwjgl_opengl_Display.c. That calls windows' SetCursorPos, which presumably does correctly handle RDP things.
1.13
LWJGL 3 doesn't have a general equivalent to Mouse.setGrabbed, other than with nuklear (which is not used by Minecraft). MouseHelper was completely rewritten to manage mouse positions itself (per forge's MCPConfig). The code now looks something like this (these names are my guesses based on previous MCP names for similar code):
public void setIngameFocus() {
if (this.minecraft.isDisplayActive()) {
if (!this.inGameHasFocus) {
if (!Minecraft.IS_RUNNING_ON_MAC) {
KeyBinding.updateKeyBindState();
}
this.inGameHasFocus = true;
this.mouseX = (double)(this.minecraft.mainWindow.getWidth() / 2);
this.mouseY = (double)(this.minecraft.mainWindow.getHeight() / 2);
GLFW.glfwSetCursorPos(this.minecraft.mainWindow.getHandle(), this.mouseX, this.mouseY);
GLFW.glfwSetInputMode(this.minecraft.mainWindow.getHandle(), 208897 /* GLFW_CURSOR */, 212995 /* GLFW_CURSOR_DISABLED */);
this.minecraft.displayGuiScreen((GuiScreen)null);
this.minecraft.leftClickCounter = 10000;
}
}
}
public void setIngameNotInFocus() {
if (this.inGameHasFocus) {
this.inGameHasFocus = false;
GLFW.glfwSetInputMode(this.minecraft.mainWindow.getHandle(), 208897 /* GLFW_CURSOR */, 212993 /* GLFW_CURSOR_NORMAL */);
this.mouseX = (double)(this.minecraft.mainWindow.getWidth() / 2);
this.mouseY = (double)(this.minecraft.mainWindow.getHeight() / 2);
GLFW.glfwSetCursorPos(this.minecraft.mainWindow.getHandle(), this.mouseX, this.mouseY);
}
}These methods are each called once to change the current state (and actually match the general structure of methods declared in Minecraft before). In any case, LWJGL3's GLFW component is a pretty thin wrapper around GLFW itself, so I won't bother to link to it directly. Per glfwSetInputMode's documentation, GLFW_CURSOR_DISABLED is basically the same thing as grabbed cursor, and glfwSetCursorPos behaves differently in that mode. So, the order of setting the position before disabling, but re-enabling before moving the cursor is correct, as part of the fix to MC-121705.
The implementation of glfwSetInputMode is here. Note the call to _glfwPlatformGetCursorPos(window, &window->virtualCursorPosX, &window->virtualCursorPosY). Those fields are declared here and are described as "virtual cursor position when cursor is disabled"; they're used in glfwGetCursorPos and in glfwSetCursorPos when in the disabled mode.
glfwSetInputMode calls the platform-depended _glfwPlatformSetCursorMode; for windows that is this. That in turn calls functions that enable and disable the cursor, which stores the cursor position and centers the cursor. It also registers a raw input device (using windows' RegisterRawInputDevices). Apart from that last step, this more or less matches LWJGL2.
The differences come in the way it handles mouse movement. The handler for WM_MOUSEMOVE is the equivalent to LWJGL2's system... but it doesn't run when in the disabled cursor mode. Instead the raw input handler, WM_INPUT, is used. It looks for changes in cursor position and computes the deltas, and then calls _glfwInputCursorPos with the previous position plus the delta. However, in no part of the process does it move the cursor back to the center; it just keeps it in the same position. I suspect that's the cause of the issue. ignore that, it does center it, hm.
Unfortunately I'm going to need to fix my RDP setup to test further... but this is a start at least.
I got an RDP setup working, and it seems like this is rather a case of RDP doing something weird; a poorly documented MOUSE_VIRTUAL_DESKTOP flag in the raw mouse input is set and it seems glfw doesn't process that (as it's not documented well). I've created GLFW issue #1276 for this and will be testing further later.
OK, I've fixed it on the GLFW end and submitted a pull request. Once that's merged and GLFW releases (and then LWJGL updates their GLFW version and Mojang updates their LWJGL version, which might take a while) this should be fixed ingame. I've also made a modified version JSON for 18w21b that uses the fixed version. To use that, you'd create a new folder in .minecraft/versions named 18w21b-glfwtest, and then save that file as 18w21b-glfwtest.json within that folder. Then you should be able to launch that version in the launcher. If you still experience issues with that, let me know.
@@unknown , I've tested your modified GLFW and it does indeed fix the issue with both RDP and KaVoom. Your efforts are stunningly impressive. I appreciate that this will take some time to filter it's way into the official release.
GLFW merged the change to fix RDP at https://github.com/glfw/glfw/pull/2431 (I'm pretty sure it's included in the GLFW 3.4 release). LWJGL hasn't yet updated to the new GLFW version as far as I can tell though, so Mojang can't update to a new LWJGL version to fix it just yet.
Still occurs on 1.21.
Luckily LWJGL updated GLFW in version 3.3.4, hopefully Mojang will update LWJGL to version 3.3.4 to fix this issue as soon as possible.
Oooh, I might be able to diagnose this a bit, since I've looked at RDP's internals a bit for my (now more or less abandoned) mcrdp project. Unfortunately I don't have a functioning setup at the moment, so I can't reproduce the issue myself.
Anyways, RDP has aPointer Position Update packet(§2.2.9.1.1.4.2), part of theServer Pointer Update PDU. And, MC got mouse input in 1.12.2 and below by continuously moving the mouse to the center and using offsets from the center to rotate (that's more specifically an LWJGL2 function). Thatshouldlead to that PDU being sent to realign the cursor (though, latency might lead to weird things). If that doesn't happen, though, then you'll just continually spin around (MC-81818 presents a similar case where the cursor isn't centered again, but without RDP).I'm not sure whether LWJGL3 still uses the move cursor to center system. If it still does, it might be that there's a bug that causes RDP to no longer pick up the movement and center it again. If not, it might be the case that it's detecting movement in a way RDP doesn't understand. I'll investigate what I can.(Update, that wasn't the cause and it does move it back to the center in a way RDP understands; see later comments for my current understanding of it)