Mouse click position always lags a few frames behind the crosshair

Ugh. This is getting so annoying. Why do mouse clicks have to register using a mouse position from ages ago? If your mouse control is any good, you'll often move the mouse and then click at the right time in one motion. If you try to do that now, you'll usually hit at a past mouse position.

The one good thing is that the delay roughly counteracts the latency of my system. That means the click will register roughly where the crosshair was physically displayed on the monitor at the time the mouse button was physically clicked. But this property is very rarely helpful when it comes at the huge cost of the loss of coherence. Just don't go there.

I can not understand what you mean. I can not reproduce it.
It could be that you are out of attack range.

@unknown: It's still there in the release. I'm not out of attack range. In fact, if you watch the video where I missed a lot at first, you'll see all misses (except possibly one) were at a smaller distance than the eventual hit. The determining factor that made me hit was that I was leading a little with the crosshair. It's odd that the hitbox is effectively offset by about the width of a slime, no?

Maybe it only stands out when you have shooter skills. If you do, it's quite egregious. If you don't, that's why I made these videos to highlight it. You can frame-step through them if needed. You can also categorize your own footage by lead.

I should emphasize that it applies to all 3D aiming and clicking at about the same time; I've misplaced quite a few blocks due to this. This test is merely the least defendable one, chosen so knee-jerk excuses don't make sense.

Man, this is such a pain. I've been playing a little again lately. I just can't help clicking as soon as my crosshair is over the mob/block/chest/whatever I want to hit, meaning I'll often hit the thing my crosshair was over a moment before instead. Why can't such a simple thing work?

Duplicate of MC-62463, MC-63986, and MC-65439

MC-62463 is definitely not the same thing. No unloaded chunks or permanently unhittable mobs involved. If you and the mob just stop moving for a second or you manage to lead correctly, hits will register.

MC-63986 is reportedly 1.7.10. That version didn't suffer from quite as much lag as I reported here. My report also applies to blocks, although I can imagine people misdiagnosing this because they take more time to place blocks; this issue becomes most glaring during faster action.

MC-65439 is too vague. Who knows what's going on there.

There are most likely multiple hit detection issues. They're not all this specific one that I could analyze in some detail.

I'll see if I can update my reports to 1.8.5 soon-ish.

I'm noticing this a lot with block placement too since updating to 1.9. Clicking when the crosshair is over the edge of a block closer to you while you're moving the mouse towards an adjacent block will usually put it on the other block instead of the one I wanted, which doesn't happen nearly as much in past versions.

This is so much worse when you haven't played in a while and are no longer used to it. Still there in 1.9.4. How can this remain under the radar?

Please test in next snapshot.

Looks like you've fixed the pitch axis but not the yaw axis.

I must say this is hilarious. See the movie I just attached.

We've (at least, I have; I'm not sure about others) been using the technique described on MC-99604 to reproduce:

Easily reproducible by spinning around, looking at the ground, pausing the game and observing cursor position vs block outline position.

It seems that you can no longer reproduce it that way, but you can reproduce it with high-speed video.

I must say this is hilarious.

How so?

I'd never tried pausing before. I've always done it by looking (and possibly clicking) in real time. Frame by frame analysis is just for more detailed analysis and to convince others. Pausing does show a difference between 1.10.2 and 16w39c. In the snapshot the selection box snaps in place right as the pause menu comes up.

I just did some more checks. It turns out pitch was already/still instant all the way back in the original 1.8 release! (The oldest I can easily test because the launcher doesn't list snapshots of that era.) I could make the same kind of video there or in any intermediate release. I just hadn't noticed all along. And now returning to 16w39c, I do notice the delay is shorter. I hadn't noticed that jumping straight into the snapshot after not playing for a while. Just not quite gone as you can tell.

What's hilarious is that I end up clicking in a position I was never near, not just a position where I was a moment ago. It's just so blatant and incontrovertible. I should've seen that earlier. This is what happens when most movements are horizontal, and you don't have a reason to believe vertical movements are different. Bit of an egg of Columbus once someone notices.

TL;DR: Disregard my previous comment. Check whatever is different between pitch and yaw. Make yaw like pitch. That should fix the brunt of it.

I've been thinking a little about the big picture. One more thing is that mouse clicks and key presses are processed before mouse moves. Another is that clicks and presses are only processed per tick, with no particular synchronization to frames, currently having an opposite effect in the vertical direction. These don't result in such massive visual inconsistencies as described in this bug report, but do affect the overall feedback loop between player and game. If you can't synchronize on a finer level, biasing toward using newer mouse deltas relative to clicks (as with vertical movements at tick level) does seem to work better.

I guess it's because most actions consist of homing in geometrically as per Fitts's law, and clicking as soon as you're 'resting' on the target. Clicking and immediately moving away is less common. And really, in cases where you're trying to click on an object moving unpredictably (as happens in all kinds of games including Minecraft), move-then-click-ASAP is the only workable timing. Anything else artificially increases the reaction time of tracking. Games should adapt their order of operations to optimize this. Again to the extent that it's necessary to quantize: ideally all input devices would report at millisecond-level precision and games would use that information, even if it's in between frames or ticks or whatever.

I've had a look in MCP. The class called EntityLivingBase in MCP has a method called getLook. Before 1.8 it returned a vector based on rotationYaw and rotationPitch (and the prev* variants which I won't mention). In 1.8 there was apparently some refactoring. getLook was moved to the superclass, Entity. EntityLivingBase still implements its own getLook, doing the same thing but using rotationYawHead instead of rotationYaw (but still using rotationPitch). That change to rotationYawHead is the problem.

rotationYawHead appears to be the delayed/smoothed yaw used to draw the head model, running on ticks rather than frames and clearly lagging behind in F5 mode. Interestingly, there is no such thing as rotationPitchHead; rotationPitch is used to draw the head. Indeed the pitch of the head model tracks the crosshair's pitch perfectly, unlike yaw. This was the same before 1.8. The difference is that 1.8 started using that delayed direction to perform actions such as attacking and placing blocks.

Thanks for the info, will look into it

I'm having a hard time getting a good visual on reproducing the issue.

However I've done some changes that at least allow me to properly shop at a fence while running around it, so please check next snapshot 🙂

@unknown This might be a good visual representation: https://youtu.be/HTw87opulQg Taken from MC-99604

Grum said he's already shopping [sic] at a fence. Sounds like he's coded an aimbot to make up for his eye–hand coordination. So cheaty! If you're going to use an aimbot anyway, may I suggest testing it on iron bars or glass panes? Those are even thinner.

Some suggestions:

1. Set the frame rate limiter to 10 fps to see frames more clearly. Observe how in 16w39c but not 1.10.2, the selection box tends to lead your crosshair when walking around without moving the mouse. On some frames the crosshair isn't even in it! This is regardless of frame rate; just easier to see at 10 fps.

2. Use bright blocks without clutter to make the selection box stand out more. A sandstone plane is good. A standard world, not so much.

3. Slow the passage of time à la a tick speed mod. That should highlight tick-based shenanigans which are responsible for the brunt of the problem.

4. Draw a marker at the precise hit point. Like the selection box, but more prominent and not coarsely quantized to a nearby block. Should be easy to see that it's off for this particular problem when looking around.

I've had a closer look at the game loop. Here's a rough outline of relevant parts. This is based on MCP 9.30 (folder called mcp931 for some reason) for Minecraft 1.10 but likely valid for a large range of versions.

repeat for number of ticks (generally 0 or 1, but can be greater at low frame rate)
    update selection box (without interpolation; this is exactly on a tick so interpolation wouldn't even do anything)
    read and handle mouse clicks and key presses
    tick the world (causing players' rotationYawHead to be set to rotationYaw)
end repeat
read mouse delta and adjust player's direction
update selection box (this time with interpolation)
draw scene

Interpolation turns out not to do anything for rotationYaw and rotationPitch, because Entity's setAngles updates the prev* variables to be mathematically identical. (With rare and minimal rounding error; don't ask me why it's done the way it's done.) It does affect rotationYawHead. My guess is that Grum disabled interpolation for the final selection box update in 16w36c. That just kills the interpolation delay of the visual selection box. No good at all because it now jerks around on ticks and doesn't match the interpolated camera position. It doesn't do anything for actions (which are deferred until the next tick) at all.

I've had a little look at the use of rotationYawHead. It appears to exist primarily for mobs. That might make 1.8's implementation more correct overall depending on how they use getLook. If you want to fix players without changing mob behavior (for better or worse), you can override setAngles in EntityLivingBase (or one of its descendants used for players) to set rotationYawHead and prevRotationYawHead to the new rotationYaw after calling super.setAngles. setAngles is only ever called to update the direction the player is facing with mouse deltas (on frames!), so it won't affect any other entities.

I was also thinking of further improvements, but this is enough for now. This will bring it back to the former state which was at least acceptable.

In all of this I wonder what everything is called in the actual source rather than MCP. I guess an analysis based on MCP identifiers is still much more helpful than no analysis, haha!

Seems to not be (fully?) fixed in 16w40a. I clicked on a chest while moving the cursor to the top left really fast and I got the GUI while not looking at the chest. For better visibility the chest GUI is transparent in the resource pack.

That observation alone is not enough to confirm this bug - there's a short delay between opening a chest and the GUI appearing.

It's not apparent to me that 16w40a is different from 16w39c at all. I also reproduced the video on my first try.

Indeed the chest is no proof. This bug does affect it, but a necessary round trip to the server is added on top, even in singleplayer. That round trip doesn't cause misclicks so it's fine.

I so want this fixed. It's been here for years, actively irritating me all along no matter what I'm doing unlike most other bugs. The one mitigating factor is that I'm not playing regularly. Please just end it.

confirming it's not fixed in 16w40a

https://youtu.be/fFWpNegZX9s

Please reopen, this is not fixed.

I cannot replicate the behavior for myself, can you think of a way of showing the issue?

Here. Comparing all of 5 versions, including two of my suggested fixes. https://www.youtube.com/watch?v=dxr1ypzRLkY First fix: remove EntityLivingBase's getLook implementation. Second fix: override setAngles in EntityLivingBase to also set rotationYawHead and prevRotationYawHead.

You can use Cheat Engine to slow down a game to see it better. My computer sucks and I can't record it btw. Maybe some of you can?

Did as you wanted. javaw.exe is set to 10% speed and it's pretty clearly visible. But I wasn't able to clearly show a situation where the crosshair was already displayed away from the block, but I was still able to break it by clicking then. Maybe this is just a visual problem with the block marker.

And here I actually got the opposite: The block marker is before the crosshair. Created that without Cheat Engine, by flying fast in Creative mode. But this might be MC-5024.

If you slow it down, keep the frame rate way up. This is pretty much a frame-perfect slideshow, which doesn't reveal much to the untrained eye. I only suggested 10 fps to make it easier to pick out obviously wrong stills. At a tenth of normal speed there is no need for that; more frames in between the two updates per second will make it more glaring. It's not just a visual problem with the selection box. The box is exactly where the broken block is, just in a slightly different sense than it was before the purported fix.

The screenshot with the leading box is exactly what I described. At high speeds it starts leading overall because interpolation is applied to the player's position but no longer to the player position used to calculate the selection box. Also part of the purported fix. MC-5024 is irrelevant: that just describes how the hot spot of the crosshair is the top left of the intersection of its two lines.

Easy way to reproduce from MC-109254

Confirmed 16w43a myself.

As discussed earlier storage containers aren't a reliable test. There's an additional round trip to the server before the container UI pops up, separate from this bug. You've really moved your crosshair after the click as far as that is concerned. It can't cause misclicks like this bug.

-I think I've fixed the problem - see paragraph 4.-

I would disagree with using Cheat Engine as a viable testing tool. While its use is effective, the way speedhacking messes with the games clocking can severely bias results. For example, speedhacking to a low speed can cause the game to report extremely high frame rates, I've hit as high as 4000, but neither my CPU nor GPU are near powerful enough to get more than 200ish while on the nether roof with minimum settings and optifine, 4000 fps is out of the question under any circumstances. While the frame counting code is completely seperate from the raytrace/rotation code, I'd imagine frame counting isn't the only thing speedhacking interferes with.
As for the use of opening containers to test the bug, yes I am aware that this test includes the delay of waiting for the server to respond with an open inventory packet, BUT the client should be sending a block/inventory interact packet for the block the cursor is currently pointing at, not the block you were looking at one to two ticks in the past. While you could still move your mouse between the time you click and the time the inventory gui appears, the delay is very low when on singleplayer worlds and the bug is still present there. Same applies to attacking entities while spinning very fast. You should be able to attack the entity under your crosshair, no matter how long you've had your mouse still or how long you've been looking at the entity you are trying to attack. In close-quarters melee combat situations, where players are rapidly moving past each other, flicking sideways very quickly to hit the player moving past you will not work as the objectMouseOver will be lagging well behind where your cursor is pointing for a tick or two.
The biggest problem with testing this bug, since it occurs every frame/tick and only for a few frames/ticks, is that there will always be an element of human error. The problem is only obvious when quickly rotating, but rotating that quickly means you as a human have less time to see the effect - or lack thereof - of your actions on the world, and trying to pinpoint exactly where your cursor was at the instant the game registered your click is all the more difficult.

All this said, I think I have fixed the problem, or at the very least reduced it to being close enough to the behavior of 1.7.10 that it's almost unnoticeable.

This video shows a comparison between vanilla 1.10, vanilla 1.10 OptiFine HD Ultra C1, and modded OptiFine HD Ultra C1 (modded by me). While these do not cover all affected versions, it does demonstrate the bug, and I have tested and observed the bug in all vanilla major/minor updates from 1.8 to 1.10.2 and in snapshot 16w43a, and I am assuming that the bug has been present since the 1.8 snapshot that introduced split-threaded rendering/game ticking. The video also includes my process of ensuring the latest official versions of 1.10 and OptiFine, as well as my process for installing the modified version of OptiFine, to prove that it was not staged.

https://youtu.be/ibmXGcl8UfU

I created the mod using MCP 9.31 for Minecraft 1.10, and all code references use the de-obfuscation mappings included with MCP 9.31. I would include line number references, but seeing as I decompiled OptiFine, they would not match those present in vanilla. The modifications I made are as follows:

In net.minecraft.client.renderer.EntityRenderer

Within updateCameraAndRender(float, long), cut the entire code block starting with the line this.mc.mcProfiler.startSection("mouse"); and ending with the line marking the next occurrence of this.mc.mcProfiler.endSection(); (both lines included in the cut), then paste the code block in a new method public void mouseMovedRotatePlayer(boolean flag). In place of the code you just cut, call mouseMovedRotatePlayer(flag); (variable flag holds the value returned by Display.isActive();.

Within renderWorld(float, long), find the method call this.getMouseOver(partialTicks); and replace it with an if/else statement:

if (this.mc.getRenderViewEntity() == this.mc.thePlayer) {
    this.getMouseOver(1.0F);
} else {
    this.getMouseOver(partialTicks);
}

Within orientCamera(float), find the if code block starting with

if (!this.mc.gameSettings.debugCamEnable)

Add these lines directly before that code block

float partTickTemp = partialTicks;
if (entity == mc.thePlayer) {
    partialTicks = 1.0F;
}

and add this line directly after that code block

partialTicks = partTickTemp;

If you have source code including OptiFine, the not debugCamEnable conditional will be present within another if block, and a second time after the if block as an else-if. In this case, add the above code above/below the entire if/else-if block.

In net.minecraft.client.Minecraft

WIthin runTickMouse(), add the line

this.entityRenderer.mouseMovedRotatePlayer(Display.isActive());

without any condition.

If allowed, I could upload a .zip with the modified .class files and instructions on how to install them and the version of optifine I used so others can test the fix and/or look for bugs introduced by the fix.

Oh dear. This is a lot to debunk. I'm shortening quotes to keep it manageable. Read the original for full context. I also suggest that people read more of the comments. Everything gets reiterated all the time as soon as it's no longer in the latest 5.

I would disagree with using Cheat Engine as a viable testing tool. While its use is effective, the way speedhacking messes with the games clocking can severely bias results. For example, speedhacking to a low speed can cause the game to report extremely high frame rates, I've hit as high as 4000, […]

I'm not familiar with the way it slows the clock down. If it's continuous it should be fine. Of course you're going to see high reported frame rates because time as experienced by everything including fps code is slowed down. On my Mac I don't have a ready-made solution to do this kind of thing. Oddly enough, editing all System.currentTimeMillis and System.nanoTime calls (dividing the result by 10) in MCP didn't have any effect. I'm not sure what went wrong.

As for the use of opening containers to test the bug, yes I am aware that this test includes the delay of waiting for the server to respond with an open inventory packet, BUT the client should be sending a block/inventory interact packet for the block the cursor is currently pointing at, not the block you were looking at one to two ticks in the past.

This bug is to the order of a single tick, the same as containers on the integrated server. The container test has sensitivity but no specificity, making it next to useless.

In close-quarters melee combat situations, where players are rapidly moving past each other, flicking sideways very quickly to hit the player moving past you will not work as the objectMouseOver will be lagging well behind where your cursor is pointing for a tick or two.

One.

The biggest problem with testing this bug, since it occurs every frame/tick and only for a few frames/ticks, is that there will always be an element of human error.

There's never a moment that it doesn't occur. The effects just show up at a timescale that's slightly short. I'll bet most people who are any good with the mouse can feel the difference, though. What's more difficult is to reason about it and rule out human error. I've done all of it.

All this said, I think I have fixed the problem, or at the very least reduced it to being close enough to the behavior of 1.7.10 that it's almost unnoticeable.

I'll have a look at it and its ramifications later. Note that I have already provided some concise fixes, attacking the change in 1.8 snapshots at its core. Your patch seems very fragile.

I am assuming that the bug has been present since the 1.8 snapshot that introduced split-threaded rendering/game ticking.

Entities were wrongly refactored in a way that's orthogonal to the threading. It does appear to be coincident.

While the fix I mentioned eliminates most, maybe nearly all of the problem, I should clarify that it is not a 100% solution that doesn't present any bugs of its own.

When I first decompiled to try and fix the bug myself (maybe a week ago), the first thing I noticed was that while all the render code was factoring in the partialTick time to interpolate entity movement, camera rotation, and targetted block/entity between ticks, the mouse's movement was being checked and the player's camera rotation was being changed every frame, rather than every tick (this is why I assume the bug has been present since the snapshot introducing multithreading).

My first instinct was to do as was suggested in one of the duplicates of this bug report and just change getMouseOver(partialTicks) to getMouseOver(1.0F), which helped very slightly but didn't do much of anything to fix the problem. Seeing as the bug appeared to be caused by the render code expecting player rotation changes only on ticks, I undid my edit to getMouseOver and cut the block of code handling mouse movement and player rotation out of the render code and pasted it in the tick code (at the beginning of Minecraft.runTickMouse()). This fixed the problem entirely, but introduced new problems: input lag (because mouse position is checked less often) and jittery camera rotation, especially while moving.

I now saw why the mouse movement handling had been in the render code (to smooth mouse movement and decrease input lag), so I cut the code out of Minecraft.runTickMouse() and pasted it in a new public method that is called from both Minecraft.runTickMouse() and EntityRenderer.updateCameraAndRender(), once again changing getMouseOver(partialTicks) to getMouseOver(1.0F). Problem almost solved, but then I remembered the camera orientation itself relied on the player's rotation and added the lines temporarily setting partialTicks to 1.0F during the gl calls to rotate the camera. That left me with the 1.7.10-like behavior seen in the video in my previous comment.

After running those changes for a while I noticed it was causing problems with jittery camera rotation while spectating others. This is why I added the conditionals to set partialTicks to 1.0F, because while my mouse movement is updated every frame, the rotation of an entity being spectated is updated every tick, so partialTicks could not be ignored while the player was not the renderViewEntity.

After playing with these final modifications for the past day, the bug seems fixed, or at least restored to 1.7.10 behavior and the only side effect I've noticed is jumpy yaw rotation when rotating a boat with A or D while riding it (I haven't looked at the code yet, but can almost guarantee this is because the boat updates player rotation every tick, while the edits I've made expect rotation changes every frame, ignoring the need to interpolate rotation while riding a boat). While adding a check for the player riding a boat to the conditional for setting partialTicks to 1.0F might fix this problem, it could also re-introduce the bug specifically when the player is riding a boat, though I have not attempted to test this yet. As far as I know this only happens on boats, but it likely happens with anything that change's player rotation directly on a per-tick basis (and I'm not sure of anything else in the vanilla game that does this).

On an entirely separate note, as I've been thinking about how to fix the boat issue, I came up with another idea. The problem seems to stem from the mouse movement directly changing the player's rotation every frame rather than every tick, and the rest of the code only expecting rotation changes during a tick. For entities whose movements can be controlled by the mouse (afaik only EntityPlayerSP), create two new float variables - rotationPitchSinceTick and rotationYawSinceTick - and modify those every frame rather than directly altering the player's rotation. They would be factored into EntityRenderer.orientCamera(float) and EntityRenderer.getMouseOver(float) (and possibly Entity.getLook(), if a new) so that the objectMouseOver and camera rotation follow the mouse movement properly. Each tick they would be added to the player's actual rotation and reset to 0. Theoretically, this would remove the conflict between the code that changes the player's rotation every tick and the code that handles mouse movement and changes the player's rotation every frame. It would also be much less likely to break if/when the code is refactored in the future. These changes would be an alternative to the edits I suggested earlier, though It is also possible that it would require (at least in part) the some of the edits I suggested earlier (particularly the handling of mouse movement both in-tick and before rendering the next frame).

I tried applying your fix. Got a compile error because there's no `partialTicks` in `public void mouseMovedRotatePlayer(boolean flag)`. Adding it as an argument and filling in `1.0F` from both call sites didn't have the desired effect, nor did `partialTicks` from `updateCameraAndRender`. The selection box was now lagging in a different way. At least I didn't manage to misplace a block the way I did in my recent demonstrations.

The thing is, this kind of patch is basically litter. It's hard to analyze the effects when you're all over an unfamiliar codebase without explaining the high-level effects (or even knowing?). You've already found side effects yourself. It isn't a full fix, either. For proper coding you have to get down to the core, not massage things until something kind of happens. I appreciate the effort, though.

Go back in the comments a bit for an analysis that actually gets down to the core and elaborates on the surrounding structure.

My first instinct was to do as was suggested in one of the duplicates of this bug report and just change `getMouseOver(partialTicks)` to `getMouseOver(1.0F)`, which helped very slightly but didn't do much of anything to fix the problem.

That's the behavior of Grum's 'fix' which doesn't do anything useful for reasons I gave earlier.

Wow I am incredibly blind. I read through most of the comments but missed both of yours.

That change to rotationYawHead is the problem.

I copied Entity.getLook(float) to EntityPlayer to override EntityLivingBase.getLook(float) and it is more consistent than my fix by far (like 1.7 as you had hinted at) what problems did you see with this/what reasons could keep this from being included in the snapshot?

Also, Marcono1234 (or any other moderator/helper), replace the "code analysis" link to my comment with jonathan2520's comment on 05/Oct/16; it does a much better job at pinpointing the issue than I have.

Difference between my original modification and the modification based on jonathan2520's analysis: https://youtu.be/_zgG9PDNfWk

what problems did you see with this/what reasons could keep this from being included in the snapshot?

No problems, per se. More of a precaution that mobs can turn their head (rotationYawHead) independently. There would be some side effects there, but then both getLook versions have been in the game for years each. I think the current behavior is generally better, going by the way it's used in the code, making my second or third suggested fix most suitable. (Third is to re-override in EntityPlayer as you have done now, which I had only mentioned in my YouTube video description for some reason.)

Also, Marcono1234 (or any other moderator/helper), replace the "code analysis" link to my comment with jonathan2520's comment on 05/Oct/16; it does a much better job at pinpointing the issue than I have.

This being my report, I could do it myself. I'd just rather convince you than stomp on you. Maybe I'll move some information there.

You can no longer see the blue line when the hitboxes are drawn, that pretty much means it is 100% fixed now! 😃

That's F3+B combined with F5, for the uninitiated. Sounds like a fix. I'm looking forward to testing the next snapshot when it's out and hopefully confirming that it's fixed.

1.11-pre1 (and now the release) is mostly correct! I'm still marking it because disabled interpolation from a previous purported fix is still there. The thing where the selection box tends to lead you when you sprint. In MCP parlance, renderWorld needs getMouseOver(partialTicks), not getMouseOver(1.0F). As I mentioned before, interpolation conveniently doesn't do anything for angles updated by setAngles, so once you're using those it doesn't incur any mouse look delay.

What's wrong? I think it's fixed now.

Let this be fixed please ;D

Hard to verify it IS fixed, looks better but meh 😞

Can confirm that this seems to be fixed from my testing with a 10x slowdown factor.

I agree it's fixed in 16w50a. Well, as good as it was before 1.8 when problems weren't so blatant. My remarks about further improvements still stand. But let's call it fixed for now. Thanks for sticking with it, Grum!