I have also experienced this issue.
If a copper chest is being constantly searched, it freezes any copper golem attempting to reach it, therefore slowing it down tremendously. It won’t search for any other copper chests in the meantime which makes that copper golem useless.
I’m only using 4-5 copper golems in that hole, and one copper golem outside of it.
This is definitely something that survival players would notice.
Note:
This bug exists starting in 1.21.5 up to and including the latest version and latest snapshot. 1.21.4 does not have this issue.
This is in 1.21.9-rc1 for clarification.