Villagers not harvesting correctly

I've built a villager-powered crop farm. While the wheat and beetroot work perfectly, the carrot and potato ones seem to keep failing me.
I fill the villager's inventory with one of the crops (we'll say carrots to start with) and he goes off planting. While he may harvest occasionally, if I leave the area and then come back, he will stop harvesting and refuse to restart,
He does throw stuff to the central villager, but then will not harvest - but if i manually break a crop, the villager does race across to replant, so his inventory definitely has stuff in it.
The carrots and potatoes are all above the wheat and beetroot, and all levels only have one composter.
As per attached photo - wheat is yellow glass, beetroot red, carrots orange and potatoes brown.

https://drive.google.com/open?id=1ZdcNkuj2uwMhbL0dLZQO-HKknfqHW6KM
For copy of world. I'm guessing this might be a unique thing to my world, so any ideas welcome.

I do have a command block system for one-player sleep detection and skipping to day.