I noticed that when a villager harvested crops, he would trample the farmland turning it back into dirt.
Steps to Reproduce: (This can be reproduced in creative)
Make an area of hydrated farmland and plant seeds
Spawn a farmer villager
Bonemeal the crops
As the villager harvests the crops, notice the soil turning back to dirt occasionally
I can confirm