Farmer Villagers don't harvest crops

I created a brand new world and tried to get Brown-Robed Villagers to harvest and replant crops, but I can't get it to work at all. I've set up villages for them and everything. You can see my set up below. The Villager does use the doors and goes in and out of the houses.