Pillagers Spawn in Soul Sand with Nether Wart Growing

Basically, I have large netherwart farms above ground next to wheat and carrot and potato farms. Pillagers continue to spawn in spots that I have planted netherwart. Not sure if this is technically a bug, but no other mobs spawn there, and its really annoying when my crops are continually trampled by an otherwise protected area.