Pillagers holding anything but a crossbow is friendly.

when a pillager is summoned holding anything but a crossbow, they will not do anything but stare at you and wander about. however when one with a crossbow is summoned it fires at you. if this is not intentional i would imagine it should run at you and punch you like skeletons do.
this was produced with the pillagers not on my team with the command /summon minecraft:pillager ~ ~ ~ {}. they still do not attack even when attacked themselves, and ignore villagers. I rate this as low priority because this is not likely to happen in the game without commands, however can potentially get in the way of some.