Mona Juntus

Maybe, but also happens in vanilla-only single player worlds.
IMO it's just a matter of world time.

How long does it take to happen in SSP?

I've seen this bug occur in Singleplayer in 1.8.9, and it seems to be related to the removal of mods, as it has occurred each time I've removed a mod (any mod, so far it does not seem to matter which) from the mod folder and loaded up a save that had been in use when the mod was still in, even if any relevant mod items/blocks had been removed.

It does not occur when adding mods, just when removing them.

I have previously only noticed it happen in multiplayer servers, as those tend to use mods on their end.

A thing of note, the bug affects banners from both Vanilla Minecraft and Thaumcraft 5.

Edit: I'd recommend to back up saves to zip files, and make at least two new worlds when testing mods out, as the bug may only affect the last two worlds that have been in use, even if they hadn't technically been in use with the mods installed.
I believe I've confirmed that beyond those two worlds, no other are affected.
If anyone else can confirm this, that would be helpful.

Edit 2: For SMP it appears that once a certain uptime value is reached, banners stop swaying until the uptime is reset in level.dat with an NBT editor.

@[Mod] CubeTheThird: Thanks, I'll keep it there in the future.

Confirmed for 16w03a.