Some faces of the world border are rendered in front of other faces.
To Reproduce
Make sure that Graphics is set to Fabulous! in Video Settings.
Use
/worldborder center ~ ~ /worldborder set 10
to set up the world border.
Move out of the world borders in spectator mode and observe.
Also happening for me, Nvidia GeForce GTX 1050 Ti and Intel Core i7-3770 @ 3.4 GHz