Unlike all other mob loot, the nautilus drop isn't 100% with high looting ench sword.

When using high looting enchanted sword on any other mob with rare loot it will always drop it, but the nautilus’s shell is still rare to drop. See image of wither skele always dropping skulls but not shell from nautilus.