Testing each Shadow Quality setting. I consistently find performance from best to worst goes: Low > Ultra > High > Medium. Medium being the worst performing mode.
Considering that Visuals Mode in Handheld uses Medium Shadow Quality, it would make sense to either change it to Ultra, or find whatever is causing Medium and High to run worse.
Attached Images show Medium running at 29.5fps, and Ultra running at 32.1fps, same settings otherwise.
Thank you for helping us improve Minecraft! We saved your files: