Vibrant Visuals FPS decrease when looking at water in Favor Visuals mode and FPS decrease when I open up any Xbox Menu or get any notification in Favor Performance mode

I recently tested the Vibrant Visuals feature on my Xbox Series X and noticed that my fps degrades when I look at water while the preset is set to Favor Visuals mode. I changed it to Favor Performance mode and that seemed to fix the issue but then when I open up any Xbox menu or get any notification, the fps degrades again.