Low fps when using vibrant visuals while in a xbox party

(Im on an xbox series s) Whenever I'm in a world using vibrant visuals and in a xbox party, I constantly have very low fps and performance. I'm using the lowest settings for vibrant visuals as well. This also occurs whenever I'm using the xbox side menu. I've also tried this on a newly created world, and it still has the same performance issues. I'm not sure if its just the Series S that has this issue or the Series X also has it.