Nintendo switch lags when joining friend’s world (not realm)

Every time I try to join a regular world (NOT REALM) with a friend the play lag is so bad that we are unable to play, build, etc.