In permissions I gave a group the inability to join zero groups. But if they are invited they are able to join. Also even though I created group permissions to not create a group, someone can transfer ownership to a user that cannot create a group.
Note: After leaving the group and testing again, the same user cannot be added because the error they are already in the max number of groups. But it initially let's users get around the permissions.
I asked in the Xenforo discussion about having forums where users that aren't members of the social groups can post too, and Ozzy47 told me it's probably not possible.
I'd just like to recommend this to be an option for future updates, I know in most cases people won't use it for their forums but I think it's something pretty basic that should be possible to do easily, and I personally would like to have it in my forum.
Also if Snog or anyone knows how to do this in any way, please contact me.