Originally Posted by
GrnEyedDvl
(...)
There is no "optimizing" vBulletin with a forum structure 1200 forums deep. The software has too many inherent design issues to continue down that path. Most specifically, the way the forumcache works with forum vs user permissions.
The permission structure of vBulletin is actually very, very granular, which allows us to do some of the things we do. Unfortunately that also means that every one of those permissions (both on forums and on users) has to be parsed every time a user reads a thread or enters a forum. And we have users that by default are part of 3-4 user groups. Staff and modders are members of more than that, then for every customization there is generally a user group tied to that. Because the permissions system is how you give people the extra eye candy, or take stuff away, or grant access to certain forums.
Every custom user title color, thats a user group.
Every badge, that's a user group.
Every moderation level (no rep, no D&D posting, no avatar, no sig, etc) is a user group.
Various suspension levels, those are user groups.
Every hosted mod Dev Forum is a user group.
Every Curia Award (Artifex, Patrician, Divus, etc) is a user group.
Emperor Commodus, you are a member of 3 groups. I am a member of 5. Halie Satanus is a member of 6. Leonardo is a member of 2. AnthoniusII is a member of 5. Mak is a member of 8. Some people are members of a dozen.
Gigantus is a member of 15. Biggest one I found with a quick look, since searching that isnt built in. I have to manually type someone's name in to look at it.
Anyways, there are about 300+ (am not going to count them) individual permissions in 35 categories/sections (I did count those) that can be assigned. Every single one of them has to be set, for every single user group. Even though most of them are the same across every single user group, some of them are not. And some of them may conflict. For instance one group CAN do something, but another CANNOT. So now the groups have a hierarchy, so your Primary group permission overrides the other, whichever option that is.
So for Commodus, the software has to query 900+ permissions, churn through them and apply them correctly, BEFORE it displays anything on your screen. So the custom query for that is a nightmare. Its actually several queries just for one group. At least 35, one for each category such as Blogs, Awards, Polls, Private Messages, etc. Some of the categories have multiple queries built in depending what you are doing. But we will go with 35 for now.
So every time you do ANYTHING on the site, Commodus is generating a minimum of 105 individual database queries just to establish his permissions. Gigantus is generating 525, at a minimum.
To get away from all those queries there is a function called the forumcache. Basically when you login it caches your permissions so it doesnt have to look them up every time. That's great in theory, until you add forum permissions in as well.
Every single forum can be assigned permissions based on every single user group. So it also caches 1200 forums x 35 queries per user group x 500 something user groups which is 20 million something queries, for you to view this thread. Yes, those are all cached when you login. But it has to actually READ that cache to assign your permissions. So its happening in RAM which is really fast, but its still a really big file and it takes time to read.
On top of that there is this crappy thing called Forum Masks which lets you assign an individual user specific permissions for every single forum no matter what group he is in. So take those 20 million queries and multiply it by every user. Those were used quite a bit back in the early days, because its a fast and easy way to grant someone permission into something. But its seriously overhead intensive. Its fine for a forum with a few hundred members, but it doesn't scale well at all. I believe I nuked every single one of those years ago
Back in 2008 Simetrical and I took a hard look at this with vBulletin 3. If you lok through the Tech forums you can see some of that stuff. Some is not public as it probably isn't a good idea to post some technical information like that in public. He actually rewrote part of the forumcache to make it more efficient, but there is a limit to what can be done. vBulletin 4 actually included some of his improvements, and added a few of their own. At that time the forumcache (per user) was something like 50 megs. Its probably more like 150 megs now. And it has to be parsed every single time you click.
That is the root of our problems. It always has been.
The biggest source of user groups and forum permissions is the Hosted Mod Dev Forums. Remember, this is exponential.
This is one reason most sites (or most workplaces) do NOT have insane amounts of permission groups. Aside from the manpower overhead to manage it, its a ton of processing power. And time. We have tons of groups with 1 or 3 members because of how the forums are setup.
What is particularly painful to me, is that because x number of idiots are currently in some form of moderation points we have to process a ton of info for their user groups. That is one reason why most places just ban your dumb ass when you misbehave, rather than spending a bunch of manpower and processing power dealing with you. Its small potatoes at the scale we are talking about here, since its 4-5 user groups, but it still bothers me.
So, add all that up and the bottom line is that ANY forum structure is going to work in a similar fashion. Some might be better at it than others, but you won't know that for certain until you spend some $ and actually convert. In my opinion, the forum structure is not sustainable the way we currently do things. On any platform.
(...)