Page 2 of 4 FirstFirst 1 2 3 4 LastLast
Results 21 to 40 of 66

Thread: VBulletin 4 Upgrade Thread

  1. #21
    Squid's Avatar Opifex
    Artifex Technical Staff

    Join Date
    Feb 2007
    Location
    Frozen waste lands of the north
    Posts
    16,999
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    OK so the thread keywords have been successfully rebuilt, so I'm going to post an announcement and take down the site for a couple of minutes to upgrade the downloads plugin and get that back online. After that most of the major functionality will have been restored.

    Under the patronage of Roman_Man#3, Patron of Ishan
    Click for my tools and tutorials
    "Two things are infinite: the universe and human stupidity; and I'm not sure about the universe." -----Albert Einstein

  2. #22
    Squid's Avatar Opifex
    Artifex Technical Staff

    Join Date
    Feb 2007
    Location
    Frozen waste lands of the north
    Posts
    16,999
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Needless to say that did not go according to plan. I will provide more details in a little while.

    Under the patronage of Roman_Man#3, Patron of Ishan
    Click for my tools and tutorials
    "Two things are infinite: the universe and human stupidity; and I'm not sure about the universe." -----Albert Einstein

  3. #23
    Squid's Avatar Opifex
    Artifex Technical Staff

    Join Date
    Feb 2007
    Location
    Frozen waste lands of the north
    Posts
    16,999
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    OK, so the upgrade crapped out in the middle of a table rename, which had all sorts of interesting ramifications, instead of having renamed some of the tables to their new names, the tables had been copied but not properly, so I needed to clean up the db and fix the script before finishing the upgrade, this took a fair bit of time. Downloads should be available to everyone on a read-only basis at the moment, in other words uploads are currently not available.

    Under the patronage of Roman_Man#3, Patron of Ishan
    Click for my tools and tutorials
    "Two things are infinite: the universe and human stupidity; and I'm not sure about the universe." -----Albert Einstein

  4. #24
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Did you get the script updated, and when do you think you will be taking another shot at this? I need to do some database clobbering of my own to get Sphinx up and running so I need to plan around the stuff you need to do.

  5. #25
    Squid's Avatar Opifex
    Artifex Technical Staff

    Join Date
    Feb 2007
    Location
    Frozen waste lands of the north
    Posts
    16,999
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    I fixed the script and the upgrade of downloads ran through to the end. Just a bunch of new setting I need to set in order to get uploads up again, I just haven't had time since then. Go ahead and clobber the db to get search working again.

    Under the patronage of Roman_Man#3, Patron of Ishan
    Click for my tools and tutorials
    "Two things are infinite: the universe and human stupidity; and I'm not sure about the universe." -----Albert Einstein

  6. #26
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    I will do that after dinner tonight.

  7. #27
    Augustus Lucifer's Avatar Life = Like a beanstalk
    Citizen Vault Staff

    Join Date
    Aug 2006
    Location
    Mrs Robinson's
    Posts
    10,493
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    So when can I clobber the DB to run 100,000 instances of vB table Sudoku?

  8. #28
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Th beta of sphinx is built against CentOS and threw hundreds of warnings when I compiled it. It did eventually compile but it will not run. I am going to reindex the default search function which will take quite a bit of time and then see if I can figure out what is going on with sphinx when we move to mjollnir.

  9. #29
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Actually I just found some stuff on this that I will try tomorrow.

  10. #30
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    How would you guys feel about implementing Google search on the forums as opposed to Sphinx?

  11. #31
    Squid's Avatar Opifex
    Artifex Technical Staff

    Join Date
    Feb 2007
    Location
    Frozen waste lands of the north
    Posts
    16,999
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Only problem with it is it couldn't be used to search private forums since google can't crawl there.

    Under the patronage of Roman_Man#3, Patron of Ishan
    Click for my tools and tutorials
    "Two things are infinite: the universe and human stupidity; and I'm not sure about the universe." -----Albert Einstein

  12. #32
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Yeah that would be a problem. However I did find two options other than the beta plugin that is on the vBulletin forums.

    This one is a paid package for $150. I am not exactly fond of that idea, but supposedly it does work well
    https://marketplace.digitalpoint.com...tin-4.870/item


    And I also found a link to this in the vBulletin 3 forums with a small note that it had been updated to run with vb4. I will be trying this on Pandora first.
    https://github.com/rcdesign/vb-sphinx_search

  13. #33
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Sphinx 1.11 was never officially released, and some idiot built this tool to only run on that version and never bothered to upgrade it to something else.
    **Attention!** This software **requires sphinx 1.11**. It's currently available only from SVN repository [[http://code.google.com/p/sphinxsearch/source/checkout]].

    The link listed there is down for maintenance "for up to 2 hours" though I first looked at it well over 2 hours ago. The good news is that someone took the standard vBulletin search rebuild tool and turned it into a standalone script you can run from the command line, which will be much faster to run since it isnt trying to refresh a web page every 2 seconds before it starts the next cycle. I grabbed that and threw it in /usr/local/sbin and am running it now. Its about 8% done on the post table. You can do all tables or choose specific ones, dump or retain the current indices, set how many to grab at a time, etc. Its currently processing 10,000 posts per cycle (default setting for vBulletin is 250) and isnt completely killing the site while it runs.

    The command to run it is:
    sudo php5 /usr/local/sbin/searchindex.php

  14. #34
    Augustus Lucifer's Avatar Life = Like a beanstalk
    Citizen Vault Staff

    Join Date
    Aug 2006
    Location
    Mrs Robinson's
    Posts
    10,493
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    I can't seem to look at all of the files in that eXtExplorer thing. When I view the "forums" directory, it cuts off after a certain number, and most of that is filled with log files, so everything after list.php is cutoff. Can't even seem to search in or for the missing files.

  15. #35
    Squid's Avatar Opifex
    Artifex Technical Staff

    Join Date
    Feb 2007
    Location
    Frozen waste lands of the north
    Posts
    16,999
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    At the bottom of that page is a page select mechanism which allows you to view the next page of entries for a directory.

    Under the patronage of Roman_Man#3, Patron of Ishan
    Click for my tools and tutorials
    "Two things are infinite: the universe and human stupidity; and I'm not sure about the universe." -----Albert Einstein

  16. #36
    Augustus Lucifer's Avatar Life = Like a beanstalk
    Citizen Vault Staff

    Join Date
    Aug 2006
    Location
    Mrs Robinson's
    Posts
    10,493
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    That worked to see the other files. Still having issues actually downloading or editing them, since the server seems to hang. Related to the CPU spikes issue?

  17. #37
    Squid's Avatar Opifex
    Artifex Technical Staff

    Join Date
    Feb 2007
    Location
    Frozen waste lands of the north
    Posts
    16,999
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Probably, although things appear to be a bit better since I added a line to lighttpd.conf.

    Under the patronage of Roman_Man#3, Patron of Ishan
    Click for my tools and tutorials
    "Two things are infinite: the universe and human stupidity; and I'm not sure about the universe." -----Albert Einstein

  18. #38
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Yeah I think we were getting banged pretty hard by that. I made a few other changes as well, some temporary.

    1. Knocked max_execution_time back to 30 seconds. I had it set really high because of the PM downloads issue. I knew we couldnt leave it that high for long. Basically it just lets stuff stack up in the queue if we have a bunch of stuff running for a long time and it just keeps spawning fast-cgi processes to deal with new requests.

    2. Lowered the amount of resources that lighttpd can use.

    3.. I broke search intentionally. Leave it broken for a while today please until I can look at some other stuff. Basically we know that the built in vb search function sucks, but I havent been able to move us to Sphinx for various reasons. So we rebuilt all the search indices and one of those was a full post index of 11.5 million posts or whatever it is we currently have. Operating on the assumption that the search function is eating a lot of resources I broke search and then took a look at the search options in the admincp.

    admincp > Settings > Options > Message Searching Options


    Minimum Word Length was 2, thats ridiculous. Its now 4 and I copied the sphinx stopwords into the list of words to be ignored on a search.

    Max Results was 5000, is now 500.

    Turned off wildcards.

    Minimum time between searches set to 30 seconds.

    We may need to turn on the Queue so that the search index isnt trying to rebuild for every single post. We can cronjob it to run every 10 minutes or so.


    As I am typing this loads are 14-15 and CPU use is down. Last week when we were on vb3 loads and CPU were less than they are now, which suggests that there is still something wrong with vbulletins forumcache. We may want to look at hacking the forumcache up for vb4. That one is a bit beyond me +but Dragooon was involved in that and he has been around a bit lately. Dragooon if you happen to read this let me know what you think on that.

    Lets leave search down for a while and see if we start piling up on load again. If we do then this obviously wasnt the problem, but the more I think about it we were not having these issues before we rebuilt the indices.

  19. #39
    GrnEyedDvl's Avatar Barackolypse Now
    Artifex Technical Staff

    Join Date
    Jan 2007
    Location
    Denver CO
    Posts
    22,809
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    Ok I think that was my fault. I installed xdebug and when I turned it on load jumped up to like 120!

    I might have gotten some good info out of it though, I need to dig through /tmp/xdebug for a while.

  20. #40
    Squid's Avatar Opifex
    Artifex Technical Staff

    Join Date
    Feb 2007
    Location
    Frozen waste lands of the north
    Posts
    16,999
    Tournaments Joined
    0
    Tournaments Won
    0

    Default Re: VBulletin 4 Upgrade Thread

    For the forum cache hack, dragoon left some instructions to follow to recreate it and that coupled with the git logs from the old forums it should be not unduly difficult to recreate.

    Under the patronage of Roman_Man#3, Patron of Ishan
    Click for my tools and tutorials
    "Two things are infinite: the universe and human stupidity; and I'm not sure about the universe." -----Albert Einstein

Page 2 of 4 FirstFirst 1 2 3 4 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •