Beta Testing: An Introduction

[Roman_Man#3]

Hey everybody. I will be organizing the beta tester team, so if you guys have any questions, feel free to ask me.

There is just one thing I'd like to emphasize before we begin. Releasing any material or information without the teams consent will be dealt with harshly. Thank you.

And now, onto beta testing. There is really 3 types of testing. Battle Testing, Graphic/Interface Testing, and Strat Play Testing. There are guidelines for each one set by DVK that will allow him and the rest of the coders to be able to exactly pinpoint any problems you report.

You will each have your own beta testing thread. Look at King Nud's Report thread for an idea of what it should be.

Now, when making a report, it would help a lot if you could put a Header on your post, saying whether the report is for Battles, Graphics, or the Strat Map. Just say the type of report in bold at the beginning of the post.

Now, onto the guidelines:

[Roman_Man#3]

CUSTOM BATTLES:

1. The custom battles in RS2 are fully functional for many factions, whether there are unit cards or not. Custom Battles can eliminate problems with units before a player ever gets to the Campaign, so it is a Beta Testers job to pound the crap out of Custom battles for every funtional faction.

2. Write down what units you recruit to play, and play against.

3. If you get a CTD in a Custom battle and there is no error (which assumes you, as a Beta Tester always play with -show_err enabled in your RS shortcut), there is a specific procedure for finding the culprit:

a. Load the same faction you were playing, and the exact same units you used before, and then load only one known good unit from another faction. ALL of Carthage's units are perfect, so I always use one of theirs. This eliminates any possibility of two units causing the trouble.
b. If it CTD's again, load one or two units at a time from the units you recruited, and test against the same Carthage or known good unit. No CTD, cross them off, and keep doing this until you either find the one (by process of elimination) that is causing the problem...or, you find nothing.
c. In that case, it was the units from the OTHER faction, which you wrote down.
d. Do the same thing with this faction, eliminating the good ones until you find the bad.

4. If you get an error message, report the error message exactly as displayed. I just write them down. Screen shots aren't necessary.

5. If you got an in-battle CTD during a Campaign, load the same units in Custom Battle and test them as above.

[Roman_Man#3]

GRAPHICAL or INTERFACE TESTING:

1. You should be testing everything in this mod. Opening things, clicking on them, reading the tool tips and reporting any misspellings or text that makes no sense or is just wrong. Read unit descriptions and look for 'garbage text'....the file is just huge and I could spend hours and hours checking every line for garbage machine code. But it needs to be done.

2. Building browser.....click and right click on all buildings. Bring up dialogues and descriptions. Read them and look for mistakes. If you get a CTD clicking on one, write down and report WHAT building did that. If you can't remember, restart and go back in there and do it again.

3. Characters, General's, Family Trees, etc......look, click, move, change, otherwise mess with and torture, looking for any CTD's (usually traits in this area). Remember what caused it, write down and report.

4. Banners, Icons, flags above cities.....obvious things that sometimes just don't noticed. Check them....we are going to be changing them all in the future, and the more 'eyes that look' the easier it will be to find any oversights.

I can't think of anything else, but feel free. Anyone have a passion for organizing this, because it is important.

[Roman_Man#3]

STRAT PLAY:

1. Money. How much money is a faction making. Is it excessive? Is it enough? Is it too frustrating to get ahead? Do buildings cost too much? Too little? Some of this is 'subjective' obviously, so it isn't up to the beta tester to 'decide on his own' it's OK. It's part of the report. The faction makes xxxxx denarii per turn, it's steady, it builds up, it isn't enough. These are the kinds of things that allow fine tuning and adjustments. They build 'overall statistics' that can be looked at in an overview sorta way and allow better corrections.

2. The look and feel of things. Do cities look realistically built up, or insufficiently built. Is the starting position, configuration of a faction too easy, too hard, just right? If a LOT of people report this sort of thing, you can get a statistic of some kind based on player abilities or known preferrences.

3. CTD's of any kind. This is very frustrating. THAT you get a CTD is of no importance whatsoever. WHAT caused the CTD is the important thing. So a CTD needs to be reported in an orderly fashion:

a. Name of city and region where the CTD happened if it's on the strat map.
b. Exact composition of all armies involved.
c. Name of faction attacking.
d. Name of faction being attacked.
e. Name of the winner.
f. If for a city\town...occupy, enslave, or exterminate does not matter. The CTD's are caused by the units and traits when cities are involved. (or a rare RTW bug that is just...oh well)

4. You need to save often...and before every battle. Why? So if you get a CTD in one, you can replay it and find out what armies were involved, and EXACTLY what units were in them. Then:

a. Name of both faction's involved and what units they had.
b. Names of any 'named characters' involved so that if the saved game has to be anaylsed, we know what to look for.
c. Look for any strange behavior or 'look' in the units. An animation that has gone screwy in a battle CAN cause a CTD...I've seen it happen, and it can't be fixed. And more than likely, it can't be repeated, because it's a wandering and rare occurrance.
d. Knowing what units were involved in that battle leads to the next step.

[tone]

CUSTOM BATTLES:

a. Load the same faction you were playing, and the exact same units you used before, and then load only one known good unit from another faction. ALL of Carthage's units are perfect, so I always use one of theirs. This eliminates any possibility of two units causing the trouble.
b. If it CTD's again, load one or two units at a time from the units you recruited, and test against the same Carthage or known good unit. No CTD, cross them off, and keep doing this until you either find the one (by process of elimination) that is causing the problem...or, you find nothing.
c. In that case, it was the units from the OTHER faction, which you wrote down.
d. Do the same thing with this faction, eliminating the good ones until you find the bad.

Just a word of caution on this.

If the issue is a sprite issue it'll probably play absolutely fine with the same units the second time as for some reason it seems able to ignore missing sprites second time round. If, after following the procedure aove, you are no wiser, particularly if there was no error message, then play a custom battle with a few units on each side you know for sure are working and have never had a crash when you played with them. This "flushes" out the memory.

Now go back to trying to find the fault.

You can basically otherwise follow the process Roman_Man described as long as you add this "flushing" step in between.