Results 1 to 16 of 16

Thread: The Beta testers handbook

  1. #1
    Halie Satanus's Avatar Emperor of ice cream
    Join Date
    Nov 2005
    Location
    London
    Posts
    19,995
    Blog Entries
    1

    Default The Beta testers handbook

    Foreword; Beta testing is an integral part of any mods progress, a mod team, no matter how big, can rarely spare the time to properly test every aspect of a mod. Hiring testers who can be trusted is a big step. For most teams this will be the first time they have passed their labors onto a complete stranger and they will hope, fingers crossed, they find people who will take the role seriously.

    The main factor is always time. The team will often be waiting for your feedback before they can progress so taking a two week holiday the day you DL a mod isn't going to go down to well. Bear in mind you may be new to the team but the modders themselves will have already put hundreds of hours into getting the mod to the beta stage. ect ect..

    1: Play the faction you are asked to play rather than the one you want play. The Mod team have taken you on because they need feedback on a specific faction and while playing as the big boys may be tempting balancing requires all factions are given equal attention.
    2: Do not tweak stats on the version your reporting on. Ok so maybe your Spartan generals are not the 'uber' killing machines you feel they should be. Tweaking them to your tastes won't give the team a true reflection of how they will preform in the final release.
    3: Don't give unauthorized people access to the beta. You may feel an overwhelming desire to share you beta with other members, but imagine if the beta falls into the hands of someone else who has an overwhelming desire to share, before you know it the beta is sitting on a thousand desk tops.
    4: Being honest; the team hope they have made a well balanced mod which looks great and rocks your world, they are in fact quietly confident this is the case. If not say so and why and, if you can, ways to make it better. Endless posts of ''oh my god you guys rock, I love you, this is the best thing since garlic bread was invented,'' though it may be a nice ego boost to some Isn't really that helpful.
    5: 50 turns syndrome; (unless they state otherwise) Chances are the mod team has tested the first 50 turns of the faction you've been asked to play a hundred times. They would probably like you to skip the 20 page summery of how you got out of the financial hole of the start position and, how you defeated ten local rebel cities.
    6: Dropping out; Sometimes going A.W.O.L. just can't be helped, please try and drop a PM to the team if you think you have to take off, it's not uncommon for a team to be waiting for a report that is never going to come which can be very frustrating.
    7: CTD's and reporting them. You almost certainly will come across a few CTD's when testing. Assuming the team have done some testing, the campaign loads up and all seems well. Common issues arise once characters start moving around and interacting with other elements on the map, ie; settlements or other characters. There are a few things you need to do here which will help the team bug hunt.

    A; Save your game after each turn.
    B; It's a good idea to keep a list of your armies units, especially those moving around the map.
    C; If attacked or attacking it's a good idea to make a list of the opposing army.
    8; Also, try to always provide screen shots. Saying "I saw snow in the desert" is a first step, providing the screen shot may provide visual information a text explanation does not.
    9; save game each year / turn - with different descriptive names for each like "sev-hh-y980" = to mean Severin campaign, on hard, hard at year 980 etc.... so you don't overwrite old ones and you can find relevant ones in-case someone asks you to look back... (this is general good practice, but not something I often managed to persuade most of testers to do, Halie was excellent beta tester and exceptionally funny) ..

    Ending up with one final save / autosave which proves that it won't load / advance to next year isn't always exceptionally useful, you may need to go back to look at prior saves if available....

    And finally, try everything. Don't just do the basic things everyone will do. Even if it's against your playing style. Things like trying to move anywhere around the map can help find mapping problems like blocked pass, rivers unable to be cross...

    See posts below for contributors to the OP' and other tips for beta testers.
    Last edited by Halie Satanus; January 18, 2010 at 06:15 AM.

  2. #2

    Default Re: The Beta testers handbook. WIP

    Don't give unauthorized people access to the beta.
    RIP Calvin, you won't be forgotten.

  3. #3
    Halie Satanus's Avatar Emperor of ice cream
    Join Date
    Nov 2005
    Location
    London
    Posts
    19,995
    Blog Entries
    1

    Default Re: The Beta testers handbook. WIP

    A'yup, good one.

    If anyone sees anywhere I can revise or add to the OP let me know. It would be nice to have specific sections for traits, balancing, economics, ect, what to look for when testing..
    Last edited by Halie Satanus; January 05, 2010 at 03:49 AM.

  4. #4

    Default Re: The Beta testers handbook. WIP

    check the animation of the units and if any unit unit's appearance or fighting is not the way it should, check the map for errors, if the settlements have troops inside, if some units are really strong while other's aren't (meaning a wrong balance maybe), spelling mistakes in description's or cities or units if some units create a CTD. that's what came to my mind now, i am sure there are more

    Macedonia was,is and will always be Hellenic.
    it is nice to be important, but it is more important to be nice.

  5. #5

    Default Re: The Beta testers handbook. WIP

    If for any reason you need to drop out, say so. I've had to do this myself, but it always helps to know who is committed and who isn't. There's no shame in having to leave, but dropping off the face of the earth annoys all concerned.
    Son of PW

  6. #6
    Halie Satanus's Avatar Emperor of ice cream
    Join Date
    Nov 2005
    Location
    London
    Posts
    19,995
    Blog Entries
    1

    Default Re: The Beta testers handbook. WIP

    @salaminomaxos79

    Yep, if we get more specifics I'll create a section for units, maps ect.

    @Theodotos I

    Added..

  7. #7
    Augustus Lucifer's Avatar Life = Like a beanstalk
    Patrician Citizen

    Join Date
    Aug 2006
    Location
    Mote of Dust
    Posts
    10,725

    Default Re: The Beta testers handbook. WIP

    Always, always, always attach log files whenever you come across a CTD or other issue that can be replicated. Most good teams will include in their pre-release builds a CFG that has some rather heavy log writing in it. You don't need to interpret this necessarily, but other people do.

    Another very important thing is reporting what turn you are on when things occur. 'I don't know' is useless. 'Early game' is fairly useless. 'Around Turn X' is marginally useful. 'Turn 20' is useful. I've created a handy script which is very simple and will help beta testers and modders alike know exactly which turn they're on(attached to this post, works for M2 would need to be adapted for Rome probably). For M2, copy its contents to the very end of your campaign_script.txt right before wait_monitors and end_script.

    Some notes about the script. I suggest that modders expand this script to also log turn start/turn end and export to log, so if an error occurs during the turn change it can be seen which faction turn it occurs in. I can't do this because every mod uses different internal faction names(well I could, but I don't want to write a program for something that simple right now). Another important thing to note is what we consider 'Turn 1' or the opening turn, is considered 'Turn 0' by the game internally and commands like I_TurnNumber, so it's good practice to subtract one from reported numbers. The script only logs up to Turn 500, though the situations where not only are beta testers playing past that, but they can replicate a CTD that happens on a turn after 500, are infinitesimal.
    Last edited by Augustus Lucifer; January 06, 2010 at 04:34 PM.

  8. #8
    Halie Satanus's Avatar Emperor of ice cream
    Join Date
    Nov 2005
    Location
    London
    Posts
    19,995
    Blog Entries
    1

    Default Re: The Beta testers handbook. WIP

    Always with the extra inch AL..

    This could get serious, but at the moment I think I want to keep it fairly basic and user friendly for people who don't have a clue. As you probably know good beta testers are a rare commodity, but it's not beyond most people. I hope this (basic ) giude might entice people to offer their time..

    The first two points are good though and I'll defiantly simplify them into layman's terms..

  9. #9

    Default Re: The Beta testers handbook. WIP

    Maybe, report everything that seems strange. Most of the time it won't be a problem, but it can be something that is easily overlooked : shield angle for units, patches of snow in weird locations, strange textures. Finding CTD is good, but if you want to be an excellent beta-tester, finding the little wrong details is the way to do. This mean you have to look at everything closely : don't skip reading the descriptions, have a close look at your units when deploying...

    Also, try to always provide screenshots (it won't be possible for CTD, but for the rest...). Saying "I saw snow in the desert" is a first step, providing the screenshot is best.

    And finally, try everything. Don't just do the basic things everyone will do. Even if it's against your playing style. Things like trying to move anywhere around the map can help find mapping problems like blocked pass, rivers unabled to be cross...
    Under the patronage of apple.
    Patron of ybbon66.

  10. #10
    Axalon's Avatar She-Hulk wills it!
    Join Date
    Sep 2008
    Location
    Sverige
    Posts
    1,273

    Default Re: The Beta testers handbook. WIP

    Hi guys, this is a good thread Halie Satanus, have some rep on 'ol me for creating it.

    For what it is worth: Yes... keep it simple and basic here. That seems to be a good idea.

    - Cheers

  11. #11
    Spartan198's Avatar Protector Domesticus
    Join Date
    Feb 2008
    Location
    California, USA
    Posts
    4,753

    Default Re: The Beta testers handbook. WIP

    Good topic, Halie. It should help to make more great beta testers in the near future.

  12. #12

    Default Re: The Beta testers handbook. WIP

    save game each year / turn - with different descriptive names for each like "sev-hh-y980" = to mean Severin campaign, on hard, hard at year 980 etc.... so you don't overwrite old ones and you can find relevant ones in-case someone asks you to look back... (this is general good practice, but not something I often managed to persuade Halie or most of testers to do )

    ending up with one final save / autosave which proves that it won't load / advance to next year isn't always exceptionally useful, you may need to go back to look at prior saves if available....

  13. #13
    Halie Satanus's Avatar Emperor of ice cream
    Join Date
    Nov 2005
    Location
    London
    Posts
    19,995
    Blog Entries
    1

    Default Re: The Beta testers handbook. WIP

    Added as point 9..

  14. #14
    grandpa's Avatar Elite Bytebumbler
    Join Date
    Nov 2009
    Location
    Augusta, Ga
    Posts
    556

    Default Re: The Beta testers handbook. WIP

    Thank you all for your wise input. If ever given the priveledge to test a Beta, I have a better understanding of what is desired. This has been extremely helpful.

    "Work smart or be tuff", American Inventor Herbert Willaims. An amazing man. Inventor of the Openhydro Tidal energy machine, located in Orkney Islands. www.openhydro.com

    It doesn't cost a dime to be nice yet has the highest return
    on investment. grandpa

  15. #15
    Halie Satanus's Avatar Emperor of ice cream
    Join Date
    Nov 2005
    Location
    London
    Posts
    19,995
    Blog Entries
    1

    Default Re: The Beta testers handbook. WIP

    Glad it's helped, all you need do now is find a mod in need of testers and apply. I imagine Roma Surrectum is still in need.

    Take care ..

  16. #16
    Laetus
    Join Date
    Feb 2013
    Location
    anty881209wong 140219
    Posts
    1

    Default Re: The Beta testers handbook. WIP

    Hi guys, this is a good thread Halie Satanus, have some rep on 'ol me for creating it.

    For what it is worth: Yes... keep it simple and basic here. That seems to be a good idea.

    - Cheers

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •