Gamasutra-Creative Assembly's 'fairly brutal' approach to achieving high Metacritic scores

[camper-futter]

Riekopo posted this in officiall TW-forums and it attracted my attention.

"Total War developer Creative Assembly shoots for a 90 percent Metacritic score for its games, and has a "brutal" approach to getting there, the studio tells Gamasutra in our latest feature.

Metacritic's average review scores have become increasingly important for developers in recent years, as many publishers and consumers refer to those numbers as an indicator of a game's quality, as well as the quality of a studio's output. In some cases, those scores can determine whether developers receive bonus payments from publishers.

Creative Assembly's studio director Tim Heaton says that his team goes through a rigorous process when deciding what features to include in a game. The Sega subsidiary will cut some features immediately, but it will prototype and judge the quality of others before deciding to ditch them. And while prototyping ideas and examining their process, the group will also abandon features that it feels will take too long to turn into a high-quality addition.

"Through production, actually, we do what we call 'Metacritic analysis,'" says Heaton. "We will break those features down into subsets, and we both look at it from a player's point of view, and a reviewer's point of view, and we'll weigh certain features as to how we see players and reviewers look at them, and they'll build up to a 100 percent score, and then we'll judge where we feel we are on those individual feature sets, and see the momentum on those and the velocity on those, too."

He adds, "And so if we see one flat line and it's not where we want it to be, we then will cut it. Well, we'll cut it really late in the day. I think teams are really scared about doing 90 percent of the work and then cutting it. It's kind of like, 'Well, it's nearly finished; I... I've done all the work! Please don't cut it! I'm sure I can make it better.' And we're fairly brutal on that."

Heaton says he would much rather reject a feature that the studio invested resources in, than have it left in the game and affecting its quality. "You know, every step of the way -- from the beginning to the end -- we're talking about a 90 percent Metacritic," the Creative Assembly director emphasizes. "That's our goal. That's what we tell Sega. And we communicate that through graphs, basically, of where we think we are."

He continues, "We build into that also, on that Metacritic analysis, external events. So if we think we've done a really great PR job, if there's an individual event that we've done really good, we might add, you know, a .5 percent Metacritic. If we think it's ed up or somebody's not done their job right, or miscommunicated something, or whatever, we'll see that in our Metacritic analysis. And we share that with Sega on a weekly basis, so that they can figure out how we're doing, too."

Heaton admits that cutting a feature that might have been nearly completed can affect the studio's morale, but he believes the team eventually buys into the idea that these decisions are for the best: "Certainly some elements of a team -- and this always happens with every team I've ever worked with -- just go, 'Come on, just give me a game design document. Just tell me what I need to do and then I will do it to the best of my ability.' And we slowly, hopefully, educate people that's just not the best way.

"And we will enter a fog of ideas for quite a long time, and some of those things will have risks against them right up until the end, and then we might pull them. It pisses people off, absolutely, but it's for the best. But nothing makes the team prouder than delivering a 90 percent game and selling two million copies. So that's the bottom line, and people do come to understand that."

The full feature interview, in which Heaton talks more about Creative Assembly's process for building high-quality games, is live now on Gamasutra."

[GeorgePatton]

nobody cares about metacritic or fake articles from blogs.

[camper-futter]

nobody cares about metacritic or fake articles from blogs.

You expressed serious concerns. How do you reason them ?

[Holger Danske]

nobody cares about metacritic or fake articles from blogs.

Keep up your good work, you'll eventually convince people that the situation is under control and the infidels (bugs) have been crushed...

[Humble Warrior]

GeorgePatton is trolling. And won`t listen to any kind of reason. He just don`t want to know that CA take Metacritic seriously so it`s worthy of discussion. You can`t reason with someone like that.

[BunnyPoopCereal]

nobody cares about metacritic or fake articles from blogs.

If that were true this thread would have never been made in the first place.

[Destraex]

Sounds like any other large corporate. More about method than any sort of effort to make a game that people want and love. Or a combat system that is realistic and fun....
No wonder the combat system is so screwed up. I bet this is how guard mode and javelin mode got cut.
But I have to say over all the game is good still and will be better with mods.

[alhoon]

More about method than any sort of effort to make a game that people want and love.

They have such a method and they just told us which quality-control method they use: They try to get good metacritic scores. That's not a bad method. It kinda failed them this time but it's still a good method to make a game that people want and love, cause those games theoretically score high on metacritic.

[Maelbox]

I think this describes a lot of the issues the studio is having from a design standpoint. Rather than seeking to improve on what they've already got, they cut it instead, try to redo it and sometimes leave it out of the game entirely. I'd love to see less cutting (family tree, options in battle like guard mode, etc.) and more improving on what we've got so that if it isn't up to your required standard improve it until it is, as well as innovating and adding new features simultaneously. I find that the game feels empty and while it appears to have some depth I honestly think it's an illusion if anything. There's no need to treat your audience like simpletons. As long as its intuitive and engaging, they will sit down and learn how to play even the most complex game and that's even more true with strategy fans.

[VektorT]

As long as its intuitive and engaging, they will sit down and learn how to play even the most complex game and that's even more true with strategy fans.

Indeed you said it all.
Most Paradox games was a pain in the ass the first time I played and I rage quitted sometimes 'cause I was lost. When I finally got then, they was incredibly awesome. Those games are complex and hard to master, but very challenging and every play through feels different. Total War is losing it to me title after title...

[Kahvipannu]

I have been saying this all the time, and I say it again, stop giving Metacritics hits. Stop it, it's bad for the industry, it needs to vanish. Meta selects who it approves to be considered a legit review, they have applications that the reviewer needs to get thru, the scores are not real. 4 stars isn't 80%... They pull most of the stuff from theyr hat in that regards. There is multiple examples of shady activity at Meta like not changing review scores after the actual reviewer changed it later, becouse "principles" or something... I can't think one good thing they do to the industry, publishers put the success of studios in the hands of people who are not capable of showing it the right way... It's ridiculous...

[IlluminatiRex]

I have been saying this all the time, and I say it again, stop giving Metacritics hits. Stop it, it's bad for the industry, it needs to vanish. Meta selects who it approves to be considered a legit review, they have applications that the reviewer needs to get thru, the scores are not real. 4 stars isn't 80%... They pull most of the stuff from theyr hat in that regards. There is multiple examples of shady activity at Meta like not changing review scores after the actual reviewer changed it later, becouse "principles" or something... I can't think one good thing they do to the industry, publishers put the success of studios in the hands of people who are not capable of showing it the right way... It's ridiculous...

If that doesn't work I'd advocate starting a petition to have Metacritic taken down.

[raest]

I have been saying this all the time, and I say it again, stop giving Metacritics hits. Stop it, it's bad for the industry, it needs to vanish. Meta selects who it approves to be considered a legit review, they have applications that the reviewer needs to get thru, the scores are not real. 4 stars isn't 80%... They pull most of the stuff from theyr hat in that regards. There is multiple examples of shady activity at Meta like not changing review scores after the actual reviewer changed it later, becouse "principles" or something... I can't think one good thing they do to the industry, publishers put the success of studios in the hands of people who are not capable of showing it the right way... It's ridiculous...

to add to your rants... this is one of the chief reasons why we get all these crappy "popamole" games: the gaming industry doesn't have a single "serious" reviewer and there's no such thing as "gaming journalism" because the vast majority of these "journalists" are hacks that get paid for PR or are just incompetent. i.e. the games industry doesn't have a "Roger Ebert" so to speak (as much as i dislike him), but instead relies on, as you well put it, shady crap like metacritic

[leemyster]

to add to your rants... this is one of the chief reasons why we get all these crappy "popamole" games: the gaming industry doesn't have a single "serious" reviewer and there's no such thing as "gaming journalism" because the vast majority of these "journalists" are hacks that get paid for PR or are just incompetent. i.e. the games industry doesn't have a "Roger Ebert" so to speak (as much as i dislike him), but instead relies on, as you well put it, shady crap like metacritic

Though I haven't researched too much into these non gaming media reviewers like e.g. 'AngryJoe' (so cant tout for him being perfect) I think that reviewers who get people to listen to them because they can trust them and not because they are part of someones favourite gaming media site is the way forward.

It's an open secret that there is corruption in gaming media but consumers will still read their reviews, visit their sites etc. However, if more community orientated reviewers like angryjoe (others do exist of course) where corrupt then they're viewship would disappear as people watch them for amongst other things their impartiality.

[Jackal2233]

As long as its intuitive and engaging, they will sit down and learn how to play even the most complex game and that's even more true with strategy fans.

YES!

[Kahvipannu]

I have to add for my earlier post, there really also isn't any mutual agreement of reviewing games. Every single review is an opinion of single person. The review will be including "facts" about the game, like how good it is technically, and mostly subjectives arguments. Some people value other areas of things, and other the opposite. One might value story over mechanics, other other way around. These influence the outcome, and a review is no good review if these two things, facts and opinions are not clear, and it is not clear to the reader what kind of person the reviewer is.

One might influence the score more on one department of the game, like visuals, one might not care about it at all. This way the actual score is totally irrelevant, everybody determines themselves what affects, and how much to the overall score. What is important is the context of the review, and the reviewer....

I just see red when people worship the scores... And the whole Metacritic is based on these meaningles numbers, or stars, or whatever that have zero relation with each other how they are determined. And they get to choose who they aprove on theyr list? And they put more weight on certain sites, not caring about the actual reviewers, and rounding for example 5 number systems to 100 system? And people and specially publishers care about that stuff? "88 isn't enough, you needed 90..."

Rant end... Sadface.

[jtfjtfjtf]

79/3.9 on metacritic. That's very bad. They concentrated too much on being average for everyone and not enough on really making people say "This is my favorite game ever."

[Strattios]

If they really believed this, then why did we get a game with so many half finished features? At least in S2 there was more focus on art and you knew it. Not R2. Where is the focus? Just big and lots like empire but with cheesy Hollywood styling thrown in?

[vnth]

aim for 90% on metacritic
universally panned by fans

wow, this makes for some really great memes.

[Unchained]

In this regard, the game still does not seem to have achieved its intended goals, as there were some pretty decent reviews, Angry Joe for example I think was fair. I think that the best thing to come from this is that we now know what sites to watch for future reference and what sites are paid shills.

It's also safe to say that at this point the game did not meet the objective from the fan reviews.

It's fascinating though that the game companies are so obsessed with Metacritic though. Perhaps they think they will get more sales with inflated ratings?

Regardless, I wish that the focus was on development not marketing. Dear CA, create a solid product before launch. Over promises and lack of delivery is a sure root to long term failure and the margins are tight as they are. Early release should never be a priority. Stability and polish first.

Sent from my SGH-T889 using Tapatalk 4