The 11th Company 40K Podcast

Welcome to the 11th Company BLOG. The 11th Company is a Warhammer 40K podcast dedicated to players, strategies, and tactics.

You can download our episodes at the website, from ITunes, several podcast sites, or connect directly to the RSS Feed. We try to release a new Episode every Monday Night. Check it out!

Forums: http://the11thcompany.freeforums.org

PODCAST WEBSITE: http://www.tangtwo.com/11thcompany

PODCAST RSS FEED: http://www.tangtwo.com/11thcompany/podcasts/the11thcompany.xml

Podcast Archive: http://the11thcompany.libsyn.com/webpage


Search This Blog

Thursday, June 7, 2012

GT Appearance Scoring Pt 2


Wow!  Since I started with the first part in this series, which amounts to an introduction to what I want to do with Appearance Scoring at our GT, I have discovered that the Rabbit Hole gets very, very, deep indeed!  It feels like what started out as an innocent review of the system has now made my head explode!

Where to start?

Well, since I've now come up with 4 different topics which all surround Appearance Scoring and Best Overall scoring, I'll go ahead and introduce them here.  From there, I'll talk about a simple conclusion for each, and then, I'll address the granular specifics behind each in their own article.


  • Best in Class, encouraging good participation
  • Appearance/Sports Scoring based on the competition rather than an arbitrary rubric.  How much better did you do versus everyone else in the room, not against some unproven, mostly arbitrary, non-researched number.
  • Judging Appearance Scores with a rubric and BEYOND.  Truth is, rubrics really suck at determining whose army really is the best.  They are very good for weeding out contestants but very BAD at classifying the best.  We can do better.
  • Redefining how to calculate Best Overall.  Turns out, when you really get to the bottom of the Appearance Scoring rabbit hole, the entire notion of Best Overall and how it is calculated begins to change.  More to come!


Appearance Scoring Rubric, Attainable Appearance Scores, Generalship, and Encouraging Better Participants... AKA  Best in Class


In the first part of this series, I had originally wanted to modify the Appearance Scoring rubric so that it was attainable for most people, rather than serving a purpose of only to granulate the best from the worst in the room.  Why?

1)  First and foremost, because at it currently stands, most Appearance Rubrics set a bar for the Best Overall category which is insurmountable by the average hobbyist.  Best Overall, after all, will combine your Appearance and Generalship scores.  Thus, if you aren't an above average hobbyist, once you lose a game, your ability to place in Best Overall just went along with it.  So, for the average and below, it's all about winning your games.

2)  Because of that, if you know you will never be a top notch hobbyist, this provides zero encouragement for you to even bother getting serious about painting your army.  The reality is, you either win on Generalship alone or you pack it up and go home.

But it doesn't stop there!  The same exact two points can be made for people who don't have a lot of Generalship skills.  If they know they will never be a top player, if they spot an army that's better than theirs, well, they can pack it up right there!

What's even worse than that?  What if you know you will never be a top notch player nor a top hobbyist?  Well, other than just being social, which should be a strong motivator for anyone with their head in the right place about attending GTs, there isn't much encouragement for you to show up to begin with.

Does this sound like a familiar drum beat?

Well, good grief.  All I wanted to do was encourage people to paint better, and all of a sudden, I'm opening up a philosophical can of tournament worms.

So, part of my misconception about Best Overall is that one of its intents is to reward people for all aspects of the hobby.  That part is true, but the misconception is that I had thought it to be designed for the average hobbyist to be able to compete at a high level because he could bring some Generalshiip game and some Appearance game.  Turns out, that's not the case.

The reality is, Best Overall rewards the most talented person in the room.  It is still as utterly unattainable to the average hobbyist as Best General or Best Appearance.  The truth is, Best Overall will always go to the person who is highly above average in the room in all categories, or utterly exceptional in one category while being average-above average, at a minimum, in another.

Thus, if you can't be above average in every category, no point in trying for Best Overall.  Thus, for guys like me, that makes us really not want to spend to much effort, tournament wise, on making sure our armies look good on the table.  Likewise, for those exceptional hobbyists out there who barely know the rules of the game, they just show up to go through the motions of a game of 40K, while in reality, trying to win Best Appearance, and generate a series of lop-sided or generally un-challenging games for their opponents.

In both cases, everyone loses.  The issue is that we really do want to encourage people to bring their A game, even if their A game amounts to other people's D game!  We all win in that case.  We get better looking armies at our tournaments and funner, more challenging games for everybody involved.

This is, of course, an unrealistic ideal, but if you don't try, you definitely won't succeed.

Best in Class


Through the power of math-fu and a little thought, we can take some steps to make a meaningful tournament goal actually attainable to average player.  We all love winners.  So, you aren't going to just give out a booby prize.  The guy still has to earn it.  And by earning, that means he will still have to be ABOVE AVERAGE.  So, how is this going to work, again?

As said in the last article, one of the genius ideas given to us by the NoVA Open is putting people into Generalship brackets on Day 2.  This allows them to compete for a meaningful Generalship prize with their peers.  It creates an actually attainable Generalship goal for the average player.  Very cool!

Now, what if we also created an actually attainable Appearance Score and Sports Score for the average player that can't be completely over-shadowed by the talent in the room?  Then, we could combine these three scores, and presto, we have a new prize class which is basically Best Overall 'for the rest of us'.  I'll call it, Best in Class.



We take your Generalship score from the games you play in your bracket only.  Great.  This is you versus your peers.  This excludes earlier games where you may have clubbed or been clubbed as the case may be.  Average versus average, at least in theory anyways.  Check!

Next, we generate an Appearance Rubric 'for the rest of us'.  (Or more realistically, we add a cut score to our current rubric)  This new rubric encourages you to bring a great looking, table-top quality army, one in which anyone, with enough effort and time, could actually do if they felt so inclined, sprinkle in just a little reference or two for some talent, and there, an actually attainable appearance score that lets everyone compete towards a "great table-top quality" rather than towards "BLOW YOUR FREAKIN' MIND AWESOME!".  It also just so happens to lay out the road map for what a good army should have in it... you  know... for those of us who are artistically clueless.

I didn't invent this Appearance idea.  Other GTs did.  They did it when they started introducing appearance rubrics which make the first 90 points easy to get, and the last 10 difficult (or the first 15 and last 5.. or whatever).  This way, your top talent still gets a higher score, but the score they receive won't blow everyone else out of the water.  Or, maybe you just set an appearance cap for Best in Class, which is what I am leaning towards right now.

THIS WILL NOT REPLACE BEST APPEARANCE NOR BEST OVERALL.  Keep that in mind.  You will still need another way to do this, and that's to come in more articles.

Last by not least, you look at Sports scores which our Sport Scoring system already normalizes for us.

Now, you have a path to reasonably attain a high Generalship score by only counting games against your peers, a path to reasonably attain a high Appearance Score by providing a clear, easy to follow, rubric with attainable goals, and a path to reasonably attain a high Sports Score because Sports is already normalized with most people getting average to begin with.

Best in Class.

Thoughts?

6 comments:

  1. I'll probably have more to say, but I'm just going to throw in the quick corollary here.

    If you devalue appearance score, you're telling the average GAMERS who are pretty darned good hobbyists that they shouldn't even bother attending ... unless they know they're the best painter there, b/c as soon as they aren't, they know they can't even try to win Best Overall due to their generalship being "average."

    I'm still going to reiterate my overarching concern, which is that Best Overall really needs to be Best Overall ... the second it's "Some Average or better hobbyist who score most of the available easy-rubric points and also won a ton of games," you're not evaluating Best Overall at all. you might as well call it "Guy who scored somewhere around 2nd or 3rd in Generalship and also had ok paint scores." Yadda yadda.

    I think you're overall going in the right direction ... and I see you saying it won't replace Best Overall. Again, just want to make sure you're forcibly curving appearance scores so that the guy who gets the first 90 points has ALMOST no chance at Best Overall, if any ole guy can get the first 90.


    As a PS, our average appearance score at NOVA discounting zero's this last year was a handful of decimal points below 50. Gotta give Geoff credit for that.

    ReplyDelete
    Replies
    1. Can't speak for Mike on this one. He is probably just referring to removing zeros to show that his rubric was pretty well designed to give an average of 50/100, which if you keep in the people who didn't paint their stuff, causes that average to drop.

      I can tell you that when I get to Appearance Scores calculation and its relationship to Best Overall, in Article #3 on this series, I will be removing zeroes because they represent something which is not possible in either the Generalship or Sports scores, that being the "opt out".

      I'll talk about that in that article about how the "opt out" really damages Appearance Scoring if you try to factor it in in relation to Generalship and Sports when you go to calc Best Overall.

      Delete
    2. @MVB

      You gotta give me a chance to go over everything. There's a lot to write, and if I put it all in a single blog post people complain that it's too long. :( Either way, I get it. :p You can't devalue appearance scores, and skewing a rubric for easy points while not adjusting for that will indeed devalue appearance scores.

      Turns out, what gets even more interesting is you can't devalue Generalship or, for us anyways, Sports either. Appearance Scores really become an 800 lb gorilla because of the nature of rubrics, having a max score achievable by multiple people, having people opt out, and the potential, although unlikely at a 256 man level, for a wild variation in appearance scores in producing an over-powering or under-whelming effect on Best Overall, just by the nature of it.

      Talk about that in the next one.

      Also, awesome that he was able to predict scores that accurately for your rubric last years. He's got a good eye for the stuff. :)

      Delete
  2. Haha, you're totally right Neil - I DO need to give you the chance to present the whole thing.

    I also think you've hit on some issues and some insights that I hadn't to this point, which is excellent.

    Andrew, per what Neil said, I removed 0's b/c you can't evaluate the effectiveness of an appearance matrix based upon people who refused to be scored, and/or did not paint at all and therefore received default 0's.

    ReplyDelete
  3. I wish I could add something constructive, but honestly, when I read scoring/seeding discussions, I feel like I am listening to a Mentat from Dune.

    /popcorn

    ReplyDelete

Due to spam, all comments are moderated.

Pat

Note: Only a member of this blog may post a comment.