This week on the podcast we are starting a 3 part series on
Appearance Scoring for our GT. The
reason is that I am trying to come up with a smart way of accomplishing a few
goals with Appearance Scoring at our GT. What we ultimately want is a way to encourage people to a high, table-top standard, encourage players to compete for the Best Overall category, and finally, of course, provide a great atmosphere for the true Best Appearance competitors to really show their stuff.
Now, before I even get started, I suppose I should start with a disclaimer. For some reason, anytime you start talking about Appearance and Sports scoring, some groups of people feel the need to exclaim that this kind of thing is too subjective and should really not be scored. Let me go ahead and take care of that now.
Appearance Scoring is subjective? THANK YOU CAPTAIN OBVIOUS! Now, that we got that out of system, let's move on and see if we can apply our collective brain power to solving the problem rather than simply stating that there is a problem. (As a more serious life lesson for any who haven't thought about this yet, the money, power, and women go to the guy who finds solutions not to the guy who points out problems. We are all smart enough to recognize problems. We AREN'T all smart enough to solve them.)
We just finished recording for this week over an hour of back-and-forth conversation revolving mostly around the starting concept of Appearance Scoring. Although a good portion of that conversation may very well get edited out for the rambling and circular talk, I found that conversation to be VERY useful to me when determining what it is exactly that I want Appearance Scoring to do at my GT.
1) Reduce "butt-hurt" that gets associated with Appearance Scoring. Nothing generates "butt-hurt" at a tournament quite like Appearance Scoring. People get down-right hostile when they feel like their army should have scored higher than it did, ESPECIALLY higher than that guy's army over there!
2) Encourage people who will NEVER be the BEST to still strive for a high table-top quality.
3) Encourage people (just like me!) to honestly compete for Best Overall, instead of just hanging it up because we know that our Appearance Scores will not be adequate enough.
4) Create an arena where the people who CAN be the BEST a place for their work to be truly graded at the level they need, rather than at the level of the "rest of us". Do this, while not making it impossible for the "rest of us" to win Best Overall.
5) Give players an honest way of knowing up front how they will perform, Appearance wise. To me, this is just like players knowing what the tournament packet is before they show up with their army.
So, it's no secret that I'm a total fan boi of the NoVA Open. The NoVA Open concept has blessed us with two very good ideas when it comes to tournaments which need to be reflected on at this point before continuing to talk about Appearance Scoring.
Best Overall Category - NoVA calls this Renaissance Man. (I prefer Best Overall because it sounds a lot less pretentious to me... I said I was a fan boi not a drone. :P Also, if you haven't bought a 2012 ticket yet, you should. They don't have a whole lot left.) Best Overall, in my opinion, in the best overall thing to happen to the Indy tournament scene. The concept is that the top winner of your tournament is not the guy who wins all of his games. The concept is that your top winner is the guy who most ably reflects all aspects of our great hobby. Thus, this combines elements of Generalship, Appearance, and for us, Sportsmanship. So, to win Best Overall, the top tournament prize and spot, you need to bring you game not only on the table-top, but also in the hobby.
Brackets - Another NoVA brain-child which I find to be genius. This concept is that in a multi-round GT, you will use the first half of the GT to essentially qualify for what will become your actual tournament brackets in the second half. This way, when you actually start to compete to win prizes, awards, etc., you are doing so against your PEERS rather than having ZERO chance to compete because the people who are the Best Generals just drown everyone else out.
So, what does this have to do with Appearance Scoring, and more importantly, who cares?
What I want is a method for EVERYONE to be able to compete for Best Overall and not simply get canned because they can't paint very well. This is important for people JUST LIKE ME, whose hobby skills BLOW. You see, this is no different than the guys who feel like they can never win on Generalship because they will never be as good as others. They don't want to swim with sharks, but let me tell you, NEITHER DO I. At the end of the day, we should model bracketing in some manner that let's us all compete and more importantly, HAVE FUN.
Second, the goal is Best Overall. Being able to compete with your peers increases your chance of scoring the coveted Best Overall. If I have an honest shot at winning Best Overall because I don't have to paint like Rembrandt, I might actually make an effort to do so. Likewise, if a true hobbyist has a shot at Best Overall by learning how to use his army, he might actually TRY rather than just hanging it up.
So, what direction are we headed in this series?
What I would like to see is the creation of an Appearance Scoring rubric that is designed for the average guy. Something people like me can strive for. This score is what will ultimately apply to my Best Overall ranking. This will encourage guys like me to give it an honest try, and at the end of the day, we all win because guys like me will start showing up with good table-top quality armies rather than half-assed paint jobs.
Second, since the Appearance Scoring rubric will be for the average person, I expect anyone with a serious shot at a Best Appearnace to easily sweep past the rubric. These guys get a top score in painting, which they should, and THEN, they go off onto a separate path for Best Appearance. This leads into a much more subjective grading system performed by their PEERS. (More on this later). This way, they get the judgement they deserve rather than a silly rubric which will have the audacity to compare their master pieces to guys like me!
Conclusion
In Part 2, I'll start talking about the formation of a rubric as well as what happens to the Best Appearances. I'll also continue the discussion around Best Overall because there are some subtle problems with scoring Best Overall to a rubric but not Best Appearance, mainly not giving ENOUGH credit to the Best Appearance players towards the Overall score.
Careful ground to tread, is all I will share. Remember, the overall Best General in NOVA format is the guy who wins every game - who excels at generalship, with subordinate awards for generalship within peer groups.
ReplyDeleteFor Best Overall, if you overly escalate the rubric so that "average" painters are scoring very highly on it, and not far below peerless painters, you'll get into a very rough situation where your Best Overall does NOT reflect a true Best Overall, even if everyone feels like they can get high paint scores.
To wit, with the Generalship component largely being a reflection of record, and the largest record distributions being toward the middle inherently (4-4 in an 8-game format is the most common result), you see people struggling to win Best Overall at 4-4. And rightly so ... your chances improve the higher you go, so 5-3 with perfect appearance, or 6-2 with slightly not perfect appearance, or 7-1 with good appearance, or 8-0 with ok appearance.
If the rubric changes so that average painters are scoring above the 50th percentile, you're devaluing the contribution of a high paint score ... that's to say, your highest distribution in appearance needs to also be around the 50th percentile, or you're devaluing the appearance contribution to the hobby.
This applies however you do your splits ... and applies to sportsmanship also.
While it's admirable to require only mediocre performance in sportsmanship and appearance, you can't claim this is fair unless it also only requires mediocre performance in competition. That's to say, if you're sticking with an even contribution from all components of the hobby for Best Overall (and you really should), you'll have to "cap" competitive score to boot ... that's to say, that a max score would be what you score with a 6-2 (let's say), so a 6-2 finisher gets a 100% competitive score, and an 8-0 gets a 100% competitive score ... IF you are going to escalate everyone's hobby score via a rubric designed around "average" hobbyists.
Said another way, a truly average or sub-average painter should NOT be able to very easily win Best Overall ... in fact it should be nearly impossible, and happen as a result of NO superior painters performing well competitively. If you artificially lower the bar for high score in appearance, you weaken the value of painting to your consideration of Best Overall, even if you keep the % contribution of each score the same.
$.02
In addendum, the long and short is what is your goal?
ReplyDeleteIf Appearance is just as important as Generalship and Sportsmanship to your Best Overall, you need to achieve a highest concentration at the 50th percentile, just like you do with Competitiveness, and you also need to do so for Sportsmanship (or not use it as part of Best Overall, since it is not uniformly subjective).
If Generalship is MORE important, then your suggested change is totally reasonable. But, from what I'm hearing, you're not making a claim that Generalship is more important to the hobby than Appearance.
Same is true for sports, bee tee dubs. If sportsmanship contributes, that also needs to achieve 50th percentile as most common, among all results.
Noted and well understood. Re-worded from above, I got an hour long conversation to edit down at this point which is circular, back-and-forth about how the rubric needs to still be reflective of your top level appearance scorers so that their contribution to best overall isn't brought down. I'll be addressing that next article I hope along with some potential simulation numbers. Right now considering everything from a sliding scale to other caps (see below), to a whole host of other ideas.
ReplyDeleteGeneralship is not more important. All things equal. Capping generalship score is also a distinct possibility being considered. The rammifications I'll get to see first hand with NoVA this year since you are stopping calculation at Round 6, which will net a group of people who are 6/0 rather than one guy who is hopefully 8/0. Not sure if a cap is totally necessary or not but definitely can't dismiss out of hand.
The goal re-worded from above then, is to encourage people to do their best with appearance scoring. As it stands, most Appearance Scoring works like this towards Best Overall...
Can you compete with painting enough to be well above average? No? Then, it's not worth putting the effort into trying and/or BUY a paint job then.
Likewise, this is how people feel about competing for Generalship. e.g. Are you a good player? No? Then no sense bothering to try and get better because you'll just get crushed by guys who get 10 practice games a week anyways. This is why your bracket concept is genius. It gives people a reason to play. Appearance scoring, in my opinion, needs a prod to do better as well rather than just giving up out of hand.
Should also add that likewise for Appearance and Generalship, if Best Overall is basically unattainable for the average person, they have no real incentive to compete for it.
ReplyDeleteSomething about that seems contradictory to what Best Overall is intended to represent, to me. It also seems entirely appropriate for Best Overall at the same time.
It should come as no surprise that I am actually a big fan of the "Best Overall" Category - but I am also not sure you can attribute it's invention to The Nova format. This has been done for years by the guys running Da Grand Waaagh GT out here in California.
ReplyDeleteHowever, with all of that in mind, I don't think it should qualify maybe quite as heavy as generalship, but that is just my opinion. Though you touched on the subject of people disagreeing with their subjective score on painting, the other problem with it is people will NEVER agree on just how much it should be weighted.
I would highly recommend reaching out to the guys who have run Da Grand Waaagh GT (Dave Sackl and Dan Benevidez are probably the best two to talk to regarding this - I can get you contact info if you are interested), as they have a pretty black and white scoring rubric for painting and conversion. It's about as clear as you can get in regards to this kind of thing, unless someone comes up with something new and revolutionary.
Anyway, I've been a big fan of their format for awhile since I really think it celebrates the entire hobby as opposed to who can just beat face.
Thanks for the lead. I'll check out their rubric and system in just a bit.
DeleteI'm really hoping, like we have done with a lot of others things via the Podcast, to be able to tap the brains of a large group of people for creative ideas on the subject.
You happen to have a link to their appearance rubric?
DeleteI do not - write to Dan Benavidez – WH40K Head Judge: grotmaster@yahoo.com
DeleteHe can get it to you.
My father has built and restored classic show cars for the last 40 years. When he brings one of his cars to an event, he almost always puts a small card on the windshield that says "Please do not judge". This means he is interested in showing off his car, but not entering into any of the appearance contests, typically due to the subjective nature of such evaluations.
ReplyDeleteI have pondered a similar take in our GT events. I like the idea of simply stepping aside, and allowing your work to be appreciated for what it is, and not in comparison to the work of your peers.
Unfortunately, such an action would make a mess of the "overall" category.
I don't necessarily have any suggestion or comment to make. I just always saw my father's action to be, well, classy.
Appearance scoring in this area is very unique in that one can opt out without negatively effecting anyone else in the tournament. This is not possible for Sports and Generalship where opting out of participation negatively effects other participants.
DeleteI, of all people, have no problem with someone taking a 0 on appearance because they don't want to be judged. You for reasons which we already know, me because then I don't have to stand there for 30 seconds be berated by a judge because my paint scores blow, which is not a pleasant feeling for someone who is visually handicapped to begin with. :P
Now, the second part is the part I'm most interested in. I'd be a bit more willing to get my 30 seconds of berating if I actually had a snowball's chance in hell of that appearance score meaning something.
More importantly, I'd be maybe more inclined to really put forth some effort to make my army look better if I thought that there was some possible way that doing so would net me something. I'm not intrinsically motivated to "hobby better" in much the same way that a lot of hobbyists are not intrinsically motivated to "game better".
The social aspect of that though is that if I'm not motivated to hobby, and the tournament doesn't require it, I'm a dirt bag for bringing a crappy army. For the hobbyist who isn't intrinsically motivated to game and so doesn't really try very hard but just shows up for his appearance scores (or choice not to have them), it's okay because he's not hurting anybody. :P (this entire thought process was a tangent btw... only here because well.. you know me by now and the tangent thing is my MO...)
Anyways, I don't have a problem with people opting out of appearance scoring... either because you want to be classy, or because you don't want to get berated. :P
After some more lunch break consideration, the real issue is that I want Best Overall to both be:
ReplyDelete1) The Best Overall
2) A top prize, REALISTIC, chance for ANYONE.
It may very well be possible that these 2 things are mutually exclusive.
It might very well be better then to create a 4th secondary top prize, in line with Best General, Best Appearance, and Best Sports. A 4th track which is basically a normalized Best Overall.
Neil, just an FYI - we are not artificially capping competitive score.
ReplyDeleteThe range will still be normalized around 3-3 by the end of Round 6, so it's simply possible for more than one person to "max" competitive contribution, which is true for Appearance already anyway.
What you're suggesting would have us finish Round 6, and score everyone the same for competitive who had a record between 4-2 and 6-0, and that's the problem I foresee.
If the "average" score is not a 50 or thereabouts, your appearance scores aren't normalized, and are higher (or lower) in value, and so the contribution is not equal.
I think finding a way to inspire people to TRY to paint better is a noble goal ... but I think it's one that is better accomplished by creating a rubric that it's easier to climb up ... than by artificially enhancing scores.
All you do at that point is DISCOURAGE creating truly beautiful armies, while encouraging even poor painters to "paint to rubric." I think ideally you want to set a situation where getting a 45/100 is not all that difficult, getting a 50-60/100 requires effort above and beyond the normal, and it gets more difficult from there.
Just as it's unfair to create a situation where a 4-2 scores the same or near-same competitive contribution as a 5-1 or 6-0, it's unfair for an average painter to score the same or near same as a very good painter.
The reiteration of this is ... AWESOME painters are all going to max your score, and that's actually OK. Very Good painters, however, are going to meaningfully distinguish themselves from average less than they would in a "fair" system, because the proposed idea may accelerate the "average" to an above average score. This devalues painting as a whole, and over-values competitive contribution.
Effectively, if your average for Competitive is a 50/100, and your average for Painting is a 75/100, painting is less valuable. They have to both hit the same average for it to be normalized, and you can't do that in a "painting is easier to score highly in" rubric unless you proportionally overvalue the average score for competitiveness.
I think you can find a way to encourage better painting without dramatically altering the rubric ... and I don't think average painters should have a very realistic chance at Best Overall, unless the field happens to be comprised almost entirely of average painters (at least the field of 50% winrate and higher participants).
What about raffle tickets and/or extra swag for hitting "tiers" on the rubric? What about simply explaining to people more clearly that every new tier on the rubric they hit is going to compensate that much more comprehensively for a handful of losses?
I think your GOAL here is EXCEPTIONAL ... and it's one I maybe hadn't really given as much thought to, unfairly. I'm just concerned the execution is actually hurting the painting side of the hobby just as much as helping, in that you're proposing something which will inherently devalue the impact of those who are above average to excellent painters. They will be less able to incur losses than previously, putting greater emphasis on competitiveness as a component of Best Overall.
Another thought I just had is - what if tiering the rubric is exactly what you do.
DeleteCreate the real rubric, and use it in its granular entirety for the sake of APPEARANCE AWARDS. But, just as Wins/Losses are tiered to a # equal to the # of rounds+1, separate your Appearance Rubric into tiers. Anyone who is in Tier 1 is equivalent to a 0-6, in Tier 2 is a 1-5, Tier 3 is equivalent to a 2-4, and if you get to Tier 7 you're equivalent to a 6-0.
You could do something along those lines that would, if judged properly, still create a normalized average .... but you could structure it in such a way that it encourages people to try and just get to that next tier ... a new painting goal by the year or something, perhaps. Even the change in numbers and attainables might give people more momentum.
Thinking out loud here.
The idea is in the right direction. Encourage even poor painters to do their best. I'll have to digest that for a bit. You could bracket painting scores just like Generalship scores are already done. Or you could bracket around known cut scores.
DeleteI'd have to think on it a bit too and bounce some ideas and see what you could gain from that. I don't think you could offer a prize for winning a painting bracket because it would feel like a booby prize, unlike Generalship brackets where you still EARN that reward even in the bottom bracket. However, bracketing appearance scores might play in the "Best in Class" idea that I'm circling now.
I'm toying with an idea over "Best in Class" award which is basically a mini-Best Overall, which seeks to reward people in an attainable way (keeps people interested) rather than only rewarding the talented people in the room, not just for Appearance but also for Generalship. In regards to Appearance Scoring, then, I could have a reasonably attainable rubric for Best in Class and a rubric geared towards rewarding talented painters going towards Best Overall. For Best in Class Generalship, I could then just use Generalship produced from your Bracket, something which I would get away with since only 6 round tournament and all players play all 6. This grants you a Generalship score relative to the 'average' which is your peer group along with an appearance rubric that is also geared towards the average player. (Haven't thought enough about Sports yet to work on that :P)
This might kill two birds with one stone. Give me a reasonably attained appearance score that MATTERS while also helping me keep players interested in playing even after they lose their first game in the Bracket by being able to look them in the eye and go, "keep going man! You're still in it for Best in Class!"
(likewise, for a guy like me who will NEVER be able to compete at an appearance level, it keeps me interested instead of drinking away my last 2 rounds at Adepticon because I lost, albeit a very close and super fun game against a top level GT player, which I would travel to Chicago to play again, in Round 2 :P)
Another factor at play here, and a different line of thought entirely, when thinking about this is that your exceptional hobbyist were going to max your rubric, anyways. So, a painting rubric designed as such really only has meaning to your poor-above average hobbyists, not your exceptional ones.
The point there is that at some point, your exceptional painters are going to be judged outside of a printed rubric, anyways.
Just a QUICK note on that very last comment, b/c I think your mind is going in generally the right directions:
ReplyDeleteThe problem for increasing the score for your "average" hobbyists is that it devalues the max score of the top guys. So, upping the "average" artificially HURTS the exceptional painters, and defeats the balance involved with Best Overall.
A final note for the IC guy who posted ... I don't think (or hope) Neil was saying we invented Best Overall. The NOVA has arguably the SOFTEST Best Overall in the entire country, by far I think, but we certainly didn't invent the notion. I just personally didn't like seeing "Best Overall" go to the Best Painted Facebeater.
That's to say, when 30/100 points are appearance, and 60/100 are generalship (with the other 10 being sports/whatever), and the "average" appearance score on the rubric is 20 and the average generalship score is 30, the winners of Best Overall were always the guys at or near the top of Generalship, and also endowed with a slightly prettier or at least average army.
We didn't do anything other than try to make it a "true" split between appearance/generalship, which also already existed in a VERY rare/small subset of tournaments that at a glance "appeared" to over-weight hobby-side scores.
So, more thoughts over the evening.
DeleteThe Best in Class idea is really resonating with me right now because it represents an element that is otherwise not represented in the current Best Overall tract. Best Overall seeks to reward the most well-rounded talent at the tournament, which is awesome. What I'm really wanting after some more introspection here is a meaningful tract in a tournament that is designed to be attainable by anyone because it matches you to the norm rather than to the most talented people in the room. Its secondary mission then is to act as a meaningful win path that will encourage people to play their best, hobby their best, and be an overall boon to the hobby/tournament even if they will never be able to attain the BEST status.
In short, it's a best overall that anyone can hope to attain and thus gives the "rest of us" a reasonable goal to shoot for. Going to work out what that would look like sometime today.
Second, and back more directly to Appearance Scores. Two things.
First, we get that making the rubric easier negatively impacts the values given to, actually, both your highest and lowest scores in the room. (The ZERO actually becomes even more of a hindrance towards overall performance just as the 100% because less meaningful)
However, this idea only actually holds water if your average appearance score is at or near 50% from your rubric. On a lark, I checked my numbers from last year and saw that of the 400 points we could award, average turned out to be 183. However, this number gets dragged down because of the excessive amount of 0's, from unpainted stuff to people who just opt out. Removing the 0's, the average is: 241.
So, as it turns out, using the rubric from last year which was designed to granulate up to exceptional, we were over-valuing Generalship in the Best Overall tract, anyways. Not by a lot, but still by a significant margin.
It then occurs to me that this may very well be the most likely scenario. (would require study to prove) I can reason that it's certainly not unlikely to either already be over-valuing or under-valuing appearance scores simply because you can't guarantee a norm on appearance to begin with.
For your Generalship tract, an average is guaranteed because of the W/L system. The majority of people will always end up "average" because the W/L system drives it to be that way.
Appearance, however, has no such inherent driving mechanism towards average. The "average", if your rubric is as objective as it can be, is actually going to be dictated by the types of armies that show up.
Using an extreme example to prove a point, if 10 players show up with exceptional armies, average armies, or just not painted, your Appearance Scores now become arbitrary in terms of Best Overall selection. Generalship is all that would matter.
So, the nutshell then is, there may already BE a problem with over-valuing or under-valuing Generalship scores in the current system because of a rubric. The point being is that ANY rubric is actually going to run this risk.
As a result, my thinking now is that perhaps an appropriate Appearance score for the Best Overall tract needs to be forcibly normalized (bell curve, basically) just like Generalship is already forcibly normalized, inherently, by the W/L system.
Again, just bouncing ideas off the wall here. Not trying to write a proof.
Not that I run events or anything, but I've been leaning more towards skipping best overall and painting scores.
ReplyDeleteDo Best General. W/L or battle points, whatever.
Do Best Sport if you want to. (whole 'nother conversation)
Do Player's Choice award(s) for appearance. Yes, it's subjective; embrace it as a feature rather than as a bug. Have minimum standards for attending, but let the "this is awesome" votes come from the players. Harder to do at larger events, but for smaller ones where everyone can conceivably see everyone else's I think it's workable.