So, I'm really interested now how one could go about setting up a tournament format in order to appeal to the amateur competitor. Remember that the amateur competitor is the person defined basically as someone who wants to play in a competitive event but wants to play with people of like or equal skill.
This really should be something we should strive for: pairing players of like skill against each other. This creates an environment that is both competitive yet fun for all involved.
So, I've been putting some serious thought into how this might get accomplished. The biggest challenge, of course, is how does one determine the skill level of a player so that they can be matched together?
Well, the first thought that comes into mind is a preliminary event that must occur in order to rate and/or rank players. From there, we could make determinations about like skills.
So, I wrote a computer algorithm which would randomly explore 3 types of pre-liminary matches for me. It would then conclude a final ranking system for pairings for an actual tournament. It would run the experiment 5 times and find an average of the ability of each algorithm to properly place individuals in the actual tournament.
In less complex terms.... take 64 players and rank them from worst to best in terms of skill. Now, pretend that we want to run a tournament where we put "like skilled" players together. Let's say we are allowed to run 4 brackets. That puts 16 players in each bracket. If our algorithm for determining their placement works, players 1 - 16 (the worst to the 16th worst player in our group of 64) should all be in a bracket together. Likewise, players 48 - 64 should all be in a bracket of 16 together as the best players. Hope that makes sense!
So, the computer program I wrote is going to randonmize and test 3 algorithms which seeks to create 4 brackets of 16 as described above. It then calculates the % of players in each bracket that were properly placed. In other words, if the worst players in the lot somehow ends up in the "top bracket" this creates an error both in the top and bottom brackets. What we then measure is the % of players in each bracket that are properly placed. The algorithm which achieves the highest average %'s in each bracket is the superior yard stick for player placement.
Each Algorithm assumes you will play 4 games as a prelim.
Algorithm #1 is a random algorithm based on W/L pairings. Round 1 is an entirely random matching of players. From there, Rounds 2, 3, and 4 randomly match players together based on their current W/L record. So, for example, in Round 2 everyone who won in Round 1 are randomly matched as is everyone who lost in Round 1. In Round 3, everyone who is currently 2-0 gets randomly paired, everyone who is 1-1 gets randomly paired, and everyone who is 0-0 gets randomly paired. Also, a random amount of battle points is assigned each round for 0-9 for the loser and 11-20 for the winner.
Algorithm #2 pairs individuals based on W/L pairings and battle points acrued using a seeding algorithm. Round 1 is entirely random. Also, each round, battle points are assigned for 0-9 for the loser and 11-20 for the winner. This is important because those battle points are then used to seed for the following rounds. From there, like W/L records are used, but instead of being randomly pairied, the top battle points player in the bracket is paired with the worst battle points player in the bracket.
Algorithm #3 is the same thing as Algorithm #2 expect instead of using seeding, the top players in each bracket are paired, then the 3rd and 4th players in each bracket are paired, etc.
No science experiment is complete without a control. The Control Algorithm in this case is randomly pairing each player, every round, and at the end, assigning them to brackets based on W/L record and Battle Points just like the other algorithms do.
The results of all of this are actually quite disheartening! Take a look.
Algorithm #1 Average Correct Placement %:
- Low Bracket: 78.756%
- Middle / Low Bracket: 41.25%
- Middle / High Bracket: 42.5%
- High Bracket - 75%
Algorithm #2 Average Correct Placement %:
- Low Bracket: 60%
- Middle / Low Bracket: 35%
- Middle / High Bracket: 33.75%
- High Bracket: 62.5%
Algorithm #3 Average Correct Placement %:
- Low Bracket: 75%
- Middle / Low Bracket: 38.75%
- Middle / High Bracket: 43.75%
- High Bracket: 76.25%
Control Algorithm Average Correct Placement%
- Low Bracket: 65%
- Middle / Low Bracket: 41.25%
- Middle / High Bracket : 42.5%
- High Bracket: 67.5%
So, Algorithm #1 and #3 are almost identical in results. Algorithm #2 and the Control show no significant differences. It would appear that all algorithms are much better at placing players into the Lowest and Highest brackets, but the middle brackets are a total crap shoot.
In all, I'm not really satisfied with ANY of the results. I would like to be able to correctly place around 75% (or 12 out of 16) correctly in EACH bracket. Otherwise, it's not a very reliable method for placing people for the actual tournament. 12 out of 16 means that 2 players in every bracket have been displaced.
Any ideas out there?
:)
ReplyDeleteYour posts make my brain hurt.
ReplyDeleteLooks good to me, but I am not really worth asking. I just don't have the mathematical know-how to add anything useful.
I assume that to be guaranteed a winner you need to play until there are only one undefeated player, and to be really sure you need a league. Like sports. So with that in mind we look at a 40k tournament with 3-7 games. This clearly limits us and the question then becomes, how do we take "shortcuts"?
ReplyDeleteYour approach kind of reminds me a bit about a bucket sort. I'm assuming you have some kind of comp sci like background since you speak of programming and algorithms :)
My thinking is that a plan of attack would be to avoid a random first round. That way we should be able to eliminated branches faster. I have to ideas for that:
1. Use RankingsHQ and start all players in that order (with unranked players randomized, and equally ranked players randomized).
2. Go through a bunch of big previous tournaments and rank the armies according to win/loss ratios (assuming that a brand new codex has a 100% win ratio for initial seeding). Rank players according to their armies.
I have *no* idea on how to simulate these two approaches per se, but I think that sometimes you just have to use heuristics and live in our imperfect world :)
(As a side note, have you played with genetic algorithms for this?)
I think that the biggest hold up is lack of enough rounds to get a statistically correct answer. Given enough pre-rounds, we would certainly, eventually settle on the correct result. I'm thinking that basically, 3-4 games just simply isn't enough to properly rank everyone. 3-4 rounds wouldn't be enough for a genetic algorithm to properly breed us an answer either because you only got 3-4 generations. I'm not sure how many rounds you would have to go statistically to derive an appropriate answer, but the results here are basically showing that 4 is not enough.
ReplyDeleteSo, yes! Short-cuts is exactly what we need. I think the issue with going to RankingsHQ or even previous events is data collection not to mention what to do with players who aren't ranked. That's going to be a huge PITA for a TO.
My next idea might be to have players vote for themselves as to the bracket they "think" they should belong to before the pre-lim. Then, run the 3-4 games and see what happens.
I was thinking of using a genetic algorithm to find the matching algorithm, but that might just be a bit too geeky:)
ReplyDeleteGoing the RankingHQ route doesn't have to be much work at all, if RHQ would add a little support for it. It would simultaneously promote RHQ. It could go something like this:
1. When you register you include your RHQ id.
2. Come game day (or a day before) the TO inserts the IDs in a special form on RHQ plus entries for anyone without an account.
3. Out comes a tournament schedule. Printable and viewable online. Presto!
Sounds manageable. I haven't looked at RHQ lately so I don't know how many are registered.
Getting people to rank themselves. Reminds me of when they asked a bunch of people if they were in the top half of drivers. For the Internet search fearful, it was well above 50% who thought they were great behind the wheel.