Tiered Competitive Event Part 3b

Still trying to figure out the best way to group like skilled players prior to an actual tournament! Last time, I ran some algorithms which would attempt to place 64 players into appropriate brackets (say a Bronze, Silver, Gold, and Platinum) based on the skill level of the player. The idea of course is to put like skilled players together into brackets of 16 whereupon they can compete to win in the bracket with players close to their skills.

Read the previous post to see what the 4 algorithms used were. Basically put, I wasn't really satisfied with any of it!

So, I decided to try a new algorithm this week to see what my results would look like as well as a serious modification to the old algorithms to try and get better numbers.

In a nutshell, here's how it works! Take 64 people and rate them from 1 to 64, where 1 is the worst player in the group and 64 the best. The idea is, if our preliminary matching works, when it's time for the actual tournament, players 1 - 16 will be in the bronze bracket, 17 - 32 will be in the silver, 33 - 48 in the gold, and 49-64 in the platinum. If it fails, players of non-like skill will be scattered all over the brackets.

So, the first new thing I wanted to try this week was adjusting battle points based on the proximity of the players to each other in skill level. The idea here is that players are NEAR LIKE skill shouldn't end up with much of a battle point spread whereas players who are no where near each other in terms of skills should end up with a blow out. For example, if player #46 plays a game against player #50, the game should be pretty close with battle points reflecting that. Whereas, if player #60 and player #8 play each other, it should be a complete blow out where Player #60 gets full points and player #8 gets 0 points.

Using this theory, I changed the way the original 4 algorithms calculated battle points which resulted in the following: (the number in parentheses is the original % correct)

--- Percentages reflect the % of players correctly placed in that bracket
Algorithm #1:
Bronze: 80% (78.756%)
Silver: 50% (41.25%)
Gold: 58.75% (42.5%)
Platinum: 86.25% (75%)

Algorithm #2:
Bronze: 66.25 (60%)
Silver: 43.75 (35%)
Gold: 51.25 (33.75%)
Platinum: 71.25 (62.5%)

Algorithm #3:
Bronze: 78.75 (75%)
Silver: 57.5 (38.75%)
Gold: 61.25 (43.75%)
Platinum: 82.5 (76.25%)

Algorithm #4:
Bronze: 75 (65%)
Silver: 53.75 (41.25%)
Gold: 45 (42.5%)
Platinum: 66.25 (67.5%)

Here's some food for thought on this idea. First, all algorithms got better across the board, some with substantial gains in placement. This is completely expected because we are attempting to model "reality" now with battle points and not just a random occurrence. However, this "reality" has a few problems.....

..... First, we are assuming that somehow we have devised a set of missions that will lend itself to the idea of "spreading' battle points based on player skill.....

..... Second, we are assuming that players will play to their ability each game....

So, aside from those caveats, this appears to work much better than last time, but it's still not really satisfactory! Satisfactory is a 75% on average in every category!

So, I decided that I would incorporate the concept of "Player Choice" into a new algorithm we'll call Algorithm #5. Player Choice is the idea that players will have some "say" in which bracket they want to attempt to play in.

Imagine that you walk in the door of the tournament. I tell you that for Day 1, we are doing pre-lim matches that will decide placement into the actual tournament which will take place on Day 2. There will be 4 brackets, designed to group players of like skill together to ensure everyone has fun and a good chance at winning in their bracket. I tell you that before we start the pre-lim matches, I would like for you to tell me which bracket you think you fit into:

Bronze Bracket - A bracket reserved for new players or players with little to no game skill or experience. You expect to lose every game you play today!

Silver Bracket - A bracket reserved for players who know the game but don't consider themselves strong gamers. You expect to do your best today but definitely not win all your games.

Gold Bracket - A bracket reserved for players who play well but don't consider themselves "great". You might be the guy that knows the game well but doesn't consistently do well at tournaments. You expect to win most of your games today!

Platinum Bracket - A bracket reserved for players who consider themselves good players or who are looking for the challenge of playing against top gamers. You normally place well in tournaments and could go 4-0!

So, you place yourself! Thing is, I'm not so dumb as to believe that people will spontaneously or magically pick the right spot for themselves. First reason is that they don't know the other players very well. Second, people don't always do such a good job at being honest with themselves. :)

That being known, algorithm #5 takes over. First, I assign each player to a "chosen" bracket. The way this is done is by looking at the player #, where Player #1 is the worst, and Player #64 the best, and have each player pick their own category. I do this by stating that each player has a 50% chance to choose correctly, a 35% chance to pick a category that is either 1 higher OR lower than their actual placement, and a 15% chance to choose a category that is either 2 higher or 2 lower than where they actually are.

From there, we match players in each chosen bracket for 4 games. We simply, randomly match the players in each bracket, recording Battle Points using the same system as above where players of like skill are close in battle points and players with huge gaps create blow outs.

At the end, we try to determine a calculated bracket for each player. To do this, we examine their battle points which should be representative of how well they did in their chosen bracket. If they have a lot of points, we can assume they might be playing against players they don't need to be playing with. Likewise, this is also true if they don't have very many. So, if a player has less than 10 Battle Points or greater than 70 (where the max battle points to be earned each round is 20), we adjust their bracket by 2 either down or up, respectively. If they have 11-25 or 55-70, we adjust one bracket. Otherwise, they are in the bracket they chose.

Lastly, we sort by Calculated Bracket and then by total Battle Points. This gives us an order between 1 and 64 for all players. We then break them into the 4 brackets with the first 16 being in the bronze bracket, the next 16 in silver, etc.

Algorithm #5:
Bronze: 78.75
Silver: 60
Gold: 68.75
Platinum: 85

So, I looked at this, and it is better than the other algorithms but only ever so slightly. Mostly, it's better in the Silver and Gold brackets which is where we have weakness. It is worth mentioning that some data sets placed 100% and 93% in some categories which hasn't happened yet with the other algorithms.

The next thing I wanted to test was to see if maybe I wasn't so pessimistic on people's ability to place themselves. So, this time I said players have a 70% chance to pick correctly, a 20% chance to mis-place by 1 bracket, and a 10% chance to mis-place by 2 brackets.

Algorithm #5 Modified:
Bronze: 85
Silver: 73.75
Gold: 76.25
Platinum: 86.25

Wow! First time we got numbers that hit the satisfactory mark! Sweet! Problem is....

.... We are still relying on people to properly place themselves. The more accurately you place yourself, the better the matching system works. Problem is, it's complete speculation as to how well people can place themselves! What would you think people would do?.....

.... Still got the issues with battle points......