I have a whole lot of photos to work my way through from yesterdays Sur5al Worlds, but I havent been able to settle down to it today. Somethings been troubling me about the NZRDA rankings. Ive felt from the start that there was something wrong with the algorithm, but I just couldnt put my finger on it. I figured it must work, because its based on the WFTDA system, which seems to be working for the US/International game. Except that I now realise that it doesnt. What was troubling me was the very low ranking scores some teams have. At the bottom of the list, Whenua Fatales Roller Derby League have a score of 20. Given the way the NZRDA algorithm works, a team with what should be an average rating of 150 would need to outscore the Fatales 7.5 to 1 in a regular game in order to maintain their rating. I doubt any team in New Zealand could achieve that. In fact, my own algorithm predicts that even top-of-the-table Pirate City Rollers, playing with home track advantage, would only beat them to the tune of about 5.5 to 1 (a scoreline of perhaps 275 to 50) and the median team by 1.6 to 1, give or take, (a scoreline of say 240 to 150). Pirates might manage to take them by 7.5 to 1 on a good day, but to maintain their own rating, PCR would have to win by a whopping 12.6 to 1. In short, the Fatales NZRDA ranking makes them unplayable for any team that wants to maintain their ranking. This morning I finally cracked what was wrong. The Game Weights. The NZRDA ranking algorithm is fundamentally broken because a) it includes Game Weight in the wrong place (as does the WFTDA algorithm) and b) it weights short games below 1 - and not just below 1, way, way down at 0.5 (whereas the WFTDA rankings use only weights from 1 to 1.35, which reduces and somewhat masks the damage done by (a)). Because Game Weight is included on both sides of the calculation of Game Points, the total points awarded for a game dont add up to a balanced adjustment to team Rankings. This means that a yet-to-be-established tournament game gives out more points than the teams involved are worth - playing in a tournament will likely increase your ranking even if you do badly, while a short bout is worth only half the points the teams involved are worth - both teams will almost certainly lose rankings points. If PCR were to play WFRDL in a short bout they would have to win by over 25 to 1 to maintain their ranking! And if they did, the Fatales ranking would be absolutely shredded, instead of also being maintained. In short, playing short games kills your ranking and it looks like thats what has happened to the Fatales and Bay City Rollers. If the algorithm doesnt change, this spells the end of short-bout multi-header mini-tournaments, which have up to now been a major feature of the New Zealand roller derby calendar. So how should it work? Well its fair enough that short games should be worth less than full games, and that tournament games should be worth more. But the weight shouldnt be applied to the Game Score, which is a measure of the relative strength of the teams involved, it should be applied to the impact the Game Score (relative strength) has in the (Weighted) Average Game Score (overall ranking). That is essentially how my own algorithm works. My algorithm also differs from the NZRDA system in that its a relative/multiply system instead of an absolute/add and average system, but either approach should be more-or-less sound in that respect, if properly implemented. I think the proportional-shares-of-3-points element is probably not quite right too, but its not nearly as broken as the weights issue.
Posted on: Sat, 11 Oct 2014 22:51:55 +0000