I've learned quite a few things about TF2 stats over the last few years which have never seen the light of day and which nobody else uses. I'm gonna dump this knowledge before the game is completely dead in the occasional thread.
The first one is a stat that deals with a couple of outstanding problems with kill and damage stats. Currently the commonly used 2 are a variation on a ratio like KD or KAD, and the damage output stat DPM. Both of these have problems, the ratio stats can be enormously inflated by passive play. Take the following two example players:
Kills Deaths KD
Plyr A 15 5 3
Plyr B 3 1 3
Clearly Player A has had a far bigger impact on the game here but the cowardly Player B who hid the whole game and just picked off a few players has exactly the same KD suggesting they have achieved something equally important. This is an extreme example, and for the most part players in competitive matches don't tend to do this, but when compiling event statistics there are always less effective players that pop up near the top because they play conservatively.
Because players tend not to milk their KD too hard though, it does make it a pretty good stat to compare to because it's fairly strong as a win/loss indicator. If something happened like players being made aware that they were being ranked on KD in some way then they might start to try to distort it.
Similarly with DPM a player that religiously spams chokes and corners will pick up a ton of extra damage that doesn't really contribute anything particularly significant to their team, and probably helps their opponents build uber. A player that only causes damage when taking fights has more meaningful damage stats, and the crossbow means that damage built up from spamming almost never sticks.
My goal was to find something that had more relevance to the result, a performance stat that had a clear mathematical correlation to the final result. The first attempt was moderately successful, taking the logic that high damage meant that a player was being more aggressive (even spamming chokes requires a player to be positioned slightly riskily) then combining that with KD would produce something useful.
Attempt 1: DPM * KD
This was OK, there's a tendency to favour heal heavy spamming pockets and demos that build up damage, but if they're not winning fights they're also punished hard. Similarly it also sidelines scouts a little because their damage stats tend not to be competitive with the spamming classes. It also has a reasonable correlation to games won and lost. The problem with this isn't really that it's particularly inaccurate, it's that it's conceptual basis doesn't describe a game winning advantage. Damage is something that happens by playing the game, and it's being modified to link it to something relevant.
Returning to basic concepts, most games have 3 elements to consider when modelling them - material, position and time. In TF2 5cp or koth material is largely dominant, having more players is almost always a big deal, and time and positional considerations only decisively trump those in niche situations. And also we don't have stats for positional considerations from logs, so there's nothing we can do for that anyway.
So if we're thinking about material, how does that manifest itself in TF2? Teams are denied material (players, any charged up abilities over time like uber and banners, etc) by players being killed and sent to the spawn queue. So we're really looking for a player stat that reflects how much extra time on the field they generated for their team with a player advantage in the game, and this is kills. Because their own frag counts against their team, it's a net frags statistic.
Net Frags per minute: (Kills - Deaths) / Time Played
This looks extremely simple, almost childishly so, but the simple +/- approach creates something that links more clearly to in-game advantages than the other ratios and secondary activity stats do, and has a stronger link to wins and losses.
So if we return to our example players
Kills Deaths Net Kills
Plyr A 15 5 +10
Plyr B 3 1 +2
Player A's vast additional contribution to the game is now reflected here. Making the stat per minute makes it possible to compare time spent off classing, and compare from game to game, and it is very simple to read. Statistically it correlates with win loss the best from the candidate stats including KD.
It has some obvious weaknesses. It's susceptible to kill stealing, there are some situations where it's right to die, and a player that baits out their team can still inflate their personal stats at their team's expense. However it doesn't promote meaningless spam or passive play, in fact it promotes finding the maximum point of aggression you can without compromising survivability and working with your team. It also tends to reward scouts which seems right as they're broadly considered the most powerful class in 6v6.
If we look at a real world example http://logs.tf/1739763 pot8o is on +12 whereas YppY is on +8, they're only separated by 0.2 in KD but pot8o is a clear 50% ahead on Net Kills. Given pot8o was doing this from the low heals flank scout position it's a creditable performance. Much of it may have been cleanup work from the sacrifice of others but if he doesn't hit those shots then they lose the game. He also had a solid stint on sniper.
When looking at logs and considering the net kills (and per minute when looking at class breakdowns) I've found it does help to throw a new light on things and provides a solid link to the in-game advantages a player achieved.
I've learned quite a few things about TF2 stats over the last few years which have never seen the light of day and which nobody else uses. I'm gonna dump this knowledge before the game is completely dead in the occasional thread.
The first one is a stat that deals with a couple of outstanding problems with kill and damage stats. Currently the commonly used 2 are a variation on a ratio like KD or KAD, and the damage output stat DPM. Both of these have problems, the ratio stats can be enormously inflated by passive play. Take the following two example players:
[code]
Kills Deaths KD
Plyr A 15 5 3
Plyr B 3 1 3
[/code]
Clearly Player A has had a far bigger impact on the game here but the cowardly Player B who hid the whole game and just picked off a few players has exactly the same KD suggesting they have achieved something equally important. This is an extreme example, and for the most part players in competitive matches don't tend to do this, but when compiling event statistics there are always less effective players that pop up near the top because they play conservatively.
Because players tend not to milk their KD too hard though, it does make it a pretty good stat to compare to because it's fairly strong as a win/loss indicator. If something happened like players being made aware that they were being ranked on KD in some way then they might start to try to distort it.
Similarly with DPM a player that religiously spams chokes and corners will pick up a ton of extra damage that doesn't really contribute anything particularly significant to their team, and probably helps their opponents build uber. A player that only causes damage when taking fights has more meaningful damage stats, and the crossbow means that damage built up from spamming almost never sticks.
My goal was to find something that had more relevance to the result, a performance stat that had a clear mathematical correlation to the final result. The first attempt was moderately successful, taking the logic that high damage meant that a player was being more aggressive (even spamming chokes requires a player to be positioned slightly riskily) then combining that with KD would produce something useful.
[b]Attempt 1: DPM * KD[/b]
This was OK, there's a tendency to favour heal heavy spamming pockets and demos that build up damage, but if they're not winning fights they're also punished hard. Similarly it also sidelines scouts a little because their damage stats tend not to be competitive with the spamming classes. It also has a reasonable correlation to games won and lost. The problem with this isn't really that it's particularly inaccurate, it's that it's conceptual basis doesn't describe a game winning advantage. Damage is something that happens by playing the game, and it's being modified to link it to something relevant.
Returning to basic concepts, most games have 3 elements to consider when modelling them - material, position and time. In TF2 5cp or koth material is largely dominant, having more players is almost always a big deal, and time and positional considerations only decisively trump those in niche situations. And also we don't have stats for positional considerations from logs, so there's nothing we can do for that anyway.
So if we're thinking about material, how does that manifest itself in TF2? Teams are denied material (players, any charged up abilities over time like uber and banners, etc) by players being killed and sent to the spawn queue. So we're really looking for a player stat that reflects how much extra time on the field they generated for their team with a player advantage in the game, and this is kills. Because their own frag counts against their team, it's a net frags statistic.
[b]Net Frags per minute: (Kills - Deaths) / Time Played[/b]
This looks extremely simple, almost childishly so, but the simple +/- approach creates something that links more clearly to in-game advantages than the other ratios and secondary activity stats do, and has a stronger link to wins and losses.
So if we return to our example players
[code]
Kills Deaths Net Kills
Plyr A 15 5 +10
Plyr B 3 1 +2
[/code]
Player A's vast additional contribution to the game is now reflected here. Making the stat per minute makes it possible to compare time spent off classing, and compare from game to game, and it is very simple to read. Statistically it correlates with win loss the best from the candidate stats including KD.
It has some obvious weaknesses. It's susceptible to kill stealing, there are some situations where it's right to die, and a player that baits out their team can still inflate their personal stats at their team's expense. However it doesn't promote meaningless spam or passive play, in fact it promotes finding the maximum point of aggression you can without compromising survivability and working with your team. It also tends to reward scouts which seems right as they're broadly considered the most powerful class in 6v6.
If we look at a real world example http://logs.tf/1739763 pot8o is on +12 whereas YppY is on +8, they're only separated by 0.2 in KD but pot8o is a clear 50% ahead on Net Kills. Given pot8o was doing this from the low heals flank scout position it's a creditable performance. Much of it may have been cleanup work from the sacrifice of others but if he doesn't hit those shots then they lose the game. He also had a solid stint on sniper.
When looking at logs and considering the net kills (and per minute when looking at class breakdowns) I've found it does help to throw a new light on things and provides a solid link to the in-game advantages a player achieved.
Thought the thread is gonna be about hidden weapon stats. Good thing that it's not.
Thought the thread is gonna be about hidden weapon stats. Good thing that it's not.
does this mean I can make excuses for getting 150 dpm on product because if so please keep doing this.
does this mean I can make excuses for getting 150 dpm on product because if so please keep doing this.
Cyanicdoes this mean I can make excuses for getting 150 dpm on product because if so please keep doing this.
You should say you only cause damage that matters and point to your fabulous positive net kills statistics
Pro tip: If your net kills are also shit, change the subject
[quote=Cyanic]does this mean I can make excuses for getting 150 dpm on product because if so please keep doing this.[/quote]
You should say you only cause damage that matters and point to your fabulous positive net kills statistics
Pro tip: If your net kills are also shit, change the subject
Disregard K/D or medpicks. On roamer the most important stat is health packs. Denying the enemy team heals and making your own medic's job easier clearly shows that you are a very smart teamplayer.
Disregard K/D or medpicks. On roamer the most important stat is health packs. Denying the enemy team heals and making your own medic's job easier clearly shows that you are a very smart teamplayer.
A stat that i always felt like it could be interesting to see is damage done/heals received.
It would shows how efficient a player is and help the meds to chose a different healing order.
The downside is that uber building could really fuck this up, but it would still be great to see it for the other players
A stat that i always felt like it could be interesting to see is [i]damage done/heals received[/i].
It would shows how efficient a player is and help the meds to chose a different healing order.
The downside is that uber building could really fuck this up, but it would still be great to see it for the other players
cirloA stat that i always felt like it could be interesting to see is damage done/heals received.
It would shows how efficient a player is and help the meds to chose a different healing order.
The downside is that uber building could really fuck this up, but it would still be great to see it for the other players
Damage done/ Damage taken might fix this
[quote=cirlo]A stat that i always felt like it could be interesting to see is [i]damage done/heals received[/i].
It would shows how efficient a player is and help the meds to chose a different healing order.
The downside is that uber building could really fuck this up, but it would still be great to see it for the other players[/quote]
Damage done/ Damage taken might fix this
The comparisons I use to judge player performances when looking at logs are usually damage done vs damage taken, heals received and deaths (net damage is a decent indicator for performance in team fights for most roles, maybe not short leash pocket; heals received is a good indicator of how many resources they use; and deaths is obviously a good indicator of the proportion of time they're affecting the game as far as stats go). This gives a bit more context to their performance, and then I'll try to build in my knowledge of player styles and tendencies to gauge how they performed according to their style.
I think this works pretty well with the exception of high level demo, where a lot more impact in slow games can be had with purely area denial - I've found it's often the case in structured games that good demo performances aren't always reflected in the stats.
Edit: This doesn't work well with offclassing, but I don't use it for statistical analysis in the same way you would Jon :D
The comparisons I use to judge player performances when looking at logs are usually damage done vs damage taken, heals received and deaths (net damage is a decent indicator for performance in team fights for most roles, maybe not short leash pocket; heals received is a good indicator of how many resources they use; and deaths is obviously a good indicator of the proportion of time they're affecting the game as far as stats go). This gives a bit more context to their performance, and then I'll try to build in my knowledge of player styles and tendencies to gauge how they performed according to their style.
I think this works pretty well with the exception of high level demo, where a lot more impact in slow games can be had with purely area denial - I've found it's often the case in structured games that good demo performances aren't always reflected in the stats.
Edit: This doesn't work well with offclassing, but I don't use it for statistical analysis in the same way you would Jon :D
nuzeI think this works pretty well with the exception of high level demo, where a lot more impact in slow games can be had with purely area denial - I've found it's often the case in structured games that good demo performances aren't always reflected in the stats.
If you want the worst example look at Mike playing pocket for the pre i49 Epsilon. Adjusting for healing in a linear way makes him the worst player in the league because his medic almost never died. He received so much healing the possible output can simply never keep up. Epsilon basically hit the rate limit for killing they could output in a game of TF2.
Essentially healing is a diminishing return for pocket players, the more they get the less effect it has on what they can produce. This would need a polynomial equation to model at the very least. Another problem is that it's also a two way relationship, do they get more healing because they're a great player killing all enemies, or do they get more because they're being healed by Mirelin and nobody can kill him? And of course it's susceptible to the pace of the game. Around i49 it was brutally fast with every advantage pushed to the limit. Now there are long stretches of healing that produce no kills or even much damage at all.
These are the kind of things your brain can fuzzily just take into account in a vague way when assessing a log, but producing something concrete that can reliably create meaningful comparisons gets very complicated very quickly.
[quote=nuze]I think this works pretty well with the exception of high level demo, where a lot more impact in slow games can be had with purely area denial - I've found it's often the case in structured games that good demo performances aren't always reflected in the stats.[/quote]
If you want the worst example look at Mike playing pocket for the pre i49 Epsilon. Adjusting for healing in a linear way makes him the worst player in the league because his medic almost never died. He received so much healing the possible output can simply never keep up. Epsilon basically hit the rate limit for killing they could output in a game of TF2.
Essentially healing is a diminishing return for pocket players, the more they get the less effect it has on what they can produce. This would need a polynomial equation to model at the very least. Another problem is that it's also a two way relationship, do they get more healing because they're a great player killing all enemies, or do they get more because they're being healed by Mirelin and nobody can kill him? And of course it's susceptible to the pace of the game. Around i49 it was brutally fast with every advantage pushed to the limit. Now there are long stretches of healing that produce no kills or even much damage at all.
These are the kind of things your brain can fuzzily just take into account in a vague way when assessing a log, but producing something concrete that can reliably create meaningful comparisons gets very complicated very quickly.
Does this mean valve has been thinking about comp this whole time
Does this mean valve has been thinking about comp this whole time
How do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can
How do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can
Its very easy to determine how much space was created, the best roamer is the one with the most deaths at the end of the map.
Its very easy to determine how much space was created, the best roamer is the one with the most deaths at the end of the map.
I'd say the best one is the one who could best sketch -from memory -the door they're meant to have been watching all game
I'd say the best one is the one who could best sketch -from memory -the door they're meant to have been watching all game
AdebisiHow do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can
I imagine he'd survive and get kills if he could, and probably doesn't play as deliberately uselessly as suggested. I'd compare them to other roamers.
[quote=Adebisi]How do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can[/quote]
I imagine he'd survive and get kills if he could, and probably doesn't play as deliberately uselessly as suggested. I'd compare them to other roamers.
GentlemanJonAdebisiHow do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can
I imagine he'd survive and get kills if he could, and probably doesn't play as deliberately uselessly as suggested. I'd compare them to other roamers.
I don't think it is useless, just intangible.
[quote=GentlemanJon][quote=Adebisi]How do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can[/quote]
I imagine he'd survive and get kills if he could, and probably doesn't play as deliberately uselessly as suggested. I'd compare them to other roamers.[/quote]
I don't think it is useless, just intangible.
AdebisiI don't think it is useless, just intangible.
Missing everything every single time you bomb would be useless. He doesn't do that, and seeing as it's based on numbers, I don't really have a model for it
[quote=Adebisi]I don't think it is useless, just intangible.[/quote]
Missing everything every single time you bomb would be useless. He doesn't do that, and seeing as it's based on numbers, I don't really have a model for it
GentlemanJonAdebisiI don't think it is useless, just intangible.
Missing everything every single time you bomb would be useless. He doesn't do that, and seeing as it's based on numbers, I don't really have a model for it
Presumably not literally. But probably more than other roamers who statistically look better.
I guess you can't quantify distraction
[quote=GentlemanJon][quote=Adebisi]I don't think it is useless, just intangible.[/quote]
Missing everything every single time you bomb would be useless. He doesn't do that, and seeing as it's based on numbers, I don't really have a model for it[/quote]
Presumably not literally. But probably more than other roamers who statistically look better.
I guess you can't quantify distraction
great post! i've always despised logs for their inability to show overall effectiveness
great post! i've always despised logs for their inability to show overall effectiveness
AdebisiHow do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can
You don't really need to die to make space, you can just strafe in a loop or jump in and jump back instantly just to have the enemy look at you for a while and take the focus out of your team. If you just jump in the middle of the enemy's team and die instantly without even dealing good damage for your team to follow up it's just not worth it.
[quote=Adebisi]How do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can[/quote]
You don't really need to die to make space, you can just strafe in a loop or jump in and jump back instantly just to have the enemy look at you for a while and take the focus out of your team. If you just jump in the middle of the enemy's team and die instantly without even dealing good damage for your team to follow up it's just not worth it.
AmarokAdebisiHow do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can
You don't really need to die to make space, you can just strafe in a loop or jump in and jump back instantly just to have the enemy look at you for a while and take the focus out of your team. If you just jump in the middle of the enemy's team and die instantly without even dealing good damage for your team to follow up it's just not worth it.
I didn't say anything about dying.
[quote=Amarok][quote=Adebisi]How do you account for players like Muuq who, by his own admission, bomb in, miss everything, but make a tonne of space for others ?
Genuine question. I don't think you can[/quote]
You don't really need to die to make space, you can just strafe in a loop or jump in and jump back instantly just to have the enemy look at you for a while and take the focus out of your team. If you just jump in the middle of the enemy's team and die instantly without even dealing good damage for your team to follow up it's just not worth it.[/quote]
I didn't say anything about dying.
AdebisiPresumably not literally.
That's the problem when you're doing data analysis, non-literal descriptions aren't really going to have any relationship to the information available.
Given there's no way to clearly detect a bomb from the logs and I'm not going to get into parsing demos, I'd probably do something like look at the result of team fights x seconds after any given time he died in a certain situation - all players up or whatever - and see if there was something there, see if his team mates had better performance compared to other times the teams were equal and he hadn't just died.
If the space created doesn't produce clearly better performance for his team it's not worth it. It's would be very noisy data though so you probably wouldn't get much, you could do something blunter like compare team stats like net kills when just the roamer was down and look at who is performing best in what is presumably the post-bomb world.
There's a few other ways to approach it that aren't possible in TF2 for other reasons.
[quote=Adebisi]Presumably not literally. [/quote]
That's the problem when you're doing data analysis, non-literal descriptions aren't really going to have any relationship to the information available.
Given there's no way to clearly detect a bomb from the logs and I'm not going to get into parsing demos, I'd probably do something like look at the result of team fights x seconds after any given time he died in a certain situation - all players up or whatever - and see if there was something there, see if his team mates had better performance compared to other times the teams were equal and he hadn't just died.
If the space created doesn't produce clearly better performance for his team it's not worth it. It's would be very noisy data though so you probably wouldn't get much, you could do something blunter like compare team stats like net kills when just the roamer was down and look at who is performing best in what is presumably the post-bomb world.
There's a few other ways to approach it that aren't possible in TF2 for other reasons.
Cheers. I was just interested to see what you'd do.
I'm an econometrician, as it happens
Cheers. I was just interested to see what you'd do.
I'm an econometrician, as it happens
AdebisiCheers. I was just interested to see what you'd do.
I'm an econometrician, as it happens
Well, what I would really do is point to Mike posting pocket soldier numbers from the roamer role and tell them to get good
[quote=Adebisi]Cheers. I was just interested to see what you'd do.
I'm an econometrician, as it happens[/quote]
Well, what I would really do is point to Mike posting pocket soldier numbers from the roamer role and tell them to get good
#22 thinking about it some more, it might be possible to build up some kind of profile matching log events to stv observations to reliably identify roamer initiations that don't result in their death so you can include times it's just a distraction. Logs don't even include positional information for damage though so it's hard to imagine it being possible.
Ideally you could do a basketball style +/- which would just absorb all that unrecorded activity into a general contribution to positive achievements like capping points, but TF2 players are never substituted so it doesn't really work. It might be possible with offclasses which muuki would be particularly useful for. Adjusting for lopsided numbers of players would be hard though.
I've also considered making a win probability model that looks at things like score, time left, territory owned and players up and calculates win chances, and therefore the win value of any action players take. Not much point investing that kind of time given the way the game is going though.
#22 thinking about it some more, it might be possible to build up some kind of profile matching log events to stv observations to reliably identify roamer initiations that don't result in their death so you can include times it's just a distraction. Logs don't even include positional information for damage though so it's hard to imagine it being possible.
Ideally you could do a basketball style +/- which would just absorb all that unrecorded activity into a general contribution to positive achievements like capping points, but TF2 players are never substituted so it doesn't really work. It might be possible with offclasses which muuki would be particularly useful for. Adjusting for lopsided numbers of players would be hard though.
I've also considered making a win probability model that looks at things like score, time left, territory owned and players up and calculates win chances, and therefore the win value of any action players take. Not much point investing that kind of time given the way the game is going though.
GentlemanJon#22 thinking about it some more, it might be possible to build up some kind of profile matching log events to stv observations to reliably identify roamer initiations that don't result in their death so you can include times it's just a distraction. Logs don't even include positional information for damage though so it's hard to imagine it being possible.
Ideally you could do a basketball style +/- which would just absorb all that unrecorded activity into a general contribution to positive achievements like capping points, but TF2 players are never substituted so it doesn't really work. It might be possible with offclasses which muuki would be particularly useful for. Adjusting for lopsided numbers of players would be hard though.
I've also considered making a win probability model that looks at things like score, time left, territory owned and players up and calculates win chances, and therefore the win value of any action players take. Not much point investing that kind of time given the way the game is going though.
those are some fucking nutty statistics but i think that's going a bit too in depth. does any other game have stats this complicated?
[quote=GentlemanJon]#22 thinking about it some more, it might be possible to build up some kind of profile matching log events to stv observations to reliably identify roamer initiations that don't result in their death so you can include times it's just a distraction. Logs don't even include positional information for damage though so it's hard to imagine it being possible.
Ideally you could do a basketball style +/- which would just absorb all that unrecorded activity into a general contribution to positive achievements like capping points, but TF2 players are never substituted so it doesn't really work. It might be possible with offclasses which muuki would be particularly useful for. Adjusting for lopsided numbers of players would be hard though.
I've also considered making a win probability model that looks at things like score, time left, territory owned and players up and calculates win chances, and therefore the win value of any action players take. Not much point investing that kind of time given the way the game is going though.[/quote]
those are some fucking nutty statistics but i think that's going a bit too in depth. does any other game have stats this complicated?
Cyanicthose are some fucking nutty statistics but i think that's going a bit too in depth. does any other game have stats this complicated?
They might do really soon....
[quote=Cyanic]those are some fucking nutty statistics but i think that's going a bit too in depth. does any other game have stats this complicated?[/quote]
They might do really soon....
your next challenge is to come up with a WAR stat gjon
your next challenge is to come up with a WAR stat gjon