VulcanYou can determine how hard a game is by calculating the average hours of top players. And how many hours are need to be considered "pro". Average quake hours 15,000+, average tf2 hours 8000, average csgo hours 5000. In that regard I would definitely consider quake the hardest game.
Some of the smarter people may have also seen a pattern in the way they are arranged, it seems the harder the game the less of a competitive scene it has. I think this is because harder games are more confusing and the average audience won't be able to keep up, leading to the esport not being as popular
You're forgetting that most cs pros have been playing since well before 2009 when steam started recording hours. Given that quite a few cs players have well over 5k hours just since 2009 I think its fair to put that average higher for them. Also saying that the harder the game is the smaller its scene is just needlessly jerking off tf2. Quake had one of the biggest esports scenes but its dried up and fallen behind newer games where players obviously have fewer hours, and I'd bet the people with the most hours in any game are starcraft players