WariIt definitely feeds into what you noted about more spectator-friendly games though. It's exciting to see who's the best at Quake, but it definitely gets stale seeing the same 2-3 players destroy anyone who even tries to take them down, because they've had at least a 20 year headstart. It'd be nice if all competitive games could have unlimited skill ceilings and actual "Who's gonna win?" competition at the top level, but it's not that possible.
I think it's quite possible, the sport just has to be really popular over a long time. In chess, top super Grandmasters (e.g. 2750+ peak rating) barely ever lose even against other GMs, and still there's many of them and constantly new people reaching this level. If you don't count Carlsen obliterating everybody, it's pretty close to the ideal situation you described.
So I guess all that needs to happen is tf2 to stay popular with millions of players for thousands of years