Valve wants to dip their toes in the competitive market for the same reason that DotA 2 and League of Legends are so successful as competitive games.
Competitive players are more invested in their game, play the game more, and are more likely to spend money on the game. Casual players who don't enjoy playing competitively but enjoy watching it, fans of the competitive game, are also more invested in the product as a whole, and are less likely to stop playing and more likely to invest in items for it.
If they implement a lobby system that players enjoy, they'll be more invested in the game and more likely to spend money on it if they feel a drive to do well in that system as opposed to just messing around in a pub for an hour here or there.
I'm not sure any of this will have a real big effect considering the TF2 playerbase has already dropped quite a bit
TF2 is honestly set up perfectly to be an FPS esport but it's just gone so long without developer support for competition that the game has grown stale in general since everyone has played it so much. With Valve moving all their development assets to other products the process will just continue, most likely, which is a shame...
There are not any other FPS where you have intuitive concepts like roles (similar to positions on a football team), a visual style that doesn't look cheap or ugly but is easy to understand (over hyperealistic FPS games where you need to be able to discern a couple of pixels to see a guy in a bush), combined use of hitscan and projectile weapons (which is great for spectators since things like airshots are awesome). It's even got a number of easily identifiable objectives (CP points, uber) that allow commentators to break down action into something more understandable for viewers.
Valve spent too much time making the public game different than the competitive game to really salvage that, which annoys me greatly because five years later they seem to have realized the potential of esports when they were developing DotA 2. If they wanted to test some of their stuff, like tournament tickets, they could have done that in TF2 if they had ever stopped to consider since they use TF2 as their testbed anyway. Oh well.
It's true that making the competitive game more similar to pubs will be a net positive. At this point it's probably too late but considering how highlander is growing faster than 6s, maybe there's at least a little hope.