synchroIt gets said every time this sort of thread pops about, but it's worth noting again: competitive TF2 is a pretty small fraction of the TF2 player base.
If every starting player of every ESEA team this season played the game today, that's still less than 1% of the daily concurrent TF2 players - which itself is less than 13% of the DotA2 players.
The reason these numbers are important are because, even at the most generous numbers for highlander players, retired players, etc., the number of competitive TF2 players is most likely still under 5% of the total TF2 player base, and competitive TF2 is not the same as regular pub TF2. The distinction here is that DotA, LoL, SC2, etc. all have the same game being played between pub games and comp. games.
So that being said, the typical person watching a stream of a game is probably(and this part is totally my own assumption) a player of that game, or at least somebody interested in playing it. This means that less than 5% of the total TF2 player base (which, again, is tiny compared to most other popular streaming games) is probably even interested in these streams to begin with. As you said, LANs and such change this a little bit, but the unfortunate fact of the matter is simply that the viewers just don't exist. Sad, but true.
But the thing that really bugs me is how the pub streams are virtually non-existant, 90% or more of stream viewers on tf2 are on competitive streams. I know the competitive community is a small fraction of the whole playerbase but still, it is really odd not to have consistent or big pub streamers, STAR_ was streaming the other day, I saw he had around 1k stream viewers, a laughable number compared to the 300k subscribers he has on youtube.
The competitive community would benefit a lot from pub streamers putting the game out there and making it noticeable for other players as well.
TF2 having a tenth on CS:GO numbers in viewers despite having almost double the playerbase is strange at minimum, even if the competitive is completely different from pub, its still strange there are no popular casual pub streamers.
If our streams have really "sticky" content as john said, I can't really get why we aren't seeing bigger and more noticeable increases in accordance to the amount of content that is being put out there, which is definitely a lot bigger than some years or even months ago.