Steam groups are pretty much pointless honestly, as someone who made one, i stopped using it because people kept messaging me to stop spamming them.
That being said, there is absolutely no reason not to follow every streamer. Every time I find a new tf2 streamer I follow them. What people don't seem to get is you can turn off email notifications, so you aren't being spammed. Let me say that again in bold print so that people actually read it because it was in Slin's OP at the bottom and posted by other people, and yet there are still people who don't get it. YOU CAN TURN OFF EMAIL NOTIFICATIONS FROM TWITCH AND YOU WON'T GET SPAMMED and then nothing happens to you and all a follow does is helps a streamer out. That's it. You are in no way affected. I honestly don't even remember half the people i follow, but when they pop up on my twitch page, I usually tune in to check it out.
So please, just turn off email notification, and you're life won't be changed in the slightest.