happens to me too... annoying as fuck

Have you tried another game? Our is it just TF2?

I had the exact problem and tried a number of things before I worked out a fix, which turned out to be very simple after spending a lot of time on this!

1) Make sure your config.cfg isn't set to read-only.
2) Open TF2, go into advanced options etc. and change something. i.e. I changed 'automatically rezoom sniper rifle'.

It will rewrite your config file, and for whatever reason this fixes it. Has worked for me and others with the same issue.

Note: I think I did the initial create local server/join server before I did this, but pretty sure it's not necessary.

That seems to have fixed it for now. Will keep an eye on it for a little while.

Thanks