I was looking into this same exact thing with 640x360, 960x540, and 1024x576 a few days ago. I'm running Intel integrated graphics, so custom res' will probably be more flexible for you.
Basically, you'll only be able to run fullscreen resolutions that your monitor recognizes, but even then there's no guarantee that it'll display correctly. Look into your monitor's 'available modes', and you'll have to either work with whatever's in there or add your res to it & restart your pc.
Edit: can confirm 896 x 504 works in tf2 http://steamcommunity.com/profiles/76561198027288558/screenshot/645503094103886503 just gotta add that res to your monitor's list of modes to run in, or else the game won't recognize it. Also make sure you're playing on a 16:9 monitor, playing on a 4:3 monitor and wanting black bars won't work.