What bitrate are you using when you upload?

Lots of people in tf2 these days upload their videos at 1440p, even if it's 1080 upscaled, because youtube has a much higher bitrate allowance for 1440p videos.

For this clip I uploaded a raw clip from Shadowplay. This is what my settings look like

https://i.imgur.com/QSKt5io.png

altho it doesn't record in 1440p since I only use a 1080p monitor. Should I still render edited videos in 1440p and if so what render settings should I use?

Personally I still render in 1440p from Vegas for stuff recorded in 1080 simply for the extra bitrate allowance on youtube.

I've heard from other people that they've had the same problem with uploading a raw video from shadowplay, with max bitrate too, I think the problem may be due to how the raw video is encoded.

I suggest encoding to h265 with ffmpeg and upscaling to 1440p, don't record source files at 1440p unless that's what you're actually running at.
I use OBS replay buffer recording at 25 Mbps and the quality is visually fine, you should be able to lower your bit rate that far in shadowplay and save a lot of disk space. Could you list your specs? I can help you get a faster encode if I know what hardware you're using, CPU and GPU specifically.

If I had to guess, I'd say youtube is nuking your bitrate because your recording's bitrate is excesssively high, if you upload a file ~2500 Kbps this should be fine, and as Kermit said youtube will not nuke the quality of videos above 1080p so hard. If I remember correctly, Shadowplay does not create videos with 1440p resolution unless the source feed is actually that high, so you will need to convert it yourself.

I have a i7-7700K, 16 gigs of ram, and a GTX 1080

Yea it pretty much comes down to youtube nuking the bitrate on anything 1080p60 and lower, remember dealing with this kind of thing on some melee videos i worked on before.

If you just upscale to 1440p60 and upload it'll already look better, on top of all the things Adnurak mentioned. Youtube's compression is super intensive, raw 1080p footage looks similar in quality (usually better) compared to 4k footage uploaded and watched on youtube due to compression, and it gets even worse if the video is uploaded at 1080p. This is the reason a lot of youtubers will say to watch the videos in 4k even on 1080p monitors n such

this will produce a 1440p upscaled video of decent quality for youtube, using your GPU's nvenc capability to encode it quickly and painlessly, just substitute the timestamps for the time you want to use from the video, the path for the location of the source file, and the path to where you want to save the new one, if you want to convert the entire clip, remove the -ss and -to arguments, it will work fine without them but it's nice to be able to trim the fat and it will reduce the encode time. I much prefer ffmpeg to trim short clips for upload like this, rather than opening up beefy editing software just for a 30 second clip.

I also have this problem when uploading full length videos to my youtube channel (usually about 3-4 minutes long). Should I still encode those up to 1440p with those settings?

Well practically yes. Iirc uploading 1440p videos will force youtube to use the vp9 codec instead of avc which is what you should aim for. You can check by right clicking on a video and show stats for nerds.

yeah, I would, it improves the final result dramatically