i once had the problem that my graphics card would clock down to like 500MHz because usage with chris frames was only 20%. This made for a really stuttery experience.
You surely are cpu limited as well so install something like msi afterburner and monitor you gpu clock. If it's too low turn on some graphcis settings.
Also, core affinity. It might work