What class are they giving up for the heavy? Scout or soldier?
The main thing you want to keep in mind is that they are going to be much slower. If you know the heavy is at choke, then push through flank, and vice versa. The heavy will be really slow to rotate, which means you should be able to overwhelm whatever classes you find there.
Also, the enemy team will generally have to either tank heals on the heavy (leaving all the other classes without significant buffs), or else he won't be healed at all. If you uber into the combo, you should focus the heavy right away with everyone, because once he's dead, the rest of their team should be unbuffed and you can kill them. If you do what I said before, and rotate to the other side of where the heavy is holding, then you can focus down all the other classes which aren't being healed with your combo, and then focus down the heavy last.
Also, someone once told me that heavies are great for winning you fights that you were already winning, and losing fights that you were already losing. Just don't let them dictate the play with their heavy, take control of the match for yourselves. Same goes if a team runs full time sniper.