i think the biggest problem with HL is how it's advertised as the "intro 2 competitiv (4 nubes!!)" and therefore causes players new to comp to go there. many of these players are probably turned off by how shite HL is or how toxic they are towards 6s (personal experience here), and might get a negative impression of competitive and ignore it completely. while i can't speak for everyone, my first experience with comp tf2 was like this, and i only started caring about actual competitive tf2 when I saw b4nny's stream on r/tf2, and played a few lobbies after seeing how much better 6s looked. even before i played HL, i had a terrible impression of 6s simply because of the stigma attached to 6s (wrongfully so) by the pubbers, r/tf2, and the sort
while i think this is becoming better now, i believe when people say they want HL to be killed off they really want the negative effects on 6s/actual competitive that it brings about by tricking pubbers into thinking they can play like their pubs in comp to be taken care of