We just had this discussion in ozfortress for the ultiduo cup we just held and it was about 75:25 in favor of xbow being allowed simply because the medic might as well be a spectator without it and the gamemode becomes slow as all hell.
It's undeniable that being able to arrow up hurt players allows teams to start pushes faster and that can't be a bad thing. Plus, does anyone actually want to play medic without the crossbow? It's about the only enjoyable weapon the class has. Without it you're just a no aim healbot who has to hit surfs occasionally. With it, you can make some clutch plays.
There are definitely some absolutely fucking terrible weapons currently allowed but the xbow is fun, well skill-indexed, and adds to the game. The only real reason people could want to remove it is because they want to play 2009 tf2 again. But that's never coming back.