So after having tested all that, if found that @AElai script was working wonders.
Although, I would like to know if it s possible for me to change the hidden viewmodel slot depending on the class I'm playing.
IE, for scout, I would turn off primary and leave secondary on, but for soldier, I would have slot1 and 3 on and slot 2 turned off.
Is it just a matter of changing the order of those values only ?
----
alias qs_12 "slot1;slot2;tog_vm12; alias tog_vm12 incrementvar r_drawviewmodel 0 1 1; alias tog_vm13 r_drawviewmodel 1"
alias qs_13 "slot1;slot3;tog_vm13; alias tog_vm13 incrementvar r_drawviewmodel 0 1 1; alias tog_vm12 r_drawviewmodel 1"
bind mouse5 qs_12
bind mouse4 qs_13
----
Thanks again to everyone who posted :)