No I don't mind them in party/sports/racing games either.
I don't think most people would either but Sega's insistence in putting them in the main Sonic game series, making them talk a loads of shite and them sometimes they even let you control them when you just want to be playing as sonic.
Waluigi has never had his own game. He is just a roster filler. He stays out of the way.
Wario has a successful and imaginative platforming and mini games series.
The babies, as annoying as they are where used in Yoshies Island, a game loved by many. Though they tend to be relegated to roster filler status now.
Toads are just Mushroom Kingdom NPC's that give out gameplay hints usually.
Generally as rubbish as some of Nintendo's characters can be....they are generally inoffensive, stay out of the way and don't talk.
In comparison to
Shadow has a terrible game spin off.
Silver is a shitty character from one of the worst games ever made.
And SEGA insists on hiring voice actors and paying them to read mind numbing drivel.
It's no wonder people don't like them.
Not all of them are bad I agree, I mean Blaze the Cat was a good addition in sonic Rush, mainly because she just offered a different style of play, when used in that context I am totally fine with new additions.
(also cart limitations ment very little use of voices!)
They havn't been as bad recently with Generations, Colours and even Sonic 4.
They just went through a pretty terrible patch last gen and the start if this gen with the quality of their games and charachters.