The idea of the veto is to ensure that there is a chance for players to remove games that will be insanely one sided.
One sided games are no fun to watch by anyone. Nor are they any fun to take part in.
The veto will ensure there is more fun to be had by players, and by anyone who happens to be watching it at the time. Not less.
With only one veto allowed each per challenge though it shouldn't be able to be abused.