Joel,
You may have a bad blower motor, but it is rare for a blower motor to give you the symptoms that you described because the blower motor does not have overload/overheat protection built into it. If it develops an open circuit, it will just quit and rarely restart, or if it develops a short it will blow the fuse.
If the motor quits on all speeds, it is not related to the high blower relay, however it could be one of several switches, relays, or connectors that power the blower circuit. Before I replaced the blower motor, I would be certain that it still had 12V to the motor after it quit to be sure that the motor is the problem.
Gerald