Hi Drew,
I guess I'm not following what you are trying to do. Why would you want to remove a vote?
The votes are stored in the SurveyOptions table.
The tracking as to if someone has voted or not is stored in a cookie if that option is chosen in module setttings or in the Profile table using the Personalization object if the "1 Vote/Registered User" method is chosen.
Using a cookie will allow anonymous user's to vote, but they can defeat the one vote per user if they delete the cookie.
Using the 1 Vote/Registered User option will store the users vote in the DB so it is more accurate and secure way of making sure that each user only gets one vote.
Keep in mind also that if the Survey has expired, then you will only see results after that, because no more votes can be taken. The expiration for the survey is set in the module settings at the same place as the Vote Tracking.
Also realize that each module is a "Survey", and that you can have multiple questions on one survey. The action says "Add Survey", but you are really just adding questions to that survey.
So, if you want to start a new survey, you should add another module. I know this isn't the best design, but that is where it's at right now.