I have flown around balloons before, it is cool, but I couldn't tell if they were waving friendly waves at me, or trying to get me to go away, for fear of my rotorblades chopping their balloons to shreds.
There is a guy around here that has a single seat balloon, it is pretty cool.