There's always gaffer tape. Gaffer tape is similar to duct tape, and often confused with duct tape. For many uses, gaffer tape is a much superior product: it's cloth tape rather than plastic tape, so it tears much more easily while offering excellent strength, and it has a synthetic adhesive, so usually when the tape is peeled off there is no adhesive residue left behind. The downside of gaffer tape is that it costs much more than duct tape. For those in the US, Guitar Center stocks gaffer tape for about $10 and up. I probably wouldn't buy a roll just to strap a pump to a bike, but once I bought a roll I found all sorts of other uses for it, such as taping drawers shut when a toddler was visiting, where duct tape just wouldn't have been acceptable.
As a bonus, gaffer-taping a mini-pump to the frame is consistent with The Rules, according to Rule 30, "when no C02 cannisters [sic] are available and your pockets are full of spare kit and energy gels." (I'm skeptical about the picture actually, because that looks like duct tape rather than gaffer tape.)