More expensive brake pads will have a wear line. Replace these when you hit the line. Not so expensive ones should be replaced when the protrusions from the main block have been worn away.
Another decent way to tell is by how far you're rolled out your barrel adjuster. If it's out more than 5mm or so, it's probably time to replace the pad.
More importantly than any of these indicators, if you see any metal showing through the pad, it's time to replace.