Bicycles are inherently stable because of their geometry. The geometry causes the bicycle to always turn into the direction it begins to lean, which keeps it upright. The reason is best illustrated through a concept known as counter-steering.
Counter steering is how all two wheel vehicles turn. When you want to turn towards the left, you turn the handlebars a little to the right. The friction of the wheels pulls the bottom of the bike towards the right, which initiates a lean towards the left. The handle bars then begin to swing towards the left to track through the turn.
When it's time to stop the turn, you turn the handlebars a little more to the left. That pulls the bottom of the bike further towards the left, which brings the bottom of the bike directly under the center of gravity and thus stopping the turn.
On many bikes and at low speeds, the counter steering effect can be unnoticed by many riders. However, at high speeds, or with heavier vehicles such as motor cycles it is more significant.
So, how does this work where there is no rider? It is because of the rake in the fork and the rail it causes. If you trace an imaginary line through the axis of your fork to the ground, it will hit the ground ahead of where the wheel contacts the ground.
Because the wheel contacts the ground behind the steering axis, the wheel will always feel a force from the road trying to bring it to center, pointing straight ahead. When the bike is tipped to one side, the forces begin to push the wheel to the side that the bike is tipped.
So all these forces add up. The rake in the fork makes the bike want to go straight forward. And when it feels a bump in one direction or the other, the counter steering will tend to bring the bike the other direction. Then the fork rake will begin pushing the front wheel further away, which will then straighten the bike out, because of the counter steering.