It's actually an awesome thing to do! It's basically the best way to exercise for fat loss, and to keep it off. 

This concept is called Fasted Cardio.

This vid explains it better. I don't know how to embed it like the user above.

https://www.youtube.com/watch?v=O8rN5_Ace-8