It’s hard to give a general figure for how much more expensive a self built custom bike will be, but it’s relatively easy (although time consuming) to figure out how expensive your build will be. Just make a spreadsheet of all the components you will need, and research the best prices online.
Don’t forget to factor in all the minor components like grips, cable and brake housing. Also factor in all the special tools you will need to buy (or rent from a local bike co-op).
The other thing to consider is the investment you’ll have to make in learning all about component compatibility. There are some compatibility things that are not trivial to figure out.
You will also have to learn how to install and adjust every single component on the bike. That's a fairly significant investment in time, both in research and learning and the fiddling around trying to get stuff working right.
If learning all that is part of your goal (and it’s a completely reasonable one) all well and good, just know what you are getting into.
It’s also possible to scour Craigslist, EBay and local forums for parts, but you really have to know what you are looking for to make sure you get compatible components that are not worn out.