Focusing more on the empirical part of the question, consider the link that R. Chung gave in his first comment to Adam Rice's answer. That's the introductory chapter to Andrew Coggan's book, training and racing with a power meter. I believe Coggan may go into more detail in a later chapter, but I don't have the actual book.
Starting on page 9, Coggan described how normalized power is derived. As we know, average power is one measure of how demanding a ride was, but a ride with very variable efforts is more physiologically demanding than a ride at the same power. In the lab, they can measure blood lactate level. Coggan proposed that blood lactate level is a better measure of physiological demand than power. Lactic acid is produced by our muscles during exercise, but it is not practical to measure on the bike (yet?). Coggan explained:
This choice was made because many physiological responses (e.g., muscle glycogen and blood glucose utilization, catecholamine levels, ventilation) tend to parallel changes in blood lactate during exercise – in this context, then, blood lactate levels can be viewed as an overall index of physiological stress.
Coggan and unnamed collaborators had a number of trained cyclists exercise in a lab at various effort levels, above and below their lactate threshold. The power you can produce at your lactate threshold is probably around the same as your cycling functional threshold power.
After that, they had a number of datapoints for percent of lactate threshold and power. They used a technique like linear regression to relate the two variables. The relationship was not linear, and Coggan says:
Perhaps not surprisingly, an exponential function provided the best fit, but a power function of the following form proved to be nearly as good:
blood lactate (% of lactate at LT) = power (% of power at LT)3.90; R2 = 0.806, n=76
Based on these data, a 4th-order function was used in the algorithm for determining the IF (the exponent was rounded from 3.90 to 4.00 for simplicity’s sake).
So, to trust the concept of normalized power, you need at least two things. Most important, you need to trust that blood lactate level (normalized to % of lactate threshold) is a good proxy for physiological or metabolic cost. It doesn't have to be a perfect proxy, or even the best proxy, it just has to be good enough.
This is probably less important than the above precept, but you also need to be able to trust the equation above that relates power to blood lactate level. It is possible that the equation they use is flawed, or that it is good in most conditions but flawed in some. A term of art used in (at least) statistics and econometrics is that the functional form of the model needs to be correct. Without access to Coggan's raw data and/or to other data obtained the same way, it's hard to refute the equation. Take note of R. Chung's comments in this answer and Adam Rice's; there are other formulae that may be better (NB: I have not assessed the evidence that these are better formulae, I merely state that they exist and that they might be better.)
Some other considerations are that the sample of cyclists needs to be representative. I believe Coggan's sample was trained cyclists. Less trained individuals might have different physiological responses, e.g. maybe they are less efficient at clearing lactic acid, and thus NP might over- or under-state physiological demand. In addition, if the testing protocols differ materially from real-world experience, the formula might work less well in real life. It's not possible to assess those factors from where I sit.
I am not raising these points to critique the concept of NP. I have no background in exercise science. I'm merely attempting to show the trail of evidence, to show what would be required to believe the evidence, and to state some ways that the sample might not translate to every athlete. Some skepticism of any formula is always justified, but I am also not saying that you should not trust the concept of NP. Without being able to articulate specific defects in the formula, my stance is that I'll accept and use the concept, but I'll be aware that it could be flawed, and it might be subject to revision in the future.
As an example of different functional forms relating two physiological parameters, consider maximum heart rate, which can be used to pace efforts as well (NB: without a power meter, it would be better to find your threshold heart rate and use that rather than to try to find your max HR.) Many of us know 220-age - this is a formula used to approximate your maximum heart rate. This is something you'd get out of linear regression. When some researchers tried to find the academic source for 220-age, they found that the original source phrased their method a bit vaguely:
The formula maximum heart rate=220–age in years defines a line not far from many of the data points
Despite my last sentence, "not far from many of the data points" should sound a lot like least squares regression if you know that technique - it draws a line that minimizes the total of the squared distances between the line and each data point. In fact, the authors tried to replicate the regression result by approximately reproducing the data points in the article. They got 215.4 – 0.9147*age. Hence, 220-age was probably an approximation chosen for ease of use by non-scientists (e.g. personal trainers). Moreover, the article cites a number of other studies that tried to do the same thing in different samples, all of which gave different formulas.
As another aside, that formula actually gives you the average max HR for a certain age. Most of us will have higher or lower max HRs than the formula indicates. Some of us will have much lower or much higher max HRs. For example, the traditional formula gives a max HR of 180, and the slightly improved formula from the original dataset gives 179. I did a recent ramp power test, and my highest HR during the test was 192. That means that my actual max HR is at least 192. I know from other testing that my lactate threshold heart rate should be around 178-180 beats per minute, which is close to the average max HR for my age from the formula.
Going back to the concept of normalized power, all the formulas for normalizing power to lactate threshold will produce a formula based on the average experience of the people being studied. It might be possible that some of us (even those of us who are trained athletes) have physiological responses considerably different from that mean response. The more of an outlier you are, the less well any formula for NP will work for you. Of course, with NP, it's much harder for a person to determine if they're off from the mean response than with max HR.