How to quantify error for a multivariate expression/equation?
Answers
Answered by
0
If I understand correctly, the σx,σy,σzσx,σy,σzinclude both deviations of the model from reality and the variance due to measurement uncertainties, so they are not by themselves sufficient to define criteria for whether the model reasonably describes the data. Huge σx,σy,σzσx,σy,σzcould be due to either poor measurements or a poor model.
One needs to evaluate the uncertainties (σMiσMi) in the measured values (MiMi), ideally based on a good understanding of the measurement system; calibration measurements of known values are very helpful. If the only handle on the experimental uncertainty is the variance of the data, then one can partially estimate the uncertainty by looking at the local variation (i.e. noise) of data points that are close enough in (x,y,z) so that the changes in F(x,y,z) are believed to be negligible. e.g. If you repeatedly measure values at 10 locations, what are the 10 standard deviations of the measurements taken for each of the 10 points. (It is important not to just take repeated consecutive measurements at the same point, since that will miss errors due to drifts and repositioning. The variance of the noise will also not tell you anything about systematic errors such as those associated with instrument calibration.) Once you have a handle on the uncertainties, you can then, as Lê Dũng suggested, use metrics such as the chi-square test to quantify the global agreement between the model and the data, and local agreement can be judged by whether the differences between the experiment and model are consistent with the expected experimental uncertainties, e.g. if you plot the data and model, are their differences consistent with the error bars?
Note that you should also make off-axis measurements, since it is possible for F(x,y0,z0)F(x,y0,z0), F(x0,y,z0)F(x0,y,z0), and F(x,y0,z)F(x,y0,z) to all closely agree with the data, but have huge disagreements off-axis, e.g. along the diagonal F(x,y,z0
One needs to evaluate the uncertainties (σMiσMi) in the measured values (MiMi), ideally based on a good understanding of the measurement system; calibration measurements of known values are very helpful. If the only handle on the experimental uncertainty is the variance of the data, then one can partially estimate the uncertainty by looking at the local variation (i.e. noise) of data points that are close enough in (x,y,z) so that the changes in F(x,y,z) are believed to be negligible. e.g. If you repeatedly measure values at 10 locations, what are the 10 standard deviations of the measurements taken for each of the 10 points. (It is important not to just take repeated consecutive measurements at the same point, since that will miss errors due to drifts and repositioning. The variance of the noise will also not tell you anything about systematic errors such as those associated with instrument calibration.) Once you have a handle on the uncertainties, you can then, as Lê Dũng suggested, use metrics such as the chi-square test to quantify the global agreement between the model and the data, and local agreement can be judged by whether the differences between the experiment and model are consistent with the expected experimental uncertainties, e.g. if you plot the data and model, are their differences consistent with the error bars?
Note that you should also make off-axis measurements, since it is possible for F(x,y0,z0)F(x,y0,z0), F(x0,y,z0)F(x0,y,z0), and F(x,y0,z)F(x,y0,z) to all closely agree with the data, but have huge disagreements off-axis, e.g. along the diagonal F(x,y,z0
Similar questions