All nonlinear fitting options ignore the magnitude of the weights when computing the uncertainty on the fit coefficients

I've tried using fitnlm(), fit() and nlinfit() to fit a non-linear model to my experimental data. The data points each have a real variance and mean, and I am taking 1/variance as the weights.

I noticed that if I change the scale of the weights, say take [weights]*10 or [weights]/10, the standard error of the fit coefficients is unchanged. This doesn't make sense, if I have more precise data the fit should have less uncertainty. What is going on? How can I fit a nonlinear model where the true magnitude of the weights is properly accounted for?

Notes: You can get the error on the estimated coefficients in the following ways:

• nlinfit(): [beta,R,J,CovB,MSE,ErrorModelInfo] = nlinfit(x,y,fitfun,guess,'Weights',weights) --> coefficient stderr is sqrt(diag(CovB))
• fit(): f=fit(x,y,fitfun','StartPoint',guess,'Weights',weights) --> returns 95% confidence bounds on error coefficients, use confint to get 1-sigma error on coefficients
• fitnlm(): fobjnlm=fit(x,y,fitfun,guess,'Weights', weights) --> this directly returns the fit coefficient Standard Error in the fit object

This is real experimental data. I am taking the same data point multiple times then averaging it together to get a point with a mean, and taking the standard error to get an errorbar on each point. The more points I take to get a mean and stderr the smaller the errorbar is. My fit coefficients should have more uncertainty when i take less data and average down less, and should have less uncertainty when I take more data and average down more. However, the output of all the functions I have tried is immune to the data point errorbars. This is not right!

Thanks in advance!

EDIT: The data I have is quite complex and I can't attach it here. I have added some screenshots, though. The data takes the form of an exponentially decaying sinusoid. There are 2 sine waves which have slightly different frequencies, so they beat together. We want to determine the beat frequency with maximal precision. We initially do a rough fit to each sine wave separately (1st image). I zoomed in on the early part of the fit, we go to a lot of trouble to hit the sides and tops of the fringe and to minimize our measurement uncertainties (2nd image). Then we simultaneously fit the sum and difference waves to get a better fit for the beat (3rd and 4th images). The uncertainty on the beat frequency is one of the most important outputs of this analysis.

Answer 1

[Image Analyst]

It seems like if the variance is high (noisy section of your signal) and the weights are low, then it should pay less attention to how well the fitted curve fits there in that noisy section. So you get, possibly, a high error in that region.

While in the "good", low variance, noise-free section your weights will be high so it will pay more attention to that section. But it would already fit pretty good in that section anyway, even with uniform/default weights.

But anyway, the standard error, or MSE between fitted curve and data, should be identical I would think because all weights are proportional to the variance. You're just simply scaling them up and down by some constant factor. Why should that change the fit? I can't think of any reason.

Let's take an simplified example. Let's say the variance in the left half of your data is 500 and the variance in the right half of your data is one tenth of that, 50. So now you scale by 10 and you have 5000 and 500, or you scale by 1/10 and you have 500 and 5. Either way, the ratio between the weights on the left and right halves is 10, no matter how you scale your weights. So why should your fitted curve change?

I know the question posting page asked you already but I'm going to ask again: Please attach your original data, the model you want to fit to the data, the weights you want to apply to it, and a picture/diagram/screenshot/sketch of what curve you'd like the fit to look like so that we can better help you.

https://www.mathworks.com/matlabcentral/answers/6200-tutorial-how-to-ask-a-question-on-answers-and-get-a-fast-answer

[supernoob]

Thanks for your response!

Unfortunately, this doesn't make sense to me. I agree that, in your example, the fitted curve is unlikely to change. But the uncertainty in the fit coefficients should!

Why would we (scientists) bother to get more precise data at all? My entire PHD would be easy if I could just get less precise data points and fit the same curve to it! (Wouldn't that be nice eh)

More precise data has to result in more precise fit coefficient estimates. There's entire brances of Physics (and probably other fields) which relies on this one fact.

Take the simplest possible example: a straight line fit to two points. One dataset has tiny errorbars, one has huge errorbars. The estimate of the error on the slope must be smaler for the line made up of 2 points with tiny errorbars.

What I am failing to understand is how to extract the relevant fit coeffecient uncertainty which accurately reflects the precision of my data. All of the things I have tried are insensitive to the precision of the data points. It seems like all of the nonlinear regression options Matlab offers have this same flaw.

[Image Analyst]

I'm not seeing where fitnlm() returns uncertainties. Please show your call to fitnlm() so we can see how you assign the weights and pass them to the fitnlm() function. And please show how you're computing the uncertainties, which fitnlm() does not return.

And just saying that your weights are all scaled up or down by some factor does not change the overall weighting. Now if you had scaled the weights for only SOME of the input data up or down and left the others alone, then I can see how that might change the fit. Merely scaling every single weight up or down by the same factor does not change the data values and so would not change the fit. I think that would only happen if you scaled them differently. I'm not sure of the insides but it probably subtracts the mean of the signal and then multiples the mean-subtracted data by the weights at each location and then does the fit, then reverses the process to get the fit coefficients as it it was done on the original data. So I still don't see how multiplying the weights by a single factor will change the fit. I don't believe it should, which is corroborated by the fact that that is evidently what you say you're seeing.

I'm sure there is a statistics professor at your university who would be happy to discuss your problem with you so you should take advantage of that. It will be faster and more interactive than here.

[supernoob]

Hi, thanks for your response. I have updated the question to include information on calling each fitting function and extracting the fit coefficient errors/uncertainty.

Again, I agree that the fit should not change if all the errorbars are scaled up or down. But the error on the coefficients absolutely should. It is the error on the coefficients that I am concerned about, not the fit.

[Image Analyst]

To determine the beat frequency, shouldn't you be looking for spikes in the Fourier domain rather than trying to do this in the time or spatial domain?

[supernoob]

We have lots of reasons for not using the fourier domain. We need to do the nonlinear fit in the time domain.

[supernoob]

You can see in the images I attached that there is not enough sampling to get sufficient signal to noise in the fourier domain anyways. This is due to experimental constraints. Please trust me that this whole dataset and collection is very complex, and that the problem I have (that I need to do nonlinear fits in the time domain) is the one I actually need solved.