Sunday, November 17, 2013
Climate Time Series Smoothing and What is Statistically Proper
Since the majority of the "raw" data using is seasonally adjusted temperature anomaly you open another can of worms with "unbelievable" confidence intervals. With anomaly the actual deviation is not very sensitive to the baseline period selected to create the anomaly nor is the trend, but absolute value that the anomaly represents can have significant difference dependent on the range of absolute values being averaged to create the anomaly series. Since you are really considered with the energy not so much the temperature that should represent that energy, the F to T^4 relationship severely limits the range that can confidently be assume to have "negligible" error.
I am in a pickle.
Now we have the data and we have to make choices so how do we avoid fooling ourselves? I think with lots of comparisons and lots of humility. No matter what choices you make there will always be someone that perceives your choices are biased, because they are. That is unavoidable. So there becomes a battle for "consistency".
I may for example compare any or all of those SST regions to a Paleo reconstruction that has its own natural and collector based smoothing. Ocean core samples build over many thousands of years and selecting a high or low frequency reconstruction. I compare a low frequency paleo series with higher but not highest frequency instrumental I get one correlation and then any smoothing of the instrumental will improve the correlation. William Briggs has an excellent post on that pitfall. But even knowing the pitfall, some smoothing can be helpful if properly noted and considered.
So I think there should be a better "degree of bias" that encompasses the combined degrees of known and unknown freedom in the statistical food chain.
Just having a brain fart.