Wavelet probability analysis is a simple but powerful method for identifying the statistical significance of spectral overlap between datasets. For details on the method, see Stevenson et al. (2010), or try out the interactive toolkit.

In a nutshell, the idea is that once a wavelet transform has been applied to any time series, one can then form a probability distribution function (PDF) of the wavelet spectrum at any frequency. This is conceptually equivalent to creating a histogram of wavelet power at an arbitrary wavelet period. Then if two time series exhibit similar spectral variability, they should have similar wavelet PDFs.

Similarity between wavelet PDFs is measured using the wavelet probability index (WPI); this is simply the joint probability distribution function of two wavelet PDFs, or the area of the region over which the two PDFs overlap. The WPI is by definition between 0 and 1, and can be thought of as a percentage of overlap.

    see scripts: wpi.m

Long-term variability in a time series can be examined by studying the variation in their WPI. One can imagine that a subset of a time series will sample a certain amount of the internal variability present, and that some subsets will be more similar to one another than others: this is directly reflected in the WPI calculated between those subsets. The WPI calculated between portions of the same time series is referred to as the self-overlap.

There are two major types of self-overlap used in this toolkit, which are useful for different purposes. The first of the self-overlap measured between subsets of identical lengths, which measures the amount of scatter present between “chunks” of a time series. For example, one might be interested in how much variability is naturally present within a time series of length 50 years; then measuring this type of self-overlap will give a measure of that variability.The second type of self-overlap compares subsets of a time series of arbitrary length, to the full time series. This is a measure of the proportion of internal variability of the total data represented by a measurement of any given length.

It turns out that for both types of self-overlap, the width of the 90% confidence interval on WPI has an exponential dependence on the subset length. Not only that, but using NINO3.4 we find that for self-overlap between the full time series and subsets of the same, the slope of that dependence holds for multiple climate models. This means that one can actually predict the amount of time a model must be run to sample a certain percentage of the total ENSO variability.

    see scripts: ciregress.m, selfsim.m, selfsimtot.m, plot_selfsim.m

The same idea can be readily applied to comparisons of two time series against one another. Wavelet PDFs generated from subsamples of one time series are compared to the wavelet PDF of the other time series, and the resulting WPI distribution computed.

Comparison WPI distributions are useful for measuring the extent of agreement between time series; if the two are really measuring the same quantity, then theoretically the upper limit of their WPI distribution should approach 1 for longer subsamples. This is not always the case for model/data comparisons, indicating that model biases still remain - and that wavelet probability analysis is able to pick them up when they exist.

    see scripts: wpicomp.m, plot_wpicomp.m

Empirical hypothesis testing is used to determine the level of statistical significance of overlap between two WPI distributions. This is useful both for validating a model against observations, and for measuring relative model performance.

The necessary ingredients for hypothesis testing are simply two WPI distributions; for example, to validate a model against observations, these would be the model self-overlap WPI distribution and the model/data comparison WPI distribution. Then determining when the confidence intervals on the two distributions overlap yields the significance at which they differ:

the largest significance level at which the two confidence intervals overlap gives you the minimum significance at which they differ. If distributions are identical, then the minimum significance is 0; if they have nothing in common at all, then the minimum significance is 1.

    see scripts: emp_hyptest.m, plot_emphyptest.m