Mathematica 12.x: Don't use Accumulate or Differences on Around objects!

Bug introduced in 12.0 or earlier and persisting through 12.2 or later

It seems we’ve discovered a bug! The following is a summary of the problem and possible workarounds. Scroll down to ORIGINAL POST to see the original statement of the issue.

If a List, TimeSeries, etc. of Around objects contains any identical elements, Accumulate produces propagated uncertainties that are too large. FoldList[Plus, 0, list] and FoldList[Plus]@list are valid alternatives that do not present this issue for Lists, but these do not work with TimeSeries! Finding something which does work with TimeSeries is still a work in progress.

If a List, TimeSeries, etc. of Around objects contains consecutive identical elements, Differences produces 0., which does not have an associated uncertainty. MovingMap[Subtract[#[[2]], #[[1]]] &, list, 1] and MovingMap[Subtract[#[[2]], #[[1]]] &, #, 1] &@list are valid alternatives which do not present this issue. And they work with TimeSeries!

Analysis of these and other functions can be found here.

For now, my best solution is to create a symbolic TimeSeries to manipulate with Accumulate, Differences, Ratios, etc., and then use ReplaceAll to insert the Around data in the final step. I’m not using AroundReplace because that function took over ten minutes to process a single TimeSeries! But, one must be careful when using ReplaceAll in lieu of AroundReplace. According to the documentation,

[ReplaceAll and AroundReplace] coincide if each replaced variable appears once in the original
expression.

One must therefore perform calculations in a manner that ensures this is the case.

————————————ORIGINAL POST————————————

I am working with several TimeSeries containing Around objects. I find that the propagated uncertainties for numerical values are larger than I expect them to be. In contrast, the propagated uncertainties for symbolic values are exactly what I expect them to be.

This is also true for lists of Around objects, so I will use List to demonstrate. Note that the original data values are all integers greater than or equal to zero (0 ≤ n[i] ∈ ℤ) and their original uncertainties equal their square roots (δn[i] = √n[i])

First, generate a list of symbolic data:

symbolicData = Table[Around[n[i], Sqrt[n[i]]], {i, 10}]

Then, Accumulate these data:

symbolicTotals = Accumulate@symbolicData

The uncertainty of the last value is the square root of that value:

{symbolicTotals[[-1]]["Uncertainty"], Sqrt@symbolicTotals[[-1]]["Value"]}

Given my understanding of error propagation, this is correct.

Now, generate numerical data:

numericalData = Table[Around[#, Sqrt[#]] &@RandomInteger[10], {i, 10}]

Acculumate these data:

numericalTotals = Accumulate@numericalData

Compare the last value’s uncertainty and square root:

{numericalTotals[[-1]]["Uncertainty"], Sqrt@numericalTotals[[-1]]["Value"]}

The uncertainty is larger than the square root. This happens for any long list of numbers.

Why? And how can I fix it? (Assuming there’s an actual problem here, and not simply a lack of knowledge on my end.)

My guess is that this discrepancy is a consequence of the fact that

When Around is used in computations, uncertainties are by default propagated using a first-order series approximation, assuming no correlations.

What exactly is being approximated here, and if Around always does this, why doesn’t it affect symbolic values?

I understand that AroundReplace allows for higher-order approximations, but it starts with a symbolic expression and substitutes numerical values. I’m starting with numerical values. Is my only option to generate TimeSeries objects with symbolic values, do all my computations, and then AroundReplace the symbols with the original numerical data?

Is there a better way?

————————————UPDATE————————————

@JulienKluge, @SjoerdSmit, and @JimB have suggested and demonstrated that this is a bug in Accumulate. I’ve done my own investigation, and I agree. Accumulate produces incorrectly propagated uncertainty whenever the same Around object appears multiple times in whatever is being accumulated. This error seems to increase with the number of duplicates.

Accumulate is not the only function with an issue. When the same Around object appears consecutively in a list, Differences produces a zero with no uncertainty!

(If you’d like a copy of the notebook in which I performed this investigation, click here.)

I’ve encountered the problem with Differences before, but I’m only now recognizing it as a serious issue. I suppose bugs like this are why Around is still an experimental function…

In light of everything, I think I will try doing all my calculations on symbolic TimeSeries objects and finishing them off with AroundReplace. Hopefully, that will circumvent the Accumulate and Differences issues as well as some other minor problems I’ve encountered. If that doesn’t work, I’ll compute the uncertainty bands myself…

Wish me luck!

————————————UPDATE————————————

AroundReplace took over ten minutes to replace the values of a symbolic TimeSeries with Around objects. No other functions such as Accumulate had yet been used. So, I won’t be using AroundReplace. ReplaceAll was much faster, but the documentation suggests it might introduce its own problems when working with Around.

Following the advice of @SjoerdSmit, I am going to try using FoldList[Plus]@list and MovingMap[Subtract[#[[2]], #[[1]]] &, #, 1] &@list as alternatives to Accumulate and Differences, respectively.

I’ll report back if there are any major changes.

list1 = {Around[1., 2.], Around[1., 2.0001]};
Plus @@ list1
Fold[Plus, 0, list1]
Last @ Accumulate[list1]

list2 = {Around[1., 2.], Around[1., 2.]};
Plus @@ list2
Fold[Plus, 0, list2]
Last @ Accumulate[list2]

FoldList[Plus, 0, list2]

Accumulate[{a, b}]
Accumulate[{a, a}]

Accumulate[{a,b}]/.{a->Around[v,d],b->Around[v,d]}
Accumulate[{a,a}]/.{a->Around[v,d]}

newTimeseries = TimeSeries[xyz,TemporalRegularity->True]