There’s a recent paper in the Journal of the American Meteorological Society called An Evaluation of the Software System Dependency of a Global Atmospheric Model. The paper considers global atmospheric numerical model on machines with different hardware and software systems that were tested on 10 different computer systems having different central processing unit (CPU) architectures or compilers. They find that There exist differences in the results for different compilers, parallel libraries, and optimization levels, primarily due to the treatment of rounding errors by the different software systems.
Now, as far as I can tell, this is probably – in some sense – simply an illustration of deterministic chaos. Rather predictably Watts Up With That (WUWT) already has a post called another uncertainty for climate models – different results on different computers using the same code. The post actually says
It makes you wonder if some of the catastrophic future projection are simply due to a rounding error.
I think this indicates a mis-understanding of deterministic chaos and what this paper is actually illustrating. I’ve done some work on deterministic chaos, so thought I would briefly discuss my understanding of what is meant by this term, although – bear in mind – it has been a long time since I’ve looked at this in any detail.
It’s my impression that many think that chaos means random or means that we don’t really understand the underlying physics or mathematics. In fact, deterministic chaos is a well-defined mathematical concept. It refers to a situation in which you can completely understand and be able to model (mathematically) some system, but cannot actually determine the initial conditions with sufficient precision to be sure that any simulation of the system’s evolution will actually match what is observed.
Essentially, it is a system that is extremely sensitive to it’s initial conditions. There are, consequently, two related issues. One is that if you try to simulate (with a computer programme for example) such a system, you cannot possibly determine the initial conditions with sufficient precision to know if your simulation results will actually match what you would observe. A classic example of a chaotic system is a double pendulum, whose motion is chaotic for certain energies. It seems nice and simple but even though we can write down the exact equations describing the system, we would be unable to accurately model (for a given initial position) it’s evolution because we couldn’t possibly measure the initial state with sufficient precision.
The other issue is that because such systems are so sensitive to initial conditions, even if you do know how it behaves given one set of initial conditions, you can’t use that to estimate how it would behave if you changed the initial conditions very slightly. Hence the the term the butterfly effect. There are numerous variants of this effect, but one example might be the flap of a butterfly’s wings in China can set off a storm in Arizona. This doesn’t really mean that a butterfly flapping it’s wings will actually do this, simply that the weather system is so sensitive to initial conditions that a small change in the system can lead to a large effect at a later time.
So, really this paper is – in some sense – simply illustrating that climate models are inherently chaotic and hence even a small difference between how different architectures, or compilers, treat numbers can change the result of a simulation. Not that surprising really. That’s not to say that this isn’t an important thing to understand, simply that it doesn’t really indicate an error in the models.