Blog - The stochastic resonance program (part 1) (Rev #48, changes)

Showing changes from revision #47 to #48:
Added | ~~Removed~~ | ~~Chan~~ged

This page is a blog article in progress, written by David Tanzer. To see discussions of this article while it was being written, visit the Azimuth Forum. Please remember that blog articles need HTML, not Markdown.

*guest post by David Tanzer*</i>

At the Azimuth Code Project, we are aiming to produce educational software that is relevant to the Earth sciences and the study of climate. Our programs take the form of interactive web pages, which are written in javascript and use the JSXGraph library for interactive plotting. They allow you to experiment with the parameters of a model and view its outputs. But in order to fully understand the meaning of a program, we need to know about the concepts and theories that informed its creation. So we will also be writing articles to explain the science, the math, and the programming behind these models.

In this blog and the next, I will cover one of the scientific programs developed at Azimuth, written by Allan Erskine and Glyn Adgie. It demonstrates a phenomenon known as stochastic resonance, in which, under certain conditions, a noise signal may amplify the effect of a weak input to a signal detector. This concept was originally applied in a hypothesis about the timing of ice-age cycles, but has since found widespread applications, including the neuronal detection mechanisms of crickets and crayfish, and patterns of traffic congestion.

You see, there are a lot of interesting currents that come together in scientific programming. The report given in this blog is meant to function as a kind of “documentation” for a scientific program. I am approaching this from my background as a software developer and computer scientist, so if you find any issues with my interpretation of the science, let us know!

Suppose that we have an input signal which is fed into a signal detection mechanism. The input signal will drive the state of detection mechanism, in a possibly complex, but deterministic fashion. Further, let’s divide the states of the detector into “on” states and “off” states. In the case of a light switch, the input signal may be the force applied to the switch, the state could be the angular position of the switch, and we could say that the sign of the angle determines whether the switch is on or off.

~~ Let’s~~ Suppose that we have an input signal which feeds into a~~ periodic~~ signal~~ signal,~~ detection~~ and~~ mechanism.~~ examine~~ Suppose the~~ conditions~~ input~~ under~~ signal,~~ which~~~~ its~~~~ frequency~~~~ will~~~~ be~~~~ reflected~~ by itself, will drive the~~ flip-flopping~~ state of~~ the~~ detection~~ digital~~ mechanism,~~ output~~ in~~ signal.~~ some~~ In~~ possibly~~ particular,~~ complex, but deterministic fashion. Further, let’s~~ consider~~ divide the~~ case~~ states~~ where~~ of the~~ input~~~~ signal~~~~ is~~~~ weak,~~~~ so~~~~ that~~~~ the~~ detector~~ remains~~ into~~ in~~ “on”~~ a~~ states~~ single~~ and “off” states. This digital state is an abstraction from the concrete state.~~ How~~ In~~ can~~ the~~ we~~ case~~ amplify~~ of~~ such~~ a~~ signal,~~ light~~ so~~ switch,~~ that~~~~ it~~~~ becomes~~~~ present~~~~ in~~ the~~ output?~~~~ Well,~~~~ if~~~~ the~~ input signal~~ is~~ may~~ sufficient~~ be the force applied to~~ drive~~ the~~ detector’s~~ switch, the state~~ close~~ could~~ to~~ be the~~ boundary~~ angular~~ between~~ position of the switch, and the digital~~ states,~~ state~~ then~~ could~~ a~~ be~~ bit~~ defined as the sign of this angle.*random noise*~~ may be enough to get it to cross the boundary, into the other digital state. As the input signal moves towards another phase which would have pulled the state further away from the boundary, the system may then cross back into the state where it started. You can imagine that when the settings are tuned just right, the noise will catalyze the detector output to “stochastically oscillate” at the frequency of the input signal.~~

~~ Stochastic~~ Let’s~~ resonance~~ input~~ has~~~~ been~~~~ found~~~~ in~~ a~~ wide~~ periodic~~ range~~ signal,~~ of~~~~ natural~~ and~~ artificial~~ examine~~ systems,~~~~ including~~ the~~ signal~~ conditions~~ detection~~ under~~ mechanisms~~~~ of~~~~ neurons.~~~~ For~~~~ instance,~~~~ there~~~~ are~~~~ cells~~~~ in~~~~ the~~~~ tails~~~~ of~~~~ crayfish~~ which~~ are~~ its~~ tuned~~ frequency~~ to~~ is~~ respond~~ reflected~~ to~~ by~~ low-frequency~~~~ signals~~~~ in~~ the~~ movement~~ flip-flopping of the~~ water,~~ digital~~ presumably~~ output~~ arising~~ signal.~~ from~~ Suppose the~~ motions~~ input~~ of~~ signal~~ predators.~~ is~~ On~~ weak,~~ their~~ so~~ own,~~ that~~ these~~~~ signals~~~~ are~~~~ not~~~~ strong~~~~ enough~~~~ to~~~~ raise~~ the~~ neurons~~ detector~~ to~~ remains~~ the~~~~ firing~~~~ threshold.~~~~ But~~ in a single digital state. How can we amplify such a signal, so that it becomes present in the~~ presence~~ output?~~ of~~ Well, if the~~ right~~ input~~ amount~~ signal~~ of~~ is~~ noise,~~ sufficient~~ these~~ to~~ signals~~ drive~~ do~~~~ cause~~ the~~ neurons~~ detector’s state close to~~ fire.~~ the boundary between digital states, then a bit of*random noise* may push it over to the other digital state. We will then see a phase-dependent probability of transitions between digital states – and this itself bears the mark of the input frequency.

A stochastic resonance mechanism has been found in the signal detection mechanisms of neurons. There are, for example, cells in the tails of crayfish which are tuned to respond to low-frequency signals in the movement of the water, presumably arising from the motions of predators. These signals alone are not strong enough to raise the neurons to the firing threshold, but in the presence of the right amount of noise, they will cause the neurons to fire.

• Stochastic resonance, Azimuth Library

Stochastic resonance was originally defined in a more~~ restrictive~~ specific~~ sense,~~ form: where the~~ two-state~~ system~~ was~~ is~~ assumed~~ bistable.~~ to~~ So~~ be~~ not~~ bistable~~ only~~ –~~ are~~ where~~ the concrete states divided into two digital states, but the concrete states in each digital~~ state~~ states~~ is~~ are the~~ basin~~ basins of attraction for a stable point of~~ stable~~ equilibrium.

~~ The~~ Stochastic~~ prototypical~~~~ application~~~~ of~~~~ bistable~~~~ stochastic~~ resonance was~~ to~~ first introduced in this form, in a~~ form~~ stochastic variant of the Milankovitch theory of the~~ timing~~ ice-age cycles, which invokes a bistable model of the~~ ice~~ Earth’s~~ ages.~~ climate, and a periodic forcing function which is driven by long-duration cycles in the rotational parameters of the Earth’s movement. First let’s talk about the Milankovitch theory in its classical, deterministic form, which will set the stage for understanding the stochastic resonance hypothesis.

In the simplest form of Milankovitch theory, the climate is modeled as a bistable system, where one state is a cold,~~ snowball~~ “snowball” Earth, and the other is a hot, iceless Earth. The snowball Earth is stable because it is white and reflects solar energy, which keeps it~~ cold.~~ frozen. The iceless Earth is stable because it is dark and absorbs solar energy, which keeps it~~ hot~~~~ and~~ melted.

The forcing signals are taken to be the long-duration “Milankovitch” astronomical cycles – tilting of the Earth’s axis, precession of its axis, and variation in the eccentricity of the Earth’s orbit – which vary the amount of solar energy received in the northern latitudes.

There are three astronomical cycles that contribute to the forcing function:

• Changing of the eccentricity (ovalness) of the Earth’s orbit, with a period of 100 thousand years

• Changing of the obliquity (tilt) of the Earth’s axis, with a period of 41 thousand years

• Precession (rotary wobbling) of the Earth’s axis, with a period of 23 thousand years

These effects sum to produce a multi-frequency variation in the amount of solar energy reaching the northern latitudes.

The ice-ages are currently running on a roughly 100,000 year schedule, and, intriguingly, the three mentioned Milankovitch cycles have periods of 21, 32 and 100 (FIXME) kiloyears, respectively.

But how would these cycles trigger a change in the state of the Earth’s climate?

In the most recent forms of the Milankovitch hypothesis, a deterministic mechanism is invoked. See, for example Paillard’s paper.

In the original stochastic resonance hypothesis, by Benzi, these Milankovitch cycles produce a signal which in itself is not strong enough to change the state of the climate, but well-timed noise events, at the right phases of a Miloankovitch cycle, could trigger a state change.

The~~ timing~~ induced temperature changes, however, are not large enough to trigger a state change. According to the stochastic resonance hypothesis, it is other, random variations in the heat received up north that may trigger the climate to change states. One such source of variation may be changes in the~~ ice-ages~~ amount~~ remains~~ of~~ a~~ heat-trapping~~ great~~ gases~~ unsolved~~ in~~ problem~~ the~~ for~~ atmosphere.~~ modern~~~~ science.~~

~~ •~~ The timing of the ice-ages remains a great unsolved problem for modern science. The theory of the timing of the ice ages is a fascinating, challenging and open problem in science.~~Stochastic resonance~~~~, Azimuth Library~~

Note: here we are using the term ice age in the colloquial sense of being a period of maximal glaciation. The scientific term Ice Age refers to a major period of time that spans thousands of such “ice ages” and the warm periods between them. There have only been four such Ice Ages in the history of the Earth. Each one is characterized by a different timing pattern for the many ice ages that it contains.

The climate is modeled as a multistable system, and the forcing results from certain slowly varying, cyclical changes in astronomical variables (such as the tilt of the Earth’s axis). These are known as Milankovitch cycles, and their periods are on the scale of 10,000 to 100,000 years. Note that in our current major Ice Age, the minor ice ages occur roughly every 100,000 years.

• Milankovitch cycle, Azimuth Library

The Azimuth demo program uses a specific stochastic differential equation (SDE) to define a bistable system with a sinusoidal driving function and a noise component.~~ In~~ An~~ an~~ SDE~~ SDE,~~ specifies the derivative of the output signal~~ is~~~~ specified~~ as a function of time, the current signal value, and a noise process.

In~~ this~~ our~~ differential~~ SDE,~~ equation,~~ the deterministic part of the derivative is set to~~ the~~ a time-varying sine wave plus a bistable function of the current signal value:

DerivDeterministic(t, x) = SineWave(t, amplitude, frequency) + Bistable(x),

where Bistable(x) = x (1 - x^{2}).

Now let’s analyze the effects of these terms.

Alone, the sine wave would cause the output signal to vary sinusoidally.

The bistable polynomial has roots at -1, 0 and 1. The root at 0 is an unstable equilibrium, and -1 and 1 are stable equilibria. The basin of attraction for -1 is all the negative numbers, the basin for 1 is the positive numbers, and the point of unstable equilibrium separates the basins.

*We view each basin as one of the states of a bistable system.*

Now let’s put the sine wave and the bistable polynomial together. If the wave is relatively weak, the system will gravitate towards one of the attractors, and then continue to oscillate around it thereafter – never leaving the basin. But if the wave is large enough, the system will oscillate between the two basins.

Now let’s consider the noise as well. Suppose the sine wave was large enough to periodically pull the system close to zero – but not enough to cross over. Then a well-timed noise event could push the system over to the other side. There will be a phase-dependent probability of the noise triggering a state change, with more noise leading to transitions on more of the cycles. The flip-flopping between states will contain a “stochastic reflection” of the driving sine wave. Even more noise will cause multiple transitions within each cycle, thereby drowning out the signal and turning the output to noise.

This is the stochastic resonance effect: under the right conditions, the noise may *amplify* the effect of the input signal.

We’ve talked about the idea of stochastic resonance, touched on some of its applications, and described the mathematical model of stochastic resonance used in the Azimuth demo program. Next time we will look into this program: how to use it, how it works, and how to modify it to~~ make~~ develop new programs.

category: blog