Blog - The stochastic resonance program (part 1) (Rev #62, changes)

Showing changes from revision #61 to #62:
Added | ~~Removed~~ | ~~Chan~~ged

This page is a blog article in progress, written by David Tanzer. To see discussions of this article while it was being written, visit the Azimuth Forum. Please remember that blog articles need HTML, not Markdown.

*guest post by David Tanzer*</i>

At the Azimuth Code Project, we are aiming to produce educational software that is relevant to the Earth sciences and the study of climate. Our software takes the form of interactive web pages, which allow you to experiment with the parameters of a model and view its outputs. But in order to fully understand the meaning of a program, we need to know about the concepts and theories that inform it. So we will be writing articles to explain the science, the math, and the programming behind these models.

In this two-part series, I will cover the Azimuth stochastic resonance example program, by Allan Erskine and Glyn Adgie. Here I will outline some of the math and science behind the program, and next time I’ll dissect the program. By way of introduction, I am a software developer with research training in computer science, so this is a new field of investigation for me. Any amendments or clarifications are welcome!

Stochastic resonance is a phenomenon in which, under certain conditions, a noise source may amplify the effect of a weak signal. This concept was used in an early hypothesis about the timing of ice-age cycles, but has since been applied to a wide range of phenomena, including neuronal detection mechanisms, and patterns of traffic congestion.

Suppose that we have a signal detector whose internal, analog state is driven by an input signal, and suppose the analog states are classified into a region of “on” states and a region of “off” states. This is the digital state, abstracted from the analog state. With a light switch, we could take the force as the input signal, the angle as the analog state, and the up/down classification of the angle as the digital state.

Let’s consider the effect of a periodic input signal on the digital state. Suppose that the wave amplitude is not big enough to change the digital state, yet large enough to drive the detector’s analog state close to the digital state boundary. Then, a bit of random noise, occurring near the peak of an input cycle, may “tap” the system over to the other digital state. So, there will be a phase-dependent probability of transitions between digital states. This relationship between signal phase and~~ state~~ state-transition~~ transition~~ probabilities bears the~~ signature~~ stamp of the input frequency. The noise has*amplified* the input signal.

But it’s a~~ very~~ pretty funky amplifier!~~ See~~ Here~~ this~~ is~~ example~~ a picture from~~ the~~ an~~ Azimuth~~ article~~ on~~ by~~ stochastic~~ Glyn~~ resonance:~~ Adgie and Tim van Beek:

Stochastic resonance has been found in the signal detection mechanisms of neurons. There are, for example, cells in the tails of crayfish which are tuned to low-frequency signals in the movement of the water, generated by the motions of predators. These signals alone do not cross the firing threshold for the neurons, but with the right amount of noise, the neurons do respond to the signals.

See:

• Stochastic resonance, Azimuth Library

• Stochastic resonance in neurobiology, David Lyttle. An overview, with clear explanations.

• Increasing the Signal-to-Noise Ratio with More Noise, Glyn Adgie and Tim van Beek, Azimuth Blog.

Stochastic resonance was originally defined for the special case of a bistable system – where each digital state is the basin of attraction for a stable point of equilibrium.

~~ The~~ An~~ first~~ early application of stochastic resonance was to a~~ hypothesis~~ hypothesis,~~ concerning~~~~ the~~~~ cycling~~~~ of~~~~ the~~~~ ice-ages~~ within the framework of bistable climate~~ dynamics.~~ dynamics, about the timing of the ice-age cycles. Although~~ this~~ it~~ hypothesis~~ has not been confirmed, it remains of interest (1) historically, (2) because the the timing of the ice-age cycles remains an open problem, and (3) because the Milankovitch hypothesis upon which it rests is an active part of the current scientific research agenda.

In the bistable model, the two climate states are a cold, “snowball” Earth and a hot, iceless Earth. The snowball Earth is stable because it is white, and hence reflects solar energy, which keeps it frozen. The iceless Earth is stable because it is dark, and hence absorbs solar energy, which keeps it melted.

The Milankovitch hypothesis states that the drivers of climate state change are long-duration cycles in the solar energy received in the northern latitudes (called the “insolation”) that are caused by periodic changes in the Earth’s orbital parameters. The significance of the north is that the glaciers are concentrated there, and so a sufficient “pulse” in the northern temperatures could trigger a state change.

Three such astronomical cycles have been identified:

• Changing of the eccentricity of the Earth’s elliptical orbit, with a period of 100 kiloyears

• Changing of the obliquity (tilt) of the Earth’s axis, with a period of 41 kiloyears

• Precession (swiveling) of the Earth’s axis, with a period of 23 kiloyears

In the stochastic resonance hypothesis, the Milankovitch signal is amplified by random events to produce climate state changes. More recent forms of Milankovitch theories invoke a deterministic forcing mechanism. In a theory by Didier Paillard, the climate is modeled as having three states, called interglacial, mild glacial and full glacial, and the state changes depend on the volume of ice as well as the insolation.

See:

• Milankovitch cycle, Azimuth Library

• John Baez, Mathematics of the environment (part 10). This gives an exposition of Paillard’s theory.

~~ The~~ Anytime~~ Azimuth~~ we~~ demo~~~~ program~~~~ approximates~~~~ the~~~~ behavior~~~~ of~~~~ the~~~~ system~~~~ by~~~~ stepping~~~~ through~~~~ time.~~~~ We~~~~ will~~~~ continue~~~~ to~~~~ use~~~~ ‘differential’~~~~ and~~~~ ‘derivative’,~~~~ although~~~~ this~~~~ is~~~~ technically~~~~ incorrect,~~~~ and~~~~ the~~~~ noise~~~~ does~~~~ not~~ have a~~ derivative.~~ function~~ The~~ that~~ behavior~~ has~~ of~~ two~~ the~~ local~~ output~~ minima,~~ signal~~ we~~ is~~ can~~ modelled~~ use~~ using~~ that~~ a~~~~ stochastic~~~~ differential~~~~ equation~~~~ (SDE),~~~~ which~~~~ specifies~~~~ the~~~~ derivative~~ as a function~~ of~~ to~~ time,~~ define~~ the~~~~ current~~~~ signal~~~~ value,~~~~ and~~ a~~ random~~ bistable~~ noise~~ system.~~ process.~~ For instance, consider the function V(x) = x^{4}/4 - x^{2}/2:

The program’s SDE sets the deterministic component of the derivative to a bistable function of the current signal value plus a sine function of time:

```
plot.bistable <- function() {
x <- seq(from=-2, to=2, length.out=1001)
par(mar=c(3, 6, 2, 2))
matplot(x, x^4/4-x^2/2, type='l', ylab=expression(x^4/4-x^2/2))
}
plot.bistable()
png(filename="C:/Users/Work/Desktop/x.png")
plot.bistable()
dev.off()
```

~~ DerivDeterministic(t,~~ Imagine~~ x)~~ this~~ =~~ as~~ SineWave(t,~~ a~~ amplitude,~~ surface~~ frequency)~~ on~~ +~~ which~~ Bistable(x),~~ a metal ball rolls about. Further suppose there is an oscillating magnetic field which pulls the ball to and fro. To repeat a theme, if the amplitude is not too great, the ball will oscillate back and forth in one of the two wells. Now if there as also a bunch of noise, say if it were hailing, and the hailstones were randomly colliding with the ball, then from time to time the ball might change state to the other well.

~~ where~~ Let’s~~ Bistable(x)~~ look at the field of derivatives of this function: V’(x) = x (1 - x^{2}~~ ).~~ ) = Bistable(x)

~~ Alone,~~ The roots of this give the~~ sine~~ stable~~ wave~~ equilibria~~ would~~ at~~ cause~~ 1 and -1, and the~~ output~~ unstable~~ signal~~ equilibrium~~ to~~ at~~ vary~~ 0,~~ sinusoidally.~~ which divides the real numbers info two basins of attraction, one for -1, and the other for 1.

Now let’s analyze the bistable polynomial, which has roots at -1, 0 and 1. The root at 0 is an unstable equilibrium, and -1 and 1 are stable. The basin of attraction for -1 is all the negative numbers, the basin for 1 is the positive numbers, and the basins are naturally separated by the unstable equilibrium.

We~~ treat~~ now~~ each~~ talk~~ basin~~ about~~ as~~~~ one~~~~ of~~ the~~ digital~~ discrete~~ states~~ model,~~ of~~ which~~ a~~ is~~ bistable~~ implemented~~ system.~~ in the demo program.

~~ Now~~ The~~ let’s~~ Azimuth~~ put~~ demo~~ the~~ program~~ sine~~ implements~~ wave~~ a~~ and~~ discrete~~ the~~ model,~~ bistable~~ which~~ polynomial~~ exhibits~~ together.~~ stochastic~~ If~~ resonance.~~ the~~~~ wave~~~~ is~~~~ weak,~~~~ the~~~~ system~~~~ will~~~~ gravitate~~~~ towards~~~~ one~~~~ of~~~~ the~~~~ attractors,~~~~ and~~~~ then~~~~ continue~~~~ to~~~~ oscillate~~~~ around~~~~ it~~~~ thereafter~~~~ –~~~~ never~~~~ leaving~~~~ the~~~~ basin.~~~~ But~~~~ if~~~~ the~~~~ wave~~~~ is~~~~ large,~~~~ it~~~~ will~~~~ force~~~~ the~~~~ system~~~~ to~~~~ oscillate~~~~ between~~~~ the~~~~ two~~~~ basins.~~

~~ Now~~ We~~ let’s~~ use~~ include~~ the~~ noise.~~ potential~~ Suppose~~ function~~ the~~ just~~ sine~~ described,~~ wave~~~~ was~~~~ large~~~~ enough~~~~ at~~~~ its~~~~ peaks~~~~ to~~~~ pull~~~~ the~~~~ system~~~~ close~~~~ to~~~~ zero,~~ but~~ not~~ using~~ enough~~~~ to~~~~ cross~~~~ it~~~~ over.~~~~ Then~~~~ a~~~~ noise~~~~ event~~~~ occurring~~~~ near~~~~ the~~~~ peak~~~~ of~~~~ the~~~~ sine~~~~ wave~~~~ could~~~~ push~~~~ the~~~~ system~~~~ over~~~~ to~~~~ the~~~~ other~~~~ side.~~~~ Stochastic~~~~ resonance~~~~ will~~~~ occur.~~*discrete time*; the derivative is assumed to be constant over the interval between time points. In addition, a random number is added to the discretely sampled derivative. This gives us a discrete-time difference equation:

delta X(t) = Bistable(Xt) * delta t + SineWave(t) + randomSample(t) = Xt (1 - Xt) delta t + WaveAmp sin(t * WaveFreq) + NoiseAmp * GaussianSample(t)

~~ We’ve~~ Note~~ talked~~ that~~ about~~ this is the~~ idea~~ discrete version of~~ stochastic~~ a~~ resonance,~~~~ touched~~~~ on~~~~ some~~~~ of~~~~ its~~~~ applications,~~~~ and~~~~ described~~~~ the~~~~ mathematical~~~~ model~~~~ of~~~~ stochastic~~~~ resonance~~~~ used~~~~ in~~~~ the~~~~ Azimuth~~~~ demo~~~~ program.~~~~ Next~~~~ time~~~~ we~~~~ will~~~~ look~~~~ into~~~~ this~~~~ program:~~~~ how~~~~ to~~~~ use~~~~ it,~~~~ how~~~~ it~~~~ works,~~~~ and~~~~ how~~~~ to~~~~ modify~~~~ it~~~~ to~~~~ develop~~~~ new~~~~ programs.~~Stochastic differential equation.

*Next time* we will look into the Azimuth demo program: how to use it, how it works, and how to modify it to develop new programs.

category: blog