The Azimuth Project
Blog - network theory (part 19) (Rev #8, changes)

Showing changes from revision #7 to #8: Added | Removed | Changed

This page is a blog article in progress, written by John Baez and Jacob Biamonte. To see discussions of this article while it was being written, visit the Azimuth Forum.

joint with Jacob Biamonte

It’s time to resume the network theory series! It’s been a long time, so we’ll have to remind you of some things. Last time we started looking at a simple example: a diatomic gas.

Two atoms can recombine to form a diatomic molecule:

A+AA 2 A + A \to A_2

and conversely, a diatomic molecule can break apart into two atoms:

A 2A+A A_2 \to A + A

We can draw both these reactions using a Petri net:

where we’re writing BB instead of A 2A_2 to abstract away some detail that’s just distracting here. Or, equivalently, we can use a chemical reaction network:

Last time we looked at the rate equation for this chemical reaction network, and found equilibrium solutions of that equation. Now let’s look at the master equation, and find equilibrium solutions of that. This will serve as a review of three big theorems.

The master equation

We’ll start from scratch, in case you’re just tuning in. The master equation is all about how atoms or molecules or rabbits or wolves or other things interact randomly and turn into other things. So, let’s write write ψ m,n\psi_{m,n} for the probability that we have mm atoms of AA and nn molecule of BB in our container. These probabilities are functions of time, and master equation will say how they change.

First we need to pick a rate constant for each reaction. Let’s say the rate constant for this reaction is some number α>0\alpha > 0:

A+AB A + A \to B

while the rate constant for this reaction is some number β>0\beta > 0:

BA+A B \to A + A

Then the master equation says

ddtψ m,n = α(m+2)(m+1)ψ m+2,n1(t)αm(m1)ψ m,n(t)+ β(n+1)ψ m2,n+1(t)βnψ m,n \begin{array}{ccl} \displaystyle{ \frac{d}{d t} \psi_{m,n}} &=& \alpha (m+2)(m+1)\psi_{m+2,n-1}(t) - \alpha m(m-1) \psi_{m,n}(t) + \\ && \beta (n+1) \psi_{m-2,n+1}(t) - \beta n \psi_{m,n} \end{array}

Yuck!

Normally we don’t show you such nasty equations. Indeed the whole point of this series is to show that by packaging the equations in a better way, we can understand them using high-level concepts instead of mucking around with millions of scribbled symbols. But we thought we’d show you the engine under the hood, just once.

Each term has a meaning. For example,

α(m+2)(m+1)ψ m+2,n1(t) \alpha (m+2)(m+1)\psi_{m+2,n-1}(t)

means that the reaction A+ABA + A \to B will tend to increase the probability of there being mm AA atoms and nn BB molecules if we start with 2 more AA‘s and 1 fewer BB’s. This reaction can happen in (m+2)(m+1)(m+2)(m+1) ways if we start with m+2m+2 atoms of AA. And it happens at a probabilistic rate proportional to the rate constant for this reaction, α\alpha.

We won’t go through the rest of the terms. It’s a good exercise to do so, but there could easily be a typo in the formula, since it’s so long and messy. So let us know if you find one!

To simplify this mess, the key trick is to introduce a generating function that summarizes all the probabilities in a single power series:

Ψ= m,n0ψ m,nz 1 mz 2 n \Psi = \sum_{m,n \ge 0} \psi_{m,n} z_1^m \, z_2^n

It’s a power series in two variables, z 1z_1 and z 2z_2, since we have two chemical species: AA‘s, and BB’s.

Using this trick, the master equation looks like

ddtΨ=HΨ \frac{d}{d t} \Psi = H \Psi

where the Hamiltonian HH is a sum of terms, one for each reaction. This Hamiltonian is built from operators that annihilate and create AA‘s and BB’s. The annihilation and creation operators for AA atoms are:

a 1=z 1,a 1 =z 1 \displaystyle{ a_1 = \frac{\partial}{\partial z_1} , \qquad a_1^\dagger = z_1 }

The annihilation operator differentiates our power series with respect to the variable z 1z_1. The creation operator multiplies it by that variable. Similarly, the annihilation and creation operators for BB molecules are:

a 2=z 2,a 2 =z 2 \displaystyle{ a_2 = \frac{\partial}{\partial z_2} , \qquad a_2^\dagger = z_2 }

In Part 8 we explained a recipe that lets us stare at our chemical reaction network and write down this Hamiltonian:

H=α(a 2 a 1 2a 1 2a 1 2)+β(a 1 2a 2a 2 a 2) H = \alpha (a_2^\dagger a_1^2 - {a_1^\dagger}^2 a_1^2) + \beta ({a_1^\dagger}^2 a_2 - a_2^\dagger a_2)

As promised, there’s one term for each reaction. But each term is itself a sum of two: one that increases the probability that our container of chemicals will be in a new state, and another that decreases the probability that it’s in its original state. We get a total of four terms, which correspond to the four terms in our previous way of writing the master equation.

Puzzle: Show that this new way of writing the master equation is equivalent to the previous one.

Noether's Equilibrium theorem solutions

Now we will look for all equilibrium solutions of the master equation: in other words, solutions that don’t change with time. So, we’re trying to solve

HΨ=0 H \Psi = 0

Given the rather complicated form of the Hamiltonian, this seems tough. The challenge looks more concrete but perhaps even more scary if we go back to our original formulation. We’re looking for probabilitiesψ m,n\psi_{m,n}, nonnegative numbers that sum to one, such that

α(m+2)(m+1)ψ m+2,n1αm(m1)ψ m,n+β(n+1)ψ m2,n+1(t)βnψ m,n=0 \alpha (m+2)(m+1)\psi_{m+2,n-1}- \alpha m(m-1) \psi_{m,n} + \beta (n+1) \psi_{m-2,n+1}(t) - \beta n \psi_{m,n} = 0

for all m,n0 m,n m \ge 0 . andnn. (By the way, in case you’re wondering, we say ψ i,j\psi_{i,j} is zero if either ii or jj is negative.)

If This you equation hadn’t looks been rather reading unpleasant, all but these notes up to now, about the only good news you’d instantly see is that these it’s equations arelinear, so a linear combination of solutions is again a solution. This lets us simplify the problem using a conserved quantity.

Clearly, there’s a quantity that the reactions here don’t change:

What’s that? It’s the number of BB‘s plus twice the number of AA’s. When one BB is born, two AA’s go away. And when two AA’s are born, one BB goes away.

(Of course the secret reason is that BB is a diatomic molecule made of two AA ‘s. But you’d be able to follow the logic here even if you didn’t know that, just by looking at the chemical reaction network… and sometimes this more abstract approach is handy! Indeed, the way chemists first discovered that certain molecules are made of certain atoms is by seeing which reactions were possible.) possible and which weren’t.)

Suppose we start in a situation where we know for sure that the number of BB‘s plus twice the number of AA’s equals some number kk:

ψ m,n=0unless2m+2n=k \psi_{m,n} = 0 \; unless \; m+2n 2m+n = k

Then we know Ψ\Psi is initially of the form

Ψ= 2m+2n=kψ m,nz 1 mz 2 n \Psi = \sum_{m+2n \sum_{2m+n = k} \psi_{m,n} z_1^m z_2^n

But since the number of BB‘s plus twice the number of AA’s is conserved, if Ψ\Psi obeys the master equation it will continue to be of this form.

Put a fancier way, we know that if a solution of the master equation starts in this subspace of the space of all formal power series: series in two variables:

L k={Ψ:Ψ= 2m+2n=kψ m,nz 1 mz 2 nforsomeψ m,n} L_k = \{ \Psi: \; \Psi = \sum_{m+2n \sum_{2m+n = k} \psi_{m,n} z_1^m z_2^n for some \psi_{m,n} \}

it will remain in that subspace. So, because the master equation is linear, we can take any solution Ψ\Psi and write it as a linear combination of solutions Ψ k\Psi_k, one in each subspace L kL_k (where k=0,1,2,k = 0,1,2,\dots).

In particular, we can do this for an equilibrium solution Ψ\Psi. And then all the solutions Ψ k\Psi_k are also equilibrium solutions—since if one of them changed with time, Ψ\Psi would too.

This means we can just look for equilibrium solutions in the subspaces L kL_k. If we find these, we can get all equilibrium solutions by taking linear combinations.

We Once can we’ve formalize noticed this this, idea the a equation bit we’re by trying noting to it’s solve a doesn’t special look case so of bad anymore:Noether’s Theorem for Markov processes. We saw this theorem in Part 11, and later some of us wrote a paper about it, which goes further:

α(m+2)(m+1)ψ m+2,n1αm(m1)ψ m,n+β(n+1)ψ m2,n+1(t)βnψ m,n=0 \alpha (m+2)(m+1)\psi_{m+2,n-1}- \alpha m(m-1) \psi_{m,n} + \beta (n+1) \psi_{m-2,n+1}(t) - \beta n \psi_{m,n} = 0

Note that if the pair of subscripts m,nm, n obey 2m+n2m+n = k,thesameistruefortheotherpairsofsubscriptshere.Thatsbecauseofourconservationlaw:thenumberof, the same is true for the other pairs of subscripts here. That's because of our conservation law: the number of Bsplustwicethenumberof's plus twice the number of Asisconserved!Soourequationrelatesthevaluesof's is conserved! So our equation relates the values of \psi_{m,n}forallchoicesof for all choices of m,n$ lying on this line segment:

2m+n=k,m0,n02m+n = k , \qquad m \ge 0, n \ge 0

If you think about it a minute, you’ll see that if we know two of these values, we can keep using our equation to recursively work out all the rest.

So, there are at most two linearly independent equilibrium solutions of the master equation in each subspace L kL_k.

Why at most two? Well, we have to be a bit careful about what happens at the ends of the line segment: remember that ψ m,n\psi_{m,n} is defined to be zero when mm or nn becomes negative. But this is the sort of nitty-gritty stuff that’s much more fun to do than to watch someone else do, so we won’t work through it here.

Instead, we’ll move on to a more high-level approach to this problem, where we throw some theorems at it. But first, note observation is just that our horrible equation

α(m+2)(m+1)ψ m+2,n1αm(m1)ψ m,n+β(n+1)ψ m2,n+1(t)βnψ m,n=0 \alpha (m+2)(m+1)\psi_{m+2,n-1}- \alpha m(m-1) \psi_{m,n} + \beta (n+1) \psi_{m-2,n+1}(t) - \beta n \psi_{m,n} = 0

resembles the usual discretized form of the equation

\displaystyle {\frac{d^2 \psi}{d x^2} = 0

namely:

ψ n12ψ n+ψ n+1=0 \psi_{n-1} - 2 \psi_{n} + \psi_{n+1} = 0

And this makes sense, since we get

\displaystyle {\frac{d^2 \psi}{d x^2} = 0

by taking the heat equation:

\displaystyle \frac{\partial \psi}{\partial t} = {\frac{\partial^2 \psi}{\partial x^2}

and setting the term with a time derivative equal to zero. So what we’re doing is a lot like looking for equilibrium solutions of the heat equation. And that makes perfect sense, since the heat equation describes how heat smears out as little particles of heat randomly move around. True, there aren’t really little particles of heat, but the heat equation also describes the diffusion of any other substance as particles randomly move around undergoing Brownian motion. Similarly, our master equation describes a random walk on each line

2m+n=k,m0,n02m+n = k , \qquad m \ge 0, n \ge 0

and the equilibrium solutions arise when the probabilities ψ m,n\psi_{m,n} have diffused out as much as possible.

Noether's theorem

We simplified our task of finding equilibrium solutions of the master equation by finding a conserved quantity. The idea of simplifying problems using conserved quantities is fundamental to physics: this is why physicists are so enamored with quantities like energy, momentum, angular momentum and so on.

Nowadays physicists often use ‘Noether’s theorem’ to get conserved quantities from symmetries. There’s a very simple version of Noether’s theorem for quantum mechanics, but in Part 11 we saw a version for stochastic mechanics, and it’s that version that is relevant now. Here’s a paper which explains it in detail:

• John Baez and Brendan Fong, Noether’s theorem for Markov processes.

We don’t really need these Noether’s ideas theorem in now, what since follows, we so found you can skip the rest conserved of quantity this and section exploited if it you without want. even But noticing they’re the interesting. symmetry. Nonetheless it’s interesting to see how it relates to what we’re doing.

For the reaction we’re looking at now, the idea is that the subspaces L kL_k are eigenspaces of an operator that commutes with the Hamiltonian HH. It follows from standard math that a solution of the master equation that starts in one of these subspaces, stays in that subspace.

What is this operator? It’s built from ‘number operators’. The <b>number operator</a> for AA‘s is

N 1=a 1 a 1 N_1 = a_1^\dagger a_1

and the number operator for BB‘s is

N 2=a 2 a 2 N_2 = a_2^\dagger a_2

A little calculation shows

N 1z 1 mz 2 n=mz 1 mz 2 n,N 1z 1 mz 2 n=mz 1 mz 2 n N_1 z_1^m z_2^n = m z_1^m z_2^n, \quad N_1 z_1^m z_2^n = m z_1^m z_2^n

so the idea is that the eigenvalue of N 1N_1 is the number of AA‘s, while the eigenvalue of N 2N_2 is the number of BB’s.

As a consequence, the eigenvalue of the operator 2N 1+N 22 N_1 + N_2 is the number of BB‘s plus twice the number of AA’s:

(2N 1+N 2)z 1 mz 2 n=(2m+n)z 1 mz 2 n (2 N_1 + N_2) z_1^m z_2^n = (2m + n) z_1^m z_2^n

So, the spaces L kL_k we saw a minute ago are just the eigenspaces of this operator:

L k={Ψ:(2N 1+N 2)Ψ=kΨ} L_k = \{ \Psi : \; (2 N_1 + N_2) \Psi = k \Psi \}

As we’ve seen, solutions of the master equation that start in one of these eigenspaces will stay there. These

is, thanks to Noether’s theorem and a bit of extra work, equivalent to the fact that

Equilibrium states of the master equation

Here we will use the Anderson-Craciun-Kurtz theorem to work out the corresponding equilibrium states of the master equation. Brendon proved this in relation to what we consider here back in Part X.

Let

Ψ:=e z 1c 1+z 2c 2 \Psi := e^{z_1c_1 + z_2c_2}

Then

HΨ=[r 1(z 1 2c 2z 2c 2)+r 2(z 2c 2 2z 1 2c 1 2)]Ψ H\Psi = \left[ r_1(z_1^2c_2 - z_2c_2) + r_2(z_2c_2^2 - z_1^2c_1^2)\right]\Psi

and for Ψ0\Psi \neq 0

(r 1c 2r 2c 1 2)z 1 2+(r 2c 1 2r 1c 2)z 2=0 (r_1 c_2 - r_2 c_1^2) z_1^2 + (r_2 c_1^2 - r_1c_2)z_2 = 0

which vanishes for

r 1r 2=c 1 2c 2 \frac{r_1}{r_2} = \frac{c_1^2}{c_2}

Noether’s theorem

Now we will show how Noether’s theorem relates the conserved quantity 2N A+N B2N_A + N_B to a symmetry.

Conservation of particle number

The Hamiltonians that arise in Petri net field theory have a very particular general form. Not every Hamiltonian in this vast class preserves particle number (since we can have exponential growth or decay for instance). What we want to do is to find a good way to characterize those Hamiltonians that do preserve particle number. We want to understand symmetries in general. Those of you following the posts will recall the commutation relations from Part X. These are going to be relevant here too.

Just a reminder, the number operator for a single species is

N i=a i a i N_i = a_i^\dagger a_i

and the number operator for all the species is a sum over the single species

N= iN i N = \sum_i N_i

We will derive a few results at the end of the post. If you think we are telling the truth, you don’t need to check them, but they are there if you want to be bored with these sort of details. To get you into the mood…

It can be shown (using induction) that

[a,a k]=ka k1 [a, a^\dagger^k] = k a^\dagger^{k-1}
[a k,a ]=ka k1 [a^k,a^\dagger] = k a^{k-1}
  • exercise. Let [a,a ]=1[a, a^\dagger] = 1 be the base case and assume [a,a k]=ka k1[a, a^\dagger^k] = k a^\dagger^{k-1} and show that these assumptions imply the formula for k+1k+1 and hence or otherwise, prove the first commutation relation listed above by induction.

Now for some results.

  • (Lemma I). We arrive at the following commutation relations among number operators and general Hamiltonians in Petri net field theory
    (1)[N i,H]= τTr(τ)[n i(τ)m i(τ)]a n(τ)a m(τ) [N_i, H] = \sum_{\tau\in T} r(\tau)[n_i(\tau) - m_i(\tau)]a^\dagger^{n(\tau)}a^{m(\tau)}

This supporting lemma can be used to prove a range of things related to symmetries in the very type of Hamiltonians we are considering here.

  • (Theorem I — total particle conservation). The following quantity
    (2) τTr(τ)[n(τ)m(τ)]a n(τ)a m(τ)=0 \sum_{\tau\in T} r(\tau)[n(\tau) - m(\tau)]a^\dagger^{n(\tau)}a^{m(\tau)} = 0

    vanishes identically iff HH preserves particle number.

So to check if the total number of particles are conserved during evolution under some Hamiltonian, all one has to do is check Theorem I. The Hamiltonian we consider here does not conserve total particle number. However, the reversible reaction John did in Part 10 did.

  • (Exercise). In Network Theory Part 10 John considered the reversible reaction with Hamiltonian

    (3)H=(a b )(βbαa) H = (a^\dagger - b^\dagger)(\beta b - \alpha a)

    Use Theorem I to show that this reversible reaction conserves particle number.

  • (Theorem II — particle conservation symmetry). Given a Hamiltonian acting on kk particle species, there exists kk positive integer choices for ω n\omega_n which causes the following to vanish identically

    (4) i τTr(τ)ω i[n i(τ)m i(τ)]a n(τ)a m(τ)=0 \sum_i\sum_{\tau\in T} r(\tau)\omega_i[n_i(\tau) - m_i(\tau)]a^\dagger^{n(\tau)}a^{m(\tau)} = 0

    iff HH has a particle conservation symmetry.

As will soon be seen, this is precisely the case here. In other words, there exists ω 1\omega_1 and ω 2\omega_2 that take positive integer values which case the above quantity to vanish.

For our system to have a particle conservation symmetry, we must show that

(5)[N 1+2N 2,H]=[N 1,H]+2[N 2,H]=0 [N_1 + 2N_2, H] = [N_1, H] + 2[N_2,H] = 0

This vanishes since, from Lemma I, we calculate that

(6)[N 1,H]=2r 1a a b2r 2b aa [N_1, H] = 2r_1 a^\dagger a^\dagger b -2 r_2 b^\dagger a a
(7)[N 2,H]=r 1a a b+r 2b aa [N_2, H] = -r_1 a^\dagger a^\dagger b + r_2 b^\dagger a a

Here, Theorem II applies, are we are able to find two values, ω 1=1\omega_1=1 and ω 2=2\omega_2=2 that cause the commutator to vanish. The Hamiltonian therefore has a particle number conservation symmetry.

Master equation

Now for the master equation approach. Our Hamiltonian is given as

H=r 1(a a bb b)+r 2(b aaa a aa) H = r_1 (a^\dagger a^\dagger b - b^\dagger b) + r_2 (b^\dagger a a - a^\dagger a^\dagger a a)

and the evolution operator at time tt is given as

(8)V=e tH= k=0 H kk!t k V = e^{t H} = \sum_{k=0}^\infinity \frac{H^k}{k!}t^k

At each order in kk , we have a term that corresponds to the Hamiltonian HH acting kk times. One can think of these as alternative histories. In quantum mechanics, there is a spooky thing called coherence, where each of these histories seems to occur concurrently. In stochastic mechanics, each history only occurs with some probability. In terms of mathematical structure, the theories become closely related. While the semantic interpretation might differ, the syntactical form, given by a sum over histories unites quantum and stochastic mechanics. This enables us to e.g. apply tools from quantum mechanics to stochastic mechanics such as Feynman diagrams.

Supporting Material

Proof of Theorems I and II. Sometimes a long calculation can simplify matters. This is the case here, though we don’t want to muddy the water from what’s going on here, as this is just some algebra.

Here we will use the following notation.

a m(τ)a i =a i m i(τ)a i a m(τ) a^{m(\tau)}a^\dagger_i = a^{m_i(\tau)}_i a^\dagger_i a^{m'(\tau)}
a ia n(τ)=a n(τ)a ia i n i(τ) a_i a^\dagger^{n(\tau)} = a^\dagger^{n'(\tau)}a_i a_i^\dagger^{n_i(\tau)}

where the vector m(τ)m'(\tau) has its iith component zero which is why we are able to move term(s) a m(τ)a^{m'(\tau)} to the right. This enables us to express the more general commutation relations,

[a m(τ),a i ]=m i(τ)a i m i(τ)1a m(τ) [a^{m(\tau)},a^\dagger_i] = m_i(\tau)a_i^{m_i(\tau)-1}a^{m'(\tau)}
[a i,a n(τ)]=n i(τ)(a i ) n i(τ)1a n(τ) [a_i, a^\dagger^{n(\tau)}] = n_i(\tau) (a_i^\dagger)^{n_i(\tau)-1}a^\dagger^{n'(\tau)}

Using these relations, it follows that (is better notation possible?)

[H,a i ]= τTr(τ)(a n(τ)a m(τ))a i m i(τ)1a m(τ)m i(τ) [H, a_i^\dagger] = \sum_{\tau\in T} r(\tau)(a^\dagger^{n(\tau)} - a^\dagger^{m(\tau)})a_i^{m_i(\tau)-1} a^{m'(\tau)} m_i(\tau)

and also

[a i,H]= τTr(τ)[n i(τ)a n i(τ)1a n(τ)m i(τ)a m i(τ)1a m(τ)]a m(τ) [a_i, H] = \sum_{\tau\in T} r(\tau)[n_i(\tau)a^\dagger^{n_i(\tau)-1}a^\dagger^{n'(\tau)} - m_i(\tau)a^\dagger^{m_i(\tau)-1}a^\dagger^{m'(\tau)}]a^{m(\tau)}

These will simplify the calculation of the commutation of the Hamiltonian and the number operator.

[ ia i a i,H]= i[a i a i,H]= i(a i [a i,H][H,a i ]a i) [\sum_i a_i^\dagger a_i, H] = \sum_i [a_i^\dagger a_i, H] = \sum_i(a_i^\dagger [a_i, H] - [H, a_i^\dagger]a_i)

For this to vanish, the following quantity must vanish identically.

i(a i K 2K 1a i) \sum_i (a_i^\dagger K_2 - K_1 a_i)

where

a i K 2=a i [a i,H]= i τTr(τ)[n i(τ)a n(τ)m i(τ)a m(τ)]a m(τ) a_i^\dagger K_2 = a_i^\dagger[a_i, H] = \sum_i \sum_{\tau\in T} r(\tau)[n_i(\tau)a^\dagger^{n(\tau)} - m_i(\tau)a^\dagger^{m(\tau)}]a^{m(\tau)}
K 1a i=[H,a i ]a i= i τTr(τ)(a n(τ)a m(τ))a m(τ)m i(τ) K_1 a_i = [H, a_i^\dagger]a_i = \sum_i \sum_{\tau\in T} r(\tau)(a^\dagger^{n(\tau)} - a^\dagger^{m(\tau)})a^{m(\tau)}m_i(\tau)

Both of the theorems then follow from applications of the above.

Solution to exercises

Particle Conservation of the simple reversible reaction. In Network Theory Part 10 John considered the reversible reaction with Hamiltonian

H=(a b )(βbαa) H = (a^\dagger - b^\dagger)(\beta b - \alpha a)

We find its commutation relations with the creation (destruction) operators of both species to be

[H,a ]=α(b a ) [H, a^\dagger] = \alpha (b^\dagger - a^\dagger)
[H,a]=αaβb [H, a] = \alpha a - \beta b
[b ,H]=β(b a ) [b^\dagger, H] = \beta (b^\dagger - a^\dagger )
[b,H]=αaβb [b, H] = \alpha a - \beta b

Now we see that the particle number is conserved for this Hamiltonian by calculating

[N,H]=[a a,H]+[b b,H]=a [a,H]+[a ,H]a+b [b,H]+[b ,H]b=0 [N, H] = [a^\dagger a, H] + [b^\dagger b, H] = a^\dagger[a, H] + [a^\dagger, H]a + b^\dagger[b, H] + [b^\dagger, H]b = 0

This could have also been shown, using Theorem I directly.

New stuff

We’ve been working hard to understand the parallels and differences between quantum and stochastic mechanics. Last time we showed how the methods developed in prior posts can be used to model chemical reaction networks. This time, we are going to report the details of a battle.

This is the same quantum vs. stochastic battle we’ve talked about in prior posts, but this time we are going talk in detail about the odd nature of eigenstates in quantum mechanics, and how we can’t expect this structure in stochastic mechanics.