# Contents

## Introduction

Much of what is discussed below is either referred to as non-equilibrium thermodynamics or stochastic thermodynamics.

## Markov processes and the master equation

A continuous-time, finite state Markov process is a pair $(V,H)$ where $V$ is a finite set of states and $H : \mathbb{R}^V \to \mathbb{R}^V$ is an infinitesimal stochastic,

(1)$H_{ij} \geq 0 \; \: i \neq j \quad \text{and} \quad \; \sum_i H_{ij} = 0,$

Hamiltonian generating the time evolution of a population distribution $p(t) \in \mathbb{R}^V$ via the master equation

(2)$\frac{dp}{dt} = Hp .$

The infinitesimal stochastic condition requires that the off-diagonal components of $H$ are non-negative and that the columns sum to zero, meaning that $H_{ii} = -\sum_{j \neq i} H_{ji}$, or that the diagonal elements are equal to minus the sum of all the other elements in that column. The entry $H_{ij}$ is interpreted as the transition rate from state $j$ to state $i$. Thus the diagonal entries are equal to minus the sum of the outgoing rates from a certain state.

An equilibrium distribution $q \in \mathbb{R}^V$ is a population distribution that does not change with time,

(3)$\frac{dq}{dt} = Hq = 0.$

### Currents

In terms of the indices, the master equation reads:

(4)$\frac{dp_i}{dt} = \sum_{j} H_{ij}p_j.$

We can use the infinitesimal stochastic property of $H$ to rewrite this as:

(5)$\frac{dp_i}{dt} = \sum_{j} H_{ij}p_j - H_{ji}p_i.$

If we define the current from $j$ to $i$ as $J_{ij} = H_{ij}p_j - H_{ji}p_i$, we can write the master equation in the simple form:

(6)$\frac{dp_i}{dt} = \sum_j J_{ij}.$

Note that this definition of current requires an implicit choice that a positive current $J_{ij}$ corresponds to a net flow population into the $i^{\text{th}}$ state.

### Detailed Balance

We say an equilibrium distribution $q$ satisfies detailed balance if

(7)$H_{ij}q_j = H_{ji}q_i.$

Equilibrium distributions always satisfy $\frac{dq_i}{dt} = \sum_j J_{ij} = 0$ for all $i$, but for an equilibrium satisfying detailed balance each term in this sum vanishes identically.

### Non-equilibrium steady states

For an equilibrium not satisfying detailed balance there can be non-zero individual currents flowing $J_{ij} \neq 0$ in such a way that $\sum_j J_{ij} = 0$ is still satisfied. Such an equilibrium is typically referred to as a non-equilibrium steady state.

### Representation of Markov processes using labelled graphs

One can represent a Markov process as a directed graph $(V,E,s,t)$, where $V$ is a finite set of states, $E$ is a finite set of edges, and $s,t : E \to V$ are source and target maps respectively, along with a map $r : E \to [0,\infty)$ giving the labels or the transition rates associated to each edge, i.e. for an edge $e \in E$, $r(e) = H_{ t(e) s(e) }$.

### The cycle basis

Given a labelled, directed graph $G=(V,E,s,t,r)$ one can forget the directedness of the edges and consider the undirected graph $(V,E)$. A spanning tree is then a connected subgraph containing all the vertices $V$ of $G$, but with no cycles. A connected undirected graph with no cycles is called a tree. Given a spanning tree $T = (V,E_T )$ the leftover edges $E-E_T$ form the set of chords of the graph. Adding any chord to the spanning tree produces a cycle. The set of cycles obtained when adding each of the chords individually to a given spanning tree provides one choice of a cycle basis for the underlying graph $G$.

For Markov processes one cannot discard the information regarding the direction of the edges. Similarly we will need a directed cycle basis. To achieve this we choose an arbitrary orientation on all of the cycles in our basis, such as clockwise. It is easy to see that for a graph $G$ with $n$-vertices $|V|=n$ a spanning tree $T=(V,E_T)$ on those n-vertices will have $n-1$ edges, $|E_T| = n-1$. Therefore we have $\nu = |E|-|E_T| = |E| - n + 1$ chords and hence $\nu$ elements in our cycle basis. Therefore let us denote the cycle basis by $\{C_1,C_2,....C_{\nu} \}$.

Given an arbitrary directed cycle $C$ on the graph $G$ we can write $C = \sum_{\alpha=1}^{\nu} (C,C_{\alpha})C_{\alpha}$. Here we have introduced an inner product on cycle space $(C,C_{\alpha}) = S_{\alpha}(C)S_{\alpha}(C_{\alpha}) \quad 1 \leq \alpha \leq \nu$, where

(8)$S_{\alpha}(C) = \left\{ \begin{array}{ccc} +1 & \text{if} & \alpha || C \\ -1 & \text{if} & -\alpha || C \\ 0 & \text{if} & \alpha \notin C. \end{array} \right.$