A continuoustime Markov chain (CTMC) is a continuous stochastic process in which, for each state, the process will change state according to an exponential random variable and then move to a different state as specified by the probabilities of a stochastic matrix. An equivalent formulation describes the process as changing state according to the least value of a set of exponential random variables, one for each possible state it can move to, with the parameters determined by the current state.
An example of a CTMC with three states is as follows: the process makes a transition after the amount of time specified by the holding time—an exponential random variable , where i is its current state. Each random variable is independent and such that , and . When a transition is to be made, the process moves according to the jump chain, a discretetime Markov chain with stochastic matrix:
Equivalently, by the property of competing exponentials, this CTMC changes state from state i according to the minimum of two random variables, which are independent and such that for where the parameters are given by the Qmatrix
Each nondiagonal entry can be computed as the probability that the jump chain moves from state i to state j, divided by the expected holding time of state i. The diagonal entries are chosen so that each row sums to 0.
A CTMC satisfies the Markov property, that its behavior depends only on its current state and not on its past behavior, due to the memorylessness of the exponential distribution and of discretetime Markov chains.
Let be a probability space, let be a countable nonempty set, and let ( for "time"). Equip with the discrete metric, so that we can make sense of right continuity of functions . A continuoustime Markov chain is defined by:^{[1]}
Note that the row sums of are 0: or more succinctly, . This situation contrasts with the situation for discretetime Markov chains, where all row sums of the transition matrix equal unity.
Now, let such that is measurable. There are three equivalent ways to define being Markov with initial distribution and rate matrix : via transition probabilities or via the jump chain and holding times.^{[5]}
As a prelude to a transitionprobability definition, we first motivate the definition of a regular rate matrix. We will use the transitionrate matrix to specify the dynamics of the Markov chain by means of generating a collection of transition matrices on (), via the following theorem.
Theorem: Existence of solution to Kolmogorov backward equations.^{[6]} — There exists such that for all the entry is differentiable and satisfies the Kolmogorov backward equations:

(0) 
We say is regular to mean that we do have uniqueness for the above system, i.e., that there exists exactly one solution.^{[7]}^{[8]} We say is irregular to mean is not regular. If is finite, then there is exactly one solution, namely and hence is regular. Otherwise, is infinite, and there exist irregular transitionrate matrices on .^{[a]} If is regular, then for the unique solution , for each , will be a stochastic matrix.^{[6]} We will assume is regular from the beginning of the following subsection up through the end of this section, even though it is conventional^{[10]}^{[11]}^{[12]} to not include this assumption. (Note for the expert: thus we are not defining continuoustime Markov chains in general but only nonexplosive continuoustime Markov chains.)
Let be the (unique) solution of the system (0). (Uniqueness guaranteed by our assumption that is regular.) We say is Markov with initial distribution and rate matrix to mean: for any nonnegative integer , for all such that for all
.^{[10]}


(1) 
Using induction and the fact that we can show the equivalence of the above statement containing (1) and the following statement: for all and for any nonnegative integer , for all such that for all such that (it follows that ),

(2) 
It follows from continuity of the functions () that the trajectory is almost surely right continuous (with respect to the discrete metric on ): there exists a null set such that .^{[13]}
Let be right continuous (when we equip with the discrete metric). Define
let
be the holdingtime sequence associated to , choose and let
be "the state sequence" associated to .
The jump matrix , alternatively written if we wish to emphasize the dependence on , is the matrix
We say is Markov with initial distribution and rate matrix to mean: the trajectories of are almost surely right continuous, let be a modification of to have (everywhere) rightcontinuous trajectories, almost surely (note to experts: this condition says is nonexplosive), the state sequence is a discretetime Markov chain with initial distribution (jumpchain property) and transition matrix and (holdingtime property).
We say is Markov with initial distribution and rate matrix to mean: for all and for all , for all and for small strictly positive values of , the following holds for all such that :
where the term is if and otherwise , and the littleo term depends in a certain way on .^{[15]}^{[16]}
The above equation shows that can be seen as measuring how quickly the transition from to happens for , and how quickly the transition away from happens for .
Communicating classes, transience, recurrence and positive and null recurrence are defined identically as for discretetime Markov chains.
Write P(t) for the matrix with entries p_{ij} = P(X_{t} = j  X_{0} = i). Then the matrix P(t) satisfies the forward equation, a firstorder differential equation
where the prime denotes differentiation with respect to t. The solution to this equation is given by a matrix exponential
In a simple case such as a CTMC on the state space {1,2}. The general Q matrix for such a process is the following 2 × 2 matrix with α,β > 0
The above relation for forward matrix can be solved explicitly in this case to give
Computing direct solutions is complicated in larger matrices. The fact that Q is the generator for a semigroup of matrices
is used.
The stationary distribution for an irreducible recurrent CTMC is the probability distribution to which the process converges for large values of t. Observe that for the twostate process considered earlier with P(t) given by
as t → ∞ the distribution tends to
Observe that each row has the same distribution as this does not depend on starting state. The row vector π may be found by solving
with the constraint
The image to the right describes a continuoustime Markov chain with statespace {Bull market, Bear market, Stagnant market} and transitionrate matrix
The stationary distribution of this chain can be found by solving , subject to the constraint that elements must sum to 1 to obtain
The image to the right describes a discretetime Markov chain modeling PacMan with statespace {1,2,3,4,5,6,7,8,9}. The player controls PacMan through a maze, eating pacdots. Meanwhile, he is being hunted by ghosts. For convenience, the maze shall be a small 3x3grid and the monsters move randomly in horizontal and vertical directions. A secret passageway between states 2 and 8 can be used in both directions. Entries with probability zero are removed in the following transitionrate matrix:
This Markov chain is irreducible, because the ghosts can fly from every state to every state in a finite amount of time. Due to the secret passageway, the Markov chain is also aperiodic, because the monsters can move from any state to any state both in an even and in an uneven number of state transitions. Therefore, a unique stationary distribution exists and can be found by solving , subject to the constraint that elements must sum to 1. The solution of this linear equation subject to the constraint is The central state and the border states 2 and 8 of the adjacent secret passageway are visited most and the corner states are visited least.
For a CTMC X_{t}, the timereversed process is defined to be . By Kelly's lemma this process has the same stationary distribution as the forward process.
A chain is said to be reversible if the reversed process is the same as the forward process. Kolmogorov's criterion states that the necessary and sufficient condition for a process to be reversible is that the product of transition rates around a closed loop must be the same in both directions.
One method of finding the stationary probability distribution, π, of an ergodic continuoustime Markov chain, Q, is by first finding its embedded Markov chain (EMC). Strictly speaking, the EMC is a regular discretetime Markov chain. Each element of the onestep transition probability matrix of the EMC, S, is denoted by s_{ij}, and represents the conditional probability of transitioning from state i into state j. These conditional probabilities may be found by
From this, S may be written as
where I is the identity matrix and diag(Q) is the diagonal matrix formed by selecting the main diagonal from the matrix Q and setting all other elements to zero.
To find the stationary probability distribution vector, we must next find such that
with being a row vector, such that all elements in are greater than 0 and = 1. From this, π may be found as
(S may be periodic, even if Q is not. Once π is found, it must be normalized to a unit vector.)
Another discretetime process that may be derived from a continuoustime Markov chain is a δskeleton—the (discretetime) Markov chain formed by observing X(t) at intervals of δ units of time. The random variables X(0), X(δ), X(2δ), ... give the sequence of states visited by the δskeleton.