The technique uses the *paraxial approximation* of ray optics, i.e., all rays are assumed to be at a small angle (θ) to the optical axis of the system. The approximation is valid as long as sin(θ)≈θ (where θ is measured in radians).

The technique is based on two reference planes perpendicular to the optical axis of the system. At the first plane, a light ray crosses the plane at a distance *x*_{1} from the optical axis at an angle θ_{1}. Some distance along the optical axis, at the second plane, the ray crosses at distance *x*_{2} and an angle θ_{2}.

These quantities are related by the expression:

- .

For example, if there is free space between the two planes, the ray transfer matrix is given by:

- ,

- ,

- ,

- .

- .

RTM analysis is particularly used when modelling the behaviour of light in optical resonators, such as those used in lasers. At its simplest, an optical resonator consists of two identical facing mirrors of 100% reflectivity and radius of curvature *R*, separated by some distance *d*. For the purposes of ray tracing, this is equivalent to a series of identical thin lenses of focal length *f*=*R*/2, each separated from the next by length *d*. This construction is known as a *lens duct* or *lens waveguide*. The RTM of each section of the waveguide is thus **M**=**LS** as shown above, substituting *f*=*R*/2.

RTM analysis can now be used to determine the *stability* of the waveguide (and equivalently, the resonator). That is, it can be determined under what conditions light travelling down the waveguide will be periodically refocussed and stay within the waveguide. To do so, we can find all ray vectors where the output of each section of the waveguide is equal to the input vector multiplied by some real or complex constant λ:

- .

- ,

After *N* passes through the system, we have:

- .

- ,

The technique may be generalised for more complex resonators by constructing a suitable matrix **M** for the cavity from the matrices of the components present.

The matrix formalism is also useful to describe Gaussian optics. If we have a Gaussian beam of wavelength λ, radius of curvature *R* and beam radius ω, it is possible to define a "complex beam parameter" *q* by:

- .

- ,