A radiologist reading a chest CT finds a nodule, a small, round growth in the lung, under 3 cm across, that can be an early cancer, and clicks on it. The GUI records the click as an index into the image array: column 330, row 200, slice 40. That is enough to find the voxel again in the image file, but the surgeon planning the operation needs different answers:
To answer the first two questions we need to turn an image index into a position in the patient, in millimeters. This requires the affine (or affine transformation matrix). Measuring the nodule's size also needs care, because voxels are rarely cubes. Finding the same nodule in next year's scan takes one more step, registration.
What the file stores and what the surgeon needs. Left: a point marked in the left lung of an axial CT slice is stored as an index into the array, with no units and no directions. Right: to act on it, we need its side, its size and its position in the patient in millimeters, which the geometry of the image provides. CT: TotalSegmentator test data (Wasserthal et al.), Apache License 2.0.
The surgeon's first question, left lung or right, is about direction. Directions in the body are named after the patient: left–right, anterior–posterior (front to back) and superior–inferior (head to feet) (figure). CT and MRI files give every position in millimeters along these three axes.
The patient axes of LPS (DICOM) and RAS (NIfTI), seen from the front.
DICOM calls this the patient-based coordinate system: the axes follow the patient, but the point (0, 0, 0) is left to the equipment. On CT it typically lies at the center of the bore within the slice, and at a table position fixed at the start of the scan. It has no anatomical meaning and usually differs between sessions.
Two common conventions give these axes opposite signs. DICOM, the format scanners write, uses LPS: $x$, $y$ and $z$ grow toward the patient's left, back and head. NIfTI, the usual research format, uses RAS: right, front and head. A point keeps its $z$, and its $x$ and $y$ change sign. LPS and RAS name only the directions of the axes. They do not say where (0, 0, 0) is, or whether the coordinates belong to the scanner, to another image, or to a template such as MNI space.
The orientation says which way each array axis points in the patient, as a unit vector. In our axial CT, in LPS, moving along a row (increasing $i$) goes toward the patient's left, so $\mathbf{u}_i=(1,0,0)$. Moving down a column (increasing $j$) goes toward the back, $\mathbf{u}_j=(0,1,0)$, and moving to the next slice (increasing $k$) goes toward the head, $\mathbf{u}_k=(0,0,1)$. In a tilted slice a vector has several nonzero components, the cosines of its angles with $x$, $y$ and $z$, hence the name direction cosines. The pixels do not show them; the file's header stores them.
So an orientation error, such as a wrong patient position entered at the scanner or a bug in conversion code, raises no warning: the file loads and looks like a scan. Flip one axis and the patient is mirrored, and our nodule would be reported in the right lung. Flip two, as when LPS is read as RAS, and the slice turns 180°, with the spine at the top (figure).
One axial CT slice drawn three ways by a radiological viewer, which puts the patient's right (R) on the left of the screen and anterior (A) at the top. Left: correct; the liver is on the patient's right. Middle: the left–right axis flipped, a mirror image; the liver appears on the left. Right: x and y negated, as when LPS is read as RAS; a 180° rotation, with the spine at the top. CT: Alextogus, Wikimedia Commons, CC0.
Directions alone do not place the nodule: we also need to know how far apart the voxels are and where the array starts. The spacing along an axis is the distance between the centers of neighboring voxels. It is the sampling interval $\Delta$ of Eq. 2.13.1, now with one value per axis: $\Delta_i$, $\Delta_j$ and $\Delta_k$. In our CT, each slice has 512 × 512 pixels covering a field of view 358 mm wide, so neighboring pixels are 0.7 mm apart, and the slices are 2.5 mm apart (2.4 CT — Computed tomography).
We also need to know the location of the origin $\mathbf{o}$, the position in the patient of the first voxel, index (0, 0, 0). In our CT, $\mathbf{o}=(-175,-180,-300)$ mm in LPS. Because the indices grow toward the patient's left, back and head, the first voxel is the opposite corner of the scanned volume: right, front, feet. Like every position in patient coordinates, $\mathbf{o}$ is measured from the scanner's (0, 0, 0).
With this information we can place the nodule: start at the origin and step along each array axis by the index times the spacing. The axes of our CT run along $x$, $y$ and $z$, so voxel (330, 200, 40) lies at $x=-175+330\times0.7=56$ mm, $y=-180+200\times0.7=-40$ mm and $z=-300+40\times2.5=-200$ mm in LPS, about 5 cm to the patient's left of the center of the scan, in the left lung (figure below). In general, voxel $(i,j,k)$ lies at $\mathbf{x}=\mathbf{o}+i\,\Delta_i\,\mathbf{u}_i+j\,\Delta_j\,\mathbf{u}_j+k\,\Delta_k\,\mathbf{u}_k$. Appending a 1 to the index, $(i,j,k,1)$, turns this sum into one matrix product, and the matrix is the affine $\mathbf{A}$:
$$ \begin{pmatrix} x\\ y\\ z\\ 1\end{pmatrix}= \underbrace{\begin{pmatrix} \Delta_i u_{i,x} & \Delta_j u_{j,x} & \Delta_k u_{k,x} & o_x\\ \Delta_i u_{i,y} & \Delta_j u_{j,y} & \Delta_k u_{k,y} & o_y\\ \Delta_i u_{i,z} & \Delta_j u_{j,z} & \Delta_k u_{k,z} & o_z\\ 0 & 0 & 0 & 1 \end{pmatrix}}_{\mathbf{A}} \begin{pmatrix} i\\ j\\ k\\ 1\end{pmatrix}\tag{2.14.1} $$
Read $\mathbf{A}$ column by column: the first three columns are the array axes, each a direction scaled by its spacing, and the last is the origin. Applied to every voxel, $\mathbf{A}$ places the whole image in the patient (figure, panel 3). Its inverse $\mathbf{A}^{-1}$ goes the other way, from a position in the patient to an index, usually a fractional one between voxel centers. We need it whenever we look up a point of one image in another (below).