Imagine photographing your hand and then taking an X-ray of it. In the photograph, you can see skin, fingernails, and perhaps a few veins. In the X-ray, the bones stand out. It may seem obvious, but why do these images reveal different structures when the hand is the same?
The conventional photograph records visible light reflected from the hand. The radiograph records X-rays transmitted through it. The tissues in your hand interact differently with these signals, so each image reveals different properties of the same subject. We call these different imaging modalities because they differ in the physical interactions and measurements.
Examples of the imaging modalities covered in this module. Top row: chest X-ray (2.2; Stillwaterising, CC0); mammogram, MLO view, with a spiculated mass (2.3; Bahman Rasuli, Radiopaedia.org, rID 91430, CC BY-NC-SA 3.0); contrast-enhanced CT of the abdomen at the level of the kidneys (2.4; Alextogus, CC0); FDG PET-CT of a patient with lymphoma (2.5; autoPET data, CC BY 4.0); T2-weighted MRI of the brain (2.6; Novaksean, CC BY-SA 4.0); ultrasound of a 12-week fetus (2.7; Wolfgang Moroder, CC BY-SA 3.0). Bottom row: OCT of the retina with fluid beneath it (2.12; Ashley Ireland, CC BY-SA 3.0); cultured neuroblastoma cells in phase-gradient contrast (2.8; ZEISS Microscopy, CC BY 2.0); endothelial cells in fluorescence (2.9; Jakob Zbaeren, Nikon Small World); H&E-stained rectal mucosa (2.10; Nephron, CC BY-SA 4.0); spatial transcriptomics of a colorectal cancer section, each 8 µm bin colored by its expression cluster (2.11; 10x Genomics, CC BY 4.0, adapted). Images with no other source named are from Wikimedia Commons.
A biomedical image is a spatial representation of physical measurements (of photons, sound waves, magnetic fields, etc) that we use to study a biological system or support medical care. These images let us examine structures and processes all the way from individual cells to whole organisms. Conventional photographs can be biomedical images too: a smartphone photo of a skin lesion is one if it is used for diagnosis.
Choose a location in either of the hand images above. What does the value of that pixel represent? The pixels do not directly represent skin or bone. A pixel value reflects how the tissues interact with the measured signal and how those measurements are processed.
Think of an image as a rule: give it a location, and it returns the value or values associated with that location. The figure shows this for a cell under a microscope. A location on the slide goes in, and one, three, or more than a hundred values come out, depending on the optics and detector.
For a two-dimensional grayscale image,
$$ I:\Omega\rightarrow\mathbb{R},\qquad\Omega\subseteq\mathbb{R}^{2}.\tag{2.1.1} $$
Or, more generally,
$$ I:\Omega\rightarrow\mathbb{R}^{k},\qquad\Omega\subseteq\mathbb{R}^{n}.\tag{2.1.2} $$
Here, $\Omega$ is the field of view: the region of physical space that the image covers. A location $(x,y)$ in $\Omega$ is a real position, measured in millimeters or micrometers. In the figure above, $\Omega$ is the patch of slide inside the dashed box, about 50 µm across. In the radiograph of the hand, $\Omega$ is the surface of the detector. The field of view is not the specimen. It can crop the specimen: the cells continue beyond the dashed box. It can also flatten it: the hand is three-dimensional, but the detector is flat. How the image arises depends on the modality, which is the focus of this module.
In Eq 2.1.2, which describes the image as a function, $n$ coordinates determine the spatial location and $k$ image values come out: $n$ is the number of spatial dimensions of the field of view, and $k$ is the number of values at each location, called channels. In the cell figure the field of view $\Omega$ has two spatial coordinates $n=2$, and the resulting images have different numbers of channels $k$: 1, 3, and 100 or more. A CT scan introduces a third dimension to $\Omega$: it assigns one value to each location in a three-dimensional region of the body, so $n=3$ but it only provides a single intensity channel $k=1$.
A digital image stores these values on a finite grid of locations. An RGB image with height $H$ and width $W$ is stored as an $H\times W\times 3$ array. Each grid element is called a pixel in two dimensions and a voxel in three.
Digitization, going from an analog signal to a digital one, has two steps: sampling, which measures the signal at a finite grid of locations, and quantization, which rounds each value to one of a finite set of levels. We explain both in 2.13 Sampling and quantization.