Pixels and channels

A grid of numbers

A pixel is one sample of light. A grayscale image stores one number per pixel. A colour image stores three, one for red, green and blue. A 224 by 224 colour image is therefore 150,528 numbers, and a batch of 32 of them is nearly five million. A batch is the group of images processed together. That count is why vision models are memory-bound long before they are compute-bound. Memory-bound means the cost is set by moving data, not by arithmetic.

Coordinates start at the top left. The x axis runs right and the y axis runs down. That matches how images are stored, not how graphs are drawn. A point is addressed as row then column, so the first index selects the height and the second selects the width. Swapping them gives a transposed image that still looks like an image.

Resolution is a sampling decision, not a property of the scene. Downsampling throws away detail permanently. Upsampling invents values that were never measured. A model trained on 224-pixel crops has no idea what the same object looks like at 4,000 pixels, unless the pipeline resizes consistently.

Channels and layout

An image library usually returns height, width and channels: (224, 224, 3). A channel is one colour plane, such as red or blue. A framework usually wants channels, height and width: (3, 224, 224), with a batch axis in front. The numbers are identical; only the order of the axes changes. A transpose at the boundary is the whole conversion. Forgetting it is the most common shape error in vision.

Channel order is a separate promise. PIL and most cameras give RGB; OpenCV gives BGR by default. A model trained on RGB and fed BGR sees red and blue swapped. That is not an error and does not crash. It performs worse, and the cause is invisible in the loss curve. Convert explicitly at the point where the image enters your code.

The batch axis adds a fourth dimension, (N, C, H, W). A single image is (C, H, W). Adding the batch axis with unsqueeze gives (1, C, H, W), and the model sees the same rank it saw during training. Most shape errors at inference are a missing batch axis, not a wrong model.

From a file to a tensor

from PIL import Image
import numpy as np

image = Image.open("cat.png").convert("RGB")
array = np.asarray(image)
print(array.shape, array.dtype)          # (height, width, 3) uint8

tensor = np.transpose(array, (2, 0, 1)) / 255.0
print(tensor.shape, tensor.min(), tensor.max())

Range, dtype and the bugs they cause

An 8-bit image holds integers from 0 to 255. Adding ten to a white pixel does not clip to 255 in NumPy. It wraps to 9, because the result is stored in the same unsigned byte. A byte is the 8-bit storage unit, and unsigned means it cannot hold a negative number. Brightness adjustments, blends and normalisation all need a float copy first. Normalisation rescales values to a standard range. The wrap is silent and produces a plausible-looking but wrong image.

Frameworks expect float values, usually scaled to zero through one and then standardised by a mean and a standard deviation. The scaling is not cosmetic. A raw 0 to 255 input into a layer with small weights produces enormous activations and saturated nonlinearities. Know the range at every boundary, and assert it once at the entrance.

Dtype is the other half. Dtype means the numeric type of each value. A float64 array silently doubles memory and can slow a GPU path that expects float32. A bool mask is not an image, and an integer label array is not a feature map. A feature map is a grid of responses produced by a layer. Print dtype, shape, min and max once per batch in development. Four numbers catch most pipeline bugs before the model does.

Metadata is part of the image

A JPEG can carry an orientation flag that tells the viewer to rotate it. The pixels are stored sideways; the flag makes them look upright. A loader that ignores EXIF gives you a rotated image and a model that learned the wrong orientation. EXIF is the metadata stored inside the file. Most libraries expose the flag, and most pipelines forget it.

Colour profiles and alpha channels are the other two quiet inputs. An alpha channel stores transparency, so an image with one has four channels, not three. A model that expects three will either fail or silently take the first three. Resizing also changes the aspect ratio, which is width divided by height, unless you choose the crop deliberately. These are not edge cases. They are the normal state of real image files.

A loader is part of the model contract. The same resize interpolation, the same colour conversion and the same EXIF handling must run at training and inference. Otherwise the model sees a distribution it was never trained on. Interpolation is how new pixels are estimated when an image is resized. Save the preprocessing configuration next to the weights. It is as much a part of the model as the parameters.