From notebook to service

Two different programs

A notebook is optimised for exploration: cells run out of order, state accumulates, and the interesting output is a number on screen. A service is optimised for repetition: the same input gives the same output, there is no hidden state, latency is bounded and failure has a defined shape. Moving between them is a rewrite of the boundary, not a copy of the cells.

The most common mistake is to import the notebook into the service. It works on the author’s machine because the missing cells were run earlier. In production the first request fails on a name that was never defined, and the traceback points at a line that looks correct.

The audit is simple: restart the kernel and run every cell in order. If the notebook fails, it was never a program. If it succeeds, the run order is still implicit, so the next step is to move the logic into functions and call them from one entry point. A cell that depends on a variable created three cells earlier is a function argument waiting to be written down.

The inference contract

The service promises an input schema, an output schema, a latency budget and an error behaviour. The schema is not decoration: it decides what happens to a missing field, a string where a number was expected, or a category the model has never seen. Validate at the edge and reject with a reason.

The output contract matters just as much. A prediction without its model version, its units and its confidence is hard to debug and impossible to compare after a rollback. Return the score, the label and the version together.

Errors need their own contract. A malformed request is the caller’s fault and should be a 4xx with a field name; a model that cannot load is the service’s fault and should be a 5xx that pages someone. Returning 200 with a default prediction for a bad row hides the failure until a human notices the numbers are wrong.

A typed endpoint with FastAPI

from fastapi import FastAPI
from pydantic import BaseModel

class Request(BaseModel):
    user_id: str
    amount: float

app = FastAPI()

@app.post("/score")
def score(request: Request) -> dict:
    return {"user_id": request.user_id, "risk": model.predict(request.amount)}

Separate training from serving

Training may read a warehouse, take hours and use a GPU. Serving reads one row, takes milliseconds and must not import the training notebook. Share the feature code and the model definition; do not share the globals, the file paths or the plotting.

A good split is three functions: train returns the parameters, predict takes the parameters and one row, and main wires them together. The first two are testable without a network or a file system, which is the whole point.

The shared part is the feature transformation, not the training loop. Put it in a small library with its own tests, import it from both paths, and version it with the model. If the two paths need different code, that difference is a decision to document, not a detail to discover in production.

The artefact is the handoff

A model is code plus weights plus preprocessing plus the version of the library that produced it. Save all four together, give the bundle an immutable name, and load it by name. A pickle without its preprocessing is a bug waiting for the first real request.

The artefact also gives you a rollback. If the previous bundle still exists and still loads, undoing a bad release is a pointer change. If it does not, the rollback is a rebuild under pressure, which is a different and much worse operation.

Name the artefact with the model, the data snapshot and the commit, and load it through a registry rather than a path. The registry gives you a list of what exists, who promoted it and when. A directory of pickles with names like final_v2 is a registry that has stopped being honest.