The Experiment

Lesson 4 of 6. A second repository, and a second project. The answer is on branch step-4-experiment of plesty-demo-pol-pl.

take_rows.py took four rows and printed them. That is a script, not an experiment: nothing was recorded, nothing could be resumed, and nothing could watch it.

An experiment fixes that. It reads a configuration, checks the instruments it needs are the ones on the other end, works out the full list of steps up front, runs them in order, and writes each result to disk the moment it has it. This lesson builds one.

What the framework gives you

You do not write any of that machinery. plesty.lib.experiment provides it, and an experiment is a subclass that fills in two things — what the steps are, and what one step does:

The measurement is a new project, separate from the bench: the bench is the instruments, and one bench runs many experiments over its life.

Scaffold it

Same tool as lesson 1, different template:

mkdir plesty-demo-pol-pl && cd plesty-demo-pol-pl
uv run plesty init experiment

Installed pre-push hook: uv run plesty check
Initialized experiment project at: …/plesty-demo-pol-pl
Python module name: demo_pol_pl
Import as: plesty.demo_pol_pl

The name is taken from the directory again, and again there is no git init to do — the scaffold makes the repository and its first commit.

What experiment gives you that default did not:

It also prints what to do next, which is the outline of this lesson: implement build_plan() and the step methods, name the instruments the run needs, then preview the plan without hardware.

Take it at its word before writing anything:

uv sync
uv run python -m plesty.demo_pol_pl --dry-run

Plan: 3 step(s), content hash 909903064651…
  measure[x=0]: measure_point({'x': 0})
  measure[x=1]: measure_point({'x': 1})
  measure[x=2]: measure_point({'x': 2})

An experiment that runs before you have written a line of it — three example steps calling an empty method. Nothing is connected, which is what --dry-run is for.

What is missing is the measurement. Build it from the bottom: one row first, then the schedule that repeats it.

Bring the rig across

The step needs the rig, so the rig comes with you. Copy the file you wrote in lesson 3 into this package:

cp ../plesty-demo-bench/plesty/demo_bench/rig.py plesty/demo_pol_pl/rig.py

Not a line of it changes. It was written against an interface, not against a script, and this is the first time that pays.

The placeholder rig, and why it goes

plesty init does not know what instruments you have, so it wrote a rig with one imaginary instrument in it. __main__.py opens with this:

RIG = {"device": ["identity"]}

Read it as a sentence: this experiment needs one sub-device, called device, and its server must have an operation called identity. A name, and the operations that name must answer to. build_devices below it turns that into the sub-device configuration you met in lesson 3 —

{"device": {"requires": ["identity"]}}

— which is the same requires key you wrote three of. Your DEVICES is this declaration already made, for real instruments, with timeouts beside it:

    "hwp": {
        "timeout_ms": 5000,
        "connect_deadline_s": 10,
        "requires": ["home_stage", "move_absolute", "get_position"],
    },

So RIG is a second, emptier copy of something the project already has, and two declarations of one rig is one too many — the one you do not edit is the one that goes stale. Delete RIG.

That leaves __main__.py with nothing to build, which is what the file you copied is for. At the top, the scaffold imported the generic composite it was assembling by hand:

from plesty.lib.device.composite_device import CompositeDevice

Nothing assembles a composite here any more, so that import goes, and the one that replaces it is your own class:

from .rig import DemoRig

Then build_devices returns the rig instead of assembling a config:

def build_devices() -> DemoRig:
    """Build the rig this experiment measures with.

    No address appears here, and no sub-device either: ``DemoRig`` declares
    all three and reads ``HWP_ADDRESS``, ``SPEC_ADDRESS`` and ``PM_ADDRESS``
    from the environment itself, so a deployment is configured by its ``.env``
    and the experiment never knows a hostname. Called again on every start-up
    retry, so an edited ``.env`` or a server started in the meantime is picked
    up.
    """
    return DemoRig()

The second name, and why it goes too

Under RIG the scaffold left one more:

#: Sub-devices the run can do without — the operator may skip these when they
#: fail the preflight. Everything else stops the run.
OPTIONAL: frozenset[str] = frozenset()

This is a different idea from RIG, which is why it is worth a moment. RIG says what the rig is; OPTIONAL says which parts of it the measurement could survive losing. Name an instrument here and, if its server fails the check at start-up, the operator is offered "go on without it" instead of just "abort" — a monitor thermometer, one camera of four.

This measurement has no such instrument. Three instruments, all load-bearing: a sweep with no powermeter is not this sweep. Delete OPTIONAL as well.

That leaves the call the two of them fed, further down:

    devices, _ = connect_rig(lambda _skip: build_devices())

connect_rig always hands the builder a set of sub-devices to leave out — that is how the "go on without it" answer reaches the rig, on the retry after the operator gives it. With nothing optional, that set is always empty, so the argument is taken and ignored. The optional= argument and the if skipped: message the scaffold wrote around this call go with OPTIONAL.

One last thing is now unused: the --device-address argument, which overrode an address the rig resolves for itself. Delete the parser.add_argument block that defines it.

Which leaves .env, the last file. Replace the scaffold's single DEVICE_ADDRESS:

HWP_ADDRESS=tcp://localhost:5652
SPEC_ADDRESS=tcp://localhost:5653
PM_ADDRESS=tcp://localhost:5651

Use your own ports. plesty-server status prints what the bench allocated. Lesson 3 showed what a wrong one looks like, and it does not look like a wrong port.

Then the two variables under them, which are about where the measurement lands. These are not yours to choose: they have to be the disk the spectrometer is allowed to write to, which it will tell you if you ask it.

uv run python -c "
from plesty.lib.service import build_client
with build_client('tcp://localhost:5653') as spec:
    print(spec.get_data_path())
"

/Users/Shared/demo-bench/frames

That is the DEVICE_DATA_PATH you gave it when you declared it in lesson 2, and a server started with one will write nowhere else. Put it in .env, twice:

PLESTY_DATA_DIR=/Users/Shared/demo-bench/frames
PLESTY_DATA_MOUNT=/Users/Shared/demo-bench/frames

Twice, because on a real bench the instrument is on another machine and the two spellings of one disk differ: D:\data as the acquiring host writes it, /Volumes/lab-data as this machine reads it. PLESTY_DATA_DIR is the writer's spelling — the one the instrument is told — and PLESTY_DATA_MOUNT the reader's, which is also where this experiment puts the run directory. One machine, one path, written twice.

Choose your own directory instead and the first thing the run does is refuse, naming both paths:

RuntimeError: ValueError: Data path '/home/you/tmp/bench/frames/demo_pol_pl_20260821-213806/raw'
is outside the allowed roots ['/home/you/.plesty/demo-bench/frames'].

The instrument is guarding its own disk, and it is right to: a device that accepted any path it was sent would write a measurement wherever a typo pointed. The fix is always the same — make PLESTY_DATA_DIR the root in the second list, or a directory beneath it.

Nothing to install for any of that: the scaffold pinned a plesty-lib new enough for the rig it wrote, which is the same one the rig you copied needs.

One row

This is the atomic unit: everything the measurement does, once. In plesty/demo_pol_pl/experiment.py, measure_row replaces the scaffold's measure_point:

    def measure_row(self, angle_deg: float, exposure_s: float) -> dict[str, Any]:
        """Measure one row: turn the plate, expose, read the power.

        The returned dictionary is persisted as one line of ``records.jsonl``,
        which is what the monitors read. Everything needed to interpret the row
        is in the row — including the angle the stage reports having reached,
        which is not always the angle that was asked for.

        Args:
            angle_deg:  Half-wave-plate angle for this row.
            exposure_s: Spectrometer exposure time in seconds.

        Returns:
            The row document: angles, frame path, power, and the window the
            reading covered.
        """
        reported_deg = self.devices.set_hwp(angle_deg)
        t_start = _now()
        row = self.devices.acquire_row(exposure_s)
        return {
            "hwp_deg": angle_deg,
            "hwp_reported_deg": reported_deg,
            "exposure_s": exposure_s,
            "t_start": t_start,
            "t_end": _now(),
            **row,
        }

with a timestamp helper at module level, above the class, and the import it needs at the top of the file — the scaffold's imports cover everything else this lesson writes:

import datetime

def _now() -> str:
    """The current UTC instant, as the records store it."""
    return datetime.datetime.now(datetime.UTC).isoformat()

Two things are worth pausing on.

Returning a dictionary is the whole of persistence. There is no "write the record" call to make. A step that returns something non-None has it appended to records.jsonl along with its provenance — the step id, the operation, the parameters it was called with. A step that returns None is journalled as complete and stores nothing, which is what you want for a step that only moves something.

The record carries both angles. hwp_deg is what the plan asked for; hwp_reported_deg is what the stage says it reached. On a real bench those differ, and the second one is the one the measurement actually happened at. Recording only the first is the kind of loss you discover a year later.

Notice also what measure_row does not contain: no address, no port, no device command. It asks the rig for two things. Everything about how those reach three servers was settled in lesson 3.

Set up before the first row

Two things are the same for every row, so they are pushed once per run rather than once per row: where the frames go, and how the instruments are set. The scaffold left a setup hook for exactly this:

    async def setup(self) -> None:
        """Connect the rig and preflight it, then tell it where to write.

        The inherited setup connects the composite and preflights it — every
        sub-device must expose the operations its ``requires`` names, or the
        run stops before anything is measured.

        Two things are then pushed once per run rather than once per row.
        The frame directory, because a frame belongs to the run that asked
        for it: the spectrometer writes into this run's ``raw`` folder on the
        shared disk and sends back a path, never an image. And the
        acquisition settings, which do not change during a sweep — a write
        per row would be one round trip per row buying nothing.
        """
        await super().setup()
        if self.raw_dir:
            # Only the spectrometer writes files; naming it is honest, and
            # routing the stage and the meter would fail or mean nothing.
            self.devices.set_data_path(self.raw_dir, devices=["spec"])
        self.devices.configure(
            exposure_s=float(self.config.get("exposure_s", 0.3)),
            wavelength_nm=float(self.config.get("wavelength_nm", 780.0)),
        )

super().setup() is what connects the rig, so both calls after it have something to talk to. teardown already disconnects; it needs nothing added.

Images never cross the network

That one set_data_path line is where the platform's rule about measurement data is enforced, and it is worth stating plainly: a device sends back a path, never a blob. A spectrum, a camera frame, a waveform — the instrument writes it to disk itself, and what travels over ZMQ is the string saying where. Nothing else scales: a 4-megapixel camera at 10 Hz is 80 MB/s, and a network that carries it has nothing left for the commands that drive the measurement.

The experiment is what decides where, and it decides per run:

Naming devices=["spec"] is not an optimisation. The call fails rather than silently doing nothing, so routing a stage that writes no files would abort the run — in a rig where one instrument writes, saying which one is the honest form.

The guard matters too: with no shared disk configured, raw_dir is None, nothing is routed, and the spectrometer keeps writing wherever it was told to at declare time. The run still works; the frames are simply not gathered per run.

And the metadata that makes those files interpretable never goes to the disk beside them. It goes in the record — the row measure_row returns, carrying the angle asked for, the angle reached, the exposure, the window, and the path. A frame with no record is an orphan; the record is what makes it data.

The plan

One row is settled. A plan is how many of them, and in what order: the whole list of steps, worked out before the first one runs. Each step has an id, an operation, and the parameters that operation is called with.

Deciding everything up front is what makes a run resumable. The plan is written to disk and content-hashed; a resumed run rebuilds it and compares, so a configuration edited halfway through is refused rather than quietly stitched into the middle of a measurement.

It also means build_plan() must be deterministic and device-free — no timestamps, no random ids, and nothing asked of an instrument.

The configuration

The plan is built by build_plan(), in code. config/default.yaml is where its parameters are set — the values you change between runs without touching the module. Replace the example points: 3:

# The half-wave plate sweep. 0-180 degrees in 2-degree steps is 91 rows,
# which is the grid the replayed dataset was measured on.
hwp:
  start_deg: 0.0
  stop_deg: 180.0
  step_deg: 2.0

# Spectrometer exposure per row, and the wavelength the powermeter corrects
# for. The emission this measures sits around 780 nm.
exposure_s: 0.3
wavelength_nm: 780.0

Tuning values go here, in a tracked file, because they are part of what a run was. Addresses and credentials never do — those are .env, which is not tracked.

Building the steps

Back in experiment.py, replace build_plan:

    def build_plan(self) -> Plan:
        """Build the deterministic measurement schedule from the configuration.

        One step per half-wave-plate angle. Step ids must be stable across
        calls — no timestamps or randomness — and params must stay
        JSON-serializable, because the plan is written to disk and compared
        against on resume.
        """
        hwp = self.config.get("hwp", {})
        exposure_s = float(self.config.get("exposure_s", 0.3))
        steps = [
            Step(
                id=f"hwp[{angle:06.2f}]",
                op="measure_row",
                params={"angle_deg": angle, "exposure_s": exposure_s},
            )
            for angle in _angles(
                float(hwp.get("start_deg", 0.0)),
                float(hwp.get("stop_deg", 180.0)),
                float(hwp.get("step_deg", 2.0)),
            )
        ]
        return Plan(steps, config=self.config)

op="measure_row" is the method you just wrote — a step says what to call and with which arguments, nothing more. One of the contract tests checks that every op resolves to a real method, so a typo fails at plesty check rather than at row 47 of a measurement.

The angles come from a second module-level helper, beside _now:

def _angles(start_deg: float, stop_deg: float, step_deg: float) -> list[float]:
    """The sweep angles, endpoint included.

    Counted rather than accumulated: adding ``step_deg`` 90 times drifts, and
    a step id built from a drifted angle would not match the one the plan was
    resumed from.
    """
    count = int(round((stop_deg - start_deg) / step_deg))
    return [round(start_deg + index * step_deg, 6) for index in range(count + 1)]

The docstring is the whole reason it is not a while angle <= stop. Step ids are compared as text on resume, and hwp[180.00] built from an angle that drifted to 179.99999999 is a different step.

Preview it — still with nothing connected:

uv run python -m plesty.demo_pol_pl --dry-run

Plan: 91 step(s), content hash bcc1b48f020f…
  hwp[000.00]: measure_row({'angle_deg': 0.0, 'exposure_s': 0.3})
  hwp[002.00]: measure_row({'angle_deg': 2.0, 'exposure_s': 0.3})
  …
  hwp[180.00]: measure_row({'angle_deg': 180.0, 'exposure_s': 0.3})

91 steps, endpoints included, and no instrument was touched to produce them. This is the cheapest way to check a configuration before committing an instrument to it.

Name the experiment

Runs are named after the experiment, and the base class falls back to the class name — which here is Experiment, so every run would be called experiment_20260821-191701. Set it once, in __init__:

        # Without a name the base takes the class name, and every run would be
        # called `experiment_<timestamp>`. The name is what a run directory is
        # called and what the viewer filters on, so it says which measurement.
        kwargs.setdefault("name", "demo_pol_pl")
        super().__init__(devices=devices, run_root=run_root, **kwargs)

Lesson 6 filters runs by that name. It is easier to set now than to rename a directory of finished measurements later.

Run it

Bring the bench up if it is down, then:

uv run python -m plesty.demo_pol_pl

… | plesty.lib.device.composite_device | DemoRig: connecting sub-device(s) hwp, spec, pm
… | plesty.lib.device.composite_device | hwp: - -> connecting (tcp://localhost:5652, timeout 5000 ms, deadline 10 s)
… | plesty.lib.device.composite_device | DemoRig ready in 0.0 s: hwp=…, spec=…, pm=…
… | plesty.lib.device.composite_device | Preflight hwp @ tcp://localhost:5652: ok (SIM-APT (S/N: 55123456, firmware: 1.0.0))
… | plesty.lib.device.composite_device | Preflight spec @ tcp://localhost:5653: ok (Mock LightField Spectrometer (no hardware))
… | plesty.lib.device.composite_device | Preflight pm @ tcp://localhost:5651: ok (THORLABS,PM100D,MOCK0001,1.0.0)
… | demo_pol_pl | Run demo_pol_pl_20260821-191701: 91 step(s), run dir …/frames/demo_pol_pl_20260821-191701, raw frames …/frames/demo_pol_pl_20260821-191701/raw
… | demo_pol_pl | Setup: connecting devices
… | demo_pol_pl | Step 1/91 hwp[000.00] (measure_row)
… | demo_pol_pl | Step 1/91 done in 0.5 s
…
… | demo_pol_pl | Teardown: disconnecting devices
… | demo_pol_pl | Run demo_pol_pl_20260821-191701 completed (91 steps) in 42.3 s.
Run completed: demo_pol_pl_20260821-191701

About forty seconds. The line naming the run is the one to keep: it is the directory everything lands in.

The three Preflight lines are the start-up the scaffold wrote for you, and they happen before the run has an id. connect_rig builds the rig, asks each server who it is, and checks it has the operations requires names — the ones you listed per instrument in lesson 3. They come round a second time after Setup: connecting devices, because the base class checks the rig it was handed rather than trusting whoever built it: one round trip per instrument, and an experiment constructed in a notebook gets the same guarantee as this one.

When the rig is not there

Change one port in .env to something with nothing behind it, and start again:

… | plesty.lib.experiment.preflight | The rig is not ready — 1 device(s) failed the preflight:
  - pm: Sub-device 'pm' unreachable at tcp://localhost:5659 after 10.1 s: RuntimeError(…)
      address tcp://localhost:5659 (from PM_ADDRESS in .env / the environment)
Check that the pm server(s) are running and that PM_ADDRESS point at them
(a wrong port connects fine but is not the expected device).
[r] retry (after fixing .env or starting the server)  [a] abort
>

It names the instrument, the address, and the variable the address came from — which is the one thing you cannot work out from a stack trace. Fix .env in another window, press r, and it builds the rig again from scratch; the server you just started is picked up the same way.

Answer a and the process stops on Aborted by the operator: pm not ready. Somewhere with no terminal to answer on — a cron job, CI — it does not ask at all:

Not an interactive terminal — aborting. Fix the addresses or start the server(s) and run again.
Nothing was measured — not ready: pm.

Either way: non-zero exit, no run directory, nothing half-measured. That is the point of preflighting at all: the alternative is a four-hour sweep that fails on row 47 because a server was never up.

What a run leaves behind

One directory on the shared disk, named after the run:

frames/demo_pol_pl_20260821-191701/
  plan.json        the frozen schedule and the configuration it came from
  journal.jsonl    what happened, in order: started, each step, finished
  records.jsonl    one line per step that returned something
  raw/             91 spectra, written by the spectrometer itself

Three text files and the blobs they point at, side by side because they describe one measurement — but written by different machines. The three are append-only and written by this process as the run goes; raw/ is written by the instrument, over no network at all.

The records are what the next two lessons read:

{"index": 0, "saved_at": "…", "type": "value",
 "provenance": {"step_id": "hwp[000.00]", "op": "measure_row",
                "params": {"angle_deg": 0.0, "exposure_s": 0.3}},
 "value": {"hwp_deg": 0.0, "hwp_reported_deg": 0.0, "exposure_s": 0.3,
           "t_start": "…", "t_end": "…",
           "data_file": "…/demo_pol_pl_20260821-191701/raw/mock_…-731414.spe",
           "power_w": 6.112988e-06}}

Everything needed to interpret that row is in the row, and the frame itself is not: only its path travels. provenance is the half the framework adds — which step produced this, calling what, with which arguments — so a row can be traced back to the plan that asked for it without keeping the plan open beside it.

Check a few rows against the measurement being replayed:

uv run python -c "
import json
rows = [json.loads(l) for l in open('<run_dir>/records.jsonl')]
for i in (0, 6, 30, 90):
    v = rows[i]['value']
    print(f\"{v['hwp_deg']:6.1f}  {v['power_w']*1e6:8.3f} uW\")
"

   0.0     6.113 uW
  12.0     0.263 uW
  60.0    24.149 uW
 180.0     5.784 uW

Those are the powers measured on 4 August 2026, to the microwatt. You have re-run a real measurement.

Fix the tests

uv run pytest

4 failed, 6 passed

Not a problem — the arithmetic of having changed the measurement. The six that pass are the five contract tests and the config loader, which check the shape of an experiment and keep passing throughout. The four that fail are the scaffold's own: two written against the RIG you deleted, two that run the experiment against a fake composite which has none of the rig's methods.

Replacing them is mostly writing a fake rig, and the interesting part is how little it needs:

class FakeRig:
    """Duck-typed stand-in for :class:`~plesty.demo_pol_pl.rig.DemoRig`."""

    def connect_all(self) -> None: ...
    def disconnect_all(self) -> None: ...
    def release(self) -> None: ...
    def configure(self, exposure_s: float, wavelength_nm: float) -> None: ...
    def set_data_path(self, path: str, devices: list[str] | None = None) -> None: ...
    def set_hwp(self, angle_deg: float) -> float: ...
    def acquire_row(self, exposure_s: float) -> dict[str, object]: ...

Seven methods, no server, no ZMQ, no dataset. The experiment was written against the rig's interface rather than against instruments, so the whole run — plan, journal, records, resume — is testable in a tenth of a second. The full version is on the answer branch.

One of the four is worth keeping rather than deleting: the rig test, rewritten against the rig you brought across.

def test_build_devices_declares_all_three_instruments() -> None:
    """The rig start-up builds is the whole rig, with what the preflight checks."""
    # Patched: building a rig for real would open a client per sub-device.
    with patch.object(CompositeDevice, "__init__", return_value=None) as composite_init:
        build_devices()
    config = composite_init.call_args.args[0]
    assert set(config) == {"hwp", "spec", "pm"}
    assert all(config[name]["requires"] for name in config)

The patch is the whole trick, and it is worth reading twice: a DemoRig() built for real opens a client per sub-device and would need three servers running to test a dictionary.

Worth adding one the scaffold could not have written, because the routing is yours:

def test_frames_are_routed_into_the_run(tmp_path: Path) -> None:
    """The spectrometer is sent this run's raw directory, and only that."""
    devices = FakeRig()
    experiment = Experiment(
        devices=devices, config=CONFIG, run_root=tmp_path,
        data_dir=str(tmp_path / "share"),
    )
    run_id = asyncio.run(experiment.run())
    assert devices.data_paths == [(f"{tmp_path / 'share'}/{run_id}/raw", ["spec"])]

Where a run's frames land is a claim about the measurement, not a detail — a sweep whose spectra went to last week's folder is a sweep you have to take on trust. This test is what stops that being possible without anyone noticing.

11 passed

Then the gates:

uv run plesty check

Checking against standard: pixel (no release tag yet — prototype default)
  ✓ Metadata & Namespace
  ✓ Code Hygiene (lint)
  ✓ Code Hygiene (format)
  ✓ Code Hygiene (types)

All checks passed.

Resume

Stop a run with Ctrl-C partway through, and it tells you how to pick it up:

Run interrupted — resume with --resume <run_id>.

uv run python -m plesty.demo_pol_pl --resume <run_id>

… | demo_pol_pl | Resuming run demo_pol_pl_20260821-191025: 25/91 steps already completed.
… | demo_pol_pl | Run demo_pol_pl_20260821-191025 completed (91 steps) in 30.7 s.

The journal is what makes that work: steps recorded as complete are skipped, and the plan is rebuilt and compared against the stored one first. Edit config/default.yaml and try to resume, and it refuses — a run is one measurement or it is nothing.

On a real bench this is the difference between losing an evening to a stalled instrument and losing four minutes.

What you left out

This experiment is deliberately smaller than plesty-pol-pl, the production module for this measurement. What that one adds, and this one does not:

None of that changes the shape you have built. It is the same build_plan, the same step methods returning documents, the same journal.

Next

You have a measurement that records what it did. The next two lessons are about watching it: the monitor contract, and the viewer that docks three of them into one window.