The Monitor

Lesson 5 of 6. No new instrument, no change to the experiment. The answer is on branch step-5-monitor of plesty-demo-pol-pl.

The sweep takes forty seconds, so you watched it by waiting. A real one takes minutes or even hours, and waiting is not a plan: the point of watching is to find out early — that the sample drifted, that the alignment is off, that row 12 is where the signal disappeared and never came back.

This lesson builds the view. It touches nothing you have written: the experiment does not know it is being watched, and cannot be disturbed by it.

Three pieces, and only one of them is yours

Splitting it three ways is what lets one view work everywhere. The monitor never mentions a toolkit, so the same class runs in a window, in a test, or in a script that renders an MP4 on a headless machine. And because it is fed rows rather than reading files, it does not care whether they arrive live or are read back from a run that finished last week.

The record file is the stream

There is no subscription to set up, no port, no callback the experiment has to register. The run already writes records.jsonl, one line per completed step, and a line appears only when its step is fully persisted — the write is the commit. RunSource tails that file by byte offset.

Three consequences worth having in mind before you write anything:

• Attach whenever you like. A source opened halfway through hands you every row from the start on its first poll, so nothing that already happened is lost by starting late.
• Watching costs the run nothing. The source only reads, and after the first poll it reads only the bytes appended since the last one — however long the run has been going, and however many monitors are looking.
• A finished run is watched exactly like a live one. Same file, same code; the run does not have to be over for you to look, and it does not have to be still going either.

A record, and the frame it becomes

Two words do a lot of work in this lesson, and they are not the same thing. A record is a line the run wrote — one completed step, in the experiment's own vocabulary, carrying everything that step measured. You met one at the end of lesson 4; this is the row for 2°:

{"index": 1, "saved_at": "…", "type": "value",
 "provenance": {"step_id": "hwp[002.00]", "op": "measure_row",
                "params": {"angle_deg": 2.0, "exposure_s": 0.3}},
 "value": {"hwp_deg": 2.0, "hwp_reported_deg": 2.0, "exposure_s": 0.3,
           "t_start": "…", "t_end": "…",
           "data_file": "…/demo_pol_pl_20260821-191701/raw/mock_…-731542.spe",
           "power_w": 4.602e-06}}

Seven measured keys, one of them a path to a spectrum too big to inline. The source unwraps value, and that inner dict is what has to be narrowed to the channels a view actually draws.

A frame is what a monitor is fed — the channels it declared, and nothing else:

Frame(values={"hwp_deg": 2.0, "power_w": 4.602e-06},
      seq=1, timestamp="…", source="demo_pol_pl_20260821-191701",
      context={"record": "1", "run_dir": "…/frames/demo_pol_pl_20260821-191701"})

values is the part you draw from, by name: frame["hwp_deg"]. The rest is stamped on by the source and never validated — source says which run it came from, and context carries what is worth passing along but not plotting.

Two of those fields mean something narrower than they look. seq counts the frames a source has delivered, so it does not have to match the record's index — not every record becomes a frame, for a reason the next section comes to. And timestamp is when the frame was made, which is the moment of the poll, not when the step was measured; the step's own clock stayed in the record, as the t_start and t_end this view chose not to declare. Frames are immutable, so a monitor that wants a history keeps its own.

Seven keys in the record, two in the frame. Closing that gap is a job of its own, and it is the last thing you write once the monitor itself is done.

What a Monitor is

Monitor is an abstract base class, and the split is deliberate — you describe the view, the base class runs it:

One declaration and two methods is the whole obligation. That is why the class below is short, and why nothing in it mentions a file, a window, or a device.

Everything else the base class already does. You call these; you do not write them:

Write the monitor

A new file, plesty/demo_pol_pl/monitors.py, and everything it needs comes from one place:

"""Live views over a polarization sweep."""

from typing import Any, Optional

from plesty.lib.monitor import DataSource, Frame, Monitor, TraceData, TraceSpec

plesty.lib.monitor is toolkit-free — importing it pulls in no GUI, which is why this file can be imported by a test, a script, or a headless render job. The five names are the whole vocabulary of the contract: Monitor to subclass, TraceSpec to declare a drawable, TraceData to fill it, Frame for one row in, and DataSource for the stream the monitor may be handed.

A monitor declares what it consumes, declares what it draws, and projects one onto the other — input_schema, traces(), update(), and nothing else is required of it.

class MalusMonitor(Monitor):
    """Power against half-wave-plate angle, filling in as the sweep runs."""

    input_schema = {
        "hwp_deg": {"dtype": "float", "unit": "deg", "description": "Plate angle asked for"},
        "power_w": {"dtype": "float", "unit": "W", "description": "Power during the exposure"},
    }

    def traces(self) -> list[TraceSpec]:
        """Declare the one curve this view draws."""
        return [
            TraceSpec(
                key="malus",
                kind="line",
                label="Power",
                x_label="HWP angle (deg)",
                y_label="Power (uW)",
                color="cyan",
            )
        ]

traces() is read once, when a renderer attaches, and it is where the axis labels and the units live — a plot whose y-axis says Power (uW) says it because the monitor declared it, not because someone typed it into a window. color="cyan" is a PLESTY palette token rather than a hex value, so the same view is legible in whichever theme the window is running.

Then the per-row half:

    def update(self, frame: Frame) -> dict[str, TraceData]:
        """Append one row to the curve and hand back the whole of it."""
        self._angles_deg.append(float(frame["hwp_deg"]))
        self._powers_uw.append(float(frame["power_w"]) * 1e6)
        return {"malus": TraceData(x=list(self._angles_deg), y=list(self._powers_uw))}

update reads two lists that nothing has created yet, so the class needs an __init__ — it goes between input_schema and traces():

    def __init__(self, source: Optional[DataSource] = None, **params: Any) -> None:
        """Start with an empty curve; the view keeps its own history."""
        super().__init__(source, **params)
        self._angles_deg: list[float] = []
        self._powers_uw: list[float] = []

super().__init__ first: it is what validates the schema and reads traces(), and the lists have to exist by the time the first frame arrives. Miss this and the first row raises AttributeError: 'MalusMonitor' object has no attribute '_angles_deg'.

Two lists of your own are two lists the base class knows nothing about, which is the one obligation that comes with accumulating. Add the override the reset() row warned about, at the end of the class:

    def reset(self) -> None:
        """Drop the curve, then let the base class clear history and renderers."""
        self._angles_deg.clear()
        self._powers_uw.clear()
        super().reset()

Without it, clearing the view empties the frame history and the renderers while the curve survives in your two lists, and the next row redraws the whole old sweep on top of the new one.

The accumulation belongs here, not in the renderer. It is what makes the view replayable: the same rows in the same order give the same picture, whether they arrive one per second from a live run or all at once from a finished one. A renderer that kept the history instead would draw a different curve depending on when you opened it.

update is pure. It reads the frame and its own state, and returns what to draw. It does not touch a device, a file or a widget — which is why a monitor can be tested in a millisecond, and why the same one can be driven by a GUI timer or a for loop without knowing the difference.

Point it at a run

Nothing to start, nothing to configure: give the monitor the run directory from lesson 4 and ask it to take whatever is there.

First, the path. RUN_DIR is a real directory you have to fill in — the one lesson 4 printed as Run completed: demo_pol_pl_…. Set it and check it in the same terminal you will run the rest from:

RUN_DIR=~/tmp/frames/demo_pol_pl_20260821-191701   # <- yours, from lesson 4
wc -l "$RUN_DIR/records.jsonl"                     # 91 for a completed sweep

Do not skip that second line. A source pointed at a directory that does not exist raises nothing — a viewer is allowed to start before its experiment does, so it simply waits, and an unset or mistyped RUN_DIR gives you rows: 0 and then a bare KeyError: 'malus' from a monitor that never received a frame. wc says so immediately instead.

uv run python -c "
from plesty.lib.monitor import RunSource
from plesty.demo_pol_pl.monitors import MalusMonitor

monitor = MalusMonitor(RunSource('$RUN_DIR'))
print('rows:', monitor.pump())
"

FrameValidationError: MalusMonitor: unknown channel(s) ['data_file', 'exposure_s',
'hwp_reported_deg', 't_end', 't_start']; declared: ['hwp_deg', 'power_w'].

Say which columns are which

That is the contract working, and it is worth hitting once. measure_row returns seven keys — the two you draw, plus the angle the stage reported, the exposure, the two timestamps, and the path to the spectrum — and a source with no mapper hands each record over exactly as the run wrote it. A view declares the two keys it draws, and the library will not quietly hand it the rest. So something has to project one onto the other. That something is a mapper — a plain function, handed one unwrapped record, returning the channel mapping a monitor declared. It goes at the end of monitors.py, outside the class and unindented: it is handed to a source rather than called on a monitor, so nesting it in the class body would make record arrive as self and break the import the next command needs.

def malus_row(record: dict[str, Any]) -> Optional[dict[str, Any]]:
    """Project one sweep row onto the channels ``MalusMonitor`` declares."""
    if "hwp_deg" not in record or "power_w" not in record:
        return None
    return {"hwp_deg": record["hwp_deg"], "power_w": record["power_w"]}

This is the only place the view knows what this experiment calls its columns, which is what makes a monitor reusable across experiments that measure the same kind of thing with different names for it. Here it looks like ceremony, because MalusMonitor declares the names this experiment already uses and the function mostly copies them across. The point shows the moment the view is not yours: the shared series view in plesty-common-monitors declares nothing but x and y, and series_mapper("hwp_deg", "power_w", y_scale=1e6) is the whole of what it takes to aim it at this sweep — no subclass, no edit, and the watts become microwatts on the way through.

And the refusal is worth the function it costs. A library that quietly kept the keys it recognized would let a renamed column through as a plot that keeps drawing, half-empty or frozen at the last good row, and you would find out in the analysis. Failing on the first row is how a schema mismatch stays cheap. Returning None drops a record — a plan that starts with a bookkeeping step, or a row that recorded something else entirely, is skipped rather than crashed on. That is the missing piece from earlier: a dropped record never becomes a frame, which is why seq can run behind the record's index. This sweep drops nothing, so here the two happen to agree.

Run it again

Same command and the same RUN_DIR, with the mapper handed to the source:

uv run python -c "
from plesty.lib.monitor import RunSource
from plesty.demo_pol_pl.monitors import MalusMonitor, malus_row

monitor = MalusMonitor(RunSource('$RUN_DIR', mapper=malus_row))
print('rows:', monitor.pump())
curve = monitor.snapshot()['malus']
print(list(curve.x)[:4], [round(v, 3) for v in curve.y][:4])
"

rows: 91
[0.0, 2.0, 4.0, 6.0] [6.113, 4.602, 3.028, 2.135]

pump() is the whole loop: ask the source what is new, validate it, project it, hand it to the renderers. It returns how many rows it took, which is also how you know whether anything happened.

Watch it live

snapshot() returned the curve as numbers. To see it drawn, the monitor needs a renderer — and the library ships one that plots, so this is the part you do not write.

It is an optional extra, because a monitor itself never imports Qt:

uv add "plesty-lib[gui]"

A second new file, plesty/demo_pol_pl/viz.py. It is the whole viewer:

"""A window on a polarization sweep."""

import sys
from pathlib import Path

from plesty.lib.monitor import RunSource
from plesty.lib.ui import MonitorPanel, run

from .monitors import MalusMonitor, malus_row


def main() -> int:
    """Open a window on the run directory named on the command line.

    Returns:
        The Qt exit code, or 2 when the run directory is missing.
    """
    if len(sys.argv) != 2:
        print("usage: python -m plesty.demo_pol_pl.viz <run_dir>", file=sys.stderr)
        return 2

    run_dir = Path(sys.argv[1])
    if not (run_dir / "records.jsonl").exists():
        print(f"No records.jsonl in {run_dir}", file=sys.stderr)
        return 2

    monitor = MalusMonitor(RunSource(run_dir, mapper=malus_row))
    return run([MonitorPanel(monitor, area="top")], title="Malus curve")


if __name__ == "__main__":
    sys.exit(main())

The check on records.jsonl is there because of how RunSource behaves: a run directory that does not exist is not an error, it is a source that has nothing yet, and a window that opens empty and stays empty tells you nothing about which of the two you are looking at. Four lines turn that into a message.

uv run python -m plesty.demo_pol_pl.viz "$RUN_DIR"

Where the picture comes from

Two names in that command do the work, and neither of them draws anything itself.

MonitorPanel is a monitor plus a widget. When the window builds it, it does one line that matters (plesty/lib/ui/monitor_panel.py):

self.renderer = self.monitor.attach(PlotRenderer(self.theme, parent))

PlotRenderer is the code that plots. It lives in plesty/lib/ui/qt/plot.py and is built on pyqtgraph.

It is the first renderer you have met — the third row of the table at the top of this lesson, the piece that answers where does it go?. A renderer is a small class with two methods a monitor calls on it:

That is the entire interface, which is why the library can also ship one that writes video frames instead of pixels — you will meet it in lesson 6 — and why a test can attach one that only records what it was handed.

run() is the window around all this: it lays the panels out and starts a timer.

So the loop, once a second twice over:

1. The timer fires MonitorPanel.tick(), which is one line: self.monitor.pump().
2. pump() polls RunSource and submits whatever rows are new.
3. submit() validates each frame and calls your update(), which returns the curve.
4. submit() hands that return value to every attached renderer — here, PlotRenderer.draw().
5. draw() puts the numbers into pyqtgraph.

Step 5 is the one you were looking for, and it is short:

for key, payload in data.items():
    spec = monitor.trace_spec(key)          # what did the monitor declare this to be?
    if spec.kind == "image":
        self._draw_image(key, payload)      # → ImageItem.setImage(array)
    elif spec.kind in ("line", "scatter"):
        self._draw_curve(key, payload)      # → PlotDataItem.setData(x, y)

Your TraceSpec(key="malus", kind="line", …) is what that lookup finds, so TraceData(x=…, y=…) becomes setData(x, y) on a curve. Declare kind="image" instead and the same payload goes to setImage — which is how the spectral map in lesson 6 is drawn, with no new renderer and no window code.

The rest of the picture comes from the same declaration. setup() reads traces() once and builds one pyqtgraph item per spec, then takes the axis labels off the first: Power (uW) is on the axis because you typed it in traces(). color="cyan" is looked up in the theme rather than painted as a hex value, which is why the plot matches the rest of the shell.

That is the whole reason the monitor never mentions Qt. It says what to draw; PlotRenderer knows how, for every monitor on the platform.

What you are looking at

The emission is polarized, and a half-wave plate in front of a fixed analyser transmits a cos² in twice the plate angle — so one period every 90°, two across the 0–180° sweep. The null lands at 16° and the peak at 60° rather than at 0° and 45°, because the plate's zero is not the analyser's axis. That offset is a property of the bench, and seeing it in the first thirty seconds is the point of watching at all.

Two things to try:

Open the window on a sweep that is still running. In a second terminal, start the experiment again — the same command as lesson 4:

uv run python -m plesty.demo_pol_pl

It announces its new run directory on the way past, in the line that begins Run demo_pol_pl_…:

… | demo_pol_pl | Run demo_pol_pl_20260824-104233: 91 step(s), run dir …/frames/demo_pol_pl_20260824-104233, raw frames …
… | demo_pol_pl | Step 1/91 hwp[000.00] (measure_row)

Let it get a third of the way through, then open a window on that directory from your first terminal:

uv run python -m plesty.demo_pol_pl.viz ~/tmp/frames/demo_pol_pl_20260824-104233

The curve appears complete up to that moment on the very first poll, and then keeps extending as the sweep continues. Nothing that happened before you looked was missed: RunSource opened records.jsonl at the beginning and read everything already written.

Then open the same window on the finished run from earlier — your original $RUN_DIR. The whole curve is drawn at once and then nothing more arrives. Same file, same code: a finished measurement is a live one that stopped.

Next

One curve, from one channel of the record. The next lesson goes after the part of the measurement you have not seen yet — the spectra themselves, 1340 numbers a row, in files the record only points at. You will write the reader that opens them and the two views that draw them, and dock all three monitors into one window.