Hub
Docs

Plesty Documentation

Data Flow

Where a measurement's data goes — a plan of atomic steps, one folder per run as the only place the programs meet, and one shared disk every PC sees under its own path.

Each run has one folder, and that folder is the only place the programs meet. The experiment and its instruments write it; everything else reads it; the only thing written back in is a request to pause or stop.

Writers on the left — the experiment, its devices, its analyzers, PLESTY Bench — each writing its own files into the run folder; readers on the right — live views, the analysis step, the run's report, anyone later — only reading it.

Every file is plain text or a plain array, and the logs are append-only, so a run needs no database or running program to be read — a folder copied onto a laptop years later still reads.

A plan of atomic steps

Before anything moves, the experiment writes down its plan — every step, in order — and never changes it. Each step is atomic: it finishes and its record is committed, or it counts as not done.

The plan as a row of steps. Each finished step commits one line to records.jsonl. Between steps the run reads control.json, where pause and stop act; during a step only abort is checked, every half second.

That one rule gives the rest:

  • Stop anywhere, continue later. Resuming skips every committed step. The plan is content-hashed, so a configuration edited halfway is refused, not stitched in.
  • Readers see only whole results. A line in records.jsonl exists only once everything its step produced is on disk.
  • Control is a file too. PLESTY Bench's Pause, Stop and Abort write control.json; the run acts on it at the next step boundary — or at once, for Abort, and that step is repeated on resume.

journal.jsonl beside the records says what happened — started, paused, resumed, a note like "helium refilled" — while the records say what was measured, each line naming the step and the call that produced it.

Large files never cross the network twice

A spectrometer or camera produces megabytes per step. Sending that through the device server to the experiment and then to disk would move every file twice. Instead the device writes its file straight into the run's raw/ folder, and only the file's path travels back to be recorded.

For a device on another PC to do that, the run folder is on a shared data disk — and every PC sees that disk under its own path:

The spectrometer's PC writes a spectrum under G:lesty; only the path relative to the disk is recorded; the experiment's PC joins it to its own root, /mnt/plesty.

So no PC is told where another sees the disk. Each knows only its own root, PLESTY_DATA — the shared disk as that PC mounts it — and every recorded path is relative to it. A device writes only inside its bench's folder on the disk (DEVICE_DATA_ROOT); Benches, Fleets & the Share has the table of what each variable holds. PLESTY Bench checks before each run that all benches really see the same folder. A run on one PC whose devices save no files needs none of this.