Laboratories & science · Case study

Importing scientific data is a provenance problem, not an upload button.

Raw artifacts moved through archive, fingerprinting, parsing, validation and correlation without losing provenance.

Case study context

The project and the problem

The laboratory used scientific instruments from different vendors, each exporting measurements through its own text, spreadsheet, XML, image or proprietary file format. A file extension did not establish scientific meaning, and parsing alone could not prove which instrument, sample and method produced a value. A silent import failure could also be mistaken for a genuine absence of observations.

How the process worked

Each source was archived immutably and checksummed before format fingerprinting selected a versioned parser. Parsed records were normalized and validated, while sample and equipment correlation remained a separate controlled decision; ambiguous or failed inputs entered quarantine for review and replay against the preserved source.

Governed ingestion

Turn instrument files into trusted measurements without losing the evidence.

Use the controls below to inspect the key stages, constraints and validation decisions in that process.

Governed instrument ingestion workbench

Instrument artifact

ELL-02_20260830_143214.se

Instrument ELL-02 · Ellipsometer

Received
Archived original
1.8 MB · ART-2026-00814
SHA-256
8f4b…c219 · verified
REVIEW REQUIRED
Canonical record

Extracted measurement preview

Canonical measurement fields and their exact locations in the archived source artifact
FieldValueTypeExact source
sample_idS-24-01842-BtextHeader / SampleID
thickness47.9 nmdecimalData row 18 / Psi fit
fit_error0.013decimalModel / MSE
methodThickness v3referenceHeader / Method
Open raw source
[Acquisition]
Instrument=ELL-02
SampleID=S-24-01842-B
Method=Thickness v3

[Data row 18]
Thickness_A=479
FitError=0.013
Parser diagnostics
Format
Signature matched
Units
Å → nm
Required fields
All 12 present
Warnings
0
Correlation review

Link to experiment context

The artifact names sample S-24-01842-B, but two open runs reference that sample.

Candidate runs

Parsing and validation passed. Confirm the correct experiment run before publication.

Append-only history

Provenance ledger

A replay appends a new attempt. It never overwrites this record.

AttemptTimestampParserStageOutcomeRecord
Attempt 01v2.4.1CorrelationAwaiting linkImmutable

Operating constraints

  • Immutable raw evidence
  • Format ambiguity
  • Parser versions
  • Correlation uncertainty

What changed

  • Preserved raw evidence
  • Visible parsing diagnostics
  • Controlled replay
  • Measurement lineage to source locations

Evidence retained: Checksums, parser versions, validation diagnostics and source locations remained linked to every imported measurement.

Start with the problem

Have a similar operating problem?

Share the environment, constraint and result you need. We can discuss the engineering path without exposing confidential details.

Discuss a similar challenge