Datasets¶

A campaign writes a folder that stands on its own:

from spicefault import Dataset

dataset = Dataset("data/rc_lowpass")

dataset.verify()                 # [] if the files match the manifest and each other
dataset.provenance(1234)         # fault, seed key, realised values, versions of one sample
dataset.netlist(1234)            # the netlist that was simulated for it
dataset.reproduce(n=50)          # simulate samples again and compare with what is stored

X, y = dataset.to_ml(target="fault_location")       # for any ML or statistical tool
analysis = dataset.analysis()                       # a ReliabilityAnalysis

• netlist rebuilds a sample from the values stored in the table, so it needs nothing but the folder.
• reproduce rebuilds the experiment from metadata.json and reports, per sample, whether the definition, the drawn values and the status are the same, and the largest difference in the measurements and the waveform. On the machine and simulator version that wrote the dataset the differences are expected to be zero.
• Functions cannot be stored. If the experiment used custom or joint variations, or custom measurements, pass it again: dataset.reproduce(experiment=experiment).
• The library has no ML dependency: to_ml returns NumPy arrays.

split_by_replica(samples, test_fraction, seed) returns train/test row positions while keeping all conditions of each replica together and stratifying by fault_id. split_by_magnitude(samples, test_magnitudes) holds out every row at the selected fault magnitudes. Both return NumPy index arrays. Derived labels can be added with dataset.update_columns(frame, note=...); it updates the sample fingerprint and appends the operation to manifest history, while definitions, parameters and measurements remain protected.

Specifications¶

Whether a fault was injected and whether the circuit still meets its specifications are two different questions. Dataset.label answers the second from limits that stay with the dataset:

from spicefault import Specification

limits = [
    Specification("spec_cmrr_db", minimum=89.0),
    Specification("spec_noise_uvpp", maximum=30.0),
]
dataset.label(limits, note="IEC 60601-2-25 limits")

dataset.samples[["ok_spec_cmrr_db", "compliant", "violated"]]
dataset.specifications           # the limits, read back from the manifest

It writes ok_<name> for each specification, compliant and violated (the names of those that are not met), stores the limits in the manifest and appends to its history. Calling it again with other limits relabels the dataset without simulating anything. A value that could not be computed, also because the simulation failed, counts as a violation. A specification belongs to the drawn circuit: with several operating conditions, the rows of one circuit get the same labels, read in the conditions that measure each quantity.

The instrument¶

The stored measurements are those of the simulator: noise-free. What an instrument would read is computed from them afterwards, so its noise, resolution and range are parameters of a study that need no new simulation.

from spicefault import Instrument, Reading

instrument = Instrument({"gain_db": Reading(noise=0.02, resolution=0.01, limits=(0, 40))})

observed = dataset.observe(instrument, seed=7)         # a table; the dataset is not modified
dataset.statistics(instrument=instrument)
dataset.yield_report(limits, instrument=instrument)     # the yield a test with it would find
dataset.analysis(instrument=instrument)                 # detection on what it reads

A Reading adds Gaussian noise, rounds to the resolution and clips to the limits, in that order; each is optional. The noise of a value depends only on the seed, the measurement and the sample, so the observed table is the same for any order or subset of the rows. The instrument is recorded with each result (attrs["instrument"], analysis.instrument), and in a reliability analysis the uncertainty of its readings is the noise floor. One dataset then gives, for instance, the detectability of a fault against the noise of the instrument.

One row per drawn circuit¶

When operating conditions have their own measurements, the table has one row per circuit and condition, and each row holds only the measurements of its condition. Dataset.cases joins them:

cases, waveforms = dataset.cases(waveform_from="service")
cases.attrs["features"]          # the measurement columns, of every condition

X, y = dataset.to_ml(by_case=True)
analysis = dataset.analysis(by_case=True, compliant="compliant")

There is one row per fault and replica: the definition, the labels and the realised parameters once, and the measurements of every condition side by side. A measurement that several conditions make gets one column per condition, <name>_<condition>. sim_ok is true only if every condition succeeded. waveform_from may be omitted when a single condition stores the waveform.

Columns of the samples table¶