Dataset

spicefault.dataset.dataset

The dataset of a campaign as an object: traceable, verifiable and reproducible.

Provenance dataclass

Everything that identifies one sample and what produced it.

Dataset

A dataset folder written by FaultCampaign.

samples has one row per simulation, failed ones included; waveforms, if any, is aligned with it row by row, with NaN where a simulation failed or its operating condition stores no waveform. The file holds no row for those conditions: waveforms is then a StoredWaveforms, which reads like the full array and gives the array of the file as waveforms.stored. metadata is the definition of the experiment and manifest the record of the run.

ok property

True for the samples whose simulation gave a usable result.

specifications property

The specifications the dataset was last labelled with.

verify()

Problems with the files or their consistency; an empty list if there are none.

The files must match the fingerprints of the manifest, the table must have one row per planned sample, and failed samples must carry no result.

update_columns(frame, note='')

Add or replace derived label columns without invalidating dataset integrity.

label(specifications, note='')

Label every sample as meeting its specifications or not, and keep the limits.

It writes ok_<name> for each specification, compliant (all of them are met) and violated (the names of those that are not, separated by commas), stores the limits in the manifest and appends to its history. Calling it again with other limits relabels the dataset; nothing is simulated.

A specification is a property of the drawn circuit, so the rows of one circuit under its operating conditions get the same labels: the value is read in the conditions that measure it. A value that could not be computed, also because the simulation failed, is a violation. Returns the labels written.

cases(waveform_from=None)

(cases, waveforms): one row per drawn circuit, across its operating conditions.

The table of samples has one row per simulation, so a circuit drawn once and simulated under several conditions is several rows, each with the measurements of its condition. Here they are joined: the definition of the circuit, its labels and its realised parameters once, then the measurements of every condition side by side. A measurement that several conditions make gets one column per condition, <name>_<condition>. sim_ok is true only if every condition succeeded, and cases.attrs["features"] lists the measurement columns. Labels are read from the row of the first condition.

waveforms are those of the condition waveform_from, aligned with the cases; it may be omitted when a single condition stores them. None if the dataset has no waveforms.

observe(instrument, seed=0)

The samples as an instrument reads their measurements: with its noise, resolution and range (see Instrument).

Nothing is simulated and the dataset is not modified: the stored values stay noise-free, and the same dataset serves any instrument. statistics, yield_report and analysis take the instrument directly.

extremes(features=None)

The lowest and the highest value of each measurement, and the samples they are in.

One row per fault, operating condition and measurement, over the successful simulations: minimum, minimum_sample, maximum, maximum_sample. These are the extremes of the circuits that were simulated. Over the corners of a tolerance band they are the worst case of the band only where the measurement is monotonic in each parameter (see corners); over a random sample they are what happened to be drawn. In neither case are they a proven worst case.

statistics(features=None, quantiles=(0.01, 0.5, 0.99), confidence=0.95, by_case=False, instrument=None, seed=0)

The distribution of each measurement: one row per fault, operating condition and measurement, with the mean, the spread, the quantiles and their intervals (see spicefault.statistics.describe), and the failed simulations counted. For a dataset sampled by Latin hypercube or Sobol sequence the estimates are given and the intervals are left empty: see attrs["intervals"].

With by_case the rows are per fault, over one row per drawn circuit (see cases), which a dataset whose conditions measure different things needs for the count of failed simulations to be that of the circuits.

With instrument, the measurements are those it reads, with the noise drawn from seed; the instrument is recorded in the attrs of the result.

yield_report(specifications=None, fault_id=HEALTHY_ID, confidence=0.95, interval='wilson', instrument=None, seed=0)

Yield of the fault-free circuits: the fraction that meets each specification and all of them, with intervals, bounds for the failed simulations and the margin to each limit (see spicefault.statistics.yield_report).

specifications default to those the dataset was labelled with; others can be given, and nothing is simulated or written. The yield is computed per drawn circuit, each quantity read in the operating condition that measures it. With fault_id it is the same report for the circuits with that fault. With instrument it is the yield that a test with that instrument would find, which differs from the true one by its false rejects and its escapes.

netlist(sample_id)

The netlist that was simulated for a sample, rebuilt from what is stored.

The realised values are read from the table, not drawn again, so this works whatever variations the experiment used.

experiment(**overrides)

The experiment that produced the dataset, rebuilt from its metadata.

Custom or joint variations, custom measurements and custom backends cannot be stored and must be passed again (variations=, measurements=, simulator=).

reproduce(sample_ids=None, n=20, experiment=None, seed=0)

Simulate samples again and compare them with what is stored.

By default n samples chosen at random. One row per sample: - definition: same fault, condition and seed key; - parameters: the redrawn values are exactly the stored ones; - status: same simulation status; - max_abs_diff, max_rel_diff: largest difference over the measurements; - waveform_abs_diff: largest difference over the waveform, if stored. On the platform and simulator version that wrote the dataset the differences are expected to be zero; elsewhere they measure the dependence on both.

to_ml(target='fault_id', features=None, waveforms=False, drop_failed=True, by_case=False, waveform_from=None)

(X, y) for a statistical or machine-learning model.

X is the table of measurements (or of features), or the waveforms with waveforms, for the samples of the conditions that store one. y is a column of the samples, by default the fault identifier (fault_type and fault_location are the usual alternatives), or the result of a function of the samples. The library does not depend on any ML framework.

With by_case there is one row per drawn circuit instead of one per simulation, with the measurements of all its operating conditions and the waveform of the condition waveform_from (see cases).

analysis(features=None, by_case=False, instrument=None, seed=0, **kwargs)

A ReliabilityAnalysis of the dataset.

With by_case it is made on one row per drawn circuit (see cases), which a dataset whose operating conditions measure different things needs. With instrument, detection works on the measurements as it reads them, and the uncertainty of its readings is the noise_floor of the analysis unless one is given; the instrument is kept as analysis.instrument.

spicefault.dataset.specification

Specifications: the limits a circuit must meet, kept apart from what was simulated.

Specification dataclass

A limit on one column of a dataset, usually a measurement: minimum <= value <= maximum, with either bound left open.

A value that is not finite does not meet the specification: a quantity that could not be computed is not known to be within its limits.

column property

The label column that says whether each sample meets it.

met(values)

True where a value is finite and within the limits.

spicefault.dataset.manifest

The manifest of a dataset: the record of the run that produced it.

Manifest dataclass

How and with what a dataset was produced, and the fingerprint of its files.

summary holds the counts the application adds (completed, failed, status counts); in the file they are written next to the other fields. user is what the application passed as metadata, and source the git commit of the project that ran the campaign (see git_source), absent if there was none. The manifest is written last: its presence marks a complete dataset.

verify(folder)

Files that are missing or no longer match their recorded fingerprint.

file_record(path)

Size and SHA-256 of a file, read in blocks.

git_source(folder=None)

The commit of the git repository that holds folder, by default the working directory, and whether its tracked files had uncommitted changes.

This is the version of the project that ran the campaign, not of the library. An empty record outside a repository, or where git is not available.

spicefault.dataset.store

Reading and writing of a dataset directory.

A dataset is a directory with: - samples.parquet one row per simulation; - waveforms.npy float32 [n_stored, n_points] (optional): one row per sample whose operating condition stores a waveform, in sample order. When every condition stores one, that is one row per sample; - metadata.json the definition of what was simulated: enough to regenerate it; - manifest.json the record of the run: versions, counts, fingerprints of the files; - any other file the application adds, such as the source netlist.

StoredWaveforms

Waveforms aligned with the samples, read from a file that holds only some rows.

A campaign in which some operating conditions store no waveform writes only the rows of the others. This reads them as the full array would be read: indexed by sample, with NaN where nothing is stored, without holding those rows in memory. stored is the array of the file and rows the sample of each of its rows. Only the samples can be indexed in place; np.asarray gives the full array.

assemble(stems, out_dir, waveform_rows=None)

Join the chunks written by a campaign into out_dir; returns (samples, waveforms).

With waveform_rows, one flag per sample, only the waveforms of the flagged samples are written, in sample order.

read_waveforms(path, samples, manifest, mmap=False)

The waveforms of a dataset aligned with samples, or None if it stores none.

manifest is the record of the dataset: its waveform_conditions, if any, are the operating conditions whose samples have a row in the file.

load_metadata(path)

The definition of the campaign that wrote the dataset.

load_dataset(path, drop_failed=True, ok_column='sim_ok')

Return (samples, waveforms, manifest). Failed simulations are dropped on request.