Reliability

spicefault.reliability.analysis

Reliability analysis of the fault responses of a campaign.

Every figure here is conditional on the features used, on the measurement model already applied to them, on the normal variation of the campaign and on its operating condition (docs/RELIABILITY_METRICS.md). Metric identifiers (M1, M2, ...) refer to that document.

FaultResponse dataclass

The simulated response of one fault condition: the unit of the analysis.

features holds the successful simulations, one row each; n_total counts the failed ones too. record is the record of the fault, when the analysis has it.

Detectability dataclass

M1. table has one row per fault; the false-alarm rate was measured on n_evaluation healthy samples, the thresholds were set on n_calibration.

Ambiguity dataclass

M9b, at one threshold of d'.

ReliabilityAnalysis

Metrics computed from the samples of one campaign, at one operating condition.

samples has one row per simulation, with the fault identifier in column fault (healthy for the fault-free ones), the measurements in the features columns and, optionally, a boolean column ok (False for failed simulations) and a boolean column compliant (the circuit meets its specifications).

  • noise_floor: standard deviation of the measurement noise of each feature; a lower bound for spreads.
  • healthy_split: fraction of the healthy samples used to set thresholds; the rest measure the false-alarm rate. None uses all of them for both, which biases the false-alarm rate downwards.
  • faults: fault records (Fault.metadata()), for the metrics that need the component or the magnitude of each fault.

from_dataset(path, features=None, **kwargs) classmethod

Analysis of a dataset written by FaultCampaign. The features default to its measurements, and the fault records are taken from its manifest.

by(column)

One analysis per value of column, e.g. per operating condition (M11).

Each has its own healthy reference, drawn from the samples of that value.

counts()

Samples requested, simulated successfully and failed, per condition.

dictionary()

(centre, spread) of every feature per condition: median and IQR / 1.349.

flags(alpha=0.01, detector=None)

Decision of the detector for every successful sample, in the order of samples.

detectability(alpha=0.01, detector=None)

P(detected | fault), with the detector's thresholds set on healthy samples.

Per fault: p_detect over the successful simulations, with its interval, and bound_low, bound_high: the same proportion over all the simulations requested, counting every failed one as undetected and as detected. Failed simulations are not missing at random, so the bounds are what can be claimed when there are any.

standardised_shift(per_feature=False)

d of every fault against healthy: |median shift| / pooled robust spread.

Returns d' (the largest d over the features) and the feature that gives it, or the whole fault x feature table with per_feature. Univariate and conservative: a map of where the information is, not a detection probability.

auc()

Area under the ROC curve of the best single feature, per fault.

auc is max over features of max(AUC, 1 - AUC). Choosing the best of n_features is optimistic, more so with many features and few samples.

failure_probability()

P(the circuit does not meet its specifications | condition), healthy included.

For the healthy condition, 1 minus this probability is the yield. This is a conditional probability, not a failure rate in time: a campaign says nothing about how often a fault occurs.

diagnostic_coverage(alpha=0.01, weights=None, detector=None, n_boot=1000)

Fraction of the failures that are detected, over the fault conditions.

DC = sum_k w_k P(detected and failed | f_k) / sum_k w_k P(failed | f_k), and the escape rate is 1 - DC. With equal weights (the default) this is a statement about the fault list, not about field behaviour; weights proportional to the occurrence of each fault, from a cited source, make it one.

The false-reject rate is P(detected | compliant), over the healthy samples not used for the thresholds and over the fault samples that still comply.

severity_response(alpha=0.01, detector=None)

Detection probability of every graded fault against its magnitude.

One row per fault that has a magnitude, with the components and type that define its family. Needs the fault records.

minimum_detectable(alpha=0.01, beta=0.1, by='severity')

Smallest simulated severity (or magnitude) detected with probability >= 1 - beta.

Per family (fault type and components) and direction (sign of the magnitude). monotone is False when a larger simulated value falls below 1 - beta, in which case the smallest value does not describe the family. The result cannot be finer than the grid of simulated values, which is given.

separation()

Pairwise d' between all conditions, healthy included (M9a).

ambiguity(threshold=3.0, component_of=None)

Which conditions, and which components, cannot be told apart (M9b).

component_of maps each fault to the component to locate; by default the components of its record, joined by "+".

detectability_across(analyses, alpha=0.01, beta=0.1, detector=None)

Detection probability of every fault in several analyses, side by side.

The analyses are campaigns that differ in one thing: the tolerance scale (M8, robustness) or the operating condition (M11). Each is judged against its own healthy reference. Columns: one per analysis, then - worst_case: the smallest detection probability; - range: largest minus smallest; - lost: the analyses where the fault falls below 1 - beta although it is detected in the first one.

robustness(analyses, alpha=0.01, beta=0.1, detector=None)

Detectability against the tolerance scale (M8).

analyses maps each tolerance scale (1 is the declared design) to the analysis of the campaign simulated at that scale. Columns: the detection probability at each scale, in increasing order, then - loss: detection probability at the smallest scale minus at the largest; - critical_tolerance: the largest simulated scale at which the fault is still detected with probability >= 1 - beta, NaN if at none.

spicefault.reliability.detection

Decision rules: what turns a vector of measurements into "not healthy".

Detector

Bases: Protocol

fit(healthy)

Set the thresholds from healthy samples, [n, features].

flag(x)

True for the rows of x judged not healthy.

LimitTest dataclass

Limit test: a sample is flagged when any feature is outside its limits.

Each feature gets the central interval of the healthy samples at tail probability alpha / (2 d) per side, so that the d features together keep a false-alarm rate of about alpha (Bonferroni; exact if the features were independent and the limits known). The limits are estimated, and with small alpha / (2 d) they sit close to the extreme healthy samples: the realised false-alarm rate must be measured on healthy samples that were not used here.

spicefault.reliability.sensitivity

Local sensitivity of the measurements to the components, and what it bounds.

local_sensitivity(circuit, measurements, config, parameters=None, simulator=None, step=0.01)

Change of each measurement for a +1 % change of each parameter, at the nominal circuit.

Central differences with relative step step: S = (m(x (1 + h)) - m(x (1 - h))) / (2 h) * 0.01. Rows are measurements; columns are component for a value and component.parameter otherwise. parameters defaults to the values of the resistors, capacitors and inductors.

It is valid near the nominal point only, and it is noise-free.

normalised_sensitivity(sensitivity, spread)

Sensitivity in units of the healthy spread of each measurement (z per 1 % change).

A value of 1 means that a 1 % deviation of the component moves the measurement by one standard deviation of the healthy population. Measurements without spread are dropped.

testability_rank(z, deviation_pct=10.0, threshold=3.0)

Number of independent directions along which a small deviation is visible.

Counts the singular values s of the normalised sensitivity matrix with s * deviation_pct >= threshold: directions along which a deviation of that size moves the measurements by at least threshold healthy standard deviations. It bounds how many components can be told apart by deviations of that size.

collinear_groups(z, cos_threshold=0.99, deviation_pct=10.0, threshold=3.0)

(groups of components with parallel sensitivity vectors, insensitive components).

A component is insensitive when a deviation of deviation_pct moves the measurements by less than threshold healthy standard deviations (Euclidean norm). The rest are linked when the absolute cosine of their sensitivity vectors reaches cos_threshold, since then a small deviation of one can be mimicked by the other.

spicefault.reliability.structure

Separation between fault conditions and the ambiguity structure it implies.

connected_groups(names, pairs)

Connected components of the graph on names with edges pairs, largest first.

pairwise_shift(centre, spread, floor=None)

Pairwise d' between conditions: max over features of |c_i - c_j| / pooled spread.

centre and spread have one row per condition and one column per feature. floor is a per-feature lower bound of the spread (the measurement noise), which keeps numerically identical clouds from looking infinitely separable.

ambiguity_groups(d, threshold=3.0)

Connected components of the graph linking conditions with d' < threshold.

Linking is transitive, so a chain of similar conditions merges into one group: the groups are an upper bound of the ambiguity.

confusable_components(d, component_of, threshold=3.0)

For each component, the other components with a condition closer than threshold.

Not transitive: it answers "a case of this component could be mistaken for which others?". A component with an empty list can be located without ambiguity.

component_groups(d, component_of, threshold=3.0)

Transitive closure of confusable_components: groups of mutually linked components.

spicefault.reliability.statistics

Estimators used by the reliability metrics, with their uncertainty.

wilson_interval(k, n, confidence=0.95)

Wilson score interval of a proportion k / n. (nan, nan) if n is 0.

clopper_pearson_interval(k, n, confidence=0.95)

Exact (conservative) interval of a proportion k / n, by inverting the binomial tails.

robust_spread(x, axis=0)

IQR / 1.349: equal to the standard deviation for a normal cloud, and not inflated by the few extreme values of a fault that saturates an output.

auc(fault, healthy)

P(fault > healthy) + P(fault = healthy) / 2, the Mann-Whitney statistic.

bootstrap_interval(statistic, n_boot=1000, confidence=0.95, seed=0)

Percentile interval of statistic(rng), which resamples with the generator it gets.