1.0 KiB
ADR 0001 Deterministic Evaluation Before Probabilistic Analysis
Status
Accepted for the 0.2.0 research baseline.
Context
AAISA needs measurable rule quality before it introduces probabilistic or model-backed analyzers. Evaluations that depend on network services, mutable model versions, or hidden prompts would make early results difficult to reproduce and could expose sensitive artifacts.
Decision
The first evaluation contract uses local synthetic fixtures, explicit expected rule counts, content-derived corpus identity, deterministic rule execution, and machine-readable TP, FP, and FN accounting. Probabilistic analyzers remain outside the default runtime until isolation, provenance, confidence, and reproducibility requirements are approved.
Consequences
Contributors can reproduce baseline behavior without external services. Regression detection is straightforward and safe for public fixtures. The baseline cannot measure semantic attacks that require model behavior, and its metrics must not be generalized to production accuracy.