Add reproducible evaluation and provenance research

2026-09-17 10:40:29 -05:00
parent d9fc3ca18b
commit 68f02d035f
53 changed files with 1432 additions and 14 deletions
@@ -0,0 +1,21 @@
+# Evaluation Harness
+
+## Reproduction
+
+```bash
+PYTHONPATH=src python -m aaisa evaluate research/corpus-v1/manifest.json --output evaluation.json
+```
+
+The evaluator scans each artifact independently and compares observed findings with exact expected counts. It reports true positives, false positives, false negatives, precision, and recall when defined.
+
+## Reproducibility controls
+
+- Corpus paths cannot escape the manifest directory.
+- Symlinked fixtures are rejected.
+- Duplicate case identifiers and unknown rule identifiers are rejected.
+- The corpus identity covers manifest bytes, fixture paths, and fixture SHA-256 digests.
+- Timestamps are excluded from evaluation output.
+
+## Interpretation
+
+The foundation corpus is a 12-case deterministic conformance suite. It is not a representative sample and does not establish field precision, recall, exploitability, or production readiness.