diff --git a/Evaluation-Harness.-.md b/Evaluation-Harness.-.md new file mode 100644 index 0000000..fa65ab3 --- /dev/null +++ b/Evaluation-Harness.-.md @@ -0,0 +1,21 @@ +# Evaluation Harness + +## Reproduction + +```bash +PYTHONPATH=src python -m aaisa evaluate research/corpus-v1/manifest.json --output evaluation.json +``` + +The evaluator scans each artifact independently and compares observed findings with exact expected counts. It reports true positives, false positives, false negatives, precision, and recall when defined. + +## Reproducibility controls + +- Corpus paths cannot escape the manifest directory. +- Symlinked fixtures are rejected. +- Duplicate case identifiers and unknown rule identifiers are rejected. +- The corpus identity covers manifest bytes, fixture paths, and fixture SHA-256 digests. +- Timestamps are excluded from evaluation output. + +## Interpretation + +The foundation corpus is a 12-case deterministic conformance suite. It is not a representative sample and does not establish field precision, recall, exploitability, or production readiness.