From 09ea653103d5d1c1497f95cfc10e7b27b3851ce5 Mon Sep 17 00:00:00 2001 From: duaneAI - AutoCode Model <6+contributor-8@noreply.dtechtechresearch.com> Date: Thu, 17 Sep 2026 15:42:23 +0000 Subject: [PATCH] Publish Evaluation-Harness research wiki page --- Evaluation-Harness.-.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 Evaluation-Harness.-.md diff --git a/Evaluation-Harness.-.md b/Evaluation-Harness.-.md new file mode 100644 index 0000000..fa65ab3 --- /dev/null +++ b/Evaluation-Harness.-.md @@ -0,0 +1,21 @@ +# Evaluation Harness + +## Reproduction + +```bash +PYTHONPATH=src python -m aaisa evaluate research/corpus-v1/manifest.json --output evaluation.json +``` + +The evaluator scans each artifact independently and compares observed findings with exact expected counts. It reports true positives, false positives, false negatives, precision, and recall when defined. + +## Reproducibility controls + +- Corpus paths cannot escape the manifest directory. +- Symlinked fixtures are rejected. +- Duplicate case identifiers and unknown rule identifiers are rejected. +- The corpus identity covers manifest bytes, fixture paths, and fixture SHA-256 digests. +- Timestamps are excluded from evaluation output. + +## Interpretation + +The foundation corpus is a 12-case deterministic conformance suite. It is not a representative sample and does not establish field precision, recall, exploitability, or production readiness.