22 lines
894 B
Markdown
22 lines
894 B
Markdown
# Evaluation Harness
|
|
|
|
## Reproduction
|
|
|
|
```bash
|
|
PYTHONPATH=src python -m aaisa evaluate research/corpus-v1/manifest.json --output evaluation.json
|
|
```
|
|
|
|
The evaluator scans each artifact independently and compares observed findings with exact expected counts. It reports true positives, false positives, false negatives, precision, and recall when defined.
|
|
|
|
## Reproducibility controls
|
|
|
|
- Corpus paths cannot escape the manifest directory.
|
|
- Symlinked fixtures are rejected.
|
|
- Duplicate case identifiers and unknown rule identifiers are rejected.
|
|
- The corpus identity covers manifest bytes, fixture paths, and fixture SHA-256 digests.
|
|
- Timestamps are excluded from evaluation output.
|
|
|
|
## Interpretation
|
|
|
|
The foundation corpus is a 12-case deterministic conformance suite. It is not a representative sample and does not establish field precision, recall, exploitability, or production readiness.
|