Add reproducible evaluation and provenance research

README.md

@@ -17,6 +17,11 @@ This repository begins in **Public Contributing** status. It is a research found
 - Detection of common prompt-injection and tool-exfiltration instructions
 - Detection of unpinned Python dependencies and mutable container image tags
 - Validation of model and dataset provenance metadata
+- Agent-tool authorization checks for wildcard, shell, and unrestricted network capabilities
+- SLSA provenance v1 binding checks and CycloneDX 1.7 ML-BOM transparency checks
+- Versioned rule catalog with standards references and per-rule explanations
+- Reproducible corpus evaluation with per-rule precision and recall accounting
+- Stable finding fingerprints for JSON and SARIF result correlation
 - JSON and SARIF 2.1.0 reports
 - Deterministic local CLI with documented exit codes
 - Standard-library-only runtime and unit-test suite
@@ -30,7 +35,9 @@ python3 -m venv .venv
 source .venv/bin/activate
 python -m pip install -e .
 aaisa rules
+aaisa explain AAISA-AGENT-001
 aaisa scan ./examples --format json --output report.json
+aaisa evaluate research/corpus-v1/manifest.json --output evaluation.json
 ```
 
 Run directly from a checkout without installing:
@@ -51,6 +58,22 @@ aaisa init-config --output aaisa.toml
 
 The default configuration excludes version-control and build directories, limits each artifact to 2 MiB, and reports findings at low severity or higher. See [`aaisa.example.toml`](aaisa.example.toml).
 
+## Reproducible evaluation
+
+The `evaluate` command executes every case in a versioned corpus manifest, compares observed rule counts with explicit ground truth, and emits:
+
+- a digest covering the manifest and all fixture content;
+- per-case expected and actual results;
+- false-positive and false-negative counts;
+- precision and recall where the relevant denominator is nonzero;
+- a nonzero exit code when any case diverges.
+
+```bash
+PYTHONPATH=src python -m aaisa evaluate research/corpus-v1/manifest.json
+```
+
+The included corpus is a deterministic conformance suite, not a statistically representative security benchmark. See [`research/corpus-v1/README.md`](research/corpus-v1/README.md) and [`docs/RESEARCH_NOTE_001_EVALUATION_AND_PROVENANCE.md`](docs/RESEARCH_NOTE_001_EVALUATION_AND_PROVENANCE.md).
+
 ## Safety model
 
 AAISA is intentionally constrained:
@@ -72,6 +95,8 @@ src/aaisa/          Python package and CLI
 tests/              Standard-library unit tests
 examples/           Safe sample artifacts
 docs/               Architecture, threat model, and roadmap
+schemas/            Machine-readable report, corpus, and evaluation contracts
+research/           Versioned benchmark manifests and safe synthetic fixtures
 .gitea/             Contributor issue and pull-request templates
 RESEARCH_BRIEF.md   Research purpose, methods, and evaluation plan
 CONTRIBUTING.md     Development and review workflow
@@ -82,6 +107,8 @@ CONTRIBUTING.md     Development and review workflow
 ```bash
 python -m unittest discover -s tests -v
 PYTHONPATH=src python -m aaisa scan examples --format json
+PYTHONPATH=src python -m aaisa evaluate research/corpus-v1/manifest.json
+python scripts/verify.py
 ```
 
 All changes should include tests for new behavior. Detection rules must document intent, evidence, likely false positives, and remediation guidance. Rules that expose matched credentials, execute artifacts, or make network requests will not be accepted into the default rule set.