Add reproducible evaluation and provenance research

This commit is contained in:
2026-09-17 10:40:29 -05:00
parent d9fc3ca18b
commit 68f02d035f
53 changed files with 1432 additions and 14 deletions
+11
View File
@@ -9,6 +9,7 @@ python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .
python -m unittest discover -s tests -v
python scripts/verify.py
```
No external dependency is required for the foundation. Discuss new runtime dependencies before adding them.
@@ -38,6 +39,16 @@ Every new rule must document:
Rules must not execute scanned content, access the network, modify source files, or place full credential values in reports.
Detailed metadata, test, evidence, and reference requirements are defined in [`docs/RULE_AUTHORING.md`](docs/RULE_AUTHORING.md).
## Evaluation corpus changes
Corpus fixtures must be synthetic, nonfunctional, and safe to store publicly. Do not add live secrets, personal information, executable malware, or content that requires detonation. Each case must declare exact expected rule counts. Run the evaluator twice and confirm equivalent parsed results before proposing a corpus revision.
```bash
PYTHONPATH=src python -m aaisa evaluate research/corpus-v1/manifest.json
```
## Commit guidance
Use a concise imperative subject, such as `Add SPDX provenance rule`. Keep unrelated changes separate. Reference the relevant issue in the pull request.