Add reproducible evaluation and provenance research
This commit is contained in:
@@ -9,6 +9,7 @@ python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
python -m pip install -e .
|
||||
python -m unittest discover -s tests -v
|
||||
python scripts/verify.py
|
||||
```
|
||||
|
||||
No external dependency is required for the foundation. Discuss new runtime dependencies before adding them.
|
||||
@@ -38,6 +39,16 @@ Every new rule must document:
|
||||
|
||||
Rules must not execute scanned content, access the network, modify source files, or place full credential values in reports.
|
||||
|
||||
Detailed metadata, test, evidence, and reference requirements are defined in [`docs/RULE_AUTHORING.md`](docs/RULE_AUTHORING.md).
|
||||
|
||||
## Evaluation corpus changes
|
||||
|
||||
Corpus fixtures must be synthetic, nonfunctional, and safe to store publicly. Do not add live secrets, personal information, executable malware, or content that requires detonation. Each case must declare exact expected rule counts. Run the evaluator twice and confirm equivalent parsed results before proposing a corpus revision.
|
||||
|
||||
```bash
|
||||
PYTHONPATH=src python -m aaisa evaluate research/corpus-v1/manifest.json
|
||||
```
|
||||
|
||||
## Commit guidance
|
||||
|
||||
Use a concise imperative subject, such as `Add SPDX provenance rule`. Keep unrelated changes separate. Reference the relevant issue in the pull request.
|
||||
|
||||
Reference in New Issue
Block a user