Add reproducible evaluation and provenance research
This commit is contained in:
+11
-7
@@ -8,22 +8,26 @@
|
||||
- [x] JSON and SARIF reports
|
||||
- [x] Unit tests and contributor guidance
|
||||
- [x] Threat model and research brief
|
||||
- [ ] Publish schema files for JSON and rule metadata
|
||||
- [ ] Add deterministic corpus manifests
|
||||
- [x] Publish schema files for scan reports, corpus manifests, and evaluation results
|
||||
- [x] Add deterministic corpus manifests and content-derived corpus identity
|
||||
- [x] Publish a versioned rule catalog with standards references
|
||||
- [x] Add stable finding fingerprints across JSON and SARIF
|
||||
|
||||
## Evaluation harness
|
||||
|
||||
- [ ] Define positive, negative, and ambiguous fixture classes
|
||||
- [ ] Add per-rule precision and recall measurement
|
||||
- [x] Define initial positive and negative fixture classes
|
||||
- [x] Add per-rule precision and recall measurement
|
||||
- [ ] Add runtime and peak-memory benchmarks
|
||||
- [ ] Add report reproducibility checks
|
||||
- [x] Add report reproducibility checks
|
||||
- [ ] Establish release-blocking redaction tests
|
||||
|
||||
## Artifact expansion
|
||||
|
||||
- [ ] SPDX and CycloneDX SBOM parsing
|
||||
- [ ] Model card and dataset card schemas
|
||||
- [ ] Agent tool-definition and permission analysis
|
||||
- [x] Initial CycloneDX ML-BOM model-card and digest checks
|
||||
- [x] Initial agent tool-definition and permission analysis
|
||||
- [x] Initial SLSA provenance v1 binding checks
|
||||
- [ ] Model card and dataset card validation profiles
|
||||
- [ ] Infrastructure-as-code provenance checks
|
||||
- [ ] Signed artifact and attestation verification
|
||||
|
||||
|
||||
Reference in New Issue
Block a user