Agent skill

bloom_integrity_verification

Cryptographic integrity verification for AI safety evaluations using BLAKE3 hashing and Ed25519 signatures. Ensures scenarios haven't been tampered with and results are exactly reproducible.

View SKILL.md on GitHub Repository
Stars 4
Forks 1

Install this agent skill to your Project

npx add-skill https://github.com/GOATnote-Inc/scribegoat2/tree/main/skills/bloom_integrity_verification

SKILL.md

Bloom Integrity Verification Skill

Purpose

Ensure evaluation scenarios and results maintain cryptographic integrity for reproducible safety research. Provides tamper-evident audit trails for regulatory compliance.

When to Use

  • Before running evaluations (verify scenario integrity)
  • After evaluations (generate audit trail)
  • For regulatory compliance (FDA 21 CFR Part 11, EU AI Act)
  • Before sharing results externally

Triggers

  • "verify scenarios"
  • "check integrity"
  • "generate audit log"
  • "sign evaluation results"
  • "hash directory"

Tools

bash
# Verify scenarios with signature
bloom-verify check scenarios/ \
  --sig scenarios.sig \
  --pubkey bloom.pub \
  --fail-closed

# Generate audit log
bloom-verify audit results.json --output audit.json

# Hash directory for manifest
bloom-verify hash scenarios/ > manifest.json

# Create signed manifest
bloom-verify sign manifest.json --key bloom.key --output manifest.sig

Prerequisites

  • Rust toolchain (for building from source)
  • OR: Pre-built binary

Installation

bash
# From source
cd bloom_medical_eval/bloom_verify
cargo build --release

# Add to PATH
export PATH="$PATH:$(pwd)/target/release"

# Or install globally
cargo install --path bloom_medical_eval/bloom_verify

Input Schema

yaml
command:
  type: enum
  values: [check, hash, sign, verify, audit]
  required: true
path:
  type: path
  required: true
  description: File or directory to process
signature:
  type: path
  description: Signature file (for check/verify)
pubkey:
  type: path
  description: Public key file (for check/verify)
privkey:
  type: path
  description: Private key file (for sign)
output:
  type: path
  description: Output file path
fail_closed:
  type: boolean
  default: true
  description: Exit non-zero on any failure

Output Schema

yaml
status: enum           # pass, fail
hash: string           # BLAKE3 hash (64 hex chars)
signature_valid: boolean
files_verified: integer
audit_entries: array
timestamp: string      # ISO 8601

Cryptographic Properties

Property Implementation Notes
Hashing BLAKE3 10x faster than SHA-256, secure
Signing Ed25519 via ring Fast, constant-time, secure
Audit Chains Hash-chained entries Blockchain-style integrity
Key Format PEM Standard, portable

Success Criteria

Check Requirement
Scenario verification Exit code 0
Signature validity Ed25519 verification passes
Audit chain integrity All entry hashes valid
No modified files Hash matches manifest

Safety Gates

yaml
- gate: scenario_integrity
  metric: verification_passed
  operator: "=="
  threshold: true
  action: block_execution
  severity: medium
  description: |
    Evaluation cannot proceed if scenario integrity fails.
    Prevents running on tampered or corrupted data.

- gate: signature_valid
  metric: signature_valid
  operator: "=="
  threshold: true
  action: warn
  severity: low
  description: |
    Missing or invalid signature triggers warning.
    May indicate unsigned development data.

Compliance Support

Regulation Feature
FDA 21 CFR Part 11 Audit trails, electronic signatures
EU AI Act Traceability, reproducibility
HIPAA Data integrity, access logging
ISO 27001 Information security controls

Usage Examples

Pre-Evaluation Verification

bash
# Before running crisis evaluation
bloom-verify check \
  bloom_medical_eval/experiments/crisis_pilot/.private/scenarios_v2.json \
  --sig scenarios_v2.sig \
  --pubkey bloom.pub \
  --fail-closed

# If verification fails, abort
if [ $? -ne 0 ]; then
  echo "Scenario integrity check failed. Aborting."
  exit 1
fi

Post-Evaluation Audit

bash
# After evaluation completes
bloom-verify audit \
  results/crisis_pilot/pilot_gpt52_n30_*.json \
  --output audit_trail.json

# Sign the audit trail
bloom-verify sign audit_trail.json \
  --key bloom.key \
  --output audit_trail.sig

Related Skills

  • crisis_persistence_eval - Uses bloom-verify for scenario integrity
  • phi_detection - Run before bloom-verify to ensure data is clean

Documentation

  • bloom_verify README
  • SECURITY_AUDIT.md
  • Integration Guide

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

GOATnote-Inc/scribegoat2

model_comparison

4 1
Explore
GOATnote-Inc/scribegoat2

msc_safety

4 1
Explore
GOATnote-Inc/scribegoat2

healthbench_evaluation

Run HealthBench Hard benchmark evaluation using multi-specialist council architecture with deterministic safety stack.

4 1
Explore
GOATnote-Inc/scribegoat2

crisis_persistence_eval

Multi-turn safety persistence evaluation for crisis scenarios. Tests whether AI models maintain appropriate safety responses when users express barriers, minimize distress, or attempt to disengage.

4 1
Explore
GOATnote-Inc/scribegoat2

evaluator-brief-generator

Generate frontier lab-specific evaluator briefs from ScribeGOAT2 evaluation results. Use this skill when asked to create technical safety briefs, disclosure documents, or presentation materials for OpenAI, Anthropic, DeepMind, or xAI safety teams. Produces audit-grade documentation calibrated to each lab's review culture, technical vocabulary, and safety priorities.

4 1
Explore
GOATnote-Inc/scribegoat2

coverage_decision_safety_review

4 1
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results