RAI Toolkit
VerifiedEvidence-backed AI review gates for LLM apps. Run compliance-aware evals, adversarial probes, policy checks, and produce JSON/HTML approval records linking findings to evidence.
Frameworks Covered
NIST AI RMFEU AI ActMIT AI Risk Repository
Services
compliance-evaluationred-teamingpolicy-checksreporting
About RAI Toolkit
RAI Toolkit is an open-source toolkit by Weights & Biases that bridges the gap between evaluation platforms and governance tools. It provides evidence-backed review gates for LLM applications, combining technical evaluation with documented approval decisions.
Features
- Compliance-aware evaluation mapped to NIST AI RMF, EU AI Act, and MIT AI Risk Repository
- 32 curated adversarial probe templates (jailbreaks, prompt injection, PII extraction, bias probes)
- YAML-based custom policy encoding with 13 starter policies
- Reviewer-pinned findings from manual chat probes
- JSON/HTML evidence-backed reports with content-hashed reproducibility
- Industry presets: healthcare, financial services, government, general
Framework Coverage
- MIT AI Risk Repository (24 categories, 7 domains)
- NIST AI RMF 1.0 (Govern / Map / Measure / Manage)
- EU AI Act (Articles 9-15, high-risk requirements)
Installation
pip install rai-toolkit
Loading reviews...