feat: Add XSTest safety benchmark resource server

[dcfarris]

Summary

• Adds XSTest safety benchmark (Rottger et al., NAACL 2024) as a new resource server
• 450 hand-crafted prompts (250 safe, 200 unsafe) that test whether models correctly comply with safe requests and refuse unsafe ones
• Two verification modes: string-match classification (from the paper) and LLM-as-judge with the paper's 3-class taxonomy (full compliance / full refusal / partial refusal)
• Judge is optional (use_judge config flag), defaults to string-match, falls back to string-match on judge errors
• Includes per-category aggregate reporting script and documentation

What's in this PR

• resources_servers/xstest/app.py - resource server with string-match + LLM judge
• resources_servers/xstest/configs/xstest.yaml - config with optional judge model server
• resources_servers/xstest/prompt_templates/xstest_judge.txt - judge prompt from the paper
• resources_servers/xstest/data/example.jsonl - 5 example prompts for smoke testing
• resources_servers/xstest/tests/test_app.py - 24 tests across string-match, verdict parsing, and judge integration
• resources_servers/xstest/scripts/aggregate_results.py - results aggregation (overall score, unsafe rate, over-refusal rate, per-category breakdown, judge error tracking)
• resources_servers/xstest/README.md - input schema, reward scoring tables, usage examples

Validation

• Smoke tested end-to-end with ng_collect_rollouts on all 450 prompts
• Tested both string-match and LLM-as-judge verification modes
• 0% judge empty rate, <1% judge error rate in end-to-end runs

Test plan

• pytest resources_servers/xstest/tests/test_app.py - 24 tests passing
• ruff check and ruff format clean
• End-to-end rollout collection on example data and full dataset
• Full 450-prompt run with LLM judge, aggregate reporting verified
• ng_test +entrypoint=resources_servers/xstest
• ng_prepare_data validation
• Dataset upload to registry

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.