Skip to content

fix: improve swamp-model description with explicit extension-model boundary#811

Merged
stack72 merged 1 commit intomainfrom
test-model-skill
Mar 21, 2026
Merged

fix: improve swamp-model description with explicit extension-model boundary#811
stack72 merged 1 commit intomainfrom
test-model-skill

Conversation

@stack72
Copy link
Contributor

@stack72 stack72 commented Mar 21, 2026

Summary

  • Improve swamp-model skill description to better trigger on method-related
    queries (validation, transform, enrichment) while staying scoped to swamp
    domain terms. Added concrete definition of what swamp models are: "structured
    automation units that define typed schemas, methods, and outputs for data
    processing".
  • Add explicit boundary with swamp-extension-model: "Do NOT use when the user
    wants to build, create, or implement a custom model type, Zod schema, or
    TypeScript model — that is swamp-extension-model". This prevents false
    triggers on queries like "Build a Zod schema for my custom validation model".
  • Fix "Delete this model from the repository" eval expectation from false to
    true — model delete is explicitly in the skill's trigger keywords.
  • Trigger eval: 75% → 81% (passing). Remaining miss ("Run the validation
    method on this JSON data") is a zero-context edge case that passes
    intermittently with EVAL_RUNS=3.
  • tessl review: 91% average (description 82%, content 100%) — passes.

Test plan

  • EVAL_RUNS=1 deno run eval-skill-triggers --skill swamp-model — 81%, passes
  • npx tessl skill review .claude/skills/swamp-model — 91% average
  • No false triggers on swamp-extension-model boundary queries
  • Negatives all pass except probabilistic variance on EVAL_RUNS=1

🤖 Generated with Claude Code

…undary

## Summary

- Improve swamp-model skill description to better trigger on method-related
  queries (validation, transform, enrichment) while staying scoped to swamp
  domain terms. Added concrete definition of what swamp models are: "structured
  automation units that define typed schemas, methods, and outputs for data
  processing".
- Add explicit boundary with swamp-extension-model: "Do NOT use when the user
  wants to build, create, or implement a custom model type, Zod schema, or
  TypeScript model — that is swamp-extension-model". This prevents false
  triggers on queries like "Build a Zod schema for my custom validation model".
- Fix "Delete this model from the repository" eval expectation from false to
  true — `model delete` is explicitly in the skill's trigger keywords.
- Trigger eval: 75% → 81% (passing). Remaining miss ("Run the validation
  method on this JSON data") is a zero-context edge case that passes
  intermittently with EVAL_RUNS=3.
- tessl review: 91% average (description 82%, content 100%) — passes.

## Test plan

- [x] `EVAL_RUNS=1 deno run eval-skill-triggers --skill swamp-model` — 81%, passes
- [x] `npx tessl skill review .claude/skills/swamp-model` — 91% average
- [x] No false triggers on swamp-extension-model boundary queries
- [x] Negatives all pass except probabilistic variance on EVAL_RUNS=1

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial Review

Critical / High

None.

Medium

None.

Low

  1. .claude/skills/swamp-model/evals/trigger_evals.json: The old note for the "Delete this model" entry explained nuance ("borderline; context determines if this is about removing from system vs managing the definition"). The new note ("model delete is a listed trigger keyword") is simpler but loses that nuance. If the eval framework ever uses notes for disambiguation, the reduced context could matter. Practically irrelevant since notes are informational only.

Verdict

PASS — Metadata-only changes to a skill description and eval expectations. The eval fix correctly aligns "Delete this model from the repository" with the skill's explicit "model delete" trigger keyword. No executable code is modified; nothing to break.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: LGTM

Clean, well-scoped skill definition update. No blocking issues.

What changed:

  • Improved swamp-model skill description with concrete definition ("structured automation units") and expanded trigger keywords (validation method, transform method, enrichment model, etc.)
  • Added explicit boundary with swamp-extension-model to prevent false triggers
  • Fixed "Delete this model from the repository" eval: should_trigger corrected from false to truemodel delete is clearly listed in the skill's trigger keywords and Quick Reference table

Verified:

  • JSON eval file is valid
  • YAML frontmatter is well-formed
  • Eval expectation change is logically correct — delete is a lifecycle operation covered by this skill
  • No TypeScript code changes, so no type/lint/test/import concerns apply

No suggestions — this is a straightforward improvement.

@stack72 stack72 merged commit a9f0b9a into main Mar 21, 2026
9 checks passed
@stack72 stack72 deleted the test-model-skill branch March 21, 2026 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant