[Feature Request]: Create a catalog component for prebuilt detection rules

[nastasha-solomon]

Prerequisites

• I have searched existing issues to ensure this feature hasn't already been requested
• I have tested using the latest version of docs-builder

What problem are you trying to solve?

Elastic maintains 1,600+ prebuilt detection rules in elastic/detection-rules, but users must leave the core prebuilt rules documentation to browse them. They either need to navigate to the Prebuilt detection rules reference docs (which is cumbersome to navigate and lacks proper context), use the detection-rules-explorer GitHub Pages app, or read the raw TOML files on GitHub. There should be a browsable catalog inside the official docs at elastic.co/docs.

Proposed Solution

Replace the external detection-rules-explorer with a native docs-content catalog page that:

1. Displays all prebuilt detection rules organized by MITRE ATT&CK tactic.
2. Shows key metadata per rule: name, description, technique IDs, rule type, severity, and GitHub source link.
3. Auto-refreshes daily via GitHub Actions when rules change upstream.
4. Uses native docs-content patterns — no custom JavaScript.
5. Enables the eventual retirement of the detection-rules-explorer repo.

Examples and Research

Example

elastic/docs-content#5379

PRD: Prebuilt Detection Rule Catalog in docs-content

Approach

Why this design

• docs-content is content-only — built by docs-builder with MyST Markdown. No custom React/JS components are possible.
• docs-content already uses Python for data sync — the Google Sheets workflow (.github/scripts/sheet2docs/sync_sheet.py) establishes the pattern: Python script + GitHub Actions + CSV files + csv-include directive.
• The detection-rules-explorer's data pipeline (parseRuleData.ts) proves the approach: download the tarball, parse TOML, extract metadata. We port this logic to Python following the established docs-content pattern.

What we can replicate from the explorer

Explorer feature	docs-content equivalent
Browsable rule list	CSV tables organized by tactic section
Tag-based filtering	Tactic sections with heading anchors (jump links)
Rule detail (name, desc, severity, type, MITRE mapping)	CSV columns per rule
Link to GitHub source	Markdown link in Source column
Search	docs site global search indexes rule names
Auto-refresh (daily cron)	GitHub Actions daily cron

What we intentionally defer

• Client-side search/filter/sort within tables (platform limitation)
• Individual rule detail pages (these exist in the detection-rules docs repo via cross-link)
• Integration package rules (DGA, problemchild, etc. — matching current explorer behavior)

Architecture

elastic/detection-rules (GitHub)
        │
        │  GitHub Actions (daily at 03:00 UTC)
        │  sync-detection-rules.yml
        ▼
.github/scripts/sync-detection-rules/sync_rules.py
        │  Downloads tarball → parses TOML → generates CSVs
        ▼
solutions/security/detect-and-alert/prebuilt-rule-catalog/
        ├── initial-access.csv        (170 rules)
        ├── execution.csv             (326 rules)
        ├── persistence.csv           (353 rules)
        ├── privilege-escalation.csv   (225 rules)
        ├── defense-evasion.csv        (464 rules)
        ├── credential-access.csv      (203 rules)
        ├── discovery.csv              (138 rules)
        ├── lateral-movement.csv       (105 rules)
        ├── collection.csv             (55 rules)
        ├── command-and-control.csv    (144 rules)
        ├── exfiltration.csv           (61 rules)
        ├── impact.csv                 (86 rules)
        ├── reconnaissance.csv         (13 rules)
        ├── resource-development.csv   (8 rules)
        ├── other.csv                  (69 rules)
        └── _all-rules.csv            (1,664 rules)
        │
        ▼
prebuilt-rule-catalog.md  →  csv-include per tactic section

Files created/modified

docs-content (primary)

File	Action	Purpose
.github/scripts/sync-detection-rules/sync_rules.py	New	Python script: fetch tarball, parse TOML, generate CSVs
.github/scripts/sync-detection-rules/requirements.txt	New	Dependencies: requests, toml
.github/workflows/sync-detection-rules.yml	New	Daily cron workflow, creates PR with updated CSVs
solutions/security/detect-and-alert/prebuilt-rule-catalog.md	New	Catalog page with csv-include per tactic
solutions/security/detect-and-alert/prebuilt-rule-catalog/*.csv	New	One CSV per tactic + combined
solutions/toc.yml	Modified	Added catalog page under prebuilt-rules
solutions/security/detect-and-alert/prebuilt-rules.md	Modified	Added link to catalog

detection-rules-explorer (transitional)

File	Action	Purpose
parseRuleData.ts	Modified	Added CSV generation for transitional use

CSV schema

Column	Source	Example
Name	rule.name	Windows Script Interpreter Executing Process via WMI
Description	rule.description (truncated to 160 chars)	Detects WScript/CScript spawning a child process via WMI...
Technique	rule.threat[].technique[].id (comma-separated)	T1047
Rule Type	Derived from rule.type + rule.language	EQL
Severity	rule.severity	High
Source	Constructed GitHub link	GitHub ↗

Retirement plan for detection-rules-explorer

Once the docs-content catalog is live and validated:

1. Update the explorer's README to redirect to the docs catalog
2. Update the explorer site to show a banner linking to docs
3. Archive the detection-rules-explorer repo

Alternative Solutions

I considered referencing https://elastic.github.io/detection-rules-explorer/, but that goes against goal of keeping users in the Elastic docs. The repo that hosts the app's functionality is also not maintained -- the person who built it is no longer at Elastic.

Additional Context

To create this proof of concept, I collab'ed with @Mikaayenson and we used cursor + claude.

How important is this feature to you?

Nice to have

[nastasha-solomon]

@bmorelli25 I just remembered a couple of Slack threads and GH issues where folks were asking about the 9.x prebuilt rules changelog, so it might be worth bumping the importance of this feature:

• [Security Solution] Uncomment prebuilt rule changelog link kibana#213709 (grabbed from this Slack thread)
• https://github.com/elastic/enhancements/issues/25573
• https://github.com/elastic/docs-content-internal/issues/307
• https://elastic.slack.com/archives/C011H2BCRRN/p1770737454209739

[nastasha-solomon]

Also just noticed that the POC doesn't include a column for the update version, which the 8.x docs have, so prob need to tweak the design a bit.

Will create a separate page for this (example) The audience for this content has slightly different goals from the ones visiting the catalogue page.

[bmorelli25]

@nastasha-solomon can you give me more of a sense of what external resources are needed to move forward with this? I agree that this is probably more important than just a nice to have. What kind of input would you need from docs eng or the Detection team in order for us to move forward with this?

[nastasha-solomon]

Umm...I might not actually need anything. I think Cursor already created the fetching, parsing, and creating/updating scripts (see elastic/docs-content#5379), so I should just be able to make this page happen myself.

Feel free to close this issue!