feat(llm): Serve markdown inline via content negotiation#16023
feat(llm): Serve markdown inline via content negotiation#16023
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
sitemap isnt effective enough - 9000+ links gonna take a slightly diff approach |
d57f0a8 to
951505e
Compare
b22dfed to
347bff8
Compare
Change LLM/AI client detection from redirect to rewrite, so markdown content is served at the same URL when clients send Accept: text/markdown. Previously clients were redirected to .md URLs. Generate hierarchical sitemaps for LLM discovery: - Root index.md includes Sentry overview and links to top-level sections - Each section index page lists its child pages - Platform pages list guides separately from other pages - Guide pages list their sub-pages The DOCS_ORIGIN is now environment-aware for develop.sentry.dev support. Co-Authored-By: Claude <noreply@anthropic.com>
74b099d to
58168ce
Compare
Replaces the regex-based script tag removal with proper AST-based removal using unist-util-remove. This fixes CodeQL security alerts for: - js/incomplete-multi-character-sanitization - js/bad-tag-filter The AST-based approach is more robust and handles all edge cases that the regex could miss. Co-Authored-By: Claude <noreply@anthropic.com>
Replace existsSync check followed by read/write with try/catch around readFile. This eliminates the time-of-check to time-of-use race condition that CodeQL flagged as js/file-system-race. Co-Authored-By: Claude <noreply@anthropic.com>
1. Fix cache key calculation to exclude scripts - restores original caching behavior while keeping AST-based script removal for the actual processing. 2. Fix Quick Links for developer docs - conditionally show different quick links based on NEXT_PUBLIC_DEVELOPER_DOCS environment variable. 3. Upload modified parent files to R2 - parent index files with appended child page listings are now uploaded to R2 CDN. Co-Authored-By: Claude <noreply@anthropic.com>
006b0c6 to
cda5c4d
Compare
Add guidesIdx > 0 check to prevent invalid parent path calculation when guides directory appears at path position 0. Co-Authored-By: Claude <noreply@anthropic.com>
| # Force markdown via query param | ||
| curl "https://docs.sentry.io/platforms/python/?format=md" |
There was a problem hiding this comment.
We should also add curl "https://docs.sentry.io/platforms/python.md, right?
There was a problem hiding this comment.
is this file dead at this point? im gonna go spend some tim eon this repo to review state of it
There was a problem hiding this comment.
I think i added the query param ages back, and then setup the automatic switch over to .md. It all still works now, but with the auto-negotiation, I dont think its going to be used much.
I think we use it on the copy page as markdown button though.
File should be updated with the new flows for accessing the MD pages.
There was a problem hiding this comment.
this file looks like it was an LLM artifact? i didnt look at it, but is this actually supposed to be in the repo? (its a markdown file in a scripts directory is why i ask)
) Moves LLM documentation from `scripts/LLM_MARKDOWN_SERVING.md` to the contributing guide at `docs/contributing/pages/llm-support.mdx`. The old file was an implementation-focused artifact from #16023 that lived in an odd location and was incomplete (missing `.md` extension access method, per feedback from @BYK and @codyde). The new page covers our broader approach to designing docs for LLMs: - Principles (ties to existing minimalist/accuracy approach) - How we support LLMs (content negotiation, copy button, navigation) - Writing considerations for contributors Refs #16023 Co-authored-by: Claude <noreply@anthropic.com>
Serve markdown content inline when clients request
Accept: text/markdown, instead of redirecting to.mdURLs. This enables proper HTTP content negotiation for LLM/AI clients.Changes
Middleware: Switch from
redirect()torewrite()so markdown is served at the same URL.Sitemap generation: Generate hierarchical navigation for LLM discovery:
Testing