Skip to content

Anthropic: Messages Batch API not instrumented #108

@AbhiPrasad

Description

Gap

The Anthropic Messages Batch API (client.messages.batches) is not instrumented. The Messages and AsyncMessages wrapper classes only wrap create() and stream(). Accessing client.messages.batches on a wrapped client falls through via Wrapper.__getattr__ to the original unwrapped resource — batch operations work but produce no spans.

The Batch API allows submitting up to 100,000 message requests for asynchronous processing at 50% cost. It is a GA production API.

What is missing

At minimum:

  • client.messages.batches.create() — should create a span logging the batch ID, number of requests, model, and other configuration
  • client.messages.batches.results() — could optionally create spans for individual results when retrieved

Lower priority:

  • client.messages.batches.retrieve() / list() / cancel() / delete() — administrative operations

Braintrust docs status

not_found — the Anthropic integration page at braintrust.dev/docs/integrations/ai-providers/anthropic does not mention the Batch API.

Upstream sources

Local files inspected

  • py/src/braintrust/wrappers/anthropic.py:
    • Messages class (line 146): only defines stream() and create() methods
    • AsyncMessages class (line 57): only defines create() and stream() methods
    • Neither class references batches
  • py/src/braintrust/wrappers/test_anthropic.py — no batch-related test cases
  • Grep for batches across the wrappers directory: zero results in anthropic files

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions