[FEATURE] Add metadata field to messages for stateful context tracking

[kevmyung]

Problem Statement

Currently, key information related to messages (usage, metrics, etc.) is not stored alongside the message when it is saved to history

For example, when performing context engineering in ConversationManager:

• Cannot identify which messages consume the most context
• Difficult to implement smart truncation/compaction strategies

Since this information is not included in messages, there is no way to access it statelessly.

Proposed Solution

1. Add a _metadata field to Message to allow storing related information
2. For assistant messages, store the usage/metrics provided by the model by default

message = {
    "role": "assistant",
    "content": [...],
    "_metadata": {
        "usage": {...},
        "metrics": {...}
    }
}

• _metadata serves as a generic extension point for future information
• All model providers already create new dicts with only required fields (content, role, tool_calls) when formatting messages for API calls, so _metadata is naturally excluded without additional logic
• Compatible with existing SessionMessage serialization, no session manager changes required

Use Case

1. Implement context size-based dynamic truncation strategies in ConversationManager
2. Analyze token usage of past conversations after session restoration
3. Track and monitor costs per conversation

Alternatives Solutions

Store metadata in agent.state instead of message:

agent.state = {
    "message_metadata": {
        0: {"usage": {...}},
        1: {"usage": {...}},
        ...
    }
}

Pros:

• No changes to message structure

Cons:

• Requires synchronization between messages and metadata (deletion, truncation)
• Index-based mapping can break after sliding window operations
• Separate lookup needed to access metadata for a specific message

Storing metadata alongside the message is more natural since they share the same lifecycle.

Additional Context

Similar functionality exists in LangChain: Link

Happy to create a PR for this feature.