Skip to content

Propagate trace context for entity operations#654

Open
cgillum wants to merge 2 commits intomainfrom
fix/entity-trace-context
Open

Propagate trace context for entity operations#654
cgillum wants to merge 2 commits intomainfrom
fix/entity-trace-context

Conversation

@cgillum
Copy link
Member

@cgillum cgillum commented Mar 1, 2026

Summary

Entity operations triggered from orchestrations don't propagate distributed trace context because SendEntityMessageAction lacks a parentTraceContext field, and entity operation events (EntityOperationSignaledEvent, EntityOperationCalledEvent) also lack this field. This causes entity operations to appear disconnected from their parent orchestration traces in distributed tracing tools.

Changes

Proto updates (see microsoft/durabletask-protobuf#64)

  • Add parentTraceContext field to SendEntityMessageAction
  • Add parentTraceContext field to EntityOperationSignaledEvent and EntityOperationCalledEvent

SDK sending side

  • Set ParentTraceContext in ProtoUtils.ConstructOrchestratorResponse for entity message actions, matching existing behavior for tasks and sub-orchestrations

SDK receiving side

  • Extract parentTraceContext from EntityOperationSignaledEvent / EntityOperationCalledEvent in ToEntityBatchRequest, populating OperationRequest.TraceContext

Tests

  • Unit tests for trace context propagation on the sending side (5 tests)
  • Unit tests for trace context extraction on the receiving side (3 tests)

Merge note

Do not merge until the protobuf change (microsoft/durabletask-protobuf#64) has been properly merged and vendored.

Follow-up work

  • A corresponding server-side change is needed in the Durable Task Scheduler backend to propagate SendEntityMessageAction.ParentTraceContext to EntityOperationSignaledEvent/CalledEvent.ParentTraceContext on entity operation events.
  • The SDK entity worker (OnRunEntityBatchAsync) needs span creation for entity operations to complete the distributed tracing story.

Fixes #653

Entity operations triggered from orchestrations don't propagate
distributed trace context because SendEntityMessageAction lacks a
parentTraceContext field. This causes entity operations to appear
disconnected from their parent orchestration traces.

Changes:
- Vendor updated proto with parentTraceContext field on SendEntityMessageAction
  (see microsoft/durabletask-protobuf#64)
- Set ParentTraceContext in ProtoUtils.ConstructOrchestratorResponse for
  entity message actions (matching existing behavior for tasks and
  sub-orchestrations)
- Add unit tests for entity trace context propagation

Note: A corresponding server-side change is also needed in the Durable Task
Scheduler backend to propagate SendEntityMessageAction.ParentTraceContext
to OperationRequest.traceContext on entity work items.

Fixes #653
Copilot AI review requested due to automatic review settings March 1, 2026 16:23
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a distributed tracing gap where entity operations dispatched from orchestrations lacked trace context propagation. Without ParentTraceContext on SendEntityMessageAction, entity spans appeared as disconnected root traces in distributed trace viewers (e.g., Aspire Dashboard, Jaeger) rather than as children of the orchestration that triggered them.

Changes:

  • Added parentTraceContext field (field 5) to the SendEntityMessageAction proto message, consistent with ScheduleTaskAction and CreateSubOrchestrationAction
  • Set ParentTraceContext = CreateTraceContext() in ProtoUtils.ConstructOrchestratorResponse for entity message actions, matching the existing behavior for activities and sub-orchestrations
  • Added unit tests verifying trace context propagation for signal/call entity, null-activity fallback, non-entity-converted actions, and unique span ID generation

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/Grpc/orchestrator_service.proto Adds parentTraceContext field (index 5) to SendEntityMessageAction
src/Shared/Grpc/ProtoUtils.cs Sets ParentTraceContext on entity message actions; incidental character encoding change in unrelated comments
test/Worker/Grpc.Tests/ProtoUtilsTraceContextTests.cs New test class covering trace context propagation for entity operations

Comment on lines 1059 to +1068
@@ -1063,7 +1065,7 @@ internal static T Base64Decode<T>(this MessageParser parser, string encodedMessa
}
}

// If the value starts with the 'dto:' prefix, it may represent a DateTime value � attempt to parse it.
// If the value starts with the 'dto:' prefix, it may represent a DateTime value � attempt to parse it.
Copy link

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment at this line (and line 1068) contains a Unicode replacement character (U+FFFD, rendered as ?) where there was previously a dash character (likely an en-dash ). This is a character encoding corruption introduced by this PR and should be replaced with the intended dash character or a plain hyphen-minus (-) to ensure the source file has clean encoding.

Copilot uses AI. Check for mistakes.
…quest

Add parentTraceContext fields to EntityOperationSignaledEvent and
EntityOperationCalledEvent proto messages, and extract them in
ToEntityBatchRequest to populate OperationRequest.TraceContext.

This is the receiving side of the trace context propagation: once the
DTS backend populates parentTraceContext on entity operation events
(from SendEntityMessageAction.ParentTraceContext), the SDK will
correctly extract and use the trace context for entity operations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Entity operations don't propagate trace context from parent orchestration

2 participants