feat(dsl): implement workflow tool mode handling and MCP server node #212
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7074d5305c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
69
to
73
| async execute() { | ||
| // The Docker runner handles execution. | ||
| // At runtime, the workflow engine will resolve the dynamic endpoint. | ||
| return { | ||
| endpoint: 'http://localhost:8080', // Placeholder |
There was a problem hiding this comment.
In the workflow runner, only component.execute is called; Docker execution is not triggered automatically (see run-component.activity.ts, which expects components to call runComponentWithRunner themselves). This execute implementation just returns a placeholder endpoint and never starts the container, so workflows that include this node will not actually launch an MCP server and will point to http://localhost:8080 even if nothing is running. Unless an external server is already up, the node will silently succeed but downstream tool calls will fail.
Useful? React with 👍 / 👎.
Comment on lines
175
to
145
| let encryptedCredentials: string | undefined; | ||
| if (authToken) { | ||
| const encryptionMaterial = await this.encryption.encrypt(authToken); | ||
| encryptedCredentials = JSON.stringify(encryptionMaterial); |
There was a problem hiding this comment.
registerRemoteMcp encrypts a raw authToken string, but getToolCredentials later does JSON.parse on the decrypted payload. For non-JSON tokens, this throws and credentials are dropped, so remote MCP auth tokens become unretrievable. Either JSON.stringify the token before encryption or make getToolCredentials accept plain strings for remote-mcp tools.
Useful? React with 👍 / 👎.
betterclever
force-pushed
the
eng-97/workflow-tool-mode
branch
2 times, most recently
from
f79081a to
190a3d9
Compare
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Add port mapping support to DockerRunner - Refactor mcp-server component to use dynamic runner config and exposed ports - Fix trace event mapping in E2E tests - Add container cleanup to mcp-tool-mode E2E test - Ensure local MCP registration uses actual containerId and endpoint - Fix workflow definition validation by adding node positions in test Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
betterclever
force-pushed
the
eng-97/workflow-tool-mode
branch
from
190a3d9 to
ca18b5e
Compare
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
… MCP proxying - Replace deprecated SSEClientTransport with StreamableHTTPClientTransport - Fix lint errors (trailing whitespace in constructor and emitProgress) - Gateway currently executes components inline (to be refactored to Temporal) Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Add executeToolCallSignal and toolCallCompletedSignal for MCP tool calls - Add getToolCallResult query for polling tool execution results - Refactor callComponentTool to signal workflow instead of inline execution - Add queryWorkflow method to TemporalService - Tool calls now execute on worker with full Docker/secrets/storage support Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
…ation - Refactor component tool execution to run on Temporal workers via signals/queries - Implement validation for workflow run access and organization ownership - Add comprehensive telemetry: log tool execution (STARTED, COMPLETED, FAILED) to trace repository - implement robust external MCP proxying with 30s timeouts and exponential backoff retries - Add support for tool filtering via allowedTools header - Add E2E test for MCP gateway tool discovery and execution Signed-off-by: Antigravity <antigravity@google.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Extract X-Run-Id and X-Allowed-Tools headers in McpGatewayController - Pass organizationId and allowedTools to McpGatewayService - Add basic protocol version validation - Fix type casting for MCP transport request handling Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
…eway - Add McpAuthService to manage short-lived, run-bounded session tokens - Implement McpAuthGuard for RFC 6750 (Bearer) compliance and AuthInfo injection - Refactor McpGatewayController to use native MCP AuthInfo instead of internal AuthContext - Add internal endpoint /internal/mcp/generate-token for session token issuance - Update E2E tests to validate the complete secure handshake and tool execution flow - Fix type safety issues in MCP transport integration Signed-off-by: Antigravity <antigravity@google.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
…script harness - Ensure component 'parameters' are passed through tool registration and execution signals - Correctly map agent 'arguments' to component 'inputs' in runComponentActivity - Fix race condition in logic-script harness by ensuring output directory exists before write - Update E2E gateway test to reflect correct registration and execution pattern - Clean up debug logs and resolve linting errors across gateway and worker Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Detailed plan for enabling graph-based tool→agent binding: - Phase 1: Compiler tracks tool→agent edges - Phase 2: Runtime passes connectedToolNodeIds to agent - Phase 3: Agent queries MCP Gateway for tools - Phase 4: Gateway filters tools by nodeIds - Phase 5: E2E tests Linear-issue: ENG-132 Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Phase 1 of ENG-132: - Added connectedToolNodeIds to WorkflowNodeMetadata (backend/worker) - Added tools input port to AI agent component - Included connectedToolNodeIds in RunComponentActivityInput metadata Linear-issue: ENG-132 Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Phase 2 of ENG-132: - Modified compiler to collect connectedToolNodeIds from graph edges - Updated validator to allow multiple edges to 'tools' port - Virtualized 'tools' output port for nodes in tool mode - Updated DTO schemas to support tool mode metadata - Added unit test to verify tool->agent binding Linear-issue: ENG-132 Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Phase 3 of ENG-132: - Updated shipsecWorkflowRun to extract connectedToolNodeIds from node metadata - Included connectedToolNodeIds in activity metadata for agent discovery - Synchronized worker types for workflow execution Linear-issue: ENG-132 Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Phase 4 of ENG-132: - Added nodeIds filter to ToolRegistryService.getToolsForRun - Updated McpGatewayService to support tool scoping by nodeId - Included allowedNodeIds in session token and metadata - Updated InternalMcpController to allow scoped token generation Linear-issue: ENG-132 Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Phase 3 of ENG-132: - Updated ExecutionContextMetadata to include connectedToolNodeIds and organizationId - Modified runComponentActivity to inject metadata into the execution context - Ensured organizationId is available for agent tool discovery Linear-issue: ENG-132 Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Phase 5 of ENG-132: - Added McpGatewayClient support to AI agent component - Implemented runtime tool discovery via MCP Gateway - Scoped tool discovery using connectedToolNodeIds - Integrated discovered tools with agent reasoning loop - Added session token generation helper Linear-issue: ENG-132 Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Add ApiKeysModule import to McpModule to fix DI issues - Cache gateway servers/transports by runId + allowedNodeIds for agent isolation - Implement buildMcpToolSchema to convert JSON Schema to Zod format - Fix MCP tool content handling (extract text from array results) - Add proper imports for StreamableHTTPClientTransport and Client Tests now pass: - Agent can run with no tools - Multiple agents have isolated tool sets based on graph connections This enables workflow graphs to connect tool-mode nodes directly to agents, with the agent automatically discovering and using only its connected tools. Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Explicitly type tool execution to return Promise<string> - Add comprehensive fallback conversions for tool result content - Handle all formats: arrays, strings, objects, primitives - Update test to use more reliable httpbin.org endpoint Tool discovery is working correctly (tests 1 & 3 pass). Test 2 fails due to tool execution issues, not discovery. Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Create DebugLogger utility for structured JSON logging to /tmp/shipsec-debug/ - Separate heartbeat logs to dedicated file (not printed to console) - Add view-debug-logs.ts script for easy log viewing/filtering - Convert agent gateway functions to use DebugLogger - Remove verbose console heartbeat logs (15-second polling logs) - Add getRecentLogs, getLogsByContext, getLogsByLevel utilities Benefits: - Cleaner console output (no heartbeat spam) - All debug context centralized in one place - Easy filtering by context, level, or search terms - Structured JSON format for programmatic access Debug logs now available at: /tmp/shipsec-debug/worker.log Usage: bun scripts/view-debug-logs.ts [filter] [line-count] Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Log tool invocation arguments to debug-logger - Log MCP result type and content before conversion - Log final tool execution result with preview - Log errors with stack traces for debugging Root cause identified in Test 2: MCP protocol timeout on external endpoint The tool is called successfully, gateway discovery works, but the MCP callTool() times out waiting for the HTTP endpoint response. This is NOT a tool discovery or integration issue - it's endpoint latency. Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Adds the new OpenCode agent component with configurable 'providerConfig'. Refactors common gateway token logic into a shared 'utils.ts' used by both AI Agent and OpenCode. Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Add zai-coding-plan to LLMProviderSchema with apiKey and modelId support - Fix OpenCode component to use proper model format (provider/modelId) - Configure Z.AI provider with apiKey in provider.options - Fix MCP server config to use type: "remote" instead of transport: "http" - Remove unused env var API key handling in favor of provider config Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Use sh -c with properly quoted prompt string to handle multi-word prompts - Escape single quotes in prompt to prevent shell injection - Add current-state.md documenting investigation and findings - Add opencode E2E test Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
…er script - Remove --quiet flag (doesn't exist in opencode 1.1.34), use --log-level ERROR - Use wrapper script approach to handle prompt file reading inside container - Set entrypoint to /bin/sh to override default opencode entrypoint - Fix test assertions to check outputSummary.report instead of output.report - Update current-state.md with resolution details E2E tests now passing: 2 pass, 0 fail Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
…ided
When a custom systemPrompt is provided, the task was not being included
because the {{TASK}} placeholder only exists in the default template.
Now the task is always appended to ensure OpenCode receives the full prompt.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Add optional baseUrl and headers properties to zai-coding-plan provider - Add zai-coding-plan to ModelProvider type in ai-agent.ts - This fixes TypeScript build errors when using the new provider Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
… d.ts files - Remove emitDeclarationOnly: true from worker/tsconfig.json - This allows TypeScript to emit both .js and .d.ts files - Fixes backend typecheck errors when importing from @shipsec/studio-worker/workflows - The worker still uses source files directly via bun, so .js files don't interfere Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
- Rename unused 'error' to '_error' in catch block to satisfy ESLint - Apply code formatting from linter to opencode.ts Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>
feat: MCP AWS servers and proxy (ENG-103) Consolidating the work in 1 PR from the stack
…nt-ui feat: tool mode UI + agent orchestration (ENG-101/102) Consolidating the work in one PR
feat: OpenCode agent component (ENG-100) Consolidating the work in final PR
feat: implement tool-mode orchestration (ENG-132) Consolidating
feat(mcp): implement MCP Gateway for internal and external tools Consolidating
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
WorkflowNodeMetadatato supportmode('normal' | 'tool') andtoolConfig.core.mcp.servercomponent (Docker-based) to allow workflows to start and manage external tool providers.Stacked on: #208 (ENG-96)