Enable using speech processor in Docker by using HTTP #11
+422
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
This depends on #12 |
mgaido91
added
documentation
mgaido91
changed the title
sarapapi
reviewed
Feb 4, 2026
Contributor
sarapapi
left a comment
sarapapi
left a comment
There was a problem hiding this comment.
I just have a few questions/clairification comments, but in general, it's good for me
| docker run --rm --gpus=all -p 8080:8080 http_speech_processor | ||
| ``` | ||
|
|
||
| And then, you can use `simulstream` setting the proxy HTTP processor to access your |
Contributor
There was a problem hiding this comment.
Suggested change
| And then, you can use `simulstream` setting the proxy HTTP processor to access your | |
| And then, you can use `simulstream` by setting the proxy HTTP processor to access your |
Contributor
Author
There was a problem hiding this comment.
here simulstream is the command, not the tool
| @@ -0,0 +1,32 @@ | |||
| # Example of Docker Speech Processor | |||
|
|
|||
| This folder contains a Dockerfile that is a working example of how to build a Docker | |||
Contributor
There was a problem hiding this comment.
Suggested change
| This folder contains a Dockerfile that is a working example of how to build a Docker | |
| This folder contains a [Dockerfile](examples/http_docker/Dockerfile) that is a working example of how to build a Docker |
| --metrics-log-file $YOUR_OUTPUT_JSONL_FILE | ||
| ``` | ||
|
|
||
| Please notice that this example Dockerfile runs a Canary sliding window speech processor. |
Contributor
There was a problem hiding this comment.
Suggested change
| Please notice that this example Dockerfile runs a Canary sliding window speech processor. | |
| Please notice that [this Dockerfile example](examples/http_docker/Dockerfile) runs a Canary sliding window speech processor. |
| self.close_session(session_id) | ||
|
|
||
| def shutdown(self) -> None: | ||
| """ |
Contributor
There was a problem hiding this comment.
Either you add comments like this to all methods, or you drop this one that is pretty redundant
|
|
||
| def get_speech_chunk_size(self, session_id): | ||
| processor = self.speech_processor_manager.get(session_id) | ||
| self._send_json_response(200, {"speech_chunk_size": processor.speech_chunk_size}) |
Contributor
Author
There was a problem hiding this comment.
these are stardard HTTP codes. 204 means no content, 200 is just "everything good". This is standard HTTP protocol...
| processor = self.speech_processor_manager.get(session_id) | ||
| output = processor.process_chunk( | ||
| np.frombuffer(base64.b64decode(waveform), dtype=np.float32)) | ||
| self._send_json_response(200, { |
| def put_source_language(self, session_id, language): | ||
| processor = self.speech_processor_manager.get(session_id) | ||
| processor.set_source_language(language) | ||
| self._send_json_response(204) |
Contributor
There was a problem hiding this comment.
Add a comment about the meaning of the response
| yaml_config(args.speech_processor_config), server_config.pool_size, server_config.ttl | ||
| ) | ||
| speech_processor_loading_time = time.time() - speech_processor_loading_time | ||
| LOGGER.info(f"Loaded speech processor in {speech_processor_loading_time:.3f} seconds") |
Contributor
There was a problem hiding this comment.
This is the time to load the model, also, right?
Co-authored-by: sarapapi <57095209+sarapapi@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why is needed?
For sharing and reusing speech processors, it is convenient to be able to run speech processors in docker, so that the full environment is available. This is important e.g. for IWSLT campaigns, where organizers need to run participants' solutions.
What does the PR do?
It creates a new gRPC-based speech processor and a HTTP server that exposes a configured speech processor. In this way, the simulstream server/inference can be run setting the HTTP-based speech processor adn configuring it to communicate with the HTTP server that can run in a Docker container.
How is this documented?
Updated documentation and added an example to build a docker in the repo.
How was the PR tested?
Manual runs.