Skip to content

fix: add missing __aiter__ to TextGenerationStream for async for loops#196

Merged
adambalogh merged 1 commit intoOpenGradient:mainfrom
amathxbt:fix/streaming-async-aiter-missing
Mar 23, 2026
Merged

fix: add missing __aiter__ to TextGenerationStream for async for loops#196
adambalogh merged 1 commit intoOpenGradient:mainfrom
amathxbt:fix/streaming-async-aiter-missing

Conversation

@amathxbt
Copy link
Contributor

Bug

TextGenerationStream implements __anext__ for async iteration but is missing the required __aiter__ protocol method.

Any code using async for chunk in stream: when _is_async=True will crash with:

TypeError: 'TextGenerationStream' object is not an async iterable

This silently breaks all async streaming usage — a critical runtime regression.

Root Cause

Python's async iteration protocol requires both __aiter__ (to return the iterator) and __anext__ (to yield items). Only __anext__ was implemented.

From the Python data model docs:

An object must implement both __aiter__ and __anext__ to be an async iterable.

Fix

Added __aiter__ method that validates the _is_async flag and returns self:

def __aiter__(self):
    """Return async iterator (required for async for loops)."""
    if not self._is_async:
        raise TypeError("Use __iter__ for sync iterators")
    return self

Impact

  • Fixes async for chunk in stream: pattern documented in SDK examples and README
  • No behaviour change for sync iteration
  • Low risk, zero-dependency change

Testing

async def test_async_stream():
    stream = TextGenerationStream(_iterator=some_async_gen(), _is_async=True)
    async for chunk in stream:  # Would previously raise TypeError
        print(chunk)

Without __aiter__, async for chunk in stream: raises
TypeError: 'TextGenerationStream' object is not an async iterable.
This makes async streaming completely broken when _is_async=True.
@amathxbt
Copy link
Contributor Author

Hey @adambalogh this is something critical you need to take action quickly guys !

@adambalogh adambalogh merged commit 027d82b into OpenGradient:main Mar 23, 2026
4 checks passed
@adambalogh
Copy link
Collaborator

Thanks for your contribution!

@amathxbt
Copy link
Contributor Author

There's still some pr opened can you check them on your X dm or let me provide them here too

Hey team,Just pushed 4 bug fix PRs to the OpenGradient-SDK repo — all critical issues found during a code audit. Here's a quick summary:🔴 PR #196 — Async Streaming Completely Broken
TextGenerationStream was missing the aiter method, meaning async for chunk in stream: would crash with a TypeError for anyone using async streaming. Fixed ✅
🔗 https://t.co/0ZwOlWzmNo

🔴 PR #197 — TEE Metadata Lost in Streaming Responses
StreamChunk.from_sse_data() was silently dropping tee_id, tee_endpoint, and tee_payment_address — always returning None even when the server sent them. This made it impossible to verify which TEE served a streaming request. Fixed ✅
🔗 https://t.co/LcZaGCpMFI

🟠 PR #198 — run_with_retry Silently Returns None
Passing max_retries=0 caused an empty loop and a silent None return instead of executing the transaction — leading to confusing downstream crashes. Added proper validation + safety net. Fixed ✅
🔗 https://t.co/Ymdk9FAjnr

🟡 PR #199 — Public Types Missing from all
InferenceResult, ModelOutput, ModelRepository, and FileUploadResult were imported but not listed in all — breaking wildcard imports and IDE auto-complete. Fixed ✅
🔗 https://t.co/2RraRDHDpx Would appreciate a quick review when you get a chance 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants