fix: activation memory config crashes get_default() with OpenAI #1051
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Two bugs in
default_config.pythat crashget_default()whenenable_activation_memory=True:Bug 1:
act_meminjected into MOSConfig (extra=forbid)get_default_config()addsact_memdict to the config passed toMOSConfig(**config_dict), butMOSConfighas noact_memfield and inheritsextra="forbid"fromBaseConfig:Bug 2: KV cache with
openaibackendget_default_cube_config()hardcodesextractor_llmbackend to"openai"for KV cache memory, butKVCacheMemoryConfigvalidator requireshuggingface/huggingface_singleton/vllm(KV cache extracts internal attention tensors viabuild_kv_cache, which needs local model access):Reproduction
Fix
act_memfromget_default_config()— theenable_activation_memorybool flag is sufficient forMOSConfig;act_memconfig belongs in MemCube config only.get_default_cube_config(), require explicitactivation_memory_backendkwarg (e.g."huggingface") instead of hardcoding"openai". Skipact_memgracefully when no compatible local backend is provided.Test plan
get_default(openai_api_key="sk-...", enable_activation_memory=True)no longer crashesget_default(openai_api_key="sk-...", enable_activation_memory=True, activation_memory_backend="huggingface", activation_memory_llm_config={...})creates valid act_mem configget_default(openai_api_key="sk-...")continues to work as before