fix: ivf optimize fix

[iaojnh]

No description provided.

[feihongxu0824]

Please add tests to cover this fix and prove it's solid.

[iaojnh]

@greptile

[greptile-apps]

Greptile Summary

Fixed IVF optimization issues related to data type handling and metadata usage:

• Updated CosineConverter to use original_type_ instead of hardcoded DT_FP32, enabling support for FP16 and other data types
• Corrected metadata handling in MixedStreamerReducer::add_vec_with_builder to use original_query_meta_ for doc cache instead of target_streamer_query_meta
• Ensured vector reversion when using target builder with reformer
• Simplified converter application logic in IndexBuild()
• Added test coverage for IVF search with cosine normalization

Confidence Score: 5/5

• Safe to merge - fixes critical bugs in data type handling and metadata usage
• All changes are targeted bug fixes with clear intent: replacing hardcoded type checks with proper variable references, fixing metadata confusion in doc cache, and adding test coverage. The changes improve code correctness and flexibility without introducing risks.
• No files require special attention

Important Files Changed

Filename	Overview
src/core/quantizer/cosine_converter.cc	Replaced hardcoded DT_FP32 checks with original_type_ to support multiple data types (FP16, FP32)
src/core/mixed_reducer/mixed_streamer_reducer.cc	Fixed metadata handling in builder path - now correctly uses original_query_meta_ for doc cache and ensures reversion when needed
tests/core/algorithm/ivf/ivf_searcher_test.cc	Added test case for IVF search with cosine normalization converter

_{Last reviewed commit: 1d22fbe}

[greptile-apps]

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[iaojnh]

@greptile

[greptile-apps]

Greptile Summary

This PR fixes data type handling issues in IVF optimization code, specifically addressing metadata mismatches and hardcoded type assumptions.

Key Changes:

• Fixed metadata mismatch in MixedStreamerReducer::add_vec_with_builder where target_streamer_query_meta was incorrectly passed to PushToDocCache instead of original_query_meta_ (which matches the buffer creation)
• Ensured proper data conversion when using target_builder_ with reformer by setting need_revert = true
• Simplified converter logic by removing redundant need_convert check in IndexBuild()
• Made CosineConverter work with multiple data types (FP16, FP32) by replacing hardcoded DT_FP32 checks with original_type_
• Added comprehensive test TestSimpleCosine to verify cosine similarity search with IVF indexing and converters

Confidence Score: 4/5

• This PR is safe to merge with minimal risk after thorough testing
• The changes are focused bug fixes that address real data type handling issues. The metadata mismatch fix in PushToDocCache is critical and correct. The CosineConverter changes properly generalize from FP32-only to supporting multiple types. However, these changes affect core index building and conversion paths, so comprehensive testing is important before merging.
• Pay close attention to src/core/mixed_reducer/mixed_streamer_reducer.cc as it handles critical index building logic

Important Files Changed

Filename	Overview
src/core/mixed_reducer/mixed_streamer_reducer.cc	Fixed metadata mismatch in PushToDocCache, ensures proper data conversion with target_builder_, and simplified converter logic
src/core/quantizer/cosine_converter.cc	Replaced hardcoded DT_FP32 with original_type_ to support FP16 and other data types
tests/core/algorithm/ivf/ivf_searcher_test.cc	Added comprehensive test for cosine similarity search with IVF indexing and converters

_{Last reviewed commit: 4700dd8}