fix(rust): enable Union type cross-language serialization between Rust and Java

[ariesdevil]

Why?

Struct fingerprint mismatch for enum fields

When computing struct version hash for cross-language compatibility, Java and C++ treat enum fields as nullable=true by default. However, Rust's proc-macro cannot determine at compile time whether a field type is an enum, causing fingerprint mismatch.

Cross-language Union serialization fails

Java's AbstractObjectSerializer expects to read type_id for non-final fields, but Rust/C++ skip type_id for Union fields per xlang spec. This caused Type id 104 not registered errors when Java tried to deserialize Rust-serialized Union data.

What does this PR do?

Rust changes:

• Union-compatible enum handling for xlang mode
• Fix field_need_write_type_info() to handle UNION TypeId

Java changes:

• AbstractObjectSerializer: Skip type_id read for Union types
• UnionSerializer: Add read()/write() delegating to xread()/xwrite()

Tests:

• Add testUnionXlang for Rust enum <-> Java Union2 interoperability

Related issues

Does this PR introduce any user-facing change?

[ ] Does this PR introduce any public API change?
[x] Does this PR introduce any binary protocol compatibility change?
Note: Struct version hash for structs containing enum fields will now match Java/C++.

Benchmark

Others

It's really hard to determine this bug 😭...

[chaokunyang]

@ariesdevil Most of such bugs are caused by the type system differences between laungages.

I'm refactoring the whole xlang serialization to mark all fields not null by default, I think this refactor will resolve most such tricky issue.

here is the principle:

Java:

1. All fields are not null except Optional
2. All fields don't track ref by default

Rust

1. All fields are not null except Option/RcWeak/ArcWeak
2. Only Rc/Arc/RcWeak/ArcWeak track ref

Go

1. all fields are not null by default, except pointer
2. slice/map/interface are not null by default

C++

1. All fields are not null except Optional, shared_ptr/unique_ptr are not null by default
2. shared_ptr track ref by default

Python

1. All fields are not null except Optional type hints
2. All fields don't track ref by default

For native serialization, keep current nullable/ref principles for max compatibility

[ariesdevil]

@chaokunyang Thanks for informing me of the changes. I updated the PR desc and some code, PTAL again.

[chaokunyang]

Hi @ariesdevil , I finished xlang nullable refactor in #3093. It conflict with your pr, could you megre main branch and resolve the conflict? And since all fields are not null in rust by default except Optional, maybe the problem is more simple.

For enum, java and c++ will take it as not null by default now.

[chaokunyang]

Is this xlang param really necessary? could we avoid add this parameter?

[chaokunyang]

It's not in hotpath, add a new parameter is OK. Maybe we should remove this method in future, and handle type id idrectly in type_resolver.rs

[ariesdevil]

Yes, it's necessary, for union-compatible enum, xlang specific returned TypeId:

• xlang=true -> return TypeId::UNION for cross language mode
• xlang=false return enum TypeId otherwise

[chaokunyang]

LGTM

[ariesdevil]

Thanks for your patient review, I learned a lot.

[chaokunyang]

@ariesdevil #3107 refactored java ObjectSerializer, we don't need special code path for enum now

[ariesdevil]

@ariesdevil #3107 refactored java ObjectSerializer, we don't need special code path for enum now

@chaokunyang Wonderful job, happy new year : )