diff --git a/src/main/java/org/apache/sysds/runtime/instructions/ooc/OOCEvictionManager.java b/src/main/java/org/apache/sysds/runtime/instructions/ooc/OOCEvictionManager.java index f5ae7573b0a..099a26ebd9d 100644 --- a/src/main/java/org/apache/sysds/runtime/instructions/ooc/OOCEvictionManager.java +++ b/src/main/java/org/apache/sysds/runtime/instructions/ooc/OOCEvictionManager.java @@ -46,41 +46,63 @@ import java.util.concurrent.locks.ReentrantLock; /** - * Eviction Manager for the Out-Of-Core stream cache - * This is the base implementation for LRU, FIFO - * - * Design choice 1: Pure JVM-memory cache - * What: Store MatrixBlock objects in a synchronized in-memory cache - * (Map + Deque for LRU/FIFO). Spill to disk by serializing MatrixBlock - * only when evicting. - * Pros: Simple to implement; no off-heap management; easy to debug; - * no serialization race since you serialize only when evicting; - * fast cache hits (direct object access). - * Cons: Heap usage counted roughly via serialized-size estimate — actual - * JVM object overhead not accounted; risk of GC pressure and OOM if - * estimates are off or if many small objects cause fragmentation; - * eviction may be more expensive (serialize on eviction). - *

- * Design choice 2: - *

- * This manager runtime memory management by caching serialized - * ByteBuffers and spilling them to disk when needed. + * Eviction Manager for the Out-Of-Core (OOC) stream cache. *

- * * core function: Caches ByteBuffers (off-heap/direct) and - * spills them to disk - * * Eviction: Evicts a ByteBuffer by writing its contents to a file - * * Granularity: Evicts one IndexedMatrixValue block at a time - * * Data replay: get() will always return the data either from memory or - * by falling back to the disk - * * Memory: Since the datablocks are off-heap (in ByteBuffer) or disk, - * there won't be OOM. + * This manager implements a high-performance, thread-safe buffer pool designed + * to handle intermediate results that exceed available heap memory. It employs + * a partitioned eviction strategy to maximize disk throughput and a + * lock-striped concurrency model to minimize thread contention. + * + *

1. Purpose

+ * Provides a bounded cache for {@code MatrixBlock}s produced and consumed by OOC + * streaming operators (e.g., {@code tsmm}, {@code ba+*}). When memory pressure + * exceeds a configured limit, blocks are transparently evicted to disk and restored + * on demand, allowing execution of operations larger than RAM. + * + *

2. Lifecycle Management

+ * Blocks transition atomically through three states to ensure data consistency: + * * - * Pros: Avoids heap OOM by keeping large data off-heap; predictable - * memory usage; good for very large blocks. - * Cons: More complex synchronization; need robust off-heap allocator/free; - * must ensure serialization finishes before adding to queue or make evict - * wait on serialization; careful with native memory leaks. + *

3. Eviction Strategy (Partitioned I/O)

+ * To mitigate I/O thrashing caused by writing thousands of small blocks: + * + * + *

4. Data Integrity (Re-hydration)

+ * To prevent index corruption during serialization/deserialization cycles, this manager + * uses a "re-hydration" model. The {@code IndexedMatrixValue} container is never + * removed from the cache structure. Eviction only nulls the data payload. Loading + * restores the data into the existing container, preserving the original {@code MatrixIndexes}. + * + *

5. Concurrency Model (Fine-Grained Locking)

+ * */ + public class OOCEvictionManager { // Configuration: OOC buffer limit as percentage of heap