Skip to content

WAL Runtime

This page explains the current internal WAL runtime ownership in org.hestiastore.index.segmentindex.wal.

WalRuntime remains the stable compatibility-facing entry point, but it is no longer the implementation home for every WAL concern.

Ownership boundaries

  • WalRuntime: public facade for open(), recover(), appendPut(), appendDelete(), onCheckpoint(), statsSnapshot(), and close()
  • WalMetadataCatalog: owns format.meta, checkpoint.meta, temp-file promotion, metadata validation, and WAL segment discovery
  • WalSegmentCatalog: owns active segment rotation, retained-byte accounting, checkpoint cleanup, and runtime segment inventory
  • WalRecordCodec: owns WAL record encoding, CRC handling, and record-body decoding
  • WalRecoveryManager: owns replay scan flow, invalid-tail detection, corruption-policy handling, and checkpoint clamp behavior during recovery
  • WalWriter: owns append-path orchestration above storage append, segment admission, metrics, and durability-policy delegation
  • WalSyncPolicy: owns ASYNC vs SYNC vs GROUP_SYNC durability behavior, pending-sync batching, durable LSN tracking, and sync-failure state
  • WalRuntimeMetrics: owns WAL append/sync/corruption counters and snapshot assembly

Concurrency model

  • Append producers submit through the bounded ArrayBlockingQueue using its interruptible timed offer. Producers no longer hold the WAL state monitor or wait in fixed park intervals while queue capacity is unavailable.
  • A fair admission read/write lock lets producers offer concurrently. Close takes the write side before marking the runtime closed and placing the stop marker, so every accepted append stays ahead of that marker.
  • The single append worker remains the ordering authority. It assigns LSNs and mutates WAL writer/catalog state under the WAL monitor. Segment indexes obtain its named daemon thread factory from their per-index ExecutorRegistry, while WalRuntime retains stop, drain, and join ownership.
  • In SYNC mode, records drained into one worker batch share one physical sync. Their futures are completed only after that sync succeeds, so acknowledgements retain synchronous durability.
  • Recovery, checkpoint, cleanup, worker append, and durability-state changes remain serialized by the WAL monitor.
  • In delayed GROUP_SYNC mode, the append worker polls the queue only until the pending sync deadline. It also checks that deadline after every worker batch, so continuous append traffic cannot postpone durability indefinitely.
  • durableLsn() remains available as a best-effort read without requiring callers to hold the monitor.

Metadata and catalog model

The current extraction step introduces one internal catalog view for WAL:

  • format metadata from wal/format.meta
  • checkpoint metadata from wal/checkpoint.meta
  • discovered WAL segment inventory from wal/*.wal

This is intentionally still WAL-local. It is the bridge toward the broader index-state catalog work tracked in backlog item 84.

Stable invariants

  • Segment file naming remains <20-digit-base-lsn>.wal.
  • Checkpoint LSN remains monotonic at runtime.
  • Recovery still validates monotonic LSN ordering across segment boundaries.
  • Close never silently abandons an append accepted before its stop marker.
  • TRUNCATE_INVALID_TAIL vs FAIL_FAST semantics remain unchanged.
  • Public WalRuntime metrics and log event names remain unchanged.