Skip to content

Architecture

This section is organized by responsibility so cross-cutting engine topics live at the top level and component internals stay grouped under dedicated section indexes.

High-level system view

HestiaStore is organized around a SegmentIndex orchestration layer that routes operations across stable segments, keeps hot state in memory, and exposes runtime metrics without letting monitoring code touch index files directly.

HestiaStore high-level architecture

Source: system-overview.plantuml

Main runtime components

  • SegmentIndex is the public engine entry point. It owns request routing, write buffering, flush/compaction scheduling, split orchestration, and runtime metrics.
  • Key-to-segment map resolves which segment should serve a key range so the index can route reads and writes without scanning every segment.
  • SegmentRegistry is the lifecycle and cache boundary between SegmentIndex and loaded segment data. It decides when segment resources are created, reused, evicted, and closed.
  • Segment is the stable storage shard. A segment combines a main SST, a sparse index, a Bloom filter, and delta cache files to balance write cost and lookup latency.
  • Monitoring adapters consume SegmentIndex.metricsSnapshot() and runtime management APIs. They observe the engine, but they do not read or rewrite segment files directly.

High-level data flow

  • Write path: requests enter SegmentIndex, land in the write buffer or partitioned ingest layer, and are later drained into segment storage through flush and compaction work.
  • Read path: SegmentIndex routes a key through the key-to-segment map, obtains the target segment through the registry, and then reads segment structures such as delta cache, Bloom filter, sparse index, and main SST.
  • Observability path: monitoring integrations read immutable runtime snapshots from the engine and expose them through REST JSON or external metrics systems.

Cross-cutting topics

Component sections

  • Monitoring — runtime monitoring bridge and management API contracts.
  • SegmentIndex — top-level index orchestration: read/write paths, caching, performance, and index concurrency.
  • Segment — central place for segment internals: file layout, delta cache, Bloom filter, sparse/scarce index, and segment lifecycle.
  • Registry — segment registry state machine, cache-entry model, and concurrent loading/unloading flows.