Overview

A Mink deployment has four components. Nodes serve clients and hold only caches and rebuildable working copies. An object store holds every record. A SQL database holds the metadata log that coordinates the nodes. A lakehouse catalog holds the tiered history.

Anatomy of a node

Each node runs the same stack. An Arrow Flight listener serves typed batches, lookups and admin over gRPC, and a Kafka listener serves Kafka clients. Behind them, the node service decides whether this node owns a bucket or redirects, opens tablets under an epoch, and hosts the background loops. The coordinator runs on whichever node holds the lease. Every part reads cluster state from the same metadata view and proposes changes to the same command log. The s3stream engine moves records to and from object storage.

Tables and buckets

The unit of storage is a bucket. A table has bucket_count buckets, or bucket_count per partition when partitioned. Each bucket is one s3stream stream, an ordered log of record batches with offsets assigned by the bucket's leader. A primary-key table adds a KV tablet per bucket that materializes the latest row per key and writes its changelog into the same stream.

Table type	Log tablet	KV tablet	Reads
Log table	Yes	No	Log scan, tail
Primary-key table	Yes, holds the changelog	Yes	Log scan, snapshot scan, lookup, prefix lookup, union

See Tables for the schema and options, Log tablets and KV tablets for the storage.

Ownership

Each bucket has one leader node. The coordinator assigns leaders round-robin across live nodes at create time, re-leads the buckets of a dead node on its next tick, and moves buckets between nodes when an operator runs mink cluster rebalance. The leader opens the stream with the bucket's leader epoch. S3stream rejects appends from any older epoch, so a replaced node cannot write. A request for a bucket that arrives at another node gets a redirect to the leader over Flight, or is forwarded when the write is table-routed. See Ownership and failover.

The metadata log

Cluster state is a single ordered log of commands in the SQL database: streams and objects for s3stream, node registrations, the catalog of databases, tables and partitions, bucket leaders, KV and lake snapshots, producer and group offsets. Every node appends commands and tails the log, applying each command to the same in-memory state machine. The lease holder takes periodic snapshots and truncates the log behind them. See Metadata.

Data paths

• Write. The client routes each row to a bucket by key hash or round-robin and sends the batch to the leader. The leader assigns offsets, appends to the stream, waits for WAL durability on object storage, then updates the KV working copy for primary-key tables. See Writes.
• Read. A log read serves batches up to the high watermark from the s3stream cache or the object store. A snapshot read walks the KV working copy. A tail read waits for new data. See Reads.
• Tiering. A worker on the lease holder copies log data and KV state into Iceberg per table on a freshness schedule and records the tiered log offset per bucket in the snapshot. See Tiering.
• Union. The lake snapshot plus the log tail after its recorded offset, per bucket, with primary-key deduplication. See Union read.

Background work

Loop	Where	Interval
WAL upload to stream objects	every node, s3stream	wal_upload_interval, 10s
KV snapshot upload	bucket leader	kv_snapshot_interval, 10m
Log retention trim	bucket leader	log_retention_interval, 5m
Coordinator tick: re-lead orphans, prune KV snapshots, auto partitions, expire offsets, tiering schedule	lease holder	coordinator_tick, 5s
Tiering worker	lease holder	tiering_poll_interval, 30s
Metadata snapshot and truncation, dead-object cleanup	lease holder	lease TTL derived
Heartbeat and lease renewal	every node	lease_ttl / 4