Overview

A Mink deployment is Mink nodes next to a Kafka cluster, a Postgres database and an S3-compatible bucket. Nodes hold only caches. Any node can stop at any time, and the others take over its work.

Component	Holds
Kafka	The records, the only copy
Postgres	Tables, indexed offset ranges, data files and their statistics, snapshots, leases, members
Bucket	Iceberg table metadata, manifest lists, manifests and statistics files
Node	Caches of built data files and metadata objects
Schema Registry, optional	The schemas typed columns come from

Anatomy of a node

A node runs any of four roles, all four by default.

Role	Runs	Listens on
catalog	The Iceberg REST catalog	catalog_address
s3	The S3 endpoint that serves metadata objects and builds data files	s3_address
indexer	Indexers, writers and topic following
admin	The admin API, the console and /metrics	admin_address

Background work

Work	Runs on	Does
Membership	Every node	Registers the node in Postgres. See Cluster.
Topic following	indexer nodes	Follows each table's topic in Kafka: new partitions, deleted or recreated topics
Lease sharing	indexer nodes	Spreads each table's partition leases over the indexing nodes, and gives each table one writer. See Cluster.
Indexer	Holder of a partition lease	Reads the partition's committed records once and records data files. See Indexing.
Writer	Holder of a table's writer lease	Commits snapshots, assembles small files, publishes statistics, expires snapshots and collects old objects. See Snapshots.