Skip to content

Roadmap ​

Planned: Mink writes ended time buckets to Iceberg and serves one catalog over that history and Kafka.

History in Iceberg ​

Planned. Not built.

BuiltPlanned
TableThe records Kafka holdsThe Iceberg history and the newer offsets in Kafka, with no gap or overlap between them
Data filesBuilt from Kafka on read, and cachedEnded, assembled time buckets written to the bucket as Parquet. Newer buckets still built from Kafka.
File sizesAssembled up to assembled_bytesWritten slowly, after assembly, as large files only. No small files and no compaction jobs.
BucketIceberg metadata onlyIceberg metadata, and the history's data files
HistoryAs far back as Kafka's retentionKept after Kafka's retention deletes the records
FreshnessEvery record committed when the table loadsUnchanged
CatalogRead-onlyRead-only. Mink is the only writer.
one table, one cataloghistoryParquet in the bucket, large files onlynewer offsetsbuilt from Kafka on readoldestlast written bucketlatest