remove_tombstones flag exists but the compaction manager never sets it to True — explore what conditions make tombstone removal correctDate: 2026-05-28
Time: 19:02
I have enough from the observations to give you a thorough answer. Let me trace through what's here.
sstable-and-compaction/sstable.py:merge_sstables — Trace the full merge logic including timestamp-based conflict resolution to understand how "newer wins" interacts with tombstone propagationlog-structured-merge-tree/lsm.py:compact — The only place tombstones are actually removed — compare its full-merge approach with the CompactionManager's partial-merge approachhash-index-storage/bitcask.py — Bitcask's merge() at line 195 removes tombstones freely because it rebuilds the entire keydir from one merged file — another example of full compaction making removal safeleveled-compaction-tombstone-policy — How production systems like LevelDB/RocksDB track "bottommost level" to decide when tombstone removal is safe during leveled compactionmulti-leader-replication/multi_leader.py — The replication-aware tombstone lifecycle: when distributed systems need tombstone TTLs beyond just local compaction safetytombstone-removal-requires-full-key-coverage — A tombstone for key K can only be safely removed during compaction if every SSTable that could contain an older entry for K is included in that compaction run; otherwise the deleted key can be resurrected from a surviving SSTablecompaction-manager-never-removes-tombstones — The CompactionManager in sstable-and-compaction/sstable.py always calls mergesstables with the default removetombstones=False, making it conservatively correct but causing unbounded tombstone accumulationlsm-compact-removes-tombstones-safely — The LSM Tree's compact() method (lsm.py:319) removes tombstones because it performs a full merge of all SSTables, guaranteeing no surviving SSTable can contain a superseded live valuedistributed-tombstone-removal-needs-replication-convergence — In replicated systems like the multi-leader module (multi_leader.py), tombstones cannot be safely removed until all replicas have received the delete, adding a replication-convergence constraint beyond the local compaction-coverage constraint