Topic: The current truncate rewrites files in place without atomic rename; explore what a crash-safe truncation would look like (write-to-temp + fsync + rename)

Date: 2026-05-28

Time: 18:20

I now have a thorough understanding of the codebase. Let me write the explanation.

Topics to Explore

[function] write-ahead-log/wal.py:readrecord — The CRC validation and partial-read handling in this function determines what "corrupt" vs "torn write" means during recovery, which directly affects whether a crash-interrupted temp file would be detected
[function] b-tree-storage-engine/btree.py:commit — Compare the B-tree's all-or-nothing WAL truncation (fsync data file, then truncate WAL) with the standalone WAL's partial truncation — understand why one is crash-safe and the other isn't
[general] directory-fsync-semantics — On Linux (ext4, XFS) and macOS (APFS), directory fsync has different guarantees; explore when os.fsync(dir_fd) is actually necessary for rename durability
[file] hash-index-storage/bitcask.py — The compaction method has the same class of crash-safety bug (delete-then-rename without atomic swap); trace the exact window where data loss occurs
[general] wal-segment-deletion-ordering — When truncation deletes entire WAL segments, the order of unlink() calls matters for crash safety; explore whether deleting oldest-first vs newest-first changes the recovery invariants

wal-truncate-not-crash-safe — truncate() in write-ahead-log/wal.py rewrites surviving records in place via open(f, "wb"), so a crash mid-rewrite permanently loses both the original and replacement data
btree-wal-truncate-is-safe — The B-tree's WAL commit() only truncates after fsyncing the data file, making truncation idempotent — a crash mid-truncation simply replays the WAL again on recovery
atomic-rename-requires-same-filesystem — os.rename() is atomic only when source and destination are on the same filesystem; crash-safe truncation must write the temp file in the same directory as the WAL
crash-safe-truncation-three-step — The correct pattern is write-to-temp + fsync temp + rename over original + fsync directory, ensuring the old file is never destroyed until the replacement is fully durable on disk
orphaned-tmp-means-original-intact — If a .wal.tmp file exists on startup, the atomic rename never completed, so the original .wal file is authoritative and the temp file should be deleted