Chapter 3
Filesystem Fundamentals
Filesystems are the primary interface for storing and accessing data. This chapter covers the semantics every Linux filesystem shares and one local implementation behind them; Network & Distributed Filesystems takes the remote case. For systems engineers working in production environments, understanding filesystem internals isn't academic—it's essential for making informed decisions about performance, reliability, and operational characteristics.
The chapter builds up in four movements. First the fundamentals: what Linux filesystem semantics promise, how IO moves through the kernel, the page cache, and memory-mapped files—none of them local-only, which is why this is a chapter about fundamentals rather than about local filesystems. Then an implementation to reason about concretely: ZFS, with a deep dive into recordsize. With that in hand we look at what a local filesystem actually guarantees—synchronization, resiliency, and performance. We close with filesystem hacks worth knowing and the observability tools for debugging all of the above.
We'll go deep into implementation details, examining kernel code, on-disk formats, and operational trade-offs. By the end, you'll understand not just what different filesystems do, but why they behave the way they do—and how to debug them when things go wrong.
# Chapter Sections
Atomic rename, unlink with open file descriptors, O_TMPFILE, file locking (flock vs fcntl), and sparse files. POSIX operations that reliable software depends on.
IO Modes: Buffered vs Direct IOUnderstanding buffered IO, direct IO (O_DIRECT), synchronous IO flags (O_SYNC, fsync), and how different filesystems handle these modes.
The Linux Page CacheHow the page cache works, read-ahead mechanisms, dirty page writeback, and memory reclaim. Essential for understanding Linux filesystem performance.
Memory-Mapped FilesHow mmap() works, its relationship to the page cache, when writes actually reach disk, and mmap coherence across local and network filesystems.
ZFS: A Modern Local FilesystemEnd-to-end checksumming, copy-on-write architecture, integrated RAID, and snapshots. Understand ZFS's unique features, RAM requirements, and when to choose it over traditional filesystems.
Deep Dive: ZFS recordsizeWhat recordsize controls, compression interaction, workload matching, performance measurement, and code walkthrough.
Synchronization GuaranteesCoordinating concurrent access across processes on one host. POSIX semantics, advisory and mandatory locking, and what the kernel does and does not serialize for you.
Resiliency GuaranteesSurviving crashes, power loss, and corruption. Journaling, copy-on-write, fsync semantics, and checksumming.
Performance CharacteristicsBlock allocation, metadata overhead, caching strategies, and IO patterns on local storage.
Filesystem HacksBattle-tested techniques: squashfs on NFS, tmpfs for performance testing, loop devices, bind mounts, sparse files, and FUSE. Unconventional approaches that solve real problems.
Observability and DebuggingDevice and pool statistics with iostat, blktrace, and zpool iostat, plus eBPF/bpftrace scenarios. Telling a slow device from a stalled writeback from an application waiting on its own fsync.