On a local filesystem, processes on the same host share one page cache, so a read that follows a completed write normally observes it. Each NFS client has its own cache. NFS therefore uses close-to-open consistency: a writer's changes may become visible earlier, but are guaranteed to be visible to another client after the writer closes and the reader subsequently opens the file.

This section explores exactly what that guarantee means at the protocol level, what it costs in terms of network round-trips, what it does not guarantee, and how it affects the common atomic rename pattern.

# The Close-to-Open Contract

Close-to-open (CTO) provides two guarantees:

• On close: All modified data and metadata are flushed from the client's cache to the server before close() returns.
• On open: The client revalidates its cached copy against the server before open() returns. If the file changed on the server, the client purges its stale cache.

Together, these ensure a sequential handoff: if client A closes, then client B opens the same file, B is guaranteed to see everything A wrote. This is weaker than POSIX—which guarantees visibility after every write()—but strong enough for many workflows where files are written by one process and later consumed by another.

# Protocol Mechanics

To understand the cost of close-to-open, we need to look at the NFS RPCs involved. Let's trace what happens when client A writes a file and client B reads it afterward.

What happens on close()

When the application calls close(), the NFS client must push all dirty pages to the server. This involves two types of RPCs:

WRITE transfers data from the client's page cache to the server. The server may buffer these in its own RAM. COMMIT asks the server to flush the written data to stable storage (disk). Without COMMIT, a server crash could lose buffered writes. The NFS client sends COMMIT at close time so that close-to-open consistency includes durability—not just visibility.

Cost: One WRITE RPC per dirty page (or group of pages, depending on wsize), plus one COMMIT. For a file with N dirty pages, that's N+1 round-trips at minimum [1]. This is why closing a large dirty file over NFS can be slow—the latency is hidden inside close(), and applications that don't check the return value of close() may miss write errors entirely.

What happens on open()

When client B calls open(), the NFS client checks whether its cached attributes for the file are still valid:

The GETATTR RPC fetches the file's current attributes from the server. If the mtime or change attribute differs from what the client has cached, the client invalidates its page cache. The actual data isn't fetched until the application calls read()—open just ensures the cache is known to be stale.

Cost: One GETATTR round-trip on every open(). This is the price of CTO consistency. For workloads that open many small files (e.g., a build system traversing source trees), these GETATTRs can become a significant bottleneck.

# The Attribute Cache

Outside of open(), the NFS client does not revalidate on every operation. Instead, it caches file attributes for a tunable duration. During this window, stat() calls return cached data without contacting the server, and read() calls may serve stale page cache contents.

The actual cache duration is adaptive between min and max: the client uses a heuristic based on how recently the file was modified. A file that hasn't changed in a long time gets cached closer to max; a frequently-changing file gets cached closer to min.

Important: The attribute cache only affects operations between open() and close(), and operations that don't go through open() at all (like stat()). The CTO guarantee overrides the attribute cache at the open/close boundaries. But if your application polls a file with stat() without reopening it, you'll see stale attributes for up to acregmax seconds.

# What Close-to-Open Does NOT Guarantee

CTO is a narrow contract. Several common expectations from local filesystems break on NFS:

• No mid-file consistency: If clients A and B both have the same file open, A's writes are not guaranteed to be visible to B until A closes and B subsequently reopens. During the open window, B may read stale data from its local page cache. There is no invalidation protocol between clients.
• Weak mmap() consistency: Memory-mapped regions only partly follow close-to-open. Closing a writable descriptor does flush pages dirtied through a mapping, but munmap() does not count as a close, stores made after the last writable close() are never flushed by it, and readers using mmap() may never see updates from other clients, even after reopening. See mmap coherence across filesystems for the full story.
• No write ordering between files: If client A writes to file X then writes to file Y, another client may see Y's changes before X's changes. Each file's close-to-open is independent—there is no cross-file ordering guarantee.
• No directory consistency: Directory listings (readdir()) are subject to the attribute cache. A newly created file may not appear in ls output on another client for up to acdirmax seconds, and opening by name can also use a cached positive or negative directory entry unless lookup caching is revalidated or disabled.

# Atomic Rename over NFS

The atomic rename pattern—finish a unique new file in the target directory, then replace the target—provides namespace atomicity. On NFS, the RENAME RPC is a single server-side operation, so processes resolving the target do not observe a partly replaced directory entry. Content handoff to the server is a separate step.

The problem is that when you call rename(), your data might not have reached the server yet. The NFS client's page cache may still hold dirty pages from your writes. The RENAME RPC tells the server to swap directory entries—but if the data pages haven't been flushed, the renamed file may be empty or incomplete from the perspective of other clients.

Let's trace what goes wrong, and then what goes right.

Wrong: rename() without flushing

Because rename() doesn't trigger a flush of the source file's dirty pages, the server may rename a file whose data hasn't arrived yet. On a local filesystem this can't happen—both write() and rename() operate on the same page cache. On NFS, the page cache is local to the client, and the RENAME RPC operates on the server.

Illustration: stabilize contents before rename

The diagram traces the case in which the server acknowledges a WRITE as unstable. flush_to_kernel(new_file) corresponds to flushing userspace buffers (fflush() in C), while stabilize_contents(new_file) requests writeback and stability (fsync() in C). Because this WRITE reply is UNSTABLE, the client must send COMMIT before RENAME. If the server instead replies that the WRITE is stable, the separate COMMIT is unnecessary.

This is not a complete crash-durability recipe. Whether the renamed directory entry survives a client or server failure depends on the deployed NFS version, client, server, backing filesystem, and requested operations. That durability question is distinct from the namespace atomicity provided by RENAME and is outside this example's scope.

Cost of the illustrated stabilization path

If one WRITE carries a small configuration file, this model has two request/reply pairs when the WRITE reply is stable (WRITE, RENAME), or three when COMMIT is required (WRITE, COMMIT, RENAME). These counts describe the illustrated RPC shape, not a wall-clock formula: writeback can begin earlier, clients can pipeline requests, and latency depends on the network and server.

[1] Modern NFS clients pipeline WRITEs, sending multiple RPCs concurrently up to the wsize window. The actual wall-clock time depends on network bandwidth and server throughput, not just round-trip count.
[2] close(new_file) generates zero RPCs in this illustration because stabilization has completed. Without that preceding step, close may trigger WRITE RPCs and, for data acknowledged unstable, the required COMMIT; stable WRITE replies omit COMMIT.

# NFSv4 Delegations

NFSv4 introduced delegations—a mechanism where the server grants a client exclusive (or read) access to a file. While a client holds a delegation, it can cache aggressively without revalidating on every open(), because the server guarantees no other client is modifying the file. If another client tries to access the file, the server issues a callback to recall the delegation, forcing the holder to flush and relinquish its cached state.

Delegations preserve NFSv4's consistency semantics while reducing open/close validation traffic for uncontended files; a conflicting access causes recall and synchronization before the other client proceeds.

# Practical Implications

Common symptoms of CTO surprises

• Stale reads: Application on client B reads old data even though client A has written new data. Usually caused by B not reopening the file, or by the attribute cache serving stale attrs between opens.
• Empty files after rename: The target was replaced before the new file's contents were handed to the server. Complete that handoff before replacement, using operations whose guarantees match the deployed client and server.
• Distributed build failures on NFS: A consumer on another client can observe stale data or names if it accesses output before the writer has completed its close-to-open handoff or while directory-entry caching is stale. Processes on the same Linux NFS client share its page cache.
• "File not found" after create: A stale directory or negative-dentry cache can temporarily hide a new name from ls and can also make pathname lookup return ENOENT.

Tuning and mount options

Before deployment, define the application's correctness and outage requirements, including cache visibility, locking, write stability, and retry or blocking behavior. Select candidate mount behavior from those requirements, then test it with representative workloads and server or network failures. Transport and connection options can affect clients and servers differently, so measured results on the intended path should drive performance choices.

Debugging

When you suspect CTO-related issues:

• nfsstat -c — shows client-side RPC statistics. Look at GETATTR, WRITE, and COMMIT counts. A sudden spike in GETATTRs may indicate cache invalidation storms.
• mountstats — reports per-mount NFS event/byte counts, per-operation RPC statistics, and transport statistics.
• rpcdebug -m nfs -s all — enables implementation-dependent kernel NFS debug messages in the system log; verbose, so clear flags after targeted use.
• strace — shows the application's syscall sequence, but not a one-to-one RPC mapping; correlate it with mount statistics, kernel tracepoints, or packet capture.