NFS Stale Handle - ReadyNAS

nfs_rnas_stale_handle_writeup.md

NFS "Stale File Handle" That Survives exportfs -ra and an NFS Service Restart (ReadyNAS OS 6.10.10)

TL;DR

One specific subdirectory under an NFSv3 export started throwing Stale file handle on the client, while every sibling export and every other mount on the same box kept working fine. Neither exportfs -ra nor toggling NFS off/on in the ReadyNAS admin panel cleared it. What finally fixed it: giving the data a brand-new directory inode on the server by creating a new folder, moving the contents into it, and mounting the new path instead of the old one. A rename-in-place almost certainly would not have worked, since a rename typically keeps the same inode.

Environment

• Server: Netgear ReadyNAS 2100, ReadyNAS OS 6.10.10 (NFSv3 export only - no NFSv4 support on this firmware)
• Client: Linux Mint desktop, NFSv3 mount via /etc/fstab, mount options rw,bg,nosuid,vers=3
• Export in /etc/exports:

  "/data/<share>" *(insecure,insecure_locks,no_subtree_check,crossmnt,anonuid=1000,anongid=1000,all_squash,rw,async) <client-list>(...,no_root_squash,crossmnt,async)
  

• Mounted subpath: <nas-ip>:/data/<share>/Downloads → /home/<user>/Downloads

Symptom

• ssh into the NAS and ls the directory: contents all present, totally normal.
• ReadyNAS web admin file browser: contents all present, totally normal.
• sudo mount -av on the client: claims /home/<user>/Downloads : successfully mounted.
• File manager (Nemo) on the client: mount point is empty, reports not mounted.
• Running mount -av again: byte-for-byte identical output, including the "successfully mounted" line.

Verbose mount showed the real error:

mount.nfs: trying text-based options 'addr=<nas-ip>'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying <nas-ip> prog 100003 vers 3 prot TCP port 2049
mount.nfs: mount(2): Stale file handle
mount.nfs: Protocol not supported for <nas-ip>:/data/<share>/Downloads on /home/<user>/Downloads

Two things worth flagging for anyone who finds this later:

1. "successfully mounted" was a lie - or rather, misleading. With the bg mount option, if the foreground mount attempt fails, mount.nfs forks a background process to keep retrying and the parent returns success immediately. mount -a reports success based on the backgrounded process launching cleanly, not on the mount actually being up. If the underlying cause never resolves, the backgrounded retries fail silently forever and the mount point just sits empty.

2. The final summary line ("Protocol not supported") was a red herring. The client always probes NFSv4.2 → 4.1 → 4.0 first (all correctly refused since this firmware only serves v3), then falls back and negotiates v3, where the real Stale file handle error shows up. But the top-level summary error printed at the end of the attempt gets inherited from the first failed version negotiation rather than the actual terminal error. The real cause is always the Stale file handle line earlier in the trace, not whatever the last line says.

Why only one subdirectory, and why SSH/GUI showed nothing wrong

NFS file handles are opaque tokens the server hands out (roughly: filesystem ID + inode + generation number), and the client kernel caches them per mount point. SSH and the web GUI both do fresh path lookups on every access - they never touch a cached NFS handle, so neither had any way to notice that the client's cached handle for that one directory no longer matched a valid object on the server. Every sibling export mounted fine because their handles were untouched; only the one directory whose underlying inode had apparently changed was affected.

What didn't fix it

• exportfs -ra on the NAS (refreshes the export table, but not whatever was cached at the nfsd level for that specific handle)
• Toggling NFS off and back on for the whole share in the ReadyNAS admin panel
• umount -f -l on the client before retrying

Both of the NAS-side attempts still hit the exact same Stale file handle error on retry. That ruled out a simple stale export table entry - this was something the server was holding onto at a lower level for that specific directory's handle, and neither a re-export nor a service bounce touched it.

The fix

1. Toggled NFS off in the ReadyNAS admin panel.
2. Created a new folder with a different name (Download, singular) in the same parent, via the web admin panel - a genuinely new directory object, not a rename.
3. Moved the old folder's contents into the new folder (also via the web panel).
4. Toggled NFS back on.
5. Mounted the new path (.../Download instead of .../Downloads) from the client.
6. Confirmed via mount -a -vvv that it came up as already mounted with no errors.

Why this worked

A brand-new directory gets a brand-new inode. A brand-new inode means nfsd has to issue a brand-new file handle the first time any client asks for it - there's no old, poisoned handle left for it to collide with. Moving the contents into a new directory (rather than renaming the old directory) was the important part: a rename typically preserves the original inode, which would likely have carried the stale handle right along with it.

Takeaways

• If Stale file handle survives both exportfs -ra and a full NFS service toggle, stop trying export-table-level fixes - the problem is likely a handle cached against a specific inode, not the export configuration.
• Full reboot of the server (or client) probably clears this too, since it flushes all kernel-level RPC/NFS state - but that's not always an option (e.g. active rsync jobs mid-transfer).
• Moving data into a freshly created directory and re-pointing the mount is a viable non-destructive workaround that avoids downtime.
• A rename-in-place is not equivalent to this fix and likely won't help, since it doesn't necessarily change the inode.