Mixed-Protocol Permissings

Context

NAS: I run TrueNAS SCALE on a Dell R730 which is running Proxmox has the Host OS with TrueNAS SCALE as a VM with Disks passed through
K8S: I run a 3 node cluster of Minisforum MS-01s running Talos Linux has the OS
PC: The PCs on network are primarily Windows 11, With one MacOS Laptop
Other: Phones and Tables are running iOS

What is NFS?

NFS handles permissions by enforcing access controls based on user and group IDs (UIDs and GIDs), which must match between the client and server for permissions to be applied correctly. Here’s how it works:

UID/GID Mapping NFS relies on the UID and GID of users rather than usernames. When a client accesses a file, the server checks the UID/GID associated with that file and compares it with the UID/GID of the client’s user. For NFS permissions to work seamlessly, the user IDs on both the client and server must match. For example, if a user has UID 1000 on the server, they should also have UID 1000 on the client to access files with appropriate permissions.
File System Permissions NFS uses the traditional UNIX file permissions model (read, write, execute for user, group, and others) to control access. When a client tries to access a file, NFS uses the file’s permissions on the server to determine whether the action is allowed. For example, if a file is readable only by the owner, only users with the matching UID can read it from any NFS client.
Root Squashing For security, NFS uses a feature called "root squashing," which maps the root user (UID 0) on the client to a less-privileged user on the server, typically "nobody" (UID 65534). This prevents a root user on one client from having unrestricted access to the files on the NFS server. Root squashing can be disabled if necessary, but it’s generally recommended to leave it enabled for security.
NFS Export Options When exporting a directory, the server administrator can set options that control client access: rw/ro: Determines whether the client has read-write (rw) or read-only (ro) access. no_root_squash: Disables root squashing, granting root access to the client. all_squash: Maps all client users to a single UID/GID on the server, commonly used for anonymous access. anonuid and anongid: Specify which UID and GID to use for squashed users.
Kerberos for Authentication (NFSv4) With NFSv4, administrators can use Kerberos for stronger authentication and authorization controls. Kerberos enforces security policies more rigorously than basic UID/GID matching, allowing for encryption and mutual authentication, which is particularly valuable in multi-user environments.
File Locking NFS supports file locking (via lockd in older versions or native in NFSv4) to prevent multiple clients from modifying the same file simultaneously. This locking system respects permissions, so only clients with write access can obtain exclusive locks on files.

What is SMB?

SMB (Server Message Block) handles permissions differently from NFS, with a focus on integrating Windows-style permissions and authentication. Here’s how it works:

User Authentication and Mapping SMB is heavily integrated with Windows authentication. Users authenticate with usernames and passwords, and their access is based on their permissions within the Windows domain or local server configuration. In Linux environments, SMB servers (like Samba) can map Windows users to local UNIX users, translating Windows permissions into UNIX-like permissions. However, in an Active Directory (AD) environment, SMB permissions are typically managed centrally by AD.
Share-Level and File-Level Permissions Share-Level Permissions: These control access to the shared resource (the folder being shared over SMB). Administrators can set permissions like Full Control, Change, and Read at the share level, determining what actions users can perform on the shared folder. File-System (NTFS) Permissions: Once users access a share, file-level permissions are enforced based on NTFS permissions, which provide a granular level of control with attributes such as Read, Write, Execute, Modify, and Full Control.
Access Control Lists (ACLs) SMB uses Access Control Lists (ACLs) to manage permissions at a fine-grained level. Each file and folder can have an ACL with multiple entries (ACEs), specifying permissions for individual users or groups. These ACLs allow a high level of customization, where administrators can specify exactly what each user or group is allowed to do (e.g., read, write, delete, or modify a file).
Permission Inheritance NTFS permissions support inheritance, where subfolders and files inherit permissions from their parent folder. This feature simplifies permission management by applying consistent rules across a directory structure, although it can be overridden for specific files or subfolders if needed. This inheritance mechanism is helpful in situations where, for example, a shared folder’s access needs to be consistent across all contained files.
Kerberos and NTLM Authentication SMB typically uses Kerberos or NTLM (NT LAN Manager) for authentication, especially within a domain environment. Kerberos is preferred for its enhanced security and support for mutual authentication. With Kerberos, user credentials are verified by a trusted server, adding an extra layer of security, especially in multi-user, multi-device environments.
File Locking and Opportunistic Locking (OpLocks) SMB supports file locking, which prevents multiple users from editing the same file at once, preserving file integrity. Opportunistic Locking (OpLocks) allows SMB clients to cache file data locally for improved performance, but only if no other clients are accessing the file. This caching mechanism is critical in optimizing performance for frequently accessed files while ensuring data consistency.
Granular Permission Levels Unlike traditional UNIX permissions, SMB allows for a wide range of permission combinations, giving administrators control over read, write, execute, delete, and modify permissions at a highly granular level. These permissions allow for tailored access, providing flexibility in environments where users need different levels of control over shared files and directories.

Challenge

It is generally not recommended to the same files shared in TrueNAS Scale with BOTH NFS and SMB shares. This is due to the complex nature of how each system handles permissions and they often don't play nice with each other.

However, I wanted my containers to be able to access files on the NAS and I also wanted my windows user to be able to access those same files. There are many example of why I wanted this but here are a couple of simple ones

Obsidian - Notes
1. I use Obsidian, the vault is stored on my NAS. I want to use SyncThing to sync this file across many devices and have the .md files editable by any of those devices.
2. Synthing (inside k8s) is used to sync the folder on my NAS that has the obsidian vault across to other devices.
3. The Vault is created and edited on many devices across operating systems
Manyfold - 3D Library
1. I use Manyfold as a library for my 3D model files (STL). It is hosted inside k8s and uses a Library on my NAS
2. I want to drop files onto my NAS from a Network share on my Windows PC

Problem Example - Obsidian

For this example I will use Obsidian
I opened up my network share on my Windows machine for my NAS
Created a new folder called Syncthing with a sub folder called Obsidian
Next, I hope over to my terminal and shell into my Syncthing container and take a look at that folder

/files $ ls -la
total 22
drwxrwx---    3 568      568              3 Nov 12 05:55 .
drwxr-xr-x    1 root     root            52 Nov 12 08:34 ..
drwx------    4 3000     568              4 Nov 12 08:39 obsidian

You can see here that the folder is owned by UID 3000 (That is the SMB User ID for my Windows user in TrueNAS) and that it has the 568 (apps) Group
This is becuase I use 568 for all my k8s PUID/PGID as thats the default used by Onedr0p in his containers of which I use many
Attempting to go into that folder from inside the container results in a permissions error.
The containers PUID/GUID is 568/568 but as you can see from drwx------ groups do not have permission to read or write. Only owner (which is user 3000)
drwx------
1. d = directory
2. the next 9 letters correspond to permissions for 3 sets
  1. Owner (or user)
  2. Group
  3. Other (or world)
3. In this example you can see that there are permissions for Onwer (first 3 letters) and not for Group or Other
4. Looking at the Owners Permissions, we can convert this into a more commonly read numberical version
  1. r = read (4)
  2. w = write (2)
  3. x = execute (1)
  4. Total = 7
5. With that understood, we can read the permissions as 700
  1. Full permissions for Owner, no permissions for Group or Other
What we need here is 770
1. Read, Write and Execute for BOTH User and Group
But when we are creating the files from windows, the files/directories are being set to 700, not 770. How do we fix this?
In TrueNAS you need to make sure you have your permissions set correctly.
Caveat, I am not a file permissions expert and there may be things wrong about how I have mine set. Please leave a comment if you see an issue.
TrueNAS Dataset with both SMB and NFS Shares
In my NFS Share I have Advanced options set
1. Maproot User = apps (apps is user id 568 in TrueNAS which matches my containers)
2. Maproot Group = apps (apps is group 568 in TrueNAS which matches my containers)
3. Security = SYS
4. Network = My LANs CIDR
Back in the Dataset, Editing permissions and they look like this. Take note of the Flags for group@ - apps
Setting the following flags on the @apps group
1. File Inherit
  1. This flag ensures that any new files created within a directory will inherit the ACL entries of the parent directory.
  2. This is useful when you want all files within a directory to have consistent permissions without manually setting them for each file.
2. Directory Inherit
  1. Similar to File Inherit, but specifically for directories.
  2. This flag ensures that new subdirectories created within the current directory inherit the same ACLs as the parent directory.
  3. This helps maintain consistent permissions across nested directories.
3. Inherit Only
  1. When set, this flag indicates that the ACL entries are only for inheritance purposes and do not affect the permissions of the current directory or file.
  2. This is helpful for setting up permissions that will only apply to children of this directory, not to the directory itself.
Because the Dataset is owned by apps:apps (568:568) it means that files and directories created by other users will inherit the same permissions. This means that we move from our drwx------ or 0700 permission to drwxrwx--- or 0770
This resolves our persmissions issue. The Windows user and the linux container both have persmissions to Read/Write and Execute files/directories in that dataset