DirectPV discovery and init

DirectPV discovery and init.md

Challenges

1. Get drives from each node and identify drives for usability. DirectPV uses sysfs (/sys), udev (/run/udev/data) and procfs (/proc) on each node to collect all necessary information for this and they are stored in directpvnodes CRD.

2. Uniquely identify each drive not by drive name as it may change on node reboots/drive swaps/removal. DirectPV computes a hash using values collected for each drive to make sure it is unique across all nodes. This hash is called ID which is also stored in directpvnodes CRD.

3. Add/Remove of nodes and drives. DirectPV does not monitor addition or deletion of nodes and drives instead it collects information on demand to update directpvnodes CRD.

Discovery

When discover API or command is called, below steps are performed.

1. It refreshes directpvnodes CRD to accomodate node and drive changes. A controller on each node monitors this request and refreshes respective content in directpvnodes CRD. https://github.com/minio/directpv/blob/master/pkg/admin/refresh.go#L69 line of code triggers refresh request for the node.
2. We wait for refresh on each node completion by watching directpvnodes CRD. This is done at https://github.com/minio/directpv/blob/master/cmd/kubectl-directpv/discover.go#L252
3. Upon step(2) completion, it generates InitConfig data saved to passed or default drives.yaml file.

Init

When init API or command is called, below steps are performed.

1. DirectPV creates a directpvinitrequests CRD for the given InitConfig YAML and it returns RequestID to monitor the completion. init command waits for the RequestID for the completion whereas init API is not.

Drive states

As mentioned in the Challenges, the drive name/state may change in between discovery and init. init fails in that case. init never touches wrong drive.

Feel free to ask any questions.