Skip to content

Instantly share code, notes, and snippets.

@ihcsim
Last active January 24, 2025 22:10
Show Gist options
  • Save ihcsim/0dd606facf8553785ba87f39589ba7db to your computer and use it in GitHub Desktop.
Save ihcsim/0dd606facf8553785ba87f39589ba7db to your computer and use it in GitHub Desktop.

Reset RKE2 on Harvester

The rke2 server --cluster-reset command can be used to reset etcd to a single-node cluster, with its data intact.

See https://docs.rke2.io/datastore/backup_restore

Stop the RKE2 server:

sudo systemctl stop rke2-server

Take a snapshot of etcd:

 sudo /opt/rke2/bin/rke2 etcd-snapshot save --name <snapshot-name>

Run rke2 server --cluster-reset:

$ sudo /opt/rke2/bin/rke2 server --cluster-reset
# ...
INFO[0050] Waiting for etcd server to become available  
{"level":"warn","ts":"2025-01-24T21:45:13.859426Z","logger":"etcd-client","caller":"[email protected]/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc001258b40/127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
WARN[0050] Failed to get apiserver address from etcd: context deadline exceeded 
INFO[0055] Defragmenting etcd database                  
INFO[0055] etcd data store connection OK                
INFO[0055] ETCD server is now running                   
INFO[0055] rke2 is up and running                       
INFO[0055] Waiting for API server to become available   
INFO[0055] Saving cluster bootstrap data to datastore   
WARN[0055] Bootstrap key already exists                 
INFO[0055] Defragmenting etcd database                  
INFO[0055] Reconciling bootstrap data between datastore and disk 
INFO[0055] Cluster reset: backing up certificates directory to /var/lib/rancher/rke2/server/tls-1737755119 
WARN[0055] Updating bootstrap data on disk from datastore 
INFO[0055] certificate CN=etcd-peer signed by CN=etcd-peer-ca@1737751587: notBefore=2025-01-24 20:46:27 +0000 UTC notAfter=2026-01-24 21:45:19 +0000 UTC 
INFO[0055] certificate CN=etcd-server signed by CN=etcd-server-ca@1737751587: notBefore=2025-01-24 20:46:27 +0000 UTC notAfter=2026-01-24 21:45:19 +0000 UTC 
INFO[0055] Shutting down kubelet and etcd               
ERRO[0055] Kubelet exited: signal: killed               
INFO[0060] Pod for etcd not synced (pod sandbox not found), retrying 
INFO[0060] Managed etcd cluster membership has been reset, restart without --cluster-reset flag now. Backup and delete ${datadir}/server/db on each peer etcd server and rejoin the nodes 

Restart RKE2 server:

sudo systemctl start rke2-server
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment