Created
August 2, 2024 22:29
-
-
Save pingud98/0c7f6b5b424d8ab2ee96e93418ca3335 to your computer and use it in GitHub Desktop.
How I converted (and restored) three different influxdb databases into a single instance
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The situation: I have built some cosmic ray detectors that store the data in influxdb on device, using a Raspberry Pi with SD Card. The SD cards died en route as a result of power cycling. So the expedition crew changed them periodically. Also we got some data in real time via Starlink, but only for a couple of hours a day. | |
The result: I have my online influxdb, which has all events received via starlink. I have three SD cards with varying degrees of 'offline' data, in the format of raw influxdb files (shards, not portable etc.) | |
I was getting failures with the following errors in the influxd status: | |
août 03 00:15:53 beast influxd[4142]: ts=2024-08-02T22:15:53.720580Z lvl=info msg="Failed to open> | |
This is due to incorrect permissions to /var/lib/influx, the fix is below. | |
The solution: | |
1) Copy the files off the SD cards. This requires a linux instance, mounting the cards and then modifying the permissions (sudo chmod -R 777 /var/lib/influx/ and then sudo chown user:user /var/lib/influx/) and copying the files to a safe place (sudo cp -r /var/lib/influx/ ~/Documents/influx_backup_number_etc) | |
2) install influxd on the host machine (sudo apt-get install influxdb) | |
3) copy the files back to the /var/lib/influx directory | |
4) Reset the permissions as here https://docs.influxdata.com/influxdb/v1/introduction/install/ | |
(sudo chmod -R 755 /var/lib/influxdb/ and sudo chmod -R 700 /var/lib/influxdb/wal/ and sudo chown -R influxdb:influxdb /var/lib/influxdb/) | |
5) Restart the local influxd (sudo systemctl restart influxd) | |
6) Create a portable backup in a new safe place (influxd backup -portable ~/Documents/influx_2_portable_bak) | |
7) next step - follow the instructions here to merge everything into a master dataset: https://community.home-assistant.io/t/influxdb-and-merging-multiple-backups/515602 | |
Useful sources: | |
https://community.influxdata.com/t/recovering-data-after-a-disk-failure/3590 | |
https://docs.influxdata.com/influxdb/v1/tools/influx_inspect/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment