Recovering data from an SD card

The node normally collects all data from the ethoscopes. Read the data straight from an ethoscope’s card when:

• the ethoscope no longer works;
• the backup failed, or the data were never copied to the node;
• you want to archive a card before writing a new image to it;
• you want large video files quickly, without copying them over Wi-Fi.

Taking the card out

1. Shut the ethoscope down from the node.
2. Unplug its power and take out the SD card.
3. Insert the card in a Linux computer (the node will do). It is usually mounted automatically; find where with lsblk or ls /run/media/$USER/.

The card has a small boot partition (FAT32, about 128 MB) and a larger root partition, which holds the data. Older Arch Linux images had three partitions, of which the data partition was the largest (about 20 GB).

Results and videos (SQLite images)

On ethoscopes that save results as SQLite (the default since late 2025), copy the files directly from the root partition:

• results: /ethoscope_data/results/<machine_id>/<ETHOSCOPE_NNN>/<date_time>/
• videos: /ethoscope_data/videos/<machine_id>/<ETHOSCOPE_NNN>/<date_time>/

Older images kept the videos under /ethoscope_data/results too.

To hand the files to the node, copy each folder to the same place in the node’s /ethoscope_data/results or /ethoscope_data/videos. For example, a video recorded on 1 December 2021 by ethoscope 175 is in a folder such as 175f0ab9fc5f4572859da127fe59ed4b/ETHOSCOPE_175/2021-12-01_09-43-36, which goes inside /ethoscope_data/videos/175f0ab9fc5f4572859da127fe59ed4b/ETHOSCOPE_175/ on the node.

Videos are recorded as 10-minute chunks with the extension .h264, which most players cannot open. The node can merge them into MP4 files: see Videos.

MariaDB databases (older images)

Ethoscopes running older images kept their results in a MariaDB database on the card, in /var/lib/mysql/. The script recover_mariadb_from_sdcard.sh, in accessories/databases of the repository, starts MariaDB in Docker on those files, lets it recover them, exports the data and converts them into a standard ethoscope SQLite file.

You need a Linux computer with Docker running (docker info; start it with sudo systemctl start docker), a card reader, and sqlite3.

1. Take the card out and mount it, as above. Check that you have the root partition: it contains var/lib/mysql/ with folders such as ETHOSCOPE_XXX_db.

   ls /run/media/$USER/<partition-id>/var/lib/mysql/

2. Run the script with sudo (it needs Docker and access to the MySQL folder):

   sudo ./recover_mariadb_from_sdcard.sh /run/media/$USER/<partition-id>

3. Wait. The script detects the MariaDB version, starts the container, lets MariaDB recover the files (one to three minutes), exports the database with mysqldump, converts it to SQLite and saves the file. Large databases (2–3 GB) take five to ten minutes.

4. Check the result. The script prints the output file and the number of tables:

   sqlite3 /ethoscope_data/results/.../your_database.db "SELECT name FROM sqlite_master WHERE type='table';"
   sqlite3 /ethoscope_data/results/.../your_database.db "SELECT * FROM METADATA;"

The file is saved where the node keeps results, /ethoscope_data/results/<machine_id>/<machine_name>/<date_time>/<date_time>_<machine_id>.db. To save it elsewhere, give the folder as a second argument:

sudo ./recover_mariadb_from_sdcard.sh /run/media/$USER/<partition-id> /custom/output/path

If something goes wrong

• “No ethoscope database found”: you pointed to the boot partition. Use the root partition, which contains var/lib/mysql/.

• “Container stopped unexpectedly” or a version mismatch: the script detects the MariaDB version, but you can force it:

  # older ethoscopes (before 2024)
  MARIADB_IMAGE=mariadb:10.6 sudo ./recover_mariadb_from_sdcard.sh /run/media/$USER/<partition-id>
  # newer ethoscopes
  MARIADB_IMAGE=mariadb:10.11 sudo ./recover_mariadb_from_sdcard.sh /run/media/$USER/<partition-id>

• “Docker permission denied”: run the script with sudo, or add yourself to the docker group (sudo usermod -aG docker $USER, then log out and in again).

• Timeout during recovery: large databases may need longer than the three minutes allowed. Run the script again; the second attempt is often faster.

• Several cards with the same UUID: cards written from the same image can share a filesystem UUID and mount at the same path. Process one card at a time.

• Other problems: read the container log with docker logs ethoscope_recovery_db, check that the card is mounted and readable and that there is enough disk space, then report the issue.

The same folder holds mariadb2sqlite.sh, which converts the MariaDB database on a running ethoscope, and sql2sqlite.sh, which converts mysqldump output to SQLite and is used by the recovery script.