Differences

This shows you the differences between two versions of the page.

--- backing_up_your_data [2021/11/17 09:47] – [Crontab on Linux] atolstov
+++ backing_up_your_data [2023/05/02 10:43] (current) – atolstov
@@ Line 3: / Line 3: @@
-{{ ::data-backup.png?nolink&60|}}
+{{ network:data-backup.png?nolink&60|}}\\
+\\
+\\
 <WRAP center round important 80%>
-Though you might use industrial storage solutions for the WebHMI, still __there is always a risk of loosing your data__ in some cases:
+Though you might use industrial storage solutions for the WebHMI, still __there is always a risk of losing your data__ in some cases:
-  *Just forget to make a copy of the project - this is most common case.  Backup files are created on local WebHMI storage for your convinience, but it's up to you to save the latest valuable changes on your PC or other storage.
+  *Just forget to make a copy of the project - this is the most common case.  Backup files are created on local WebHMI storage for your convenience, but it's up to you to save the latest valuable changes on your PC or other storage.
-  *Occasionnally clear all collected data, e.g. when restoring your backup with unchecked "keep data" option
+  *Occasionally clear all collected data, e.g. when restoring your backup with unchecked "keep data" option
   *Flash drive wearing out
-  *Abrupt power off during critical file operation (which you can not control) - on virtual machine or WebHMI with failed backup battery
+  *Abrupt power off during critical file operation (which you can not control) - on a virtual machine or WebHMI with a failed backup battery
-  *Filesystem/DB operations failure, which still have a minor chances to happen. That's why RAID technologies exist.
+  *Filesystem/DB operations failure, which still has a minor chance to happen. That's why RAID technologies exist.
 </WRAP>
+There is only one example of how to **avoid the consequences of data loss** considered here. Using shell script via crontab utility.
-There is only one example how to **avoid the consequences of data loss**.
-===== Crontab on Linux =====
-Here is an example of how to setup your Linux machine to automatically save files from the ''/log'' folder on the WebHMI's storage:
+===== Download all present logs once =====
+At the very start, it is a good idea to download all the data.
+<code bash backup-ALL.sh>
+#!/bin/bash
+# Create a folder ~/webhmi-log-cron, with the /tmp sub-folder.
+HOST=192.168.1.1
+LOG_DIR="$HOME/webhmi-logs-backup"
+mkdir -p $LOG_DIR/tmp
+cd $LOG_DIR/tmp
+# Get all logs (including today, which is, however, obviously partial)
+wget --user=admin --password=webhmi -nH --cut-dirs=1 "ftp://$HOST/log/log*.sqlite3"
+# Archive to save storage space
+gzip -f -v *
+# Copy out of tmp .gz file and do not overwrite if destination filename already exists
+cp -r --backup=t log-*.sqlite3.gz ../.
+# Clear tmp folder
+rm $LOG_DIR/tmp/*
+</code>
+===== Crontab on Linux =====
+Here is an example of how to set up your Linux machine to automatically save files from the ''/log'' folder on the WebHMI's storage:
 <code bash backup.sh>
 #!/bin/bash
-# Create a folder ~/WEBHMI/log_backup, with the /tmp sub-folder.
+# Create a folder ~/webhmi-logs-backup, with the /tmp sub-folder.
-LOG_DIR="$HOME/WEBHMI/log_backup"
+HOST=192.168.1.1
-# download the yesterday's log file to backup storage
+LOG_DIR="$HOME/webhmi-logs-backup"
+cd $LOG_DIR/tmp
+# download yesterday's log file to backup storage
 yesterday=$(date +"%Y-%m-%d" -d yesterday)
-cd $LOG_DIR
-# because login and pass for the ftp connection are unencrypted you can
+# because login and pass for the FTP connection are unencrypted you can
 # restrict read/write access mode to the file with chmod command)
-wget --user=admin --password='webhmi' ftp://192.168.1.1/log/log-$yesterday.sqlite3
+wget --user=admin --password='webhmi' ftp://$HOST/log/log-$yesterday.sqlite3
 #archive to save storage space
-gzip -v log-$yesterday.sqlite3
+gzip -f -v log-$yesterday.sqlite3
+# Copy out of tmp .gz file and do not overwrite if destination filename already exists
+cp -r --backup=t log-$yesterday.sqlite3.gz ../.
+# Clear tmp folder
+rm $LOG_DIR/tmp/*
 </code>
 <WRAP center round tip 60%>
-Do not forget grant additional run permissions
+Do not forget to grant additional run permissions
 </WRAP>
-  chmod o+x /path_to_run/backup.sh
+  chmod u+x /path_to_run/backup.sh