System Automation

Tutorial Javascript HTML Beginners

So I have been writing about streaming our workflow, but this does not have to be about coding. How about our productivity on our systems? In this article, I focus on scheduling jobs on Unix-based systems, mostly for securing data.

Schedule Jobs with Cron

What in the world is Cron? Well: "...cron is a time-based job scheduler in Unix-like computer operating systems. Users who set up and maintain software environments use cron to schedule jobs to run periodically at fixed times, dates, or intervals."

A streamlined workflow may include many scheduled jobs to manage a user system. I am going to touch on three jobs I have on my system:

  • Backups
  • Empty Trashcan
  • Back up my cron jobs.

So, we use the default editor for cron, to edit schedules: crontab -e. If you are interested, I suggest you read this tutorial on the scheduling syntax.

Backups

I am pathological about daily backups. I was a small business owner for twenty years, and data is a product of work and therefore an investment. So, lost data represents lost money.

My System: I have a Linux laptop with a separate drive for my home directory. On a Linux system, this includes all your personal files and your configurations of most programs or packages you install. If you reinstall the operating system, it's easy because all your files are separate from the operating system. I have a second mass storage drive where I keep large files: videos, downloads, VMs.

So, I back up my home drive every evening at 11:30 pm. We open crontab -e and add the following:

# Daily System backup using RSYNC
30 23 * * * rsync --archive --delete --progress --human-readable --exclude-from '~/bin/rsync-exclude-list.txt' ~/ /media/username/Backup/home-backup | tee ~/rsync.log

I back up my storage drive on Saturday evenings:

# Weekly backup of Storage Drive
30 20 * * 6 rsync --archive --delete --progress --human-readable /media/storage /media/webrev/Backup/storage-drive | tee ~rsync-storage.log

Notice the 6 at the beginning of the line? Saturday is represented as a 6.

Trashcan

What about the trashcan, also known as the recycle bin? Do you ever remember to empty it? I have created a cron job so I do not have to worry about it.

trash-cli: There is a nice little package called Trash-cli I have installed. It works on most Linux systems. Install on Ubuntu with sudo apt install trash-cli. Refer to the repo for more details.

Add to our cron jobs to run on the first day of the month:

# Empty trash monthly - first day of the month 12AM
0 0 1 * * /usr/bin/trash-empty

Backup Crontab

We are running a backup of the home directory, but our cron configuration is in the operating system partition, and if we reinstall the OS, our configuration will be lost. The command crontab -l will output the crontab configuration. We can easily automate this task and save it to our home directory, which is backed up separately:

# Backup my crontab file - second day of the month 12AM
0 0 2 * * /usr/bin/crontab -l > ~my_crontab.backup

What other automation jobs are you running? Send me a message and let me know.