Introduction

Backing up your data is essential to prevent loss due to accidental deletions, hardware failures, or cyber threats. S3-compatible object storage, including providers like Amazon S3, DigitalOcean Spaces, Vultr Object Storage, and others, offers a reliable, scalable, and cost-effective backup solution. In this guide, we will walk you through the steps to back up data from an Ubuntu server to an S3-compatible storage bucket efficiently.

Prerequisites

Before proceeding, ensure that you have:

• An Ubuntu server with sudo privileges
• An account with an S3-compatible object storage provider (Amazon S3, DigitalOcean Spaces, Vultr Object Storage, etc.)
• AWS CLI or s3cmd installed and configured
• Sufficient storage space in your bucket

Step 1: Install AWS CLI or s3cmd

To interact with S3-compatible storage from the command line, install either the AWS CLI or s3cmd:

Installing AWS CLI

sudo apt update
sudo apt install awscli -y

Verify the installation:

aws --version

Installing s3cmd

sudo apt install s3cmd -y

Step 2: Configure CLI for Your Storage Provider

Different providers have different endpoints. To configure AWS CLI for a non-AWS provider:

aws configure set aws_access_key_id YOUR_ACCESS_KEY
aws configure set aws_secret_access_key YOUR_SECRET_KEY
aws configure set region YOUR_REGION
aws configure set s3.endpoint-url https://your-storage-provider-url

For s3cmd, configure it by running:

s3cmd --configure

Provide the appropriate API keys, endpoint, and region for your provider.

Step 3: Create a Storage Bucket (If Not Already Created)

To create a new bucket using AWS CLI:

aws s3 mb s3://your-backup-bucket-name

For s3cmd:

s3cmd mb s3://your-backup-bucket-name

Step 4: Backup Files to S3-Compatible Storage

Using AWS CLI

To upload a specific file:

aws s3 cp /path/to/your/file s3://your-backup-bucket-name/

To upload an entire directory:

aws s3 cp /path/to/your/directory s3://your-backup-bucket-name/ --recursive

Using s3cmd

To sync files using s3cmd:

s3cmd sync /path/to/your/directory s3://your-backup-bucket-name/

Step 5: Automate Backups with Shell Scripts

1️⃣ Create backup_s3.sh (WordPress Files Backup)

Run:

nano /home/ubuntu/backup_s3.sh

Add this:

#!/bin/bash

# Define variables
S3_BUCKET="s3://bucket_name/folder/"
WP_CONTENT="/var/www/html/wp-content/"
LOG_FILE="/home/ubuntu/backup_s3.log"

# Run s3cmd sync with optimizations
/usr/bin/s3cmd sync --skip-existing --no-check-md5 "$WP_CONTENT" "$S3_BUCKET" > "$LOG_FILE" 2>&1

Save and exit (CTRL + X, then Y, then ENTER).

Then, make it executable:

chmod +x /home/ubuntu/backup_s3.sh

2️⃣ Create backup_db_s3.sh (Database Backup)

Run:

nano /home/ubuntu/backup_db_s3.sh

Add this:

#!/bin/bash

# Define variables
DB_NAME="db_name"
DB_USER="db_user"
DB_PASSWORD="db_pwd"
BACKUP_PATH="/home/ubuntu/db-backups"
S3_BUCKET="s3://bucket_name/folder_name/"
LOG_FILE="/home/ubuntu/backup_db.log"

# Create backup directory if not exists
mkdir -p "$BACKUP_PATH"

# Dump database
BACKUP_FILE="$BACKUP_PATH/${DB_NAME}_$(date +%F_%H-%M-%S).sql"
mysqldump -u "$DB_USER" -p"$DB_PASSWORD" "$DB_NAME" > "$BACKUP_FILE"

# Upload to S3
/usr/bin/s3cmd put "$BACKUP_FILE" "$S3_BUCKET" > "$LOG_FILE" 2>&1

# Optional: Remove backups older than 7 days
find "$BACKUP_PATH" -type f -name "*.sql" -mtime +7 -exec rm {} \;

Save and exit.

Then, make it executable:

chmod +x /home/ubuntu/backup_db_s3.sh

Step 6: Schedule Automated Backups with Cron Jobs

3️⃣ Verify Cron Job

Check your cron jobs:

crontab -l

It should show:

0 8 * * * /home/ubuntu/backup_s3.sh
0 8 * * * /home/ubuntu/backup_db_s3.sh

4️⃣ Restart Cron Service

To ensure cron runs properly:

sudo systemctl restart cron

5️⃣ Check Logs After Execution

File backup log:

cat /home/ubuntu/backup_s3.log

Database backup log:

cat /home/ubuntu/backup_db.log

Conclusion

Regular backups to S3-compatible object storage ensure data safety, scalability, and accessibility. By using AWS CLI or s3cmd, you can efficiently back up and restore critical files from your Ubuntu server. Automating backups with cron jobs further streamlines the process, reducing manual effort and enhancing data protection.

Need more automation? Consider integrating cloud provider-specific tools like lifecycle policies for efficient storage management.