Logo

dev-resources.site

for different kinds of informations.

DevOps On A Dime: Using Docker Compose with Cloudflare, Traefik, Nginx, Wordpress, MariaDB, Redis and using AWS S3 for backups

Published at
8/16/2022
Categories
docker
s3
shellscript
devops
Author
mcsmythe
Categories
4 categories in total
docker
open
s3
open
shellscript
open
devops
open
Author
8 person written this
mcsmythe
open
DevOps On A Dime: Using Docker Compose with Cloudflare, Traefik, Nginx, Wordpress, MariaDB, Redis and using AWS S3 for backups

Well one thing is for sure, the cloud and automation have completely changed tech and honestly it’s for the better. Gone are the salad days of obsessively scouring online searching the best deal (cheapest) parts to build a server, then waiting for the parts along the occasional order mix up back and forth. After all the parts arrive the fun really begins. You get to grab the tools and bolt that bad boy together until you get the fingers crossed moment that it powers on. Obviously you didn’t order a distro so you install your own full Linux distro install along with all the necessary repo packages (that you probably wrote down on a piece of paper) to say get a web server up and running. Then comes the fun networking setup if you colo the server or set it up on your LAN. You get the idea!

Everything today is completely different. It is lighting fast, insanely efficient and honestly mind blowing in your ability to scale your code. But there are few gotchas that you need to be aware of and that is greater complexity and cost. At first glance the increased complexity of using tools for automation, containerization and container orchestration might be a bit daunting, but when fully embraced they are a complete game changer in the world of automation. For me the biggest gotcha is cost. For big business it is a no brainer to utilize all the offerings of the cloud but I don't have NASA's budget. I have champagne tastes but definitely on a beer budget.

So maybe instead of trying to scale up a complex Kubernetes cluster for this project I just wanted to keep it lean, mean and as cheap as possible. Armed with some shell scripts, a docker compose file, some cron jobs, an AWS S3 Bucket and 1 cheap VPS I was able to tackle this project and have a nice lil' stack humming along as well.

Well enough of the rambling. Let’s get to it. I was recently approached by a friend looking to build a Wordpress site for her small business. Sure I could have suggested she just use some shared hosting like Blue Host that has a Cpanel and a one click wordpress, but I wanted to help her out while automating the process as much as I could. If you could use any of this info in your projects, feel free to grab what you want.

Provisioning the Ubuntu Server:

The Repo is here if you just want to skip all the rambling and just want to copy and paste the code pull it down there. I don’t want to overwhelm anyone. I just want to highlight some key points that could have saved me quite a bit of time and not completely pull all my hair out trying to troubleshoot some issues.

Spin up a VPS (I'm using Digital Ocean for this project), use ssh-keygen -t rsa to setup up a keypair. Then ssh into your Linux host. You will want to sudo apt update && upgrade and then add a user sudo adduser. Then you will run the setup.sh shell script which will add docker, docker-compose, setup UFW Firewall Rules and Fail2Ban. The script is below:

#!/bin/bash

apt-get update
apt-get upgrade -y
apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y
curl -L "https://github.com/docker/compose/releases/download/1.28.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
apt update
apt install fail2ban -y
systemctl enable fail2ban.service
apt install ufw
ufw allow ssh
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable
reboot

Enter fullscreen mode Exit fullscreen mode

Next you will want to make a copy of your Fail2ban config

sudo cp /etc/fail2ban/jail.{conf,local} and add the following to /etc/fail2ban/jail.local file:

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
ignoreip = 127.0.0.1 
Enter fullscreen mode Exit fullscreen mode

Even with ssh-keys it is advised to disable root login as well as change your ssh port to another port and update those settings in UFW firewall. Update the settings in /etc/ssh/sshd_config:

Port (Insert Port number to ssh in with)
PermitRootLogin no
AllowUsers username
Enter fullscreen mode Exit fullscreen mode

Save and exit the file. Reload your ssh rules. Then you are ready to log into your server on a specific port with your username.

ssh -p PORTNUMBER username@ipaddress

Docker Compose Stack

Create a folder to store the docker-compose file. Fill out your passwords and settings in the .env file to be picked up by the docker-compose file. You will need to input your domain name next to server_name in /conf/nginx-wp/wp.conf. Also input your domain in /conf/traefikdynamic/routers.yml. Here is the docker-compose.yml file I use for my wordpress SEO projects:

---
version: '3.7'
services:
  traefik:
    image: traefik:${TRAEFIKVERSION}
    restart: unless-stopped
    ports:
      - target : 80
        published : 80
        protocol: tcp
        mode : host
      - target : 443
        published : 443
        protocol: tcp
        mode : host
    volumes:
      - ./conf/acme.json:/acme.json
      - ./conf/traefik.yml:/etc/traefik/traefik.yml:ro
      - ./conf/traefikdynamic:/etc/traefik/dynamic:ro
      - ./logs/traefik.log:/etc/traefik/applog.log
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      CF_API_EMAIL: ${CF_API_EMAIL} 
      CF_API_KEY: ${CF_API_KEY}
      CF_DNS_API_TOKEN: ${CF_DNS_API_EMAIL}
      CF_ZONE_API_TOKEN: ${CF_ZONE_API_TOKEN}
      TRAEFIK_PILOT_DASHBOARD: "false"
      TZ: "America/New_York"

  sqlwp:
    image: mariadb:${MARIADBVERSION}
    restart: unless-stopped
    volumes:
      - ./conf/custom-mysql.cnf:/etc/mysql/conf.d/custom-mysql.cnf:ro
      - /etc/localtime:/etc/localtime:ro
      - datasqlwp:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQLROOT}
      MYSQL_USER: ${MYSQLUSER}
      MYSQL_PASSWORD: ${MYSQLPASSWORD}
      MYSQL_DATABASE: ${MYSQLDB}
      TZ: "America/New_York"

  nginxwp:
    image: nginx:${NGINXVERSION}
    restart: unless-stopped
    volumes:
      - ./conf/nginx-wp:/etc/nginx:ro
      - /etc/localtime:/etc/localtime:ro
      - datanginxlogs:/var/log/nginx
      - datawp:/var/www/html
    links:
      - wp

  wp:
    image: wordpress:${WPVERSION}
    restart: unless-stopped
    volumes:
      - ./conf/php.ini:/usr/local/etc/php/conf.d/custom.ini
      - /etc/localtime:/etc/localtime:ro
      - datawp:/var/www/html
    depends_on:
      - sqlwp
      - redis
    environment:
      WORDPRESS_DB_HOST: sqlwp
      WORDPRESS_DB_USER: ${MYSQLUSER}
      WORDPRESS_DB_PASSWORD: ${MYSQLPASSWORD}
      WORDPRESS_DB_NAME: ${MYSQLDB}
      WORDPRESS_TABLE_PREFIX: ${MYSQLTABLEPREFIX}
      WORDPRESS_CONFIG_EXTRA: |
        /* Redis Ojbect Cache */
        define( 'WP_REDIS_HOST', 'redis' );
        define( 'WP_REDIS_PORT', 6379 );
      TZ: "America/New_York"

  redis:
    image: redis:${REDISVERSION}
    restart: unless-stopped
    command: redis-server --maxmemory 1024mb --maxmemory-policy allkeys-lru --requirepass changemeWithALongPassword --appendonly yes --bind redis
    environment:
      TZ: "America/New_York"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - dataredis:/data

volumes:
  datanginxlogs:
  dataredis:
  datasqlwp:
  datawp:

Enter fullscreen mode Exit fullscreen mode

This stack is pretty cool. You have your reverse proxy being handled by Traefik, and webserver running over Nginx. For SSL you can use Letsencypt or Cloudflare. I prefer to use Cloudflare to manage my domains and use it my Docker Compose files in regards to SSL certs. Redis is a great in-memory cache and if you find a nice light, responsive, minimal wordpress theme this stack will do the trick for optimizing SEO loading time.

AWS S3 Backup

The final piece is backing up your images, plugins and daily backups to remote storage. People seem to forget about this one, but being able to automatically back up your data is crucial. Trust me on this! Well I try to use AWS as much as possible (within my budget obviously) and S3 is a great storage option for back ups. First you need to install AWSCLI to your linux host https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

After setting up your aws profile, sign into AWS IAM and create a user that has programmatic access and the proper S3 bucket policy. I use the Wordpress Updraft plugin which makes a backup up my plugins, themes, uploads, and the database. Here is the simple shell script I use for my AWS S3 backups:

#!/bin/bash

tar -zcf /path/to/backups/daily/container-backup-$(date +%Y-%m-%d).tar.gz -C /var/lib/docker/volumes/container/_data/wp-content/ updraft
find path/to/backups/daily/* -mtime +7 -delete

# rsync via aws-cli to remote s3 bucket
aws s3 sync /path/to/backups/daily s3://bucketname
Enter fullscreen mode Exit fullscreen mode

You will need to change the folder path names and the S3 bucket name, but basically this simple shell script copies the updraft volume to a specific path name and dates it with year-month-date convention and holds up to 7 days of backups. It then syncs and pushes the backup file over to an S3 bucket.

Lastly you will need to setup a cron job. This gave me bit of a headache but without going into too much boring details (shout out to the always helpful folks at StackedOverflow) I was able to figure it out. Below is the crontab -e that I use:

SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# m h  dom mon dow   command
33 22 * * * /root/scripts/aws-s3-backup.sh

Enter fullscreen mode Exit fullscreen mode

If you made it this far, then congrats cause that was a lot of stuff! I’m sure I lost some folks interest, but if someone (like me) can use some of this information to build a nice little stack at an affordable price then great. I spent quite a bit of time (along the occasional impulsive thought of chucking my laptop out the window now and again) putting this project together for my friend's website.

Too be honest this project might not necessarily be labeled a true DevOps project, but if your on a limited budget and your working on a small project sometimes the KISS strategy (Keep It Simple Stupid) works alright in a pinch. What used to take (at least to me) almost a lifetime to setup now only takes a few minutes. That kind of automation is pretty awesome.

shellscript Article's
30 articles in total
Favicon
Bash Script Series: Automating Log Analysis with Bash Script or Shell Script
Favicon
Shell script is such a powerful
Favicon
Automate React App Deployment to AWS S3 with a Simple Bash Script
Favicon
Sending Dollar Quotes via a Telegram Bot
Favicon
Script for Easy Git Branch Management
Favicon
Why Is Bash scripting an important tool to learn as a Cloud DevOps engineer?
Favicon
Some notes on Bash Shell Script
Favicon
Taskfile: a modern alternative to Makefile
Favicon
Automation with Bash scripts
Favicon
Mastering Shell Scripting: Your Ultimate Cheat Sheet for Efficient Automation
Favicon
Manage AWS Lambda Environment Variables with CLI Shell Script
Favicon
I am trying to run script from action workflow. How to input value from workflow into script for further computation.
Favicon
How to resize and optimize images with bash script
Favicon
Teste sua conexão com um simples utilitário de linha de comando
Favicon
Consultando ajuda para os comandos no terminal
Favicon
Arte com texto ASC II no terminal comando figlet
Favicon
BASH - What is Shell ____(1)
Favicon
Linux Shell Script set fixed IP
Favicon
DevOps On A Dime: Using Docker Compose with Cloudflare, Traefik, Nginx, Wordpress, MariaDB, Redis and using AWS S3 for backups
Favicon
Building a Web server in Bash, part I - sockets
Favicon
Building a Web server in Bash, part III - Login
Favicon
Building a Web server in Bash, part II - parsing HTTP
Favicon
Getting Started With Shell Scripting(Shell Programming)
Favicon
Automatizar Tarefas
Favicon
Crossposting with a single script: Crossposter.sh
Favicon
Basic of Shell Scripting
Favicon
Linux make install command
Favicon
The easiest way to parse arguments using getoptions for bash and shell scripts
Favicon
NEPTI Script v1.01
Favicon
So, you want to open two files in vim?

Featured ones: