Your virtual private servers (VPS) is under brute-force attacks by SSH protocol, or bad bots crawling your site and searching locations like admin panels, index.php files, etc? I tried to find solution for protecting projects in WEB. His name - Fail2ban.

Note: this is not completely 100 percent protection, but is better than nothing.

Here some examples of bad bots' headers:

Go-http-client/1.1
python-requests/2.32.3
Python/3.11 aiohttp/3.9.3
Python-urllib/3.8
python-httpx/0.27.0
Ruby
curl/7.61.1
libwww-perl/5.820
lychee/0.11.1

What is Fail2ban? This is software for protecting services, connected to network, like Apache, Nginx, OpenSSH, Postfix, Asterisk, and so on. Fail2ban protect from brute-force attacks, incorrect authentication attempts, bad-bots crawling, etc...

First you need to install Fail2ban. Before installation please see official installation guide on GitHub. Maybe something has been changed after this article published.

How to install in Debian/Ubuntu:

sudo apt update && sudo apt upgrade -y
sudo apt install fail2ban -y

How to install in CentOS/CentOS Stream:

sudo yum update -y
sudo yum install epel-release -y && sudo yum install fail2ban -y

Start and enable in autorun Fail2ban service:

sudo systemctl start fail2ban
sudo systemctl enable fail2ban

Create new /etc/fail2ban/jail.local file and put next configurations:

[DEFAULT]
ignoreip = 127.0.0.1/8 192.168.0.0/24 your_external_address
findtime = 10m
maxretry = 3
bantime = 3600m

Here you may change values as you need.

Configuration for protect OpenSSH service:

[sshd]
enabled = true
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s

Configuration for Nginx authentication protection:

[nginx-http-auth]
enabled  = true
port    = http,https
logpath = %(nginx_error_log)s

For limitation HTTP requests:

[nginx-limit-req]
enabled  = true
port    = http,https
logpath = %(nginx_error_log)s

For protection from badbots copy apache-badbots file's example:

sudo cp /etc/fail2ban/filter.d/apache-badbots.conf /etc/fail2ban/filter.d/nginx-badbots.conf

In file /etc/fail2ban/filter.d/nginx-badbots.conf add to the end of badbotscustom variable heeded user-agents:

badbotscustom = |Custom-AsyncHttpClient|^\-$|Go-http-client\/.*|python-requests\/.*|Python\/.*\ aiohttp\/.*|python-httpx\/.*|python-requests\/.*|Scrapy\/.*|Python-urllib\/.*|curl\/.*|lychee\/.*|\*|

And put to end of the file /etc/fail2ban/jail.local following config lines for activating jail:

[nginx-badbots]
enabled  = true
port    = http,https
logpath = %(nginx_access_log)s
findtime = 10m
maxretry = 1
bantime = 3600m

Save and exit from text editor, restart Fail2ban service for apply changes:

sudo systemctl restart fail2ban

Check how your jails working:

sudo fail2ban-client status sshd
sudo fail2ban-client status nginx-http-auth
sudo fail2ban-client status nginx-limit-req
sudo fail2ban-client status nginx-bad-request
sudo fail2ban-client status nginx-badbots

Are my posts is helpful? You may support me on Patreon.