1. Introduction §

If you are an OpenBSD running an OpenSMTP email server, you may want to ban IPs used by bots trying to bruteforce logins. OpenBSD doesn't have fail2ban available in packages, and sshguard isn't extensible enough to support the multiline log format used by OpenSMTP.

Here is a short script that looks for authentication failures in /var/mail/maillog and will add the IPs into the PF table bot after too many failed login.

2. Setup §

2.1. PF §

Add this rule to your PF configuration:

block in quick on egress from <bot> to any

This will block any connection from banned IPs, on all ports, not only smtp. I see no reason to allow them to try other doors.

2.2. Script §

Write the following content in an executable file, this could be /usr/local/bin/ban_smtpd but this doesn't really matter.

#!/bin/sh

TRIES=10
EXPIRE_DAYS=5

awk -v tries="$TRIES" '
	/ smtp connected / {
    		ips[$6]=substr($9, 9)
	}

	/ smtp authentication / && /result=permfail/ {
    		seen[ips[$6]]++
	}

	END {
    		for(ip in seen) {
        		if(seen[ip] > tries) {
            			print ip
    			}
		}
	}' /var/log/maillog | xargs pfctl -T add -t bot

# if the file exists, remove IPs listed there
if [ -f /etc/mail/ignore.txt ]
then
    cat /etc/mail/ignore.txt | xargs pfctl -T delete -t bot
fi

# remove IPs from the table after $EXPIRE_DAYS days
pfctl -t bot -T expire "$(( 60 * 60 * 24 * $EXPIRE_DAYS ))"

This parses the maillog file, so by default it has a rotation every day, you could adapt the script to your log rotation policy to match what you want, users failing with permfail are banned after some tries, configurable with $TRIES.

I added support for an ignore list, to avoid blocking yourself out, just add IP addresses in /etc/mail/ignore.txt.

Finally, banned IPs are unbanned after 5 days, you can change it using the variable EXPIRE_DAYS.

2.3. Cronjob §

Now, edit root's crontab, you want to run this script at least every hour, and get a log if it fails.

~ * * * * -sn /usr/local/bin/ban_smtpd

This cron job will run every hour at a random minute (defined each time crond restarts, so it stays consistent for a while). The periodicity may depend on the number of scan your email server receives and also the log size vs the CPU power.

3. Conclusion §

This would be better to have an integrated banning system supporting multiple logfiles / daemons, such as fail2ban, but in the current state it's not possible. This script is simple, fast, extensible and does the job.