Author Topic: Using Fail2Ban to catch the ones that fell through the .htaccess net!  (Read 3476 times)

0 Members and 1 Guest are viewing this topic.

Offline Admin

  • Administrator
  • Full Member
  • *****
  • Posts: 212
  • Karma: +0/-0
    • View Profile
If you have a good .htaccess file, it will stop most of the bad guys, but every now and then one manages to squeeze through the cracks. DenyHosts will probably help with any SSH attempts, but why not go the whole way and install Fail2ban as well?

Fail2ban has the ability to stop much more than SSH attempts as you'll see in a moment. Make sure you have python installed before you download and make sure that you can restart the apache service without a problem, you'll see why later.

Get the right rpm from http://www.fail2ban.org/wiki/index.php/Downloads and Firefox should install it for you and there's loads of good info on http://www.fail2ban.org/wiki/index.php/HOWTOs as well.

Once installed, all you need to do is configure it and for the novice, here is my configuration which keeps 99.99% of the acne-tidden morons that don't have a life OUT of my system.

The configuration files are in /etc/fail2ban. If you downloaded the right rpm, fail2ban.conf is probably OK. Just the log level which is OK at 3 and the log file and socket.

Now for the important stuff - jail.conf

Code: [Select]
# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
# $Revision: 747 $
#

# The DEFAULT allows a global definition of the options. They can be override
# in each jail afterwards.

[DEFAULT]

# IMPORTANT "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not
# ban a host which matches an address in this list. Several addresses can be
# defined using space separator.
ignoreip = 127.0.0.1 192.168.0.0/24

# "bantime" is the number of seconds that a host is banned. Setting bantime to -1 means forever!
bantime  = -1

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
findtime  = 600

# "maxretry" is the number of failures before a host get banned.
maxretry = 3

# "backend" specifies the backend used to get files modification. Available
# options are "gamin", "polling" and "auto". This option can be overridden in
# each jail too (use "gamin" for a jail and "polling" for another).
#
# gamin:   requires Gamin (a file alteration monitor) to be installed. If Gamin
#          is not installed, Fail2ban will use polling.
# polling: uses a polling algorithm which does not require external libraries.
# auto:    will choose Gamin if available and polling otherwise.
backend = auto


# This jail corresponds to the standard configuration in Fail2ban 0.6.
# The mail-whois action send a notification e-mail with a whois request
# in the body.

[ssh-iptables]
enabled  = true
filter   = sshd
action   = iptables[name=SSH, port=ssh, protocol=tcp]
##           sendmail-whois[name=SSH, dest=root, sender=fail2ban@mail.com]
             sendmail-whois[name=SSH, dest=fred@somewhere.com, sender=fail2ban]
logpath  = /var/log/secure
maxretry = 2
bantime = -1

[sasl-iptables]
enabled  = true
filter   = sasl
backend  = polling
action   = iptables[name=sasl, port=smtp, protocol=tcp]
           sendmail-whois[name=sasl, dest=fred@somewhere.com, sender=fail2ban]
logpath  = /var/log/mail.log
bantime = -1

[ssh-tcpwrapper]
enabled     = true
filter      = sshd
action      = hostsdeny
              sendmail-whois[name=SSH, dest=fred@somewhere.com, sender=fail2ban]
ignoreregex =
logpath     = /var/log/secure.log
bantime = -1

[apache-tcpwrapper]
enabled  = true
filter = apache-auth
action   = hostsdeny
           sendmail-whois[name=APACHE, dest=fred@somewhere.com, sender=fail2ban]
logpath  = /var/log/httpd/*error_log
maxretry = 2
bantime = -1

[vsftpd]
enabled  = true
filter   = vsftpd
action   = iptables[name=VSFTPD, port=ftp, protocol=tcp]
           sendmail-whois[name=VSFTPD, dest=fred@somewhere.com]
logpath  = /var/log/vsftpd.log
maxretry = 2
bantime  = -1

[apache-badbots]
enabled  = true
filter   = apache-badbots
action   = iptables-multiport[name=BadBots, port="http,https"]
           sendmail-buffered[name=BadBots, lines=5, dest=fred@somewhere.com, sender=fail2ban]
logpath  = /var/log/httpd/*access_log
bantime  = -1
maxretry = 1

[webmin-auth]
enabled = true
filter = webmin-auth
action = iptables[name=webmin, port=10000, protocol=tcp]
sendmail-whois[name=WEBMIN, dest=fred@somewhere.com, sender=fail2ban]
logpath = /var/log/secure
bantime = -1
maxretry = 1

[php-url-fopen]
enabled = true
#port = http,https
filter = php-url-fopen
logpath = /var/log/httpd/*access_log
maxretry = 1
action = iptables-multiport[name=PHP-fopen, port="http,https", protocol=tcp]
sendmail-whois[name=php-attack, dest=fred@somewhere.com, sender=fail2ban]

[webserver-w00tw00t]
enabled  = true
port     = http,https
filter   = webserver-w00tw00t
# !!! Keep in mind to specify the correct web server log here:
logpath  = /var/log/httpd/*access_log
action   = iptables-multiport[name=w00tw00t, port="http,https"]
           sendmail-whois[name=w00tw00t, dest=fred@somewhere.com, sender=fail2ban]
maxretry = 1
bantime  = -1

[apache-404-slow]
enabled = true
port = http,https
filter = apache-404-slow
action = iptables[name=WWW, port=http, protocol=tcp]
sendmail-whois[name=www, dest=fred@somewhere.com, sender=fail2ban]
logpath = /var/log/httpd/*access.log
bantime = -1
findtime = 172800
maxretry = 3

[apache-404]
enabled = true
filter = apache-404
action = iptables[name=WWW, port=http, protocol=tcp]
sendmail-whois[name=www, dest=fred@somewhere.com, sender=fail2ban]
logpath = /var/log/httpd/*access_log
maxretry = 3
bantime = -86000

For every entry inside the [ ] square brackets, there must be a filter in /etc/fail2ban/filter.d

Most of these come pre-written with fail2ban, but i've added a few of my own which I found by trawling on Google such as : -

php-url-fopen.conf
Code: [Select]
# Fail2Ban configuration file
#
# Author: Arturo 'Buanzo' Busleiman <buanzo@buanzo.com.ar>
# Version 2
# fixes the failregex so REFERERS that contain =http:// don't get blocked
# (mentioned by "fasuto" (no real email provided... blog comment) in this entry:
# http://blogs.buanzo.com.ar/2009/04/fail2ban-filter-for-php-injection-attacks.html#comment-1489
#

[Definition]

# Option:  failregex
# Notes.:  regex to match this kind of request:
#
# 66.185.212.172 - - [26/Mar/2009:08:44:20 -0500] "GET /index.php?n=http://eatmyfood.hostinginfive.com/pizza.htm? HTTP/1.1" 200 114 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
#
failregex = ^<HOST> -.*"(GET|POST).*\?.*\=http\:\/\/.* HTTP\/.*$

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =

and
webserver-wootwoot.conf
Code: [Select]
[Definition]
failregex = ^<HOST> .*"GET \/w00tw00t\.at\.ISC\.SANS\..+\:\).*?"

ignoreregex =

very much modified

apache-badbots.conf
Code: [Select]
Fail2Ban configuration file
#
# List of bad bots fetched from http://www.user-agents.org
# Generated on Sun Feb 11 01:09:15 EST 2007 by ./badbots.sh
#
# Author: Yaroslav Halchenko
#
# $Revision: 668 $
#

[Definition]

badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider
badbots = atSpider/1\.0|autoemailspider|China Local Browse 2\.6|ContentSmartz|DataCha0s/2\.0|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 +http\://letscrawl\.com/|Lincoln State Web Browser|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|MVAClient|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|sogou spider|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|WebVulnCrawl\.blogspot\.com/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00

# Option:  failregex
# Notes.:  Regexp to catch known spambots and software alike. Please verify
#          that it is your intent to block IPs which were driven by
#          abovementioned bots.
# Values:  TEXT
#
failregex = ^<HOST> -.*"(GET|POST).*HTTP.*"(?:%(badbots)s|%(badbotscustom)s)"$

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =

The important things are : -

Make sure that your jail is looking at the correct logfile especially apache, because some distros use apache2 and others httpd
Make sure that you have the right email address to send alerts to you

Once you have configured your jails and your filters, restart apache. It will probably fail to start!

Troubleshooting if apache fails to start

In /etc/fail2ban/jail.conf set every item to enabled = false.
Now apache will start
Starting from the first item set enabled = true and restart apache. If it starts, go to the next item and set that to enabled = true. If it fails to start, set it back to false and go on to the next one.
When you have done with them all, you can go back to all the ones set to false and start to find out WHY they don't work. You'll probably find that the email address is wrong or the logfile isn't correct or you made a typo.

Once you have them all working and a decent .htaccess file, you should be able to keep the morons out. We also use Smoothwall Express and if one gets banned by fail2ban, the IP is also added to the Smoothwall block list.

Yeah, sure I know that most of these attempts come from poor innocent users <sob> whose computers have been taken over by the botnets, but if users are so stupid or cheap that they don't bother to secure their systems, they shouldn't have a computer in the first place!

If you have any problems, I'm always glad to help and if you have any new filters that you made, please post them here and if you are a script-kiddy or would-be hacker - GET OFF MY SITE AND GET A GIRLFRIEND OR A LIFE!
« Last Edit: May 18, 2011, 08:03:50 AM by Admin »