Fail2ban - How to detect the client IP in Apache Logs

Question

As much as I like Fail2ban's concept, I'm giving up on it because it's too difficult to configure filters.

I'm looking to create an Apache-404 filter simply to detect IPs causing excessive 404 errors while trying to hit random pages.

For exmaple - I have the following different logs formats, how can I detect the IP:

sub.domain.com:443 145.86.60.76 - - [12/May/2022:08:35:00 +0300] "GET /folder/filepath/styles.js?t=KA9B HTTP/1.1" 404 2212 "https://sub.domain.com/path" "Mozilla/5.0 (iPhone; CPU iPhone OS 15_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko)"

99.39.28.218 - - [12/May/2022:02:39:33 +0000] "GET /js/amcharts/amcharts.js HTTP/1.1" 404 64258 "https://sub.domain.com/path" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36"

Is there an easy way to build and test regex in Fail2ban? The fail2ban-regex command is not informative. It says when a line is matched but doesn't say which portion is actually matching the . I ran into cases where Fail2ban jailed its own domain IP.

I admit I'm not that good with regex and it's a bit difficult for me to get things to work.

This is my last resort before I move on.

Answer 1

you can give a try to crowdsec. There is already an apache2 parser and scenario that detect this behavior.

You can easily install it using the documentation. It will automatically detect apache2 and install for you the apache2 collections which contains the wanted scenario you are looking for.

And you can benefit from the power of the community by getting others IPs that already attacks on the same services you are running (based on the scenarios you installed using crowdsec).

There is also a discord community where you can ask questions if you are stuck somewhere.