Why is this IP being blocked?

I have a user who cannot access the ISPC3 panel login. I checked iptables and this user's IP is not listed.

Code:

iptables -L -n --line | grep "123.45.67.89"                 * returns nothing
iptables-save | grep 123.45.67.89                            * returns nothing

However, I did find this in the server log.

Code:

May  9 20:23:03 myhost kernel: [10176836.363422] Firewall: *TCP_IN Blocked* IN=venet0 OUT= MAC= SRC=123.45.67.89 DST=123.99.88.77 LEN=64 TOS=0x00 PREC=0x00 TTL=48 ID=59477 DF PROTO=TCP SPT=49681 DPT=8080 WINDOW=65535 RES=0x00 SYN URGP=0

The source IP 123.45.67.89 does match the user's IP. Why does it not show up in iptables and how do I unblock it?

 

So this is interesting. Fail2ban is working, and it is configured to log level 3, but the log file is empty. It exists, but it is 0 bytes. I've gone back through every one that exists and they are all empty. Here's fail2ban.conf:

Code:

# Option:  loglevel

# Notes.:  Set the log level output.
#          1 = ERROR
#          2 = WARN
#          3 = INFO
#          4 = DEBUG
# Values:  NUM  Default:  3
#
loglevel = 3

# Option:  logtarget
# Notes.:  Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
#          Only one log target can be specified.
#          If you change logtarget from the default value and you are
#          using logrotate -- also adjust or disable rotation in the
#          corresponding configuration file
#          (e.g. /etc/logrotate.d/fail2ban on Debian systems)
# Values:  STDOUT STDERR SYSLOG file  Default:  /var/log/fail2ban.log
#
logtarget = /var/log/fail2ban.log

Even if it were fail2ban though, wouldn't you still see the jails in the iptables output?

 

Ok, I think I'm on to something here. I noticed my own IP address in the ALLOWIN chain which got me to thinking. I changed my internet connection and I can't access the site now either. The problem isn't that my client IP is being blocked, it's that it's not being whitelisted. It appears to be denying all requests (potentially over port 8080) and allowing whitelisted IP addresses. The problem is, I don't recall ever setting up whitelisting.

Code:

# iptables -L -v -n | grep 123.99.88.77
4327  393K ACCEPT     all  --  !lo    *       123.99.88.77        0.0.0.0/0         
8181 9557K ACCEPT     all  --  *      !lo     0.0.0.0/0            123.99.88.77

# iptables-save | grep 123.99.88.77
-A ALLOWIN -s 123.99.88.77/32 ! -i lo -j ACCEPT
-A ALLOWOUT -d 123.99.88.77/32 ! -o lo -j ACCEPT

At this point, I believe I can whitelist the customer IP and have it work:

Code:

iptables -A ALLOWIN -s 123.45.67.89/32 ! -i lo -j ACCEPT 
iptables -A ALLOWOUT -d 123.45.67.89/32 ! -o lo -j ACCEPT

But that still doesn't suggest where the whitelisting configuration came from in the first place. I'm not seeing any rule that would block all traffic on port 8080 by default. But then again, I'm not seeing a rule that allows it either. I need to go see how strict the opening rules are at the top of the chain.

 

Ok, I think I narrowed down what was happening. Denyhosts was also playing into this problem. But I've got another issue as well. The server is seeing a different external IP than what is expected. Whatismyip.com is showing a different IP than the one that appears in the server logs. Whitelisting the one in the server logs allows access temporarily, but that IP keeps changing even though Whatismyip continues showing the same IP address all along. The user is not connecting through a proxy. How can I determine the IP address that the user is actually using? Or better yet, how can I open the firewall for traffic on port 8080? I've added an iptables rule opening port 8080, but it has no effect. I also added the user's IP to hosts.allow for the httpd service, but that did not have any effect either. I think it's because I'm whitelisting the wrong address.

 

After digging deeper, I realized that port 8080 was not allowed in csf.conf. I've allowed it now and will post back if the issue continues. I'm still curious about the IP not being recognized properly though if anyone can shed any light on that.