Since install of ISPConfig 3 out-bound traffic fails (eventually)

I like ISPConfig 3, it does the things I need. However, I have a continuous issue where any 'out-going' server created http calls / requests from the web server are eventually blocked (fire-walled?). This occurs within a time frame from about 10 minutes to up to half a day - then all out-going HTTP queries (outbound) like; [apt-get update], [aptitude update], [XML Sitemaps], PHP5-CURL calls to external sites just start to FAIL with ERROR 404. The time is never constant - it is typically about an hour.

I used the "The Perfect Server - Ubuntu 8.10 [ISPConfig 3]" template for both install, in addition to this I added the optional DNSMASQ application. All events are correct and smooth - except fot the 'time-delayed' blocking of Server initiated HTTP calls to other (external) sites.

I have found the only solution is to re-boot the server (with ISPConfig 3) and then all works well - for a variable time (nearly always less than half a day)? The other event that I noticed is that by using the recent Ubuntu Jaunty Jackalope (Ubuntu 9.04) release, that the time for the problem to occur is typically much less than with older Ubuntu Intrepid 8.10.

I have tried two clean installs; One with Ubuntu 8.10 + ISPConfig 3, and the other Ubuntu 9.04 + ISP Config 3. A total clean install (disk format and clean software install) was applied in both cases.

Outside queries coming in to the server are never a problem, just any calls (http) made from within the server (to other sites). I have tried this with both the ISPConfig 3 'Firewall' both enabled and disabled.

I have checked the Ubuntu error logs, the iptables, and my router; as yet nothing appears to be causing the [time-based] out-going HTTP request BLOCK! I am guessing' it is likely related to a internal ISPConfig 3 CRON event - or a problem with either my router (the logs do not show this), or a issue with using VMWare Workstation 6.5 in Bridged Mode (connected directly to the physical network)? Incoming traffic request's never present a problem.

I have included several screen dumps of know events:

root@server1:/home/administrator# aptitude update
Err http://security.ubuntu.com jaunty-security Release.gpg
Could not resolve 'security.ubuntu.com'
Err http://security.ubuntu.com jaunty-security/main Translation-en_AU
Could not resolve 'security.ubuntu.com'
Err http://security.ubuntu.com jaunty-security/restricted Translation-en_AU
Could not resolve 'security.ubuntu.com'
Err http://security.ubuntu.com jaunty-security/universe Translation-en_AU
Could not resolve 'security.ubuntu.com'
Err http://security.ubuntu.com jaunty-security/multiverse Translation-en_AU
Could not resolve 'security.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty Release.gpg
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty/main Translation-en_AU
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty/restricted Translation-en_AU
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty/universe Translation-en_AU
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty/multiverse Translation-en_AU
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty-updates Release.gpg
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty-updates/main Translation-en_AU
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty-updates/restricted Translation-en_AU
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty-updates/universe Translation-en_AU
Could not resolve 'au.archive.ubuntu.com'
Err http://au.archive.ubuntu.com jaunty-updates/multiverse Translation-en_AU
Could not resolve 'au.archive.ubuntu.com'
Reading package lists... Done

root@server1:/home/administrator# netstat -tap
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 localhost.localdo:10024 *:* LISTEN 2365/amavisd (maste
tcp 0 0 localhost.localdo:10025 *:* LISTEN 3362/master
tcp 0 0 *:mysql *:* LISTEN 2450/mysqld
tcp 0 0 localhost.localdo:spamd *:* LISTEN 2525/spamd.pid
tcp 0 0 *:http-alt *:* LISTEN 3523/apache2
tcp 0 0 *:www *:* LISTEN 3523/apache2
tcp 0 0 server1.peterbowey:2002 *:* LISTEN 2339/sshd
tcp 0 0 *:ftp *:* LISTEN 3376/pure-ftpd (SER
tcp 0 0 localhost.locald:domain *:* LISTEN 2321/dnsmasq
tcp 0 0 *:smtp *:* LISTEN 3362/master
tcp 0 0 *:https *:* LISTEN 3523/apache2
tcp 62 0 localhost.localdo:35001 localhost.localdo:10025 CLOSE_WAIT 2476/amavisd (ch1-a
tcp 0 0 localhost.localdo:mysql localhost.localdo:40203 ESTABLISHED 2450/mysqld
tcp 0 0 localhost.localdo:40203 localhost.localdo:mysql ESTABLISHED 2476/amavisd (ch1-a
tcp 0 148 server1.peterbowey:2002 192.168.0.3:2935 ESTABLISHED 22438/sshd: adminis
tcp 62 0 localhost.localdo:34993 localhost.localdo:10025 CLOSE_WAIT 2472/amavisd (ch1-a
tcp 0 0 localhost.localdo:mysql localhost.localdo:40195 ESTABLISHED 2450/mysqld
tcp 0 0 localhost.localdo:40195 localhost.localdo:mysql ESTABLISHED 2472/amavisd (ch1-a
tcp6 0 0 [::]:imaps [::]:* LISTEN 3246/couriertcpd
tcp6 0 0 [::]op3s [::]:* LISTEN 3284/couriertcpd
tcp6 0 0 [::]op3 [::]:* LISTEN 3262/couriertcpd
tcp6 0 0 [::]:imap2 [::]:* LISTEN 3224/couriertcpd
tcp6 0 0 [::]:ftp [::]:* LISTEN 3376/pure-ftpd (SER
root@server1:/home/administrator#

---------------------------------------------------------------------------------------------------


ISPConfig 3 firewall on = iptables -L
----------------------------------

root@server1:/home/administrator# iptables -L
Chain INPUT (policy DROP)
target prot opt source destination
fail2ban-ssh tcp -- anywhere anywhere multiport dports ssh
DROP tcp -- anywhere 127.0.0.0/8
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
DROP all -- 224.0.0.0/4 anywhere
PUB_IN all -- anywhere anywhere
PUB_IN all -- anywhere anywhere
PUB_IN all -- anywhere anywhere
PUB_IN all -- anywhere anywhere
DROP all -- anywhere anywhere

Chain FORWARD (policy DROP)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
DROP all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
PUB_OUT all -- anywhere anywhere
PUB_OUT all -- anywhere anywhere
PUB_OUT all -- anywhere anywhere
PUB_OUT all -- anywhere anywhere

Chain INT_IN (0 references)
target prot opt source destination
ACCEPT icmp -- anywhere anywhere
DROP all -- anywhere anywhere

Chain INT_OUT (0 references)
target prot opt source destination
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere

Chain PAROLE (13 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere

Chain PUB_IN (4 references)
target prot opt source destination
ACCEPT icmp -- anywhere anywhere icmp destination-unreachable
ACCEPT icmp -- anywhere anywhere icmp echo-reply
ACCEPT icmp -- anywhere anywhere icmp time-exceeded
ACCEPT icmp -- anywhere anywhere icmp echo-request
PAROLE tcp -- anywhere anywhere tcp dpt:ftp-data
PAROLE tcp -- anywhere anywhere tcp dpt:ftp
PAROLE tcp -- anywhere anywhere tcp dpt:ssh
PAROLE tcp -- anywhere anywhere tcp dpt:smtp
PAROLE tcp -- anywhere anywhere tcp dpt:domain
PAROLE tcp -- anywhere anywhere tcp dpt:www
PAROLE tcp -- anywhere anywhere tcp dptop3
PAROLE tcp -- anywhere anywhere tcp dpt:imap2
PAROLE tcp -- anywhere anywhere tcp dpt:https
PAROLE tcp -- anywhere anywhere tcp dpt:2002
PAROLE tcp -- anywhere anywhere tcp dpt:mysql
PAROLE tcp -- anywhere anywhere tcp dpt:http-alt
PAROLE tcp -- anywhere anywhere tcp dpt:webmin
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:mysql
DROP icmp -- anywhere anywhere
DROP all -- anywhere anywhere

Chain PUB_OUT (4 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere

Chain fail2ban-ssh (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere


Do you have any thoughts to share on debugging this problem?

Regards,

Peter Bowey

 

Solution discovered!

Thanks Till,

I appreciate your input and time to respond to this problem!

I did some further investigation based on your thoughts of likely DNS problems:

1) – PING worked fine on all external IPs
2) – DNS lookups failed at the given variable time frame (1+ hour(s))

I concluded that the added DNSMASQ was the likely problem – as it does both DHCP + DNS caching and lookups. As I run a dedicated server with a fixed IP and its own hosted / dedicated [nameserver], I changed the following DNSMASQ configuration as per:

interface=eth0​

no-dhcp-interface=eth0​

This change was made some 5 hours past, and so far all my ISPConfig based Server out-bound HTTP requests from the Server are completing (no 404 errors).

I conclude that the DHCP lease / expire times of the original dnsmasq were the problem, and I certainly only wanted DNS caching – and not DHCP.

Based on some other users thoughts and experiences, I finally decided to use PDNSD (proxy DNS cache). This works fine, and does offer good outbound DNS caching, even across server restarts.


Thanks for creating ISPConfig 2 and 3 – I have used them both and they are both great products. Very different in each V2 & V3 version, but well programmed!

Regards,
Peter

Peter Bowey Computer Solutions
69 Sutherland Ave, Hayborough,
Victor Harbor, SA, Australia, 5211
Ph: (08) 8552 8630
Fax: (08) 8552 9185
Mobile: 0414 440 575
EMAIL: [email protected]
WebSite: www.pbcomp.com.au

 

The server is doing something odd with MyDNS and I can't determine what it is. I'm using a recursive resolver set up in my /etc/mydns.conf rather than adding more software (DNSMasq) to the system because I have a full resolving bind9 setup running on another machine for my customers.

Like you, at some time interval that I haven't yet determined MyDNS stops using the resolver and will not resolve external domains. A simple /etc/init.d/mydns stop; /etc/init.d/mydns start solves the problem ... for a while. I still haven't figured out WHAT is causing it to stop resolving. I'll probably have to write a script that does lookups every few minutes and logs the time to see when it fails, I can't sit and watch the machine.

 

Did you'll figure out the root cause for this problem? I have the same issue on my VPS (Ubuntu 8.04 & ISPConfig 3.0.1.3). After an indefinite period all outgoing traffic is blocked. I end up with the pretty much the same iptables rules posted in the first post in this thread. I dont remember setting up the iptables with these rules. Does ISPConfig write anything to the iptables? I tried flushing the iptables, but that crashed my VPS.

So any help/input is appreciated.

Thanks
-Masky