Bug in Apache 2.2.3 on Debian Etch ?

Discussion in 'Server Operation' started by Hans, Jul 5, 2007.

  1. Hans

    Hans Moderator Moderator

    I have noticed that Apache 2.2.3 on Debian Etch and ISPConfig 2.2.14 fails/restarts more often then it did with the previous Apache version on Debian Sarge.
    From time to time Apache 2.2.3 fails, and Monit has to start it again.

    Everytime it happens, i only see this within /var/log/apache2/error.log:

    [Thu Jul 05 20:08:10 2007] [notice] caught SIGTERM, shutting down
    [Thu Jul 05 20:08:12 2007] [notice] Apache/2.2.3 (Debian) PHP/5.2.0-8+etch4 mod_ssl/2.2.3 OpenSSL/0.9.8c mod_perl/2.0.2 Perl/v5.8.8 configured -- resuming normal operations ssl/2.2.3

    It is very difficult for me to find the cause of this, because i can not find any other errors within the Apache2 logfiles. I also had a look at the log files wich belong to the websites, but i can't find any related error.

    Today i found a more serious error within /var/log/apache2/error.log:

    Bad file descriptor: apr_socket_accept: (client socket)

    /var/log/apache2/error.log
    [Thu Jul 05 10:03:22 2007] [notice] Graceful restart requested, doing restart
    [Thu Jul 05 10:03:22 2007] [error] (9)Bad file descriptor: apr_socket_accept: (client socket) lient socket)
    [Thu Jul 05 10:03:22 2007] [error] (9)Bad file descriptor: apr_socket_accept: (client socket) lient socket)
    [Thu Jul 05 10:03:22 2007] [error] (9)Bad file descriptor: apr_socket_accept: (client socket) lient socket)
    [Thu Jul 05 10:03:22 2007] [notice] Apache/2.2.3 (Debian) PHP/5.2.0-8+etch4 mod_ssl/2.2.3 OpenSSL/0.9.8c mod_perl/2.0.2 Perl/v5.8.8 configured -- resuming normal operations ssl/2.2.3 OpenSSL/0.9.8c mod_perl/2.0.2 Perl/v5.8.8 configured -- resuming normal operations\\\

    I tried to find out what it means, but i can't find the answer and i don't know what could be the cause.

    I found out that quiet some users had also these errors.
    And even i found a bug report:

    http://bugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=400918

    Does anyone out here has the same experience as me?
    Can anyone confirm this behaviour of Apache 2.2.3 on Debian Etch and tell me how to avoid these errors?

    Thanks in advance!
     
    Last edited: Jul 5, 2007
  2. daveb

    daveb Member

    Hans, I too have Apache 2.2.3 on Debian Etch and ISPConfig 2.2.14 running and I took a look at my log files going back some time and see no apache2 failures or errors.
     
  3. Hans

    Hans Moderator Moderator

    Thank you for your reply.

    The strange is also that every night at 5:01 am Apache Fails, one minute later (at 5:02 am) i see that monit has restarted Apache again.

    As far as i know, i do not have cronjobs running at that time.
     
  4. falko

    falko Super Moderator Howtoforge Staff

    I have no idea what's causing this, but if you find out, please post it. I'm very interested in this.
     
  5. daveb

    daveb Member

    Well Hans today I upgraded the php5 on the etch server and afterwards took a look at my log files and found this.
    Code:
    [Sat Jul 07 12:03:06 2007] [notice] Graceful restart requested, doing restart
    [Sat Jul 07 12:03:06 2007] [error] (9)Bad file descriptor: apr_socket_accept: (client socket)
    [Sat Jul 07 12:03:08 2007] [notice] Apache configured -- resuming normal operations
    [Sat Jul 07 12:03:08 2007] [warn] long lost child came home! (pid 2996)
    The other day I went back quite awhile in the logs looking for this due to your post and found nothing like this until today. Like I said today I applied the security updates to php5 from deb then did a reboot for the first time in months. If you find anything on this please post.
    I found this http://issues.apache.org/bugzilla/show_bug.cgi?id=42829
     
    Last edited: Jul 8, 2007
  6. Hans

    Hans Moderator Moderator

    @ Daveb,

    Thank you for your useful report.
    What i read in your information and also what i found on the internet is that these kind of problems could occur when Apache is listening to more then 1 port. In our case the ports 80 & 443. But i have to say that i did not have any better results when i removed port 443 within /etc/apache2/ports.conf.

    I have to say that i had only once that error which says:
    9)Bad file descriptor: apr_socket_accept: (client socket)

    Yesterday, i did the php5 security update too and this update did not cause any errors for me. (i had that Bad file descriptor error before the php5-security update).

    As i mentioned before, Apache was failing more often on Etch then it did on Sarge.

    In the mean time i found out that i had to increase the value to 256 for MaxClients within /etc/apache2/apache2.conf.

    If i keep the default value of 0 for MaxRequestsPerChild Apache2 fails too often. Now i have a value set to 10000, which works fine until now. Apache2 does not fail anymore.
     
    Last edited: Jul 8, 2007
  7. daveb

    daveb Member

    Good Deal Hans, what you change the values for prefork or worker mpm or both?
    I just changed the values for both. Thanks I will post if I continue to see error or find something more as a fix.
     
    Last edited: Jul 8, 2007
  8. Hans

    Hans Moderator Moderator

    The server has 2 GB internal memory and the system does not make use of /swap so that's nice.

    On the moment i have the following settings for Apache 2.2.3 within /etc/apache2/apache2.conf:

    Timeout 300

    KeepAlive On

    MaxKeepAliveRequests 500

    KeepAliveTimeout 4

    <IfModule prefork.c>
    StartServers 5
    MinSpareServers 5
    MaxSpareServers 10
    MaxClients 256
    MaxRequestsPerChild 10000

    <IfModule worker.c>
    StartServers 2
    MaxClients 150
    MinSpareThreads 25
    MaxSpareThreads 75
    ThreadsPerChild 25
    MaxRequestsPerChild 0
    </IfModule>

    <IfModule perchild.c>
    NumServers 5
    StartThreads 5
    MinSpareThreads 5
    MaxSpareThreads 10
    MaxThreadsPerChild 20
    MaxRequestsPerChild 0
    AcceptMutex fcntl
    </IfModule>
     
  9. daveb

    daveb Member

    Hans I made the changes as you suggested sorta but I am curious as to something. I am no apache guru so bare with me, but my conf is a bit different. So thats why I am asking since we both run apache 2.2.3 on etch.
    here is what i have in my conf.
    <IfModule mpm_prefork_module>
    and
    <IfModule mpm_worker_module>
    but no
    <IfModule perchild.c>

    not
    <IfModule prefork.c>
    or
    <IfModule worker.c>
    or
    <IfModule perchild.c>

    so my question is that due to how apache was installed or something?
    here is what I have running
    /etc/apache2# httpd -V -l
    Server version: Apache/2.2.3
    Server built: Mar 27 2007 15:06:55
    Server's Module Magic Number: 20051115:3
    Server loaded: APR 1.2.7, APR-Util 1.2.7
    Compiled using: APR 1.2.7, APR-Util 1.2.7
    Architecture: 32-bit
    Server MPM: Prefork
    threaded: no
    forked: yes (variable process count)
    Server compiled with....
    -D APACHE_MPM_DIR="server/mpm/prefork"
    -D APR_HAS_SENDFILE
    -D APR_HAS_MMAP
    -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
    -D APR_USE_SYSVSEM_SERIALIZE
    -D APR_USE_PTHREAD_SERIALIZE
    -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
    -D APR_HAS_OTHER_CHILD
    -D AP_HAVE_RELIABLE_PIPED_LOGS
    -D DYNAMIC_MODULE_LIMIT=128
    -D HTTPD_ROOT=""
    -D SUEXEC_BIN="/usr/lib/apache2/suexec"
    -D DEFAULT_PIDLOG="/var/run/apache2.pid"
    -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
    -D DEFAULT_LOCKFILE="/var/run/apache2/accept.lock"
    -D DEFAULT_ERRORLOG="logs/error_log"
    -D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types"
    -D SERVER_CONFIG_FILE="/etc/apache2/apache2.conf"
     
  10. Hans

    Hans Moderator Moderator

    httpd -V -l output

    Hi Daveb,

    I am not an Apache guru either. My best reference is http://httpd.apache.org/ :) and i never added the IfModule worker.c & IfModule perchild.c manually.

    Here is my httpd -V -l output:

    Apache/2.2.3
    Server built: Mar 27 2007 14:57:24
    Server's Module Magic Number: 20051115:3
    Server loaded: APR 1.2.7, APR-Util 1.2.7
    Compiled using: APR 1.2.7, APR-Util 1.2.7
    Architecture: 64-bit
    Server MPM: Prefork
    threaded: no
    forked: yes (variable process count)
    Server compiled with....
    -D APACHE_MPM_DIR="server/mpm/prefork"
    -D APR_HAS_SENDFILE
    -D APR_HAS_MMAP
    -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
    -D APR_USE_SYSVSEM_SERIALIZE
    -D APR_USE_PTHREAD_SERIALIZE
    -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
    -D APR_HAS_OTHER_CHILD
    -D AP_HAVE_RELIABLE_PIPED_LOGS
    -D DYNAMIC_MODULE_LIMIT=128
    -D HTTPD_ROOT=""
    -D SUEXEC_BIN="/usr/lib/apache2/suexec"
    -D DEFAULT_PIDLOG="/var/run/apache2.pid"
    -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
    -D DEFAULT_LOCKFILE="/var/run/apache2/accept.lock"
    -D DEFAULT_ERRORLOG="logs/error_log"
    -D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types"
    -D SERVER_CONFIG_FILE="/etc/apache2/apache2.

    Note: i have a 64bit system.

    How is your Apache server running on the moment does it fail form time to time or is everything fine right now?

    (I hope that Apache 2.2.4 will be available soon for Etch, because this is a bugfix release).
     
  11. daveb

    daveb Member

    Thanks Hans, as of today I have yet to see any new activity in error.log since i changed those values.
    Thanks for the reply though. I was wondering if you had a 64 bit system or not when I posted. Funny how our build dates are 8 mins apart.
     

Share This Page