How can I fix recurring PHP 503 Errors on my Apache, mod_proxy_fcgi, php-fpm Server?

Discussion in 'Server Operation' started by Riemu, Aug 28, 2015.

  1. Riemu

    Riemu New Member

    Hey,

    i'm having trouble with my php-fpm setup. Maybe you guys can point me in the right direction.
    First off, everything is working fine. But every now and then, I'll get 503 Errors. Those errors are gone once I reload the website. They only ever appear on php sites and are not isolated to one domain or one framework. I am receiving the 503 errors in PHPmyAdmin, Wordpress and Typo3. Those are the 3 sites I have tested. They are on seperate vhosts and have different php-fpm pools, but they share the same php-fpm master prozess.

    The server I am running is Apache 2.4 (MPM-Event Workers), without mod_php or cgi/fastcgi. Instead, I'm using mod_proxy and mod_proxy_fcgi to pass every .php file to my php-fpm process.
    One thing to note is, the server is not yet in production, so there is little to no traffic at all. Server Hardware is strong, 12 VCores und 32 GB Ram.

    My mod_proxy and mod_proxy_fcgi setups are default - I haven't changed anything there.


    My vhost config (the proxy part):

    <FilesMatch "\.php$">
    SetHandler "proxy:unix:///opt/php-5.6.11/var/run/php5-fpm-mywebsite.sock|fcgi://mywebsite/"
    </FilesMatch>
    <Proxy fcgi://mywebsite/ enablereuse=on retry=0>
    </Proxy>​

    Note: I have had max=10 in the Proxy directive before and it seemed to produce the 503 error more often. Now that I have removed max=10, it seems to occur less. Might just be coincidence though.


    My PHP-FPM Pool Config (the relevant parts):

    listen = var/run/php5-fpm-mywebsite.sock
    listen.owner = mywebsite
    listen.group = www-data
    listen.mode = 0660
    listen.backlog = 65535

    user = mywebsite
    group = www-data
    listen.allowed_clients = 127.0.0.1

    pm = ondemand
    pm.max_children = 20
    pm.process_idle_timeout = 15s

    request_terminate_timeout = 300s
    rlimit_files = 131072
    rlimit_core = unlimited
    catch_workers_output = no


    My PHP-FPM Config (the relevant parts):

    emergency_restart_threshold = 10
    emergency_restart_interval = 1m
    process_control_timeout = 10
    events.mechanism = epoll​

    PHP.ini
    My PHP.ini for the PHP-FPM Master Process.
    Everything not listed here, is either default php settings or should not be relevant:

    memory_limit = 400M
    upload_max_filesize = 20M
    post_max_size = 20M
    max_execution_time = 600
    max_input_time = -1
    max_input_vars = 10000
    suhosin.get.max_vars = 10000
    suhosin.post.max_vars = 10000

    [Zend]
    zend_extension=/opt/php-5.6.11/lib/php/extensions/no-debug-non-zts-20131226/ioncube.so
    zend_extension=opcache.so
    opcache.revalidate_freq=0
    ;opcache.validate_timestamps=0
    opcache.max_accelerated_files=50000
    opcache.memory_consumption=256
    opcache.interned_strings_buffer=16
    opcache.fast_shutdown=1

    [APC]
    extension=apcu.so
    apc.enabled=1
    apc.shm_segments = 1
    apc.shm_size=256M
    apc.ttl=7200
    apc.user_ttl=7200
    apc.gc_ttl=3600
    apc.stat=1
    apc.enable_cli=0
    apc.file_update_protection=2
    apc.max_file_size=2M
    apc.include_once_override=0
    apc.mmap_file_mask=/tmp/apc.XXXXXX
    apc.cache_by_default=1
    apc.use_request_time=1
    apc.slam_defense=0
    apc.stat_ctime=0
    apc.canonicalize=1
    apc.write_lock=1
    apc.report_autofilter=0
    apc.rfc1867=0
    apc.rfc1867_prefix =upload_
    apc.rfc1867_name=APC_UPLOAD_PROGRESS
    apc.rfc1867_freq=0
    apc.rfc1867_ttl=3600
    apc.lazy_classes=0
    apc.lazy_functions=0

    extension=memcache.so
    extension=memcached.so​

    Note: Memcached has 1 GB of memory allocated to it.

    Apache Log
    The actual error message from the apache error.log. The error message if always the same. (I have enabled verbose proxy logging):

    [proxy:debug] [pid 141760:tid 140526898214656] mod_proxy.c(1159): [client myclient] AH01143: Running scheme unix handler (attempt 0), referer: hxxx://mywebsite/website/typo3/install/index.php?TYPO3_INSTALL[type]=cleanup
    [proxy_fcgi:debug] [pid 141760:tid 140526898214656] mod_proxy_fcgi.c(879): [client myclient] AH01076: url: fcgi://mywebsite//var/www/html/mywebsite/htdocs/typo3site/website/index.php proxyname: (null) proxyport: 0, referer: hxxx://mywebsite/website/typo3/install/index.php?TYPO3_INSTALL[type]=cleanup
    [proxy_fcgi:debug] [pid 141760:tid 140526898214656] mod_proxy_fcgi.c(886): [client myclient] AH01078: serving URL fcgi://mywebsite//var/www/html/mywebsite/htdocs/typo3site/website/index.php, referer: hxxx://mywebsite/website/typo3/install/index.php?TYPO3_INSTALL[type]=cleanup
    [proxy:debug] [pid 141760:tid 140526898214656] proxy_util.c(2147): AH00942: FCGI: has acquired connection for (mywebsite)
    [proxy:debug] [pid 141760:tid 140526898214656] proxy_util.c(2200): [client myclient] AH00944: connecting fcgi://mywebsite//var/www/html/mywebsite/htdocs/typo3site/website/index.php to mywebsite:8000, referer: hxxx://mywebsite/website/typo3/install/index.php?TYPO3_INSTALL[type]=cleanup
    [proxy:debug] [pid 141760:tid 140526898214656] proxy_util.c(2237): [client myclient] AH02545: fcgi: has determined UDS as /opt/php-5.6.11/var/run/php5-fpm-mywebsite.sock, referer: hxxx://mywebsite/website/typo3/install/index.php?TYPO3_INSTALL[type]=cleanup
    [proxy:debug] [pid 141760:tid 140526898214656] proxy_util.c(2409): [client myclient] AH00947: connected //var/www/html/mywebsite/htdocs/typo3site/website/index.php to httpd-UDS:0, referer: hxxx://mywebsite/website/typo3/install/index.php?TYPO3_INSTALL[type]=cleanup
    [proxy_fcgi:error] [pid 141760:tid 140526898214656] [client myclient] AH01067: Failed to read FastCGI header, referer: hxxx://mywebsite/website/typo3/install/index.php?TYPO3_INSTALL[type]=cleanup
    [proxy_fcgi:error] [pid 141760:tid 140526898214656] (104)Connection reset by peer: [client myclient] AH01075: Error dispatching request to : , referer: hxxx://mywebsite/website/typo3/install/index.php?TYPO3_INSTALL[type]=cleanup
    [proxy:debug] [pid 141760:tid 140526898214656] proxy_util.c(2162): AH00943: FCGI: has released connection for (mywebsite)​


    Now my question:

    How can I fix recurring PHP 503 Errors on my Webserver?


    My thoughts:
    • Maybe mod_proxy_fcgi in UDS mode. But isn't it bad to deactive UDS, performance wise? Can I tweak anything there?
    • mod_proxy or mod_proxy_fcgi not working correctly with php-fpm or not set up properly?
    • APC or ZendOPCache or Memcached messing things up? I wouldn't say its the Memory allocated to them, because there is hardly anything going on on the server and there is plenty of free memory
    • Some Problems with the php.ini config?
    • Some Problems with the php-fpm config or php-fpm pool config?

    I'm not an export on those things, so I'm having a hard time figuring things out. Apache with php-fpm doesn't seem that common either, most google results are nginx based, which doesn't help me much.

    Maybe someone here can help me?


    Thanks alot!!
     
  2. TiTex

    TiTex Member

    is there any special reason why you are going the proxy_fcgi way ?
    the first thing you should try in my opinion is installing and trying mod_php , if that's ok i think you can exclude apache & application fault , then try mod_fastcgi ... this way you can exclude php-fpm config issues
    so maybe there's a bug in proxy_fcgi

    also per documentation you should not use unix sockets as handler , use the standard inet address handler like fcgi://127.0.01 and see if that solves the problem
     
    Last edited: Aug 29, 2015

Share This Page