A Working Nginx Catchall with ISPConfig 3.

Discussion in 'Tips/Tricks/Mods' started by renders, Dec 9, 2010.

  1. renders

    renders New Member

    Hi all.. I thought that I would share my working config after following the The NGINX Catchall for ISPconfig 3 howto.

    I have 7 sites running, 2 drupal, 2 workpress and 3 custom php. This is running on a openvz vps with 16gb and 8 cpus. You should adjust your install accordingly. I sourced most of this info from the nginx forums..

    It will pass thru all vhosts created within ispconfig with no impact. But, if you manually create vhost files for each site, nginx will serve the static files directly, greatly improving the speed of your sites. Apache is configured with 2048 max clients but with this config, barely uses 100. Yet nginx was running 4000 connections recently.

    I have replaced the configs suggested in the howto with:

    nginx.conf
    - main conf file
    ** needed the worker_rlimit_nofile to eliminate open file errors, default is 1024


    Code:
    user www-data;
    worker_processes  3;
    
    error_log  /var/log/nginx/error.log;
    pid        /var/run/nginx.pid;
    worker_rlimit_nofile  4096;
    
    events {
        worker_connections  2048;
        use epoll;
    }
    
    
    
    http {
        index index.php index.html index.htm;
        root  /var/www/;
        error_page   502 503 504  /50x.html;
        
        server_names_hash_max_size 512;
        server_names_hash_bucket_size 128;
        
        include       /etc/nginx/mime.types;
        default_type  application/octet-stream;
         
        client_header_timeout   10m;
        client_body_timeout     10m;
        send_timeout            10m;
        connection_pool_size           512;
        client_header_buffer_size       1k;
        large_client_header_buffers     4 2k;
        request_pool_size               4k;
        client_body_temp_path /var/lib/nginx/body 1 2;  
        
        gzip  on;
        gzip_http_version 1.0;
        gzip_min_length  1100;
        gzip_types text/plain text/css image/x-icon application/x-javascript application/javascript text/javascript application/atom+xml application/xml;
        gzip_buffers 16 8k;    
        gzip_vary         on;
        gzip_disable "MSIE [1-6]\.(?!.*SV1)";
        gzip_proxied  any;
        gzip_comp_level 2;
    
        output_buffers  1 32k;
        postpone_output 1460;
        sendfile        on;
        #tcp_nopush     on;
        keepalive_timeout       75 20;
        tcp_nodelay        off;
        ignore_invalid_headers  on;
       
        server_tokens off;
    
        access_log	  off;
        error_log     /var/log/nginx/error_log;
    
        #default nginx virtual host server
        #it's used if there is no other matching nginx virtual host found
        server {      
    	listen	80 default;
    	#you can access nginx internal stats using lynx or alike console based web browser     
    	#at http://your.external.interface.ip/nginx_status address
    	location /nginx_status {
    	    stub_status on;
    	    access_log   off;
    	    allow 127.0.0.1;
    	    deny all;
    	}
    	
            #default proxy settings for each virtual host
            include /etc/nginx/proxy.conf;
        }
        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
    }
    
    proxy.conf
    - common proxy conf file.
    ** set the client_max_body_size to match apache max_file_size

    Code:
    location / {
       proxy_pass         http://127.0.0.1:82/;
       proxy_redirect     off;
       proxy_set_header   Host             $host;
       proxy_set_header   X-Real-IP        $remote_addr;
       proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
       client_max_body_size       64m;
       client_body_buffer_size    128k;
       proxy_connect_timeout      90;
       proxy_send_timeout         90;
       proxy_read_timeout         90;
       proxy_buffer_size          4k;
       proxy_buffers              4 32k;
       proxy_busy_buffers_size    64k;
       proxy_temp_file_write_size 64k;
    }
    proxy_failback.conf
    - if nginx cannot serve the file, pass it on to apache.

    Code:
    #proxy options can't be set inside if directive 
    proxy_set_header   Host             $host;
    proxy_set_header   X-Real-IP        $remote_addr;
    proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
     if (!-f $request_filename) {
     break;
     proxy_pass http://127.0.0.1:82;
    }
    
    If you want nginx to serve static files directly, you need to add the following for each vhost in the sites-available folder, linked to the sites-enabled folder
    ** PNG is removed due to a realtime file issue, I hope to resolve this properly.

    Code:
    server {
    #   listen 80;
       server_name www.domain.com domain.com;
        #default proxy settings shared are among all virtual hosts
        access_log off;
        include /etc/nginx/proxy.conf;
          location ~* ^.+.(jpe?g|gif|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|html?|txt|tar|mid|midi|wav|bmp|rtf|js|swf|avi|mp3)$ {
               #forcing browser to cache locally static content for 1 day
               expires 1d;
               root /var/www/domain.com/web/;
               #access_log  /var/log/ispconfig/httpd/domain.com/access.log;
               access_log  off;
               #graceful fallback in case if static content doesn't exist
    	   include /etc/nginx/proxy_fallback.conf;
          }
    }
    Good luck, I hope you find this helpful.

    Rob
     
    Last edited: Dec 9, 2010
  2. renders

    renders New Member

    BTW, I am constantly tweaking and improving.. So if you have any suggestions, please share..

    Rob
    ;)
     
  3. renders

    renders New Member

    Found a typo in the nginx.conf file

    Change the line as follows

    error_log /var/log/nginx/error.log;

    Otherwise your error log will never be rotated and could get VERY large!

    Rob
     
  4. renders

    renders New Member

    Error 400 Bad Request

    I was getting this on some browsers and only on one site.. Quite strange but searching google came up with the following fix that works, something to due with cookie size.

    Change the line in Nginx.conf as follows

    large_client_header_buffers 4 2k

    increase the second number to 8k

    large_client_header_buffers 4 8k

    regards,

    Rob
     
  5. tio289

    tio289 Member

    Hi, I have few notes to your config files, specifically proxy.conf file.

    Directives:
    1. client_max_body_size and client_body_buffer_size have to be out of the location, directly in server section.
    2. proxy_connect_timeout cannot be more than 75s - look on reference http://wiki.nginx.org/HttpProxyModule#proxy_connect_timeout
    3. proxy_send_timeout set to match php max_input_type
    4. proxy_read_timeout set to match php max_execution_time

    I changed buffers too, because I got this error:
    Code:
    upstream sent too big header while reading response header from upstream
    File should look like this:
    Code:
    client_max_body_size       64m;
    client_body_buffer_size    128k;
    location / {
       proxy_pass         http://127.0.0.1:82/;
       proxy_redirect     off;
       proxy_set_header   Host             $host;
       proxy_set_header   X-Real-IP        $remote_addr;
       proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
       proxy_connect_timeout      60;
       proxy_send_timeout         90;
       proxy_read_timeout         90;
       proxy_buffer_size          32k;
       proxy_buffers              8 16k;
       proxy_busy_buffers_size    64k;
       proxy_temp_file_write_size 64k;
    }
    
     
    Last edited: Apr 15, 2011

Share This Page