Slow browsing in HTTPS

Discussion in 'Installation/Configuration' started by cremos, Jan 21, 2021.

  1. cremos

    cremos Member

    1 /I have nothing in the "global settings" of the haproxy regarding SSL.
    2 / I have nothing about SSL in /etc/apache2/mods-enabled/ssl.conf.
    3 / Include Optional conf-enabled / *. Conf no SSL option.
    4 / Include Optional sites-enabled
    SSL configurations are found by default in vhost.

    I'm doing an apache2ctl -S
    the first vhost is dev.dsden60.ac-amiens.fr

    Code:
    192.168.236.50:443     is a NameVirtualHost
             default server dev.dsden60.ac-amiens.fr (/etc/apache2/sites-enabled/100-dev.dsden60.ac-amiens.fr.vhost:110)
             port 443 namevhost dev.dsden60.ac-amiens.fr (/etc/apache2/sites-enabled/100-dev.dsden60.ac-amiens.fr.vhost:110)
                     alias dev.dsden60.ac-amiens.fr
             port 443 namevhost gdmat.dsden60.ac-amiens.fr (/etc/apache2/sites-enabled/100-gdmat.dsden60.ac-amiens.fr.vhost:116)
                     alias gdmat.dsden60.ac-amiens.fr
    
    I have just modified the option as follows in dev.dsden60.ac-amiens.fr:
    Code:
    SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
    this applied to all sites after checking on Qualys SSL Labs site:
    Code:
    Overall Rating = A & Protocol Support = TLS 1.3 TLS 1.2 and always "Protocol or cipher suite mismatch" errors with TLS 1.3 protocol
    
    We went from 17s to 10s of SSL negotiation, capture attached.
    I do not understand why by acting on the first vhost 443 by default this has repercussions on all the sites (vhost).

    I understand that the default server is just the first one encountered for the port because Apache parses the config / vhost files.
    But why by acting on the first vhost 443 by default this has an impact on all the sites (vhost).
     
    Last edited: Jan 25, 2021
  2. Steini86

    Steini86 Active Member

    Without having a look on the system I can't really help you. You might consider using the professional support and let them have a look on your server: https://www.ispconfig.org/get-support/?type=ispconfig
    I still believe you have a problem with your HAProxy as I have never seen that behavior with apache. This problem does not occur on the standard configuration. So, you somehow have messed with your config files. Given the time you have already invested, it might almost be faster to restart from scratch. Start with a minimal system and verify the results after each step.

    What have you changed in that file? In the standard configuration, there should be a lot about SSL in that file ;)
     
  3. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    there doesn't seem too much wrong with the haproxy config, it can be as simple as:

    Code:
    frontend https
           bind *:443
           mode tcp
           default_backend bk_app
    
    backend bk_app
          mode tcp
          server TWXAPP01  <twxapp01IP>:<port>
    
    maybe remove the extraneous bits you have configured : 'check-ssl verify none' and see if that makes any difference.

    but the concern i have with it is that your frontend doesn't specify any backend, not even as a default. It may be that it matches them as the frontend and backend have the same name, it's been years since i've used haproxy, so my memories hazy on that, but this is what concerns me about it:
     
  4. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    ok, forget that about the backend not specified, just looked at the full haproxy cfg you attached, that does specify it.

    perhaps the problem is you also set 'option ssl-hello-chk'
    it looks like that's an old option that only uses sslv3, and manually constructs the tcp frame.
    try removing that, or replace it with the check-ssl directive, it replaces the old ssl-hello-chk. It actually uses OpenSSL
     
  5. cremos

    cremos Member

    Hello !
    I always had the default configuration from the beginning and after upgrading from 3.1.15p2 to 3.2.2. The only modification made is in the vhost and I then reset via Ispconfig> Tools> Resync your websites.

    Indeed, I must have a problem with the Haproxy which takes 10s to respond and 10 for SSL.
    In private browsing I practically have timed out errors

    2021-01-25 19_58_43-Lycée Madeleine Michelis - Amiens.png

    I commented the line :
    Code:
            ## vérifie la connexion ainsi que sa capacité à gérer les connexions SSL (SSLv3 en particulier).
            ## option ssl-hello-chk
    
    We always have a long time.
     
    Last edited: Jan 26, 2021
  6. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    as already stated. ssl-hello-chk is SSLv3 only. that's your service health check.
    sslv3 is disabled on apache on your server (if it isn't, it bloody well should be). so it's always going to fail. which probably lines up quite nicely with what you're seeing, i'm betting the timeout is set at 10 seconds.

    try removing option ssl-hello-chk or replace it with check-ssl or tcp-check.
     
  7. cremos

    cremos Member

    1/ I confirm that the ssl-hello-chk option is indeed disabled in the backend.
    Code:
            ## vérifie la connexion ainsi que sa capacité à gérer les connexions SSL (SSLv3 en particulier).
            ## option ssl-hello-chk
    
    2/ SSLv3 is indeed disabled throughout vhost:
    Code:
    SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
    3/ in /etc/apache2/mods-enabled/ssl.conf
    Code:
    SSLProtocol all -SSLv3
    4/ I also have in the backend the option:
    Code:
    server panel3_https.in 192.168.236.50:443 check-ssl verify none
     
    Last edited: Jan 26, 2021
  8. Steini86

    Steini86 Active Member

    From the manual:
    I would try with the most simple haproxy setup (if you can't bypass it for a test). Then add your options one after the other and observe the results.
    And while you are at it, you should really upgrade your HAProxy setup. There is a valid chance, that your problem is solved then.
     
  9. cremos

    cremos Member

    I can't understand why the first default vhost on port 443 takes over all vhost?
    Let me explain, when I modify an SSL option SSLProtocol example this applies to all vhost.

    Code:
    192.168.236.50:443     is a NameVirtualHost
             default server dev.dsden60.ac-amiens.fr (/etc/apache2/sites-enabled/100-dev.dsden60.ac-amiens.fr.vhost:110)
             port 443 namevhost dev.dsden60.ac-amiens.fr (/etc/apache2/sites-enabled/100-dev.dsden60.ac-amiens.fr.vhost:110)
    
    When I modify an SSL option in any vhost it is not taken into account.
    Code:
             port 443 namevhost madeleine.michelis-amiens.lyc.ac-amiens.fr (/etc/apache2/sites-enabled/100-madeleine.michelis-amiens.lyc.ac-amiens.fr.vhost:111)
                     alias madeleine.michelis-amiens.lyc.ac-amiens.fr
    
    Does this correspond to normal behavior?
     
  10. cremos

    cremos Member

    Another behavior, without putting the option "check-ssl verify none" in the backend to access via port 8080 to the management of Ispconfig that returns an error via my browser:
    Code:
    Bad Request
    Your browser sent a request that this server could not understand.
    Reason: You're speaking plain HTTP to an SSL-enabled server port.
    Instead use the HTTPS scheme to access this URL, please.
    ######
    Without the check-ssl verify none option the display is slower. I left the conf in the lightest haproxy to no avail. I need to plan to upgrade my server and update the version of my haproxy to 2.1 or 2.3.
     
    Last edited: Jan 27, 2021
  11. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    your using http mode for the 8080 port frontend and backend. that will not put an https request through unchanged.

    where exactly is this information being found? this is not the format of the information in the vhost configuration or the systemctl service status.

    exactly how are you creating these websites and their certificates?

    this is what the start of a vhost confguration should look like:
    Code:
    <Directory /var/www/domainname.co.uk>
                    AllowOverride None
                                    Require all denied
                    </Directory>
    <VirtualHost *:80>
                          DocumentRoot /var/www/clients/client1/web25/web
    
                    ServerName domainname.co.uk
                    ServerAlias www.domainname.co.uk
                    ServerAdmin [email protected]
    
    Code:
    <Directory /var/www/madeleine.michelis-amiens.lyc.ac-amiens.fr>
            AllowOverride None
                    Require all denied
            </Directory>
    
    <VirtualHost 192.168.236.50:80>
    
                        DocumentRoot /var/www/clients/client5/web85/web
              
            ServerName madeleine.michelis-amiens.lyc.ac-amiens.fr
            ServerAlias madeleine.michelis-amiens.lyc.ac-amiens.fr
            ServerAdmin [email protected]
    
    notice that in my example. the servername and serveralias are different, with the serveralias having the www prefix.
    whilst in your vhost, the servername and serveralias are identical, which is just weird. that serveralias line should not exist in your vhost configuration. i'm not aware that it, in itself, would cause any issues, but it is indicative of some fundamental misconfiguration in your system somewhere.
     
  12. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    My old experiences say that too much levels of config your apache2 server plus the duplicated SSLProtocol All (which in my mind should be on either vhost or ssl.conf and not both) could lead to delay as well.
     
  13. cremos

    cremos Member

    I constantly have errors in all the apache logs of vhost with OCSP stapling
    Code:
    [ssl:error] [pid 8950] AH01941: stapling_renew_response: responder error
    ssl_engine_init.c(1750): AH10083: Init: (madeleine.michelis-amiens.lyc.ac-amiens.fr:443) mod_md support
    is unavailable.
    ssl_util_stapling.c(761): AH01951: stapling_cb: OCSP Stapling callback called
    
    Reference to vhost: 100-madeleine.michelis-amiens.lyc.ac-amiens.fr.vhost
    Code:
    SSLUseStapling on
    
    </VirtualHost>
    
    <IfModule mod_ssl.c>
            SSLStaplingCache shmcb:/var/run/ocsp(128000)
    </IfModule>
    
     
    Last edited: Jan 27, 2021
  14. cremos

    cremos Member

    I checked OCSP Stapling using this command: echo QUIT | openssl s_client -connect madeleine.michelis-amiens.lyc.ac-amiens.fr:443 -status
    and received this response which shows it working:
    Code:
    CONNECTED(00000003)
    depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
    verify return:1
    depth=1 C = US, O = Let's Encrypt, CN = R3
    verify return:1
    depth=0 CN = madeleine.michelis-amiens.lyc.ac-amiens.fr
    verify return:1
    OCSP response:
    ======================================
    OCSP Response Data:
        OCSP Response Status: trylater (0x3)
    ======================================
    ---
    Certificate chain
     0 s:CN = madeleine.michelis-amiens.lyc.ac-amiens.fr
       i:C = US, O = Let's Encrypt, CN = R3
     1 s:C = US, O = Let's Encrypt, CN = R3
       i:O = Digital Signature Trust Co., CN = DST Root CA X3
    ---
    Server certificate
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
    subject=CN = madeleine.michelis-amiens.lyc.ac-amiens.fr
    
    issuer=C = US, O = Let's Encrypt, CN = R3
    
    ---
    No client certificate CA names sent
    Peer signing digest: SHA256
    Peer signature type: RSA-PSS
    Server Temp Key: X25519, 253 bits
    ---
    SSL handshake has read 3594 bytes and written 433 bytes
    Verification: OK
    ---
    New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
    Server public key is 4096 bit
    Secure Renegotiation IS NOT supported
    Compression: NONE
    Expansion: NONE
    No ALPN negotiated
    Early data was not sent
    Verify return code: 0 (ok)
    
    I checked for the cache files mentioned in the "SSLStaplingCache shmcb vhost:/var/run/ocsp" but the path does not exist (/var/run/ ocsp). Does it have to exist?

    Certificate test on the Qualyb SSL Lab website:
    OCSP stapling No INTOLERANT
     
    Last edited: Jan 27, 2021
  15. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    i don't think the non-existence of /var/run/ocsp is a problem, i don't have it on any of my servers, and i don't have any problems with ssl or slowness, and the stapling config in the vhost configurations matches yours.

    that said, for :
    Code:
    Server certificate
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
    subject=CN = madeleine.michelis-amiens.lyc.ac-amiens.fr
    
    are the certificate details really empty? or did you remove all that data from your post?
    if i run that status check on domains on my server i get all the certificate info included there.

    and for
    Code:
    OCSP response:
    ======================================
    OCSP Response Data:
       OCSP Response Status: trylater (0x3)
    ======================================
    
    i don't get trylater, i get successful, along with a lot more data:
    Code:
    OCSP response:
    ======================================
    OCSP Response Data:
        OCSP Response Status: successful (0x0)
        Response Type: Basic OCSP Response
        Version: 1 (0x0)
        Responder Id: C = US, O = Let's Encrypt, CN = R3
        Produced At: Jan 25 03:01:00 2021 GMT
        Responses:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: <data removed>
          Issuer Key Hash: <data removed>
          Serial Number: <data removed>
        Cert Status: good
        This Update: Jan 25 03:00:00 2021 GMT
        Next Update: Feb  1 03:00:00 2021 GMT
    
        Signature Algorithm: sha256WithRSAEncryption
             39:<data removed>:06
    ======================================
    
    
     
  16. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    when you ran
    Code:
    echo QUIT | openssl s_client -connect madeleine.michelis-amiens.lyc.ac-amiens.fr:443 -status
    
    where did you run it? what ip does that host resolve for that domain name?

    even if you ran it on your webserver, if that's resolving it to the external ip it's still sending in back out to come in via the haproxy instance, so it's not necessarily a fault on the webserver being detected.

    you may need to set the domain and it's local private ip address in the webservers hosts file so that it doesn't use dns to resolve the domain name. and then run the openssl command again from the webservers cli. see what response you get then without any involvement from haproxy.
     
  17. Monica Benson

    Monica Benson New Member

    You are probably tunneling all traffic, not just traffic destined for the remote LAN... Find and compare the best payday loan and cash advance https://maybeloan.com/cash-advance
     
    Last edited: Feb 9, 2021
  18. cremos

    cremos Member

    So I disable OCSP.
    Then redo a certificate test on the Qualyb SSL Lab site: I no longer have the error: "protocol or cipher suite mismatch"
    I also have as info:
    Code:
    Revocation status Validation error
    OCSP ERROR: Exception: connect timed out [http://r3.o.lencr.org]
     
  19. cremos

    cremos Member

    Performance test with the chrome debugger:
    We went from 10s to 20.12ms for the SSL test

    2021-01-27 18_12_38-Lycée Madeleine Michelis - Amiens.png

    I no longer have an error: Lighthouse returned error: failed_document_request (net :: err_times_out).
    On the performance test via the site: https://developers.google.com/
     
    Last edited: Jan 27, 2021
  20. cremos

    cremos Member

    Command executed from the haproxy:
    Code:
    echo QUIT | openssl s_client -connect madeleine.michelis-amiens.lyc.ac-amiens.fr:443 -status
    
    CONNECTED(00000004)
    depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
    verify return:1
    depth=1 C = US, O = Let's Encrypt, CN = R3
    verify return:1
    depth=0 CN = madeleine.michelis-amiens.lyc.ac-amiens.fr
    verify return:1
    OCSP response: no response sent
    ---
    Certificate chain
     0 s:CN = madeleine.michelis-amiens.lyc.ac-amiens.fr
       i:C = US, O = Let's Encrypt, CN = R3
     1 s:C = US, O = Let's Encrypt, CN = R3
       i:O = Digital Signature Trust Co., CN = DST Root CA X3
    ---
    Server certificate
    -----BEGIN CERTIFICATE-----
    MIIGWzCCBUOgAwIBAgISBNZWcYpdbkfO7uL/OU65/5BOMA0GCSqGSIb3DQEBCwUA
    MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
    EwJSMzAeFw0yMTAxMDQwODE4MDFaFw0yMTA0MDQwODE4MDFaMDUxMzAxBgNVBAMT
    Km1hZGVsZWluZS5taWNoZWxpcy1hbWllbnMubHljLmFjLWFtaWVucy5mcjCCAiIw
    DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBANxSHVzUsbpKkOj+ZBiX+F4xDjMi
    WnPn4n/bLIIuhiAfgfkprS61efSVarLOiK1c2avjNEgS2ZLhexSrn5SwZjx+Vcrc
    u88VbM3fe/DKDkZgBeBQShBHeFcZMb0aUhe1yCnPMegKYB5Z9OG0ZDeezguN/CnC
    7HOs3DWOBiouss7ctHrCsGxcqFTBSjw1VBCmNO3JFQ+agpQTt8Zyp1TP4W/PWB/v
    Lv46XdNo1r1WNlXkw7MmdwRrU8DSvzZ/GIVYRZDjh+E1PiWM0riG4gpf969Il56V
    PCjodnYxI8luA67tSUEKb/ZPdqr3Y1gfZyN+m8MZXNS/sRLC2Cazce4bIsL/Mgh0
    QQfZeljGd6MHoA7CtXJYpbAjEJDL1tyNHOLGuNcvpR0stoxVt3qCcvhj1zsZis6R
    qajIWM5OQTsSIzHOAYwX30FzhKqEVT7RFizR/MRFx0W6iWLWEJpw4gYPDpOvPjE7
    C1jcrauTbESNaVLxGH1bIi82TX9ztmJbDS6oxdcbAgLa6uHPAgdcQGVLsBr3dn6T
    8x/EGw6L9+N3VS0MDkRLv5c0c+zJ4y3t0YvNthqilu9pd73JMnCYb+gTYf2THXo6
    c7qyMB54aurI7Olk4H73D1mvLVbCXk/dooIAgyxVp657cTMqMzrzq9hQD0yfUvZR
    UGwdhOvADSCbo2hzAgMBAAGjggJmMIICYjAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0l
    BBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYE
    FGMUBsKWBRvTIzTqikw+OhuA13hPMB8GA1UdIwQYMBaAFBQusxe3WFbLrlAJQOYf
    r52LFMLGMFUGCCsGAQUFBwEBBEkwRzAhBggrBgEFBQcwAYYVaHR0cDovL3IzLm8u
    bGVuY3Iub3JnMCIGCCsGAQUFBzAChhZodHRwOi8vcjMuaS5sZW5jci5vcmcvMDUG
    A1UdEQQuMCyCKm1hZGVsZWluZS5taWNoZWxpcy1hbWllbnMubHljLmFjLWFtaWVu
    cy5mcjBMBgNVHSAERTBDMAgGBmeBDAECATA3BgsrBgEEAYLfEwEBATAoMCYGCCsG
    AQUFBwIBFhpodHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCCAQUGCisGAQQB1nkC
    BAIEgfYEgfMA8QB2AESUZS6w7s6vxEAH2Kj+KMDa5oK+2MsxtT/TM5a1toGoAAAB
    dsywaBAAAAQDAEcwRQIhAIANWgdinIg6PWyhbDWYWHXLgn60Gk5qGNpjxTfu8D6o
    AiA6bOIxGCt/3gArJ1buwYEivQoD7C5VW+KdWjBJZZ64DwB3APZclC/RdzAiFFQY
    CDCUVo7jTRMZM7/fDC8gC8xO8WTjAAABdsywaAAAAAQDAEgwRgIhAIqmCx8w6Ub4
    rji8ttcGb4M89WLpUfb8JZ5l+AHpeudnAiEA1bkhNN5tQ3mHobGQviZWLKWAzghb
    fSvu28cHuMIjdgYwDQYJKoZIhvcNAQELBQADggEBADgnrn7t6CJh+NJW/9NuMx64
    rUEINRMxUYVn4tMEx7G4UGaE1kRzFd32n8YIg62OLlw0OeVGHeBvRHsJjr/TQk+a
    CSi98S4LeE4nWk6w5x5jzy899XJMnvOUU6ooIcs3IDnw+8db7j/+Dcy2LnJ4fYaE
    yL2d3etFosI4eVYoBxP0e+uzJDFiWLBB8PeiabFUSkjGWkISZ+I8CvhFfuth2NYm
    ZxWJxmB61Y0f7D9fTrDAU3cY+2nXtmpGJWpclU4MitGfvJzGtXL9FEHuZLKB7MFM
    mPEBO5FcJV4sVqf2mQee9XbER4RiTfYUPPYc0s2Cb9Gyzpi7igaIT5ebr+pCgKo=
    -----END CERTIFICATE-----
    subject=CN = madeleine.michelis-amiens.lyc.ac-amiens.fr
    
    issuer=C = US, O = Let's Encrypt, CN = R3
    ---
    No client certificate CA names sent
    Peer signing digest: SHA256
    Peer signature type: RSA-PSS
    Server Temp Key: X25519, 253 bits
    ---
    SSL handshake has read 3581 bytes and written 433 bytes
    Verification: OK
    ---
    New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
    Server public key is 4096 bit
    Secure Renegotiation IS NOT supported
    Compression: NONE
    Expansion: NONE
    No ALPN negotiated
    Early data was not sent
    Verify return code: 0 (ok)
    ---
    DONE
    
     

Share This Page