I’m getting a number of these per day, one burst every 5-10 minutes. I’ve been reluctant to change the SSL settings from standard to not risk angering the SSLLabs and other security metrics.
Compared to most, this system is not very busy, but has lots of many hours long connections vs millions on single transactions. We used to run haproxy with SSL pass thru. We converted to SSL termination in/out over the weekend and now are getting some reports that people can’t access the site, but haven’t gathered enough information to determine any commonalities or platforms or anything to debug with.
I have these settings in my global config for SSL:
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3 no-tlsv10 no-tls-tickets
tune.ssl.maxrecord 1460
tune.ssl.lifetime 600
tune.ssl.cachesize 1000000
tune.ssl.default-dh-param 2048
I do have HTTP/2 enabled.
All of the errors come up with a “/2” after the site name:
Oct 15 22:24:14 firehawk haproxy[5229]: 203.188.238.29:13882 [15/Oct/2020:22:24:14.443] www.example.com/2: SSL handshake failure
Oct 15 22:24:14 firehawk haproxy[5229]: 203.188.238.29:13882 [15/Oct/2020:22:24:14.443] www.example.com/2: SSL handshake failure
Oct 15 22:24:22 firehawk haproxy[5229]: 203.188.238.29:14945 [15/Oct/2020:22:24:22.001] www.example.com/2: SSL handshake failure
Oct 15 22:24:22 firehawk haproxy[5229]: 203.188.238.29:14945 [15/Oct/2020:22:24:22.001] www.example.com/2: SSL handshake failure
Oct 15 22:24:22 firehawk haproxy[5229]: 203.188.238.29:15073 [15/Oct/2020:22:24:22.794] www.example.com/2: SSL handshake failure
Oct 15 22:24:22 firehawk haproxy[5229]: 203.188.238.29:15073 [15/Oct/2020:22:24:22.794] www.example.com/2: SSL handshake failure
Oct 15 22:24:34 firehawk haproxy[5227]: 203.188.238.29:17370 [15/Oct/2020:22:24:33.670] www.example.com/2: SSL handshake failure
Oct 15 22:24:34 firehawk haproxy[5227]: 203.188.238.29:17370 [15/Oct/2020:22:24:33.670] www.example.com/2: SSL handshake failure
Oct 15 22:24:34 firehawk haproxy[5229]: 203.188.238.29:17543 [15/Oct/2020:22:24:34.458] www.example.com/2: SSL handshake failure
Oct 15 22:24:34 firehawk haproxy[5229]: 203.188.238.29:17543 [15/Oct/2020:22:24:34.458] www.example.com/2: SSL handshake failure
Which leads me to believe this is an HTTP/2 issue, but I don’t see why they wouldn’t renegotiate as HTTP/1.1. We have OCSP stapling enabled, SSLLabs gives us an “A” – so all the usual SSL issues should be in good working order.
I don’t know how to turn on a log for cipher mismatch, but I am looking the cipher used on successful logins. These connections are being shut so hard, I wish there was more info.
Please, any advice on where to look or how to identify the kinds of clients having the issue would be greatly appreciated!
thanks in advance!
haproxy -vv
HA-Proxy version 2.2.4-1ppa1~bionic 2020/10/02 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.2.4.html
Running on: Linux 4.15.0-118-generic #119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020 x86_64
Build options :
TARGET = linux-glibc
CPU = generic
CC = gcc
CFLAGS = -O2 -g -O2 -fdebug-prefix-map=/build/haproxy-couRLx/haproxy-2.2.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wall -Wextra -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-stringop-overflow -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference
OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_SYSTEMD=1
Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT +PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.1.1 11 Sep 2018
Running on OpenSSL version : OpenSSL 1.1.1 11 Sep 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.3
Built with network namespace support.
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity(“identity”), deflate(“deflate”), raw-deflate(“deflate”), gzip(“gzip”)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.31 2018-02-12
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 7.5.0
Built with the Prometheus exporter as a service
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as cannot be specified using ‘proto’ keyword)
fcgi : mode=HTTP side=BE mux=FCGI
: mode=HTTP side=FE|BE mux=H1
h2 : mode=HTTP side=FE|BE mux=H2
: mode=TCP side=FE|BE mux=PASS
Available services :
prometheus-exporter
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
[CACHE] cache
[FCGI] fcgi-app
1 post - 1 participant