@pkaroluk wrote:
Hi, When I chose url_param balance method I got an “Error in the HTTP2 framing layer”, but in http1.1 it works. Whereas when I change the balancing mode, e.g. roundrobin instead url_param then it’s okay so probably it’s an issue of usage url_param in http2. I checked several versions of 1.8.3 -> 1.8.8 and it is the same.
# haproxy -vv HA-Proxy version 1.8.8-1.el7 2018/04/19 Copyright 2000-2018 Willy Tarreau <willy@haproxy.org> Build options : TARGET = linux2628 CPU = generic CC = gcc CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -fno-strict-overflow -Wno-unused-label OPTIONS = USE_ZLIB=1 USE_DL=1 USE_OPENSSL=1 USE_LUA=1 USE_SYSTEMD=1 USE_PCRE=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 Built with OpenSSL version : OpenSSL 1.0.2k-fips 26 Jan 2017 Running on OpenSSL version : OpenSSL 1.0.2k-fips 26 Jan 2017 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2 Built with Lua version : Lua 5.3.4 Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND Encrypted password support via crypt(3): yes Built with multi-threading support. Built with PCRE version : 8.32 2012-11-30 Running on PCRE version : 8.32 2012-11-30 PCRE library supports JIT : no (USE_PCRE_JIT not set) Built with zlib version : 1.2.7 Running on zlib version : 1.2.7 Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip") Built with network namespace support. Available polling systems : epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use epoll. Available filters : [SPOE] spoe [COMP] compression [TRACE] trace
Sample config:
# /etc/haproxy/haproxy.cfg global chroot /var/lib/haproxy daemon group haproxy log 127.0.0.1 local2 debug maxconn 210000 nbproc 2 pidfile /var/run/haproxy.pid spread-checks 2 ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!DHE:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl-default-bind-options ssl-min-ver TLSv1.0 no-tls-tickets stats socket /var/lib/haproxy/stats uid 0 gid 0 mode 0440 process 1 stats bind-process all tune.bufsize 16384 tune.h2.max-concurrent-streams 100 tune.maxrewrite 1024 tune.ssl.cachesize 100000 tune.ssl.default-dh-param 2048 tune.ssl.lifetime 600 ulimit-n 500000 user haproxy defaults log global maxconn 120000 mode http option redispatch option http-server-close option dontlognull retries 3 stats enable timeout http-request 302s timeout queue 60s timeout connect 5s timeout client 302s timeout server 302s timeout check 1s frontend https bind 0.0.0.0:443 ssl crt /etc/ssl/server.pem alpn h2,http/1.1 bind 0.0.0.0:80 mode http bind-process 1 default_backend http2_backend maxconn 120000 option httplog option http-server-close option forwardfor except 127.0.0.0/8 timeout client 30s tcp-request inspect-delay 5s tcp-request content accept if { req_ssl_hello_type 1 } reqadd X-Forwarded-Proto:\ https reqadd Https:\ on backend http2_backend mode http balance url_param bh bind-process 1 hash-type consistent option redispatch option http-server-close server http1.1 127.0.0.1:81 check listen stats bind 10.254.0.10:888 process 1 mode http stats uri / stats enable stats show-node stats refresh 20s stats show-legends
Requests in url_param:
# curl -v -I -k --http2 -XGET https://10.254.0.10/index.html * Trying 10.254.0.10... * TCP_NODELAY set * Connected to 10.254.0.10 (10.254.0.10) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Client hello (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: C=US; ST=CA; L=LA; O=ACME Inc.; OU=IT; CN=example.com * start date: May 9 21:03:10 2018 GMT * expire date: May 9 21:03:10 2019 GMT * issuer: C=US; ST=CA; L=LA; O=ACME Inc.; OU=IT; CN=example.com * SSL certificate verify result: self signed certificate (18), continuing anyway. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x7ffe1e000000) > GET /index.html HTTP/2 > Host: 10.254.0.10 > User-Agent: curl/7.54.0 > Accept: */* > * Connection state changed (MAX_CONCURRENT_STREAMS updated)! * Closing connection 0 * TLSv1.2 (OUT), TLS alert, Client hello (1): curl: (16) Error in the HTTP2 framing layer # curl -v -I -k --http1.1 -XGET https://10.254.0.10/index.html * Trying 10.254.0.10... * TCP_NODELAY set * Connected to 10.254.0.10 (10.254.0.10) port 443 (#0) * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Client hello (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use http/1.1 * Server certificate: * subject: C=US; ST=CA; L=LA; O=ACME Inc.; OU=IT; CN=example.com * start date: May 9 21:03:10 2018 GMT * expire date: May 9 21:03:10 2019 GMT * issuer: C=US; ST=CA; L=LA; O=ACME Inc.; OU=IT; CN=example.com * SSL certificate verify result: self signed certificate (18), continuing anyway. > GET /index.html HTTP/1.1 > Host: 10.254.0.10 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK HTTP/1.1 200 OK < Date: Wed, 09 May 2018 22:28:55 GMT Date: Wed, 09 May 2018 22:28:55 GMT < Server: Apache/2.4.6 (CentOS) Server: Apache/2.4.6 (CentOS) < Last-Modified: Wed, 09 May 2018 21:57:36 GMT Last-Modified: Wed, 09 May 2018 21:57:36 GMT < ETag: "1ca-56bccfda5fc1c" ETag: "1ca-56bccfda5fc1c" < Accept-Ranges: bytes Accept-Ranges: bytes < Content-Length: 458 Content-Length: 458 < Content-Type: text/html; charset=UTF-8 Content-Type: text/html; charset=UTF-8 < * Excess found in a non pipelined read: excess = 458 url = /index.html (zero-length body) * Connection #0 to host 10.254.0.10 left intact
Dump from h2c:
-> SETTINGS(0) - ACK SETTINGS_INITIAL_WINDOW_SIZE: 1073741824 SETTINGS_ENABLE_PUSH: 0 SETTINGS_MAX_CONCURRENT_STREAMS: 100 <- SETTINGS(0) - ACK SETTINGS_MAX_CONCURRENT_STREAMS: 100 -> WINDOW_UPDATE(0) Window size increment: 1073676289 <- SETTINGS(0) + ACK {empty} -> HEADERS(1) + END_STREAM + END_HEADERS :method: GET :path: /index.html?bh=1233 :scheme: https :authority: 10.254.0.10:443 user-agent: curl/7.54.0 accept: */* -> SETTINGS(0) + ACK {empty} <- RST_STREAM(1) Error code: NO_ERROR <- GOAWAY(0) Last stream id: 1 Error code: NO_ERROR Error while reading next frame: EOF Closing connection.
After changing to roundrobin:
# curl -v -I -k --http2 -XGET https://10.254.0.10/index.html * Trying 10.254.0.10... * TCP_NODELAY set * Connected to 10.254.0.10 (10.254.0.10) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Client hello (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: C=US; ST=CA; L=LA; O=ACME Inc.; OU=IT; CN=example.com * start date: May 9 21:03:10 2018 GMT * expire date: May 9 21:03:10 2019 GMT * issuer: C=US; ST=CA; L=LA; O=ACME Inc.; OU=IT; CN=example.com * SSL certificate verify result: self signed certificate (18), continuing anyway. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x7f845e80d800) > GET /index.html HTTP/2 > Host: 10.254.0.10 > User-Agent: curl/7.54.0 > Accept: */* > * Connection state changed (MAX_CONCURRENT_STREAMS updated)! < HTTP/2 200 HTTP/2 200 < date: Wed, 09 May 2018 22:49:11 GMT date: Wed, 09 May 2018 22:49:11 GMT < server: Apache/2.4.6 (CentOS) server: Apache/2.4.6 (CentOS) < last-modified: Wed, 09 May 2018 21:57:36 GMT last-modified: Wed, 09 May 2018 21:57:36 GMT < etag: "1ca-56bccfda5fc1c" etag: "1ca-56bccfda5fc1c" < accept-language: bytes accept-language: bytes < content-length: 458 content-length: 458 < content-type: text/html; charset=UTF-8 content-type: text/html; charset=UTF-8 < * Excess found in a non pipelined read: excess = 458 url = /index.html (zero-length body) * Connection #0 to host 10.254.0.10 left intact # curl -v -I -k --http1.1 -XGET https://10.254.0.10/index.html * Trying 10.254.0.10... * TCP_NODELAY set * Connected to 10.254.0.10 (10.254.0.10) port 443 (#0) * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Client hello (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use http/1.1 * Server certificate: * subject: C=US; ST=CA; L=LA; O=ACME Inc.; OU=IT; CN=example.com * start date: May 9 21:03:10 2018 GMT * expire date: May 9 21:03:10 2019 GMT * issuer: C=US; ST=CA; L=LA; O=ACME Inc.; OU=IT; CN=example.com * SSL certificate verify result: self signed certificate (18), continuing anyway. > GET /index.html HTTP/1.1 > Host: 10.254.0.10 > User-Agent: curl/7.54.0 > Accept: */* > < HTTP/1.1 200 OK HTTP/1.1 200 OK < Date: Wed, 09 May 2018 22:49:08 GMT Date: Wed, 09 May 2018 22:49:08 GMT < Server: Apache/2.4.6 (CentOS) Server: Apache/2.4.6 (CentOS) < Last-Modified: Wed, 09 May 2018 21:57:36 GMT Last-Modified: Wed, 09 May 2018 21:57:36 GMT < ETag: "1ca-56bccfda5fc1c" ETag: "1ca-56bccfda5fc1c" < Accept-Ranges: bytes Accept-Ranges: bytes < Content-Length: 458 Content-Length: 458 < Content-Type: text/html; charset=UTF-8 Content-Type: text/html; charset=UTF-8 < * Excess found in a non pipelined read: excess = 458 url = /index.html (zero-length body) * Connection #0 to host 10.254.0.10 left intact
Dump from h2c:
-> SETTINGS(0) - ACK SETTINGS_MAX_CONCURRENT_STREAMS: 100 SETTINGS_INITIAL_WINDOW_SIZE: 1073741824 SETTINGS_ENABLE_PUSH: 0 -> WINDOW_UPDATE(0) Window size increment: 1073676289 -> HEADERS(1) + END_STREAM + END_HEADERS :method: GET :path: /index.html?bh=1233 :scheme: https :authority: 10.254.0.10:443 user-agent: curl/7.54.0 accept: */* <- SETTINGS(0) - ACK SETTINGS_MAX_CONCURRENT_STREAMS: 100 -> SETTINGS(0) + ACK {empty} <- SETTINGS(0) + ACK {empty} <- HEADERS(1) - END_STREAM + END_HEADERS :status: 200 date: Thu, 10 May 2018 15:18:03 GMT server: Apache/2.4.6 (CentOS) last-modified: Wed, 09 May 2018 21:57:36 GMT etag: "1ca-56bccfda5fc1c" accept-language: bytes content-length: 458 content-type: text/html; charset=UTF-8 <- DATA(1) + END_STREAM {458 bytes} Error while reading next frame: EOF Closing connection.
And some logs in http1.1/http2:
http2_backend/http1.1 0/0/0/1/1 200 720 - - ---- 1/1/0/0/0 0/0 "GET /index.html?bh=1 HTTP/1.1" May 10 01:49:57 localhost haproxy[24227]: 10.254.0.1:64065 [10/May/2018:01:49:57.403] https~ http2_backend/<NOSRV> 0/-1/-1/-1/0 -1 0 - - CH-- 1/1/0/0/3 0/0 "GET /index.html?bh=1 HTTP/1.1" May 10 01:49:57 localhost haproxy[24227]: Connect from 10.254.0.1:64066 to 10.254.0.10:888 (stats/HTTP) May 10 01:50:03 localhost haproxy[24227]: 10.254.0.1:64087 [10/May/2018:01:50:03.132] https~ http2_backend/<NOSRV> 0/-1/-1/-1/0 -1 0 - - CH-- 2/1/0/0/3 0/0 "GET /index.html?bh=1 HTTP/1.1" May 10 01:50:25 localhost haproxy[24227]: 10.254.0.1:64190 [10/May/2018:01:50:25.344] https~ http2_backend/<NOSRV> 0/-1/-1/-1/0 -1 0 - - CH-- 2/1/0/0/3 0/0 "GET /index.html?bh=12 HTTP/1.1"
Has anyone ever used url_param balance method in http2 traffic?
Thanks
Pawel
Posts: 1
Participants: 1