Haproxy slow upload speed

so i ran into a really strange situation.

haproxy upload speed are insanely slow.

posting a 400MB video file to an php script takes around 1.3 minutes
this virtual machine is not receiving production traffic. isolated test setup. so CPU and RAM is all fine.

to make sure Haproxy was the issue and not network/backend server i installed nginx and configured it as a reverse proxy (with proxy_request_buffering off; ofcourse) then uploading the 400MB video file takes around 8 seconds… the network internaly and externaly is 2 gbit interface.

i tested it on haproxy 1.8 2.2 and 2.5 everywhere i have the same slow speeds.

haproxy:

  /         iface                   Rx                   Tx                Total
  ==============================================================================
             eth0:        4482.29 KB/s           22.86 KB/s         4505.15 KB/s
               lo:           1.50 KB/s            1.50 KB/s            3.00 KB/s
  ------------------------------------------------------------------------------
            total:        4483.79 KB/s           24.37 KB/s         4508.15 KB/s

and then nginx:

  |         iface                   Rx                   Tx                Total
  ==============================================================================
             eth0:       57731.82 KB/s          232.15 KB/s        57963.97 KB/s
               lo:           1.30 KB/s            1.30 KB/s            2.61 KB/s
  ------------------------------------------------------------------------------
            total:       57733.13 KB/s          233.45 KB/s        57966.58 KB/s

here is my config.

global
	log /dev/log	local0
	log /dev/log	local1 notice
	chroot /var/lib/haproxy
	stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
	stats timeout 30s
	user haproxy
	group haproxy
	daemon

	# Default SSL material locations
	ca-base /etc/ssl/certs
	crt-base /etc/ssl/private

	# Default ciphers to use on SSL-enabled listening sockets.
	# For more information, see ciphers(1SSL). This list is from:
	#  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
	# An alternative list with additional directives can be obtained from
	#  https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
	ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
	ssl-default-bind-options no-sslv3

	#CORS Module https://github.com/haproxytech/haproxy-lua-cors
	lua-load /etc/haproxy/cors.lua

defaults
	log	global
	mode	http
	option	dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
	errorfile 400 /etc/haproxy/errors/400.http
	errorfile 403 /etc/haproxy/errors/403.http
	errorfile 408 /etc/haproxy/errors/408.http
	errorfile 500 /etc/haproxy/errors/500.http
	errorfile 502 /etc/haproxy/errors/502.http
	errorfile 503 /etc/haproxy/errors/503.http
	errorfile 504 /etc/haproxy/errors/504.http

frontend http
    bind *:80
    mode http
    option  httplog
    default_backend servers-staging
    timeout client          1m
    redirect scheme https code 301 if !{ ssl_fc }

frontend https
    bind *:443 ssl crt /etc/haproxy/SSL/Bundles/ no-sslv3 alpn h2,http/1.1
    tcp-request content accept if { req_ssl_hello_type 1 }
    mode http
    option  httplog
    default_backend servers-staging
    timeout client          1m

    option forwardfor if-none

backend servers-staging
        option forwardfor if-none
        mode http
        timeout server 300s
        balance roundrobin
        option forwardfor if-none
        server localhost 10.0.0.233:8080

Please advice.

2 posts - 2 participants

Read full topic