Performance problem with nbthread

@Vlad wrote:

Hi all,

I need an advice regarding multithreading configuration.

We are using haproxy 2.0.12 on a CentOS8 virtual machine (VMware) with 16 GB of RAM, 4 vCPUs (1 core each) and 1GBPs NIC. This setup has only one frontend (http mode, SSL only) and two backends (http). Currently, the traffic is quite small: we saw maximum up to 400 concurrent connections, with maximal request rate of 42 connections/sec and maximal consumed network bandwidth of 10 MBit/s. But in future we need to handle up to ~5000 concurrent connections, maybe 10000.

And here begins the problem: with the current setup, haproxy consumes up to 35% of CPU power when nbproc 1 and nbthread 4 are set. As soon as I comment out nbthread line and switch to nbproc 4, the CPU load disappears completely: maximum 2% of all 4 CPU’s power is used by haproxy.

I would leave it “as is” with nbproc, but it causes some problems with “independent” stick tables, dedicated stats page for each process etc… So I definitely need to use multithreading.

Here is our config:

global
    maxconn         10000
    stats socket    /var/run/haproxy.stat mode 600 level admin
    log             127.0.0.1:514 local2
    chroot          /var/empty
    pidfile         /var/run/haproxy.pid
    user            haproxy
    group           haproxy
    ssl-default-bind-options no-tlsv13
    ssl-default-bind-ciphers 'HIGH:!aNULL:!MD5'
    tune.ssl.default-dh-param 4096
    tune.ssl.cachesize 1000000
    tune.ssl.lifetime 600
    tune.ssl.maxrecord 1460
    nbproc 1
    nbthread 4
    daemon

defaults
    option contstats
    retries 3 

frontend WEB
    bind            192.168.0.25:80
    bind            192.168.0.25:443 ssl crt /Certs/domain1.pem crt /Certs/domain2.pem
    mode            http
    timeout         http-request 5s 
    timeout         client 30s
    log             global
    option          httplog
    option          dontlognull
    option          forwardfor
    monitor-uri     /healthcheck
    maxconn         8000
    timeout client  30s
    http-request capture req.hdr(Host) len 20

    %%%Some ACLs are defined here%%%

    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"
    http-response set-header X-Frame-Options "SAMEORIGIN"
    http-response set-header X-XSS-Protection "1; mode=block"
    http-response set-header X-Content-Type-Options "nosniff"
    http-response set-header X-Permitted-Cross-Domain-Policies "none"
    http-response set-header X-Robots-Tag "all"
    http-response set-header X-Download-Options "noopen"

    # Do not allow more than 10 concurrent tcp connections per IP, or 15 connections in 3 seconds
    tcp-request content reject if { src_conn_rate(Abuse) ge 15 }
    tcp-request content reject if { src_conn_cur(Abuse) ge 10 }
    tcp-request connection track-sc1 src table Abuse

    # Redirect HTTP to HTTPS
    redirect        scheme https code 301 if !{ ssl_fc } 
    default_backend Web-Pool


backend Web-Pool
    mode            http
    balance         roundrobin
    retries         2
    option redispatch
    timeout connect 5s
    timeout server  30s
    timeout queue   30s
    option forwardfor
    option httpchk  HEAD /
    http-check      expect status 200
    cookie          DYNSRV insert indirect nocache
    fullconn        4000 
    http-request set-header X-Client-IP %[src]
    server          httpd01 192.168.0.30:80 check weight 1 inter 2000 rise 2 fall 2 minconn 0 maxconn 0 on-marked-down shutdown-sessions
    server          httpd02 192.168.0.31:80 check weight 2 inter 2000 rise 2 fall 2 minconn 0 maxconn 0 on-marked-down shutdown-sessions

backend Abuse
    stick-table type ip size 1m expire 30m store conn_rate(3s),conn_cur,gpc0,http_req_rate(10s),http_err_rate(20s)

With multi-process config, I use the following settings:
nbproc 4
cpu-map 1 0
cpu-map 2 1
cpu-map 3 2
cpu-map 4 3

I believe something is just wrong in my configuration… Could anybody help me to find the cause of this problem?

Thank you.

Posts: 1

Participants: 1

Read full topic