HA Proxy - Gracefully close connections on servers

@bdudick-cocon wrote:

We are running into a snag in our deployment process. While removing servers from rotation, persistent connections are dropped. We are currently using cookie based persistence. We’d like the connections to gracefully move to another server, instead of being dropped.

Here is what we are doing to remove a server from rotation:

1. Change the state of the server to DRAIN (via socat command). This command disallows any new connections to be on the server, however persistent connections are still hitting our server.
2. Change “health.html” contents to “DOWN”. This marks the server as “DOWN”, but all connections are dropped and users bounced to another server.

We are unable to determine the step we are missing between #1 & #2. We have tried the following:

• Incorporating the “MAINT” status
• Setting the maxconn value on a server to -1
• Renaming the “health.html” file instead of changing the contents. This causes the server to be marked as “NOLB”

Does anyone have any suggestions?

Below is the HA Proxy config:

    global
        maxconn 30000
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
        stats timeout 30s
        user haproxy
        group haproxy
        daemon
        nbthread 48

        tune.bufsize 32768
        tune.ssl.cachesize 30000
        tune.ssl.lifetime  600

        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
        ssl-default-bind-options no-sslv3

        stats socket ipv4@127.0.0.1:9999 level admin
        stats socket /var/run/haproxy.sock mode 666 level admin

    defaults
            log     global
            mode    http
            option  httplog
            option  dontlognull
            timeout connect 121000
            timeout client  121000
            timeout server  121000
            errorfile 400 /etc/haproxy/errors/400.http
            errorfile 403 /etc/haproxy/errors/403.http
            errorfile 408 /etc/haproxy/errors/408.http
            errorfile 500 /etc/haproxy/errors/500.http
            errorfile 502 /etc/haproxy/errors/502.http
            errorfile 503 /etc/haproxy/errors/503.http
            errorfile 504 /etc/haproxy/errors/504.http

    frontend fe_main
            bind :80
            bind :443 ssl crt /etc/cc-ssl/[redacted].pem crt /etc/cc-ssl/[redacted].pem
            reqadd X-Forwarded-Proto:\ https

            http-request redirect scheme https unless { ssl_fc }

            default_backend be-https

    frontend stats
            bind *:8404
            stats enable
            stats uri /stats

    backend be-https
            balance roundrobin
            cookie NUMID insert indirect nocache
            option httpchk GET /health.html HTTP/1.1\r\nHost:\ www
            http-check disable-on-404
            http-check expect string UP
            default-server inter 3s fall 2 rise 2 slowstart 5m
            server s1 10.10.10.1:443 ssl verify none check cookie 1
            server s2 10.10.10.2:443 ssl verify none check cookie 2
            server s3 10.10.10.3:443 ssl verify none check cookie 3
            server s4 10.10.10.4:443 ssl verify none check cookie 4

Posts: 2

Participants: 1

Read full topic