Haproxy And Percona Cluster Transaction Commit Failing

@zebidiah wrote:

Hi,

I have an issue with my loadbalanced percona setup. The issue only rears its head when there is a large amount of data being inserted into the database during a transaction, it fails at the commit stage, it seems as if the connection times out but it happens no matter what I raise the timeouts to. There is no issue when I hit one of the percona nodes directly, and I have set the haproxy config to write to 1 node only. As it works when I hit the percona node directly it seems the issue is with my haproxy configuration. Here it is, any help greatly appreciated:

global
        log /dev/log    local0
        log /dev/log    local1 notice
        maxconn 4096
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
        stats timeout 30s
        user haproxy
        group haproxy
        daemon

        # Default SSL material locations
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        # Default ciphers to use on SSL-enabled listening sockets.
        # For more information, see ciphers(1SSL). This list is from:
        #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
        # An alternative list with additional directives can be obtained from
        #  https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
        ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
        ssl-default-bind-options no-sslv3

defaults
        log     global
        mode    tcp
        option  tcplog
        option  dontlognull
        option  tcp-smart-accept
        option  tcp-smart-connect
        retries 3
        option  redispatch
        maxconn 40000
        timeout check   3500ms
        timeout queue   3500ms
        timeout connect 3500ms
        timeout client  10000ms
        timeout server  10000ms

listen stats
        bind 10.100.2.11:9000
        mode http
        stats enable
        stats hide-version
        stats realm HAproxy-Statistics
        stats uri /haproxy_stats
        stats auth 

listen percona_cluster_write
        bind 10.100.2.11:3306
        mode tcp
        timeout client 1m
        timeout server 1m
        balance leastconn
        option mysql-check
        server DB01-QA 10.100.3.11:3306 send-proxy check inter 12000 rise 3 fall 3
        server DB01-QA 10.100.3.12:3306 send-proxy check inter 12000 rise 3 fall 3 backup
        server DB01-QA 10.100.3.13:3306 send-proxy check inter 12000 rise 3 fall 3 backup

listen percona_cluster_read
        bind 10.100.2.11:3307
        mode tcp
        balance leastconn
        option mysql-check
        server DB01-QA 10.100.3.11:3306 send-proxy check inter 12000 rise 3 fall 3
        server DB02-QA 10.100.3.12:3306 send-proxy check inter 12000 rise 3 fall 3
        server DB03-QA 10.100.3.13:3306 send-proxy check inter 12000 rise 3 fall 3

Here is error I get from application:

Fatal error encountered during command execution.
MySql.Data.MySqlClient.MySqlException (0x80004005): Fatal error encountered during command execution. —> MySql.Data.MySqlClient.MySqlException (0x80004005): Fatal error encountered attempting to read the resultset. —> MySql.Data.MySqlClient.MySqlException (0x80004005): Reading from the stream has failed. —> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count)
at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
at MySql.Data.MySqlClient.NativeDriver.GetResult(Int32& affectedRow, Int64& insertedId)
at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, Boolean force)
at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
at MySql.Data.MySqlClient.MySqlCommand.ExecuteNonQuery()
at MySql.Data.MySqlClient.MySqlTransaction.Commit()

Posts: 1

Participants: 1

Read full topic