Race Condition between two frontends

Hello,

I have configured two Frontend in my haproxy config. My service is running on ECS Container.

global
  maxconn 2000
  maxsessrate 700

frontend monitoring
  bind localhost:8080
  mode http
  maxconn 200
  monitor-uri /

frontend service
  mode tcp
  maxconn 1000
  rate-limit sessions 500

Monitoring Frontend been using to confirm that Haproxy running on ECS task is healthy, we are sending periodically traffic on localhost:8080 if the output is not 200 then we will kill the Task assuming Haproxy is not healthy and spin up a new one.

So while doing the load test on my ECS task, I’m sending high amount of traffic on service Frontend and due to that all queue is full with the traffic coming for service Frontend .
Since, queue is full connection request to monitoring Frontend is rejected or getting delayed. On this condition no response been received back from monitoring and due to that we mark ECS task as unhealthy.

My backend is only consuming ~500 connections.

Can someone help how I can I resolve this issue?

Thank You
Bhavesh

3 posts - 2 participants

Read full topic