HAProxy HTTP frontend and backend

Hi ,
We have an HAProxy setup running in Production for some time which supports access to Confluent Kafka cloud purpose in TCP for both Kafka brokers , port 9092 and Kafka Admin API , port 443 ,
as explained - both are TCP and that setup works for a while …
following is the snippet from the configuration -

frontend ccloud
mode tcp
bind *:9092
bind *:443
log global

    tcp-request inspect-delay 5s
    tcp-request content accept if { req.ssl_hello_type 1 }

**** there are several backends here additionally at the same format … brokers 1-8

backend broker1-nonprod
mode tcp
server broker1 b1-confluent.cloud:9092 check resolvers mynameservers init-addr last,libc,none
…
…
…
…
backend bootstrap
mode tcp
server bootstrap confluent.cloud:9092 check resolvers mynameservers init-addr last,libc,none

backend admin_api
mode tcp
server admin_api admin_api.confluent.cloud:443 check resolvers mynameservers init-addr last,libc,none

additionally to that we have a frontend for the stats API (again that’s also been running successfully for a while)

frontend stats
mode http
bind *:8404
http-request use-service prometheus-exporter if { path /metrics }
stats enable
stats uri /stats
stats refresh 20s

Now - we wanted to add support to use HAProxy to connect to a different backend in Confluent to support Rest API calls using HTTPS , note that the URL to connect is same URL as the other general bootstrap server URL for Confluent used in the TCP 9092 - for instance in the example here -
URL is - confluent.cloud

The difference is the protocol + port ,
as in the new addition we need to support that URL in port 443 HTTPS

So we added the following -

frontend ccloud_rest
mode http
bind *:443
use_backend rest_proxy_cloud

backend rest_proxy_cloud
server rest_proxy_cloud confluent.cloud:443 check resolvers mynameservers init-addr last,libc,none

Tried also to add several adjustments such as -
mode http - in the backend
or - default_backend rest_proxy_cloud
and some other adjustments …

Issue is that none of the attempt work in routing the request to the backend of the Rest-Proxy in the cloud -

• not seeing anything in the logs with that frontend / backend (anything that can be added to the configuration to add it to the logs) ?

• not getting a correct response when invoking a request to that cloud from the client -
  getting response - The underlying connection was closed: The connection was closed unexpectedly.

• when looking in the stats page - able to detect that there are incoming sessions in the frontend for that new frontend ccloud_rest , however in the backend not getting any sessions , does that say that the sessions are not being routed correctly to the appropriate backend ?

• on the same stats page , on the frontend - getting that the response is some 4XX HTTP response code , not getting the accurate code , only 4XX
  is there an option to tell the exact code or response returned by that call ?

Can anyone explain the reason why it’s not working or how to debug it based on the configuration that posted here ?

Thanks

1 post - 1 participant

Read full topic