HAProxy + Pfsense: Combining modes TCP and HTTP for SSL

Hello!

I’ve spent quite a bit of time between a rock and a hard place trying to be able to connect with SSL to a couple of services running on my home LAN using HAProxy on Pfsense in TCP mode. I have no issues with HTTPS connections (accessing web front-ends) in HTTP mode, but I cannot for the life of me figure out how to get the TCP mode to work in conjunction with HTTP mode.

I have read this post, which at face value seems to be what I’m going for, but I am doing everything in the Pfsense UI and am really struggling to figure out what I’m supposed to do, and where, when reading the config output. Ideally, a kind soul can help me figure out what to do in the UI to get this working.

Current setup:

• I have a 2 servers running in a separate different VLANs for different purposes: pihole, proxmox, etc.
• Created wildcard certificate for mydomain.com / *.mydomain.com in ACME
• Set up HAProxy backends for these servers + frontend with SSL offloading
• I can access https://pihole.mydomain.com, https://proxmox.mydomain.com, etc. all with zero issues — happy me… except for this new problem:
• 1 service does not use HTTP mode; I need to connect to it using TCP mode i.e. ssl://node.mydomain.com:50002

I can use my desktop software to connect directly to the server today by typing the IPv4 ssl://192.168.16.3 on port 50002 and the connection is successful + traffic is encrypted using a self-signed certificate on the server.

However, I want to connect to ssl://node.mydomain.com:50002via a reverse proxy for SSL termination, as the mobile apps I use to connect require a valid SSL certificate. I’ve thus ignored the self-signed certificate and am connecting at tcp://192.168.16.3:50001 on my LAN. fwiw I only want to do all of this on LAN; I’m not planning on making these servers available on the Internet, but it doesn’t change the requirement of these apps to use a valid SSL certificate.

I’ve set up HAProxy backend to point index to 192.168.16.3 on port 50001 with Encrypt (SSL) unchecked and SSL checks unchecked:

On the frontend, I already have the Listen Address for the VLAN interfaces on port 443 (for the services with webUIs), so I added a new frontend, chose the relevant VLAN interface, set port 50002 and SSL offloading checked:

This current setup doesn’t work. I cannot connect to the server in the apps; I receive a “connection failed: illegal state error (I/O error during request processing)” message. Per the thread referenced above, it seems I need to do something about capturing the tcp traffic and redirecting it… I’m not sure.

It’s been several weeks that I’m fiddling around here with this. Any help would be much appreciated!

Below is the output of my HAProxy config (sensitive parts marked [redacted]):

# Automaticaly generated, dont edit manually.
# Generated on: 2024-10-13 13:26
global
	maxconn			100
	log			/var/run/log	local0	debug
	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
	uid			80
	gid			80
	nbthread			1
	hard-stop-after		15m
	chroot				/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	4096
	log-send-hostname		HAProxy[redacted]
	server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
	bind 127.0.0.1:2200 name localstats
	mode http
	stats enable
	stats refresh 10
	stats admin if TRUE
	stats show-legends
	stats uri /haproxy/haproxy_stats.php?haproxystats=1
	timeout client 5000
	timeout connect 5000
	timeout server 5000

resolvers globalresolvers
	nameserver Pihole 192.168.88.2:53
	resolve_retries 3
	timeout retry 1s
	timeout resolve 10s

frontend [redacted]
	bind			192.168.88.1:443 name 192.168.88.1:443   ssl crt-list /var/etc/haproxy/[redacted].crt_list  
	bind			192.168.16.1:443 name 192.168.16.1:443   ssl crt-list /var/etc/haproxy/[redacted].crt_list  
	mode			http
	log			global
	option			socket-stats
	option			http-keep-alive
	timeout client		30000
	acl			pi	var(txn.txnhost) -m str -i pi.[redacted]
	acl			pxmx	var(txn.txnhost) -m str -i pxmx.[redacted]
	acl			mp	var(txn.txnhost) -m str -i mp.[redacted]
	acl			[redacted]	var(txn.txnhost) -m str -i [redacted].[redacted]
	acl			aclcrt_[redacted]	var(txn.txnhost) -m reg -i ^([^\.]*)\.[redacted](:([0-9]){1,5})?$
	http-request set-var(txn.txnhost) hdr(host)
	use_backend pi_ipvANY  if  pi aclcrt_[redacted]
	use_backend pxmx_ipvANY  if  pxmx aclcrt_[redacted]
	use_backend mp_ipvANY  if  mp aclcrt_[redacted]
	use_backend [redacted]_ipvANY  if  [redacted] aclcrt_[redacted]

frontend [redacted]_TCP
	bind			192.168.16.1:50002 name 192.168.16.1:50002   ssl crt-list /var/etc/haproxy/[redacted]_TCP.crt_list  
	mode			tcp
	log			global
	timeout client		30000
	tcp-request inspect-delay	5s
	acl			node	req.ssl_ver gt 0
	tcp-request content accept if { req.ssl_ver gt 0 }
	use_backend node_ipvANY  if  node 

backend pi_ipvANY
	mode			http
	id			100
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	load-server-state-from-file	global
	server			pi 192.168.88.2:80 id 101 check inter 1000  resolvers globalresolvers 

backend pxmx_ipvANY
	mode			http
	id			104
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	load-server-state-from-file	global
	server			pxmx 192.168.16.2:8006 id 101 ssl check inter 1000  verify none resolvers globalresolvers 

backend mp_ipvANY
	mode			http
	id			107
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	load-server-state-from-file	global
	server			mp 192.168.16.3:4081 id 108 ssl check inter 1000  verify none resolvers globalresolvers 

backend [redacted]_ipvANY
	mode			http
	id			109
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	load-server-state-from-file	global
	server			[redacted] 192.168.16.3:4001 id 110 ssl check inter 1000  verify none resolvers globalresolvers 

backend node_ipvANY
	mode			tcp
	id			102
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	load-server-state-from-file	global
	server			node 192.168.16.3:50001 id 103  resolvers globalresolvers

1 post - 1 participant

Read full topic