Is there a way to use a forwarding proxy as a backend (including authentication) in HAProxy?

Question

I am quite new to HAProxy and want to achieve the following setup / packet flow:

Client -> HAProxy (as reverse proxy) -> Forwarding Proxy (HAProxy, IIS, Squid...) -> Internet -> server.example.com

I would like to have encrypted connections with TLS/SSL from the Client -> HAProxy and from HAProxy -> server.example.com

This means, that the forwarding proxy needs to support the HTTP CONNECT method, to establish a TCP tunnel and transmits packets without trying to interpret them. Over this TCP port I should be able to send bytes to server.example.com - TLS/SSL encrypted, so HTTPS.

Furthermore it could be, that authentication against the forwarding proxy is needed e.g. HTTP Basic authentication.

The software stack of my test setup is as follows:

Client Firefox 76.0.1
HA-Proxy version 1.6.3 2015/12/25
Squid Cache: Version 3.5.27

I have setup this HAProxy configuration:

global
        # Standard settings                                                                                                                                                                                        
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        user haproxy
        group haproxy
        daemon

        # Tuning                                                                                                                                                                                                   
        maxconn 2000

        # Default SSL material locations                                                                                                                                                                           
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        ssl-default-bind-options no-sslv3

        # Ensure a secure enough DH paramset                                                                                                                                                                       
        tune.ssl.default-dh-param 2048


defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        # Redispatch 503 errors on backends, reducing the number of 503 reaching clients                                                                                                                           
        option  redispatch
        # We want to stall clients for as long as possible, before giving                                                                                                                                          
        # up with 503:                                                                                                                                                                                             
        timeout connect 5m
        # Clients must be acceptably responsive                                                                                                                                                                    
        timeout client  1m
        # Server not as much...                                                                                                                                                                                    
        timeout server  5m


# HTTPS server                                                                                                                                                                                                     

frontend https-in                      
        bind :443 ssl crt-ignore-err all crt /etc/haproxy/ssl/certkey.pem

        # Don't serve HTTP directly, but redirect to same URL in https                                                                                                                                             
        redirect scheme https code 301 if !{ ssl_fc }

        default_backend backend-proxy

backend backend-proxy
        # Create the Authorization / Proxy-Authorization header value                                                                                                                                              
        # echo -n "user:password" | base64                                                                                                                                                                         

        http-request add-header Proxy-Authorization "Basic dXNlcjpwYXNzd29yZA=="
                                                                                                                                                                         
        # We need to use the CONNECT method                                                                                                                                                                        
        http-request set-method CONNECT

        # The proxyserver needs to know the full server name                                                                                                                                                       
        http-request set-path server.example.com:443

        server proxy 192.168.1.1:8080

In my test setup I use a Squid server as a forwarding proxy with the following configuration:

acl SSL_ports port 443

acl Safe_ports port 80          # http
acl Safe_ports port 443         # https

acl CONNECT method CONNECT

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost

auth_param basic program /usr/lib/squid3/basic_ncsa_auth /etc/squid/squid-passwd
acl basic_client proxy_auth REQUIRED
http_access deny !basic_client
http_access allow basic_client
http_access deny all

http_port 8080

coredump_dir /var/spool/squid

Using the Squid forwarding proxy in a regular browser on the same subnet including authentication works fine.

So when the first request comes in from the Client at the HAProxy it gets forwarded via the backend backend-proxy to the Forward Proxy (Squid). The CONNECT succeeds, as I see in the Squid log.

1591166403.966  60146 192.168.1.10 TCP_TUNNEL/200 39 CONNECT server.example.com:443 test HIER_DIRECT/6.7.8.9 -

(IP addresses were replaced with generic values)

The HAProxy log shows, that the correct backend is used:

Jun  3 08:39:57 localhost haproxy[3547]: 192.168.1.20:39398 [03/Jun/2020:08:38:56.855] https-in~ backend-proxy/proxy 154/0/13/209/60375 200 39 - - ---- 0/0/0/0/0 0/0 "GET /someurl HTTP/1.1"

(IP addresses were replaced with generic values)

So far so good. But I am unable to establish a successful communication to server.example.com from my Client. I think I have to use a second/other backend, which will not mangle the requests any more (exchange method and path) but instead use the given TCP port from the forwarding proxy to transmit the request.

How can I save the 'state' of the communication to my backend / proxy server in HAProxy, so the request could be resend to another backend?

How to extract and use the TCP port from the response of the Forwarding Proxy?

Is there a way to check, if the TCP tunnel on the Forwarding Proxy is still opened or do I need to request it using CONNECT every time before I want to use it?

EDIT: I solved the situation by using stunnel as an intermediary to handle the TCP tunnel creation with CONNECT against the Forwarding Proxy.

Answer 1

If you have an upstream HTTP proxy (like squid) (not a socks proxy) and you want to have haproxy accept connections and open the tunnel thru the upstream proxy (such that the haproxy clients do not support doing so themselves via http/CONNECT method) on behalf of the clients, then this functionality does not exist in haproxy today.

I crated a branch that does this via server keyword proxy-tunnel.

The below example config will behave such that clients of haproxy that connect to port 20025 (haproxy) will result in haproxy establishing a http/CONNECT tunnel via the upstream proxy 172.16.0.99:50443 to 172.16.0.2:2023:

listen SMTP-20025
    bind 0.0.0.0:20025
    server TEST_SERVERVIA_PROXY 172.16.0.2:2023 proxy-tunnel 172.16.0.99:50443