Performance Degradation in libpq/pgbench Due to OpenSSL 3.x Global Lock Contention

Core Problem

When using pgbench --connect (which tests new connection establishment rate) against pgbouncer with SCRAM-SHA-256 authentication, performance drops catastrophically compared to connecting directly to postmaster:

• Direct to postmaster (SSL disabled): ~3,000 new connections/second, pgbench at ~25% CPU
• Via pgbouncer (SSL disabled, SCRAM auth): ~350 new connections/second, pgbench consuming 1000% CPU (10 cores saturated)

This is a 10x degradation — pgbouncer, which should be faster for connection pooling, actually makes the pgbench client itself the bottleneck. The root cause is global lock contention inside OpenSSL 3.x's provider architecture triggered by libpq's use of the legacy HMAC_Init_ex() API.

Architectural Root Cause

OpenSSL 3.x Provider Architecture Change

In OpenSSL 1.1.1 (e.g., RHEL8), HMAC_Init_ex() was a straightforward call with minimal locking. In OpenSSL 3.0+, this function was retrofitted for backward compatibility but now implicitly:

1. Queries OpenSSL's internal provider registry via EVP_MAC_fetch() and EVP_MD_fetch()
2. These fetch routines acquire an internal global read/write lock to locate the SHA-256 digest implementation
3. In a multi-threaded application like pgbench (10 threads), this creates a serialization point

Why pgbouncer Triggers It More Than postmaster

When connecting directly to postmaster, the server performs most of the SCRAM computation. The client-side SCRAM work is relatively light per connection, and the bottleneck is postmaster's fork-per-connection model (~3k/s ceiling).

With pgbouncer, the server-side connection establishment is nearly free (connections are pooled), so the client can attempt connections at a much higher rate. This exposes the client-side SCRAM computation as the new bottleneck — specifically the HMAC_Init_ex() calls inside pg_fe_sendauth() in libpq, which now contend on OpenSSL's global provider lock across all 10 pgbench threads.

Confirmation

Switching pgbouncer's auth_type to "plain" (bypassing SCRAM/HMAC entirely) yielded 58,000 new connections/second — demonstrating that the SCRAM/OpenSSL path is definitively the bottleneck, not pgbouncer itself.

Proposed Solutions and Design Tradeoffs

Ideal Fix: Migrate to EVP_MAC Provider-Aware API

The modern approach would be:

• Call EVP_MAC_fetch() + EVP_MAC_CTX_new() once (at initialization)
• Use EVP_MAC_CTX_dup() per connection/thread (avoids repeated provider lookups)

This eliminates repeated global lock acquisition by caching the provider lookup result.

Blocker: OpenSSL Version Support Policy

The reason libpq uses HMAC_Init_ex() today is historical — PostgreSQL needed to support OpenSSL down to 1.0.1, and the EVP_MAC API didn't exist until OpenSSL 3.0. The thread from 2020 (referenced by Jakub) explicitly chose the older interface for compatibility.

Path Forward: Drop OpenSSL 1.1.1 Support in PG20

Jacob Champion raises the possibility of dropping OpenSSL 1.1.1 support for PostgreSQL 20, which would make the API modernization straightforward. However, this creates a secondary challenge:

LibreSSL Complication

Daniel Gustafsson identifies a critical constraint: LibreSSL still uses the 1.1.1 API surface. Any migration to OpenSSL 3.x APIs requires first separating the OpenSSL and LibreSSL code paths. Daniel has a patch in progress to accomplish this separation, which would serve as groundwork for the modernization.

Impact Assessment

• Affected scope: Any heavily multi-threaded libpq client using SCRAM or MD5 authentication on OpenSSL 3.x systems (not just pgbench)
• Not affected: Systems using OpenSSL 1.1.1 (RHEL8 and similar legacy distributions)
• No workaround exists short of using a legacy OpenSSL version or plain-text authentication
• md5 auth is also affected since it also calls HMAC_Init_ex()

Current Status

This is filed as a known limitation report without a fix or patch. The resolution path requires:

1. Daniel's patch separating OpenSSL/LibreSSL code paths
2. A decision to drop OpenSSL 1.1.1 support (likely PG20)
3. Migration of HMAC code to the provider-aware EVP_MAC API