Technical Analysis: Use ssup_datum_*_cmp for int2, oid, and oid8 Sort Support

Core Problem

PostgreSQL's tuplesort infrastructure includes a radix sort fast path (commit ef3c3cf6d02) that provides significant speedups for in-memory sorts. However, this optimization dispatches based on function pointer identity — it checks whether the comparator assigned to a sort support state is one of the known ssup_datum_int32_cmp or ssup_datum_unsigned_cmp functions. If a type uses a functionally equivalent but distinct local comparator, the radix sort path is never triggered, and the sort falls back to the general-purpose comparison-based quicksort.

The types int2, oid, and oid8 all define custom local fastcmp helper functions in nbtcompare.c that are semantically identical to the canonical helpers but are distinct function pointers:

• int2 uses a local comparator that performs signed 32-bit comparison — identical to ssup_datum_int32_cmp
• oid uses a local comparator that performs unsigned datum comparison — identical to ssup_datum_unsigned_cmp
• oid8 (a newer type added January 2026) inherited the same pattern from oid

This is a gap left over from the original 2021-2022 development of the ssup_datum_*_cmp helpers (commit 6974924347c, April 2022), which covered int4, int8, timestamp, date, and abbreviated-key types but did not address int2 or oid. The oid8 type was added after that work but before radix sort landed, and copied the existing oid pattern.

Why This Matters Architecturally

Radix Sort Dispatch Mechanism

The radix sort optimization in tuplesort works by recognizing specific comparator function pointers at sort initialization time. This design was chosen because:

1. It avoids adding new fields to the sort support state structure
2. It guarantees the radix sort implementation matches the comparison semantics
3. It's zero-cost when not triggered (just a pointer comparison)

The downside is that any type using a functionally-identical-but-distinct comparator silently misses the optimization with no warning. This is exactly the situation this patch addresses.

int2 and ssup_datum_int32_cmp

The choice to use ssup_datum_int32_cmp (rather than a hypothetical ssup_datum_int16_cmp) for int2 is deliberate and correct:

• Every int16 value fits losslessly in a Datum-width value after sign extension
• int32_cmp triggers 4-byte radix passes rather than the 8-byte passes that ssup_datum_signed_cmp would use
• This matches the approach already used for int4, providing consistent performance characteristics

Performance Impact

The benchmarks show 27-28% speedup on 10M-row single-key sorts, bringing int2 and oid performance in line with int4 and int8 baselines. This is a substantial improvement for what is essentially a one-line change per type (replacing the comparator assignment).

Proposed Solution

The patch is minimal and surgical:

1. Replace comparator assignments: In each type's _sortsupport() function, replace the assignment of the local fastcmp function with the canonical helper (ssup_datum_int32_cmp for int2, ssup_datum_unsigned_cmp for oid and oid8)

2. Remove dead code: Delete the now-unused local fastcmp helper functions from nbtcompare.c

3. No behavioral change: The helpers produce identical comparison results; the only difference is that the radix sort path now recognizes these types as eligible

Design Tradeoffs

Why not all types?

The author explicitly notes that float4/float8 and varlena types cannot be trivially switched because:

• Float types require special NaN handling in comparisons
• Varlena types have locale-dependent comparison semantics

These types would need more invasive changes to their sort support to participate in radix sort.

Risk Assessment

The risk is extremely low:

• The canonical helpers are already well-tested through int4, int8, timestamp, etc.
• The local helpers being replaced are provably identical in behavior
• Full regression test suite passes with no changes needed to expected outputs

Open Questions

This is a new patch submission with no responses yet, so several questions remain:

1. Whether a committer will want to verify the semantic equivalence formally (e.g., through code inspection or additional test coverage)
2. Whether this should be back-patched (likely not, as it's a performance improvement, not a bug fix)
3. Whether there are other types in contrib or extensions that could benefit from the same pattern