Ternary Reloption Type: Consolidating the "unset" Semantics

The Architectural Problem

PostgreSQL's reloption subsystem historically provided a small set of primitive types: bool, int, real, enum, and string. A pattern has emerged where boolean reloptions need a third semantic state beyond on/off: an "unset" or "use default/auto" state. This third state is needed because:

1. Inheritance/override semantics: A table-level reloption may need to distinguish "explicitly set to false" from "not set, inherit GUC or hardcoded default." This matters for vacuum_truncate, where you want VACUUM (TRUNCATE ...) command-level options to override the reloption only when the reloption wasn't explicitly chosen.
2. Auto/heuristic modes: vacuum_index_cleanup and GiST's buffering reloption expose an auto value where the system decides at runtime. Here the third state is user-visible and user-selectable.

These needs were being solved ad-hoc with two different mechanisms:

• Shadow boolean flag — vacuum_truncate uses a companion vacuum_truncate_set field to record whether the user set it. This doubles the reloption storage and scatters "is-set" logic through the code.
• Enum reloption — vacuum_index_cleanup and buffering are declared as enums with {on, off, auto} labels. This works but loses the boolean-value-parsing niceties (accepting t, true, 1, yes, y and their false variants) and produces inconsistent code paths for what is fundamentally "boolean + unset."

Nikolay Shaplov's patch unifies these by introducing a first-class ternary reloption type backed by a new enum pg_ternary { PG_TERNARY_FALSE, PG_TERNARY_TRUE, PG_TERNARY_UNSET }.

Design of the Ternary Type

Placement of the enum

The original patch put the ternary enum in c.h, reasoning by analogy with bool. Timur Magomedov pushed back suggesting a dedicated header. Álvaro Herrera ultimately moved it to postgres.h and added the PG_ prefix (PG_TERNARY_*) to avoid symbol collisions with third-party code — a sensible defensive move given c.h/postgres.h are transitively included by virtually everything, including extensions.

Parser behavior

The ternary type reuses the boolean value parser (parse_bool), so vacuum_index_cleanup and buffering now accept all boolean aliases (1, 0, t, f, yes, no, FAL, etc.) that the enum implementation rejected. This is a minor user-visible behavior change, but a strictly permissive one.

The unset_alias mechanism

Where the third state must be user-selectable (as in auto), the reloption definition specifies an alias string. The parser accepts that string as a third legal literal mapping to PG_TERNARY_UNSET. Where no alias is provided (as in vacuum_truncate), the user can only type on/off; the unset state is reachable only by not specifying the option at all.

This is the subtle point Nathan Bossart flagged as a naming tension: from the user's perspective, when no alias is defined, the type looks boolean — it's only "ternary" in its internal representation. Álvaro sharpened the critique: even with an alias, the system eventually resolves to on or off at runtime; the user can never make the system "behave as if set to half." The type is ternary in state space, not in semantics.

Removal of default_val

Álvaro removed the default-value slot for ternaries with the argument: if a ternary defaults to true or false, you could have achieved identical behavior with a plain bool reloption — because specifying no default effectively means "the default is 'unset.'" Shaplov objected using a hypothetical prefer_XXX_optimization: yes/no/never where the natural default of yes is distinct from the "never" (opt-out heuristic) sentinel. This disagreement was left unresolved at commit time; Álvaro committed without default_val and Shaplov accepted deferring the question until a concrete use case appears.

Error Message Quality

The weakest area of the patch. The enum implementation of vacuum_index_cleanup used to produce a message enumerating the three valid labels. The unified ternary path initially produced a generic "invalid value for boolean option" message, losing the auto hint. Shaplov's resolution: split the error reporting based on whether the ternary has an unset_alias:

• No alias → "invalid value for boolean option" (truthful: the user sees only bool)
• With alias → "invalid value for option ...; valid values are 'on', 'off' and '%s'"

This preserves informativeness precisely where the user needs it while avoiding the misleading word "ternary" in user-facing messages.

Commit Structuring Critique

Álvaro delivered a direct methodological critique that is worth preserving as general guidance: the original split was by idea development layers (tests for existing behavior → new type → alias feature → more tests), which made the patch series readable as a tutorial but nonsensical as individual commits in git log. Specifically, 0001 was "tests for behavior being replaced" — which as a standalone commit makes no sense because it tests behavior that the next commit removes. Álvaro squashed 0001+0002 plus the relevant tests from 0004, and committed. The principle: each commit should be coherent and make sense in isolation, assuming arbitrary commits land between it and its siblings.

What Actually Got Committed

Álvaro pushed part 1 (the ternary infrastructure + vacuum_truncate conversion) in January 2026, removing the vacuum_truncate_set shadow flag. The second part — converting vacuum_index_cleanup and GiST buffering from enum to ternary with unset_alias="auto" — was rebased and resubmitted separately (May 2026) for a new commitfest entry.

Significance

This is a small but architecturally clarifying refactor. It doesn't add user-visible functionality beyond accepting more boolean aliases; its value is in:

1. Eliminating the shadow-flag anti-pattern for "is this reloption explicitly set?"
2. Providing a canonical answer to a recurring reloption design question.
3. Reducing code duplication and the risk that future vacuum_truncate-like options reinvent yet another mechanism.

The naming remains slightly awkward (Nathan and Álvaro both noted "ternary" overstates what the type offers users), but no better name was proposed.