Reject HEADER with binary and json COPY formats by option presence

First seen: 2026-05-31 01:57:06+00:00 · Messages: 4 · Participants: 4

Latest Update

2026-06-04 · claude-opus-4-6

Incremental Update: New Responses and Emerging Disagreement

Three new messages since the prior analysis, introducing a notable counter-position from a senior contributor.

Summary of New Activity

  1. Tingchuan Sun (+1, no new technical content) — agrees with presence-based validation, says patch looks good.
  2. Shinya Kato — generally approves (LGTM) but requests additional test coverage: specifically COPY-level test cases for format json, header 0 and format binary, header 0 to complement the existing file_fdw tests.
  3. Masao Fujii — pushes back on the premise of the patch, arguing that PostgreSQL does not have a consistent rule mandating presence-based validation. He observes that both validation styles (presence-based and value-based) already coexist in the codebase, and questions whether a code change is warranted.

Key Technical Disagreement

Fujii's objection is significant because it challenges the core argument cited in the prior analysis — that VACUUM's behavior establishes a precedent for presence-based validation. Fujii contends that no such universal rule exists, and the mixed status quo is acceptable. This reframes the discussion from "bug fix to match documentation" to a broader design policy question about whether PostgreSQL should standardize on one validation approach.

This creates a potential blocker: if the community cannot agree that presence-based rejection is the correct policy, the patch may stall or require a documentation-side fix instead (i.e., weakening the docs to say the option "has no effect" rather than "is not allowed").

History (1 prior analysis)
2026-06-01 · claude-opus-4-6

Technical Analysis: Reject HEADER with binary and json COPY formats by option presence

Core Problem

The thread identifies an inconsistency between PostgreSQL's documentation and implementation regarding how the HEADER option interacts with binary and json COPY formats. The documentation states categorically that "This option is not allowed when using binary or json format," implying that the mere presence of the option should trigger an error. However, the actual implementation only rejects the option based on its value — specifically, HEADER '0' (a falsy value) is silently accepted while HEADER '1' (a truthy value) is correctly rejected.

Architectural Significance

This is fundamentally a question about option validation semantics in PostgreSQL's command processing layer. The issue touches on a broader design principle: should PostgreSQL validate options based on their presence (syntactic) or their semantic effect (value-based)?

The distinction matters because:

  1. Documentation fidelity: If the docs say an option "is not allowed," users expect any specification of it to fail, regardless of value.
  2. Consistency across subsystems: The thread demonstrates that VACUUM already follows presence-based validation (e.g., BUFFER_USAGE_LIMIT 0 with FULL is rejected even though 0 effectively disables the strategy). COPY should follow the same pattern.
  3. Future extensibility: Allowing falsy values to pass silently creates a precedent where other options might inconsistently accept "no-op" values in invalid contexts, making behavior harder to reason about.

Technical Details of the Bug

The validation code in the COPY path currently checks the effective value of the header option (likely checking if header > 0 or similar) rather than checking the header_specified boolean flag that already exists in the parsing infrastructure. The header_specified variable is set whenever the HEADER option appears in the command syntax, regardless of its value.

Current (incorrect) behavior:

-- Rejected (correct): header is truthy in binary mode
CREATE FOREIGN TABLE ft (...) OPTIONS (format 'binary', header '1');
-- ERROR: cannot specify HEADER in BINARY mode

-- Accepted (incorrect): header is falsy in binary mode
CREATE FOREIGN TABLE ft (...) OPTIONS (format 'binary', header '0');
-- CREATE FOREIGN TABLE (should error)

Expected behavior after fix:

Both should be rejected because the HEADER option is present, regardless of its value being effectively a no-op.

Proposed Solution

The fix is described as straightforward: change the validation check from inspecting the header option's value to checking the header_specified flag. This flag is already tracked during option parsing, making the implementation minimal and low-risk.

The relevant code path is in the COPY option validation (likely in src/backend/commands/copy.c or src/backend/commands/copyfrom.c and src/backend/commands/copyto.c), where after all options are parsed, incompatible combinations are checked. The condition guarding the "cannot specify HEADER in BINARY mode" error needs to be changed from something like:

if (opts->header_line && opts->binary)
    ereport(ERROR, ...);

to:

if (header_specified && opts->binary)
    ereport(ERROR, ...);

And similarly for the json format check.

Broader Context

The author references a prior thread ([1]) about a similar issue with the COPY command directly (as opposed to file_fdw), suggesting this pattern of value-based rather than presence-based validation may exist in multiple code paths. The issue was discovered while testing the "file_fdw: Support multi-line HEADER option" patch, indicating active development in this area that could compound the inconsistency if left unaddressed.

Design Tradeoff

There's a philosophical tension here:

  • Presence-based rejection (proposed): Strict, documentation-conformant, prevents confusion, consistent with VACUUM behavior
  • Value-based rejection (current): More permissive, arguably harmless since HEADER 0 is effectively a no-op

The argument for presence-based rejection is stronger because: (a) it matches documentation, (b) it matches precedent in other commands, (c) it prevents tools/scripts from accidentally including meaningless options that might later cause confusion during debugging or migrations.