Proposal: Conflict log history table for Logical Replication

First seen: 2025-08-05 12:24:01+00:00 · Messages: 407 · Participants: 11

Latest Update

2026-06-04 · claude-opus-4-6

Round Update: v44/v45 Posted, Naming Settled, New Bugs Surface

This round sees significant progress on multiple fronts: naming conventions finalized per Amit's direction, patches v44 and v45 posted, Peter Smith's enum-vs-string proposal rejected, and several new bugs discovered by Nisha and Zsolt.

Naming Decisions Finalized (Amit, June 1)

Amit settled the naming bikeshed with concrete directives:

Dilip incorporated all of these in v44.

Peter Smith's enum→string Proposal Rejected

Peter proposed changing conflict_log_destination from an enum (log/table/all) to a comma-separated string like the publish option, arguing it's more future-proof for N-way combinations. Amit rejected this: all remains useful as shorthand, and the current enum can be extended to multi-value later if needed.

Patch v44/v45 Structure

Patches reorganized: 0002 merged into 0001, 0004 merged into 0003. v45 adds Vignesh's rebased pg_upgrade and \dRs+ patches (0003, 0004). Key v44 changes:

New Bugs Discovered

1. ProcessPendingConflictLogTuple() failure masks original error (Nisha) If CLT insertion fails in the PG_CATCH path (e.g., relation OID invalid), PG_RE_THROW() is never reached. The original conflict error is lost; only the secondary CLT error is reported. Worker retries indefinitely showing only the CLT error, never the actual conflict.

2. disable_on_error path: CLT failure prevents subscription disable (Nisha) ProcessPendingConflictLogTuple() is called before the subscription is disabled. If it fails, the subscription never disables and the worker loops forever. Nisha suggests moving it after the disable logic.

3. Premature "logged to" message wording (Nisha) Error message says "Conflict details are logged to the conflict log table" before the write actually occurs. Should say "will be logged" since CLT write is deferred.

4. remote_commit_ts NULL under streaming=on (Nisha) When streaming=on, the commit timestamp isn't populated in the CLT because apply_handle_stream_commit() doesn't set remote_commit_ts = commit_data.committime in the TRANS_LEADER_APPLY case.

5. Inconsistent UPDATE error for superuser vs. owner (Nisha) Superuser gets the descriptive "cannot modify or insert data into conflict log table" error. The subscription owner (non-superuser) gets a generic "permission denied for table" because ACL check fires before the CLT-specific check.

6. Stale name references (Zsolt) Several code locations and commit messages still use old pg_conflict_log_for_subid_<oid> naming instead of pg_conflict_log_<subid>.

7. Missing xid/commit_ts in apply_handle_begin_prepare (Zsolt) apply_handle_begin_prepare doesn't set remote_xid/remote_commit_ts, potentially leaving them uninitialized for prepared transaction conflicts.

Test Infrastructure Issue (Nisha)

The poll_query_until() test for CLT content was silently broken in v44 — when the table doesn't exist, it retries until timeout rather than failing immediately. Nisha suggests wrapping in ok() for proper TAP reporting.

History (2 prior analyses)
2026-06-01 · claude-opus-4-6

Monthly Summary: Conflict Log History Table for Logical Replication — May 2026

Overview

May 2026 saw this proposal mature from a design-settled but implementation-rough state into extensive late-stage review, with multiple patch versions (v30–v40) posted. The pg_conflict schema architecture held firm under scrutiny, but testing revealed several serious functional bugs that remain unresolved at month's end. The month was dominated by code review, corner-case discovery, and incremental refinement rather than architectural pivots.

Architecture (Settled)

The core design is now stable:

  • pg_conflict schema (OID 1382): A system-reserved namespace analogous to pg_toast, housing auto-generated pg_conflict_log_<subid> tables per subscription. Protected via IsConflictClass/IsSystemClass checks blocking user modification.
  • conflict_log_destination = log | table | all: Bitmask-based subscription option controlling where conflict data flows.
  • JSON-based multi-unique-conflict model: Single row per conflict with local_conflicts JSON array column holding all colliding local tuples.
  • Deferred insertion for ERROR-level conflicts: Tuple stashed in worker memory, inserted in PG_CATCH block after transaction rollback.
  • Publication exclusion: is_publishable_class() rejects pg_conflict namespace relations.

Key Policy Decisions This Month

Schema Rename: Won't-Fix

Shveta reopened whether renaming pg_conflict should be blocked. Amit ruled against making it a special case — other reserved schemas (pg_toast, pg_catalog) allow superuser rename with equivalent breakage. Consensus formed that this is a pre-existing broader issue.

allow_system_table_mods + DROP COLUMN on CLT: Won't-Fix

Shlok Kyal demonstrated that ALTER TABLE ... DROP COLUMN succeeds on CLTs under allow_system_table_mods=on. Vignesh showed even real system catalogs like pg_type crash under the same GUC. Accepted as consistent with existing behavior under this developer-only GUC.

conflict_id / conflict_logged_at Column: Deferred Post-Commit

Shveta advocated for a timestamp column as partition key; Amit preferred a BIGINT identity column for manual resolution. Both explicitly deferred to avoid scope creep.

Ownership Transfer Policy: Unresolved

When ALTER SUBSCRIPTION OWNER TO targets a role without pg_create_subscription, the new owner cannot access their own CLT. Amit favored disallowing the transfer; Shveta argued it's analogous to toast table behavior and should be documented only. Dilip's patch implements the permissive approach, but Amit has not yet approved it.

Subscription-OID Preservation: Stays in Patchset

Shveta proposed splitting subscription-OID preservation for pg_upgrade into a separate thread (shared with the "origin migration" effort). Amit agreed it should eventually separate but keeps it here for now.

Serious Bugs Found (Unresolved at Month's End)

1. Missing TOAST Support — Apply Worker Crashes on Large Tuples

CLT created without TOAST table. JSON columns cannot store tuple representations exceeding ~8KB, causing ERROR: row is too big crashes. The conflict is lost entirely — not logged to CLT or server log.

2. Parallel Apply Workers Bypass CLT Entirely

Conflicts detected by parallel/streaming workers only reach the server logfile. The deferred-insertion mechanism doesn't exist in the parallel worker code path.

3. disable_on_error=true Suppresses CLT Logging

When a subscription disables on error, the PG_CATCH deferred-insertion path doesn't fire, losing exactly the ERROR-level conflicts operators most need to investigate.

4. remote_tuple and replica_identity Columns Swapped

Column assignment bug: replica identity key written to remote_tuple, full remote tuple written to replica_identity.

5. ATTACH PARTITION Exploit

CLT can be attached as a partition to a user table, then published — bypassing is_publishable_class(). Needs guards analogous to TOAST tables blocking ATTACH PARTITION.

Bugs Found and Fixed

  • Uninitialized log_dest_clt/log_dest_logfile (v30): Stack garbage causing assertion failures in conflict.c. Fixed by initialization at declaration. This same bug class recurred in v34 (third occurrence), motivating Peter Smith's macro proposal (CONFLICTS_LOGGED_TO_TABLE(dest)).
  • Use-after-free in DropSubscription (v34-0005): form->subconflictlogrelid accessed after tuple freed by CatalogTupleDelete.
  • CLT owned by wrong user on ALTER-time creation: GetUserId() used instead of subscription owner.

pg_dump/pg_upgrade Issues

  • Test assertion uses unqualified to_regclass() that always returns NULL (vacuous test)
  • dobj.dump value overwritten by fall-through control flow
  • Unconditional ALTER SUBSCRIPTION dump even for default values
  • Duplicate assignment copy-paste error
  • NULL output for missing column values in older PG versions

Code Organization Evolution

  • CLT schema definition (ConflictLogSchema[], ConflictLogColumnDef) moved entirely into conflict.c — no header exposure
  • Debate ongoing: drop_conflict_log_table → drop_subscription_dependencies in subscriptioncmds.c (Peter) vs. keeping in conflict.c (current)
  • Peter proposed destination-check macros to structurally prevent recurring initialization bugs
  • TupleDesc caching suggested by Shveta for build_conflict_tupledesc() (performance optimization)
  • Potential relcache corruption flagged: BlessTupleDesc mutating index RelationGetDescr directly

Patch Versions This Month

Version Date (approx) Key Changes
v30 Early May Base version entering review
v31 Early May Fixes uninitialized bool crash
v33 Mid May Major refactoring
v34 Mid May Addresses protection gaps
v35-v36 Late May Peter Smith's code-style review addressed
v37 Late May pg_dump fixes, code reorganization
v38 Late May Multi-thread review fixes
v39-v40 May 25-26 Incremental cleanup

Status at Month's End

The architecture is stable but the implementation has 3-5 serious functional bugs outstanding (TOAST, parallel workers, disable_on_error, column swap, partition exploit). Code review is in late stages with Peter Smith, Shveta, Nisha, and Vignesh all actively reviewing. The ownership-transfer policy question remains open pending Amit's response to Dilip's permissive implementation. Documentation structure is being reworked per Peter's proposal to break §29.8 into subsections.

2026-06-01 · claude-opus-4-6

Round Update: Naming Bikeshed Only — No Substantive Progress

This round contains a single message from Peter Smith responding to what appears to be a naming discussion about the conflict_log_destination option. His contribution is purely terminological:

  • He argues against removing "log" from the option/variable names, noting that "conflict log" clearly denotes "the log of the conflict" whereas bare "conflict" is ambiguous (could refer to the conflict itself or where it occurred).
  • He floats alternative option names (conflict_reporting, conflict_data_destination) but immediately notes any rename would cascade through all code, docs, and acronyms (CLT → CRT, CDD, etc.), making it impractical at this stage.

No technical substance, no new bugs, no design decisions, no new patches, no responses to the three outstanding serious bugs (missing TOAST support, parallel worker bypass, swapped columns).