injection_points: Switch wait/wakeup to use atomics rather than latches

First seen: 2026-05-28 02:43:29+00:00 · Messages: 14 · Participants: 4

Latest Update

2026-06-04 · claude-opus-4-6

Incremental Update: New Discussion on Wait Point Detection Without PGPROC

The thread has advanced with a focused sub-discussion on how to detect that a wait point has been reached when there is no PGPROC (and thus no pg_stat_activity wait event visibility). This is a practical problem for writing reliable tests against the new atomic-based mechanism.

New Technical Proposal: mmap'd Shared State File

Andrey Borodin proposes a novel approach: memory-map inj_state to a fixed file (PGDATA/injection_points.shm). This would allow TAP test scripts to directly:

  1. Poll the name[] array in the mapped file to detect that a backend has reached a wait point
  2. Write to wait_counts[] to wake the backend

This is architecturally different from Michael's earlier LOG-polling idea — it provides direct shared-memory access from the test harness rather than indirect observation through log output.

Michael's Response: Portability Concerns

Michael raises the WIN32 portability issue: mmap() is POSIX-only, and Windows requires CreateFileMapping()/MapViewOfFile(). He floats two possible directions:

  1. A generic interface that extensions could use for similar cross-process shared state exposure
  2. A portable abstraction layer added specifically within the injection_points module

This suggests the discussion is evolving toward a broader infrastructure question about how test tooling can observe/interact with server-internal shared memory from outside the server process.

Validation of the Core Patch

Borodin confirms the patch works for his postmaster-kill corruption testing scenario. His test methodology involves iteratively sending SIGKILL to cluster PIDs and waiting for shared memory release before attempting recovery — exactly the use case that motivated the latch-to-atomics switch.

Practical Testing Methodology Debate

The exchange about sleep(1) vs. reliable detection highlights a real buildfarm concern: hardcoded sleeps are both too slow for fast machines and unreliable on slow machines. Any solution must work across the full spectrum of buildfarm animals. This constrains the design space — LOG polling introduces I/O latency and parsing complexity, while mmap'd state offers low-latency direct observation but adds portability burden.

History (1 prior analysis)
2026-06-01 · claude-opus-4-6

Injection Points: Switching Wait/Wakeup from Latches to Atomics

Core Problem

The injection_points testing infrastructure in PostgreSQL provides a mechanism for test code to synchronize backends at specific execution points — one backend waits at an injection point while another wakes it up. The existing implementation uses condition variables (which internally rely on latches) for this synchronization.

The fundamental limitation: condition variables and latches require a PGPROC slot. A backend only has a PGPROC slot after it has been assigned one from the proc array — which happens after authentication and initial setup. This means the injection_points wait/wakeup mechanism cannot be used in:

  1. Postmaster context — the postmaster never has a standard PGPROC slot or access to DSM segments in the same way backends do.
  2. Pre-authentication code paths — before a backend has acquired its PGPROC slot.
  3. Post-ProcKill scenarios — once a backend has released its PGPROC slot during shutdown (the immediate motivating case from the ProcKill thread).

This creates a significant gap in test coverage: any code that runs outside the PGPROC-attached lifecycle of a backend cannot be tested with injection point synchronization.

Proposed Solution

Michael Paquier's patch replaces the condition variable mechanism with atomic counters combined with a polling loop with exponential backoff:

Architecture Change

Before:

- Wait side: ConditionVariableWait() on inj_state->wait_point (requires latch/PGPROC)
- Wakeup side: Increment wait_counts[index] under spinlock, then ConditionVariableBroadcast()

After:

- Wait side: Polling loop checking pg_atomic_read_u32(&inj_state->wait_counts[index])
  with escalating sleep (10μs → 100ms max)
- Wakeup side: pg_atomic_fetch_add_u32(&inj_state->wait_counts[index], 1)

Key Design Characteristics

  1. Exponential backoff: Starting at 10μs and maxing at 100ms. This balances CPU usage (no tight spin loop) against responsiveness (fast machines see wakeups within microseconds to low milliseconds).

  2. CHECK_FOR_INTERRUPTS() in the wait loop: Required for tests like the autovacuum test in test_misc that depend on signal processing during waits.

  3. Context-independence: Atomic operations work without any PGPROC, DSM, or latch infrastructure — they operate directly on shared memory words.

  4. Loss of wait event visibility: Without a PGPROC, there's no way to report wait events in pg_stat_activity. The proposed workaround is emitting a LOG entry when entering the wait state, with TAP tests polling server logs instead.

Technical Tradeoffs

Responsiveness vs. Universality

The core tradeoff is clear: latches provide instant wakeup (event-driven via kernel signaling mechanisms like SetLatch → write() to self-pipe or signalfd), while atomic polling introduces bounded latency. Robert Haas initially pushed back precisely because this goes against the general PostgreSQL design principle of replacing polling loops with event-driven wakeups.

However, this is test infrastructure only — it never runs in production. The 100ms maximum delay is acceptable for test synchronization where the alternative is having no synchronization mechanism at all.

Atomicity of Wakeup vs. Slot Identity

Andrey Borodin raised a subtle correctness concern: in the wakeup path, the slot index is determined under the spinlock, but the atomic increment happens after the lock is released. Between these two operations, the slot's identity could theoretically change (if the injection point is detached and re-attached to a different name). While Borodin acknowledges this isn't a problem in "correctly written tests," it's a defensive programming concern. Moving the atomic increment back under the lock would eliminate this window.

Additional Motivation: Corruption Testing

Andrey Borodin reveals an important secondary use case: testing postmaster death scenarios. When the postmaster is kill-9'd while a backend waits on a ConditionVariable, the LWLock release cascade can cause the checkpointer to flush dirty buffers that haven't been WAL-logged — actual data corruption. By using atomics instead, the wait doesn't hold any LWLocks, making it safe to test crash scenarios without inducing false corruption.

Backpatch Considerations

Michael proposes backpatching to v17 (when injection_points were introduced). This is aggressive for infrastructure changes but motivated by wanting future test patches that depend on this capability to also be backpatchable.