Incremental Update: New Participant Shares Concrete Implementation Work
What's New
Greg Burd has responded to Pavel Stehule's original post with a concrete (though incomplete) implementation effort: replacing PostgreSQL's flex/bison parser infrastructure with "Lime," a parser generator that supports runtime grammar modification.
Technical Significance
This is the first substantive response to the thread and introduces two concrete artifacts:
- Lime parser generator (https://codeberg.org/gregburd/lime) — A standalone parser generator project designed to support runtime grammar extensibility
- PostgreSQL integration branch (https://github.com/gburd/postgres/tree/lime) — An actual (unstable) branch attempting to replace PostgreSQL's flex+bison parser with Lime
This is architecturally significant because it represents a fundamentally different approach than DuckDB's:
- DuckDB's approach: Keep the core parser intact, add a fallback/intercept mechanism for extensions to handle unparseable statements
- Greg's approach: Replace the entire parser generator infrastructure with one that natively supports runtime grammar modification, allowing extensions to inject new grammar productions directly
The second approach is far more ambitious — it doesn't just add a hook for unrecognized statements, it proposes that extensions could modify the actual grammar rules at runtime. This would theoretically allow extensions to add new keywords, new expression types, or new statement forms that integrate seamlessly with existing grammar.
Practical Status
Greg explicitly states the branch is "not yet stable or ready" — this is exploratory/proof-of-concept work, not a patch submission. However, it demonstrates that at least one developer is actively working on the problem beyond just discussing it conceptually.
Open Questions This Raises
- What is the performance profile of a runtime-modifiable parser vs. compile-time-generated Bison tables?
- How would grammar conflicts between multiple extensions be detected and resolved at runtime?
- Does replacing the entire parser infrastructure introduce unacceptable risk to PostgreSQL's parsing correctness guarantees?
- Is a full parser replacement realistic given the decades of battle-tested behavior in the current gram.y?