Skip to content

Update grammar to cover array spreads, variant type spreads and newer operators. #267

Open
mvaled wants to merge 6 commits intorescript-lang:mainfrom
mvaled:main
Open

Update grammar to cover array spreads, variant type spreads and newer operators. #267
mvaled wants to merge 6 commits intorescript-lang:mainfrom
mvaled:main

Conversation

@mvaled
Copy link
Copy Markdown

@mvaled mvaled commented Apr 11, 2026

Disclosure: This PR was generated with AI assistance and reviewed by the author. The author's goal is to improve the Emacs rescript-ts-mode (jjlee/rescript-mode#19).

Summary

This PR updates the Tree-sitter grammar to better match recent ReScript language features across three areas:

  1. ReScript 12 operator support
  2. Variant spread syntax
  3. Array spread expressions

What changed

1. ReScript 12 operator support

Added grammar support for the newer operator syntax introduced in ReScript 12, including:

  • modulo: %

  • bitwise operators:

    • &&&
    • |||
    • ^^^
    • ~~~
  • shift operators:

    • <<
    • >>
    • >>>

2. Variant spread syntax

Added support for:

  • variant type spreads, e.g.

    type base = Start | Stop | Pause
    type extended = | ...base | Resume
  • variant pattern spreads, e.g.

    let handle = (event: extended) =>
        switch event {
        | ...base as core => Console.log2("base", core)
        | Resume => Console.log("resuming")
        }

3. Array spread expressions

Added support for spread elements inside array literals, including multiple spreads, e.g.

let animals = ["🐶", "🐱", "🐷"]
let moreAnimals = [...animals, "🐔", "🐴", "🐮", ...otherAnimals]

This brings the grammar in line with array spread syntax available since ReScript v11.

mvaled and others added 4 commits April 10, 2026 16:23
Add parser support for the ReScript 12 operator improvements that were
missing from the grammar.

This covers modulo, bitwise operators, and shift operators, and adds
corpus and highlight updates for the new syntax. Unrelated untracked
files are left out of the commit.
Add grammar support for variant type spreads and variant pattern
spreads.

This adds corpus coverage for type declarations and expressions,
including a dedicated variant pattern spread example.

Also ignore local uv lockfiles via .git/info/exclude.
Allow spread elements in array expressions.

Add corpus coverage for array spread syntax using the ReScript 11.1
style example, including multiple spreads in the same array literal.
The ".." token for extensible variants (e.g. `type t = ..`) was
included as a choice in the `_type_identifier` rule alongside
`type_identifier` and `type_identifier_path`.  Because
`_type_identifier` is inlined, the tokenizer would see the three-dot
spread operator "..." and greedily match ".." first, leaving a stray
"." behind.  This made every variant type spread (`| ...Foo.t`)
produce an ERROR node, breaking font-lock for the rest of the file.

Extensible variants are syntactically distinct: `type t = ..` is the
only valid use of ".." in a type position — it cannot appear inside
generics (`option<..>` is invalid) or be combined with other type
constructors.  Modelling it as a separate `extensible_type` node
branching directly from `type_binding` reflects this and removes the
ambiguity from the tokenizer entirely.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@nojaf
Copy link
Copy Markdown
Member

nojaf commented Apr 15, 2026

Hello @mvaled, thanks for this PR!
Could you check the failing tests in CI?
I've asked @aspeddro for a review as he knows most about this stuff.

@mvaled
Copy link
Copy Markdown
Author

mvaled commented Apr 15, 2026

Hello @mvaled, thanks for this PR! Could you check the failing tests in CI? I've asked @aspeddro for a review as he knows most about this stuff.

Hi @nojaf, @aspeddro,

Fixed. The problem was I used a different tree-sitter version that the one in the CI.

I took the liberty to update the .github/workflows/ci.yml to pin the tree-sitter version, and the peerDependency in package.json.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Tree-sitter ReScript grammar and associated artifacts to support newer ReScript language constructs (ReScript 12 operators, variant spreads, and array spreads), improving parsing and editor integrations (e.g., rescript-ts-mode).

Changes:

  • Extend the grammar to parse variant type spreads, variant pattern spreads, and array spread elements.
  • Add parsing support for ReScript 12 operators (%, bitwise ops, and shift ops) with precedence updates.
  • Regenerate/update Tree-sitter artifacts (node types, generated grammar JSON) and adjust CI/runtime tooling versions.

Reviewed changes

Copilot reviewed 8 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
grammar.js Implements grammar rule updates for operators, variant spreads, array spreads, and extensible variant types.
src/grammar.json Regenerated grammar JSON reflecting new/updated grammar rules and precedences.
src/node-types.json Regenerated node-types including new nodes/tokens (e.g., extensible_type, spread/pattern nodes, operator tokens).
test/corpus/type_declarations.txt Adds/updates corpus coverage for extensible variants and variant type spreads.
test/corpus/expressions.txt Adds corpus coverage for variant pattern spreads, modulo, bitwise operators, shift operators, and array spreads.
queries/highlights.scm Adds highlighting for newly supported operators.
package.json Updates tree-sitter peer dependency version requirement.
package-lock.json Lockfile update to match package.json peer dependency change.
.github/workflows/ci.yml Ensures CI runs when Node/tooling files change and pins the Tree-sitter CLI ref used in CI.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread queries/highlights.scm
Comment thread package.json Outdated
Remove '%' from the unconditional operator query so leading percent signs in
extension expressions like %foo and %%foo are not highlighted as operators.

Highlight '%' only in binary_expression, matching how other context-sensitive
operators such as '<', '>', and '/' are handled.
Pin the GitHub Actions tree-sitter CLI to v0.26.3 so parser generation
matches the version declared in the package metadata, and regenerate
src/parser.c with that version.

Keep the runtime tree-sitter peer dependency at ^0.21.1. The CLI and the
runtime are versioned separately, so the CI toolchain update should not raise
the consumer runtime requirement.

Include package.json, package-lock.json, and .nvmrc in the CI path filters so
toolchain-only changes still trigger the workflow.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants