Frequently Asked Questions

My pattern does not work, why?

1. Use the Playground: Test your pattern in the ast-grep playground.
2. Check for Valid Code: Make sure your pattern is valid code that tree-sitter can parse.
3. Ensure Correctness: Use a pattern object to ensure your code is correct and unambiguous.
4. Explore Examples: See ast-grep's catalog for more examples.

The most common scenario is that you only want to match a sub-expression or one specific AST node in a whole syntax tree. However, the code fragment corresponding to the sub-expression may not be valid code. To make the code can be parsed by tree-sitter, you probably need more context instead of providing just code fragment.

For example, if you want to match key-value pair in JSON, writing "key": "$VAL" will not work because it is not a legal JSON.

Instead, you can provide context via the pattern object. See playground code.

rule:
  pattern:
    context: '{"key": "$VAL"}'
    selector: pair

The idea is that you can write full an valid code in the context field and use selector to select the sub-AST node.

This trick can be used in other languages as well, like C and Go.

My Rule does not work, why?

Here are some tips to debug your rule:

• Use the ast-grep playground to test your rule.
• Simplify your rule to the minimal possible code that reproduces the issue.
• Confirm pattern's matched AST nodes are expected. e.g. statement and expression are different matches. This usually happens when you use follows or precedes in the rule.
• Check the rule order. The order of rules matters in ast-grep especially when using meta variables with relational rules.

CLI and Playground produce different results, why?

The CLI and Playground may use different tree-sitter parsers. There are two main reasons why the results may differ:

• Parser Version: The CLI may use a different version of the tree-sitter parser than the Playground. Playground parsers are updated less frequently than the CLI, so there may be differences in the results.
• Text Encoding: The CLI and Playground may use different text encodings. CLI uses utf-8, while the Playground uses utf-16. The encoding difference may cause different fallback parsing during error recovery.

To debug the issue, you can use the --debug-query flag in the CLI to see the parsed AST nodes and meta variables.

sg run -p <PATTERN> --debug-query ast

The debug output will show the parsed AST nodes and you can compare them with the Playground. You can also use different debug formats like cst or pattern.

If you find there are different results, it is usually caused by incomplete code snippet in the pattern. To fix the issue, you can provide a complete context code via the pattern object.

rule:
  pattern:
    context: 'int main() { return 0; }'
    selector: function

See Pattern Deep Dive for more context.

Text encoding impacts tree-sitter error recovery.

Tree-sitter is a robust parser that can recover from syntax errors and continue parsing the rest of the code. The exact strategy for error recovery is implementation-defined and uses a heuristic to determine the best recovery strategy. See tree-sitter issue for more details.

Text-encoding will affect the error recovery because it changed the cost of different recovery strategies.

Found inconsistency?

If you find the inconsistency between CLI and Playground, please open an issue in the Playground repository.

MetaVariable does not work, why?

1. Correct Naming: Start meta variables with the $ sign, followed by uppercase letters (A-Z), underscores (_), or digits (1-9).
2. Single AST Node: A meta variable should be a single AST node. Avoid mixing meta variables with other text in one AST node. For example, mix$OTHER_VAR or use$HOOK will not work.
3. Named AST Nodes: By default, a meta variable matches only named AST nodes. Use double dollar signs like $$UNNAMED to match unnamed nodes.

Multiple MetaVariable does not work

Multiple meta variables in ast-grep, such as $$$MULTI, are lazy. They stop matching nodes if the first node after them can match.

For example, foo($$$A, b, $$$C) matches foo(a, c, b, b, c). $$$A stops before the first b and only matches a, c.

This design follows TypeScript's template literal types (${infer VAR}Literal) to ensure multiple meta variables always produce a match or non-match in linear time.

Pattern cannot match my use case, how?

Patterns are a quick and easy way to match code in ast-grep, but they might not handle complex code. YAML rules are much more expressive and make it easier to specify complex code.

I want to pattern match function call starts with some prefix string, how can I do that?

It is common to find function name or variable name following some naming convention like a function must starts with specific prefix.

For example, React Hook in JavaScript requires function names start with use. Another example will be using io_uring in Linux asynchronous programming.

You may start with pattern like use$HOOK or io_uring_$FUNC. However, they are not valid meta variable names since the AST node text does not start with the dollar sign.

The workaround is using constraints in YAML rule and regex rule.

rule:
  pattern: $HOOK($$$ARGS)
constraints:
  HOOK: { regex: '^use' }

Example usage.

MetaVariable must be one single AST node

Meta variables cannot be mixed with prefix/suffix string . use$HOOK and io_uring_$FUNC are not valid meta variables. They are parsed as one AST node as whole, and ast-grep will not treat them as valid meta variable name.

Why is rule matching order sensitive?

ast-grep's rule matching is a step-by-step process. It matches one atomic rule at a time, stores the matched meta-variable, and proceeds to the next rule until all rules are matched.

Rule matching is ordered because previous rules' matched meta-variables can affect later rules. Only the first rule can specify what a $META_VAR matches, and later rules can only match the content captured by the first rule without modifying it.

Let's see an example. Suppose we want to find a recursive function in JavaScript. This rule can do the trick.

rule.yml

id: recursive-call
language: JavaScript
rule:
  all:
  - pattern: function $F() { $$$ }
  - has:
      pattern: $F()
      stopBy: end

match.js

function recurse() {
  foo()
  recurse()
}

The rule works because the pattern function $F() { $$$ } matches first, capturing $F as recurse. The later has rule then looks for a recurse() call based on the matched $F.

If we swap the order of rules, it will produce no match.

id: recursive-call
language: JavaScript
rule:
  all:
  - has:  # N.B. has is the first rule
      pattern: $F()
      stopBy: end
  - pattern: function $F() { $$$ }

In this case, the has rule matches first and captures $F as foo since foo() is the first function call matching the pattern $F(). The later rule function $F() { $$$ } will only find the foo declaration instead of recurse.

TIP

Using all to specify the order of rule matching can be helpful when debugging YAML rules.

What does unordered rule object imply?

A rule object in ast-grep is an unordered dictionary. The order of rule application is implementation-defined. Currently, ast-grep applies atomic rules first, then composite rules, and finally relational rules.

If your rule depends on using meta variables in later rules, the best way is to use the all rule to specify the order of rules.

kind and pattern rules are not working together, why?

The most common scneario is that your pattern is parsed as a different AST node than you expected. And you may use kind rule to filter out the AST node you want to match. This does not work in ast-grep for two reasons:

1. tree-sitter, the underlying parser library, does not offer a way to parse a string of a specific kind. So kind rule cannot be used to change the parsing outcome of a pattern.
2. ast-grep rules are mostly independent of each other, except sharing meta-variables during a match. pattern will behave the same regardless of another kind rule.

To specify the kind of a pattern, you need to use pattern object.

For example, to match class field in JavaScript, a kind + pattern rule will not work:

# these are two separate rules
pattern: a = 123          # rule 1
kind: field_definition    # rule 2

This is because pattern a = 123 is parsed as assignment_expression. Pattern and kind are two separate rules. And using them together will match nothing because no AST will have both assignment_expression and field_definition kind at once.

Instead, you need to use pattern object to provide enough context code for the parser to parse the code snippet as field_definition:

# this is one single pattern rule!
pattern:
  context: 'class A { a = 123 }' # provide full context code
  selector: field_definition     # select the effective pattern

Note the rule above is one single pattern rule, instead of two. The context field provides the full unambiguous code snippet of class. So the a = 123 will be parsed as field_definition. The selector field then selects the field_definition node as the effective pattern matcher.