Deep Dive into Svelte (0) — What is an Abstract Syntax Tree?

Introduction

This series of articles focuses on exploring the inner workings and implementation of Svelte, aiming to give readers a deeper understanding of Svelte’s compilation mechanism and code generation. Since Svelte’s compilation process involves parsing code, this article will primarily discuss what an Abstract Syntax Tree (AST) is and elaborate on its role and importance.

What is an Abstract Syntax Tree (AST)?

Let’s start with the definition from Wikipedia:

In computer science, an abstract syntax tree (AST), or just syntax tree, is a tree representation of the abstract syntactic structure of source code written in a programming language. Each node of the tree denotes a construct occurring in the source code. The syntax is “abstract” in the sense that it does not represent every detail appearing in the real syntax.

The reason ASTs are so important is that we need a structured way to describe programming languages (or markup languages) so that computers can manipulate them easily. Let’s start by looking at some HTML:

<h1>這是一個標題</h1>
<p>這是一個段落</p>
<ul>
  <li data-item="1">清單列表</li>
  <li data-item="2">清單列表</li>
  <li data-item="3">清單列表</li>
</ul>

Tags like <xxx> are called tags in HTML, much like a clothing label summarizing information about a garment. The tag also records information about what this HTML element represents.

The reason we need to convert strings into a tree structure is that while a markup language like HTML is intuitive and structured for humans, it is just a plain string to a computer. Therefore, we need to parse the string beforehand and build a tree-like data structure for easier manipulation. The HTML above can be represented as an AST:

Once HTML is converted into a tree structure, we can traverse each node using tree traversal algorithms to perform corresponding operations, making it easy to search for specific nodes. For example, if I want to search for li in the tree, I can traverse it with a tree algorithm and return the node whenever tagName equals li.

This applies not only to HTML; other programming languages follow a similar process based on their defined grammars. Languages and query formats like SQL and GraphQL also convert raw strings into an AST before performing any further operations.

Representing an Abstract Syntax Tree

In the previous section, we used a diagram to represent an AST. However, an AST can also store additional information. Taking HTML as an example:

• Whether there are attributes, which should also be stored within the node if present
• Whether it is a self-closing tag (like <input />, <video />, etc.)
• Parsing position (line and column numbers)

If you’d like to inspect real-world ASTs more closely, check out AST Explorer, which supports parsing a wide variety of languages into ASTs. Here is an example using HTML:

Next, let’s take a look at a JavaScript AST:

Implementing an Abstract Syntax Tree

There are mainly two approaches to implementing an AST parser: one is to define the grammar yourself and implement the parser directly in code (handwritten); the other is to define grammar rules (such as BNF) and use a generator like yacc or PEG.js to generate the parser.

I previously wrote articles on how to build a JSON parser from scratch. If you’re interested, feel free to check them out:

• Building a JSON Parser from Scratch (1)
• Building a JSON Parser from Scratch (2)

My previous article introducing linaria also touched upon some AST principles, which you might find useful as well.

However, implementing an accurate AST is not easy. While it’s relatively straightforward for simpler markup languages like HTML, creating a fully compliant AST for languages like C, Java, or JavaScript takes time and strict adherence to specifications. If you’re doing it as an exercise, I’d recommend implementing just a subset of the syntax.

In fact, within Svelte, apart from HTML and the Svelte template syntax where custom parsers are implemented, others like JavaScript and CSS leverage existing, battle-tested third-party parsers. Later articles will cover how to build a simple Svelte syntax parser, so we won’t dive too deep here.

If you can’t wait to see the implementation, you can check out my implementation in tiny-svelte, or look directly at Svelte’s source code.

Uses of an Abstract Syntax Tree

While ASTs are crucial, a syntax tree on its own can’t accomplish much—compiling code still requires a code generation step. Beyond compilation, ASTs can also be used for:

• Syntax highlighting
• Code autocompletion
• Prettifying / formatting
• ESLint

(Added by former colleague @kai):

• Writing Babel plugins
• Static analysis (e.g., TypeScript)
• Code transformation / refactoring (codemods)

All these tools are built on top of ASTs, which speaks volumes about their importance. With an AST in hand, it feels like anything is possible—the possibilities are endless.

Conclusion

This article explained the existence and purpose of Abstract Syntax Trees, aiming to give you a foundational understanding of what they are. Since Svelte’s compilation process begins by turning source code into an AST, understanding ASTs is essential for following along with the upcoming articles.