How it works
Phiki uses TextMate grammar files to tokenize your code and then highlights them using Visual Studio Code themes. This allows for a high degree of highlighting accuracy and makes your code look like it was highlighted in your favourite code editor.1
Tokenization
Phiki starts by tokenizing your code using a TextMate grammar file.This breaks your code into smaller pieces of text with detailed “scope” information.
A scope is a dot-separated string that describes the type of text. For example, in PHP, a variable might have the scope
variable.other.php.2
Highlighting
Once Phiki has tokenized your code it starts to highlight the tokens using the Visual Studio Code theme of your choice.The scopes from each token are used to match against a ruleset in the theme file. This ruleset defines the background color, foreground color, and font style for the token.The tokens are transformed into a special
HighlightedToken object that contains the raw token and the “settings” / style information.3
Structuring
After producing a set of highlighted tokens, Phiki then creates an AST-like structure to represent the desired HTML output.This intermediate structure allows for easier manipulation of the final output without requiring complex string operations.
4
Transformation
If required, Phiki can then transform the intermediate structure. This is typically where extensions will hook into the process to modify the AST and add additional features such as line highlighting, class names, etc.
5
Rendering
Finally, Phiki takes the final AST and “stringifies” it into HTML.This HTML can then be outputted directly to the browser, inside of a template file, or saved to disk.