blob: ad98c0f3c0d25632534be8ec70afaeec96841ee4 [file] [view] [edit]
Lexical Structure
=================
This section describes the detailed lexical structure of the Lean
language. Many readers will want to skip this section on a first
reading.
Lean input is processed into a stream of tokens by its scanner, using
the UTF-8 encoding. The next token is the longest matching prefix of
the remaining input.
```
token: `symbol` | `command` | `ident` | `string` | `char` | `numeral` |
: `decimal` | `quoted_symbol` | `doc_comment` | `mod_doc_comment` |
: `field_notation`
```
Tokens can be separated by the whitespace characters space, tab, line
feed, and carriage return, as well as comments. Single-line comments
start with ``--``, whereas multi-line comments are enclosed by ``/-``
and ``-/`` and can be nested.
Symbols and Commands
====================
.. *(TODO: list built-in symbols and command tokens?)*
Symbols are static tokens that are used in term notations and
commands. They can be both keyword-like (e.g. the :keyword:`have
<structured_proofs>` keyword) or use arbitrary Unicode characters.
Command tokens are static tokens that prefix any top-level declaration
or action. They are usually keyword-like, with transitory commands
like :keyword:`#print <instructions>` prefixed by an additional
``#``. The set of built-in commands is listed in the :numref:`Chapter
%s <other_commands>` section.
Users can dynamically extend the sets of both symbols (via the
commands listed in :numref:`quoted_symbols`) and command tokens (via
the :keyword:`[user_command] <attributes>` attribute).
.. _identifiers:
Identifiers
===========
An *atomic identifier*, or *atomic name*, is (roughly) an alphanumeric
string that does not begin with a numeral. A (hierarchical)
*identifier*, or *name*, consists of one or more atomic names
separated by periods.
Parts of atomic names can be escaped by enclosing them in pairs of French double quotes ``«»``.
```lean
def Foo.«bar.baz» := 0 -- name parts ["Foo", "bar.baz"]
```
```
ident: `atomic_ident` | `ident` "." `atomic_ident`
atomic_ident: `atomic_ident_start` `atomic_ident_rest`*
atomic_ident_start: `letterlike` | "_" | `escaped_ident_part`
letterlike: [a-zA-Z] | `greek` | `coptic` | `letterlike_symbols`
greek: <[α-ωΑ-Ωἀ-῾] except for [λΠΣ]>
coptic: [ϊ-ϻ]
letterlike_symbols: [℀-⅏]
escaped_ident_part: "«" [^«»\r\n\t]* "»"
atomic_ident_rest: `atomic_ident_start` | [0-9'ⁿ] | `subscript`
subscript: [₀-₉ₐ-ₜᵢ-ᵪ]
```
String Literals
===============
String literals are enclosed by double quotes (``"``). They may contain line breaks, which are conserved in the string value.
```
string : '"' `string_item` '"'
string_item : `string_char` | `string_escape`
string_char : [^\\]
string_escape: "\" ("\" | '"' | "'" | "n" | "t" | "x" `hex_char` `hex_char`)
hex_char : [0-9a-fA-F]
```
Char Literals
=============
Char literals are enclosed by single quotes (``'``).
```
char: "'" `string_item` "'"
```
Numeric Literals
================
Numeric literals can be specified in various bases.
```
numeral : `numeral10` | `numeral2` | `numeral8` | `numeral16`
numeral10 : [0-9]+
numeral2 : "0" [bB] [0-1]+
numeral8 : "0" [oO] [0-7]+
numeral16 : "0" [xX] `hex_char`+
```
Decimal literals are currently only being used for some :keyword:`set_option <options>` values.
```
decimal : [0-9]+ "." [0-9]+
```
Quoted Symbols
==============
In a fixed set of commands (:keyword:`notation
<notation_declarations>`, :keyword:`local notation
<notation_declarations>`, and :keyword:`reserve
<notation_declarations>`), symbols (known or unknown) can be quoted by
enclosing them in backticks (`````). Quoted symbols are used by these
commands for registering new notations and symbols.
```
quoted_symbol : "`" " "* `quoted_symbol_start` `quoted_symbol_rest`* " "* "`"
quoted_symbol_start: [^0-9"\n\t `]
quoted_symbol_rest : [^"\n\t `]
```
A quoted symbol may contain surrounding whitespace, which is
customarily used for pretty printing the symbol and ignored while
scanning.
While backticks are not allowed in a user-defined symbol, they are
used in some built-in symbols (see :ref:`quotations`), which are
accessible outside of the set of commands noted above.
Doc Comments
============
A special form of comments, doc comments are used to document modules
and declarations.
```
doc_comment: "/--" ([^-] | "-" [^/])* "-/"
mod_doc_comment: "/-!" ([^-] | "-" [^/])* "-/"
```
Field Notation
==============
Trailing field notation tokens are used in expressions such as
``(1+1).to_string``. Note that ``a.toString`` is a single
:ref:`identifier <identifiers>`, but may be interpreted as a field
notation expression by the parser.
```
field_notation: "." ([0-9]+ | `atomic_ident`)
```