← playground

Muon spec

muon, mu object notation

Mu object notation, Muon, is a permissive textual format for structured data, which is also a substrate for describing Mu source code via additional higher-level parsing. Muon is a superset of Mu source code.

Muon structure is designed to reflect human visual perception of source code and to be loosely compatible with other structured formats (e.g. json).

Its top-level entity is a "sequence", <seq>. A standalone Muon file is a <seq> (maybe empty one).

whitespace and newline handling

Muon is indentation-insensitive, but it is sensitive to newlines, \n. Any ebnf rules defined below:

Allowed whitespace characters are (\x20), \r, \t.

\r\n in muon text is treated as a whitespace character \r, followed by <sep>. \r\n <-> \n conversions do not change the parsed muon structure.

Muon has one position-dependent rule: a #! at byte offset 0 opens a shebang comment (see <comm>), so a file may start with #!/usr/bin/env elly. Everywhere else # is the <prefixed> sigil. Nothing else in the notation depends on absolute position.

To convert multi-line representation into one-line representation (e.g. for minification):

grammar

<seq>

A <seq> consists of zero or more "chains", <chain>, separated by separator <sep> (can be a newline or a comma interchangeably, to format long chains as lines and short chains as a comma-separated lists in one line).

Zero chains is a valid empty seq.

Preceding/trailing/duplicate separators are valid, to allow for blank lines visually, hanging commas in lists, newline before EOF, etc. The content of a sequence is defined by chains inside of it, whereas separators only delimit chains. Chains by their definition are non-empty, but empty space between <sep> is allowed and does not produce a chain.

(,,,)      // no chains inside
(a b, c)   // two chains: `a b`, `c`
(, ,,a b, , c,,)  // same

In source code, a sequence corresponds to the inner content of a module, a <parens>, a <braces>, a <brackets>, etc.

<sep> ::= "\n" | ","
<seq> ::= <chain>? (<sep> <chain>?)*

<chain>

A chain, <chain>, is a juxtaposition of one or more items: runs of atoms, <run>, plus the between-items kinds, <punct> and <comm>. Whitespace between chain items is optional but significant: two atoms written with no whitespace between them are one item (a <run>), the same two atoms with whitespace are two items.

Visually, a chain is a "sentence" of "items": atoms glued with no whitespace form one item (a <run>), and a <punct> or <comm> stands between items.

A chain cannot be empty (but a place between two <sep> in a <seq> can be, this does not produce a chain).

In source code, a chain is roughly an "expression"/"statement". Source code can treat multiple adjacent chains as one expression (e.g. to allow visual separation of declaration and definition of a function, to spread long expressions) according to simple unambiguous rules.

<chain> ::= (<run> | <punct> | <comm>)+

<run>

A run, <run>, is a sequence of <atom>s written with no whitespace between them. When two or more glue, they form one chain item; a lone atom is not a run — it stays a bare item, so a chain of spaced atoms builds the same tree it built before runs existed.

Both sides of every item boundary are decided by adjacency: x.0 is one run, x .0 is two items. A <punct> and a <comm> are not atoms: they end a run and join none, so a:b stays three items, x|>f keeps the |> free, and a run never crosses a <sep>.

Any atom may start or continue a run, and a sigil binds tighter than the gluing: & and . each take the one atom that follows, and the run groups afterwards, so &x.0 is the run &x .0, not &(x.0), and .a.b is the run .a .b, not .(a.b).

(* no whitespace between the atoms of a run *)
<run>  ::= <atom>+
<atom> ::= <str> | <sym> | <parens> | <brackets> | <braces> | <prefixed>

<item>

An item is one of the visually atomic atoms — a number/string literal, a symbolic name, or something clearly delineated by an opening and a closing element — or one of the two between-items kinds, a <punct> or a <comm>:

Atoms written with no whitespace between them are grouped into one item, a <run> (see <run>); a <punct> and a <comm> never join a run.

A prefixed item is a leading sigil (&, ., or #) immediately followed (no whitespace) by any <item>. Muon assigns it no meaning — it is structure only, just like a : (<punct>) or -1 (<sym>) — leaving the interpretation to higher levels (e.g. Elly uses &<item> to introduce a binding, .<item> for a symbol / projection, and #<item> for its list / map literals; see elly-spec.md). The . sigil applies only when the dot is glued to an item; a dot not followed by an item is a <punct> (see <prefixed> and <punct> below).

<item> ::= <atom> | <punct> | <comm>

Item nesting (via parens, brackets, braces, prefixed items, or runs) is unbounded: the reference parser walks nesting with an explicit heap stack rather than recursion, so depth is limited only by available memory, never by the parser's own call stack. An implementation may still impose its own depth cap (RFC 8259 §9 permits one), but Muon's grammar sets none.

<prefixed>

A prefixed item is a sigil directly attached to an item. The sigils are &, ., and #. There is no whitespace between the sigil and the item. A sigil takes exactly the one item that follows, before any run gluing: &x.0 is the run &x .0 (a & over x, glued to a .0), not &(x.0), and .a.b is the run .a .b, not .(a.b).

Muon assigns none of the sigils any meaning — a prefixed item is structure only, interpreted by higher levels (e.g. Elly uses &<item> for a binding, .<item> for a symbol / projection, and #<item> for its list / map literals; see elly-spec.md).

The sigils differ in how they handle a dangling sigil (one not directly followed by an item):

(* no whitespace between sigil and item *)
<prefixed> ::= <sigil> <item>
<sigil>    ::= "&" | "." | "#"   (* "." only when right-glued to an item; else <punct> *)

<comm>

Comments are valid atomic items which preserve their inner unicode textual content literally. They are preserved at the Muon level and usually are ignored by higher level semantics. This still allows to reuse them in tools like formatters.

Visually, comments are C-style // one-line and /*-*/ multiline comments. A third form, #!, is a comment only at byte offset 0 — a shebang line — so a Muon file may begin with #!/usr/bin/env elly. It runs to the end of its line, like //, and is stored raw with its #!. Anywhere but offset 0, # is the <prefixed> sigil (see <prefixed>), so this is the notation's one position-dependent rule.

Comments are not nested:

<comm> ::=
  | "//" (any character except "\n")*
  | "/*" (any character sequence without "*/") "*/"
  | "#!" (any character except "\n")*   (* only at byte offset 0 *)

Newlines inside multiline comments are not <sep> and are preserved.

<str>

String literals are wrapped in ". They contain any unescaped Unicode characters (except a literal newline or unescaped ") or escape sequences starting with \.

Escape sequences are Muon's own — \", \n, \t, \r, \\ — plus the rest of JSON's (RFC 8259 §7): \/, \b, \f, and \uXXXX (exactly four hex digits). Any other character after \, or a \u not followed by four hex digits, is a syntax error, open to changes in the future.

Escapes are validated but not decoded: the inner slice is stored raw (backslash and all), consistent with the zero-copy design. In particular Muon does not combine \uXXXX surrogate pairs or interpret code points — that is decoding, a higher layer's job (see docs/todo/elly-json.md), so a lone surrogate is accepted at this layer. This makes Muon a parse-level superset of JSON strings: every valid JSON string literal parses.

<str>     ::= "\"" (<strchar> | <esc>)* "\""
<strchar> ::= (* any utf-8 character except unescaped '"', '\', or literal newline *)
<esc>     ::= "\" ("\"" | "n" | "t" | "r" | "\\" | "/" | "b" | "f" | "u" <hex> <hex> <hex> <hex>)
<hex>     ::= "0".."9" | "a".."f" | "A".."F"

TODO: \u{NNNN}, \xNN encoding? — an explicit future question, separate from the JSON \uXXXX above.

<sym>

Symbols are any (non-whitespace) sequences of one or more of:

A symchar is exactly a character that can occur inside a single atomic literal — a number or an identifier. That is why -, + are symchars (-1, +123) but : and . are not: neither occurs inside a literal, so : is a <punct> (below) and . is a sigil-or-punct (see <prefixed> and <punct>). A consequence is that a . breaks a symbol: foo.bar is not one symbol but the run foo .bar (<sym> glued to a .-prefixed item), and 3.14 is the run 3 .14.

They can encode:

(* no whitespace *)
<sym> ::= <symchar>+
<symchar> ::= "0".."9" | "A".."Z" | "a".."z"
            | `_` | `-` | `+`

TODO: +, - genuinely belong to both classes (they are operators too), but they stay symchars so numeric literals stay whole. A consequence is that a token cannot straddle the two classes by maximal munch: :+ is <punct> : then <sym> +, and a future -> would not munch as one token. Cross-class multi-char tokens (like ->, or / once it joins a class) would be handled as explicit lexical exceptions, the way // and /* already are. (. used to be a symchar too, for 3.14 / foo.bar; it has been split out as a sigil/punct — float-literal notation is deferred until Elly grows a Num type.)

<punct>

Punctuation, <punct>, is a maximal run of one or more punctchars — non- whitespace characters that stand between items rather than occurring inside a literal. The punctchars are :, =, |, >, and <. Like <sym>, Muon assigns <punct> no meaning: they are just structure, read by higher levels (e.g. Elly reads key : value as a map entry, x = 1 as a binding, x |> f as a pipe, < e / > e as ordering patterns, or a future => as an arrow; see elly-spec.md).

Punctuation is maximal munch like <sym>, so :: and == are each a single <punct> (write : : / = = for two), and — since :, =, |, >, and < share the class — :=, |=, |>, =>, and <= munch as one punct too. Adding a further punctchar lets a new operator fall out the same way, with no per-operator rule.

A . is also a <punct>, but a special one: it is a standalone single-character punct and is not a punctchar, so it never joins the :/=/| run (:. is <punct> : then <punct> ., not one token). A . is this punct only when it is not glued to an item; a . directly followed by an item is instead the sigil of a <prefixed> (see <prefixed>). Muon gives the . punct no meaning either — a higher level (Elly) reads a spaced . as an application / composition combinator.

(* no whitespace *)
<punct> ::= <punctchar>+ | "."
<punctchar> ::= ":" | "=" | "|" | ">" | "<"

<parens>

A <parens> is a <seq> wrapped in parentheses. It's used for grouping chains together. Its exact semantics is defined by higher levels (e.g. in Elly).

Empty parens, ()/( )/(,,)/etc, is valid and wraps an empty sequence.

Since the content of a <parens> is a <seq>, they can spread over multiple lines:

(a, (b, c))  
// is equivalent to
(a
(b,c))
// is equivalent to
(
  a
  (b, c)
)
// is equivalent to
(
  a
  (
    b
    c
  )
)

Definition:

<parens> ::= "(" <seq> ")"

<brackets>

A <brackets> is a <seq> wrapped in brackets. Like <parens>, it only groups chains; Muon assigns it no meaning, leaving it to higher levels (e.g. Elly reads a #-prefixed <brackets> — #[], #[.x], #[1, .two] — as a runtime list value; see docs/elly-spec.md).

Empty brackets, []/[ ]/[,,]/etc, is valid and wraps an empty sequence. Its content being a <seq>, a <brackets> can spread over multiple lines exactly like a <parens>. Juxtaposition without whitespace glues an atom to the next, so m[k] is one run, a <sym> glued to a <brackets> (mirroring f(x) as a run of <sym> and <parens>); a space — m [k] — un-glues it into two items.

<brackets> ::= "[" <seq> "]"

<braces>

A <braces> is a <seq> wrapped in braces. Like <parens> and <brackets>, it only groups chains; Muon assigns it no meaning (e.g. Elly reads a #-prefixed <braces> — #{}, #{ .k: v } — as a map literal; see docs/elly-spec.md).

Empty braces, {}/{ }/{,,}/etc, is valid and wraps an empty sequence, and it follows the same multi-line and juxtaposition rules as a <parens> and a <brackets>.

<braces> ::= "{" <seq> "}"

json compatibility

Requires TODOs: