Can the existing "vt100" model of terminal emulators be progressively enhanced?

Goals for rich program output:

  • programs should exchange semantically rich information, where pure text is enhanced by (semantic/presentational) metainformation;
    • at the very least: hyperlinked parts of output.
    • better: nested structures (of data/metadata)
  • terminals should make use of the metadata.
  • it should be somewhat backwards-compatible with vt100.
  • compatible with UTF-8 streams.

The state of the art

UTF-8 does not redefine the first 32 characters (CC0, control plane 0), they can keep their TTY meaning. Control plane 1 control characters (0x80-0x9F) clash with UTF-8 (are they even well understood by ttys?). Unicode has its own CC1 though.

# ASCII:
ESC = \x1B = \e = ^[                # Escaping sequence
FS = \x1C                           # File Separator
GS = \x1D                           # Group Separator
RS = \x1E                           # Record Separator
US = \x1F                           # Field Separator

These separators are a great idea for table-structured information (CSV, SQL table), not used in the POSIX world (are they used in IBM OSes?), can be used orthogonally. Linux/vt100 heavily uses ESC.

# Escape sequences
CSI = ESC [                         # Control Sequence Intro
OSC = ESC ]                         # Operating System Command
SOS = ESC X                         # Start Of String
ST = ESC \                          # String Terminator

Linux kernel console_codes man page. Xterm control sequences

Neither Linux, nor xterm do not really use OSC, there is a lot of custom space in there.

Hyperlinks in terminal emulators: OSC 8 allows to introduce hyperlinks in terminals:

<OSC>8;;http://example.com<ST>link text<OSC>8;;<ST>

support for this feature is young, but popular.

Option: json tty

Program output, i.e. the data itself, could be JSON-structured.

Naively: just spit out JSON by default or when the receiving end can accept it. Humans will see the mess and deal with it manually.

More invasive, vt100-incompatible change: mark up data structure with repurposed control characters: e.g. some of CC0 could substitute { } , :, etc (BSON-style). Advantages: precomputed object length (although this is not always possible, e.g. when streaming).

Option: xml tty

The idea is: progressively enhance pure text output with XML markup (the entire web works this way already!). Metadata make data structured, but it's trivial to ignore the structure.

The markup should be semantic, it's up to the user to style it via "CSS" (or ignore the markup altogether).

Applications:

  • for ps output: mark it up as a table, add hyperlinks to OS entities. Make all these ugly tabs and pseudographical tables go away.
  • once terminal emulator understands the structure of a table, it can be interactive (sort by, hide column, scroll, etc).
  • displaying source code: add as much semantic information as you like (tokens, AST nodes, version history/state, etc), the text is still the same.
  • MIME-types for output and custom handlers (e.g. pdf, hex editors, foldable JSON, ...).
  • <img src="data:image/png,..."> instead of sixel!
  • <!DOCTYPE html> directly in terminal.
  • progress bars like in PowerShell.
  • shell <-> emulator interaction becomes a boring streaming XML protocol.
    • <shell:prompt>, <shell:input>, <shell:output>, <shell:stderr>, foldable and searchable output
    • multiplexed ptys become just another protocol.
    • GUI integration becomes flexible and styleable.

Compatibility with unaware receivers:

  • filter metadata out (trivial)
  • make this markup hidden.

Hiding metadata can work well with pseudotty receivers. One possible trick is to use ESC 7 ("Save cursor position"), print metainformation and then clear it with ESC 8 ("Restore cursor position") and ESC [ 0 J ("Clear from cursor to the end of screen"): 

# let's define
<SOM> = <ESC>7               # Start Of Metadata, 2B
<EOM> = <ESC>8<ESC>[0J       # End Of Metadata,   6B

then use this like:

<SOM>invisible metainformation<EOM>regular data

SOM can be like < in XML, EOM like >.

If this becomes popular enough: it can be abbreviated to kernel-supported short and sweet sequences:

<SOM> = <ESC>"<"
<EOM> = <ESC>">"

The inside of metadata can be just XML:

<SOM>a href="http://example.org"<EOM>link text<SOM>/a<EOM>

Hyperlinks can be special-cased:

<SOH> = <ESC>"@"                # Start HYperlink
<EOH> = <ESC>";"                # End HYperlink

# example:
<SOH>http://example.com<ST>link text<EOH>

JSON-like structure markup