This document describes the grammar of Morel (constants, identifiers, expressions, patterns, types, declarations), and then lists its built-in operators, types, functions. Properties affect the execution strategy and the behavior of the shell. The shell is Morel's interactive read-eval-print loop.
Query expressions (from, exists, forall) are described in more
detail in the query reference.
This reference is based on Andreas Rossberg's grammar for Standard ML. While the files have a different notation, they are similar enough to the two languages.
Morel aims to be compatible with Standard ML. It extends Standard ML in areas related to relational operations. Some of the features in Standard ML are missing, just because they take effort to build. Contributions are welcome!
In Morel but not Standard ML:
- Queries (expressions starting with
exists,forallorfrom) withcompute,distinct,except,group,intersect,into,join,order,require,skip,take,through,union,unorder,where,yield,yieldAllsteps andinandofkeywords elem,implies,notelembinary operatorscurrent,elements,ordinalnilary operatorstypeoftype operatortype_stringexpression operatorlab =is optional inexprowrecord.labas an alternative to#lab record; for tuples,tuple.1,tuple.2etc. as an alternative to#1 tuple,#2 tuple- postfix method-call syntax
exp.f ()andexp.f arg, wherefis a function whose first parameter is namedself - identifiers and type names may be quoted
(for example,
`an identifier`) - record modifiers:
replace(functional update for record values, as OCaml'swith),extend,remove,rename, theirorpairs,allandlenient - overloaded functions may be declared using
overandinst - checked types: a
checkcondition on a type declaration, thecheckexpression operator, and theasandasOptconversion operators - attributes (
[@attr]/[@@attr]/[@@@attr]) based on OCaml (*)line comments (syntax as SML/NJ and MLton)(**...*)doc comments (from OCaml)
In Standard ML but not in Morel:
wordconstantlongididentifier- references (
refand operators!and:=) - exceptions:
handleand user-definedexceptiondeclarations (raiseand built-in exceptions are supported) whileloop- data type replication (
type) withtypeindatatypedeclaration- abstract type (
abstype) - structures (
structure) - signature refinement (
where type) - signature sharing constraints
- local declarations (
local) - operator declarations (
nonfix,infix,infixr) openbeforeandooperators
con → int integer
| float floating point
| char character
| string string
int → [~]num decimal
| [~]0xhex hexadecimal
float → [~]num.num floating point
| [~]num[.num]e[~]num
scientific
char → #"ascii" character
string → "ascii*" string
num → digit digit* number
hex → (digit | letter) (digit | letter)*
hexadecimal number (letters
may only be in the range A-F)
ascii → ... single non-" ASCII character
or \-headed escape sequence
A string literal may span lines. A newline in a string is part of the
string; but a \ at the end of a line is a line continuation: the
backslash, the newline, and the spaces and tabs that begin the next
line are ignored. For example,
val text = "This is a very \
long string split \
across lines.";gives "This is a very long string split across lines.".
id → letter (letter | digit | ''' | _)*
alphanumeric
| symbol symbol* symbolic (not allowed for type
variables or module language
identifiers)
symbol → !
| %
| &
| $
| #
| +
| -
| /
| :
| <
| =
| >
| ?
| @
| \
| ~
| `
| ^
| '|'
| '*'
var → '''(letter | digit | ''' | _)*
unconstrained
''''(letter | digit | ''' | _⟩*
equality
lab → id identifier
num number (may not start with 0)
exp → con constant
| [ op ] id value or constructor identifier
| exp1 exp2 application
| exp1 id exp2 infix application
| '(' exp ')' parentheses
| '(' exp1 , ... , expn ')' tuple (n ≠ 1)
| { [ exprow ] } record
| { exp modifier1 ... modifiern }
modified record (n ≥ 1)
| #lab record selector
| '[' exp1 , ... , expn ']' list (n ≥ 0)
| '(' exp1 ; ... ; expn ')' sequence (n ≥ 2)
| let dec in exp1 ; ... ; expn end
local declaration (n ≥ 1)
| exp . lab () postfix call (no argument)
| exp1 . lab exp2 postfix call (with argument)
| exp : type type annotation
| exp as typ conversion (raises on failure)
| exp asOpt typ conversion (answers an option)
| exp check match checked expression
| exp1 andalso exp2 conjunction
| exp1 orelse exp2 disjunction
| if exp1 then exp2 else exp3
conditional
| case exp of match case analysis
| raise exp exception raising
| type_string exp type of exp as a string (exp is not evaluated)
| fn match function
| current current element (only valid in a query step)
| elements elements of current group (only valid in compute)
| ordinal element ordinal (only valid in a query step)
| exp1 over exp2 aggregate (only valid in compute)
| from [ scan1 , ... , scans ] step1 ... stept [ terminalStep ]
relational expression (s ≥ 0, t ≥ 0)
| exists [ scan1 , ... , scans ] step1 ... stept
existential quantification (s ≥ 0, t ≥ 0)
| forall [ scan1 , ... , scans ] step1 ... stept require exp
universal quantification (s ≥ 0, t ≥ 0)
| exp expAttr1 ... expAttrn
attributed expression (n ≥ 1)
exprow → exprowItem [, exprowItem ]*
expression row
exprowItem → [ lab = ] exp
modifier → assignVerb exprow assign to fields
| assignVerb all exp assign to every field of exp
| remove [ or skip ] lab [, lab ]*
remove fields
| rename lab = lab [, lab = lab ]*
rename fields
assignVerb → extend [ or ( skip | replace [ lenient ] ) ]
| replace [ lenient ] [ or skip ]
match → matchItem [ '|' matchItem ]*
match
matchItem → pat => exp
scan → pat in exp [ on exp ] iteration
| pat = exp single iteration
| pat unbounded scan
step → distinct distinct step
| except [ distinct ] exp1 , ... , expe
except step (e ≥ 1)
| group exp1 [ compute exp2 ]
group step
| intersect [ distinct ] exp1 , ... , expi
intersect step (i ≥ 1)
| [ left | right | full ] join scan1 , ... , scans
join step (s ≥ 1)
| order exp order step
| skip exp skip step
| take exp take step
| through pat in exp through step
| union [ distinct ] exp1 , ... , expu
union step (u ≥ 1)
| where exp filter step
| yield exp yield step
| yieldAll exp yieldAll step
terminalStep → into exp into step
| compute exp compute step
groupKey → [ id = ] exp
agg → [ id = ] exp [ of exp ]
Inside braces, an expression may be followed by modifiers, each applied to the record that the modifier before it produced:
{{comm = 2.0, ename = "Shaggy", sal = 10.0}
replace sal = sal * 12.0
extend tax = sal * 0.3
remove comm};
> val it = {ename="Shaggy",sal=120,tax=36} : {ename:string, sal:real, tax:real}A modifier is one or two verbs, an optional lenient, an optional
all, and an operand. The verbs are replace, remove and skip,
which act on a label the record has, and extend and skip, which act
on one it does not have. A single verb makes the other case a
compile-time error; a pair joined by or handles both, and since each
verb names its own case the pair is unordered. skip does nothing, and
takes whichever case the other verb does not.
| modifier | label exists | label absent |
|---|---|---|
extend |
error | add |
replace |
replace | error |
remove |
remove | error |
extend or replace |
replace | add |
extend or skip |
keep | add |
replace or skip |
replace | skip |
remove or skip |
remove | skip |
Assignment preserves a field's type; lenient relaxes that check, and
is allowed only after replace or extend or replace. all exp
applies the verbs to every field of the record exp instead of to
a list of assignments. rename gives the value of each right-hand label
to the left-hand one, and removes the right-hand labels; labels may
therefore be swapped or chained.
Within one modifier the assignments are simultaneous, so
{r replace i = j, j = i} swaps two fields; a later modifier sees the
result of an earlier one, so {r replace i = j replace j = i} does not.
An assignment sees the fields of the record the modifier is applied to,
and they shadow the enclosing environment.
A modifier always applies to a base expression, so it belongs to the
braces it is written in, never to a field of the record being built:
the arguments of a modifier are plain fields, and a modifier keyword
among them starts another modifier of the same record, so in
{r replace i = 1 remove j} the remove applies to r, not to 1. To
modify a field's value, give it braces of its own --
{a = 1, b = {b remove x}} -- or, as elsewhere, leave the label out
and let the field take the one its base implies:
{a = 1, {b remove x}}.
extend, remove, rename and replace are reserved words, and need
back-ticks to be used as identifiers. all, lenient and or are
not: they are keywords only in the positions above, and are ordinary
identifiers everywhere else.
A check clause on a type declaration states a condition that every value
of the type satisfies:
type nat = int check i => i >= 0;
val n: nat = 5;
> val n = 5 : nat
val bad: nat = ~1;
> uncaught exception Constraint [~1 is not a valid nat]
A clause may be repeated, and each adds a condition. The match may have several branches and need not be exhaustive; a value that no branch matches does not have the type. The value the match binds may be destructured.
A condition must be closed: it may refer only to the value it is given and to the standard basis. That is what lets a checked type be interned like any other type -- two are the same type when their conditions are textually equal -- and it settles what a condition means when the names it used are re-bound. Shadowing a basis name does not evade this: the binding decides, not the name.
A checked type is erased. Its representation is that of the type it
abbreviates, and everything that examines a type structurally -- choosing an
overload, aggregating, printing -- behaves as it does for the base type. So
widening is free and narrowing is checked: using a nat as an int needs no
coercion, and n - 100 is ordinary int subtraction with type int.
A condition is claimed only where the type says so, and is checked wherever a value flows into a claim: a binding, a function parameter, an ascription, a conversion, a datatype constructor, and inside a composite value, where the message names the component that failed. A scan over a checked type conjoins the condition into the query, so a generator enumerates only values of the type.
Two conversion operators ask rather than claim. exp as typ converts, raising
Constraint if the condition does not hold; exp asOpt typ answers NONE
instead. Both bind as loosely as :, are left-associative, and may be mixed
with it.
A condition may also be written on an expression, exp check match, which
adds to what the expression already claims rather than replacing it, and on
any type, not only one that is named. check binds more loosely than
anything else in a type or an expression, so parentheses put the condition on
a component instead.
A function type may not be claimed -- honouring the claim would mean inserting a check at every call site -- and a parameterized type may not carry a condition. Both are reported rather than silently ignored.
check and asOpt are reserved words, and need back-ticks to be used as
identifiers.
pat → con constant
| _ wildcard
| [ op ] id variable
| [ op ] id [ pat ] construction
| pat1 id pat2 infix construction
| '(' pat ')' parentheses
| '(' pat1 , ... , patn ')' tuple (n ≠ 1)
| { [ patrow ] } record
| '[' pat1 , ... , patn ']' list (n ≥ 0)
| pat : type type annotation
| id as pat layered
patrow → '...' wildcard
| lab = pat [, patrow] pattern
| id [ : type ] [, patrow] label as variable
typ → var variable
| [ typ ] id constructor
| '(' typ [, typ ]* ')' id constructor
| '(' typ ')' parentheses
| typ1 -> typ2 function
| typ1 '*' ... '*' typn tuple (n ≥ 2)
| { [ typrow ] } record
| typeof exp expression type
| typ check match1 ... check matchn
checked type (n ≥ 1)
| typ expAttr1 ... expAttrn
attributed type (n ≥ 1)
typrow → lab : typ [, typrow] type row
dec → vals valbind value
| fun funbind function
| type typbind type
| datatype datbind data type
| signature sigbind signature
| over id overloaded name
| dec declAttr1 ... declAttrn
attributed declaration (n ≥ 1)
| floatingAttr floating attribute
| empty
| dec1 [;] dec2 sequence
valbind → pat = exp [ and valbind ]*
destructuring
| rec valbind recursive
| inst valbind overload instance
funbind → funmatch [ and funmatch ]*
clausal function
funmatch → funmatchItem [ '|' funmatchItem ]*
funmatchItem → [ op ] id pat1 ... patn [ : type ] = exp
nonfix (n ≥ 1)
| pat1 id pat2 [ : type ] = exp
infix
| '(' pat1 id pat2 ')' pat'1 ... pat'n [ : type ] = exp
infix (n ≥ 0)
typbind → [ vars ] id = typ [ and typbind ]*
abbreviation
datbind → datbindItem [ and datbindItem ]*
data type
datbindItem → [ vars ] id = conbind
conbind → conbindItem [ '|' conbindItem ]*
data constructor
conbindItem → id [ of typ ]
vals → val
| '(' val [, val]* ')'
vars → var
| '(' var [, var]* ')'
expAttr → '[@' attrName [ payload ] ']'
expression / type attribute
declAttr → '[@@' attrName [ payload ] ']'
declaration attribute
floatingAttr → '[@@@' attrName [ payload ] ']'
floating attribute
attrName → id [ . id ]*
dotted attribute name
payload → exp expression payload
| : type type payload
sigbind → id = sig spec end [ and sigbind ]*
signature
spec → val valdesc value
| type typdesc abstract type
| type typbind type abbreviation
| datatype datdesc data type
| exception exndesc exception
| empty
| spec1 [;] spec2 sequence
valdesc → id : typ [ and valdesc ]*
value specification
typdesc → [ vars ] id [ and typdesc ]*
type specification
datdesc → datdescItem [ and datdescItem ]*
datatype specification
datdescItem → [ vars ] id = conbind
exndesc → id [ of typ ] [ and exndesc ]*
exception specification
A signature defines an interface that specifies types, values, datatypes, and exceptions without providing implementations. Signatures are used to document module interfaces and, in future versions of Morel, will be used to constrain structure implementations.
Signature declarations appear at the top level (see grammar in Declarations).
A signature body contains specifications that describe the interface:
Value specifications declare the type of a value without defining it:
val empty : 'a stack
val push : 'a * 'a stack -> 'a stackType specifications can be abstract (no definition) or concrete (type alias):
type 'a stack (* abstract type *)
type point = real * real (* concrete type alias *)
type ('k, 'v) map (* abstract with multiple params *)Datatype specifications describe algebraic datatypes:
datatype 'a tree = Leaf | Node of 'a * 'a tree * 'a treeException specifications declare exceptions:
exception Empty (* exception without payload *)
exception QueueError of string (* exception with payload *)A simple signature with abstract type and value specifications:
signature STACK =
sig
type 'a stack
exception Empty
val empty : 'a stack
val isEmpty : 'a stack -> bool
val push : 'a * 'a stack -> 'a stack
val pop : 'a stack -> 'a stack
val top : 'a stack -> 'a
endMultiple signatures declared together using and:
signature EQ =
sig
type t
val eq : t * t -> bool
end
and ORD =
sig
type t
val lt : t * t -> bool
val le : t * t -> bool
endThe current implementation supports parsing and pretty-printing signatures but does not yet support:
- Structure declarations that implement signatures
- Signature refinement (
where type) - Signature sharing constraints
- Signature inclusion (
include)
These features may be added in future versions.
This grammar uses the following notation:
| Syntax | Meaning |
|---|---|
| symbol | Grammar symbol (e.g. con) |
| keyword | Morel keyword (e.g. if) and symbol (e.g. ~, "(") |
| [ term ] | Option: term may occur 0 or 1 times |
| [ term1 | term2 ] | Alternative: term1 may occur, or term2 may occur, or neither |
| term* | Repetition: term may occur 0 or more times |
| 's' | Quotation: Symbols used in the grammar — ( ) [ ] | * ... — are quoted when they appear in Morel language |
| Operator | Precedence | Meaning |
|---|---|---|
| ~ | prefix 8 | Negate |
| * | infix 7 | Multiplication |
| / | infix 7 | Division |
| div | infix 7 | Integer division |
| mod | infix 7 | Modulo |
| + | infix 6 | Plus |
| - | infix 6 | Minus |
| ^ | infix 6 | String concatenate |
| :: | infixr 5 | List cons |
| @ | infixr 5 | List append |
| <= | infix 4 | Less than or equal |
| < | infix 4 | Less than |
| >= | infix 4 | Greater than or equal |
| > | infix 4 | Greater than |
| = | infix 4 | Equal |
| <> | infix 4 | Not equal |
| elem | infix 4 | Member of list |
| notelem | infix 4 | Not member of list |
| := | infix 3 | Assign |
| o | infix 3 | Compose |
| andalso | infix 2 | Logical and |
| orelse | infix 1 | Logical or |
| implies | infix 0 | Logical implication |
~ binds tighter than every infix operator, so ~x div 2 is
(~x) div 2, and looser than function application, so ~f x negates
the result of the call.
abs is a built-in function (not an operator, because it uses function
syntax rather than prefix or infix syntax). It is overloaded: its type is
int -> int when applied to an int argument, and real -> real when
applied to a real argument. It is equivalent to Int.abs and Real.abs
respectively.
Primitive: bool, char, int, real, string, unit
Datatype:
datatype 'a descending = DESC of 'a(in structureRelational)datatype ('l, 'r) either = INL of 'l | INR of 'r(in structureEither)datatype 'a list = nil | :: of 'a * 'a list(in structureList)datatype 'a option = NONE | SOME of 'a(in structureOption)datatype 'a order = LESS | EQUAL | GREATER(in structureGeneral)
Eqtype:
eqtype 'a bag = 'a bag(in structureBag)eqtype 'a vector = 'a vector(in structureVector)
Exception:
Bind(in structureGeneral)Chr(in structureGeneral)Div(in structureGeneral)Domain(in structureGeneral)Empty(in structureList)Error(in structureInteract)Option(in structureOption)Overflow(in structureOption)Size(in structureGeneral)Subscript(in structureGeneral)Unordered(in structureIEEEReal)
Each property is set using the function Sys.set (name, value),
displayed using Sys.show name,
and unset using Sys.unset name.
Sys.showAll () shows all properties and their values.
| Name | Type | Default | Description |
|---|---|---|---|
| banner | string | Morel version ... | Startup banner message displayed when launching the Morel shell. |
| colorScheme | string option | NONE | Color scheme for syntax highlighting in the shell: a built-in scheme ('dark', 'light' or 'none'), or a user-defined scheme. If unset, the scheme is deduced from the environment. |
| directory | file | Path of the directory that the 'file' variable maps to in this connection. | |
| excludeStructures | string | ^Test$ | Regular expression that controls which built-in structures are excluded from the environment. |
| hybrid | bool | false | Whether to try to create a hybrid execution plan that uses Apache Calcite relational algebra. |
| inlinePassCount | int | 5 | Maximum number of inlining passes. |
| lineWidth | int option | 79 | When printing, the length at which lines are wrapped. Must not be negative; NONE means that lines are not wrapped. |
| matchCoverageEnabled | bool | true | Whether to check whether patterns are exhaustive and/or redundant. |
| matchStrict | bool | false | Whether the script-test harness compares output verbatim, rather than modulo whitespace and bag-element order. |
| maxUseDepth | int option | 50 | How deeply the 'use' command may nest. Must not be negative; NONE means no limit. |
| now | string option | NONE | Overrides the current time. Value is an ISO-8601 string (e.g. '2024-01-01T00:00:00Z'). If not set, the system clock is used. |
| output | enum | classic | How values should be formatted. "classic" (the default) prints values in a compact nested format; "tabular" prints values in a table if their type is a list of records. |
| printDepth | int option | 5 | When printing, the depth of nesting of recursive data structure at which ellipsis begins. Must not be negative; NONE means that values are printed in full. |
| printLength | int option | 12 | When printing, the length of lists at which ellipsis begins. Must not be negative; NONE means that lists are printed in full. |
| productName | string | morel-java | Name of the Morel product. |
| productVersion | string | 0.9.0 | Current version of Morel. |
| rangeMaxLength | IntInf.int | 16777215 | Largest number of values that expanding a range may produce. Must be positive. |
| relationalize | bool | false | Whether to convert to relational algebra. |
| scriptDirectory | file | Path of the directory where the 'use' command looks for scripts. When running a script, it is generally set to the directory that contains the script. | |
| stringDepth | int option | 70 | When printing, the length of strings at which ellipsis begins. Must not be negative; NONE means that strings are printed in full. |
| stringFold | int option | NONE | In tabular mode, the column width at which long strings are folded across multiple lines. Must be positive; NONE disables folding. |
| terminalBackground | string option | NONE | The terminal's background color, of the form 'rgb:RRRR/GGGG/BBBB'. Set by the shell at startup; used to deduce the color scheme when 'colorScheme' is unset. |
| timeZone | string option | NONE | Overrides the local timezone. Value is a timezone ID (e.g. 'UTC' or 'America/New_York'). If not set, the JVM default timezone is used. |
When the hybrid property is set, Morel tries to translate each query
into Apache Calcite relational algebra,
falling back to its own evaluator for any part it cannot translate.
Limitations include:
minandmaxcannot be pushed down forwordvalues or composite values (tuples, records, lists, and datatypes such asoption).minByandmaxBycannot be pushed down.
Morel includes an interactive shell (a read-eval-print loop, or REPL).
Launch it by running morel with no arguments:
$ ./morel
morel-java version 0.9.0 (java version "25", JLine terminal, xterm-256color)
- "Hello, world!";
val it = "Hello, world!" : string
-
At the - prompt, type an expression or declaration followed by a
semicolon, and Morel evaluates it and prints the result. A statement may
span several lines; while it is incomplete, the prompt changes to =.
Type Ctrl-D (or the exit command) to quit.
The shell remembers the commands you type, so that you can recall and
edit earlier commands using the up and down arrow keys. History is saved
between sessions in the file ~/.morel/history, and is reloaded when the
shell starts. If the ~/.morel directory does not exist, the shell
creates it.
Each line in the history file consists of a Unix timestamp, a colon, and the text of a command. If the file cannot be created — for example, if the home directory is not writable — the shell prints a warning and continues without saving history for that session.
In an interactive terminal, the shell highlights the code you type, coloring each token according to its category: keyword, symbol, numeric, constant, string, comment, and type variable. (Plain identifiers are left in the terminal's default color.)
The color scheme is chosen by the colorScheme property,
whose value is one of:
dark— colors tuned for a dark terminal background;light— colors tuned for a light terminal background;none— no highlighting;- the name of a user-defined scheme.
If the property is unset (the default), the scheme is deduced from the
environment by Sys.deduceColorScheme:
none if the terminal does not support color (if standard output is not a
terminal, NO_COLOR is set, or TERM is dumb), otherwise light or
dark according to the terminal's background. The shell determines the
background by asking the terminal for its background color (the OSC 11
escape sequence); outside the shell, or if the terminal does not answer, it
falls back to the COLORFGBG environment variable, and otherwise to
dark.
Choose a scheme at startup with the --color-scheme flag:
$ ./morel --color-scheme=lightor at any time from within the shell:
- Sys.set ("colorScheme", "none");
A future release will let you define your own schemes as properties files
in ~/.morel/color-schemes/.