marianoguerra/pure-py/eval

marianoguerra/pure-py/eval does not have a README file

Dispatch

pub(all) enum Dispatch {
  Taken(@moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value], Array[@marianoguerra/pure-py/ast.Stmt
Stmt])
  FellThrough
  DispatchStuck(String)
}

What a match statement does with a value.

Host

source

pub struct Host {
  write : (String) -> Unit
  argv : Array[String]
  modules : Map[String, @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value]]
  call : async (String, Array[@marianoguerra/pure-py/value.Value
Value]) -> @marianoguerra/pure-py/value.Outcome
Outcome noraise
}

What the host supplies: everything about a PurePy run that is not pure.

PurePy is a pure language, but a RUN is not a pure thing: it prints, it reads its arguments, it loads modules, and -- when it is embedded -- it calls out to whatever the application wants to expose. Those four are the whole of it, and they are all here, so an embedder can see the surface at once instead of discovering it a call at a time.

Output. write receives what print produced, one call per print, while the run is still going. A host that streams to a terminal writes it out; a host that wants a transcript accumulates.
Arguments. argv is what sys.argv reads. argv[0] is the program's own name, as in Python.
Modules. modules are importable modules the host defines, beyond the five the specification predefines. Their members are values, and a member may be a function the host implements.
Foreign calls. call answers a call to one of those functions. It is given the name the host registered and the arguments the guest passed, and returns an Outcome: a value, an abort, or -- for anything it does not recognise or cannot do -- an operation the semantics does not cover.
It is async, so it may also answer LATER: a host that has to read a socket, await a promise or ask a person suspends, and the whole run parks where it stood -- mid-expression, inside a call, anywhere -- and resumes on the value the host eventually supplies. Nothing about the guest changes: PurePy has no await, and a program cannot tell a call that answered late from one that answered at once. See run_with for how a synchronous embedder drives a run that can park.

Everything else about a run is already a value: the program is a SourceTree, so a host that keeps its guest code in a database, a zip file or a string never touches the filesystem.

Note what is NOT here. There is no way for the host to mutate a guest value, because there is no mutation; no way to install a callback the guest invokes implicitly, because there are no hooks; and no ambient authority at all -- a guest can reach exactly the modules the host handed it. A host that gives no modules gives a program that can only compute and print.

Suspension adds nothing to that list. A parked run is not a concurrent one: there is one guest, it is at exactly one point, and the host holds the only continuation. Resuming it twice is the host resuming it twice, which is the host's bug and not an escape from the semantics.

Host::member_names

source

fn Host::member_names(self : Host) -> Map[String, Array[String]]

The names of each host module's members.

The CHECKER needs these and not the values: a from hostmod import f has to type-check before it can run, and the checker has no business knowing what f is. This is the one place the two sides of a host module meet.

Host::module_names

source

fn Host::module_names(self : Host) -> Array[String]

The modules a program may import that have no source: the five the specification predefines, and the host's own.

Host::new

source

fn Host::new(write? : (String) -> Unit, argv? : Array[String], modules? : Array[HostModule], call? : async (String, Array[@marianoguerra/pure-py/value.Value
Value]) -> @marianoguerra/pure-py/value.Outcome
Outcome noraise) -> Host

A host.

Every part has a default that does nothing observable: output is dropped, sys.argv is empty, no modules are defined, and a foreign call is an operation the semantics does not cover. A host supplies the parts it wants.

A call that answers on the spot is written exactly as it was before it could do otherwise: a plain function is a valid async one.

HostModule

source

pub(all) struct HostModule {
  name : String
  members : Array[(String, @marianoguerra/pure-py/value.Value
Value)]
}

A module the host defines: a name the guest can import, and its members.

A member is any Value. @value.host_fn("name") makes one the guest can call and the host answers by name.

Interp

source

pub struct Interp {
  tree : @marianoguerra/pure-py/program.SourceTree
SourceTree
  host : Host
  loaded : Map[String, @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value]]
  loading : Array[String]
  depth : Int
  max_depth : Int
  fuel : Int
  max_steps : Int
  profile : @marianoguerra/pure-py/profile.Profile
Profile
  in_module : String
  abort_at : Site?
}

Interp::apply

source

async fn Interp::apply(self : Interp, callee : @marianoguerra/pure-py/value.Value
Value, actual : Array[@marianoguerra/pure-py/value.Value
Value]) -> @marianoguerra/pure-py/value.Outcome
Outcome noraise

eval-call-lambda, eval-call-def, eval-call-prim and their arity rules.

An entry into the machine, so that an embedder calling a guest function gets the same unbounded recursion a guest call inside the machine gets. The depth this call takes is counted there, by the call boundary it pushes.

Interp::dispatch

source

async fn Interp::dispatch(self : Interp, env : @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value], v : @marianoguerra/pure-py/value.Value
Value, cases : Array[@marianoguerra/pure-py/ast.MatchCase
MatchCase]) -> Dispatch noraise

dispatch(ρ, v, p⃗, s⃗): the first case whose pattern matches, with its bindings; no bindings and pass if none does.

Interp::eval_body

source

async fn Interp::eval_body(self : Interp, body : Array[@marianoguerra/pure-py/ast.Stmt
Stmt], env : @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value]) -> @marianoguerra/pure-py/value.StmtResult
StmtResult noraise

Interp::eval_expr

source

async fn Interp::eval_expr(self : Interp, e : @marianoguerra/pure-py/ast.Expr
Expr, env : @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value]) -> @marianoguerra/pure-py/value.Outcome
Outcome noraise

Expression evaluation: Figures 4.5 to 4.10.

Every rule that needs a condition -- if, and, or, a conditional expression, assert -- requires an actual True or False. Python's truthiness is not PurePy's, so if 5: has no rule and the run is undefined. That is one of the conformance suite's dynamically excluded tests, and it is the reason Stuck is threaded everywhere.

Where an abort happened is recorded by sited, at each of the six places an expression can produce one. Not by wrapping this function: it recurses and it is async, so a step after the recursive call is a continuation allocated per expression rather than a tail call, and that measured 21% on a program that does nothing but call.

Interp::eval_seq

source

async fn Interp::eval_seq(self : Interp, items : ArrayView[@marianoguerra/pure-py/analysis.Statement
Statement], env : @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value]) -> @marianoguerra/pure-py/value.StmtResult
StmtResult noraise

Statement evaluation: Figures 4.3 and 4.4.

A sequence's result is the bindings the SEQUENCE made, not the whole environment: eval-seq composes assigns ρ' with what follows, and the caller overrides its own environment with the result. That is what keeps a function body's locals out of its caller.

Interp::load

source

async fn Interp::load(self : Interp, q : String) -> Result[@moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value], @marianoguerra/pure-py/value.Outcome
Outcome] noraise

Module and program evaluation: Figures 4.11 to 4.13.

Loading mirrors the static rules: the import prefix first, then the body, under builtins. An import loads the ancestors of its target from the root down, each before its descendant.

The one departure is the cache, and it is about OUTPUT rather than values. The spec says loading is deterministic and needs no cache, which is true of what a module loads TO; it is not true of what loading PRINTS. Two imports of a module that prints would print twice, and CPython prints once. The suite has a test for it.

Interp::match_pattern

source

async fn Interp::match_pattern(self : Interp, env : @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value], p : @marianoguerra/pure-py/ast.Pattern
Pattern, v : @marianoguerra/pure-py/value.Value
Value) -> @marianoguerra/pure-py/value.MatchResult
MatchResult noraise

Pattern matching: Figure 4.2.

The rules are narrower than "anything else fails", and the narrowness is deliberate. eval-pat-list-no fires when the value is NEITHER a list nor a tuple, or is a list of the wrong length -- so a list pattern against a TUPLE is covered by no rule at all and the run is undefined, while a tuple pattern against an integer is an honest no-match. A literal pattern against a list reaches eq between unrelated kinds and is undefined too. The conformance suite has a test for each of the three.

Interp::new

source

fn Interp::new(tree : @marianoguerra/pure-py/program.SourceTree
SourceTree, host? : Host, max_depth? : Int, profile? : @marianoguerra/pure-py/profile.Profile
Profile, max_steps? : Int) -> Interp

Interp::predefined

source

fn Interp::predefined(self : Interp, q : String) -> @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value]?

The environment a predefined module loads to, or None if q is not one.

typing exposes Any alone. Figure 2.7 also lists Callable; the reference checker does not, and the signature and the environment have to agree or a from-import would type-check and then fail to run.

Interp::resolve_class

source

async fn Interp::resolve_class(self : Interp, e : @marianoguerra/pure-py/ast.Expr
Expr, env : @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value]) -> @marianoguerra/pure-py/context.ClassEntry
ClassEntry? noraise

resolve-class(ρ, q): the class a qualified name stands for.

Interp::run

source

async fn Interp::run(self : Interp) -> RunResult noraise

⇒ κ (Figure 4.13): evaluate a program by loading its main module.

RunResult

source

pub(all) enum RunResult {
  Finished
  Terminated(@marianoguerra/pure-py/value.Termination
Termination, Site?)
  Undefined(String)
  Suspended
} derive(@moonbitlang/core/debug.Debug
Debug)

How a run ended, or that it has not ended yet.

Sink

source

pub struct Sink {
  buf : StringBuilder
}

Somewhere to put output when the caller just wants the transcript.

run_program uses one of these; run_with does not, because a host that supplies its own write has somewhere better to put it.

Sink::new

source

fn Sink::new() -> Sink

Sink::text

source

fn Sink::text(self : Sink) -> String

Sink::write

source

fn Sink::write(self : Sink, text : String) -> Unit

Site

source

pub(all) struct Site {
  in_module : String
  span : @marianoguerra/pure-py/basic.Span
Span
} derive(Eq, @moonbitlang/core/debug.Debug
Debug)

Where an abort happened: the module, and the span within it.

The specification says an implementation "must agree on which kind a run yields, though how it reports one is not prescribed" (operational-semantics.tex). This is reporting, then, and not semantics: a Site never changes which Termination a run yields, and two runs that abort at different places with the same kind still agree.

Site::to_display

source

fn Site::to_display(self : Site) -> String

module:line:col, the site as a person reads it.

default_max_depth

source

let default_max_depth : Int

How deep a PurePy call stack may go by default.

PurePy has no loops, so recursion is the only iteration a guest has, and this is the bound on it. Past it a run ends undefined -- a call stackdeeper than 10000 -- which is an answer a person can read, a page can render and a program can be rewritten around.

One number, on every backend. It used to be two, 500 on a native thread and 12 on the three that run on a JavaScript engine, because the evaluator recursed and the real limit was the HOST's stack: a JavaScript engine holds two orders of magnitude fewer frames than a native thread, and overflowing one throws a RangeError no MoonBit code can catch, taking a browser tab with it. That number could not be chosen, only measured -- and re-measured whenever the evaluator changed shape, since a frame is sized by the whole function.

The evaluator keeps its continuation on the heap now (machine.mbt), so the host's stack does not grow with the guest's recursion and there is nothing left for the backend to decide. Ten thousand is a policy: deep enough that no reasonable program meets it, shallow enough that a runaway stops in an array of ten thousand frames rather than in an out-of-memory. A caller who wants a different one passes max_depth.

default_max_steps

source

let default_max_steps : Int

How many steps a run may take by default.

A step is one move of the machine in machine.mbt: evaluate a sub-expression, hand a value to a frame, enter a call, bind a statement. max_depth bounds how DEEP a guest may go and this bounds how LONG it may take, which are different runaways and only one of them used to have an answer.

PurePy has no loops, so for a long time the only way to run forever was to recurse, and max_depth caught that. It is not the only way: a comprehension over a long sequence loops without recursing, and used to run to completion however long that took -- which on a browser tab is a page that stops answering, with no message and nothing to interrupt it.

A hundred million is a policy, like max_depth's ten thousand: far more than any program a person is waiting on, few enough that a runaway ends in an answer rather than in a hang. A host that knows better passes max_steps -- a notebook cell wants a much smaller one, and a batch job on a native thread may want none of it.

It is DETERMINISTIC, which a wall-clock timeout would not be. Two runs of the same program over the same host answers take the same number of steps and stop in the same place, so a guest that hits the limit hits it reproducibly and a test can pin it.

region_bindings

source

fn region_bindings(env : @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value], region : Array[@marianoguerra/pure-py/ast.Stmt
Stmt], in_module : String) -> @moonbitlang/core/immut/hashmap.HashMap
HashMap[String, @marianoguerra/pure-py/value.Value
Value]

{f⃗: v⃗}: every definition of a region, as closures over the same environment.

in_module is the module the region was written in, which the closures carry so that an abort inside one is reported against the right file.

run_program

source

fn run_program(tree : @marianoguerra/pure-py/program.SourceTree
SourceTree, argv? : Array[String], max_depth? : Int, profile? : @marianoguerra/pure-py/profile.Profile
Profile, max_steps? : Int) -> (String, RunResult)

Evaluate a program and answer with its transcript and how it ended.

The simple path: output is collected and handed back. A caller that wants output as it happens, its own modules, or its own functions supplies a Host and uses run_with.

The host this builds answers no foreign call, so this run cannot suspend and the result is never Suspended. The transcript is complete.

run_with

source

fn run_with(tree : @marianoguerra/pure-py/program.SourceTree
SourceTree, host : Host, max_depth? : Int, done? : (RunResult) -> Unit, profile? : @marianoguerra/pure-py/profile.Profile
Profile, max_steps? : Int) -> RunResult

Evaluate a program against a host, from a synchronous caller.

Output goes wherever the host sends it, so nothing is returned but how the run ended.

A host call may suspend -- park the run and answer later -- and a synchronous caller cannot wait for it. So this returns when the run ends OR when it suspends, whichever comes first, and there are two ways to learn the answer:

The return value is the answer when the run ended here, and Suspended when it did not.
done is called exactly once with the answer whenever the run really ends: before this returns when nothing suspended, and from inside the host's own continuation when something did.

A host that always answers immediately can ignore done and read the return value, which is what every caller before suspension existed did. A host that suspends should pass done, because the value returned here is not the answer.

An embedder that is itself asynchronous does not need any of this: Interp::run is an async function and can simply be awaited.

Source Files

Site source Report issue Packages Build queue Skills Statistics

#Dispatch

#Host

#Host::member_names

#Host::module_names

#Host::new

#HostModule

#Interp

#Interp::apply

#Interp::dispatch

#Interp::eval_body

#Interp::eval_expr

#Interp::eval_seq

#Interp::load

#Interp::match_pattern

#Interp::new

#Interp::predefined

#Interp::resolve_class

#Interp::run

#RunResult

#Sink

#Sink::new

#Sink::text

#Sink::write

#Site

#Site::to_display

#default_max_depth

#default_max_steps

#region_bindings

#run_program

#run_with

Source Files

Dispatch

Host

Host::member_names

Host::module_names

Host::new

HostModule

Interp

Interp::apply

Interp::dispatch

Interp::eval_body

Interp::eval_expr

Interp::eval_seq

Interp::load

Interp::match_pattern

Interp::new

Interp::predefined

Interp::resolve_class

Interp::run

RunResult

Sink

Sink::new

Sink::text

Sink::write

Site

Site::to_display

default_max_depth

default_max_steps

region_bindings

run_program

run_with