This reference describes the implemented native runtime. Historical paths and internal version labels are retained where relevant. Later extension chapters supersede older limitations.
This extends the optimized utility interpreter in restored/. No originals or preserved baselines were changed. LANGUAGE.md and UTILITIES.md still describe the older commands; this document supersedes their former scalar-only function limitation.
Build and trust boundary
Linux, C++17, GCC 13, libcurl, nlohmann JSON, and SQLite's linked C API are used. Install libsqlite3-dev (and existing libcurl4-openssl-dev, nlohmann-json3-dev, pkg-config, compiler dependencies), then make release sanitize. SQLite 3.45.1 was exercised; use SQLite 3.41 or newer for progress callbacks during preparation as well as execution. No SQLite CLI, shell invocation, downloaded extension, service, or remote app is involved.
--allow-db is off by default and independent of --allow-files and --allow-network. These are trusted-script feature gates, not a sandbox. DB paths resolve against the process working directory; opening permits creation/modification of the named database and its SQLite journal/WAL sidecars, follows OS permissions/symlinks, and is not directory confinement. Do not run untrusted scripts or point these examples at production databases. Tests use disposable local directories and the existing suites' loopback-only HTTP fixtures.
New reserved command/function/parameter names: foreach, returnvalue, assert, date, csv, db. file create is a new subcommand. The CLI accepts test in place of run, and --allow-db. Existing math evaluation/caching/formatting, % scalar semantics, scalar concatenation, and legacy error ordering remain tested against preserved executables. Structured args that formerly errored are now accepted by calls, not by scalar concatenation.
Snapshot iteration and structured functions
foreach $value $source
...
end foreach
foreach $key $value $source
...
end foreach
call $destination function $structured_argument ...
returnvalue value
assert left operator right
Declare every destination first. source is a native string array or a typed JSON array/object. Native values remain ordinary scalar strings. JSON elements remain typed JSON, including null, booleans, numbers and strings: use json text to obtain a scalar. Keys are scalar strings: zero-based decimal indices for arrays, lexicographically sorted object keys for maps. Empty sources leave destinations unchanged. Key and value destinations must differ; either may alias the source.
Iteration takes an entry-time snapshot. Mutating/replacing the source, mutating a map-valued element, or changing the loop variables cannot alter that snapshot. continue advances, break exits the nearest loop; lexical try/catch unwinds discarded snapshots without leaving stale iterators. Return and exit release all invocation-local snapshots.
Named parameters receive the original type. Immutable JSON trees may share backing storage, but every mutation makes a bounded copy; native arrays copy their strings. Returns and arguments are independent values, not references. returnvalue $x preserves any scalar/native-array/JSON value, including through recursion or a destination/source alias. returnvalue %1 and forwarding %1 to a function also preserve the original argument type. Ordinary print %1, math %1, and return %1 ... still require scalar arguments. Named parameter mutation never changes the original positional argument. Existing return concatenates scalar tokens with no separator; falling off a function still returns an empty scalar. var redeclaration remains a no-op.
Snapshots, positional arguments and named/local values are conservatively charged against the existing 2 MiB frame bound, including shared JSON logical storage. Argument collection has a separate 2 MiB bound; native arrays retain their 16,384-item bound. JSON retains depth 64, 16,384 nodes, 1 MiB accounted storage/encoded text and safe integers through ±9,007,199,254,740,991. Call depth 64, block depth 128, source/token/line bounds and the normal instruction budget remain. Large retained snapshots can therefore reject otherwise individually valid JSON values. Structured copies are bounded, not free memory aliases.
assert uses the same six operators and numeric-or-lexical comparison rules as if. Failure produces a source-located assertion failed error, catchable with try. Resource failures remain uncatchable.
Actual CLI tests
./build/vanilla1 test "examples/worklog_test.v"
./build/vanilla1 run "examples/worklog_test.v"
test FILE discovers all test_* functions in the loaded program/includes, sorted by name. Each must have zero arguments. A main is optional and is not run in test mode. Each test gets a fresh interpreter, source load, expression caches, locals, environment overlay, random seed, clocks, handle tables and all budgets. One failed test does not prevent later tests, including after a fatal budget failure. exit is a failed test, not a false pass. Output contains PASS name/FAIL name and a count summary; exit status is 1 for any failure or no discovered tests, 0 only for a nonempty all-pass run. Source-located failures go to stderr. Output is flushed and checked.
This is runtime state isolation, not OS/process isolation: stdin position, inherited environment, filesystem/database writes and network side effects are external state. Use unique temporary paths for tests that write. Unclosed database handles roll back active transactions on test destruction. Explicitly committed changes persist.
Dates: explicit UTC/fixed offsets only
date parse $epoch "2024-02-29" UTC
date parse $epoch "2024-02-29T12:34:56" "+05:30"
date format $local_text $epoch "-05:00"
date diff $seconds $end_epoch $start_epoch
Parse accepts exactly YYYY-MM-DD (midnight) or YYYY-MM-DDTHH:MM:SS; no embedded zone suffix, whitespace, fractional seconds, loose padding or leap seconds. Years are 0001–9999, with Gregorian leap-year/month/day validation. Zones are UTC, Z, or a signed HH:MM offset through ±14:00 (minutes 00–59, ±14 requires :00). Named IANA zones, DST transitions and leap seconds are not implemented, and are rejected rather than guessed.
Epochs are integral Unix seconds. Format produces a zero-padded local YYYY-MM-DDTHH:MM:SS without a zone suffix; the required zone operand is explicit and independent of host TZ. Format rejects a resulting local year outside 0001–9999. Epoch operands are bounded to -62135647200 through 253402351200 seconds, encompassing the supported civil-year range with offset margins; difference returns signed end - start seconds, not calendar months or DST-adjusted days. Every accepted timestamp/difference in this range is exact in the interpreter's binary64 integer range. Parse/format do not consult the wall clock.
CSV
csv parse $rows $text
csv stringify $text $rows
Rows are a typed JSON array of nonempty arrays of JSON strings. Empty input yields []. Parsing accepts commas, doubled quotes, quoted embedded CR/LF, LF or CRLF record endings, and empty/trailing fields. A final record separator does not add a phantom row. A blank record is one empty field. Bare CR outside a quoted field, quotes in an unquoted field, junk after closing quotes and unfinished quotes are errors. Ragged rows are allowed; no header inference or numeric conversion occurs.
Stringify quotes every cell, doubles embedded quotes, and emits CRLF after each row. It rejects nonstring cells, nonarray rows and empty rows (unrepresentable without ambiguity). Parse/stringify support destination/source aliases. Input/output limits are 1 MiB, with conservative incremental storage/node checks and the existing JSON/frame bounds. CSV parse has no delimiter/dialect autodetection.
Formula injection is not sanitized: strings beginning =, +, -, or @ are faithfully preserved. Quoting CSV is not spreadsheet formula protection. Treat exports as data or separately escape them before opening in a spreadsheet.
file create "path" text requires --allow-files and creates a new file with Linux O_EXCL (mode 0600, subject to umask). Existing regular files, directories and even dangling symlinks are rejected atomically. This makes the example's export no-clobber without relying only on an existence precheck. It uses existing 1 MiB value / 16 MiB cumulative write limits. Creation is not transactional replacement or durability/fsync: an I/O failure may leave a partial newly created file; parent directories are not sandboxed.
SQLite
json parse $params "[]"
db open $handle "local work log.db"
db exec $changed $handle "CREATE TABLE jobs(id INTEGER PRIMARY KEY, note TEXT, cents INTEGER)" $params
json parse $params "[\"O'Brien\",12500]"
db exec $changed $handle "INSERT INTO jobs(note,cents) VALUES(?,?)" $params
json parse $params "[]"
db query $rows $handle "SELECT id,note,cents FROM jobs ORDER BY id" $params
db begin $handle
db commit $handle
db rollback $handle
db close $handle
All operations require --allow-db; filenames are value operands, not SQL fragments. Quote filenames in scripts and at the CLI. :memory: is supported; file: URI paths and NULs are rejected. Open is read/write/create; close invalidates the numeric handle and rolls back uncommitted work. At most eight live DB handles exist per interpreter. Handles are runtime-scoped and must not be confused with file handles.
exec and query prepare exactly one statement. Trailing comments/whitespace are allowed; a second statement or malformed tail is rejected before stepping the first. Parameters must be a typed JSON array with exactly SQLite's parameter-slot count. Use ordinary ? placeholders in left-to-right order; named/indexed SQLite placeholders also bind by SQLite slot number, not JSON key. Strings (including embedded NUL) use explicit lengths and SQLITE_TRANSIENT; null maps to SQL NULL, booleans to integer 0/1, integer JSON to int64, fractional JSON to double. Containers and BLOBs are unsupported.
query accepts only read-only statements with columns, returning a JSON array of objects. Column aliases must be unique, and query order is whatever explicit SQL ORDER BY guarantees. SQL integer/float/text/NULL retain their corresponding JSON type; booleans read back as integers. BLOBs, nonfinite values and integers outside JSON's exact-safe range fail. exec rejects all result-bearing statements, including RETURNING; it returns SQLite's changes64 count (the last DML change count, not a DDL count or a total including trigger changes).
Every exec is enclosed in an internal savepoint. A failed statement, including INSERT OR FAIL, cannot leave partial statement writes. Any error in an operation on a valid DB handle aborts that handle's active transaction, including earlier writes since db begin, prepare/bind/type/authorization/result-bound errors and interrupted queries. Cleanup disables the progress handler before rollback. db begin/commit/rollback expose simple nonnested transactions; SQL transaction/savepoint commands are denied so scripts cannot interfere with internal savepoints. Argument/destination/invalid-handle errors that occur before selecting a connection cannot roll back an unknown connection. Errors elsewhere in the script do not automatically roll back an otherwise still-live handle if the script catches them; use db rollback or let the runtime destroy the handle.
Security controls: SQLite authorizer denies ATTACH, DETACH, every PRAGMA, SQL transaction/savepoint controls, CREATE/DROP VIRTUAL TABLE, and load_extension/readfile/writefile functions. This also blocks VACUUM INTO's attach path. Extension loading is disabled, defensive mode is enabled, and trusted schema is disabled. The authorizer remains installed during stepping/reprepare. No host callbacks or custom SQL extensions are registered. SQL data parameters do not become SQL code. There is deliberately no shell escape.
Bounds:
- SQL source 64 KiB; SQL values/rows 1 MiB; columns 128; expression depth 64; compound selects 16; parameters 256; compiled VM ops 25,000; attached DBs 0; trigger depth 16; LIKE/GLOB pattern 4096 bytes.
- Query result at most 1024 rows, 16,384 nodes, 1 MiB conservatively accounted structured bytes; cumulative results 16 MiB per interpreter. General JSON/frame limits may reject earlier.
- SQLite allocator hard limit 64 MiB per process; private connection caches request 2 MiB each; temporary storage is memory. These do not cap total process RSS, existing database size, cumulative persistent DB growth, OS caches or journal bytes. Script instruction limits still bound the number of statements.
- SQLite progress callback every approximately 1000 VM instructions; at most 1000 callbacks per operation (approximately one million VM instructions), further restricted by the remaining script instruction budget. Callback VM work is charged to the script budget on success and error. Operations below callback granularity still cost their script instruction. A steady-clock 2-second cooperative deadline applies to prepare/step, with 100 ms busy timeout for locks. Interruptions are fatal resource errors, not catchable retries. SQLite/OS operations that do not call the progress callback are not a hard real-time guarantee; use an external process timeout for adversarial inputs.
Primary API references consulted during implementation: SQLite official c3ref/set_authorizer.html, c3ref/progress_handler.html, c3ref/prepare.html and security.html on www.sqlite.org (fetched directly; the generic extraction/browser backend failed). Linked system SQLite is not rebuilt with sanitizers; the interpreter and its wrapper are.
Persisted example
Run in a fresh disposable working directory; replace the interpreter/example paths with absolute paths if needed:
/root/vanilla1/build/vanilla1 --allow-db run "/root/vanilla1/examples/worklog.v" -- initialize "demo work log.db"
/root/vanilla1/build/vanilla1 --allow-db run "/root/vanilla1/examples/worklog.v" -- add "demo work log.db" "2024-02-29" "Install" 12 12500
/root/vanilla1/build/vanilla1 --allow-db run "/root/vanilla1/examples/worklog.v" -- add "demo work log.db" "2024-02-29" "Repair" 3 3499
/root/vanilla1/build/vanilla1 --allow-db run "/root/vanilla1/examples/worklog.v" -- show "demo work log.db"
/root/vanilla1/build/vanilla1 --allow-db --allow-files run "/root/vanilla1/examples/worklog.v" -- export "demo work log.db" "demo export.csv"
Each command is an independent process. initialize creates the table and errors if it already exists. add appends a uniquely numbered entry, so repeated dates/jobs never silently overwrite an entry. Dates are strict UTC civil days; job strings are JSON-encoded then SQL-bound, never SQL-concatenated. Mileage is an integer 0–1,000,000; expenses are integer cents, 0–1,000,000,000 per entry (no decimal currency parsing). Assertions reject fractional/nonfinite/bad values. A job with zero expense records mileage/work alone; expense-only entries can have zero miles. Rows are read in ID order, totals use structured helpers and foreach, and CSV round-trips are asserted before export. Supported result limits deliberately make this a small local demonstration, not an unbounded production ledger: over 1024 returned entries errors rather than truncates.
The regression suite adds two same-day entries, including quoted/comma/multiline jobs, reopens via independent processes, verifies SQLite values using Python's SQLite reader, verifies CSV using Python's independent CSV reader, tests no-clobber export and reinitialization, rejects malformed dates/fractional cents, and verifies the database remains unchanged by rejected adds. Exact combined transcript: worklog-demo.txt (and worklog-demo-asan.txt). The concise examples/worklog_test.v demonstrates functions, typed results, foreach, mutation isolation and assertions without filesystem permissions.