safe_wren

A nearly-complete implementation of the Wren language (wren.io) in Rust.

The original https://github.com/wren-lang/wren from wren.io is refered to as wren_c to disambiguate from this safe_wren.

Similaries to wren_c

• Passes ~90% of wren_c tests
• Exposes ~90% of wren_c C API

Differences from wren_c

• Never crashes to bad input (but can currently still be timed-out via infinite loops)
• Reference-counted, no garbage collected (in progress).
• Stops at first compile error (instead of continuing)
• Requires utf8 input and strings are always utf8 (does not allow invalid utf8 bytes)
• Does not allow overriding allocator (yet)
• Missing opt-meta, and class attributes
• About 2x slower than wren_c on some microbenchmarks (should be compariable after GC work completes)

Usage

From an existing C project:

cargo build --release produces target/release/libsafe_wren.a and target/release/libsafe_wren.so which are drop-in replacements for wren.a and wren.so and compatible with wren.h found at (wren_c/src/include/wren.h).

From Rust:

Add a dependency to your cargo.toml, e.g.

[dependencies] safe_wren = "0.1.0"

Development

There are two binaries: - wren_test -- used for testing uses only public API - wren_debug -- used for debugging vm, uses private calls.

cargo run FILENAME_OR_STRING will run wren_test against a file or string.

cargo run --bin=wren_debug FILENAME_OR_STRING will run wren_debug

wren_debug sub-commands: tokenize Dumps token stream. compile Dumps compiler bytecode. interpret Similar to no arguments, excepts prints VM state after run.

python3 util/test.py will run the tests, including updating test_results/* with error text from any failed tests. test.py will also update test_results/passes.txt with the list of passing tests.

test_results/test_expectations.txt lists all currently skipped tests and why.

Work yet to do

Launch list?

• Example using rust API
• Example using C API
• Publish to Cargo
• Announce to wren-lang
• Generate and publish Rust docs.

Ordered goals?

• Fix delta blue (closure error!)
• fix localoutsidemethod.wren
• Time the tests / make faster (next is vec::alloc from method calls)
• Fancier test_expectations system ** Config / Expectation pairs (c | FAIL, RUST | TIMEOUT)
• Teach utils/test.py how to easily switch between rust and c_rust and c
• remove all uses of 'as' (use into() instead).
• validate_superclass could now use ClassSource to validate internal, etc.
• String codepoint APIs (including String.iterate)
• wrong line numbers for foreign method runtime errors.
• attributes
• rust implementation of meta package.
• continue after failure during compiling?
• \x should not round-trip through char.
• Garbage Collection?
• Sort methods to match wren_c order?
• Variable should use a different type for each scope type.
• fuzz both wrenc and safewren and compare output?
• Look at some of the slow-unit fuzz results * fuzz/artifacts/fuzz_target_1/slow-unit-63ea01d2d5ba869bdb889c3b51b21350d5a4ffea (lookup_symbol should be a hash) * fuzz/artifacts/fuzztarget1/slow-unit-355b25c3fc10bfd14a363cf737abf3a07bab4a1e (needless stack resizing)
• wrendebug interpret wrenc/test/language/staticfield/nestedclass.wren does out of bound lookup in lineforpc.

Leads to pursue

• Making InputManager an Iterator, could make easier the "skip until" pattern?
• https://docs.rs/oncecell/1.8.0/oncecell/
• https://docs.rs/anyhow/1.0.41/anyhow/
• Try optimizing for size: https://github.com/johnthagen/min-sized-rust
• To try for really-small size, no-std + https://doc.rust-lang.org/alloc/
• Box has the same layout as a C pointer, possibly simplifying our FFI? https://stackoverflow.com/questions/62338832/how-to-hold-rust-objects-in-rust-code-created-through-c https://doc.rust-lang.org/nomicon/ffi.html#representing-opaque-structs seems to imply so?

Benchmarking notes

• Current numbers show safewren to be about 2.5x-9x slower than wrenc across the various microbenchmarks. Unclear what real-world effect this would have.
• Value::clone is apparent in many benchmarks.
• Using move/drain semantics when calling args from the stack could help avoid needing to clone values when converting them with tryintoX. Or at least make tryintoX use move semenatics and the clone explicit in the caller.
• classforvalue under call_method shows up at ~5% on several benchmarks.
• map_numeric heavily tests Value::PartialEq
• binarytrees leans heavily (at least 20% of time) on ptr::dropin_place (deallocation) of RefCell. GC would reduce this.
• mapstring spends 57% of time in core::stringplus, and 20% of time truncating the stack.

wren_c bugs

• closures/functions defined in wren_core.wren end up with a null class pointer?
• If you yield from the root, it gets set to state=OTHER, presumably later you might be able to call things on it?
• WrenConfiguration likely leaked for each WrenVM constructed/destructed?