Testing

A service is tested two ways: its tools as plain Rust functions, or its built image on the host it runs on.

Unit tests

Off WebAssembly the crate is an ordinary library, so cargo nextest run runs its tools natively, with no toolchain target and no host. crab new leaves tests to the developer. Add a test such as tests/tools.rs for the tools you keep, and update its export names and arguments when changing those tools:

use berm_lang::test;

#[test]
fn echo_answers_its_argument() {
    let out = test::call(my_service::berm_tool_echo, br#"{"query":"hi"}"#).unwrap();
    assert_eq!(out, br#"{"query":"hi"}"#);
}

berm_lang::test stands in for the host:

• test::call(export, args) runs a tool's berm_tool_<name> export, and answers Err with the message of a tool that failed. The host's bound on arguments and results is not applied.
• test::answer arranges what another service answers, test::respond what an HTTP request answers by method and URL, test::var what a var holds, and test::fetched lists the requests sent, {secret:NAME} as written. test::forget drops them.
• test::emitted and test::traced answer what the last call logged and traced.
• A syscall with no stand-in panics, naming it. State, the database and who called have none.

Native tests do not measure WebAssembly CPU or linear memory and do not validate runtime budgets. Before publishing or deploying, follow Sizing a budget.

On the host

berm_host::test::TestHost runs the built image under the real engine, against edges the test holds: fetches answer what the test arranged, secrets are what it granted, the clock stands where it set it, and wakes run when the test fires them. It is behind the test feature of berm-host, from the same repository and tag as berm-lang:

[dev-dependencies]
anyhow = "1"
berm-host = { git = "https://github.com/crabtalk/crab", tag = "…", features = ["test"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

use anyhow::Result;
use berm_host::test::{self, TestHost};

#[tokio::test]
async fn a_call_runs_on_the_host() -> Result<()> {
    let host = TestHost::open(0).await?;
    host.deploy("my-service", &test::build("my-service")?).await?;
    let ran = host.call("my-service", "echo", br#"{"query":"hi"}"#).await?;
    let result: Result<String, String> = ran.result.into();
    assert_eq!(result, Ok(r#"{"query":"hi"}"#.to_owned()));
    Ok(())
}

• test::build(package) builds the image as crab build does.
• TestHost::open(now) starts with nothing deployed and its clock at now, in milliseconds since the epoch; set_now and advance move it.
• arrange(method, url, status, body) answers a fetch, and fetched lists what was sent. grant(service, secret, host, value) gives a service a secret for one host.
• call runs a tool with no caller, as a wake does, whatever its access; call_for runs it for a user. Both answer a Ran: the tool's result, whether it refused its caller, and the meter of the whole chain.
• call_with_client takes a berm::Client { key, ip } to supply the outside client's daily key and IP address.
• wakes lists what the service armed and fire runs them. logs, state and sql read back what calls committed.

Replaying a failed call

crab calls my-service --outcome failed
crab replay my-service 1234
crab replay my-service 1234 --build

A call whose tool failed keeps its input and a tape of every answer it got from outside: each syscall in order, the clock and the random source included, and no secret. crab replay runs it again in the CLI's own process, from the image it ran, each syscall answered from the tape, so a failure in production fails the same way here, printing its logs.

Use --build from the service's crate to check a fix against the same tape. It builds the current code instead of fetching the recorded image; it still refuses syscall requests that differ from the tape.

• A request that differs from the recorded one stops the replay, naming the syscall: the code took another path than it did.
• --on default runs the call's tool, arguments and caller on the local host instead, against its own state, with no tape.
• What a tape keeps, its limits and who reads it: Replaying.