Quickstart

Suppose the crate's binary is called my-foo and this binary takes a file path as positional argument. This first example shows the basic usage of the high-level API with the #[binary_benchmark] attribute:

extern crate gungraun;
macro_rules! env { ($m:tt) => {{ "/some/path" }} }
use gungraun::prelude::*;

#[binary_benchmark]
#[bench::some_id("foo.txt")]
fn bench_binary(path: &str) -> gungraun::Command {
    gungraun::Command::new(env!("CARGO_BIN_EXE_my-foo"))
        .arg(path)
        .build()
}

binary_benchmark_group!(
    name = my_group,
    benchmarks = bench_binary
);

fn main() {
main!(binary_benchmark_groups = my_group);
}

The gungraun::prelude contains the most important macro definitions and structs. If you want to try out this example with your crate's binary, put the above code into a file in $WORKSPACE_ROOT/benches/binary_benchmark.rs. Next, replace my-foo in env!("CARGO_BIN_EXE_my-foo") with the name of a binary of your crate.

Note the env! macro is a rust builtin macro and CARGO_BIN_EXE_<name> is documented in rust stdlib.

You should always use env!("CARGO_BIN_EXE_<name>") to determine the path to the binary of your crate. Do not use relative paths like target/release/my-foo since this might break your benchmarks in many ways. The environment variable does exactly the right thing and the usage is short and simple.

Lastly, adjust the argument of the Command and add the following to your Cargo.toml:

[[bench]]
name = "binary_benchmark"
harness = false

Running

cargo bench

presents you with something like the following:

binary_benchmark::my_group::bench_binary some_id:("foo.txt") -> target/release/my-foo foo.txt
  Instructions:              342129|N/A             (*********)
  L1 Hits:                   457370|N/A             (*********)
  LL Hits:                      734|N/A             (*********)
  RAM Hits:                    4096|N/A             (*********)
  Total read+write:          462200|N/A             (*********)
  Estimated Cycles:          604400|N/A             (*********)

Gungraun result: Ok. 1 without regressions; 0 regressed; 0 filtered; 1 benchmarks finished in 0.49333s

As opposed to library benchmarks, binary benchmarks have access to a low-level API. Here is roughly the same setup as above, written with the low-level API:

extern crate gungraun;
macro_rules! env { ($m:tt) => {{ "/some/path" }} }
use gungraun::prelude::*;
use gungraun::{BinaryBenchmark, Bench};

binary_benchmark_group!(
    name = my_group,
    benchmarks = |group: &mut BinaryBenchmarkGroup| {
        group.binary_benchmark(BinaryBenchmark::new("bench_binary")
            .bench(Bench::new("some_id")
                .command(gungraun::Command::new(env!("CARGO_BIN_EXE_my-foo"))
                    .arg("foo.txt")
                    .build()
                )
            )
        )
    }
);

fn main() {
main!(binary_benchmark_groups = my_group);
}

If in doubt, use the high-level API. You can still migrate to the low-level API easily if you need to. The other way around is more involved.

Gungraun Guide

Quickstart