benchmark methodology
Each comparison records workload inputs, hardware, cache state, timing boundaries and outcomes. Results must include failures and regressions.
available results
The first two records are exploratory measurements from 5 October 2026, with one sample per platform. Both ran public Rust code successfully through the customer execution path. They are insufficient to establish current-release performance, typical speedup or cost savings.
Downloads contain timing summaries and the original record digest. Internal operational identifiers and obsolete cost-model values are omitted. Complete raw evidence is not yet publicly available.
workload inputs
A publishable comparison identifies the repository, exact commit, lockfile or resolved dependency set, compiler, target, profile, features, Cargo arguments and outcome. If one differs, state it prominently and narrow the conclusion.
hardware and cache state
Record vCPU, memory, architecture, CPU model, region and runner class for both sides. Equal vCPU counts do not equal identical cores. Report cold, dependency-warm and build-warm measurements separately. State exactly which caches are enabled and what was restored.
timing boundaries
The newer harness defines Sorus time-to-result from client invocation and GitHub-hosted time-to-result from job creation. It identifies excluded harness overhead. These boundaries still need to be declared next to the result; they must not be retroactively assigned to the older command-to-outcome observations.
Show source upload, queueing, guest startup, execution and delivery when reliable component timings exist. Missing measurements remain missing, not zero.
samples and failures
A release series must have a declared sample count and repeatable run procedure. Publish the distribution and justified summaries only when enough samples exist. Keep timeouts, refusals, platform errors and regressions in the evidence; do not cherry-pick the best run or call one observation a median.
cost comparisons
Customer-price comparisons require actual published or agreed prices, billing rules, dates and a representative usage scenario. An internal host-cost model is not a Sorus price. No cost comparison is currently published in this library.
publication checklist
- A named, qualified release and the exact invocation on both sides.
- Fixed source and declared dependency resolution, or a visible mismatch.
- Hardware, environment and cache declarations.
- Timing boundaries and excluded intervals.
- Sample count, all outcomes and justified statistical summaries.
- Downloadable scrubbed evidence with integrity digests and reproduction instructions.
- A freshness review; stale results remain historical.
- No implication that a measured project's maintainers are customers or endorse Sorus.
propose a workload
Submit a repository for evaluation. Public case studies, logo use and customer attribution are agreed separately.