Product Assurance Engineer

Are you a curious, detail-oriented person who loves shipping polished products? We are seeking a Product Assurance Engineer to work on a multi-disciplinary team to assure that Oxide ships the highest quality product to our customers.

You will contribute to the practice of reliability-oriented testing: designing experiments that surface latent bugs, availability gaps, and performance cliffs before customers encounter them.

AS A PRODUCT ASSURANCE ENGINEER WORKING AT OXIDE, YOU WILL:

* Design and execute workload-driven stress tests, with a focus on workloads that replicate realistic production traffic patterns such as high connection-rate workloads, micro-burst traffic, large-flow saturation, multicast, and adversarial packet patterns.

* Instrument, observe, and correlate metrics that help characterize and pinpoint system issues.

* Drive chaos-monkey-style tests to simulate common hardware failure modes and control plane outages.

* Investigate availability and reliability gaps, and produce reproducible failure reports.

* Work with engineers across the software stack to triage findings and verify fixes and mitigations.

* Improve the assurance infrastructure itself: scripts, harnesses, chaos tooling, and automated regression suites.

YOU WILL THRIVE IN THIS ROLE IF YOU:

* Have deep familiarity with troubleshooting traffic issues at different network layers and characterising performance and reliability of network data paths.

* Are comfortable with low-level debugging, reading packet captures and reasoning from raw protocol state.

* Enjoy hands-on work with test infrastructure.

* Have a track record of finding non-obvious reliability bugs in complex systems.

* Are proficient in at least one scripting or systems language (Python, Rust, Go) for building test harnesses and automating workload generation.

* Enjoy writing: clear bug reports, knowledge articles, and technical documentation.

BEFORE APPLYING FOR THIS ROLE, YOU SHOULD:

* Learn enough about our product to understand our approach.

* Listen to Hiring Processes with Gergely Orosz to familiarize yourself with the Oxide hiring process.

* Browse our public Requests for Discussion to get a flavor for how we work.

* Listen to some of our episodes of Oxide and Friends to get a sense for how we operate. A few recommendations:

* Rack-scale Networking to understand our approach to networking and some of the details of our implementation

* Hell is other networks to learn about our disposition to customer problems

* The Network Behind the Network on the specific challenges of our service processor network

* The Tale of Reverso on a particularly gnarly networking bug unearthed by a manufacturing test fixture

* Systems Software in the Large on a multi-year effort to develop full-system update

* When Nine Nines Isn’t Enough on a multi-year search for an evasive failure

* RFDs: The Backbone of Oxide on how we got to our writing-intensive culture

Remote positions are open to candidates worldwide. We do ask that your workday overlaps with Pacific Time for at least four hours.