About the Role

As QA Engineer you will own quality for an AI-enabled decision-support system: the test plan, the automated coverage, and the evidence that what was delivered is what was specified. Product defines the acceptance criteria. You build the executable version of it, and you hold engineering, platform, and everyone else delivering the work accountable to it.

This role sits on the product side of the organization, not the engineering side. You will report through the Product Lead and take direction from product, while working with engineering and platform every day. A tester embedded in engineering tends to verify what was built, because the implementation is what they see. This role verifies what was specified. A test suite written from the code will pass while still missing what the customer asked for.

That reporting line does not lower the engineering bar. You will write code and build real automation rather than managing test cases, and you will need enough fluency in the application code to test it well. As the first dedicated quality engineering hire you are establishing the approach rather than inheriting one, so we want someone with a clear point of view on how testing should work. We build with current tooling and expect the same, including AI assistance in our own engineering practice.

This is a fully remote role with occasional travel (up to 25%) to DEFCON AI HQ, customer sites, and vendor facilities as required.

Key Responsibilities

Test Strategy and Acceptance Criteria

Own the system test plan covering functional, integration, performance, and load testing, and maintain it as a formal deliverable
Translate the acceptance criteria set by product into executable tests
Review acceptance criteria with product before development begins, and identify requirements that are ambiguous or not verifiable as written
Hold engineering, platform, and partner teams accountable to the acceptance criteria, and define what evidence must exist before a change is considered complete
Work alongside the engineering team day to day, close enough to test their work well and independent enough to report what the testing shows
Plan and coordinate user acceptance testing with the customer, and track findings through to resolution

Test Automation and Coverage

Build and maintain automated test suites, and integrate them into the CI/CD pipeline as release gates
Establish and hold an automated coverage standard, and report against it
Build performance and load testing that validates the system under its expected user load
Maintain regression coverage across releases

Measurement and Evidence

Build and maintain labeled evaluation sets, working with subject-matter experts whose time is limited
Design and run sampled audits confirming that automated decisions were correct
Measure and report what the system misses alongside what it gets right
Measure how often users override the system, and report the trend
Document how each committed measure is computed, on what data, and what evidence accompanies a release
Produce this evidence independently of the people building the models

Release Governance

Maintain the version inventory across builds, models, configurations, and thresholds
Produce test reports, evaluation records, and rollback criteria for each release
Work with platform engineering to confirm that test environments match production closely enough for results to be meaningful
Hold the gate: no release ships without the evidence its acceptance requires

Required Qualifications

5+ years in software quality engineering, test automation, or software engineering with substantial dedicated test ownership
A clear point of view on how testing should be done, and the judgment to know which practices are worth holding to. You will be setting the standard here rather than following one
Strong coding ability in a language and framework used for test automation, for example Python with pytest, or JavaScript or TypeScript with Playwright, Cypress, or Selenium. Python is what most of this stack is written in. You should also be able to read and reason about the application code under test
Routine use of AI-assisted development, with informed judgment about where it adds value and where its output requires verification. We expect AI tooling to be part of how test coverage gets built here, not an occasional experiment
Demonstrated experience building automated test suites and integrating them into CI/CD as release gates
Experience owning a test plan through review and acceptance
Demonstrated habit of testing against the requirement rather than the implementation, including working directly with product owners or end users to establish what correct actually means
Experience with performance and load testing, including establishing what a result does and does not demonstrate
Working statistical literacy, including sampling design and the confidence a measurement can support
Experience measuring what a system misses, not only what it gets right
Willingness to report a result the team did not want, and to keep the measurement independent of the people whose work it evaluates
Ability to gate work owned by people who do not report to you, and to earn enough credibility with engineers that the gate holds
Clear, audit-ready written documentation, since much of the output of this role is evidence that others read
Willingness to hold a release, and explain the reason, to people who want it shipped
US Citizenship Required
Active US Secret clearance. The work is performed in a controlled government cloud environment and requires a favorable investigation and CAC eligibility from the start
Elevated security requirements apply to portions of this work and are discussed during the interview process
Willingness to travel up to 25% to customer sites, DEFCON AI HQ, and vendor facilities as required

Preferred Qualifications

Clearance: active Top Secret
Environment: government DevSecOps, RMF, or ATO environments, and experience producing test evidence for a formal accreditation or audit
Automation: contract and API testing, test data management, and building test infrastructure others can extend
AI-assisted testing: using language models to generate and maintain test coverage, and the practices that keep generated tests meaningful rather than merely passing
Performance: JMeter, k6, Locust, or comparable
Evaluation: validating machine learning or model output, including groundedness and citation checking, and analyzing where users disagreed with a system recommendation
Domain: systems where an incorrect result carries real cost, and where testing had to satisfy an outside reviewer rather than only an internal one
Governance: model cards, NIST AI RMF, or comparable responsible AI practice
Accessibility: Section 508 and WCAG verification

What Success Looks Like

A test plan the customer accepts, and that holds up as the system grows
Acceptance criteria that are verifiable before development starts
Automated coverage meeting its standard and running as a release gate
Sampled audits designed to catch an incorrect automated decision
Performance evidence that reflects realistic load
A release trail complete enough to reconstruct why any version shipped

What We Offer:

A fully remote, results-based environment
Competitive salary, bonus, and equity package
100% employer paid, comprehensive health insurance including medical, dental, and vision for you and your family
Unlimited PTO, with your manager's approval
Flexible work environment where you manage your work day
14 weeks of fully-paid parental leave

Salary Range: $135,000-$170,000. This represents the typical salary range for this position based on experience, skills, and other factors.

Our Red Cell Partners Benefits:

For full-time roles

Career track opportunity with potential for rapid advancement with strong performance as the firm grows
100% employer paid, comprehensive health care including medical, dental, and vision for you and your family.
Paid maternity and paternity for 14 weeks at employees' normal pay.
Unlimited PTO, with management approval.
Opportunities for professional development and continued learning.
Optional 401K, FSA, and equity incentives available.
Mental health benefits are available through Tara Mind.
Cost effective GLP-1 solutions available through Crux.

Red Cell Partners

New Jobs

QA Engineer - Clearance Required