Role Overview

We are hiring a hands-on Senior Software Engineer in Test to design and operationalize a scalable, automation-first quality framework across our software and AI-driven systems. This role owns test strategy, infrastructure, and execution, ensuring high-confidence releases across APIs, cloud services, data pipelines, and AI/ML components.

The mandate is twofold: (1) build robust, modern testing systems and (2) embed a pragmatic culture of quality that keeps pace with rapid product development.

Key Responsibilities

1) Test Architecture & Infrastructure
· Design and implement a unified test framework across:
· Backend services and APIs
· Cloud platforms and distributed systems
· Data pipelines and data quality layers
· AI application and evaluation systems
· Define test environments, mocking/simulation strategies, and synthetic data generation.
· Build and maintain CI/CD pipelines.
· Integrate testing deeply into CI/CD pipelines with clear gating signals.

2) Automated Testing & Tooling
· Build and maintain automated test suites:
· Unit, integration, system, regression, and performance testing
· Develop test orchestration, reporting dashboards, and failure triage workflows.
· Ensure tests are deterministic, reproducible, and fast enough for developer iteration.

3) Data & Pipeline Validation
· Establish validation strategies for data pipelines:
· Schema validation, anomaly detection, and data integrity checks
· Build automated tests for ETL workflows and downstream system dependencies.
· Ensure reproducibility between offline experimentation and production behavior.

4) Debugging & Root Cause Analysis
· Lead investigation of complex failures across services, data, and AI layers.
· Establish structured approaches to failure classification and regression prevention.

5) AI/ML Testing & Evaluation
· Build continuous evaluation pipelines tied to model releases.
· Define acceptance criteria and release gates for AI features (beyond traditional QA).
· Develop benchmarking tools for comparing models across datasets and scenarios.

6) Quality Culture & Process
· Introduce scalable quality practices:
· Shift-left testing and testability in design
· Definition of done includes validation and observability
· Partner with engineering and product to define:
· Measurable quality metrics (e.g., defect escape rate, test signal quality)
· Release criteria aligned with risk
· Balance thoroughness with speed—avoid over-engineering test systems.

Required Qualifications

• 6–10+ years in software engineering, software testing or SDET.

• Strong programming skills (Python required; experience with backend systems preferred).

• Proven track record building test frameworks and automation from scratch.

• Deep understanding of:

• API testing, distributed systems, and cloud architectures

• CI/CD systems (e.g., GitHub Actions, Jenkins)

• Test methodologies (boundary testing, fuzzing, fault injection)

• Experience validating AI/ML systems is a plus, including:

• Model evaluation metrics and tradeoffs

• Dataset validation and management

• Experiment tracking tools (e.g., MLflow, Weights & Biases, or equivalent)

• Experience with LLM or computer vision evaluation is a plus.

Preferred Qualifications

• Experience testing data-intensive systems or analytics platforms.

• Familiarity with data engineering tools and workflows.

• Experience with performance, scalability, and reliability testing.

• Exposure to observability tooling (logs, metrics, tracing) for test validation.

• Experience working in fast-paced product environments with evolving requirements.

• Design and implement evaluation frameworks for AI/ML systems:

• Model performance (accuracy, precision/recall, etc.)

• Robustness, edge cases, and failure modes

• Data quality, drift detection, and dataset versioning

What We’re Looking For

· Builder mindset: Creates frameworks and tools, not just test cases.
· Systems thinker: Understands interactions across APIs, data, and AI layers.
· Pragmatic operator: Applies the right level of rigor for the stage of product maturity.
· Quality driver: Elevates engineering standards without becoming a bottleneck.
· Hands-on depth: Writes code, debugs systems, and owns outcomes.

Success Metrics (First 6–12 Months)

· Production-grade automated test framework integrated into CI/CD.
· Clear, adopted AI evaluation framework used across AI applications releases.
· Reduction in escaped defects and regression incidents.
· Measurable improvement in data pipeline reliability and validation coverage.
· Increased developer confidence in releases without slowing iteration speed.

Why Join Covalent

At Covalent, you’ll work alongside world-class scientists and engineers in a dynamic, collaborative environment. We empower our team members to take ownership of their work, innovate constantly, and engage directly with customers, shaping the future of technology.