Role Overview

We are hiring a hands-on Senior Software & AI Test Engineer to design and operationalize a scalable, automation-first quality framework across our software and AI-driven systems. This role owns test strategy, infrastructure, and execution, ensuring high-confidence releases across APIs, cloud services, data pipelines, and AI/ML components.
The mandate is twofold: (1) build robust, modern testing systems and (2) embed a pragmatic culture of quality that keeps pace with rapid product development.

Key Responsibilities

1) Test Architecture & Infrastructure
· Design and implement a unified test framework across:
· Backend services and APIs
· Cloud platforms and distributed systems
· Data pipelines and data quality layers
· AI application and evaluation systems
· Define test environments, mocking/simulation strategies, and synthetic data generation.
· Build and maintain CI/CD pipelines.
· Integrate testing deeply into CI/CD pipelines with clear gating signals.

2) Automated Testing & Tooling
· Build and maintain automated test suites:
· Unit, integration, system, regression, and performance testing
· Develop test orchestration, reporting dashboards, and failure triage workflows.
· Ensure tests are deterministic, reproducible, and fast enough for developer iteration.

3) Data & Pipeline Validation
· Establish validation strategies for data pipelines:
· Schema validation, anomaly detection, and data integrity checks
· Build automated tests for ETL workflows and downstream system dependencies.
· Ensure reproducibility between offline experimentation and production behavior.

4) Debugging & Root Cause Analysis
· Lead investigation of complex failures across services, data, and AI layers.
· Establish structured approaches to failure classification and regression prevention.

5) AI/ML Testing & Evaluation
· Build continuous evaluation pipelines tied to model releases.
· Define acceptance criteria and release gates for AI features (beyond traditional QA).
· Develop benchmarking tools for comparing models across datasets and scenarios.

6) Quality Culture & Process
· Introduce scalable quality practices:
· Shift-left testing and testability in design
· Definition of done includes validation and observability
· Partner with engineering and product to define:
· Measurable quality metrics (e.g., defect escape rate, test signal quality)
· Release criteria aligned with risk
· Balance thoroughness with speed—avoid over-engineering test systems.

Required Qualifications

· 6–10+ years in software testing, SDET, or quality engineering.
· Strong programming skills (Python required; experience with backend systems preferred).
· Proven track record building test frameworks and automation from scratch.
· Deep understanding of:
· API testing, distributed systems, and cloud architectures
· CI/CD systems (e.g., GitHub Actions, Jenkins)
· Test methodologies (boundary testing, fuzzing, fault injection)
· Hands-on experience validating AI/ML systems, including:
· Model evaluation metrics and tradeoffs
· Dataset validation and management
· Experiment tracking tools (e.g., MLflow, Weights & Biases, or equivalent)
· Experience with LLM or computer vision evaluation is a plus.

Preferred Qualifications

· Experience testing data-intensive systems or analytics platforms.
· Familiarity with data engineering tools and workflows.
· Experience with performance, scalability, and reliability testing.
· Exposure to observability tooling (logs, metrics, tracing) for test validation.
· Experience working in fast-paced product environments with evolving requirements.
· Design and implement evaluation frameworks for AI/ML systems:
· Model performance (accuracy, precision/recall, etc.)
· Robustness, edge cases, and failure modes
· Data quality, drift detection, and dataset versioning

What We’re Looking For

· Builder mindset: Creates frameworks and tools, not just test cases.
· Systems thinker: Understands interactions across APIs, data, and AI layers.
· Pragmatic operator: Applies the right level of rigor for the stage of product maturity.
· Quality driver: Elevates engineering standards without becoming a bottleneck.
· Hands-on depth: Writes code, debugs systems, and owns outcomes.

Success Metrics (First 6–12 Months)

· Production-grade automated test framework integrated into CI/CD.
· Clear, adopted AI evaluation framework used across AI applications releases.
· Reduction in escaped defects and regression incidents.
· Measurable improvement in data pipeline reliability and validation coverage.
· Increased developer confidence in releases without slowing iteration speed.

Why Join Covalent

At Covalent, you’ll work alongside world-class scientists and engineers in a dynamic, collaborative environment. We empower our team members to take ownership of their work, innovate constantly, and engage directly with customers, shaping the future of technology.