Staff Test Engineer - AI
Outreach is seeking a Staff AI Test Engineer to lead quality initiatives for their agentic AI revenue platform. The role involves designing evaluation frameworks for non-deterministic AI outputs and integrating them into CI/CD pipelines. The candidate will work closely with Data Science and Engineering teams to establish quality standards for LLM-based agent behaviors. This is a senior-level position requiring deep expertise in test automation, ML pipelines, and AI evaluation methodologies.
50k new jobs listed every day. Install TAL to find more jobs like this.

Experience
7-12 years
Function
Quality Assurance
Work mode
Onsite, India
Company
Tier 2
What you will work on
Outreach is seeking a Staff AI Test Engineer to lead quality initiatives for their agentic AI revenue platform. The role involves designing evaluation frameworks for non-deterministic AI outputs and integrating them into CI/CD pipelines. The candidate will work closely with Data Science and Engineering teams to establish quality standards for LLM-based agent behaviors. This is a senior-level position requiring deep expertise in test automation, ML pipelines, and AI evaluation methodologies.
TAL's take
High-impact, senior-level role focusing on cutting-edge GenAI quality strategies at an established company.
Very clear, well-defined responsibilities focusing on GenAI quality, LLM evaluation, and platform testing.
Must haves
- 7-12 years software development or test automation experience
- B.S. in Computer Science or related field
- Strong Python programming skills
- Experience testing complex distributed systems and API layers
- Hands-on experience with Databricks for ML pipelines
- Experience with MLflow for experiment tracking and model versioning
Tools and skills
Nice to have: genai, llm-as-judge, langgraph, langchain, rag, aws, gcp, azure, docker, kubernetes, prompt engineering.
About the company
Outreach is an established mid-stage product company in the sales engagement space, though not typically categorized as top-tier FAANG/Unicorn status in India.
Posts mentioning Outreach
Title: Data Scientist (~2 YOE) – Built planning tools, pipelines, and AI system. Need honest feedback on profile.
Hi all, Looking for practical feedback on my profile before I start applying. I’ll keep this structured so it’s easier to evaluate. --- 1) Planning Tools / Web Applications Problem: Forecasting workflows were fragmented and heavily Excel-driven: - Multiple data sources (orders, shipments, different forecast versions) - Manual merging, lookups, and adjustments - No way to simulate scenarios or compare forecasts cleanly - Different planners using different methods → inconsistency What I built: - Two internal applications for planning workflows: - A planning tool integrating 8+ data sources - A forecasting simulator supporting multi-level editing (high → granular) Key capabilities: - Real-time scenario simulation - Side-by-side comparison of multiple forecast types - Hierarchical adjustments across levels - SQL write-back for persistence Scale: - Processes ~150K+ records per cycle - Used in monthly planning cycles by multiple teams Impact: - Removed fragmented Excel workflows - Enabled consistent decision-making across users - Reduced manual effort and improved visibility into forecast behavior --- 2) Automation & Data Pipelines Problem: Core workflows were manual and repetitive: - Multi-file Excel processing - Data cleaning + merging across systems - Version tracking errors - High effort per cycle (1–4 hours depending on workflow) What I built: - Multiple pipelines automating end-to-end workflows Examples: - Large-scale consolidation pipeline: - Input: ~1M+ rows across 20+ files - Output: clean, unified dataset (~75% reduction) - 2nd pipeline: - Replaced a 23-step manual process - Standardized inconsistent formats across datasets - 3rd automation processing: - Automated unpivoting, enrichment, and version tracking Impact: - Reduced processing time from hours → minutes per cycle - Eliminated manual errors (copy-paste, lookup mistakes) - Standardized workflows across users --- 3) Power BI / Monitoring Problem: Recent data (orders/shipments) showed inconsistencies, but: - No visibility into changes over time - Hard to identify where data drift was happening What I built: - Power BI dashboards with: - Hierarchical filters - Drill-down views - Month-over-month comparison Scale: - ~30K+ records analyzed Impact: - Enabled early detection of data inconsistencies - Helped planners validate inputs before forecasting - Improved trust in upstream data --- 4) Side Project (AI System) What I built: - AI-powered job assistant system Features: - Scrapes job postings - Scores relevance using LLMs - Generates tailored resume points and outreach messages - Tracks applications Tech: - FastAPI backend - LLM routing (cloud + local fallback) - SQLite storage Goal: - Build a system-driven workflow (not just model usage) --- My concern Most of my work sits at the intersection of: - forecasting - data systems - workflow automation I’m trying to move into: 👉 Applied Data Scientist / Product-oriented roles --- Questions 1. Does this profile look too niche (forecasting-heavy)? 2. Does “building systems around data” help or hurt for DS roles? 3. What’s the biggest gap you see (if any)? Would really appreciate honest feedback. Thanks.
🚨 B2B Startups & Agencies 🚨
I have access to a special appolo.io API that lets me access unlimited leads. Comment if you want me to hit you up with unlimited leads at 5$ per 1000 leads Useful for SaaS, Start-ups, Marketing, Sales and Outreach (drop email to receive a sample)
How to enter into VC after 1.5 YoE in Mgmt Consulting
I am a Mechanical Engineer from one of the top IITs. Got a day 1 consulting job (tier 1 firm). It's been 1.5 years in the job and I want to switch to VC. Due promotion by the start of 2025. Recently interviewed with a new US based early stage VC firm (remote job), cleared the first interview and the case study, final interview didn't go amazingly well. They just wanted one Associate and I wasn't the best of the lot, got rejected. My job role actually says 'Business Analyst ' so I get outreaches for Analytics roles mostly. How do I enter into VC in India (early stage smaller fund-size VC firms preferred) ? What are the do's and don'ts? What would be the best time to target such roles? How should I prepare? Who should I reach out to and how? How should I shape my profile? Will it pay me as well as consulting? What are other options that I can consider apart from consulting (I was thinking about Strategy roles in startups/ founder's office). GV fam help me please!