Noida · On-site

AI QA Engineer

Full-timePosted 27 Oct

Salary: ₹4.3–11.3 LPAEstimated from AmbitionBox data for this role across companies. The employer has not stated pay.
Experience: 2–3 years
Eligible batches: 2023 · 2024
Degree: B.S., M.S.
Location: NoidaOn-site

About the role

apexanalytix is looking for an AI QA Engineer to own the quality and safety of its AI‑driven products for Global 2000 clients. Based in Noida, the role blends data‑science knowledge with test‑automation expertise to validate LLM features, RAG pipelines, and ML models.

Responsibilities

Define and execute test strategies for AI solutions, covering LLMs, agentic workflows, and retrieval‑augmented pipelines.
Create test plans, cases and defect reports while performing exploratory testing of AI features.
Establish acceptance criteria and quality metrics for non‑deterministic outputs such as accuracy, hallucination rate and cost.
Maintain golden datasets, ground‑truth labels and regression suites in collaboration with subject‑matter experts.
Build automated evaluation pipelines (DeepEval, Ragas, promptfoo, LangSmith) that run in CI/CD for every model or prompt change.
Conduct adversarial and safety testing, including prompt injection, jailbreaks and PII leakage checks.

Eligibility

B.S. or M.S. in Computer Science, Data Science, AI/ML, Statistics or a related discipline.
2–3 years of experience in AI/ML, data science or AI‑focused quality assurance.
At least one year of hands‑on test automation for production ML systems.

Skills

Strong Python programming and PyTest for test automation.
API testing experience with CI/CD integration.
Statistical analysis and experimentation knowledge.
SQL proficiency and familiarity with data‑validation tools such as Great Expectations or Evidently.
Understanding of LLM behavior, prompting and retrieval‑augmented generation.
Experience evaluating GenAI applications using tools like DeepEval or Ragas.

Resume keywords

Terms from this listing. Many screening tools match exact words, so mirror the ones you genuinely have.

PythonPytestAPI testingCI/CDSQLStatisticsGreat expectationsEvidentlyDeepevalRagasDockerKubernetesMLOpsLangChainPlaywright

Tailor my resume with these →

Benefits

Comprehensive health and wellness programs through the BeWell@apex initiative.
Mentor program, internal training portal, and tuition‑assistance for certifications.
Opportunities to work on cutting‑edge AI products for Global 2000 customers.

Apply now

You'll apply on ats.rippling.com

Find a referralPeople at apexanalytix on LinkedIn

Curated and verified by CareersAt.Tech. You never pay to apply. Report listing

About apexanalytix

Apex Analytix provides a technology platform for supplier onboarding, risk management and profit recovery for large enterprises. It serves Fortune 500 and Global 2000 companies, protecting over $10 trillion in annual spend. The company combines AI and a large data set to deliver touchless risk monitoring.

Founded 1988 · 201-500 employees · HQ Greensboro, USA

More about apexanalytix

Similar roles

Browse all jobs