Thriventree Consulting Pvt. Ltd is urgently hiring an AI QA Engineer with strong experience in QA Automation, Generative AI (GenAI), Large Language Models (LLMs), Agentic AI, and AI testing.
This opportunity is suitable for experienced QA professionals with 7+ years of QA experience who have hands-on expertise in Python-based automation and are comfortable testing modern AI/LLM applications, including RAG and multi-agent workflows.
๐ Location: Bangalore โ Hybrid
๐ผ Experience: 7+ Years in QA
โณ Availability: Immediate Joiners / Less than 15 Days Preferred
If you have experience validating AI outputs, testing hallucinations, designing AI evaluation metrics, and building automation frameworks, this could be a relevant opportunity to explore.
๐ Position Details
๐ข Company: Thriventree Consulting Pvt. Ltd
๐ผ Position: AI QA Engineer โ Agentic AI / GenAI
๐ Job ID: Not Mentioned
๐ Location: Bangalore โ Hybrid
๐จโ๐ป Experience Required: 7+ Years QA
โณ Joining Preference: Immediate Joiners / Less than 15 Days Preferred
๐ Qualification: Not Mentioned
๐ข Employment Type: Not Mentioned
๐ Eligibility: Experienced QA professionals with relevant AI/GenAI and automation testing experience
๐ ๏ธ Mandatory Skills
โ
Python
โ
Pytest / Unittest
โ
QA Automation
โ
GenAI / LLMs
โ
LLM Output Validation
โ
Hallucination Testing
โ
Agentic AI / Multi-Agent Testing
โ
RAG Testing
โ
API & Integration Testing
โ
Precision / Recall / F1 Metrics
โ
CI/CD
โ
Git
โ
Debugging
โ
Test Automation Framework Development
โ
Regression & End-to-End Testing
๐ Position Overview
The AI QA Engineer role focuses on testing and validating applications powered by Generative AI, LLMs, Agentic AI, and RAG architectures.
The selected candidate will be responsible for developing Python-based test automation frameworks and creating comprehensive test scenarios for AI-powered applications.
A key part of the role involves evaluating LLM responses for accuracy, relevance, consistency, and hallucinations. The role also requires testing Agentic AI and multi-agent workflows, where multiple AI agents may interact to complete a task.
Candidates will also work on API, integration, regression, and end-to-end testing while contributing to AI evaluation using metrics such as Precision, Recall, and F1-score.
Strong debugging skills and experience integrating automated tests into CI/CD pipelines will also be important for maintaining production-quality applications.
๐ฏ Key Responsibilities
๐น Build and maintain Python-based QA automation frameworks.
๐น Develop automated test cases using Pytest/Unittest.
๐น Test and validate LLM-generated responses for accuracy and relevance.
๐น Identify and test potential LLM hallucinations and incorrect outputs.
๐น Design test scenarios for Generative AI applications.
๐น Validate Agentic AI and multi-agent workflows.
๐น Perform testing of RAG-based applications and workflows.
๐น Conduct API and integration testing for AI-powered services.
๐น Execute regression and end-to-end testing to ensure application quality.
๐น Develop and apply AI evaluation metrics such as Precision, Recall, and F1-score.
๐น Integrate automated testing into CI/CD pipelines.
๐น Debug automation failures and application issues.
๐น Improve test coverage and help ensure production-quality software.
๐ข About the Company
Thriventree Consulting Pvt. Ltd is the organization associated with this hiring opportunity. The provided job details focus on opportunities in AI QA, GenAI, Agentic AI, and automation testing.
Additional information about company size, products, clients, awards, benefits, or work culture was not provided in the job details.
๐ฐ Salary Range
๐ต Salary: Not Mentioned
๐งช Expected Interview Rounds
The exact interview process has not been mentioned in the job posting. Candidates may generally expect some combination of:
1๏ธโฃ Resume Shortlisting
2๏ธโฃ HR Screening
3๏ธโฃ Technical Assessment
4๏ธโฃ Technical Interview
5๏ธโฃ Managerial/Final Discussion
6๏ธโฃ HR Discussion & Documentation
โ ๏ธ These are general expected rounds, not confirmed Thriventree Consulting interview rounds.
๐ Preparation Tips for AI QA Engineer
If you’re planning to apply, consider preparing the following areas:
โ Revise Python fundamentals, especially concepts used in automation.
โ Practice writing automated tests using Pytest and Unittest.
โ Understand QA automation framework design and maintenance.
โ Learn how to validate LLM-generated responses.
โ Prepare concepts related to LLM hallucination testing.
โ Understand GenAI testing strategies and evaluation techniques.
โ Revise Agentic AI and multi-agent architecture concepts.
โ Understand the fundamentals of RAG testing, including retrieval and response validation.
โ Practice API and integration testing.
โ Understand Precision, Recall, and F1-score and how they can be used for AI evaluation.
โ Revise CI/CD and Git workflows.
โ Practice debugging automation failures and identifying root causes.
๐ Resume Tips
โ Highlight your 7+ years of QA experience clearly.
โ Mention Python automation experience prominently.
โ Add Pytest/Unittest experience if applicable.
โ Highlight experience with GenAI, LLMs, or AI testing.
โ Mention hands-on experience with LLM output validation and hallucination testing.
โ Include experience with Agentic AI / multi-agent systems.
โ Highlight RAG testing experience if applicable.
โ Mention API and integration testing expertise.
โ Add experience with CI/CD pipelines and Git.
โ Include measurable automation achievements such as improved test coverage, reduced execution time, or automation percentage where applicable.
โ Tailor your resume specifically toward AI QA / GenAI QA Engineer roles.
โ Frequently Asked Questions
Q1. What is the position available?
The position is AI QA Engineer โ Agentic AI / GenAI.
Q2. How much experience is required?
The requirement is 7+ years of QA experience.
Q3. Where is the job located?
The position is based in Bangalore with a hybrid work arrangement.
Q4. Are immediate joiners preferred?
Yes. Immediate joiners or candidates with less than 15 days’ notice period are preferred.
Q5. Is Python mandatory?
Yes. Python is listed among the mandatory skills.
Q6. Is AI/LLM testing experience required?
Yes. The requirements specifically include GenAI/LLMs, LLM output validation, hallucination testing, Agentic AI/multi-agent testing, and RAG testing.
Q7. Is API testing part of the role?
Yes. API and integration testing are included in the mandatory skills and responsibilities.
Q8. Are CI/CD and Git required?
Yes. Both CI/CD and Git are listed as mandatory skills.
Q9. What AI evaluation metrics should candidates know?
The job details specifically mention Precision, Recall, and F1-score.
Q10. How can candidates apply?
Candidates can share their CV with vanitha.s@thriventree.com.
๐ Why Consider This Opportunity?
โจ Work in the emerging AI Quality Engineering domain.
โจ Apply QA automation skills to GenAI and LLM-based applications.
โจ Gain exposure to LLM output validation.
โจ Work on hallucination testing.
โจ Explore testing strategies for Agentic AI and multi-agent systems.
โจ Work with RAG-based applications.
โจ Apply AI evaluation metrics such as Precision, Recall, and F1-score.
โจ Work with Python-based automation frameworks.
โจ Apply API, integration, regression, and end-to-end testing skills.
โจ Work with CI/CD-based automation workflows.
๐ฉ How to Apply
๐ง Email Your Resume: vanitha.s@thriventree.com
๐ Application Link: Not Mentioned
๐ Suggested Subject Line: Application for AI QA Engineer โ Bangalore
๐ Include in Your Email:
โข Position Applying For: AI QA Engineer
โข Total Experience: 7+ Years QA
โข Current Location
โข Notice Period / Availability
โข Updated Resume
โข Contact Number
โข Relevant AI/GenAI & Automation Experience
โณ Immediate Joiners / Candidates available within 15 days are preferred.
๐ข Know someone who is an experienced QA professional looking for opportunities in AI/GenAI testing? Share this opening with your friends, colleagues, and professional network.
