Are you an experienced QA professional who has moved beyond traditional web and API testing and worked directly with AI, LLM, Generative AI or Conversational AI applications? A new AI Quality Engineer opportunity is open for professionals with 5โ9 years of experience, strong automation skills and genuine hands-on AI testing experience.
This is not a conventional QA automation position. The requirement specifically emphasizes professionals who have tested AI-powered applications rather than simply using tools such as ChatGPT or Copilot during regular software development projects.
๐ Role: AI Quality Engineer
๐ผ Experience: 5โ9 Years
๐ Work Mode: Remote
โก Notice Period: Immediate Joiners Only
๐ค Focus: AI/LLM/Generative AI/Conversational AI Testing
๐ง Apply: sonali.shraiya@programming.com
๐ข About the Company
The employer is not identified in the vacancy shared, so candidates should verify the company name, official website and employment details with the recruiter before proceeding.
This means the successful candidate is likely to work at the intersection of Quality Engineering, AI evaluation, automation, API testing and software engineering.
๐ Why Is This Role Good for Experienced QA Engineers?
That creates an exciting career path for experienced QA professionals.
In this role, you could gain practical experience in:
๐ค LLM and Generative AI testing
๐ฌ Conversational AI validation
๐ง Prompt and model regression testing
๐ AI-agent workflow testing
๐ ๏ธ Tool-calling validation
๐ AI evaluation datasets
โ๏ธ Automation framework development
๐ API and backend testing
๐ End-to-end AI workflow validation
๐ ๏ธ Skills Required
Candidates should have strong traditional QA fundamentals combined with AI-testing knowledge.
AI Testing: LLM testing, Generative AI testing, chatbot testing, conversational AI, prompt testing and response evaluation.
Conversational Testing: Intent recognition, intent routing, context retention, multi-turn conversations, fallback scenarios and escalation workflows.
AI Evaluation: Relevance, correctness, groundedness, hallucination detection, response consistency and model regression.
Agent Testing: Tool calling, workflow orchestration, agent-to-agent interactions and failure handling.
Automation: Playwright, Selenium or Cypress.
Programming: Python, JavaScript, Java or C#.
API: Postman, REST APIs, authentication, request/response validation and backend integration.
Database: SQL, data validation and backend verification.
Testing Methodologies: Functional, integration, regression and end-to-end testing.
Additional exposure to LangGraph, LangChain, MCP, Azure OpenAI and AI Agents can provide an advantage.
๐ฏ Expected Interview Rounds
Round 1 โ Recruiter Screening: Experience, AI-testing background, notice period, remote-work suitability and compensation expectations.
Round 2 โ QA/Automation Technical Round: Selenium/Playwright/Cypress, API testing, SQL, automation frameworks and software-testing methodologies.
Round 3 โ AI Testing Round: This is likely to be the most important round. Expect scenario-based questions around hallucinations, prompt regression, context retention, intent routing, groundedness and AI response evaluation.
Round 4 โ Practical/Design Discussion: You may be asked to design an AI testing strategy or framework for a chatbot or AI agent.
Round 5 โ Managerial/Final Round: Ownership, communication, problem-solving and experience handling production-quality issues may be evaluated.
๐ฐ Salary Range
The vacancy does not mention a fixed salary.
For 5โ9 years of experienced AI/QA automation professionals in India, a broad market-oriented expectation could be approximately โน12โ30+ LPA, depending heavily on AI-testing depth, automation expertise, location, employer type and seniority.
This is a market estimate, not a confirmed salary range for this vacancy.
๐ฅ Preparation Tips
Don’t prepare like a traditional Selenium interview.
Take a hypothetical customer-support AI chatbot and create a complete testing strategy.
Test:
๐น Incorrect intent classification
๐น Context loss between messages
๐น Prompt injection attempts
๐น Hallucinated answers
๐น Unsupported claims
๐น Poorly grounded responses
๐น Tool/API failures
๐น Authentication failures
๐น Fallback and escalation
๐น Conversation loops
๐น Model-version regression
Then design an automated evaluation approach.
Also understand the difference between traditional pass/fail testing and AI evaluation using metrics such as relevance, groundedness, correctness and consistency.
๐ Resume Tips
Your resume must clearly demonstrate that you have actually tested AI systems.
Avoid:
โ โUsed ChatGPT for testing.โ
Instead write:
โ โDesigned automated regression scenarios for LLM-powered conversational workflows, validating intent routing, context retention and response quality.โ
Strong keywords to include naturally:
AI Quality Engineering | LLM Testing | Generative AI Testing | Conversational AI | AI Agents | Prompt Testing | Model Regression | AI Evaluation | Groundedness | Hallucination Testing | Playwright | Selenium | Cypress | Python | REST API | Postman | SQL | E2E Testing | LangChain | LangGraph | MCP
๐ฉ How to Apply
Interested candidates meeting the 5โ9 years requirement and possessing genuine AI-testing experience can share their updated CV at:
๐ง sonali.shraiya@programming.com
โก Immediate joiners only according to the vacancy.
Before applying, make sure your resume specifically demonstrates hands-on experience with AI/LLM/Chatbot testing. Candidates whose only AI exposure is using ChatGPT or Copilot for ordinary development/testing work may not match the requirement.
๐ Final Takeaway
This opportunity is particularly relevant for experienced QA professionals who want to transition from traditional automation into AI Quality Engineering.
The key differentiator isn’t simply knowing ChatGPTโit is understanding how AI systems fail and how those failures can be systematically detected, measured, automated and prevented.
If you can combine strong QA fundamentals with LLM evaluation, conversational testing, AI-agent validation and modern automation, this type of role can position you strongly for the rapidly growing AI testing market. ๐ค๐
