Are you an experienced QA professional who has moved beyond traditional web and API testing and worked directly with AI, LLM, Generative AI or Conversational AI applications? A new AI Quality Engineer opportunity is open for professionals with 5β9 years of experience, strong automation skills and genuine hands-on AI testing experience.
This is not a conventional QA automation position. The requirement specifically emphasizes professionals who have tested AI-powered applications rather than simply using tools such as ChatGPT or Copilot during regular software development projects.
π Role: AI Quality Engineer
πΌ Experience: 5β9 Years
π Work Mode: Remote
β‘ Notice Period: Immediate Joiners Only
π€ Focus: AI/LLM/Generative AI/Conversational AI Testing
π§ Apply: sonali.shraiya@programming.com
π’ About the Company
The employer is not identified in the vacancy shared, so candidates should verify the company name, official website and employment details with the recruiter before proceeding.
This means the successful candidate is likely to work at the intersection of Quality Engineering, AI evaluation, automation, API testing and software engineering.
π Why Is This Role Good for Experienced QA Engineers?
That creates an exciting career path for experienced QA professionals.
In this role, you could gain practical experience in:
π€ LLM and Generative AI testing
π¬ Conversational AI validation
π§ Prompt and model regression testing
π AI-agent workflow testing
π οΈ Tool-calling validation
π AI evaluation datasets
βοΈ Automation framework development
π API and backend testing
π End-to-end AI workflow validation
π οΈ Skills Required
Candidates should have strong traditional QA fundamentals combined with AI-testing knowledge.
AI Testing: LLM testing, Generative AI testing, chatbot testing, conversational AI, prompt testing and response evaluation.
Conversational Testing: Intent recognition, intent routing, context retention, multi-turn conversations, fallback scenarios and escalation workflows.
AI Evaluation: Relevance, correctness, groundedness, hallucination detection, response consistency and model regression.
Agent Testing: Tool calling, workflow orchestration, agent-to-agent interactions and failure handling.
Automation: Playwright, Selenium or Cypress.
Programming: Python, JavaScript, Java or C#.
API: Postman, REST APIs, authentication, request/response validation and backend integration.
Database: SQL, data validation and backend verification.
Testing Methodologies: Functional, integration, regression and end-to-end testing.
Additional exposure to LangGraph, LangChain, MCP, Azure OpenAI and AI Agents can provide an advantage.
π― Expected Interview Rounds
Round 1 β Recruiter Screening: Experience, AI-testing background, notice period, remote-work suitability and compensation expectations.
Round 2 β QA/Automation Technical Round: Selenium/Playwright/Cypress, API testing, SQL, automation frameworks and software-testing methodologies.
Round 3 β AI Testing Round: This is likely to be the most important round. Expect scenario-based questions around hallucinations, prompt regression, context retention, intent routing, groundedness and AI response evaluation.
Round 4 β Practical/Design Discussion: You may be asked to design an AI testing strategy or framework for a chatbot or AI agent.
Round 5 β Managerial/Final Round: Ownership, communication, problem-solving and experience handling production-quality issues may be evaluated.
π° Salary Range
The vacancy does not mention a fixed salary.
For 5β9 years of experienced AI/QA automation professionals in India, a broad market-oriented expectation could be approximately βΉ12β30+ LPA, depending heavily on AI-testing depth, automation expertise, location, employer type and seniority.
This is a market estimate, not a confirmed salary range for this vacancy.
π₯ Preparation Tips
Don’t prepare like a traditional Selenium interview.
Take a hypothetical customer-support AI chatbot and create a complete testing strategy.
Test:
πΉ Incorrect intent classification
πΉ Context loss between messages
πΉ Prompt injection attempts
πΉ Hallucinated answers
πΉ Unsupported claims
πΉ Poorly grounded responses
πΉ Tool/API failures
πΉ Authentication failures
πΉ Fallback and escalation
πΉ Conversation loops
πΉ Model-version regression
Then design an automated evaluation approach.
Also understand the difference between traditional pass/fail testing and AI evaluation using metrics such as relevance, groundedness, correctness and consistency.
π Resume Tips
Your resume must clearly demonstrate that you have actually tested AI systems.
Avoid:
β βUsed ChatGPT for testing.β
Instead write:
β βDesigned automated regression scenarios for LLM-powered conversational workflows, validating intent routing, context retention and response quality.β
Strong keywords to include naturally:
AI Quality Engineering | LLM Testing | Generative AI Testing | Conversational AI | AI Agents | Prompt Testing | Model Regression | AI Evaluation | Groundedness | Hallucination Testing | Playwright | Selenium | Cypress | Python | REST API | Postman | SQL | E2E Testing | LangChain | LangGraph | MCP
π© How to Apply
Interested candidates meeting the 5β9 years requirement and possessing genuine AI-testing experience can share their updated CV at:
π§ sonali.shraiya@programming.com
β‘ Immediate joiners only according to the vacancy.
Before applying, make sure your resume specifically demonstrates hands-on experience with AI/LLM/Chatbot testing. Candidates whose only AI exposure is using ChatGPT or Copilot for ordinary development/testing work may not match the requirement.
π Final Takeaway
This opportunity is particularly relevant for experienced QA professionals who want to transition from traditional automation into AI Quality Engineering.
The key differentiator isn’t simply knowing ChatGPTβit is understanding how AI systems fail and how those failures can be systematically detected, measured, automated and prevented.
If you can combine strong QA fundamentals with LLM evaluation, conversational testing, AI-agent validation and modern automation, this type of role can position you strongly for the rapidly growing AI testing market. π€π
