Are you a QA professional looking to move into the rapidly evolving world of Generative AI, Large Language Models and Agentic AI? Nagarro is hiring an AI/LLM QA Engineer for a remote opportunity in India. This role is particularly suited to testers who understand modern automation and want to build expertise in evaluating AI-powered applications, LLM responses and autonomous AI-agent workflows.
The hiring requirement focuses on LLM and prompt-based testing, AI Agent/Agentic AI testing, Playwright with TypeScript and Python. Experience with Promptfoo or similar LLM evaluation tools, functional/integration/E2E testing and identifying hallucinations, edge cases and failure scenarios is an additional advantage.
📌 Job Details
🏢 Company: Nagarro
💼 Role: AI/LLM QA Engineer
📍 Location: Remote – India
🤖 Domain: AI / GenAI / LLM / Agentic AI Testing
🎭 Automation: Playwright
📘 Languages: TypeScript & Python
🧠 Core Testing: LLM & Prompt-Based Testing
🧩 AI Testing: AI Agents / Agentic AI
🔎 Evaluation Tools: Promptfoo or similar – Good to Have
🧪 Testing: Functional, Integration & E2E
⚠️ Key Focus: Hallucinations, edge cases and AI failure scenarios
📩 Apply: santhanaselvi.s@nagarro.com
🏢 About Nagarro
Nagarro is a global digital engineering and technology company working across software engineering, cloud, data, AI and other technology areas. Its India careers platform highlights challenging projects, continuous learning, a flat hierarchy and opportunities to work with evolving technologies.
💻 What Will You Work On?
The role moves beyond traditional UI automation. As an AI/LLM QA Engineer, you may test applications where the expected output can vary depending on prompts, context and model behavior.
LLM testing can involve creating prompt suites, evaluating responses, checking hallucinations, testing adversarial or ambiguous inputs and identifying situations where the model produces incomplete or misleading results. For an AI agent, testing can additionally involve checking whether the agent selects the appropriate tool, follows the intended workflow, handles failures and produces the expected final outcome.
The requirement for Playwright + TypeScript + Python means traditional automation remains an important part of the role. You may use browser automation for end-to-end validation while Python can support evaluation scripts, test utilities, data processing or AI-testing workflows.
🌟 Why This Role Is Good for Experience
This opportunity can help an experienced QA professional expand from conventional automation into AI quality engineering. The combination of Playwright, TypeScript and Python gives you a conventional automation foundation, while LLM and Agentic AI testing introduces a newer testing discipline.
You can gain practical experience in areas such as prompt evaluation, AI reliability, hallucination detection, agent workflow validation, regression testing for model behavior and automated LLM evaluation.
🧠 Skills Required
🤖 LLM Testing: Prompt suites, response evaluation, context validation and regression testing.
✍️ Prompt Engineering: System prompts, user prompts, context, constraints and structured outputs.
🧩 Agentic AI Testing: Tool calling, multi-step workflows, agent decisions, state management and failure recovery.
🎭 Playwright: Locators, fixtures, assertions, browser contexts, parallel execution and debugging.
📘 TypeScript: Types, interfaces, async/await, promises, modules and reusable automation code.
🐍 Python: OOP, collections, APIs, data processing and test automation utilities.
🔎 LLM Evaluation: Accuracy, relevance, consistency, groundedness and hallucination detection.
🧪 Functional/E2E: UI workflows, integration validation and end-to-end scenarios.
⚠️ Edge-Case Testing: Ambiguous prompts, missing context, malformed input and unexpected user behavior.
🔧 CI/CD: Git, pipelines and automated regression execution are valuable additions.
🎯 Expected Interview Rounds at Nagarro
💻 Round 1 – Screening/Assessment: Resume discussion, QA fundamentals, automation experience and potentially coding or technical assessment.
🤖 Round 2 – AI/LLM Technical: LLM concepts, prompt testing, hallucinations, evaluation methodology and AI-agent scenarios.
🎭 Round 3 – Automation: Playwright, TypeScript, Python, framework design and E2E testing.
🧩 Round 4 – Scenario/Project Discussion: You may be asked to design a test strategy for an LLM chatbot or an AI agent and explain how you would detect failures.
👔 Round 5 – HR/Client Discussion: Communication, project experience, remote collaboration and role alignment.
These are preparation areas rather than a confirmed sequence for the current vacancy.
💰 Salary Range
The hiring post does not specify a salary range for this AI/LLM QA Engineer position. Public Nagarro salary submissions show substantial variation by QA role, experience and location. For example, reported QA Engineer compensation ranges from approximately ₹4.13 lakh to ₹8 lakh per year in base pay in the broader dataset, while reported Senior QA Engineer figures extend higher; these figures are not specific to this AI/LLM vacancy.
Nagarro’s reported QA Automation Engineer entries also span approximately ₹4 lakh–₹10 lakh/year, but these are employee-submitted figures across different experience levels rather than an official range for this vacancy. Candidates should confirm the actual CTC with the recruiter.
🔥 Preparation Tips
🤖 Test an LLM like a QA engineer: Create 20–30 prompts covering normal, ambiguous, incomplete and adversarial inputs and define what a successful response should look like.
⚠️ Study hallucinations: Prepare examples of factual fabrication, unsupported claims and incorrect context usage, and explain how you would detect them.
🧩 Test an AI agent: Design scenarios for successful tool use, incorrect tool selection, unavailable tools, repeated calls, timeouts and partial failures.
🎭 Strengthen Playwright: Practice fixtures, page objects, network interception, parallel execution and API/UI integration.
📘 Practice TypeScript: Focus on async/await, interfaces, types, generics and clean reusable test utilities.
🐍 Use Python for evaluation: Practice JSON processing, API calls, dataset handling and automated comparison of expected versus generated outputs.
🔎 Learn LLM evaluation tools: Explore Promptfoo or similar frameworks and understand how evaluation datasets and assertions can be used for regression testing.
📊 Think probabilistically: Unlike traditional deterministic tests, AI testing often requires thresholds, rubrics and multiple evaluation criteria.
📄 Resume Tip
Your resume should immediately communicate the transition from traditional QA to AI quality engineering. A strong headline could be: “QA Automation Engineer | Playwright | TypeScript | Python | LLM & GenAI Testing.”
Place LLM testing, Agentic AI testing, Playwright, TypeScript and Python prominently if you genuinely have hands-on experience. Describe AI-testing projects using concrete examples: prompt datasets created, hallucinations identified, evaluation criteria designed, agent workflows tested or regression suites automated.
If you have used Promptfoo, LangChain, LLM APIs, vector databases, RAG applications or AI-agent frameworks, include them in a dedicated project section and be prepared to explain exactly what you tested.
📩 How to Apply
📩santhanaselvi.s@nagarro.com
Interested candidates should prepare an updated resume emphasizing AI/LLM testing, Agentic AI, Playwright, TypeScript and Python.
📧 Send your resume to: santhanaselvi.s@nagarro.com
🚀 Final Takeaway
This Nagarro opening combines modern automation engineering with AI quality engineering. The most distinctive requirements are LLM/prompt testing, Agentic AI testing, Playwright, TypeScript and Python, with Promptfoo-style evaluation and hallucination analysis providing additional relevance. For QA professionals moving toward AI, the role calls for a different testing mindset: defining quality criteria for variable model outputs, systematically probing failure modes and validating complete AI-agent workflows.
