This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a QA Engineer – GTM Applications based in India.
This role sits at the forefront of quality engineering for next-generation Go-To-Market systems powered by AI and large-scale enterprise integrations. You will be part of a highly technical team working across AI-driven workflows, Salesforce ecosystems, Slack applications, and cloud-native architectures to ensure reliable, secure, and scalable delivery. The position requires rethinking traditional QA approaches to accommodate non-deterministic AI systems, where evaluation is based on probabilistic outputs rather than fixed outcomes. You will design advanced testing frameworks, define quality standards for agentic AI workflows, and ensure robust validation across complex distributed systems. This is a high-impact role where QA becomes a core enabler of innovation, directly influencing the reliability of AI-powered business processes. You will collaborate closely with engineers, product owners, and AI specialists in a fast-paced, experimentation-driven environment.
Accountabilities:
- Define and own the end-to-end QA strategy for agentic AI and GTM workflows, establishing quality standards for non-deterministic and LLM-powered systems.
- Design AI-specific evaluation frameworks covering hallucination detection, prompt scoring, response faithfulness, and agent task completion metrics.
- Develop and maintain scalable automated test suites in Python using frameworks such as pytest and Robot Framework across unit, integration, and system levels.
- Build and execute RAG pipeline testing strategies, including retrieval accuracy, semantic relevance, context grounding, and end-to-end response validation.
- Design QA automation for integrated systems including Salesforce, Slack applications, APIs, and cloud-based GTM tools.
- Integrate automated testing into CI/CD pipelines (GitHub Actions, Jenkins, Copado) to enforce quality gates on every deployment.
- Conduct performance, load, and scalability testing for LLM-powered APIs, analyzing latency, throughput, and degradation under stress.
- Perform security and adversarial testing, including prompt injection resistance, data leakage prevention, and validation with DevSecOps teams.
- Establish regression strategies for probabilistic systems using statistical thresholds, variance analysis, and snapshot-based validation techniques.
- Collaborate with engineering and product teams from requirements definition through release to ensure testability and quality alignment.
- Own API contract testing across multiple internal and third-party systems including Salesforce, Marketo, Snowflake, Gong, Clari, and G-Suite.
- Drive defect lifecycle management including triage, root cause analysis, regression prevention, and continuous improvement of QA processes.
- Promote a shift-left quality culture by embedding testing practices early in development and enabling engineers to build testable systems.
Requirements:
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field.
- 6+ years of experience in QA, SDET, or test automation roles within complex enterprise environments.
- Strong hands-on experience in Python for building scalable test automation frameworks.
- Proven experience testing AI/ML or LLM-based systems with understanding of non-deterministic behavior and evaluation challenges.
- Expertise with test automation tools such as pytest, Selenium, Playwright, Postman, or equivalent frameworks.
- Strong understanding of LLM behavior, including prompt sensitivity, temperature effects, token limits, and failure modes.
- Hands-on experience with Salesforce testing, including APIs, Lightning Web Components, and event-driven architectures.
- Strong API testing expertise including contract validation, schema verification, and error handling.
- Experience integrating automated testing into CI/CD pipelines to enforce continuous quality gates.
- Familiarity with RAG evaluation metrics and tools such as RAGAS or LangSmith for AI system assessment.
- Experience with performance testing tools such as k6, Locust, or JMeter for high-scale systems.
- Working knowledge of security testing principles including OWASP guidelines and prompt injection mitigation strategies.
Benefits:
- Competitive compensation with market-leading equity opportunities
- Comprehensive physical and mental wellness programs
- Generous paid vacation, holidays, and parental leave policies
- Strong professional development and continuous learning opportunities
- Remote-friendly and flexible work arrangements (#LI-Remote)
- Employee resource groups and global community initiatives
- Inclusive and high-performance engineering culture
- Opportunity to work on cutting-edge AI, cybersecurity, and enterprise-scale systems