QA Automation and AI Lead

We are looking for a hands-on QA Automation & AI Lead to build AI-driven quality engineering for web applications. The role involves building AI-driven frontend test automation with Playwright and LLMs, designing evaluation frameworks for LLM products, and providing technical leadership.

• Build AI-driven frontend test automation, primarily using Playwright + LLMs, to test real user journeys.
• Design frameworks where AI can help generate test cases, execute tests, validate outcomes, and analyze failures.
• Build evaluation frameworks for LLM products, including prompt evaluation, hallucination testing, regression testing, and output quality.
• Make intelligent decisions about which model to use where, balancing accuracy, latency, context size, and cost.
• Optimize token consumption for frontend automation instead of sending entire pages/context to expensive models unnecessarily.
• Identify where to use deterministic automation vs. LLM-based automation.
• Explore AI-assisted approaches to make manual and exploratory testing faster and more effective.
• Remain highly hands-on while providing technical direction and guiding engineers implementing the automation platform.

• Strong software engineering and test automation experience.
• Hands-on experience with LLMs and AI-powered applications.
• Strong experience with Playwright or similar browser automation frameworks.
• Experience with prompt/LLM evaluation, hallucination testing, and non-deterministic systems.
• Good understanding of token optimization, context management, model selection, latency, and LLM costs.
• Strong understanding of QA principles, test strategy, regression testing, and edge cases.
• Ability to prototype quickly and turn experiments into scalable automation frameworks.
• Strong technical leadership skills while remaining an individual contributor.