W

AI - Software Development Engineer in Test (AI - SDET)

Web com Canada · 2 Locations · posted 10 Nov 2026
Full-timeIT & Software

About the job

This role goes beyond traditional QA; the ideal candidate should be capable of validating, evaluating, and improving AI-powered systems, agents, copilots, and LLM-driven workflows while building scalable automation frameworks and intelligent quality solutions. You will work closely with Engineering, AI/ML, Platform, Product, and DevOps teams to ensure reliability, safety, observability, scalability, and quality of modern AI-native applications and Agentic systems.Who Are We.  Newfold Digital is a leading web technology company serving nearly seven million customers globally. Established in 2021 through the combination of leading web services providers Endurance Web Presence and Web.com Group, our portfolio of brands includes: Bluehost, BigRock, ResellerClub, CrazyDomains, HostGator, Network Solutions, Register.com, Web.com and many others.   We help customers of all sizes build a digital presence that delivers results. With our extensive product offerings and personalized support, we take pride in collaborating with our customers to serve their online presence needs. Key Responsibilities AI/LLM Quality Engineering • Design, validate, and optimise testing strategies for AI-powered applications, copilots, chatbots, RAG pipelines, and autonomous AI agents. • Develop AI evaluation (Evals) frameworks for:Hallucination detectionGroundednessToxicity & bias validationResponse qualityContext relevancePrompt regression testingDeterminism & consistencySafety and guardrail validation • Build automated validation pipelines for LLM outputs using structured assertions, semantic scoring, embeddings, and evaluation metrics. • Create synthetic datasets and benchmark datasets for AI testing scenarios. • Test multi-agent orchestration workflows and agent memory/context retention. • Validate MCP (Model Context Protocol) integrations and tool interoperability. • Work on Context Engineering, Prompt Engineering, and Prompt Optimisation strategies. • Evaluate latency, token usage, model…

Read the full description on Web com Canada's site ↗

Skills

Upload your CV on LitCareer to see which of these you match.

Similar jobs

S
Design Verification Infrastructure Engineer - Software
SiFive India · Hsinchu, Taiwan · 15 Nov 2026
S
Software Engineer - Platform Technologies
SiFive India · 4 Locations · 15 Nov 2026
S
Datacenter System Software Architect
SiFive India · 50 Locations · 15 Nov 2026
S
Confidential Computing Software Architect
SiFive India · 13 Locations · 15 Nov 2026
S
Principal Design Verification Engineer – CPU Core Frontend
SiFive India · 2 Locations · 15 Nov 2026
S
Platform Technologies Language & Frontend Design Engineer
SiFive India · 3 Locations · 15 Nov 2026

Collected from Web com Canada's official careers site by LitCareer. Always apply on the employer's own site; LitCareer never charges you to apply.