SDE2 - API Platform

Sarvam AI · Bengaluru · posted 29 Oct 2026
Full-timeOther3+ years

About the job

ABOUT THE ROLE Sarvam's model APIs (LLM, ASR, TTS, Vision, Translate) are how developers and enterprises ship products on our foundation models. Our OpenAI-compatible LLM APIs route every request across multiple model deployments, with streaming, failover, metering, billing, and rate-limiting all part of the same stack. Today it's FastAPI and Python; we're in the middle of rewriting it in Go. We're hiring a Software Engineer (SDE 2) to own core parts of this platform. The features. The reliability. The performance. This is an IC role. WHAT YOU'LL DO - Building and evolving the LLM API platform, from the moment a request hits the edge to the response that goes back out - The Python→Go rewrite: building services in Go alongside the existing Python stack, without compromising reliability - Streaming infrastructure: SSE for Chat Completions and Responses, long-lived connections, clean handling of cancellations and client disconnects - Routing and failover: weighted provider chains, retries with backoff, and latency-driven admission control across model deployments - Protocol compatibility: keeping the API faithful to the OpenAI spec across inference servers (vLLM, SGLang, Dynamo) that each behave slightly differently - The commercial layer: metering, billing, and rate-limiting, engineered to the same bar as the model APIs themselves - Performance and reliability for a multi-tenant platform serving long-context agentic and coding workloads - Observability (logging, metrics, tracing), so the team can debug production confidently and quickly - Integration and load testing: the harness that lets us ship fast without breaking customer-facing APIs - Close partnership with the Inference and MLOps teams on production readiness for new model launches - Design-first, documentation-first engineering culture within the team: RFCs before code, decisions written down, no tribal knowledge WHAT WE'RE LOOKING FOR - 3+ years building production backend…

Read the full description on Sarvam AI's site ↗

Skills

Upload your CV on LitCareer to see which of these you match.

Similar jobs

0
Senior Network Automation Engineering
0200 Ares Operations · Mumbai, India · 15 Nov 2026
0
Associate, Pipeline Management
0200 Ares Operations · Mumbai, India · 15 Nov 2026
0
Vice President, Corporate Technology
0200 Ares Operations · Mumbai, India · 15 Nov 2026
0
Sr. Associate (Alternative Credit)
0200 Ares Operations · Mumbai, India · 15 Nov 2026
0
Associate- Expense Process
0200 Ares Operations · Mumbai, India · 15 Nov 2026
0
Analyst, Shared Business Services (Secondaries Real Estate)
0200 Ares Operations · Mumbai, India · 15 Nov 2026

Collected from Sarvam AI's official careers site by LitCareer. Always apply on the employer's own site; LitCareer never charges you to apply.