Bengaluru · On-site

SDE2 - API Platform

Full-timePosted 29 Oct

Salary: Not disclosed
Experience: 3–6 years
Eligible batches: 2020 – 2023
Location: BengaluruOn-site

About the role

Sarvam AI is looking for a Software Engineer (SDE2) to own and evolve its LLM API platform, handling everything from edge request intake to final response. The team is transitioning the stack from Python/FastAPI to Go while maintaining high reliability and performance.

Responsibilities

Design and build core components of the LLM API platform, ensuring low‑latency, multi‑tenant operation.
Participate in the Python‑to‑Go rewrite, developing new services without disrupting existing functionality.
Implement streaming capabilities (SSE/WebSockets) and manage long‑lived connections with proper back‑pressure handling.
Develop routing, failover, and latency‑driven admission control across model deployments.
Maintain API compatibility with OpenAI specifications across various inference servers.
Build commercial features such as metering, billing, and rate‑limiting.
Provide observability through logging, metrics, and tracing, and lead integration and load testing efforts.

Free interview prep

Prepare for the interview

All lists

Picked for this role's title and skills: the questions that come up most, each with a short answer to say out loud.

Backend interview questions40 questions · ≈ 19 min to revise

Asked most often · 18 of 40

Python interview questions40 questions · Data structures, Functions, OOP

Eligibility

Minimum three years of experience building production backend systems.
Strong proficiency in Python, especially asyncio and FastAPI or similar frameworks.
Willingness to work with Go and contribute to the language transition.
Hands‑on experience with PostgreSQL and Redis, including schema design and performance tuning.
Familiarity with Kubernetes for running services in production.

Skills

PythonGoFastAPIAsyncioPostgreSQLRedisKubernetesSSEWebSockets

Resume keywords

Terms from this listing. Many screening tools match exact words, so mirror the ones you genuinely have.

Asked for

Python
Go
FastAPI
Asyncio
PostgreSQL
Redis
Kubernetes
SSE
WebSockets

Nice to have

LLM
Vllm
Sglang
Billing
Startup

Tailor my resume with these

Benefits

Opportunity to work on cutting‑edge AI infrastructure that powers large‑scale applications in India.
High ownership and impact from day one within a fast‑moving, high‑talent team.
Collaboration with researchers, engineers, and product leaders in an AI‑first environment.

Apply now

You'll apply on jobs.ashbyhq.com

Find a referralPeople at Sarvam AI on LinkedIn

Apply link checked by CareersAt.Tech and re-tested nightly. You never pay to apply. Report listing

About Sarvam AI

AI-powered operations

11-50 employees · HQ Bengaluru, India

More about Sarvam AI

Similar roles

More backend jobs