Atul V

Software Engineer - AI Platform

Mindtickle • Vollzeit

Jul 2024 - Present • 2 yrs 1 mo

− Architected and solely owned; a production-grade, vendor-agnostic LLM Gateway in Go (multiple- microservices on AWS EKS) supporting synchronous, streaming, and async inference over gRPC; sustains 100+ req/sec peak across 10+ AI product teams, serving as the core AI infrastructure layer org-wide. − Enabled multi-cloud LLM integration across AWS Bedrock, Azure OpenAI, and Google Vertex AI, abstracting provider differences so teams switch models without code changes; reduced AI feature onboarding from days to hours. − Built and shipped a production MCP server exposing Mindtickle tools (search, copilot) as first-class MCP tools to BETA customers, with JWT-based auth and fine-grained scope control for secure external access. − Designed and delivered multi-agent agentic AI workflows powering Seller Behaviour Intel and Deal Guides; orchestrating LLM agents to autonomously research companies, gather signals, and synthesize structured sales intelligence outputs. − Established end-to-end LLM evaluation pipelines from scratch using RAGAS, Maxim, and Braintrust for production AI features - setting quality benchmarks, tracking regressions, and driving iterative prompt improvements with product teams. − Built distributed orchestration primitives: model routing, async execution, usage metering, and business event emission using Kafka (AWS MSK), AWS Lambda, and Redis. − Implemented end-to-end observability with Datadog for latency, cost, and LLM usage tracking; provisioned monitors via Terraform, improving reliability and cost transparency of AI workloads across the org. − Deployed services on AWS EKS with GitLab CI/CD and GitOps-based prompt versioning, enabling zero-downtime multi-region releases and A/B evaluation of LLM responses in production. − Wrote integration and load tests for LLM inference services, validating reliability of streaming and async workloads under production traffic.