EXL—Senior Software Engineer - AI Gateway & Agentic AI Solutions

Job description

Description: Top 5 Responsibilities - Architect and implement the AI Gateway. Design and build an Azure-native LLM gateway in C# providing unified ingress across multiple model providers (Azure OpenAI, Azure AI Foundry models, and others) - covering intelligent routing, fallback/load balancing, rate limiting and token-quota enforcement, semantic caching, and centralized API key/secret management. Reference the LiteLLM feature set as the functional bar to hit. - Build the observability and governance layer. Implement request/response tracing, prompt/completion logging, token and cost metering, and latency dashboards - the Langfuse-equivalent half of the stack - using Azure Monitor, Application Insights, OpenTelemetry, and APIM's native LLM logging/token-metric policies (or a self-hosted Langfuse instance where warranted). - Deliver multiple Agentic AI PoC scenarios. Using C# and the Microsoft Agent Framework, build a portfolio of distinct agentic patterns (single-agent tool use, multi-agent orchestration, human-in-the-loop workflows, RAG-grounded agents) mapped to real client business use cases - not one deep PoC, but several breadth-covering scenarios that demonstrate different capabilities. - Own technical delivery on client engagements end-to-end. Run architecture proposals, hands-on build, live demos, and production-readiness assessments directly with client technical stakeholders; translate ambiguous business asks into scoped, demoable agentic scenarios. - Package the work as reusable engineering assets. Turn the gateway and agent scenarios into templates, SDKs, or IaaC that can be re-deployed across environments rather than rebuilt from scratch each time - even though this is a client-facing role, the artifacts should outlive any single client environment. Requirements: Top 5 Required Skills / Background - Deep C#/.NET engineering background (12-15+ years). Production-grade API/service development, async patterns, dependency injection, testing discipline, and comfort operating without a large surrounding platform team. - Hands-on Azure platform expertise. Azure API Management (including GenAI/LLM-specific policies: token-limit, token-metric, semantic-caching), Azure OpenAI / Azure AI Foundry, Azure Monitor / Application Insights, Key Vault, and either Azure Functions/App Service or AKS for hosting gateway services. - Working knowledge of LLM gateway and observability platforms. Familiarity with LiteLLM, Langfuse, or comparable tools (Portkey, Kong AI Gateway) - not to operate them directly, but to translate their proven patterns (routing, fallback, semantic caching, token metering, tracing) into an Azure/C# implementation. - Microsoft Agent Framework experience, or a direct path into it. Hands-on work with MAF, or strong prior experience in Semantic Kernel and/or AutoGen (MAF's predecessors) - multi-agent orchestration, tool/function calling, thread/state management, and ideally MCP/A2A protocol exposure. - Client-facing solution engineering track record. Demonstrated ability to architect, prototype, and present technical PoCs directly to enterprise client stakeholders, defend design trade-offs live, and adapt scope under engagement time pressure. Responsibilities: Top 5 Required Skills / Background - Deep C#/.NET engineering background (12-15+ years). Production-grade API/service development, async patterns, dependency injection, testing discipline, and comfort operating without a large surrounding platform team. - Hands-on Azure platform expertise. Azure API Management (including GenAI/LLM-specific policies: token-limit, token-metric, semantic-caching), Azure OpenAI / Azure AI Foundry, Azure Monitor / Application Insights, Key Vault, and either Azure Functions/App Service or AKS for hosting gateway services. - Working knowledge of LLM gateway and observability platforms. Familiarity with LiteLLM, Langfuse, or comparable tools (Portkey, Kong AI Gateway) - not to operate them directly, but to translate their proven patterns (routing, fallback, semantic caching, token metering, tracing) into an Azure/C# implementation. - Microsoft Agent Framework experience, or a direct path into it. Hands-on work with MAF, or strong prior experience in Semantic Kernel and/or AutoGen (MAF's predecessors) - multi-agent orchestration, tool/function calling, thread/state management, and ideally MCP/A2A protocol exposure. - Client-facing solution engineering track record. Demonstrated ability to architect, prototype, and present technical PoCs directly to enterprise client stakeholders, defend design trade-offs live, and adapt scope under engagement time pressure.