About the Role
We are seeking an experienced and visionary Lead Software Engineer specializing in Generative AI and Large Language Models (LLMs) to spearhead cutting-edge AI initiatives within our mission-critical enterprise environments. In this high-impact role, you will be responsible for architecting, developing, fine-tuning, and deploying advanced GenAI solutions directly within secure, air-gapped on-premises infrastructures.
You will collaborate closely with cross-functional technical teams, enterprise architects, and engineering leadership to deliver scalable, privacy-focused machine learning systems. By leading the adoption of state-of-the-art open-weight models, you will ensure high-throughput, low-latency performance while meeting rigorous enterprise compliance, security, and data governance standards.
Key Responsibilities:
- Architect, build, and optimize enterprise-grade Generative AI applications and custom LLM-driven workflows.
- Lead the end-to-end lifecycle of local GenAI solutions, from model evaluation to self-hosted serving and fine-tuning open-weight foundation models (such as Llama 3, Mistral, and Qwen).
- Implement high-performance model serving infrastructures using modern engines like vLLM, TensorRT-LLM, and Triton Inference Server.
- Configure and manage local GPU compute, CUDA environments, and containerized orchestration across Docker and Kubernetes/OpenShift platforms.
- Design secure, air-gapped integration pipelines ensuring strict data privacy and isolation for proprietary enterprise assets.
- Develop advanced prompt engineering frameworks, Retrieval-Augmented Generation (RAG) pipelines, and context-aware systems.
- Establish rigorous evaluation, monitoring, and benchmarking metrics to track model inference latency, accuracy, and resource utilization.
- Mentor engineering team members on AI/ML best practices, code quality, and state-of-the-art model serving methodologies.
Required Qualifications & Skills:
- Bachelor’s or Master’s degree in Computer Science, Data Science, AI, or a related quantitative field.
- 7+ years of overall software engineering experience, including 4+ years of dedicated hands-on experience in Machine Learning, Deep Learning, and AI.
- Expert-level proficiency in Python and deep expertise with AI frameworks such as PyTorch, TensorFlow, Hugging Face (transformers, peft, datasets, accelerate), spaCy, and Scikit-Learn.
- Hands-on experience deploying and fine-tuning open-weight foundation models (e.g., Llama, Mistral, Gemma, Qwen).
- Proven expertise with self-hosted local serving frameworks such as vLLM, Ollama, TensorRT-LLM, or Triton.
- Strong background in on-premises Linux systems, containerization, and local GPU orchestration using Docker and Kubernetes (Rancher, OpenShift, or microK8s).
- Demonstrated track record of deploying at least one end-to-end GenAI application into a production enterprise environment.
Preferred Qualifications:
- Experience developing production-grade RAG systems using vector databases and semantic search engines.
- Familiarity with Kore.AI or conversational AI enterprise platforms.
- Deep understanding of parameter-efficient fine-tuning (PEFT/LoRA/QLoRA) and model quantization techniques.
- Experience configuring and tuning bare-metal NVIDIA GPU clusters and network-attached storage for AI workloads.