Mounika – Python, LLM, RAG

Enterprise-scale evaluation platform for production LLMs and RAG systems, measuring response correctness, hallucinations, grounding quality, bias, and safety across open-source and commercial language models including LLaMA, Mistral, and ChatGPT APIs. Supports continuous regression testing, retrieval validation, and scalable experiment tracking for GenAI applications, processing millions of evaluation records in production.

• Redesigned the evaluation architecture from batch-based to event-driven using Kafka, decoupling retrieval from evaluation so scoring, human feedback, and monitoring could be built and scaled independently.
• Implemented hybrid BM25 + dense retrieval with cross-encoder re-ranking, improving retrieval relevance by ~20%.
• Built structure-aware document chunking (headings, tables, sections; 500–800 tokens) with document-type and client metadata for consistent RAG context quality.
• Designed Kafka-based event-driven multi-agent orchestration with Redis shared state for loosely coupled ingestion, extraction, retrieval, and drafting agents.
• Built a complementary evaluation strategy combining a gold dataset with an LLM-judge scorer and human escalation on low-confidence judgments.
• Implemented automated regression testing triggered by changes to prompts, embeddings, or retrieval logic; tracked experiments with MLflow and Weights & Biases.
• Deployed FastAPI services on AWS EKS with Docker, Kubernetes, and GitHub Actions CI/CD, reducing manual evaluation effort by ~40%.
• Processed millions of evaluation records using Apache Spark, PySpark, and Kafka.

Large-scale real-time recommendation and personalization platform delivering product and content recommendations across web and mobile applications for millions of users. End-to-end ML pipelines, real-time streaming infrastructure, and scalable inference services drove measurable improvements in user engagement, recommendation relevance, and business conversion.

• Designed and implemented a real-time recommendation system serving millions of users, covering collaborative filtering, content-based, hybrid recommendation, and ranking models using XGBoost and LightGBM — increasing click-through rate by ~25% and conversion by 18%.
• Built scalable end-to-end ML pipelines using Python, PySpark, and Apache Spark to process clickstream, browsing, search, and transaction datasets.
• Built low-latency recommendation services using Kafka and Spark Structured Streaming for real-time inference.
• Designed feature engineering pipelines with Apache Airflow supporting both offline model training and online prediction services.
• Applied transformer-based NLP models (BERT, RoBERTa, DistilBERT) to improve semantic understanding of search queries, product descriptions, and customer reviews.
• Deployed scalable inference APIs using FastAPI, gRPC, Docker, Kubernetes, and AWS cloud infrastructure.
• Implemented CI/CD pipelines with GitHub Actions and Jenkins for automated model deployment and service releases.

MLOps and model quality platform automating machine learning model training, validation, deployment, monitoring, and performance analysis. Integrated data quality checks, experiment tracking, explainability tooling, A/B testing, and CI/CD workflows to improve model reliability and streamline production ML operations across large-scale pipelines.

• Built ML operations workflows supporting automated model training, validation, deployment, and monitoring.
• Implemented CI/CD pipelines using GitHub Actions and Jenkins for automated model and service releases.
• Performed A/B testing and tracked experiments using MLflow to evaluate model performance across production iterations.
• Implemented SHAP and LIME for model explainability and feature-level analysis.
• Used Great Expectations to validate data quality and identify issues in large-scale ML pipelines.
• Optimized Spark and PySpark workloads to improve pipeline performance and reduce cloud infrastructure costs.

NLP and semantic search platform improving search understanding and product recommendation relevance across large-scale e-commerce applications. Leveraged transformer-based models to understand customer queries, product information, and reviews, enabling accurate semantic retrieval, ranking, and personalized recommendations.

• Applied transformer-based BERT, RoBERTa, and DistilBERT models to improve semantic understanding of customer search queries, product descriptions, and customer reviews.
• Rebuilt the core retrieval pipeline from pure semantic search to hybrid BM25 + query rewriting and normalization after real usage revealed users submitted fragmented, keyword-poor queries rather than clean natural-language inputs.
• Developed recommendation explanation pipelines using SHAP-based feature attribution to improve user trust and search relevance transparency.
• Built collaborative filtering, content-based, and hybrid ranking models using XGBoost and LightGBM, increasing search-driven CTR by ~25%.
• Designed feature engineering pipelines with Apache Airflow for offline model training and online prediction services.
• Deployed inference APIs using FastAPI, gRPC, Docker, Kubernetes, and AWS.