Ayodele – Python, OpenAI, PyTorch

Frontier AI labs need reliable human evaluation to fine-tune models and validate their safety. Ayodele owns end-to-end evaluation workflows that catch failures automated metrics miss, feeding training data used in enterprise agent deployments.

• Owned end-to-end RLHF evaluation workflows for frontier model fine-tuning, authoring structured quality assessments across accuracy, instruction-following, and safety dimensions;
• Identified failure modes in agentic and multi-turn AI flows that automated metrics missed, directly shaping training data used in enterprise AI agent deployment cycles;
• Built contextual evaluation frameworks and guardrail test suites covering output constraint enforcement, safety filtering, and policy compliance;
• Caught behavioral regressions across fine-tuning checkpoints before production release, reducing escaped defects and establishing evaluation best practices adopted across the team.

An enterprise labor management platform for a Fortune 50 retailer, piloted at a distribution center and designed for phased rollout across the DC network. It gives operations leaders real-time visibility into team member productivity and performance across inbound, outbound, warehousing, and packing workflows.

• Built event-driven data pipelines processing millions of operational transactions daily across ingest, enrichment, and snapshot workflows on Kotlin/Kafka;
• Developed production observability dashboards in Grafana to track pipeline health across ingestion throughput, processing latency, consumer lag, and error/retry rates;
• Implemented enrichment and snapshot-publishing pipelines feeding downstream productivity and performance metrics for distribution center operations across inbound, outbound, warehousing, and packing workflows;
• Built retry and dead-letter-queue handling for fault-tolerant event processing at scale;
• Worked within internal Kubernetes platform to deploy and operate services supporting a phased pilot-to-scale rollout.

• Built and productionized agentic services with a durable execution engine supporting retries (5 max attempts), timeouts, deduplication, fan-out, and deterministic replay.
• Orchestrated 100+ concurrent agent workflow runs with PostgreSQL persistence and 16 state transitions per run.

• Designed control plane UI for monitoring AI agent conversations: event history (26+ events per run), execution timeline, pending tasks, call stack, worker management, cancellation, and versioning.
• Implemented MCP integration for agent tool routing. Built RAG-powered assistant with inline source citations. Online endpoints with autoscaling and rollback strategies.

A fraud detection system built on a large-scale transaction dataset with over 6.3 million records, designed to hold up in production rather than score well only on paper. It focuses on reliable fraud modeling, leakage-free validation, and serving workflows that support multi-tenant deployment.

• Trained an initial fraud classifier (XGBoost/PyTorch ensemble) that scored a near-perfect 0.998 ROC AUC, then identified it as a red flag rather than a win;
• Diagnosed the cause: a feature encoding post-transaction account balances was leaking the label, a synthetic-data artifact that does not exist in real banking systems;
• Rebuilt the pipeline with a leakage-free feature set, including transaction velocity, new-account risk, and behavioral patterns, plus a time-based train/test split instead of random splitting;
• The corrected model's PR AUC dropped to 0.64, a deliberately more honest number reflecting a model trained on signals a real fraud investigator would trust;
• Built FastAPI/Uvicorn multi-tenant serving with per-tenant threshold tuning and observability, using PyTorch/Hugging Face with Accelerate for data-parallel training.

A community platform needed users to find relevant content and get personalised answers across a large volume of user-generated posts. Ayodele built and deployed a LangGraph AI agent with semantic search, integrated into the platform's production backend.

• Implemented a LangGraph-powered AI agent with autonomous tool use, multi-turn reasoning, and structured decision graphs in Python;
• Integrated the agent into a production Node.js/Express backend with PostgreSQL persistence;
• Developed a vector-embedding semantic search layer using pgvector and cosine similarity indexing for contextual retrieval across user-generated content;
• Combined AI agent workflows with retrieval-augmented generation to deliver personalized experiences at scale;
• Engineered REST APIs and a GraphQL endpoint in Node.js with PostgreSQL, reducing data over-fetching by 30%;
• Containerized the full stack with Docker and automated CI/CD via GitHub Actions with test suites;
• Collaborated with cross-functional stakeholders on requirements and deployment.

Devices and AI org building ML-powered discovery and semantic search systems across multiple consumer platforms, serving 24M+ users.

• Owned productionization of ML semantic search model outputs into scalable retrieval workflows across iOS, Android, and web.
• Partnered with backend engineers on API design and data modeling for ML-driven discovery features.
• Built Redshift SQL dashboards and QuickSight visualizations instrumenting ML feature event pipelines across platform surfaces.
• Identified and resolved cross-functional infrastructure blockers, escalating to Principal Engineers where needed to accelerate release velocity.