Marcello – Python, LLM, AI agent development

Developed an AI assistant for a legal document editing company to support grammar and content review, enhancing editorial efficiency and enabling teams to focus on more complex, high-value tasks.

• Designed and implemented a validation pipeline, establishing accuracy benchmarks as a baseline for continuous model evaluation and improvement;
• Developed and refined prompt engineering strategies to enhance response quality and consistency;
• Built and maintained the request processing pipeline, ensuring reliable and efficient handling of AI-driven tasks;
• Optimized model outputs through iterative tuning, evaluation, and AI benchmarking techniques;
• Integrated external services via APIs, including the OpenAI API, to enable seamless AI functionality;
• Developed AI-driven solutions using LangChain, focusing on scalable AI agent development in Python.

LLM-powered extractor agent for converting diverse document formats into structured data, with continuous benchmarking to ensure reliability.

• Built and integrated an LLM-powered extractor agent (Claude 3.5) into new and existing systems;
• Developed pipelines for extracting data from diverse document formats (TXT, Excel, text PDFs, image PDFs) and converting them into structured outputs for database ingestion;
• Implemented input data normalization workflows (PDFs, images, text → JSON) to ensure consistent processing;
• Designed and maintained a validation test bench to monitor performance and prevent regression in extraction accuracy;
• Conducted LLM model validation using benchmark datasets and defined evaluation metrics;
• Applied prompt engineering techniques to improve extraction quality and reliability;
• Implemented tool calling mechanisms to enhance agent capabilities and workflow orchestration;
• Integrated external services and systems via API integrations.

A chatbot assistant for a fantasy sports app powered by LLMs, designed with RAG, memory management, and vector databases to improve contextual understanding and user interaction.

• Built and integrated an LLM-powered chatbot assistant (GPT-3.5 Turbo) into a fantasy sports application using LangChain;
• Designed and evaluated different agent architectures, including prompt-based agents, tool calling, and OpenAI tools;
• Implemented agent memory management to support context-aware and multi-turn interactions;
• Developed and validated RAG (Retrieval-Augmented Generation) pipelines using vector store databases;
• Conducted experiments to compare approaches and optimize chatbot performance and reliability;
• Designed and optimized data storage structures to improve retrieval efficiency and overall agent performance;
• Performed stress testing and validation to ensure robustness and scalability of the chatbot system.

Built human-like game-playing agents with competitive performance using reinforcement learning and neural network optimization for a major U.S. game development company.

• Developed and maintained AI agents capable of human-like gameplay at a high skill level using Deep Reinforcement Learning;
• Designed and optimized deep learning models (neural network architectures) using PyTorch and TensorFlow;
• Implemented and trained machine learning pipelines in Python, focusing on agent performance and stability;
• Applied machine learning and data science techniques to evaluate and improve gameplay behavior;
• Conducted experiments and performance analysis to refine models and training strategies;
• Collaborated with game design teams, providing simulation results and insights to support data-driven decisions.