Eric – BigQuery, Apache Spark, Snowflake

• Client & Stakeholders: Eric's shipped product with real paying subscribers, so he was the product owner, engineer, and support.
• Business Problem & Ambiguity: A consumer app calling a vision model on every scan has an unbounded cost surface, where one runaway user or one bug breaks the economics. Recognition accuracy had to be proven, not assumed.
• Solution Built: The entire product solo. React Native and TypeScript client, Node API, Postgres, authentication, payments, and App Store release. AI vision recognition measured against a benchmark harness Eric wrote, with a second recognition lane behind a feature flag for input types the first lane handled poorly. Cost defended in layers through per-tier quotas, deduplication caching, and daily spend breakers.
• Measurable Outcome: Live on the App Store with 98.3 percent top-1 recognition accuracy and per-scan cost held under a cent.

• Designed, built, and shipped the entire product solo, from the React Native and TypeScript client through the Node API, Postgres, authentication, payments, and App Store release;
• Measured AI vision recognition against a benchmark harness Eric wrote rather than assuming accuracy, reaching 98.3 percent top-1;
• Added a second recognition lane behind a feature flag for the input types the first lane handled poorly;
• Defended an unbounded per-scan cost surface in layers through per-tier quotas, deduplication caching, and daily spend breakers, holding per-scan cost under a cent;
• Shipped subscription billing and ran the product as owner, engineer, and support for real paying subscribers.

• Client & Stakeholders: The ad trading desk using the platform daily, plus the engineering organization.
• Business Problem & Ambiguity: Working out which audiences a campaign should target took traders hours of manual research, sometimes spread over days. Internal AI processes were only accessible to engineers. Exposing them through an agent raised open questions about security, data access, and answer quality that had no template to follow.
• Solution Built: A chatbot for traders built on MCP servers we wrote in house, so a trader describes a campaign in plain language and the agent executes the matching process. Around it, a full agent harness with QA loops that verify answers, access control and prompt injection defense, and self-healing context repositories built from company data through vectorization and similarity scoring. Extended the same approach into the development lifecycle with context repositories that refresh through CI/CD, indexed so coding agents find the right context without wasting tokens.
• Measurable Outcome: Campaign targeting research went from hours or days to minutes. Engineering teams work from continuously current context instead of stale documentation, with managed token costs.

• Built a trader-facing chatbot on MCP servers written in house, so a trader describes a campaign in plain language and the agent executes the audience matching process;
• Built the agent harness around it, including QA loops that verify answers before they reach the trader;
• Implemented access control and prompt injection defense for an agent exposed to a non-engineering audience;
• Built self-healing context repositories from company data using vectorization and similarity scoring, so the agent answers from current context rather than stale documentation;
• Extended the same pattern into the development lifecycle with context repositories that refresh through CI/CD, indexed so coding agents reach the right context without burning tokens;
• Cut campaign targeting research from hours or days of manual work down to minutes.

A daily ad-analytics data product that ingests about 1 TB per day of ad attribution and campaign data from a third-party measurement vendor and turns it into canonical analytical datasets powering a partner analytics team's executive dashboards. Eric led the data engineering side of the partnership end to end, from ingestion architecture and data modeling through metric definitions and delivery.

• Architected the multi-stage daily ingestion flow (vendor S3 drops into GCS, converted to Parquet, exposed as BigQuery external tables) normalizing irregular CSV/TSV vendor exports into clean data models;
• Authored dbt-based production data models and complex multi-hundred-line SQL transformations turning ambiguous stakeholder requirements into reliable analytical surfaces;
• Owned column-level decisions, metric definitions, and data-integrity trade-offs across millions of daily attribution events;
• Built a 7-step Airflow (Cloud Composer) delivery DAG exporting curated datasets from BigQuery through GCS to partner S3 with row-count integrity validation as a delivery gate;
• Diagnosed and fixed a partner-reported impossible conversion funnel, a partition-key dedup bug spanning 9 SQL locations, and validated the fix against raw attribution data;
• Pioneered the team's Airflow adoption, architecting the integration layer to the centrally hosted deployment that 5+ services now extend;
• Established a configuration-driven BigQuery backup DAG using dynamic task mapping where protecting a new table is a config-only change.

A contextual advertising intelligence platform that analyzes the content of millions of publisher web pages to power privacy-safe ad targeting without cookies. The system crawls publisher URLs at internet scale, generates LLM embeddings for every page, scores them against an advertising taxonomy with cosine similarity, and serves the resulting segments to demand-side and supply-side ad platforms. Eric owned the end-to-end data pipeline, from crawl to embedding to the TB-scale analytical tables that partners consume.

• Designed and shipped a 5-stage event-driven Kafka pipeline of FastAPI microservices covering orchestration, site discovery, page fetching, HTML parsing, and content-mapping ETL, replacing a legacy cron and queue fan-out;
• Achieved 99%+ storage reduction (1.15 TB down to 9.88 GB) and 500x+ query speedup on a multi-TB BigQuery table via custom NTILE-based partitioning and nested STRUCT schemas;
• Parallelized embedding and cosine-similarity scoring over a 10-15 TB dataset using Dask distributed dataframes, with PySpark jobs for the heaviest re-scoring runs;
• Backfilled OpenAI embeddings for about 13.47 million URLs in a single coordinated operation with deduplicating MERGE logic;
• Authored a tiktoken pre-flight and adaptive-batch retry pattern for the OpenAI API plus SHA-256 content-hash caching, delivering 40%+ ETL speedup on re-runs;
• Operated a high-concurrency async web crawler (aiohttp) with robots.txt compliance and differentiated retry strategies for transient vs permanent errors;
• Built a shared Python utility library on an internal PyPI used by every service, covering Kafka wrappers with DLQ and back-pressure, BigQuery MERGE upsert helpers, and a Vault secrets client;
• Deployed all services on Kubernetes with Docker, and built GCP Monitoring and PagerDuty alerting from scratch with real-time log-based data-integrity alerts;
• Cut new-supply onboarding latency from 3+ weeks to a few days via parallelized async processing and Kafka retry/DLQ architecture.

• Client & Stakeholders: Ad tech platform operating as a startup unit inside a large media organization. Stakeholders were the executive leadership funding the AI systems and the data science team whose modeling approach needed to reach production.
• Business Problem & Ambiguity: The platform needed model-derived context for over 50 million pages of supply. The prototype cost roughly ten cents per page, about five million dollars per full run, and took around thirty days on a single machine. Supply grows continuously, so this was a structural blocker with no defined path to production. Deciding which parts of the workload actually needed model inference was itself the open question.
• Solution Built: Re-engineered the LLM and embedding process around clustering and contextual expansion, so the model runs only on the roughly fifty thousand pages that define matchable context and everything else inherits from them. Rebuilt processing as a staged Kafka pipeline. Layered reliability using OpenAI structured outputs for schema enforcement plus a low-cost judge model that catches the half percent of outputs that pass validation but are unusable, before they corrupt downstream matching.
• Measurable Outcome: Inference cost per full run went from about five million dollars to about five thousand, a 99.9 percent reduction. Runtime went from thirty days to under three.

• Re-engineered an LLM and embedding process around clustering and contextual expansion, restricting model inference to the roughly 50,000 pages that define matchable context while over 50 million pages inherit from them;
• Rebuilt the processing path as a staged Kafka pipeline, taking a full run from about thirty days on a single machine to under three;
• Enforced output schemas with OpenAI structured outputs and added a low-cost judge model that catches the roughly half percent of outputs which pass validation but are unusable, before they corrupt downstream matching;
• Partnered with the data science team to move their modeling approach from prototype into production, reporting progress to the executive leadership funding the work;
• Cut inference cost per full run from about five million dollars to about five thousand, a 99.9 percent reduction.

An enterprise FinOps and IT-service analytics platform unifying cloud spend across Azure, AWS, and GCP with ServiceNow ITSM operational data into one governed dimensional model. It gave 15+ business units tailored cost visibility and turned over $1M of cloud spend into tracked, remediated savings.

• Engineered multi-cloud (Azure, AWS, GCP) FinOps dimensional models in Power BI with row-level security, visualizing $1M+ cloud spend with per-resource cost-saving recommendations;
• Unified analytics across 10+ departments and 50K+ ITSM tickets through a centralized dimensional model orchestrated with Apache Airflow DAGs, cutting report duplication by 40%;
• Built an automated dashboard integrating ServiceNow ticket data to track remediation progress, visualizing $100K+ in captured savings;
• Defined and enforced cross-cloud tagging standards as the data-quality contract for cost attribution across 3 platforms;
• Built a Python-based data quality framework enforcing schema integrity, type validation, and null-rate thresholds across 10,000+ records feeding ML input pipelines;
• Applied SQL and Python text analytics over ITSM ticket exports to surface recurring root causes, improving visibility by 30%.

• Client & Stakeholders: Pharmaceutical companies, research labs, and universities, under NDA on names. Counterparts ranged from technical teams to non-technical executives who had to sign off before anything was built.
• Business Problem & Ambiguity: This was the first wave of enterprise AI adoption. Clients knew they wanted AI but not what was feasible, what it would cost, or where it fit their business. The offerings themselves did not exist yet and had to be defined, sold, and then delivered.
• Solution Built: As part of the firm's AI Guild, built out how clients would implement AI in their businesses, shaped the programs and offerings, sold them alongside the AI leads, and delivered the work we sold. Delivered production machine learning including similarity and matching models on healthcare claims data and location-based analysis on health data, deployed on Snowflake and Azure. Navigated security reviews, data access negotiation, and executive sign-off in regulated environments.
• Measurable Outcome: Production ML deployments running for regulated enterprise clients, and engagements sold and delivered that established AI offerings within the practice.

• Shaped how enterprise clients would implement AI in their businesses during the first wave of adoption, defining programs and offerings that did not exist yet;
• Sold engagements alongside the AI leads and delivered the work that was sold;
• Delivered production machine learning for pharmaceutical, research lab, and university clients, including similarity and matching models on healthcare claims data;
• Built location-based analysis on health data, deployed on Snowflake and Azure;
• Navigated security reviews, data access negotiation, and non-technical executive sign-off in regulated environments before any build started;
• Ran discovery directly with counterparts ranging from technical teams to executives with no AI background.

• Client & Stakeholders: A major shipping and logistics company. IT leadership and business operations stakeholders on the client side.
• Business Problem & Ambiguity: A legacy Oracle estate was limiting analytics and AI readiness. Migration scope across source systems was unclear at the start and had to be made concrete through discovery with the client's teams.
• Solution Built: Led the migration to Azure Data Factory pipelines feeding Azure SQL, modernizing legacy ETL into a platform ready for analytics and AI workloads.
• Measurable Outcome: Data latency reduced 30 to 50 percent, with the modernized platform in production.

• Led the migration of a legacy Oracle estate onto Azure Data Factory pipelines feeding Azure SQL;
• Ran discovery with the client's IT leadership and business operations teams to turn an unclear migration scope into a concrete plan across source systems;
• Modernized legacy ETL into a platform ready for analytics and AI workloads;
• Reduced data latency by 30 to 50 percent and left the modernized platform running in production.

A secure, multi-tenant analytics platform for a private equity investment firm, where positions, transactions, and portfolio data flowed from legacy systems into a modern Azure lakehouse serving OLAP analytics under SOC 2 requirements. Eric owned the transformation layer and the platform evaluation for the modernization.

• Built PySpark transformation jobs on Azure Databricks processing multi-GB datasets from Azure Data Lake into curated star-schema layers organized as Delta Lake bronze/silver/gold medallion tables, cutting nightly batch windows by roughly 60%;
• Tuned Databricks job clusters for skewed joins and partition pruning with broadcast hints, Z-ORDER, and autoscaling, and used Delta time travel to reproduce and debug bad loads;
• Led a Snowflake vs Microsoft Fabric platform evaluation, building proof-of-concept ingestion pipelines and authoring complex Snowflake SQL across window functions, VARIANT handling, and clustering keys to benchmark cost and performance;
• Modularized the dimensional transformation layer as dbt models with source-freshness and schema tests, deployed through Azure DevOps CI/CD;
• Designed star schemas with slowly-changing-dimension patterns across 150+ database fields, enforcing SOC 2-compliant tenant isolation;
• Architected Azure Data Factory ETL/ELT pipelines into Azure SQL DB and Data Lake, reducing data latency by 50% with idempotent, schema-evolution-safe design;
• Built infrastructure as code with ARM templates and Terraform modules deploying the full data stack across isolated dev, QA, and prod environments, covering Data Lake, dimensional SQL DBs, Cosmos DB, and Databricks workspaces.

The data backbone for a 1,000+ user, 20+ location enterprise migration program, consolidating user, machine, and workspace records from more than 20 source systems into a single validated dataset that drove scheduling, remediation, and leadership reporting.

• Built automated Python (pandas) and SQL ETL pipelines cleaning, validating, and consolidating 50K+ records from 20+ source systems, achieving 95% data readiness;
• Designed SQL Server staging schemas and SSIS-based ingestion flows with reusable unique-identifier systems and data-quality rules (dedup, referential checks, exception queues);
• Built Tableau dashboards over the consolidated dataset tracking engagement and readiness metrics, surfacing risks early across 7 deployment scenarios and 15+ scheduling holds;
• Led migration data strategy across 20+ site IT leads, standardizing collection templates and integrity practices program-wide.