Onkar
From Sweden (UTC+2)
Lemon.io stats
1
offers now 🔥Onkar – Airflow, Databricks, Python
Onkar is a senior data engineer with 12 years in software and 9+ years in data and platform engineering across retail, banking, insurance, and consumer comparison platforms. His core stack is GCP (BigQuery, Pub/Sub, Dataflow, Composer) and Azure Databricks, with dbt, Airflow, Kafka, Spark, and Python. As the founding data engineer of a global retailer’s order-lifecycle team, he replaced a nightly Oracle warehouse with an event-driven platform handling about 25 million order events a day across 36 countries. Onkar owns architecture end-to-end, from medallion design and orchestration to GDPR and data-retention controls, and brings mature CI/CD habits to every pipeline.
12 years of commercial experience in
Main technologies
Additional skills
Direct hire
PossibleReady to get matched with vetted developers fast?
Let’s get started today!Experience Highlights
Senior Data Engineer
A Swiss consumer comparison platform that helps people compare and choose insurance, financial products, telecom plans, cars, and real estate. It runs on a central data platform that brings real-time and batch data from product and ad-tech sources into a cloud data warehouse. Curated data layers feed business dashboards as well as newer AI and agent-based workflows.
- Build Airflow workflows that move real-time and batch workloads into the BigQuery raw layer, where incremental dbt models pick up the latest changes and build the silver and gold layers used by BI and AI workflows.
- Built a new source integration into the central data warehouse.
- Migrated the company's ad-tech platforms onto the central data platform.
- Built agentic AI solutions on top of the platform using MCP servers, RAG, and vector databases.
- Worked AI-first with Claude Code connected through MCP servers to GitHub, Atlassian, Databricks Unity Catalog, and BigQuery, planning designs before coding.
Senior Data Engineer
A lakehouse data platform for the range and product domain of a global home-furnishing retailer, giving business teams and analysts one trusted source for BI and advanced analytics. It combines real-time product and range data streamed from Kafka with batch sources and models it into curated datasets for dashboards and data science across markets.
- Built the Azure Databricks lakehouse for the Range domain, with Spark Structured Streaming from Kafka and Delta Live Tables feeding bronze, silver, and gold layers for BI and advanced analytics.
- Implemented idempotent MERGE into the silver layer to deduplicate at-least-once events, and scheduled Delta OPTIMIZE and VACUUM on high-volume streaming tables on top of Databricks' own maintenance.
- Model data in Postgres and Databricks Unity Catalog and build SQL workflows with Lakeflow and dbt.
- Maintain CI/CD with tests and coverage on every commit and dbt tests in the pipeline, with orchestration in Airflow and Azure Data Factory.
Founding Data Engineer
An event-driven order-lifecycle data platform for a global home-furnishing retailer, replacing a nightly-loaded on-premises Oracle decision-support warehouse that left the business working with stale data. Source microservices emit an event on every order change, which flows through streaming ingestion into a cloud data warehouse and on to curated data products and Power BI dashboards for fulfillment and global business decisions. The platform handles about 25 million order events per day across 36 countries, with data stored in a 25-30 TB warehouse.
- As the founding data engineer of the order-lifecycle team, built the first data product end-to-end on GCP with a Data Mesh approach, replacing a nightly Oracle warehouse with fresh, event-driven data at about 25 million order events per day across 36 countries.
- Designed ingestion through Pub/Sub, Dataflow (Apache Beam) and Cloud Run into BigQuery, with dbt and Dataform building the silver and gold layers: bronze kept as strings to survive locale issues, and silver deduplicating at-least-once events with SCD history.
- Ran the new pipeline in parallel with the legacy warehouse before moving historical data over, avoiding a risky big-bang switch.
- Orchestrated pipelines in Cloud Composer (Airflow) with one DAG per source, sensors so hourly jobs wait for nightly loads, and dbt incremental models triggered by Airflow.
- Built data lifecycle and compliance controls for the 25-30 TB warehouse: Terraform-managed partitioning with 5-year expiration, GCS storage-class tiering, and a GDPR DAG that masks PII across all layers.
- Set up the team's first CI/CD (GitHub Actions with tests and coverage on every commit) and data-sharing guidelines, so separate companies in the group share data only through curated gold layers.
Lead Data Engineer
A digital transformation program for the order-fulfillment area of a global home-furnishing retailer, moving on-premises data and streaming sources to the cloud. It provides ingestion pipelines, large-scale data warehouses, and data models that power reporting and dashboards for fulfillment teams worldwide.
- Led data engineering as a consultant for the order-fulfillment digital transformation, migrating on-premises data and streaming sources to Google Cloud with Dataflow (Apache Beam) and Pub/Sub ETL jobs.
- Designed and built data ingestion pipelines and data models for large-scale BigQuery and Cloud Spanner warehouses.
- Built data models and visualizations in Data Studio and Power BI for fulfillment stakeholders.
- Worked with microservice teams on big data processing in Python and Java, using dbt for SQL transformations.
Senior Data Engineer
A private-banking data platform for a global bank, modernizing how client and transaction data is processed and served to internal applications. It ingests bank transactions in real time and in batch from on-premises Hadoop clusters and cloud sources, stores them in scalable data repositories, and serves them to UIs and APIs through an in-memory data layer. It also includes a knowledge graph that links banking entities for analysis.
- Built real-time bank transaction processing with Kafka and Spark Structured Streaming in Scala as part of modernizing the private-banking data platform.
- Designed the data repository in Apache HBase and supported front-end and back-end developers serving data to UIs and APIs through GemFire (Apache Geode).
- Built batch and streaming pipelines on an on-premises Cloudera Hadoop cluster with Hive and Sqoop, alongside AWS and Google Cloud sources.
- Built a bank knowledge graph database on Neo4j.
Middle Data Platform Engineer
A cloud data lake for US insurance companies, replacing scattered legacy data stores with one platform for analytics and BI. It ingests batch and real-time data, transforms it via ETL pipelines, and exposes it for ad hoc SQL and business data marts.
- Built a data lake on AWS S3 and Hortonworks Data Platform, using Apache NiFi for ETL and Apache Hive for ad hoc SQL and BI.
- Built real-time ingestion with Amazon Kinesis and Spark Streaming (PySpark).
- Built Hive data marts for business reporting and analytics.
- Delivered across AWS and Google Cloud in a startup-like consulting environment, working with Spark, Hadoop/MapReduce, Java, and Python.
Junior Back-end Developer
A business intelligence and data-migration product for the Product Lifecycle Management (PLM) space, used by engineering, aerospace, and defense manufacturers. It connects to enterprise PLM systems, enabling customers to visualize their engineering data and migrate it between platforms without custom one-off projects.
- Built Java/J2EE connectors and adapters for Siemens Teamcenter and PTC Windchill, letting enterprise customers such as General Electric and General Dynamics Mission Systems visualize and migrate their PLM data.
- Worked on the company's proprietary BI and data-migration engine across business intelligence, data migration, and integration projects on Oracle and PostgreSQL.
- Ran requirements analysis and designed modules using object-oriented modeling (OOMD), then implemented them in Java.
- Delivered customized solutions for clients, handled client-side queries, and worked with QA to resolve issues during release cycles.