Onkar – Airflow, Databricks, Python, experts in Lemon.io

Onkar

From Sweden (UTC+2)flag

Data Engineer|Senior
Lemon.io stats
1
offers now 🔥

Onkar – Airflow, Databricks, Python

Onkar is a senior data engineer with 12 years in software and 9+ years in data and platform engineering across retail, banking, insurance, and consumer comparison platforms. His core stack is GCP (BigQuery, Pub/Sub, Dataflow, Composer) and Azure Databricks, with dbt, Airflow, Kafka, Spark, and Python. As the founding data engineer of a global retailer’s order-lifecycle team, he replaced a nightly Oracle warehouse with an event-driven platform handling about 25 million order events a day across 36 countries. Onkar owns architecture end-to-end, from medallion design and orchestration to GDPR and data-retention controls, and brings mature CI/CD habits to every pipeline.

12 years of commercial experience in
Aerospace
Analytics
Banking
Data analytics
Defense
E-commerce
Fintech
Insurance
Manufacturing
Retail
Supply chain
AI software
Main technologies
Airflow
5 years
Databricks
3 years
Python
9 years
SQL
9 years
PySpark
8 years
DBT
3 years
GCP
9 years
Azure Data Factory
3 years
Azure Data Lake
3 years
Additional skills
Microsoft Azure
CI/CD
Apache Kafka
Apache Spark
Apache Hadoop
AWS
ETL
MySQL
Kubernetes
Java
BigQuery
Data Modeling
Data Warehouse
AI agent development
Terraform
Neo4j
Direct hire
Possible
Ready to get matched with vetted developers fast?
Let’s get started today!

Experience Highlights

Senior Data Engineer
Aug 2026 - Ongoing2 months
Project Overview

A Swiss consumer comparison platform that helps people compare and choose insurance, financial products, telecom plans, cars, and real estate. It runs on a central data platform that brings real-time and batch data from product and ad-tech sources into a cloud data warehouse. Curated data layers feed business dashboards as well as newer AI and agent-based workflows.

Responsibilities:
  • Build Airflow workflows that move real-time and batch workloads into the BigQuery raw layer, where incremental dbt models pick up the latest changes and build the silver and gold layers used by BI and AI workflows.
  • Built a new source integration into the central data warehouse.
  • Migrated the company's ad-tech platforms onto the central data platform.
  • Built agentic AI solutions on top of the platform using MCP servers, RAG, and vector databases.
  • Worked AI-first with Claude Code connected through MCP servers to GitHub, Atlassian, Databricks Unity Catalog, and BigQuery, planning designs before coding.
Project Tech stack:
GCP
BigQuery
Airflow
DBT
Python
AI agent development
SQL
Databricks
Senior Data Engineer
May 2025 - Ongoing1 year 5 months
Project Overview

A lakehouse data platform for the range and product domain of a global home-furnishing retailer, giving business teams and analysts one trusted source for BI and advanced analytics. It combines real-time product and range data streamed from Kafka with batch sources and models it into curated datasets for dashboards and data science across markets.

Responsibilities:
  • Built the Azure Databricks lakehouse for the Range domain, with Spark Structured Streaming from Kafka and Delta Live Tables feeding bronze, silver, and gold layers for BI and advanced analytics.
  • Implemented idempotent MERGE into the silver layer to deduplicate at-least-once events, and scheduled Delta OPTIMIZE and VACUUM on high-volume streaming tables on top of Databricks' own maintenance.
  • Model data in Postgres and Databricks Unity Catalog and build SQL workflows with Lakeflow and dbt.
  • Maintain CI/CD with tests and coverage on every commit and dbt tests in the pipeline, with orchestration in Airflow and Azure Data Factory.
Project Tech stack:
Microsoft Azure
Databricks
PySpark
SQL
Python
DBT
Apache Kafka
Airflow
Azure Data Factory
Founding Data Engineer
Jul 2024 - Jun 202510 months
Project Overview

An event-driven order-lifecycle data platform for a global home-furnishing retailer, replacing a nightly-loaded on-premises Oracle decision-support warehouse that left the business working with stale data. Source microservices emit an event on every order change, which flows through streaming ingestion into a cloud data warehouse and on to curated data products and Power BI dashboards for fulfillment and global business decisions. The platform handles about 25 million order events per day across 36 countries, with data stored in a 25-30 TB warehouse.

Responsibilities:
  • As the founding data engineer of the order-lifecycle team, built the first data product end-to-end on GCP with a Data Mesh approach, replacing a nightly Oracle warehouse with fresh, event-driven data at about 25 million order events per day across 36 countries.
  • Designed ingestion through Pub/Sub, Dataflow (Apache Beam) and Cloud Run into BigQuery, with dbt and Dataform building the silver and gold layers: bronze kept as strings to survive locale issues, and silver deduplicating at-least-once events with SCD history.
  • Ran the new pipeline in parallel with the legacy warehouse before moving historical data over, avoiding a risky big-bang switch.
  • Orchestrated pipelines in Cloud Composer (Airflow) with one DAG per source, sensors so hourly jobs wait for nightly loads, and dbt incremental models triggered by Airflow.
  • Built data lifecycle and compliance controls for the 25-30 TB warehouse: Terraform-managed partitioning with 5-year expiration, GCS storage-class tiering, and a GDPR DAG that masks PII across all layers.
  • Set up the team's first CI/CD (GitHub Actions with tests and coverage on every commit) and data-sharing guidelines, so separate companies in the group share data only through curated gold layers.
Project Tech stack:
Python
GCP
BigQuery
Apache Kafka
SQL
Data Warehouse
Data Modeling
DBT
Airflow
Terraform
Lead Data Engineer
Nov 2019 - Jul 20244 years 7 months
Project Overview

A digital transformation program for the order-fulfillment area of a global home-furnishing retailer, moving on-premises data and streaming sources to the cloud. It provides ingestion pipelines, large-scale data warehouses, and data models that power reporting and dashboards for fulfillment teams worldwide.

Responsibilities:
  • Led data engineering as a consultant for the order-fulfillment digital transformation, migrating on-premises data and streaming sources to Google Cloud with Dataflow (Apache Beam) and Pub/Sub ETL jobs.
  • Designed and built data ingestion pipelines and data models for large-scale BigQuery and Cloud Spanner warehouses.
  • Built data models and visualizations in Data Studio and Power BI for fulfillment stakeholders.
  • Worked with microservice teams on big data processing in Python and Java, using dbt for SQL transformations.
Project Tech stack:
BigQuery
GCP
Python
Java
DBT
PowerBI
Senior Data Engineer
May 2018 - Oct 20191 year 5 months
Project Overview

A private-banking data platform for a global bank, modernizing how client and transaction data is processed and served to internal applications. It ingests bank transactions in real time and in batch from on-premises Hadoop clusters and cloud sources, stores them in scalable data repositories, and serves them to UIs and APIs through an in-memory data layer. It also includes a knowledge graph that links banking entities for analysis.

Responsibilities:
  • Built real-time bank transaction processing with Kafka and Spark Structured Streaming in Scala as part of modernizing the private-banking data platform.
  • Designed the data repository in Apache HBase and supported front-end and back-end developers serving data to UIs and APIs through GemFire (Apache Geode).
  • Built batch and streaming pipelines on an on-premises Cloudera Hadoop cluster with Hive and Sqoop, alongside AWS and Google Cloud sources.
  • Built a bank knowledge graph database on Neo4j.
Project Tech stack:
AWS
Apache Hadoop
Apache Kafka
Apache Spark
Scala
Neo4j
Hive
Middle Data Platform Engineer
Mar 2017 - May 20181 year 2 months
Project Overview

A cloud data lake for US insurance companies, replacing scattered legacy data stores with one platform for analytics and BI. It ingests batch and real-time data, transforms it via ETL pipelines, and exposes it for ad hoc SQL and business data marts.

Responsibilities:
  • Built a data lake on AWS S3 and Hortonworks Data Platform, using Apache NiFi for ETL and Apache Hive for ad hoc SQL and BI.
  • Built real-time ingestion with Amazon Kinesis and Spark Streaming (PySpark).
  • Built Hive data marts for business reporting and analytics.
  • Delivered across AWS and Google Cloud in a startup-like consulting environment, working with Spark, Hadoop/MapReduce, Java, and Python.
Project Tech stack:
AWS
Apache Hadoop
Apache Spark
Java
Python
ETL
Hive
Apache Kafka
Junior Back-end Developer
Jun 2014 - Mar 20172 years 9 months
Project Overview

A business intelligence and data-migration product for the Product Lifecycle Management (PLM) space, used by engineering, aerospace, and defense manufacturers. It connects to enterprise PLM systems, enabling customers to visualize their engineering data and migrate it between platforms without custom one-off projects.

Responsibilities:
  • Built Java/J2EE connectors and adapters for Siemens Teamcenter and PTC Windchill, letting enterprise customers such as General Electric and General Dynamics Mission Systems visualize and migrate their PLM data.
  • Worked on the company's proprietary BI and data-migration engine across business intelligence, data migration, and integration projects on Oracle and PostgreSQL.
  • Ran requirements analysis and designed modules using object-oriented modeling (OOMD), then implemented them in Java.
  • Delivered customized solutions for clients, handled client-side queries, and worked with QA to resolve issues during release cycles.
Project Tech stack:
Java
SQL

Education

2014
Computer Engineering
Bachelor's
2026
Databricks Certified Data Engineer Associate
Certificate
2024
Google Cloud Certified Data Engineer Professional
Certificate

Languages

Swedish
Intermediate
Hindi
Advanced
English
Advanced

Hire Onkar or someone with similar qualifications in days
All developers are ready for interview and are are just waiting for your requestdream dev illustration
Copyright © 2026 lemon.io. All rights reserved.