Logo
Juan – Databricks, Python, GCP, experts in Lemon.io

Juan

From Colombia (GMT-5)

flag
Data EngineerSenior
Hire developer
7 years of commercial experience
Banking
Cloud computing
Consulting services
Consumer goods
Data analytics
Gambling
Retail
Lemon.io stats

Juan – Databricks, Python, GCP

Juan David Barreto is a focused and passionate individual who transitioned from Software Engineering to Data Engineering, showcasing adeptness in implementing complex solutions. With prior experience as a Lead Data Engineer, he seamlessly blends people management with technical expertise. His proficiency in SQL and Python, coupled with project management abilities, denotes senior-level expertise and suitability for challenging roles.

Main technologies
Databricks
3 years
Python
7 years
GCP
3 years
SQL
7 years
Additional skills
Big Data
Apache Spark
Data Modeling
Data Warehouse
BigQuery
Apache Airflow
API
PySpark
ETL
AWS
MySQL
Ready to start
ASAP
Direct hire
Potentially possible

Experience Highlights

Data Architect
Dec 2022 - Jan 20241 year 1 month
Project Overview

The project was developed for the biggest hard discount retail company in Colombia. The main challenge was to utilize Lakehouse, unify all the company's data in a single source of truth, and replace a legacy on-premises data warehouse that processed more than 200 million transactions daily.

Skeleton
Skeleton
Skeleton
Responsibilities:

Juan's successes include the following achievements:

  • Designed the architecture for the whole solution;
  • Implemented the data ingestion processes from the different data sources;
  • Build the data pipelines to transform the data;
  • Modeled the data warehouse layer;
  • Data Governance.
Project Tech stack:
Databricks
Big Data
AWS
PySpark
Apache Airflow
Amazon S3
Senior Data Engineer
Dec 2022 - Oct 202310 months
Project Overview

The project was developed for a popular pet e-commerce in Colombia. The main idea was to unify the company's data in a single source of truth, allowing the company to have a curated repository ready to use by BI tools in almost real-time.

This solution helped the company to democratize their data, allowing all the different business areas to access its data without any intermediates.

Skeleton
Skeleton
Skeleton
Responsibilities:

Juan achieved the following:

  • Designed the architecture for the whole solution;
  • Implemented the data ingestion processes from the different data sources;
  • Build the data pipelines to transform the data;
  • Modeled the data warehouse layer;
  • Helped create several dashboards to track important KPIs;
  • Data Governance.
Project Tech stack:
Databricks
AWS
Cloud Computing
Cloud Architecture
PySpark
Apache Spark
Amazon S3
Hive
Senior Data Engineer
Dec 2021 - Jun 20225 months
Project Overview

This project intended automating the creation of different reports that tracked the performance KPIS from the slot machines deployed in several casinos across the USA. All the data was ingested into BigQuery, allowing the team to automate the different transformations needed for the reports, which allowed the company to have a single source of truth and replace man-made Excel files with dashboards in Tableau.

Skeleton
Skeleton
Skeleton
Responsibilities:

Among others, Juan managed such responsibilities as:

  • Developed several data pipelines using BigQuery Stored procedures;
  • Migrated all the business logic from Excel files to SQL queries;
  • Helped build different dashboards tracking important KPIs.
Project Tech stack:
BigQuery
SQL
Python
Cloud Computing
Tableau
Senior Data Engineer
Dec 2020 - Oct 202110 months
Project Overview

Migration of the historical data from COBOL-indexed files to the Hadoop ecosystem for the largest credit bureau in the country. Additionally, reimplementation of all the business rules in Apache Spark and Scala.

Skeleton
Skeleton
Skeleton
Responsibilities:

Juan has successfully implemented the following tasks:

  • Ingested all the historical data to the company HDFS;
  • Recreated the business rules using Apche Spark;
  • Deployed the new pipelines to a production cluster.
Project Tech stack:
Scala
Apache Spark
Apache Hadoop

Education

2016
Mechatronics
B.S in Mechatronics Engineering
2022
Master in Data Analytics
M.S in Data Analytics
2024
Databricks Certified Data Engineer Associate
Databricks Certified Data Engineer Associate

Copyright © 2024 lemon.io. All rights reserved.