Zafer – Apache Spark, SQL, Python

A high-throughput cybersecurity analytics platform processing approximately 50TB of data daily. It provides near real-time analytics capabilities for security operations. I led the data engineering architecture, owning the complete migration from legacy batch processing to a low-latency streaming infrastructure.

• Orchestrated the shift from a legacy batch ETL system to a high-throughput near real-time processing infrastructure.
• Achieved and maintained a 15-minute latency SLO for about 50TB/day of cybersecurity analytics data.
• Spearheaded technical coordination with internal consumers and dependent parties.
• Implemented platform engineering standards using Infrastructure-as-Code with Terraform.
• Designed container orchestration workflows with Docker and Kubernetes.

A scalable data discovery and lakehouse search platform providing high-performance queryable data services for internal consumers. I acted as the lead engineer and domain expert, owning the infrastructure architecture from the S3 data layer to the containerized deployment workflows.

• Engineered a scalable data lake search platform utilizing Trino over AWS S3.
• Spearheaded technical coordination with internal consumers and dependent parties.
• Implemented platform engineering standards using Infrastructure-as-Code with Terraform.

An enterprise data modernization initiative migrating on-premise legacy systems to a scalable cloud architecture. It decouples and transfers datasets to resolve structural scalability limits.

• Executed a complete lift-and-shift of an on-premise legacy ETL system (Hadoop, Teradata, PL/SQL, in-house orchestration) to the cloud.
• Engineered the migration of large-scale data pipelines from legacy Teradata-backed Hadoop clusters.
• Operated PySpark workloads directly on YARN.
• Orchestrated the decoupling and transfer of on-premise datasets to AWS S3 and Snowflake using Apache Airflow.

An enterprise-wide metadata management platform that establishes end-to-end data lineage across the organization's data assets. It serves internal data teams to track data provenance and discovery.

• Developed an in-house data catalog project using Java from a forked Apache Atlas repository.
• Deployed and configured the Apache Atlas infrastructure.
• Implemented functionalities to manage data dictionaries, flag PII, and categorize sensitive data.

A data privacy and regulatory compliance project aimed at identifying, anonymizing, and removing Personally Identifiable Information (PII) from legacy platforms.

• Contributed to GDPR compliance efforts by migrating and anonymizing sensitive user records within the Hadoop ecosystem.
• Applied GDPR practices for the anonymization of legacy data according to C1/C2 data privacy classifications.
• Designed specific data models for handling and structuring sensitive data securely.

Enterprise data warehouse and business intelligence implementations for banking and insurance clients.

• Directed end-to-end Enterprise Data Warehouse and Business Intelligence implementations for banking and insurance clients.
• Architected Data Marts in adherence to Inmon and Kimball dimensional modeling methodologies.
• Optimized mission-critical EDW refresh processes through query tuning and ETL redesign, reducing nightly load durations from 13 hours to 7 hours.
• Led the technical and communication parts of the projects to ensure that deliveries are valid and on track.