Faisal – BigQuery, Python, SQL

A digital banking application providing financial services and account management for retail customers. The platform included data pipelines for synchronizing production databases, generating automated daily reports, and transferring data to external partners. The project involved end-to-end ownership of data orchestration and pipeline infrastructure, cloud-based deployments, ingestion workflows, backend services, and data usage and cost monitoring.

• Handled data pipelines and automation for internal and external uses;
• Automated pipelines with Prefect for daily email reports, external report handover, and daily data retrieval from production databases;
• Reported and analyzed BigQuery cost usage;
• Designed and implemented the Prefect architecture in GCP using Terraform;
• Implemented scheduled automations for data reporting and pipelines;
• Fixed issues and implemented features in existing services written in Java;
• Integrate APIs, creating a webhook ingester using Rust, and fetch data from FTP/SFTP for our data pipelines.

An internal analytics data mart was developed for a division of a large insurance organization. The platform sourced data from a centralized data warehouse, processed it through multiple structured data layers, and stored it in Parquet format to support divisional reporting and analytics. The project focused on building reliable data transformation workflows and managing data workloads using Databricks and Python, while supporting business requirements and coordination across different teams.

• Gathered and analyzed user requirements to translate business needs into data solutions;
• Designed and implemented data pipelines from source systems to the analytics data mart;
• Developed data transformation and processing logic across multiple storage layers;
• Managed and optimized data workloads using Databricks.

A peer-to-peer lending platform supporting loan underwriting through applicant evaluation, credit risk assessment, and borrower data management. The system aggregated financial and geospatial data from multiple sources, providing clean datasets for analytics and lending decisions. The project included data orchestration, automated data extraction, financial data services, geospatial processing, and deployment of containerized evaluation and machine learning tools.

• Ensured data analyst and data science teams had reliable datasets for their workflows;
• Maintained and enhanced a legacy data orchestration system built with Airflow and Kedro;
• Built automated data ingestion workflows from multiple sources, including Google Sheets, into the data lake;
• Configured daily Airflow DAGs for scheduled data processing, monitoring, and email reporting;
• Processed GIS data to generate geospatial features for client evaluation;
• Developed and deployed client evaluation services using Docker on Linux;
• Built and containerized a machine learning model for production use;
• Created a Streamlit-based interface for machine learning inference.

A social media intelligence and analytics platform designed to monitor digital campaigns, brand engagement, and visual content for marketing clients. The system collected and processed social media data from multiple sources, storing structured datasets in a cloud warehouse to support campaign reporting and content analysis. The project involved data collection services, cloud data management, web scraping, and machine learning for image classification.

• Collected and processed social media data from multiple sources, including web scrapers and the Facebook Graph API;
• Managed and maintained the BigQuery data warehouse for structured analytics data;
• Developed and deployed data collection applications on remote Linux environments;
• Orchestrated cloud-based data processing workflows and retrieved processed results;
• Designed and architected a machine learning solution for image classification.