Zarreen – Python, AI, LLM

An AI-powered system designed to monitor elderly individuals in elderly care facilities. The system processes and analyzes monitoring video footage in real-time, identifying any alarming actions by patients that require human intervention. This solution is being built in partnership with a nursing facility in Quebec.

• Worked as the Lead AI Scientist, spearheading the research and development.
• Developed a context-aware multi-modal video understanding model using LLMs, vision transformers (ViT), and Active learning.
• Processed real-time video streams using advanced computer vision techniques.
• Implemented a mechanism for obfuscating faces from video frames to ensure patient privacy.

The platform enables customizable content creation, improves clinical training, and ensures data security. It is ideal for nursing schools, healthcare institutions, and regulatory bodies seeking scalable, AI-powered training tools. It includes three components: a teacher, a student, and an interactive avatar module for communication training.

• Designed the project from beginning to end and developed the complete Teachers and Students Modules as the Lead Research Scientist.
• Managed a team of two researchers and facilitated regular client collaboration as the Project Manager.
• Employed RAG, CAG (context-augmented generation), LoRA fine-tuning, hallucination mitigation, prompt engineering, and safety guardrails to the framework.
• Customized an open-source multi-modal LLM to fit the use case.
• Integrated GraphRAG for retrieval augment generation, improving the model responses significantly with factual correctness.
• Extensively used Agentic AI frameworks like LangChain and LangGraph.
• Developed a fully-featured interface using Python and Flask as the frontend.

The project is one of the first initiatives in Canada to integrate custom, fully local large language models (LLMs) as AI-powered virtual assistants in a real classroom setting. It is assessed using standard NLP and machine learning metrics, including BERTScore, BLEU, and F1-score, by comparing AI-generated outputs with teacher-annotated references. Composed of 2 core components:

1. Teacher Module: This component supports educators in generating lab materials and evaluating student submissions.
2. Student Module: Designed as an interactive learning assistant, this chatbot helps students understand course material, solve exercises, and locate relevant content. Crucially, it does not provide direct answers but encourages learning through hints and conceptual guidance.

• Designed the project from beginning to end and developed core features as the Lead Research Scientist.
• Managed a team of two researchers and facilitated regular client collaboration as the Project Manager.
• Employed RAG, CAG (context-augmented generation), LoRA fine-tuning, hallucination mitigation, prompt engineering and safety guardrails to the framework.
• Developed a fully-featured interface using Python and Flask as the frontend.
• Deployed the solution to Microsoft Azure using Docker inside the client environment.
• Extensively used Agentic AI frameworks like LangChain and LangGraph.

This paper investigates the effectiveness of LoRA fine-tuning of LLMs in capturing the shift of fine-tuning datasets from the initial pre-trained data distribution. Findings reveal cases in which low-rank fine-tuning falls short in learning such shifts. This, in turn, produces non-negligible side effects, especially when fine-tuning is adopted for toxicity mitigation in pre-trained models, or in scenarios where it is essential to provide fair models. Through comprehensive empirical evidence on several models, datasets, and tasks, we show that low-rank fine-tuning inadvertently preserves undesirable biases and toxic behaviors. We also show that this extends to sequential decision-making tasks, emphasizing the need for careful evaluation to promote responsible LLM development.

• Examined the impact of Low-Rank Fine-Tuning (LoRA) on the fairness of language models by evaluating their disparate impacts across demographic groups and mitigating biases using counterfactuals.
• Ran experiments with Llama-2 and OPT models and their fine-tuned counterparts using varying LoRA ranks and conducting a comprehensive ablation study.
• Modified the HONEST benchmark dataset, designed initially to detect hurtful sentence completions based on gender, to evaluate bias related to race and religion. Additionally, generated counterfactuals to fine-tune models for bias mitigation and open-sourced the data on HuggingFace.
• The results directly contributed to novel insights showing that low-rank fine-tuning inadvertently preserves undesirable biases and toxic behaviors and extends them to downstream tasks like sequential decision-making.

This recommender system developed for a Quebec-based fintech incubator to match startup founders with suitable advisors using detailed criteria. It leverages a semi-supervised machine learning model trained on multimodal data, including questionnaires, user profiles, company info, experience, and other metadata. Integrated into the client’s workflow before an email campaign, the system boosted click-through and follow-up rates by 27%.

• Developed a hybrid recommendation system for matching startups with advisors using NLP techniques, BERT, and dimensionality reduction as the lead scientist.
• Designed the end-to-end framework for training a semi-supervised model using text and numeric data.
• Created a robust data processing pipeline to handle highly sparse, heterogeneous data using topic modeling, embeddings, PCA, UMAP and TF-IDF.
• The proposed semi-supervised solution outperformed the baseline model using collaborative filtering regarding accuracy and performance.

A part of the Shell company, which deals with advertising on Shell Recharge electric vehicle charging stations.

• Led and co-led multiple research and deployment efforts of core modules.
• Built small-scale proof-of-concepts and demos using open-access and first-party datasets for internal teams.
• Worked at the intersection of high-quality research, data science, and applied machine learning to produce innovative ML solutions for EV charging stations.
• Developed projects using deep learning for predictions, optimization, time-series analysis, and NLP.

The company is a global leader in advanced technologies for the Defence, Aerospace, and Cyber & Digital sectors.

• Developed proof of concepts (PoC) from customer data using cutting-edge machine learning algorithms.
• Developed an end-to-end orchestration pipeline of a machine learning PoC.
• Held hands-on demo of PoC in production using Docker, Kubernetes, Argo, MLFlow, Kubeflow, Seldon Core, AWS Lambda, Prometheus, and Grafana.
• Conducted a thorough analysis of MLOps frameworks.

Real-time weld quality inspection using deep learning and computer vision.

• Developed a CNN model (Convolutional Neural Network) for automated weld-quality detection in real-time from ultrasonic B-Scans.
• The model is deployed in a real production facility in the automotive parts company Narmco, Alabama, and one of the largest car manufacturers in the world, BMW.