Ibrahim – PyTorch, LLM, Python

This project employs the Llama-2-7B model, fine-tuned for question-answering using two datasets: physics books and a scientific paper. The models adapted to the domains and provided representative answers.

• Fine-tuned Llama-2-7B for domain-specific question answering across physics and scientific text corpora.
• Conducted self-supervised pre-training on domain text and supervised fine-tuning on generated Q&A datasets.
• Built automated scripts using LangChain and Llama-2 Chat to generate and structure domain-specific Q&A pairs.
• Applied parameter-efficient fine-tuning (PEFT) and QLoRA to reduce trainable parameters from 7B to ~34M.
• Evaluated model performance using Perplexity and ROUGE metrics for both domain adaptations.
• Authored the paper “Evaluation of Fine-Tuning Llama-2 for Domain-Specific Question Answering”, accepted at MedPRAI-24.

The project involved developing a BERT-based model to classify job proposal texts into categories such as 'very_poor', 'poor', 'fair', 'good', and 'excellent'.

• Designed and fine-tuned the BERT model with LoRA, achieving 0.96 accuracy on training data and 0.89 on evaluation data.
• Led end-to-end development, from data preprocessing to model deployment on Hugging Face.
• Created insightful visualizations to monitor training progress and model performance.

The project was an experimental POC of structural model pruning for optimizing neural networks, gaining higher inference FPS.

• Conducted an empirical study on structural pruning to optimize neural networks for edge devices.
• Applied DepGraph-based pruning to the SwinTransformer-Tiny model on an ImageNet-1k subset.
• Trained, pruned, and fine-tuned models to evaluate the impact of structural pruning on performance and efficiency.
• Achieved up to 45% faster inference while maintaining accuracy comparable to the original model.
• Analyzed parameter reduction and performance trade-offs across multiple pruning configurations.
• Authored the paper “Optimizing Neural Networks for Edge Devices via Structural Pruning: An Empirical Study with DepGraph”, accepted at CCIDSA Conference.

The project was a solution, implemented directly from a published paper to predict 3D objects from fisheye cameras without training images, using our vanilla model trained on rectilinear images.

• Implemented a fisheye image correction pipeline enabling accurate 3D object detection using existing pre-trained models.
• Transformed fisheye images into cylindrical projections to achieve shift invariance for improved detection consistency.
• Performed inference on cylindrical images with a pre-trained 3D object detector and converted outputs to real 3D space.
• Ported the initial Python prototype to C++ and integrated it into the main AI stack for production use.
• Reduced R&D time by reusing existing models without additional image training.