profile picture

Amr Kayid

🪼 @ 🇨🇦 🇪🇺 🇬🇧 🇺🇸 🇩🇪 🇪🇬

training 🐬 🐋 🐳 🐘 🦒 on SUPERCOMPUTERSSSSS tpuwu 🛸
& building powerful better version of baymax baymax 💓 🥺
& I spend sometime understanding
my 🧠 @ smol lab medialab 👀 mit

I'm Amr Kayid, a fresh Computer Science and Engineering Grad from the German University in Cairo, Egypt 🇪🇬. I am currently a machine learning researcher 👨🏻‍🔬 with FOR.ai doing research on advanced A.I. topics and working with amazing researchers from Google Brain, University of Oxford, and MILA. My work involved around autonomous systems 🤖, imitation learning 👀, hierarchical reinforcement learning 📚, and neural network optimization 🕸. Adding to that, collaborating with OATML Oxford Research Group on autonomous driving 🚘 and Bayesian deep learning 🤔. I am also a Research Engineer 😎 and Senior UCSF Fellowship Recipient ✌️ at OpenMined Organization. I am working and doing research on advanced federated learning 👁, neural architecture search 🕵🏻‍♂️ with differential privacy, and dynamic federated learning 👨🏻‍💻. I am also collaborating with research scientists from Harvard and CMU in the theory and applications of Learning Systems, researching multimodal and multitask learning, aiming to build the Infrastructure for Intelligence 🕸. I was a visiting student at the Technical University of Munich (TUM) where I worked on my bachelor project, associated with the Human Brain Project Organization, in the area of distributed deep reinforcement learning. Recently, I participated in DeepMind EEML Summer School 2020 as an undergraduate student, the school focus was on Deep Learning and Reinforcement Learning. My research interests are multi-agent reinforcement learning, autonomous systems & self-driving cars, federated learning, multitask & meta learning, and Human-Centered AI .

Education

Mar. 2020 - Sep. 2020
Spent spectacular 6 months in the lovely Munich City. I was fortunate enough to work on my Bachelor Project/Thesis in the area of Distributed Deep Reinforcement Learning. The project was held in the chair of Robotics, Artificial Intelligence, and Embedded Systems. This project is associated with the Human Brain Project Organization and Neurorobotics Platform Team.

During my stay in Munich, I was lucky to join multiple events and hackathons including HackRoboy with amazing Roboy Team and spending time at this amazing building UnternehmerTUM GmbH. Visiting Max Planck Institute of Quantum Optics and discovering the amazing research they are doing in Quantum Machine Learning, Quantum Computing and Tensor Networks. Adding to that, spending wonderful time at the Chair for Computer Aided Medical Procedures & Augmented Reality ( CAMP) at their Virtual Reality Labs with my friend Omar. 😎
My favorite place there was Olympiapark! ❤️

Best Memories

Oct. 2015 - Jul. 2020
guc

The journey that changed my whole life!

Best Memories

Jul. 2020
One of the best things that happened to me in 2020 when I received the acceptance email to attend DeepMind EEML Summer School! Starting in 1st of July, during my university exams and OpenMined fellowship, I was full of excitement and enthusiasm to attend the school and I had a great time learning from the best researchers in the world of Deep Learning and Reinforcement Learning from various companies and universities including DeepMind, Facebook AI (FAIR), Google Research, McGill University, Warsaw University, and University College London. This 10 days was full of fun, learning, discovering new fields and getting pieces of advices and mentorship! Thank you EEML!! ❤️


Research & Work Experience

May 2021 - Jun 2024

✨ An incredible 3-Year Journey! ✨

Back in 2019, while juggling undergrad life, fate led me to the brilliant FORai research group founded by Aidan Gomez, Ivan Zhang, and Bryan M. Li. Little did I know, diving into fascinating RL projects would introduce me to the amazing Nick Frosst, who gave me a life-changing intro to Cohere. That single connection reshaped my path forever! Fast forward, I joined Cohere as an early employee and founding team member, riding the rocket ship 🚀 from a cozy crew of 25 to a dynamic family of 250+ talented folks. This journey wasn't just about numbers – it was about building something extraordinary with some of the brightest, kindest, and most inspiring minds of our time. 🧠✨

💻 From Code to Impact: Technical Adventures

• Built cutting-edge distributed training frameworks for scaling massive language models using JAX, Ray, TPUs, and GPUs 🛠️
• Trained, evaluated, and shipped Cohere's multilingual embeddings model – enabling AI to speak my native language 🌍❤️
• Scaled semantic search to mind-blowing billion-scale efficiency, achieving a 1,000x cost reduction with quantization magic! 🪄🔍
• Designed an ironclad evaluation framework to measure performance across 250+ tasks alongside an epic team. 💪

💞 Legendary Co:lleagues & Forever Friends

Every moment with the incredible Adrien Morisot, Siddhartha Rao Kamalakara, Sylvie Shi, Bharat Venkitesh, Acyr Locatelli, Dwarak Talupuru, Elliott Choi, Jay Alammar, and so many more was pure gold. 🏅 The countless discussions, late-night breakthroughs, and shared laughs? Absolutely unforgettable. ❤️

Tech Arsenal: JAX, TPUs, PyTorch, Triton, PySpark, Beam 🛠️
Focus Areas: Large Language Models, Transformers, Semantic Search, Distributed Systems, Model Optimization 🌐

This chapter of my life was nothing short of legendary 🔥

Mar. 2020 - Now
We're an open, distributed research group working on significantly improving learning systems and applying them to the toughest problems humanity faces. We do this by advancing theory and also building exceptional open source tools. Manifold Computing is a fairly unique research lab, formed from the thesis that the biggest scientific problems we face as a civilization will require progress on numerous fronts. One of these is improving the theory and applicability of modern learning systems, which despite significant progress, do not scale well to multimodal, heirarchical and data starved problems. Manifold Computing is built on numerous collaborations, with folks from well known research institutions like Harvard, UCL, TUM, Georgia Tech, industry organizations like Google, OpenAI, and Facebook. you're interested in building the Infrastructure of Intelligence, reach out! My current work there focus on Research on Creating, Managing, and Understanding Large,Sparse, Multitask Neural Networks 🕸 & Working on new algorithms for Neural Architecture Search with Differential Privacy 🕵🏻‍♂️
Jan. 2020 - Now
OpenMined is an open-source community whose goal is to make the world more privacy-preserving by lowering the barrier-to-entry to private AI technologies. Working on various open-source projects ranging between python development, multi-language libraries, federated learning, infrastructure management. Adding to that, being a Research Engineer working with Research scientists on multimodal and multitask learning, neural architecture search with differential privacy, and federated reinforcement learning. Moreover, I recently got accepted into 👉🏻 OpenMined-UCSF Project for Data-Centric Federated Learning as Senior Fellow and I am Leading the Infrastructure team for multiple cloud deployment of our PyGrid Platform as Peer-to-peer Platform for Decentralized Data Science.
Oct. 2019 - Now
We are a multi-disciplinary team of scientists and engineers who like doing research for fun. Our only objective is to publish good machine learning research that is useful and interesting. Our collaborators include researchers from research institutions such as Google Brain, University of Oxford, and Vector Institute for Artificial Intelligence. Currently, Collaborating with OATML Oxford Research Group on autonomous driving and Bayesian deep learning, Building OATomobile: A research framework for autonomous driving for doing novel research, Experimenting with CARLA simulator for novel research in reinforcement learning and autonomous driving with Uncertainty-Aware Policy Distillation. Building a Highly modular codebase for reinforcement learning models training, testing and visualization. Researching curriculum learning, hierarchical reinforcement learning, selfdriving cars, and neural network optimization.
Apr. 2020 - Oct. 2020
Creating artistic robots that can draw incredible portrait paintings with neural style transfer Developing automated pipeline for image modification & enhancement methods using GANs and computer vision techniques Helping with creating simulated environments for the robotic arm and training RL algorithms Building a distributed pipeline system for integrating the software and hardware parts of the robots
Jun. 2019 - Oct. 2019
AI Engineer (Voice Emotion Recognition) was Responsible for Viva Robot's Visual and Emotional Perception, Implemented speech emotion recognition system using the State of the Art methods for viva robot
Mar. 2019 - Sep. 2019
Worked on Neurorobotics for the next Generation of Intelligent Robots, Worked with the team of the neurorobotics platform on building new environments and simulations for distributed reinforcement learning agents


Publications



Places where I lived and visited

🇨🇦
Currently living in Canada with bears 🐻 and moose 🦌
🇩🇪
Spent time at the Technical University of Munich (TUM) and exploring the beauty of Germany
🇸🇦
Spent time with family in Riyadh and performed Umrah in my favorite cities Mecca and Medina 🤍
🇪🇬
During University and undergraduate studies
🇪🇬
My hometown and where I grew up with my family and friends.