Introduction GPUs as main accelerators for deep learning training tasks suffer from under-utilization. Under-utilization means in the first place means spending more money on purchasing the GPU. Second, it means wasting energy. Third, if it was used in a better way, it could mean higher performance! Authors of AntMan [1] propose…

Introduction Training deep learning models is a heavy task from computation and memory requirement perspective. Enterprises, research and development teams shared GPU clusters for this purpose. Usually, there is a resource manager and scheduler (e.g., SLURM, LFS, Kubernetes, Apache YARN, etc.) on the clusters to get the jobs and allocate GPUs…

This post reviews machine learning, its different subbranches very concisely as introduction, then delve into metrics. It is important to be careful when choosing and using or coming up with new metrics evaluating how well a ML approach performs. Machine Learning (ML) ML is about learning patterns in data instead of explicitly programming…

This post’s aim to spread the word for disabling MIG mode of all GPUs when using Nsight systems for profiling a deep learning model developed with PyTorch. If MIG is not disabled, you will experience observing errors talking about an error caused by a device that is in MIG mode followed by a bunch of bucket errors!

This post briefly and with an example shows how to profile a training task of a model with the help of PyTorch profiler. Developers use profiling tools for understanding the behavior of their code to be able to optimize it. TensorFlow framework provides a good ecosystem for machine learning developers…

When “traditional programming” or “programming” terms are used, we usually mean using programming languages like python, Java, C#, etc. to reshape or re-express a programmer’s solution for a problem in a way to be executable by a computer system. …

This post reviews how to build the experimenting environment with conda for deep learning. I compile the commands to ease the purpose of setting up the environment. Step 1: Get miniconda installed Conda is a package or environment management tool that eases the life of developers by eliminating the necessity of changing versions of packages…

Introduction and History This post delves into the MIG feature of NVIDIA GPU devices. GPUs are identified as the primary parallel general purpose processors for tasks like rendering graphics and training deep learning models. If you need to learn more about how these processors work, read the following post: How do GPUs work? This post reviews all that is needed to understand how GPU executes code. The assumption is that you know the basics of…medium.com

This post reviews all that is needed to understand how GPUs execute code. The assumption is that you know the basics of computer architecture to grasp this concept fully. This post starts with Flynn’s taxonomy, then goes into the single instruction multiple data (SIMD) processor. Afterwards, x86 instruction extension for…