How to run docker containers to use NVIDIA GPUs from the system in RHEL 9.x?

I am a software engineer. Documenting my DevOps journey
Search for a command to run...

I am a software engineer. Documenting my DevOps journey
No comments yet. Be the first to comment.
Introduction SSH is method of securely connect to remote server and execute command on that server. To provide user access to remote server either password or SSH key needs to be shared. But this becomes cumbersome when there are many users as well a...

Introduction In this article, we ran a Python application in AWS Lambda using Docker. Now we want to add some licencing mechanism by keeping the access key or any secret key to another service or application to restrict the use of this application. H...

Introduction I have been using Ubuntu for a long time on Intel Core i5 system. The daily routine was to pull the code from the Git repositories, build docker images, push it to Dockerhub and run docker containers with those images in the production U...

Introduction In the last article, we discovered how to monitor a NodeJs application with Prometheus and Grafana. In that, we learned to collect metrics from the application like the rate of requests in the last 5 minutes, the rate of 5xx responses, b...

Introduction Application monitoring is essential in the production environment. Once the application is deployed, the traffic to the application starts increasing, after some time the application starts responding slowly or sometimes it gets failed. ...

In this article I will tell you how to run docker containers to use NVIDIA GPUs from the system in Red Hat Enterprise Linux (RHEL) 9. x
We are going to achieve this with the help of NVIDIA Container Toolkit.
The NVIDIA Container Toolkit allows users to run GPU accelerated containers. The toolkit includes a container runtime library and utilities to automatically configure containers to leverage NVIDIA GPUs.
NVIDIA drivers installed.
Please read an article from the below link to install NVIDIA drivers on RHEL 9.x
https://amiitn.hashnode.dev/install-nvidia-drivers-on-red-hat-enterprise-linux-9x
Docker engine installed.
Please read an article from the below link to install the Docker engine on RHEL 9.x
https://amiitn.hashnode.dev/install-docker-engine-on-rhel-9x
Use the following command to download the official NVIDIA Container Toolkit Repository from NVIDIA
curl -s -L https://nvidia.github.io/nvidia-docker/rhel9.0/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

Use the following command to download the official NVIDIA Container Toolkit Repository from NVIDIA
sudo dnf clean expire-cache && sudo dnf install -y nvidia-container-toolkit

After successful installation of the NVIDIA Container Toolkit restart the docker engine with the below command
sudo systemctl restart docker
To test the docker container is using NVIDIA GPU, run a base CUDA container with following command
sudo docker run --rm --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
This container should result in a console output shown below:

If the output shows NVIDIA GPU and CUDA version details as above, we can say that the docker container can use NVIDIA GPUs. While running the docker container just need to pass '--gpus all' parameter to run the command.
By installing NVIDIA Container Toolkit we made NVIDIA GPU available to the docker container by passing '--gpus all' parameter to run the command.