Introduction

In this article I will tell you how to run docker containers to use NVIDIA GPUs from the system in Red Hat Enterprise Linux (RHEL) 9. x

We are going to achieve this with the help of NVIDIA Container Toolkit.

NVIDIA Container Toolkit?

The NVIDIA Container Toolkit allows users to run GPU accelerated containers. The toolkit includes a container runtime library and utilities to automatically configure containers to leverage NVIDIA GPUs.

Prerequisite

• NVIDIA drivers installed.
  Please read an article from the below link to install NVIDIA drivers on RHEL 9.x
  https://amiitn.hashnode.dev/install-nvidia-drivers-on-red-hat-enterprise-linux-9x

• Docker engine installed.
  Please read an article from the below link to install the Docker engine on RHEL 9.x
  https://amiitn.hashnode.dev/install-docker-engine-on-rhel-9x

Download NVIDIA Container Toolkit Repository

Use the following command to download the official NVIDIA Container Toolkit Repository from NVIDIA

curl -s -L https://nvidia.github.io/nvidia-docker/rhel9.0/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

Install NVIDIA Container Toolkit

Use the following command to download the official NVIDIA Container Toolkit Repository from NVIDIA

sudo dnf clean expire-cache && sudo dnf install -y nvidia-container-toolkit

After successful installation of the NVIDIA Container Toolkit restart the docker engine with the below command

sudo systemctl restart docker

Verify docker container using NVIDIA GPUs

To test the docker container is using NVIDIA GPU, run a base CUDA container with following command

sudo docker run --rm --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi

This container should result in a console output shown below:

If the output shows NVIDIA GPU and CUDA version details as above, we can say that the docker container can use NVIDIA GPUs. While running the docker container just need to pass '--gpus all' parameter to run the command.

Conclusion

By installing NVIDIA Container Toolkit we made NVIDIA GPU available to the docker container by passing '--gpus all' parameter to run the command.