# How to run docker containers to use NVIDIA GPUs from the system in RHEL 9.x?

## Introduction

In this article I will tell you how to run docker containers to use NVIDIA GPUs from the system in Red Hat Enterprise Linux (RHEL) 9. x

We are going to achieve this with the help of NVIDIA Container Toolkit.

## NVIDIA Container Toolkit?

The NVIDIA Container Toolkit allows users to run GPU accelerated containers. The toolkit includes a container runtime library and utilities to automatically configure containers to leverage NVIDIA GPUs.

## **Prerequisite**

* NVIDIA drivers installed.  
    Please read an article from the below link to install NVIDIA drivers on RHEL 9.x  
    [https://amiitn.hashnode.dev/install-nvidia-drivers-on-red-hat-enterprise-linux-9x](https://amiitn.hashnode.dev/install-nvidia-drivers-on-red-hat-enterprise-linux-9x)
    
* Docker engine installed.  
    Please read an article from the below link to install the Docker engine on RHEL 9.x  
    [https://amiitn.hashnode.dev/install-docker-engine-on-rhel-9x](https://amiitn.hashnode.dev/install-docker-engine-on-rhel-9x)
    

## Download NVIDIA Container Toolkit Repository

Use the following command to download the official NVIDIA Container Toolkit Repository from NVIDIA

```bash
curl -s -L https://nvidia.github.io/nvidia-docker/rhel9.0/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1678616897125/7e14e22d-2c2b-402a-9ede-15a5c4b1edc3.png align="center")

## Install NVIDIA Container Toolkit

Use the following command to download the official NVIDIA Container Toolkit Repository from NVIDIA

```bash
sudo dnf clean expire-cache && sudo dnf install -y nvidia-container-toolkit
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1678617033719/5a4ef33f-9419-4076-be8e-849d0c8dc71f.png align="center")

After successful installation of the NVIDIA Container Toolkit restart the docker engine with the below command

```bash
sudo systemctl restart docker
```

## Verify docker container using NVIDIA GPUs

To test the docker container is using NVIDIA GPU, run a base CUDA container with following command

```bash
sudo docker run --rm --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
```

This container should result in a console output shown below:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1678617831056/b1d00320-7c8a-402a-b88b-4c8ae5fc4c87.png align="center")

If the output shows NVIDIA GPU and CUDA version details as above, we can say that the docker container can use NVIDIA GPUs. While running the docker container just need to pass '--gpus all' parameter to run the command.

## Conclusion

By installing NVIDIA Container Toolkit we made NVIDIA GPU available to the docker container by passing '--gpus all' parameter to run the command.