So, I saw this Network Chuck video and wanted to have a go on the SteamDeck: Network Chuck - Containerising Ollama to run DeepSeek inside a constrained container
His video goes into using GPU's attached to the host, but he's using a NVidia GPU and the SteamDeck has AMD, so here's the detail on using NVidia or AMD GPU's in docker: running on a amd gpu
Ok, onto nitty gritty stuff..
So I run Bazzite on my Deck because I use it as my desktop computer, as well as a games platform, I spend more time in the Desktop mode than I do in the main BigScreen games mode, so with Bazzite I can get aruond the Read Only Operating System to sideload things like applications, or user permissions.
Anyway, that's a long way to say 'Docker is an arse on Bazzite SteamDeck', but you know what isn't an arse? Podman. So I have issues with the Docker company, they seem to be in the habit of land-grabbing and making themselves seen as the only-shop-in-town for containers, which is not the truth. Containers is its own thing, Docker is just one way to interact with them. The Docker toolset (and that's all it is) is a build tooling and a management layer for running container instances. That's it.
:) yeah fair play, but you got this far so I'll push my luck a little further... Podman is a like-for-like replacement tooling for Docker, the biggest thing it does is in the management layer its much more local, it's less about 'this big pool of things' and much more about a group of 'this one ephemeral thing', which is awesome for development..
So, Podman is a like-for-like replacement to docker, which means anytime we see the word docker in some bash script we replace with podman, so where Network Chuck says:
bash
docker run -d \
--gpus all \
-v ollama:/root/.ollama \
-p 11434:11434 \
--security-opt=no-new-privileges \
--cap-drop=ALL \
--cap-add=SYS_NICE \
--memory=8g \
--memory-swap=8g \
--cpus=4 \
--read-only \
--name ollama \
ollama/ollama
..I replace with:
bash
podman run -d \
-v ollama:/root/.ollama \
-p 11434:11434 \
--device /dev/kfd \
--device /dev/dri \
--security-opt=no-new-privileges \
--cap-drop=ALL \
--cap-add=SYS_NICE \
--memory=8g \
--memory-swap=8g \
--cpus=4 \
--read-only \
--name ollama \
--replace \
ollama/ollama:rocm
...and it works...
So, quick breakdown of the differences:
- docker gets changed to podman
- --gpus all is for nvidia, I don't got me that
- --device /dev/kfd and --device /dev/dri are used because I have an AMD GPU
- ollama/ollama is swapped out for ollama/ollama:rocm because I have an AMD GPU
Then when that finishes building and runs the service in the background we can issue commands to it to tell it which version of DeepSeek to use. I started with the 1.5 and worked up to 14b using the following commands:
bash
podman exec -it ollama ollama run deepseek-r1:1.5b
podman exec -it ollama ollama run deepseek-r1:7b
podman exec -it ollama ollama run deepseek-r1:8b
podman exec -it ollama ollama run deepseek-r1:14b
Realistically the smallest is the only one that works in any timely fashion, 14b fails to run up and 7b and 8b are as slow as drawn out speech.