Docker Pruning on Your Host

What You’ll Learn

What a dangling image is and how it impacts disk space
How unused Docker volumes can accumulate and adversely affect your system
How to use the docker prune command to reclaim unused and orphaned resources
A few best practices for conserving disk space on a Docker host

What You’ll Need

An understanding of the basics of Docker Administration
Access to a host running Docker

Docker is a common tool for developers and system administrators to manage containerized applications and services. However, as you use Docker over time, you may notice that your system consumes a lot of disk space.

In Docker, a “dangling image” is an untagged image not used by any containers. One common way this can occur is when a new image takes the tag of an older image, leaving the older one untagged. For example, suppose you have an image tagged as my-awesome-app:latest. If you build a new image and tag it as my-awesome-app:latest, the older image will lose its tag and become a dangling image.

barweiss@host:~/my-awesome-app-project$ docker images
REPOSITORY       TAG       IMAGE ID       CREATED         SIZE
my-awesome-app   latest    ea112f0f77da   2 minutes ago   1.49GB

# Rebuild an new version of the app as latest
barweiss@host:~/my-awesome-app-project$ docker build -t my-awesome-app:latest .
[+] Building 51.9s (10/10) FINISHED
=> [internal] load build definition from dockerfile
<-- Output Ommitted -->

barweiss@host:~/my-awesome-app-project$ docker images
REPOSITORY       TAG       IMAGE ID       CREATED         SIZE
<none>           <none>    ea112f0f77da   2 minutes ago   1.49GB
my-awesome-app   latest    7bc2610f0fef   2 minutes ago   1.49GB

Why worry about dangling images? First, they take up disk space. Second, having too many dangling images can make it hard to know which images you actually need, causing clutter and confusion. Additionally, unused images could pose a security risk if they contain vulnerabilities. This is not a good situation to be in; there is nothing worse than maintaining a bloated and cluttered system.

Docker images are not the only objects that can become stale and consume resources. When you are using Docker and you see heavy and gradual use of storage space, this can be due to unused Docker objects like:

Containers
Volumes
Networks
Images (which we just talked about 😊)

Docker provides a built-in solution to address this situation called “Docker pruning,” which helps you clean up these unused objects, thereby conserving system resources.

What Is Docker Pruning?

Docker pruning is the process of removing unused Docker objects to free up system resources—primarily storage space. Docker can prune:

Containers: Stopped containers that are not in use
Volumes: Volumes not associated with any containers
Networks: Networks not being used by any containers
Images: Images not associated with any running or stopped containers, including dangling images

How Do You Prune Your Docker Images?

Before running the docker image prune command, be sure to tag and account for all the images that you want to keep. Also, identify images that are not needed, and ensure that they are not associated with any running or stopped containers. Docker provides a command to filter out dangling images, as shown in this example:

barweiss@host:~/my-awesome-app-project$ docker images -f "dangling=true"
REPOSITORY    TAG       IMAGE ID             CREATED         SIZE
<none>        <none>    40046f75d6ee         3 minutes ago   300MB
<none>        <none>    341640cdfda9         3 minutes ago   450MB

After verifying the images identified as dangling, you can remove them using the following command:

barweiss@host:~/my-awesome-app-project$ docker image prune

You will receive the following warning. If you are comfortable with the changes, type y and enter to continue with the pruning:

WARNING! This will remove all dangling images.
Are you sure you want to continue? [y/N] y
Deleted Images:
deleted: 40046f75d6ee
deleted: 341640cdfda9

Total reclaimed space: 750MB

To skip the warning—useful when scripting—you can add the -f or --force option to proceed without confirmation. Use this with caution.

Maintaining a Large Local Image Repository

If you’re dealing with many Docker images on your local hosts repository, labels and filters are invaluable tools for organization. We used the filter command earlier to isolate dangling images. Adding labels to your images is another way to stay organized. Labels are simple tags that you can add during the build process with a --label flag or directly in your Dockerfile. Once labeled, filters help you sort through your images. For example, to list or prune images with specific labels, you can use commands like docker images --filter "label=project=my-awesome-app".

Docker’s pruning feature includes a --filter flag, allowing you to clean up specific resources. This flag can be used with various Docker prune commands to set conditions based on time or labels. For instance, to remove images from a specific project that are older than an hour, use docker image prune --filter "label=project=my-awesome-app" --filter "until=1h". Always double-check what you’re about to prune, especially in production environments, to avoid unintended deletions. Measure twice, cut once.📐✂️

Docker Volumes and Clutter

Docker volumes are used to persist data and state between container runs. While this is incredibly useful for many applications, it can also lead to the accumulation of unused or “orphaned” volumes that are no longer associated with a running or stopped container. These unused volumes can consume valuable disk space and may pose security risks.

The most immediate concern with unused volumes is disk space. Over time, as you create and destroy containers, you may forget about the volumes that were once attached to them. These volumes can take up a significant amount of disk space, especially if they contain large datasets or logs.

Unused volumes can also pose a security risk. Sensitive data might be left exposed in these volumes, and if an unauthorized user gains access to your Docker host, they may be able to read or modify this data. Additionally, old volumes might contain configurations or files that are out of date and have known security vulnerabilities.

Pruning Docker Volumes

⚠️ As with pruning Docker images, the same precautions should be taken before issuing the prune command. Make sure that you have identified all the volumes you want to keep so that you don’t inadvertently delete something you didn’t intend to.

Docker provides a straightforward way to remove unused volumes: the docker volume prune command. This command will prompt you for confirmation and then remove all volumes not used by at least one container.

Here’s how to run it:

barweiss@host:~/my-awesome-app-project$ docker volume prune

Just like when you prune Docker images, you’ll be prompted to confirm the action:

WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N]

After confirming, Docker will proceed to remove unused volumes and display a summary:

Deleted Volumes:
volume1
volume2
volume3

Total reclaimed space: 1.24GB

As with the Docker images, you can also use filters to prune volumes selectively—for example, to prune volumes that are labeled for a specific project:

barweiss@host:~/my-awesome-app-project$ docker volume prune --filter "label=project=myProject"

Things to Keep In Mind About Keeping a Docker Host Healthy

In Docker management, regular pruning stands out as more than just an uncluttering measure; it’s essential for both system performance and security. To maintain a healthy Docker environment, consider these best practices:

Regular pruning: Automate the removal of unused resources through scheduled tasks like cron jobs.
Labeling: Consistently label your Docker items to simplify filtering and management, especially in large-scale setups.
Disk monitoring: Use tools like the docker system df command to keep track of disk usage.
Resource limits: Set CPU and memory caps on running containers to prevent resource hogging.
Security scans: Regularly check images for vulnerabilities, either through your container registry or third-party tools.
Audit and review: Periodically go through your containers, images, and volumes to remove unnecessary or outdated items.
Log management: Implement a log rotation strategy to prevent logs from eating up all your disk space.

Automating Pruning

In larger deployments and production environments, manual pruning becomes impractical because of the high volume of containers, images, and volumes. Automation is essential for consistently freeing up resources and maintaining a secure, efficient system.

For complex needs, custom scripts using Docker’s API or CLI can identify resources for pruning. These scripts can be scheduled to run regularly. The simplest automation method, however, is scheduling cron jobs to execute Docker prune commands at set intervals.

For instance, to initiate a systemwide prune every day at midnight, your crontab entry could look like:

#Nightly Docker System Prune
0 0 * * * /usr/bin/docker system prune -af

The docker system prune command will prune all elements of Docker, which includes containers, images, networks, and volumes. The -a will clear all unused and dangling elements, and -f will force the prune without providing you a warning. Use this command with extreme caution!

As with automating any process, especially a process that deletes something, consider the following:

Environment: Exercise extra caution in production settings. Test scripts and cron jobs in a staging environment first.
Timing: Opt for times when the system is least busy to minimize service impact.
Logging and monitoring: Maintain logs of pruned items and set up alerts for issues during the process.
Exclusions: Implement logic to exclude essential resources from being pruned.

Orchestration Tools

In larger deployments, orchestration tools like Kubernetes or Docker Swarm become increasingly important. These platforms often come with their own built-in mechanisms for resource cleanup. Additionally, you can weave pruning commands directly into your CI/CD pipeline. As your Docker operations grow, you’ll find yourself relying less on standalone Docker commands and more on these comprehensive tools.

For even more control, specialized solutions like Portainer and Rancher, as well as third-party monitoring tools, offer nuanced management of Docker resources, including options for automated cleanup.

Final Words

Congratulations on completing the tutorial! 🎉 You have taken one more step in maintaining a well-functioning Docker host environment!

This tutorial touched on some basics of Docker pruning, especially dangling images and unused volumes, which can consume storage on your host. It is important to know that pruning in Docker serves multiple purposes that enhance both system performance and security, and is sometimes an overlooked detail when introducing beginners to Docker. In summary, pruning helps with:

Disk space: Reclaiming valuable storage space by removing unused images, containers, and volumes
Memory: Freeing up RAM by getting rid of stopped containers that consume system resources
Performance: Speeding up Docker commands and operations by eliminating clutter from unused objects

In addition to operational efficiency, pruning has security benefits. Unused containers and images can be potential security risks, harboring vulnerabilities that might be exploited in the future. Therefore, regular pruning is not just good for system performance; it’s also crucial for maintaining a secure Docker environment.

Learn More

Ready to become a Docker and cloud expert? Cisco has you covered. From our beginner-friendly Docker 101 course at Cisco Developer Learning Labs Center to the DEVOPS learning path at Cisco U., we offer a comprehensive suite of resources to boost your skills. Don’t miss out—join the DevNet Certifications Community now!