In this tutorial, we will demonstrate how containers efficiently use storage on the underlying guest operating system.
What You’ll Learn
- How Docker containers use and share the guest OS storage
- What Docker images are
- How Docker containers use Docker images as their read-only file systems
- How Docker containers can share read-only file systems
- How Docker containers have their own read/write (R/W) layer
- How Docker containers retrieve a unified view of their file system
What You’ll Need
- Basic understanding of Docker and containerization
- Docker Desktop installed
A driving factor that is increasing the adoption of containerization is that you can have multiple containers (and thus multiple applications) safely running on the same guest OS. When multiple containers are running on the same guest OS, those containers can share and utilize the server’s hardware resources (CPU, memory, network I/O, and storage). In the image below, container A and container B are both running on the same guest OS. Therefore, they can both utilize and share the server’s hardware resources, including storage.
Docker images are a core concept of containerization. A docker container is created from a Docker image. You can think of Docker images as templates from which we can spawn our containers. For example, the image below illustrates how a Docker container gets created. The Docker image (blue box on the left) is like our template. It is downloaded from Docker’s repository to the guest OS. Next, the user runs the docker run command (blue arrow) to create a container (green box on right). When the user runs the docker run command, they must specify the Docker image (Ubuntu 23.04) from which they want to create the container.
Let’s see container creation in action. Open a terminal CLI session from your respective OS and run the command below to create a container named Container1. Create the container from the Ubuntu 23.04 image.
C:\>docker run -itd --name Container1 ubuntu:23.04 bash
Unable to find image 'ubuntu:23.04' locally
23.04: Pulling from library/ubuntu
361237ddf358: Pull complete
Digest: sha256:9279f41cc6e4df8f87b13ac17c2c6f2a280fd3ca2638d18f8dc94b774486909f
Status: Downloaded newer image for ubuntu:23.04
8c8841d75a669757a2914222228f9fb54b687719ae826d85c322bde5282cac82
If you have never used the Ubuntu 23.04 image, the Docker image should be downloaded to your guest OS. Confirm that the Docker image is downloaded to your guest OS by running the command below.
Also, run the command to confirm that Container1 was created from the Ubuntu 23.04 image.
Next, attach into Container1 and run the command to view Container1’s file system.
C:\>docker attach Container1
root@8c8841d75a66:/# ls
bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var
root@8c8841d75a66:/#
When running the ls command from within Container1, how is Container1 reading its file system? Surprisingly, containers use the Docker image that they were created from as their read-only file systems. When running the ls command from within Container1, Container1 reads its file system from the read-only Ubuntu 23.04 image. Previously, I mentioned that Docker images were like templates from which we can spawn our containers. However, there is a lot more to Docker images. They also play a significant role in container storage architecture. Docker images are essentially read-only files that act as read-only file systems for their respective containers.
Let’s detach from Container1 by typing CTRL p q.
Next, run the command below to try to delete the Ubuntu 24.03 image. What do you think will happen if we try to delete Container1’s file system?
C:\>docker image rm ubuntu:23.04
Error response from daemon: conflict: unable to remove repository reference "ubuntu:23.04" (must force) - container 8c8841d75a66 is using its referenced image d710383bd1ef
When attempting to delete the Ubuntu 24.03 image, Docker prints out an error because Container1 currently is using the Ubuntu 24.03 image as its read-only file system.
Previously, we created only one container from a Docker image. However, we can create multiple containers from the same Docker image.
Next, create Container2 from the same Ubuntu 24.03 image.
C:\>docker run -itd --name Container2 ubuntu:23.04 bash
4a964ca2a7a89ad933b071389e67d8854f3df872ddeb93250b0549ede74810b3
Run the command to confirm that both Container1 and Container2 are using the same image.
Run the command below to confirm the amount of storage space Container1 and Container2 utilize on the underlying guest OS. How much storage space does Container1 utilize? How much storage space does Container2 utilize?
C:\>docker ps -s
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
4a964ca2a7a8 ubuntu:23.04 "bash" 7 minutes ago Up 7 minutes Container2 0B (virtual 70.3MB)
8c8841d75a66 ubuntu:23.04 "bash" 58 minutes ago Up 58 minutes Container1 0B (virtual 70.3MB)
Container1 and Container2 each take up zero bytes of space on the underlying guest OS. Yes, that’s correct; the containers themselves take up no storage space on the underlying guest OS! You see, containers created from the same Docker image on the same guest OS will share the same read-only file system.
How can this happen? The containers take up zero bytes of space because their read-only file system is stored in the Docker image from which they were created. Based on the output below, we can see the Ubuntu 24.03 image they share is 70.3 MB.
We can also confirm the size of the Ubuntu 23.04 image by running the command below.
We can also use another command to confirm that the containers are taking up zero bytes of storage.
The image below depicts our current setup from a container storage perspective. The containers are taking up 0 MB of storage space on the underlying guest OS. All the files and folders that the containers currently see within their file system are actually stored inside the Docker image from which the container was created. The container’s Docker image is consuming 70.3 MB of storage space on the underlying guest OS.
The concept of containers sharing the same read-only file system is an incredibly powerful benefit when using containerization. When we create more containers from the same Ubuntu 23.04 read-only image, guess what? We still only utilize 70.3 MBs of storage space.
Create Container3 and Container4 from the same image.
C:\>docker run -itd --name Container3 ubuntu:23.04 bash
bfff1f921a6fa0823395374c1cec8118dd22cedf106434ca9f542e9dda1eea2a
C:\>docker run -itd --name Container4 ubuntu:23.04 bash
325a2584e5762610e6c6b3563b06dcc95cbc477c42052269d1f98c705696b867
How much storage space do you think Container3 and Container4 will take up on the underlying guest OS? Run the command below to check.
Even though we created more containers, because the containers are sharing the same Docker image, those containers do not take up additional storage space on the underlying guest OS. We have four containers and only one file system that the containers can share; this is data deduplication at its finest!
Go ahead and stop, and then remove Container3 and Container4.
C:\>docker stop Container3 Container4
Container3
Container4
C:\>docker rm Container3 Container4
Container3
Container4
Confirm that Container1 and Container2 are still running on the guest OS and that Container3 and Container4 have been removed.
C:\>docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4a964ca2a7a8 ubuntu:23.04 "bash" 47 minutes ago Up 47 minutes Container2
8c8841d75a66 ubuntu:23.04 "bash" 2 hours ago Up 2 hours Container1
You might be asking: If containers created from the same image share the same read-only file system, how can each individual container have its own unique files? This is where R/W layers come into play.
Although containers can share the same image, each container will have its own R/W layer. In the image below, Container1 is able to read its file system from the Ubuntu 23.04 image. However, if you were to create a new file within Container1, that new file would get stored in Container1’s R/W layer. Each container’s R/W layer is stored in a unique directory on the underlying guest OS. Because each container has its own R/W layer, containers can have files that are only available to that respective container.
Let’s write data to Container1’s R/W layer. Attach into Container1 and create a file called Hello_Uncle_Bob.
C:\>docker attach Container1
root@8c8841d75a66:/# truncate -s 10m Hello_Uncle_Bob
Confirm that you can see the Hello_Uncle_Bob file in Container1’s file system.
root@8c8841d75a66:/# ls
Hello_Uncle_Bob boot etc lib lib64 media opt root sbin sys usr
bin dev home lib32 libx32 mnt proc run srv tmp var
Detach from Container1 by typing CTRL p q.
Container1’s Hello_Uncle_Bob file will be stored in Container1’s R/W layer. Because Container2 does not have access to Container1’s R/W layer, Container2 should not be able to view the Hello_Uncle_Bob file. Confirm that Container2 cannot see the Hello_Uncle_Bob file by attaching into Container2 and viewing its file system.
C:\>docker attach Container2
root@4a964ca2a7a8:/# ls
bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var
When running the command inside Container2, we do not see the Hello_Uncle_Bob file; we only see the files and folders within the Ubuntu 23.04 image, which is acting as Container2’s read-only file system.
Detach from Container2 by typing CTRL p q.
Run the command below to confirm how each container is utilizing storage. How much storage space do you think Container1 is utilizing? How much storage space do you think Container2 is utilizing? How much storage space do you think the Ubuntu 23.04 image is utilizing?
When running the docker ps -s command above, we can see that Container1’s R/W layer is taking up 10.5 MB of storage space on the underlying guest OS. When we add 10.5 MB to 70.3 MB (the size of the Ubuntu image), we get a total of 80.7 MB displayed. (10.5 plus 70.3 actually equals 80.8, so the “80.7” number that is displayed must be rounded down.)
When running the docker ps -s command, we can also see that Container2’s R/W layer is taking up 0 MB of storage space on the underlying guest OS. When we add 0 MB to 70.3 MB (the size of the Ubuntu image), we get a total of 70.3 MB displayed.
How do a container’s R/W layer and read-only file system work in tandem? Docker containers use an overlay file system (OverlayFS). The container’s R/W layer will act as an overlay, and the container’s read-only file system will act as an underlay. The container’s R/W layer is placed on top of or “over” the container’s read-only file system to merge into a single unified view of the container’s file system.
Referencing the image below, Container1’s unified view of the file system is constructed by merging the contents of Container1’s R/W layer and the contents of Container1’s read-only file system. When you run the ls command from within Container1, you are viewing the unified view of the file system.
Confirm that Docker is using the overlay2 storage driver by running the command below. When you run the command, you will receive a lot of output. Parse through the output and look for Storage Driver: overlay2.
C:\>docker info
<snip>
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 3
Server Version: 24.0.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
<snip>
Now, you might be asking: Where are Container1’s R/W layer, read-only file system, and unified view of the file system stored in the underlying guest OS? Run the command below to find out.
C:\>docker inspect Container1
When you run the command, you will receive a lot of output. Parse through the output and look for the following outlined in the screenshot below.
Referencing the image above, UpperDir reflects the path to Container1’s R/W layer, LowerDir reflects the path to Container1’s read-only file system, and MergedDir reflects the path to Container1’s unified view of its file system. OverlayFS takes the contents of the UpperDir directory and the contents of the LowerDir directory and merges them into the MergedDir directory. The MergedDir directory provides a single unified view of the file system for the respective container.
Attach into Container1 and run the command to view Container1’s file system.
root@8c8841d75a66:/# ls
Hello_Uncle_Bob boot etc lib lib64 media opt root sbin sys usr
bin dev home lib32 libx32 mnt proc run srv tmp var
When running the ls command above, Container1 will retrieve the files and folders from the MergedDir directory that reside on the underlying guest OS.
Detach from Container1 by typing CTRL p q.
Let’s clean up your work. Try to delete your Ubuntu image. The deletion should fail. Why do you think we are unable to delete the Ubuntu image?
C:\>docker image rm ubuntu:23.04
Error response from daemon: conflict: unable to remove repository reference "ubuntu:23.04" (must force) - container 8c8841d75a66 is using its referenced image d710383bd1ef
Remember, the Ubuntu image currently is being used as the read-only file system for both Container1 and Container2. Let’s try to stop and then delete Container1 and Container2.
C:\>docker stop Container1 Container2
Container1
Container2
C:\>docker rm Container1 Container2
Container1
Container2
Now that Container1 and Container2 are deleted, let’s try to delete the Ubuntu image.
C:\>docker image rm ubuntu:23.04
Untagged: ubuntu:23.04
Untagged: ubuntu@sha256:9279f41cc6e4df8f87b13ac17c2c6f2a280fd3ca2638d18f8dc94b774486909f
Deleted: sha256:d710383bd1efcb401c8e0aca2c596b3b93ab021d1e001b4e90c394af54773632
Deleted: sha256:b6ecb2c0d9b4ec9282af5a67caf5e397305998b1c5d1b6aa117dfbac2a6cf4e9
The Ubuntu image should now be deleted.
Nice work! You now understand how containers utilize storage and how they access their file systems.