Docker is a very interesting project. It promises to alleviate the pain of running and deploying modern, scalable server applications. I want to believe the hype, but I have some reservations about using it for real projects. In this post, I am going to start from the beginning and point out things that surprise me about docker's behavior. Then I am going to revisit the docs and devise experiments to clarify concepts I don't understand.
This post will examine the basics of working with containers and images - the foundation upon which all other docker features are built. It is not comprehensive, but instead focuses on what is most interesting or important to me.
If you plan to follow along, you will benefit from a running docker installation. For installing docker locally, I found Docker Machine very easy to install and configure. If you would prefer, Digital Ocean can launch a droplet with the latest Docker already installed and ready for immediate use.
I am using Docker 1.6.0 for the examples in this post.
Lines that start with the ~$ prompt are from my local shell and lines that start with root@ are
from inside the docker container.
My container is gone! Where did it go?
Below I am running my first docker container. I use the -i (interactive) and -t (pseudo TTY)
flags together, which let me interact with the container like I interact with my terminal.
~$ docker run -it ubuntu /bin/bash
root@1f608dc4e5b4:/# echo hello docker > /message.txt
root@1f608dc4e5b4:/# cat /message.txt
hello docker
root@1f608dc4e5b4:/# exit
Notice I created an important file, /message.txt, before exiting my container. I would like to
read this file back now.
~$ docker run -it ubuntu cat /message.txt
cat: /message.txt: No such file or directory
Uh oh! There is no such file. I run docker ps to list my containers, but there is no such container.
It turns out, the command I am looking for is docker ps -a, which lists all of my containers, running or not.
~$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b3d8c3ef31a0 ubuntu:latest "cat /message.txt" 3 minutes ago Exited (1) 3 minutes ago admiring_lalande
1f608dc4e5b4 ubuntu:latest "/bin/bash" 4 minutes ago Exited (0) 3 minutes ago insane_wright
That's interesting. It looks like I created two containers, not just one. Why is that?
Docker images combine a set of file system layers to form a single file system using something
called a Union file system. From a user perspective,
the combined layers appear as a single file system. Docker image layers are read-only, so when you
run a container via the docker run command, Docker creates a read-write layer for you to work with.
In our example, we specify the ubuntu image in our docker run command, so the file system layers
in the ubuntu image form the starting point for our container.
Now to answer why there are two containers. Each time you run a container, Docker creates a new
read-write layer. The read-write layer for one container is not shared with other containers.
Becase we used the docker run command twice, we now have two separate containers, each with a
distinct read-write layer. While the containers do share a common set of base layers from the
ubuntu image, the read-write layer for each container is unique.
How do I access the data in my old container?
Now that we know our containers still exist, let's retrieve the contents of our message file.
Looking back at the output of docker ps -a, we see that the second container, the one with
/bin/bash in the command column, is the container that has /message.txt file. We can reference
this container by it's ID, 1f608dc4e5b4, or by its name, insane_wright.
For the rest of the examples, I will refer to containers by name because I find the container names
Docker generates both more entertaining and easier to remember than container IDs. If you prefer,
you can name your container via the --name argument of the docker run command.
Let's restart our container to view our message file.
~$ docker start -ai insane_wright
root@1f608dc4e5b4:/# cat /message.txt
hello docker
root@1f608dc4e5b4:/# exit
I used the docker start command with the -a (attach) argument to view stdout and the -i
(interactive) argument to write to stidn. The container starts with the same read-write layer
in the same state as when we exited the container. Hooray! We can view our file again.
How do I save and reusue the data in my container?
We found our data, but it would be useful to save and access our data from other containers.
We can do this by creating an image from our container with the docker commit command.
~$ docker commit -a "Jordan Bach" -m "saved my message" insane_wright jbgo/message:v0.0.1
1c9195e4c24c894a274f857a60b46fb828ee70ff0e78d18017dfb79c5bf68409
This command turns the read-write layer from our insane_wright container into a read-only layer.
Docker places this layer on top of the layers from the ubuntu image on which we originally based the
container. This new image is stored in the jbgo/message repository (more on repositories later)
with the v0.0.1 tag. Tags are useful for distinguish different versions of the same image.
If you are following along, you should replace jbgo with your
Docker Hub user name. If you don't have a
Docker Hub account yet, it is definitely worth signing up if you plan to work with docker at all
in the future.
Let's verify that our new image exists.
root@docker-tmp:~# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
jbgo/message v0.0.1 1c9195e4c24c 18 seconds ago 188.3 MB
ubuntu latest b7cf8f0d9e82 4 days ago 188.3 MB
Awesome! I see two images. The first image is our new jbgo/message:v0.0.1 image, and the second
image is ubuntu:latest, which Docker automatically downloaded from
Docker Hub when we ran our first container.
Now we can start containers with our new image and they will have the /message.txt file in them.
~$ docker run -it jbgo/message:v0.0.1 cat /message.txt
hello docker
If we list our containers again, we can see our new container using our new image.
~$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
01bd4c098203 jbgo/message:v0.0.1 "cat /message.txt" 10 seconds ago Exited (0) 9 seconds ago hopeful_lovelace
b3d8c3ef31a0 ubuntu:latest "cat /message.txt" 33 minutes ago Exited (1) 33 minutes ago admiring_lalande
1f608dc4e5b4 ubuntu:latest "/bin/bash" 34 minutes ago Exited (0) 15 minutes ago insane_wright
I find it interesting that docker ps shows the most recently created container first. If you
only want to see the most recent container, you can use docker ps with the -l (latest) argument.
How do I backup my image?
At this point, we have a custom docker image that contains a message file. We can run new containers with this image and those containers can read and write to the message file. But what if we lose our docker host to a crash or a security compromise? How can we still keep our image?
We need a docker registry - a repository where we can upload and download docker images. Docker provides the Docker Hub registry, which is free for public use and paid for private use. We can also run our own registry if we desire. I will use the Docker Hub registry, since it is free and ready to use. If you're following along, remember to replace jbgo with your actual Docker Hub username in the upcoming examples.
Let's upload our image to the Docker registry.
~$ docker push jbgo/message
The push refers to a repository [jbgo/message] (len: 1)
1c9195e4c24c: Image push failed
Please login prior to push:
Username: jbgo
Password:
Email: [email protected]
WARNING: login credentials saved in /root/.dockercfg.
Login Succeeded
The push refers to a repository [jbgo/message] (len: 1)
1c9195e4c24c: Image already exists
b7cf8f0d9e82: Image successfully pushed
2c014f14d3d9: Image successfully pushed
a62a42e77c9c: Image successfully pushed
706766fe1019: Image successfully pushed
Digest: sha256:b2a98b19e06a4df91150f8a2cd47a6e440cbc13b39f1fc235d46f97f631ad117
Because this was my first time pushing to the docker registry on this host, docker prompts me for my login credentials.
Here is my image on Docker Hub: https://registry.hub.docker.com/u/jbgo/message/
You can download this image on other docker hosts with the docker pull command.
However, you may notice a problem. I tagged my image incorrectly, and docker can't find my image.
~$ docker pull jbgo/message
Pulling repository jbgo/message
FATA[0007] Tag latest not found in repository jbgo/message
Let's fix it. I did not specify a tag in my pull command (as in docker pull jbgo/message:v0.0.1),
so docker attempts
to use the latest tag by default. While I could pull a specific tag, for now I prefer to use the
default, so I will create a latest tag for my image. The v0.0.1 tag will continue to exist and
may be beneficial for people who don't always want to use the latest version of my image.
~$ docker tag jbgo/message:v0.0.1 jbgo/message:latest
Now I can push my image with the new tag.
~$ docker push jbgo/message:latest
The push refers to a repository [jbgo/message] (len: 1)
1c9195e4c24c: Image already exists
b7cf8f0d9e82: Image already exists
2c014f14d3d9: Image already exists
a62a42e77c9c: Image already exists
706766fe1019: Image already exists
Digest: sha256:cc2fbbb2029c6402cea639b2454da08ef05672da81176ae97f57d4f51be19fc3
Notice that the push was faster this time because docker recognizes image layers that already exist in the registry and only uploads new layers. Let's try to pull the image again to see if it works.
~ $ docker pull jbgo/message
latest: Pulling from jbgo/message
1c9195e4c24c: Already exists
706766fe1019: Already exists
a62a42e77c9c: Already exists
2c014f14d3d9: Already exists
b7cf8f0d9e82: Already exists
Digest: sha256:cc2fbbb2029c6402cea639b2454da08ef05672da81176ae97f57d4f51be19fc3
Status: Downloaded newer image for jbgo/message:latest
That works, but because we're on the same host, docker did not actually download anything.
For fun I'm going to spin up a new docker host and download the image
to the new host. You can do this with docker-machine or your favorite cloud provider, or you
can skip this step if you trust me.
docker2:~$ docker pull jbgo/message
latest: Pulling from jbgo/message
706766fe1019: Pull complete
a62a42e77c9c: Pull complete
2c014f14d3d9: Pull complete
b7cf8f0d9e82: Pull complete
1c9195e4c24c: Already exists
Digest: sha256:cc2fbbb2029c6402cea639b2454da08ef05672da81176ae97f57d4f51be19fc3
Status: Downloaded newer image for jbgo/message:latest
Based on the command output, we see that docker actually downloaded the image layers this time. Because computers don't operate on blind faith, let's once again verify our message file is present.
docker2:~$ docker run jbgo/message cat /message.txt
hello docker
Success!
What do I do with containers when I'm done with them?
Ok, I think I'm beginning to understand how to work with containers and images. However, I'm creating a bunch of containers and images I don't need anymore. I would like to get rid of these to free up some disk space and unclutter the output of my docker commands.
Below is how you can remove one or more containers.
~$ docker rm hopeful_lovelace insane_wright admiring_lalande
hopeful_lovelace
insane_wright
admiring_lalande
Be sure to verify that they are actually gone.
~$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
At this point, you can no longer recover these contaienrs. Fortunately, we already created an image
for the container we cared about and pushed the image to a docker registry. As long as we don't
explicity docker rm a container we can still recover the container and its file system contents.
This was one of my major concerns when I first tried docker, but now that I know more about how
docker containers and images work, I no longer worry about what happens to my containers when I am
done with them.
Wait a minute! Why did docker push more than one image?
If you are paying attention, you noticed that when we pushed the jbgo/message image the first time, docker uploaded more than one image to the registry. How could this be?
~$ docker push jbgo/message:v0.0.1
...
1c9195e4c24c: Image already exists
b7cf8f0d9e82: Image successfully pushed
2c014f14d3d9: Image successfully pushed
a62a42e77c9c: Image successfully pushed
706766fe1019: Image successfully pushed
We can use the docker history command to give us some clues.
~$ docker history jbgo/message:v0.0.1
IMAGE CREATED CREATED BY SIZE
1c9195e4c24c 33 minutes ago /bin/bash 108 B
b7cf8f0d9e82 4 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
2c014f14d3d9 4 days ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/ 1.895 kB
a62a42e77c9c 4 days ago /bin/sh -c echo '#!/bin/sh' > /usr/sbin/polic 194.5 kB
706766fe1019 4 days ago /bin/sh -c #(nop) ADD file:777fad733fc954c0c1 188.1 MB
It seems that we did only create one image. By using docker inspect on the first image, we can
see that I'm the author and that the commit message is present as a comment on the image.
~$ docker inspect 1c9195e4c24c
[{
"Architecture": "amd64",
"Author": "Jordan Bach",
"Comment": "saved my message",
"Config": {
What does the history look like for the ubuntu image?
~$ docker history ubuntu
IMAGE CREATED CREATED BY SIZE
b7cf8f0d9e82 4 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
2c014f14d3d9 4 days ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/ 1.895 kB
a62a42e77c9c 4 days ago /bin/sh -c echo '#!/bin/sh' > /usr/sbin/polic 194.5 kB
706766fe1019 4 days ago /bin/sh -c #(nop) ADD file:777fad733fc954c0c1 188.1 MB
Ah, mystery solved! Those are the mysterious images in my image. It appears that when I pushed to my Docker Hub account, docker copied the ubuntu layers to my jbgo/message repository. The copy ensures that my image always contains the exact same layers. Otherwise, my image could break if docker did not copy all of the layers into my repository and a new, incompatible version of the ubuntu image was published.
How do I delete an image?
We already deleted our containers, but we still have images we don't want anymore. Let's delete those, too.
docker2:~$ docker rmi jbgo/message
Error response from daemon: Conflict, cannot delete 1c9195e4c24c because the container 2ea39e64a130 is using it, use -f to force
FATA[0000] Error: failed to remove one or more images
Well, crap! I don't have any containers running. But I do have some stopped containers.
docker2:~$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2ea39e64a130 jbgo/message:latest "cat /message.txt" 2 minutes ago Exited (0) 2 minutes ago lonely_jones
This is for your own good. If docker allowed you to delete an image that a container uses, you would never be able to start that container again. In our case, we really want to delete the image, so let's delete the container first.
docker2:~$ docker rm lonely_jones
lonely_jones
Now we can delete the image. Notice the command is docker rmi for remove image as opposed to the
shorter docker rm. which is just for containers.
docker2:~$ docker rmi jbgo/message
Untagged: jbgo/message:latest
Deleted: 1c9195e4c24c894a274f857a60b46fb828ee70ff0e78d18017dfb79c5bf68409
Deleted: b7cf8f0d9e82c9d96bd7afd22c600bfdb86b8d66c50d29164e5ad2fb02f7187b
Deleted: 2c014f14d3d95811df672ddae2af376f9557f6b8f5623e3e3f8c5ca3f6ff42e6
Deleted: a62a42e77c9c3626118dc411092d23cf658220261acdafe21a7589a8eeba627e
Deleted: 706766fe101906a1a6628173c2677173a5f8c6c469075083f3cf3a8f5e5eb367
As always, let's verify that the image is really gone.
docker2:~$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
Conclusion
Congratulations! That was a lot of poking and prodding to understand how docker behaves. There are many more wonderful, useful features of docker that I hope to explore with you soon.
