Home · Book Reports · 2024 · Docker Book, The

designates my notes. / designates important. / designates very important.

Thoughts

Pages from pdf.

Exceptional Quotes

Table of Contents

• 01: Introduction
• 02: Installing Docker
• 03: Getting Started with Docker
• 04: Working with Docker images and repositories
• 05: Testing with Docker
• 06: Building services with Docker
• 07: Docker Orchestration and Service Discovery
• 08: Using the Docker API
• 09: Getting help and extending Docker

· Chapter 01 - Introduction

page 19:

• Docker also encourages service-oriented and microservices architectures. Docker recommends that each container run a single application or process.

• This should be interesting…

page 22:

• Images are the building blocks of the Docker world. You launch your containers from images. Images are the “build” part of Docker’s life cycle. They have a layered format, using Union file systems, that are built step-by-step using a series of instructions. For example:

  • Add a file. • Run a command. • Open a port.

• You can consider images to be the “source code” for your containers. They are highly portable and can be shared, stored, and updated.

• Docker stores the images you build in registries. There are two types of registries: public and private.

page 23:

• Docker helps you build and deploy containers inside of which you can package your applications and services. As we’ve just learned, containers are launched from images and can contain one or more running processes. You can think about images as the building or packing aspect of Docker and the containers as the running or execution aspect of Docker.

• A Docker container is: • An image format. • A set of standard operations. • An execution environment.

page 24:

• In addition to solitary containers we can also run Docker containers in stacks and clusters, what Docker calls swarms. The Docker ecosystem contains two more tools:

• Docker Compose - which allows you to run stacks of containers to represent application stacks, for example web server, application server and database server containers running together to serve a specific application.

• Docker Swarm - which allows you to create clusters of containers, called swarms, that allow you to run scalable workloads.

page 26:

• the recreation of state may often be cheaper than the remediation of state.

· Chapter 02 - Installing Docker

· Chapter 3 - Getting Started with Docker

page 62:

page 75:

• If your container has stopped because of a failure you can configure Docker to restart it using the restart flag. The –restart flag checks for the container’s exit code and makes a decision whether or not to restart it. The default behavior is to not restart containers at all.

page 79:

• There isn’t currently a way to delete all containers, but you can slightly cheat with a command like the following:

# Listing 3.37: Deleting all containers

sudo docker rm -f `sudo docker ps -a -q`

• This command will list all of the current containers using the docker ps command. The -a flag lists all containers, and the -q flag only returns the container IDs rather than the rest of the information about your containers. This list is then passed to the docker rm command, which deletes each container. The -f flag force removes any running containers. If you’d prefer to protect those containers, omit the flag.

· Chapter 04 - Working with Docker images and repositories

page 83:

page 84:

• This pattern is traditionally called “copy on write”

• Local images live on our local Docker host in the /var/lib/docker directory. Each image will be inside a directory named for your storage driver; for example, aufs or devicemapper. You’ll also find all your containers in the /var/lib/docker/containers directory.

page 94:

• The first method of creating images uses the docker commit command. You can think about this method as much like making a commit in a version control system. We create a container, make changes to that container as you would change code, and then commit those changes to a new image.

page 97:

• We don’t recommend the docker commit approach. Instead, we recommend that you build images using a definition file called a Dockerfile and the docker build command. The Dockerfile uses a basic DSL (Domain Specific Language) with in- structions for building Docker images. We recommend the Dockerfile approach over docker commit because it provides a more repeatable, transparent, and idem- potent mechanism for creating images.

page 98:

# Listing 4.21: Creating a sample repository

mkdir static_web
cd static_web
touch Dockerfile

# Listing 4.22: Our first Dockerfile

# Version: 0.0.1
FROM ubuntu:18.04
LABEL maintainer="james@example.com"
RUN apt-get update; apt-get install -y nginx
RUN echo 'Hi, I am in your container' \
    >/var/www/html/index.html
EXPOSE 80

• The Dockerfile contains a series of instructions paired with arguments. Each instruction, for example FROM, should be in upper-case and be followed by an argument: FROM ubuntu:18.04. Instructions in the Dockerfile are processed from the top down, so you should order them accordingly.

page 99:

• Docker runs a container from the image.

• An instruction executes and makes a change to the container.

• Docker runs the equivalent of docker commit to commit a new layer.

• Docker then runs a new container from this new image.

• The next instruction in the file is executed, and the process repeats until all instructions have been executed.

• This means that if your Dockerfile stops for some reason (for example, if an instruction fails to complete), you will be left with an image you can use. This is highly useful for debugging: you can run a container from this image interactively and then debug why your instruction failed using the last image created.

page 100:

• By default, the RUN instruction executes inside a shell using the command wrapper /bin/sh -c. If you are running the instruction on a platform without a shell or you wish to execute without a shell (for example, to avoid shell string munging), you can specify the instruction in exec format:

# Listing 4.23: A RUN instruction in exec form

RUN [ "apt-get", " install", "-y", "nginx" ]

• We use this format to specify an array containing the command to be executed and then each parameter to pass to the command.

• Next, we’ve specified the EXPOSE instruction, which tells Docker that the application in this container will use this specific port on the container. That doesn’t mean you can automatically access whatever service is running on that port (here, port 80) on the container. For security reasons, Docker doesn’t open the port automatically, but waits for you to do it when you run the container using the docker run command.

• You can expose ports at run time with the docker run command with the --expose option.

page 102:

# Listing 4.24: Running the Dockerfile

cd static_web
sudo docker build -t="jamtur01/static_web" .

page 103:

# Listing 4.25: Tagging a build

sudo docker build -t="jamtur01/static_web:v1" .

• If you don’t specify any tag, Docker will automatically tag your image as latest.

• The trailing . tells Docker to look in the local directory to find the Dockerfile. You can also specify a Git repository as a source for the Dockerfile as we see here:

# Listing 4.26: Building from a Git repository

sudo docker build -t="jamtur01/static_web:v1" \
    github.com/turnbullpress/docker-static_web

• Here Docker assumes that there is a Dockerfile located in the root of the Git repository.

page 104:

• If a file named .dockerignore exists in the root of the build context then it is interpreted as a newline-separated list of exclusion patterns. Much like a .gitignore file it excludes the listed files from being treated as part of the build context, and therefore prevents them from being uploaded to the Docker daemon. Globbing can be done using Go’s filepath.

page 106:

# Listing 4.30: Bypassing the Dockerfile build cache

sudo docker build --no-cache -t="jamtur01/static_web" .

page 111:

# Listing 4.37: The docker port command

sudo docker port 6751b94bb5c0 80

> 0.0.0.0:49154

• We’ve specified the container ID and the container port for which we’d like to see the mapping, 80, and it has returned the mapped port, 49154.

# Listing 4.39: Exposing a specific port with -p

sudo docker run -d -p 80:80 --name static_web_80 jamtur01 static_web nginx -g "daemon off;"

• This will bind port 80 on the container to port 80 on the local host. It’s important to be wary of this direct binding: if you’re running multiple containers, =only one container can bind a specific port on the local host. This can limit Docker’s flexibility.

page 112:

# Listing 4.41: Binding to a specific interface

sudo docker run -d -p 127.0.0.1:80:80 --name static_web_lb jamtur01/static_web nginx -g "daemon off;"

• Here we’ve bound port 80 of the container to port 80 on the 127.0.0.1 interface on the local host. We can also bind to a random port using the same structure.

# Listing 4.42: Binding to a random port on a specific interface

sudo docker run -d -p 127.0.0.1::80 --name static_web_random jamtur01/static_web nginx -g "daemon off;"

page 113:

• Docker also has a shortcut, -P, that allows us to publish all ports we’ve exposed via EXPOSE instructions in our Dockerfile.

# Listing 4.43: Exposing a port with docker run

sudo docker run -d -P --name static_web_all jamtur01/static_web nginx -g "daemon off;"

page 114:

# Listing 4.45: Specifying a specific command to run
sudo docker run -i -t jamtur01/static_web /bin/true

• This would be articulated in the Dockerfile as: (with CMD instruction)

# Listing 4.46: Using the CMD instruction

CMD ["/bin/true"]

page 115:

# Listing 4.47: Passing parameters to the CMD instruction

CMD ["/bin/bash", "-l"]

page 117:

• Closely related to the CMD instruction, and often confused with it, is the ENTRYPOINT instruction.

• The ENTRYPOINT instruction provides a command that isn’t as easily overridden. Instead, any arguments we specify on the docker run command line will be passed as arguments to the command specified in the ENTRYPOINT.

# Listing 4.51: Specifying an ENTRYPOINT

ENTRYPOINT ["/usr/sbin/nginx"]

# Listing 4.52: Specifying an ENTRYPOINT parameter

ENTRYPOINT ["/usr/sbin/nginx", "-g", "daemon off;"]

page 118:

# Listing 4.54: Using docker run with ENTRYPOINT

sudo docker run -t -i jamtur01/static_web -g "daemon off;"

• We’ve rebuilt our image and then launched an interactive container. We specified the argument -g “daemon off;”. This argument will be passed to the command specified in the ENTRYPOINT instruction, which will thus become /usr/sbin/nginx -g “daemon off;”.

page 119:

# Listing 4.55: Using ENTRYPOINT and CMD together

ENTRYPOINT ["/usr/sbin/nginx"]
CMD ["-h"]

• Now when we launch a container, any option we specify will be passed to the Nginx daemon; for example, we could specify -g “daemon off”; as we did above to run the daemon in the foreground. If we don’t specify anything to pass to the container, then the -h is passed by the CMD instruction and returns the Nginx help text: /usr/sbin/nginx -h.

• This allows us to build in a default command to execute when our container is run combined with overridable options and flags on the docker run command line.

• The WORKDIR instruction provides a way to set the working directory for the con- tainer and the ENTRYPOINT and/or CMD to be executed when a container is launched from the image.

page 120:

# Listing 4.56: Using the WORKDIR instruction

WORKDIR /opt/webapp/db
RUN bundle install
WORKDIR /opt/webapp
ENTRYPOINT [ "rackup" ]

• Here we’ve changed into the /opt/webapp/db directory to run bundle install and then changed into the /opt/webapp directory prior to specifying our ENTRYPOINT instruction of rackup.

• You can override the working directory at runtime with the -w flag, for example:

# Listing 4.57: Overriding the working directory

sudo docker run -ti -w /var/log ubuntu pwd

> /var/log

• The ENV instruction is used to set environment variables during the image build process. For example:

# Listing 4.58: Setting an environment variable in Dockerfile

ENV RVM_PATH /home/rvm/

page 121:

# Listing 4.61: Setting multiple environment variables using ENV

ENV RVM_PATH=/home/rvm RVM_ARCHFLAGS="-arch i386"

• We can also use these environment variables in other instructions.

# Listing 4.62: Using an environment variable in other Dockerfile instructions

ENV TARGET_DIR /opt/app
WORKDIR $TARGET_DIR

• Here we’ve specified a new environment variable, TARGET_DIR, and then used its value in a WORKDIR instruction.

page 123:

• The USER instruction specifies a user that the image should be run as; for example:

# Listing 4.65: Using the USER instruction

USER nginx

page 124:

• The VOLUME instruction adds volumes to any container created from the image. A volume is a specially designated directory within one or more containers that bypasses the Union File System to provide several useful features for persistent or shared data:

  • Volumes can be shared and reused between containers. • A container doesn’t have to be running to share its volumes. • Changes to a volume are made directly. • Changes to a volume will not be included when you update an image. • Volumes persist even if no containers use them.

# Listing 4.67: Using the VOLUME instruction

VOLUME ["/opt/project"]

• This would attempt to create a mount point /opt/project to any container created from the image.

page 125:

# Listing 4.68: Using multiple VOLUME instructions

VOLUME ["/opt/project", "/data" ]

• The ADD instruction adds files and directories from our build environment into our image; for example, when installing an application. The ADD instruction specifies a source and a destination for the files, like so:

# Listing 4.69: Using the ADD instruction

ADD software.lic /opt/application/software.lic

• When ADD’ing files Docker uses the ending character of the destination to deter- mine what the source is. If the destination ends in a /, then it considers the source a directory. If it doesn’t end in a /, it considers the source a file.

page 126:

• Lastly, the ADD instruction has some special magic for taking care of local tar archives. If a tar archive (valid archive types include gzip, bzip2, xz) is specified as the source file, then Docker will automatically unpack it for you:

page 127:

• The COPY instruction is closely related to the ADD instruction. The key difference is that the COPY instruction is purely focused on copying local files from the build context and does not have any extraction or decompression capabilities.

page 128:

• The LABEL instruction adds metadata to a Docker image. The metadata is in the form of key/value pairs. Let’s see an example.

# Listing 4.73: Adding LABEL instructions

LABEL version="1.0"
LABEL location="New York" type="Data Center" role="Web Server"

page 129:

• The STOPSIGNAL instruction sets the system call signal that will be sent to the container when you tell it to stop. This signal can be a valid number from the kernel syscall table, for instance 9, or a signal name in the format SIGNAME, for instance SIGKILL.

• The ARG instruction defines variables that can be passed at build-time via the docker build command. This is done using the –build-arg flag. You can only specify build-time arguments that have been defined in the Dockerfile.

# Listing 4.75: Adding ARG instructions

ARG build
ARG webapp_user=user

• The second ARG instruction sets a default, if no value is specified for the argument at build-time then the default is used. Let’s use one of these arguments in a docker build now.

page 130:

# Listing 4.76: Using an ARG instruction

docker build --build-arg build=1234 -t jamtur01/webapp .

• As the jamtur01/webapp image is built the build variable will be set to 1234 and the webapp_user variable will inherit the default value of user.

page 131:

• The SHELL instruction allows the default shell used for the shell form of commands to be overridden. The default shell on Linux is ‘["/bin/sh", “-c”] and on Windows is [“cmd”, “/S”, “/C”].

• The HEALTHCHECK instruction tells Docker how to test a container to check that it is still working correctly. This allows you to check things like a web site being served or an API endpoint responding with the correct data, allowing you to identify issues that appear, even if an underlying process still appears to be running normally.

• When a container has a health check specified, it has a health status in addition to its normal status. You can specify a health check like:

# Listing 4.78: Specifying a HEALTHCHECK instruction

HEALTHCHECK --interval=10s --timeout=1m --retries=5 CMD curl http://localhost || exit 1

page 133:

• The ONBUILD instruction adds triggers to images. A trigger is executed when the image is used as the basis of another image (e.g., if you have an image that needs source code added from a specific location that might not yet be available, or if you need to execute a build script that is specific to the environment in which the image is built).

page 135:

# Listing 4.84: A new ONBUILD image Dockerfile

FROM ubuntu:18.04
LABEL maintainer="james@example.com"
RUN apt-get update; apt-get install -y apache2
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_PID_FILE /var/run/apache2.pid
ENV APACHE_RUN_DIR /var/run/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
ONBUILD ADD . /var/www/
EXPOSE 80
ENTRYPOINT ["/usr/sbin/apachectl"]
CMD ["-D", "FOREGROUND"]

• We now have an image with an ONBUILD instruction that uses the ADD instruction to add the contents of the directory we’re building from to the /var/www/ directory in our image. This could readily be our generic web application template from which I build web applications.

· Chapter 05 - Testing with Docker

page 157:

# Listing 5.9: Running our first Nginx testing container

sudo docker run -d -p 80 --name website -v $PWD/website:/var/www/html/website jamtur01/nginx nginx

page 158:

• the -v option is new. This new option allows us to create a volume in our container from a directory on the host.

• Volumes are specially designated directories within one or more containers that bypass the layered Union File System to provide persistent or shared data for Docker. This means that changes to a volume are made directly and bypass the image. They will not be included when we commit or build an image.

• Volumes can also be shared between containers and can persist even when containers are stopped.

• We can also specify the read/write status of the container directory by adding either rw or ro after that directory

page 163:

# Listing 5.16: Dockerfile for our Sinatra container

FROM ubuntu:18.04
LABEL maintainer="james@example.com"
ENV REFRESHED_AT 2016-06-01
RUN apt-get update -yqq; apt-get -yqq install ruby ruby-dev build -essential redis-tools
RUN gem install --no-rdoc --no-ri sinatra json redis
RUN mkdir -p /opt/webapp
EXPOSE 4567
CMD [ "/opt/webapp/bin/webapp" ]

• You can see that we’ve created another Ubuntu-based image, installed Ruby and RubyGems, and then used the gem binary to install the sinatra, json, and redis gems. The sinatra and json gems contain Ruby’s Sinatra library and support for JSON. The redis gem we’re going to use a little later on to provide integration to a Redis database.

• We’ve also created a directory to hold our new web application and exposed the default WEBrick port of 4567.

• Finally, we’ve specified a CMD of /opt/webapp/bin/webapp, which will be the bi- nary that launches our web application.

page 174:

• Docker’s internal network is not an overly flexible or powerful solution. We’re mostly going to discuss it to introduce you to how Docker networking functions. We don’t recommend it as a solution for connecting containers.

• The more realistic method for connecting containers is Docker Networking.

• Docker Networking can connect containers to each other across different hosts.

• Containers connected via Docker Networking can be stopped, started or restarted without needing to update connections.

• With Docker Networking you don’t need to create a container before you can connect to it. You also don’t need to worry about the order in which you run containers and you get internal container name resolution and discovery inside the network.

page 175:

• internal networking.

• Every Docker container is assigned an IP address, provided through an interface created when we installed Docker. That interface is called docker0. Let’s look at that interface on our Docker host now.

page 176:

• Every time Docker creates a container, it creates a pair of peer interfaces that are like opposite ends of a pipe (i.e., a packet sent on one will be received on the other). It gives one of the peers to the container to become its eth0 interface and keeps the other peer, with a unique name like vethec6a, out on the host machine. You can think of a veth interface as one end of a virtual network cable. One end is plugged into the docker0 bridge, and the other end is plugged into the container. By binding every veth* interface to the docker0 bridge, Docker creates a virtual subnet shared between the host machine and every Docker container.

page 177:

• But there’s one other piece of Docker networking that enables this connectivity: firewall rules and NAT configuration allow Docker to route between containers and the host network.

page 178:

# Listing 5.43: Docker iptables and NAT

sudo iptables -t nat -L -n

page 180:

• So, while this initially looks like it might be a good solution for connecting our containers together, sadly, this approach has two big rough edges: Firstly, we’d need to hard-code the IP address of our Redis container into our applications.

page 181:

• Secondly, if we restart the container, Docker changes the IP address. Let’s see this now using the docker restart command (we’ll get the same result if we kill our container using the docker kill command).

• Container connections are created using networks. This is called Docker Networking and was introduced in the Docker 1.9 release. Docker Networking allows you to setup your own networks through which containers can communicate. Essentially this supplements the existing docker0 network with new, user managed networks. Importantly, containers can now communicate with each across hosts and your networking configuration can be highly customizable.

page 182:

# Listing 5.49: Creating a Docker network

sudo docker network create app

page 183:

• In addition to bridge networks, which exist on a single host, we can also create overlay networks, which allow us to span multiple hosts. You can read more about overlay networks in the Docker multi-host network documentation.

• You can list all current networks using the docker network ls command.

page 184

• And you can remove a network using the docker network rm command.

# Listing 5.52: Creating a Redis container inside our Docker network

sudo docker run -d --net=app --name db jamtur01/redis

page 186:

# Listing 5.54: Linking our Redis container

cd sinatra
sudo docker run -p 4567 --net=app --name network_test -t -i jamtur01/sinatra /bin/bash
root@305c5f27dbd1:/#

• We’ve launched a container named network_test inside the app network. We’ve launched it interactively so we can peek inside to see what’s happening. As the container has been started inside the app network, Docker will have taken note of all other containers running inside that network and populated their addresses in local DNS. Let’s see this now in the network_test container.

• We first need the dnsutils and iputils-ping packages to get the nslookup and ping binaries respectively.

# Listing 5.56: DNS resolution in the network_test container

root@305c5f27dbd1:/# nslookup db
Server: 127.0.0.11
Address:127.0.0.11#53

Non-authoritative answer:
Name: db
Address: 172.18.0.2

• We see that using the nslookup command to resolve the db container it returns the IP address: 172.18.0.2. A Docker network will also add the app network as a domain suffix for the network, any host in the app network can be resolved by hostname.app, here db.app. Let’s try that now.

page 187:

# Listing 5.57: Pinging db.app in the network_test container

root@305c5f27dbd1:/# ping db.app
PING db.app (172.18.0.2) 56(84) bytes of data.
64 bytes from db (172.18.0.2): icmp_seq=1 ttl=64 time=0.290 ms

• In our case we just need the db entry to make our application function. To make that work our webapp’s Redis connection code already uses the db hostname.

page 188:

# Listing 5.58: The Redis DB hostname in code

redis = Redis.new(:host => 'db', :port => '6379')

page 190:

• You can also add already running containers to existing networks using the docker network connect command.

page 192:

• We can also disconnect a container from a network using the docker network disconnect command.

• Containers can belong to multiple networks at once so you can create quite complex networking models.

page 193:

• Section on using docker with Jenkins CI

· Chapter 06 - Building services with Docker

page 226:

• A volume is a specially designated directory within one or more containers that bypasses the Union File System to provide several useful features for persistent or shared data:

  • Volumes can be shared and reused between containers. • A container doesn’t have to be running to share its volumes. • Changes to a volume are made directly. • Changes to a volume will not be included when you update an image. • Volumes persist even when no containers use them.

• This allows you to add data (e.g., source code, a database, or other content) into an image without committing it to the image and allows you to share that data between containers.

page 227:

• Volumes live on your Docker host, in the /var/lib/docker/volumes directory. You can identify the location of specific volumes using the docker inspect command.

# Listing 6.11: Creating an Apache container
sudo docker run -d -P --volumes-from james_blog jamtur01/apache
09a570cc2267019352525079fbba9927806f782acb88213bd38dde7e2795407d

• This looks like a typical docker run, except that we’ve used a new flag: --volumes -from. The --volumes-from flag adds any volumes in the named container to the newly created container. This means our Apache container has access to the com- piled Jekyll site in the /var/www/html volume within the james_blog container we created earlier. It has that access even though the james_blog container is not running. As you’ll recall, that is one of the special properties of volumes. The container does have to exist, though.

page 231:

• the --rm flag, which is useful for single-use or throw- away containers. It automatically deletes the container after the process running in it is ended.

• This example could also work for a database stored in a volume or similar data. Simply mount the volume in a fresh container, perform your backup, and discard the container you created for the backup.

page 264:

# Listing 6.62: Using docker kill to send signals
sudo docker kill -s <signal> <container>

• This will send the specific signal you want (e.g., a HUP) to the container in question rather than killing the container.

• I didn’t take any notes on the docker-compose section.

· Chapter 07 - Docker Orchestration and Service Discovery

page 267:

• With Docker Compose, we define a set of containers to boot up, and their runtime properties, all defined in a YAML file. Docker Compose calls each of these containers “services” which it defines as:

• A container that interacts with other containers in some way and that has specific runtime properties.

• Again, very few notes takes because this isn’t what I am interested in. Focus is on ‘consul’.

page 317:

• Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single virtual Docker host.

· Chapter 08 - Using the Docker API

page 332:

• There are three specific APIs in the Docker ecosystem.

  • The Registry API - provides integration with the Docker registry, which stores our images.
  • The Docker Hub API - provides integration with the Docker Hub.
  • The Docker Engine API - provides integration with the Docker daemon.

page 333:

• Firstly, we need to remember the Engine API is provided by the Docker daemon. By default, the Docker daemons binds to a socket, unix:///var/run/docker.sock, on the host on which it is running. The daemon runs with root privileges so as to have the access needed to manage the appropriate resources.

• The docker group is root-equivalent and should be limited to those users and applications that absolutely need it.

page 334:

# Listing 8.1: Querying the Docker API locally
curl --unix-socket /var/run/docker.sock http:/info
{"ID":"PH4R:BT7H:44F6:GQGP:FS2O:7OZO:HQ2P:NSVF:MK27:NBGZ:N3VP:
    K2O5","Containers":3,"ContainersRunning":3,"ContainersPaused"
    :0,"ContainersStopped":0,"Images":3,"
. . .
}

• Some stuff on securing the API with SSL/TLS.

· Chapter 09 - Getting help and extending Docker