Introduction to docker, containers, docker compose, dockerfile

In this article will cover up basic understanding of Containerization and why we even need docker, and how it add value in application development cycle. And will also cover basic tutorial and commands to get started with docker.

Photo by Ian Taylor on Unsplash

what is docker ?

Docker is essentially a toolkit that enables developers to build, deploy, run, update and stop containers using simple commands and work-saving automation through a single API.

what is containers ?

It is lightweight software components that bundles the application, its dependencies, and its configurations in single image. It is isolated from all other processes on the host system. It can be run and access from local machine, virtual machine or deploy of cloud mean it is portable without having dependencies on host machine. If there are multiple containers with same application it can be isolated from each other for testing, development and other purposes.

why needs of Containerization ?

To ease the problem of infrastructure maintains, updation, changes for application deployment, testing of application, efficiently and fast deployment of applications, efficiently use of available resources and reduce the cost of infrastructure for applications etc..

Container vs Image:

1. Images can exist without containers, but container needs to run an image. Container is a running environment of image. So, container have dependence on image and use image to construct a run-time environment and run an application.
2. A docker image is unchangeable file that bundles the source code, libraries, dependencies, and other files needs to run an application.
3. Application image: Postgres, mongodb, redis, etc..

Difference between docker and virtual machine:

As bother docker and virtual machine are virtualisation tools.
Main difference between both is that docker virtualize only application layer and uses host machine kernel as docker doesn’t have its kernel. But virtual machine have application layer and os kernel of it hence it virtualise complete operating system.
Docker image is much smaller in it size compare to virtual machine.
Docker container start and run much faster compare to virtual machine.
VM of any OS can run on any OS host but not the same case for docker as if linux docker image try to run on windows os host which is not possible.

Benefits of docker:

1. Reproducibility
2. Local experiments
3. Integration tests (CI/CD) like GitHub actions
4. Running pipelines on the cloud (AWS Lambda, google functions)
5. Spark
6. Serverless (AWS Lambda, Google functions)

Solution that Docker provides:

1. Build Image: Consistently package everything your application needs to run.
2. Ship Image: Easily ships these images to runtimes in cloud or on your local developer machine.
3. Run Image: Easily and consistently execute your application.

Steps to install and get started with commands of docker:

1. Install docker from official platform based on operating system like windows, mac os etc..
   https://docs.docker.com/get-docker/
2. Once installation is complete run this basic command in terminal/command prompt to check if installation is done properly:

$ docker run -d -p 80:80 docker/getting-started

Here’s some more info about command:

• -d - run the container in detached mode (in the background)
• -p 80:80 - map port 80 of the host to port 80 in the container
• docker/getting-started - the image to use

3. Can find different official images with details on docker hub for different type of application development attached link below and command to download any image from docker hub:
https://hub.docker.com/

Docker pull syntax:
$ docker pull image_name 
Example:
$ docker pull postgres

4. Commands to run/stop/restart or detail about running images in docker:

List running containers and details of containers:
syntax:
$ docker ps
systax:
$ docker ps -a
Docker run in detached mode:
syntax:
$ docker run -d image_name
Example:
$ docker run -d Postgres
Restart/stop container:
syntax:
$ docker start/stop container_id
Example:
$ docker start/stop c332b396610a

5. Commands to remove/kill the container or image in docker:

To kill/remove all existing containers:
$ docker rm $(docker ps -a -q)
To kill/remove container:
$ Docker rm container_id
To kill/remove image:
$ Docker rmi image_id

6. Commands to run terminal of docker container:

To get the terminal of docker container: -it stands for interactive terminal
Syntax:
$ docker exec -it container_id /bin/bash
or
$ docker exec -it container_id /bin/sh
To check container id:
$ docker ps
Inside the terminal of docker container to check environment variable:
syntax:
$ Env
To exit the terminal of docker container:
$ exit

7. Multiple container can run on host machine. Container and host machine can be connected by port binding. Container can have duplicate ports but host can’t have because of conflicts. Port binding can be done at time of docker run command.

Command for port binding between host and container:
syntax:
$ docker run -p host_port: container port container_image_name
Example: 
$ docker run -p 6000:6379 c332b396610a

8. To check network in docker and create new network to connect multiple container with each other.

To check networks of docker:
syntax:
$ docker network ls
To create network in docker:
syntax:
$ docker network create network_name
Example:
$ docker network create mongo-network

docker example command to use created network, set port binding, set name of container, define password and user of database in image.

docker run -d \
-p 27017:27017 \
-e MONGO_INITDB_ROOT_USERNAME=admin \
-e MONGO_INITDB_ROOT_PASSWORD=password \
--name mongodb \
--net mongo-network \
mongo

Next Topic in docker series is Docker compose:

An application can have multiple container running different services. It could be repetitive and tedious to start and manage various container manually, so docker compose is tool to define and running multiple container docker applications.

It is very structured way to maintain common docker command in proper format and it is very easy to edit script. Docker compose take care of creating a common network also.

Have attached example docker script where we download and run our application image from docker registry like AWS EKS and run on specific port and install, run mongo and mongo-express image for database setup for our application and setup persistent data volume in it.

Command to run docker compose:

Docker compose start command:
$ docker-compose -f mongo.yaml up
Docker compose stop command:
$ docker-compose -f mongo.yaml down

Docker File:

Docker file is a blueprint for creating docker image.

Example script of dockerfile:

• Docker file start by bashing it on another image( install node)
• Run — execute any linux command (inside container)
• Copy — command executed on host to copy files from host to container
• CMD — entry point command

To build a docker image for the application:
$ docker build -t my-app:1.0 .
The dot “.” at the end of the command denotes location of the Dockerfile.

Docker volumes:

It used for data persistence. Folder in physical host system is mounted into the virtual file system of docker. Data automatically get replicated.

3 type of volume in docker:

1. Host volumes: We decide where on the host file system the reference is made.

syntax:
$ docker run -v host path:container path
Example: 
$ docker run -v /home/mount/data:/var/lib/mysql/data

2. Anonymous volumes: For each container a folder is generated that gets mounted and it is automatically created by docker.

syntax:
$ docker run -v container path
Example:
$ docker run -v /var/lib/mysql/data

3. Named volumes: we can reference the volume by name.recommend to use in production.

syntax:
$ docker run -v volume_name:container path
Example:
$ docker run -v storage1:/var/lib/mysql/data

Conclusion:

Tried to cover all major fundamental concept around docker, Containerization of application and basic practical tutorial to have hands-on exposure with docker. Have attached link of project of developing application using docker.
https://github.com/bharatc9530/Data-Engineering/tree/main/docker-project

Reference:

1. https://docs.docker.com/get-docker/
2. https://hub.docker.com/

Thank you for reading. Please let me know if you have any feedback.

I welcome feedback and constructive criticism and can be reached on Linkedin.