有时候我们在 Mac OS X  或 Windows 平台下需要开发以 Linux 为运行时的应用，IDE 或可直接使用 Docker 容器，或 SSH 远程连接。本地命令行下操作虽然可以用 docker exec 连接正在运行的容器，但 IDE 远程连接的话 SSH 总是一种较为通用的连接方式，所以我们希望做一个能进行 SSH 连接的 Docker 容器。因为是本地运行的 Docker，我们想直接用 root 连接，以获得在容器中最大的运行权限。下面以 ubuntu:2004 基础镜像为例，看如何安装启用 ssh 服务以及允许 root 连接。

创建允许 root + 密码登陆的镜像

我们创建一个基本的 Dockerfile 文件，内容为 Read More

不觉一晃还是在五年前记录过一篇 Dockerfile 中命令的两种书写方式的区别，其中提到过 Dockerfile 中可选择用 ENTRYPOINT 或 CMD 来启动进程，并且在 ENTRYPOINT 和 CMD 都支持  exec, shell 和增强型 shell 方式。如果同时有 ENTRYPOINT 和  CMD(或 docker 运行时的 CMD), 则 CMD 将为 ENTRYPOINT 提供参数。

在原来那篇文中认为 shell 无法接收到 docker stop 或  docker -s SIGTERM 发来的信号，也许是随着 Docker 版本的变迁，Docker 变得越发聪明了起来，无论何种格式的 ENTRYPOINT, 都能够收到 SIGTERM 信号，比如在 Java 的 ShutdownHook 能捕捉到该信号，得以在进程停止之前作必要的清理工作。

进行本文相关研究的主因是部署在 ECS(Fargate) 中的 Java Web 服务，Task 总是因为 OutOfMemoryError 被杀掉，而在应用程序日志中却见不着半点线索说 JVM 的 OutOfMemoryError，即使后来给 Fargate 配上了 EFS, 加了 +XX:+HeapDumpOnOutOfMemoryError XX:HeapDumpPath=/efs JVM参数，在任务被 kill 时在 /efs 上从来就都没生成过内存映像文件。最后发现是因为 JVM 的 -Xmx 配置太高，留给 Fargate 容器的太少的缘故。 Read More

一谈到 Docker 容器，按照以往的惯性思维，那就是 Linux 容器(LXC)，和 Windows 没多大关系，顶多也就是在 Windows 的 Linux 虚拟机中跑 Docker 容器。

不过自从 Windows Server 2016 开始，出现了 Windows 原生的 Docker 容器，它再也不只是 Linux 下的专利了。Docker 容器中可以运行 Windows 系统了, 每个 Windows 容器共享宿主机的 Windows 内核(--isolation=process,)，或使用一个高度优化虚拟机中的 Windows 内核(--isolation=hyperv)。

我们说自 Windows Server 2016 开始，包括现在的 Windows Server 2019, Windows Server 2022, 还有桌面系统的 Windows 10 和  11 上 借助于 Docker Desktop 也能跑 Windows 容器。

原本在 Windows 桌面版上安装 Docker Desktop 就能用来运行 Linux 容器，由此可知在 Windows 桌面版上(如 Windows 7, 10, 11) 可运行两种类型的容器

1. Linux 容器: 每个容器运行的是 Linux 实例，用 cgcroups 命名空间隔离资源。默认的，使用 Docker Desktop 的 LinuxEngine
2. Windows 容器：容器中运行的是 Windows 实例，进程隔离模式是容器共享主 机的 Windows 内核，Hyper-V 隔离模式是容器使用高度优化虚拟机的内核。需启用 Windows 的 Hyper-V 特性，并切换 Docker Desktop 使用 WindowsEngine

Read More

AWS 的 Lambda 在 2020-12-01 开始支持用 Docker 镜像存放代码，见 New for AWS Lambda - Container Image Support。AWS Lambda 最初的对发布包的限制是 50M, 解压后(因为执行前需要解压缩)不能超过 250M，对于压缩比小于 1/5 的包来说，要突破 50M 部署包的限制就要用 2018-11-29 推出的层(layer), 即把 Lambda 的依赖可以组织为层，每个 Lambda 可引用最多 5 个层，但最终 Lambda 加上层所解压后的大小仍然有 250 M 的限制。

对于使用了大量依赖的 Lambda，比如 Python 中用了 Pandas 之类的数学分析包，250M 的大小是不够的，所以才有了 Docker 镜像化的 Lambda, 镜像的大小限制一下蹦到 10G，要构建出一个 10G Lambda 用的 Linux 镜像, 那绝对是个巨兽，至少目前是超越我的想像力，除非往里面塞入大量的业务数据。关于 Lambda 有哪些限制，请参阅 Lambda quotas。

介绍完 Lambda 引入 Docker 镜像的背景后，本文接下来只关注如何构建一个 Python Lambda 镜像，对于如何部署 Docker 化的 Lambda, 不在本文的范围之内。主要的参考文档为 AWS Lambda 官方的 Deploy Python Lambda functions with container images. Read More

最近本人在 Mac OS X 下安装的 Docker Desktop 经常蹦出要升级的窗口，Skip this update PRO 不能用，Install & Restart  也总是失败，所以每次只能 SNOOZE。原来觉得事有蹊跷，后来才知道这或许与近日吵得沸沸扬扬的 Docker Desktop 收费计划有关。Docker Desktop 是专门为 Windows 和 Mac OS X 定制的，Mac OS X 和 Windows  都是开发人员用的平台。Linux 下可以不用 Docker Desktop，Linux 下 Docker 之间甚至一层窗户纸都没有，实际部署的 Docker 服务也是运行在 Linux 下的。

墙里边微信正在谋划着对聊天记录的云存储收年费，外边是 Docker 在向大公司和开发人员下手，这难道是美国加州版的共同富裕？ Read More

Continue to advance to Kubernetes, the last article Docker Swarm In Action, After understanding Swarm, it is necessary to get familiar with Docker Compose. Docker Swarm forms a cluster of Docker hosts. As long as the Manager node is notified when the service is deployed, it will automatically find the corresponding node to run containers. Compose is another concept entirely. It organizes multiple associated containers into a whole for deployment, such as a load balance container, multiple web containers, and a Redis cache container to make up a whole bundle.

Revealed by its name, Compose, the concept of container orchestration was officially established. Later, Kubernetes, which we will learn, it's a tool for higher-level organization, operation and management of containers. Because Compose organizes the containers, it can start multiple associated containers with one command, instead of starting one container separately.

Regarding the installation of Docker Compose, Docker Desktop under Mac OS X comes with docker-compose; since Docker Compose is written in Python, we can use  pip install docker-compose  to install it, and use its commands after installation docker-compose. Read More

Before officially entering into Kubernetes, I hope to know something about the earlier Docker Swarm, although it is basically no long used currently. Swarm provides a cluster of Docker hosts. There is at least one Manager (or more) in the cluster, plus 0 or more Workers. The main idea is that a Docker service (container) is executed by the Swarm cluster manager, and it will find the corresponding host in the cluster to execute the container. Both Manager and Worker can to run Docker containers, but only Manager can execute management commands.

The following two pictures shows the difference between running Docker containers on a host and a Swarm cluster host.

Run all containers in one host

=>

Run container together

Read More

Many of my demos about Kafka, Docker, Python, Kubernates, and etc. are made with Vagrant virtual machines. It is time to write a blog for some frequently used Vagrant commands. Vagrant is a member of the HashiCorp family. Others of HashiCorp's famous tools include  Terraform , Consul ,  Vault , Boundary , Packer , Nomad and Waypoint . 

Speaking of Vagrant, I have to mention the similar Docker, in fact, they are quite different, while they both give people the external feeling that they are command line control Linux. Vagrant is essentially a virtual machine shell, allowing us to use Vagrant commands to interact with the virtual machine more conveniently, instead of switching back and forth between the host machine and the virtual machine, it is more convenient to manage multiple virtual machines in one single terminal; while Docker is a container, The essence of a container is a process on the host machine, but it is isolated from the file system, process, network, etc. of the process with a namespace, making the container process look like a virtual OS.

Vagrant is a tool for the development environment, and Docker is a tool for the deployment environment; Vagrant operates a standard Linux or Windows operating system, and the Docker is very critical on image size. Docker image is usually a trimmed system, just has necessary command to run our service. Since Vagrant corresponds to a virtual machine, the operation status with Vagrant and the installed software will be retained after Vagrant virtual machine shutdown, while the operations status in Docker are all for the current container (copy-on-write), which does not affect the corresponding image , Unless it is committed as a new image with docker commit.

Understand that Vagrant is just the shell of a virtual machine, so it requires different virtual machine implementations, such as VirtualBox, Hyper-V, VMware, etc., and we can use Vagrant to interact with Docker as well. Vagrant can support multiple Operation Systems. Read More