Before officially entering into Kubernetes, I hope to know something about the earlier Docker Swarm, although it is basically no long used currently. Swarm provides a cluster of Docker hosts. There is at least one Manager (or more) in the cluster, plus 0 or more Workers. The main idea is that a Docker service (container) is executed by the Swarm cluster manager, and it will find the corresponding host in the cluster to execute the container. Both Manager and Worker can to run Docker containers, but only Manager can execute management commands.
The following two pictures shows the difference between running Docker containers on a host and a Swarm cluster host.
Run all containers in one host=>
Run container together
Many of my demos about Kafka, Docker, Python, Kubernates, and etc. are made with Vagrant virtual machines. It is time to write a blog for some frequently used Vagrant commands. Vagrant is a member of the HashiCorp family. Others of HashiCorp's famous tools include Terraform , Consul , Vault , Boundary , Packer , Nomad and Waypoint .
Speaking of Vagrant, I have to mention the similar Docker, in fact, they are quite different, while they both give people the external feeling that they are command line control Linux. Vagrant is essentially a virtual machine shell, allowing us to use Vagrant commands to interact with the virtual machine more conveniently, instead of switching back and forth between the host machine and the virtual machine, it is more convenient to manage multiple virtual machines in one single terminal; while Docker is a container, The essence of a container is a process on the host machine, but it is isolated from the file system, process, network, etc. of the process with a namespace, making the container process look like a virtual OS.
Vagrant is a tool for the development environment, and Docker is a tool for the deployment environment; Vagrant operates a standard Linux or Windows operating system, and the Docker is very critical on image size. Docker image is usually a trimmed system, just has necessary command to run our service. Since Vagrant corresponds to a virtual machine, the operation status with Vagrant and the installed software will be retained after Vagrant virtual machine shutdown, while the operations status in Docker are all for the current container (copy-on-write), which does not affect the corresponding image , Unless it is committed as a new image with docker commit.
Understand that Vagrant is just the shell of a virtual machine, so it requires different virtual machine implementations, such as VirtualBox, Hyper-V, VMware, etc., and we can use Vagrant to interact with Docker as well. Vagrant can support multiple Operation Systems. Read More
The most common way to manage a remote machine is SSH (Unix/Linux, Mac) or PowerShell/RDP (Windows), which requires the remote machine to open the corresponding access port and firewall, credentials or SSH Key. When selecting an EC2 instance on AWS console, we can click the "Connect" button, which provides three connection options:- EC2 Instance Connect: Requires EC2 to be configured with SSH Key, sshd is started, ssh inbound port allowed by Security Group,
ec2-instance-connectinstalled(sudo yum install ec2-instance-connect) - Session Manager: This is what we are going to talk about next. sshd is not required(SSH key is not needed of cause). Security Group only requires outbound port 443.
- SSH client: Client SSH to EC2 instance, start sshd, allow inbound ssh port 22 by Security Group, use SSH Key or username and password in AMI, or configure to login with domain account after joining the domain.
AWS Session Manager provides access to EC2 instances through a browser or AWS CLI, and even machines or virtual machines in the local datacenter (requires advanced-instances tier support) , and no longer depends on SSH.Session Manager Overview
Session Manager determines who can or cannot be accessed by the IAM access policy. It can be forwarded through the local port, the operation log in the session can be recorded as an audit, and can configure to send a message to Amazon EventBridge or SQS when session open or closed. The session log encrypted by a KMS key. Read More- EC2 Instance Connect: Requires EC2 to be configured with SSH Key, sshd is started, ssh inbound port allowed by Security Group,
前方许多有关于 Kafka, Docker, Python 和 Kubernates 的文章都是在 Vagrant 虚拟机中做的 Demo,经常用到的一些 Vagrant 命令是时候有必要写篇日志记录下来。Vagrant 是 HashiCorp 家族中的一员,HashiCorp 旗下著名的工具还有 Terraform, Consul, Vault, Boundary, Packer, Nomad 和 Waypoint.
说起 Vagrant,不得不提起与之仿佛类似的 Docker,其实它们相差还是比较大的,只因它们给人的外在感觉都是命令行控制 Linux。Vagrant 实质是一个虚拟机的外挂,让我们更方便的用 Vagrant 命令与虚拟机交互,而不用在宿主机与虚拟机间来回切换,管理多个虚拟机就更得心应手了; 而 Docker 是一个容器,容器的本质是宿主机上的一个进程,只是用命名空间与该进程的文件系统,进程,网络等进行了隔离,使得该容器进程看似一个虚拟 OS。
Vagrant 是开发环境的部署工具, 而 Docker 是运行环境的部署工具; Vagrant 操作的是一个标准的 Linux 或 Windows 操作系统,而 Docker 的镜像考虑到发布服务的个头,通常是一个裁剪的系统,去除了服务器非必要的命令。既然 Vagrant 对应的是虚拟机,那么在 Vagrant 中的操作,安装的软件在 Vagrant 退出后都会保留下来,而 Docker 中操作的都是当前容器(copy-on-write),并不影响所对应的镜像, 除非用 docker commit 固化为新的镜像.
明白了 Vagrant 只是一个虚拟机的皮,那他在不同的硬件平台或操作系统下需要与不同的 Provider, 如 VirtualBox, Hyper-V, VMware 等配合工作,还能用 Vagrant 来操作 Docker。
有了 Vagrant, 从此不再需要下载不同操作系统的 ISO 安装镜像文件,耗时的逐步安装操作系统,也不用手工的下载别人安装好并导出的虚拟机文件,一切有点类似 Docker 一样从远程公共仓库中选择系统即可。
Read More
记录一下在 Vue.js 项目中使用 vue-prism-editor 进行语法高亮显示代码,并兼具代码编辑功能。JavaScript 中有几个比较典型的语法高亮显示的代码库,例如 SHJS, SyntaxHighlighter, Rainbow, highlight.js。本博客曾经使用过 SyntaxHighlighter 来显示日志中的示例代码,现今所采用的是基于 PHP 的 Crayon Syntax Highlighter。除了只为了高亮显示代码外,有时候还需处理在线编辑代码,许多年前试用过 CodeMirror, 而今天要上手 Prism.js 也能够支持代码编辑。
官方 Demo: prism-editor.netlify.com
Codesandbox: https://codesandbox.io/s/61yrlnlnmn。
为了给 Vue.js 项目提供方便,有人专门做了一个 Vue.js 的插件 vue-prism-editor,本文直接使用该插件,而非直接使用 Prism.js。并且项目蓝本用@vue/cli 4.4.6的命令$ vue create vue-prism-editor-demo
创建的,创建过程可参考 Flask 和 Vue.js 开发及整合部署实例 的 创建 Vue 项目一节,此文简单些,就只选择default (babel, eslint)默认项来创建 Read More
熟悉了传统的 C++/Java 类定义的风格,来感受一下 Python 是如何定义类的。本篇是阅读 《The Quick Python Book》第二版关于类定义的笔记,由原书内容进一步引申,不过是依照本人的思考顺序来组织的。在理解 Python 类定义的同时头脑中应该闪现出 JavaScript/Java 如何定义类的情景。最简单的类定义
class MyClass:
由于
passclass MyClass后面要有个冒号,而冒号后总得有点东西才能表示该类定义结束了,于是放个pass当占位符。Python 也像 Java 一样,有一个根类,叫做 object,例如上面的定义1>>> MyClass.__bases__ 2(<class 'object'>,) 3>>> import inspect 4>>> inspect.getmro(MyClass) 5(<class '__main__.MyClass'>, <class 'object'>)
我们能看到它隐式的基类是object, 而不用显式的声明为class MyClass(object)。看到__bases__属性是一个 Tuple, 意识到 Python 是支持多重继承的。 Read More
现实中有这样的用法,创建一批在线程池中运行的 CompletableFuture 实例,然后等待它们全部执行完再继续后面的操作。比如说 AWS 的 Lambda, 单单提交任务到线程池,不等待所有任务全部完成便退出主线程的话,AWS 便认为 Lambda 执行完毕,无视线程池中正在执行的任务而强行结束该 Lambda 实例。
以往我们通常的作法如下
1ExecutorService threadPool = Executors.newFixedThreadPool(10); 2List<CompletableFuture<Void>> futures = IntStream.rangeClosed(1, 10000) 3 .mapToObj(n -> 4 CompletableFuture.runAsync(() -> { 5 System.out.println("done " + n); 6 }, threadPool)).collect(toList()); 7 8futures.forEach(CompletableFuture::join); 9 10System.out.println("all done");如果所有的任务均无异常,上面的代码能得到预想的结果,只要上面打印出
all done的话真的就是表明所有的任务都完成了。但是在循环 joinfutures中的每一个 CompletableFuture 时,只要碰到任意一个任务有异常时,便立即抛出给外部线程,不在乎是否还有其他任务正在执行。此时,如果外部未予捕获,当然 Read More
一年前解决了 WordPress 在 Linux 下不能发送邮件的问题,后来有段时间很正常,我的网站接收邮件的邮箱是 Gmail 的。但是近来,网站上有留言时极少收到通知邮件,怀疑是 Debian 下的exim4又不能正常工作了。但是试了如下的 PHP 脚本$ php -a
从命令行上并没有提示任何的错误,但是检查我的 Gmail 信箱,死活就是收不到邮件。即使直接用
Interactive mode enabled
php> mail('<my_gmail_account>@gmail.com', 'test subject', 'test content');
php > exitmail命令echo Hello World | mail -v -s Test <my_gmail_account>@gmail.com
也是不行的。
查看日志文件/var/log/exim4/mainlog, 发现有下面的错误信息 Read More
上一篇 使用 SQL Server 的 uniqueidentifier 字段类型 了解了 SQL Server 中如何使用 uniqueidentifier 字段类型后,现在来看下 PostgreSQL 中如何使用 uuid 字段类型。在 PostgreSQL 的字段类型是uuid了,所以创建一个带有uuid字段的表是CREATE TABLE customers (
我们这里设置 id 字段类型为
id uuid PRIMARY KEY,
name VARCHAR(36)
);uuid, 并且它是一个主键。也可以应用函数指定它的默认值,下面将会讲述到。
然后用 SQL 语句来向该表插入记录 Read More
SQL Server 自 2008 版起引入了uniqueidentifier字段,它存储的是一个 UUID, 或者叫 GUID,内部存储为 16 个字节。SQL Server 可用两个函数来生成uniqueidentifier, 分别是NEWID()和NEWSEQUENTIALID(), 后者只能用作字段的默认值。Java 也有一个 UUID 工具类java.uti.UUID,UUID.randomUUID().toString()生成一个随机的 UUID 字符串,在java.util.UUID也是用两个long字段表示内部状态。
SQL Server 的uniqueidentifier类型字段表明了内部如何存储,在我们操作它时,它的外在表现形式都是一个固定格式 `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx` 的字符串,不区分大小写的。
本文所使用的 SQL Server 是 2017 版,通过 Docker 来启动的docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=yourStrong(!)Password' -p 1433:1433 -d microsoft/mssql-server-linux:2017-latest
然后我们创建一个带有uniqueidentifier类型字段的表 Read More