「Kubernetes」- NGINX master process died (-1): signal: illegal instruction (core dumped)

  CREATED BY JENKINSBOT

问题描述

环境概述

系统环境: CentOS Linux release 7.4.1708 (Core)
软件版本: Kubernetes v1.12.1
  Docker version 18.06.1-ce, build e68fc7a
  Helm v3.0.0-beta.3
网络信息: k8s-master[1-3]:10.10.50.91 – 93
  node[1-6]:10.10.50.81 – 86
  VIP:10.10.50.90

现象概述

在集群中部署 Nginx Ingress Controller 组件,但是在某两个节点(node3、node4)中nginx-ingress-controller无法启动。错误信息如下:

NGINX master process died (-1): signal: illegal instruction (core dumped)
-------------------------------------------------------------------------------
W1207 07:56:02.153812       8 nginx.go:39]
-------------------------------------------------------------------------------
NGINX master process died (-1): signal: illegal instruction (core dumped)
-------------------------------------------------------------------------------
W1207 07:56:02.165902       8 nginx.go:39]
-------------------------------------------------------------------------------
NGINX master process died (-1): signal: illegal instruction (core dumped)
-------------------------------------------------------------------------------
W1207 07:56:02.179189       8 nginx.go:39]
-------------------------------------------------------------------------------
NGINX master process died (-1): signal: illegal instruction (core dumped)
-------------------------------------------------------------------------------
W1207 07:56:02.190344       8 nginx.go:39]
-------------------------------------------------------------------------------
NGINX master process died (-1): signal: illegal instruction (core dumped)
-------------------------------------------------------------------------------
W1207 07:56:02.202655       8 nginx.go:39]
-------------------------------------------------------------------------------

还有另外壹点,新版Nginx Ingress Controller镜像中,使用Openresty程序。

原因分析

遇到这种问题肯定去搜索呀。既然有错误日志,而且还是NGINX输出的错误日志,这就说明这是壹个可预见的错误。

然后我找到下面两篇讨论:
1)process exit with code 132 #39
2)0.25.0 crashes at backend reload maybe due to recent OpenResty changes #4311

在上述讨论中,问题已经很清楚。作者在程序中使用了新CPU指令集(SSE 4.2),作者也不打算支持旧CPU型号。

解决办法

目前,我们能够想到的解决办法有两个:
1)将节点迁移到新服务器(那些 CPU 支持 SSE 4.2 指令集的服务器),或者将 Pod 调度到新节点(如果使用 DaemonSet 部署,则仅能迁移虚拟机到新服务器)
2)而实际上,Openresty程序也有“No SSE 4.2”版本。(但是不知道 Nginx Ingress Controller 镜像有没有“No SSE 4.2”版本)

很显然我们选择第一个方案,直接将节点迁移到新物理机中。

参考文献

process exit with code 132 #39
0.25.0 crashes at backend reload maybe due to recent OpenResty changes #4311