修改日志等级
# 通过 Docker 部署 docker exec -ti <container_id> loglevel --set debug # debug info trace
[ERROR] error syncing ‘system-library’
... rancher-server | 2022/08/01 09:47:03 [ERROR] error syncing 'system-library': handler system-image-upgrade-catalog-controller: upgrade cluster c-m-dqgvppq8 system service alerting failed: template system-library-rancher-monitoring incompatible with rancher version or cluster's [c-m-dqgvppq8] kubernetes version, requeuing rancher-server | 2022/08/01 09:47:03 [ERROR] error syncing 'system-library': handler system-image-upgrade-catalog-controller: upgrade cluster c-m-dqgvppq8 system service alerting failed: template system-library-rancher-monitoring incompatible with rancher version or cluster's [c-m-dqgvppq8] kubernetes version, requeuing ...
原因分析:
原始的 Catelog system-library 是 Rancher 自带的,使用 release-v2.6 分支;
解决方案:
修改系统 Catelog 配置,将 system-library 的 branche 设置为 master;
部署 Helm 应用失败;helm-operation … ErrImagePull …
问题描述:在 Rancher 中,通过 Helm 部署应用失败;
原因分析:我们看到 helm-operation-6dvgh 拉取镜像失败,我们猜测是:helm operation Pod 负责 Helm 部署,而镜像拉取失败导致出现该错误;
# kubectl get pods -n cattle-system NAME READY STATUS RESTARTS AGE cm-acme-http-solver-4r5n4 1/1 Running 0 7d7h helm-operation-6dvgh 0/2 ErrImagePull 0 51m rancher-5ddfb86964-2d9kt 1/1 Running 0 17d rancher-5ddfb86964-98bpc 1/1 Running 0 17d rancher-5ddfb86964-j7q4z 1/1 Running 0 17d rancher-webhook-565d58fffd-rjkk6 1/1 Running 0 17d # kubectl describe pod helm-operation-6dvgh ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 45m default-scheduler Successfully assigned cattle-system/helm-operation-6dvgh to cn-hangzhou.172.18.3.203 Warning Failed 34m kubelet Error: ImagePullBackOff Warning Failed 34m kubelet Failed to pull image "rancher/shell:v0.1.18": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/rancher/shell:v0.1.18": failed to copy: read tcp 172.18.3.203:43858->104.18.121.25:443: read: connection reset by peer Normal BackOff 34m kubelet Back-off pulling image "rancher/shell:v0.1.18" Warning Failed 11m (x2 over 34m) kubelet Error: ErrImagePull Normal BackOff 11m (x3 over 34m) kubelet Back-off pulling image "rancher/shell:v0.1.18" Warning Failed 11m (x3 over 34m) kubelet Error: ImagePullBackOff Warning Failed 11m kubelet Failed to pull image "rancher/shell:v0.1.18": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/rancher/shell:v0.1.18": failed to copy: read tcp 172.18.3.203:45852->104.18.125.25:443: read: connection reset by peer Normal Pulling 11m (x3 over 45m) kubelet Pulling image "rancher/shell:v0.1.18" ...
解决方案:我们尝试修改 rancher/shell 镜像的地址:
// 我们尝试修改 Helm Chart values.yaml 文件 // postDelete.image.repository,但似乎并未生效,从参数名来看也与该参数无关; // 根据社区反馈:https://forums.rancher.com/t/how-can-i-change-the-rancher-shell-image/36630 // 应该修改 CR 的配置 # kubectl edit settings.management.cattle.io ... default: rancher/shell:v0.1.10 ...