Containerd containerd_insecure_registries not working

[gvrosun]

This is my offline.yml entry
containerd_insecure_registries:
"repo:5000": "http://repo:5000"

I got error when running ansible using docker container
TASK [containerd ｜ Create registry directories] ****************************************************************************************************************************
fatal: [master1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'key'\n\nThe error appears to be in '/root/r2-new-nexus/test.yml': line 7, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - name: containerd ｜ Create registry directories\n ^ here\n"}
fatal: [worker1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'key'\n\nThe error appears to be in '/root/r2-new-nexus/test.yml': line 7, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - name: containerd ｜ Create registry directories\n ^ here\n"}

[gvrosun]

I found solution
Need to update '/kubespray/roles/container-engine/containerd/tasks/main.yml'

- name: containerd ｜ Create registry directories
  file:
    path: "/etc/containerd/certs.d/{{ item.key }}"
    state: directory
    mode: 0755
    recurse: true
  with_items: "{{ query('dict', containerd_insecure_registries) }}"
  when: containerd_insecure_registries is defined

- name: containerd ｜ Write hosts.toml file
  blockinfile:
    path: "/etc/containerd/certs.d/{{ item.key }}/hosts.toml"
    owner: "root"
    mode: 0640
    create: true
    block: |
      server = "{{ item.value }}"
      [host."{{ item.value }}"]
        capabilities = ["pull", "resolve", "push"]
        skip_verify = true
  with_items: "{{ query('dict', containerd_insecure_registries) }}"
  when: containerd_insecure_registries is defined

[floryut]

@gvrosun Could you check instead if this PR fixes your issue #9651 ?

[gvrosun]

Can anyone tell me how to setup containerd_insecure_registries variable?

[teriyakichild]

I think this is a better solution:

diff --git a/roles/container-engine/containerd/tasks/main.yml b/roles/container-engine/containerd/tasks/main.yml
index ecea9dd75..c057a08e2 100644
--- a/roles/container-engine/containerd/tasks/main.yml
+++ b/roles/container-engine/containerd/tasks/main.yml
@@ -117,7 +117,7 @@
     state: directory
     mode: 0755
     recurse: true
-  with_items: "{{ containerd_insecure_registries }}"
+  with_dict: "{{ containerd_insecure_registries }}"
   when: containerd_insecure_registries is defined
 
 - name: containerd ｜ Write hosts.toml file
@@ -131,7 +131,7 @@
       [host."{{ item.value }}"]
         capabilities = ["pull", "resolve", "push"]
         skip_verify = true
-  with_items: "{{ containerd_insecure_registries }}"
+  with_dict: "{{ containerd_insecure_registries }}"
   when: containerd_insecure_registries is defined
 
 # you can sometimes end up in a state where everything is installed

[lengrongfu]

fix pr: #9729

[bmartron]

Hello,
Thanks for the tips, work for me
Changing with_items by with_dict in roles/container-engine/containerd/tasks/main.yml
This is solve the Ansible issue we have also an issue witch the new configuration file in cert.d in containerd

[gvrosun]

Is there any way to skip ensure docker-ce repository public key is installed & ensure docker-ce repository is enabled tasks

[bmartron]

Hello,
To solve the issue i also have to remove mirror line in /etc/containerd/config.toml
To remove mirror

version = 2
root = "/var/lib/containerd"
state = "/run/containerd"
oom_score = 0

[grpc]
max_recv_message_size = 16777216
max_send_message_size = 16777216

[debug]
level = "info"

[metrics]
address = ""
grpc_histogram = false

[plugins]
[plugins."io.containerd.grpc.v1.cri"]
sandbox_image = "registry.k8s.io/pause:3.8"
max_container_log_line_size = -1
enable_unprivileged_ports = false
enable_unprivileged_icmp = false
[plugins."io.containerd.grpc.v1.cri".containerd]
default_runtime_name = "runc"
snapshotter = "overlayfs"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
runtime_engine = ""
runtime_root = ""
base_runtime_spec = "/etc/containerd/cri-base.json"

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
        systemdCgroup = true
[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = "/etc/containerd/certs.d"

~
With this config the ansible task is ok and pulling from unsecure registry is ok

[jkanasu]

Hello, To solve the issue i also have to remove mirror line in /etc/containerd/config.toml To remove mirror

version = 2 root = "/var/lib/containerd" state = "/run/containerd" oom_score = 0

[grpc] max_recv_message_size = 16777216 max_send_message_size = 16777216

[debug] level = "info"

[metrics] address = "" grpc_histogram = false

[plugins] [plugins."io.containerd.grpc.v1.cri"] sandbox_image = "registry.k8s.io/pause:3.8" max_container_log_line_size = -1 enable_unprivileged_ports = false enable_unprivileged_icmp = false [plugins."io.containerd.grpc.v1.cri".containerd] default_runtime_name = "runc" snapshotter = "overlayfs" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes] [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" runtime_engine = "" runtime_root = "" base_runtime_spec = "/etc/containerd/cri-base.json"

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
        systemdCgroup = true
[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = "/etc/containerd/certs.d"

~ With this config the ansible task is ok and pulling from unsecure registry is ok

This behaviour is true, the kubeadm/kubelet fails to initialize if the mirrors portion is in the config.toml
I fixed in the kubespray templates/config.toml.j2 file by removing the logic to add mirrors for insecure registries

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.