host-ctr: add fallback image parsing for special regions #3138

jpmcb · 2023-05-23T21:20:52Z

Issue number:
N/a

Description of changes:

    host-ctr: add temporary workaround for bypassing checks for parsing ECR ref

    In order to support pulling images from ECR repositories from
    il-central-1, we need to generate our own canonical ECR image reference
    since the region is not officially supported by aws-go-sdk yet.

    Co-authored-by: Erikson Tung <[email protected]>
    Signed-off-by: John McBride <[email protected]>

This is a continuation of #3127:

includes unit testing
abit of a refactor to make testing ecr resolution easier (without having to create a dummy containerd client)
nested error handling
checks for valid mappings in the ecrRefPrefixMapping

Testing done:

Launched aws-k8s-1.26 instance in il-central-1:
All ECR container images get pulled down by host-ctr successfully

[root@admin]# apiclient get os                                        
{                                                                     
  "os": {                                                             
    "arch": "x86_64",                                                 
    "build_id": "86513dc3",                                           
    "pretty_name": "Bottlerocket OS 1.15.0 (aws-k8s-1.26)",           
    "variant_id": "aws-k8s-1.26",                                     
    "version_id": "1.15.0"                                            
  }                                                                   
}                                                                     
[root@admin]# apiclient get settings.kubernetes.pod-infra-container-image                                                                    
{                                                                     
  "settings": {                                                       
    "kubernetes": {                                                   
      "pod-infra-container-image": "066635153087.dkr.ecr.il-central-1.amazonaws.com/eks/pause:3.1-eksbuild.1"                                
    }                                                                 
  }                                                                   
}                                                                     
[root@admin]# apiclient get settings.host-containers.admin.source                                                                            
{                                                                     
  "settings": {                                                       
    "host-containers": {                                              
      "admin": {                                                      
        "source": "288123944683.dkr.ecr.il-central-1.amazonaws.com/bottlerocket-admin:v0.10.1"                                               
      }                                                               
    }                                                                 
  }                                                                   
}                                                                     
[root@admin]# apiclient get settings.host-containers.control.source   
{                                                                     
  "settings": {                                                       
    "host-containers": {                                              
      "control": {                                                    
        "source": "288123944683.dkr.ecr.il-central-1.amazonaws.com/bottlerocket-control:v0.7.2"                                              
      }                                                               
    }                                                                 
  }                                                                   
}                                                                     
...
bash-5.1# ctr -a /run/containerd/containerd.sock -n k8s.io images ls  
REF                                                                               TYPE                                                      DIGEST                                                                  SIZE      PLATFORMS               LABELS                          
066635153087.dkr.ecr.il-central-1.amazonaws.com/eks/pause:3.1-eksbuild.1          application/vnd.docker.distribution.manifest.list.v2+json sha256:1cb4ab85a3480446f9243178395e6bee7350f0d71296daeb6a9fdd221e23aea6 292.4 KiB linux/amd64,linux/arm64 io.cri-containerd.image=managed 
ecr.aws/arn:aws:ecr:il-central-1:066635153087:repository/eks/pause:3.1-eksbuild.1 application/vnd.docker.distribution.manifest.list.v2+json sha256:1cb4ab85a3480446f9243178395e6bee7350f0d71296daeb6a9fdd221e23aea6 292.4 KiB linux/amd64,linux/arm64 io.cri-containerd.image=managed 
sha256:106a8e54d5eb3f70fcd1ed46255bdf232b3f169e89e68e13e4e67b25f59c1315           application/vnd.docker.distribution.manifest.list.v2+json sha256:1cb4ab85a3480446f9243178395e6bee7350f0d71296daeb6a9fdd221e23aea6 292.4 KiB linux/amd64,linux/arm64 io.cri-containerd.image=managed

Launched in us-west-2:
All container images that gets pulled through host-ctr do get pulled down:

bash-5.1# journalctl -u kubelet
May 23 21:38:41 i-0bccbc2c09e312610.etung.test systemd[1]: Starting Kubelet...
May 23 21:38:41 i-0bccbc2c09e312610.etung.test host-ctr[1169]: time="2023-05-23T21:38:41Z" level=info msg="Image does not exist, proceeding to pull image from source." ref="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause:3.1-eksbuild.1"
May 23 21:38:41 i-0bccbc2c09e312610.etung.test host-ctr[1169]: time="2023-05-23T21:38:41Z" level=info msg="pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause:3.1-eksbuild.1"
May 23 21:38:42 i-0bccbc2c09e312610.etung.test host-ctr[1169]: time="2023-05-23T21:38:42Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause:3.1-eksbuild.1"
May 23 21:38:42 i-0bccbc2c09e312610.etung.test host-ctr[1169]: time="2023-05-23T21:38:42Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:us-west-2:602401143452:repository/eks/pause:3.1-eksbuild.1"
May 23 21:38:42 i-0bccbc2c09e312610.etung.test host-ctr[1169]: time="2023-05-23T21:38:42Z" level=info msg="tagging image" img="602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause:3.1-eksbuild.1"
...
bash-5.1# systemctl status  host-containers@admin host-containers@control
● [email protected] - Host container: admin
     Loaded: loaded (/x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-05-23 21:38:41 UTC; 6min ago
   Main PID: 1129 (host-ctr)
      Tasks: 9 (limit: 9251)
     Memory: 31.1M
        CPU: 404ms
     CGroup: /system.slice/system-host\x2dcontainers.slice/[email protected]
             └─ 1129 /usr/bin/host-ctr run --container-id=admin --source=328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1 --superpowered=true --registry-config=/etc/host-containers/host-ctr.toml

May 23 21:38:44 i-0bccbc2c09e312610.etung.test host-ctr[1129]: time="2023-05-23T21:38:44Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 21:38:44 i-0bccbc2c09e312610.etung.test host-ctr[1129]: time="2023-05-23T21:38:44Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 21:39:01 i-0bccbc2c09e312610.etung.test host-ctr[1129]: time="2023-05-23T21:39:01Z" level=info msg="tagging image" img="328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1"
May 23 21:39:01 i-0bccbc2c09e312610.etung.test host-ctr[1129]: time="2023-05-23T21:39:01Z" level=info msg="Container does not exist, proceeding to create it" ctr-id=admin
May 23 21:39:01 i-0bccbc2c09e312610.etung.test host-ctr[1129]: time="2023-05-23T21:39:01Z" level=info msg="container task does not exist, proceeding to create it" container-id=admin
May 23 21:39:01 i-0bccbc2c09e312610.etung.test host-ctr[1129]: time="2023-05-23T21:39:01Z" level=info msg="successfully started container task"
....

● [email protected] - Host container: control
     Loaded: loaded (/x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-05-23 21:38:41 UTC; 6min ago
   Main PID: 1135 (host-ctr)
      Tasks: 9 (limit: 9251)
     Memory: 27.2M
        CPU: 681ms
     CGroup: /system.slice/system-host\x2dcontainers.slice/[email protected]
             └─ 1135 /usr/bin/host-ctr run --container-id=control --source=328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-control:v0.7.2 --superpowered=false --registry-config=/etc/host-containers/host-ctr.toml

....

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

jpmcb · 2023-05-23T21:47:31Z

Built a custom AMI and deployed. Successfully starts up with admin container enabled (note this is in the us-west-2 partition)

bash-5.1# systemctl status host-containers@admin
● [email protected] - Host container: admin
     Loaded: loaded (/aarch64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-05-23 21:36:56 UTC; 2min 16s ago
   Main PID: 519 (host-ctr)
      Tasks: 10 (limit: 4491)
     Memory: 42.5M
     CGroup: /system.slice/system-host\x2dcontainers.slice/[email protected]
             └─ 519 /usr/bin/host-ctr run --container-id=admin --source=328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1 --superpowered=true --registry-config=/etc/host-containers/host-ctr.toml

May 23 21:36:58 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:36:58Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 21:36:58 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:36:58Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:37:04Z" level=info msg="tagging image" img="328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1"
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:37:04Z" level=info msg="Container does not exist, proceeding to create it" ctr-id=admin
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:37:04Z" level=info msg="container task does not exist, proceeding to create it" container-id=admin
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:37:04Z" level=info msg="successfully started container task"
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: Created symlink /root/.config/systemd/user/admin.target.wants/[email protected], pointing to /etc/systemd/user/[email protected].
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: Created symlink /root/.config/systemd/user/admin.target.wants/[email protected], pointing to /etc/systemd/user/[email protected].
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: Created symlink /root/.config/systemd/user/admin.target.wants/sshd.service, pointing to /etc/systemd/user/sshd.service.
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: Startup finished in 21ms.

Logs for host-ctr look good and containers are being pulled:

May 23 21:36:57 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:36:57Z" level=info msg="Image does not exist, proceeding to pull image from source." ref="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 21:36:57 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:36:57Z" level=info msg="pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 21:36:57 ip-172-31-41-104.us-west-2.compute.internal host-ctr[520]: time="2023-05-23T21:36:57Z" level=info msg="Image does not exist, proceeding to pull image from source." ref="ecr.aws/arn:aws:ecr:us-west-2:994959692891:repository/chronicle:latest"
May 23 21:36:57 ip-172-31-41-104.us-west-2.compute.internal host-ctr[520]: time="2023-05-23T21:36:57Z" level=info msg="pulling with Amazon ECR Resolver" ref="ecr.aws/arn:aws:ecr:us-west-2:994959692891:repository/chronicle:latest"
May 23 21:36:58 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:36:58Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 21:36:58 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:36:58Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 21:37:00 ip-172-31-41-104.us-west-2.compute.internal host-ctr[520]: time="2023-05-23T21:37:00Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:994959692891:repository/chronicle:latest"
May 23 21:37:00 ip-172-31-41-104.us-west-2.compute.internal host-ctr[520]: time="2023-05-23T21:37:00Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:us-west-2:994959692891:repository/chronicle:latest"
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[520]: time="2023-05-23T21:37:04Z" level=info msg="tagging image" img="994959692891.dkr.ecr.us-west-2.amazonaws.com/chronicle:latest"
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[520]: time="2023-05-23T21:37:04Z" level=info msg="Container does not exist, proceeding to create it" ctr-id=chronicle
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:37:04Z" level=info msg="tagging image" img="328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1"
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:37:04Z" level=info msg="Container does not exist, proceeding to create it" ctr-id=admin
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[520]: time="2023-05-23T21:37:04Z" level=info msg="container task does not exist, proceeding to create it" container-id=chronicle
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:37:04Z" level=info msg="container task does not exist, proceeding to create it" container-id=admin
May 23 21:37:04 ip-172-31-41-104.us-west-2.compute.internal host-ctr[519]: time="2023-05-23T21:37:04Z" level=info msg="successfully started container task"

zmrow

😶‍🌫️

jpmcb · 2023-05-23T21:53:06Z

Force pushed for less verbose commit message title / contents.

sources/host-ctr/cmd/host-ctr/main.go

jpmcb · 2023-05-23T22:39:04Z

Force pushed to address @bcressey's comments!

parseImageURISpecialRegions now returns `(ecr.ECRSpec, error)
call canonical within the same block

sources/host-ctr/cmd/host-ctr/main.go

jpmcb · 2023-05-23T22:48:33Z

Re-tested on us-west-2 - all's well!

ash-5.1# systemctl status  host-containers@admin host-containers@control
● [email protected] - Host container: admin
     Loaded: loaded (/aarch64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-05-23 22:45:21 UTC; 1min 46s ago
   Main PID: 1001 (host-ctr)
      Tasks: 9 (limit: 4491)
     Memory: 19.8M
     CGroup: /system.slice/system-host\x2dcontainers.slice/[email protected]
             └─ 1001 /usr/bin/host-ctr run --container-id=admin --source=328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1 --superpowered=true --registry-config=/etc/host-containers/host-ctr.toml

May 23 22:45:22 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: time="2023-05-23T22:45:22Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 22:45:22 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: time="2023-05-23T22:45:22Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 22:45:23 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: time="2023-05-23T22:45:23Z" level=info msg="tagging image" img="328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1"
May 23 22:45:23 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: time="2023-05-23T22:45:23Z" level=info msg="Container does not exist, proceeding to create it" ctr-id=admin
May 23 22:45:23 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: time="2023-05-23T22:45:23Z" level=info msg="container task does not exist, proceeding to create it" container-id=admin
May 23 22:45:23 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: time="2023-05-23T22:45:23Z" level=info msg="successfully started container task"
May 23 22:45:23 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: Created symlink /root/.config/systemd/user/admin.target.wants/[email protected], pointing to /etc/systemd/user/[email protected].
May 23 22:45:23 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: Created symlink /root/.config/systemd/user/admin.target.wants/[email protected], pointing to /etc/systemd/user/[email protected].
May 23 22:45:23 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: Created symlink /root/.config/systemd/user/admin.target.wants/sshd.service, pointing to /etc/systemd/user/sshd.service.
May 23 22:45:24 ip-172-31-43-225.us-west-2.compute.internal host-ctr[1001]: Startup finished in 22ms.

● [email protected] - Host container: control
     Loaded: loaded (/aarch64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-05-23 22:44:48 UTC; 2min 19s ago
   Main PID: 511 (host-ctr)
      Tasks: 10 (limit: 4491)
     Memory: 53.2M
     CGroup: /system.slice/system-host\x2dcontainers.slice/[email protected]
             └─ 511 /usr/bin/host-ctr run --container-id=control --source=328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-control:v0.7.2 --superpowered=false --registry-config=/etc/host-containers/host-ctr.toml

May 23 22:44:53 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:52 INFO [Registrar] Starting registrar module
May 23 22:44:53 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:52 INFO [EC2Identity] no registration info found for ec2 instance, attempting registration
May 23 22:44:53 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:53 INFO [EC2Identity] EC2 registration was successful.
May 23 22:44:53 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:53 INFO [CredentialRefresher] credentialRefresher has started
May 23 22:44:53 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:53 INFO [CredentialRefresher] Starting credentials refresher loop
May 23 22:44:53 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:53 INFO EC2RoleProvider Successfully connected with instance profile role credentials
May 23 22:44:53 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:53 INFO [CredentialRefresher] Next credential rotation will be in 32.03332778276667 minutes
May 23 22:44:54 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:54 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] [WorkerProvider] Worker ssm-agent-worker is not running, starting worker process
May 23 22:44:54 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:54 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] [WorkerProvider] Worker ssm-agent-worker (pid:16) started
May 23 22:44:54 ip-172-31-43-225.us-west-2.compute.internal host-ctr[511]: 2023-05-23 22:44:54 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] Monitor long running worker health every 60 seconds

In order to support pulling images from ECR repositories from il-central-1, we need to generate our own canonical ECR image reference since the region is not officially supported by aws-go-sdk yet. Co-authored-by: Erikson Tung <[email protected]> Signed-off-by: John McBride <[email protected]>

jpmcb · 2023-05-23T23:05:52Z

Force pushed again to address Ben's comments - re-tested in us-west-2:

bash-5.1#  systemctl status  host-containers@admin host-containers@control
● [email protected] - Host container: admin
     Loaded: loaded (/x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-05-23 23:04:15 UTC; 1min 8s ago
   Main PID: 2308 (host-ctr)
      Tasks: 11 (limit: 9148)
     Memory: 19.4M
     CGroup: /system.slice/system-host\x2dcontainers.slice/[email protected]
             └─ 2308 /usr/bin/host-ctr run --container-id=admin --source=328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1 --superpowered=true --registry-config=/etc/host-containers/host-ctr.toml

May 23 23:04:16 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: time="2023-05-23T23:04:16Z" level=info msg="pulled image successfully" img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 23:04:16 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: time="2023-05-23T23:04:16Z" level=info msg="unpacking image..." img="ecr.aws/arn:aws:ecr:us-west-2:328549459982:repository/bottlerocket-admin:v0.10.1"
May 23 23:04:17 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: time="2023-05-23T23:04:17Z" level=info msg="tagging image" img="328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-admin:v0.10.1"
May 23 23:04:17 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: time="2023-05-23T23:04:17Z" level=info msg="Container does not exist, proceeding to create it" ctr-id=admin
May 23 23:04:17 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: time="2023-05-23T23:04:17Z" level=info msg="container task does not exist, proceeding to create it" container-id=admin
May 23 23:04:17 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: time="2023-05-23T23:04:17Z" level=info msg="successfully started container task"
May 23 23:04:18 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: Created symlink /root/.config/systemd/user/admin.target.wants/[email protected], pointing to /etc/systemd/user/[email protected].
May 23 23:04:18 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: Created symlink /root/.config/systemd/user/admin.target.wants/[email protected], pointing to /etc/systemd/user/[email protected].
May 23 23:04:18 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: Created symlink /root/.config/systemd/user/admin.target.wants/sshd.service, pointing to /etc/systemd/user/sshd.service.
May 23 23:04:18 ip-172-31-11-103.us-west-2.compute.internal host-ctr[2308]: Startup finished in 19ms.

● [email protected] - Host container: control
     Loaded: loaded (/x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/[email protected]; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-05-23 23:03:08 UTC; 2min 15s ago
   Main PID: 1764 (host-ctr)
      Tasks: 9 (limit: 9148)
     Memory: 61.0M
     CGroup: /system.slice/system-host\x2dcontainers.slice/[email protected]
             └─ 1764 /usr/bin/host-ctr run --container-id=control --source=328549459982.dkr.ecr.us-west-2.amazonaws.com/bottlerocket-control:v0.7.2 --superpowered=false --registry-config=/etc/host-containers/host-ctr.toml

May 23 23:03:12 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:12 INFO [Registrar] Starting registrar module
May 23 23:03:12 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:12 INFO [EC2Identity] no registration info found for ec2 instance, attempting registration
May 23 23:03:12 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:12 INFO [EC2Identity] EC2 registration was successful.
May 23 23:03:12 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:12 INFO [CredentialRefresher] credentialRefresher has started
May 23 23:03:12 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:12 INFO [CredentialRefresher] Starting credentials refresher loop
May 23 23:03:12 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:12 INFO EC2RoleProvider Successfully connected with instance profile role credentials
May 23 23:03:12 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:12 INFO [CredentialRefresher] Next credential rotation will be in 31.77499348088333 minutes
May 23 23:03:13 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:13 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] [WorkerProvider] Worker ssm-agent-worker is not running, starting worker process
May 23 23:03:14 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:13 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] [WorkerProvider] Worker ssm-agent-worker (pid:18) started
May 23 23:03:14 ip-172-31-11-103.us-west-2.compute.internal host-ctr[1764]: 2023-05-23 23:03:13 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] Monitor long running worker health every 60 seconds

jpmcb · 2023-05-23T23:13:16Z

Tests look good in other regions - Launched aws-k8s-1.26 instance in il-central-1 👍🏼

[1.14.x] Cherry-pick #3137, #3138, #3139

jpmcb changed the title ~~host-ctr: add temporary workaround for bypassing checks for parsing E…~~ host-ctr: add temporary workaround for bypassing checks for parsing ECR images for special regions (take 2) May 23, 2023

etungsten mentioned this pull request May 23, 2023

host-ctr: add support for pulling private ECR images from il-central-1 #3127

Closed

etungsten marked this pull request as ready for review May 23, 2023 21:45

etungsten approved these changes May 23, 2023

View reviewed changes

zmrow approved these changes May 23, 2023

View reviewed changes

jpmcb changed the title ~~host-ctr: add temporary workaround for bypassing checks for parsing ECR images for special regions (take 2)~~ host-ctr: add fallback image parsing for special regions May 23, 2023

jpmcb force-pushed the host-ctr-special-reg branch from 86513dc to 641f2d5 Compare May 23, 2023 21:52

bcressey reviewed May 23, 2023

View reviewed changes

sources/host-ctr/cmd/host-ctr/main.go Outdated Show resolved Hide resolved

sources/host-ctr/cmd/host-ctr/main.go Outdated Show resolved Hide resolved

jpmcb force-pushed the host-ctr-special-reg branch from 641f2d5 to 35bf366 Compare May 23, 2023 22:28

yeazelm approved these changes May 23, 2023

View reviewed changes

jpmcb force-pushed the host-ctr-special-reg branch from 35bf366 to 6d30d45 Compare May 23, 2023 22:38

jpmcb requested a review from bcressey May 23, 2023 22:42

bcressey reviewed May 23, 2023

View reviewed changes

sources/host-ctr/cmd/host-ctr/main.go Outdated Show resolved Hide resolved

sources/host-ctr/cmd/host-ctr/main.go Outdated Show resolved Hide resolved

sources/host-ctr/cmd/host-ctr/main.go Outdated Show resolved Hide resolved

jpmcb force-pushed the host-ctr-special-reg branch from 6d30d45 to feabba8 Compare May 23, 2023 22:54

yeazelm approved these changes May 23, 2023

View reviewed changes

jpmcb requested a review from bcressey May 23, 2023 23:13

bcressey approved these changes May 23, 2023

View reviewed changes

etungsten merged commit a84e6cd into bottlerocket-os:develop May 23, 2023

etungsten mentioned this pull request May 24, 2023

[1.14.x] Cherry-pick #3137, #3138, #3139 #3140

Merged

jpmcb deleted the host-ctr-special-reg branch May 24, 2023 03:09

etungsten added a commit that referenced this pull request May 24, 2023

Merge pull request #3140 from etungsten/1.14.1-cherrypicks

95ed132

[1.14.x] Cherry-pick #3137, #3138, #3139

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

host-ctr: add fallback image parsing for special regions #3138

host-ctr: add fallback image parsing for special regions #3138

jpmcb commented May 23, 2023 •

edited by etungsten

Loading

jpmcb commented May 23, 2023 •

edited

Loading

zmrow left a comment

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023

host-ctr: add fallback image parsing for special regions #3138

host-ctr: add fallback image parsing for special regions #3138

Conversation

jpmcb commented May 23, 2023 • edited by etungsten Loading

jpmcb commented May 23, 2023 • edited Loading

zmrow left a comment

Choose a reason for hiding this comment

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023

jpmcb commented May 23, 2023 •

edited by etungsten

Loading

jpmcb commented May 23, 2023 •

edited

Loading