[Bug] [Container Image Builder] When AWSTOE fails, it says logs can be found in /var/lib/amazon/toe/ but folder does not exist.

[psantus]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
Build a container recipe.
Not sure this is relevant, but I'm using "Amazon Linux AMI 2023.0.20230627 x86_64 ECS HVM EBS" (ami-0a6c1a11150f7af05 in eu-central-1) as the bage image for my build machine.

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
When AWSTOE fails for whatever reason TOE_LOGS in /tmp/imagebuilder_service/TOE_LOGS look like this

{
    "executionId": "3b1ce2ae-1687-11ee-a8af-0242ac110002",
    "status": "failed",
    "failedStepCount": 1,
    "executedStepCount": 1,
    "ignoredFailedStepCount": 0,
    "failureMessage": "Document arn:aws:imagebuilder:eu-central-1:454249771573:component/mariadb-linux/1.0.0/1 failed!",
    "logUrl": "/var/lib/amazon/toe/TOE_2023-06-29_14-14-16_UTC-0_3b1ce2ae-1687-11ee-a8af-0242ac110002"
}

Problem is: folder /var/lib/amazon/toe/ does not even exist.

Are you currently working around this issue?
As a consequence, I needed to manually replay TOE after connecting to the builder VM.

## 1. Install AWSTOE 

sudo curl https://awstoe-us-east-1.s3.us-east-1.amazonaws.com/latest/linux/386/awstoe -o /usr/bin/awstoe

sudo chmod +x /usr/bin/awstoe

## 2. Download component data 

sudo touch document.yml
sudo chmod 777 document.yml
sudo aws imagebuilder get-component --component-build-version-arn arn:aws:imagebuilder:eu-central-1:aws:component/mariadb-linux/1.0.0/1 --query "component.data" --output text > document.yml

## 3. Run document 

sudo su
awstoe validate --documents document.yml

{
    "validationStatus": "success",
    "message": "Document(s) [document.yml] is/are valid."
}

awstoe run --documents document.yml
{
    "executionId": "af3dc7dc-1692-11ee-b11f-0a750e1b65ac",
    "status": "success",
    "failedStepCount": 0,
    "executedStepCount": 3,
    "ignoredFailedStepCount": 0,
    "failureMessage": "",
    "logUrl": "/tmp/imagebuilder_service/TOE_2023-06-29_15-36-15_UTC-0_af3dc7dc-1692-11ee-b11f-0a750e1b65ac"
}

Then as you can see logs are located in /tmp/imagebuilder_service/ (and there they DO exist :)

Additional context
Not sure this is relevant, but I'm using

• Amazon Linux 2023 AMI (cf. above)
• I have the issue with all base docker images I tried (one is based on Ubuntu, the other one on AlmaLinux minimal)