Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated build ATS to 9.2.4 #7958

Merged
merged 10 commits into from
Jul 10, 2024
Merged

Updated build ATS to 9.2.4 #7958

merged 10 commits into from
Jul 10, 2024

Conversation

ntheanh201
Copy link
Contributor

@ntheanh201 ntheanh201 commented Mar 8, 2024

  • Upgrade tsb build ATS 9.2.4
  • Change docker-compose: build astats_over_http from the local source code instead of the cloned code

Which Traffic Control components are affected by this PR?

  • ATS

What is the best way to verify this PR?

  • Build ATS with pkg: ./pkg -o -b -v ats
  • The ATS version should be built with 9.2.4 instead of 8.0.x

If this is a bugfix, which Traffic Control versions contained the bug?

PR submission checklist

@ntheanh201 ntheanh201 changed the title Update build ATS 9.2.3 Updated build ATS to 9.2.3 Mar 8, 2024
@limited
Copy link
Contributor

limited commented Mar 8, 2024

Maybe this is a problem with building on OS X, but I run into an error following the test instructions

./pkg -o -b -v ats
Building ats.
[+] Building 45.9s (18/18) FINISHED                                                                                                                    docker:desktop-linux
 => [ats internal] load build definition from Dockerfile-tsb                                                                                                           0.0s
 => => transferring dockerfile: 2.63kB                                                                                                                                 0.0s
 => [ats internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                        0.0s
 => [ats internal] load metadata for docker.io/library/rockylinux:8                                                                                                    1.1s
 => [ats auth] library/rockylinux:pull token for registry-1.docker.io                                                                                                  0.0s
 => [ats internal] load build context                                                                                                                                  0.0s
 => => transferring context: 44B                                                                                                                                       0.0s
 => [ats tsb-build 1/6] FROM docker.io/library/rockylinux:8@sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980                                    4.8s
 => => resolve docker.io/library/rockylinux:8@sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980                                                  0.0s
 => => sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980 547B / 547B                                                                             0.0s
 => => sha256:be879ad24fd5387ed135b99ebf0622c323afab20ff7f1967d6f06e5dbf07ee31 529B / 529B                                                                             0.0s
 => => sha256:c49dbacab8723f9007f4cc25a5d701c40f2ba3597d8fce029830876326b6fbeb 1.49kB / 1.49kB                                                                         0.0s
 => => sha256:a49f4b3e1553c4468c366b42fd1cde2a27729bd7ab13162ad061af2bd1ef9268 72.15MB / 72.15MB                                                                       2.0s
 => => extracting sha256:a49f4b3e1553c4468c366b42fd1cde2a27729bd7ab13162ad061af2bd1ef9268                                                                              2.7s
 => [ats tsb-build 2/6] RUN yum clean all && yum -y install git &&  git config --global --add safe.directory '*'                                                      23.3s
 => [ats stage-1 2/7] RUN YUM -y install git sudo                                                                                                                     27.3s
 => [ats tsb-build 3/6] COPY GO_VERSION /                                                                                                                              0.0s
 => [ats tsb-build 4/6] RUN curl -L https://dl.google.com/go/go$(cat /GO_VERSION).linux-amd64.tar.gz | tar -C /usr/local -xz && ln -s /usr/local/go/bin/go /usr/bin/g  3.0s
 => [ats tsb-build 5/6] RUN mkdir -p /go/src/github.com/comcast/tsb && cd /go/src/github.com/comcast/tsb && git init && git remote add origin https://github.com/comc  0.8s
 => [ats tsb-build 6/6] RUN cd /go/src/github.com/comcast/tsb && CGOENABLED=0 go build -o /usr/bin/tsb                                                                12.0s
 => [ats stage-1 3/7] RUN ECHO -e "#!/usr/bin/env bash\necho docker-compose is not supported in this container and the build command will not work.\nexit 1" > /usr/b  0.2s
 => [ats stage-1 4/7] RUN CHMOD +x /usr/bin/docker-compose                                                                                                             0.3s
 => [ats stage-1 5/7] COPY --FROM=TSB-BUILD /usr/bin/tsb /usr/bin/tsb                                                                                                  0.0s
 => [ats stage-1 6/7] RUN GIT config --global user.email "[email protected]"                                                                               0.1s
 => [ats stage-1 7/7] RUN GIT config --global user.name "Apache Traffic Control"                                                                                       0.2s
 => [ats] exporting to image                                                                                                                                           0.4s
 => => exporting layers                                                                                                                                                0.4s
 => => writing image sha256:4ec40d3f4f2ac2e4648169feeae1ad8412840377d4cc9cb48aad6384ab2a5d9f                                                                           0.0s
 => => naming to docker.io/library/build-ats                                                                                                                           0.0s
realpath: illegal option -- e
usage: realpath [-q] [path ...]
realpath: illegal option -- e
usage: realpath [-q] [path ...]
unknown flag: --services
parsing /Users/eric.friedrich/Development/trafficcontrol/version: Top-level object must be a mapping
parsing /Users/eric.friedrich/Development/trafficcontrol/version: Top-level object must be a mapping
Results in 'dist':
total 0
Results in 'dist':
total 0

@ntheanh201
Copy link
Contributor Author

Maybe this is a problem with building on OS X, but I run into an error following the test instructions

./pkg -o -b -v ats
Building ats.
[+] Building 45.9s (18/18) FINISHED                                                                                                                    docker:desktop-linux
 => [ats internal] load build definition from Dockerfile-tsb                                                                                                           0.0s
 => => transferring dockerfile: 2.63kB                                                                                                                                 0.0s
 => [ats internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                        0.0s
 => [ats internal] load metadata for docker.io/library/rockylinux:8                                                                                                    1.1s
 => [ats auth] library/rockylinux:pull token for registry-1.docker.io                                                                                                  0.0s
 => [ats internal] load build context                                                                                                                                  0.0s
 => => transferring context: 44B                                                                                                                                       0.0s
 => [ats tsb-build 1/6] FROM docker.io/library/rockylinux:8@sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980                                    4.8s
 => => resolve docker.io/library/rockylinux:8@sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980                                                  0.0s
 => => sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980 547B / 547B                                                                             0.0s
 => => sha256:be879ad24fd5387ed135b99ebf0622c323afab20ff7f1967d6f06e5dbf07ee31 529B / 529B                                                                             0.0s
 => => sha256:c49dbacab8723f9007f4cc25a5d701c40f2ba3597d8fce029830876326b6fbeb 1.49kB / 1.49kB                                                                         0.0s
 => => sha256:a49f4b3e1553c4468c366b42fd1cde2a27729bd7ab13162ad061af2bd1ef9268 72.15MB / 72.15MB                                                                       2.0s
 => => extracting sha256:a49f4b3e1553c4468c366b42fd1cde2a27729bd7ab13162ad061af2bd1ef9268                                                                              2.7s
 => [ats tsb-build 2/6] RUN yum clean all && yum -y install git &&  git config --global --add safe.directory '*'                                                      23.3s
 => [ats stage-1 2/7] RUN YUM -y install git sudo                                                                                                                     27.3s
 => [ats tsb-build 3/6] COPY GO_VERSION /                                                                                                                              0.0s
 => [ats tsb-build 4/6] RUN curl -L https://dl.google.com/go/go$(cat /GO_VERSION).linux-amd64.tar.gz | tar -C /usr/local -xz && ln -s /usr/local/go/bin/go /usr/bin/g  3.0s
 => [ats tsb-build 5/6] RUN mkdir -p /go/src/github.com/comcast/tsb && cd /go/src/github.com/comcast/tsb && git init && git remote add origin https://github.com/comc  0.8s
 => [ats tsb-build 6/6] RUN cd /go/src/github.com/comcast/tsb && CGOENABLED=0 go build -o /usr/bin/tsb                                                                12.0s
 => [ats stage-1 3/7] RUN ECHO -e "#!/usr/bin/env bash\necho docker-compose is not supported in this container and the build command will not work.\nexit 1" > /usr/b  0.2s
 => [ats stage-1 4/7] RUN CHMOD +x /usr/bin/docker-compose                                                                                                             0.3s
 => [ats stage-1 5/7] COPY --FROM=TSB-BUILD /usr/bin/tsb /usr/bin/tsb                                                                                                  0.0s
 => [ats stage-1 6/7] RUN GIT config --global user.email "[email protected]"                                                                               0.1s
 => [ats stage-1 7/7] RUN GIT config --global user.name "Apache Traffic Control"                                                                                       0.2s
 => [ats] exporting to image                                                                                                                                           0.4s
 => => exporting layers                                                                                                                                                0.4s
 => => writing image sha256:4ec40d3f4f2ac2e4648169feeae1ad8412840377d4cc9cb48aad6384ab2a5d9f                                                                           0.0s
 => => naming to docker.io/library/build-ats                                                                                                                           0.0s
realpath: illegal option -- e
usage: realpath [-q] [path ...]
realpath: illegal option -- e
usage: realpath [-q] [path ...]
unknown flag: --services
parsing /Users/eric.friedrich/Development/trafficcontrol/version: Top-level object must be a mapping
parsing /Users/eric.friedrich/Development/trafficcontrol/version: Top-level object must be a mapping
Results in 'dist':
total 0
Results in 'dist':
total 0

Lemme try it with MacOS, with Linux (Ubuntu) it's working

@mikeV02
Copy link
Contributor

mikeV02 commented Mar 10, 2024

Maybe this is a problem with building on OS X, but I run into an error following the test instructions

./pkg -o -b -v ats
Building ats.
[+] Building 45.9s (18/18) FINISHED                                                                                                                    docker:desktop-linux
 => [ats internal] load build definition from Dockerfile-tsb                                                                                                           0.0s
 => => transferring dockerfile: 2.63kB                                                                                                                                 0.0s
 => [ats internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                        0.0s
 => [ats internal] load metadata for docker.io/library/rockylinux:8                                                                                                    1.1s
 => [ats auth] library/rockylinux:pull token for registry-1.docker.io                                                                                                  0.0s
 => [ats internal] load build context                                                                                                                                  0.0s
 => => transferring context: 44B                                                                                                                                       0.0s
 => [ats tsb-build 1/6] FROM docker.io/library/rockylinux:8@sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980                                    4.8s
 => => resolve docker.io/library/rockylinux:8@sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980                                                  0.0s
 => => sha256:c464612ef7e3d54d658c3eaa4778b5cdc990ec7a4d9ab63b0f00c9994c6ce980 547B / 547B                                                                             0.0s
 => => sha256:be879ad24fd5387ed135b99ebf0622c323afab20ff7f1967d6f06e5dbf07ee31 529B / 529B                                                                             0.0s
 => => sha256:c49dbacab8723f9007f4cc25a5d701c40f2ba3597d8fce029830876326b6fbeb 1.49kB / 1.49kB                                                                         0.0s
 => => sha256:a49f4b3e1553c4468c366b42fd1cde2a27729bd7ab13162ad061af2bd1ef9268 72.15MB / 72.15MB                                                                       2.0s
 => => extracting sha256:a49f4b3e1553c4468c366b42fd1cde2a27729bd7ab13162ad061af2bd1ef9268                                                                              2.7s
 => [ats tsb-build 2/6] RUN yum clean all && yum -y install git &&  git config --global --add safe.directory '*'                                                      23.3s
 => [ats stage-1 2/7] RUN YUM -y install git sudo                                                                                                                     27.3s
 => [ats tsb-build 3/6] COPY GO_VERSION /                                                                                                                              0.0s
 => [ats tsb-build 4/6] RUN curl -L https://dl.google.com/go/go$(cat /GO_VERSION).linux-amd64.tar.gz | tar -C /usr/local -xz && ln -s /usr/local/go/bin/go /usr/bin/g  3.0s
 => [ats tsb-build 5/6] RUN mkdir -p /go/src/github.com/comcast/tsb && cd /go/src/github.com/comcast/tsb && git init && git remote add origin https://github.com/comc  0.8s
 => [ats tsb-build 6/6] RUN cd /go/src/github.com/comcast/tsb && CGOENABLED=0 go build -o /usr/bin/tsb                                                                12.0s
 => [ats stage-1 3/7] RUN ECHO -e "#!/usr/bin/env bash\necho docker-compose is not supported in this container and the build command will not work.\nexit 1" > /usr/b  0.2s
 => [ats stage-1 4/7] RUN CHMOD +x /usr/bin/docker-compose                                                                                                             0.3s
 => [ats stage-1 5/7] COPY --FROM=TSB-BUILD /usr/bin/tsb /usr/bin/tsb                                                                                                  0.0s
 => [ats stage-1 6/7] RUN GIT config --global user.email "[email protected]"                                                                               0.1s
 => [ats stage-1 7/7] RUN GIT config --global user.name "Apache Traffic Control"                                                                                       0.2s
 => [ats] exporting to image                                                                                                                                           0.4s
 => => exporting layers                                                                                                                                                0.4s
 => => writing image sha256:4ec40d3f4f2ac2e4648169feeae1ad8412840377d4cc9cb48aad6384ab2a5d9f                                                                           0.0s
 => => naming to docker.io/library/build-ats                                                                                                                           0.0s
realpath: illegal option -- e
usage: realpath [-q] [path ...]
realpath: illegal option -- e
usage: realpath [-q] [path ...]
unknown flag: --services
parsing /Users/eric.friedrich/Development/trafficcontrol/version: Top-level object must be a mapping
parsing /Users/eric.friedrich/Development/trafficcontrol/version: Top-level object must be a mapping
Results in 'dist':
total 0
Results in 'dist':
total 0

Don't quote me here, but I believe the issue with "realpath" being different in MacOS (BSD) than GNU realpath in Linux. That command is used in some bash scripts, for example, the pkg.sh script, not related with the PR itself.

@smalenfant
Copy link
Contributor

Was this build working prior to your change?

@ntheanh201
Copy link
Contributor Author

Was this build working prior to your change?

It has to be something with the OS, with Linux I see there's no problem

@limited
Copy link
Contributor

limited commented Mar 12, 2024 via email

@ntheanh201
Copy link
Contributor Author

I'll retry on non Mac and see how it goes. I am usually able to build pkg as part of my release verification

On Mon, Mar 11, 2024 at 12:18 PM The Anh Nguyen @.> wrote: Was this build working prior to your change? It has to be something with the OS, with Linux I see there's no problem — Reply to this email directly, view it on GitHub <#7958 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGRNNBJP6EN3CATLPRID7TYXXKNHAVCNFSM6AAAAABEM62OSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBYHAZTINZXG4 . You are receiving this because you commented.Message ID: @.>

Did you take a look?

@zrhoffman zrhoffman added Traffic Server related to Apache Traffic Server dependencies Pull requests that update a dependency file labels Mar 21, 2024
@zrhoffman zrhoffman mentioned this pull request Apr 3, 2024
4 tasks
@zrhoffman
Copy link
Member

@limited As noted by @mikeV02, ./pkg -o fails on macOS because macOS does not have realpath (although this problem is not caused by #7958).

I added realpath support on macOS to pkg in #7981, if anyone wants to review.

@ntheanh201
Copy link
Contributor Author

Can you review this PR after #7981 was merged? @zrhoffman

Copy link
Member

@zrhoffman zrhoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very close, Just a couple small nits :)

@limited The build issue should be fixed now that #7981 is merged, does #7958 build for you?

traffic_server/_tsb/run.sh Outdated Show resolved Hide resolved
traffic_server/_tsb/repos.yml Outdated Show resolved Hide resolved
@ntheanh201 ntheanh201 changed the title Updated build ATS to 9.2.3 Updated build ATS to 9.2.4 Jun 10, 2024
Copy link
Member

@zrhoffman zrhoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good besides the tag pattern updates to update now that we have a proper release tag. Thanks!

traffic_server/_tsb/trafficserver.spec Outdated Show resolved Hide resolved
@ntheanh201
Copy link
Contributor Author

@zrhoffman can you re-run the checks? I rebuild it and it works

Copy link
Member

@zrhoffman zrhoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

The CI is taking a long time for some reason, but I tested locally and it works.

As a note, the RPM still is built as 9.2.4-rc0, but that seems to be because ATS's version 9.2.4 was tagged incorrectly. So everything looks fine in #7958. :)

@zrhoffman zrhoffman merged commit b65e7a1 into apache:master Jul 10, 2024
23 of 24 checks passed
@ntheanh201
Copy link
Contributor Author

ntheanh201 commented Jul 10, 2024

Looks good, thanks!

The CI is taking a long time for some reason, but I tested locally and it works.

As a note, the RPM still is built as 9.2.4-rc0, but that seems to be because ATS's version 9.2.4 was tagged incorrectly. So everything looks fine in #7958. :)

Actually, sometimes I see it will be like that, today I tried and the tag was 9.2.4 without rc0

@zrhoffman
Copy link
Member

Actually, sometimes I see it will be like that, today I tried and the tag was 9.2.4 without rc0

Does this command print the 9.2.4 tag for you?

git describe --match=9.2.4 9.2.4

For me, it outputs

fatal: No annotated tags can describe '90fbf13db0858cef0e0a094f445d846b60a4c1ef'.
However, there were unannotated tags: try --tags.

. However,

git describe --match=9.2.4-rc0 9.2.4

and

git describe --match=9.2.3 9.2.3

work.

@ntheanh201
Copy link
Contributor Author

ntheanh201 commented Jul 10, 2024

Actually, sometimes I see it will be like that, today I tried and the tag was 9.2.4 without rc0

Does this command print the 9.2.4 tag for you?

git describe --match=9.2.4 9.2.4

For me, it outputs

fatal: No annotated tags can describe '90fbf13db0858cef0e0a094f445d846b60a4c1ef'.
However, there were unannotated tags: try --tags.

. However,

git describe --match=9.2.4-rc0 9.2.4

and

git describe --match=9.2.3 9.2.3

work.

for me, it outputs:

git describe --long --tags --match='*[0-9.][0-9.][0-9.]'
9.2.4-0-g90fbf13db

But after I removed the ats folder and rebuild ats

@zrhoffman
Copy link
Member

for me, it outputs:

git describe --long --tags --match='*[0-9.][0-9.][0-9.]'
9.2.4-0-g90fbf13db

But after I removed the ats folder and rebuild ats

When I run that command in a fresh clone of the ATS repo, I get

4.1.0-12031-g073fb7bea

@zrhoffman
Copy link
Member

I opened apache/trafficserver#11529, hopefully the release manager can investigate.

@ntheanh201
Copy link
Contributor Author

for me, it outputs:

git describe --long --tags --match='*[0-9.][0-9.][0-9.]'
9.2.4-0-g90fbf13db

But after I removed the ats folder and rebuild ats

When I run that command in a fresh clone of the ATS repo, I get

4.1.0-12031-g073fb7bea

Did you checkout to this commit?: git checkout 90fbf13db0858cef0e0a094f445d846b60a4c1e

@ntheanh201
Copy link
Contributor Author

run this command and gets the result: git describe --long --tags --match=9.2.4 9.2.4
9.2.4-0-g90fbf13db

@zrhoffman
Copy link
Member

You're right, I get that result, and git describe --tags --match=9.2.4 9.2.4 does return 9.2.4. But I still thing ATS tagged it wrong, because --match should not be necessary. See:

$ git describe --tags 9.2.3
9.2.3

even though 9.2.3 and 9.2.3-rc0 refer to the same commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file Traffic Server related to Apache Traffic Server
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants