{toolchain} GCC 10.3.0 #12521

boegel · 2021-04-05T10:08:35Z

(created using eb --new-pr)
WIP because final GCC 10.3.0 is not available yet, using first release candidate (10.2.1) for now; final 10.3 release expected later this week (cfr. https://gcc.gnu.org/pipermail/gcc/2021-April/235257.html) (all good now, final 10.3.0 release is there)

boegel · 2021-04-05T10:12:06Z

@boegelbot please test @ generoso
CORE_COUNT=16

boegelbot · 2021-04-05T10:15:07Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=12521 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12521 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 16490

Test results coming soon (I hope)...

- notification for comment with ID 813321070 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2021-04-05T11:38:32Z

Test report by @boegelbot
SUCCESS
Build succeeded for 11 out of 11 (11 easyconfigs in total)
generoso-c1-s-1 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/be76b66f522517e33ad5996ac45dce75 for a full test report.

boegel · 2021-04-08T17:59:49Z

@boegelbot please test @ generoso
CORE_COUNT=16

boegelbot · 2021-04-08T18:00:08Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=12521 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12521 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 16511

Test results coming soon (I hope)...

- notification for comment with ID 816025310 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2021-04-08T18:05:56Z

Test report by @boegelbot
FAILED
Build succeeded for 3 out of 11 (11 easyconfigs in total)
generoso-c1-s-1 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/c25ded996464fc08311c3bb98bae63f5 for a full test report.

edit (by @boegel): sources failed to download because mirrors haven't caught up yet, I'll seed in the sources on generoso...

boegelbot · 2021-04-08T18:15:07Z

@boegelbot: I noticed your comment, but I only dance when @akesandgren or @bartoldeman or @boegel or @branfosj or @casparvl or @Micket or @migueldiascosta or @smoors or @verdurin tells me (for now), I'm sorry...

- notification for comment with ID 816029120 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegel · 2021-04-08T18:20:09Z

@boegelbot please test @ generoso
CORE_COUNT=16

boegelbot · 2021-04-08T18:25:07Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=12521 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12521 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 16512

Test results coming soon (I hope)...

- notification for comment with ID 816038162 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegel · 2021-04-08T19:05:06Z

Test report by @boegel
SUCCESS
Build succeeded for 11 out of 11 (11 easyconfigs in total)
node3501.doduo.os - Linux RHEL 8.2, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/61a3b978f4186ceb8eb6c43d8d003e11 for a full test report.

Micket · 2021-04-08T19:13:42Z

Test report by @Micket
SUCCESS
Build succeeded for 11 out of 11 (11 easyconfigs in total)
vera-c1 - Linux centos linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake), Python 2.7.5
See https://gist.github.com/71648b7fc12df8bbe8e1b32da33a9665 for a full test report.

boegel · 2021-04-08T19:15:54Z

Test report by @boegel
SUCCESS
Build succeeded for 11 out of 11 (11 easyconfigs in total)
node3416.kirlia.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz (cascadelake), Python 3.6.8
See https://gist.github.com/b847f04fe8ed30b789cfe869acf84144 for a full test report.

SebastianAchilles

About binutils: Is there a reason to use version 2.35 and version 2.36.1?

easybuild/easyconfigs/f/flex/flex-2.6.4-GCCcore-10.3.0.eb

easybuild/easyconfigs/h/help2man/help2man-1.48.3-GCCcore-10.3.0.eb

easybuild/easyconfigs/m/M4/M4-1.4.18-GCCcore-10.3.0.eb

easybuild/easyconfigs/b/Bison/Bison-3.7.6-GCCcore-10.3.0.eb

…th GCCcore/10.3.0

boegel · 2021-04-08T19:39:04Z

@boegelbot please test @ generoso
CORE_COUNT=16

boegel · 2021-04-08T19:43:15Z

Test report by @boegel
SUCCESS
Build succeeded for 11 out of 11 (11 easyconfigs in total)
node3123.skitty.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/bf746a2b5c5ac41d50d575b4c89d1d6d for a full test report.

boegelbot · 2021-04-08T20:00:09Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=12521 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12521 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 16515

Test results coming soon (I hope)...

- notification for comment with ID 816107035 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegel · 2021-04-08T20:24:47Z

Test report by @boegel
SUCCESS
Build succeeded for 11 out of 11 (11 easyconfigs in total)
select-pika-c6gd-4xlarge-0001 - Linux centos linux 8.3.2011, AArch64, ARM UNKNOWN (graviton2), Python 3.6.8
See https://gist.github.com/6bb86938282e0812b141c85bb5c4b8f0 for a full test report.

akesandgren · 2021-04-09T22:27:58Z

Test report by @akesandgren
FAILED
Build succeeded for 1 out of 9 (10 easyconfigs in total)
b-cn0235.hpc2n.umu.se - Linux Ubuntu 20.04, x86_64, Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, Python 3.8.5
See https://gist.github.com/0164131f3a3ea695722e657f889d8388 for a full test report.

Micket · 2021-04-09T22:56:39Z

Test report by @Micket
SUCCESS
Build succeeded for 10 out of 10 (10 easyconfigs in total)
vera-c1 - Linux centos linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake), Python 2.7.5
See https://gist.github.com/4b7e15eae493ed00caf80c455aaf096f for a full test report.

akesandgren · 2021-04-10T06:54:41Z

GCCcore itself requires perl during install phase !!

mkdir -p -- /home/a/ake/Public/easybuild/software/GCCcore/10.3.0/nvptx-none/bin
rel=$(perl -MFile::Spec -e 'print File::Spec->abs2rel(@ARGV)' /home/a/ake/Public/easybuild/software/GCCcore/10.3.0/bin /home/a/ake/Public/easybuild/software/GCCcore/10.3.0/nvptx-none/bin) && \
for p in ar as ld ranlib; do \
  rm -f /home/a/ake/Public/easybuild/software/GCCcore/10.3.0/nvptx-none/bin/"$p" && \
  ln -s "$rel"/nvptx-none-"$p" /home/a/ake/Public/easybuild/software/GCCcore/10.3.0/nvptx-none/bin/"$p"; \
done
/bin/sh: 1: perl: not found
make: *** [Makefile:103: install] Error 127

So we do need Perl-base at SYSTEM level

Or a patch that removes the perl usage there. It simply wants to make relative symlinks, and we know what they should be instead...

boegel · 2021-04-10T08:19:52Z

@akesandgren Is it worth blocking this PR over that? Is this a problem new to GCC 10.3?

If not, I suggest we open an issue to follow up on this...
On most systems, perl will be there as an OS package, so this is unlikely to cause trouble very often?

Flamefire · 2021-04-10T08:43:18Z

@boegel If perl is already required, I'd suggest to include makeinfo again to avoid the "hack". In which case I'd say this is a blocker.
How hard is it to get a core-perl at system level? So just the absolut minimum to decrease size and dependencies, maybe just the perl interpreter or so?

Alternatively: Add perl to the system-dependencies as we do with other packages already.

And to be blunt: What does Spack do? Always good to see how others handle the problem ;)

boegel · 2021-04-10T09:55:27Z

@boegel If perl is already required, I'd suggest to include makeinfo again to avoid the "hack". In which case I'd say this is a blocker.

Hmm, yeah, I guess we can add back makeinfo then too. Except if Perl requires makeinfo... ;)

How hard is it to get a core-perl at system level? So just the absolut minimum to decrease size and dependencies, maybe just the perl interpreter or so?

Probably not difficult, but Perl being the very first thing that gets built is going to be annoying...

Alternatively: Add perl to the system-dependencies as we do with other packages already.

That makes most sense to me, since in practice it shouldn't make much difference (because perl is usually there), and if it does happen to be missing, then you'll get a clear error.

And to be blunt: What does Spack do? Always good to see how others handle the problem ;)

No trace of perl in the Spack package for GCC, so it seems like they don't register this requirement either.

boegel · 2021-04-10T10:55:04Z

Actually, since makeinfo requires additional Perl modules which are not a part of the Perl standard library (see #12067), we should probably stick to the MAKEINFO=true hack?

Micket · 2021-04-10T11:04:49Z

I'm for just treating it as a OS dep, at least for now, so that we can get things started with GCCcore 10.3.0. And then let Åke sort this out for all the versions of GCC where we enable nvptx :)

I'm also for just keeping makeinfo out of system level packages if possible. It really doesn't add any benefit to the stuff we use to just boostrap the real stuff. In fact, I don't think the manuals/documentation in modules really serve much purpose at all in a module system. I just would look up documentation online anyway for any software, and I think users do that as well.

justbennet · 2021-04-10T11:57:32Z

I think people are having trouble with including an EB-built Perl, and I certainly did not intend to try to force installation of one. I was trying to ask whether it should be done, or whether the system dependency should be documented.

I was purposely starting from a Red Hat 8 'minimal' installation to see what was required from the system packages, and this came up.

For what my opinion is worth, I think it would be fine to make the two Perl libraries a system dependency, so long as the error is detected and can be clearly stated along with the solution, or, preferably, just listed so that people can have an OS equivalent to a requirements.txt file to avoid the build error.

So far as I know, there isn't a list of what is needed from the system, and I was probably using an artificially small installation, but such a list could (should?) be made, which would render this moot. Regardless of the outcome of this, a list of system dependencies would be worth having, wouldn't it?

Sorry for creating a storm in the teacup.

akesandgren · 2021-04-10T13:00:12Z

makeinfo at the basic level does not require extra perl modules, see my comment in #12067

Micket · 2021-04-10T19:05:32Z

I'm still in favor of not including a makeinfo at SYSTEM level if we can avoid it, even if we have a perl. It just doesn't add anything of value for bootstrapping the toolchain.

And there is plenty of time to resolve the perl-dependency for GCCcore before 4.3.5 (since the problem isn't new with 10.3, it is surely also present in 10.2 and probably several earlier versions as well).

boegel · 2021-04-12T08:06:13Z

I've added perl as an OS dependency for GCCcore 10.3.0, but strictly speaking that's not exactly what we need: perl is only needed when building GCC?

boegel · 2021-04-12T08:27:34Z

I've added perl as an OS dependency for GCCcore 10.3.0, but strictly speaking that's not exactly what we need: perl is only needed when building GCC?

Actually, I guess that doesn't matter much, since an OS dependency is really only checking during the installation, there's no runtime check added to the module file for these...

SebastianAchilles

As far as I understood perl is only need when compiling GCCcore with nvptx support.

SebastianAchilles · 2021-04-12T09:06:28Z

easybuild/easyconfigs/g/GCCcore/GCCcore-10.3.0.eb

+    '0e135e1cc7cec701beea9d7d17a61bab34cfd496b4b555930016b98db99f922e',  # GCCcore-9.3.0_gmp-c99.patch
+]
+
+osdependencies = ['perl']


Suggested change

osdependencies = ['perl']

# needed for nvptx

if withnvptx:

osdependencies = ['perl']

Isn't osdeps a () list?

Just looked it up: I can be a STRING_OR_TUPLE_LIST.

https://github.com/easybuilders/easybuild-framework/blob/develop/easybuild/framework/easyconfig/easyconfig.py#L843-845

Do you prefer a tuple list?

We use [] elsewhere, so should be [] here too, for consistency.

The if withnvptx is not going to work though, unless it's moved down below the withnvptx = True...

Yes, my idea was to move this if withnvptx below withnvptx = True. In that way it's more clear why we need perl when someone just looks at the easyconfig in the future.
Also this could make it more clear that people can choose to disable nvptx in case perl is it not available (or if they don't want to install it). Although personally I think nvptx offload support is a really nice feature 😉

SebastianAchilles

@boegel This is what I actually had in mind.

SebastianAchilles · 2021-04-12T10:45:43Z

easybuild/easyconfigs/g/GCCcore/GCCcore-10.3.0.eb

+    '0e135e1cc7cec701beea9d7d17a61bab34cfd496b4b555930016b98db99f922e',  # GCCcore-9.3.0_gmp-c99.patch
+]
+
+osdependencies = ['perl']


Suggested change

osdependencies = ['perl']

easybuild/easyconfigs/g/GCCcore/GCCcore-10.3.0.eb

boegel · 2021-04-12T17:14:22Z

@boegelbot please test @ generoso
CORE_COUNT=16

boegelbot · 2021-04-12T17:15:09Z

@boegel: Request for testing this PR well received on generoso

PR test command 'EB_PR=12521 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12521 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 16525

Test results coming soon (I hope)...

- notification for comment with ID 817982484 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

SebastianAchilles · 2021-04-12T18:18:29Z

Test report by @SebastianAchilles
SUCCESS
Build succeeded for 10 out of 10 (10 easyconfigs in total)
jrlogin11.jureca - Linux centos linux 8.2.2004, x86_64, AMD EPYC 7742 64-Core Processor, Python 3.6.8
See https://gist.github.com/6188c8c2a7fbfc38547f6b80fee954d2 for a full test report.

boegelbot · 2021-04-12T18:37:24Z

Test report by @boegelbot
SUCCESS
Build succeeded for 10 out of 10 (10 easyconfigs in total)
generoso-c1-s-2 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/148fcc03b0654c5fa52129643045691a for a full test report.

Micket · 2021-04-13T01:33:02Z

Test report by @Micket
SUCCESS
Build succeeded for 10 out of 10 (10 easyconfigs in total)
vera-c1 - Linux centos linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake), Python 2.7.5
See https://gist.github.com/d5d46025fa8168dedcab2ee9f18aad48 for a full test report.

Micket

lgtm

@akesandgren

@akesandgren said he had no complaints about the current state of the PR.

Micket · 2021-04-13T13:58:06Z

Going in, thanks @boegel!

easyconfigs for GCC 10.2.1 (release candidate for GCC 10.3)

2ad2939

boegel added the update label Apr 5, 2021

boegel added this to the 4.x milestone Apr 5, 2021

update to use final GCC 10.3.0 release

aa08527

boegel changed the title ~~{toolchain} GCC 10.3.0 (WIP)~~ {toolchain} GCC 10.3.0 Apr 8, 2021

SebastianAchilles reviewed Apr 8, 2021

View reviewed changes

use correct binutils version for Bison/flex/help2man/M4/zlib built wi…

c142d03

…th GCCcore/10.3.0

easybuilders deleted a comment from boegelbot Apr 8, 2021

boegel modified the milestones: 4.x, next release (4.3.4), release after 4.3.4 Apr 8, 2021

add Perl as OS dependency for GCC 10.3.0

6e32609

SebastianAchilles reviewed Apr 12, 2021

View reviewed changes

Perl is only required when building with NVPTX support

c295cc4

robert-mijakovic mentioned this pull request Apr 12, 2021

{mpi}[GCCcore/10.3.0] OpenMPI v4.1.1 #12566

Merged

Micket approved these changes Apr 13, 2021

View reviewed changes

Micket merged commit 0a3f0f3 into easybuilders:develop Apr 13, 2021

boegel deleted the 20210405120828_new_pr_GCC1021 branch April 13, 2021 18:12

{toolchain} GCC 10.3.0 #12521

{toolchain} GCC 10.3.0 #12521

Conversation

boegel commented Apr 5, 2021 • edited Loading

boegel commented Apr 5, 2021

boegelbot commented Apr 5, 2021

boegelbot commented Apr 5, 2021

boegel commented Apr 8, 2021

boegelbot commented Apr 8, 2021

boegelbot commented Apr 8, 2021 • edited by boegel Loading

boegelbot commented Apr 8, 2021

boegel commented Apr 8, 2021

boegelbot commented Apr 8, 2021

boegel commented Apr 8, 2021

Micket commented Apr 8, 2021

boegel commented Apr 8, 2021

SebastianAchilles left a comment

Choose a reason for hiding this comment

boegel commented Apr 8, 2021

boegel commented Apr 8, 2021

boegelbot commented Apr 8, 2021

boegel commented Apr 8, 2021

akesandgren commented Apr 9, 2021

Micket commented Apr 9, 2021

akesandgren commented Apr 10, 2021 • edited Loading

boegel commented Apr 10, 2021

Flamefire commented Apr 10, 2021

boegel commented Apr 10, 2021

boegel commented Apr 10, 2021

Micket commented Apr 10, 2021

justbennet commented Apr 10, 2021

akesandgren commented Apr 10, 2021

Micket commented Apr 10, 2021

boegel commented Apr 12, 2021

boegel commented Apr 12, 2021

SebastianAchilles left a comment

Choose a reason for hiding this comment

SebastianAchilles Apr 12, 2021

Choose a reason for hiding this comment

akesandgren Apr 12, 2021

Choose a reason for hiding this comment

SebastianAchilles Apr 12, 2021

Choose a reason for hiding this comment

boegel Apr 12, 2021

Choose a reason for hiding this comment

SebastianAchilles Apr 12, 2021

Choose a reason for hiding this comment

SebastianAchilles left a comment

Choose a reason for hiding this comment

SebastianAchilles Apr 12, 2021

Choose a reason for hiding this comment

boegel commented Apr 12, 2021

boegelbot commented Apr 12, 2021

SebastianAchilles commented Apr 12, 2021

boegelbot commented Apr 12, 2021

Micket commented Apr 13, 2021

Micket left a comment

Choose a reason for hiding this comment

Micket commented Apr 13, 2021

boegel commented Apr 5, 2021 •

edited

Loading

boegelbot commented Apr 8, 2021 •

edited by boegel

Loading

akesandgren commented Apr 10, 2021 •

edited

Loading