LoadError und Failed to build gem native extension when using Asciidoctor gems

[CSV-Tom]

I am in the process of evaluating the new Asciidoctor-Gradle-Toolchain 4.0.3. I have encountered two challenges that I could not solve right away. Perhaps I took a wrong turn somewhere. Could you provide me with feedback or point me in the right direction where I might have gone wrong?

I managed to solve the first issue. I'm really struggling with the second one.

Environment

java --version

openjdk 21.0.3 2024-04-16
OpenJDK Runtime Environment (build 21.0.3+9-Ubuntu-1ubuntu120.04.1)
OpenJDK 64-Bit Server VM (build 21.0.3+9-Ubuntu-1ubuntu120.04.1, mixed mode, sharing)

./gradlew --version

------------------------------------------------------------
Gradle 8.9
------------------------------------------------------------

Build time:    2024-07-11 14:37:41 UTC
Revision:      d536ef36a19186ccc596d8817123e5445f30fef8

Kotlin:        1.9.23
Groovy:        3.0.21
Ant:           Apache Ant(TM) version 1.10.13 compiled on January 4 2023
Launcher JVM:  21.0.3 (Ubuntu 21.0.3+9-Ubuntu-1ubuntu120.04.1)
Daemon JVM:    /usr/lib/jvm/java-21-openjdk-amd64 (no JDK specified, using current Java home)
OS:            Linux 5.15.0-116-generic amd64

Additionally tested on Ubuntu 22.04 and Ubuntu 24.04 within Docker.

Example Code

plugins {
    id 'org.asciidoctor.editorconfig' version '4.0.3'
    id 'org.asciidoctor.jvm.convert' version '4.0.3'
    id 'org.asciidoctor.jvm.gems' version '4.0.3'
}

repositories {
    mavenCentral()
    ruby.gems()
}

dependencies {
    asciidoctorGems 'rubygems:rouge:3.30.0'
    asciidoctorGems 'rubygems:asciimath:2.0.5'
    asciidoctorGems 'rubygems:asciidoctor-lists:1.1.2'    
    // asciidoctorGems 'rubygems:asciidoctor-mathematical:0.3.5' 
}

asciidoctorj {
    requires 'rouge'
    requires 'asciimath'
    requires 'asciidoctor-lists'
    // requires 'asciidoctor-mathematical'

    jrubyVersion '9.4.8.0'

    modules {
        diagram {
            version '2.3.1'
        }
        pdf {
            version '2.3.17'
        }
    }
}

asciidoctorEditorConfig {
    attributes \
        'build-gradle': layout.projectDirectory.file('build.gradle')

    additionalAttributes asciidoctorj
    additionalAttributes layout.projectDirectory.file("asciidoctor.config")
}

asciidoctor {

    logDocuments = true

    outputOptions {
        backends = ['pdf']
    }

    attributes asciidoctorEditorConfig.attributes

    dependsOn  asciidoctorGemsPrepare, asciidoctorEditorConfig
}

build.dependsOn asciidoctor

Error Message 1: LoadError: (LoadError) no such file to load -- asciidoctor-lists

1 gem installed

> Task :asciidoctor
Exception in thread "main" org.jruby.exceptions.LoadError: (LoadError) no such file to load -- asciidoctor-lists
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1184)
        at RUBY.require(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85)
        at RUBY.<main>(<script>:1)

> Task :asciidoctor FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':asciidoctor'.
> Process 'command '/usr/lib/jvm/java-21-openjdk-amd64/bin/java'' finished with non-zero exit value 1

Solution:
The problem could be solved by adding withGemJar asciidoctorGemsJar within the Asciidoctor task:

asciidoctor {

    logDocuments = true

    outputOptions {
        backends = ['pdf']
    }

    attributes asciidoctorEditorConfig.attributes

    dependsOn  asciidoctorGemsPrepare, asciidoctorEditorConfig

    withGemJar 'asciidoctorGemsJar' 
}

Error Message 2: ERROR: Failed to build gem native extension.

The following error occurs after asciidoctor-mathematical has been included:

Successfully installed concurrent-ruby-1.3.3
Successfully installed i18n-1.14.5
Successfully installed ruby-enum-0.9.0
Building native extensions. This could take a while...
ERROR:  Error installing /home/<user>/.gradle/caches/modules-2/files-2.1/rubygems/mathematical/1.6.18/7ca3dcf196407c547dae654654e86f892cb8c493/mathematical-1.6.18.gem:
        ERROR: Failed to build gem native extension.

    current directory: /tmp/Test/build/.asciidoctorGems/gems/mathematical-1.6.18/ext/mathematical
/usr/lib/jvm/java-21-openjdk-amd64/bin/java -cp /home/<user>/.gradle/caches/modules-2/files-2.1/org.jruby/jruby-complete/9.4.8.0/812a4440518de50a3222ac85e9677e92e9c1272a/jruby-complete-9.4.8.0.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.jruby/jruby-complete/9.4.8.0/812a4440518de50a3222ac85e9677e92e9c1272a/jruby-complete-9.4.8.0.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.asciidoctor/asciidoctoreditorconfig/4.0.3/6f422e92e195722fd82b86d7511c228b1e38395e/asciidoctoreditorconfig-4.0.3.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.asciidoctor/asciidoctor-gradle-jvm-gems/4.0.3/e637e8b1f47b3cf282ab739860f4ccff8d7338d4/asciidoctor-gradle-jvm-gems-4.0.3.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.asciidoctor/asciidoctor-gradle-jvm/4.0.3/874d0e4e8a95f5717840d6e77883218a5ba7ba86/asciidoctor-gradle-jvm-4.0.3.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.asciidoctor/asciidoctor-gradle-base/4.0.3/922cc25a499464b43a0251851024589186bd72c4/asciidoctor-gradle-base-4.0.3.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle.jruby/jrubygradle-resolver/1.0.2/5207c025513ca5354eb158cf688d74693cde5c22/jrubygradle-resolver-1.0.2.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant-herd/3.0.1/55006d9403215cbb7418bb995fe4b39ca37e7367/grolifant-herd-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant80/3.0.1/dbc44df8a728cea14b6e11f18899b672b711bd4a/grolifant80-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant70/3.0.1/ae6bcc05cd765f8e17fcf2e8b5431b9415a76940/grolifant70-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant60/3.0.1/c0785d9f66a5b5f0fe00bb2c4fad674067b814db/grolifant60-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant50/3.0.1/38b0b6d39529c2a71f21c53be6d88da048f1b348/grolifant50-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant40-legacy-api/3.0.1/4274623a7e4c24769457a9363404bf640a30058/grolifant40-legacy-api-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant40/3.0.1/c370a49f41cc24b823fe584eaf3064777d3f2ff8/grolifant40-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant-core/3.0.1/d4898e72cdbf2233d60f4614000b87c38edc86a9/grolifant-core-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.ysb33r.gradle/grolifant-rawhide/3.0.1/343168621aa8ccd937c5259ea6d1b431555941c8/grolifant-rawhide-3.0.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.asciidoctor/asciidoctorj-api/2.5.7/37ea651d5f1ba2ad3d1c09eb2356b17f35c15bb5/asciidoctorj-api-2.5.7.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.6/5b6f921f1810bdf90e25471968f741f87168b64/xz-1.6.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.github.http-builder-ng/http-builder-ng-okhttp/1.0.3/f5a767dc67c1e60daba8cd974007aeef06068fbd/http-builder-ng-okhttp-1.0.3.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.ratpack/ratpack-core/1.6.1/5d6041a76c83769f09164d485609c30e26d2a958/ratpack-core-1.6.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.codehaus.groovy/groovy-nio/3.0.13/bfdc9d5c9ca5dc1efb0f9543dd732b7c20647fcd/groovy-nio-3.0.13.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.github.rburgst/okhttp-digest/3.1.0/c6b2fd76753c25f8f611d3b55fcb520b5b6f3aeb/okhttp-digest-3.1.0.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-collections4/4.4/62ebe7544cb7164d87e0637a2a6a2bdc981395e8/commons-collections4-4.4.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.github.http-builder-ng/http-builder-ng-core/1.0.3/ad2b181931a4c451f8f456c7b5eaffc9fda8fee6/http-builder-ng-core-1.0.3.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.squareup.okhttp3/okhttp/4.11.0/436932d695b2c43f2c86b8111c596179cd133d56/okhttp-4.11.0.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.ratpack/ratpack-exec/1.6.1/76566346af31fc2df3a30828d02ab4a76218d25e/ratpack-exec-1.6.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec-http/4.1.32.Final/b9218adba7353ad5a75fcb639e4755d64bd6ddf/netty-codec-http-4.1.32.Final.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-handler/4.1.32.Final/b4e3fa13f219df14a9455cc2111f133374428be0/netty-handler-4.1.32.Final.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.sun.activation/javax.activation/1.2.0/bf744c1e2776ed1de3c55c8dac1057ec331ef744/javax.activation-1.2.0.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.github.ben-manes.caffeine/caffeine/2.6.2/c8fd8817f9d2b2ced82f8968e0dd943aab557de0/caffeine-2.6.2.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.javassist/javassist/3.22.0-GA/3e83394258ae2089be7219b971ec21a8288528ad/javassist-3.22.0-GA.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-guava/2.9.8/b7f7819800f8ebe51a03dd951636f6dbc6245ce6/jackson-datatype-guava-2.9.8.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-jdk8/2.9.8/bcd02aa9195390e23747ed40bf76be869ad3a2fb/jackson-datatype-jdk8-2.9.8.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.datatype/jackson-datatype-jsr310/2.9.8/28ad1bced632ba338e51c825a652f6e11a8e6eac/jackson-datatype-jsr310-2.9.8.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.9.8/11283f21cc480aa86c4df7a0a3243ec508372ed2/jackson-databind-2.9.8.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.9.8/a1c807329eb0c75976aeb5961a506b3516ffeae3/jackson-dataformat-yaml-2.9.8.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.yaml/snakeyaml/1.23/ec62d74fe50689c28c0ff5b35d3aebcaa8b5be68/snakeyaml-1.23.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.codehaus.groovy/groovy/3.0.13/7652b22000c2cebbfdcfdf530a64e41baa518cb/groovy-3.0.13.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/xml-resolver/xml-resolver/1.2/3d0f97750b3a03e0971831566067754ba4bfd68c/xml-resolver-1.2.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.ratpack/ratpack-base/1.6.1/2fd3ffd5fa3bd0e5aed91814cf7e9a8c5b6d56db/ratpack-base-1.6.1.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-epoll/4.1.32.Final/9261e2930ab30681230a1192878dc2d17525350d/netty-transport-native-epoll-4.1.32.Final-linux-x86_64.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-codec/4.1.32.Final/8f32bd79c5a16f014a4372ed979dc62b39ede33a/netty-codec-4.1.32.Final.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport-native-unix-common/4.1.32.Final/3d2aea6a2873b0a6503645dc296bc0ce6d01a25a/netty-transport-native-unix-common-4.1.32.Final.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-transport/4.1.32.Final/d5e5a8ff9c2bc7d91ddccc536a5aca1a4355bd8b/netty-transport-4.1.32.Final.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-buffer/4.1.32.Final/46ede57693788181b2cafddc3a5967ed2f621c8/netty-buffer-4.1.32.Final.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.reactivestreams/reactive-streams/1.0.2/323964c36556eb0e6209f65c1cef72b53b461ab8/reactive-streams-1.0.2.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.9.0/7c10d545325e3a6e72e06381afe469fd40eb701/jackson-annotations-2.9.0.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.9.8/f5a654e4675769c716e5b387830d19b501ca191/jackson-core-2.9.8.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio-jvm/3.2.0/332d1c5dc82b0241cb1d35bb0901d28470cc89ca/okio-jvm-3.2.0.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.6.20/dab8089bca6ac0e394c37281ea8cff2f99acd421/kotlin-stdlib-jdk8-1.6.20.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.6.20/f8629f336bad4001c89e9cffa5ef3d4b5d0f5e22/kotlin-stdlib-jdk7-1.6.20.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.6.20/6cedc143badbb4f1c6b7f5a340b04edff1743208/kotlin-stdlib-1.6.20.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/21.0/3a3d111be1be1b745edfa7d91678a12d7ed38709/guava-21.0.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-resolver/4.1.32.Final/3e0114715cb125a12db8d982b2208e552a91256d/netty-resolver-4.1.32.Final.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/io.netty/netty-common/4.1.32.Final/e95de4f762606f492328e180c8ad5438565a5e3b/netty-common-4.1.32.Final.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.6.20/27b4562b6713d70f458c6d7ea39aadacb8e6a92b/kotlin-stdlib-common-1.6.20.jar:/home/<user>/.gradle/caches/modules-2/files-2.1/org.jetbrains/annotations/13.0/919f0dfe192fb4e063e7dacadee7f8bb9a2672a9/annotations-13.0.jar org.jruby.Main -I uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib extconf.rb
extconf failedBad file descriptor - /bin/sh

Gem files will remain installed in /tmp/Test/build/.asciidoctorGems/gems/mathematical-1.6.18 for inspection.
Results logged to /tmp/Test/build/.asciidoctorGems/extensions/universal-java-21/3.1.0/mathematical-1.6.18/gem_make.out
Successfully installed asciidoctor-mathematical-0.3.5
4 gems installed

> Task :asciidoctor
Exception in thread "main" org.jruby.exceptions.LoadError: (MissingSpecError) Gem::MissingSpecError
        at RUBY.activate_dependencies(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1453)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1981)
        at RUBY.activate_dependencies(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1439)
        at RUBY.activate(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1421)
        at RUBY.try_activate(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems.rb:211)
        at RUBY.require(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:153)
        at RUBY.<main>(<script>:1)
Caused by: org.jruby.exceptions.LoadError: (MissingSpecError) Gem::MissingSpecError
        at RUBY.to_specs(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/dependency.rb:311)
        at RUBY.activate_dependencies(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1451)
        ... 6 more
Caused by: org.jruby.exceptions.LoadError: (MissingSpecError) Gem::MissingSpecError
        at RUBY.activate_dependencies(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1453)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1981)
        at RUBY.activate_dependencies(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1439)
        at RUBY.activate(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1421)
        at RUBY.try_activate(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems.rb:205)
        ... 2 more
Caused by: org.jruby.exceptions.LoadError: (MissingSpecError) Gem::MissingSpecError
        at RUBY.to_specs(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/dependency.rb:311)
        at RUBY.activate_dependencies(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb:1451)
        ... 6 more
Caused by: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- asciidoctor-mathematical
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1184)
        at RUBY.require(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85)
        ... 1 more

> Task :asciidoctor FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':asciidoctor'.
> Process 'command '/usr/lib/jvm/java-21-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

[ysb33r]

This is due to a change in asciidoctorj itself, but basically if you do not need to use the require keyword as raw GEMs are now bundled into a temporary JAR. That JAR is on the load path for asciidoctorj. Just comment out

    requires 'rouge'
    requires 'asciimath'
    requires 'asciidoctor-lists'

[CSV-Tom]

If I comment out the following directives, their transformation into the PDF will no longer be performed.

Example 1: Commented-out for example asciidoctor-lists

asciidoctorj {
    ...     
    // requires 'asciidoctor-lists' Commented out
    ...
}

PDF Result oflist-of::listing[]:

List of code snippets

list-of::listing[]

Example 2: Commented-in for example asciidoctor-lists

asciidoctorj {

    ...     
    requires 'asciidoctor-lists'
    ...
}

PDF Result of list-of::listing[]:

List of code snippets

Create a basic PDF document using Prawn

Is a connection or configuration still missing here?

Unfortunately, this approach does not resolve the second error message mentioned earlier.

[9ao9ai9ar]

Error message 1 is related to issue #718, and I've successfully used markslater's workaround to load pygments.rb. Both OP's solution and ysb33r's suggestion would lead to some other errors for me.