Retrieving a SmallRyeConfig instance fails on Android

[caberger]

On Android the following Statement crashes:

SmallRyeConfig config = ConfigProvider.getConfig().unwrap(SmallRyeConfig.class);

with a ProviderNotFoundException.

Propably the reason is that io.smallrye.config.AbstractLocationConfigSourceLoader does not pass the ClassLoader that has been passed in to consumeAsPaths(...) and so ClassPathUtils::processAsPath uses (ClassLoader) null as argument instead.

In addition to that AbstractLocationConfigSourceLoader::tryClassPath() does not catch the ProviderNotFoundException, which is a RunTimeException, not an IOException.

SmallRye config works perfectly on Android when I use a self-coded ConfigSourceProvider, which does nothing more than use getClass().getClassLoader() to load the property files.

Doing that SmallRye Config can perfectly be used in Android with java 17 by adding implementation("io.smallrye.config:smallrye-config:3.4.4") to build.gradle.kt, and adding a resources/ folder to src/main with e.g. an application.properties and a META-INF/microprofile-config.properties file.
If you need an example Android App to reproduce the problem please tell me.

After fixing the above small Problem SmallRye config would be a perfect microprofile config implementation also for Android , in my opinion the best and most mature way to configure Android Apps.

[radcortez]

Yes, please provide a reproducer. Thanks!

[caberger]

Propably the root of the problem is in principle not smallrye config itself, but the fact that Android is missing the jar file provider for java.nio.file, so using ConfigProvider.getConfig().unwrap(SmallRyeConfig.class) crashes. The problem of missing jar file support has been reported very long ago in Jar and Zip FileSystemProvider not present on API 26+.
You can find the reproducer in caberger/android-jetpack-compose-smallrye-config, please read the readme.md

[radcortez]

Unfortunately, I need to update my version of IntelliJ does not support your reproducer, so I couldn't run it. I did create one Android project from a template, added SR Config and it seemed to work.

Can you please, paste me the stacktrace you get? Thanks!

[caberger]

I repeated your steps an I confirm that the problem does not appear in an new Android Start Project, when you retrieve a configuration only, as long as there is no microprofile-config.properties file.

To reproduce, please switch in the Project pane from "Android" to "Project" and add a folder app/src/main/resources/META-INF/microprofile-config.properties

Just for now I added this info and the stack trace to the readme in the github expample project caberger/android-jetpack-compose-smallrye-config

There you can also find details about how to create the project and reproduce the problem.

[radcortez]

I did add a file, but apparently, it was not being picked up. I can now see the stacktrace.

Not sure what we can do... it is unfortunate that Android does not support nio jar, but URLConnection is still able to read the file from a jar, so I don't see any big reason as to why not support it in the filesystem.

I can try to figure out if it is possible to achieve the behaviour we have right now and get rid of the filesystem jar call.

[caberger]

Well, I think the Problem is in ClassPathUtils::processAsPath()
It parses into an internal representation of an url, e.g. it searches for an exclamation mark '!'. In my opinion the format of this URL should be left private to the ClassLoader implementation of the platform.

What about changing consumeAsPaths to consumeAsInputStreams, like in the following:

    public class MainActivity extends ComponentActivity {
    private static final String TAG = MainActivity.class.getName();

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        try {
            consumeAsInputStreams(getClass().getClassLoader(), "META-INF/microprofile-config.properties", inputStream -> {
                var properties = new Properties();
                try {
                    properties.load(inputStream);
                    // TODO: use these properties for config
                    properties.forEach((key, value) -> Log.i(TAG, String.format("%s -> %s", key, value))); // <-- remove this
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            });
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        //...
    }
    void consumeAsInputStreams(ClassLoader cl, String resource, Consumer<InputStream> consumer) throws IOException {
        final var resources = cl.getResources(resource);
        while (resources.hasMoreElements()) {
            final var res = resources.nextElement();
            try (var is = res.openStream()) {
                consumer.accept(is);
            }
        }
    }
}

It works on Android, the full app is here.

ouch, I understand to change this would touch very stable, well tested and often used code ...
... still we do not know what the future and other platforms will do with their internal URL formats...

[radcortez]

Well, I think the Problem is in ClassPathUtils::processAsPath() It parses into an internal representation of an url, e.g. it searches for an exclamation mark '!'. In my opinion the format of this URL should be left private to the ClassLoader implementation of the platform.

Yes, but it does it, because the URI is from a jar scheme, returned by the Classloader, and a jar URI scheme uses ! to represent an entry. The issue as you stated initially is that Android does not support a fs provider for jar. The reason we do this is to avoid locking the jar on Windows.

We also support passing a jar URI as a location to load a configuration.

Finally, we cannot rely on the CL getResources to load the profile-aware files. We pick up the URI of the main file and rewrite the URI with the expected profile name to load it from the same location to make sure the order is retained. I guess we could load all the resources, but then we would be required to match them manually to make sure the order is retained.

ouch, I understand to change this would touch very stable, well tested and often used code ...
... still we do not know what the future and other platforms will do with their internal URL formats...

Correct. The URL format is not the issue, but Android not supporting the jar fs. Not exactly sure how we can fix this... rewriting the loading implementation would be tricky, but I would also like to fix this issue.

[caberger]

Yes, I absolutely agree to your arguments.

It is hard to touch that code.

• The exception appears in tryClassPath, which has only 56% test coverage (zip of jacoco html report attached). This is one of the lowest test coverage percentage in the whole project.

• I could not find a test that asserts that resources are only loaded with ClassPathUtils and not
  directly by CL (Windows Jar Locking Problem) - maybe I just missed it. If we rewrite to using CL this test should propably be added before that.

• the 2nd statement in tryClassPath contains SecuritySupport.getContextClassLoader(), which uses a deprecated API. I suppose this will be changed soon anyway, so better not touch this code until the deprecated API has been removed. Additionally this statement also has a missed branch in the Jacoco report.

So maybe it is better to prepare those things first and only after that touch the good and reliable existing CL/FileSystem code.

[radcortez]

• The exception appears in tryClassPath, which has only 56% test coverage (zip of jacoco html report attached). This is one of the lowest test coverage percentage in the whole project.
• I could not find a test that asserts that resources are only loaded with ClassPathUtils and not
  directly by CL (Windows Jar Locking Problem) - maybe I just missed it. If we rewrite to using CL this test should propably be added before that.

The coverage should be a bit higher:
https://sonarcloud.io/component_measures?id=smallrye_smallrye-config&metric=coverage&selected=smallrye_smallrye-config%3Aimplementation%2Fsrc%2Fmain%2Fjava%2Fio%2Fsmallrye%2Fconfig%2FAbstractLocationConfigSourceLoader.java

Not all the tests are inside the implementation folder.

• the 2nd statement in tryClassPath contains SecuritySupport.getContextClassLoader(), which uses a deprecated API. I suppose this will be changed soon anyway, so better not touch this code until the deprecated API has been removed. Additionally this statement also has a missed branch in the Jacoco report.

Yes, we would need to remove SecurityManager, but at the moment, we are still supporting Java 11, and application servers that use SmallRye Config rely on SecurityManager, so we cannot remove it just yet.

So maybe it is better to prepare those things first and only after that touch the good and reliable existing CL/FileSystem code.

Feel free to add additional tests. You can find most of the testes about the classpath loading here: https://github.com/smallrye/smallrye-config/blob/27b00d0f9f61969777cf72b8bcddc5bdce80d0c2/testsuite/extra/src/test/java/io/smallrye/config/test/location/PropertiesLocationTest.java.