SentenceTransformer Unrecognized option: -c

[iprovalo]

Describe the bug
Getting this exception

Unrecognized option: -c
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

When calling
mainJepInterpreter.eval("sentence_transformer = SentenceTransformer(\"sentence-transformers/gtr-t5-large\")");

The next statement is executing correctly:
mainJepInterpreter.eval("output = sentence_transformer.encode('test')[0]");

To Reproduce
Java Code:

import jep.Interpreter;
import jep.JepConfig;
import jep.MainInterpreter;
import jep.SharedInterpreter;

import java.util.concurrent.*;

public class JepTest {
    private static ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();

    /**
     * Pass venv and python version:
     *          "/Users/XXX/venv_python_3.9_latest" "3.9"
     *
     *  Tested with jep 4.0.3
     *
     * 1. Activate your venv
     * 2. Install jep in that virtual environment: `python -m pip install jep==4.0.3`
     * 3. Install components you want to use with your python code: `python -m pip install sentence_transformers`
     * */
    public static void main(String[] args) {
        JepInterpreter jepInterpreter = JepInterpreter.getInstance(true, args[0], args[1]);
        JepTest.scheduleWithBoolean(jepInterpreter, true);
        System.exit(0);
    }


    public static Boolean scheduleWithBoolean(Callable<Boolean> callable, Boolean isDebug) {
        Boolean returnValue = false;
        Future<Boolean> future = scheduledExecutorService.schedule(callable, 0l, TimeUnit.MILLISECONDS);
        try {
            returnValue = future.get();
        } catch (Throwable e) {
            System.err.println("Could not process on a thread");
            if(isDebug) e.printStackTrace();
        }
        return returnValue;
    }


    private static class JepInterpreter implements Callable<Boolean> {
        private final Boolean isDebug;
        private final String venv;
        private final String pythonVersion;
        private Interpreter mainJepInterpreter;

        private static JepInterpreter uniqueInstance;

        public static synchronized JepInterpreter getInstance(Boolean isDebug, String venv, String pythonVersion) {
            if (uniqueInstance == null) {
                uniqueInstance = new JepInterpreter(isDebug, venv, pythonVersion);
            }
            return uniqueInstance;
        }

        private JepInterpreter(Boolean isDebug, String venv, String pythonVersion) {
            this.isDebug = isDebug;
            this.venv = venv;
            this.pythonVersion = pythonVersion;
        }

        private Boolean init() {
            String venvPythonPathLocalMac = this.venv;
            if (isDebug) System.out.println("VENV_PYTHON_PATH_LOCAL_MAC: "+venvPythonPathLocalMac);
            String sitePackages = venvPythonPathLocalMac + "/lib/python"+this.pythonVersion+"/site-packages";
            if (isDebug) System.out.println("Site Packages in venv: "+sitePackages);
            String pythonVenvExec = venvPythonPathLocalMac + "/bin/python3";
            if (isDebug) System.out.println("Executable in venv: " + pythonVenvExec);
            System.setProperty("jep.python", pythonVenvExec);
            System.setProperty("python.home", venvPythonPathLocalMac);
            String jepPath = sitePackages + "/jep/libjep.jnilib";
            if (isDebug) System.out.println("jep lib in venv: "+ jepPath);
            MainInterpreter.setJepLibraryPath(jepPath);
            JepConfig jepConf = new JepConfig();
            jepConf.addIncludePaths(sitePackages);
            jepConf.addIncludePaths("src/main/python/components/");
            SharedInterpreter.setConfig(jepConf);

            System.out.println("Java jep.python=" + System.getProperty("jep.python"));
            System.out.println("Java python.home=" + System.getProperty("python.home"));

            mainJepInterpreter = new SharedInterpreter();

            //TODO: generic settings for all python components:
            mainJepInterpreter.eval("import os");
            mainJepInterpreter.eval("import sys");
            mainJepInterpreter.eval("import io");

            mainJepInterpreter.eval("venv_path = '"+venvPythonPathLocalMac+"'");

            mainJepInterpreter.eval("output = io.StringIO()");
            mainJepInterpreter.eval("sys.stdout = output");

            mainJepInterpreter.eval("print(sys.executable)");
            String output = (String) mainJepInterpreter.getValue("output.getvalue()");
            System.out.println("Python sys.executable=" + output);

            mainJepInterpreter.eval("print(sys.path)");
            output = (String) mainJepInterpreter.getValue("output.getvalue()");
            System.out.println("Python sys.path=" + output);

            mainJepInterpreter.eval("import logging");

            if (isDebug)
                mainJepInterpreter.exec("logging.basicConfig(level=logging.DEBUG, format=\"%(asctime)s %(threadName)s (%(module)s:%(lineno)d) %(levelname)s: %(message)s\")");
            else {
                mainJepInterpreter.exec("logging.basicConfig(level = logging.INFO, format = \"%(asctime)s (%(module)s:%(lineno)d) %(levelname)s: %(message)s\")");
            }

            long start = System.currentTimeMillis();
            mainJepInterpreter.eval("from sentence_transformers import SentenceTransformer");
            mainJepInterpreter.eval("sentence_transformer = SentenceTransformer(\"sentence-transformers/gtr-t5-large\")");
            mainJepInterpreter.eval("output = sentence_transformer.encode('test')[0]");
            long end = System.currentTimeMillis();
            output = (String) mainJepInterpreter.getValue("str(output)");
            if (isDebug)
                System.out.println(output + " (Took " + (end - start) + "ms)");

            return true;
        }

        @Override
        public Boolean call() {
            return init();
        }

    }

}

Cannot reproduce from jep console:

from sentence_transformers import SentenceTransformer
sentence_transformer = SentenceTransformer("sentence-transformers/gtr-t5-large")

Expected behavior
No exception thrown.

Environment (please complete the following information):

• OS Platform, Distribution, and Version: macOS, Sonoma 14.2.1
• Python Distribution and Version: 3.9
• Java Distribution and Version: openjdk 19.0.2 2023-01-17
• Jep Version: 4.0.3
• Python packages used (e.g. numpy, pandas, tensorflow): sentence_transformers

[bsteffensmeier]

Unrecognized option: -c
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

This error indicates that the JVM was not even created but your comments indicate specific lines are the problem. I do not understand how those lines would execute if jvm creation failed.

[iprovalo]

@bsteffensmeier that’s my confusion as well. The jvm starts and completes with the right output. The error is printed out but seems to be ignored.

[bsteffensmeier]

It looks like sentence transformer is using a multiprocessing mechanism which is trying to spawn new processes. I suspect it is trying to start python processes however since jep is a java process it may be spawning java processes instead which does not work. It looks like multiprocessing.set_executable() can be used to point to the correct python executable. I am not sure if torch mukltiprocessing is using multiprocessing internally but it is worth a try.

[iprovalo]

@bsteffensmeier your explanation makes sense, thank you!

Indeed, adding two lines after the initial imports fixes the issue:

           //TODO: generic settings for all python components:
            mainJepInterpreter.eval("import os");
            mainJepInterpreter.eval("import sys");
            mainJepInterpreter.eval("import io");

           //Added these two lines as a fix:
            mainJepInterpreter.eval("import multiprocessing");
            mainJepInterpreter.eval("multiprocessing.set_executable('"+pythonVenvExec+"')");