require('libraryName')

[EToreo]

Is there a way to have sol2 make available lua-written libraries for "include"?

For example, I have a library I would like to use called "differed". It consists of one file "differed.lua" but expects to be required: local deferred = require('deferred'). I would like a way to do something similar to lua.open_libraries on the file and have it available for require. Is this possible with sol2?

[ThePhD]

I have thought about making a lua.require( ... ); function. It seems like it would take key argument (the key for the module / whatever you want to open) and then the next argument would either be a script that returns or loads an argument or a file. Note that lua has something similar for the C API, but it does not take script files and the like. Potential API:

// essentially wraps up luaL_requiref
lua.require("woofers", open_woofers_library ); // opens using any kind of function

lua.require_script("woofers", "return { bark = 20 }" ); 
// or something, no idea how that would work
lua.require_file("deferred", "deferred.lua" ); // fictional

I have no idea how I'd make the other 2 work. In all honestly, I'd probably just call lua.open_libraries( sol::lib::require); and then do lua.script( stringkey + "require(" + filename + ")" ); internally, like the true plebeian I am.

[EToreo]

I would love to see lua.require_script("woofers", "return { bark = 20 }" );. Your suggestion about adding sol::lib::require doesn't work for me because I'm being very cautious about what I expose to my lua users. Anything that can cause any IO, or open up the potential for IO can't be given to them. Its a great suggestion otherwise.

[ThePhD]

Question: what happens if this fails? Do we just toss an error?

[ThePhD]

Nevermind, I'm a dumb. Of course we just lua_error out.

[EToreo]

Lol, sounds good to me. I can't imagine it failing for too many reasons.

[ThePhD]

You're all set:

    std::string code = "return { modfunc = function () return 221 end }";

    sol::state lua;
    sol::table thingy1 = lua.require_script("thingy", code);
    sol::table thingy2 = lua.require_script("thingy", code);

    int val1 = thingy1["modfunc"]();
    int val2 = thingy2["modfunc"]();
    // val1 == val2 == 221
    bool reference_the_same_thing = thingy1 == thingy2; // true

[ThePhD]

You can also require_file and just plain require. The plain one takes a function to call in case it needs to open the module.

[EToreo]

This is really cool and I will pick it up once I get sol compiling on windows.

[syed-ahmed]

Hi, I think I have a similar issue. I need to run a torch/luajit script from a C++ program and have just started learning about lua C api and sol2 library. My script initially looks like:

require 'nn'
require 'cunn'
require 'cudnn'

model = torch.load("<path to serialized file>")
#take an input from C++
#do more stuff
#return an output from C++ 

Can you give an example how the 'require' portion of my script would work using sol2?
Thank you!

[ThePhD]

I don't know where require when called from Lua goes looking, so I can't duplicate the functionality of it exactly. IIRC, it has code to load C libraries of certain names and also to check other local settings, so I am fairly unsure if I can give you an example. What are 'nn', 'cunn', and 'cudnn' ? Script files? C Libraries DLLs?

If you need exactly those libraries, try just running lua.script("require 'nn'");. I know this is not ideal, but since I don't exactly know what those are I can't offer anymore sound advice.

Note that even if sol2 doesn't do exactly what you want, if you do it yourself using the C API and find out how you can show me and I can spend some time working it into the library.

[syed-ahmed]

Hi @ThePhD . Thank you for your reply. So these nn, cunn, cudnn are packages that gets installed by luarocks during the torch installation process. Luarocks calls 'make' on these folders: https://github.com/torch/distro/tree/master/extra. I am not sure where exactly these packages are lying. Here is the installation script: https://github.com/torch/distro/blob/master/install.sh

I don't know if that helps (i'm a beginner) but my end goal is to write a C++ command line interface that does the following:

---TODO:
---Write the lua C++ interface that passes an input tensor from c++ to lua and then fetches
---the output tensor from lua

1. Get input image from source
2. Feed processed image as a tensor in the lua program
    a. Get Input tensor
    b. Feed into model
    c. return output tensor
3. Get output tensor

An example that does something similar is the torch-android demo and it's relevant code is here. The torch android demo says

• Loading of lua packages from the apk directly.
• This is done by writing a custom package.loader Reference: http://www.lua.org/manual/5.1/manual.html#pdf-package.loaders The loader is in torchandroid.cpp as loader_android

So I'm wondering if there is a way I can use package.loader in sol2 that can load the require 'nn', 'cunn' packages before my lua script is run.

[syed-ahmed]

Here's an error I get when I compiled the C++ program loading a lua script with those require 'nn', require 'cunn' lines: attempt to call global 'require' (a nil value), when I just use lua.script("require 'nn'");.
Thinking if there is a to load 'nn', 'cunn' packages like you do in lua.open_libraries(sol::lib::base)

[Nava2]

Here is how I load libraries:

sol::state lua;
        lua.open_libraries(sol::lib::base, sol::lib::package);
        const std::string package_path = lua["package"]["path"];
        lua["package"]["path"]
            = package_path + (!package_path.empty() ? ";" : "") + test::scripts_path("proc/valid/") + "?.lua";

The lua["package"]["path"] variable is what the Lua engine uses to look for scripts or files. You'll need to adjust the path to look for your libraries in the LuaRocks packages.

[syed-ahmed]

Hi @Nava2 , thanks for your reply. I did a luarocks path and got the following:

export LUA_PATH='/home/ubuntu/.luarocks/share/lua/5.1/?.lua;/home/ubuntu/.luarocks/share/lua/5.1/?/init.lua;/home/ubuntu/torch/install/share/lua/5.1/?.lua;/home/ubuntu/torch/install/share/lua/5.1/?/init.lua;./?.lua;/home/ubuntu/torch/install/share/luajit-2.1.0-beta1/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua'

export LUA_CPATH='/home/ubuntu/.luarocks/lib/lua/5.1/?.so;/home/ubuntu/torch/install/lib/lua/5.1/?.so;/home/ubuntu/torch/install/lib/?.so;./?.so;/usr/local/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so'

I browsed to /home/ubuntu/torch/install/share/lua/5.1/?.lua and found the .lua scripts for all the 'nn', 'cunn' packages installed by lua rocks. Can you help me modify your lines using these path? I don't quite understand how the lua["package"]["path"] is working.

[Nava2]

Does this help?

[syed-ahmed]

@Nava2 Sorry, that didn't really help. Like I understand in my case, it could be package.path = package.path .. ';/usr/local/share/lua/5.1/?/?.lua'? So integrating with your code, should it be:

const std::string package_path = lua["package"]["path"];
        lua["package"]["path"]
            = package.path .. ';/usr/local/share/lua/5.1/?/?.lua';

[Nava2]

Close, you'd want to do:

lua["package"]["path"] = package_path + ";/usr/local/share/lua/5.1/?/?.lua";

[syed-ahmed]

@Nava2 Sorry for a late reply. Was working on something different. I tried your suggestion. However, it loads the exact same paths as this line does

lua.open_libraries(sol::lib::base, sol::lib::package);

Eitherway, I get the following error:

terminate called after throwing an instance of 'sol::error'
  what():  lua: error: caught (...) exception
Aborted (core dumped)

The error specifically happens in the lines - require 'nn', require 'cunn', require 'cudnn' - in the lua script. I have made sure the path for those modules are lib package. Any help is appreciated.

[Nava2]

What does the lua code print(package.path) print if you run it after you add the extra package_paths?

Could you show your full (relevant) code?

[syed-ahmed]

@Nava2
Here is my code for the lua script called avg.lua:

print (package.path)
require 'nn'
require 'cudnn'
require 'cunn'
require 'torch'

M = torch.load("/home/ubuntu/Desktop/model.net")

Here is the C++ code:

#include <iostream>
#include "sol.hpp"

int main() {

    sol::state lua;

    lua.open_libraries(sol::lib::base, sol::lib::package);
    const std::string package_path = lua["package"]["path"];
    lua["package"]["path"] = package_path + ";/usr/local/share/lua/5.1/?/?.lua";

    lua.script_file("avg.lua");

    std::string important_string = lua["M"];
    std::cout << important_string << std::endl;

}

And here's my build command:

g++-4.9 -std=c++14 -W -Wall -g -o torchwrapper torchwrapper.cpp -I/home/ubuntu/torch/install/include -L/home/ubuntu/torch/install/lib -lluajit

Following is the output of print(package.path) before I get the error:

/home/ubuntu/.luarocks/share/lua/5.1/?.lua;/home/ubuntu/.luarocks/share/lua/5.1/?/init.lua;/home/ubuntu/torch/install/share/lua/5.1/?.lua;/home/ubuntu/torch/install/share/lua/5.1/?/init.lua;./?.lua;/home/ubuntu/torch/install/share/luajit-2.1.0-beta1/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/local/share/lua/5.1/?/?.lua

[Nava2]

And you've verified the package nn lives at: /usr/local/share/lua/5.1/nn/nn.lua?

[syed-ahmed]

@Nava2 I actually don't have a /usr/local/share/lua/5.1 (it gets added with the extra package path). nn is living in /home/ubuntu/torch/install/share/lua/5.1/nn/init.lua