Julia script creation system custom images

compile/README.md

@@ -0,0 +1,57 @@
+The Julia script available in this folder (`create_gridap_image.jl`) lets one to create a custom sysimage of `Gridap.jl` using the so-called [`PackageCompiler.jl`](https://github.com/JuliaLang/PackageCompiler.jl) Julia package. See [documentation page](https://julialang.github.io/PackageCompiler.jl/dev/sysimages/) for more details.
+
+The Julia script at hand provides a set of command-line-arguments (CLAs) in order to customize the creation process of the sysimage. You can see the list of options available using the following command:
+
+```bash
+$ julia create_gridap_image.jl -h 
+```
+
+In order to create the sysimage, the script clones the Git repository of `Gridap.jl` and that of its [Tutorials](https://github.com/gridap/Tutorials), and runs one of the tutorials as requested by the user (or the `validation.jl` tutorial by default). By default, the script will generate the sysimage from the code corresponding to the last commit pushed to the `master` branch of both Git repositories. However, via appropriate CLAs (see help message), one may select the particular version of both projects to be cloned by specifying the corresponding Git release tags. For example, if one wants to generate a sysimage of Gridap `v0.10.4` from Tutorials `v0.10.0`, one has to execute the following command:
+
+
+```bash
+$ julia -O3 --check-bounds=no create_gridap_image.jl -g v0.10.4 -t v0.10.0 -n Gridapv0.10.4.so
+```
+
+Another example is:
+
+```bash
+julia -O3 --check-bounds=no create_gridap_image.jl -g v0.14.1 -t v0.13.0 --tutorial-name stokes.jl -n Gridapv0.14.1.so
+```
+
+Obviously, one has to select a version of the Tutorials package compatible with the version selected for Gridap, **otherwise the image creation process will fail**. Note that in the above command, we are also specifying the name of the image to be generated, which by default, is created in the directory from which the script is executed
+(the path where the image is generated can be customized as well via the corresponding CLA).
+
+Once the image is generated, one may use it to execute a Julia script that uses Gridap as follows:
+
+```bash
+$ julia -J Gridapv0.10.4.so gridap_script.jl
+```
+
+or run an interactive Julia session as:
+
+```bash
+$ julia -J Gridapv0.10.4.so
+```
+
+### Lessons learned, caveats
+
+1. **The sysimage works and is useful when `Gridap.jl` is being used but not being modified**, e.g., while developing `GridapDistributed.jl` or `GridapODEs.jl`. If you want to develop `Gridap.jl`, use [`Revise.jl`](https://github.com/gridap/Gridap.jl/wiki/REPL-based-workflow#editing-small-parts-of-the-project).  It is also very useful for batch computing environments (e.g., a supercomputer), as you may significantly reduce the amount of CPU time spent by your jobs.
+
+2. Also note the following from the [documentation page](https://julialang.github.io/PackageCompiler.jl/dev/sysimages/) of `PackageCompiler.jl`: *It should be clearly stated that there are some drawbacks to using a custom sysimage, thereby sidestepping the standard Julia package precompilation system. The biggest drawback is that packages that are compiled into a sysimage (including their dependencies!) are "locked" to the version they where at when the sysimage was created. This means that no matter what package version you have installed in your current project, the one in the sysimage will take precedence. This can lead to bugs where you start with a project that needs a specific version of a package, but you have another one compiled into the sysimage.*
+
+ 3. We have observed that the resulting image does not contain all the code whose compilation is triggered from the tutorial. The code that is compiled during execution time can be spotted with, e.g.,
+    ```bash
+    $ julia --trace-compile=stderr -q -J Gridapv0.10.4.so validation.jl
+    ```
+    As an example, I got the following on my machine:
+    ```
+    $ julia --trace-compile=stderr ../../Tutorials/src/validation.jl 2>&1 | wc -l 
+    2920   
+    $ julia --trace-compile=stderr -J./Gridap.so ../../Tutorials/src/validation.jl 2>&1 | wc -l
+    825
+    ```
+
+4. In most cases, the user is willing to execute a Gridap driver that is not a tutorial (nor a minor modification of it). While he/she may still use the sysimage generated by this script, it may (likely) happen that the user's particular driver triggers the compilation of code that is not within the image. If the amount of code that is compiled on run-time cannot be tolerated, then you can use the approach followed in this repo: https://github.com/fverdugo/Poisson.jl. Essentially, to create an ad-hoc Julia environment for your particular driver, in which you create the sysimage from it. 
+
+5. See also issue [#276](https://github.com/gridap/Gridap.jl/issues/276).

compile/create_gridap_image.jl

@@ -0,0 +1,145 @@
+#!/usr/bin/env julia
+#
+# Examples:
+#      julia create_gridap_image.jl -h
+#      julia create_gridap_image.jl -g v0.10.4
+
+using Pkg
+Pkg.add("ArgParse")
+Pkg.add("PackageCompiler")
+using ArgParse
+using LibGit2
+using PackageCompiler
+const do_not_clone_gridap_flag="--do-not-clone-gridap"
+const do_not_clone_tutorials_flag="--do-not-clone-tutorials"
+
+function parse_commandline()
+    s = ArgParseSettings()
+    do_not_clone_gridap_flag="--do-not-clone-gridap"
+    do_not_clone_tutorials_flag="--do-not-clone-tutorials"
+    @add_arg_table! s begin
+        "--image-name", "-n"
+        help = "The name of the Gridap.jl custom system image that will be created"
+        arg_type = String
+        default = "Gridap.so"
+        "--image-path", "-p"
+        help = "The relative or absolute PATH where the Gridap.jl custom system image will be created"
+        arg_type = String
+        default = "./"
+        "--gridap-tag", "-g"
+        help = """Gridap.jl Git repo Tag string with the source code from which the Gridap.jl custom system
+                  image will be created. This option is ignored if $(do_not_clone_gridap_flag) flag IS PASSED.
+               """
+        arg_type = String
+        default = "master"
+        "--tutorials-tag", "-t"
+        help = "Tutorials Git repo Tag string with the base code line that will be executed in order to generate
+                the Gridap.jl custom system image. This option is ignored if $(do_not_clone_tutorials_flag) flag IS PASSED"
+        arg_type = String
+        default = "master"
+        "--gridap-path"
+        help = """If $(do_not_clone_gridap_flag) flag IS NOT PASSED, the relative or absolute PATH where to
+                  clone the Gridap.jl Git repo (Warning: Removed if it exists!).
+                  If $(do_not_clone_gridap_flag) flag IS PASSED, the relative or absolute PATH where an existing
+                  Gridap.jl source directory tree can be found.
+               """
+        arg_type = String
+        default = "/tmp/Gridap.jl/"
+        "--tutorials-path"
+        arg_type = String
+        default = "/tmp/Tutorials/"
+        help = """If $(do_not_clone_tutorials_flag) flag IS NOT PASSED, the relative or absolute PATH where to
+                  clone the Tutorials Git repo (Warning: Removed if it exists!).
+                  If $(do_not_clone_tutorials_flag) flag IS PASSED, the relative or absolute PATH where an existing
+                  Tutorials source directory tree can be found.
+               """
+        "--do-not-clone-gridap"
+        help = "Do not clone the Gridap.jl Git repo, but instead use an existing source directory."
+        action = :store_true
+        "--do-not-clone-tutorials"
+        help = "Do not clone the Tutorials Git repo, but instead use an existing source directory."
+        action = :store_true
+        "--tutorial-name"
+        help = "The name of the Julia script (including the .jl extension) with the tutorial from which the sys image is created."
+        default = "validation.jl"
+        arg_type = String
+    end
+    return parse_args(s)
+end
+
+function clone_and_checkout_tag(
+    repo_url::AbstractString,
+    repo_path::AbstractString,
+    git_tag::AbstractString,
+)
+    rm(repo_path; force = true, recursive = true)
+    repo = LibGit2.clone(repo_url, repo_path)
+    if (lowercase(git_tag) != "master")
+        @assert git_tag in LibGit2.tag_list(repo) "Wrong Git Tag $(git_tag). Available choices for $(repo_url) are $(LibGit2.tag_list(repo))"
+        gt=LibGit2.GitTag(repo, git_tag)
+        commit_hash=LibGit2.target(gt)
+        LibGit2.checkout!(repo, string(commit_hash))
+    end
+end
+
+
+function main()
+    parsed_args = parse_commandline()
+    image_name = parsed_args["image-name"]
+    image_path = parsed_args["image-path"]
+    gridap_tag = parsed_args["gridap-tag"]
+    tutorials_tag = parsed_args["tutorials-tag"]
+    gridap_path = parsed_args["gridap-path"]
+    tutorials_path = parsed_args["tutorials-path"]
+    tutorial_name = parsed_args["tutorial-name"]
+    if (! parsed_args["do-not-clone-gridap"])
+      clone_and_checkout_tag(
+        "https://github.com/gridap/Gridap.jl",
+        gridap_path,
+        gridap_tag,
+      )
+    end
+    if (! parsed_args["do-not-clone-tutorials"])
+      clone_and_checkout_tag(
+       "https://github.com/gridap/Tutorials",
+       tutorials_path,
+       tutorials_tag,
+      )
+    end
+
+    @info "Creating system image for Gridap.jl#$(gridap_tag) object file at: '$(image_path)'"
+    @info "Building Gridap.jl#$(gridap_tag) into system image: $(joinpath(image_path,image_name))"
+
+    start_time = time()
+
+    Pkg.activate(gridap_path)
+    Pkg.instantiate(verbose = true)
+
+    pkgs = Symbol[]
+    push!(pkgs, :Gridap)
+
+
+    if VERSION >= v"1.4"
+        append!(pkgs, [Symbol(v.name) for v in values(Pkg.dependencies()) if v.is_direct_dep],)
+    else
+        append!(pkgs, [Symbol(name) for name in keys(Pkg.installed())])
+    end
+
+    tutorial_script_path=joinpath(tutorials_path,"src",tutorial_name)
+    if ! isfile(tutorial_script_path)
+      @error "$(tutorial_script_path) not found or not a file"
+    end
+
+    # use package compiler
+    PackageCompiler.create_sysimage(
+        pkgs,
+        sysimage_path = joinpath(image_path,image_name),
+        precompile_execution_file = tutorial_script_path,
+    )
+
+    tot_secs = Int(floor(time() - start_time))
+    @info "Created system image object file at: $(joinpath(image_path,image_name))"
+    @info "System image build time: $tot_secs sec"
+end
+
+main()