Skip to content

Latest commit

 

History

History
191 lines (122 loc) · 6.25 KB

README.md

File metadata and controls

191 lines (122 loc) · 6.25 KB
Raw

Clara Framework multi-language application demo

Build status

Example of a Clara application wrapping a multi-language legacy library into services.

OpenCV is used to simulate a legacy application that detects faces and pupils on a given set of images. For simplicity the images are stored in a ZIP file. In a real-world scenario the dataset would be saved into a more complex file format, like HDF5.

Application architecture

The application is presented as a multi-project.

legacy subproject

The legacy subproject simulates the legacy application.

The Java implementation presents I/O classes and a simple image detection program:

  • The ImageReader class opens a ZIP file and read images from it.
  • The ImageWriter class opens a ZIP file and writes images to it.
  • The Image class stores the content and name of a file image.
  • The FaceDetector class finds faces in a image.

The C++ implementation adds a new detection algorithm:

  • The Image class stores the content and name of a file image.
  • The PupilDetector class finds pupils in a image.

The legacy subproject creates the demo-legacy-0.1.jar and libdemo_legacy.so artifacts.

data subproject

When the services are developed to be shared with other developers and investigation groups, it is a good idea to present the data types used by the services into its own package.

Orchestrators can then add a dependency on the data type package to send requests to the services, without downloading all the services implementation dependencies.

The data subproject creates the demo-data-0.1.jar and libdemo_data.so artifacts.

services subproject

The final Clara services are just a tiny layer over the legacy code. The most important thing to consider is thread-safety of the legacy classes. Clara services will run in a multi-thread environment and developers must ensure thread-safety.

The Java implementation provides the ImageReaderService and the ImageWriterService to open the ZIP dataset files and read or save the images. The services extend the standard Clara I/O services AbstractEventReaderService and AbstractEventWriterService.

The Java FaceDetectorService and the C++ PupilDetectorService pass any received image to its wrapped detection class.

The services subproject creates the demo-services-0.1.jar and libpupils_detector.so, artifacts.

Dependencies

Install Clara Java and Clara C++ into $CLARA_HOME.

Install OpenCV 4 following these instructions.

Build and deploy

Build and deploy the Java services into $CLARA_HOME:

$ ./gradlew
$ ./gradlew deploy

Build and deploy the C++ services into $CLARA_HOME:

$ mkdir build
$ cd build
$ cmake ..
$ make
$ make install

Note: The build may fail if OpenCV cannot be found in a standard location.

  • For Gradle, set the custom OpenCV install prefix in the opencvDir property:

    $ ./gradlew -PopencvDir=/app/opencv4/ [ deploy ]

  • For CMake, set the custom OpenCV install prefix in the OpenCV_ROOT variable:

    $ cmake -DOpenCV_ROOT=/app/opencv4 ..

Running the application

Using the Clara Shell

Use the clara-shell to run the services locally with a set of ZIP files (each containing many images). The shell provides a high-level interface to configure and start the different Clara components required to run an application.

  1. Start the Clara shell:

    $ $CLARA_HOME/bin/clara-shell

  2. Define the application within a services.yaml file. There is one file ready in the root of the repository.

    clara> set servicesFile services.yaml

  3. Define the data set with the input files to be processed. Put a list with their names into a text file. There is a files.list file ready in the root of the repository.

    clara> set fileList files.list

  4. Define the locations for the input data set and the output result (relative to where the shell was started):

    clara> set inputDir resources
clara> set outputDir /tmp/out

    Optionally you can change the number of parallel threads used by the services to process requests

    clara> set threads <NUM_THREADS>

  5. Start the data processing. This will start the main Java DPE, a C++ DPE if the C++ service is listed in services.yaml, and it will run the standard orchestrator to process the dataset.

    clara> run local

  6. Inspect the output files in the output directory.

You can put the above shell commands into a script and run the script instead. There is a demo.clara script ready in the root of the repository:

$ $CLARA_HOME/bin/clara-shell demo.clara

Starting the Clara DPEs and orchestrator manually

All Clara components started by the Clara shell can be run manually.

Use the standard clara-orchestrator to run the services with a set of ZIP files (each containing many images):

  1. Start the main Java DPE:

    $ $CLARA_HOME/bin/j_dpe

  2. Start the C++ DPE (if the application uses C++ services):

    $ $CLARA_HOME/bin/c_dpe --fe-host localhost

  3. Define the application with a services.yaml file. There is one file ready in the root of the repository.

  4. Define the data set with the input files to be processed. Put a list with the names into text file. There is a files.list file ready in the root of the repository.

  5. Run the orchestrator with the application configuration file, the input and output directories, and the list of input files:

    $ $CLARA_HOME/bin/clara-orchestrator -F -i ./resources -o /tmp/out services.yaml files.list

    To change the number of parallel threads used by the services to process requests, pass the -t <NUM_THREADS> option.

  6. Inspect the output files in the output directory.

Example