Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allow setting runtime handles #997

Closed
wants to merge 2 commits into from
Closed

allow setting runtime handles #997

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cbb5aeb
C-API
fs-eire Oct 18, 2024
6cdd29f
impl
fs-eire Oct 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions src/ort_genai_c.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ typedef enum OgaElementType {
typedef struct OgaResult OgaResult;
typedef struct OgaGeneratorParams OgaGeneratorParams;
typedef struct OgaGenerator OgaGenerator;
typedef struct OgaRuntimeSettings OgaRuntimeSettings;
fs-eire marked this conversation as resolved.
Show resolved Hide resolved
typedef struct OgaModel OgaModel;
// OgaSequences is an array of token arrays where the number of token arrays can be obtained using
// OgaSequencesCount and the number of tokens in each token array can be obtained using OgaSequencesGetSequenceCount.
Expand Down Expand Up @@ -149,6 +150,27 @@ OGA_EXPORT OgaResult* OGA_API_CALL OgaLoadAudios(const OgaStringArray* audio_pat

OGA_EXPORT void OGA_API_CALL OgaDestroyAudios(OgaAudios* audios);

/*
* \brief Creates a runtime settings instance to be used to create a model.
* \param[out] out The created runtime settings.
* \return OgaResult containing the error message if the creation of the runtime settings failed.
*/
OGA_EXPORT OgaResult* OGA_API_CALL OgaCreateRuntimeSettings(OgaRuntimeSettings** out);
/*
* \brief Destroys the given runtime settings.
* \param[in] settings The runtime settings to be destroyed.
*/
OGA_EXPORT void OGA_API_CALL OgaDestroyRuntimeSettings(OgaRuntimeSettings* settings);

/*
* \brief Sets a specific runtime handle for the runtime settings.
* \param[in] settings The runtime settings to set the device type.
* \param[in] handle_name The name of the handle to set for the runtime settings.
* \param[in] handle The value of handle to set for the runtime settings.
* \return OgaResult containing the error message if the setting of the device type failed.
*/
OGA_EXPORT OgaResult* OGA_API_CALL OgaRuntimeSettingsSetHandle(OgaRuntimeSettings* settings, const char* handle_name, void* handle);

/*
* \brief Creates a model from the given configuration directory and device type.
* \param[in] config_path The path to the model configuration directory. The path is expected to be encoded in UTF-8.
Expand All @@ -158,6 +180,16 @@ OGA_EXPORT void OGA_API_CALL OgaDestroyAudios(OgaAudios* audios);
*/
OGA_EXPORT OgaResult* OGA_API_CALL OgaCreateModel(const char* config_path, OgaModel** out);

/*
* \brief Creates a model from the given configuration directory, runtime settings and device type.
* \param[in] config_path The path to the model configuration directory. The path is expected to be encoded in UTF-8.
* \param[in] settings The runtime settings to use for the model.
* \param[in] device_type The device type to use for the model.
* \param[out] out The created model.
* \return OgaResult containing the error message if the model creation failed.
*/
OGA_EXPORT OgaResult* OGA_API_CALL OgaCreateModelWithRuntimeSettings(const char* config_path, OgaRuntimeSettings* settings, OgaModel** out);

/*
* \brief Destroys the given model.
* \param[in] model The model to be destroyed.
Expand Down
Loading