feat: VoyageAI encoder

[xRiddin]

New Feature: Integration of super fast and accurate VoyageAI Embeddings into semantic-router

[jamescalam]

thanks @xRiddin — running again

[jamescalam]

@xRiddin you can run poetry lock --no-update to recreate the poetry.lock file, after committing that it should fix the test issue above

[xRiddin]

@jamescalam done 👍

[xRiddin]

@jamescalam also the the maximum length of the list is 128 for Voyage AI api. If the user feeds an insane amount of data inside the lists, it will likely throw an error. Need to find a workaround for that.

[jamescalam]

Running:

poetry run pytest tests/unit/encoders/test_voyageai.py

Returns errors (same as being returned in the tests here), seem to only be mocking problems afaict — @xRiddin are you able to resolve? Let us know if you want us to jump in and help on the remaining mocking/tests

I also moved voyageai from a required install to an optional install, and modified the import in the encode file accordingly. Also renamed client to _client to align with other API-based encoders (for example the Mistral encoder)

[xRiddin]

@jamescalam I'm not able to resolve the mocking problems, any help on the remaining tests are very much appreciated.

[kurtismassey]

Just to help with the test problems, the test is looking to assert the error message outputs as "VOYAGE API client failed to initialize. Error: Initialization error" but because e.args returns a tuple the output from the failure is actually "Unable to connect to VoyageAI ('Initialization error',): Initialization error".

To pass the assertion you could update your test assertion to match the args output or you could update the actual error message to ValueError(f"VOYAGE API client failed to initialize. Error: {e}") which should pass

[kurtismassey]

The last test problems seem to be due to the handling of exceptions in the exponential backoff, currently it will stop on the first exception and not retry at all due to the exception being raised straight away. To get it to continue for the specified number of retries as the test requires you could just add a line to continue the retries before the raise which should pass those other tests

except Exception as e:
                logger.error(f"VoyageAI API call failed. Error: {error_message}")
                if j < 6:
                    sleep(2 ** j)
                    continue
                raise ValueError(f"VoyageAI API call failed. Error: {e}") from e