README.md

marqo-go (Unofficial Go Client)

Marqo is more than a vector database, it's an end-to-end vector search engine for both text and images. Vector generation, storage and retrieval are handled out of the box through a single API. No need to bring your own embeddings.

Features

• Easy connection to Marqo with minimal setup
• Index creation and management
• Document upsertion and retrieval
• Advanced search capabilities
• Index statistics and health monitoring

Getting Started

Prerequisites

• Docker (Ensure that Docker has at least 8GB memory and 50GB storage)

Installation

1. Marqo requires docker. To install Docker go to the Docker Official website. Ensure that docker has at least 8GB memory and 50GB storage.

2. Use docker to run Marqo for Mac users:

## will start marqo-os and marqo service mentioned in docker compose.
docker compose up -d

3. Get the Marqo Go client:

go get github.com/ganeshdipdumbare/marqo-go@latest

Usage

Example

package main

import (
	"fmt"

	marqo "github.com/ganeshdipdumbare/marqo-go"
)

// Document represents a document

type Document struct {
	ID          string `json:"_id"`
	Title       string `json:"title"`
	Description string `json:"description"`
	Genre       string `json:"genre"`
}

func main() {

	marqoClient, err := marqo.NewClient("http://localhost:8882")
	if err != nil {
		panic(err)
	}
	model, err := marqoClient.GetModels()
	if err != nil {
		panic(err)
	}
	fmt.Printf("Model: %+v\n", model)

	indexes, err := marqoClient.ListIndexes()
	if err != nil {
		panic(err)
	}

	fmt.Printf("Indexes: %+v\n", indexes)

	// create index
	resp, err := marqoClient.CreateIndex(&marqo.CreateIndexRequest{
		IndexName: "test1",
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("CreateIndexResponse: %+v\n", resp)

	// create document
	documents := []Document{
		{
			ID:          "1",
			Title:       "The Great Gatsby",
			Description: "The Great Gatsby is a 1925 novel by American writer F. Scott Fitzgerald.",
			Genre:       "Novel",
		},
		{
			ID:          "2",
			Title:       "The Catcher in the Rye",
			Description: "The Catcher in the Rye is a novel by J. D. Salinger, partially published in serial form in 1945–1946 and as a novel in 1951.",
			Genre:       "Novel",
		},
	}
	docInterface := make([]interface{}, len(documents))
	for i, v := range documents {
		docInterface[i] = v
	}

	upsertResp, err := marqoClient.UpsertDocuments(&marqo.UpsertDocumentsRequest{
		IndexName: "test1",
		Documents: docInterface,
		TensorFields: []string{
			"title",
			"description",
		},
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("UpsertDocumentsResponse: %+v\n", upsertResp)

	// delete documents
	deleteResp, err := marqoClient.DeleteDocuments(&marqo.DeleteDocumentsRequest{
		IndexName:   "test1",
		DocumentIDs: []string{"1", "2"},
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("DeleteDocumentsResponse: %+v\n", deleteResp)

	// Refresh index
	refreshResp, err := marqoClient.RefreshIndex(&marqo.RefreshIndexRequest{
		IndexName: "test1",
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("RefreshIndexResponse: %+v\n", refreshResp)

	// Get index stats
	statsResp, err := marqoClient.GetIndexStats(&marqo.GetIndexStatsRequest{
		IndexName: "test1",
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("GetIndexStatsResponse: %+v\n", statsResp)

	// Get index settings
	settingsResp, err := marqoClient.GetIndexSettings(&marqo.GetIndexSettingsRequest{
		IndexName: "test1",
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("GetIndexSettingsResponse: %+v\n", *settingsResp.IndexDefaults.Model)

	// Get index health
	healthResp, err := marqoClient.GetIndexHealth(&marqo.GetIndexHealthRequest{
		IndexName: "test1",
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("GetIndexHealthResponse: %+v\n", healthResp)

	// Search
	searchQuery := "The Great Gatsby"
	searchResp, err := marqoClient.Search(&marqo.SearchRequest{
		IndexName: "test1",
		Q:         &searchQuery,
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("SearchResponse: %+v\n", searchResp)

	// Bulk search
	//searchQuery := "The Great Gatsby"
	indexName := "test1"
	bulkSearchResp, err := marqoClient.BulkSearch(&marqo.BulkSearchRequest{
		Queries: []marqo.SearchRequest{
			{
				Index: &indexName,
				Q:     &searchQuery,
			},
		},
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("BulkSearchResponse: %+v\n", bulkSearchResp)

}

Improvements

• Add unit tests for all the APIs.
• Add end-to-end examples for search.

Contributing

Contributions are welcome! Please open an issue or submit a pull request for any improvements or bug fixes.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgements

• Marqo for providing the vector search engine.
• Go for the programming language.