Comsos DB as storage backend

[marius-stanescu-archive360]

Could you also add Cosmos DB as a storage backend?
Related to #638 and #1105.

[jpkrohling]

We have already Elasticsearch, Cassandra, Badger and in-memory in the main repository. To support other storage backends, we have support for storage plugins (#1461). We might be missing the documentation of how to develop one such plugin, but the PR introducing that feature also added an example:

jaeger/examples/memstore-plugin/main.go

Lines 30 to 65 in d3ee676

	var configPath string
	func main() {
	flag.StringVar(&configPath, "config", "", "A path to the plugin's configuration file")
	flag.Parse()
	if configPath != "" {
	viper.SetConfigFile(path.Base(configPath))
	viper.AddConfigPath(path.Dir(configPath))
	}
	v := viper.New()
	v.AutomaticEnv()
	v.SetEnvKeyReplacer(strings.NewReplacer("-", "_", ".", "_"))
	opts := memory.Options{}
	opts.InitFromViper(v)
	grpc.Serve(&memoryStore{store: memory.NewStore()})
	}
	type memoryStore struct {
	store *memory.Store
	}
	func (ns *memoryStore) DependencyReader() dependencystore.Reader {
	return ns.store
	}
	func (ns *memoryStore) SpanReader() spanstore.Reader {
	return ns.store
	}
	func (ns *memoryStore) SpanWriter() spanstore.Writer {
	return ns.store
	}

Right now, we are not hosting plugins in the main repository, but given enough traction, we can certainly host it at this organization.

As such, I'm closing this issue, but do let us know in case you develop a plugin for Cosmos DB!

[omerlh]

Hey, @marius-stanescu I started to play a bit with the GRPC plugin API in dotnet, should be pretty simple to write a plugin. Is that something you're interested at?

[yurishkuro]

I am going to re-open this and link from #638. We can close once we have a clear answer.

I was under impression that CosmosDB has a wire protocol compatible with Cassandra, is it not the case?

[yurishkuro]

There are some discussions about CosmosDB here: #1105 and #1472

[yurishkuro]

Pining @sagaranand015 @nutharsh @486 @lukasmrtvy who previously commented on issues related to CosmosDB - did anyone get it to work?

[sagaranand015]

@yurishkuro : Checked the cosmosDb connection with cassandra API and as mentioned in #1105, it works by removing the SnappyCompressor in Cassandra connection. Also, the keyspace creation using templates do not work because of the commas(which Cosmos db does not support)

How do you think we should proceed here? A Plugin approach to Cosmos db or a storage backend altogether?

Also, should the compressor be forced or should it be driven by config? (This seems like a change we can make before going to plugin/storage backend development).

[yurishkuro]

@sagaranand015 these changes sound very minor. I don't think it makes sense to treat it as a new storage backend, just parameterize the existing code to allow to control for these variations.

[sagaranand015]

@yurishkuro :
So, this boils down to 2 action items:

1. Make the Cassandra Compressor drive from config. How about adding --cassandra.compression=(True/False) config from flags? Default being SnappyCompressor if --cassandra.compression is true(true by default) (Issue: Do not force cassandra protocol's compressor #1105)
2. Change the Storage backend documentation to point out the following changes for CosmosDB connection with Cassandra API:
   a. --cassandra.compression flag to be false
   b. change the generated keyspace templates to remove unsupported commas.

If the above sounds good, will raise a PR for the same. Please let me know. Thanks

[yurishkuro]

@sagaranand015 yes, sounds good