scalecube-events

[ronenhamias]

scalecube-events:

Motivation:
When a service needs data owned by another service, there are two main strategies to obtain the sought data:

1. A service can ask for the data from the service that owns it, and wait until the data is sent back to it. This is a point-to-point communication pattern.

2. A service can subscribe to "events" or "topic" of events and in case a publisher publish on this topic handle the events and data. this is publish-subscribe communication model.

Scalecube currently gives answer for the current pattern of communication between services

• request response
• request many
• request channel.
• fire and forget

scalecube does not answer pubsub communication pattern.
such as multicast, unicast and topic notion communication patterns between services.

such capability is commonly provided by a message broker. Various technologies that can be used as a message broker exist, such as Google Cloud Pub/Sub, Kafka, and RabbitMQ.

the pub-sub api aims to allow such communication pattern and abstracts the specific message broker technologies, and makes it dead simple for services to share events. supports an implementation of the Message Broker API that uses Kafka, and other implementations may become available in the future.

a business example for pubsub is when node wish to broadcast stock-quotes rate to all rate consumers in the cluster. in such case the rate is a "public stream" and its same rate for all subscribers.

on the other hand Aeron publisher subscriber offer the notion of broadcasting which comes more intuitive when using UDP transport.

in case of PubSub patterns we can provide alternative channels for broadcasting messages and managing discovery and failures in the cluster on top of low latency high volume transport such as Aeron.

1. using aeron multicast and unicast.
2. we should discover the publishers location on the cluster and create a "topic registry".
3. when publish or broadcast we discover the endpoints.

publish
send singlecast message to a single subscriber using routing such as round robin or custom routing algo.

broadcast
send multicast message to all subscribers currently running in the cluster.

// Add an event service subscriber
ms.events().topics("test1","test2").subscribe(payload -> {
  
});

ms.events().topics("/scalecube/topic1", "/scalecube/topic2").publish(payload);

ms.events().topics("test1", "test2").boradcast(payload);

Sugar:

@Service("greetings")
public interface GreetingService {
   
     @ServiceTopic()  
     void sayHello(Flux<Greeting> greetings);

}

TopicPublisher pub = ms.events().topic("/greetings/sayHello");
pub.publish(new Greeting("John"));

see some poc:
https://github.com/ronenhamias/scalecube-pubsub

background:
https://hackernoon.com/events-as-first-class-citizens-8633e8479493

[snripa]

@ronenhamias what about onMessage instead of subscribe not to be confused with reactor's API?

[snripa]

and, do you have any suggestions regarding implementation?

[segabriel]

I think it will be better use separated operator to specify topic/queue name. And we can subscribe to more than one topic:

ms.events().topics("test1", "test2").publish("Hello world!").subscribe();

[ronenhamias]

@segabriel @snripa

IMHO implementation should be based on Aeron transport.

1. using aeron multicast and unicast.
2. we should publish the handlers location on the cluster in the same way we discover service methods in the cluster.
3. when publish or broadcast we discover the endpoints and using aeron publish to these endpoints/

regards your suggestions for the api it makes sense.

publish
send singlecast message to a single subscriber using routing such as round robin or custom routing algo.

broadcast
send multicast message to all subscribers currently running in the cluster.

// Add an event service subscriber
ms.events().topics("test1","test2").subscribe(message -> {
  
});

ms.events().topics("test1", "test2").publish("Hello world!").subscribe();

ms.events().topics("test1", "test2").boradcast("Hello world!").subscribe();

Sugar:

@Service("greetings")
public interface GreetingService {
   
     @ServiceTopic()  
     void sayHello(Flux<Greeting> greetings);

}

TopicPublisher pub = ms.events().topic("/greetings/sayHello");
pub.publish(new Greeting("John"));

[artem-v]

Terminology:

Connections:

PubSub classes:

Topic management:

Topic Registry functions:

Subscriber functions:

Publisher registration and msg sending:

Publisher deregistration: