Restate logging utilities

[tillrohrmann]

In order to avoid duplicate log outputs on replay, we could offer the user some logging utility that only outputs log statements when not being in replay mode. That way, users wouldn't see their logging statements output multiple times in the presence of suspension points.

[tillrohrmann]

@slinkydeveloper how is this problem solved in the Java SDK?

[slinkydeveloper]

@slinkydeveloper how is this problem solved in the Java SDK?

We don't need any logging facade ourselves, we rely on log4j2 and we use the log4j2 context to propagate relevant restate metadata: https://github.com/restatedev/sdk-java#logging

I have to expand that guide, but there's a way you can configure log4j2 to filter out logs (e.g. logs on replay), print logs as json, and so forth.

[tillrohrmann]

We don't need any logging facade ourselves, we rely on log4j2 and we use the log4j2 context to propagate relevant restate metadata: https://github.com/restatedev/sdk-java#logging

I have to expand that guide, but there's a way you can configure log4j2 to filter out logs (e.g. logs on replay), print logs as json, and so forth.

This is great :-) I've created restatedev/documentation#182 to update the logging documentation for the Java SDK.

[slinkydeveloper]

I've been looking around for logging solutions in nodejs. This is what I've found:

• AWS Lambda provides its own logger tool: https://docs.powertools.aws.dev/lambda/typescript/latest/core/logger/
• There's a bunch on famous logging tools, notably winston. A setup working with the lambda request id: https://betterdev.blog/aws-lambda-logging-best-practices/#simple_logger_config
• Most of these APIs have the following look:

import Logger from `log-library`;

// in global context
let logger = new Logger(loggerConf);

export const handler = async (event, context) => {
  logger.info()
};

If we wanna follow the same API style, we probably need to use the AsyncLocalStorage to carry around the "replaying" state. Otherwise we need to either provide the logger from the RestateContext, or define our own logger interface and get RestateContext as parameter.

I think a winston integration is doable where we provide a custom "format" that disables entries in "replay". See https://github.com/winstonjs/winston#filtering-info-objects. From my understanding the same format can also add contextual informations. The usage could look like:

import winston from 'winston';
import * as restate from "@restatedev/restate-sdk";

const logger = winston.createLogger({
    level: process.env.LOG_LEVEL || 'info',
    format: winston.format.combine(
      restate.winstonFormat(), // This filters logs on replay and adds restate invocation id
      winston.format.json(),
    ),
    transports: [
        new winston.transports.Console(),
    ],
});

export class GreeterService implements TestGreeter {
  async greet(request: TestRequest): Promise<TestResponse> {
    const ctx = restate.useContext(this);

    // state
    let seen = (await ctx.get<number>("seen")) || 0;
    seen += 1;

    await ctx.set("seen", seen);

    logger.info('Updating logger to ${seen}');

    // return the final response
    return TestResponse.create({
      greeting: `Hello ${request.name} no.${seen}!`,
    });
  }
}

[slinkydeveloper]

The alternative solution is to provide our own logger interface, e.g. based on the built-in console, that the user can access with context.console.

This logger implementation will then have access to the state machine object, in order to filter out logs in replaying mode and to add contextual info. Probably we also need to provide a way to switch between json/text format logging, as json logging is an important feature for many deployment scenarios.

Example:

import * as restate from "@restatedev/restate-sdk";

export class GreeterService implements TestGreeter {
  async greet(request: TestRequest): Promise<TestResponse> {
    const ctx = restate.useContext(this);

    // state
    let seen = (await ctx.get<number>("seen")) || 0;
    seen += 1;

    await ctx.set("seen", seen);

    ctx.logger.info('Updating logger to ${seen}');

    // return the final response
    return TestResponse.create({
      greeting: `Hello ${request.name} no.${seen}!`,
    });
  }
}

IMO we should integrate with another logging framework, rather than building our own, but I guess we can also have both if we want to.