Project Tracking: Client Instrumentation

[martinkuba]

Description

We believe that in order for OpenTelemetry to be adopted across the board, it needs to have a good support for client applications. Client instrumentation has some unique challenges that are currently not well supported.

Our initial goals include

• define which signal(s) are used for different types of client telemetry
• define semantic conventions for different types of client events
• define semantic conventions for client-specific resources
• define how sessions are handled in SDKs and represented across the wire

Beyond these, we expect that this will be a long-term ongoing project, which might include development of client-optimized SDKs. Also, given that there are many types of client applications/environments, this group may branch into multiple groups over time.

Project Board

https://github.com/orgs/open-telemetry/projects/19/views/1

Deliverables

• Semantic conventions for (most common) client events
  • draft PR for two browser events
• Semantic conventions for (most common) client resources
  • Ephemeral Resources OTEP
• Logs/Events API + SDK in the Javascript SDK
  • specification PR
  • API draft implementation in JS
• Logs/Events API + SDK for Android
• Logs/Events API + SDK for Swift
• Implementation of sessions in the Javascript SDK
• Research + proposal for how browser SDK should be optimized
  • sandbox repository for experimentation

Staffing / Help Wanted

Expertise required: browser instrumentation, mobile instrumentation

The group currently has several regular contributors who have expertise in browser instrumentation. Most of the focus so far has been on browser and shared concerns across all types of client applications. We need more participation from mobile in order to move mobile instrumentation forward.

Required staffing

Project lead(s): @martinkuba, @scheler, @MSNev

TC sponsoring members:

Meeting Times

• SIG meeting times: Wednesdays 8am PT
• Secondary meeting times: Tuesdays 9am PT

Timeline

Timeline: 6 - 12 months

Labels

[The tracking issue should be properly labeled to indicate what parts of the specification it is focused on.]

Linked Issues and PRs

[All PRs, Issues, and OTEPs related to the project should link back to the tracking issue, so that they can be easily found.]

[tigrannajaryan]

I added this issue to the top-level board https://github.com/orgs/open-telemetry/projects/29/views/1

Also unassigned from myself. Should be assigned to the workgroup's lead who will be keeping the issue up to date.

[reyang]

Is this a proposal for having a SIG or a project?
What's the relationship between this proposed project/SIG and the Specification: Logs SIG?
Why would a "Client Instrumentation" project/SIG try to deliver "Specification for Logs/Events API"? Would that be specific to client instrumentation only, and how would a library (which can be used on client/server/etc.) benefit from such API?

[tigrannajaryan]

@reyang to clarify, I asked @martinkuba to open the tracking issue so that we can try to follow the new process for projects. However, I guess we should have first discussed if it needs to be a project or a SIG as you rightfully asked. I am not sure we have a good litmus test to decide.

[tigrannajaryan]

Why would a "Client Instrumentation" project/SIG try to deliver "Specification for Logs/Events API"?

It should not be a "Client Instrumentation" deliverable, it should be a dependency. The Logs and Events API is a Logging SIG deliverable and is in progress there.

[scheler]

@tigrannajaryan @reyang what is the difference between a SIG and a project? I suggest we keep this a SIG as we need to work out many different independent topics, which could each be a project. This SIG will mostly be a Spec SIG, do you think we should move this SIG under Specification SIGs? On prototyping and implementation, we will be working close with the respective language SIGs but this SIG is mostly about how we do things in a standard way across different for client-side telemetry (RUM) SDKs (Browser, Android, iOS, etc).

[tigrannajaryan]

The difference is not well-defined. Project typically have limited time duration, they have a specific goal and once achieved are considered done. SIGs typically are expected to last as long as OpenTelemetry itself exists.

[reyang]

The TC (Technical Committee) had a discussion this morning, here goes our recommendation:

1. For things that are generic (e.g. design the log/event API), it should be part of the existing specification SIGs, as @tigrannajaryan has explained here Project Tracking: Client Instrumentation #2734 (comment).
2. For semantic conventions that are domain specific (client resource, client telemetry), these need to be aligned with the overall semantic convention direction, which the Instrumentation SIG is driving. @jsuereth is exploring how to help the Instrumentation SIG to move forward. Meanwhile this "Client Instrumentation" group can work in parallel, with the understanding that semantic convention will take time to become Stable. We also encourage this group to work closely with the Instrumentation SIG and help to move things forward.

Here are some of my personal suggestions:

1. The "Client" term is vague and confusing, e.g. is CLI (command line tools) considered as client or not? I suggest either have a better name or clarify/define what "client" is.
2. The implementation (e.g. Android) should happen in the language specific implementation SIG (e.g. for Android, maybe OpenTelemetry Java or OpenTelemetry C++ depending on whether JNI / reverse-JNI is needed). It should not end up with a separate project such as "OpenTelemetry Java (Android)".

[scheler]

@jsuereth @reyang @bogdandrutu @tigrannajaryan can we have a TC member assigned to this project? We are stuck again with open-telemetry/opentelemetry-js#3222 which needs a future-proof path to [Ephemeral Resource Attributes[(https://github.com/open-telemetry/oteps/pull/208) - this has been pending for a long time and only a TC member could help take this forward.

The RUM use-cases are different from what OTel supports today - OTel is currently best designed for data flow through services, but the data generated at the source all belong to different traces and yet require data/attributes common to them. Examples include session id, current page url, etc.

The alternates suggested to us so far all force us to fit the RUM use-cases into the current design, but it's not healthy long term. If we were evolve OTel to support these use-cases now is a very good time to look into this.

[jsuereth]

Regarding a TC member to support this project, we'll raise this in the next TC meeting. One big issue we have is the TC meeting itself conflicts with your SiG meeting times (every other session).

For now, I'd ask the Client-SiG (and instrumentation authors) to drive a solution for (request, session, etc.) scoped-attribute attachment to signals. I don't think OTEP 208 is the right solution (can comment on the OTEP). I think what the client-instrumentation team needs is a way to define scoped-attributes that isn't at conflict with how instrumentation authors view writing instrumentation (i.e. something more akin to automatic Baggage attribute inclusion). I'd suggest specifically looking at attaching attributes to context rather than Resource and building out an automatic interaction between Context + Signals here. I tried to do something previously (metric-specific) with Baggage, but I think this is the missing link for what your SiG needs. Would be happy to meet offline to brainstorm/discuss. Unfortuantely, your SiG is during an unmovable meeting for me.

[scheler]

@jsuereth We can definitely change the SIG meeting time to accommodate a TC member's schedule. The core members of this SIG (myself, @martinkuba and @MSNev) meet on Tuesdays 9am PT as well (This is on the OTel public calendar as RUM OTEP working group) in addition to Wednesdays 8am PT. If this time works for you, can you join us next week? Otherwise, we can try and meet later in the day as well - please let us know a couple times next week that work for you. We could go over with you what our thinking is and what we have tried so far.

[scheler]

@jsuereth any update on having a TC member help us on this project? Did you get a chance to raise it in a TC meeting?

[austinlparker]

@dyladan will be GC Facilitator/Sponsor.

This is a new responsibility the GC is adopting to better support cross-functional projects in OpenTelemetry. There will be a document in the community repo in the future detailing these responsibilities -- in general, the facilitator will help ensure that required TC sponsors are available and assigned, and will be a resource for project maintainers.

[danielgblanco]

As a change since the previous comment, it is me that's working as GC liason for this project. However, as we move towards tracking projects in the Main Backlog board and this SIG already has a Project Document, I'm going to go ahead and close this issue.

@martinkuba, @scheler, @MSNev can you confirm the contents of the Project Document still match the deliverables for the client-side instrumentation SIG? At least initially. If the SIG needs to be permanent or not can be discussed later. If deliverables don't match, can you please update them?