Expand GenAI project to cover instrumentation libraries

[lmolkova]

Based on the discussions in LLM semconv group, we'd like to suggest expanding the scope of the group to cover instrumentation libraries.

The long-term goal of the project is:

• provide space within otel for different GenAI observability vendors to collaborate on GenAI instrumentations
• provide official otel-hosted or native/first-party instrumentations for popular GenAI components covering current https://github.com/traceloop/openllmetry scope and transfer corresponding PyPI packages ownership to OTel
• evolve GenAI semantic conventions to meet generic and unique GenAI observability needs (already covered with existing LLM semconv)

We intend to add instrumentations one-by-one and limit features to those documented in semantic conventions. Instrumentations will evolve along with semconv, but may provide additional hooks or opt-in features allowing to customize them to specific vendor needs.

We're going to request opentelemetry-genai repo for the coordination/project management, but may request additional repos later to host WIP gen-ai instrumentation libraries when using existing contrib repos is problematic. with the long-term goal to contribute them upstream or merge to contrib once possible.

We're looking for engineers to work on instrumentation libraries, especially in JavaScript and existing JS-contrib approvers/maintainers to help with code review, best practices and general OTel.

/cc @alolita @tedsuo @trask @nirga @drewby @karthikscale3 @lzchen

[karthikscale3]

Thank you for bringing this topic up @lmolkova . From Langtrace, myself and my colleague, Ali are happy to contribute/maintain/approve the channges.

[lmolkova]

Tagging more people in the community who might be interested to work on instrumentation libraries @AmanAgarwal041 @cartermp @codefromthecrypt @langchain4j @trentm

[nirga]

@lmolkova happy to assist here! I think my co-founder @galkleinman would also want to participate

[alizenhom]

@lmolkova More than happy to help here!

[xrmx]

Would it be possible to move this to 9AM PST? It would be a bit more easier to partecipate from central europe timezones :)

[lmolkova]

Sure, I updated the doc - will start a poll to find more times. BTW we have another meeting at 8am Thu CET, would that one work for you?

[drewby]

Does the meeting time change (for rest of world) when the clocks move back an hour in November?

[lmolkova]

I believe time stays the same in the timezone it's specified.

community/README.md

Lines 130 to 131 in ad757c7

	> All meeting hours between 2pm and 1am UTC are in Pacific Time (PT), with Daylight Saving Time.
	> All meeting hours between 2am and 1pm UTC in UTC+8, without Daylight Saving Time.

[danielgblanco]

Very happy to see this taking shape!

[codefromthecrypt]

A couple org/scope related questions which are possibly just because I am new to how things are structured. Please forgive if they are misplaced.

[codefromthecrypt]

in otel terminology we have both maintainer and approver defined, are we saying in these repos there is no difference? Cool with me, just curious.

[lmolkova]

we don't usually separate maintainers and approvers in project templates.

For this project I believe we'll work in the following repos:

• central repo for issues/discussions/maybe prototypes/artifacts, but probably no implementation code - opentelmetry-genai (new).
  • Maintainers/approvers listed here will be the same for this repo
• semantic-conventions and corresponding semconv-llm-approvers which are codeowners for GenAI files. I think we need to expand this group and include more people who've been active and consistent contributors to semantic conventions.
• opentelemetry-js-contrib maintainers/approvers for JS will become component owners, I hope @trentm would keep me honest here
• opentelemtery-python-contrib - similar component-ownership model
• new repos - it seems that we would need to start python work in a separate repo and if it's the case, we'll need to decide. I updated the list and added existing Otel Python/JS maintainers to maintainers for potential new repos.

[patcher9]

Great to see this in motion, Few nits as to keep vendor neutrality from this proposal. Me and @AmanAgarwal041 are happy to contribute instrumentations from https://github.com/openlit/openlit aswell

[codefromthecrypt]

I used to maintain js and python for zipkin and currently dusting off my skills, reviewing and maintaining these languages again.

If you need another approver who will pull changes local, test them, verify against current practice, etc before clicking ✔️ , feel free to use me. There's a lot of work to do, and I suspect this will help keep wheels in APAC moving.

[codefromthecrypt]

Suggested change

	- *looking for existing OTel JS approvers/maintainers for general JS & OTel feedback*
	- @codefromthecrypt
	- *looking for existing OTel JS approvers/maintainers for general JS & OTel feedback*

[lmolkova]

It'd be great to have you!

There are requirements for approvers/maintainers status in the community
which boils down to the history of contributions.

The list in this PR is an initial set and is expected to evolve based on the existing process.

[codefromthecrypt]

It was worth a shot!

Meanwhile, I'll open an issue on the community project because there is a chicken/egg condition which is a special case to new projects looking to acquire new people. For example, it discusses "the codebase" whereas no code exists yet. I'll take that offline.

[patcher9]

Same comment tbh here aswell
#2326 (comment)

[codefromthecrypt]

Suggested change

	- @xrmx
	- @xrmx
	- @codefromthecrypt

[patcher9]

Would preferably want to be more than a contributor as this would affect our SDK aswell like others

Suggested change

	- @xrmx
	- @xrmx
	- @patcher9
	- @AmanAgarwal041

[lmolkova]

There are some requirements around approver/maintainer status in the community
https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md

which boils down to the history of previous contributions.

A good place to start would be to first request to join OTel org: https://github.com/open-telemetry/community/issues/new?assignees=&labels=area%2Fgithub-membership&projects=&template=membership.md&title=REQUEST%3A+New+membership+for+%3Cyour-GH-handle%3E

and after making some contributions ask to become an approver.

[patcher9]

Can I ask previous contributions to OTel for everyone in the Codeowners/approvers list here (Not taking a shot at anyone or personal, Just curious)? Just wanna get a grasp on how much would we need to be since @AmanAgarwal041 already has a JS PR for OpenAI Instrumentation in review much like the Python one

[xrmx]

@patcher9 I think you are adding yourself to the maintainer list where currently there's me and Leighton that are already maintainers for the Python SIG.
AFAIK people listed as component / code owners are people that already contributed to semconv-llm.

[codefromthecrypt]

@patcher9 maybe focus on the approvers part as "maintainer" is a higher bar. I'm not sure when this doc needs to land, but someone could propose you or aman as a community member now. You might already be a member before this doc merges. Many people have only been members 6 months or less.

Once a member, you can choose to try to make the case that some approvers have not worked on code (instrumentation) and not this semver doc or visa versa. Also, the case about the code contribution already underway, as well points about this project aiming to widen the community since it has such a large new scope, and obviously you folks already have experience in it. I'd vote for you, though I don't think approver status is a vote thing at this point.

[patcher9]

Yeah approvers is what I had in mind my bad (Maintainers I understand why we cant be assigned). This to an extent does seem very closed source tbh even for a new project (we can say its an extension but if we look at it the work has already been started for the JS OpenAI SDK so not sure what the criteria is again) and alsp considering how the language was changed.
Im fine with this just that I don't completely agree (Feel free to resolve)

[patcher9]

Suggested change

	- @nirga
	- @nirga
	- @patcher9
	- @AmanAgarwal041

[drewby]

Does the meeting time change (for rest of world) when the clocks move back an hour in November?

[trentm]

@lmolkova Not sure if it was clear, but I am an OTel JS maintainer.

[jack-berg]

Component/code owner isn't a formal concept in the opentelemetry project. In opentelemetry-java-contrib, the bar is fairly low:

• must be a member of the OpenTelemetry organization (required for github automation)
• must be responsive to issues / PRs
• must express interest in being a code owner

Not sure what the criteria is for other *-contrib repositories. It looks like opentelemetry-collector-contrib has some documentation on the role, but it boils down to "the person is interested and the existing code owners / maintainers approve".

Can / should we broaden this group of code owners to the group of people that meet this loose criteria (member of Otel org, ready to respond to issues / PRs, interested)? In my experience with contrib repos, the problem has been underparticipation, not over participation.

[gyliu513]

We may need to add LangSmith here as well

[gyliu513]

@lmolkova I want to nominate myself to join OTEL community as a member, but I only did some PR reviews and attend some LLM Semantic Convention meetings, what do you think of counting in the effort that we contribute to https://github.com/traceloop/openllmetry/tree/main/packages as well? All of the instrumentation code are based on OpenTelemetry, and I think this may become the code base for the python instrumentation code, it will be great if we can count in those efforts for someone to apply to be a member of otel community, what do you think? Thanks!