Open Analytics

[jjmachan]

We track very basic usage metrics to guide us to figure out what our users want, what is working and what's not. As a young startup, we have to be brutally honest about this which is why we are tracking these metrics. We are also an Open Startup, which is a product or company which operates in the open and shares its statistics publically.

All the data and the code we track will be open-sourced soon

• add open analytics to ragas #51
• open bigquery table for public #50

If you don't want to send tracking info, you can easily disable it by setting RAGAS_DO_NOT_TRACK to True.

Keeping this issue open for feedback from the community and further discussions.

Refs

• What does it mean to be an Open Startup? | HackerNoon
• Why being an open startup matters | Cal.com Blog

[epec254]

When will this data be opened? :-)

[jjmachan]

Yeah will open that up as soon as I figure out the right Bigquery permissions 😂 (if you have some experience please do let me know)

also curious as to what u were looking for in the data, so that we can prioritize this better

[epec254]

thanks, i want to understand how people are evaluating their rags + any metadata about the rags themselves these steps will do it for you - https://www.youtube.com/watch?v=kMuTYjHGM1s - allUsers is the service principal to grant.

…

On Thu, Oct 12, 2023 at 11:43 PM Jithin James ***@***.***> wrote: Yeah will open that up as soon as I figure out the right Bigquery permissions 😂 (if you have some experience please do let me know) also curious as to what u were looking for in the data, so that we can prioritize this better — Reply to this email directly, view it on GitHub <#49 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACDPWDHQCSM35SHO5D2PD2LX7DPKZANCNFSM6AAAAAA2AANVDI> . You are receiving this because you commented.Message ID: ***@***.***>

-- Eric Peter ***@***.*** | 214.600.3056

[jjmachan]

Ohh I think you will not find any of that information in that, it just has the count of evaluations run, nothing more

I will checkout the resource, thanks for sharing :)

[epec254]

Gentle bump on your promise to open source this data! Even a CSV file dump would honor your commitment :-)

[jjmachan]

we're a bit tied down at the moment but we could work with you to make it happen. Would it be possible for you to volunteer some time to make it possible? would really really appreciate the help 🙏🏽

[epec254]

Sure! How can I help?

…

On Thu, Nov 16, 2023 at 10:11 PM Jithin James ***@***.***> wrote: we're a bit tied down at the moment but we could work with you to make it happen. Would it be possible for you to volunteer some time to make it possible? would really really appreciate the help 🙏🏽 — Reply to this email directly, view it on GitHub <#49 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACDPWDFJAWKC5EIIXGBAYHDYE35YJAVCNFSM6AAAAAA2AANVDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVHAYDAMBQG4> . You are receiving this because you commented.Message ID: ***@***.***>

[jjmachan]

could you give me your email ID? I'll ping you personally and we can set it up? or at the very least export the CSV for you

(apologies for the delay)

[epec254]

No worries! First initial (e) and last name (Peter) at Google’s email service ;)

…

On Wed, Nov 29, 2023 at 6:04 AM Jithin James ***@***.***> wrote: could you give me your email ID? I'll ping you personally and we can set it up? or at the very least export the CSV for you (apologies for the delay) — Reply to this email directly, view it on GitHub <#49 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACDPWDF5OENGPJ4DUWRKVW3YG46HHAVCNFSM6AAAAAA2AANVDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZRHE2TONBYHA> . You are receiving this because you commented.Message ID: ***@***.***>

[IbrahimAlrayes]

Is there any update on publishing the data?