metrics: add debug capabilities

[mtanzim]

Thank you for contributing to the Cohere Toolkit!

• PR title: "area: description"

  • Where "area" is whichever of interface, frontend, model, tools, backend, etc. is being modified. Use "docs: ..." for purely docs changes, "infra: ..." for CI changes.
  • Example: "deployment: add Azure model option"

• PR message: Delete this entire checklist and replace with

  • Description: a description of the change
  • Issue: the issue # it fixes, if applicable
  • Dependencies: any dependencies required for this change

• Add tests and docs: Please include testing and documentation for your changes

• Lint and test: Run make lint and make run-tests

AI Description

This PR introduces a new helper script, metrics_helper.py, which contains functions for interacting with the Cohere Platform's user, agent, and conversation management endpoints. It also includes updates to the backend code related to user and agent management, and modifications to the metrics middleware and reporting functionality.

helper_scripts/metrics_helper.py

A new helper script has been added, which provides functions for creating, updating, listing, and deleting users and agents, as well as initiating a chat session and performing related operations.

src/backend/routers/agent.py and src/backend/routers/user.py

The update_agent and update_user functions have been modified to include the line request.state.agent = agent and request.state.user = UserSchema.model_validate(db_user), respectively. This ensures that the request state is updated with the modified agent or user details.

The delete_agent and delete_user functions have also been updated to include the line request.state.agent = agent and request.state.user = UserSchema.model_validate(user), respectively, to capture the state before deletion.

src/backend/schemas/metrics.py

A new class MetricsSignal has been added, which contains a signal attribute of type MetricsData.

src/backend/services/metrics.py

The dispatch function signature has been updated to include the call_next parameter as call_next: Callable.

The get_data function has been renamed to get_event_data, and it now returns a value of type MetricsData.

The get_user_id function now includes a return type annotation, Union [str, None].

The report_metrics function has been updated to accept a MetricsSignal object as an argument and includes additional error handling for cases where the signal is not a dictionary.

Two new functions, wrap_and_log_data and run_loop, have been added to handle the logging and reporting of metrics data.

The run_loop function has been moved from its previous location within the MetricsMiddleware class to the module's global scope.

The report_metrics function is now asynchronous, and the run_loop function has been updated to use it accordingly.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 85.45455% with 8 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@028432c). Learn more about missing BASE report.

Files	Patch %	Lines
src/backend/services/metrics.py	81.39%	8 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #260   +/-   ##
=======================================
  Coverage        ?   86.66%           
=======================================
  Files           ?      149           
  Lines           ?     5549           
  Branches        ?        0           
=======================================
  Hits            ?     4809           
  Misses          ?      740           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.