Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Anthropic request headers helper #233

Merged
merged 5 commits into from
Aug 21, 2024
Merged

Anthropic request headers helper #233

merged 5 commits into from
Aug 21, 2024

Conversation

enzowritescode
Copy link
Contributor

@enzowritescode enzowritescode commented Aug 19, 2024

Description

  • Add helper function to set common headers for Anthropic requests
  • Fix variables colliding with imported package

@@ -19,8 +19,7 @@ func (p *Plugin) GetMetrics() metrics.Metrics {
}

func (p *Plugin) metricsMiddleware(c *gin.Context) {
metrics := p.GetMetrics()
if metrics == nil {
if p.GetMetrics() == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be nicer to use the variable below instead of putting the p.GetMetrics() into the if

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metrics variable was colliding with the metrics package and the variable was only being used for the if statement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you saying like this? This is something I had actually wondered about since I saw p.GetMetrics() being called several times

func (p *Plugin) metricsMiddleware(c *gin.Context) {
	llmMetrics := p.GetMetrics()
	if llmMetrics == nil {
		c.Next()
		return
	}
	llmMetrics.IncrementHTTPRequests()
	now := time.Now()

	c.Next()

	elapsed := float64(time.Since(now)) / float64(time.Second)

	status := c.Writer.Status()

	if status < 200 || status > 299 {
		llmMetrics.IncrementHTTPErrors()
	}

	endpoint := c.HandlerName()
	llmMetrics.ObserveAPIEndpointDuration(endpoint, c.Request.Method, strconv.Itoa(status), elapsed)
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup like that.

@@ -203,3 +199,9 @@ func (c *Client) MessageCompletion(completionRequest MessageRequest) (*ai.TextSt

return &ai.TextStreamResult{Stream: output, Err: errChan}, nil
}

func (c *Client) prepareRequestHeaders(req *http.Request) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this helper is necessary. But if we do want to keep it maybe call it setAnthropicHeaders and pull in the streaming ones with a boolean parameter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good. Yeah it was repeated logic so I figured not repeating it would be better, especially because then there were less hard-coded strings and consolidating into one place would eliminate the argument for making constants to reduce the number of hard-coded strings.

Copy link
Contributor Author

@enzowritescode enzowritescode Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just remembered my reasoning for originally calling it prepareRequestHeaders().
using "set" typically denotes that you are setting an instance variable on the calling object, whereas in this case I am setting values of something being passed in.

@crspeller crspeller merged commit 3ecb618 into master Aug 21, 2024
5 checks passed
@crspeller crspeller deleted the add-athropic-req-helper branch August 21, 2024 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants