-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Anthropic request headers helper #233
Conversation
server/metrics.go
Outdated
@@ -19,8 +19,7 @@ func (p *Plugin) GetMetrics() metrics.Metrics { | |||
} | |||
|
|||
func (p *Plugin) metricsMiddleware(c *gin.Context) { | |||
metrics := p.GetMetrics() | |||
if metrics == nil { | |||
if p.GetMetrics() == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be nicer to use the variable below instead of putting the p.GetMetrics()
into the if
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The metrics
variable was colliding with the metrics package and the variable was only being used for the if statement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you saying like this? This is something I had actually wondered about since I saw p.GetMetrics()
being called several times
func (p *Plugin) metricsMiddleware(c *gin.Context) {
llmMetrics := p.GetMetrics()
if llmMetrics == nil {
c.Next()
return
}
llmMetrics.IncrementHTTPRequests()
now := time.Now()
c.Next()
elapsed := float64(time.Since(now)) / float64(time.Second)
status := c.Writer.Status()
if status < 200 || status > 299 {
llmMetrics.IncrementHTTPErrors()
}
endpoint := c.HandlerName()
llmMetrics.ObserveAPIEndpointDuration(endpoint, c.Request.Method, strconv.Itoa(status), elapsed)
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup like that.
server/ai/anthropic/client.go
Outdated
@@ -203,3 +199,9 @@ func (c *Client) MessageCompletion(completionRequest MessageRequest) (*ai.TextSt | |||
|
|||
return &ai.TextStreamResult{Stream: output, Err: errChan}, nil | |||
} | |||
|
|||
func (c *Client) prepareRequestHeaders(req *http.Request) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure this helper is necessary. But if we do want to keep it maybe call it setAnthropicHeaders
and pull in the streaming ones with a boolean parameter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That sounds good. Yeah it was repeated logic so I figured not repeating it would be better, especially because then there were less hard-coded strings and consolidating into one place would eliminate the argument for making constants to reduce the number of hard-coded strings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just remembered my reasoning for originally calling it prepareRequestHeaders()
.
using "set" typically denotes that you are setting an instance variable on the calling object, whereas in this case I am setting values of something being passed in.
Description