自定义渠道参数传递不完整 #397
Labels
bug
Something isn't working
Comments
MartialBE
added a commit
that referenced
this issue
Oct 30, 2024
Some parameter types are changed to pointer types, which can correctly pass values to the provider. - Update `convertFromChatOpenai` function in `hunyuan/chat.go` to use the actual values for `TopP` and `Temperature` parameters. - Update `getChatRequestBody` function in `groq/chat.go` to use the actual values for `N` parameter. - Update `Option` struct in `ollama/type.go` to use the actual values for `TopP` and `Temperature` fields. - Update `PaLMChatRequest` struct in `palm/type.go` to use the actual values for `TopP` and `Temperature` fields. - Update `GeminiChatGenerationConfig` struct in `gemini/type.go` to use the actual values for `TopP`, `Temperature`, and `TopK` fields. - Update `XunfeiParameterChat` struct in `xunfei/type.go` to use the actual values for `Temperature` and `TopK` fields. - Update `ClaudeRequest` struct in `claude/type.go` to use the actual values for `Temperature`, `TopP`, and `TopK` fields. - Update `BaichuanChatRequest` struct in `baichuan/type.go` to use the actual values for `Temperature`, `TopP`, and `TopK` fields. - Update `convertFromChatOpenai` function in `tencent/chat.go` to use the actual values for `Temperature` and `TopP` parameters. - Update `MiniMaxChatRequest` struct in `minimax/type.go` to use the actual values for `Temperature` and `TopP` fields. - Update `convertFromChatOpenai` function in `zhipu/chat.go` to use the actual values for `Temperature` and `TopP` parameters. - Update `BaiduChatRequest` struct in `baidu/type.go` to use the actual values for `Temperature`, `TopP`, and `PenaltyScore` fields. - Update `MistralChatCompletionRequest` struct in `mistral/type.go` to use the actual values for `Temperature`, `TopP`, and `N` fields. - Update `CohereRequest` struct in `cohere/type.go` to use the actual values for `Temperature`, `MaxTokens`, `K`, `P`, `Seed`, `FrequencyPenalty`, and `PresencePenalty` fields.
|
repetition_penalty 有这个参数么？ 应该是 |
有些推理框架（如vllm）有超出open api reference的参数（repetition_penalty），是否可以在自定义渠道这里直接传上游所有的参数，网关直接传递下游的接口返回异常即可。这样可以比较大的兼容各种类openai的接口？ |
|
后面我再看看怎么处理比较好 |
另外，#397 中 *float64 是字段只有在有值时才会输出，null 时会被省略吗（我对go不熟悉）？我的建议是，字段null或默认值会随着下游的渠道/推理框架接口的变动而变动。最好是在网关只进行异常捕捉返回，不做严格的限制，要求上游请求的时候自己进行限制和调整。这样后期项目需要不停改动的地方也会比较小。 |
|
是的 指针在没有传入该值时是不会传给供应商的。如果不传，那么就是默认供应商的值了，所以现在的改动是合理的。 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
例行检查
问题描述
自定义渠道或openai渠道，当接口传入参数例如："temperature": 0.0, "repetition_penalty": 1.2 下游接口接受不到这两个参数？
复现步骤
使用最新的docker
预期结果
向下游传递所有上游请求的参数，如果参数错误向上传递？
The text was updated successfully, but these errors were encountered: