cneicy/new-api-custom
1
0 Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity

feat: publish sanitized custom.97 source

Browse Source
This commit is contained in:
cneicy committed 2026-10-27 20:43:39 +08:00
1 parent 8e6ed0317b
commit 86c0276156
1022 files changed
+69903 -19207
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

relaykit/README.md
+100 -63
View File
@@ -1,58 +1,73 @@
# RelayKit
RelayKit 是从 [new-api](https://github.com/QuantumNous/new-api) 中拆分出的独立 Go 模块，提供常用大模型文本协议的 DTO、请求转换、响应转换和流式事件转换。
RelayKit 是一个 Go 模块，在 OpenAI Chat Completions、OpenAI Responses、Anthropic Messages 和 Gemini `generateContent` 四种协议之间转换请求、非流式响应和流式事件。
它只负责协议层的数据建模与语义转换，不包含 HTTP 服务、上游请求发送、渠道调度、鉴权、计费或数据库逻辑。因此可以脱离 new-api 主模块，嵌入其他 Go 网关或代理服务。
## 能力
- 在 OpenAI Chat Completions、OpenAI Responses、Anthropic Messages 和 Gemini `generateContent` 之间转换
- 同时支持请求、非流式响应和增量流式响应
- 自动根据 DTO 类型识别源协议，并选择内置的直接或多跳转换路径
- 返回转换器 ID、质量等级、实际转换步骤和统一 usage，方便审计与调试
- 支持常用的文本、多模态内容、工具调用、推理内容和 usage 映射
- 作为独立 Go module 构建，不依赖 new-api 主模块、Gin、数据库或全局设置
它从 [new-api](https://github.com/QuantumNous/new-api) 中拆出，模块路径是 `github.com/QuantumNous/new-api/relaykit`，运行时依赖是 uuid、lo、gjson 和 sjson 四个包。它做协议数据的建模和转换，HTTP 收发、SSE 读写、渠道选择、鉴权和计费由宿主程序完成，所以可以嵌进其他 Go 网关或代理。
## 支持矩阵
以下四种文本协议支持任意两种格式之间的转换：
四种协议两两之间都能转换，请求、非流式响应和流式响应覆盖同一张表：
| 源格式 \ 目标格式 | OpenAI Chat | OpenAI Responses | Claude Messages | Gemini |
|---|---:|---:|---:|---:|
| OpenAI Chat | — | Good | Fair | Fair |
| OpenAI Responses | Good | — | Fair | Fair |
| Claude Messages | Fair | Fair | — | Discouraged |
| Gemini | Fair | Fair | Discouraged | — |
| OpenAI Chat | | `good` | `fair` | `fair` |
| OpenAI Responses | `good` | | `fair` | `fair` |
| Claude Messages | `fair` | `fair` | | `discouraged` |
| Gemini | `fair` | `fair` | `discouraged` | |
质量等级表示协议之间的语义匹配程度：
`good` 表示两种协议结构接近，字段基本一一对应。`fair` 表示主要内容都能转，协议独有的字段以诊断报告。其中 Gemini 转 Responses 的请求和响应、Claude 转 Responses 的响应、Responses 转 Gemini 的响应经 OpenAI Chat 中转。`discouraged` 表示 Claude 和 Gemini 之间请求和响应都经 OpenAI Chat 中转两跳，丢字段的机会更多。实际走的路径在转换结果的 `Steps` 里，等级在 `Quality` 里。
- `Good`：两种协议的核心结构较接近
- `Fair`：主要能力可转换，但部分协议特性可能需要适配或无法完整保留
- `Discouraged`：目前需要经过中间协议转换，语义损失风险更高
## 功能
请求、非流式响应和流式响应均覆盖上述矩阵。实际采用的路径可从转换结果的 `Steps` 和 `Quality` 字段中读取。
### 自动选路
`ConvertRequest` 和 `ConvertResponse` 按传入 DTO 的具体类型判断源协议，`NewResponseStreamState` 按参数里的源格式和目标格式，都查表选直接转换器或多跳路径。结果里有源格式、目标格式、转换器 ID、质量等级、实际经过的每一步和诊断列表。要固定路径时用 `ConvertRequestVia`，要按 ID 执行时用 `ConvertRequestByID`、`ConvertResponseByID` 和 `NewResponseStreamStateByID`。
### 工具调用
function tool 的定义、调用和结果在四种协议之间互转。Responses 的 custom tool 和带 namespace 的工具发往 Chat、Claude、Gemini 上游时改成普通 function tool，编码方式记在 `convmeta.ResponsesToolState` 里，响应转换时按这份记录还原成 Responses 的形状。托管 web search 按目标协议重新编码：Chat 用 `web_search_options`，Responses、Claude 和 Gemini 各用自己协议的 web search 工具定义，宿主可以通过 `Options.WebSearch` 为某个上游改写编码。其他托管工具在目标协议里表达不了的部分，转换照常完成，缺失写进诊断。
### 推理设置
reasoning effort、thinking budget 和是否返回思考内容用 `dto.ReasoningConversionState` 在转换步骤之间传递，多跳路径上的每一步都能读到精确的 budget 和 effort。Claude 的 max_tokens 和 budget 超出模型范围时会调整，Gemini 的 thinkingConfig 按模型归一化，这些改动都以诊断返回，例如 `claude_max_tokens_raised` 和 `claude_budget_adjusted`。
### 多模态
图片和文件内容块在协议之间转换。需要下载 URL 或解析 data URL 时，RelayKit 调用宿主用 `SetMediaResolver` 注册的函数，未注册时转换返回错误。
### Usage
每个响应转换都返回统一的 `dto.Usage`，含输入、输出、缓存命中，以及 Claude 缓存写入按 5 分钟和 1 小时的拆分。`Usage.BillingUsage` 同时带着上游原始的 usage 结构和它来自哪个协议，宿主可以按上游的计量规则计费。流式转换把分散在多个事件里的 usage 累加到 `ResponseStreamState`。`dto.MergeUsageNonZero` 一类函数合并两份 usage 时，后到的非零字段覆盖已有值，后到的零值保留已有的计数。
### 流式
每条上游流对应一个 `ResponseStreamState`，跨事件的工具调用拼接、usage 和结束状态都存在这里。上游结束后调用 `FinalizeStreamResponse`，转换器在这一步补发终止事件和最终 usage。目标是 Responses 时，上游中途出错可以调用 `FailResponsesStream` 生成 Responses 协议的错误事件，`ResponseStreamOptions.EmitSequenceNumber` 打开后每个事件带 `sequence_number`。
### 宿主钩子
JSON 编解码默认用 `encoding/json`，宿主可以在启动时用 `kitutil.SetCodec` 换成别的实现，DTO 上自定义的 `MarshalJSON` 和 `UnmarshalJSON` 也走同一个编解码器。数据形状异常的日志通过 `kitutil.SetLogging` 和 `kitutil.SetSystemErrorLogging` 交给宿主，默认写到 stderr。
## 安装
RelayKit 要求 Go 1.25.1 或更高版本。
RelayKit 要求 Go 1.25.1 或更高版本。tag 的形式是 `relaykit/vX.Y.Z`，`@latest` 解析到最新的一个。
```bash
go get github.com/QuantumNous/new-api/relaykit@latest
```
主要包：
| 包 | 用途 |
| 包 | 内容 |
|---|---|
| `relaykit/dto` | 各协议的请求、响应、流式事件和 usage DTO |
| `relaykit/types` | 协议格式、错误、文件来源及共享类型 |
| `relaykit/dto` | 四种协议的请求、响应、流式事件和 usage 结构，以及 new-api 宿主共用的渠道、定价和用户设置结构 |
| `relaykit/types` | 协议格式、错误、诊断、文件来源等共享类型 |
| `relaykit/relayconvert` | 请求、响应和流式转换入口 |
| `relaykit/relayconvert/convmeta` | 与宿主实现解耦的转换上下文和选项 |
| `relaykit/reasonmap` | 不同协议之间的结束原因映射 |
| `relaykit/relayconvert/convmeta` | 转换上下文接口 `Meta`、默认实现 `Values` 和选项 `Options` |
| `relaykit/relayconvert/reasoning` | 推理设置的解析、合并和渲染，包括模型名尾缀解析 |
| `relaykit/relayconvert/kitutil` | JSON 编解码和日志钩子 |
| `relaykit/reasonmap` | Claude 和 OpenAI 之间的结束原因映射 |
## 快速开始
下面将 OpenAI Chat Completions 请求转换为 Claude Messages 请求：
下面把 OpenAI Chat Completions 请求转换成 Claude Messages 请求：
```go
package main
@@ -102,11 +117,11 @@ func main() {
}
```
`ConvertRequest` 根据请求的具体 DTO 类型推断源格式。传入原始 JSON、`map[string]any` 或不受支持的 DTO 会返回错误。
`ConvertRequest` 按请求的具体 DTO 类型推断源格式。传入 nil、原始 JSON、`map[string]any` 或其他类型时返回错误。
### 非流式响应
响应转换使用相同的目标格式模型：
响应转换用同一套目标格式参数：
```go
result, err := relayconvert.ConvertResponse(
@@ -134,16 +149,17 @@ usage := result.Usage
### 流式响应
流式转换可能需要跨事件保存工具调用、usage 和结束状态。每条上游流应创建独立的 `ResponseStreamState`，并在上游结束后调用 `FinalizeStreamResponse`：
每条上游流创建一个 `ResponseStreamState`，上游结束后调用 `FinalizeStreamResponse`：
```go
state, err := relayconvert.NewResponseStreamState(
types.RelayFormatOpenAI,
types.RelayFormatOpenAIResponses,
relayconvert.ResponseStreamOptions{
ID: "resp_123",
Model: "gpt-4.1",
IncludeUsage: true,
ID: "resp_123",
Model: "gpt-4.1",
IncludeUsage: true,
EmitSequenceNumber: true,
},
)
if err != nil {
@@ -169,15 +185,16 @@ for _, result := range finalResults {
}
usage := state.Usage()
diagnostics := state.Diagnostics()
```
RelayKit 不负责 SSE 的读取和写入。宿主需要将每个 SSE 事件解析为对应 DTO，并将转换结果重新编码后发送给下游。不要省略 `FinalizeStreamResponse`，部分转换器会在该阶段补发终止事件或最终 usage。
SSE 的读取和写入由宿主完成：把每个上游事件解析成对应的 DTO 传进来，再把转换结果编码后发给下游。`FinalizeStreamResponse` 每条流都要调用，部分转换器在这一步才补发终止事件和最终 usage。
## 转换上下文
大多数基础转换可以传入 `nil` 作为 `convmeta.Meta`。需要模型映射、推理适配、安全设置或流式状态时，应使用 `convmeta.Values`，或在宿主中实现 `convmeta.Meta`。
大多数基础转换的 `convmeta.Meta` 参数可以传 `nil`。需要模型名、推理设置、工具编码记录或流式状态时，用 `convmeta.Values`，或者在宿主里实现 `convmeta.Meta`。自己实现时，指针类型的每个方法都要能在 nil receiver 上返回零值，接口注释写了原因。
常用选项通过 `convmeta.Options` 按请求传入：
按请求传入的选项放在 `convmeta.Options`：
```go
meta := &convmeta.Values{
@@ -194,18 +211,39 @@ meta := &convmeta.Values{
}
```
需要注意：
| 选项 | 作用 |
|---|---|
| `Claude.DefaultMaxTokens` | 源请求没有 max_tokens 时，转 Claude 用这个函数取值 |
| `Claude.ThinkingAdapterEnabled`、`Gemini.ThinkingAdapterEnabled` | 模型名尾缀适配开关，见下文 |
| `Claude.ThinkingAdapterBudgetTokensPercentage`、`Gemini.ThinkingAdapterBudgetTokensPercentage` | 尾缀适配触发时，thinking budget 占 max_tokens 的比例 |
| `Claude.WebSearchToolVersion` | 转 Claude 时 web search 工具的版本，空值用 `web_search_20250305` |
| `Gemini.FunctionCallThoughtSignatureEnabled` | 给 function call 部分附加 thoughtSignature |
| `Gemini.SupportsImagine` | 判断模型是否支持生图，支持时切换响应模态 |
| `Gemini.SafetySetting` | 按分类返回 safetySettings 的阈值 |
| `ToolLossPolicy` | 跨协议丢失工具行为时的处理，见下文 |
| `OpenRouterDialect` | 上游是 OpenRouter 时打开，转换器会输出它额外接受的字段 |
| `WebSearch` | 为某个上游改写托管 web search 的编码 |
| `PreserveThinkingSuffix`、`PreserveEffortTail` | 判断哪些模型名要原样保留尾缀 |
- OpenAI Chat 或 OpenAI Responses 转 Claude 时，Claude 请求必须具有 `max_tokens`。源请求未提供时，需要配置 `Claude.DefaultMaxTokens`，否则转换会返回错误。
- RelayKit 不负责选择渠道或映射模型名。调用转换前，应将请求中的 `Model` 设置为目标上游使用的模型名。
- 自定义 `convmeta.Meta` 的指针实现必须保证所有方法对 nil receiver 安全，完整约束见 `convmeta.Meta` 的接口注释。
- 工具损耗策略默认是 `allow`：跨协议转换会成功，损耗以诊断形式返回。`safe` / `strict` 只在请求阶段 opt-in 拒绝；响应和流式转换无论策略如何都不会因损耗失败。
- `ThinkingAdapterEnabled` 只控制是否把已解析的推理意图渲染到 Claude / Gemini 请求上。`-thinking` / `-nothinking` / effort 尾缀等命名约定不再由转换器自动解释。
- 若你的入口仍使用这些模型名后缀，请在调用转换前自行调用 `relayconvert/reasoning` 的 `Parse*` 帮助函数，把结果写成 `dto.ReasoningConversionState`，并通过 `convmeta.Meta.ReasoningState()`（`convmeta.Values.ReasoningConversion`）传入。同时把发给上游的模型名裁成无后缀基础名。
转 Claude 时请求需要 `max_tokens`。源请求没有时，RelayKit 调用 `Claude.DefaultMaxTokens` 取值，两者都没有时返回错误。
## 多模态内容
RelayKit 按请求里的 `Model` 生成上游请求，调用前把它设成上游使用的模型名。带 `-thinking`、`-nothinking`、`-thinking-<budget>` 或 `-high` 这类尾缀的 Claude、Gemini 模型名由调用方在转换前解析：调用 `reasoning.ParseClaudeModelSuffix` 或 `reasoning.ParseGeminiModelSuffix`，`allowSuffixAlias` 参数传 `ThinkingAdapterEnabled`，把返回的 `Intent` 用 `reasoning.StateFromIntent` 写进 `Values.ReasoningConversion`，再把裁掉尾缀的模型名放进请求。开关关闭时这两个函数原样返回模型名，推理设置来自请求字段。
某些跨协议的图片转换需要下载 URL 内容或解析 data URL。宿主应在启动时配置媒体解析器：
## 诊断与损耗策略
每个转换结果都带 `Diagnostics`，每条诊断有代码、字段路径、说明、级别和源目标格式。级别 `warning` 表示展示层面的差异，`error` 表示工具行为或内容发生了变化。常见代码有 `hosted_web_search_omitted`、`trailing_assistant_omitted` 和 `unsupported_media_type`。
`Options.ToolLossPolicy` 决定请求阶段怎么处理诊断：
- `allow` 是默认值，转换完成，所有损耗以诊断返回。
- `safe` 在请求阶段遇到 `error` 级诊断时返回 `*types.ConversionLossError`。
- `strict` 在请求阶段遇到任何诊断都返回这个错误。
响应和流式转换在三种策略下都完成转换并返回诊断。流式用 `state.Diagnostics()` 读到目前为止累计的诊断，包括没有产生下游事件的那些。
## 媒体解析
跨协议的图片转换有时需要下载 URL 内容或解析 data URL。宿主在启动时注册解析函数：
```go
relayconvert.SetMediaResolver(relayconvert.MediaResolver{
@@ -214,39 +252,38 @@ relayconvert.SetMediaResolver(relayconvert.MediaResolver{
})
```
两个回调的签名由 `relayconvert.MediaResolver` 定义。需要媒体解析而未配置对应回调时，转换会明确返回错误；RelayKit 本身不会发起网络请求。
两个函数的签名见 `relayconvert.MediaResolver`。需要媒体解析而没有注册时，转换返回错误。网络请求由这两个函数发起。
## 转换结果
请求转换返回 `relayconvert.RequestResult`，响应转换返回 `relayconvert.ResponseResult`。除 `Value` 外，建议关注：
请求转换返回 `relayconvert.RequestResult`，响应转换返回 `relayconvert.ResponseResult`，字段有：
- `From` / `To`：源格式和目标格式
- `Converter`：所选转换器 ID
- `Quality`：转换质量等级
- `Steps`：直接转换或多跳转换的实际路径
- `Usage`：响应转换后的统一 token usage
- `Stream`：结果是否来自流式转换
如果需要固定转换路径，可使用 `ConvertRequestVia`；如果需要按转换器 ID 执行，可使用 `ConvertRequestByID`、`ConvertResponseByID` 和 `NewResponseStreamStateByID`。
- `Value`：转换后的 DTO
- `From`、`To`：源格式和目标格式
- `Converter`：转换器 ID
- `Quality`：质量等级
- `Steps`：实际经过的每一步
- `Diagnostics`：诊断列表
- `Usage`、`Stream`：响应结果独有，统一 usage 和是否来自流式转换
## 开发
RelayKit 必须始终保持独立可构建。修改模块后，在 `relaykit` 目录运行：
在 `relaykit` 目录运行：
```bash
GOWORK=off go test ./...
GOWORK=off go build ./...
GOWORK=off go test ./...
```
转换矩阵由 golden tests 覆盖。确认协议输出变化是预期行为后，可更新快照：
`GOWORK=off` 让模块脱离仓库的 workspace 单独构建，这是 new-api 对 RelayKit 改动的检查项。转换矩阵由 golden test 覆盖，协议输出变化确认是预期行为后更新快照：
```bash
GOWORK=off go test ./relayconvert -run TestGolden -update
```
## 版本与兼容性
## 版本
RelayKit 当前使用 `v0.x` 版本。公开 API 和 DTO 仍可能在小版本中调整，升级前请检查发布说明和实际序列化结果。协议之间并非完全同构，建议对业务实际使用的工具调用、多模态、推理和流式场景增加端到端测试。
RelayKit 处于 `v0.x`。公开 API 和 DTO 可能在小版本里变化，升级前对照发布说明和序列化结果。每对协议都有对方没有的字段，建议对业务里用到的工具调用、多模态、推理和流式场景加端到端测试。
## 许可证
relaykit/dto/channel_settings.go
+65 -4
View File
@@ -4,6 +4,7 @@ import (
"fmt"
"net/url"
"regexp"
"slices"
"strings"
"sync"
@@ -21,12 +22,46 @@ type ChannelSettings struct {
ResponsesAstraModelGuard *bool `json:"responses_astra_model_guard,omitempty"`
SystemPrompt string `json:"system_prompt,omitempty"`
SystemPromptOverride bool `json:"system_prompt_override,omitempty"`
// TaskExtendPluginKeys lists the task plugins a New API channel (type 60)
// is extended with. The upstream gateway may host many plugins, so the
// channel serves every listed plugin's models while the request still pins
// the executing plugin. TaskPluginKey remains the single type-61 binding
// and stays valid on a New API channel as well.
TaskExtendPluginKeys []string `json:"task_extend_plugin_keys,omitempty"`
// HTTPProtocol controls outbound HTTP version negotiation for this channel.
// Accepted values: "", "auto" (default), "http1".
HTTPProtocol string `json:"http_protocol,omitempty"`
// HTTP2ConnectionShards spreads HTTP/2 traffic across N independent transports
// (1-8). Zero/unset means 1. Ignored when HTTPProtocol is "http1".
HTTP2ConnectionShards int `json:"http2_connection_shards,omitempty"`
// TLSInsecureSkipVerify accepts upstream certificates that fail
// verification, such as self-signed ones, for this channel only.
TLSInsecureSkipVerify bool `json:"tls_insecure_skip_verify,omitempty"`
}
// BindsTaskPlugin reports whether the channel is bound to the task plugin,
// either through the single binding or the New API extension list.
func (s ChannelSettings) BindsTaskPlugin(key string) bool {
if key == "" {
return false
}
return s.TaskPluginKey == key || slices.Contains(s.TaskExtendPluginKeys, key)
}
// TaskPluginBindings returns the sorted, de-duplicated set of task plugins the
// channel binds through either field, so callers can compare bindings as a set.
func (s ChannelSettings) TaskPluginBindings() []string {
bindings := make([]string, 0, len(s.TaskExtendPluginKeys)+1)
if s.TaskPluginKey != "" {
bindings = append(bindings, s.TaskPluginKey)
}
for _, key := range s.TaskExtendPluginKeys {
if key != "" && !slices.Contains(bindings, key) {
bindings = append(bindings, key)
}
}
slices.Sort(bindings)
return bindings
}
func (s ChannelSettings) AstraModelGuardEnabled() bool {
@@ -73,9 +108,20 @@ const (
AwsKeyTypeApiKey AwsKeyType = "api_key"
)
type AwsApiType string
const (
AwsApiTypeInvokeModel AwsApiType = "invoke_model"
AwsApiTypeMantle AwsApiType = "mantle"
)
type ChannelOtherSettings struct {
// Exact upstream model names whose Messages input_tokens includes cache reads.
// Empty preserves standard Anthropic accounting; cache writes remain separate.
ClaudeInputIncludesCacheReadModels []string `json:"claude_input_includes_cache_read_models,omitempty"`
AzureResponsesVersion string `json:"azure_responses_version,omitempty"`
VertexKeyType VertexKeyType `json:"vertex_key_type,omitempty"` // "json" or "api_key"
VertexKeyType VertexKeyType `json:"vertex_key_type,omitempty"` // "json" or "api_key"
VertexProjectID string `json:"vertex_project_id,omitempty"` // 仅当 vertex_key_type=api_key 时使用；JSON 模式下从 service account 解析
OpenRouterEnterprise *bool `json:"openrouter_enterprise,omitempty"`
ClaudeBetaQuery bool `json:"claude_beta_query,omitempty"` // Claude 渠道是否强制追加 ?beta=true
AllowServiceTier bool `json:"allow_service_tier,omitempty"` // 是否允许 service_tier 透传（默认过滤以避免额外计费）
@@ -86,6 +132,7 @@ type ChannelOtherSettings struct {
AllowIncludeObfuscation bool `json:"allow_include_obfuscation,omitempty"` // 是否允许 stream_options.include_obfuscation 透传（默认过滤以避免关闭流混淆保护）
DisableTaskPollingSleep bool `json:"disable_task_polling_sleep,omitempty"` // 是否跳过异步任务轮询间隔
AwsKeyType AwsKeyType `json:"aws_key_type,omitempty"`
AwsApiType AwsApiType `json:"aws_api_type,omitempty"`
UpstreamModelUpdateCheckEnabled bool `json:"upstream_model_update_check_enabled,omitempty"` // 是否检测上游模型更新
UpstreamModelUpdateAutoSyncEnabled bool `json:"upstream_model_update_auto_sync_enabled,omitempty"` // 是否自动同步上游模型更新
UpstreamModelUpdateLastCheckTime int64 `json:"upstream_model_update_last_check_time,omitempty"` // 上次检测时间
@@ -134,6 +181,10 @@ const (
advancedCustomConverterOpenAIResponsesToGemini = "openai_responses_to_gemini_generate_content"
advancedCustomConverterGeminiContentToOpenAIChat = "gemini_generate_content_to_openai_chat_completions"
advancedCustomConverterOpenAIChatToGeminiContent = "openai_chat_completions_to_gemini_generate_content"
// Same values as relayconvert.ConverterClaudeMessagesToOpenAIResponses and
// relayconvert.ConverterGeminiContentToOpenAIResponses.
advancedCustomConverterClaudeMessagesToOpenAIResponses = "claude_messages_to_openai_responses"
advancedCustomConverterGeminiContentToOpenAIResponses = "gemini_generate_content_to_openai_responses"
)
const (
@@ -181,6 +232,8 @@ const (
advancedCustomEndpointPathOpenAIResponses = "/v1/responses"
advancedCustomEndpointPathOpenAIResponsesCompact = "/v1/responses/compact"
advancedCustomEndpointPathOpenAIAlphaSearch = "/v1/alpha/search"
advancedCustomEndpointPathDeepSeekToolsSearch = "/v1/tools/search"
advancedCustomEndpointPathDeepSeekToolsFetch = "/v1/tools/fetch"
advancedCustomEndpointPathClaudeMessages = "/v1/messages"
advancedCustomEndpointPathJinaRerank = "/v1/rerank"
advancedCustomEndpointPathImageGeneration = "/v1/images/generations"
@@ -306,6 +359,10 @@ func advancedCustomEndpointTypeFromIncomingPath(incomingPath string) (types.Endp
return types.EndpointTypeOpenAIResponseCompact, true
case advancedCustomEndpointPathOpenAIAlphaSearch:
return types.EndpointTypeOpenAIAlphaSearch, true
case advancedCustomEndpointPathDeepSeekToolsSearch:
return types.EndpointTypeDeepSeekToolsSearch, true
case advancedCustomEndpointPathDeepSeekToolsFetch:
return types.EndpointTypeDeepSeekToolsFetch, true
case advancedCustomEndpointPathClaudeMessages:
return types.EndpointTypeAnthropic, true
case advancedCustomEndpointPathJinaRerank:
@@ -411,7 +468,9 @@ func IsAdvancedCustomConverterAllowed(converter string) bool {
advancedCustomConverterOpenAIResponsesToOpenAIChat,
advancedCustomConverterOpenAIResponsesToGemini,
advancedCustomConverterGeminiContentToOpenAIChat,
advancedCustomConverterOpenAIChatToGeminiContent:
advancedCustomConverterOpenAIChatToGeminiContent,
advancedCustomConverterClaudeMessagesToOpenAIResponses,
advancedCustomConverterGeminiContentToOpenAIResponses:
return true
default:
return false
@@ -606,7 +665,8 @@ func validateAdvancedCustomConverterPath(index int, incomingPath string, convert
switch converter {
case advancedCustomConverterNone:
return nil
case advancedCustomConverterClaudeMessagesToOpenAIChat:
case advancedCustomConverterClaudeMessagesToOpenAIChat,
advancedCustomConverterClaudeMessagesToOpenAIResponses:
if incomingPath == "/v1/messages" {
return nil
}
@@ -624,7 +684,8 @@ func validateAdvancedCustomConverterPath(index int, incomingPath string, convert
if incomingPath == "/v1/responses" {
return nil
}
case advancedCustomConverterGeminiContentToOpenAIChat:
case advancedCustomConverterGeminiContentToOpenAIChat,
advancedCustomConverterGeminiContentToOpenAIResponses:
if strings.Contains(incomingPath, ":generateContent") || strings.Contains(incomingPath, ":streamGenerateContent") {
return nil
}
relaykit/dto/channel_settings_test.go
+4
View File
@@ -676,20 +676,24 @@ func TestChannelSettingsHTTPTransportJSONRoundTrip(t *testing.T) {
require.NoError(t, err)
assert.NotContains(t, string(encoded), "http_protocol")
assert.NotContains(t, string(encoded), "http2_connection_shards")
assert.NotContains(t, string(encoded), "tls_insecure_skip_verify")
explicit := ChannelSettings{
Proxy: "socks5://127.0.0.1:1080",
HTTPProtocol: HTTPProtocolHTTP1,
HTTP2ConnectionShards: 1,
TLSInsecureSkipVerify: true,
}
encoded, err = json.Marshal(explicit)
require.NoError(t, err)
assert.Contains(t, string(encoded), `"http_protocol":"http1"`)
assert.Contains(t, string(encoded), `"tls_insecure_skip_verify":true`)
var decoded ChannelSettings
require.NoError(t, json.Unmarshal(encoded, &decoded))
assert.Equal(t, explicit.HTTPProtocol, decoded.HTTPProtocol)
assert.Equal(t, 1, decoded.HTTP2ConnectionShards)
assert.True(t, decoded.TLSInsecureSkipVerify)
sharded := ChannelSettings{HTTP2ConnectionShards: 4}
encoded, err = json.Marshal(sharded)
relaykit/dto/claude.go
+80 -15
View File
@@ -3,6 +3,7 @@ package dto
import (
"encoding/json"
"fmt"
"maps"
"net/http"
"strings"
@@ -135,6 +136,9 @@ type ClaudeMessageSource struct {
type ClaudeMessage struct {
Role string `json:"role"`
Content any `json:"content"`
// OutputConfig carries per-message effort (beta): an effort-only system
// message has empty content and the new level in output_config.effort.
OutputConfig json.RawMessage `json:"output_config,omitempty"`
}
func (c *ClaudeMessage) IsStringContent() bool {
@@ -238,6 +242,7 @@ type ClaudeRequest struct {
TopK *int `json:"top_k,omitempty"`
Stream *bool `json:"stream,omitempty"`
Tools any `json:"tools,omitempty"`
Safeguards json.RawMessage `json:"safeguards,omitempty"`
ContextManagement json.RawMessage `json:"context_management,omitempty"`
OutputConfig json.RawMessage `json:"output_config,omitempty"`
OutputFormat json.RawMessage `json:"output_format,omitempty"`
@@ -332,9 +337,25 @@ func (c *ClaudeRequest) GetTokenCountMeta() *types.TokenCountMeta {
texts = append(texts, string(b))
}
case "tool_result":
if media.Content != nil {
b, _ := kitutil.Marshal(media.Content)
texts = append(texts, string(b))
// Count tool output as the text the model reads; JSON-encoding it
// would escape newlines and quotes and count image base64 as text.
if media.IsStringContent() {
texts = append(texts, media.GetStringContent())
break
}
for _, part := range media.ParseMediaContent() {
if source := part.ToFileSource(); source != nil {
fileType := types.FileTypeFile
if part.Type == "image" {
fileType = types.FileTypeImage
}
fileMeta = append(fileMeta, &types.FileMeta{FileType: fileType, Source: source})
} else if part.Type == "text" {
texts = append(texts, part.GetText())
} else {
b, _ := kitutil.Marshal(part)
texts = append(texts, string(b))
}
}
}
}
@@ -391,18 +412,6 @@ func (c *ClaudeRequest) SetModelName(modelName string) {
}
}
func (c *ClaudeRequest) SearchToolNameByToolCallId(toolCallId string) string {
for _, message := range c.Messages {
content, _ := message.ParseContent()
for _, mediaMessage := range content {
if mediaMessage.Id == toolCallId {
return mediaMessage.Name
}
}
}
return ""
}
// AddTool 添加工具到请求中
func (c *ClaudeRequest) AddTool(tool any) {
if c.Tools == nil {
@@ -456,6 +465,18 @@ func ProcessTools(tools []any) ([]*Tool, []*ClaudeWebSearchTool) {
normalTools = append(normalTools, &t)
case ClaudeWebSearchTool:
webSearchTools = append(webSearchTools, &t)
case map[string]any:
// Tools decoded from client JSON arrive as maps.
toolType, _ := t["type"].(string)
if strings.HasPrefix(toolType, "web_search") {
if webSearchTool, err := kitutil.Any2Type[ClaudeWebSearchTool](t); err == nil {
webSearchTools = append(webSearchTools, &webSearchTool)
}
continue
}
if normalTool, err := kitutil.Any2Type[Tool](t); err == nil {
normalTools = append(normalTools, &normalTool)
}
default:
// 未知类型，跳过
continue
@@ -473,6 +494,50 @@ type Thinking struct {
// the visible summary that was default on Opus 4.6; "omitted" (default on
// 4.7) suppresses it. Pass-through field from upstream Anthropic API.
Display string `json:"display,omitempty"`
// BlockBinding carries the client's preserved-thinking controls, such as
// {"prefix_mismatch_behavior":"drop_block"} (beta
// thinking-binding-controls-2026-08-01). It is forwarded verbatim and
// never added by the gateway.
BlockBinding json.RawMessage `json:"block_binding,omitempty"`
// Extra keeps thinking fields this struct does not model yet, so a native
// Claude request forwards them instead of silently dropping them. It is
// exported so common.DeepCopy copies it; MarshalJSON writes it back.
Extra map[string]json.RawMessage `json:"-"`
}
func (c *Thinking) UnmarshalJSON(data []byte) error {
type thinkingFields Thinking
var fields thinkingFields
if err := kitutil.Unmarshal(data, &fields); err != nil {
return err
}
var all map[string]json.RawMessage
if err := kitutil.Unmarshal(data, &all); err != nil {
return err
}
for _, known := range []string{"type", "budget_tokens", "display", "block_binding"} {
delete(all, known)
}
*c = Thinking(fields)
if len(all) > 0 {
c.Extra = all
}
return nil
}
func (c Thinking) MarshalJSON() ([]byte, error) {
type thinkingFields Thinking
data, err := kitutil.Marshal(thinkingFields(c))
if err != nil || len(c.Extra) == 0 {
return data, err
}
merged := make(map[string]json.RawMessage, len(c.Extra)+4)
maps.Copy(merged, c.Extra)
if err := kitutil.Unmarshal(data, &merged); err != nil {
return nil, err
}
return kitutil.Marshal(merged)
}
func (c *Thinking) GetBudgetTokens() int {
relaykit/dto/deepseek_tools_request.go
+34
View File
@@ -0,0 +1,34 @@
package dto
import (
"encoding/json"
"net/http"
"github.com/QuantumNous/new-api/relaykit/types"
)
// DeepSeekToolsRequest is a DeepSeek Search or Fetch API request
// (/v1/tools/search, /v1/tools/fetch). The body names no model; Model holds
// the routing name. RawBody preserves the original JSON so the body is
// forwarded with unknown fields intact.
type DeepSeekToolsRequest struct {
Model string `json:"model,omitempty"`
RawBody json.RawMessage `json:"-"`
}
func (r *DeepSeekToolsRequest) GetTokenCountMeta() *types.TokenCountMeta {
return &types.TokenCountMeta{
CombineText: string(r.RawBody),
TokenType: types.TokenTypeTokenizer,
}
}
func (r *DeepSeekToolsRequest) IsStream(_ *http.Request) bool {
return false
}
func (r *DeepSeekToolsRequest) SetModelName(modelName string) {
if modelName != "" {
r.Model = modelName
}
}
relaykit/dto/gemini.go
+52 -2
View File
@@ -2,7 +2,9 @@ package dto
import (
"encoding/json"
"maps"
"net/http"
"slices"
"strings"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
@@ -74,13 +76,47 @@ func (r *GeminiChatRequest) GetTokenCountMeta() *types.TokenCountMeta {
}
var inputTexts []string
if r.SystemInstructions != nil {
for _, part := range r.SystemInstructions.Parts {
if part.Text != "" {
inputTexts = append(inputTexts, part.Text)
}
}
}
for _, content := range r.Contents {
for _, part := range content.Parts {
if part.Text != "" {
inputTexts = append(inputTexts, part.Text)
}
if source := part.InlineData.ToFileSource(); source != nil {
mimeType := part.InlineData.MimeType
// Function calls and responses replayed in agent loops are part of the prompt.
if part.FunctionCall != nil {
args, _ := kitutil.Marshal(part.FunctionCall.Arguments)
inputTexts = append(inputTexts, part.FunctionCall.FunctionName, string(args))
}
mediaParts := []GeminiPart{part}
if part.FunctionResponse != nil {
inputTexts = append(inputTexts, part.FunctionResponse.Name)
// Count tool output as the text the model reads, not JSON-escaped.
for _, key := range slices.Sorted(maps.Keys(part.FunctionResponse.Response)) {
if text, ok := part.FunctionResponse.Response[key].(string); ok {
inputTexts = append(inputTexts, text)
continue
}
value, _ := kitutil.Marshal(part.FunctionResponse.Response[key])
inputTexts = append(inputTexts, string(value))
}
// Multimodal function responses carry media such as agent screenshots.
var responseParts []GeminiPart
if kitutil.Unmarshal(part.FunctionResponse.Parts, &responseParts) == nil {
mediaParts = append(mediaParts, responseParts...)
}
}
for _, media := range mediaParts {
source := media.InlineData.ToFileSource()
if source == nil {
continue
}
mimeType := media.InlineData.MimeType
var fileType types.FileType
if strings.HasPrefix(mimeType, "image/") {
fileType = types.FileTypeImage
@@ -91,13 +127,26 @@ func (r *GeminiChatRequest) GetTokenCountMeta() *types.TokenCountMeta {
} else {
fileType = types.FileTypeFile
}
// Image cost depends on the media resolution; a part's own level
// overrides the request-wide one.
detail := string(r.GenerationConfig.MediaResolution)
var partResolution struct {
Level string `json:"level"`
}
if kitutil.Unmarshal(media.MediaResolution, &partResolution) == nil && partResolution.Level != "" {
detail = partResolution.Level
}
files = append(files, &types.FileMeta{
FileType: fileType,
Source: source,
Detail: detail,
})
}
}
}
if len(r.Tools) > 0 {
inputTexts = append(inputTexts, string(r.Tools))
}
inputText := strings.Join(inputTexts, "\n")
return &types.TokenCountMeta{
@@ -467,6 +516,7 @@ type GeminiChatCandidate struct {
type GeminiGroundingMetadata struct {
WebSearchQueries []string `json:"webSearchQueries,omitempty"`
ImageSearchQueries []string `json:"imageSearchQueries,omitempty"`
RetrievalQueries []string `json:"retrievalQueries,omitempty"`
GroundingChunks json.RawMessage `json:"groundingChunks,omitempty"`
GroundingSupports json.RawMessage `json:"groundingSupports,omitempty"`
relaykit/dto/openai_request.go
+74 -17
View File
@@ -93,6 +93,7 @@ type GeneralOpenAIRequest struct {
ThinkingBudget json.RawMessage `json:"thinking_budget,omitempty"`
ChatTemplateKwargs json.RawMessage `json:"chat_template_kwargs,omitempty"`
EnableSearch json.RawMessage `json:"enable_search,omitempty"`
SearchOptions json.RawMessage `json:"search_options,omitempty"`
// ollama Params
Think json.RawMessage `json:"think,omitempty"`
// baidu v2
@@ -251,6 +252,14 @@ func (r *GeneralOpenAIRequest) GetTokenCountMeta() *types.TokenCountMeta {
}
}
}
// Assistant tool calls replayed in agent loops are part of the prompt.
for _, toolCall := range message.ParseToolCalls() {
if len(toolCall.Custom) > 0 {
texts = append(texts, string(toolCall.Custom))
continue
}
texts = append(texts, toolCall.Function.Name, toolCall.Function.Arguments)
}
}
tools := r.Tools
@@ -259,12 +268,17 @@ func (r *GeneralOpenAIRequest) GetTokenCountMeta() *types.TokenCountMeta {
}
for _, tool := range tools {
tokenCountMeta.ToolsCount++
if len(tool.Custom) > 0 {
texts = append(texts, string(tool.Custom))
continue
}
texts = append(texts, tool.Function.Name)
if tool.Function.Description != "" {
texts = append(texts, tool.Function.Description)
}
if tool.Function.Parameters != nil {
texts = append(texts, fmt.Sprintf("%v", tool.Function.Parameters))
parameters, _ := kitutil.Marshal(tool.Function.Parameters)
texts = append(texts, string(parameters))
}
}
//toolTokens := CountTokenInput(countStr, request.Model)
@@ -329,7 +343,8 @@ func GetOpenAIChatCapabilities(modelName, reasoningEffort string) OpenAIChatCapa
}
isGPT5Model := IsOpenAIGPT5Model(modelName)
if !isGPT5Model && !isOpenAIModelSnapshot(modelName, "gpt-6-astra") {
isGPT6SolLuna := isOpenAIModelSnapshot(modelName, "gpt-6-sol") || isOpenAIModelSnapshot(modelName, "gpt-6-luna")
if !isGPT5Model && !isGPT6SolLuna && !isOpenAIModelSnapshot(modelName, "gpt-6-astra") {
return capabilities
}
capabilities.UseMaxCompletionTokens = true
@@ -337,12 +352,15 @@ func GetOpenAIChatCapabilities(modelName, reasoningEffort string) OpenAIChatCapa
// These standard GPT-5 models default to none and support sampling only
// without reasoning. Named variants (pro, codex, chat-latest, etc.) do not
// inherit this exception. GPT-6 Astra never supports these parameters.
// inherit this exception. GPT-6 Sol and Luna follow the same rule. GPT-6
// Astra never supports these parameters.
// https://developers.openai.com/api/docs/guides/latest-model?model=gpt-5.2
// https://developers.openai.com/api/docs/guides/latest-model?model=gpt-5.4
// https://developers.openai.com/api/docs/guides/latest-model?model=gpt-6-astra
// https://developers.openai.com/api/docs/guides/latest-model?model=gpt-6-luna
supportsSampling := false
if isGPT5Model && (reasoningEffort == "" || reasoningEffort == "none") {
if reasoningEffort == "" || reasoningEffort == "none" {
supportsSampling = isGPT6SolLuna
for _, model := range []string{"gpt-5.1", "gpt-5.2", "gpt-5.4"} {
if isOpenAIModelSnapshot(modelName, model) {
supportsSampling = true
@@ -390,6 +408,19 @@ type ToolCallRequest struct {
Type string `json:"type"`
Function FunctionRequest `json:"function"`
Custom json.RawMessage `json:"custom,omitempty"`
// Native is a complete vendor tool object sent instead of the fields
// above. Function has no omitempty, so a non-function tool would
// otherwise carry "function":{"name":""}, which vendor schemas with
// additionalProperties:false reject. Never read from client JSON.
Native json.RawMessage `json:"-"`
}
func (t ToolCallRequest) MarshalJSON() ([]byte, error) {
if len(t.Native) > 0 {
return t.Native, nil
}
type Alias ToolCallRequest
return kitutil.Marshal((*Alias)(&t))
}
type FunctionRequest struct {
@@ -567,9 +598,11 @@ func (m *MediaContent) ToFileSource() types.FileSource {
}
type MessageImageUrl struct {
Url string `json:"url"`
Detail string `json:"detail,omitempty"`
MimeType string
Url string `json:"url"`
Detail string `json:"detail,omitempty"`
// MimeType is in-memory metadata for converters and token counting; it is
// not part of the Chat image_url object.
MimeType string `json:"-"`
}
func (m *MessageImageUrl) IsRemoteImage() bool {
@@ -743,9 +776,9 @@ func (m *Message) ParseContent() []MediaContent {
case ContentTypeImageURL:
imageUrl := contentItem["image_url"]
temp := &MessageImageUrl{
Detail: "high",
}
// An omitted detail stays empty: upstream defaults it to auto, which
// costs more than high on gpt-5.5 and later.
temp := &MessageImageUrl{}
switch v := imageUrl.(type) {
case string:
temp.Url = v
@@ -1103,7 +1136,12 @@ func (r *OpenAIResponsesRequest) GetTokenCountMeta() *types.TokenCountMeta {
}
if len(r.Instructions) > 0 {
texts = append(texts, string(r.Instructions))
var instructions string
if kitutil.GetJsonType(r.Instructions) == "string" && kitutil.Unmarshal(r.Instructions, &instructions) == nil {
texts = append(texts, instructions)
} else {
texts = append(texts, string(r.Instructions))
}
}
if len(r.Metadata) > 0 {
@@ -1162,6 +1200,11 @@ type Input struct {
Type string `json:"type,omitempty"`
Role string `json:"role,omitempty"`
Content json.RawMessage `json:"content,omitempty"`
// Tool-call items replayed in agent loops.
Name string `json:"name,omitempty"`
Arguments json.RawMessage `json:"arguments,omitempty"`
ToolInput json.RawMessage `json:"input,omitempty"`
Output json.RawMessage `json:"output,omitempty"`
}
type MediaInput struct {
@@ -1176,7 +1219,8 @@ type MediaInput struct {
// Reference implementation mirrors Message.ParseContent:
// - input can be a string, treated as an input_text item
// - input can be an array of objects with a `type` field
// supported types: input_text, input_image, input_file
// supported types: input_text, output_text, input_image, input_file
// - tool-call items contribute their name, arguments or input, and output as input_text
func (r *OpenAIResponsesRequest) ParseInput() []MediaInput {
if r.Input == nil {
return nil
@@ -1201,15 +1245,28 @@ func (r *OpenAIResponsesRequest) ParseInput() []MediaInput {
var inputs []Input
_ = kitutil.Unmarshal(r.Input, &inputs)
for _, input := range inputs {
if kitutil.GetJsonType(input.Content) == "string" {
content := input.Content
switch input.Type {
case "function_call":
content = input.Arguments
case "custom_tool_call":
content = input.ToolInput
case "function_call_output", "custom_tool_call_output", "local_shell_call_output":
content = input.Output
}
if input.Name != "" {
mediaInputs = append(mediaInputs, MediaInput{Type: "input_text", Text: input.Name})
}
if kitutil.GetJsonType(content) == "string" {
var str string
_ = kitutil.Unmarshal(input.Content, &str)
_ = kitutil.Unmarshal(content, &str)
mediaInputs = append(mediaInputs, MediaInput{Type: "input_text", Text: str})
}
if kitutil.GetJsonType(input.Content) == "array" {
if kitutil.GetJsonType(content) == "array" {
var array []any
_ = kitutil.Unmarshal(input.Content, &array)
_ = kitutil.Unmarshal(content, &array)
for _, itemAny := range array {
// Already parsed MediaContent
if media, ok := itemAny.(MediaInput); ok {
@@ -1228,7 +1285,7 @@ func (r *OpenAIResponsesRequest) ParseInput() []MediaInput {
continue
}
switch typeVal {
case "input_text":
case "input_text", "output_text":
text, _ := item["text"].(string)
mediaInputs = append(mediaInputs, MediaInput{Type: "input_text", Text: text})
case "input_image":
relaykit/dto/openai_response.go
+31 -1
View File
@@ -374,7 +374,9 @@ type ResponsesOutput struct {
Result string `json:"result,omitempty"`
CallId string `json:"call_id,omitempty"`
Name string `json:"name,omitempty"`
Namespace string `json:"namespace,omitempty"`
Arguments json.RawMessage `json:"arguments,omitempty"`
Input json.RawMessage `json:"input,omitempty"`
Action json.RawMessage `json:"action,omitempty"`
Queries json.RawMessage `json:"queries,omitempty"`
Results json.RawMessage `json:"results,omitempty"`
@@ -393,9 +395,24 @@ type ResponsesOutput struct {
// MarshalJSON keeps hosted-tool variants within their protocol-specific
// schemas. ResponsesOutput also represents messages, images, and function
// calls, whose fields must not leak into web_search_call or mcp_call items.
// calls, whose fields must not leak into web_search_call, mcp_call, or
// custom_tool_call items.
func (r ResponsesOutput) MarshalJSON() ([]byte, error) {
switch r.Type {
case "custom_tool_call":
input := r.Input
if len(input) == 0 {
input = json.RawMessage(`""`)
}
return kitutil.Marshal(struct {
Type string `json:"type"`
ID string `json:"id,omitempty"`
Status string `json:"status,omitempty"`
CallID string `json:"call_id"`
Name string `json:"name"`
Namespace string `json:"namespace,omitempty"`
Input json.RawMessage `json:"input"`
}{Type: r.Type, ID: r.ID, Status: r.Status, CallID: r.CallId, Name: r.Name, Namespace: r.Namespace, Input: input})
case "web_search_call":
return kitutil.Marshal(struct {
Type string `json:"type"`
@@ -425,6 +442,16 @@ func (r ResponsesOutput) MarshalJSON() ([]byte, error) {
Error: r.ItemError,
ApprovalRequestID: r.ApprovalRequestID,
})
case "reasoning":
// summary is required on a reasoning item, even before any summary text exists.
type responsesOutputAlias ResponsesOutput
if len(r.Summary) > 0 {
return kitutil.Marshal(responsesOutputAlias(r))
}
return kitutil.Marshal(struct {
responsesOutputAlias
Summary []ResponsesReasoningSummaryPart `json:"summary"`
}{responsesOutputAlias(r), []ResponsesReasoningSummaryPart{}})
default:
type responsesOutputAlias ResponsesOutput
return kitutil.Marshal(responsesOutputAlias(r))
@@ -555,6 +582,7 @@ type ResponsesStreamResponse struct {
Param string `json:"param,omitempty"`
Delta string `json:"delta,omitempty"`
Arguments *string `json:"arguments,omitempty"`
Input *string `json:"input,omitempty"`
Name string `json:"name,omitempty"`
Text *string `json:"text,omitempty"`
Item *ResponsesOutput `json:"item,omitempty"`
@@ -564,6 +592,8 @@ type ResponsesStreamResponse struct {
Obfuscation string `json:"obfuscation,omitempty"`
// - response.function_call_arguments.delta
// - response.function_call_arguments.done
// - response.custom_tool_call_input.delta
// - response.custom_tool_call_input.done
OutputIndex *int `json:"output_index,omitempty"`
ContentIndex *int `json:"content_index,omitempty"`
SummaryIndex *int `json:"summary_index,omitempty"`
relaykit/dto/token_count_meta_test.go
+100
View File
@@ -0,0 +1,100 @@
package dto
import (
"testing"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Agent loops replay tool definitions, tool calls, and tool output on every
// turn; the prompt estimate must see all of them as the model reads them.
func TestTokenCountMetaCountsAgentLoopContent(t *testing.T) {
const toolOutput = "package main\n\nfunc main() {\n\tprintln(\"hi\")\n}"
const imageData = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="
for _, tc := range []struct {
name string
raw string
request interface{ GetTokenCountMeta() *types.TokenCountMeta }
contains []string
files int
detail string
toolCount int
}{
{
name: "chat",
raw: `{"model":"m","messages":[
{"role":"user","content":[{"type":"text","text":"show main.go"},{"type":"image_url","image_url":{"url":"data:image/png;base64,` + imageData + `"}}]},
{"role":"assistant","content":null,"tool_calls":[{"id":"c1","type":"function","function":{"name":"read_file","arguments":"{\"path\":\"main.go\"}"}}]},
{"role":"tool","tool_call_id":"c1","content":"package main\n\nfunc main() {\n\tprintln(\"hi\")\n}"}],
"tools":[{"type":"function","function":{"name":"read_file","description":"Read a file","parameters":{"type":"object","properties":{"path":{"type":"string"}}}}}]}`,
request: &GeneralOpenAIRequest{},
contains: []string{"read_file", `{"path":"main.go"}`, toolOutput, "Read a file", `"properties":{"path":{"type":"string"}}`},
// An omitted detail stays empty so the estimate uses the upstream default (auto).
files: 1,
toolCount: 1,
},
{
name: "responses",
raw: `{"model":"m","instructions":"Be brief.\nUse tools.","input":[
{"type":"message","role":"user","content":[{"type":"input_text","text":"show main.go"}]},
{"type":"message","role":"assistant","content":[{"type":"output_text","text":"Reading it now."}]},
{"type":"function_call","call_id":"c1","name":"read_file","arguments":"{\"path\":\"main.go\"}"},
{"type":"function_call_output","call_id":"c1","output":"package main\n\nfunc main() {\n\tprintln(\"hi\")\n}"},
{"type":"custom_tool_call","call_id":"c2","name":"apply_patch","input":"*** Begin Patch"},
{"type":"custom_tool_call_output","call_id":"c2","output":[{"type":"input_text","text":"Done!"}]}]}`,
request: &OpenAIResponsesRequest{},
contains: []string{"Be brief.\nUse tools.", "Reading it now.", "read_file", `{"path":"main.go"}`, toolOutput, "apply_patch", "*** Begin Patch", "Done!"},
},
{
name: "claude",
raw: `{"model":"m","max_tokens":1,"messages":[
{"role":"user","content":"show main.go"},
{"role":"assistant","content":[{"type":"tool_use","id":"c1","name":"read_file","input":{"path":"main.go"}}]},
{"role":"user","content":[
{"type":"tool_result","tool_use_id":"c1","content":"package main\n\nfunc main() {\n\tprintln(\"hi\")\n}"},
{"type":"tool_result","tool_use_id":"c2","content":[{"type":"text","text":"screenshot taken"},{"type":"image","source":{"type":"base64","media_type":"image/png","data":"` + imageData + `"}}]}]}],
"tools":[{"name":"read_file","description":"Read a file","input_schema":{"type":"object","properties":{"path":{"type":"string"}}}}]}`,
request: &ClaudeRequest{},
contains: []string{"read_file", `{"path":"main.go"}`, toolOutput, "screenshot taken", "Read a file", `"properties":{"path":{"type":"string"}}`},
files: 1,
toolCount: 1,
},
{
name: "gemini",
raw: `{"systemInstruction":{"parts":[{"text":"Be brief."}]},"contents":[
{"role":"user","parts":[{"text":"show main.go"}]},
{"role":"model","parts":[{"functionCall":{"name":"read_file","args":{"path":"main.go"}}}]},
{"role":"user","parts":[{"functionResponse":{"name":"read_file","response":{"output":"package main\n\nfunc main() {\n\tprintln(\"hi\")\n}","exit_code":0}}}]},
{"role":"user","parts":[{"functionResponse":{"name":"screenshot","response":{},"parts":[{"inlineData":{"mimeType":"image/png","data":"` + imageData + `"}}]}}]}],
"generationConfig":{"mediaResolution":"MEDIA_RESOLUTION_LOW"},
"tools":[{"functionDeclarations":[{"name":"read_file","description":"Read a file"}]}]}`,
request: &GeminiChatRequest{},
contains: []string{"Be brief.", "read_file", `{"path":"main.go"}`, toolOutput, "Read a file"},
files: 1,
detail: "MEDIA_RESOLUTION_LOW",
},
} {
t.Run(tc.name, func(t *testing.T) {
require.NoError(t, kitutil.Unmarshal([]byte(tc.raw), tc.request))
meta := tc.request.GetTokenCountMeta()
for _, want := range tc.contains {
assert.Contains(t, meta.CombineText, want)
}
// Tool output and instructions are counted as raw text, not JSON-escaped,
// and images stay out of the text.
assert.NotContains(t, meta.CombineText, `\n`)
assert.NotContains(t, meta.CombineText, "map[")
assert.NotContains(t, meta.CombineText, imageData)
require.Len(t, meta.Files, tc.files)
for _, file := range meta.Files {
assert.Equal(t, types.FileTypeImage, file.FileType)
assert.Equal(t, tc.detail, file.Detail)
}
assert.Equal(t, tc.toolCount, meta.ToolsCount)
})
}
}
relaykit/dto/usage_merge.go
+18 -13
View File
@@ -264,23 +264,28 @@ func normalizeGeminiModality(modality string) string {
return strings.ToUpper(strings.TrimSpace(modality))
}
// mergeGeminiTokenDetails overlays per-modality counts the same way as the
// scalar counts: every stream frame carries a cumulative snapshot, so a
// modality the incoming frame reports replaces all earlier entries for it,
// and a modality it omits keeps its earlier value. Duplicate entries inside
// one snapshot are kept as-is; settlement sums them.
func mergeGeminiTokenDetails(current []GeminiPromptTokensDetails, incoming []GeminiPromptTokensDetails) []GeminiPromptTokensDetails {
merged := append([]GeminiPromptTokensDetails{}, current...)
indexes := make(map[string]int, len(merged))
for index, detail := range merged {
indexes[normalizeGeminiModality(detail.Modality)] = index
reported := make(map[string]bool, len(incoming))
for _, detail := range incoming {
if detail.TokenCount > 0 {
reported[normalizeGeminiModality(detail.Modality)] = true
}
}
merged := make([]GeminiPromptTokensDetails, 0, len(current)+len(incoming))
for _, detail := range current {
if !reported[normalizeGeminiModality(detail.Modality)] {
merged = append(merged, detail)
}
}
for _, detail := range incoming {
if detail.TokenCount <= 0 {
continue
if detail.TokenCount > 0 {
merged = append(merged, detail)
}
key := normalizeGeminiModality(detail.Modality)
if index, ok := indexes[key]; ok {
merged[index].TokenCount += detail.TokenCount
continue
}
indexes[key] = len(merged)
merged = append(merged, detail)
}
return merged
}
relaykit/dto/usage_merge_test.go
+79 -21
View File
@@ -72,37 +72,95 @@ func TestGeminiModalityKeysSettleConsistentlyAndDuplicateEntriesSum(t *testing.T
})
}
// A case variant in a later cumulative frame is the same modality and
// replaces the earlier value.
mergedDetails := mergeGeminiTokenDetails(
[]GeminiPromptTokensDetails{{Modality: "AUDIO", TokenCount: 10}},
[]GeminiPromptTokensDetails{{Modality: "audio", TokenCount: 15}},
)
require.Len(t, mergedDetails, 1)
assert.Equal(t, 25, mergedDetails[0].TokenCount)
assert.Equal(t, []GeminiPromptTokensDetails{{Modality: "audio", TokenCount: 15}}, mergedDetails)
streamMerged := MergeGeminiUsageMetadataNonZero(
&GeminiUsageMetadata{
PromptTokenCount: 10,
PromptTokensDetails: []GeminiPromptTokensDetails{{Modality: "AUDIO", TokenCount: 10}},
},
&GeminiUsageMetadata{
PromptTokenCount: 25,
PromptTokensDetails: []GeminiPromptTokensDetails{{Modality: "audio", TokenCount: 15}},
},
)
require.NotNil(t, streamMerged)
streamUsage, ok := NewGeminiChatBillingUsage(streamMerged).CanonicalUsage()
require.True(t, ok)
decodedUsage, ok := NewGeminiChatBillingUsage(&GeminiUsageMetadata{
// Duplicate entries inside one snapshot settle as their sum, and merging
// that snapshot again (every frame repeats it) does not add it twice.
snapshot := &GeminiUsageMetadata{
PromptTokenCount: 25,
PromptTokensDetails: []GeminiPromptTokensDetails{
{Modality: "AUDIO", TokenCount: 10},
{Modality: "audio", TokenCount: 15},
},
}).CanonicalUsage()
}
streamMerged := MergeGeminiUsageMetadataNonZero(MergeGeminiUsageMetadataNonZero(nil, snapshot), snapshot)
require.NotNil(t, streamMerged)
streamUsage, ok := NewGeminiChatBillingUsage(streamMerged).CanonicalUsage()
require.True(t, ok)
decodedUsage, ok := NewGeminiChatBillingUsage(snapshot).CanonicalUsage()
require.True(t, ok)
assert.Equal(t, decodedUsage.PromptTokensDetails.AudioTokens, streamUsage.PromptTokensDetails.AudioTokens)
assert.Equal(t, 25, decodedUsage.PromptTokensDetails.AudioTokens)
assert.Equal(t, 25, streamUsage.PromptTokensDetails.AudioTokens)
}
func TestMergeGeminiUsageMetadataStreamDetailsAreCumulativeSnapshots(t *testing.T) {
t.Parallel()
prompt := []GeminiPromptTokensDetails{
{Modality: "IMAGE", TokenCount: 1089},
{Modality: "TEXT", TokenCount: 42},
}
frames := []*GeminiUsageMetadata{
{PromptTokenCount: 1131, TotalTokenCount: 1131, PromptTokensDetails: prompt},
{
PromptTokenCount: 1131,
ToolUsePromptTokenCount: 7,
CandidatesTokenCount: 5,
TotalTokenCount: 1143,
PromptTokensDetails: prompt,
ToolUsePromptTokensDetails: []GeminiPromptTokensDetails{{Modality: "TEXT", TokenCount: 7}},
CandidatesTokensDetails: []GeminiPromptTokensDetails{{Modality: "TEXT", TokenCount: 5}},
},
{
PromptTokenCount: 1131,
ToolUsePromptTokenCount: 7,
CandidatesTokenCount: 1310,
ThoughtsTokenCount: 76,
TotalTokenCount: 2524,
PromptTokensDetails: prompt,
ToolUsePromptTokensDetails: []GeminiPromptTokensDetails{{Modality: "TEXT", TokenCount: 7}},
CandidatesTokensDetails: []GeminiPromptTokensDetails{
{Modality: "TEXT", TokenCount: 20},
{Modality: "IMAGE", TokenCount: 1290},
},
},
// A trailing frame without details keeps the last reported ones.
{PromptTokenCount: 1131, CandidatesTokenCount: 1310, ThoughtsTokenCount: 76, TotalTokenCount: 2524},
}
var merged *GeminiUsageMetadata
for _, frame := range frames {
merged = MergeGeminiUsageMetadataNonZero(merged, frame)
}
require.NotNil(t, merged)
assert.Equal(t, prompt, merged.PromptTokensDetails)
assert.Equal(t, []GeminiPromptTokensDetails{{Modality: "TEXT", TokenCount: 7}}, merged.ToolUsePromptTokensDetails)
assert.Equal(t, []GeminiPromptTokensDetails{
{Modality: "TEXT", TokenCount: 20},
{Modality: "IMAGE", TokenCount: 1290},
}, merged.CandidatesTokensDetails)
streamUsage, ok := NewGeminiChatBillingUsage(merged).CanonicalUsage()
require.True(t, ok)
nonStreamUsage, ok := NewGeminiChatBillingUsage(frames[2]).CanonicalUsage()
require.True(t, ok)
assert.Equal(t, nonStreamUsage, streamUsage)
assert.Equal(t, 1089, streamUsage.PromptTokensDetails.ImageTokens)
assert.Equal(t, 1290, streamUsage.CompletionTokenDetails.ImageTokens)
// Converted streams remember the same cumulative sidecar at several
// pipeline steps; repeated merges must not change the result.
sidecar := NewGeminiChatBillingUsage(merged)
again := MergeBillingUsageNonZero(MergeBillingUsageNonZero(sidecar, sidecar), sidecar)
repeatedUsage, ok := again.CanonicalUsage()
require.True(t, ok)
assert.Equal(t, nonStreamUsage, repeatedUsage)
}
func TestClaudeCacheCreationSubObjectZeroDoesNotReviveFlatLegacyFields(t *testing.T) {
@@ -172,8 +230,8 @@ func TestMergeClaudeUsageNonZeroPreservesBillingUsage(t *testing.T) {
t.Parallel()
currentSidecar := NewGeminiChatBillingUsage(&GeminiUsageMetadata{
PromptTokenCount: 3868,
TotalTokenCount: 3868,
PromptTokenCount: 3868,
TotalTokenCount: 3868,
CachedContentTokenCount: 20,
})
incomingSidecar := NewGeminiChatBillingUsage(&GeminiUsageMetadata{
relaykit/dto/user_settings.go
+2
View File
@@ -3,6 +3,7 @@ package dto
type UserSetting struct {
NotifyType string `json:"notify_type,omitempty"` // QuotaWarningType 额度预警类型
QuotaWarningThreshold float64 `json:"quota_warning_threshold,omitempty"` // QuotaWarningThreshold 额度预警阈值
QuotaWarningEnabled *bool `json:"quota_warning_enabled,omitempty"` // 是否接收余额不足提醒，未设置时接收
WebhookUrl string `json:"webhook_url,omitempty"` // WebhookUrl webhook地址
WebhookSecret string `json:"webhook_secret,omitempty"` // WebhookSecret webhook密钥
NotificationEmail string `json:"notification_email,omitempty"` // NotificationEmail 通知邮箱地址
@@ -11,6 +12,7 @@ type UserSetting struct {
GotifyToken string `json:"gotify_token,omitempty"` // GotifyToken Gotify应用令牌
GotifyPriority int `json:"gotify_priority"` // GotifyPriority Gotify消息优先级
UpstreamModelUpdateNotifyEnabled bool `json:"upstream_model_update_notify_enabled,omitempty"` // 是否接收上游模型更新定时检测通知（仅管理员）
ChannelNotifyEnabled *bool `json:"channel_notify_enabled,omitempty"` // 是否接收渠道通知（仅管理员），未设置时 root 接收、其他管理员不接收
AcceptUnsetRatioModel bool `json:"accept_unset_model_ratio_model,omitempty"` // AcceptUnsetRatioModel 是否接受未设置价格的模型
RecordIpLog bool `json:"record_ip_log,omitempty"` // 是否记录请求和错误日志IP
SidebarModules string `json:"sidebar_modules,omitempty"` // SidebarModules 左侧边栏模块配置
relaykit/relayconvert/convmeta/meta.go
+87
View File
@@ -5,6 +5,8 @@
package convmeta
import (
"strings"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/types"
)
@@ -40,6 +42,15 @@ type Meta interface {
// nil receiver may return a temporary initialized state.
EnsureClaudeConvertInfo() *ClaudeConvertInfo
// ResponsesToolState returns the Responses tool encoding recorded by the
// latest request conversion, or nil when none was recorded. Response
// converters read it to restore Responses-only tool call shapes.
ResponsesToolState() *ResponsesToolState
// SetResponsesToolState replaces that record; nil clears it. Request
// conversion calls it on every attempt so retries never inherit a record
// from another channel.
SetResponsesToolState(state *ResponsesToolState)
// GetSendResponseCount / IncrSendResponseCount expose the shared
// downstream-chunk counter (the host may also increment it).
GetSendResponseCount() int
@@ -80,6 +91,60 @@ type ClaudeStreamToolCall struct {
Started bool
}
// ResponsesToolState records how Responses-only tool definitions were encoded
// for the upstream protocol, so the matching response can be restored.
type ResponsesToolState struct {
// CustomToolNames lists Responses custom (freeform) tools that were sent
// upstream as function tools taking one string "input" argument. Function
// calls with these names are custom tool calls.
CustomToolNames map[string]struct{}
// Namespaces maps an upstream function name to the Responses tool
// namespace it was flattened from; see NamespacedToolName.
Namespaces map[string]string
}
// CustomToolInputArgument is the single function argument that carries a
// Responses custom tool input through an upstream function call.
const CustomToolInputArgument = "input"
// DefaultResponsesToolNamespace is the namespace Codex declares its ordinary
// tools in. OpenAI returns calls to its tools by bare name without a
// namespace field, so its tools keep their names upstream.
const DefaultResponsesToolNamespace = "functions"
// NamespacedToolName is the upstream function name of a tool declared in a
// Responses tool namespace: "<namespace>__<name>", or the bare name for the
// default namespace.
func NamespacedToolName(namespace string, name string) string {
if namespace == "" || namespace == DefaultResponsesToolNamespace {
return name
}
return namespace + "__" + name
}
// IsCustomTool reports whether name was encoded from a Responses custom tool.
func (s *ResponsesToolState) IsCustomTool(name string) bool {
if s == nil {
return false
}
_, ok := s.CustomToolNames[name]
return ok
}
// ResponsesToolName restores the Responses namespace and tool name of an
// upstream function name. Names that were not flattened from a namespace
// return an empty namespace and the name unchanged.
func (s *ResponsesToolState) ResponsesToolName(upstreamName string) (string, string) {
if s == nil {
return "", upstreamName
}
namespace, ok := s.Namespaces[upstreamName]
if !ok {
return "", upstreamName
}
return namespace, strings.TrimPrefix(upstreamName, namespace+"__")
}
const (
LastMessageTypeNone = "none"
LastMessageTypeText = "text"
@@ -101,6 +166,7 @@ type Values struct {
EstimatePromptTokens int
ClaudeConvertInfo *ClaudeConvertInfo
ResponsesTools *ResponsesToolState
SendResponseCount int
ConversionChain []types.RelayFormat
@@ -182,6 +248,19 @@ func (v *Values) EnsureClaudeConvertInfo() *ClaudeConvertInfo {
return v.ClaudeConvertInfo
}
func (v *Values) ResponsesToolState() *ResponsesToolState {
if v == nil {
return nil
}
return v.ResponsesTools
}
func (v *Values) SetResponsesToolState(state *ResponsesToolState) {
if v != nil {
v.ResponsesTools = state
}
}
func (v *Values) GetSendResponseCount() int {
if v == nil {
return 0
@@ -240,6 +319,14 @@ func OptionsOf(m Meta) *Options {
return m.ConvOptions()
}
// ResponsesToolStateOf is a nil-safe reader for Meta.ResponsesToolState.
func ResponsesToolStateOf(m Meta) *ResponsesToolState {
if m == nil {
return nil
}
return m.ResponsesToolState()
}
// ReasoningStateOf is a nil-safe reader for Meta.ReasoningState.
func ReasoningStateOf(m Meta) *dto.ReasoningConversionState {
if m == nil {
relaykit/relayconvert/convmeta/options.go
+29 -6
View File
@@ -23,6 +23,12 @@ type Options struct {
// it from the channel type.
OpenRouterDialect bool
// WebSearch encodes hosted web search for the upstream dialect. Nil, or a
// nil result, keeps the default: web_search_options on Chat and the native
// hosted tool on Responses, Claude, and Gemini. The host sets it from the
// channel adaptor's Init.
WebSearch WebSearchEncoder
// PreserveThinkingSuffix reports models whose -thinking/-nothinking/effort
// suffix must be kept on the outgoing model name (host blacklist lookup).
// Nil means "never preserve".
@@ -34,9 +40,13 @@ type Options struct {
}
type ClaudeOptions struct {
// ThinkingAdapterEnabled controls whether suffix-derived reasoning intent
// is rendered onto Claude thinking / output_config. Suffix parsing itself
// is the host entry layer's job (standalone users call Parse* themselves).
// ThinkingAdapterEnabled is the host thinking-suffix adapter toggle. The
// host passes it to reasoning.ParseClaudeModelSuffix, so -thinking,
// -nothinking, -thinking-<budget>, and effort tails such as -high on
// claude-* names become reasoning intent only while it is on; when off the
// model name is sent verbatim. Explicit @ modifiers are unaffected. Suffix
// parsing itself is the host entry layer's job (standalone users call
// Parse* themselves).
ThinkingAdapterEnabled bool
// ThinkingAdapterBudgetTokensPercentage sizes thinking budget_tokens as a
// fraction of max_tokens when the adapter fires.
@@ -57,9 +67,12 @@ type ClaudeOptions struct {
}
type GeminiOptions struct {
// ThinkingAdapterEnabled controls whether suffix-derived reasoning intent
// is rendered onto Gemini thinkingConfig. Suffix parsing itself is the
// host entry layer's job (standalone users call Parse* themselves).
// ThinkingAdapterEnabled is the host thinking-suffix adapter toggle. The
// host and gemini_chat request conversion pass it to
// reasoning.ParseGeminiModelSuffix, so -thinking, -nothinking,
// -thinking-<budget>, and effort tails such as -high on gemini-* names
// become reasoning intent only while it is on; when off the model name is
// sent verbatim. Explicit @ modifiers are unaffected.
ThinkingAdapterEnabled bool
// ThinkingAdapterBudgetTokensPercentage sizes thinkingBudget as a fraction
// of maxOutputTokens when the adapter fires.
@@ -70,6 +83,12 @@ type GeminiOptions struct {
// SupportsImagine reports whether the model supports image generation
// (switches response modalities). Nil means "never".
SupportsImagine func(modelName string) bool
// SupportsJSONSchema reports whether the upstream accepts JSON Schema in
// parametersJsonSchema and responseJsonSchema for the model. A schema that
// Gemini's typed Schema cannot hold without loss is then sent there
// unchanged; otherwise it is cleaned to the typed Schema and the loss is
// reported as a diagnostic. Nil means "never".
SupportsJSONSchema func(modelName string) bool
// SafetySetting returns the harm threshold for a category. Nil or empty
// return means no safetySettings are attached.
SafetySetting func(category string) string
@@ -86,6 +105,10 @@ func (o *GeminiOptions) SupportsImagineModel(modelName string) bool {
return o != nil && o.SupportsImagine != nil && o.SupportsImagine(modelName)
}
func (o *GeminiOptions) SupportsJSONSchemaModel(modelName string) bool {
return o != nil && o.SupportsJSONSchema != nil && o.SupportsJSONSchema(modelName)
}
func (o *GeminiOptions) SafetySettingFor(category string) string {
if o == nil || o.SafetySetting == nil {
return ""
relaykit/relayconvert/convmeta/websearch.go
+95
View File
@@ -0,0 +1,95 @@
package convmeta
import (
"encoding/json"
"fmt"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/types"
)
// WebSearch is the protocol-neutral hosted web-search specification decoded
// from the source request.
type WebSearch struct {
Location *ApproximateLocation
AllowedDomains []string
BlockedDomains []string
SearchContextSize string
MaxUses *int
AllowedCallers []string
ResponseInclusion string
ExternalWebAccess *bool
ReturnTokenBudget json.RawMessage
}
type ApproximateLocation struct {
City string
Region string
Country string
Timezone string
}
// WebSearchCall is one hosted web-search definition about to be encoded for
// the target request. Exactly one of Chat, Responses, Claude, Gemini is set;
// the encoder may write top-level fields of that request directly.
type WebSearchCall struct {
Source types.RelayFormat
Target types.RelayFormat
// NativeType is the source tool type, for example web_search_preview,
// web_search_20250305, or googleSearch.
NativeType string
// Index is the position in the source tool list. Chat Completions
// carries one hosted web search, so a Chat encoder sees only the first
// definition; the other targets call the encoder once per definition.
Index int
// Forced reports that the source tool_choice named the web-search tool.
Forced bool
Spec WebSearch
Chat *dto.GeneralOpenAIRequest
Responses *dto.OpenAIResponsesRequest
Claude *dto.ClaudeRequest
Gemini *dto.GeminiChatRequest
}
// WebSearchEncoding is what an encoder produced for one WebSearchCall.
type WebSearchEncoding struct {
// Tool is appended to the target tool list in the target protocol's own
// shape. A Chat target requires a dto.ToolCallRequest (any other type
// fails the conversion); the others take a JSON-marshalable value (usually
// map[string]any). Nil means the encoder wrote top-level request fields
// instead.
Tool any
// Omitted reports that no hosted search reaches the upstream. relaykit
// records hosted_web_search_omitted (error severity) so ToolLossPolicy
// can reject it, and appends no tool.
Omitted bool
// Diagnostics are merged into the conversion result. Error-severity
// diagnostics count as semantic loss: under the channel's "safe" or
// "strict" ToolLossPolicy they reject the request, so return
// ConversionDiagnosticError only when the search behavior the client
// asked for is actually lost, and ConversionDiagnosticWarning for tuning
// that does not change whether or what is searched.
Diagnostics []types.ConversionDiagnostic
}
// WebSearchEncoder encodes hosted web search for an upstream whose dialect
// differs from the target protocol's default. A nil result keeps relaykit's
// default encoding; an error fails the conversion and is reserved for
// encodings that cannot be produced at all, never for ordinary loss.
type WebSearchEncoder func(call WebSearchCall) (*WebSearchEncoding, error)
// Path is the diagnostic path of the definition in the source tool list.
func (c WebSearchCall) Path() string {
return fmt.Sprintf("tools[%d]", c.Index)
}
// SemanticLoss builds an error-severity diagnostic for this definition.
func (c WebSearchCall) SemanticLoss(code, message string) types.ConversionDiagnostic {
return types.ConversionDiagnostic{Code: code, Path: c.Path(), Message: message, Severity: types.ConversionDiagnosticError}
}
// PresentationLoss builds a warning-severity diagnostic for this definition.
func (c WebSearchCall) PresentationLoss(code, message string) types.ConversionDiagnostic {
return types.ConversionDiagnostic{Code: code, Path: c.Path(), Message: message, Severity: types.ConversionDiagnosticWarning}
}
relaykit/relayconvert/internal/claude_messages/to_oai_chat_req.go
+119 -12
View File
@@ -7,8 +7,10 @@ import (
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
"github.com/QuantumNous/new-api/relaykit/relayconvert/internal/convdiag"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/relayconvert/reasoning"
"github.com/QuantumNous/new-api/relaykit/types"
)
const (
@@ -154,7 +156,13 @@ func ClaudeMessagesRequestToOpenAIChat(ctx context.Context, claudeRequest dto.Cl
}
}
for _, claudeMessage := range claudeRequest.Messages {
type unnamedToolResult struct {
index int
id string
}
toolNames := make(map[string]string)
var unnamedToolResults []unnamedToolResult
for messageIndex, claudeMessage := range claudeRequest.Messages {
openAIMessage := dto.Message{
Role: claudeMessage.Role,
}
@@ -168,7 +176,10 @@ func ClaudeMessagesRequestToOpenAIChat(ctx context.Context, claudeRequest dto.Cl
var toolCalls []dto.ToolCallRequest
mediaMessages := make([]dto.MediaContent, 0, len(content))
for _, mediaMsg := range content {
for blockIndex, mediaMsg := range content {
if _, exists := toolNames[mediaMsg.Id]; !exists {
toolNames[mediaMsg.Id] = mediaMsg.Name
}
switch mediaMsg.Type {
case "text", "input_text":
message := dto.MediaContent{
@@ -178,12 +189,50 @@ func ClaudeMessagesRequestToOpenAIChat(ctx context.Context, claudeRequest dto.Cl
}
mediaMessages = append(mediaMessages, message)
case "image":
imageData := fmt.Sprintf("data:%s;base64,%s", mediaMsg.Source.MediaType, mediaMsg.Source.Data)
mediaMessage := dto.MediaContent{
Type: "image_url",
ImageUrl: &dto.MessageImageUrl{Url: imageData},
url := claudeImageSourceURL(mediaMsg.Source)
if url == "" {
sourceType := ""
if mediaMsg.Source != nil {
sourceType = mediaMsg.Source.Type
}
convdiag.Add(ctx, types.ConversionDiagnostic{
Code: "unsupported_media_source",
Path: fmt.Sprintf("messages[%d].content[%d].source", messageIndex, blockIndex),
Message: fmt.Sprintf("OpenAI Chat Completions cannot reference a Claude image with source type %q; the image was omitted", sourceType),
Severity: types.ConversionDiagnosticError,
})
continue
}
mediaMessages = append(mediaMessages, mediaMessage)
mediaMessages = append(mediaMessages, dto.MediaContent{
Type: "image_url",
ImageUrl: &dto.MessageImageUrl{Url: url},
})
case "document":
if mediaMsg.Source != nil && mediaMsg.Source.Type == "text" {
// A plain-text document keeps its content as a Chat text part.
mediaMessages = append(mediaMessages, dto.MediaContent{Type: "text", Text: kitutil.Interface2String(mediaMsg.Source.Data)})
continue
}
if mediaMsg.Source == nil || mediaMsg.Source.Type != "base64" || mediaMsg.Source.MediaType != "application/pdf" {
sourceType, mediaType := "", ""
if mediaMsg.Source != nil {
sourceType, mediaType = mediaMsg.Source.Type, mediaMsg.Source.MediaType
}
convdiag.Add(ctx, types.ConversionDiagnostic{
Code: "unsupported_media_source",
Path: fmt.Sprintf("messages[%d].content[%d].source", messageIndex, blockIndex),
Message: fmt.Sprintf("OpenAI Chat Completions carries only base64 PDF and plain-text documents, not a %q source of type %q; the document was omitted", sourceType, mediaType),
Severity: types.ConversionDiagnosticError,
})
continue
}
mediaMessages = append(mediaMessages, dto.MediaContent{
Type: dto.ContentTypeFile,
File: &dto.MessageFile{
FileName: "document.pdf",
FileData: "data:application/pdf;base64," + kitutil.Interface2String(mediaMsg.Source.Data),
},
})
case "tool_use":
toolCall := dto.ToolCallRequest{
ID: mediaMsg.Id,
@@ -197,7 +246,7 @@ func ClaudeMessagesRequestToOpenAIChat(ctx context.Context, claudeRequest dto.Cl
case "tool_result":
toolName := mediaMsg.Name
if toolName == "" {
toolName = claudeRequest.SearchToolNameByToolCallId(mediaMsg.ToolUseId)
unnamedToolResults = append(unnamedToolResults, unnamedToolResult{index: len(openAIMessages), id: mediaMsg.ToolUseId})
}
oaiToolMessage := dto.Message{
Role: "tool",
@@ -207,9 +256,12 @@ func ClaudeMessagesRequestToOpenAIChat(ctx context.Context, claudeRequest dto.Cl
if mediaMsg.IsStringContent() {
oaiToolMessage.SetStringContent(mediaMsg.GetStringContent())
} else {
mediaContents := mediaMsg.ParseMediaContent()
encodedJSON, _ := kitutil.Marshal(mediaContents)
oaiToolMessage.SetStringContent(string(encodedJSON))
content, media := claudeToolResultToChat(mediaMsg.ParseMediaContent())
oaiToolMessage.SetStringContent(content)
// A Chat tool message only carries text. Images from the tool result
// join the user message this Claude message becomes, which lands right
// after the tool batch and keeps the tool messages contiguous.
mediaMessages = append(mediaMessages, media...)
}
openAIMessages = append(openAIMessages, oaiToolMessage)
}
@@ -218,7 +270,7 @@ func ClaudeMessagesRequestToOpenAIChat(ctx context.Context, claudeRequest dto.Cl
if len(toolCalls) > 0 {
openAIMessage.SetToolCalls(toolCalls)
}
if len(mediaMessages) > 0 && len(toolCalls) == 0 {
if len(mediaMessages) > 0 {
openAIMessage.SetMediaContent(mediaMessages)
}
}
@@ -226,11 +278,66 @@ func ClaudeMessagesRequestToOpenAIChat(ctx context.Context, claudeRequest dto.Cl
openAIMessages = append(openAIMessages, openAIMessage)
}
}
for _, result := range unnamedToolResults {
*openAIMessages[result.index].Name = toolNames[result.id]
}
openAIRequest.Messages = openAIMessages
return &openAIRequest, nil
}
// claudeToolResultToChat maps a structured tool_result content array onto Chat Completions,
// where a tool message may only carry text. Text blocks stay on the tool message and image
// blocks come back in Chat shape for the caller to place on the following user message;
// stringifying them instead would hand base64 image data to the upstream text tokenizer.
// Arrays holding any other block type keep the historical stringified form so nothing is lost.
func claudeToolResultToChat(blocks []dto.ClaudeMediaMessage) (string, []dto.MediaContent) {
texts := make([]string, 0, len(blocks))
media := make([]dto.MediaContent, 0, len(blocks))
for _, block := range blocks {
switch {
case block.Type == "text" || block.Type == "input_text":
if text := block.GetText(); text != "" {
texts = append(texts, text)
}
case block.Type == "image":
url := claudeImageSourceURL(block.Source)
if url == "" {
return requestToJSONString(blocks), nil
}
media = append(media, dto.MediaContent{Type: "image_url", ImageUrl: &dto.MessageImageUrl{Url: url}})
default:
return requestToJSONString(blocks), nil
}
}
switch {
case len(texts) == 0 && len(media) == 0:
return requestToJSONString(blocks), nil
case len(texts) == 0:
// Upstreams reject empty tool content; the images ride on the following user message.
return "[image]", media
default:
return strings.Join(texts, "\n"), media
}
}
// claudeImageSourceURL is the Chat image_url for a Claude image source: a data
// URL for base64 data or the URL of a url source. Other sources, such as Files
// API references, have no Chat equivalent and yield "".
func claudeImageSourceURL(source *dto.ClaudeMessageSource) string {
if source == nil || source.Type == "file" {
return ""
}
if source.Url != "" {
return source.Url
}
data := kitutil.Interface2String(source.Data)
if data == "" {
return ""
}
return fmt.Sprintf("data:%s;base64,%s", source.MediaType, data)
}
func requestToJSONString(v any) string {
b, err := kitutil.Marshal(v)
if err != nil {
relaykit/relayconvert/internal/claude_messages/to_oai_chat_req_test.go
+151
View File
@@ -0,0 +1,151 @@
package claudemessages
import (
"context"
"testing"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
"github.com/samber/lo"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestClaudeMessagesRequestToOpenAIChatToolResultContent(t *testing.T) {
const imageData = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="
const dataURL = "data:image/png;base64," + imageData
imageBlock := dto.ClaudeMediaMessage{
Type: "image",
Source: &dto.ClaudeMessageSource{Type: "base64", MediaType: "image/png", Data: imageData},
}
textBlock := func(text string) dto.ClaudeMediaMessage {
return dto.ClaudeMediaMessage{Type: "text", Text: lo.ToPtr(text)}
}
tests := []struct {
name string
content any
wantContent string
wantImages []string
}{
{
name: "string passes through",
content: "done",
wantContent: "done",
},
{
name: "text and image keep text on tool message",
content: []dto.ClaudeMediaMessage{textBlock("screenshot taken"), imageBlock},
wantContent: "screenshot taken",
wantImages: []string{dataURL},
},
{
name: "image only uses placeholder",
content: []dto.ClaudeMediaMessage{imageBlock},
wantContent: "[image]",
wantImages: []string{dataURL},
},
{
name: "url image source is forwarded as is",
content: []dto.ClaudeMediaMessage{{
Type: "image",
Source: &dto.ClaudeMessageSource{Type: "url", Url: "https://example.com/shot.png"},
}},
wantContent: "[image]",
wantImages: []string{"https://example.com/shot.png"},
},
{
name: "text blocks join with newline",
content: []dto.ClaudeMediaMessage{textBlock("first"), textBlock(""), textBlock("second")},
wantContent: "first\nsecond",
},
{
name: "unknown block keeps whole array",
content: []dto.ClaudeMediaMessage{textBlock("see doc"), {Type: "document", Data: "opaque"}},
wantContent: `[{"type":"text","text":"see doc"},{"type":"document","data":"opaque"}]`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ClaudeMessagesRequestToOpenAIChat(context.Background(), dto.ClaudeRequest{
Model: "claude-test",
MaxTokens: lo.ToPtr(uint(64)),
Messages: []dto.ClaudeMessage{
{Role: "user", Content: "take a screenshot"},
{Role: "assistant", Content: []dto.ClaudeMediaMessage{
{Type: "tool_use", Id: "toolu_1", Name: "screenshot", Input: map[string]any{}},
}},
{Role: "user", Content: []dto.ClaudeMediaMessage{
{Type: "tool_result", ToolUseId: "toolu_1", Content: tt.content},
textBlock("what do you see?"),
}},
},
}, &convmeta.Values{})
require.NoError(t, err)
require.Len(t, got.Messages, 4)
tool := got.Messages[2]
assert.Equal(t, "tool", tool.Role)
assert.Equal(t, "toolu_1", tool.ToolCallId)
if tt.wantImages == nil && tt.wantContent[0] == '[' {
assert.JSONEq(t, tt.wantContent, tool.StringContent())
} else {
assert.Equal(t, tt.wantContent, tool.StringContent())
}
user := got.Messages[3]
assert.Equal(t, "user", user.Role)
parts := user.ParseContent()
require.Len(t, parts, len(tt.wantImages)+1)
for i, wantURL := range tt.wantImages {
assert.Equal(t, dto.ContentTypeImageURL, parts[i].Type)
require.NotNil(t, parts[i].GetImageMedia())
assert.Equal(t, wantURL, parts[i].GetImageMedia().Url)
}
assert.Equal(t, dto.ContentTypeText, parts[len(parts)-1].Type)
assert.Equal(t, "what do you see?", parts[len(parts)-1].Text)
})
}
}
func TestClaudeMessagesRequestToOpenAIChatKeepsParallelToolResultsContiguous(t *testing.T) {
const dataURL = "data:image/png;base64,iVBORw0KGgo="
got, err := ClaudeMessagesRequestToOpenAIChat(context.Background(), dto.ClaudeRequest{
Model: "claude-test",
MaxTokens: lo.ToPtr(uint(64)),
Messages: []dto.ClaudeMessage{
{Role: "user", Content: "screenshot and read the note"},
{Role: "assistant", Content: []dto.ClaudeMediaMessage{
{Type: "text", Text: lo.ToPtr("Let me look.")},
{Type: "tool_use", Id: "toolu_shot", Name: "screenshot", Input: map[string]any{}},
{Type: "tool_use", Id: "toolu_read", Name: "read_file", Input: map[string]any{"path": "note.txt"}},
}},
{Role: "user", Content: []dto.ClaudeMediaMessage{
{Type: "tool_result", ToolUseId: "toolu_shot", Content: []dto.ClaudeMediaMessage{{
Type: "image",
Source: &dto.ClaudeMessageSource{Type: "base64", MediaType: "image/png", Data: "iVBORw0KGgo="},
}}},
{Type: "tool_result", ToolUseId: "toolu_read", Content: "note contents"},
}},
},
}, &convmeta.Values{})
require.NoError(t, err)
roles := make([]string, 0, len(got.Messages))
for _, message := range got.Messages {
roles = append(roles, message.Role)
}
assert.Equal(t, []string{"user", "assistant", "tool", "tool", "user"}, roles)
assert.Len(t, got.Messages[1].ParseToolCalls(), 2)
assert.Equal(t, "[image]", got.Messages[2].StringContent())
assert.Equal(t, "toolu_shot", got.Messages[2].ToolCallId)
assert.Equal(t, "note contents", got.Messages[3].StringContent())
assert.NotContains(t, got.Messages[2].StringContent()+got.Messages[3].StringContent(), "base64")
parts := got.Messages[4].ParseContent()
require.Len(t, parts, 1)
assert.Equal(t, dto.ContentTypeImageURL, parts[0].Type)
require.NotNil(t, parts[0].GetImageMedia())
assert.Equal(t, dataURL, parts[0].GetImageMedia().Url)
}
relaykit/relayconvert/internal/claude_messages/to_oai_chat_resp.go
+10
View File
@@ -19,6 +19,8 @@ type ClaudeResponseInfo struct {
Created int64
Model string
ResponseText strings.Builder
// ToolUseCount counts streamed tool_use blocks for the output-token fallback.
ToolUseCount int
Usage *dto.Usage
Done bool
@@ -524,6 +526,9 @@ func FormatClaudeResponseInfo(claudeResponse *dto.ClaudeResponse, oaiResponse *d
if claudeResponse.Delta.Thinking != nil {
claudeInfo.ResponseText.WriteString(*claudeResponse.Delta.Thinking)
}
if claudeResponse.Delta.PartialJson != nil {
claudeInfo.ResponseText.WriteString(*claudeResponse.Delta.PartialJson)
}
}
} else if claudeResponse.Type == "message_delta" {
if claudeResponse.Usage != nil {
@@ -552,6 +557,11 @@ func FormatClaudeResponseInfo(claudeResponse *dto.ClaudeResponse, oaiResponse *d
claudeInfo.Done = true
} else if claudeResponse.Type == "content_block_start" {
// Tool calls are most of an agent turn's output; count them for usage fallback.
if block := claudeResponse.ContentBlock; block != nil && block.Type == "tool_use" {
claudeInfo.ResponseText.WriteString(block.Name)
claudeInfo.ToolUseCount++
}
} else {
return false
}
relaykit/relayconvert/internal/claude_messages/to_oai_responses_req.go
+7 -1
View File
@@ -10,6 +10,7 @@ import (
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
"github.com/QuantumNous/new-api/relaykit/relayconvert/internal/convdiag"
relaymedia "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/media"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/relayconvert/reasoning"
)
@@ -177,6 +178,7 @@ func claudeMessagesToResponsesInput(messages []dto.ClaudeMessage) (json.RawMessa
contentParts = append(contentParts, map[string]any{
"type": "input_image",
"image_url": source,
"detail": relaymedia.DefaultResponsesImageDetail,
})
}
case "document":
@@ -298,7 +300,11 @@ func claudeToolResultToResponsesOutput(content any) (any, error) {
parts = append(parts, map[string]any{"type": "input_text", "text": block.GetText()})
case "image":
if source := claudeSourceURL(block.Source); source != "" {
parts = append(parts, map[string]any{"type": "input_image", "image_url": source})
parts = append(parts, map[string]any{
"type": "input_image",
"image_url": source,
"detail": relaymedia.DefaultResponsesImageDetail,
})
}
case "document":
if source := claudeSourceURL(block.Source); source != "" {
relaykit/relayconvert/internal/gemini_chat/to_oai_chat_req.go
+83 -20
View File
@@ -9,8 +9,10 @@ import (
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
"github.com/QuantumNous/new-api/relaykit/relayconvert/internal/convdiag"
"github.com/QuantumNous/new-api/relaykit/relayconvert/internal/jsonutil"
relaymedia "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/media"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/relayconvert/reasoning"
"github.com/QuantumNous/new-api/relaykit/types"
)
func GeminiGenerateContentRequestToOpenAIChat(ctx context.Context, geminiRequest *dto.GeminiChatRequest, info convmeta.Meta) (*dto.GeneralOpenAIRequest, error) {
@@ -72,7 +74,7 @@ func GeminiGenerateContentRequestToOpenAIChat(ctx context.Context, geminiRequest
callHistory := newGeminiFunctionCallHistory(geminiRequest.Contents)
var messages []dto.Message
for _, content := range geminiRequest.Contents {
for contentIndex, content := range geminiRequest.Contents {
message := dto.Message{
Role: convertGeminiRoleToOpenAI(content.Role),
}
@@ -80,7 +82,7 @@ func GeminiGenerateContentRequestToOpenAIChat(ctx context.Context, geminiRequest
var mediaContents []dto.MediaContent
var toolCalls []dto.ToolCallRequest
var reasoningTexts []string
for _, part := range content.Parts {
for partIndex, part := range content.Parts {
if part.Text != "" {
if part.Thought {
reasoningTexts = append(reasoningTexts, part.Text)
@@ -91,24 +93,20 @@ func GeminiGenerateContentRequestToOpenAIChat(ctx context.Context, geminiRequest
Text: part.Text,
}
mediaContents = append(mediaContents, mediaContent)
} else if part.InlineData != nil {
mediaContent := dto.MediaContent{
Type: "image_url",
ImageUrl: &dto.MessageImageUrl{
Url: fmt.Sprintf("data:%s;base64,%s", part.InlineData.MimeType, part.InlineData.Data),
Detail: "auto",
MimeType: part.InlineData.MimeType,
},
}
mediaContents = append(mediaContents, mediaContent)
} else if part.FileData != nil {
mediaContent := dto.MediaContent{
Type: "image_url",
ImageUrl: &dto.MessageImageUrl{
Url: part.FileData.FileUri,
Detail: "auto",
MimeType: part.FileData.MimeType,
},
} else if part.InlineData != nil || part.FileData != nil {
mediaContent, reason := geminiMediaToChatContent(part.InlineData, part.FileData)
if reason != "" {
kind := "inlineData"
if part.InlineData == nil {
kind = "fileData"
}
convdiag.Add(ctx, types.ConversionDiagnostic{
Code: "unsupported_media_type",
Path: fmt.Sprintf("contents[%d].parts[%d].%s", contentIndex, partIndex, kind),
Message: reason + "; the part was omitted",
Severity: types.ConversionDiagnosticError,
})
continue
}
mediaContents = append(mediaContents, mediaContent)
} else if part.FunctionCall != nil {
@@ -208,6 +206,71 @@ func GeminiGenerateContentRequestToOpenAIChat(ctx context.Context, geminiRequest
return openaiRequest, nil
}
// chatImageMediaTypes are the image formats Chat Completions accepts.
var chatImageMediaTypes = map[string]struct{}{
"image/png": {},
"image/jpeg": {},
"image/webp": {},
"image/gif": {},
}
// geminiMediaToChatContent maps Gemini inlineData or fileData onto the Chat
// content part for its MIME type: PNG, JPEG, WebP, or GIF images inline or by
// URL, PDF as an inline file, text files as text, and WAV or MP3 as inline
// audio. Whether the model accepts the part is left to the upstream. A
// non-empty reason means the media has no Chat content part.
func geminiMediaToChatContent(inlineData *dto.GeminiInlineData, fileData *dto.GeminiFileData) (dto.MediaContent, string) {
if inlineData == nil {
mimeType := strings.ToLower(strings.TrimSpace(fileData.MimeType))
if mimeType == "image/jpg" {
mimeType = "image/jpeg"
}
if _, supported := chatImageMediaTypes[mimeType]; mimeType != "" && !supported {
return dto.MediaContent{}, fmt.Sprintf("Chat Completions content cannot reference a Gemini file of MIME type %q", fileData.MimeType)
}
return dto.MediaContent{
Type: dto.ContentTypeImageURL,
ImageUrl: &dto.MessageImageUrl{Url: fileData.FileUri, Detail: "auto", MimeType: mimeType},
}, ""
}
mimeType := strings.ToLower(strings.TrimSpace(inlineData.MimeType))
if mimeType == "image/jpg" {
mimeType = "image/jpeg"
}
if _, supported := chatImageMediaTypes[mimeType]; supported {
return dto.MediaContent{
Type: dto.ContentTypeImageURL,
ImageUrl: &dto.MessageImageUrl{Url: fmt.Sprintf("data:%s;base64,%s", mimeType, inlineData.Data), Detail: "auto", MimeType: mimeType},
}, ""
}
switch {
case strings.HasPrefix(mimeType, "image/"):
return dto.MediaContent{}, fmt.Sprintf("Chat Completions content accepts only PNG, JPEG, WebP, and GIF images, not %q", inlineData.MimeType)
case mimeType == "application/pdf":
return dto.MediaContent{
Type: dto.ContentTypeFile,
File: &dto.MessageFile{FileName: "document.pdf", FileData: "data:application/pdf;base64," + inlineData.Data},
}, ""
case relaymedia.IsTextMimeType(mimeType):
text, ok := relaymedia.DecodeText(inlineData.Data)
if !ok {
return dto.MediaContent{}, fmt.Sprintf("Gemini inlineData of MIME type %q could not be decoded as UTF-8 text", inlineData.MimeType)
}
return dto.MediaContent{Type: dto.ContentTypeText, Text: text}, ""
case mimeType == "audio/wav" || mimeType == "audio/x-wav" || mimeType == "audio/wave":
return dto.MediaContent{
Type: dto.ContentTypeInputAudio,
InputAudio: &dto.MessageInputAudio{Data: inlineData.Data, Format: "wav"},
}, ""
case mimeType == "audio/mp3" || mimeType == "audio/mpeg":
return dto.MediaContent{
Type: dto.ContentTypeInputAudio,
InputAudio: &dto.MessageInputAudio{Data: inlineData.Data, Format: "mp3"},
}, ""
}
return dto.MediaContent{}, fmt.Sprintf("Gemini inlineData of MIME type %q has no Chat Completions content part", inlineData.MimeType)
}
type geminiPendingFunctionCall struct {
id string
name string
relaykit/relayconvert/internal/media/media.go
+40 -3
View File
@@ -1,10 +1,13 @@
package media
import (
"errors"
"sync"
"context"
"encoding/base64"
"errors"
"strings"
"sync"
"unicode/utf8"
"github.com/QuantumNous/new-api/relaykit/types"
)
@@ -44,3 +47,37 @@ func DecodeBase64FileData(base64String string) (string, string, error) {
}
return resolver(base64String)
}
// DefaultResponsesImageDetail is used when the source protocol carries no detail
// hint. Sending the default explicitly also supports strict Responses validators
// that reject input_image parts without detail.
const DefaultResponsesImageDetail = "auto"
// ResponsesImageDetail preserves supported image detail levels and defaults
// missing or unrecognized hints to "auto".
func ResponsesImageDetail(detail string) string {
switch normalized := strings.ToLower(strings.TrimSpace(detail)); normalized {
case "low", "high", "auto", "original":
return normalized
default:
return DefaultResponsesImageDetail
}
}
// IsTextMimeType reports whether a MIME type is plain text that a target can
// carry as text: text/* and application/json, ignoring parameters.
func IsTextMimeType(mimeType string) bool {
mediaType, _, _ := strings.Cut(strings.ToLower(strings.TrimSpace(mimeType)), ";")
mediaType = strings.TrimSpace(mediaType)
return strings.HasPrefix(mediaType, "text/") || mediaType == "application/json"
}
// DecodeText decodes base64 file data as text. It reports false when the data
// is not valid base64 or the bytes are not UTF-8.
func DecodeText(base64Data string) (string, bool) {
decoded, err := base64.StdEncoding.DecodeString(strings.TrimSpace(base64Data))
if err != nil || !utf8.Valid(decoded) {
return "", false
}
return string(decoded), true
}
relaykit/relayconvert/internal/oai_chat/to_claude_messages_req.go
+24 -17
View File
@@ -2,7 +2,6 @@ package oaichat
import (
"fmt"
"strings"
"context"
@@ -13,6 +12,7 @@ import (
sharedclaude "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/shared/claude"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/relayconvert/reasoning"
"github.com/QuantumNous/new-api/relaykit/types"
)
func OpenAIChatRequestToClaudeMessages(c context.Context, info convmeta.Meta, textRequest dto.GeneralOpenAIRequest) (*dto.ClaudeRequest, error) {
@@ -265,24 +265,27 @@ func OpenAIChatRequestToClaudeMessages(c context.Context, info convmeta.Meta, te
if source == nil {
continue
}
base64Data, mimeType, err := relaymedia.ResolveBase64Data(c, source, "formatting image for Claude")
if err != nil {
return nil, fmt.Errorf("get file data failed: %s", err.Error())
// Claude content blocks carry images, PDF documents, and text
// documents; audio and video have no Claude block.
reason := fmt.Sprintf("Claude Messages cannot carry %s content", mediaMessage.Type)
var block dto.ClaudeMediaMessage
if mediaMessage.Type != dto.ContentTypeInputAudio && mediaMessage.Type != dto.ContentTypeVideoUrl {
base64Data, mimeType, err := relaymedia.ResolveBase64Data(c, source, "formatting image for Claude")
if err != nil {
return nil, fmt.Errorf("get file data failed: %s", err.Error())
}
block, reason = sharedclaude.MediaBlock(base64Data, mimeType, mediaMessage.Type == dto.ContentTypeImageURL)
}
claudeMediaMessage := dto.ClaudeMediaMessage{
Source: &dto.ClaudeMessageSource{
Type: "base64",
},
if reason != "" {
convdiag.Add(c, types.ConversionDiagnostic{
Code: "unsupported_media_type",
Path: "messages.content",
Message: reason + "; the part was omitted",
Severity: types.ConversionDiagnosticError,
})
continue
}
if strings.HasPrefix(mimeType, "application/pdf") {
claudeMediaMessage.Type = "document"
} else {
claudeMediaMessage.Type = "image"
}
claudeMediaMessage.Source.MediaType = mimeType
claudeMediaMessage.Source.Data = base64Data
claudeMediaMessages = append(claudeMediaMessages, claudeMediaMessage)
claudeMediaMessages = append(claudeMediaMessages, block)
continue
}
}
@@ -303,6 +306,10 @@ func OpenAIChatRequestToClaudeMessages(c context.Context, info convmeta.Meta, te
})
}
}
if len(claudeMediaMessages) == 0 {
// Every part was omitted; Claude rejects an empty content array.
claudeMediaMessages = append(claudeMediaMessages, dto.ClaudeMediaMessage{Type: "text", Text: kitutil.GetPointer("...")})
}
claudeMessage.Content = claudeMediaMessages
}
claudeMessages = append(claudeMessages, claudeMessage)
relaykit/relayconvert/internal/oai_chat/to_claude_messages_resp.go
+43 -43
View File
@@ -241,9 +241,6 @@ func StreamResponseOpenAI2Claude(openAIResponse *dto.ChatCompletionsStreamRespon
state.FinishReason = *chosenChoice.FinishReason
}
var claudeResponse dto.ClaudeResponse
var isEmpty bool
claudeResponse.Type = "content_block_delta"
if len(chosenChoice.Delta.ToolCalls) > 0 {
toolCalls := chosenChoice.Delta.ToolCalls
if state.LastMessagesType != convmeta.LastMessageTypeTools {
@@ -339,55 +336,58 @@ func StreamResponseOpenAI2Claude(openAIResponse *dto.ChatCompletionsStreamRespon
state.Index = state.ToolCallBaseIndex + len(state.ToolCalls) - 1
}
} else {
// Some upstreams put the last reasoning token and the first answer
// text in one chunk: the thinking block closes before the text block opens.
reasoning := chosenChoice.Delta.GetReasoningContent()
textContent := chosenChoice.Delta.GetContentString()
if reasoning != "" || textContent != "" {
if reasoning != "" {
if state.LastMessagesType != convmeta.LastMessageTypeThinking {
stopOpenBlocksAndAdvance()
idx := state.Index
claudeResponses = append(claudeResponses, &dto.ClaudeResponse{
Index: &idx,
Type: "content_block_start",
ContentBlock: &dto.ClaudeMediaMessage{
Type: "thinking",
Thinking: kitutil.GetPointer[string](""),
},
})
}
state.LastMessagesType = convmeta.LastMessageTypeThinking
claudeResponse.Delta = &dto.ClaudeMediaMessage{
if reasoning != "" {
if state.LastMessagesType != convmeta.LastMessageTypeThinking {
stopOpenBlocksAndAdvance()
idx := state.Index
claudeResponses = append(claudeResponses, &dto.ClaudeResponse{
Index: &idx,
Type: "content_block_start",
ContentBlock: &dto.ClaudeMediaMessage{
Type: "thinking",
Thinking: kitutil.GetPointer[string](""),
},
})
}
state.LastMessagesType = convmeta.LastMessageTypeThinking
claudeResponses = append(claudeResponses, &dto.ClaudeResponse{
Index: kitutil.GetPointer[int](state.Index),
Type: "content_block_delta",
Delta: &dto.ClaudeMediaMessage{
Type: "thinking_delta",
Thinking: &reasoning,
}
} else {
if state.LastMessagesType != convmeta.LastMessageTypeText {
stopOpenBlocksAndAdvance()
idx := state.Index
claudeResponses = append(claudeResponses, &dto.ClaudeResponse{
Index: &idx,
Type: "content_block_start",
ContentBlock: &dto.ClaudeMediaMessage{
Type: "text",
Text: kitutil.GetPointer[string](""),
},
})
}
state.LastMessagesType = convmeta.LastMessageTypeText
claudeResponse.Delta = &dto.ClaudeMediaMessage{
},
})
}
if textContent != "" {
if state.LastMessagesType != convmeta.LastMessageTypeText {
stopOpenBlocksAndAdvance()
idx := state.Index
claudeResponses = append(claudeResponses, &dto.ClaudeResponse{
Index: &idx,
Type: "content_block_start",
ContentBlock: &dto.ClaudeMediaMessage{
Type: "text",
Text: kitutil.GetPointer[string](""),
},
})
}
state.LastMessagesType = convmeta.LastMessageTypeText
claudeResponses = append(claudeResponses, &dto.ClaudeResponse{
Index: kitutil.GetPointer[int](state.Index),
Type: "content_block_delta",
Delta: &dto.ClaudeMediaMessage{
Type: "text_delta",
Text: kitutil.GetPointer[string](textContent),
}
}
} else {
isEmpty = true
},
})
}
}
claudeResponse.Index = kitutil.GetPointer[int](state.Index)
if !isEmpty && claudeResponse.Delta != nil {
claudeResponses = append(claudeResponses, &claudeResponse)
}
appendCitationDeltas(chosenChoice.Delta.Annotations)
if doneChunk || state.Done {
relaykit/relayconvert/internal/oai_chat/to_gemini_chat_req.go
+3 -6
View File
@@ -186,11 +186,8 @@ func OpenAIChatRequestToGeminiGenerateContent(c context.Context, textRequest dto
geminiRequest.GenerationConfig.ResponseMimeType = "application/json"
if len(textRequest.ResponseFormat.JsonSchema) > 0 {
var jsonSchema dto.FormatJsonSchema
if err := kitutil.Unmarshal(textRequest.ResponseFormat.JsonSchema, &jsonSchema); err == nil {
cleanedSchema := sharedgemini.RemoveAdditionalProperties(jsonSchema.Schema, 0)
geminiRequest.GenerationConfig.ResponseSchema = cleanedSchema
}
sharedgemini.ApplyResponseSchema(c, &geminiRequest.GenerationConfig, textRequest.ResponseFormat.JsonSchema,
"response_format.json_schema.schema", opts.Gemini.SupportsJSONSchemaModel(upstreamModelName))
}
}
@@ -339,7 +336,7 @@ func OpenAIChatRequestToGeminiGenerateContent(c context.Context, textRequest dto
return nil, fmt.Errorf("get file data from '%s' failed: %w", source.GetIdentifier(), err)
}
if _, ok := sharedgemini.SupportedMimeTypes[strings.ToLower(mimeType)]; !ok {
if !sharedgemini.IsSupportedMimeType(mimeType) {
return nil, fmt.Errorf("mime type is not supported by Gemini: '%s', url: '%s', supported types are: %v", mimeType, source.GetIdentifier(), sharedgemini.SupportedMimeTypesList())
}
relaykit/relayconvert/internal/oai_chat/to_oai_responses_req.go
+20 -4
View File
@@ -9,6 +9,7 @@ import (
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/internal/convdiag"
relaymedia "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/media"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/relayconvert/reasoning"
"github.com/samber/lo"
@@ -227,9 +228,14 @@ func ChatCompletionsRequestToResponsesRequest(ctx context.Context, req *dto.Gene
"text": part.Text,
})
case dto.ContentTypeImageURL:
var sourceDetail string
if img := part.GetImageMedia(); img != nil {
sourceDetail = img.Detail
}
contentParts = append(contentParts, map[string]any{
"type": "input_image",
"image_url": normalizeChatImageURLToString(part.ImageUrl),
"detail": relaymedia.ResponsesImageDetail(sourceDetail),
})
case dto.ContentTypeInputAudio:
contentParts = append(contentParts, map[string]any{
@@ -237,10 +243,20 @@ func ChatCompletionsRequestToResponsesRequest(ctx context.Context, req *dto.Gene
"input_audio": part.InputAudio,
})
case dto.ContentTypeFile:
contentParts = append(contentParts, map[string]any{
"type": "input_file",
"file": part.File,
})
// Responses carries the file fields on the input_file part itself.
inputFile := map[string]any{"type": "input_file"}
if file := part.GetFile(); file != nil {
if file.FileName != "" {
inputFile["filename"] = file.FileName
}
if file.FileData != "" {
inputFile["file_data"] = file.FileData
}
if file.FileId != "" {
inputFile["file_id"] = file.FileId
}
}
contentParts = append(contentParts, inputFile)
case dto.ContentTypeVideoUrl:
contentParts = append(contentParts, map[string]any{
"type": "input_video",
relaykit/relayconvert/internal/oai_chat/to_oai_responses_req_test.go
+31
View File
@@ -40,6 +40,37 @@ func TestChatCompletionsRequestToResponsesRequestInstructionsAndTools(t *testing
assert.Equal(t, "function_call_output", gjson.GetBytes(got.Input, "3.type").String())
}
func TestChatCompletionsRequestToResponsesRequestImageDetail(t *testing.T) {
tests := []struct {
name string
imageURL any
want string
}{
{name: "object without detail defaults to auto", imageURL: map[string]any{"url": "https://example.test/a.png"}, want: "auto"},
{name: "low detail is preserved", imageURL: map[string]any{"url": "https://example.test/a.png", "detail": "low"}, want: "low"},
{name: "high detail is preserved", imageURL: map[string]any{"url": "https://example.test/a.png", "detail": "high"}, want: "high"},
{name: "auto detail is preserved", imageURL: map[string]any{"url": "https://example.test/a.png", "detail": "auto"}, want: "auto"},
{name: "original detail is preserved", imageURL: map[string]any{"url": "https://example.test/a.png", "detail": "original"}, want: "original"},
{name: "unknown detail falls back to auto", imageURL: map[string]any{"url": "https://example.test/a.png", "detail": "bogus"}, want: "auto"},
{name: "plain string url defaults to auto", imageURL: "https://example.test/a.png", want: "auto"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ChatCompletionsRequestToResponsesRequest(context.Background(), &dto.GeneralOpenAIRequest{
Model: "gpt-test",
Messages: []dto.Message{{Role: "user", Content: []any{
map[string]any{"type": "image_url", "image_url": tt.imageURL},
}}},
})
require.NoError(t, err)
assert.Equal(t, "input_image", gjson.GetBytes(got.Input, "0.content.0.type").String())
assert.Equal(t, tt.want, gjson.GetBytes(got.Input, "0.content.0.detail").String())
})
}
}
func TestChatCompletionsRequestToResponsesRequestPreservesPromptCacheKey(t *testing.T) {
t.Run("present", func(t *testing.T) {
key := "session-\"quoted\"\\path\n世界"
relaykit/relayconvert/internal/oai_chat/to_oai_responses_resp.go
+53 -3
View File
@@ -8,6 +8,7 @@ import (
"time"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
)
@@ -24,9 +25,14 @@ const (
responsesEventOutputItemDone = "response.output_item.done"
responsesEventFunctionArgsDelta = "response.function_call_arguments.delta"
responsesEventFunctionArgsDone = "response.function_call_arguments.done"
responsesEventCustomToolInputDelta = "response.custom_tool_call_input.delta"
responsesEventCustomToolInputDone = "response.custom_tool_call_input.done"
responsesEventReasoningSummaryPartAdded = "response.reasoning_summary_part.added"
responsesEventReasoningSummaryDelta = "response.reasoning_summary_text.delta"
responsesEventReasoningSummaryDone = "response.reasoning_summary_text.done"
responsesEventReasoningSummaryPartDone = "response.reasoning_summary_part.done"
responsesOutputTypeFunctionCall = "function_call"
responsesOutputTypeCustomToolCall = "custom_tool_call"
responsesOutputTypeMessage = "message"
responsesOutputTypeReasoning = "reasoning"
responsesIncompleteReasonContentFilter = "content_filter"
@@ -34,6 +40,12 @@ const (
)
func ChatCompletionsResponseToResponsesResponse(resp *dto.OpenAITextResponse, id string) (*dto.OpenAIResponsesResponse, *dto.Usage, error) {
return ChatCompletionsResponseToResponsesResponseWithTools(resp, id, nil)
}
// ChatCompletionsResponseToResponsesResponseWithTools also restores function
// calls that tools marks as Responses custom tools into custom_tool_call items.
func ChatCompletionsResponseToResponsesResponseWithTools(resp *dto.OpenAITextResponse, id string, tools *convmeta.ResponsesToolState) (*dto.OpenAIResponsesResponse, *dto.Usage, error) {
if resp == nil {
return nil, nil, errors.New("response is nil")
}
@@ -93,7 +105,10 @@ func ChatCompletionsResponseToResponsesResponse(resp *dto.OpenAITextResponse, id
}
for i, toolCall := range choice.Message.ParseToolCalls() {
toolOutput, err := chatToolCallToResponsesOutput(toolCall, id, i, responseOutputStatus(out))
if (toolCall.Type == "" || toolCall.Type == "function") && strings.TrimSpace(toolCall.Function.Name) == "" {
continue
}
toolOutput, err := chatToolCallToResponsesOutput(toolCall, id, i, responseOutputStatus(out), tools)
if err != nil {
return nil, nil, err
}
@@ -202,18 +217,31 @@ func responseStatusString(resp *dto.OpenAIResponsesResponse) string {
return strings.TrimSpace(status)
}
func chatToolCallToResponsesOutput(toolCall dto.ToolCallRequest, responseID string, index int, status string) (dto.ResponsesOutput, error) {
func chatToolCallToResponsesOutput(toolCall dto.ToolCallRequest, responseID string, index int, status string, tools *convmeta.ResponsesToolState) (dto.ResponsesOutput, error) {
callID := strings.TrimSpace(toolCall.ID)
if callID == "" {
callID = fmt.Sprintf("%s_call_%d", responseID, index)
}
if toolCall.Type == "" || toolCall.Type == "function" {
namespace, name := tools.ResponsesToolName(toolCall.Function.Name)
if tools.IsCustomTool(toolCall.Function.Name) {
return dto.ResponsesOutput{
Type: responsesOutputTypeCustomToolCall,
ID: callID,
Status: status,
CallId: callID,
Name: name,
Namespace: namespace,
Input: chatArgumentsRawMessage(customToolInputFromArguments(toolCall.Function.Arguments)),
}, nil
}
return dto.ResponsesOutput{
Type: responsesOutputTypeFunctionCall,
ID: callID,
Status: status,
CallId: callID,
Name: toolCall.Function.Name,
Name: name,
Namespace: namespace,
Arguments: chatArgumentsRawMessage(toolCall.Function.Arguments),
}, nil
}
@@ -226,6 +254,28 @@ func chatToolCallToResponsesOutput(toolCall dto.ToolCallRequest, responseID stri
}, nil
}
// customToolInputFromArguments unwraps the raw custom tool input from the
// {"input": ...} arguments a Chat function call carries. Arguments of any other
// shape are returned unchanged so the model's output is never dropped.
func customToolInputFromArguments(arguments string) string {
var value map[string]any
if err := kitutil.UnmarshalJsonStr(arguments, &value); err != nil {
return arguments
}
input, ok := value[convmeta.CustomToolInputArgument]
if !ok {
return arguments
}
if text, ok := input.(string); ok {
return text
}
raw, err := kitutil.Marshal(input)
if err != nil {
return arguments
}
return string(raw)
}
func chatArgumentsRawMessage(arguments string) []byte {
raw, err := kitutil.Marshal(arguments)
if err != nil {
relaykit/relayconvert/internal/oai_chat/to_oai_responses_resp_test.go
+410
View File
@@ -4,6 +4,8 @@ import (
"testing"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/samber/lo"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -41,6 +43,54 @@ func TestChatCompletionsResponseToResponsesPreservesTextToolCallsAndUsage(t *tes
assert.Equal(t, `"{\"q\":\"x\"}"`, string(resp.Output[1].Arguments))
}
func TestChatCompletionsResponseToResponsesDropsEmptyNameToolCalls(t *testing.T) {
usage := dto.Usage{PromptTokens: 20, CompletionTokens: 5, TotalTokens: 25}
usage.PromptTokensDetails.CachedTokens = 7
usage.BillingUsage = dto.NewOpenAIChatBillingUsage(&usage)
message := dto.Message{Role: "assistant"}
message.SetToolCalls([]dto.ToolCallRequest{
{
ID: "call_invalid",
Type: "function",
Function: dto.FunctionRequest{
Arguments: `{"ok":true}`,
},
},
{
ID: "call_whitespace",
Function: dto.FunctionRequest{Name: " \t\n", Arguments: `{}`},
},
{
ID: "call_valid",
Type: "function",
Function: dto.FunctionRequest{
Name: "lookup",
Arguments: `{"q":"x"}`,
},
},
})
resp, convertedUsage, err := ChatCompletionsResponseToResponsesResponse(&dto.OpenAITextResponse{
Usage: usage,
Choices: []dto.OpenAITextResponseChoice{{
Message: message,
FinishReason: "tool_calls",
}},
}, "resp_1")
require.NoError(t, err)
require.Len(t, resp.Output, 1)
assert.Equal(t, "call_valid", resp.Output[0].CallId)
assert.Equal(t, "lookup", resp.Output[0].Name)
assert.Equal(t, `"{\"q\":\"x\"}"`, string(resp.Output[0].Arguments))
assert.Equal(t, 20, convertedUsage.InputTokens)
assert.Equal(t, 5, convertedUsage.OutputTokens)
assert.Equal(t, 25, convertedUsage.TotalTokens)
require.NotNil(t, convertedUsage.InputTokensDetails)
assert.Equal(t, 7, convertedUsage.InputTokensDetails.CachedTokens)
assert.Equal(t, usage.BillingUsage, convertedUsage.BillingUsage)
}
func TestChatCompletionsResponseToResponsesEmitsReasoningSummaryBeforeText(t *testing.T) {
message := dto.Message{Role: "assistant", Content: "final answer"}
message.ReasoningContent = lo.ToPtr("thinking summary")
@@ -153,6 +203,366 @@ func TestChatCompletionsStreamToResponsesEventsAggregatesUsageAndToolArgs(t *tes
assert.Equal(t, `"{\"q\":\"x\"}"`, string(events[9].Payload.Response.Output[1].Arguments))
}
func TestChatCompletionsStreamToResponsesEmitsReasoningSummaryPartLifecycle(t *testing.T) {
state := NewChatToResponsesStreamState("resp_1", "gpt-test")
stop := "stop"
var events []ChatToResponsesStreamEvent
events = append(events, mustResponsesEventsFromChatChunk(t, state, &dto.ChatCompletionsStreamResponse{
Id: "chatcmpl_1", Model: "gpt-test",
Choices: []dto.ChatCompletionsStreamResponseChoice{{
Delta: dto.ChatCompletionsStreamResponseChoiceDelta{ReasoningContent: lo.ToPtr("think")},
}},
})...)
events = append(events, mustResponsesEventsFromChatChunk(t, state, &dto.ChatCompletionsStreamResponse{
Choices: []dto.ChatCompletionsStreamResponseChoice{{
Delta: dto.ChatCompletionsStreamResponseChoiceDelta{Content: lo.ToPtr("answer")},
FinishReason: &stop,
}},
})...)
events = append(events, FinalizeChatCompletionsStreamToResponses(state)...)
types := make([]string, 0, len(events))
for _, event := range events {
types = append(types, event.Type)
}
assert.Equal(t, []string{
responsesEventCreated,
responsesEventOutputItemAdded,
responsesEventReasoningSummaryPartAdded,
responsesEventReasoningSummaryDelta,
responsesEventOutputItemAdded,
responsesEventOutputTextDelta,
"response.output_text.done",
responsesEventOutputItemDone,
responsesEventReasoningSummaryDone,
responsesEventReasoningSummaryPartDone,
responsesEventOutputItemDone,
responsesEventCompleted,
}, types)
// The Responses schema requires summary on every reasoning item, including
// the one announced before any summary text exists.
itemAdded, err := kitutil.Marshal(events[1].Payload)
require.NoError(t, err)
assert.Contains(t, string(itemAdded), `"summary":[]`)
partAdded := events[2].Payload
assert.Equal(t, "resp_1_reasoning_0", partAdded.ItemID)
assert.Equal(t, 0, lo.FromPtr(partAdded.OutputIndex))
assert.Equal(t, 0, lo.FromPtr(partAdded.SummaryIndex))
require.NotNil(t, partAdded.Part)
assert.Equal(t, dto.ResponsesReasoningSummaryPart{Type: "summary_text"}, *partAdded.Part)
partDone := events[9].Payload
assert.Equal(t, "resp_1_reasoning_0", partDone.ItemID)
require.NotNil(t, partDone.Part)
assert.Equal(t, dto.ResponsesReasoningSummaryPart{Type: "summary_text", Text: "think"}, *partDone.Part)
}
func TestChatCompletionsStreamToResponsesReopensItemsAfterMidStreamFinishReason(t *testing.T) {
for _, emitSequenceNumber := range []bool{false, true} {
t.Run(map[bool]string{false: "legacy", true: "sequence_numbers"}[emitSequenceNumber], func(t *testing.T) {
state := NewChatToResponsesStreamState("resp_1", "gpt-test")
state.EmitSequenceNumber = emitSequenceNumber
toolIndex := 0
toolCalls := "tool_calls"
stop := "stop"
chunks := []*dto.ChatCompletionsStreamResponse{
{Id: "chatcmpl_1", Model: "gpt-test", Choices: []dto.ChatCompletionsStreamResponseChoice{{
Delta: dto.ChatCompletionsStreamResponseChoiceDelta{ReasoningContent: lo.ToPtr("round 1")},
}}},
{Choices: []dto.ChatCompletionsStreamResponseChoice{{
Delta: dto.ChatCompletionsStreamResponseChoiceDelta{Content: lo.ToPtr("calling")},
}}},
{Choices: []dto.ChatCompletionsStreamResponseChoice{{
Delta: dto.ChatCompletionsStreamResponseChoiceDelta{ToolCalls: []dto.ToolCallResponse{{
Index: &toolIndex, ID: "call_1", Type: "function",
Function: dto.FunctionResponse{Name: "lookup", Arguments: "{}"},
}}},
}}},
{Choices: []dto.ChatCompletionsStreamResponseChoice{{FinishReason: &toolCalls}}},
{Choices: []dto.ChatCompletionsStreamResponseChoice{{
Delta: dto.ChatCompletionsStreamResponseChoiceDelta{ReasoningContent: lo.ToPtr("round 2")},
}}},
{Choices: []dto.ChatCompletionsStreamResponseChoice{{
Delta: dto.ChatCompletionsStreamResponseChoiceDelta{Content: lo.ToPtr("final")},
FinishReason: &stop,
}}},
}
var events []ChatToResponsesStreamEvent
for _, chunk := range chunks {
events = append(events, mustResponsesEventsFromChatChunk(t, state, chunk)...)
}
events = append(events, FinalizeChatCompletionsStreamToResponses(state)...)
open := map[string]bool{}
partAdded, partDone := 0, 0
for _, event := range events {
itemID := event.Payload.ItemID
if event.Payload.Item != nil {
itemID = event.Payload.Item.ID
}
switch event.Type {
case responsesEventOutputItemAdded:
assert.Falsef(t, open[itemID], "item %q added twice", itemID)
open[itemID] = true
case responsesEventOutputItemDone:
assert.Truef(t, open[itemID], "item %q done without being open", itemID)
delete(open, itemID)
case responsesEventReasoningSummaryPartAdded:
partAdded++
assert.Truef(t, open[itemID], "part added for closed item %q", itemID)
case responsesEventReasoningSummaryPartDone:
partDone++
assert.Truef(t, open[itemID], "part done for closed item %q", itemID)
case responsesEventReasoningSummaryDelta, responsesEventOutputTextDelta:
assert.Truef(t, open[itemID], "%s for %q arrived without an active item", event.Type, itemID)
}
}
assert.Empty(t, open)
assert.Equal(t, 2, partAdded)
assert.Equal(t, 2, partDone)
completed := events[len(events)-1]
require.Equal(t, responsesEventCompleted, completed.Type)
require.NotNil(t, completed.Payload.Response)
output := completed.Payload.Response.Output
require.Len(t, output, 5)
assert.Equal(t, []string{
responsesOutputTypeReasoning, responsesOutputTypeMessage, responsesOutputTypeFunctionCall,
responsesOutputTypeReasoning, responsesOutputTypeMessage,
}, []string{output[0].Type, output[1].Type, output[2].Type, output[3].Type, output[4].Type})
assert.Equal(t, "resp_1_reasoning_0", output[0].ID)
assert.Equal(t, "round 1", output[0].Summary[0].Text)
assert.Equal(t, "resp_1_msg_0", output[1].ID)
assert.Equal(t, "calling", output[1].Content[0].Text)
assert.Equal(t, "call_1", output[2].CallId)
assert.Equal(t, "resp_1_reasoning_1", output[3].ID)
assert.Equal(t, "round 2", output[3].Summary[0].Text)
assert.Equal(t, "resp_1_msg_1", output[4].ID)
assert.Equal(t, "final", output[4].Content[0].Text)
for _, item := range output {
assert.Equal(t, "completed", item.Status)
}
})
}
}
func execCustomToolState() *convmeta.ResponsesToolState {
return &convmeta.ResponsesToolState{CustomToolNames: map[string]struct{}{"exec": {}}}
}
func TestChatCompletionsResponseToResponsesRestoresCustomToolCalls(t *testing.T) {
message := dto.Message{Role: "assistant"}
message.SetToolCalls([]dto.ToolCallRequest{
{ID: "call_exec", Type: "function", Function: dto.FunctionRequest{Name: "exec", Arguments: `{"input":"echo \"hi\" && ls"}`}},
{ID: "call_wait", Type: "function", Function: dto.FunctionRequest{Name: "wait", Arguments: `{"ms":1}`}},
})
chat := &dto.OpenAITextResponse{
Model: "gpt-test",
Choices: []dto.OpenAITextResponseChoice{{Message: message, FinishReason: "tool_calls"}},
}
resp, _, err := ChatCompletionsResponseToResponsesResponseWithTools(chat, "resp_1", execCustomToolState())
require.NoError(t, err)
require.Len(t, resp.Output, 2)
custom, err := kitutil.Marshal(resp.Output[0])
require.NoError(t, err)
assert.JSONEq(t, `{"type":"custom_tool_call","id":"call_exec","status":"completed","call_id":"call_exec","name":"exec","input":"echo \"hi\" && ls"}`, string(custom))
assert.Equal(t, responsesOutputTypeFunctionCall, resp.Output[1].Type)
assert.Equal(t, `"{\"ms\":1}"`, string(resp.Output[1].Arguments))
plain, _, err := ChatCompletionsResponseToResponsesResponse(chat, "resp_1")
require.NoError(t, err)
assert.Equal(t, responsesOutputTypeFunctionCall, plain.Output[0].Type)
}
func TestCustomToolInputFromArguments(t *testing.T) {
tests := map[string]string{
`{"input":"ls -la"}`: "ls -la",
`{"input":""}`: "",
`{"input":{"cmd":"ls"}}`: `{"cmd":"ls"}`,
`{"cmd":"ls"}`: `{"cmd":"ls"}`,
`ls -la`: "ls -la",
``: "",
`{"input":"a","extra":true}`: "a",
`{"input":"\u003cdiv\u003e"}`: "<div>",
}
for arguments, want := range tests {
assert.Equalf(t, want, customToolInputFromArguments(arguments), "arguments %q", arguments)
}
}
func TestChatCompletionsStreamToResponsesRestoresCustomToolCalls(t *testing.T) {
state := NewChatToResponsesStreamState("resp_1", "gpt-test")
state.EmitSequenceNumber = true
state.Tools = execCustomToolState()
execIndex, waitIndex := 0, 1
var events []ChatToResponsesStreamEvent
for _, delta := range []dto.ToolCallResponse{
{Index: &execIndex, ID: "call_exec", Type: "function", Function: dto.FunctionResponse{Name: "exec"}},
{Index: &execIndex, Function: dto.FunctionResponse{Arguments: `{"input":"echo `}},
{Index: &execIndex, Function: dto.FunctionResponse{Arguments: `\"hi\""}`}},
{Index: &waitIndex, ID: "call_wait", Type: "function", Function: dto.FunctionResponse{Name: "wait", Arguments: `{"ms":1}`}},
} {
events = append(events, mustResponsesEventsFromChatChunk(t, state, &dto.ChatCompletionsStreamResponse{
Choices: []dto.ChatCompletionsStreamResponseChoice{
{Index: 0, Delta: dto.ChatCompletionsStreamResponseChoiceDelta{ToolCalls: []dto.ToolCallResponse{delta}}},
},
})...)
}
events = append(events, FinalizeChatCompletionsStreamToResponses(state)...)
types := make([]string, 0, len(events))
for _, event := range events {
types = append(types, event.Type)
}
assert.Equal(t, []string{
responsesEventCreated,
responsesEventOutputItemAdded,
responsesEventOutputItemAdded,
responsesEventFunctionArgsDelta,
responsesEventCustomToolInputDelta,
responsesEventCustomToolInputDone,
responsesEventOutputItemDone,
responsesEventFunctionArgsDone,
responsesEventOutputItemDone,
responsesEventCompleted,
}, types)
added, err := kitutil.Marshal(events[1].Payload.Item)
require.NoError(t, err)
assert.JSONEq(t, `{"type":"custom_tool_call","id":"call_exec","status":"in_progress","call_id":"call_exec","name":"exec","input":""}`, string(added))
assert.Equal(t, `echo "hi"`, events[4].Payload.Delta)
assert.Equal(t, "call_exec", events[4].Payload.ItemID)
require.NotNil(t, events[5].Payload.Input)
assert.Equal(t, `echo "hi"`, *events[5].Payload.Input)
assert.Equal(t, responsesOutputTypeCustomToolCall, events[6].Payload.Item.Type)
assert.Equal(t, `"echo \"hi\""`, string(events[6].Payload.Item.Input))
output := events[len(events)-1].Payload.Response.Output
require.Len(t, output, 2)
assert.Equal(t, responsesOutputTypeCustomToolCall, output[0].Type)
assert.Equal(t, `"echo \"hi\""`, string(output[0].Input))
assert.Equal(t, responsesOutputTypeFunctionCall, output[1].Type)
assert.Equal(t, `"{\"ms\":1}"`, string(output[1].Arguments))
}
func TestChatCompletionsStreamToResponsesHoldsNamelessToolUntilNameArrives(t *testing.T) {
toolIndex := 0
chunk := func(call dto.ToolCallResponse) *dto.ChatCompletionsStreamResponse {
return &dto.ChatCompletionsStreamResponse{
Choices: []dto.ChatCompletionsStreamResponseChoice{
{Index: 0, Delta: dto.ChatCompletionsStreamResponseChoiceDelta{ToolCalls: []dto.ToolCallResponse{call}}},
},
}
}
t.Run("custom tool", func(t *testing.T) {
state := NewChatToResponsesStreamState("resp_1", "gpt-test")
state.Tools = execCustomToolState()
first := mustResponsesEventsFromChatChunk(t, state, chunk(dto.ToolCallResponse{Index: &toolIndex, ID: "call_exec", Function: dto.FunctionResponse{Arguments: `{"input":`}}))
for _, event := range first {
assert.NotEqual(t, responsesEventOutputItemAdded, event.Type)
assert.NotEqual(t, responsesEventFunctionArgsDelta, event.Type)
}
second := mustResponsesEventsFromChatChunk(t, state, chunk(dto.ToolCallResponse{Index: &toolIndex, Function: dto.FunctionResponse{Name: "exec", Arguments: `"ls"}`}}))
require.Len(t, second, 1)
assert.Equal(t, responsesEventOutputItemAdded, second[0].Type)
assert.Equal(t, responsesOutputTypeCustomToolCall, second[0].Payload.Item.Type)
done := FinalizeChatCompletionsStreamToResponses(state)
require.NotEmpty(t, done)
output := done[len(done)-1].Payload.Response.Output
require.Len(t, output, 1)
assert.Equal(t, `"ls"`, string(output[0].Input))
})
t.Run("function tool flushes held arguments", func(t *testing.T) {
state := NewChatToResponsesStreamState("resp_1", "gpt-test")
first := mustResponsesEventsFromChatChunk(t, state, chunk(dto.ToolCallResponse{Index: &toolIndex, Type: "function", Function: dto.FunctionResponse{Arguments: `{"ms":`}}))
require.Len(t, first, 1)
assert.Equal(t, responsesEventCreated, first[0].Type)
second := mustResponsesEventsFromChatChunk(t, state, chunk(dto.ToolCallResponse{Index: &toolIndex, ID: "call_wait", Function: dto.FunctionResponse{Name: "wait", Arguments: `1}`}}))
require.Len(t, second, 2)
assert.Equal(t, responsesEventOutputItemAdded, second[0].Type)
assert.Equal(t, responsesOutputTypeFunctionCall, second[0].Payload.Item.Type)
assert.Equal(t, "call_wait", second[0].Payload.Item.ID)
assert.Equal(t, "call_wait", second[0].Payload.Item.CallId)
assert.Equal(t, responsesEventFunctionArgsDelta, second[1].Type)
assert.Equal(t, "call_wait", second[1].Payload.ItemID)
assert.Equal(t, `{"ms":1}`, second[1].Payload.Delta)
done := FinalizeChatCompletionsStreamToResponses(state)
output := done[len(done)-1].Payload.Response.Output
require.Len(t, output, 1)
assert.Equal(t, "call_wait", output[0].ID)
assert.Equal(t, `"{\"ms\":1}"`, string(output[0].Arguments))
})
t.Run("held tool takes the next output index when announced", func(t *testing.T) {
state := NewChatToResponsesStreamState("resp_1", "gpt-test")
state.Tools = execCustomToolState()
waitIndex := 1
var events []ChatToResponsesStreamEvent
events = append(events, mustResponsesEventsFromChatChunk(t, state, chunk(dto.ToolCallResponse{Index: &toolIndex, ID: "call_exec"}))...)
events = append(events, mustResponsesEventsFromChatChunk(t, state, chunk(dto.ToolCallResponse{Index: &waitIndex, ID: "call_wait", Function: dto.FunctionResponse{Name: "wait", Arguments: `{}`}}))...)
events = append(events, mustResponsesEventsFromChatChunk(t, state, &dto.ChatCompletionsStreamResponse{
Choices: []dto.ChatCompletionsStreamResponseChoice{{Index: 0, Delta: dto.ChatCompletionsStreamResponseChoiceDelta{Content: lo.ToPtr("done")}}},
})...)
events = append(events, mustResponsesEventsFromChatChunk(t, state, chunk(dto.ToolCallResponse{Index: &toolIndex, Function: dto.FunctionResponse{Name: "exec", Arguments: `{"input":"ls"}`}}))...)
events = append(events, FinalizeChatCompletionsStreamToResponses(state)...)
var addedIndexes []int
var addedTypes []string
for _, event := range events {
if event.Type == responsesEventOutputItemAdded {
addedIndexes = append(addedIndexes, lo.FromPtr(event.Payload.OutputIndex))
addedTypes = append(addedTypes, event.Payload.Item.Type)
}
}
assert.Equal(t, []int{0, 1, 2}, addedIndexes)
assert.Equal(t, []string{responsesOutputTypeFunctionCall, responsesOutputTypeMessage, responsesOutputTypeCustomToolCall}, addedTypes)
output := events[len(events)-1].Payload.Response.Output
require.Len(t, output, 3)
assert.Equal(t, addedTypes, []string{output[0].Type, output[1].Type, output[2].Type})
})
t.Run("name never arrives", func(t *testing.T) {
for _, tools := range []*convmeta.ResponsesToolState{nil, execCustomToolState()} {
state := NewChatToResponsesStreamState("resp_1", "gpt-test")
state.EmitSequenceNumber = true
state.Tools = tools
missingIndex, validIndex, blankIndex := 0, 1, 2
var events []ChatToResponsesStreamEvent
events = append(events, mustResponsesEventsFromChatChunk(t, state, &dto.ChatCompletionsStreamResponse{
Choices: []dto.ChatCompletionsStreamResponseChoice{{Index: 0, Delta: dto.ChatCompletionsStreamResponseChoiceDelta{ToolCalls: []dto.ToolCallResponse{
{Index: &missingIndex, ID: "call_missing", Type: "function", Function: dto.FunctionResponse{Arguments: `{"ok":true}`}},
{Index: &validIndex, ID: "call_valid", Type: "function", Function: dto.FunctionResponse{Name: "lookup", Arguments: `{"q":"x"}`}},
{Index: &blankIndex, ID: "call_blank", Function: dto.FunctionResponse{Name: " \t\n", Arguments: `{}`}},
}}}},
})...)
events = append(events, FinalizeChatCompletionsStreamToResponses(state)...)
for i, event := range events {
require.NotNil(t, event.Payload.SequenceNumber)
assert.Equal(t, i, *event.Payload.SequenceNumber)
if event.Payload.OutputIndex != nil {
assert.Equal(t, 0, *event.Payload.OutputIndex)
}
if event.Payload.Item != nil {
assert.Equal(t, "call_valid", event.Payload.Item.CallId)
}
}
output := events[len(events)-1].Payload.Response.Output
require.Len(t, output, 1)
assert.Equal(t, "call_valid", output[0].CallId)
assert.Equal(t, "lookup", output[0].Name)
}
})
}
func mustResponsesEventsFromChatChunk(t *testing.T, state *ChatToResponsesStreamState, chunk *dto.ChatCompletionsStreamResponse) []ChatToResponsesStreamEvent {
t.Helper()
events, err := ChatCompletionsStreamChunkToResponsesEvents(chunk, state)
relaykit/relayconvert/internal/oai_chat/to_oai_responses_stream_resp.go
+183 -40
View File
@@ -8,6 +8,7 @@ import (
"time"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
)
@@ -25,6 +26,9 @@ type ChatToResponsesStreamState struct {
// EmitSequenceNumber enables the required sequence_number field for current
// Responses API SSE consumers while preserving the legacy relaykit default.
EmitSequenceNumber bool
// Tools marks the Chat function calls that encode Responses custom tools;
// they are streamed back as custom_tool_call items.
Tools *convmeta.ResponsesToolState
status string
incompleteDetails *dto.IncompleteDetails
@@ -44,6 +48,17 @@ type ChatToResponsesStreamState struct {
text strings.Builder
annotations []any
reasoning strings.Builder
// usageText collects every text, reasoning, and tool argument delta sent
// to the client. It is the missing-usage estimate and tells whether any
// output reached the client.
usageText strings.Builder
// A mid-stream finish_reason closes the message and reasoning items. Deltas that
// arrive afterwards open a fresh item (next segment ordinal) instead of touching the
// closed one; the closed items are snapshotted so the final response keeps them.
textSegment int
reasoningSegment int
closedMessages []dto.ResponsesOutput
closedReasonings []dto.ResponsesOutput
}
type chatToResponsesStreamTool struct {
@@ -53,11 +68,16 @@ type chatToResponsesStreamTool struct {
CallID string
Name string
Arguments strings.Builder
Done bool
// Announced records that output_item.added was sent. Custom reports that
// the call restores a Responses custom tool; it is fixed at announcement.
Announced bool
Custom bool
Done bool
}
type chatToResponsesOutputRef struct {
Kind string
Segment int
ToolIndex int
HostedID string
}
@@ -386,12 +406,13 @@ func (s *ChatToResponsesStreamState) UsageText() string {
if s == nil {
return ""
}
return s.text.String()
return s.usageText.String()
}
func (s *ChatToResponsesStreamState) appendTextDelta(delta string) []ChatToResponsesStreamEvent {
events := s.startText()
s.text.WriteString(delta)
s.usageText.WriteString(delta)
events = append(events, s.event(responsesEventOutputTextDelta, dto.ResponsesStreamResponse{
Type: responsesEventOutputTextDelta,
OutputIndex: intPtr(s.textOutputIndex),
@@ -403,9 +424,16 @@ func (s *ChatToResponsesStreamState) appendTextDelta(delta string) []ChatToRespo
}
func (s *ChatToResponsesStreamState) startText() []ChatToResponsesStreamEvent {
if s.textDone {
s.textSegment++
s.textStarted = false
s.textDone = false
s.text.Reset()
s.annotations = nil
}
if !s.textStarted {
s.textStarted = true
s.textOutputIndex = s.nextIndex("message", -1)
s.textOutputIndex = s.nextIndex(chatToResponsesOutputRef{Kind: "message", Segment: s.textSegment})
return []ChatToResponsesStreamEvent{s.event(responsesEventOutputItemAdded, dto.ResponsesStreamResponse{
Type: responsesEventOutputItemAdded,
OutputIndex: intPtr(s.textOutputIndex),
@@ -447,10 +475,16 @@ func (s *ChatToResponsesStreamState) appendAnnotationDelta(raw []byte) ([]ChatTo
}
func (s *ChatToResponsesStreamState) appendReasoningDelta(delta string) []ChatToResponsesStreamEvent {
events := make([]ChatToResponsesStreamEvent, 0, 2)
events := make([]ChatToResponsesStreamEvent, 0, 3)
if s.reasoningDone {
s.reasoningSegment++
s.reasoningStarted = false
s.reasoningDone = false
s.reasoning.Reset()
}
if !s.reasoningStarted {
s.reasoningStarted = true
s.reasoningIndex = s.nextIndex("reasoning", -1)
s.reasoningIndex = s.nextIndex(chatToResponsesOutputRef{Kind: "reasoning", Segment: s.reasoningSegment})
events = append(events, s.event(responsesEventOutputItemAdded, dto.ResponsesStreamResponse{
Type: responsesEventOutputItemAdded,
OutputIndex: intPtr(s.reasoningIndex),
@@ -461,8 +495,16 @@ func (s *ChatToResponsesStreamState) appendReasoningDelta(delta string) []ChatTo
Summary: []dto.ResponsesReasoningSummaryPart{},
},
}))
events = append(events, s.event(responsesEventReasoningSummaryPartAdded, dto.ResponsesStreamResponse{
Type: responsesEventReasoningSummaryPartAdded,
OutputIndex: intPtr(s.reasoningIndex),
SummaryIndex: intPtr(0),
ItemID: s.reasoningID(),
Part: &dto.ResponsesReasoningSummaryPart{Type: "summary_text"},
}))
}
s.reasoning.WriteString(delta)
s.usageText.WriteString(delta)
events = append(events, s.event(responsesEventReasoningSummaryDelta, dto.ResponsesStreamResponse{
Type: responsesEventReasoningSummaryDelta,
OutputIndex: intPtr(s.reasoningIndex),
@@ -483,29 +525,15 @@ func (s *ChatToResponsesStreamState) appendToolCallDelta(toolCall dto.ToolCallRe
events := make([]ChatToResponsesStreamEvent, 0, 2)
if tool == nil {
tool = &chatToResponsesStreamTool{
ChatIndex: chatIndex,
OutputIndex: s.nextIndex("tool", chatIndex),
CallID: incomingID,
Name: strings.TrimSpace(toolCall.Function.Name),
ChatIndex: chatIndex,
CallID: incomingID,
Name: strings.TrimSpace(toolCall.Function.Name),
}
tool.ItemID = incomingID
if tool.ItemID == "" {
tool.ItemID = fmt.Sprintf("%s_call_%d", s.ID, chatIndex)
}
s.toolsByIndex[chatIndex] = tool
events = append(events, s.event(responsesEventOutputItemAdded, dto.ResponsesStreamResponse{
Type: responsesEventOutputItemAdded,
OutputIndex: intPtr(tool.OutputIndex),
ItemID: tool.ItemID,
Item: &dto.ResponsesOutput{
Type: responsesOutputTypeFunctionCall,
ID: tool.ItemID,
Status: "in_progress",
CallId: tool.callID(),
Name: tool.Name,
Arguments: []byte(`""`),
},
}))
}
if tool.Done {
return nil, fmt.Errorf("tool-call stream index %d received data after completion", chatIndex)
@@ -515,6 +543,9 @@ func (s *ChatToResponsesStreamState) appendToolCallDelta(toolCall dto.ToolCallRe
return nil, fmt.Errorf("tool-call stream index %d changed id from %q to %q", chatIndex, tool.CallID, incomingID)
}
tool.CallID = incomingID
if !tool.Announced {
tool.ItemID = incomingID
}
}
incomingName := strings.TrimSpace(toolCall.Function.Name)
if incomingName != "" {
@@ -523,18 +554,59 @@ func (s *ChatToResponsesStreamState) appendToolCallDelta(toolCall dto.ToolCallRe
}
tool.Name = incomingName
}
if toolCall.Function.Arguments != "" {
tool.Arguments.WriteString(toolCall.Function.Arguments)
delta := toolCall.Function.Arguments
tool.Arguments.WriteString(delta)
// A nameless call is invalid Responses output, so hold it back until a
// valid name arrives. Announcement also determines whether it is custom.
if !tool.Announced && tool.Name != "" {
events = append(events, s.announceTool(tool))
delta = tool.Arguments.String()
}
if tool.Announced {
// Announced calls reach the client; custom tool input at completion.
s.usageText.WriteString(delta)
}
// Custom tool input is unwrapped from the complete arguments at the end.
if tool.Announced && !tool.Custom && delta != "" {
events = append(events, s.event(responsesEventFunctionArgsDelta, dto.ResponsesStreamResponse{
Type: responsesEventFunctionArgsDelta,
OutputIndex: intPtr(tool.OutputIndex),
ItemID: tool.ItemID,
Delta: toolCall.Function.Arguments,
Delta: delta,
}))
}
return events, nil
}
// announceTool also allocates the output index, so a held tool never ends up
// behind items that were added to the stream before it.
func (s *ChatToResponsesStreamState) announceTool(tool *chatToResponsesStreamTool) ChatToResponsesStreamEvent {
tool.Announced = true
tool.OutputIndex = s.nextIndex(chatToResponsesOutputRef{Kind: "tool", ToolIndex: tool.ChatIndex})
tool.Custom = s.Tools.IsCustomTool(tool.Name)
namespace, name := s.Tools.ResponsesToolName(tool.Name)
item := &dto.ResponsesOutput{
Type: responsesOutputTypeFunctionCall,
ID: tool.ItemID,
Status: "in_progress",
CallId: tool.callID(),
Name: name,
Namespace: namespace,
Arguments: []byte(`""`),
}
if tool.Custom {
item.Type = responsesOutputTypeCustomToolCall
item.Arguments = nil
item.Input = []byte(`""`)
}
return s.event(responsesEventOutputItemAdded, dto.ResponsesStreamResponse{
Type: responsesEventOutputItemAdded,
OutputIndex: intPtr(tool.OutputIndex),
ItemID: tool.ItemID,
Item: item,
})
}
func (s *ChatToResponsesStreamState) doneDeltaEvents() []ChatToResponsesStreamEvent {
events := make([]ChatToResponsesStreamEvent, 0)
status := s.outputStatus()
@@ -550,11 +622,13 @@ func (s *ChatToResponsesStreamState) doneDeltaEvents() []ChatToResponsesStreamEv
textDone.Text = kitutil.GetPointer(s.text.String())
}
events = append(events, s.event("response.output_text.done", textDone))
closedMessage := s.messageOutput(status)
events = append(events, s.event(responsesEventOutputItemDone, dto.ResponsesStreamResponse{
Type: responsesEventOutputItemDone,
OutputIndex: intPtr(s.textOutputIndex),
Item: s.messageOutput(status),
Item: closedMessage,
}))
s.closedMessages = append(s.closedMessages, *closedMessage)
}
if s.reasoningStarted && !s.reasoningDone {
s.reasoningDone = true
@@ -573,27 +647,71 @@ func (s *ChatToResponsesStreamState) doneDeltaEvents() []ChatToResponsesStreamEv
reasoningDone.Part = nil
}
events = append(events, s.event(responsesEventReasoningSummaryDone, reasoningDone))
events = append(events, s.event(responsesEventReasoningSummaryPartDone, dto.ResponsesStreamResponse{
Type: responsesEventReasoningSummaryPartDone,
OutputIndex: intPtr(s.reasoningIndex),
SummaryIndex: intPtr(0),
ItemID: s.reasoningID(),
Part: &dto.ResponsesReasoningSummaryPart{
Type: "summary_text",
Text: s.reasoning.String(),
},
}))
closedReasoning := s.reasoningOutput(status)
events = append(events, s.event(responsesEventOutputItemDone, dto.ResponsesStreamResponse{
Type: responsesEventOutputItemDone,
OutputIndex: intPtr(s.reasoningIndex),
Item: s.reasoningOutput(status),
Item: closedReasoning,
}))
s.closedReasonings = append(s.closedReasonings, *closedReasoning)
}
for _, tool := range s.sortedTools() {
if tool.Done {
continue
}
tool.Done = true
argumentsDone := dto.ResponsesStreamResponse{
Type: responsesEventFunctionArgsDone,
OutputIndex: intPtr(tool.OutputIndex),
ItemID: tool.ItemID,
if !tool.Announced {
if tool.Name == "" {
continue
}
events = append(events, s.announceTool(tool))
if !tool.Custom && tool.Arguments.Len() > 0 {
events = append(events, s.event(responsesEventFunctionArgsDelta, dto.ResponsesStreamResponse{
Type: responsesEventFunctionArgsDelta,
OutputIndex: intPtr(tool.OutputIndex),
ItemID: tool.ItemID,
Delta: tool.Arguments.String(),
}))
}
}
if s.EmitSequenceNumber {
argumentsDone.Arguments = kitutil.GetPointer(tool.Arguments.String())
argumentsDone.Name = tool.Name
if tool.Custom {
input := customToolInputFromArguments(tool.Arguments.String())
if input != "" {
events = append(events, s.event(responsesEventCustomToolInputDelta, dto.ResponsesStreamResponse{
Type: responsesEventCustomToolInputDelta,
OutputIndex: intPtr(tool.OutputIndex),
ItemID: tool.ItemID,
Delta: input,
}))
}
events = append(events, s.event(responsesEventCustomToolInputDone, dto.ResponsesStreamResponse{
Type: responsesEventCustomToolInputDone,
OutputIndex: intPtr(tool.OutputIndex),
ItemID: tool.ItemID,
Input: kitutil.GetPointer(input),
}))
} else {
argumentsDone := dto.ResponsesStreamResponse{
Type: responsesEventFunctionArgsDone,
OutputIndex: intPtr(tool.OutputIndex),
ItemID: tool.ItemID,
}
if s.EmitSequenceNumber {
argumentsDone.Arguments = kitutil.GetPointer(tool.Arguments.String())
_, argumentsDone.Name = s.Tools.ResponsesToolName(tool.Name)
}
events = append(events, s.event(responsesEventFunctionArgsDone, argumentsDone))
}
events = append(events, s.event(responsesEventFunctionArgsDone, argumentsDone))
events = append(events, s.event(responsesEventOutputItemDone, dto.ResponsesStreamResponse{
Type: responsesEventOutputItemDone,
OutputIndex: intPtr(tool.OutputIndex),
@@ -649,8 +767,20 @@ func (s *ChatToResponsesStreamState) finalResponse() *dto.OpenAIResponsesRespons
for _, ref := range s.outputOrder {
switch ref.Kind {
case "message":
if ref.Segment < len(s.closedMessages) {
closed := s.closedMessages[ref.Segment]
closed.Status = status
output = append(output, closed)
continue
}
output = append(output, *s.messageOutput(status))
case "reasoning":
if ref.Segment < len(s.closedReasonings) {
closed := s.closedReasonings[ref.Segment]
closed.Status = status
output = append(output, closed)
continue
}
output = append(output, *s.reasoningOutput(status))
case "tool":
if tool := s.toolsByIndex[ref.ToolIndex]; tool != nil {
@@ -685,10 +815,10 @@ func (s *ChatToResponsesStreamState) createdResponse() *dto.OpenAIResponsesRespo
}
}
func (s *ChatToResponsesStreamState) nextIndex(kind string, toolIndex int) int {
func (s *ChatToResponsesStreamState) nextIndex(ref chatToResponsesOutputRef) int {
index := s.nextOutputIndex
s.nextOutputIndex++
s.outputOrder = append(s.outputOrder, chatToResponsesOutputRef{Kind: kind, ToolIndex: toolIndex})
s.outputOrder = append(s.outputOrder, ref)
return index
}
@@ -720,11 +850,11 @@ func (s *ChatToResponsesStreamState) outputStatus() string {
}
func (s *ChatToResponsesStreamState) messageID() string {
return fmt.Sprintf("%s_msg_0", s.ID)
return fmt.Sprintf("%s_msg_%d", s.ID, s.textSegment)
}
func (s *ChatToResponsesStreamState) reasoningID() string {
return fmt.Sprintf("%s_reasoning_0", s.ID)
return fmt.Sprintf("%s_reasoning_%d", s.ID, s.reasoningSegment)
}
func (s *ChatToResponsesStreamState) messageOutput(status string) *dto.ResponsesOutput {
@@ -762,12 +892,25 @@ func (s *ChatToResponsesStreamState) reasoningOutput(status string) *dto.Respons
}
func (s *ChatToResponsesStreamState) toolOutput(tool *chatToResponsesStreamTool, status string) *dto.ResponsesOutput {
namespace, name := s.Tools.ResponsesToolName(tool.Name)
if tool.Custom {
return &dto.ResponsesOutput{
Type: responsesOutputTypeCustomToolCall,
ID: tool.ItemID,
Status: status,
CallId: tool.callID(),
Name: name,
Namespace: namespace,
Input: chatArgumentsRawMessage(customToolInputFromArguments(tool.Arguments.String())),
}
}
return &dto.ResponsesOutput{
Type: responsesOutputTypeFunctionCall,
ID: tool.ItemID,
Status: status,
CallId: tool.callID(),
Name: tool.Name,
Name: name,
Namespace: namespace,
Arguments: chatArgumentsRawMessage(tool.Arguments.String()),
}
}
relaykit/relayconvert/internal/oai_responses/req_helpers.go
+25
View File
@@ -1,10 +1,12 @@
package oairesponses
import (
"context"
"fmt"
"strings"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/internal/convdiag"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/types"
)
@@ -58,6 +60,29 @@ func InputItems(raw []byte) ([]map[string]any, error) {
return responsesInputItems(raw)
}
// omitTrailingAssistantText drops the assistant turns without tool calls that
// end the converted conversation after a user turn or tool result. In
// Responses they are history and the model answers with a new item; Claude and
// Gemini would continue them as a prefill, which newer models reject with a
// 400. Codex leaves them at the end of input when it retries a stream that
// failed after an assistant message completed.
func omitTrailingAssistantText[T any](c context.Context, turns []T, isUser, isAssistantText func(T) bool) []T {
end := len(turns)
for end > 0 && isAssistantText(turns[end-1]) {
end--
}
if end == len(turns) || end == 0 || !isUser(turns[end-1]) {
return turns
}
convdiag.Add(c, types.ConversionDiagnostic{
Code: "trailing_assistant_omitted",
Path: "input",
Message: "assistant output after the last user message or tool result was omitted; the target protocol would continue it as a prefill",
Severity: types.ConversionDiagnosticWarning,
})
return turns[:end]
}
func responsesContentParts(content any) ([]map[string]any, error) {
switch typed := content.(type) {
case nil:
relaykit/relayconvert/internal/oai_responses/to_claude_messages_req.go
+85 -32
View File
@@ -3,6 +3,7 @@ package oairesponses
import (
"context"
"fmt"
"slices"
"strings"
"github.com/QuantumNous/new-api/relaykit/dto"
@@ -12,6 +13,7 @@ import (
sharedclaude "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/shared/claude"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/relayconvert/reasoning"
"github.com/QuantumNous/new-api/relaykit/types"
)
func convertOpenAIResponsesRequestToClaudeMessages(c context.Context, info convmeta.Meta, request any) (any, error) {
@@ -94,11 +96,26 @@ func OpenAIResponsesRequestToClaudeMessages(c context.Context, info convmeta.Met
itemType := strings.TrimSpace(kitutil.Interface2String(item["type"]))
switch itemType {
case ResponsesInputTypeFunctionCall:
claudeRequest.Messages = appendClaudeToolUse(claudeRequest.Messages, responsesFunctionCallItemToClaudeToolUse(item, "arguments"))
claudeRequest.Messages = appendClaudeToolUse(claudeRequest.Messages, responsesFunctionCallItemToClaudeToolUse(item))
case ResponsesInputTypeCustomToolCall:
claudeRequest.Messages = appendClaudeToolUse(claudeRequest.Messages, responsesFunctionCallItemToClaudeToolUse(item, "input"))
name := responsesCallName(item)
if name == "" {
return nil, fmt.Errorf("custom_tool_call item is missing name")
}
// The custom tool is declared as a function taking one string
// argument, so its raw input is replayed in that shape.
claudeRequest.Messages = appendClaudeToolUse(claudeRequest.Messages, dto.ClaudeMediaMessage{
Type: "tool_use",
Id: CallID(item),
Name: name,
Input: map[string]any{convmeta.CustomToolInputArgument: responsesArgumentsString(item["input"])},
})
case ResponsesInputTypeFunctionCallOutput, ResponsesInputTypeCustomToolOutput:
claudeRequest.Messages = appendClaudeToolResult(claudeRequest.Messages, responsesFunctionOutputItemToClaudeToolResult(item))
claudeRequest.Messages = appendClaudeToolResult(claudeRequest.Messages, dto.ClaudeMediaMessage{
Type: "tool_result",
ToolUseId: CallID(item),
Content: responsesToolOutputToClaude(c, item["output"]),
})
default:
sourceRole := strings.TrimSpace(kitutil.Interface2String(item["role"]))
role := responsesClaudeRole(sourceRole)
@@ -131,6 +148,11 @@ func OpenAIResponsesRequestToClaudeMessages(c context.Context, info convmeta.Met
})
}
}
claudeRequest.Messages = omitTrailingAssistantText(c, claudeRequest.Messages,
func(message dto.ClaudeMessage) bool { return message.Role == "user" },
func(message dto.ClaudeMessage) bool {
return message.Role == "assistant" && !slices.ContainsFunc(claudeMessageContentParts(message.Content), func(part dto.ClaudeMediaMessage) bool { return part.Type == "tool_use" })
})
if len(systemMessages) > 0 {
claudeRequest.System = systemMessages
@@ -181,50 +203,81 @@ func responsesInputContentToClaudeMediaMessages(c context.Context, content any)
if source == nil {
continue
}
base64Data, mimeType, err := relaymedia.ResolveBase64Data(c, source, "formatting Responses input for Claude")
if err != nil {
return nil, fmt.Errorf("get file data failed: %s", err.Error())
// Claude content blocks carry images, PDF documents, and text
// documents; audio and video have no Claude block.
reason := fmt.Sprintf("Claude Messages cannot carry %s content", partType)
var block dto.ClaudeMediaMessage
if partType == "input_image" || partType == "input_file" {
base64Data, mimeType, err := relaymedia.ResolveBase64Data(c, source, "formatting Responses input for Claude")
if err != nil {
return nil, fmt.Errorf("get file data failed: %s", err.Error())
}
block, reason = sharedclaude.MediaBlock(base64Data, mimeType, partType == "input_image")
}
claudePart := dto.ClaudeMediaMessage{
Source: &dto.ClaudeMessageSource{
Type: "base64",
MediaType: mimeType,
Data: base64Data,
},
if reason != "" {
convdiag.Add(c, types.ConversionDiagnostic{
Code: "unsupported_media_type",
Path: "input.content",
Message: reason + "; the part was omitted",
Severity: types.ConversionDiagnosticError,
})
continue
}
if strings.HasPrefix(mimeType, "application/pdf") {
claudePart.Type = "document"
} else {
claudePart.Type = "image"
}
parts = append(parts, claudePart)
parts = append(parts, block)
}
}
return parts, nil
}
func responsesFunctionCallItemToClaudeToolUse(item map[string]any, inputKey string) dto.ClaudeMediaMessage {
func responsesFunctionCallItemToClaudeToolUse(item map[string]any) dto.ClaudeMediaMessage {
return dto.ClaudeMediaMessage{
Type: "tool_use",
Id: CallID(item),
Name: strings.TrimSpace(kitutil.Interface2String(item["name"])),
Input: ObjectValue(item[inputKey], inputKey),
Name: responsesCallName(item),
Input: ObjectValue(item["arguments"], "arguments"),
}
}
func responsesFunctionOutputItemToClaudeToolResult(item map[string]any) dto.ClaudeMediaMessage {
return dto.ClaudeMediaMessage{
Type: "tool_result",
ToolUseId: CallID(item),
Content: responsesToolOutputValue(item["output"]),
// responsesToolOutputToClaude maps a function_call_output payload onto Claude
// tool_result content. A Responses content-part array becomes the text, image,
// and document blocks tool_result accepts; stringifying it instead would hand
// base64 media to the upstream text tokenizer. An array holding any other part
// type, or media that cannot be resolved, keeps the stringified form, and other
// payload shapes pass through unchanged.
func responsesToolOutputToClaude(c context.Context, value any) any {
rawParts, ok := value.([]any)
if !ok || len(rawParts) == 0 {
if value == nil {
return ""
}
return value
}
}
func responsesToolOutputValue(value any) any {
if value == nil {
return ""
labels := make([]string, 0, len(rawParts))
for _, rawPart := range rawParts {
part, isMap := rawPart.(map[string]any)
if !isMap {
return responseToolOutputToChatContent(value)
}
switch partType := strings.TrimSpace(kitutil.Interface2String(part["type"])); partType {
case "input_text", "output_text", "text":
case "input_image", "input_file", "input_audio", "input_video":
if label := "[" + strings.TrimPrefix(partType, "input_") + "]"; !slices.Contains(labels, label) {
labels = append(labels, label)
}
default:
return responseToolOutputToChatContent(value)
}
}
return value
blocks, err := responsesInputContentToClaudeMediaMessages(c, rawParts)
if err != nil {
return responseToolOutputToChatContent(value)
}
if len(blocks) == 0 {
// Every media part was omitted with a diagnostic; name what was there
// instead of sending the payload as text.
return strings.Join(labels, " ")
}
return blocks
}
func appendClaudeToolUse(messages []dto.ClaudeMessage, toolUse dto.ClaudeMediaMessage) []dto.ClaudeMessage {
relaykit/relayconvert/internal/oai_responses/to_gemini_chat_req.go
+149 -42
View File
@@ -2,6 +2,7 @@ package oairesponses
import (
"fmt"
"slices"
"strings"
"context"
@@ -13,21 +14,9 @@ import (
sharedgemini "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/shared/gemini"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/relayconvert/reasoning"
"github.com/QuantumNous/new-api/relaykit/types"
)
func convertOpenAIResponsesRequestToGeminiChat(c context.Context, info convmeta.Meta, request any) (any, error) {
responsesRequest, err := OpenAIResponsesRequestFromAny(request)
if err != nil {
return nil, err
}
prepared, err := PrepareOpenAIResponsesRequest(*responsesRequest)
if err != nil {
return nil, err
}
return OpenAIResponsesRequestToGeminiChat(c, &prepared, info)
}
func OpenAIResponsesRequestToGeminiChat(c context.Context, req *dto.OpenAIResponsesRequest, info convmeta.Meta) (*dto.GeminiChatRequest, error) {
opts := convmeta.OptionsOf(info)
if req == nil {
@@ -59,7 +48,7 @@ func OpenAIResponsesRequestToGeminiChat(c context.Context, req *dto.OpenAIRespon
if opts.Gemini.SupportsImagineModel(upstreamModelName) {
geminiRequest.GenerationConfig.ResponseModalities = []string{"TEXT", "IMAGE"}
}
if err := applyResponsesTextToGemini(req.Text, geminiRequest); err != nil {
if err := applyResponsesTextToGemini(c, req.Text, geminiRequest, opts.Gemini.SupportsJSONSchemaModel(upstreamModelName)); err != nil {
return nil, err
}
reasoningIntent, diagnostics, err := reasoning.FromOpenAIResponses(req)
@@ -134,12 +123,17 @@ func OpenAIResponsesRequestToGeminiChat(c context.Context, req *dto.OpenAIRespon
if err != nil {
return nil, err
}
// Multimodal function responses (media in functionResponse.parts) are
// documented for Gemini 3. Gemini 1.x and 2.x get tool media as ordinary
// inlineData parts after the function responses of the same turn.
modelID := strings.TrimPrefix(strings.ToLower(upstreamModelName), "models/")
toolMediaInFunctionResponse := !strings.HasPrefix(modelID, "gemini-1.") && !strings.HasPrefix(modelID, "gemini-2.")
callNames := make(map[string]string)
for _, item := range inputItems {
itemType := strings.TrimSpace(kitutil.Interface2String(item["type"]))
switch itemType {
case ResponsesInputTypeFunctionCall:
part, callID, err := responsesFunctionCallItemToGeminiPart(item)
case ResponsesInputTypeFunctionCall, ResponsesInputTypeCustomToolCall:
part, callID, err := responsesFunctionCallItemToGeminiPart(item, itemType)
if err != nil {
return nil, err
}
@@ -148,12 +142,14 @@ func OpenAIResponsesRequestToGeminiChat(c context.Context, req *dto.OpenAIRespon
callNames[callID] = part.FunctionCall.FunctionName
}
appendGeminiContentPart(geminiRequest, "model", part)
case ResponsesInputTypeFunctionCallOutput:
part, err := responsesFunctionOutputItemToGeminiPart(item, callNames)
case ResponsesInputTypeFunctionCallOutput, ResponsesInputTypeCustomToolOutput:
part, media, err := responsesFunctionOutputItemToGeminiPart(c, item, callNames, toolMediaInFunctionResponse)
if err != nil {
return nil, err
}
appendGeminiContentPart(geminiRequest, "user", part)
turn := &geminiRequest.Contents[len(geminiRequest.Contents)-1]
turn.Parts = append(turn.Parts, media...)
default:
role := responsesGeminiRole(item)
parts, err := responsesInputContentToGeminiParts(c, item["content"])
@@ -176,6 +172,11 @@ func OpenAIResponsesRequestToGeminiChat(c context.Context, req *dto.OpenAIRespon
}
}
}
geminiRequest.Contents = omitTrailingAssistantText(c, geminiRequest.Contents,
func(content dto.GeminiChatContent) bool { return content.Role == "user" },
func(content dto.GeminiChatContent) bool {
return content.Role == "model" && !slices.ContainsFunc(content.Parts, func(part dto.GeminiPart) bool { return part.FunctionCall != nil })
})
if len(systemTexts) > 0 {
geminiRequest.SystemInstructions = &dto.GeminiChatContent{
@@ -186,7 +187,7 @@ func OpenAIResponsesRequestToGeminiChat(c context.Context, req *dto.OpenAIRespon
return geminiRequest, nil
}
func applyResponsesTextToGemini(raw []byte, geminiRequest *dto.GeminiChatRequest) error {
func applyResponsesTextToGemini(c context.Context, raw []byte, geminiRequest *dto.GeminiChatRequest, jsonSchema bool) error {
responseFormat, err := RequestTextToChatResponseFormat(raw)
if err != nil {
return err
@@ -200,11 +201,7 @@ func applyResponsesTextToGemini(raw []byte, geminiRequest *dto.GeminiChatRequest
return nil
}
var jsonSchema dto.FormatJsonSchema
if err := kitutil.Unmarshal(responseFormat.JsonSchema, &jsonSchema); err != nil {
return nil
}
geminiRequest.GenerationConfig.ResponseSchema = sharedgemini.RemoveAdditionalProperties(jsonSchema.Schema, 0)
sharedgemini.ApplyResponseSchema(c, &geminiRequest.GenerationConfig, responseFormat.JsonSchema, "text.format.schema", jsonSchema)
return nil
}
@@ -243,7 +240,7 @@ func responsesContentPartToGeminiParts(c context.Context, part map[string]any) (
if err != nil {
return nil, fmt.Errorf("get file data from '%s' failed: %w", source.GetIdentifier(), err)
}
if _, ok := sharedgemini.SupportedMimeTypes[strings.ToLower(mimeType)]; !ok {
if !sharedgemini.IsSupportedMimeType(mimeType) {
return nil, fmt.Errorf("mime type is not supported by Gemini: '%s', url: '%s', supported types are: %v", mimeType, source.GetIdentifier(), sharedgemini.SupportedMimeTypesList())
}
return []dto.GeminiPart{
@@ -259,58 +256,168 @@ func responsesContentPartToGeminiParts(c context.Context, part map[string]any) (
}
}
func responsesFunctionCallItemToGeminiPart(item map[string]any) (dto.GeminiPart, string, error) {
name := strings.TrimSpace(kitutil.Interface2String(item["name"]))
func responsesFunctionCallItemToGeminiPart(item map[string]any, itemType string) (dto.GeminiPart, string, error) {
name := responsesCallName(item)
if name == "" {
return dto.GeminiPart{}, "", fmt.Errorf("function_call item is missing name")
return dto.GeminiPart{}, "", fmt.Errorf("%s item is missing name", itemType)
}
var arguments map[string]any
if itemType == ResponsesInputTypeCustomToolCall {
// The custom tool is declared as a function taking one string
// argument, so its raw input is replayed in that shape.
arguments = map[string]any{convmeta.CustomToolInputArgument: responsesArgumentsString(item["input"])}
} else {
arguments = ObjectValue(item["arguments"], "arguments")
}
callID := CallID(item)
return dto.GeminiPart{
FunctionCall: &dto.FunctionCall{
ID: callID,
FunctionName: name,
Arguments: ObjectValue(item["arguments"], "arguments"),
Arguments: arguments,
},
}, callID, nil
}
func responsesFunctionOutputItemToGeminiPart(item map[string]any, callNames map[string]string) (dto.GeminiPart, error) {
// responsesFunctionOutputItemToGeminiPart returns the functionResponse part
// and, unless mediaInParts puts them into functionResponse.parts, the tool
// media the caller sends as separate parts after the turn's function
// responses.
func responsesFunctionOutputItemToGeminiPart(c context.Context, item map[string]any, callNames map[string]string, mediaInParts bool) (dto.GeminiPart, []dto.GeminiPart, error) {
callID := CallID(item)
name := strings.TrimSpace(kitutil.Interface2String(item["name"]))
if name == "" {
name = callNames[callID]
}
output, media := responsesToolOutputToGemini(c, item["output"], mediaInParts)
response := &dto.GeminiFunctionResponse{
Name: name,
Response: GeminiResponseMap(item["output"]),
Response: output,
}
if mediaInParts && len(media) > 0 {
parts, err := kitutil.Marshal(media)
if err != nil {
return dto.GeminiPart{}, nil, fmt.Errorf("failed to marshal function response parts: %w", err)
}
response.Parts = parts
media = nil
}
if callID != "" {
id, err := kitutil.Marshal(callID)
if err != nil {
return dto.GeminiPart{}, fmt.Errorf("failed to marshal function response ID: %w", err)
return dto.GeminiPart{}, nil, fmt.Errorf("failed to marshal function response ID: %w", err)
}
response.ID = id
}
return dto.GeminiPart{
FunctionResponse: response,
}, nil
}, media, nil
}
// responsesToolOutputToGemini maps a function_call_output payload onto a Gemini
// functionResponse. A Responses content-part array keeps its text in response
// and returns its media as inlineData parts; stringifying them instead would
// hand base64 media to the upstream text tokenizer. Multimodal function
// responses (mediaInParts) carry images and documents only; separate parts
// carry every type Gemini accepts. Media that cannot be loaded or carried is
// omitted with a diagnostic. An array holding any other part type, and every
// other payload shape, keeps the historical response map.
func responsesToolOutputToGemini(c context.Context, value any, mediaInParts bool) (map[string]any, []dto.GeminiPart) {
rawParts, ok := value.([]any)
if !ok || len(rawParts) == 0 {
return GeminiResponseMap(value), nil
}
parts := make([]map[string]any, 0, len(rawParts))
for _, rawPart := range rawParts {
part, isMap := rawPart.(map[string]any)
if !isMap {
return GeminiResponseMap(value), nil
}
switch strings.TrimSpace(kitutil.Interface2String(part["type"])) {
case "input_text", "output_text", "text", "input_image", "input_file", "input_audio", "input_video":
default:
return GeminiResponseMap(value), nil
}
parts = append(parts, part)
}
texts := make([]string, 0, len(parts))
labels := make([]string, 0, len(parts))
media := make([]dto.GeminiPart, 0, len(parts))
for _, part := range parts {
partType := strings.TrimSpace(kitutil.Interface2String(part["type"]))
switch partType {
case "input_text", "output_text", "text":
if text := kitutil.Interface2String(part["text"]); text != "" {
texts = append(texts, text)
}
continue
}
kind := strings.TrimPrefix(partType, "input_")
if label := "[" + kind + "]"; !slices.Contains(labels, label) {
labels = append(labels, label)
}
source := ContentPartToFileSource(part)
if source == nil {
continue
}
// The documented MIME list for multimodal function responses has no
// audio or video, and Gemini 3 ignores audio sent there.
reason := fmt.Sprintf("Gemini function responses cannot carry %s content", kind)
if !mediaInParts || kind == "image" || kind == "file" {
base64Data, mimeType, err := relaymedia.ResolveBase64Data(c, source, "formatting Responses tool output for Gemini")
topLevelType, _, _ := strings.Cut(strings.ToLower(mimeType), "/")
switch {
case err != nil:
reason = fmt.Sprintf("the %s could not be loaded", kind)
case sharedgemini.IsSupportedMimeType(mimeType) && (!mediaInParts || topLevelType != "audio" && topLevelType != "video"):
media = append(media, dto.GeminiPart{InlineData: &dto.GeminiInlineData{MimeType: mimeType, Data: base64Data}})
continue
default:
reason = fmt.Sprintf("Gemini function responses cannot carry %s content", mimeType)
}
}
convdiag.Add(c, types.ConversionDiagnostic{
Code: "unsupported_media_type",
Path: "input.output",
Message: reason + "; the part was omitted",
Severity: types.ConversionDiagnosticError,
})
}
switch {
case len(texts) > 0:
return GeminiResponseMap(strings.Join(texts, "\n")), media
case len(media) > 0:
return map[string]any{}, media
default:
// Every media part was omitted; name what was there instead of
// sending the payload as text.
return GeminiResponseMap(strings.Join(labels, " ")), nil
}
}
func appendGeminiContentPart(req *dto.GeminiChatRequest, role string, part dto.GeminiPart) {
if len(req.Contents) > 0 && req.Contents[len(req.Contents)-1].Role == role {
if role == "model" && part.FunctionCall != nil {
parts := req.Contents[len(req.Contents)-1].Parts
insertAt := 0
parts := req.Contents[len(req.Contents)-1].Parts
insertAt := len(parts)
switch {
case role == "model" && part.FunctionCall != nil:
insertAt = 0
for insertAt < len(parts) && parts[insertAt].FunctionCall != nil {
insertAt++
}
parts = append(parts, dto.GeminiPart{})
copy(parts[insertAt+1:], parts[insertAt:])
parts[insertAt] = part
req.Contents[len(req.Contents)-1].Parts = parts
return
case part.FunctionResponse != nil:
// Tool media sent as separate parts stay after every function
// response of the turn, so a later response goes before them.
for i := len(parts) - 1; i >= 0; i-- {
if parts[i].FunctionResponse != nil {
insertAt = i + 1
break
}
}
}
req.Contents[len(req.Contents)-1].Parts = append(req.Contents[len(req.Contents)-1].Parts, part)
req.Contents[len(req.Contents)-1].Parts = slices.Insert(parts, insertAt, part)
return
}
req.Contents = append(req.Contents, dto.GeminiChatContent{
relaykit/relayconvert/internal/oai_responses/to_gemini_chat_req_preprocess.go
-101
View File
@@ -1,101 +0,0 @@
package oairesponses
import (
"strings"
"github.com/QuantumNous/new-api/relaykit/dto"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
)
const (
geminiResponsesInputTypeCustomToolCall = "custom_tool_call"
geminiResponsesInputTypeCustomToolCallOutput = "custom_tool_call_output"
geminiResponsesInputTypeFunctionCallOutput = "function_call_output"
)
const (
ResponsesInputTypeCustomToolCallOutput = geminiResponsesInputTypeCustomToolCallOutput
)
func PrepareOpenAIResponsesRequest(request dto.OpenAIResponsesRequest) (dto.OpenAIResponsesRequest, error) {
tools, err := filterGeminiResponsesTools(request.Tools)
if err != nil {
return request, err
}
request.Tools = tools
input, err := filterGeminiResponsesInput(request.Input)
if err != nil {
return request, err
}
request.Input = input
return request, nil
}
func filterGeminiResponsesTools(raw []byte) ([]byte, error) {
if !geminiRawJSONPresent(raw) || kitutil.GetJsonType(raw) != "array" {
return raw, nil
}
var tools []map[string]any
if err := kitutil.Unmarshal(raw, &tools); err != nil {
return nil, err
}
filtered := make([]map[string]any, 0, len(tools))
for _, tool := range tools {
if strings.TrimSpace(kitutil.Interface2String(tool["type"])) != "function" {
continue
}
filtered = append(filtered, tool)
}
if len(filtered) == 0 {
return nil, nil
}
return kitutil.Marshal(filtered)
}
func filterGeminiResponsesInput(raw []byte) ([]byte, error) {
if !geminiRawJSONPresent(raw) || kitutil.GetJsonType(raw) != "array" {
return raw, nil
}
var items []map[string]any
if err := kitutil.Unmarshal(raw, &items); err != nil {
return nil, err
}
skippedCustomCallIDs := make(map[string]struct{})
for _, item := range items {
if strings.TrimSpace(kitutil.Interface2String(item["type"])) != geminiResponsesInputTypeCustomToolCall {
continue
}
if callID := strings.TrimSpace(kitutil.Interface2String(item["call_id"])); callID != "" {
skippedCustomCallIDs[callID] = struct{}{}
}
}
filtered := make([]map[string]any, 0, len(items))
for _, item := range items {
itemType := strings.TrimSpace(kitutil.Interface2String(item["type"]))
switch itemType {
case geminiResponsesInputTypeCustomToolCall, geminiResponsesInputTypeCustomToolCallOutput:
continue
case geminiResponsesInputTypeFunctionCallOutput:
if _, ok := skippedCustomCallIDs[strings.TrimSpace(kitutil.Interface2String(item["call_id"]))]; ok {
continue
}
}
filtered = append(filtered, item)
}
return kitutil.Marshal(filtered)
}
func geminiRawJSONPresent(raw []byte) bool {
if len(raw) == 0 {
return false
}
return kitutil.GetJsonType(raw) != "null"
}
relaykit/relayconvert/internal/oai_responses/to_oai_chat_req.go
+180 -39
View File
@@ -5,12 +5,16 @@ import (
"encoding/json"
"errors"
"fmt"
"slices"
"strings"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
"github.com/QuantumNous/new-api/relaykit/relayconvert/internal/convdiag"
relaymedia "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/media"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/relayconvert/reasoning"
"github.com/QuantumNous/new-api/relaykit/types"
)
const (
@@ -38,7 +42,7 @@ func ResponsesRequestToChatCompletionsRequest(ctx context.Context, req *dto.Open
return nil, err
}
messages, err := responsesRequestMessagesToChat(req)
messages, err := responsesRequestMessagesToChat(ctx, req)
if err != nil {
return nil, err
}
@@ -139,7 +143,7 @@ func ValidateRequestChatUnsupportedFields(req *dto.OpenAIResponsesRequest) error
return validateResponsesRequestChatUnsupportedFields(req)
}
func responsesRequestMessagesToChat(req *dto.OpenAIResponsesRequest) ([]dto.Message, error) {
func responsesRequestMessagesToChat(ctx context.Context, req *dto.OpenAIResponsesRequest) ([]dto.Message, error) {
messages := make([]dto.Message, 0)
if rawJSONPresent(req.Instructions) {
instructions, err := responsesJSONString(req.Instructions)
@@ -168,12 +172,25 @@ func responsesRequestMessagesToChat(req *dto.OpenAIResponsesRequest) ([]dto.Mess
if err := kitutil.Unmarshal(req.Input, &items); err != nil {
return nil, fmt.Errorf("invalid input array: %w", err)
}
// Chat Completions requires the tool messages answering one assistant tool_calls
// batch to stay contiguous, so media hoisted out of tool output items is
// held back and emitted as a single user message once the batch ends.
var pendingMedia []any
for _, item := range items {
nextMessages, err := responsesInputItemToChatMessages(item, messages)
itemType := strings.TrimSpace(kitutil.Interface2String(item["type"]))
if len(pendingMedia) > 0 && itemType != responsesInputTypeFunctionCallOutput && itemType != responsesInputTypeCustomToolOutput {
messages = append(messages, dto.Message{Role: "user", Content: pendingMedia})
pendingMedia = nil
}
nextMessages, media, err := responsesInputItemToChatMessages(ctx, item, messages)
if err != nil {
return nil, err
}
messages = nextMessages
pendingMedia = append(pendingMedia, media...)
}
if len(pendingMedia) > 0 {
messages = append(messages, dto.Message{Role: "user", Content: pendingMedia})
}
return messages, nil
default:
@@ -181,39 +198,42 @@ func responsesRequestMessagesToChat(req *dto.OpenAIResponsesRequest) ([]dto.Mess
}
}
func responsesInputItemToChatMessages(item map[string]any, messages []dto.Message) ([]dto.Message, error) {
// responsesInputItemToChatMessages appends the Chat messages for one Responses input item.
// The second result carries media content parts hoisted out of a tool output item,
// already in Chat shape; the caller decides where that user message lands.
func responsesInputItemToChatMessages(ctx context.Context, item map[string]any, messages []dto.Message) ([]dto.Message, []any, error) {
itemType := strings.TrimSpace(kitutil.Interface2String(item["type"]))
switch itemType {
case responsesInputTypeFunctionCall:
toolCall, err := responsesFunctionCallItemToChatToolCall(item)
if err != nil {
return nil, err
return nil, nil, err
}
return appendToolCallToLastAssistant(messages, toolCall), nil
return appendToolCallToLastAssistant(messages, toolCall), nil, nil
case responsesInputTypeCustomToolCall:
toolCall, err := responsesCustomToolCallItemToChatToolCall(item)
if err != nil {
return nil, err
return nil, nil, err
}
return appendToolCallToLastAssistant(messages, toolCall), nil
case responsesInputTypeFunctionCallOutput:
return appendToolCallToLastAssistant(messages, toolCall), nil, nil
case responsesInputTypeFunctionCallOutput, responsesInputTypeCustomToolOutput:
callID := strings.TrimSpace(kitutil.Interface2String(item["call_id"]))
content := responseToolOutputToChatContent(item["output"])
return append(messages, dto.Message{Role: "tool", ToolCallId: callID, Content: content}), nil
content, media := responsesToolOutputToChat(ctx, item["output"])
return append(messages, dto.Message{Role: "tool", ToolCallId: callID, Content: content}), media, nil
}
role := strings.TrimSpace(kitutil.Interface2String(item["role"]))
if role == "" {
role = "user"
}
content, err := responsesInputContentToChatContent(item["content"])
content, err := responsesInputContentToChatContent(ctx, item["content"])
if err != nil {
return nil, err
return nil, nil, err
}
return append(messages, dto.Message{Role: role, Content: content}), nil
return append(messages, dto.Message{Role: role, Content: content}), nil, nil
}
func responsesInputContentToChatContent(content any) (any, error) {
func responsesInputContentToChatContent(ctx context.Context, content any) (any, error) {
if content == nil {
return "", nil
}
@@ -222,22 +242,23 @@ func responsesInputContentToChatContent(content any) (any, error) {
case string:
return value, nil
case []any:
return responsesContentPartsToChatContent(value)
return responsesContentPartsToChatContent(ctx, value)
case []map[string]any:
parts := make([]any, 0, len(value))
for _, part := range value {
parts = append(parts, part)
}
return responsesContentPartsToChatContent(parts)
return responsesContentPartsToChatContent(ctx, parts)
default:
return content, nil
}
}
func responsesContentPartsToChatContent(parts []any) (any, error) {
func responsesContentPartsToChatContent(ctx context.Context, parts []any) (any, error) {
chatParts := make([]any, 0, len(parts))
var textOnly strings.Builder
onlyText := true
droppedFile := false
for _, rawPart := range parts {
part, ok := rawPart.(map[string]any)
@@ -264,10 +285,18 @@ func responsesContentPartsToChatContent(parts []any) (any, error) {
})
case "input_file":
onlyText = false
chatParts = append(chatParts, map[string]any{
"type": dto.ContentTypeFile,
"file": responsesFilePartToChatFile(part),
})
chatPart, reason := responsesFilePartToChat(ctx, part)
if reason != "" {
convdiag.Add(ctx, types.ConversionDiagnostic{
Code: "unsupported_media_type",
Path: "input.content",
Message: reason + "; the file was omitted",
Severity: types.ConversionDiagnosticError,
})
droppedFile = true
continue
}
chatParts = append(chatParts, chatPart)
case "input_audio":
onlyText = false
chatParts = append(chatParts, map[string]any{
@@ -289,11 +318,15 @@ func responsesContentPartsToChatContent(parts []any) (any, error) {
if onlyText {
return textOnly.String(), nil
}
if len(chatParts) == 0 && droppedFile {
// Chat rejects an empty content array; name what was omitted instead.
return "[file]", nil
}
return chatParts, nil
}
func responsesFunctionCallItemToChatToolCall(item map[string]any) (dto.ToolCallRequest, error) {
name := strings.TrimSpace(kitutil.Interface2String(item["name"]))
name := responsesCallName(item)
if name == "" {
return dto.ToolCallRequest{}, errors.New("function_call item is missing name")
}
@@ -307,18 +340,26 @@ func responsesFunctionCallItemToChatToolCall(item map[string]any) (dto.ToolCallR
}, nil
}
// responsesCustomToolCallItemToChatToolCall encodes a custom tool call the same
// way its definition reaches Chat Completions: a function call whose arguments
// carry the raw input under the single "input" key.
func responsesCustomToolCallItemToChatToolCall(item map[string]any) (dto.ToolCallRequest, error) {
raw, err := kitutil.Marshal(item)
name := responsesCallName(item)
if name == "" {
return dto.ToolCallRequest{}, errors.New("custom_tool_call item is missing name")
}
arguments, err := kitutil.Marshal(map[string]string{
convmeta.CustomToolInputArgument: responsesArgumentsString(item["input"]),
})
if err != nil {
return dto.ToolCallRequest{}, err
}
return dto.ToolCallRequest{
ID: responsesCallID(item),
Type: dto.CustomType,
Custom: raw,
ID: responsesCallID(item),
Type: "function",
Function: dto.FunctionRequest{
Name: strings.TrimSpace(kitutil.Interface2String(item["name"])),
Arguments: responsesArgumentsString(item["input"]),
Name: name,
Arguments: string(arguments),
},
}, nil
}
@@ -443,6 +484,12 @@ func RequestTextToChatResponseFormat(raw json.RawMessage) (*dto.ResponseFormat,
func responsesImagePartToChatImageURL(part map[string]any) any {
if imageURL, ok := part["image_url"]; ok {
// Chat Completions takes the URL inside an object. detail is left out:
// Chat upstreams that validate it with older OpenAI SDKs reject newer
// values such as "original".
if url, ok := imageURL.(string); ok {
return map[string]any{"url": url}
}
return imageURL
}
imageURL := map[string]any{}
@@ -457,20 +504,52 @@ func responsesImagePartToChatImageURL(part map[string]any) any {
return imageURL
}
func responsesFilePartToChatFile(part map[string]any) any {
if file, ok := part["file"]; ok {
return file
// responsesFilePartToChat maps a Responses input_file onto Chat Completions,
// whose file part takes only PDF data or an uploaded file_id and no file URL.
// Files given by data or URL are loaded through the media resolver: PDF
// becomes a Chat file part with inline data and text files become a text
// part. A non-empty reason means Chat cannot carry the file.
func responsesFilePartToChat(ctx context.Context, part map[string]any) (map[string]any, string) {
fields := part
if nested, ok := part["file"].(map[string]any); ok {
fields = nested
}
file := map[string]any{}
for _, key := range []string{"file_id", "file_data", "filename", "file_url"} {
if value, ok := part[key]; ok {
file[key] = value
filename := strings.TrimSpace(kitutil.Interface2String(fields["filename"]))
source := ContentPartToFileSource(part)
if source == nil {
fileID := strings.TrimSpace(kitutil.Interface2String(fields["file_id"]))
if fileID == "" {
return nil, "input_file carries no file_data, file_url, or file_id"
}
file := map[string]any{"file_id": fileID}
if filename != "" {
file["filename"] = filename
}
return map[string]any{"type": dto.ContentTypeFile, "file": file}, ""
}
if len(file) == 0 {
return part
base64Data, mimeType, err := relaymedia.ResolveBase64Data(ctx, source, "formatting Responses input file for Chat")
if err != nil {
return nil, fmt.Sprintf("input_file could not be loaded: %s", err.Error())
}
return file
mediaType, _, _ := strings.Cut(strings.ToLower(strings.TrimSpace(mimeType)), ";")
mediaType = strings.TrimSpace(mediaType)
switch {
case mediaType == "application/pdf":
if filename == "" {
filename = "document.pdf"
}
return map[string]any{"type": dto.ContentTypeFile, "file": map[string]any{
"filename": filename,
"file_data": "data:application/pdf;base64," + base64Data,
}}, ""
case relaymedia.IsTextMimeType(mediaType):
text, ok := relaymedia.DecodeText(base64Data)
if !ok {
return nil, fmt.Sprintf("input_file of MIME type %q could not be decoded as UTF-8 text", mimeType)
}
return map[string]any{"type": dto.ContentTypeText, "text": text}, ""
}
return nil, fmt.Sprintf("OpenAI Chat Completions file parts accept only PDF, not %q", mimeType)
}
func responsesVideoPartToChatVideoURL(part map[string]any) any {
@@ -514,6 +593,17 @@ func CallID(item map[string]any) string {
return responsesCallID(item)
}
// responsesCallName is the upstream function name of a function_call or
// custom_tool_call item: a call made in a tool namespace uses the name its
// flattened definition was sent with.
func responsesCallName(item map[string]any) string {
name := strings.TrimSpace(kitutil.Interface2String(item["name"]))
if name == "" {
return ""
}
return convmeta.NamespacedToolName(strings.TrimSpace(kitutil.Interface2String(item["namespace"])), name)
}
func responsesArgumentsString(value any) string {
switch v := value.(type) {
case nil:
@@ -544,6 +634,57 @@ func responseToolOutputToChatContent(value any) any {
}
}
// responsesToolOutputToChat maps a function_call_output payload onto Chat Completions, where a
// tool message may only carry text. A Responses content-part array keeps its text on the tool
// message and returns the media parts in Chat shape for the caller to hoist into a user message;
// stringifying them instead would hand base64 image data to the upstream text tokenizer. Any
// other payload shape (string, object, plain JSON array) keeps the historical stringified form.
func responsesToolOutputToChat(ctx context.Context, value any) (any, []any) {
rawParts, ok := value.([]any)
if !ok || len(rawParts) == 0 {
return responseToolOutputToChatContent(value), nil
}
texts := make([]string, 0, len(rawParts))
mediaParts := make([]any, 0, len(rawParts))
labels := make([]string, 0, len(rawParts))
for _, rawPart := range rawParts {
part, ok := rawPart.(map[string]any)
if !ok {
return responseToolOutputToChatContent(value), nil
}
partType := strings.TrimSpace(kitutil.Interface2String(part["type"]))
switch partType {
case "input_text", "output_text", "text":
if text := kitutil.Interface2String(part["text"]); text != "" {
texts = append(texts, text)
}
case "input_image", "input_file", "input_audio", "input_video":
mediaParts = append(mediaParts, part)
label := "[" + strings.TrimPrefix(partType, "input_") + "]"
if !slices.Contains(labels, label) {
labels = append(labels, label)
}
default:
return responseToolOutputToChatContent(value), nil
}
}
if len(mediaParts) == 0 {
return strings.Join(texts, "\n"), nil
}
converted, err := responsesContentPartsToChatContent(ctx, mediaParts)
if err != nil {
return responseToolOutputToChatContent(value), nil
}
media, _ := converted.([]any)
if len(texts) == 0 {
// Upstreams reject empty tool content; the media itself rides on the hoisted user message.
return strings.Join(labels, " "), media
}
return strings.Join(texts, "\n"), media
}
func responsesRawFloat(raw json.RawMessage) (*float64, error) {
if !rawJSONPresent(raw) {
return nil, nil
relaykit/relayconvert/internal/oai_responses/to_oai_chat_req_test.go
+196 -9
View File
@@ -233,29 +233,44 @@ func TestResponsesRequestToChatCompletionsRequestToolsToolChoiceAndTextFormat(t
assert.True(t, gjson.GetBytes(got.ResponseFormat.JsonSchema, "strict").Bool())
}
func TestResponsesRequestToChatCompletionsRequestCustomToolCallPreservesRawShape(t *testing.T) {
func TestResponsesRequestToChatCompletionsRequestEncodesCustomToolHistoryAsFunction(t *testing.T) {
got, err := ResponsesRequestToChatCompletionsRequest(context.Background(), &dto.OpenAIResponsesRequest{
Model: "gpt-test",
Input: mustRawMessage(t, []map[string]any{
{
"type": "custom_tool_call",
"call_id": "call_custom",
"name": "apply_patch",
"input": "patch body",
"name": "exec",
"input": "echo \"hi\" && ls",
},
{
"type": "custom_tool_call_output",
"call_id": "call_custom",
"output": "hi",
},
}),
})
require.NoError(t, err)
require.Len(t, got.Messages, 1)
require.Len(t, got.Messages, 2)
toolCalls := got.Messages[0].ParseToolCalls()
require.Len(t, toolCalls, 1)
assert.Equal(t, dto.CustomType, toolCalls[0].Type)
assert.Equal(t, "function", toolCalls[0].Type)
assert.Equal(t, "call_custom", toolCalls[0].ID)
assert.Equal(t, "apply_patch", toolCalls[0].Function.Name)
assert.Equal(t, "patch body", toolCalls[0].Function.Arguments)
assert.Equal(t, "custom_tool_call", gjson.GetBytes(toolCalls[0].Custom, "type").String())
assert.Equal(t, "patch body", gjson.GetBytes(toolCalls[0].Custom, "input").String())
assert.Equal(t, "exec", toolCalls[0].Function.Name)
assert.Equal(t, `echo "hi" && ls`, gjson.Get(toolCalls[0].Function.Arguments, "input").String())
assert.Empty(t, toolCalls[0].Custom)
assert.Equal(t, dto.Message{Role: "tool", ToolCallId: "call_custom", Content: "hi"}, got.Messages[1])
}
func TestResponsesRequestToChatCompletionsRequestRejectsNamelessCustomToolCall(t *testing.T) {
_, err := ResponsesRequestToChatCompletionsRequest(context.Background(), &dto.OpenAIResponsesRequest{
Model: "gpt-test",
Input: mustRawMessage(t, []map[string]any{
{"type": "custom_tool_call", "call_id": "call_custom", "input": "ls"},
}),
})
require.ErrorContains(t, err, "custom_tool_call item is missing name")
}
func TestResponsesRequestToChatCompletionsRequestRejectsStatefulFields(t *testing.T) {
@@ -351,6 +366,178 @@ func TestResponsesRequestToChatCompletionsRequestRejectsMalformedPenalty(t *test
assert.Contains(t, err.Error(), "frequency_penalty")
}
func TestResponsesRequestToChatCompletionsRequestToolOutputContentParts(t *testing.T) {
const dataURL = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="
imagePart := map[string]any{"type": "input_image", "image_url": dataURL}
tests := []struct {
name string
output any
wantContent string
jsonContent bool
wantMedia []string
}{
{
name: "text and image keep text on tool message",
output: []any{map[string]any{"type": "input_text", "text": "screenshot taken"}, imagePart},
wantContent: "screenshot taken",
wantMedia: []string{dto.ContentTypeImageURL},
},
{
name: "image only uses placeholder",
output: []any{imagePart},
wantContent: "[image]",
wantMedia: []string{dto.ContentTypeImageURL},
},
{
name: "mixed media dedupes placeholder labels",
output: []any{imagePart, map[string]any{"type": "input_file", "file_id": "file_1"}, imagePart},
wantContent: "[image] [file]",
wantMedia: []string{dto.ContentTypeImageURL, dto.ContentTypeFile, dto.ContentTypeImageURL},
},
{
name: "text parts join with newline",
output: []any{
map[string]any{"type": "input_text", "text": "first"},
map[string]any{"type": "output_text", "text": ""},
map[string]any{"type": "text", "text": "second"},
},
wantContent: "first\nsecond",
},
{
name: "string passes through",
output: "done",
wantContent: "done",
},
{
name: "object stays json",
output: map[string]any{"ok": true},
wantContent: `{"ok":true}`,
jsonContent: true,
},
{
name: "plain array stays json",
output: []any{1, 2},
wantContent: `[1,2]`,
jsonContent: true,
},
{
name: "unknown part keeps whole array",
output: []any{imagePart, map[string]any{"type": "refusal", "refusal": "no"}},
wantContent: `[{"type":"input_image","image_url":"` + dataURL + `"},{"type":"refusal","refusal":"no"}]`,
jsonContent: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ResponsesRequestToChatCompletionsRequest(context.Background(), &dto.OpenAIResponsesRequest{
Model: "gpt-test",
Input: mustRawMessage(t, []map[string]any{
{"type": "function_call", "call_id": "call_1", "name": "view_image", "arguments": "{}"},
{"type": "function_call_output", "call_id": "call_1", "output": tt.output},
{"role": "user", "content": "next"},
}),
})
require.NoError(t, err)
wantLen := 3
if tt.wantMedia != nil {
wantLen = 4
}
require.Len(t, got.Messages, wantLen)
assert.Equal(t, "tool", got.Messages[1].Role)
assert.Equal(t, "call_1", got.Messages[1].ToolCallId)
if tt.jsonContent {
assert.JSONEq(t, tt.wantContent, got.Messages[1].StringContent())
} else {
assert.Equal(t, tt.wantContent, got.Messages[1].StringContent())
}
assert.Equal(t, dto.Message{Role: "user", Content: "next"}, got.Messages[wantLen-1])
if tt.wantMedia == nil {
return
}
assert.Equal(t, "user", got.Messages[2].Role)
parts := got.Messages[2].ParseContent()
require.Len(t, parts, len(tt.wantMedia))
for i, wantType := range tt.wantMedia {
assert.Equal(t, wantType, parts[i].Type)
}
require.NotNil(t, parts[0].GetImageMedia())
assert.Equal(t, dataURL, parts[0].GetImageMedia().Url)
})
}
}
func TestResponsesRequestToChatCompletionsRequestHoistsToolOutputMediaAfterToolBatch(t *testing.T) {
const dataURL = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="
imageOutput := []any{map[string]any{"type": "input_image", "image_url": dataURL}}
t.Run("parallel outputs stay contiguous", func(t *testing.T) {
got, err := ResponsesRequestToChatCompletionsRequest(context.Background(), &dto.OpenAIResponsesRequest{
Model: "gpt-test",
Input: mustRawMessage(t, []map[string]any{
{"type": "function_call", "call_id": "call_1", "name": "screenshot", "arguments": "{}"},
{"type": "function_call", "call_id": "call_2", "name": "read_file", "arguments": "{}"},
{"type": "function_call_output", "call_id": "call_1", "output": imageOutput},
{"type": "function_call_output", "call_id": "call_2", "output": "file contents"},
{"role": "user", "content": "what do you see?"},
}),
})
require.NoError(t, err)
require.Len(t, got.Messages, 5)
assert.Equal(t, "assistant", got.Messages[0].Role)
assert.Len(t, got.Messages[0].ParseToolCalls(), 2)
assert.Equal(t, dto.Message{Role: "tool", ToolCallId: "call_1", Content: "[image]"}, got.Messages[1])
assert.Equal(t, dto.Message{Role: "tool", ToolCallId: "call_2", Content: "file contents"}, got.Messages[2])
assert.Equal(t, "user", got.Messages[3].Role)
parts := got.Messages[3].ParseContent()
require.Len(t, parts, 1)
assert.Equal(t, dto.ContentTypeImageURL, parts[0].Type)
assert.Equal(t, dto.Message{Role: "user", Content: "what do you see?"}, got.Messages[4])
})
t.Run("custom tool outputs stay in the same batch", func(t *testing.T) {
got, err := ResponsesRequestToChatCompletionsRequest(context.Background(), &dto.OpenAIResponsesRequest{
Model: "gpt-test",
Input: mustRawMessage(t, []map[string]any{
{"type": "function_call", "call_id": "call_1", "name": "screenshot", "arguments": "{}"},
{"type": "custom_tool_call", "call_id": "call_2", "name": "exec", "input": "ls"},
{"type": "function_call_output", "call_id": "call_1", "output": imageOutput},
{"type": "custom_tool_call_output", "call_id": "call_2", "output": "file.txt"},
}),
})
require.NoError(t, err)
require.Len(t, got.Messages, 4)
assert.Len(t, got.Messages[0].ParseToolCalls(), 2)
assert.Equal(t, dto.Message{Role: "tool", ToolCallId: "call_1", Content: "[image]"}, got.Messages[1])
assert.Equal(t, dto.Message{Role: "tool", ToolCallId: "call_2", Content: "file.txt"}, got.Messages[2])
assert.Equal(t, "user", got.Messages[3].Role)
})
t.Run("trailing output flushes media at end of input", func(t *testing.T) {
got, err := ResponsesRequestToChatCompletionsRequest(context.Background(), &dto.OpenAIResponsesRequest{
Model: "gpt-test",
Input: mustRawMessage(t, []map[string]any{
{"type": "function_call", "call_id": "call_1", "name": "screenshot", "arguments": "{}"},
{"type": "function_call_output", "call_id": "call_1", "output": imageOutput},
}),
})
require.NoError(t, err)
require.Len(t, got.Messages, 3)
assert.Equal(t, dto.Message{Role: "tool", ToolCallId: "call_1", Content: "[image]"}, got.Messages[1])
assert.Equal(t, "user", got.Messages[2].Role)
parts := got.Messages[2].ParseContent()
require.Len(t, parts, 1)
require.NotNil(t, parts[0].GetImageMedia())
assert.Equal(t, dataURL, parts[0].GetImageMedia().Url)
})
}
func mustRawMessage(t *testing.T, value any) []byte {
t.Helper()
raw, err := kitutil.Marshal(value)
relaykit/relayconvert/internal/shared/claude/media.go
+63
View File
@@ -0,0 +1,63 @@
package claude
import (
"fmt"
"strings"
"github.com/QuantumNous/new-api/relaykit/dto"
relaymedia "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/media"
)
// imageMediaTypes are the image formats Claude accepts.
var imageMediaTypes = map[string]struct{}{
"image/jpeg": {},
"image/png": {},
"image/gif": {},
"image/webp": {},
}
// MediaBlock builds the Claude content block for resolved base64 media: an
// image in a format Claude accepts, a PDF document, or a plain-text document
// for text files. declaredImage marks media sent as an image part, which
// passes through when its MIME type is unknown. A non-empty reason means
// Claude cannot carry the media and the block must be omitted.
func MediaBlock(base64Data string, mimeType string, declaredImage bool) (dto.ClaudeMediaMessage, string) {
mediaType, _, _ := strings.Cut(strings.ToLower(strings.TrimSpace(mimeType)), ";")
mediaType = strings.TrimSpace(mediaType)
if mediaType == "image/jpg" {
mediaType = "image/jpeg"
}
switch {
case mediaType == "application/pdf":
return dto.ClaudeMediaMessage{
Type: "document",
Source: &dto.ClaudeMessageSource{Type: "base64", MediaType: mediaType, Data: base64Data},
}, ""
case relaymedia.IsTextMimeType(mediaType):
text, ok := relaymedia.DecodeText(base64Data)
if !ok {
return dto.ClaudeMediaMessage{}, fmt.Sprintf("text file of MIME type %q could not be decoded as UTF-8 text", mimeType)
}
return dto.ClaudeMediaMessage{
Type: "document",
Source: &dto.ClaudeMessageSource{Type: "text", MediaType: "text/plain", Data: text},
}, ""
case mediaType == "" && declaredImage:
return dto.ClaudeMediaMessage{
Type: "image",
Source: &dto.ClaudeMessageSource{Type: "base64", MediaType: mimeType, Data: base64Data},
}, ""
case mediaType == "application/octet-stream" && declaredImage:
return dto.ClaudeMediaMessage{}, "the image format could not be identified"
}
if _, ok := imageMediaTypes[mediaType]; ok {
return dto.ClaudeMediaMessage{
Type: "image",
Source: &dto.ClaudeMessageSource{Type: "base64", MediaType: mediaType, Data: base64Data},
}, ""
}
if strings.HasPrefix(mediaType, "image/") {
return dto.ClaudeMediaMessage{}, fmt.Sprintf("Claude accepts only JPEG, PNG, GIF, and WebP images, not %q", mimeType)
}
return dto.ClaudeMediaMessage{}, fmt.Sprintf("Claude Messages cannot carry content of MIME type %q", mimeType)
}
relaykit/relayconvert/internal/shared/claude/reasoning.go
+32 -7
View File
@@ -44,7 +44,7 @@ func ApplyReasoning(ctx context.Context, req *dto.ClaudeRequest, info convmeta.M
case req.Thinking.BudgetTokens != nil:
effort = string(reasoning.EffortFromBudget(*req.Thinking.BudgetTokens))
case req.Thinking.Type == "enabled" || req.Thinking.Type == "adaptive":
effort = string(reasoning.EffortHigh)
effort = string(reasoning.ClaudeDefaultEffort(req.Model))
}
}
info.SetReasoningEffort(effort)
@@ -81,6 +81,16 @@ func ApplyReasoning(ctx context.Context, req *dto.ClaudeRequest, info convmeta.M
intent.Effort = reasoning.EffortHigh
}
}
requestEffort, err := reasoning.ParseEffort(req.GetEfforts())
if err != nil {
return err
}
if knownClaudeModel && intent.Mode == reasoning.ModeDisabled && (intent.Effort == "" || intent.Effort == reasoning.EffortNone) &&
requestEffort != "" && requestEffort != reasoning.EffortNone {
// Effort works without thinking, so the request's own effort is
// rendered for the thinking-off state rather than left unchecked.
intent.Effort = requestEffort
}
if req.MaxTokens == nil && intent.HasStrength() {
// Adapter-provided defaults may be raised to accommodate an exact
// cross-protocol budget. Values too small for manual thinking are
@@ -117,7 +127,14 @@ func ApplyReasoning(ctx context.Context, req *dto.ClaudeRequest, info convmeta.M
if rendered.Thinking != nil {
req.Thinking = rendered.Thinking
}
if rendered.OutputEffort != "" {
outputEffort := rendered.OutputEffort
// An effort the request still carries reaches a known model only at a
// level it accepts, and is removed where the model takes none.
rewriteRequestEffort := outputEffort == "" && knownClaudeModel && requestEffort != ""
if rewriteRequestEffort {
outputEffort = reasoning.ClaudeEffort(capabilityModel, requestEffort)
}
if outputEffort != "" || rewriteRequestEffort {
outputConfig := make(map[string]any)
if len(req.OutputConfig) > 0 {
if kitutil.GetJsonType(req.OutputConfig) != "object" {
@@ -130,12 +147,20 @@ func ApplyReasoning(ctx context.Context, req *dto.ClaudeRequest, info convmeta.M
outputConfig = make(map[string]any)
}
}
outputConfig["effort"] = string(rendered.OutputEffort)
encoded, err := kitutil.Marshal(outputConfig)
if err != nil {
return fmt.Errorf("failed to marshal Claude output_config: %w", err)
if outputEffort != "" {
outputConfig["effort"] = string(outputEffort)
} else {
delete(outputConfig, "effort")
}
if len(outputConfig) == 0 {
req.OutputConfig = nil
} else {
encoded, err := kitutil.Marshal(outputConfig)
if err != nil {
return fmt.Errorf("failed to marshal Claude output_config: %w", err)
}
req.OutputConfig = encoded
}
req.OutputConfig = encoded
}
if rendered.ClearSampling {
if req.Temperature != nil || req.TopP != nil || req.TopK != nil {
relaykit/relayconvert/internal/shared/gemini/request.go
+40 -25
View File
@@ -3,6 +3,8 @@ package gemini
import (
"context"
"fmt"
"maps"
"slices"
"strconv"
"strings"
@@ -13,26 +15,43 @@ import (
"github.com/QuantumNous/new-api/relaykit/types"
)
var SupportedMimeTypes = map[string]bool{
"application/pdf": true,
"audio/mpeg": true,
"audio/mp3": true,
"audio/wav": true,
"image/png": true,
"image/jpeg": true,
"image/jpg": true,
"image/webp": true,
"image/heic": true,
"image/heif": true,
"text/plain": true,
"video/mov": true,
"video/mpeg": true,
"video/mp4": true,
"video/mpg": true,
"video/avi": true,
"video/wmv": true,
"video/mpegps": true,
"video/flv": true,
// supportedMimeTypes follows the Blob.mimeType list in the Gemini API
// reference (https://ai.google.dev/api/generate-content#Blob). Audio and
// video are listed there as audio/* and video/*; IsSupportedMimeType accepts
// them by prefix.
var supportedMimeTypes = map[string]bool{
"image/png": true,
"image/jpeg": true,
"image/jpg": true,
"image/webp": true,
"image/heic": true,
"image/heif": true,
"image/gif": true,
"image/avif": true,
"text/plain": true,
"text/html": true,
"text/css": true,
"text/javascript": true,
"text/x-typescript": true,
"text/csv": true,
"text/markdown": true,
"text/x-python": true,
"text/xml": true,
"text/rtf": true,
"application/x-javascript": true,
"application/x-typescript": true,
"application/x-python-code": true,
"application/json": true,
"application/x-ipynb+json": true,
"application/rtf": true,
"application/pdf": true,
}
// IsSupportedMimeType reports whether Gemini accepts inline data of the MIME
// type.
func IsSupportedMimeType(mimeType string) bool {
mimeType = strings.ToLower(mimeType)
return supportedMimeTypes[mimeType] || strings.HasPrefix(mimeType, "audio/") || strings.HasPrefix(mimeType, "video/")
}
var SafetySettingCategories = []string{
@@ -272,9 +291,5 @@ func HasFunctionCallContent(call *dto.FunctionCall) bool {
}
func SupportedMimeTypesList() []string {
keys := make([]string, 0, len(SupportedMimeTypes))
for key := range SupportedMimeTypes {
keys = append(keys, key)
}
return keys
return append(slices.Sorted(maps.Keys(supportedMimeTypes)), "audio/*", "video/*")
}
relaykit/relayconvert/internal/shared/gemini/schema.go
+285
View File
@@ -1,9 +1,17 @@
package gemini
import (
"context"
"encoding/json"
"fmt"
"slices"
"strconv"
"strings"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/internal/convdiag"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/types"
)
var geminiOpenAPISchemaAllowedFields = map[string]struct{}{
@@ -56,6 +64,7 @@ func cleanGeminiFunctionParametersWithDepth(params interface{}, depth int) inter
}
normalizeGeminiSchemaTypeAndNullable(cleanedMap)
sanitizeGeminiEnum(cleanedMap)
if props, ok := cleanedMap["properties"].(map[string]interface{}); ok && props != nil {
cleanedProps := make(map[string]interface{})
@@ -102,6 +111,7 @@ func cleanGeminiFunctionParametersShallow(params interface{}) interface{} {
}
}
normalizeGeminiSchemaTypeAndNullable(cleanedMap)
sanitizeGeminiEnum(cleanedMap)
delete(cleanedMap, "properties")
delete(cleanedMap, "items")
delete(cleanedMap, "anyOf")
@@ -113,6 +123,30 @@ func cleanGeminiFunctionParametersShallow(params interface{}) interface{} {
}
}
// sanitizeGeminiEnum keeps the enum constraint only when every element is a
// string, the only form Gemini's legacy Schema accepts. Any other shape is
// dropped (e.g. boolean or number elements, which valid JSON Schema allows but
// Gemini rejects with a 400). This widens the value domain for that schema, a
// deliberate lossy relaxation: the declared type is preserved, string enums
// pass through untouched.
func sanitizeGeminiEnum(schema map[string]interface{}) {
raw, exists := schema["enum"]
if !exists {
return
}
values, ok := raw.([]interface{})
if !ok {
delete(schema, "enum")
return
}
for _, value := range values {
if _, isString := value.(string); !isString {
delete(schema, "enum")
return
}
}
}
func normalizeGeminiSchemaTypeAndNullable(schema map[string]interface{}) {
rawType, ok := schema["type"]
if !ok || rawType == nil {
@@ -175,6 +209,148 @@ func normalizeGeminiSchemaTypeAndNullable(schema map[string]interface{}) {
}
}
// FunctionParameters returns the Gemini function declaration field that
// carries schema, and its value. A schema that fits Gemini's typed Schema is
// cleaned as before, and still left out when its properties are empty. Any
// other schema goes unchanged to parametersJsonSchema when jsonSchema is
// true; otherwise it is cleaned anyway and misfits names what the typed
// Schema cannot carry.
func FunctionParameters(schema any, jsonSchema bool) (field string, value any, misfits []string) {
misfits = schemaMisfits(schema)
object, _ := schema.(map[string]any)
properties, hasProperties := object["properties"].(map[string]any)
emptyObject := hasProperties && len(properties) == 0
if emptyObject {
for key, item := range object {
switch key {
case "type", "properties":
continue
case "required":
if required, _ := item.([]any); len(required) == 0 {
continue
}
}
if _, allowed := geminiOpenAPISchemaAllowedFields[key]; allowed {
misfits = append(misfits, "constraints on an empty object")
break
}
}
}
if len(misfits) > 0 && jsonSchema {
return "parametersJsonSchema", schema, nil
}
if emptyObject {
return "parameters", nil, misfits
}
return "parameters", CleanFunctionParameters(schema), misfits
}
// ApplyResponseSchema sets the structured output schema from the json_schema
// object of an OpenAI response format. A schema that fits Gemini's typed
// Schema keeps the responseSchema cleanup. Any other schema is sent unchanged
// as responseJsonSchema when jsonSchema is true; otherwise it is cleaned
// anyway and a diagnostic at path names what responseSchema cannot carry.
func ApplyResponseSchema(ctx context.Context, config *dto.GeminiChatGenerationConfig, rawFormat json.RawMessage, path string, jsonSchema bool) {
var format dto.FormatJsonSchema
if err := kitutil.Unmarshal(rawFormat, &format); err != nil {
return
}
misfits := schemaMisfits(format.Schema)
if len(misfits) > 0 && jsonSchema {
// The raw member keeps the client's key order.
var raw struct {
Schema json.RawMessage `json:"schema"`
}
if err := kitutil.Unmarshal(rawFormat, &raw); err == nil {
config.ResponseJsonSchema = raw.Schema
return
}
}
config.ResponseSchema = RemoveAdditionalProperties(format.Schema, 0)
if len(misfits) > 0 {
convdiag.Add(ctx, types.ConversionDiagnostic{
Code: "gemini_schema_keyword_dropped",
Path: path,
Message: fmt.Sprintf("Gemini response schema cannot carry %s", strings.Join(misfits, ", ")),
Severity: types.ConversionDiagnosticWarning,
})
}
}
// schemaMisfits lists what Gemini's typed Schema cannot hold of a JSON Schema
// as written: keywords outside its fields, type arrays, non-string enum
// values, tuple items, boolean schemas, and nesting deeper than
// CleanFunctionParameters keeps. additionalProperties and $schema are not
// misfits: agent tools and OpenAI strict schemas carry them on nearly every
// object, and dropping them as before keeps those requests unchanged for
// upstreams that do not pass the JSON Schema fields through.
func schemaMisfits(schema any) []string {
found := make(map[string]struct{})
collectSchemaMisfits(schema, 0, found)
misfits := make([]string, 0, len(found))
for misfit := range found {
misfits = append(misfits, misfit)
}
slices.Sort(misfits)
return misfits
}
func collectSchemaMisfits(schema any, depth int, found map[string]struct{}) {
if depth > geminiFunctionSchemaMaxDepth {
found[fmt.Sprintf("nesting deeper than %d levels", geminiFunctionSchemaMaxDepth)] = struct{}{}
return
}
value, ok := schema.(map[string]any)
if !ok {
if _, isBool := schema.(bool); isBool {
found["boolean schemas"] = struct{}{}
}
return
}
for key, item := range value {
if _, allowed := geminiOpenAPISchemaAllowedFields[key]; !allowed {
if key != "additionalProperties" && key != "$schema" {
found[key] = struct{}{}
}
continue
}
switch key {
case "type":
if _, isArray := item.([]any); isArray {
found["type arrays"] = struct{}{}
}
case "enum":
values, _ := item.([]any)
if slices.ContainsFunc(values, func(value any) bool {
_, isString := value.(string)
return !isString
}) {
found["non-string enum values"] = struct{}{}
}
case "properties":
properties, _ := item.(map[string]any)
for _, property := range properties {
collectSchemaMisfits(property, depth+1, found)
}
case "items":
entries, isTuple := item.([]any)
if !isTuple {
collectSchemaMisfits(item, depth+1, found)
continue
}
found["tuple items"] = struct{}{}
for _, entry := range entries {
collectSchemaMisfits(entry, depth+1, found)
}
case "anyOf":
variants, _ := item.([]any)
for _, variant := range variants {
collectSchemaMisfits(variant, depth+1, found)
}
}
}
}
func RemoveAdditionalProperties(schema interface{}, depth int) interface{} {
if depth >= 5 {
return schema
@@ -254,3 +430,112 @@ func OpenAIToolChoiceToConfig(toolChoice any) *dto.ToolConfig {
return nil
}
// geminiSchemaInt64Fields are Gemini Schema bounds typed int64, which the
// proto JSON encoding may send as strings.
var geminiSchemaInt64Fields = map[string]struct{}{
"maxItems": {},
"minItems": {},
"maxLength": {},
"minLength": {},
"maxProperties": {},
"minProperties": {},
}
// OpenAPISchemaToJSONSchema rewrites a schema in Gemini's OpenAPI subset as
// JSON Schema: type names are lowercased, nullable becomes a "null" type,
// example becomes examples, and int64 bounds sent as strings become numbers.
// It also returns the Gemini-only keywords JSON Schema cannot express, which
// are dropped (propertyOrdering).
func OpenAPISchemaToJSONSchema(schema any) (any, []string) {
dropped := make(map[string]struct{})
converted := openAPISchemaToJSONSchema(schema, 0, dropped)
keywords := make([]string, 0, len(dropped))
for keyword := range dropped {
keywords = append(keywords, keyword)
}
slices.Sort(keywords)
return converted, keywords
}
func openAPISchemaToJSONSchema(schema any, depth int, dropped map[string]struct{}) any {
value, ok := schema.(map[string]any)
if !ok || depth >= geminiFunctionSchemaMaxDepth {
return schema
}
converted := make(map[string]any, len(value))
nullable := false
for key, item := range value {
switch key {
case "type":
typeName, isString := item.(string)
if !isString {
converted[key] = item
continue
}
typeName = strings.ToLower(strings.TrimSpace(typeName))
if typeName != "" && typeName != "type_unspecified" {
converted[key] = typeName
}
case "nullable":
nullable, _ = item.(bool)
case "propertyOrdering":
dropped[key] = struct{}{}
case "example":
if _, exists := value["examples"]; exists {
dropped[key] = struct{}{}
continue
}
converted["examples"] = []any{item}
case "properties":
properties, isMap := item.(map[string]any)
if !isMap {
converted[key] = item
continue
}
convertedProperties := make(map[string]any, len(properties))
for name, property := range properties {
convertedProperties[name] = openAPISchemaToJSONSchema(property, depth+1, dropped)
}
converted[key] = convertedProperties
case "items":
converted[key] = openAPISchemaToJSONSchema(item, depth+1, dropped)
case "anyOf":
variants, isArray := item.([]any)
if !isArray {
converted[key] = item
continue
}
convertedVariants := make([]any, len(variants))
for index, variant := range variants {
convertedVariants[index] = openAPISchemaToJSONSchema(variant, depth+1, dropped)
}
converted[key] = convertedVariants
default:
if _, isInt64 := geminiSchemaInt64Fields[key]; isInt64 {
if text, isString := item.(string); isString {
if number, err := strconv.ParseInt(strings.TrimSpace(text), 10, 64); err == nil {
converted[key] = number
continue
}
}
}
converted[key] = item
}
}
if !nullable {
return converted
}
switch typed := converted["type"].(type) {
case string:
converted["type"] = []any{typed, "null"}
default:
if variants, isArray := converted["anyOf"].([]any); isArray {
converted["anyOf"] = append(variants, map[string]any{"type": "null"})
}
}
if values, isArray := converted["enum"].([]any); isArray && !slices.Contains(values, nil) {
converted["enum"] = append(values, nil)
}
return converted
}
relaykit/relayconvert/internal/toolconv/decode.go
+136 -6
View File
@@ -7,6 +7,7 @@ import (
"strings"
"github.com/QuantumNous/new-api/relaykit/dto"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/types"
)
@@ -142,11 +143,11 @@ func extractOpenAIResponsesRequest(request any) (any, Set, error) {
return nil, Set{}, fmt.Errorf("invalid Responses tools: %w", err)
}
for index, rawTool := range rawTools {
definition, err := decodeOpenAIResponsesDefinition(rawTool)
definitions, err := decodeOpenAIResponsesDefinitions(rawTool)
if err != nil {
return nil, Set{}, fmt.Errorf("tools[%d]: %w", index, err)
}
set.Definitions = append(set.Definitions, definition)
set.Definitions = append(set.Definitions, definitions...)
}
}
choice, err := decodeOpenAIResponsesChoice(source.ToolChoice)
@@ -155,11 +156,41 @@ func extractOpenAIResponsesRequest(request any) (any, Set, error) {
}
set.Choice = choice
input, additionalDefinitions, additionalDiagnostics, err := extractOpenAIResponsesAdditionalTools(source.Input)
if err != nil {
return nil, Set{}, err
}
set.Definitions = append(set.Definitions, additionalDefinitions...)
set.Diagnostics = append(set.Diagnostics, additionalDiagnostics...)
// Flattened namespace names can collide with other tools.
seenNames := make(map[string]struct{}, len(set.Definitions))
seenNamespaces := make(map[string]struct{})
definitions := set.Definitions[:0]
for _, definition := range set.Definitions {
if _, seen := seenNamespaces[definition.Namespace]; !seen && definition.Namespace != "" && definition.Namespace != convmeta.DefaultResponsesToolNamespace {
seenNamespaces[definition.Namespace] = struct{}{}
set.Diagnostics = append(set.Diagnostics, presentationLoss("tools", "tool_namespace_flattened",
fmt.Sprintf("tools in namespace %q are sent as functions named %q", definition.Namespace, convmeta.NamespacedToolName(definition.Namespace, "<name>"))))
}
name := definition.Name
_, duplicate := seenNames[name]
if definition.Namespace != "" && name != "" && duplicate {
set.Diagnostics = append(set.Diagnostics, semanticLoss("tools", "namespaced_tool_dropped",
fmt.Sprintf("tool %q from namespace %q shares its flattened name with another tool and was omitted", name, definition.Namespace)))
continue
}
if name != "" {
seenNames[name] = struct{}{}
}
definitions = append(definitions, definition)
}
set.Definitions = definitions
clone := *source
clone.Tools = nil
clone.ToolChoice = nil
clone.ParallelToolCalls = nil
sanitizedInput, history, err := extractOpenAIResponsesHostedHistory(source.Input)
sanitizedInput, history, err := extractOpenAIResponsesHostedHistory(input)
if err != nil {
return nil, Set{}, err
}
@@ -168,6 +199,103 @@ func extractOpenAIResponsesRequest(request any) (any, Set, error) {
return &clone, set, nil
}
// extractOpenAIResponsesAdditionalTools removes the additional_tools input
// items Codex uses to declare tools and returns their definitions. No target
// protocol has an input item that declares tools, so they join the top-level
// tool list.
func extractOpenAIResponsesAdditionalTools(input json.RawMessage) (json.RawMessage, []Definition, []types.ConversionDiagnostic, error) {
if len(input) == 0 || kitutil.GetJsonType(input) != "array" {
return input, nil, nil, nil
}
var rawItems []json.RawMessage
if err := kitutil.Unmarshal(input, &rawItems); err != nil {
return nil, nil, nil, fmt.Errorf("invalid Responses input: %w", err)
}
var (
filtered []json.RawMessage
definitions []Definition
diagnostics []types.ConversionDiagnostic
)
for index, rawItem := range rawItems {
var item struct {
Type string `json:"type"`
Tools []json.RawMessage `json:"tools"`
}
if err := kitutil.Unmarshal(rawItem, &item); err != nil || strings.TrimSpace(item.Type) != "additional_tools" {
filtered = append(filtered, rawItem)
continue
}
for toolIndex, rawTool := range item.Tools {
decoded, err := decodeOpenAIResponsesDefinitions(rawTool)
if err != nil {
return nil, nil, nil, fmt.Errorf("input[%d].tools[%d]: %w", index, toolIndex, err)
}
definitions = append(definitions, decoded...)
}
diagnostics = append(diagnostics, presentationLoss(fmt.Sprintf("input[%d]", index), "additional_tools_flattened",
"additional_tools items are sent as top-level tools; their position in the input is not preserved"))
}
if len(diagnostics) == 0 {
return input, nil, nil, nil
}
encoded, err := kitutil.Marshal(filtered)
if err != nil {
return nil, nil, nil, err
}
return encoded, definitions, diagnostics, nil
}
// decodeOpenAIResponsesDefinitions decodes one Responses tool. A namespace
// tool expands into its member tools, each renamed with
// convmeta.NamespacedToolName and described with the namespace description.
func decodeOpenAIResponsesDefinitions(raw json.RawMessage) ([]Definition, error) {
var namespace struct {
Type string `json:"type"`
Name string `json:"name"`
Description string `json:"description"`
Tools []map[string]any `json:"tools"`
}
if err := kitutil.Unmarshal(raw, &namespace); err != nil || strings.TrimSpace(namespace.Type) != "namespace" {
definition, err := decodeOpenAIResponsesDefinition(raw)
if err != nil {
return nil, err
}
return []Definition{definition}, nil
}
namespaceName := strings.TrimSpace(namespace.Name)
if namespaceName == "" {
return nil, fmt.Errorf("namespace tool requires name")
}
definitions := make([]Definition, 0, len(namespace.Tools))
for index, tool := range namespace.Tools {
name := strings.TrimSpace(kitutil.Interface2String(tool["name"]))
if name != "" {
tool["name"] = convmeta.NamespacedToolName(namespaceName, name)
}
description := strings.TrimSpace(kitutil.Interface2String(tool["description"]))
if namespaceDescription := strings.TrimSpace(namespace.Description); namespaceDescription != "" {
description = strings.TrimSpace(namespaceDescription + "\n\n" + description)
}
if description != "" {
tool["description"] = description
}
encoded, err := kitutil.Marshal(tool)
if err != nil {
return nil, err
}
definition, err := decodeOpenAIResponsesDefinition(encoded)
if err != nil {
return nil, fmt.Errorf("namespace %q tools[%d]: %w", namespaceName, index, err)
}
definition.Namespace = namespaceName
if definition.Name == "" {
definition.Name = kitutil.Interface2String(tool["name"])
}
definitions = append(definitions, definition)
}
return definitions, nil
}
func extractClaudeRequest(request any) (any, Set, error) {
source, ok := request.(*dto.ClaudeRequest)
if !ok {
@@ -428,9 +556,10 @@ func decodeGeminiDefinitions(raw json.RawMessage) ([]Definition, error) {
Name: strings.TrimSpace(kitutil.Interface2String(function["name"])),
Raw: functionRaw,
Function: &Function{
Name: strings.TrimSpace(kitutil.Interface2String(function["name"])),
Description: kitutil.Interface2String(function["description"]),
Parameters: parameters,
Name: strings.TrimSpace(kitutil.Interface2String(function["name"])),
Description: kitutil.Interface2String(function["description"]),
Parameters: parameters,
OpenAPISchema: function["parameters"] != nil,
},
})
}
@@ -610,6 +739,7 @@ func decodeOpenAIResponsesChoice(raw json.RawMessage) (*Choice, error) {
if name == "" {
return nil, fmt.Errorf("Responses function tool_choice requires name")
}
name = convmeta.NamespacedToolName(strings.TrimSpace(kitutil.Interface2String(value["namespace"])), name)
return &Choice{Mode: ChoiceNamed, Kind: KindFunction, Name: name}, nil
}
if isOpenAIResponsesWebSearchType(toolType) {
relaykit/relayconvert/internal/toolconv/encode.go
+418 -62
View File
@@ -24,16 +24,17 @@ func AttachRequest(format types.RelayFormat, request any, set Set, options *conv
)
switch format {
case types.RelayFormatOpenAI:
value, diagnostics, err = attachOpenAIChatRequest(request, set)
value, diagnostics, err = attachOpenAIChatRequest(request, set, options)
case types.RelayFormatOpenAIResponses:
value, diagnostics, err = attachOpenAIResponsesRequest(request, set)
value, diagnostics, err = attachOpenAIResponsesRequest(request, set, options)
case types.RelayFormatClaude:
value, diagnostics, err = attachClaudeRequest(request, set, options)
case types.RelayFormatGemini:
value, diagnostics, err = attachGeminiRequest(request, set)
value, diagnostics, err = attachGeminiRequest(request, set, options)
default:
value = request
}
diagnostics = append(set.Diagnostics, diagnostics...)
if err != nil {
return nil, diagnostics, err
}
@@ -47,24 +48,30 @@ func AttachRequest(format types.RelayFormat, request any, set Set, options *conv
return value, diagnostics, nil
}
func attachOpenAIChatRequest(request any, set Set) (any, []types.ConversionDiagnostic, error) {
func attachOpenAIChatRequest(request any, set Set, options *convmeta.Options) (any, []types.ConversionDiagnostic, error) {
target, ok := request.(*dto.GeneralOpenAIRequest)
if !ok || target == nil {
return nil, nil, fmt.Errorf("expected OpenAI chat completions request, got %T", request)
}
var diagnostics []types.ConversionDiagnostic
customTools, diagnostics := responsesCustomTools(set)
// chatWebSearchEncoded reports that the host encoder handled the hosted
// web search; it may have written a tools[] entry or top-level fields
// instead of web_search_options.
chatWebSearchEncoded := false
for index, definition := range set.Definitions {
switch definition.Kind {
case KindFunction:
if definition.Function == nil {
continue
}
parameters, schemaDiagnostics := jsonSchemaParameters(index, definition.Function)
diagnostics = append(diagnostics, schemaDiagnostics...)
target.Tools = append(target.Tools, dto.ToolCallRequest{
Type: "function",
Function: dto.FunctionRequest{
Name: definition.Function.Name,
Description: definition.Function.Description,
Parameters: definition.Function.Parameters,
Parameters: parameters,
Strict: definition.Function.Strict,
},
})
@@ -72,23 +79,52 @@ func attachOpenAIChatRequest(request any, set Set) (any, []types.ConversionDiagn
if set.Source == types.RelayFormatGemini {
diagnostics = append(diagnostics, geminiNativeWebSearchDiagnostics(index, definition, types.RelayFormatOpenAI)...)
}
if target.WebSearchOptions != nil {
return nil, diagnostics, fmt.Errorf("tools[%d]: multiple hosted web-search definitions cannot be represented by Chat Completions", index)
if target.WebSearchOptions != nil || chatWebSearchEncoded {
diagnostics = append(diagnostics, semanticLoss(
fmt.Sprintf("tools[%d]", index),
"duplicate_hosted_web_search",
"Chat Completions carries one hosted web search; this definition was dropped",
))
continue
}
options := &dto.WebSearchOptions{}
call := newWebSearchCall(set, index, definition, types.RelayFormatOpenAI)
call.Chat = target
tool, encoderDiagnostics, handled, err := encodeWebSearch(call, options)
if err != nil {
return nil, diagnostics, err
}
if handled {
diagnostics = append(diagnostics, encoderDiagnostics...)
chatWebSearchEncoded = true
if tool != nil {
chatTool, ok := tool.(dto.ToolCallRequest)
if !ok {
return nil, diagnostics, fmt.Errorf("%s: Chat web-search encoder returned %T, want dto.ToolCallRequest", call.Path(), tool)
}
target.Tools = append(target.Tools, chatTool)
}
continue
}
webSearchOptions := &dto.WebSearchOptions{}
if definition.WebSearch != nil {
options.SearchContextSize = definition.WebSearch.SearchContextSize
webSearchOptions.SearchContextSize = definition.WebSearch.SearchContextSize
if definition.WebSearch.Location != nil {
location := map[string]any{
"type": "approximate",
"approximate": locationMap(definition.WebSearch.Location),
}
options.UserLocation, _ = kitutil.Marshal(location)
webSearchOptions.UserLocation, _ = kitutil.Marshal(location)
}
diagnostics = append(diagnostics, openAIChatWebSearchDiagnostics(index, definition.WebSearch)...)
}
target.WebSearchOptions = options
target.WebSearchOptions = webSearchOptions
default:
if tool, handled := customTools[index]; handled {
if tool != nil {
target.Tools = append(target.Tools, tool.chatTool())
}
continue
}
diagnostics = append(diagnostics, semanticLoss(
fmt.Sprintf("tools[%d]", index),
"unsupported_hosted_tool",
@@ -97,17 +133,281 @@ func attachOpenAIChatRequest(request any, set Set) (any, []types.ConversionDiagn
}
}
normalizedChoice, allowedChoiceDiagnostics := narrowAllowedFunctionChoice(set.Choice, types.RelayFormatOpenAI)
diagnostics = append(diagnostics, unsupportedHostedHistoryDiagnostics(types.RelayFormatOpenAI, set.History)...)
if chatWebSearchEncoded && forcedWebSearch(set.Choice) {
// The encoder received Forced and expressed it in its own encoding.
set.Choice = nil
}
if len(target.Tools) == 0 {
return target, append(diagnostics, toolChoiceWithoutToolsDiagnostics(set)...), nil
}
normalizedChoice, allowedChoiceDiagnostics := narrowAllowedFunctionChoice(responsesCustomToolChoice(set.Choice, customTools), types.RelayFormatOpenAI)
choice, choiceDiagnostics := encodeOpenAIChatChoice(normalizedChoice)
target.ToolChoice = choice
target.ParallelTooCalls = set.ParallelAllowed
diagnostics = append(diagnostics, allowedChoiceDiagnostics...)
diagnostics = append(diagnostics, choiceDiagnostics...)
diagnostics = append(diagnostics, unsupportedHostedHistoryDiagnostics(types.RelayFormatOpenAI, set.History)...)
return target, diagnostics, nil
}
func attachOpenAIResponsesRequest(request any, set Set) (any, []types.ConversionDiagnostic, error) {
// newWebSearchCall describes one hosted web-search definition for the host
// encoder; the caller sets the typed target request.
func newWebSearchCall(set Set, index int, definition Definition, target types.RelayFormat) convmeta.WebSearchCall {
call := convmeta.WebSearchCall{
Source: set.Source,
Target: target,
NativeType: definition.NativeType,
Index: index,
Forced: forcedWebSearch(set.Choice),
}
if definition.WebSearch != nil {
call.Spec = *definition.WebSearch
}
return call
}
func forcedWebSearch(choice *Choice) bool {
return choice != nil && choice.Mode == ChoiceNamed && choice.Kind == KindWebSearch
}
// encodeWebSearch runs the host web-search encoder for one definition.
// handled is false when no encoder is set or it returned nil; the caller then
// keeps its default encoding and default diagnostics. When handled, the
// encoder's diagnostics replace the default ones and tool (possibly nil) is
// what the caller appends.
func encodeWebSearch(call convmeta.WebSearchCall, options *convmeta.Options) (tool any, diagnostics []types.ConversionDiagnostic, handled bool, err error) {
if options == nil || options.WebSearch == nil {
return nil, nil, false, nil
}
encoding, err := options.WebSearch(call)
if err != nil {
return nil, nil, false, fmt.Errorf("%s: %w", call.Path(), err)
}
if encoding == nil {
return nil, nil, false, nil
}
if encoding.Omitted {
return nil, append(encoding.Diagnostics, call.SemanticLoss(
"hosted_web_search_omitted",
"the upstream has no hosted web search for this protocol; the web search tool was omitted",
)), true, nil
}
return encoding.Tool, encoding.Diagnostics, true, nil
}
// appendEncodedWebSearch runs the host encoder for a Responses, Claude, or
// Gemini target. When the encoder handled the definition, its diagnostics and
// tool (if any) are appended and the caller skips its default encoding.
func appendEncodedWebSearch(tools *[]any, diagnostics *[]types.ConversionDiagnostic, call convmeta.WebSearchCall, options *convmeta.Options) (bool, error) {
tool, encoderDiagnostics, handled, err := encodeWebSearch(call, options)
if err != nil || !handled {
return false, err
}
*diagnostics = append(*diagnostics, encoderDiagnostics...)
if tool != nil {
*tools = append(*tools, tool)
}
return true, nil
}
// toolChoiceWithoutToolsDiagnostics reports the tool_choice and
// parallel_tool_calls controls omitted because no tool reached the target;
// upstreams reject either control in a request without tools.
func toolChoiceWithoutToolsDiagnostics(set Set) []types.ConversionDiagnostic {
if set.Choice == nil && set.ParallelAllowed == nil {
return nil
}
return []types.ConversionDiagnostic{presentationLoss("tool_choice", "tool_choice_without_tools",
"the target request has no function tools, so tool_choice and parallel_tool_calls were omitted")}
}
// jsonSchemaParameters returns function parameters as JSON Schema for a
// non-Gemini target, rewriting Gemini's OpenAPI schema subset.
func jsonSchemaParameters(index int, function *Function) (any, []types.ConversionDiagnostic) {
if !function.OpenAPISchema {
return function.Parameters, nil
}
parameters, dropped := sharedgemini.OpenAPISchemaToJSONSchema(function.Parameters)
if len(dropped) == 0 {
return parameters, nil
}
return parameters, []types.ConversionDiagnostic{presentationLoss(
fmt.Sprintf("tools[%d]", index),
"gemini_schema_keyword_dropped",
fmt.Sprintf("JSON Schema has no equivalent for Gemini schema keywords %s; they were dropped", strings.Join(dropped, ", ")),
)}
}
// responsesCustomTool is a Responses custom (freeform) tool sent upstream as
// a function whose only argument is the raw input string.
type responsesCustomTool struct {
name string
description string
}
// inputSchema is the parameters schema of the function that carries a custom
// tool: a single required string argument holding the raw input.
func (tool *responsesCustomTool) inputSchema() map[string]any {
return map[string]any{
"type": "object",
"properties": map[string]any{
convmeta.CustomToolInputArgument: map[string]any{
"type": "string",
"description": "Raw input for the tool.",
},
},
"required": []string{convmeta.CustomToolInputArgument},
"additionalProperties": false,
}
}
func (tool *responsesCustomTool) chatTool() dto.ToolCallRequest {
return dto.ToolCallRequest{
Type: "function",
Function: dto.FunctionRequest{
Name: tool.name,
Description: tool.description,
Parameters: tool.inputSchema(),
},
}
}
const responsesCustomToolType = "custom"
// ResponsesCustomToolNames returns the Responses custom tools that
// AttachRequest sends upstream as functions. The response side restores
// function calls with these names as custom_tool_call items.
func ResponsesCustomToolNames(set Set) map[string]struct{} {
customTools, _ := responsesCustomTools(set)
var names map[string]struct{}
for _, tool := range customTools {
if tool == nil {
continue
}
if names == nil {
names = make(map[string]struct{}, len(customTools))
}
names[tool.name] = struct{}{}
}
return names
}
// ResponsesToolNamespaces maps the upstream name of every tool flattened
// from a non-default Responses tool namespace to that namespace. The response
// side restores the namespace on calls to these tools.
func ResponsesToolNamespaces(set Set) map[string]string {
if set.Source != types.RelayFormatOpenAIResponses {
return nil
}
var namespaces map[string]string
for _, definition := range set.Definitions {
if definition.Namespace == "" || definition.Namespace == convmeta.DefaultResponsesToolNamespace || definition.Name == "" {
continue
}
if namespaces == nil {
namespaces = make(map[string]string)
}
namespaces[definition.Name] = definition.Namespace
}
return namespaces
}
// responsesCustomTools selects, by definition index, the Responses custom
// tools that the upstream receives as functions. A nil entry marks a custom
// tool that was dropped with a diagnostic; indexes without an entry fall back
// to the generic unsupported-tool loss. Request encoding and
// ResponsesCustomToolNames share this selection so the restored names always
// match the functions that were sent.
func responsesCustomTools(set Set) (map[int]*responsesCustomTool, []types.ConversionDiagnostic) {
if set.Source != types.RelayFormatOpenAIResponses {
return nil, nil
}
functionNames := make(map[string]struct{})
for _, definition := range set.Definitions {
if definition.Kind == KindFunction && definition.Function != nil {
functionNames[definition.Function.Name] = struct{}{}
}
}
var (
tools map[int]*responsesCustomTool
diagnostics []types.ConversionDiagnostic
)
for index, definition := range set.Definitions {
if definition.Kind != KindNative || definition.NativeType != responsesCustomToolType {
continue
}
tool := decodeResponsesCustomTool(definition.Raw)
if tool == nil {
continue
}
if tools == nil {
tools = make(map[int]*responsesCustomTool)
}
path := fmt.Sprintf("tools[%d]", index)
if _, exists := functionNames[tool.name]; exists {
tools[index] = nil
diagnostics = append(diagnostics, semanticLoss(path, "custom_tool_name_conflict",
fmt.Sprintf("custom tool %q shares its name with another tool, so the target protocol cannot tell their calls apart", tool.name)))
continue
}
functionNames[tool.name] = struct{}{}
tools[index] = tool
diagnostics = append(diagnostics, presentationLoss(path, "custom_tool_as_function",
fmt.Sprintf("the target protocol has no freeform tools; custom tool %q is sent as a function with one string argument and its input format becomes a description hint", tool.name)))
}
return tools, diagnostics
}
func decodeResponsesCustomTool(raw json.RawMessage) *responsesCustomTool {
var value map[string]any
if err := kitutil.Unmarshal(raw, &value); err != nil {
return nil
}
name := strings.TrimSpace(kitutil.Interface2String(value["name"]))
if name == "" {
return nil
}
parts := make([]string, 0, 3)
if description := strings.TrimSpace(kitutil.Interface2String(value["description"])); description != "" {
parts = append(parts, description)
}
parts = append(parts, `This tool takes freeform text. Put the complete raw text in the "input" argument.`)
if format, ok := value["format"].(map[string]any); ok && strings.TrimSpace(kitutil.Interface2String(format["type"])) == "grammar" {
if definition := strings.TrimSpace(kitutil.Interface2String(format["definition"])); definition != "" {
switch syntax := strings.TrimSpace(kitutil.Interface2String(format["syntax"])); syntax {
case "lark":
parts = append(parts, "The input must match this Lark grammar:\n"+definition)
case "regex":
parts = append(parts, "The input must match this regular expression:\n"+definition)
default:
parts = append(parts, fmt.Sprintf("The input must match this %s grammar:\n%s", syntax, definition))
}
}
}
return &responsesCustomTool{name: name, description: strings.Join(parts, "\n\n")}
}
// responsesCustomToolChoice rewrites a Responses {"type":"custom"} tool choice
// into a named function choice when that custom tool was sent as a function.
// Other choices are returned unchanged.
func responsesCustomToolChoice(choice *Choice, customTools map[int]*responsesCustomTool) *Choice {
if choice == nil || choice.Mode != ChoiceOpaque || choice.NativeType != responsesCustomToolType {
return choice
}
var value map[string]any
if err := kitutil.Unmarshal(choice.Raw, &value); err != nil {
return choice
}
name := convmeta.NamespacedToolName(strings.TrimSpace(kitutil.Interface2String(value["namespace"])), strings.TrimSpace(kitutil.Interface2String(value["name"])))
for _, tool := range customTools {
if tool != nil && tool.name == name {
return &Choice{Mode: ChoiceNamed, Kind: KindFunction, Name: name}
}
}
return choice
}
func attachOpenAIResponsesRequest(request any, set Set, options *convmeta.Options) (any, []types.ConversionDiagnostic, error) {
target, ok := request.(*dto.OpenAIResponsesRequest)
if !ok || target == nil {
return nil, nil, fmt.Errorf("expected OpenAI Responses request, got %T", request)
@@ -120,11 +420,13 @@ func attachOpenAIResponsesRequest(request any, set Set) (any, []types.Conversion
if definition.Function == nil {
continue
}
parameters, schemaDiagnostics := jsonSchemaParameters(index, definition.Function)
diagnostics = append(diagnostics, schemaDiagnostics...)
tool := map[string]any{
"type": "function",
"name": definition.Function.Name,
"description": definition.Function.Description,
"parameters": definition.Function.Parameters,
"parameters": parameters,
}
if definition.Function.Strict != nil {
tool["strict"] = *definition.Function.Strict
@@ -135,6 +437,15 @@ func attachOpenAIResponsesRequest(request any, set Set) (any, []types.Conversion
if set.Source == types.RelayFormatGemini {
diagnostics = append(diagnostics, geminiNativeWebSearchDiagnostics(index, definition, types.RelayFormatOpenAIResponses)...)
}
call := newWebSearchCall(set, index, definition, types.RelayFormatOpenAIResponses)
call.Responses = target
handled, err := appendEncodedWebSearch(&tools, &diagnostics, call, options)
if err != nil {
return nil, diagnostics, err
}
if handled {
continue
}
webSearch := definition.WebSearch
toolType := "web_search"
if set.Source == types.RelayFormatOpenAIResponses && definition.NativeType != "" {
@@ -182,13 +493,19 @@ func attachOpenAIResponsesRequest(request any, set Set) (any, []types.Conversion
))
}
}
if len(tools) > 0 {
encoded, err := kitutil.Marshal(tools)
if err != nil {
return nil, diagnostics, err
}
target.Tools = encoded
historyDiagnostics, err := appendHostedHistoryToOpenAIResponses(target, set)
if err != nil {
return nil, diagnostics, err
}
diagnostics = append(diagnostics, historyDiagnostics...)
if len(tools) == 0 {
return target, append(diagnostics, toolChoiceWithoutToolsDiagnostics(set)...), nil
}
encoded, err := kitutil.Marshal(tools)
if err != nil {
return nil, diagnostics, err
}
target.Tools = encoded
normalizedChoice, allowedChoiceDiagnostics := narrowAllowedFunctionChoice(set.Choice, types.RelayFormatOpenAIResponses)
choice, choiceDiagnostics, err := encodeOpenAIResponsesChoice(normalizedChoice, set.Source)
if err != nil {
@@ -200,11 +517,6 @@ func attachOpenAIResponsesRequest(request any, set Set) (any, []types.Conversion
}
diagnostics = append(diagnostics, allowedChoiceDiagnostics...)
diagnostics = append(diagnostics, choiceDiagnostics...)
historyDiagnostics, err := appendHostedHistoryToOpenAIResponses(target, set)
if err != nil {
return nil, diagnostics, err
}
diagnostics = append(diagnostics, historyDiagnostics...)
return target, diagnostics, nil
}
@@ -214,14 +526,16 @@ func attachClaudeRequest(request any, set Set, options *convmeta.Options) (any,
return nil, nil, fmt.Errorf("expected Claude Messages request, got %T", request)
}
tools := make([]any, 0, len(set.Definitions))
var diagnostics []types.ConversionDiagnostic
customTools, diagnostics := responsesCustomTools(set)
for index, definition := range set.Definitions {
switch definition.Kind {
case KindFunction:
if definition.Function == nil {
continue
}
inputSchema, err := functionParametersMap(definition.Function.Parameters)
parameters, schemaDiagnostics := jsonSchemaParameters(index, definition.Function)
diagnostics = append(diagnostics, schemaDiagnostics...)
inputSchema, err := functionParametersMap(parameters)
if err != nil {
return nil, diagnostics, fmt.Errorf("tools[%d].input_schema: %w", index, err)
}
@@ -235,6 +549,15 @@ func attachClaudeRequest(request any, set Set, options *convmeta.Options) (any,
if set.Source == types.RelayFormatGemini {
diagnostics = append(diagnostics, geminiNativeWebSearchDiagnostics(index, definition, types.RelayFormatClaude)...)
}
call := newWebSearchCall(set, index, definition, types.RelayFormatClaude)
call.Claude = target
handled, err := appendEncodedWebSearch(&tools, &diagnostics, call, options)
if err != nil {
return nil, diagnostics, err
}
if handled {
continue
}
toolType := "web_search_20250305"
if set.Source == types.RelayFormatClaude && isKnownClaudeWebSearchType(definition.NativeType) {
toolType = definition.NativeType
@@ -285,6 +608,16 @@ func attachClaudeRequest(request any, set Set, options *convmeta.Options) (any,
tools = append(tools, tool)
continue
}
if tool, handled := customTools[index]; handled {
if tool != nil {
tools = append(tools, &dto.Tool{
Name: tool.name,
Description: tool.description,
InputSchema: tool.inputSchema(),
})
}
continue
}
diagnostics = append(diagnostics, semanticLoss(
fmt.Sprintf("tools[%d]", index),
"unsupported_hosted_tool",
@@ -292,23 +625,33 @@ func attachClaudeRequest(request any, set Set, options *convmeta.Options) (any,
))
}
}
if len(tools) > 0 {
target.Tools = tools
}
normalizedChoice, allowedChoiceDiagnostics := narrowAllowedFunctionChoice(set.Choice, types.RelayFormatClaude)
choice, choiceDiagnostics := encodeClaudeChoice(normalizedChoice, set.ParallelAllowed, set.Source)
target.ToolChoice = choice
diagnostics = append(diagnostics, allowedChoiceDiagnostics...)
diagnostics = append(diagnostics, choiceDiagnostics...)
historyDiagnostics, err := appendHostedHistoryToClaude(target, set)
if err != nil {
return nil, diagnostics, err
}
diagnostics = append(diagnostics, historyDiagnostics...)
if len(tools) == 0 {
return target, append(diagnostics, toolChoiceWithoutToolsDiagnostics(set)...), nil
}
target.Tools = tools
normalizedChoice, allowedChoiceDiagnostics := narrowAllowedFunctionChoice(responsesCustomToolChoice(set.Choice, customTools), types.RelayFormatClaude)
diagnostics = append(diagnostics, allowedChoiceDiagnostics...)
forced := normalizedChoice != nil && (normalizedChoice.Mode == ChoiceRequired || normalizedChoice.Mode == ChoiceNamed)
manualThinking := target.Thinking != nil && target.Thinking.Type == "enabled"
if forced && manualThinking && set.Source != types.RelayFormatClaude {
downgraded := *normalizedChoice
downgraded.Mode = ChoiceAuto
normalizedChoice = &downgraded
diagnostics = append(diagnostics, semanticLoss("tool_choice", "forced_tool_choice_downgraded",
"Claude rejects forced tool use while manual extended thinking is enabled; tool_choice was sent as auto, so the model may answer without calling a tool"))
}
choice, choiceDiagnostics := encodeClaudeChoice(normalizedChoice, set.ParallelAllowed, set.Source)
target.ToolChoice = choice
diagnostics = append(diagnostics, choiceDiagnostics...)
return target, diagnostics, nil
}
func attachGeminiRequest(request any, set Set) (any, []types.ConversionDiagnostic, error) {
func attachGeminiRequest(request any, set Set, options *convmeta.Options) (any, []types.ConversionDiagnostic, error) {
target, ok := request.(*dto.GeminiChatRequest)
if !ok || target == nil {
return nil, nil, fmt.Errorf("expected Gemini generateContent request, got %T", request)
@@ -334,33 +677,34 @@ func attachGeminiRequest(request any, set Set) (any, []types.ConversionDiagnosti
return target, unsupportedHostedHistoryDiagnostics(types.RelayFormatGemini, set.History), nil
}
var (
functions []map[string]any
tools []map[string]any
diagnostics []types.ConversionDiagnostic
functions []map[string]any
tools []any
)
jsonSchema := options != nil && options.Gemini.SupportsJSONSchemaModel(set.Model)
customTools, diagnostics := responsesCustomTools(set)
for index, definition := range set.Definitions {
switch definition.Kind {
case KindFunction:
if definition.Function == nil {
continue
}
parameters := definition.Function.Parameters
parametersField, parameters := "parameters", definition.Function.Parameters
if parameters != nil {
cloned, err := kitutil.Any2Type[any](parameters)
if err != nil {
return nil, diagnostics, fmt.Errorf("tools[%d].parameters: %w", index, err)
}
if params, ok := cloned.(map[string]any); ok {
if properties, exists := params["properties"].(map[string]any); exists && len(properties) == 0 {
cloned = nil
}
var misfits []string
parametersField, parameters, misfits = sharedgemini.FunctionParameters(cloned, jsonSchema)
if len(misfits) > 0 {
diagnostics = append(diagnostics, presentationLoss(fmt.Sprintf("tools[%d].parameters", index), "gemini_schema_keyword_dropped",
fmt.Sprintf("Gemini function parameters cannot carry %s", strings.Join(misfits, ", "))))
}
parameters = sharedgemini.CleanFunctionParameters(cloned)
}
function := map[string]any{
"name": definition.Function.Name,
"description": definition.Function.Description,
"parameters": parameters,
"name": definition.Function.Name,
"description": definition.Function.Description,
parametersField: parameters,
}
deleteEmptyStrings(function)
functions = append(functions, function)
@@ -368,6 +712,15 @@ func attachGeminiRequest(request any, set Set) (any, []types.ConversionDiagnosti
diagnostics = append(diagnostics, presentationLoss(fmt.Sprintf("tools[%d].strict", index), "unsupported_function_strict", "Gemini does not expose OpenAI function strictness"))
}
case KindWebSearch:
call := newWebSearchCall(set, index, definition, types.RelayFormatGemini)
call.Gemini = target
handled, err := appendEncodedWebSearch(&tools, &diagnostics, call, options)
if err != nil {
return nil, diagnostics, err
}
if handled {
continue
}
if set.Source == types.RelayFormatGemini && len(definition.Raw) > 0 {
var tool map[string]any
if err := kitutil.Unmarshal(definition.Raw, &tool); err != nil {
@@ -401,19 +754,18 @@ func attachGeminiRequest(request any, set Set) (any, []types.ConversionDiagnosti
tools = append(tools, tool)
continue
}
// The established Responses-to-Gemini compatibility path removes
// free-form/unknown tools together with custom call history in
// PrepareOpenAIResponsesRequest. Keep that explicit downgrade as a
// diagnostic; other opaque tools may be server-executed and remain a
// semantic loss under the default Safe policy.
if set.Source == types.RelayFormatOpenAIResponses && (definition.NativeType == "custom" || definition.NativeType == "unknown") {
diagnostics = append(diagnostics, presentationLoss(
fmt.Sprintf("tools[%d]", index),
"custom_tool_omitted",
"Gemini cannot represent this OpenAI free-form or unknown tool; its preprocessed call history and definition were omitted",
))
if tool, handled := customTools[index]; handled {
if tool != nil {
functions = append(functions, map[string]any{
"name": tool.name,
"description": tool.description,
"parameters": sharedgemini.CleanFunctionParameters(tool.inputSchema()),
})
}
continue
}
// Other opaque tools may be server-executed and remain a semantic
// loss under the default Safe policy.
if set.Source == types.RelayFormatOpenAIResponses && definition.Kind == KindNative {
diagnostics = append(diagnostics, semanticLoss(
fmt.Sprintf("tools[%d]", index),
@@ -439,7 +791,11 @@ func attachGeminiRequest(request any, set Set) (any, []types.ConversionDiagnosti
}
target.Tools = encoded
}
config, choiceDiagnostics := encodeGeminiChoice(set.Choice)
if len(functions) == 0 {
diagnostics = append(diagnostics, unsupportedHostedHistoryDiagnostics(types.RelayFormatGemini, set.History)...)
return target, append(diagnostics, toolChoiceWithoutToolsDiagnostics(set)...), nil
}
config, choiceDiagnostics := encodeGeminiChoice(responsesCustomToolChoice(set.Choice, customTools))
target.ToolConfig = config
diagnostics = append(diagnostics, choiceDiagnostics...)
if set.ParallelAllowed != nil && !*set.ParallelAllowed {
relaykit/relayconvert/internal/toolconv/model.go
+21 -19
View File
@@ -3,6 +3,7 @@ package toolconv
import (
"encoding/json"
"github.com/QuantumNous/new-api/relaykit/relayconvert/convmeta"
"github.com/QuantumNous/new-api/relaykit/types"
)
@@ -33,26 +34,18 @@ type Function struct {
Description string
Parameters any
Strict *bool
// OpenAPISchema marks Parameters written in Gemini's OpenAPI schema
// subset (uppercase type names, nullable, propertyOrdering) rather than
// JSON Schema.
OpenAPISchema bool
}
type ApproximateLocation struct {
City string
Region string
Country string
Timezone string
}
type WebSearch struct {
Location *ApproximateLocation
AllowedDomains []string
BlockedDomains []string
SearchContextSize string
MaxUses *int
AllowedCallers []string
ResponseInclusion string
ExternalWebAccess *bool
ReturnTokenBudget json.RawMessage
}
// ApproximateLocation and WebSearch are the convmeta types, so a host
// web-search encoder receives the decoded specification unchanged.
type (
ApproximateLocation = convmeta.ApproximateLocation
WebSearch = convmeta.WebSearch
)
type Definition struct {
Kind Kind
@@ -63,6 +56,9 @@ type Definition struct {
WebSearch *WebSearch
Raw json.RawMessage
Group int
// Namespace is the Responses tool namespace the definition was declared
// in. Name and Function.Name already hold the flattened upstream name.
Namespace string
}
type ChoiceMode string
@@ -92,10 +88,16 @@ type Set struct {
ParallelAllowed *bool
NativeToolConfig json.RawMessage
History []HostedHistoryItem
// Diagnostics reports losses found while extracting the definitions; they
// are returned with the attach diagnostics.
Diagnostics []types.ConversionDiagnostic
// Model is the upstream model the definitions are attached for; target
// capability checks read it. The caller of AttachRequest sets it.
Model string
}
func (s Set) Empty() bool {
return len(s.Definitions) == 0 && s.Choice == nil && s.ParallelAllowed == nil && len(s.NativeToolConfig) == 0 && len(s.History) == 0
return len(s.Definitions) == 0 && s.Choice == nil && s.ParallelAllowed == nil && len(s.NativeToolConfig) == 0 && len(s.History) == 0 && len(s.Diagnostics) == 0
}
type HostedHistoryItem struct {
relaykit/relayconvert/internal/toolconv/policy_test.go
+178
View File
@@ -101,3 +101,181 @@ func TestSafePolicyRejectsRequestPhaseHostedToolLoss(t *testing.T) {
assert.True(t, hasDiagnosticCode(loss.Diagnostics, "unsupported_hosted_tool"))
assert.True(t, hasDiagnosticCode(diagnostics, "unsupported_hosted_tool"))
}
func codexResponsesToolsRequest(t *testing.T, toolChoice any, extraTools ...map[string]any) *dto.OpenAIResponsesRequest {
t.Helper()
tools := []map[string]any{
{
"type": "custom",
"name": "exec",
"description": "Run code.",
"format": map[string]any{"type": "grammar", "syntax": "lark", "definition": "start: /.+/"},
},
{"type": "function", "name": "wait", "parameters": map[string]any{"type": "object"}},
}
tools = append(tools, extraTools...)
rawTools, err := kitutil.Marshal(tools)
require.NoError(t, err)
request := &dto.OpenAIResponsesRequest{Model: "gpt-test", Tools: rawTools}
if toolChoice != nil {
request.ToolChoice, err = kitutil.Marshal(toolChoice)
require.NoError(t, err)
}
return request
}
// responsesCustomToolTargets are the upstream protocols that receive a
// Responses custom tool as a function taking one string argument.
var responsesCustomToolTargets = []struct {
format types.RelayFormat
request func() any
}{
{types.RelayFormatOpenAI, func() any { return &dto.GeneralOpenAIRequest{Model: "gpt-test"} }},
{types.RelayFormatClaude, func() any { return &dto.ClaudeRequest{Model: "claude-test"} }},
{types.RelayFormatGemini, func() any { return &dto.GeminiChatRequest{} }},
}
// attachedFunctions returns the function declarations and the tool choice
// that AttachRequest wrote into a Chat, Claude, or Gemini request, decoded
// from their wire JSON.
func attachedFunctions(t *testing.T, out any) ([]map[string]any, any) {
t.Helper()
var (
functions []map[string]any
choice any
err error
)
switch target := out.(type) {
case *dto.GeneralOpenAIRequest:
for _, tool := range target.Tools {
function, err := kitutil.Any2Type[map[string]any](tool.Function)
require.NoError(t, err)
functions = append(functions, function)
}
choice, err = kitutil.Any2Type[any](target.ToolChoice)
case *dto.ClaudeRequest:
functions, err = kitutil.Any2Type[[]map[string]any](target.Tools)
require.NoError(t, err)
choice, err = kitutil.Any2Type[any](target.ToolChoice)
case *dto.GeminiChatRequest:
var groups []struct {
FunctionDeclarations []map[string]any `json:"functionDeclarations"`
}
require.NoError(t, kitutil.Unmarshal(target.Tools, &groups))
require.Len(t, groups, 1)
functions = groups[0].FunctionDeclarations
choice, err = kitutil.Any2Type[any](target.ToolConfig)
default:
require.FailNow(t, "unexpected request type", "%T", out)
}
require.NoError(t, err)
return functions, choice
}
func TestResponsesCustomToolReachesEachTargetAsStringInputFunction(t *testing.T) {
t.Parallel()
const execDescription = "Run code.\n\nThis tool takes freeform text. Put the complete raw text in the \"input\" argument.\n\nThe input must match this Lark grammar:\nstart: /.+/"
inputSchema := map[string]any{
"type": "object",
"properties": map[string]any{
"input": map[string]any{"type": "string", "description": "Raw input for the tool."},
},
"required": []any{"input"},
"additionalProperties": false,
}
expected := map[types.RelayFormat]struct {
functions []map[string]any
choice any
}{
types.RelayFormatOpenAI: {
functions: []map[string]any{
{"name": "exec", "description": execDescription, "parameters": inputSchema},
{"name": "wait", "parameters": map[string]any{"type": "object"}},
},
choice: map[string]any{"type": "function", "function": map[string]any{"name": "exec"}},
},
types.RelayFormatClaude: {
functions: []map[string]any{
{"name": "exec", "description": execDescription, "input_schema": inputSchema},
{"name": "wait", "input_schema": map[string]any{"type": "object", "properties": map[string]any{}}},
},
choice: map[string]any{"type": "tool", "name": "exec"},
},
// Gemini receives the cleaned OpenAPI schema without additionalProperties.
types.RelayFormatGemini: {
functions: []map[string]any{
{"name": "exec", "description": execDescription, "parameters": map[string]any{
"type": "OBJECT",
"properties": map[string]any{
"input": map[string]any{"type": "STRING", "description": "Raw input for the tool."},
},
"required": []any{"input"},
}},
{"name": "wait", "parameters": map[string]any{"type": "OBJECT"}},
},
choice: map[string]any{"functionCallingConfig": map[string]any{"mode": "ANY", "allowedFunctionNames": []any{"exec"}}},
},
}
_, set, err := ExtractRequest(types.RelayFormatOpenAIResponses, codexResponsesToolsRequest(t, map[string]any{"type": "custom", "name": "exec"}))
require.NoError(t, err)
assert.Equal(t, map[string]struct{}{"exec": {}}, ResponsesCustomToolNames(set))
for _, target := range responsesCustomToolTargets {
t.Run(string(target.format), func(t *testing.T) {
out, diagnostics, err := AttachRequest(target.format, target.request(), set, &convmeta.Options{ToolLossPolicy: types.ConversionLossPolicySafe})
require.NoError(t, err)
functions, choice := attachedFunctions(t, out)
assert.Equal(t, expected[target.format].functions, functions)
assert.Equal(t, expected[target.format].choice, choice)
assert.True(t, hasDiagnosticCode(diagnostics, "custom_tool_as_function"))
assert.False(t, hasDiagnosticCode(diagnostics, "unsupported_hosted_tool"))
assert.False(t, hasDiagnosticCode(diagnostics, "unsupported_tool_choice"))
_, _, err = AttachRequest(target.format, target.request(), set, &convmeta.Options{ToolLossPolicy: types.ConversionLossPolicyStrict})
var loss *types.ConversionLossError
require.ErrorAs(t, err, &loss)
assert.True(t, hasDiagnosticCode(loss.Diagnostics, "custom_tool_as_function"))
})
}
}
func TestResponsesCustomToolNameConflictIsDropped(t *testing.T) {
t.Parallel()
// The custom exec loses to the function exec, so a choice naming it has no
// sent tool to point at and must stay unconverted.
request := codexResponsesToolsRequest(t, map[string]any{"type": "custom", "name": "exec"},
map[string]any{"type": "function", "name": "exec", "parameters": map[string]any{"type": "object"}},
map[string]any{"type": "custom", "name": "apply_patch"},
map[string]any{"type": "custom", "name": "apply_patch", "description": "duplicate"},
)
_, set, err := ExtractRequest(types.RelayFormatOpenAIResponses, request)
require.NoError(t, err)
assert.Equal(t, map[string]struct{}{"apply_patch": {}}, ResponsesCustomToolNames(set))
for _, target := range responsesCustomToolTargets {
t.Run(string(target.format), func(t *testing.T) {
out, diagnostics, err := AttachRequest(target.format, target.request(), set, &convmeta.Options{})
require.NoError(t, err)
functions, choice := attachedFunctions(t, out)
require.Len(t, functions, 3)
names := make([]string, 0, len(functions))
for _, function := range functions {
names = append(names, function["name"].(string))
}
assert.Equal(t, []string{"wait", "exec", "apply_patch"}, names)
// Only the custom definitions carry a description, so the kept exec
// is the function one.
assert.NotContains(t, functions[1], "description")
assert.NotContains(t, functions[2]["description"], "duplicate")
assert.Nil(t, choice)
assert.True(t, hasDiagnosticCode(diagnostics, "custom_tool_name_conflict"))
assert.True(t, hasDiagnosticCode(diagnostics, "unsupported_tool_choice"))
_, _, err = AttachRequest(target.format, target.request(), set, &convmeta.Options{ToolLossPolicy: types.ConversionLossPolicySafe})
require.Error(t, err)
})
}
}
relaykit/relayconvert/kitutil/mask.go
+138 -48
View File
@@ -1,15 +1,25 @@
package kitutil
import (
"maps"
"net"
"net/url"
"regexp"
"slices"
"strings"
)
var (
maskURLPattern = regexp.MustCompile(`(http|https)://[^\s/$.?#].[^\s]*`)
maskDomainPattern = regexp.MustCompile(`\b(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\.)+[a-zA-Z]{2,}\b`)
maskIPPattern = regexp.MustCompile(`\b(?:\d{1,3}\.){3}\d{1,3}\b`)
// maskURLPattern stops at characters a URL cannot contain unescaped, such
// as double quotes and angle or curly brackets, so a URL quoted in an error
// keeps its delimiters. Apostrophes and parentheses are valid inside a URL;
// the caller trims them, like other punctuation, only when they trail it.
maskURLPattern = regexp.MustCompile(`(?:https?|wss?)://[^\s"<>{}]+`)
maskIPPattern = regexp.MustCompile(`\b(?:\d{1,3}\.){3}\d{1,3}\b`)
// maskBracketedIPv6Pattern matches the bracketed form Go network errors use
// for IPv6 addresses, as in "dial tcp [2001:db8::1]:443", with an optional
// zone; the caller confirms the content parses as an IP.
maskBracketedIPv6Pattern = regexp.MustCompile(`\[([0-9A-Fa-f:.]*:[0-9A-Fa-f:.]*)(%[^\]\s]*)?\]`)
// maskApiKeyPattern matches patterns like 'api_key:xxx' or "api_key:xxx" to mask the API key value
maskApiKeyPattern = regexp.MustCompile(`(['"]?)api_key:([^\s'"]+)(['"]?)`)
)
@@ -53,30 +63,111 @@ func maskHostForPlainDomain(domain string) string {
return stars + "." + strings.Join(tail, ".")
}
// MaskSensitiveInfo masks sensitive information like URLs, IPs, and domain names in a string
// Example:
// http://example.com -> http://***.com
// https://api.test.org/v1/users/123?key=secret -> https://***.org/***/***/?key=***
// https://sub.domain.co.uk/path/to/resource -> https://***.co.uk/***/***
// 192.168.1.1 -> ***.***.***.***
// openai.com -> ***.com
// www.openai.com -> ***.***.com
// api.openai.com -> ***.***.com
func MaskSensitiveInfo(str string) string {
// Mask URLs
str = maskURLPattern.ReplaceAllStringFunc(str, func(urlStr string) string {
// maskURLQuery keeps the parameter names of a raw query and masks the values.
// Example: key=secret&alt=sse -> ?alt=***&key=***
func maskURLQuery(rawQuery string) string {
if rawQuery == "" {
return ""
}
values, err := url.ParseQuery(rawQuery)
if err != nil {
// If can't parse query, just mask the whole query string
return "?***"
}
if len(values) == 0 {
return ""
}
maskedParams := make([]string, 0, len(values))
for _, key := range slices.Sorted(maps.Keys(values)) {
maskedParams = append(maskedParams, key+"=***")
}
return "?" + strings.Join(maskedParams, "&")
}
// MaskSensitiveInfo masks the upstream addresses in str and leaves every other
// URL and domain readable. upstreamURLs are the channel base URL and the URL
// the gateway actually requested, which differs for routes such as absolute
// Advanced Custom paths; empty entries are skipped. Credentials and IPv4
// addresses are masked regardless of the upstream: API key values, the query
// values and userinfo of every URL, which carry secrets such as client_secret
// or key on provider endpoints, and IPs, which in network errors include the
// gateway's own address.
// Example with upstream https://api.upstream.example.com:
// https://api.upstream.example.com/v1/chat?key=secret -> https://***.com/***/***?key=***
// lookup api.upstream.example.com -> lookup ***.***.***.com
// dial tcp 192.168.1.1:443 -> dial tcp ***.***.***.***:443
// https://docs.example.org/errors -> unchanged
// https://auth.example.org/token?client_secret=s -> https://auth.example.org/token?client_secret=***
func MaskSensitiveInfo(str string, upstreamURLs ...string) string {
// Mask API keys (e.g., "api_key:test-api-key-for-redaction" -> "api_key:***")
str = maskApiKeyPattern.ReplaceAllString(str, "${1}api_key:***${3}")
upstreamHosts := make([]string, 0, len(upstreamURLs))
for _, upstreamURL := range upstreamURLs {
upstreamURL = strings.TrimSpace(upstreamURL)
if upstreamURL == "" {
continue
}
if !strings.Contains(upstreamURL, "://") {
// A base URL saved without a scheme would otherwise parse as a path.
upstreamURL = "//" + upstreamURL
}
parsed, err := url.Parse(upstreamURL)
if err != nil {
continue
}
// A fully qualified host keeps its trailing root dot; errors may omit it.
if host := strings.TrimSuffix(parsed.Hostname(), "."); host != "" {
upstreamHosts = append(upstreamHosts, host)
}
}
// Mask URLs on an upstream host, and the query values of every other URL
str = maskURLPattern.ReplaceAllStringFunc(str, func(match string) string {
urlStr := match
for urlStr != "" {
last := urlStr[len(urlStr)-1]
if strings.IndexByte(".,;:!?'", last) < 0 && (last != ')' || strings.Count(urlStr, "(") >= strings.Count(urlStr, ")")) {
break
}
urlStr = urlStr[:len(urlStr)-1]
}
suffix := match[len(urlStr):]
u, err := url.Parse(urlStr)
if err != nil {
return urlStr
return match
}
host := strings.TrimSuffix(u.Hostname(), ".")
if !slices.ContainsFunc(upstreamHosts, func(upstreamHost string) bool { return strings.EqualFold(upstreamHost, host) }) {
if u.RawQuery == "" && u.User == nil {
return match
}
// Userinfo is a credential wherever the URL points.
if u.User != nil {
scheme, afterScheme, _ := strings.Cut(urlStr, "://")
authorityEnd := strings.IndexAny(afterScheme, "/?#")
if authorityEnd < 0 {
authorityEnd = len(afterScheme)
}
urlStr = scheme + "://***@" + afterScheme[strings.LastIndex(afterScheme[:authorityEnd], "@")+1:]
}
if u.RawQuery == "" {
return urlStr + suffix
}
beforeQuery, _, _ := strings.Cut(urlStr, "?")
fragment := ""
if u.Fragment != "" {
fragment = "#" + u.EscapedFragment()
}
return beforeQuery + maskURLQuery(u.RawQuery) + fragment + suffix
}
host := u.Host
if host == "" {
return urlStr
}
// Mask host with unified logic
maskedHost := maskHostForURL(host)
if ip := net.ParseIP(host); ip != nil && ip.To4() != nil {
maskedHost = "***.***.***.***"
}
if port := u.Port(); port != "" {
maskedHost += ":" + port
}
result := u.Scheme + "://" + maskedHost
@@ -96,36 +187,35 @@ func MaskSensitiveInfo(str string) string {
result += "/"
}
// Mask query parameters
if u.RawQuery != "" {
values, err := url.ParseQuery(u.RawQuery)
if err != nil {
// If can't parse query, just mask the whole query string
result += "?***"
} else {
maskedParams := make([]string, 0, len(values))
for key := range values {
maskedParams = append(maskedParams, key+"=***")
}
if len(maskedParams) > 0 {
result += "?" + strings.Join(maskedParams, "&")
}
}
return result + maskURLQuery(u.RawQuery) + suffix
})
// Mask IP addresses: in network errors they are the resolved upstream
// address or the gateway's own, so they are masked even without an
// upstream address, and an IP upstream host is covered too. IPv6 goes
// first so an embedded IPv4 such as [::ffff:10.0.0.1] still parses.
str = maskBracketedIPv6Pattern.ReplaceAllStringFunc(str, func(match string) string {
if net.ParseIP(maskBracketedIPv6Pattern.FindStringSubmatch(match)[1]) == nil {
return match
}
return result
return "[***]"
})
// Mask domain names without protocol (like openai.com, www.openai.com)
str = maskDomainPattern.ReplaceAllStringFunc(str, func(domain string) string {
return maskHostForPlainDomain(domain)
})
// Mask IP addresses
str = maskIPPattern.ReplaceAllString(str, "***.***.***.***")
if len(upstreamHosts) == 0 {
return str
}
// Mask API keys (e.g., "api_key:test-api-key-for-redaction" -> "api_key:***")
str = maskApiKeyPattern.ReplaceAllString(str, "${1}api_key:***${3}")
// Mask an upstream host where it appears without a scheme, as in
// "dial tcp: lookup api.example.com: no such host". Longer hosts go first
// so a subdomain upstream is masked whole before its parent base host.
slices.SortFunc(upstreamHosts, func(a, b string) int { return len(b) - len(a) })
for _, host := range upstreamHosts {
if net.ParseIP(host) != nil {
continue
}
hostPattern := regexp.MustCompile(`(?i)\b` + regexp.QuoteMeta(host) + `\b`)
str = hostPattern.ReplaceAllStringFunc(str, maskHostForPlainDomain)
}
return str
}
relaykit/relayconvert/kitutil/mask_test.go
+136
View File
@@ -0,0 +1,136 @@
package kitutil
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestMaskSensitiveInfoMasksOnlyTheUpstreamAddress(t *testing.T) {
tests := []struct {
name string
upstreamURLs []string
input string
want string
}{
{
name: "no upstream address leaves hosts readable but masks IPs",
upstreamURLs: []string{"", ""},
input: `Post "https://api.upstream.example.com/v1/chat/completions": dial tcp 10.0.0.1:443: connect: connection refused`,
want: `Post "https://api.upstream.example.com/v1/chat/completions": dial tcp ***.***.***.***:443: connect: connection refused`,
},
{
name: "bracketed IPv6 addresses are masked without an upstream address",
upstreamURLs: nil,
input: `dial tcp [2a01:4f8::1]:443: i/o timeout; lookup api.example.com on [::1]:53; dial tcp [fe80::1%eth0]:443; dial tcp [::ffff:10.0.0.1]:443; items[0] and [1:2] stay`,
want: `dial tcp [***]:443: i/o timeout; lookup api.example.com on [***]:53; dial tcp [***]:443; dial tcp [***]:443; items[0] and [1:2] stay`,
},
{
name: "IPv6 upstream base URL in a WebSocket dial error",
upstreamURLs: []string{"http://[2001:db8::1]:8080"},
input: `dial failed to ws://[2001:db8::1]:8080/v1/realtime: dial tcp [2001:db8::1]:8080: connect: connection refused`,
want: `dial failed to ws://***:8080/***/***: dial tcp [***]:8080: connect: connection refused`,
},
{
name: "no upstream address still masks API key values",
upstreamURLs: nil,
input: `invalid api_key:test-api-key-for-redaction`,
want: `invalid api_key:***`,
},
{
name: "query values carry credentials on any host",
upstreamURLs: []string{"https://custom.base.example"},
input: `Post "https://aip.baidubce.com/oauth/2.0/token?grant_type=client_credentials&client_id=id-1&client_secret=secret-1": dial tcp: i/o timeout, see https://docs.example.org/errors?lang=en#retry`,
want: `Post "https://aip.baidubce.com/oauth/2.0/token?client_id=***&client_secret=***&grant_type=***": dial tcp: i/o timeout, see https://docs.example.org/errors?lang=***#retry`,
},
{
name: "parentheses inside a URL keep its query masked",
upstreamURLs: nil,
input: `failed to download https://cdn.example/image(1).png?token=secret (status 403)`,
want: `failed to download https://cdn.example/image(1).png?token=*** (status 403)`,
},
{
name: "apostrophes inside a URL keep its query and userinfo masked",
upstreamURLs: nil,
input: `failed to download https://cdn.example/John's-photo.png?token=secret from 'https://o'neil:pw@cdn.example/a?sig=1'`,
want: `failed to download https://cdn.example/John's-photo.png?token=*** from 'https://***@cdn.example/a?sig=***'`,
},
{
name: "userinfo is masked on any host",
upstreamURLs: nil,
input: `downloading https://alice:secret@cdn.example/file.png and https://bob@cdn.example/a?sig=1`,
want: `downloading https://***@cdn.example/file.png and https://***@cdn.example/a?sig=***`,
},
{
name: "query values are masked without an upstream address",
upstreamURLs: nil,
input: `request to https://generativelanguage.googleapis.com/v1beta/models?key=AIza-secret failed`,
want: `request to https://generativelanguage.googleapis.com/v1beta/models?key=*** failed`,
},
{
name: "upstream URL, its IP and other URLs",
upstreamURLs: []string{"https://api.upstream.example.com/openai"},
input: `request to https://api.upstream.example.com/openai/v1/chat/completions?key=secret failed: dial tcp 10.0.0.1:443: i/o timeout, see https://docs.example.org/errors`,
want: `request to https://***.com/***/***/***/***?key=*** failed: dial tcp ***.***.***.***:443: i/o timeout, see https://docs.example.org/errors`,
},
{
name: "bare upstream host in a DNS error",
upstreamURLs: []string{"https://API.Upstream.example.com"},
input: `dial tcp: lookup api.upstream.example.com: no such host`,
want: `dial tcp: lookup ***.***.***.com: no such host`,
},
{
name: "fully qualified base URL with a trailing dot",
upstreamURLs: []string{"https://private.example.com./v1"},
input: `request to https://private.example.com./v1/responses failed: dial tcp: lookup private.example.com.: no such host`,
want: `request to https://***.com/***/*** failed: dial tcp: lookup ***.***.com.: no such host`,
},
{
name: "base URL saved without a scheme",
upstreamURLs: []string{"relay.example.co.uk"},
input: `request to https://relay.example.co.uk/v1/messages failed: EOF`,
want: `request to https://***.co.uk/***/*** failed: EOF`,
},
{
name: "quoted URL keeps its delimiters and trailing punctuation",
upstreamURLs: []string{"https://api.upstream.example.com"},
input: `Post "https://api.upstream.example.com/v1/chat/completions": EOF; retry https://api.upstream.example.com/v1/models.`,
want: `Post "https://***.com/***/***/***": EOF; retry https://***.com/***/***.`,
},
{
name: "port does not hide a country-code TLD",
upstreamURLs: []string{"https://relay.example.co.uk:8443"},
input: `request to https://relay.example.co.uk:8443/v1/messages failed`,
want: `request to https://***.co.uk:8443/***/*** failed`,
},
{
name: "IP upstream host is fully masked",
upstreamURLs: []string{"http://10.0.0.1:8080"},
input: `request to http://10.0.0.1:8080/v1/chat failed`,
want: `request to http://***.***.***.***:8080/***/*** failed`,
},
{
name: "requested origin of an absolute route besides the base URL",
upstreamURLs: []string{"https://api.upstream.example.com", "wss://realtime.private.example.net"},
input: `dial failed to wss://realtime.private.example.net/v1/responses: dial tcp: lookup realtime.private.example.net: no such host`,
want: `dial failed to wss://***.net/***/***: dial tcp: lookup ***.***.***.net: no such host`,
},
{
name: "subdomain upstream is masked whole before its parent base host",
upstreamURLs: []string{"https://api.example.com", "wss://tenant.api.example.com"},
input: `dial tcp: lookup tenant.api.example.com: no such host`,
want: `dial tcp: lookup ***.***.***.com: no such host`,
},
{
name: "other domains and dotted identifiers stay readable",
upstreamURLs: []string{"https://api.upstream.example.com"},
input: `Unknown event response.output_text.delta from model gpt-4.1, contact support@example.org`,
want: `Unknown event response.output_text.delta from model gpt-4.1, contact support@example.org`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.want, MaskSensitiveInfo(tt.input, tt.upstreamURLs...))
})
}
}
relaykit/relayconvert/reasoning/claude.go
+138 -27
View File
@@ -26,15 +26,25 @@ type claudeCapabilities struct {
supportsManual bool
defaultThinking bool
supportsDisable bool
supportsEffort bool
supportsXHigh bool
supportsMax bool
strictSampling bool
// disableAsBetweenTools marks models whose lowest thinking setting is
// {"type":"between_tools"} instead of {"type":"disabled"}.
disableAsBetweenTools bool
supportsEffort bool
supportsXHigh bool
supportsMax bool
strictSampling bool
// offEffortAtMostHigh marks models that accept effort high or below only
// while thinking is off (disabled or between_tools).
offEffortAtMostHigh bool
// defaultEffort is the effort the model runs at when a request sets none:
// medium on Claude Opus 5.5 and Haiku 5.5 and high on every other model
// (https://platform.claude.com/docs/en/build-with-claude/effort).
defaultEffort Effort
}
func claudeCapabilitiesFor(model string) claudeCapabilities {
model = strings.ToLower(model)
capabilities := claudeCapabilities{supportsManual: true, supportsDisable: true}
capabilities := claudeCapabilities{supportsManual: true, supportsDisable: true, defaultEffort: EffortHigh}
switch {
case strings.HasPrefix(model, "claude-fable-5"),
@@ -43,6 +53,7 @@ func claudeCapabilitiesFor(model string) claudeCapabilities {
capabilities.supportsManual = false
capabilities.defaultThinking = true
capabilities.supportsDisable = false
capabilities.supportsEffort = true
capabilities.supportsXHigh = true
capabilities.supportsMax = true
capabilities.strictSampling = true
@@ -50,17 +61,41 @@ func claudeCapabilitiesFor(model string) claudeCapabilities {
capabilities.adaptive = true
capabilities.defaultThinking = true
capabilities.supportsDisable = false
capabilities.supportsEffort = true
capabilities.supportsMax = true
capabilities.strictSampling = true
case strings.HasPrefix(model, "claude-opus-5-5"),
strings.HasPrefix(model, "claude-sonnet-5-5"):
// Opus 5.5 cannot turn thinking off at all; Sonnet 5.5 turns it off
// with between_tools, which takes effort high or below only.
capabilities.adaptive = true
capabilities.supportsManual = false
capabilities.defaultThinking = true
capabilities.supportsDisable = false
capabilities.disableAsBetweenTools = strings.HasPrefix(model, "claude-sonnet-5-5")
capabilities.offEffortAtMostHigh = capabilities.disableAsBetweenTools
if strings.HasPrefix(model, "claude-opus-5-5") {
capabilities.defaultEffort = EffortMedium
}
capabilities.supportsEffort = true
capabilities.supportsXHigh = true
capabilities.supportsMax = true
capabilities.strictSampling = true
case strings.HasPrefix(model, "claude-opus-5"),
strings.HasPrefix(model, "claude-sonnet-5"),
strings.HasPrefix(model, "claude-haiku-5"),
strings.HasPrefix(model, "claude-opus-4-8"),
strings.HasPrefix(model, "claude-opus-4-7"):
capabilities.adaptive = true
capabilities.supportsManual = false
if strings.HasPrefix(model, "claude-opus-5") || strings.HasPrefix(model, "claude-sonnet-5") {
if strings.HasPrefix(model, "claude-opus-5") || strings.HasPrefix(model, "claude-sonnet-5") || strings.HasPrefix(model, "claude-haiku-5") {
capabilities.defaultThinking = true
}
// Opus 5 and Haiku 5.5 reject disabled thinking with xhigh or max.
capabilities.offEffortAtMostHigh = strings.HasPrefix(model, "claude-opus-5") || strings.HasPrefix(model, "claude-haiku-5")
if strings.HasPrefix(model, "claude-haiku-5") {
capabilities.defaultEffort = EffortMedium
}
capabilities.supportsEffort = true
capabilities.supportsXHigh = true
capabilities.supportsMax = true
@@ -97,12 +132,6 @@ func RenderClaude(model string, intent Intent, maxTokens *uint, adapterBudgetPer
diagnostics = append(diagnostics, normalizeDiagnostics...)
}
capabilities := claudeCapabilitiesFor(model)
if disabledWithEffort {
diagnostics = append(diagnostics, claudeReasoningDiagnostic(
"claude_disabled_effort_ignored",
fmt.Sprintf("model %q cannot apply effort %q while thinking is disabled; the effort was ignored", model, intent.Effort),
))
}
if !intent.HasStrength() {
if intent.IncludeThoughts != nil && capabilities.adaptive && capabilities.defaultThinking {
thinking := &dto.Thinking{Type: "adaptive"}
@@ -113,17 +142,43 @@ func RenderClaude(model string, intent Intent, maxTokens *uint, adapterBudgetPer
}
return ClaudeRender{
Thinking: thinking,
EffectiveEffort: EffortHigh,
EffectiveEffort: capabilities.defaultEffort,
ClearSampling: capabilities.strictSampling,
}, nil
}
if capabilities.defaultThinking {
return ClaudeRender{EffectiveEffort: EffortHigh, ClearSampling: capabilities.strictSampling}, nil
return ClaudeRender{EffectiveEffort: capabilities.defaultEffort, ClearSampling: capabilities.strictSampling}, nil
}
return ClaudeRender{ClearSampling: capabilities.strictSampling}, nil
}
if intent.Mode == ModeDisabled || intent.Effort == EffortNone {
// Effort works without thinking, so a requested level is sent at one
// the model accepts while thinking is off.
outputEffort := Effort("")
if disabledWithEffort && (capabilities.supportsDisable || capabilities.disableAsBetweenTools) {
outputEffort = ClaudeEffort(model, intent.Effort)
if capabilities.offEffortAtMostHigh && (outputEffort == EffortXHigh || outputEffort == EffortMax) {
outputEffort = EffortHigh
}
diagnostics = append(diagnostics, claudeEffortDiagnostics(model, intent.Effort, outputEffort)...)
}
if capabilities.disableAsBetweenTools {
// between_tools takes no other thinking field and runs at the
// default effort or the requested one, both within its "high or
// below" limit.
diagnostics = append(diagnostics, claudeReasoningDiagnostic(
"claude_thinking_disable_as_between_tools",
fmt.Sprintf("model %q rejects disabled thinking; using between_tools, its lowest thinking setting", model),
))
return ClaudeRender{
Thinking: &dto.Thinking{Type: "between_tools"},
OutputEffort: outputEffort,
EffectiveEffort: EffortNone,
ClearSampling: capabilities.strictSampling,
Diagnostics: diagnostics,
}, nil
}
if !capabilities.supportsDisable {
diagnostics = append(diagnostics, claudeReasoningDiagnostic(
"claude_thinking_disable_unsupported",
@@ -138,8 +193,7 @@ func RenderClaude(model string, intent Intent, maxTokens *uint, adapterBudgetPer
thinking.Display = "omitted"
}
}
outputEffort := Effort("")
effectiveEffort := EffortHigh
effectiveEffort := capabilities.defaultEffort
if capabilities.supportsEffort {
outputEffort = EffortLow
effectiveEffort = EffortLow
@@ -156,6 +210,7 @@ func RenderClaude(model string, intent Intent, maxTokens *uint, adapterBudgetPer
}
return ClaudeRender{
Thinking: &dto.Thinking{Type: "disabled"},
OutputEffort: outputEffort,
EffectiveEffort: EffortNone,
ClearSampling: capabilities.strictSampling,
Diagnostics: diagnostics,
@@ -175,7 +230,7 @@ func RenderClaude(model string, intent Intent, maxTokens *uint, adapterBudgetPer
effort = EffortFromBudget(*intent.BudgetTokens)
}
if effort == "" && intent.Mode == ModeEnabled {
effort = EffortHigh
effort = capabilities.defaultEffort
}
normalizedEffort := normalizeClaudeEffort(effort, capabilities)
if effort != "" && normalizedEffort != effort {
@@ -187,7 +242,7 @@ func RenderClaude(model string, intent Intent, maxTokens *uint, adapterBudgetPer
effort = normalizedEffort
effectiveEffort := effort
if effectiveEffort == "" && intent.Mode == ModeAdaptive {
effectiveEffort = EffortHigh
effectiveEffort = capabilities.defaultEffort
}
// Claude effort can be used without enabling thinking. Preserve that
@@ -227,11 +282,22 @@ func RenderClaude(model string, intent Intent, maxTokens *uint, adapterBudgetPer
))
intent.Mode = ModeEnabled
if intent.Effort == "" {
intent.Effort = EffortHigh
intent.Effort = capabilities.defaultEffort
}
}
if intent.Mode == ModeUnset {
return ClaudeRender{OutputEffort: intent.Effort, EffectiveEffort: intent.Effort, Diagnostics: diagnostics}, nil
// Known models get the effort they accept. They think only when asked,
// so a removed effort leaves none.
effort := intent.Effort
if isKnownClaudeModel(model) {
effort = ClaudeEffort(model, intent.Effort)
diagnostics = append(diagnostics, claudeEffortDiagnostics(model, intent.Effort, effort)...)
}
effectiveEffort := effort
if effectiveEffort == "" {
effectiveEffort = EffortNone
}
return ClaudeRender{OutputEffort: effort, EffectiveEffort: effectiveEffort, Diagnostics: diagnostics}, nil
}
var raisedMaxTokens *uint
if maxTokens == nil || *maxTokens <= 1024 {
@@ -289,9 +355,8 @@ func RenderClaude(model string, intent Intent, maxTokens *uint, adapterBudgetPer
} else if effectiveEffort == "" {
effectiveEffort = EffortFromBudget(budget)
}
outputEffort := Effort("")
if capabilities.supportsEffort && intent.Effort != "" {
outputEffort = normalizeClaudeEffort(intent.Effort, capabilities)
outputEffort := ClaudeEffort(model, intent.Effort)
if outputEffort != "" {
effectiveEffort = outputEffort
}
thinking := &dto.Thinking{Type: "enabled", BudgetTokens: &budget}
@@ -322,6 +387,25 @@ func claudeReasoningDiagnostic(code string, message string) types.ConversionDiag
}
}
// claudeEffortDiagnostics reports a requested effort that is sent at another
// level or removed because the model takes none.
func claudeEffortDiagnostics(model string, requested Effort, sent Effort) []types.ConversionDiagnostic {
switch {
case requested == "" || requested == EffortNone || requested == sent:
return nil
case sent == "":
return []types.ConversionDiagnostic{claudeReasoningDiagnostic(
"claude_effort_unsupported",
fmt.Sprintf("model %q does not support effort %q; the effort was removed", model, requested),
)}
default:
return []types.ConversionDiagnostic{claudeReasoningDiagnostic(
"claude_effort_adjusted",
fmt.Sprintf("model %q does not support effort %q with the selected thinking mode; using %q", model, requested, sent),
)}
}
}
// ClaudeUsesManualThinking reports whether an exact numeric budget is rendered
// as legacy extended thinking rather than being reduced to adaptive effort.
func ClaudeUsesManualThinking(model string, intent Intent) bool {
@@ -333,12 +417,39 @@ func IsKnownClaudeModel(model string) bool {
return isKnownClaudeModel(model)
}
// ClaudeDefaultEffort is the effort the model runs at when a request sets
// none.
func ClaudeDefaultEffort(model string) Effort {
return claudeCapabilitiesFor(model).defaultEffort
}
// ClaudeEffort returns the effort level the model accepts in place of the
// requested one. It returns empty when no effort is requested or the model
// takes none, which includes models missing from the capability table.
func ClaudeEffort(model string, effort Effort) Effort {
capabilities := claudeCapabilitiesFor(model)
if !capabilities.supportsEffort || effort == "" || effort == EffortNone {
return ""
}
return normalizeClaudeEffort(effort, capabilities)
}
// ResolveClaudeDefault makes the Claude model's default thinking explicit for
// a non-Claude target: no intent becomes adaptive thinking on models that
// think by default, and thinking without a strength takes the model's
// default effort.
func ResolveClaudeDefault(model string, intent Intent) Intent {
if intent.HasStrength() || !claudeCapabilitiesFor(model).defaultThinking {
return intent
capabilities := claudeCapabilitiesFor(model)
switch {
case !intent.HasStrength():
if !capabilities.defaultThinking {
return intent
}
intent.Mode = ModeAdaptive
intent.Effort = capabilities.defaultEffort
case intent.Effort == "" && intent.BudgetTokens == nil && (intent.Mode == ModeAdaptive || intent.Mode == ModeEnabled):
intent.Effort = capabilities.defaultEffort
}
intent.Mode = ModeAdaptive
intent.Effort = EffortHigh
return intent
}
relaykit/relayconvert/reasoning/gemini.go
+9 -4
View File
@@ -145,7 +145,7 @@ func RenderGemini(model string, intent Intent, maxOutputTokens *uint, adapterBud
}
hasBudget = true
} else if intent.Effort != "" {
budget = gemini25BudgetForEffort(intent.Effort)
budget = gemini25BudgetForEffort(intent.Effort, capabilities.maxBudget)
hasBudget = true
} else if intent.Mode != ModeUnset && maxOutputTokens != nil && *maxOutputTokens > 0 {
if uint64(*maxOutputTokens) > uint64(math.MaxInt) {
@@ -295,7 +295,7 @@ func NormalizeGeminiThinkingConfig(model string, generation *dto.GeminiChatGener
if capabilities.kind == geminiThinkingBudget {
if config.ThinkingLevel != "" {
budget := gemini25BudgetForEffort(intent.Effort)
budget := gemini25BudgetForEffort(intent.Effort, capabilities.maxBudget)
diagnostics = append(diagnostics, geminiReasoningDiagnostic(
"gemini_level_to_budget",
fmt.Sprintf("model %q uses thinkingBudget; thinkingLevel %q was converted to budget %d", model, config.ThinkingLevel, budget),
@@ -476,14 +476,19 @@ func EquivalentGeminiStrength(model string, left Intent, right Intent) (bool, er
return leftConfig.ThinkingBudget == nil || *leftConfig.ThinkingBudget == *rightConfig.ThinkingBudget, nil
}
func gemini25BudgetForEffort(effort Effort) int {
// gemini25BudgetForEffort follows Google's OpenAI-compatible mapping, where
// high is 24576; xhigh and max, which that mapping lacks, take the model's
// largest budget.
func gemini25BudgetForEffort(effort Effort, maxBudget int) int {
switch effort {
case EffortMinimal, EffortLow:
return 1024
case EffortMedium:
return 8192
case EffortHigh, EffortXHigh, EffortMax:
case EffortHigh:
return 24576
case EffortXHigh, EffortMax:
return maxBudget
default:
return 0
}
relaykit/relayconvert/reasoning/suffix.go
+51 -40
View File
@@ -13,7 +13,7 @@ var EffortSuffixes = []string{"-max", "-xhigh", "-high", "-medium", "-low", "-mi
var OpenAIEffortSuffixes = []string{"-max", "-xhigh", "-high", "-medium", "-low", "-minimal", "-none"}
var DeepSeekV4EffortSuffixes = []string{"-none", "-max"}
var DeepSeekV4EffortSuffixes = []string{"-none", "-low", "-max"}
var (
legacyOpenAIModelPattern = regexp.MustCompile(`^(gpt-[a-z0-9][a-z0-9._-]*|o[1-9][a-z0-9._-]*)$`)
@@ -136,12 +136,16 @@ func ParseOpenAIReasoningEffortFromModelSuffix(modelName string, preserveEffortT
return effort, baseModel
}
func ParseClaudeModelSuffix(modelName string, allowThinkingAlias bool) (string, Intent, bool, error) {
// ParseClaudeModelSuffix folds the naked claude-* aliases (-thinking,
// -nothinking, -thinking-<budget>, and effort tails such as -high) into an
// Intent. allowSuffixAlias is the host thinking-adapter toggle; when false the
// name is returned verbatim and no intent is produced.
func ParseClaudeModelSuffix(modelName string, allowSuffixAlias bool) (string, Intent, bool, error) {
prefix, bare := splitModelNamespace(modelName)
if !strings.HasPrefix(bare, "claude-") {
return modelName, Intent{}, false, nil
}
base, intent, found, err := parseProviderModelSuffix(bare, "claude-", allowThinkingAlias, true)
base, intent, found, err := parseProviderModelSuffix(bare, "claude-", allowSuffixAlias, true)
if err != nil || !found || !legacyClaudeModelPattern.MatchString(base) {
return modelName, Intent{}, false, err
}
@@ -157,7 +161,7 @@ func isKnownClaudeModel(modelName string) bool {
}
knownPrefixes := []string{
"claude-fable-5", "claude-mythos-5", "claude-mythos-preview",
"claude-opus-5", "claude-sonnet-5", "claude-opus-4-8",
"claude-opus-5", "claude-sonnet-5", "claude-haiku-5", "claude-opus-4-8",
"claude-opus-4-7", "claude-opus-4-6", "claude-sonnet-4-6",
"claude-opus-4-5", "claude-sonnet-4-5", "claude-haiku-4-5",
"claude-opus-4-1", "claude-opus-4-", "claude-sonnet-4-",
@@ -171,12 +175,14 @@ func isKnownClaudeModel(modelName string) bool {
return false
}
func ParseGeminiModelSuffix(modelName string, allowThinkingAlias bool) (string, Intent, bool, error) {
// ParseGeminiModelSuffix is the gemini-* counterpart of ParseClaudeModelSuffix
// and honours allowSuffixAlias the same way.
func ParseGeminiModelSuffix(modelName string, allowSuffixAlias bool) (string, Intent, bool, error) {
prefix, bare := splitModelNamespace(modelName)
if !strings.HasPrefix(bare, "gemini-") {
return modelName, Intent{}, false, nil
}
base, intent, found, err := parseProviderModelSuffix(bare, "gemini-", allowThinkingAlias, true)
base, intent, found, err := parseProviderModelSuffix(bare, "gemini-", allowSuffixAlias, true)
if err != nil || !found || !legacyGeminiModelPattern.MatchString(base) {
return modelName, Intent{}, false, err
}
@@ -186,13 +192,13 @@ func ParseGeminiModelSuffix(modelName string, allowThinkingAlias bool) (string,
// ParseKnownProviderModelSuffix extracts a canonical intent only when the
// origin identifies a provider family whose suffix vocabulary is defined by
// relaykit. Unknown OpenAI-compatible model names are deliberately untouched.
func ParseKnownProviderModelSuffix(modelName string, allowThinkingAlias bool) (string, Intent, bool, error) {
func ParseKnownProviderModelSuffix(modelName string, allowSuffixAlias bool) (string, Intent, bool, error) {
bare := lastModelPathSegment(modelName)
if strings.HasPrefix(bare, "claude-") {
return ParseClaudeModelSuffix(modelName, allowThinkingAlias)
return ParseClaudeModelSuffix(modelName, allowSuffixAlias)
}
if strings.HasPrefix(bare, "gemini-") {
return ParseGeminiModelSuffix(modelName, allowThinkingAlias)
return ParseGeminiModelSuffix(modelName, allowSuffixAlias)
}
return modelName, Intent{}, false, nil
}
@@ -214,37 +220,39 @@ func TrimGeminiThinkingSuffix(modelName string) (string, bool) {
return baseModel, ok && err == nil
}
func parseProviderModelSuffix(modelName string, requiredPrefix string, allowThinkingAlias bool, includeThoughts bool) (string, Intent, bool, error) {
if allowThinkingAlias {
if marker := strings.LastIndex(modelName, "-thinking-"); marker >= 0 {
baseModel := modelName[:marker]
if !strings.HasPrefix(baseModel, requiredPrefix) {
return modelName, Intent{}, false, nil
}
budget, err := strconv.Atoi(modelName[marker+len("-thinking-"):])
if err != nil {
return modelName, Intent{}, false, fmt.Errorf("invalid thinking budget suffix on model %q: %w", modelName, err)
}
intent := Intent{BudgetTokens: &budget, Source: SourceSuffix, BudgetSource: SourceSuffix}
if includeThoughts {
value := true
intent.IncludeThoughts = &value
}
return baseModel, intent, true, nil
// parseProviderModelSuffix recognises a provider family's naked model-name
// aliases. allowSuffixAlias gates all of them, including effort tails, so a
// disabled thinking adapter leaves the model name untouched.
func parseProviderModelSuffix(modelName string, requiredPrefix string, allowSuffixAlias bool, includeThoughts bool) (string, Intent, bool, error) {
if !allowSuffixAlias {
return modelName, Intent{}, false, nil
}
if marker := strings.LastIndex(modelName, "-thinking-"); marker >= 0 {
baseModel := modelName[:marker]
if !strings.HasPrefix(baseModel, requiredPrefix) {
return modelName, Intent{}, false, nil
}
if before, ok := strings.CutSuffix(modelName, "-nothinking"); ok {
baseModel := before
return baseModel, Intent{Mode: ModeDisabled, Effort: EffortNone, Source: SourceSuffix}, true, nil
budget, err := strconv.Atoi(modelName[marker+len("-thinking-"):])
if err != nil {
return modelName, Intent{}, false, fmt.Errorf("invalid thinking budget suffix on model %q: %w", modelName, err)
}
if before, ok := strings.CutSuffix(modelName, "-thinking"); ok {
baseModel := before
intent := Intent{Mode: ModeEnabled, Source: SourceSuffix}
if includeThoughts {
value := true
intent.IncludeThoughts = &value
}
return baseModel, intent, true, nil
intent := Intent{BudgetTokens: &budget, Source: SourceSuffix, BudgetSource: SourceSuffix}
if includeThoughts {
value := true
intent.IncludeThoughts = &value
}
return baseModel, intent, true, nil
}
if baseModel, ok := strings.CutSuffix(modelName, "-nothinking"); ok {
return baseModel, Intent{Mode: ModeDisabled, Effort: EffortNone, Source: SourceSuffix}, true, nil
}
if baseModel, ok := strings.CutSuffix(modelName, "-thinking"); ok {
intent := Intent{Mode: ModeEnabled, Source: SourceSuffix}
if includeThoughts {
value := true
intent.IncludeThoughts = &value
}
return baseModel, intent, true, nil
}
suffixes := []string{"-max", "-xhigh", "-high", "-medium", "-low", "-minimal", "-none"}
@@ -266,16 +274,19 @@ func parseProviderModelSuffix(modelName string, requiredPrefix string, allowThin
return baseModel, intent, true, nil
}
// ParseDeepSeekV4ThinkingSuffix maps the -none/-low/-max tails of
// deepseek-flash and deepseek-v4-* onto DeepSeek's thinking switch and
// effort. A bare name keeps DeepSeek's default (thinking on, effort high).
func ParseDeepSeekV4ThinkingSuffix(modelName string) (baseModel string, thinkingType string, effort string, ok bool) {
baseModel, suffix, ok := TrimEffortSuffixWithSuffixes(modelName, DeepSeekV4EffortSuffixes)
if !ok || !strings.HasPrefix(baseModel, "deepseek-v4-") {
if !ok || baseModel != "deepseek-flash" && !strings.HasPrefix(baseModel, "deepseek-v4-") {
return modelName, "", "", false
}
switch suffix {
case "none":
return baseModel, "disabled", "", true
case "max":
return baseModel, "enabled", "max", true
case "low", "max":
return baseModel, "enabled", suffix, true
default:
return modelName, "", "", false
}
relaykit/relayconvert/request_registry.go
+22 -35
View File
@@ -68,9 +68,9 @@ var (
const (
requestConverterClaudeToGemini = "claude_messages_to_gemini_generate_content"
requestConverterClaudeToResponses = "claude_messages_to_openai_responses"
requestConverterClaudeToResponses = ConverterClaudeMessagesToOpenAIResponses
requestConverterGeminiToClaude = "gemini_generate_content_to_claude_messages"
requestConverterGeminiToResponses = "gemini_generate_content_to_openai_responses"
requestConverterGeminiToResponses = ConverterGeminiContentToOpenAIResponses
requestConverterResponsesToClaude = ConverterOpenAIResponsesToClaudeMessages
)
@@ -84,6 +84,8 @@ const (
ConverterOpenAIResponsesToGemini = "openai_responses_to_gemini_generate_content"
ConverterGeminiContentToOpenAIChat = "gemini_generate_content_to_openai_chat_completions"
ConverterOpenAIChatToGeminiContent = "openai_chat_completions_to_gemini_generate_content"
ConverterClaudeMessagesToOpenAIResponses = "claude_messages_to_openai_responses"
ConverterGeminiContentToOpenAIResponses = "gemini_generate_content_to_openai_responses"
)
func registerBuiltinRequestConverter(spec RequestConverterSpec) {
@@ -247,11 +249,6 @@ func executeRequestSteps(c context.Context, info convmeta.Meta, from types.Relay
}
steps := make([]RequestStep, 0, len(specs))
for _, spec := range specs {
current, err = prepareRequestForStep(current, spec, target)
if err != nil {
return nil, err
}
var step RequestStep
current, step, err = executeRequestStep(c, info, spec, current)
if err != nil {
@@ -260,6 +257,7 @@ func executeRequestSteps(c context.Context, info convmeta.Meta, from types.Relay
steps = append(steps, step)
}
tools.Model = convmeta.UpstreamModelName(info)
current, toolDiagnostics, err := toolconv.AttachRequest(target, current, tools, convmeta.OptionsOf(info))
diagnostics := append(diagnosticCollector.Diagnostics(), toolDiagnostics...)
for i := range diagnostics {
@@ -284,6 +282,9 @@ func executeRequestSteps(c context.Context, info convmeta.Meta, from types.Relay
for _, step := range steps {
info.AppendRequestConversion(step.To)
}
if from == types.RelayFormatOpenAIResponses {
info.SetResponsesToolState(responsesToolState(tools))
}
}
converters := make([]string, 0, len(steps))
@@ -304,6 +305,19 @@ func executeRequestSteps(c context.Context, info convmeta.Meta, from types.Relay
}, nil
}
// responsesToolState records which Responses custom tools were sent upstream
// as functions and which tool names were flattened from namespaces, so the
// response side can restore their calls. It returns nil when there is nothing
// to restore so a retry never reuses another attempt's record.
func responsesToolState(tools toolconv.Set) *convmeta.ResponsesToolState {
names := toolconv.ResponsesCustomToolNames(tools)
namespaces := toolconv.ResponsesToolNamespaces(tools)
if len(names) == 0 && len(namespaces) == 0 {
return nil
}
return &convmeta.ResponsesToolState{CustomToolNames: names, Namespaces: namespaces}
}
func expandRequestConverterSteps(spec RequestConverterSpec) ([]RequestConverterSpec, error) {
if len(spec.StepConverters) == 0 {
if spec.Convert == nil {
@@ -353,28 +367,6 @@ func executeRequestStep(c context.Context, info convmeta.Meta, spec RequestConve
}, nil
}
func prepareRequestForStep(request any, spec RequestConverterSpec, finalTarget types.RelayFormat) (any, error) {
if spec.From != types.RelayFormatOpenAIResponses || finalTarget != types.RelayFormatGemini {
return request, nil
}
responsesRequest, ok := request.(*dto.OpenAIResponsesRequest)
if !ok {
if value, ok := request.(dto.OpenAIResponsesRequest); ok {
responsesRequest = &value
}
}
if responsesRequest == nil {
return nil, fmt.Errorf("expected OpenAI responses request, got %T", request)
}
prepared, err := oairesponses.PrepareOpenAIResponsesRequest(*responsesRequest)
if err != nil {
return nil, err
}
return &prepared, nil
}
func lookupRequestRoute(from types.RelayFormat, to types.RelayFormat) (RequestConverterSpec, bool) {
requestConverterMu.RLock()
defer requestConverterMu.RUnlock()
@@ -521,12 +513,7 @@ func convertOpenAIResponsesRequestToGeminiChat(c context.Context, info convmeta.
if err != nil {
return nil, err
}
prepared, err := oairesponses.PrepareOpenAIResponsesRequest(*responsesRequest)
if err != nil {
return nil, err
}
return oairesponses.OpenAIResponsesRequestToGeminiChat(c, &prepared, info)
return oairesponses.OpenAIResponsesRequestToGeminiChat(c, responsesRequest, info)
}
func convertResponsesRequestToChat(c context.Context, _ convmeta.Meta, request any) (any, error) {
relaykit/relayconvert/request_registry_test.go
+316 -55
View File
@@ -1,6 +1,7 @@
package relayconvert
import (
"context"
"testing"
"github.com/QuantumNous/new-api/relaykit/dto"
@@ -8,10 +9,90 @@ import (
sharedgemini "github.com/QuantumNous/new-api/relaykit/relayconvert/internal/shared/gemini"
kitutil "github.com/QuantumNous/new-api/relaykit/relayconvert/kitutil"
"github.com/QuantumNous/new-api/relaykit/types"
"github.com/samber/lo"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestClaudeMessagesToOpenAIChatPreservesAssistantTextWithToolCalls(t *testing.T) {
text := "I will look that up."
request := dto.ClaudeRequest{
Model: "claude-test",
Messages: []dto.ClaudeMessage{
{
Role: "assistant",
Content: []dto.ClaudeMediaMessage{
{Type: "text", Text: &text},
{
Type: "tool_use",
Id: "call_1",
Name: "lookup",
Input: map[string]any{"query": "weather"},
},
},
},
},
}
result, err := ConvertRequest(
context.Background(),
&convmeta.Values{},
types.RelayFormatOpenAI,
&request,
)
require.NoError(t, err)
converted, ok := result.Value.(*dto.GeneralOpenAIRequest)
require.True(t, ok)
require.Len(t, converted.Messages, 1)
assistant := converted.Messages[0]
assert.Equal(t, "assistant", assistant.Role)
toolCalls := assistant.ParseToolCalls()
require.Len(t, toolCalls, 1)
assert.Equal(t, "call_1", toolCalls[0].ID)
require.Len(t, assistant.ParseContent(), 1)
assert.Equal(t, "text", assistant.ParseContent()[0].Type)
assert.Equal(t, text, assistant.ParseContent()[0].Text)
}
func TestClaudeMessagesToOpenAIChatKeepsToolOnlyContentEmpty(t *testing.T) {
request := dto.ClaudeRequest{
Model: "claude-test",
Messages: []dto.ClaudeMessage{
{
Role: "assistant",
Content: []dto.ClaudeMediaMessage{
{
Type: "tool_use",
Id: "call_1",
Name: "lookup",
Input: map[string]any{"query": "weather"},
},
},
},
},
}
result, err := ConvertRequest(
context.Background(),
&convmeta.Values{},
types.RelayFormatOpenAI,
&request,
)
require.NoError(t, err)
converted, ok := result.Value.(*dto.GeneralOpenAIRequest)
require.True(t, ok)
require.Len(t, converted.Messages, 1)
assistant := converted.Messages[0]
assert.Empty(t, assistant.ParseContent())
toolCalls := assistant.ParseToolCalls()
require.Len(t, toolCalls, 1)
assert.Equal(t, "call_1", toolCalls[0].ID)
}
func TestRequestConverterRegistryListsSupportedTextConverters(t *testing.T) {
tests := []struct {
converter string
@@ -38,10 +119,11 @@ func TestRequestConverterRegistryListsSupportedTextConverters(t *testing.T) {
},
},
{
converter: requestConverterClaudeToResponses,
from: types.RelayFormatClaude,
to: types.RelayFormatOpenAIResponses,
quality: RequestConverterQualityFair,
converter: requestConverterClaudeToResponses,
from: types.RelayFormatClaude,
to: types.RelayFormatOpenAIResponses,
quality: RequestConverterQualityFair,
advancedCustom: true,
},
{
converter: requestConverterGeminiToClaude,
@@ -62,6 +144,7 @@ func TestRequestConverterRegistryListsSupportedTextConverters(t *testing.T) {
ConverterGeminiContentToOpenAIChat,
ConverterOpenAIChatToOpenAIResponses,
},
advancedCustom: true,
},
{
converter: requestConverterResponsesToClaude,
@@ -158,6 +241,52 @@ func TestConvertRequestClaudeToResponsesUsesDirectPath(t *testing.T) {
assert.Equal(t, []types.RelayFormat{types.RelayFormatClaude, types.RelayFormatOpenAIResponses}, info.ConversionChain)
}
func TestConvertRequestClaudeToChatResolvesToolResultNames(t *testing.T) {
req := &dto.ClaudeRequest{
Model: "claude-test",
Messages: []dto.ClaudeMessage{
{Role: "user", Content: []dto.ClaudeMediaMessage{
{Type: "tool_result", ToolUseId: "call_1", Content: "before call"},
}},
{Role: "assistant", Content: []dto.ClaudeMediaMessage{
{Type: "tool_use", Id: "call_1", Name: "first", Input: map[string]any{}},
}},
{Role: "user", Content: []dto.ClaudeMediaMessage{
{Type: "tool_result", ToolUseId: "call_1", Content: "after call"},
{Type: "tool_result", ToolUseId: "missing", Content: "unknown"},
{Type: "tool_result", ToolUseId: "call_1", Name: "explicit", Content: "named"},
}},
{Role: "assistant", Content: []dto.ClaudeMediaMessage{
{Type: "tool_use", Id: "call_1", Name: "later", Input: map[string]any{}},
}},
},
}
result, err := ConvertRequestByID(nil, nil, ConverterClaudeMessagesToOpenAIChat, req)
require.NoError(t, err)
chatReq, ok := result.Value.(*dto.GeneralOpenAIRequest)
require.True(t, ok)
require.Len(t, chatReq.Messages, 6)
for _, tt := range []struct {
index int
id string
name string
}{
{0, "call_1", "first"},
{2, "call_1", "first"},
{3, "missing", ""},
{4, "call_1", "explicit"},
} {
message := chatReq.Messages[tt.index]
assert.Equal(t, "tool", message.Role)
assert.Equal(t, tt.id, message.ToolCallId)
require.NotNil(t, message.Name)
assert.Equal(t, tt.name, *message.Name)
}
assert.Equal(t, "assistant", chatReq.Messages[1].Role)
assert.Equal(t, "assistant", chatReq.Messages[5].Role)
}
func TestConvertRequestClaudeToResponsesPreservesMixedBlockOrder(t *testing.T) {
info := &convmeta.Values{ConversionChain: []types.RelayFormat{types.RelayFormatClaude}}
stream := true
@@ -219,6 +348,43 @@ func TestConvertRequestClaudeToResponsesPreservesMixedBlockOrder(t *testing.T) {
assert.Equal(t, "continue", inputContentText(t, input[5]))
}
func TestConvertRequestClaudeToResponsesDefaultsImageDetail(t *testing.T) {
const imageData = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+Xoy8AAAAASUVORK5CYII="
req := &dto.ClaudeRequest{
Model: "gpt-test",
Messages: []dto.ClaudeMessage{
{Role: "user", Content: []dto.ClaudeMediaMessage{{
Type: "image",
Source: &dto.ClaudeMessageSource{Type: "url", Url: "https://example.test/a.png"},
}}},
{Role: "assistant", Content: []dto.ClaudeMediaMessage{{
Type: "tool_use", Id: "call_1", Name: "screenshot", Input: map[string]any{},
}}},
{Role: "user", Content: []dto.ClaudeMediaMessage{{
Type: "tool_result", ToolUseId: "call_1",
Content: []dto.ClaudeMediaMessage{{
Type: "image",
Source: &dto.ClaudeMessageSource{Type: "base64", MediaType: "image/png", Data: imageData},
}},
}}},
},
}
result, err := ConvertRequestByID(context.Background(), nil, ConverterClaudeMessagesToOpenAIResponses, req)
require.NoError(t, err)
responsesReq, ok := result.Value.(*dto.OpenAIResponsesRequest)
require.True(t, ok)
assert.JSONEq(t, `[
{"role":"user","content":[
{"type":"input_image","image_url":"https://example.test/a.png","detail":"auto"}
]},
{"type":"function_call","call_id":"call_1","name":"screenshot","arguments":"{}"},
{"type":"function_call_output","call_id":"call_1","output":[
{"type":"input_image","image_url":"data:image/png;base64,`+imageData+`","detail":"auto"}
]}
]`, string(responsesReq.Input))
}
func TestConvertRequestClaudeToResponsesDropsIncompatibleContextManagement(t *testing.T) {
req := &dto.ClaudeRequest{
Model: "gpt-test",
@@ -374,61 +540,121 @@ func TestConvertRequestViaExecutesExplicitPath(t *testing.T) {
assert.Equal(t, []types.RelayFormat{types.RelayFormatOpenAI, types.RelayFormatOpenAIResponses}, info.ConversionChain)
}
func TestConvertRequestResponsesToGeminiAppliesResponsesPreprocess(t *testing.T) {
info := &convmeta.Values{
ConversionChain: []types.RelayFormat{types.RelayFormatOpenAIResponses},
ChannelMetaAttached: true,
UpstreamModelName: "gemini-test",
// customToolInputs are custom_tool_call inputs that must reach the upstream
// as the raw string, including ones that happen to look like JSON.
var customToolInputs = []struct {
name string
input string
}{
{"json object", `{"a":1}`},
{"json array", `[1,2]`},
{"plain text", "*** Begin Patch\n*** End Patch"},
}
func customToolHistoryRequest(t *testing.T, items ...map[string]any) *dto.OpenAIResponsesRequest {
t.Helper()
input := append([]map[string]any{{"role": "user", "content": "apply the patch"}}, items...)
return &dto.OpenAIResponsesRequest{
Model: "model-test",
MaxOutputTokens: lo.ToPtr(uint(64)),
Input: mustRawMessage(t, input),
Tools: mustRawMessage(t, []map[string]any{{"type": "custom", "name": "apply_patch"}}),
}
req := &dto.OpenAIResponsesRequest{
Model: "gemini-test",
Input: mustRawMessage(t, []map[string]any{
{
"role": "user",
"content": "next turn",
},
{
"type": "custom_tool_call",
"call_id": "call_custom",
"name": "apply_patch",
"input": "patch body",
},
{
"type": "custom_tool_call_output",
"call_id": "call_custom",
"output": "ok",
},
{
"type": "function_call_output",
"call_id": "call_custom",
"output": "legacy custom output",
},
}),
Tools: mustRawMessage(t, []map[string]any{
{"type": "custom", "name": "apply_patch"},
}),
}
func TestConvertRequestResponsesToClaudeKeepsCustomToolHistory(t *testing.T) {
for _, tc := range customToolInputs {
t.Run(tc.name, func(t *testing.T) {
req := customToolHistoryRequest(t,
map[string]any{"type": "custom_tool_call", "call_id": "call_custom", "name": "apply_patch", "input": tc.input},
map[string]any{"type": "custom_tool_call_output", "call_id": "call_custom", "output": "ok"},
)
result, err := ConvertRequest(nil, &convmeta.Values{}, types.RelayFormatClaude, req)
require.NoError(t, err)
claudeReq, ok := result.Value.(*dto.ClaudeRequest)
require.True(t, ok)
require.Len(t, claudeReq.Messages, 3)
assert.Equal(t, "assistant", claudeReq.Messages[1].Role)
toolUse, err := claudeReq.Messages[1].ParseContent()
require.NoError(t, err)
assert.Equal(t, []dto.ClaudeMediaMessage{{
Type: "tool_use",
Id: "call_custom",
Name: "apply_patch",
Input: map[string]any{"input": tc.input},
}}, toolUse)
assert.Equal(t, "user", claudeReq.Messages[2].Role)
toolResult, err := claudeReq.Messages[2].ParseContent()
require.NoError(t, err)
assert.Equal(t, []dto.ClaudeMediaMessage{{Type: "tool_result", ToolUseId: "call_custom", Content: "ok"}}, toolResult)
})
}
result, err := ConvertRequest(nil, info, types.RelayFormatGemini, req)
t.Run("missing name", func(t *testing.T) {
req := customToolHistoryRequest(t, map[string]any{"type": "custom_tool_call", "call_id": "call_custom", "input": "ls"})
_, err := ConvertRequest(nil, &convmeta.Values{}, types.RelayFormatClaude, req)
require.ErrorContains(t, err, "custom_tool_call item is missing name")
})
}
require.NoError(t, err)
geminiReq, ok := result.Value.(*dto.GeminiChatRequest)
require.True(t, ok)
assert.Empty(t, geminiReq.GetTools())
require.Len(t, geminiReq.Contents, 1)
assert.Equal(t, "user", geminiReq.Contents[0].Role)
require.Len(t, geminiReq.Contents[0].Parts, 1)
assert.Equal(t, "next turn", geminiReq.Contents[0].Parts[0].Text)
assert.Equal(t, ConverterOpenAIResponsesToGemini, result.Converter)
assert.Equal(t, RequestConverterQualityFair, result.Quality)
assert.Equal(t, []RequestStep{
{
Converter: ConverterOpenAIResponsesToGemini,
From: types.RelayFormatOpenAIResponses,
To: types.RelayFormatGemini,
},
}, result.Steps)
assert.Equal(t, []types.RelayFormat{types.RelayFormatOpenAIResponses, types.RelayFormatGemini}, info.ConversionChain)
func TestConvertRequestResponsesToGeminiKeepsCustomToolHistory(t *testing.T) {
info := func() *convmeta.Values {
return &convmeta.Values{Options: &convmeta.Options{Gemini: convmeta.GeminiOptions{FunctionCallThoughtSignatureEnabled: true}}}
}
for _, tc := range customToolInputs {
t.Run(tc.name, func(t *testing.T) {
// Older Codex history may answer a custom call with function_call_output.
req := customToolHistoryRequest(t,
map[string]any{"type": "custom_tool_call", "call_id": "call_custom", "name": "apply_patch", "input": tc.input},
map[string]any{"type": "custom_tool_call_output", "call_id": "call_custom", "output": "ok"},
map[string]any{"type": "function_call_output", "call_id": "call_custom", "output": "legacy"},
)
result, err := ConvertRequest(nil, info(), types.RelayFormatGemini, req)
require.NoError(t, err)
geminiReq, ok := result.Value.(*dto.GeminiChatRequest)
require.True(t, ok)
tools := geminiReq.GetTools()
require.Len(t, tools, 1)
functions, err := kitutil.Any2Type[[]dto.FunctionRequest](tools[0].FunctionDeclarations)
require.NoError(t, err)
require.Len(t, functions, 1)
assert.Equal(t, "apply_patch", functions[0].Name)
require.Len(t, geminiReq.Contents, 3)
model := geminiReq.Contents[1]
assert.Equal(t, "model", model.Role)
require.Len(t, model.Parts, 1)
assert.Equal(t, &dto.FunctionCall{
ID: "call_custom",
FunctionName: "apply_patch",
Arguments: map[string]any{"input": tc.input},
}, model.Parts[0].FunctionCall)
var thoughtSignature string
require.NoError(t, kitutil.Unmarshal(model.Parts[0].ThoughtSignature, &thoughtSignature))
assert.Equal(t, sharedgemini.ThoughtSignatureBypassValue, thoughtSignature)
user := geminiReq.Contents[2]
assert.Equal(t, "user", user.Role)
require.Len(t, user.Parts, 2)
for index, output := range []string{"ok", "legacy"} {
assert.Equal(t, &dto.GeminiFunctionResponse{
Name: "apply_patch",
Response: map[string]any{"content": output},
ID: []byte(`"call_custom"`),
}, user.Parts[index].FunctionResponse)
}
})
}
t.Run("missing name", func(t *testing.T) {
req := customToolHistoryRequest(t, map[string]any{"type": "custom_tool_call", "call_id": "call_custom", "input": "ls"})
_, err := ConvertRequest(nil, info(), types.RelayFormatGemini, req)
require.ErrorContains(t, err, "custom_tool_call item is missing name")
})
}
func TestConvertRequestResponsesToGeminiUsesDirectConverter(t *testing.T) {
@@ -781,3 +1007,38 @@ func inputContentText(t *testing.T, item map[string]any) string {
require.True(t, ok)
return text
}
func TestConvertRequestRecordsResponsesCustomToolsForEachTarget(t *testing.T) {
codexRequest := func(tools ...map[string]any) *dto.OpenAIResponsesRequest {
return &dto.OpenAIResponsesRequest{
Model: "model-test",
MaxOutputTokens: lo.ToPtr(uint(64)),
Input: mustMarshalRequestJSON(t, "hello"),
Tools: mustMarshalRequestJSON(t, tools),
}
}
execTool := map[string]any{"type": "custom", "name": "exec", "description": "Run code", "format": map[string]any{"type": "text"}}
waitTool := map[string]any{"type": "function", "name": "wait", "parameters": map[string]any{"type": "object"}}
for _, target := range []types.RelayFormat{types.RelayFormatOpenAI, types.RelayFormatClaude, types.RelayFormatGemini} {
t.Run(string(target), func(t *testing.T) {
info := &convmeta.Values{}
_, err := ConvertRequest(nil, info, target, codexRequest(execTool, waitTool))
require.NoError(t, err)
require.NotNil(t, info.ResponsesTools)
assert.True(t, info.ResponsesTools.IsCustomTool("exec"))
assert.False(t, info.ResponsesTools.IsCustomTool("wait"))
_, err = ConvertRequest(nil, info, target, codexRequest(waitTool))
require.NoError(t, err)
assert.Nil(t, info.ResponsesTools, "a retry without custom tools must not keep the previous record")
})
}
}
func mustMarshalRequestJSON(t *testing.T, value any) []byte {
t.Helper()
raw, err := kitutil.Marshal(value)
require.NoError(t, err)
return raw
}
relaykit/relayconvert/response_registry.go
+6 -3
View File
@@ -893,7 +893,7 @@ func usageFromClaudeResponse(resp *dto.ClaudeResponse) *dto.Usage {
return nil
}
func convertOAIChatResponseToOAIResponses(_ context.Context, _ convmeta.Meta, response any) (any, *dto.Usage, error) {
func convertOAIChatResponseToOAIResponses(_ context.Context, info convmeta.Meta, response any) (any, *dto.Usage, error) {
chatResponse, err := asOAIChatResponse(response)
if err != nil {
return nil, nil, err
@@ -902,7 +902,7 @@ func convertOAIChatResponseToOAIResponses(_ context.Context, _ convmeta.Meta, re
if id == "" {
id = fmt.Sprintf("resp_%s", kitutil.GetUUID())
}
return ChatCompletionsResponseToResponsesResponse(chatResponse, id)
return oaichat.ChatCompletionsResponseToResponsesResponseWithTools(chatResponse, id, convmeta.ResponsesToolStateOf(info))
}
func convertOAIResponsesResponseToOAIChat(_ context.Context, _ convmeta.Meta, response any) (any, *dto.Usage, error) {
@@ -938,7 +938,7 @@ func newOAIChatToOAIResponsesStreamState(options ResponseStreamOptions) any {
return state
}
func convertOAIChatStreamResponseToOAIResponses(_ context.Context, _ convmeta.Meta, response any, state any) ([]any, *dto.Usage, error) {
func convertOAIChatStreamResponseToOAIResponses(_ context.Context, info convmeta.Meta, response any, state any) ([]any, *dto.Usage, error) {
chatResponse, err := asOAIChatStreamResponse(response)
if err != nil {
return nil, nil, err
@@ -947,6 +947,9 @@ func convertOAIChatStreamResponseToOAIResponses(_ context.Context, _ convmeta.Me
if !ok || streamState == nil {
return nil, nil, errors.New("OAI chat to OAI responses stream state is required")
}
if streamState.Tools == nil {
streamState.Tools = convmeta.ResponsesToolStateOf(info)
}
events, err := ChatCompletionsStreamChunkToResponsesEvents(chatResponse, streamState)
if err != nil {
return nil, nil, err
relaykit/relayconvert/response_registry_test.go
+102
View File
@@ -1,6 +1,7 @@
package relayconvert
import (
"strings"
"testing"
"github.com/QuantumNous/new-api/relaykit/dto"
@@ -699,3 +700,104 @@ func textRegistryResponsesResponse() *dto.OpenAIResponsesResponse {
func respPtr[T any](value T) *T {
return &value
}
func TestConvertResponseToResponsesRestoresRecordedCustomTools(t *testing.T) {
info := &convmeta.Values{ResponsesTools: &convmeta.ResponsesToolState{CustomToolNames: map[string]struct{}{"exec": {}}}}
chatMessage := dto.Message{Role: "assistant"}
chatMessage.SetToolCalls([]dto.ToolCallRequest{{ID: "call_exec", Type: "function", Function: dto.FunctionRequest{Name: "exec", Arguments: `{"input":"ls"}`}}})
geminiExecCall := `{"candidates":[{"finishReason":"STOP","content":{"role":"model","parts":[{"functionCall":{"name":"exec","args":{"input":"ls"}}}]}}],"usageMetadata":{"promptTokenCount":4,"candidatesTokenCount":2,"totalTokenCount":6}}`
tests := []struct {
name string
from types.RelayFormat
response any
stream []any
}{
{
name: "chat",
from: types.RelayFormatOpenAI,
response: &dto.OpenAITextResponse{
Id: "chatcmpl_1",
Model: "gpt-test",
Choices: []dto.OpenAITextResponseChoice{{Message: chatMessage, FinishReason: "tool_calls"}},
},
stream: []any{
chatStreamChunk(`{"id":"chatcmpl_1","model":"gpt-test","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_exec","type":"function","function":{"name":"exec","arguments":"{\"input\":"}}]}}]}`),
chatStreamChunk(`{"id":"chatcmpl_1","model":"gpt-test","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"ls\"}"}}]},"finish_reason":"tool_calls"}]}`),
},
},
{
name: "claude",
from: types.RelayFormatClaude,
response: &dto.ClaudeResponse{
Id: "msg_1",
Type: "message",
Role: "assistant",
Model: "claude-test",
StopReason: "tool_use",
Content: []dto.ClaudeMediaMessage{{Type: "tool_use", Id: "toolu_exec", Name: "exec", Input: map[string]any{"input": "ls"}}},
},
stream: []any{
claudeStreamChunk(`{"type":"message_start","message":{"id":"msg_1","type":"message","role":"assistant","model":"claude-test","content":[],"usage":{"input_tokens":4,"output_tokens":0}}}`),
claudeStreamChunk(`{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_exec","name":"exec","input":{}}}`),
claudeStreamChunk(`{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"input\":"}}`),
claudeStreamChunk(`{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"\"ls\"}"}}`),
claudeStreamChunk(`{"type":"content_block_stop","index":0}`),
claudeStreamChunk(`{"type":"message_delta","delta":{"stop_reason":"tool_use"},"usage":{"output_tokens":2}}`),
claudeStreamChunk(`{"type":"message_stop"}`),
},
},
{
name: "gemini",
from: types.RelayFormatGemini,
response: geminiStreamChunk(geminiExecCall),
stream: []any{geminiStreamChunk(geminiExecCall)},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, err := ConvertResponse(nil, info, types.RelayFormatOpenAIResponses, tt.response)
require.NoError(t, err)
responses, ok := result.Value.(*dto.OpenAIResponsesResponse)
require.True(t, ok)
require.Len(t, responses.Output, 1)
assert.Equal(t, "custom_tool_call", responses.Output[0].Type)
assert.Equal(t, "exec", responses.Output[0].Name)
assert.Equal(t, `"ls"`, string(responses.Output[0].Input))
state, err := NewResponseStreamState(tt.from, types.RelayFormatOpenAIResponses, ResponseStreamOptions{ID: "resp_1", Model: "model-test"})
require.NoError(t, err)
var results []ResponseResult
for _, chunk := range tt.stream {
chunkResults, err := ConvertStreamResponseChunk(nil, info, state, chunk)
require.NoError(t, err)
results = append(results, chunkResults...)
}
finals, err := FinalizeStreamResponse(nil, info, state)
require.NoError(t, err)
results = append(results, finals...)
var toolEvents []string
for _, result := range results {
event, ok := result.Value.(ChatToResponsesStreamEvent)
require.True(t, ok)
switch {
case event.Payload.Item != nil && event.Payload.Item.Type != "message":
toolEvents = append(toolEvents, event.Type+" "+event.Payload.Item.Type+" "+string(event.Payload.Item.Input))
case event.Type == "response.custom_tool_call_input.delta":
toolEvents = append(toolEvents, event.Type+" "+event.Payload.Delta)
case event.Type == "response.custom_tool_call_input.done":
require.NotNil(t, event.Payload.Input)
toolEvents = append(toolEvents, event.Type+" "+*event.Payload.Input)
case strings.Contains(event.Type, "function_call_arguments"):
toolEvents = append(toolEvents, event.Type)
}
}
assert.Equal(t, []string{
`response.output_item.added custom_tool_call ""`,
"response.custom_tool_call_input.delta ls",
"response.custom_tool_call_input.done ls",
`response.output_item.done custom_tool_call "ls"`,
}, toolEvents)
})
}
}
relaykit/relayconvert/testdata/golden/request/openai_responses_to_openai.golden.json
Vendored
+3 -1
View File
@@ -13,7 +13,9 @@
"type": "text"
},
{
"image_url": "https://example.com/cat.png",
"image_url": {
"url": "https://example.com/cat.png"
},
"type": "image_url"
}
]
relaykit/relayconvert/testdata/golden/request/openai_to_openai_responses.golden.json
Vendored
+1
View File
@@ -8,6 +8,7 @@
"type": "input_text"
},
{
"detail": "high",
"image_url": "https://example.com/cat.png",
"type": "input_image"
}
relaykit/relayconvert/text_converter_registry.go
+2 -2
View File
@@ -164,7 +164,7 @@ var builtinTextConverters = []TextConverterSpec{
},
},
{
ID: requestConverterClaudeToResponses,
ID: ConverterClaudeMessagesToOpenAIResponses,
From: types.RelayFormatClaude,
To: types.RelayFormatOpenAIResponses,
Quality: TextConverterQualityFair,
@@ -199,7 +199,7 @@ var builtinTextConverters = []TextConverterSpec{
},
},
{
ID: requestConverterGeminiToResponses,
ID: ConverterGeminiContentToOpenAIResponses,
From: types.RelayFormatGemini,
To: types.RelayFormatOpenAIResponses,
Quality: TextConverterQualityFair,
relaykit/types/endpoint_type.go
+2
View File
@@ -10,6 +10,8 @@ const (
EndpointTypeOpenAIResponse EndpointType = "openai-response"
EndpointTypeOpenAIResponseCompact EndpointType = "openai-response-compact"
EndpointTypeOpenAIAlphaSearch EndpointType = "openai-alpha-search"
EndpointTypeDeepSeekToolsSearch EndpointType = "deepseek-tools-search"
EndpointTypeDeepSeekToolsFetch EndpointType = "deepseek-tools-fetch"
EndpointTypeAnthropic EndpointType = "anthropic"
EndpointTypeGemini EndpointType = "gemini"
EndpointTypeJinaRerank EndpointType = "jina-rerank"
relaykit/types/error.go
+13 -8
View File
@@ -145,7 +145,9 @@ func (e *NewAPIError) ErrorWithStatusCode() string {
return fmt.Sprintf("status_code=%d, %s", e.StatusCode, msg)
}
func (e *NewAPIError) MaskSensitiveError() string {
// MaskSensitiveError masks the upstream addresses of the channel that produced
// the error in the error text; see kitutil.MaskSensitiveInfo.
func (e *NewAPIError) MaskSensitiveError(upstreamURLs ...string) string {
if e == nil {
return ""
}
@@ -156,14 +158,14 @@ func (e *NewAPIError) MaskSensitiveError() string {
if e.errorCode == ErrorCodeCountTokenFailed {
return errStr
}
return kitutil.MaskSensitiveInfo(errStr)
return kitutil.MaskSensitiveInfo(errStr, upstreamURLs...)
}
func (e *NewAPIError) MaskSensitiveErrorWithStatusCode() string {
func (e *NewAPIError) MaskSensitiveErrorWithStatusCode(upstreamURLs ...string) string {
if e == nil {
return ""
}
msg := e.MaskSensitiveError()
msg := e.MaskSensitiveError(upstreamURLs...)
if e.StatusCode == 0 {
return msg
}
@@ -177,7 +179,9 @@ func (e *NewAPIError) SetMessage(message string) {
e.Err = errors.New(message)
}
func (e *NewAPIError) ToOpenAIError() OpenAIError {
// ToOpenAIError masks the upstream addresses in the message; pass those of the
// channel that produced the error, or none when no channel did.
func (e *NewAPIError) ToOpenAIError(upstreamURLs ...string) OpenAIError {
var result OpenAIError
switch e.errorType {
case ErrorTypeOpenAIError:
@@ -202,7 +206,7 @@ func (e *NewAPIError) ToOpenAIError() OpenAIError {
}
}
if e.errorCode != ErrorCodeCountTokenFailed {
result.Message = kitutil.MaskSensitiveInfo(result.Message)
result.Message = kitutil.MaskSensitiveInfo(result.Message, upstreamURLs...)
}
if result.Message == "" {
result.Message = string(e.errorType)
@@ -210,7 +214,8 @@ func (e *NewAPIError) ToOpenAIError() OpenAIError {
return result
}
func (e *NewAPIError) ToClaudeError() ClaudeError {
// ToClaudeError masks the upstream addresses like ToOpenAIError.
func (e *NewAPIError) ToClaudeError(upstreamURLs ...string) ClaudeError {
var result ClaudeError
switch e.errorType {
case ErrorTypeOpenAIError:
@@ -231,7 +236,7 @@ func (e *NewAPIError) ToClaudeError() ClaudeError {
}
}
if e.errorCode != ErrorCodeCountTokenFailed {
result.Message = kitutil.MaskSensitiveInfo(result.Message)
result.Message = kitutil.MaskSensitiveInfo(result.Message, upstreamURLs...)
}
if result.Message == "" {
result.Message = string(e.errorType)
relaykit/types/relay_format.go
+1
View File
@@ -9,6 +9,7 @@ const (
RelayFormatOpenAIResponses = "openai_responses"
RelayFormatOpenAIResponsesCompaction = "openai_responses_compaction"
RelayFormatOpenAIAlphaSearch = "openai_alpha_search"
RelayFormatDeepSeekTools = "deepseek_tools"
RelayFormatOpenAIAudio = "openai_audio"
RelayFormatOpenAIImage = "openai_image"
RelayFormatOpenAIRealtime = "openai_realtime"