feat: 支持对话与设置页单图编辑
This commit is contained in:
1 parent
7e2ec28b80
commit
99e087112b
8 files changed
+361
-30
No files matched your search
@@ -7,6 +7,7 @@
|
|||||||
- **接口配置**:设置 Base URL 和 API Key,支持第三方兼容服务。
|
- **接口配置**:设置 Base URL 和 API Key,支持第三方兼容服务。
|
||||||
- **显式选择模型**:手动维护模型 ID,或从接口读取模型列表;可设置对话默认模型,也可以每次在对话中指定。
|
- **显式选择模型**:手动维护模型 ID,或从接口读取模型列表;可设置对话默认模型,也可以每次在对话中指定。
|
||||||
- **对话生图**:助手通过 `imagegen_status` 查询配置,通过 `imagegen_generate` 生成图片。
|
- **对话生图**:助手通过 `imagegen_status` 查询配置,通过 `imagegen_generate` 生成图片。
|
||||||
|
- **图片编辑**:对话工具 `imagegen_edit` 与「编辑图片」页面支持上传单张 PNG/JPEG/WebP,调用 `/images/edits`,保留输入原图并另存结果。
|
||||||
- **直接生成**:在「设置 → 生图 → 生成图片」中输入提示词、选择模型并生成。
|
- **直接生成**:在「设置 → 生图 → 生成图片」中输入提示词、选择模型并生成。
|
||||||
- **图片输出**:支持 base64 和图片 URL 响应,提供图片预览与原图下载。
|
- **图片输出**:支持 base64 和图片 URL 响应,提供图片预览与原图下载。
|
||||||
- **技能复用**:附带 imagegen 提示词规范和示例,并注册 DSH 专用的 `dsh-imagegen` 技能。
|
- **技能复用**:附带 imagegen 提示词规范和示例,并注册 DSH 专用的 `dsh-imagegen` 技能。
|
||||||
@@ -104,12 +105,35 @@ git+https://git.crash.work/cneicy/dsh-imagegen.git
|
|||||||
4. 按需展开尺寸、质量与数量设置。
|
4. 按需展开尺寸、质量与数量设置。
|
||||||
5. 点击「生成图片」,完成后预览并下载原图。
|
5. 点击「生成图片」,完成后预览并下载原图。
|
||||||
|
|
||||||
|
### 编辑已有图片(1.1.0)
|
||||||
|
|
||||||
|
先确认服务商和所选模型支持 `POST /v1/images/edits`(`edits` 为复数;Base URL 已有 `/v1` 时只追加 `/images/edits`)。支持生图不代表支持编辑。
|
||||||
|
|
||||||
|
- **对话中**:上传图片,或指定已有图片的本机路径,例如「把这张图的背景改成海边,保持人物、衣服和姿势不变,用我设置的默认模型」。助手使用附件的本机路径调用 `imagegen_edit`。
|
||||||
|
- **设置页**:进入「设置 → 生图 → 编辑图片」,选择图片并预览,选择模型,输入修改要求,然后点击「编辑图片」。
|
||||||
|
- 输入仅支持一张 PNG/JPEG/WebP,最大 20 MiB;DSH 附件服务还会检查图片能否解码及部署的像素限制。暂不支持 GIF、蒙版或多图输入。
|
||||||
|
- 原图字节会发送到所配置的服务商;请求可能收费,失败不自动重试。输出另存为新文件,输入不会被覆盖。
|
||||||
|
- 对话附件可能已经过 DSH 缩放;要使用原始分辨率,请选择本机原文件或在设置页上传原文件。
|
||||||
|
|
||||||
|
工具参数示例(路径替换为实际存在的绝对路径):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"image_path": "D:/Pictures/input.png",
|
||||||
|
"model": "gpt-image-2.5-sunburst",
|
||||||
|
"prompt": "只将背景改成夕阳下的海边,保持人物、衣服和姿势不变"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
编辑工具复用生图的 `n`、`size`、`quality`、`background`、`output_format` 可选参数;是否支持由服务商决定。设置页通过 `POST /api/imagegen/edit` 提交 multipart 表单:`image` 为文件,`input` 为上述生成参数的 JSON(不包含 `image_path`)。浏览器路由只接收上传文件,不读取请求提供的本机路径。
|
||||||
|
|
||||||
### 对话工具
|
### 对话工具
|
||||||
|
|
||||||
| 工具 | 用途 |
|
| 工具 | 用途 |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `imagegen_status` | 返回 Base URL、是否已配置、候选模型、默认模型与输出目录;不返回密钥。 |
|
| `imagegen_status` | 返回 Base URL、是否已配置、候选模型、默认模型与输出目录;不返回密钥。 |
|
||||||
| `imagegen_generate` | 以明确的模型和提示词生成图片,并返回 DSH 图片附件及原图路径。 |
|
| `imagegen_generate` | 以明确的模型和提示词生成图片,并返回 DSH 图片附件及原图路径。 |
|
||||||
|
| `imagegen_edit` | 上传 `image_path` 指定的图片,按提示词编辑并返回新图片附件和路径。 |
|
||||||
|
|
||||||
生成参数示例:
|
生成参数示例:
|
||||||
|
|
||||||
@@ -164,7 +188,7 @@ git+https://git.crash.work/cneicy/dsh-imagegen.git
|
|||||||
|
|
||||||
**模型不支持尺寸、质量或透明背景**:去掉不受支持的可选参数,或显式选择支持这些能力的模型。第三方服务也可能忽略某些参数。
|
**模型不支持尺寸、质量或透明背景**:去掉不受支持的可选参数,或显式选择支持这些能力的模型。第三方服务也可能忽略某些参数。
|
||||||
|
|
||||||
**能否用参考图或修改已有图片?** 当前只实现文生图,不支持参考图输入、图像编辑或蒙版。附带的上游 imagegen 技能资源包含编辑说明,但不代表本插件已实现编辑接口。
|
**能否用参考图或修改已有图片?** 1.1.0 起支持单张图片加文字指令编辑,调用 `/images/edits`。暂不支持蒙版和多图输入。若编辑返回 404,请检查服务商是否提供该端点;插件不会自动降级成文生图。
|
||||||
|
|
||||||
## 开发与验证
|
## 开发与验证
|
||||||
|
|
||||||
|
|||||||
+57
-10
@@ -4,7 +4,8 @@ window.__ModuleLoader__.load({ id: 'dsh-imagegen-local', factory(require) {
|
|||||||
const { useState, useEffect } = React;
|
const { useState, useEffect } = React;
|
||||||
const style = `.ig-root{color:var(--dsw-alias-label-primary);font-size:14px;max-width:920px;margin:0 auto;padding:8px 4px 28px}.ig-root *{box-sizing:border-box}.ig-root h2{font-size:21px;margin:0 0 8px}.ig-root h3{font-size:16px;margin:0 0 14px}.ig-muted{color:var(--dsw-alias-label-secondary);line-height:1.6}.ig-card{border:1px solid var(--dsw-alias-border-l1);border-radius:16px;padding:20px;margin-top:20px;background:var(--dsw-alias-bg-layer-1)}.ig-field{display:flex;flex-direction:column;gap:7px;margin:14px 0;min-width:0}.ig-field span{font-weight:500}.ig-root input,.ig-root textarea,.ig-root select{width:100%;background:var(--dsw-alias-bg-base);color:var(--dsw-alias-label-primary);border:1px solid var(--dsw-alias-border-l2);border-radius:9px;padding:10px 12px;font:inherit}.ig-root input:focus,.ig-root textarea:focus,.ig-root select:focus{outline:2px solid var(--dsw-alias-brand-primary);outline-offset:1px}.ig-root textarea{resize:vertical}.ig-row{display:flex;gap:12px;flex-wrap:wrap;align-items:center}.ig-row>.ig-field{flex:1 1 160px}.ig-root button,.ig-link{border:1px solid var(--dsw-alias-border-l2);border-radius:9px;padding:9px 15px;background:var(--dsw-alias-bg-layer-2);color:var(--dsw-alias-label-primary);cursor:pointer;font:inherit}.ig-root button:disabled{opacity:.5;cursor:default}.ig-root .ig-primary{background:var(--dsw-alias-brand-primary);color:#fff;border-color:transparent}.ig-message{margin-top:14px;white-space:pre-wrap;overflow-wrap:anywhere}.ig-error{color:var(--dsw-alias-state-error-primary)}.ig-success{color:var(--dsw-alias-state-success-primary)}.ig-gallery{display:grid;grid-template-columns:repeat(auto-fit,minmax(220px,1fr));gap:16px;margin-top:20px}.ig-gallery img{display:block;width:100%;height:auto;max-height:500px;object-fit:contain;border-radius:10px;background:var(--dsw-alias-bg-base)}.ig-gallery a{color:var(--dsw-alias-brand-primary);display:inline-block;margin-top:10px}.ig-gallery small{display:block;overflow-wrap:anywhere;margin-top:8px}.ig-check{display:flex;align-items:center;gap:8px}.ig-check input{width:auto}.ig-status{border-radius:20px;padding:4px 10px;background:var(--dsw-alias-bg-layer-2);font-size:12px}.ig-tabs{display:flex;gap:8px;margin:20px 0}.ig-root .ig-tab-active{border-color:var(--dsw-alias-brand-primary);color:var(--dsw-alias-brand-primary)}.ig-root details{margin-top:12px}.ig-root summary{cursor:pointer;color:var(--dsw-alias-label-secondary)}`;
|
const style = `.ig-root{color:var(--dsw-alias-label-primary);font-size:14px;max-width:920px;margin:0 auto;padding:8px 4px 28px}.ig-root *{box-sizing:border-box}.ig-root h2{font-size:21px;margin:0 0 8px}.ig-root h3{font-size:16px;margin:0 0 14px}.ig-muted{color:var(--dsw-alias-label-secondary);line-height:1.6}.ig-card{border:1px solid var(--dsw-alias-border-l1);border-radius:16px;padding:20px;margin-top:20px;background:var(--dsw-alias-bg-layer-1)}.ig-field{display:flex;flex-direction:column;gap:7px;margin:14px 0;min-width:0}.ig-field span{font-weight:500}.ig-root input,.ig-root textarea,.ig-root select{width:100%;background:var(--dsw-alias-bg-base);color:var(--dsw-alias-label-primary);border:1px solid var(--dsw-alias-border-l2);border-radius:9px;padding:10px 12px;font:inherit}.ig-root input:focus,.ig-root textarea:focus,.ig-root select:focus{outline:2px solid var(--dsw-alias-brand-primary);outline-offset:1px}.ig-root textarea{resize:vertical}.ig-row{display:flex;gap:12px;flex-wrap:wrap;align-items:center}.ig-row>.ig-field{flex:1 1 160px}.ig-root button,.ig-link{border:1px solid var(--dsw-alias-border-l2);border-radius:9px;padding:9px 15px;background:var(--dsw-alias-bg-layer-2);color:var(--dsw-alias-label-primary);cursor:pointer;font:inherit}.ig-root button:disabled{opacity:.5;cursor:default}.ig-root .ig-primary{background:var(--dsw-alias-brand-primary);color:#fff;border-color:transparent}.ig-message{margin-top:14px;white-space:pre-wrap;overflow-wrap:anywhere}.ig-error{color:var(--dsw-alias-state-error-primary)}.ig-success{color:var(--dsw-alias-state-success-primary)}.ig-gallery{display:grid;grid-template-columns:repeat(auto-fit,minmax(220px,1fr));gap:16px;margin-top:20px}.ig-gallery img{display:block;width:100%;height:auto;max-height:500px;object-fit:contain;border-radius:10px;background:var(--dsw-alias-bg-base)}.ig-gallery a{color:var(--dsw-alias-brand-primary);display:inline-block;margin-top:10px}.ig-gallery small{display:block;overflow-wrap:anywhere;margin-top:8px}.ig-check{display:flex;align-items:center;gap:8px}.ig-check input{width:auto}.ig-status{border-radius:20px;padding:4px 10px;background:var(--dsw-alias-bg-layer-2);font-size:12px}.ig-tabs{display:flex;gap:8px;margin:20px 0}.ig-root .ig-tab-active{border-color:var(--dsw-alias-brand-primary);color:var(--dsw-alias-brand-primary)}.ig-root details{margin-top:12px}.ig-root summary{cursor:pointer;color:var(--dsw-alias-label-secondary)}`;
|
||||||
async function api(action, data, signal) {
|
async function api(action, data, signal) {
|
||||||
const response = await fetch(`/api/imagegen/${action}`, { method: data === undefined ? 'GET' : 'POST', credentials: 'same-origin', headers: data === undefined ? {} : { 'Content-Type': 'application/json' }, body: data === undefined ? undefined : JSON.stringify(data), signal });
|
const multipart = data instanceof FormData;
|
||||||
|
const response = await fetch(`/api/imagegen/${action}`, { method: data === undefined ? 'GET' : 'POST', credentials: 'same-origin', headers: data === undefined || multipart ? {} : { 'Content-Type': 'application/json' }, body: data === undefined ? undefined : multipart ? data : JSON.stringify(data), signal });
|
||||||
const value = await response.json();
|
const value = await response.json();
|
||||||
if (!response.ok) throw new Error(value.error || `请求失败(${response.status})`);
|
if (!response.ok) throw new Error(value.error || `请求失败(${response.status})`);
|
||||||
return value;
|
return value;
|
||||||
@@ -28,6 +29,26 @@ window.__ModuleLoader__.load({ id: 'dsh-imagegen-local', factory(require) {
|
|||||||
const [message, setMessage] = useState('');
|
const [message, setMessage] = useState('');
|
||||||
const [error, setError] = useState('');
|
const [error, setError] = useState('');
|
||||||
const [results, setResults] = useState(null);
|
const [results, setResults] = useState(null);
|
||||||
|
const [image, setImage] = useState(null);
|
||||||
|
const [preview, setPreview] = useState('');
|
||||||
|
const [fileInputKey, setFileInputKey] = useState(0);
|
||||||
|
useEffect(() => {
|
||||||
|
if (!image) { setPreview(''); return; }
|
||||||
|
const url = URL.createObjectURL(image);
|
||||||
|
setPreview(url);
|
||||||
|
return () => URL.revokeObjectURL(url);
|
||||||
|
}, [image]);
|
||||||
|
function selectImage(event) {
|
||||||
|
const file = event.target.files?.[0];
|
||||||
|
setError(''); setMessage('');
|
||||||
|
if (!file) { setImage(null); return; }
|
||||||
|
if (!['image/png', 'image/jpeg', 'image/webp'].includes(file.type) || !file.size || file.size > 20 * 1024 * 1024) {
|
||||||
|
setImage(null); event.target.value = '';
|
||||||
|
setError('请选择一张 PNG、JPEG 或 WebP 图片,文件大小不超过 20 MiB,且不能为空。');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
setImage(file);
|
||||||
|
}
|
||||||
const [controller] = useState(() => ({ current: null }));
|
const [controller] = useState(() => ({ current: null }));
|
||||||
function load(value) { setConfig(value); setBaseUrl(value.baseUrl); setModels(value.models.join('\n')); setDefault(value.defaultModel); setModel(current => current || value.defaultModel); }
|
function load(value) { setConfig(value); setBaseUrl(value.baseUrl); setModels(value.models.join('\n')); setDefault(value.defaultModel); setModel(current => current || value.defaultModel); }
|
||||||
useEffect(() => { const ac = new AbortController(); api('settings', undefined, ac.signal).then(load).catch(e => { if (!ac.signal.aborted) setError(e.message); }); return () => { ac.abort(); controller.current?.abort(); }; }, []);
|
useEffect(() => { const ac = new AbortController(); api('settings', undefined, ac.signal).then(load).catch(e => { if (!ac.signal.aborted) setError(e.message); }); return () => { ac.abort(); controller.current?.abort(); }; }, []);
|
||||||
@@ -36,15 +57,33 @@ window.__ModuleLoader__.load({ id: 'dsh-imagegen-local', factory(require) {
|
|||||||
const changed = config && (baseUrl !== config.baseUrl || models !== config.models.join('\n') || defaultModel !== config.defaultModel || apiKey || clearKey);
|
const changed = config && (baseUrl !== config.baseUrl || models !== config.models.join('\n') || defaultModel !== config.defaultModel || apiKey || clearKey);
|
||||||
async function save() { await run('save', async () => { const value = await api('settings', { baseUrl, models: modelList, defaultModel, ...(apiKey ? { apiKey } : {}), clearKey }); load(value); setApiKey(''); setClearKey(false); setMessage('设置已保存,对话生图立即使用这些配置。'); }); }
|
async function save() { await run('save', async () => { const value = await api('settings', { baseUrl, models: modelList, defaultModel, ...(apiKey ? { apiKey } : {}), clearKey }); load(value); setApiKey(''); setClearKey(false); setMessage('设置已保存,对话生图立即使用这些配置。'); }); }
|
||||||
async function discover() { await run('models', async () => { const value = await api('models', {}); setModels([...new Set([...modelList, ...value.models])].join('\n')); setMessage(`已读取 ${value.models.length} 个模型。列表可能包含聊天模型,请保留支持生图的模型后保存。`); }); }
|
async function discover() { await run('models', async () => { const value = await api('models', {}); setModels([...new Set([...modelList, ...value.models])].join('\n')); setMessage(`已读取 ${value.models.length} 个模型。列表可能包含聊天模型,请保留支持生图的模型后保存。`); }); }
|
||||||
async function generate() { await run('generate', async () => { const ac = new AbortController(); controller.current = ac; try { const value = await api('generate', { model: model.trim(), prompt, n: Number(count), ...(size.trim() ? { size: size.trim() } : {}), ...(quality ? { quality } : {}) }, ac.signal); setResults(value); setMessage(`已使用 ${value.model} 生成 ${value.images.length} 张图片。`); } finally { controller.current = null; } }); }
|
async function generate() {
|
||||||
|
if (busy || controller.current || !model.trim() || !prompt.trim() || (tab === 'edit' && !image)) return;
|
||||||
|
const action = tab === 'edit' ? 'edit' : 'generate';
|
||||||
|
await run(action, async () => {
|
||||||
|
const ac = new AbortController(); controller.current = ac;
|
||||||
|
try {
|
||||||
|
const args = { model: model.trim(), prompt, n: Number(count), ...(size.trim() ? { size: size.trim() } : {}), ...(quality ? { quality } : {}) };
|
||||||
|
let body = args;
|
||||||
|
if (action === 'edit') {
|
||||||
|
body = new FormData();
|
||||||
|
body.set('image', image, image.name);
|
||||||
|
body.set('input', JSON.stringify(args));
|
||||||
|
}
|
||||||
|
const value = await api(action, body, ac.signal);
|
||||||
|
setResults(value);
|
||||||
|
setMessage(`已使用 ${value.model} ${action === 'edit' ? '编辑并生成' : '生成'} ${value.images.length} 张图片。`);
|
||||||
|
} finally { controller.current = null; }
|
||||||
|
});
|
||||||
|
}
|
||||||
if (!config) return h('div', { className: 'ig-root' }, h('h2', null, '生图'), h('p', { role: error ? 'alert' : 'status' }, error || '正在读取生图设置…'));
|
if (!config) return h('div', { className: 'ig-root' }, h('h2', null, '生图'), h('p', { role: error ? 'alert' : 'status' }, error || '正在读取生图设置…'));
|
||||||
return h('div', { className: 'ig-root' },
|
return h('div', { className: 'ig-root' },
|
||||||
h('div', { className: 'ig-row' }, h('h2', null, '生图'), h('span', { className: 'ig-status' }, config.configured ? '接口已配置' : '待配置')),
|
h('div', { className: 'ig-row' }, h('h2', null, '生图'), h('span', { className: 'ig-status' }, config.configured ? '接口已配置' : '待配置')),
|
||||||
h('p', { className: 'ig-muted' }, '配置兼容 OpenAI Images API 的服务,然后在对话中指定模型生图,或在这里直接生成。'),
|
h('p', { className: 'ig-muted' }, '配置图片服务,选择模型后生成新图片,或上传图片进行编辑。'),
|
||||||
h('div', { className: 'ig-tabs' }, ['settings', 'generate'].map(id => h('button', { key: id, type: 'button', className: tab === id ? 'ig-tab-active' : '', onClick: () => { setTab(id); setMessage(''); setError(''); } }, id === 'settings' ? '接口设置' : '生成图片'))),
|
h('div', { className: 'ig-tabs', role: 'tablist', 'aria-label': '生图功能' }, ['settings', 'generate', 'edit'].map(id => h('button', { key: id, type: 'button', role: 'tab', 'aria-selected': tab === id, disabled: !!busy, className: tab === id ? 'ig-tab-active' : '', onClick: () => { setTab(id); setMessage(''); setError(''); } }, { settings: '接口设置', generate: '生成图片', edit: '编辑图片' }[id]))),
|
||||||
tab === 'settings' ? h('div', { className: 'ig-card' },
|
tab === 'settings' ? h('div', { className: 'ig-card' },
|
||||||
h('h3', null, '接口与凭据'),
|
h('h3', null, '接口与凭据'),
|
||||||
h(Field, { label: 'Base URL', hint: '填写 API 根地址,如 https://api.openai.com/v1;插件会追加 /images/generations。' }, h('input', { value: baseUrl, onChange: e => setBaseUrl(e.target.value), placeholder: 'https://api.openai.com/v1', spellCheck: false, disabled: !!busy })),
|
h(Field, { label: 'Base URL', hint: '填写服务商提供的 API 根地址,如 https://api.openai.com/v1。' }, h('input', { value: baseUrl, onChange: e => setBaseUrl(e.target.value), placeholder: 'https://api.openai.com/v1', spellCheck: false, disabled: !!busy })),
|
||||||
h(Field, { label: 'API Key', hint: baseUrl === config.baseUrl && config.configured ? '已保存。留空保持当前密钥;密钥不会回显。' : '为这个 Base URL 输入密钥。不同地址的密钥分别保存。' }, h('input', { type: 'password', value: apiKey, onChange: e => { setApiKey(e.target.value); setClearKey(false); }, autoComplete: 'new-password', placeholder: '输入 API Key', disabled: !!busy })),
|
h(Field, { label: 'API Key', hint: baseUrl === config.baseUrl && config.configured ? '已保存。留空保持当前密钥;密钥不会回显。' : '为这个 Base URL 输入密钥。不同地址的密钥分别保存。' }, h('input', { type: 'password', value: apiKey, onChange: e => { setApiKey(e.target.value); setClearKey(false); }, autoComplete: 'new-password', placeholder: '输入 API Key', disabled: !!busy })),
|
||||||
h('label', { className: 'ig-check ig-muted' }, h('input', { type: 'checkbox', checked: clearKey, disabled: !!busy, onChange: e => { setClearKey(e.target.checked); if (e.target.checked) setApiKey(''); } }), '清除这个地址已保存的密钥'),
|
h('label', { className: 'ig-check ig-muted' }, h('input', { type: 'checkbox', checked: clearKey, disabled: !!busy, onChange: e => { setClearKey(e.target.checked); if (e.target.checked) setApiKey(''); } }), '清除这个地址已保存的密钥'),
|
||||||
h(Field, { label: '可选模型', hint: '每行一个精确模型 ID。支持手动填写第三方模型;模型是否可用由服务商决定。' }, h('textarea', { value: models, rows: 4, spellCheck: false, disabled: !!busy, onChange: e => setModels(e.target.value) })),
|
h(Field, { label: '可选模型', hint: '每行一个精确模型 ID。支持手动填写第三方模型;模型是否可用由服务商决定。' }, h('textarea', { value: models, rows: 4, spellCheck: false, disabled: !!busy, onChange: e => setModels(e.target.value) })),
|
||||||
@@ -52,16 +91,24 @@ window.__ModuleLoader__.load({ id: 'dsh-imagegen-local', factory(require) {
|
|||||||
h('div', { className: 'ig-row' }, h('button', { className: 'ig-primary', disabled: !!busy, onClick: save }, busy === 'save' ? '保存中…' : '保存设置'), h('button', { disabled: !!busy || !!changed || !config.configured, onClick: discover }, busy === 'models' ? '读取中…' : '从接口读取模型')),
|
h('div', { className: 'ig-row' }, h('button', { className: 'ig-primary', disabled: !!busy, onClick: save }, busy === 'save' ? '保存中…' : '保存设置'), h('button', { disabled: !!busy || !!changed || !config.configured, onClick: discover }, busy === 'models' ? '读取中…' : '从接口读取模型')),
|
||||||
h('p', { className: 'ig-muted' }, '先保存接口设置,再读取模型。部分服务不提供模型列表,可以直接手动填写。')
|
h('p', { className: 'ig-muted' }, '先保存接口设置,再读取模型。部分服务不提供模型列表,可以直接手动填写。')
|
||||||
) : h('div', { className: 'ig-card' },
|
) : h('div', { className: 'ig-card' },
|
||||||
h('h3', null, '生成图片'),
|
h('h3', null, tab === 'edit' ? '编辑图片' : '生成图片'),
|
||||||
h(Field, { label: '本次生成模型', hint: '选择已配置模型,也可以填写完整的自定义模型 ID。' }, h('input', { list: 'ig-model-options', value: model, disabled: !!busy, onChange: e => setModel(e.target.value), placeholder: '选择或输入模型' }), h('datalist', { id: 'ig-model-options' }, config.models.map(m => option(m)))),
|
tab === 'edit' && h('div', null,
|
||||||
h(Field, { label: '画面描述' }, h('textarea', { rows: 5, value: prompt, maxLength: 32000, disabled: !!busy, placeholder: '描述主体、风格、构图和需要出现的文字…', onChange: e => setPrompt(e.target.value) })),
|
h(Field, { label: '原图', hint: '选择一张 PNG、JPEG 或 WebP 图片,最大 20 MiB。' }, h('input', { key: fileInputKey, type: 'file', accept: 'image/png,image/jpeg,image/webp', disabled: !!busy, onChange: selectImage })),
|
||||||
|
image && h('div', null,
|
||||||
|
preview && h('img', { src: preview, alt: '待编辑的原图', style: { display: 'block', maxWidth: '100%', maxHeight: '320px', objectFit: 'contain', borderRadius: '10px' } }),
|
||||||
|
h('p', { className: 'ig-muted', style: { overflowWrap: 'anywhere' } }, image.name),
|
||||||
|
h('button', { type: 'button', disabled: !!busy, onClick: () => { setImage(null); setFileInputKey(value => value + 1); } }, '移除图片')
|
||||||
|
)
|
||||||
|
),
|
||||||
|
h(Field, { label: '本次使用模型', hint: tab === 'edit' ? '请选择支持图片编辑的模型,也可以填写完整的自定义模型 ID。' : '选择已配置模型,也可以填写完整的自定义模型 ID。' }, h('input', { list: 'ig-model-options', value: model, disabled: !!busy, onChange: e => setModel(e.target.value), placeholder: '选择或输入模型' }), h('datalist', { id: 'ig-model-options' }, config.models.map(m => option(m)))),
|
||||||
|
h(Field, { label: tab === 'edit' ? '修改要求' : '画面描述' }, h('textarea', { rows: 5, value: prompt, maxLength: 32000, disabled: !!busy, placeholder: tab === 'edit' ? '描述要修改的内容,以及需要保留的人物、构图或细节…' : '描述主体、风格、构图和需要出现的文字…', onChange: e => setPrompt(e.target.value) })),
|
||||||
h('details', null, h('summary', null, '尺寸、质量与数量'), h('div', { className: 'ig-row' },
|
h('details', null, h('summary', null, '尺寸、质量与数量'), h('div', { className: 'ig-row' },
|
||||||
h(Field, { label: '尺寸', hint: '留空使用接口默认' }, h('input', { value: size, placeholder: '如 1024x1024', disabled: !!busy, onChange: e => setSize(e.target.value) })),
|
h(Field, { label: '尺寸', hint: '留空使用接口默认' }, h('input', { value: size, placeholder: '如 1024x1024', disabled: !!busy, onChange: e => setSize(e.target.value) })),
|
||||||
h(Field, { label: '质量' }, h('select', { value: quality, disabled: !!busy, onChange: e => setQuality(e.target.value) }, option('', '接口默认'), ['auto','low','medium','high','standard','hd'].map(m => option(m)))),
|
h(Field, { label: '质量' }, h('select', { value: quality, disabled: !!busy, onChange: e => setQuality(e.target.value) }, option('', '接口默认'), ['auto','low','medium','high','standard','hd'].map(m => option(m)))),
|
||||||
h(Field, { label: '数量' }, h('select', { value: count, disabled: !!busy, onChange: e => setCount(e.target.value) }, ['1','2','3','4'].map(m => option(m))))
|
h(Field, { label: '数量' }, h('select', { value: count, disabled: !!busy, onChange: e => setCount(e.target.value) }, ['1','2','3','4'].map(m => option(m))))
|
||||||
)),
|
)),
|
||||||
h('p', { className: 'ig-muted' }, `本次请求发送至 ${config.baseUrl},按服务商规则计费。`),
|
h('p', { className: 'ig-muted' }, `本次${tab === 'edit' ? '图片与修改要求' : '请求'}发送至 ${config.baseUrl},按服务商规则计费。`),
|
||||||
h('div', { className: 'ig-row' }, h('button', { className: 'ig-primary', disabled: !!busy || !config.configured || !model.trim() || !prompt.trim(), onClick: generate }, busy === 'generate' ? '正在生成…' : '生成图片'), busy === 'generate' && h('button', { onClick: () => controller.current?.abort() }, '停止等待')),
|
h('div', { className: 'ig-row' }, h('button', { className: 'ig-primary', disabled: !!busy || !config.configured || !model.trim() || !prompt.trim() || (tab === 'edit' && !image), onClick: generate }, busy === 'edit' ? '正在编辑…' : busy === 'generate' ? '正在生成…' : tab === 'edit' ? '编辑图片' : '生成图片'), ['generate', 'edit'].includes(busy) && h('button', { onClick: () => controller.current?.abort() }, '停止等待')),
|
||||||
!config.configured && h('p', { className: 'ig-muted' }, '请先在接口设置中保存 API Key。')
|
!config.configured && h('p', { className: 'ig-muted' }, '请先在接口设置中保存 API Key。')
|
||||||
),
|
),
|
||||||
error && h('div', { role: 'alert', className: 'ig-message ig-error' }, error),
|
error && h('div', { role: 'alert', className: 'ig-message ig-error' }, error),
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
const MAX_BYTES = 64 * 1024 * 1024;
|
const MAX_BYTES = 64 * 1024 * 1024;
|
||||||
|
const MAX_EDIT_BYTES = 20 * 1024 * 1024;
|
||||||
const TIMEOUT_MS = 180_000;
|
const TIMEOUT_MS = 180_000;
|
||||||
const QUALITY = new Set(['auto', 'low', 'medium', 'high', 'standard', 'hd']);
|
const QUALITY = new Set(['auto', 'low', 'medium', 'high', 'standard', 'hd']);
|
||||||
const BACKGROUND = new Set(['auto', 'transparent', 'opaque']);
|
const BACKGROUND = new Set(['auto', 'transparent', 'opaque']);
|
||||||
@@ -52,6 +53,7 @@ async function operation(signal, work) {
|
|||||||
const timer = setTimeout(() => { timedOut = true; controller.abort(); }, TIMEOUT_MS);
|
const timer = setTimeout(() => { timedOut = true; controller.abort(); }, TIMEOUT_MS);
|
||||||
timer.unref?.();
|
timer.unref?.();
|
||||||
try {
|
try {
|
||||||
|
if (controller.signal.aborted) throw new Error('Request aborted.');
|
||||||
return await work(controller.signal);
|
return await work(controller.signal);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
if (controller.signal.aborted) throw new Error(timedOut ? 'Request timed out after 180 seconds.' : 'Request aborted.');
|
if (controller.signal.aborted) throw new Error(timedOut ? 'Request timed out after 180 seconds.' : 'Request aborted.');
|
||||||
@@ -119,6 +121,19 @@ function mediaType(bytes) {
|
|||||||
throw new Error('Image bytes are not a supported PNG, JPEG, WebP, or GIF image.');
|
throw new Error('Image bytes are not a supported PNG, JPEG, WebP, or GIF image.');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Validate one PNG/JPEG/WebP upload by signature, not full image decoding.
|
||||||
|
* Accepts Buffer/Uint8Array up to 20 MiB; returns an owned Buffer copy and MIME type.
|
||||||
|
*/
|
||||||
|
export function validateEditImage(data) {
|
||||||
|
if (!(data instanceof Uint8Array) || data.byteLength === 0) throw new Error('Edit image must be nonempty PNG, JPEG, or WebP bytes.');
|
||||||
|
if (data.byteLength > MAX_EDIT_BYTES) throw new Error('Edit image exceeds the 20 MiB upload limit.');
|
||||||
|
const bytes = Buffer.from(data);
|
||||||
|
let type;
|
||||||
|
try { type = mediaType(bytes); } catch { /* Use the upload-specific error below. */ }
|
||||||
|
if (!['image/png', 'image/jpeg', 'image/webp'].includes(type)) throw new Error('Edit image must be a PNG, JPEG, or WebP image.');
|
||||||
|
return { data: bytes, mediaType: type };
|
||||||
|
}
|
||||||
|
|
||||||
function imageUrl(value) {
|
function imageUrl(value) {
|
||||||
let url;
|
let url;
|
||||||
try { url = new URL(value); } catch { throw new Error('API returned an invalid image URL.'); }
|
try { url = new URL(value); } catch { throw new Error('API returned an invalid image URL.'); }
|
||||||
@@ -144,6 +159,23 @@ export async function generateImages({ baseUrl, apiKey, input, signal }) {
|
|||||||
const headers = { ...apiHeaders(apiKey), 'Content-Type': 'application/json' };
|
const headers = { ...apiHeaders(apiKey), 'Content-Type': 'application/json' };
|
||||||
const body = JSON.stringify(generation);
|
const body = JSON.stringify(generation);
|
||||||
if (Buffer.byteLength(body) > MAX_BYTES) throw new Error('Request exceeds the 64 MB request limit.');
|
if (Buffer.byteLength(body) > MAX_BYTES) throw new Error('Request exceeds the 64 MB request limit.');
|
||||||
|
return requestImages({ endpoint, headers, body, generation, signal });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Edit one image; input and return value share the generateImages contract. */
|
||||||
|
export async function editImages({ baseUrl, apiKey, input, image, signal }) {
|
||||||
|
const endpoint = `${validateBaseUrl(baseUrl)}/images/edits`;
|
||||||
|
const generation = validateGeneration(input);
|
||||||
|
const headers = apiHeaders(apiKey);
|
||||||
|
const upload = validateEditImage(image);
|
||||||
|
const extension = { 'image/png': 'png', 'image/jpeg': 'jpg', 'image/webp': 'webp' }[upload.mediaType];
|
||||||
|
const body = new FormData();
|
||||||
|
for (const [key, value] of Object.entries(generation)) body.set(key, String(value));
|
||||||
|
body.set('image', new Blob([upload.data], { type: upload.mediaType }), `image.${extension}`);
|
||||||
|
return requestImages({ endpoint, headers, body, generation, signal });
|
||||||
|
}
|
||||||
|
|
||||||
|
function requestImages({ endpoint, headers, body, generation, signal }) {
|
||||||
return operation(signal, async combinedSignal => {
|
return operation(signal, async combinedSignal => {
|
||||||
const budget = { remaining: MAX_BYTES };
|
const budget = { remaining: MAX_BYTES };
|
||||||
const json = await fetchJson(endpoint, { method: 'POST', headers, body, signal: combinedSignal }, budget);
|
const json = await fetchJson(endpoint, { method: 'POST', headers, body, signal: combinedSignal }, budget);
|
||||||
|
|||||||
+57
-7
@@ -1,9 +1,11 @@
|
|||||||
import { readFile, writeFile, mkdir, rename } from 'node:fs/promises';
|
import { readFile, writeFile, mkdir, rename } from 'node:fs/promises';
|
||||||
|
import { createReadStream } from 'node:fs';
|
||||||
|
import { Readable, addAbortSignal } from 'node:stream';
|
||||||
import { homedir } from 'node:os';
|
import { homedir } from 'node:os';
|
||||||
import { join, dirname } from 'node:path';
|
import { join, dirname, isAbsolute } from 'node:path';
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { createHash, randomUUID } from 'node:crypto';
|
import { createHash, randomUUID } from 'node:crypto';
|
||||||
import { validateBaseUrl, validateGeneration, generateImages, listModels } from './engine.mjs';
|
import { validateBaseUrl, validateGeneration, generateImages, editImages, validateEditImage, listModels } from './engine.mjs';
|
||||||
|
|
||||||
export const name = 'imagegen-local';
|
export const name = 'imagegen-local';
|
||||||
export const inject = ['connection', 'tools', 'credentials', 'attachments', 'skills'];
|
export const inject = ['connection', 'tools', 'credentials', 'attachments', 'skills'];
|
||||||
@@ -42,12 +44,14 @@ export async function apply(ctx) {
|
|||||||
config = next;
|
config = next;
|
||||||
return view();
|
return view();
|
||||||
}
|
}
|
||||||
async function generate(input, signal) {
|
async function generate(input, signal, image) {
|
||||||
const args = validateGeneration(input);
|
const args = validateGeneration(input);
|
||||||
const snapshot = { ...config };
|
const snapshot = { ...config };
|
||||||
const apiKey = await readKey(snapshot.baseUrl);
|
const apiKey = await readKey(snapshot.baseUrl);
|
||||||
if (!apiKey) throw new Error('请先在 设置 → 生图 中为当前 Base URL 保存 API Key。');
|
if (!apiKey) throw new Error('请先在 设置 → 生图 中为当前 Base URL 保存 API Key。');
|
||||||
const result = await generateImages({ baseUrl: snapshot.baseUrl, apiKey, input: args, signal: signalFor(signal) });
|
const options = { baseUrl: snapshot.baseUrl, apiKey, input: args, signal: signalFor(signal) };
|
||||||
|
if (image !== undefined) await ctx.attachments.validateImage(validateEditImage(image));
|
||||||
|
const result = image === undefined ? await generateImages(options) : await editImages({ ...options, image });
|
||||||
await mkdir(outputs, { recursive: true });
|
await mkdir(outputs, { recursive: true });
|
||||||
const images = [];
|
const images = [];
|
||||||
for (const item of result.images) {
|
for (const item of result.images) {
|
||||||
@@ -61,8 +65,8 @@ export async function apply(ctx) {
|
|||||||
}
|
}
|
||||||
return { model: args.model, prompt: args.prompt, images };
|
return { model: args.model, prompt: args.prompt, images };
|
||||||
}
|
}
|
||||||
function route(path, methods, handler) {
|
function route(path, methods, handler, requestBody = 'buffered') {
|
||||||
ctx.connection.fetch.register({ path, methods, requestBody: 'buffered', fetch: async request => {
|
ctx.connection.fetch.register({ path, methods, requestBody, fetch: async request => {
|
||||||
try { return await handler(request); }
|
try { return await handler(request); }
|
||||||
catch (error) {
|
catch (error) {
|
||||||
// Never include arbitrary provider responses or request headers in a browser error.
|
// Never include arbitrary provider responses or request headers in a browser error.
|
||||||
@@ -92,6 +96,22 @@ export async function apply(ctx) {
|
|||||||
return json({ models: await listModels({ baseUrl: snapshot.baseUrl, apiKey, signal: signalFor(request.signal) }) });
|
return json({ models: await listModels({ baseUrl: snapshot.baseUrl, apiKey, signal: signalFor(request.signal) }) });
|
||||||
});
|
});
|
||||||
route('/api/imagegen/generate', ['POST'], async request => json(await generate(await body(request), request.signal)));
|
route('/api/imagegen/generate', ['POST'], async request => json(await generate(await body(request), request.signal)));
|
||||||
|
route('/api/imagegen/edit', ['POST'], async request => {
|
||||||
|
const limit = 21 * 1024 * 1024;
|
||||||
|
if (Number(request.headers.get('content-length')) > limit) throw new Error('编辑上传内容过大(图片最大 20 MiB)。');
|
||||||
|
if (!request.headers.get('content-type')?.startsWith('multipart/form-data;')) throw new Error('请使用图片上传表单。');
|
||||||
|
const bytes = await boundedBytes(request.body, limit, signalFor(request.signal));
|
||||||
|
let form;
|
||||||
|
try { form = await new Response(bytes, { headers: { 'Content-Type': request.headers.get('content-type') } }).formData(); }
|
||||||
|
catch { throw new Error('图片上传表单无效。'); }
|
||||||
|
if ([...form.keys()].some(key => !['image', 'input'].includes(key)) || form.getAll('image').length !== 1 || form.getAll('input').length !== 1) throw new Error('请只上传一张图片和一组编辑参数。');
|
||||||
|
const image = form.get('image'), raw = form.get('input');
|
||||||
|
if (!image || typeof image.arrayBuffer !== 'function' || image.size > 20 * 1024 * 1024) throw new Error('请选择不超过 20 MiB 的图片。');
|
||||||
|
if (typeof raw !== 'string' || raw.length > 100000) throw new Error('编辑参数无效。');
|
||||||
|
let input;
|
||||||
|
try { input = JSON.parse(raw); } catch { throw new Error('编辑参数必须为 JSON。'); }
|
||||||
|
return json(await generate(input, request.signal, Buffer.from(await image.arrayBuffer())));
|
||||||
|
}, 'streaming');
|
||||||
route('/api/imagegen/file', ['GET'], async request => {
|
route('/api/imagegen/file', ['GET'], async request => {
|
||||||
const filename = new URL(request.url).searchParams.get('name') || '';
|
const filename = new URL(request.url).searchParams.get('name') || '';
|
||||||
if (!/^\d+-[0-9a-f-]{36}\.(png|jpg|webp|gif)$/.test(filename)) return json({ error: '图片不存在。' }, 404);
|
if (!/^\d+-[0-9a-f-]{36}\.(png|jpg|webp|gif)$/.test(filename)) return json({ error: '图片不存在。' }, 404);
|
||||||
@@ -108,7 +128,7 @@ export async function apply(ctx) {
|
|||||||
execute: () => view(),
|
execute: () => view(),
|
||||||
isConcurrencySafe: () => true,
|
isConcurrencySafe: () => true,
|
||||||
});
|
});
|
||||||
ctx.tools.register({
|
const generationTool = {
|
||||||
name: 'imagegen_generate',
|
name: 'imagegen_generate',
|
||||||
description: 'Generate images using the configured Images API. Requires an exact model explicitly chosen by the user in chat or Settings → 生图 (query imagegen_status for their default). Never silently switch models. Returns saved original file paths and inline image attachments. API calls may incur charges. Text-to-image only; no reference-image editing. Reuse dsh-imagegen skill prompt guidance.',
|
description: 'Generate images using the configured Images API. Requires an exact model explicitly chosen by the user in chat or Settings → 生图 (query imagegen_status for their default). Never silently switch models. Returns saved original file paths and inline image attachments. API calls may incur charges. Text-to-image only; no reference-image editing. Reuse dsh-imagegen skill prompt guidance.',
|
||||||
parameters: { type: 'object', properties: {
|
parameters: { type: 'object', properties: {
|
||||||
@@ -124,10 +144,40 @@ export async function apply(ctx) {
|
|||||||
output: { schema, render: (_, value) => [text({ model: value.model, prompt: value.prompt, images: value.images.map(({ path, revisedPrompt }) => ({ path, revisedPrompt })) }), ...value.images.map(image => ({ type: 'image', attachment: image.attachment }))] },
|
output: { schema, render: (_, value) => [text({ model: value.model, prompt: value.prompt, images: value.images.map(({ path, revisedPrompt }) => ({ path, revisedPrompt })) }), ...value.images.map(image => ({ type: 'image', attachment: image.attachment }))] },
|
||||||
execute: (args, exec) => generate(args, exec.signal),
|
execute: (args, exec) => generate(args, exec.signal),
|
||||||
presentCall: args => ({ card: 'generic', title: `生图 · ${args.model}`, kind: 'fetch', rawInput: args }),
|
presentCall: args => ({ card: 'generic', title: `生图 · ${args.model}`, kind: 'fetch', rawInput: args }),
|
||||||
|
};
|
||||||
|
ctx.tools.register(generationTool);
|
||||||
|
ctx.tools.register({
|
||||||
|
...generationTool,
|
||||||
|
name: 'imagegen_edit',
|
||||||
|
description: 'Edit one user-provided image using POST /images/edits. Requires an absolute local image_path (uploaded attachment host path or existing image), exact user-selected model and edit prompt. Query imagegen_status first. PNG/JPEG/WebP up to 20 MiB; no masks or multiple inputs. Sends image bytes to the configured provider and may incur charges. Preserve requested unchanged features; never substitute text-only generation or silently change models.',
|
||||||
|
parameters: { ...generationTool.parameters, properties: { ...generationTool.parameters.properties, image_path: { type: 'string', description: 'Absolute host path of the user-authorized input image; use the attachment host path for a chat upload. Never guess a path.' } }, required: ['model', 'prompt', 'image_path'] },
|
||||||
|
execute: async (args, exec) => {
|
||||||
|
validateGeneration(args);
|
||||||
|
if (typeof args.image_path !== 'string' || !isAbsolute(args.image_path)) throw new Error('image_path 必须是图片的本机绝对路径。');
|
||||||
|
let image;
|
||||||
|
try { image = await boundedBytes(createReadStream(args.image_path), 20 * 1024 * 1024, signalFor(exec.signal)); }
|
||||||
|
catch { throw new Error('无法读取编辑图片,或图片超过 20 MiB,或读取已取消。'); }
|
||||||
|
return generate(args, exec.signal, image);
|
||||||
|
},
|
||||||
|
presentCall: args => ({ card: 'generic', title: `编辑图片 · ${args.model}`, kind: 'fetch', rawInput: args }),
|
||||||
});
|
});
|
||||||
ctx.skills.register({ name: 'dsh-imagegen', description: '使用生图插件生成图片,复用 imagegen 提示词规范,支持自定义 API 与显式模型选择。', source: 'runtime', path: join(root, 'skills', 'SKILL.md'), resourceBase: { kind: 'directory', path: join(root, 'skills') }, content: await readFile(join(root, 'skills', 'SKILL.md'), 'utf8') });
|
ctx.skills.register({ name: 'dsh-imagegen', description: '使用生图插件生成图片,复用 imagegen 提示词规范,支持自定义 API 与显式模型选择。', source: 'runtime', path: join(root, 'skills', 'SKILL.md'), resourceBase: { kind: 'directory', path: join(root, 'skills') }, content: await readFile(join(root, 'skills', 'SKILL.md'), 'utf8') });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function boundedBytes(source, limit, signal) {
|
||||||
|
if (!source) throw new Error('图片内容为空。');
|
||||||
|
const stream = typeof source.getReader === 'function' ? Readable.fromWeb(source) : source;
|
||||||
|
if (signal) addAbortSignal(signal, stream);
|
||||||
|
const chunks = [];
|
||||||
|
let length = 0;
|
||||||
|
for await (const chunk of stream) {
|
||||||
|
length += chunk.length;
|
||||||
|
if (length > limit) throw new Error('图片上传超过大小限制。');
|
||||||
|
chunks.push(chunk);
|
||||||
|
}
|
||||||
|
return Buffer.concat(chunks, length);
|
||||||
|
}
|
||||||
|
|
||||||
export function validateSettings(input) {
|
export function validateSettings(input) {
|
||||||
if (!input || typeof input !== 'object' || Array.isArray(input)) throw new Error('生图设置格式无效。');
|
if (!input || typeof input !== 'object' || Array.isArray(input)) throw new Error('生图设置格式无效。');
|
||||||
const baseUrl = validateBaseUrl(input.baseUrl);
|
const baseUrl = validateBaseUrl(input.baseUrl);
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "dsh-imagegen-local",
|
"name": "dsh-imagegen-local",
|
||||||
"version": "1.0.1",
|
"version": "1.1.0",
|
||||||
"description": "生图:自定义 Images API、显式模型选择及对话生图工具",
|
"description": "生图:自定义 Images API、显式模型选择及对话生图工具",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "./lib/index.mjs",
|
"main": "./lib/index.mjs",
|
||||||
|
|||||||
+1
-1
@@ -13,7 +13,7 @@ This skill adapts the copied OpenAI imagegen skill's prompt guidance to the DSH
|
|||||||
2. The user must select the exact model, either in their request or explicitly as the default in Settings → 生图. If neither is set, ask which model to use. Never silently choose another model or fall back after an error.
|
2. The user must select the exact model, either in their request or explicitly as the default in Settings → 生图. If neither is set, ask which model to use. Never silently choose another model or fall back after an error.
|
||||||
3. Call `imagegen_generate` with the exact `model`, a clear `prompt`, and only the optional parameters that the user needs. This plugin uses the configured OpenAI-compatible Images API. Do not run the vendored CLI to bypass its configuration.
|
3. Call `imagegen_generate` with the exact `model`, a clear `prompt`, and only the optional parameters that the user needs. This plugin uses the configured OpenAI-compatible Images API. Do not run the vendored CLI to bypass its configuration.
|
||||||
4. Generated images are returned as durable image attachments and saved as original files. Show the images and mention the model and saved paths. Use `present` when a separate downloadable deliverable is useful.
|
4. Generated images are returned as durable image attachments and saved as original files. Show the images and mention the model and saved paths. Use `present` when a separate downloadable deliverable is useful.
|
||||||
5. This tool generates new images. It does not currently accept edit targets or masks. Do not claim to have edited a supplied image with text-only generation.
|
5. For editing an existing image, use `imagegen_edit` with `image_path`, `model`, and `prompt`. `image_path` must be an existing absolute host path to the user-selected image (use the provided host path for a chat attachment, or a previous result's saved path; never guess it). Only PNG/JPEG/WebP up to 20 MiB are accepted. This sends the actual image bytes to `/images/edits`; never substitute text-only generation for editing. State the requested changes and explicitly list what must remain unchanged. Results are saved separately; the input is never overwritten. No masks or multiple input images are supported yet. If no usable source path is available, ask the user to attach the image.
|
||||||
|
|
||||||
## Prompt guidance reused from imagegen
|
## Prompt guidance reused from imagegen
|
||||||
|
|
||||||
|
|||||||
+128
-1
@@ -2,7 +2,7 @@ import test from 'node:test';
|
|||||||
import assert from 'node:assert/strict';
|
import assert from 'node:assert/strict';
|
||||||
import http from 'node:http';
|
import http from 'node:http';
|
||||||
import { once } from 'node:events';
|
import { once } from 'node:events';
|
||||||
import { validateBaseUrl, validateGeneration, listModels, generateImages } from '../lib/engine.mjs';
|
import { validateBaseUrl, validateGeneration, listModels, generateImages, validateEditImage, editImages } from '../lib/engine.mjs';
|
||||||
|
|
||||||
const apiKey = 'secret-do-not-echo';
|
const apiKey = 'secret-do-not-echo';
|
||||||
const input = { model: 'explicit-third-party-model', prompt: 'a blue bird' };
|
const input = { model: 'explicit-third-party-model', prompt: 'a blue bird' };
|
||||||
@@ -31,6 +31,70 @@ async function requestJson(req) {
|
|||||||
return JSON.parse(Buffer.concat(chunks).toString());
|
return JSON.parse(Buffer.concat(chunks).toString());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
test('edit image validation enforces bytes, signatures, upload limit and owned copies', () => {
|
||||||
|
const padded = Buffer.concat([Buffer.from('prefix'), png, Buffer.from('suffix')]);
|
||||||
|
const view = new Uint8Array(padded.buffer, padded.byteOffset + 6, png.length);
|
||||||
|
const validated = validateEditImage(view);
|
||||||
|
assert.equal(validated.mediaType, 'image/png');
|
||||||
|
assert.ok(Buffer.isBuffer(validated.data));
|
||||||
|
assert.deepEqual(validated.data, png);
|
||||||
|
view.fill(0);
|
||||||
|
assert.deepEqual(validated.data, png);
|
||||||
|
for (const data of [undefined, null, 'image.png', [], new ArrayBuffer(8), new Uint16Array(8), Buffer.alloc(0), Buffer.from('GIF89a'), Buffer.from('<svg/>')]) {
|
||||||
|
assert.throws(() => validateEditImage(data), /PNG, JPEG, or WebP/);
|
||||||
|
}
|
||||||
|
const boundary = Buffer.alloc(20 * 1024 * 1024 + 1);
|
||||||
|
png.copy(boundary);
|
||||||
|
assert.throws(() => validateEditImage(boundary), /20 MiB/);
|
||||||
|
assert.equal(validateEditImage(boundary.subarray(0, -1)).data.length, 20 * 1024 * 1024);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('edits upload real multipart with singular image, correct filename/type and validated fields', async t => {
|
||||||
|
const fixtures = [
|
||||||
|
[png, 'image/png', 'png'],
|
||||||
|
[Buffer.from([255, 216, 255, 224, 0, 0]), 'image/jpeg', 'jpg'],
|
||||||
|
[Buffer.from('RIFF\x04\0\0\0WEBP', 'latin1'), 'image/webp', 'webp'],
|
||||||
|
];
|
||||||
|
let index = 0;
|
||||||
|
const specified = { ...input, prompt: '把背景换成海边', n: 2, size: 'auto', quality: 'high', background: 'opaque', output_format: 'webp' };
|
||||||
|
const base = await server(t, async (req, res) => {
|
||||||
|
const [bytes, type, extension] = fixtures[index++];
|
||||||
|
assert.equal(req.method, 'POST');
|
||||||
|
assert.equal(req.url, '/custom/v1/images/edits');
|
||||||
|
assert.equal(req.headers.authorization, `Bearer ${apiKey}`);
|
||||||
|
assert.match(req.headers['content-type'], /^multipart\/form-data; boundary=/);
|
||||||
|
const request = new Request('http://localhost', { method: 'POST', headers: req.headers, body: req, duplex: 'half' });
|
||||||
|
const form = await request.formData();
|
||||||
|
const expected = index === 1 ? specified : { ...input, n: 1 };
|
||||||
|
assert.deepEqual([...form.keys()].sort(), [...Object.keys(expected), 'image'].sort());
|
||||||
|
for (const [key, value] of Object.entries(expected)) assert.equal(form.get(key), String(value));
|
||||||
|
assert.equal(form.getAll('image').length, 1);
|
||||||
|
const file = form.get('image');
|
||||||
|
assert.equal(file.name, `image.${extension}`);
|
||||||
|
assert.equal(file.type, type);
|
||||||
|
assert.deepEqual(Buffer.from(await file.arrayBuffer()), bytes);
|
||||||
|
json(res, { data: [{ b64_json: png.toString('base64'), revised_prompt: 'edited bird' }] });
|
||||||
|
});
|
||||||
|
for (const [i, [image]] of fixtures.entries()) {
|
||||||
|
const options = i === 0 ? specified : input;
|
||||||
|
const result = await editImages({ baseUrl: `${base}/custom/v1/`, apiKey, input: { ...options, ignored: true }, image });
|
||||||
|
assert.deepEqual(result, { model: options.model, prompt: options.prompt, images: [{ data: png, mediaType: 'image/png', revisedPrompt: 'edited bird' }] });
|
||||||
|
}
|
||||||
|
assert.equal(index, 3);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('edits reject invalid images and options before any API request', async t => {
|
||||||
|
let count = 0;
|
||||||
|
const baseUrl = await server(t, (req, res) => { count++; json(res, { data: [] }); });
|
||||||
|
for (const image of [Buffer.alloc(0), Buffer.from('GIF89a'), Buffer.alloc(20 * 1024 * 1024 + 1)]) {
|
||||||
|
await assert.rejects(editImages({ baseUrl, apiKey, input, image }));
|
||||||
|
}
|
||||||
|
for (const change of [{ model: '' }, { prompt: '' }, { n: 5 }, { quality: 'invalid' }]) {
|
||||||
|
await assert.rejects(editImages({ baseUrl, apiKey, input: { ...input, ...change }, image: png }));
|
||||||
|
}
|
||||||
|
assert.equal(count, 0);
|
||||||
|
});
|
||||||
|
|
||||||
test('base URL normalization preserves API prefix and rejects unsafe parts', () => {
|
test('base URL normalization preserves API prefix and rejects unsafe parts', () => {
|
||||||
assert.equal(validateBaseUrl(' HTTPS://EXAMPLE.COM/v1/// '), 'https://example.com/v1');
|
assert.equal(validateBaseUrl(' HTTPS://EXAMPLE.COM/v1/// '), 'https://example.com/v1');
|
||||||
for (const value of ['', null, '/v1', 'ftp://example.com', 'https://a:b@example.com', 'https://example.com?x=1', 'https://example.com#x', 'https://example.com?', 'https://example.com#']) {
|
for (const value of ['', null, '/v1', 'ftp://example.com', 'https://a:b@example.com', 'https://example.com?x=1', 'https://example.com#x', 'https://example.com?', 'https://example.com#']) {
|
||||||
@@ -207,6 +271,69 @@ test('180 second timeout aborts a stalled request', async t => {
|
|||||||
assert.equal(configuredTimeout, 180_000);
|
assert.equal(configuredTimeout, 180_000);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('edit response downloads omit auth and retain output validation and bounds', async t => {
|
||||||
|
const imageBase = await server(t, (req, res) => {
|
||||||
|
assert.equal(req.headers.authorization, undefined);
|
||||||
|
res.end(png);
|
||||||
|
});
|
||||||
|
let payload = { data: [{ url: `${imageBase}/image` }] };
|
||||||
|
let oversized = false;
|
||||||
|
const baseUrl = await server(t, (req, res) => {
|
||||||
|
if (oversized) { res.writeHead(200, { 'Content-Length': 64 * 1024 * 1024 + 1 }); res.flushHeaders(); }
|
||||||
|
else json(res, payload);
|
||||||
|
});
|
||||||
|
const edit = () => editImages({ baseUrl, apiKey, input, image: png });
|
||||||
|
assert.deepEqual((await edit()).images, [{ data: png, mediaType: 'image/png' }]);
|
||||||
|
payload = { data: [{ b64_json: '!!!!' }] };
|
||||||
|
await assert.rejects(edit(), /invalid base64/);
|
||||||
|
payload = { data: [] };
|
||||||
|
await assert.rejects(edit(), /malformed image response/);
|
||||||
|
oversized = true;
|
||||||
|
await assert.rejects(edit(), /64 MB response limit/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('edit API failure is sanitized and never retried', async t => {
|
||||||
|
let count = 0;
|
||||||
|
const baseUrl = await server(t, (req, res) => { count++; res.writeHead(503); res.end(`provider-private ${apiKey}`); });
|
||||||
|
await assert.rejects(editImages({ baseUrl, apiKey, input, image: png }), error => {
|
||||||
|
assert.equal(error.message, 'API request failed with HTTP 503.');
|
||||||
|
assert.equal(error.cause, undefined);
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
assert.equal(count, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('edits cancel pending API requests, downloads and pre-aborted requests', async t => {
|
||||||
|
let count = 0;
|
||||||
|
let controller = new AbortController();
|
||||||
|
let download = false;
|
||||||
|
const imageBase = await server(t, (req, res) => {
|
||||||
|
assert.equal(req.headers.authorization, undefined);
|
||||||
|
res.write(png.subarray(0, 4));
|
||||||
|
controller.abort(new Error(apiKey));
|
||||||
|
});
|
||||||
|
const baseUrl = await server(t, (req, res) => {
|
||||||
|
count++;
|
||||||
|
if (download) json(res, { data: [{ url: `${imageBase}/slow` }] });
|
||||||
|
else controller.abort(new Error(apiKey));
|
||||||
|
});
|
||||||
|
const edit = signal => editImages({ baseUrl, apiKey, input, image: png, signal });
|
||||||
|
await assert.rejects(edit(AbortSignal.abort(apiKey)), /^Error: Request aborted\.$/);
|
||||||
|
assert.equal(count, 0);
|
||||||
|
await assert.rejects(edit(controller.signal), /^Error: Request aborted\.$/);
|
||||||
|
controller = new AbortController();
|
||||||
|
download = true;
|
||||||
|
await assert.rejects(edit(controller.signal), /^Error: Request aborted\.$/);
|
||||||
|
assert.equal(count, 2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('edits share the 180 second operation timeout', async t => {
|
||||||
|
const baseUrl = await server(t, () => {});
|
||||||
|
const original = globalThis.setTimeout;
|
||||||
|
t.mock.method(globalThis, 'setTimeout', (callback, milliseconds, ...args) => original(callback, milliseconds === 180_000 ? 20 : milliseconds, ...args));
|
||||||
|
await assert.rejects(editImages({ baseUrl, apiKey, input, image: png }), /timed out after 180 seconds/);
|
||||||
|
});
|
||||||
|
|
||||||
test('pre-aborted signal does not send an API request', async t => {
|
test('pre-aborted signal does not send an API request', async t => {
|
||||||
let count = 0;
|
let count = 0;
|
||||||
const baseUrl = await server(t, (req, res) => { count++; json(res, { data: [] }); });
|
const baseUrl = await server(t, (req, res) => { count++; json(res, { data: [] }); });
|
||||||
|
|||||||
+60
-9
@@ -1,7 +1,7 @@
|
|||||||
import test from 'node:test';
|
import test from 'node:test';
|
||||||
import assert from 'node:assert/strict';
|
import assert from 'node:assert/strict';
|
||||||
import { createServer } from 'node:http';
|
import { createServer } from 'node:http';
|
||||||
import { mkdtemp, readFile, rm } from 'node:fs/promises';
|
import { mkdtemp, readFile, writeFile, rm } from 'node:fs/promises';
|
||||||
import { tmpdir } from 'node:os';
|
import { tmpdir } from 'node:os';
|
||||||
import { join, dirname, basename } from 'node:path';
|
import { join, dirname, basename } from 'node:path';
|
||||||
|
|
||||||
@@ -27,12 +27,16 @@ test('native host integration using only a local image API', async t => {
|
|||||||
const requests = [];
|
const requests = [];
|
||||||
const server = createServer(async (request, response) => {
|
const server = createServer(async (request, response) => {
|
||||||
try {
|
try {
|
||||||
let raw = '';
|
const chunks = [];
|
||||||
for await (const chunk of request) raw += chunk;
|
for await (const chunk of request) chunks.push(chunk);
|
||||||
requests.push({ path: request.url, authorization: request.headers.authorization, body: raw ? JSON.parse(raw) : undefined });
|
const raw = Buffer.concat(chunks);
|
||||||
|
const body = request.url === '/v1/images/edits'
|
||||||
|
? await new Response(raw, { headers: { 'Content-Type': request.headers['content-type'] } }).formData()
|
||||||
|
: raw.length ? JSON.parse(raw.toString()) : undefined;
|
||||||
|
requests.push({ path: request.url, authorization: request.headers.authorization, body });
|
||||||
response.setHeader('Content-Type', 'application/json');
|
response.setHeader('Content-Type', 'application/json');
|
||||||
if (request.url === '/v1/models') response.end(JSON.stringify({ data: [{ id: 'test-image-model' }] }));
|
if (request.url === '/v1/models') response.end(JSON.stringify({ data: [{ id: 'test-image-model' }] }));
|
||||||
else if (request.url === '/v1/images/generations') response.end(JSON.stringify({ data: [{ b64_json: PNG.toString('base64'), revised_prompt: 'Local test image' }] }));
|
else if (['/v1/images/generations', '/v1/images/edits'].includes(request.url)) response.end(JSON.stringify({ data: [{ b64_json: PNG.toString('base64'), revised_prompt: 'Local test image' }] }));
|
||||||
else { response.statusCode = 404; response.end('{}'); }
|
else { response.statusCode = 404; response.end('{}'); }
|
||||||
} catch { response.statusCode = 500; response.end('{}'); }
|
} catch { response.statusCode = 500; response.end('{}'); }
|
||||||
});
|
});
|
||||||
@@ -46,7 +50,7 @@ test('native host integration using only a local image API', async t => {
|
|||||||
const ctx = {
|
const ctx = {
|
||||||
on(event, listener) { listeners.set(event, listener); },
|
on(event, listener) { listeners.set(event, listener); },
|
||||||
connection: { fetch: { register(route) {
|
connection: { fetch: { register(route) {
|
||||||
assert.equal(route.requestBody, 'buffered');
|
assert.equal(route.requestBody, route.path === '/api/imagegen/edit' ? 'streaming' : 'buffered');
|
||||||
assert.ok(!routes.has(route.path));
|
assert.ok(!routes.has(route.path));
|
||||||
routes.set(route.path, route);
|
routes.set(route.path, route);
|
||||||
return async () => routes.delete(route.path);
|
return async () => routes.delete(route.path);
|
||||||
@@ -57,7 +61,7 @@ test('native host integration using only a local image API', async t => {
|
|||||||
async modifyRecord(key, change) { validateCredentialName(key); const value = await change(records.get(key)); records.set(key, value); return value; },
|
async modifyRecord(key, change) { validateCredentialName(key); const value = await change(records.get(key)); records.set(key, value); return value; },
|
||||||
async deleteRecord(key) { validateCredentialName(key); records.delete(key); },
|
async deleteRecord(key) { validateCredentialName(key); records.delete(key); },
|
||||||
},
|
},
|
||||||
attachments: { async saveImage(image) {
|
attachments: { async validateImage(image) { assert.deepEqual(Buffer.from(image.data), PNG); }, async saveImage(image) {
|
||||||
assert.deepEqual(Buffer.from(image.data), PNG);
|
assert.deepEqual(Buffer.from(image.data), PNG);
|
||||||
assert.equal(image.mediaType, 'image/png');
|
assert.equal(image.mediaType, 'image/png');
|
||||||
const ref = { attachmentId: `test-attachment-${attachments.length}`, mediaType: image.mediaType, bytes: image.data.length, width: 1, height: 1, name: image.name };
|
const ref = { attachmentId: `test-attachment-${attachments.length}`, mediaType: image.mediaType, bytes: image.data.length, width: 1, height: 1, name: image.name };
|
||||||
@@ -80,8 +84,8 @@ test('native host integration using only a local image API', async t => {
|
|||||||
|
|
||||||
await t.test('registers dependencies, routes, tools and skill', () => {
|
await t.test('registers dependencies, routes, tools and skill', () => {
|
||||||
assert.deepEqual(host.inject, ['connection', 'tools', 'credentials', 'attachments', 'skills']);
|
assert.deepEqual(host.inject, ['connection', 'tools', 'credentials', 'attachments', 'skills']);
|
||||||
assert.deepEqual([...routes.keys()].sort(), ['/api/imagegen/file', '/api/imagegen/generate', '/api/imagegen/models', '/api/imagegen/settings']);
|
assert.deepEqual([...routes.keys()].sort(), ['/api/imagegen/edit', '/api/imagegen/file', '/api/imagegen/generate', '/api/imagegen/models', '/api/imagegen/settings']);
|
||||||
assert.deepEqual([...tools.keys()].sort(), ['imagegen_generate', 'imagegen_status']);
|
assert.deepEqual([...tools.keys()].sort(), ['imagegen_edit', 'imagegen_generate', 'imagegen_status']);
|
||||||
assert.equal(skills.length, 1);
|
assert.equal(skills.length, 1);
|
||||||
assert.equal(skills[0].name, 'dsh-imagegen');
|
assert.equal(skills[0].name, 'dsh-imagegen');
|
||||||
assert.ok(skills[0].content.length > 0);
|
assert.ok(skills[0].content.length > 0);
|
||||||
@@ -175,6 +179,53 @@ test('native host integration using only a local image API', async t => {
|
|||||||
assert.deepEqual(await readFile(value.images[0].path), PNG);
|
assert.deepEqual(await readFile(value.images[0].path), PNG);
|
||||||
assert.ok(!JSON.stringify(value).includes(key));
|
assert.ok(!JSON.stringify(value).includes(key));
|
||||||
});
|
});
|
||||||
|
await t.test('edit tool uploads original bytes and saves separate output', async () => {
|
||||||
|
const inputPath = join(home, 'input.png');
|
||||||
|
await writeFile(inputPath, PNG);
|
||||||
|
const value = await tools.get('imagegen_edit').execute({ image_path: inputPath, model: 'test-image-model', prompt: 'Change background only' }, { signal: new AbortController().signal });
|
||||||
|
const request = requests.at(-1);
|
||||||
|
assert.equal(request.path, '/v1/images/edits');
|
||||||
|
assert.equal(request.authorization, `Bearer ${key}`);
|
||||||
|
assert.equal(request.body.get('model'), 'test-image-model');
|
||||||
|
assert.deepEqual(Buffer.from(await request.body.get('image').arrayBuffer()), PNG);
|
||||||
|
assert.deepEqual(await readFile(inputPath), PNG);
|
||||||
|
assert.notEqual(value.images[0].path, inputPath);
|
||||||
|
assert.deepEqual(await readFile(value.images[0].path), PNG);
|
||||||
|
assert.equal(tools.get('imagegen_edit').output.render({}, value)[1].type, 'image');
|
||||||
|
assert.ok(!JSON.stringify(value).includes(key));
|
||||||
|
const before = requests.length;
|
||||||
|
for (const image_path of ['relative.png', join(home, 'missing.png')]) {
|
||||||
|
await assert.rejects(tools.get('imagegen_edit').execute({ image_path, model: 'test-image-model', prompt: 'edit' }, { signal: new AbortController().signal }));
|
||||||
|
}
|
||||||
|
await assert.rejects(tools.get('imagegen_edit').execute({ image_path: inputPath, prompt: 'edit' }, { signal: new AbortController().signal }), /model/);
|
||||||
|
assert.equal(requests.length, before);
|
||||||
|
});
|
||||||
|
await t.test('multipart edit route accepts one file and rejects invalid uploads before API calls', async () => {
|
||||||
|
const send = form => routes.get('/api/imagegen/edit').fetch(new Request('http://localhost/api/imagegen/edit', { method: 'POST', body: form }));
|
||||||
|
const make = () => { const form = new FormData(); form.set('input', JSON.stringify({ model: 'test-image-model', prompt: 'Edit via browser' })); form.set('image', new Blob([PNG], { type: 'image/png' }), 'input.png'); return form; };
|
||||||
|
const result = await send(make());
|
||||||
|
assert.equal(result.status, 200);
|
||||||
|
assert.deepEqual(await readFile((await result.json()).images[0].path), PNG);
|
||||||
|
const before = requests.length;
|
||||||
|
const missing = make(); missing.delete('image');
|
||||||
|
const duplicate = make(); duplicate.append('image', new Blob([PNG]), 'second.png');
|
||||||
|
const invalid = make(); invalid.set('image', new Blob(['not an image']), 'fake.png');
|
||||||
|
const unsupported = make(); unsupported.set('mask', new Blob([PNG]), 'mask.png');
|
||||||
|
const invalidJson = make(); invalidJson.set('input', '{');
|
||||||
|
const noModel = make(); noModel.set('input', JSON.stringify({ prompt: 'edit' }));
|
||||||
|
for (const form of [missing, duplicate, invalid, unsupported, invalidJson, noModel]) assert.equal((await send(form)).status, 400);
|
||||||
|
assert.equal((await call('/api/imagegen/edit', 'POST', { image_path: join(home, 'input.png') })).status, 400);
|
||||||
|
const tooLarge = new Request('http://localhost/api/imagegen/edit', { method: 'POST', headers: { 'Content-Length': String(22 * 1024 * 1024) }, body: 'x' });
|
||||||
|
assert.equal((await routes.get('/api/imagegen/edit').fetch(tooLarge)).status, 400);
|
||||||
|
// Model bytes arriving from the browser transport; cancelling Node's own
|
||||||
|
// FormData encoder mid-Blob triggers an unrelated undici closed-stream bug.
|
||||||
|
const oversized = new Request('http://localhost/api/imagegen/edit', { method: 'POST', headers: { 'Content-Type': 'multipart/form-data; boundary=test' }, body: Buffer.alloc(22 * 1024 * 1024) });
|
||||||
|
assert.equal((await routes.get('/api/imagegen/edit').fetch(oversized)).status, 400, 'streamed body limit applies even without Content-Length');
|
||||||
|
const decode = ctx.attachments.validateImage;
|
||||||
|
ctx.attachments.validateImage = async () => { throw new Error('Image decode refused'); };
|
||||||
|
try { assert.equal((await send(make())).status, 400); } finally { ctx.attachments.validateImage = decode; }
|
||||||
|
assert.equal(requests.length, before);
|
||||||
|
});
|
||||||
await t.test('all digest prefixes are valid and clearing only removes the selected endpoint', async () => {
|
await t.test('all digest prefixes are valid and clearing only removes the selected endpoint', async () => {
|
||||||
const originalRecords = new Map(records);
|
const originalRecords = new Map(records);
|
||||||
const prefixes = new Set();
|
const prefixes = new Set();
|
||||||
|
|||||||
Reference in new issue
Block a user