feat: 支持对话与设置页单图编辑

This commit is contained in:
cneicy committed 2026-10-05 04:21:45 +08:00
1 parent 7e2ec28b80
commit 99e087112b
8 files changed
+361 -30

No files matched your search

+25 -1
View File
@@ -7,6 +7,7 @@
- **接口配置**:设置 Base URL 和 API Key,支持第三方兼容服务。
- **显式选择模型**:手动维护模型 ID,或从接口读取模型列表;可设置对话默认模型,也可以每次在对话中指定。
- **对话生图**:助手通过 `imagegen_status` 查询配置,通过 `imagegen_generate` 生成图片。
- **图片编辑**:对话工具 `imagegen_edit` 与「编辑图片」页面支持上传单张 PNG/JPEG/WebP,调用 `/images/edits`,保留输入原图并另存结果。
- **直接生成**:在「设置 → 生图 → 生成图片」中输入提示词、选择模型并生成。
- **图片输出**:支持 base64 和图片 URL 响应,提供图片预览与原图下载。
- **技能复用**:附带 imagegen 提示词规范和示例,并注册 DSH 专用的 `dsh-imagegen` 技能。
@@ -104,12 +105,35 @@ git+https://git.crash.work/cneicy/dsh-imagegen.git
4. 按需展开尺寸、质量与数量设置。
5. 点击「生成图片」,完成后预览并下载原图。
### 编辑已有图片(1.1.0)
先确认服务商和所选模型支持 `POST /v1/images/edits`(`edits` 为复数;Base URL 已有 `/v1` 时只追加 `/images/edits`)。支持生图不代表支持编辑。
- **对话中**:上传图片,或指定已有图片的本机路径,例如「把这张图的背景改成海边,保持人物、衣服和姿势不变,用我设置的默认模型」。助手使用附件的本机路径调用 `imagegen_edit`。
- **设置页**:进入「设置 → 生图 → 编辑图片」,选择图片并预览,选择模型,输入修改要求,然后点击「编辑图片」。
- 输入仅支持一张 PNG/JPEG/WebP,最大 20 MiB;DSH 附件服务还会检查图片能否解码及部署的像素限制。暂不支持 GIF、蒙版或多图输入。
- 原图字节会发送到所配置的服务商;请求可能收费,失败不自动重试。输出另存为新文件,输入不会被覆盖。
- 对话附件可能已经过 DSH 缩放;要使用原始分辨率,请选择本机原文件或在设置页上传原文件。
工具参数示例(路径替换为实际存在的绝对路径):
```json
{
"image_path": "D:/Pictures/input.png",
"model": "gpt-image-2.5-sunburst",
"prompt": "只将背景改成夕阳下的海边,保持人物、衣服和姿势不变"
}
```
编辑工具复用生图的 `n`、`size`、`quality`、`background`、`output_format` 可选参数;是否支持由服务商决定。设置页通过 `POST /api/imagegen/edit` 提交 multipart 表单:`image` 为文件,`input` 为上述生成参数的 JSON(不包含 `image_path`)。浏览器路由只接收上传文件,不读取请求提供的本机路径。
### 对话工具
| 工具 | 用途 |
| --- | --- |
| `imagegen_status` | 返回 Base URL、是否已配置、候选模型、默认模型与输出目录;不返回密钥。 |
| `imagegen_generate` | 以明确的模型和提示词生成图片,并返回 DSH 图片附件及原图路径。 |
| `imagegen_edit` | 上传 `image_path` 指定的图片,按提示词编辑并返回新图片附件和路径。 |
生成参数示例:
@@ -164,7 +188,7 @@ git+https://git.crash.work/cneicy/dsh-imagegen.git
**模型不支持尺寸、质量或透明背景**:去掉不受支持的可选参数,或显式选择支持这些能力的模型。第三方服务也可能忽略某些参数。
**能否用参考图或修改已有图片?** 当前只实现文生图,不支持参考图输入、图像编辑或蒙版。附带的上游 imagegen 技能资源包含编辑说明,但不代表本插件已实现编辑接口。
**能否用参考图或修改已有图片?** 1.1.0 起支持单张图片加文字指令编辑,调用 `/images/edits`。暂不支持蒙版和多图输入。若编辑返回 404,请检查服务商是否提供该端点;插件不会自动降级成文生图。
## 开发与验证
+57 -10
View File
@@ -4,7 +4,8 @@ window.__ModuleLoader__.load({ id: 'dsh-imagegen-local', factory(require) {
const { useState, useEffect } = React;
const style = `.ig-root{color:var(--dsw-alias-label-primary);font-size:14px;max-width:920px;margin:0 auto;padding:8px 4px 28px}.ig-root *{box-sizing:border-box}.ig-root h2{font-size:21px;margin:0 0 8px}.ig-root h3{font-size:16px;margin:0 0 14px}.ig-muted{color:var(--dsw-alias-label-secondary);line-height:1.6}.ig-card{border:1px solid var(--dsw-alias-border-l1);border-radius:16px;padding:20px;margin-top:20px;background:var(--dsw-alias-bg-layer-1)}.ig-field{display:flex;flex-direction:column;gap:7px;margin:14px 0;min-width:0}.ig-field span{font-weight:500}.ig-root input,.ig-root textarea,.ig-root select{width:100%;background:var(--dsw-alias-bg-base);color:var(--dsw-alias-label-primary);border:1px solid var(--dsw-alias-border-l2);border-radius:9px;padding:10px 12px;font:inherit}.ig-root input:focus,.ig-root textarea:focus,.ig-root select:focus{outline:2px solid var(--dsw-alias-brand-primary);outline-offset:1px}.ig-root textarea{resize:vertical}.ig-row{display:flex;gap:12px;flex-wrap:wrap;align-items:center}.ig-row>.ig-field{flex:1 1 160px}.ig-root button,.ig-link{border:1px solid var(--dsw-alias-border-l2);border-radius:9px;padding:9px 15px;background:var(--dsw-alias-bg-layer-2);color:var(--dsw-alias-label-primary);cursor:pointer;font:inherit}.ig-root button:disabled{opacity:.5;cursor:default}.ig-root .ig-primary{background:var(--dsw-alias-brand-primary);color:#fff;border-color:transparent}.ig-message{margin-top:14px;white-space:pre-wrap;overflow-wrap:anywhere}.ig-error{color:var(--dsw-alias-state-error-primary)}.ig-success{color:var(--dsw-alias-state-success-primary)}.ig-gallery{display:grid;grid-template-columns:repeat(auto-fit,minmax(220px,1fr));gap:16px;margin-top:20px}.ig-gallery img{display:block;width:100%;height:auto;max-height:500px;object-fit:contain;border-radius:10px;background:var(--dsw-alias-bg-base)}.ig-gallery a{color:var(--dsw-alias-brand-primary);display:inline-block;margin-top:10px}.ig-gallery small{display:block;overflow-wrap:anywhere;margin-top:8px}.ig-check{display:flex;align-items:center;gap:8px}.ig-check input{width:auto}.ig-status{border-radius:20px;padding:4px 10px;background:var(--dsw-alias-bg-layer-2);font-size:12px}.ig-tabs{display:flex;gap:8px;margin:20px 0}.ig-root .ig-tab-active{border-color:var(--dsw-alias-brand-primary);color:var(--dsw-alias-brand-primary)}.ig-root details{margin-top:12px}.ig-root summary{cursor:pointer;color:var(--dsw-alias-label-secondary)}`;
async function api(action, data, signal) {
const response = await fetch(`/api/imagegen/${action}`, { method: data === undefined ? 'GET' : 'POST', credentials: 'same-origin', headers: data === undefined ? {} : { 'Content-Type': 'application/json' }, body: data === undefined ? undefined : JSON.stringify(data), signal });
const multipart = data instanceof FormData;
const response = await fetch(`/api/imagegen/${action}`, { method: data === undefined ? 'GET' : 'POST', credentials: 'same-origin', headers: data === undefined || multipart ? {} : { 'Content-Type': 'application/json' }, body: data === undefined ? undefined : multipart ? data : JSON.stringify(data), signal });
const value = await response.json();
if (!response.ok) throw new Error(value.error || `请求失败(${response.status})`);
return value;
@@ -28,6 +29,26 @@ window.__ModuleLoader__.load({ id: 'dsh-imagegen-local', factory(require) {
const [message, setMessage] = useState('');
const [error, setError] = useState('');
const [results, setResults] = useState(null);
const [image, setImage] = useState(null);
const [preview, setPreview] = useState('');
const [fileInputKey, setFileInputKey] = useState(0);
useEffect(() => {
if (!image) { setPreview(''); return; }
const url = URL.createObjectURL(image);
setPreview(url);
return () => URL.revokeObjectURL(url);
}, [image]);
function selectImage(event) {
const file = event.target.files?.[0];
setError(''); setMessage('');
if (!file) { setImage(null); return; }
if (!['image/png', 'image/jpeg', 'image/webp'].includes(file.type) || !file.size || file.size > 20 * 1024 * 1024) {
setImage(null); event.target.value = '';
setError('请选择一张 PNG、JPEG 或 WebP 图片,文件大小不超过 20 MiB,且不能为空。');
return;
}
setImage(file);
}
const [controller] = useState(() => ({ current: null }));
function load(value) { setConfig(value); setBaseUrl(value.baseUrl); setModels(value.models.join('\n')); setDefault(value.defaultModel); setModel(current => current || value.defaultModel); }
useEffect(() => { const ac = new AbortController(); api('settings', undefined, ac.signal).then(load).catch(e => { if (!ac.signal.aborted) setError(e.message); }); return () => { ac.abort(); controller.current?.abort(); }; }, []);
@@ -36,15 +57,33 @@ window.__ModuleLoader__.load({ id: 'dsh-imagegen-local', factory(require) {
const changed = config && (baseUrl !== config.baseUrl || models !== config.models.join('\n') || defaultModel !== config.defaultModel || apiKey || clearKey);
async function save() { await run('save', async () => { const value = await api('settings', { baseUrl, models: modelList, defaultModel, ...(apiKey ? { apiKey } : {}), clearKey }); load(value); setApiKey(''); setClearKey(false); setMessage('设置已保存,对话生图立即使用这些配置。'); }); }
async function discover() { await run('models', async () => { const value = await api('models', {}); setModels([...new Set([...modelList, ...value.models])].join('\n')); setMessage(`已读取 ${value.models.length} 个模型。列表可能包含聊天模型,请保留支持生图的模型后保存。`); }); }
async function generate() { await run('generate', async () => { const ac = new AbortController(); controller.current = ac; try { const value = await api('generate', { model: model.trim(), prompt, n: Number(count), ...(size.trim() ? { size: size.trim() } : {}), ...(quality ? { quality } : {}) }, ac.signal); setResults(value); setMessage(`已使用 ${value.model} 生成 ${value.images.length} 张图片。`); } finally { controller.current = null; } }); }
async function generate() {
if (busy || controller.current || !model.trim() || !prompt.trim() || (tab === 'edit' && !image)) return;
const action = tab === 'edit' ? 'edit' : 'generate';
await run(action, async () => {
const ac = new AbortController(); controller.current = ac;
try {
const args = { model: model.trim(), prompt, n: Number(count), ...(size.trim() ? { size: size.trim() } : {}), ...(quality ? { quality } : {}) };
let body = args;
if (action === 'edit') {
body = new FormData();
body.set('image', image, image.name);
body.set('input', JSON.stringify(args));
}
const value = await api(action, body, ac.signal);
setResults(value);
setMessage(`已使用 ${value.model} ${action === 'edit' ? '编辑并生成' : '生成'} ${value.images.length} 张图片。`);
} finally { controller.current = null; }
});
}
if (!config) return h('div', { className: 'ig-root' }, h('h2', null, '生图'), h('p', { role: error ? 'alert' : 'status' }, error || '正在读取生图设置…'));
return h('div', { className: 'ig-root' },
h('div', { className: 'ig-row' }, h('h2', null, '生图'), h('span', { className: 'ig-status' }, config.configured ? '接口已配置' : '待配置')),
h('p', { className: 'ig-muted' }, '配置兼容 OpenAI Images API 的服务,然后在对话中指定模型生图,或在这里直接生成。'),
h('div', { className: 'ig-tabs' }, ['settings', 'generate'].map(id => h('button', { key: id, type: 'button', className: tab === id ? 'ig-tab-active' : '', onClick: () => { setTab(id); setMessage(''); setError(''); } }, id === 'settings' ? '接口设置' : '生成图片'))),
h('p', { className: 'ig-muted' }, '配置图片服务,选择模型后生成新图片,或上传图片进行编辑。'),
h('div', { className: 'ig-tabs', role: 'tablist', 'aria-label': '生图功能' }, ['settings', 'generate', 'edit'].map(id => h('button', { key: id, type: 'button', role: 'tab', 'aria-selected': tab === id, disabled: !!busy, className: tab === id ? 'ig-tab-active' : '', onClick: () => { setTab(id); setMessage(''); setError(''); } }, { settings: '接口设置', generate: '生成图片', edit: '编辑图片' }[id]))),
tab === 'settings' ? h('div', { className: 'ig-card' },
h('h3', null, '接口与凭据'),
h(Field, { label: 'Base URL', hint: '填写 API 根地址,如 https://api.openai.com/v1;插件会追加 /images/generations。' }, h('input', { value: baseUrl, onChange: e => setBaseUrl(e.target.value), placeholder: 'https://api.openai.com/v1', spellCheck: false, disabled: !!busy })),
h(Field, { label: 'Base URL', hint: '填写服务商提供的 API 根地址,如 https://api.openai.com/v1。' }, h('input', { value: baseUrl, onChange: e => setBaseUrl(e.target.value), placeholder: 'https://api.openai.com/v1', spellCheck: false, disabled: !!busy })),
h(Field, { label: 'API Key', hint: baseUrl === config.baseUrl && config.configured ? '已保存。留空保持当前密钥;密钥不会回显。' : '为这个 Base URL 输入密钥。不同地址的密钥分别保存。' }, h('input', { type: 'password', value: apiKey, onChange: e => { setApiKey(e.target.value); setClearKey(false); }, autoComplete: 'new-password', placeholder: '输入 API Key', disabled: !!busy })),
h('label', { className: 'ig-check ig-muted' }, h('input', { type: 'checkbox', checked: clearKey, disabled: !!busy, onChange: e => { setClearKey(e.target.checked); if (e.target.checked) setApiKey(''); } }), '清除这个地址已保存的密钥'),
h(Field, { label: '可选模型', hint: '每行一个精确模型 ID。支持手动填写第三方模型;模型是否可用由服务商决定。' }, h('textarea', { value: models, rows: 4, spellCheck: false, disabled: !!busy, onChange: e => setModels(e.target.value) })),
@@ -52,16 +91,24 @@ window.__ModuleLoader__.load({ id: 'dsh-imagegen-local', factory(require) {
h('div', { className: 'ig-row' }, h('button', { className: 'ig-primary', disabled: !!busy, onClick: save }, busy === 'save' ? '保存中…' : '保存设置'), h('button', { disabled: !!busy || !!changed || !config.configured, onClick: discover }, busy === 'models' ? '读取中…' : '从接口读取模型')),
h('p', { className: 'ig-muted' }, '先保存接口设置,再读取模型。部分服务不提供模型列表,可以直接手动填写。')
) : h('div', { className: 'ig-card' },
h('h3', null, '生成图片'),
h(Field, { label: '本次生成模型', hint: '选择已配置模型,也可以填写完整的自定义模型 ID。' }, h('input', { list: 'ig-model-options', value: model, disabled: !!busy, onChange: e => setModel(e.target.value), placeholder: '选择或输入模型' }), h('datalist', { id: 'ig-model-options' }, config.models.map(m => option(m)))),
h(Field, { label: '画面描述' }, h('textarea', { rows: 5, value: prompt, maxLength: 32000, disabled: !!busy, placeholder: '描述主体、风格、构图和需要出现的文字…', onChange: e => setPrompt(e.target.value) })),
h('h3', null, tab === 'edit' ? '编辑图片' : '生成图片'),
tab === 'edit' && h('div', null,
h(Field, { label: '原图', hint: '选择一张 PNG、JPEG 或 WebP 图片,最大 20 MiB。' }, h('input', { key: fileInputKey, type: 'file', accept: 'image/png,image/jpeg,image/webp', disabled: !!busy, onChange: selectImage })),
image && h('div', null,
preview && h('img', { src: preview, alt: '待编辑的原图', style: { display: 'block', maxWidth: '100%', maxHeight: '320px', objectFit: 'contain', borderRadius: '10px' } }),
h('p', { className: 'ig-muted', style: { overflowWrap: 'anywhere' } }, image.name),
h('button', { type: 'button', disabled: !!busy, onClick: () => { setImage(null); setFileInputKey(value => value + 1); } }, '移除图片')
)
),
h(Field, { label: '本次使用模型', hint: tab === 'edit' ? '请选择支持图片编辑的模型,也可以填写完整的自定义模型 ID。' : '选择已配置模型,也可以填写完整的自定义模型 ID。' }, h('input', { list: 'ig-model-options', value: model, disabled: !!busy, onChange: e => setModel(e.target.value), placeholder: '选择或输入模型' }), h('datalist', { id: 'ig-model-options' }, config.models.map(m => option(m)))),
h(Field, { label: tab === 'edit' ? '修改要求' : '画面描述' }, h('textarea', { rows: 5, value: prompt, maxLength: 32000, disabled: !!busy, placeholder: tab === 'edit' ? '描述要修改的内容,以及需要保留的人物、构图或细节…' : '描述主体、风格、构图和需要出现的文字…', onChange: e => setPrompt(e.target.value) })),
h('details', null, h('summary', null, '尺寸、质量与数量'), h('div', { className: 'ig-row' },
h(Field, { label: '尺寸', hint: '留空使用接口默认' }, h('input', { value: size, placeholder: '如 1024x1024', disabled: !!busy, onChange: e => setSize(e.target.value) })),
h(Field, { label: '质量' }, h('select', { value: quality, disabled: !!busy, onChange: e => setQuality(e.target.value) }, option('', '接口默认'), ['auto','low','medium','high','standard','hd'].map(m => option(m)))),
h(Field, { label: '数量' }, h('select', { value: count, disabled: !!busy, onChange: e => setCount(e.target.value) }, ['1','2','3','4'].map(m => option(m))))
)),
h('p', { className: 'ig-muted' }, `本次请求发送至 ${config.baseUrl},按服务商规则计费。`),
h('div', { className: 'ig-row' }, h('button', { className: 'ig-primary', disabled: !!busy || !config.configured || !model.trim() || !prompt.trim(), onClick: generate }, busy === 'generate' ? '正在生成…' : '生成图片'), busy === 'generate' && h('button', { onClick: () => controller.current?.abort() }, '停止等待')),
h('p', { className: 'ig-muted' }, `本次${tab === 'edit' ? '图片与修改要求' : '请求'}发送至 ${config.baseUrl},按服务商规则计费。`),
h('div', { className: 'ig-row' }, h('button', { className: 'ig-primary', disabled: !!busy || !config.configured || !model.trim() || !prompt.trim() || (tab === 'edit' && !image), onClick: generate }, busy === 'edit' ? '正在编辑…' : busy === 'generate' ? '正在生成…' : tab === 'edit' ? '编辑图片' : '生成图片'), ['generate', 'edit'].includes(busy) && h('button', { onClick: () => controller.current?.abort() }, '停止等待')),
!config.configured && h('p', { className: 'ig-muted' }, '请先在接口设置中保存 API Key。')
),
error && h('div', { role: 'alert', className: 'ig-message ig-error' }, error),
+32
View File
@@ -1,4 +1,5 @@
const MAX_BYTES = 64 * 1024 * 1024;
const MAX_EDIT_BYTES = 20 * 1024 * 1024;
const TIMEOUT_MS = 180_000;
const QUALITY = new Set(['auto', 'low', 'medium', 'high', 'standard', 'hd']);
const BACKGROUND = new Set(['auto', 'transparent', 'opaque']);
@@ -52,6 +53,7 @@ async function operation(signal, work) {
const timer = setTimeout(() => { timedOut = true; controller.abort(); }, TIMEOUT_MS);
timer.unref?.();
try {
if (controller.signal.aborted) throw new Error('Request aborted.');
return await work(controller.signal);
} catch (error) {
if (controller.signal.aborted) throw new Error(timedOut ? 'Request timed out after 180 seconds.' : 'Request aborted.');
@@ -119,6 +121,19 @@ function mediaType(bytes) {
throw new Error('Image bytes are not a supported PNG, JPEG, WebP, or GIF image.');
}
/** Validate one PNG/JPEG/WebP upload by signature, not full image decoding.
* Accepts Buffer/Uint8Array up to 20 MiB; returns an owned Buffer copy and MIME type.
*/
export function validateEditImage(data) {
if (!(data instanceof Uint8Array) || data.byteLength === 0) throw new Error('Edit image must be nonempty PNG, JPEG, or WebP bytes.');
if (data.byteLength > MAX_EDIT_BYTES) throw new Error('Edit image exceeds the 20 MiB upload limit.');
const bytes = Buffer.from(data);
let type;
try { type = mediaType(bytes); } catch { /* Use the upload-specific error below. */ }
if (!['image/png', 'image/jpeg', 'image/webp'].includes(type)) throw new Error('Edit image must be a PNG, JPEG, or WebP image.');
return { data: bytes, mediaType: type };
}
function imageUrl(value) {
let url;
try { url = new URL(value); } catch { throw new Error('API returned an invalid image URL.'); }
@@ -144,6 +159,23 @@ export async function generateImages({ baseUrl, apiKey, input, signal }) {
const headers = { ...apiHeaders(apiKey), 'Content-Type': 'application/json' };
const body = JSON.stringify(generation);
if (Buffer.byteLength(body) > MAX_BYTES) throw new Error('Request exceeds the 64 MB request limit.');
return requestImages({ endpoint, headers, body, generation, signal });
}
/** Edit one image; input and return value share the generateImages contract. */
export async function editImages({ baseUrl, apiKey, input, image, signal }) {
const endpoint = `${validateBaseUrl(baseUrl)}/images/edits`;
const generation = validateGeneration(input);
const headers = apiHeaders(apiKey);
const upload = validateEditImage(image);
const extension = { 'image/png': 'png', 'image/jpeg': 'jpg', 'image/webp': 'webp' }[upload.mediaType];
const body = new FormData();
for (const [key, value] of Object.entries(generation)) body.set(key, String(value));
body.set('image', new Blob([upload.data], { type: upload.mediaType }), `image.${extension}`);
return requestImages({ endpoint, headers, body, generation, signal });
}
function requestImages({ endpoint, headers, body, generation, signal }) {
return operation(signal, async combinedSignal => {
const budget = { remaining: MAX_BYTES };
const json = await fetchJson(endpoint, { method: 'POST', headers, body, signal: combinedSignal }, budget);
+57 -7
View File
@@ -1,9 +1,11 @@
import { readFile, writeFile, mkdir, rename } from 'node:fs/promises';
import { createReadStream } from 'node:fs';
import { Readable, addAbortSignal } from 'node:stream';
import { homedir } from 'node:os';
import { join, dirname } from 'node:path';
import { join, dirname, isAbsolute } from 'node:path';
import { fileURLToPath } from 'node:url';
import { createHash, randomUUID } from 'node:crypto';
import { validateBaseUrl, validateGeneration, generateImages, listModels } from './engine.mjs';
import { validateBaseUrl, validateGeneration, generateImages, editImages, validateEditImage, listModels } from './engine.mjs';
export const name = 'imagegen-local';
export const inject = ['connection', 'tools', 'credentials', 'attachments', 'skills'];
@@ -42,12 +44,14 @@ export async function apply(ctx) {
config = next;
return view();
}
async function generate(input, signal) {
async function generate(input, signal, image) {
const args = validateGeneration(input);
const snapshot = { ...config };
const apiKey = await readKey(snapshot.baseUrl);
if (!apiKey) throw new Error('请先在 设置 → 生图 中为当前 Base URL 保存 API Key。');
const result = await generateImages({ baseUrl: snapshot.baseUrl, apiKey, input: args, signal: signalFor(signal) });
const options = { baseUrl: snapshot.baseUrl, apiKey, input: args, signal: signalFor(signal) };
if (image !== undefined) await ctx.attachments.validateImage(validateEditImage(image));
const result = image === undefined ? await generateImages(options) : await editImages({ ...options, image });
await mkdir(outputs, { recursive: true });
const images = [];
for (const item of result.images) {
@@ -61,8 +65,8 @@ export async function apply(ctx) {
}
return { model: args.model, prompt: args.prompt, images };
}
function route(path, methods, handler) {
ctx.connection.fetch.register({ path, methods, requestBody: 'buffered', fetch: async request => {
function route(path, methods, handler, requestBody = 'buffered') {
ctx.connection.fetch.register({ path, methods, requestBody, fetch: async request => {
try { return await handler(request); }
catch (error) {
// Never include arbitrary provider responses or request headers in a browser error.
@@ -92,6 +96,22 @@ export async function apply(ctx) {
return json({ models: await listModels({ baseUrl: snapshot.baseUrl, apiKey, signal: signalFor(request.signal) }) });
});
route('/api/imagegen/generate', ['POST'], async request => json(await generate(await body(request), request.signal)));
route('/api/imagegen/edit', ['POST'], async request => {
const limit = 21 * 1024 * 1024;
if (Number(request.headers.get('content-length')) > limit) throw new Error('编辑上传内容过大(图片最大 20 MiB)。');
if (!request.headers.get('content-type')?.startsWith('multipart/form-data;')) throw new Error('请使用图片上传表单。');
const bytes = await boundedBytes(request.body, limit, signalFor(request.signal));
let form;
try { form = await new Response(bytes, { headers: { 'Content-Type': request.headers.get('content-type') } }).formData(); }
catch { throw new Error('图片上传表单无效。'); }
if ([...form.keys()].some(key => !['image', 'input'].includes(key)) || form.getAll('image').length !== 1 || form.getAll('input').length !== 1) throw new Error('请只上传一张图片和一组编辑参数。');
const image = form.get('image'), raw = form.get('input');
if (!image || typeof image.arrayBuffer !== 'function' || image.size > 20 * 1024 * 1024) throw new Error('请选择不超过 20 MiB 的图片。');
if (typeof raw !== 'string' || raw.length > 100000) throw new Error('编辑参数无效。');
let input;
try { input = JSON.parse(raw); } catch { throw new Error('编辑参数必须为 JSON。'); }
return json(await generate(input, request.signal, Buffer.from(await image.arrayBuffer())));
}, 'streaming');
route('/api/imagegen/file', ['GET'], async request => {
const filename = new URL(request.url).searchParams.get('name') || '';
if (!/^\d+-[0-9a-f-]{36}\.(png|jpg|webp|gif)$/.test(filename)) return json({ error: '图片不存在。' }, 404);
@@ -108,7 +128,7 @@ export async function apply(ctx) {
execute: () => view(),
isConcurrencySafe: () => true,
});
ctx.tools.register({
const generationTool = {
name: 'imagegen_generate',
description: 'Generate images using the configured Images API. Requires an exact model explicitly chosen by the user in chat or Settings → 生图 (query imagegen_status for their default). Never silently switch models. Returns saved original file paths and inline image attachments. API calls may incur charges. Text-to-image only; no reference-image editing. Reuse dsh-imagegen skill prompt guidance.',
parameters: { type: 'object', properties: {
@@ -124,10 +144,40 @@ export async function apply(ctx) {
output: { schema, render: (_, value) => [text({ model: value.model, prompt: value.prompt, images: value.images.map(({ path, revisedPrompt }) => ({ path, revisedPrompt })) }), ...value.images.map(image => ({ type: 'image', attachment: image.attachment }))] },
execute: (args, exec) => generate(args, exec.signal),
presentCall: args => ({ card: 'generic', title: `生图 · ${args.model}`, kind: 'fetch', rawInput: args }),
};
ctx.tools.register(generationTool);
ctx.tools.register({
...generationTool,
name: 'imagegen_edit',
description: 'Edit one user-provided image using POST /images/edits. Requires an absolute local image_path (uploaded attachment host path or existing image), exact user-selected model and edit prompt. Query imagegen_status first. PNG/JPEG/WebP up to 20 MiB; no masks or multiple inputs. Sends image bytes to the configured provider and may incur charges. Preserve requested unchanged features; never substitute text-only generation or silently change models.',
parameters: { ...generationTool.parameters, properties: { ...generationTool.parameters.properties, image_path: { type: 'string', description: 'Absolute host path of the user-authorized input image; use the attachment host path for a chat upload. Never guess a path.' } }, required: ['model', 'prompt', 'image_path'] },
execute: async (args, exec) => {
validateGeneration(args);
if (typeof args.image_path !== 'string' || !isAbsolute(args.image_path)) throw new Error('image_path 必须是图片的本机绝对路径。');
let image;
try { image = await boundedBytes(createReadStream(args.image_path), 20 * 1024 * 1024, signalFor(exec.signal)); }
catch { throw new Error('无法读取编辑图片,或图片超过 20 MiB,或读取已取消。'); }
return generate(args, exec.signal, image);
},
presentCall: args => ({ card: 'generic', title: `编辑图片 · ${args.model}`, kind: 'fetch', rawInput: args }),
});
ctx.skills.register({ name: 'dsh-imagegen', description: '使用生图插件生成图片,复用 imagegen 提示词规范,支持自定义 API 与显式模型选择。', source: 'runtime', path: join(root, 'skills', 'SKILL.md'), resourceBase: { kind: 'directory', path: join(root, 'skills') }, content: await readFile(join(root, 'skills', 'SKILL.md'), 'utf8') });
}
async function boundedBytes(source, limit, signal) {
if (!source) throw new Error('图片内容为空。');
const stream = typeof source.getReader === 'function' ? Readable.fromWeb(source) : source;
if (signal) addAbortSignal(signal, stream);
const chunks = [];
let length = 0;
for await (const chunk of stream) {
length += chunk.length;
if (length > limit) throw new Error('图片上传超过大小限制。');
chunks.push(chunk);
}
return Buffer.concat(chunks, length);
}
export function validateSettings(input) {
if (!input || typeof input !== 'object' || Array.isArray(input)) throw new Error('生图设置格式无效。');
const baseUrl = validateBaseUrl(input.baseUrl);
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "dsh-imagegen-local",
"version": "1.0.1",
"version": "1.1.0",
"description": "生图:自定义 Images API、显式模型选择及对话生图工具",
"type": "module",
"main": "./lib/index.mjs",
+1 -1
View File
@@ -13,7 +13,7 @@ This skill adapts the copied OpenAI imagegen skill's prompt guidance to the DSH
2. The user must select the exact model, either in their request or explicitly as the default in Settings → 生图. If neither is set, ask which model to use. Never silently choose another model or fall back after an error.
3. Call `imagegen_generate` with the exact `model`, a clear `prompt`, and only the optional parameters that the user needs. This plugin uses the configured OpenAI-compatible Images API. Do not run the vendored CLI to bypass its configuration.
4. Generated images are returned as durable image attachments and saved as original files. Show the images and mention the model and saved paths. Use `present` when a separate downloadable deliverable is useful.
5. This tool generates new images. It does not currently accept edit targets or masks. Do not claim to have edited a supplied image with text-only generation.
5. For editing an existing image, use `imagegen_edit` with `image_path`, `model`, and `prompt`. `image_path` must be an existing absolute host path to the user-selected image (use the provided host path for a chat attachment, or a previous result's saved path; never guess it). Only PNG/JPEG/WebP up to 20 MiB are accepted. This sends the actual image bytes to `/images/edits`; never substitute text-only generation for editing. State the requested changes and explicitly list what must remain unchanged. Results are saved separately; the input is never overwritten. No masks or multiple input images are supported yet. If no usable source path is available, ask the user to attach the image.
## Prompt guidance reused from imagegen
+128 -1
View File
@@ -2,7 +2,7 @@ import test from 'node:test';
import assert from 'node:assert/strict';
import http from 'node:http';
import { once } from 'node:events';
import { validateBaseUrl, validateGeneration, listModels, generateImages } from '../lib/engine.mjs';
import { validateBaseUrl, validateGeneration, listModels, generateImages, validateEditImage, editImages } from '../lib/engine.mjs';
const apiKey = 'secret-do-not-echo';
const input = { model: 'explicit-third-party-model', prompt: 'a blue bird' };
@@ -31,6 +31,70 @@ async function requestJson(req) {
return JSON.parse(Buffer.concat(chunks).toString());
}
test('edit image validation enforces bytes, signatures, upload limit and owned copies', () => {
const padded = Buffer.concat([Buffer.from('prefix'), png, Buffer.from('suffix')]);
const view = new Uint8Array(padded.buffer, padded.byteOffset + 6, png.length);
const validated = validateEditImage(view);
assert.equal(validated.mediaType, 'image/png');
assert.ok(Buffer.isBuffer(validated.data));
assert.deepEqual(validated.data, png);
view.fill(0);
assert.deepEqual(validated.data, png);
for (const data of [undefined, null, 'image.png', [], new ArrayBuffer(8), new Uint16Array(8), Buffer.alloc(0), Buffer.from('GIF89a'), Buffer.from('<svg/>')]) {
assert.throws(() => validateEditImage(data), /PNG, JPEG, or WebP/);
}
const boundary = Buffer.alloc(20 * 1024 * 1024 + 1);
png.copy(boundary);
assert.throws(() => validateEditImage(boundary), /20 MiB/);
assert.equal(validateEditImage(boundary.subarray(0, -1)).data.length, 20 * 1024 * 1024);
});
test('edits upload real multipart with singular image, correct filename/type and validated fields', async t => {
const fixtures = [
[png, 'image/png', 'png'],
[Buffer.from([255, 216, 255, 224, 0, 0]), 'image/jpeg', 'jpg'],
[Buffer.from('RIFF\x04\0\0\0WEBP', 'latin1'), 'image/webp', 'webp'],
];
let index = 0;
const specified = { ...input, prompt: '把背景换成海边', n: 2, size: 'auto', quality: 'high', background: 'opaque', output_format: 'webp' };
const base = await server(t, async (req, res) => {
const [bytes, type, extension] = fixtures[index++];
assert.equal(req.method, 'POST');
assert.equal(req.url, '/custom/v1/images/edits');
assert.equal(req.headers.authorization, `Bearer ${apiKey}`);
assert.match(req.headers['content-type'], /^multipart\/form-data; boundary=/);
const request = new Request('http://localhost', { method: 'POST', headers: req.headers, body: req, duplex: 'half' });
const form = await request.formData();
const expected = index === 1 ? specified : { ...input, n: 1 };
assert.deepEqual([...form.keys()].sort(), [...Object.keys(expected), 'image'].sort());
for (const [key, value] of Object.entries(expected)) assert.equal(form.get(key), String(value));
assert.equal(form.getAll('image').length, 1);
const file = form.get('image');
assert.equal(file.name, `image.${extension}`);
assert.equal(file.type, type);
assert.deepEqual(Buffer.from(await file.arrayBuffer()), bytes);
json(res, { data: [{ b64_json: png.toString('base64'), revised_prompt: 'edited bird' }] });
});
for (const [i, [image]] of fixtures.entries()) {
const options = i === 0 ? specified : input;
const result = await editImages({ baseUrl: `${base}/custom/v1/`, apiKey, input: { ...options, ignored: true }, image });
assert.deepEqual(result, { model: options.model, prompt: options.prompt, images: [{ data: png, mediaType: 'image/png', revisedPrompt: 'edited bird' }] });
}
assert.equal(index, 3);
});
test('edits reject invalid images and options before any API request', async t => {
let count = 0;
const baseUrl = await server(t, (req, res) => { count++; json(res, { data: [] }); });
for (const image of [Buffer.alloc(0), Buffer.from('GIF89a'), Buffer.alloc(20 * 1024 * 1024 + 1)]) {
await assert.rejects(editImages({ baseUrl, apiKey, input, image }));
}
for (const change of [{ model: '' }, { prompt: '' }, { n: 5 }, { quality: 'invalid' }]) {
await assert.rejects(editImages({ baseUrl, apiKey, input: { ...input, ...change }, image: png }));
}
assert.equal(count, 0);
});
test('base URL normalization preserves API prefix and rejects unsafe parts', () => {
assert.equal(validateBaseUrl(' HTTPS://EXAMPLE.COM/v1/// '), 'https://example.com/v1');
for (const value of ['', null, '/v1', 'ftp://example.com', 'https://a:b@example.com', 'https://example.com?x=1', 'https://example.com#x', 'https://example.com?', 'https://example.com#']) {
@@ -207,6 +271,69 @@ test('180 second timeout aborts a stalled request', async t => {
assert.equal(configuredTimeout, 180_000);
});
test('edit response downloads omit auth and retain output validation and bounds', async t => {
const imageBase = await server(t, (req, res) => {
assert.equal(req.headers.authorization, undefined);
res.end(png);
});
let payload = { data: [{ url: `${imageBase}/image` }] };
let oversized = false;
const baseUrl = await server(t, (req, res) => {
if (oversized) { res.writeHead(200, { 'Content-Length': 64 * 1024 * 1024 + 1 }); res.flushHeaders(); }
else json(res, payload);
});
const edit = () => editImages({ baseUrl, apiKey, input, image: png });
assert.deepEqual((await edit()).images, [{ data: png, mediaType: 'image/png' }]);
payload = { data: [{ b64_json: '!!!!' }] };
await assert.rejects(edit(), /invalid base64/);
payload = { data: [] };
await assert.rejects(edit(), /malformed image response/);
oversized = true;
await assert.rejects(edit(), /64 MB response limit/);
});
test('edit API failure is sanitized and never retried', async t => {
let count = 0;
const baseUrl = await server(t, (req, res) => { count++; res.writeHead(503); res.end(`provider-private ${apiKey}`); });
await assert.rejects(editImages({ baseUrl, apiKey, input, image: png }), error => {
assert.equal(error.message, 'API request failed with HTTP 503.');
assert.equal(error.cause, undefined);
return true;
});
assert.equal(count, 1);
});
test('edits cancel pending API requests, downloads and pre-aborted requests', async t => {
let count = 0;
let controller = new AbortController();
let download = false;
const imageBase = await server(t, (req, res) => {
assert.equal(req.headers.authorization, undefined);
res.write(png.subarray(0, 4));
controller.abort(new Error(apiKey));
});
const baseUrl = await server(t, (req, res) => {
count++;
if (download) json(res, { data: [{ url: `${imageBase}/slow` }] });
else controller.abort(new Error(apiKey));
});
const edit = signal => editImages({ baseUrl, apiKey, input, image: png, signal });
await assert.rejects(edit(AbortSignal.abort(apiKey)), /^Error: Request aborted\.$/);
assert.equal(count, 0);
await assert.rejects(edit(controller.signal), /^Error: Request aborted\.$/);
controller = new AbortController();
download = true;
await assert.rejects(edit(controller.signal), /^Error: Request aborted\.$/);
assert.equal(count, 2);
});
test('edits share the 180 second operation timeout', async t => {
const baseUrl = await server(t, () => {});
const original = globalThis.setTimeout;
t.mock.method(globalThis, 'setTimeout', (callback, milliseconds, ...args) => original(callback, milliseconds === 180_000 ? 20 : milliseconds, ...args));
await assert.rejects(editImages({ baseUrl, apiKey, input, image: png }), /timed out after 180 seconds/);
});
test('pre-aborted signal does not send an API request', async t => {
let count = 0;
const baseUrl = await server(t, (req, res) => { count++; json(res, { data: [] }); });
+60 -9
View File
@@ -1,7 +1,7 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { createServer } from 'node:http';
import { mkdtemp, readFile, rm } from 'node:fs/promises';
import { mkdtemp, readFile, writeFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join, dirname, basename } from 'node:path';
@@ -27,12 +27,16 @@ test('native host integration using only a local image API', async t => {
const requests = [];
const server = createServer(async (request, response) => {
try {
let raw = '';
for await (const chunk of request) raw += chunk;
requests.push({ path: request.url, authorization: request.headers.authorization, body: raw ? JSON.parse(raw) : undefined });
const chunks = [];
for await (const chunk of request) chunks.push(chunk);
const raw = Buffer.concat(chunks);
const body = request.url === '/v1/images/edits'
? await new Response(raw, { headers: { 'Content-Type': request.headers['content-type'] } }).formData()
: raw.length ? JSON.parse(raw.toString()) : undefined;
requests.push({ path: request.url, authorization: request.headers.authorization, body });
response.setHeader('Content-Type', 'application/json');
if (request.url === '/v1/models') response.end(JSON.stringify({ data: [{ id: 'test-image-model' }] }));
else if (request.url === '/v1/images/generations') response.end(JSON.stringify({ data: [{ b64_json: PNG.toString('base64'), revised_prompt: 'Local test image' }] }));
else if (['/v1/images/generations', '/v1/images/edits'].includes(request.url)) response.end(JSON.stringify({ data: [{ b64_json: PNG.toString('base64'), revised_prompt: 'Local test image' }] }));
else { response.statusCode = 404; response.end('{}'); }
} catch { response.statusCode = 500; response.end('{}'); }
});
@@ -46,7 +50,7 @@ test('native host integration using only a local image API', async t => {
const ctx = {
on(event, listener) { listeners.set(event, listener); },
connection: { fetch: { register(route) {
assert.equal(route.requestBody, 'buffered');
assert.equal(route.requestBody, route.path === '/api/imagegen/edit' ? 'streaming' : 'buffered');
assert.ok(!routes.has(route.path));
routes.set(route.path, route);
return async () => routes.delete(route.path);
@@ -57,7 +61,7 @@ test('native host integration using only a local image API', async t => {
async modifyRecord(key, change) { validateCredentialName(key); const value = await change(records.get(key)); records.set(key, value); return value; },
async deleteRecord(key) { validateCredentialName(key); records.delete(key); },
},
attachments: { async saveImage(image) {
attachments: { async validateImage(image) { assert.deepEqual(Buffer.from(image.data), PNG); }, async saveImage(image) {
assert.deepEqual(Buffer.from(image.data), PNG);
assert.equal(image.mediaType, 'image/png');
const ref = { attachmentId: `test-attachment-${attachments.length}`, mediaType: image.mediaType, bytes: image.data.length, width: 1, height: 1, name: image.name };
@@ -80,8 +84,8 @@ test('native host integration using only a local image API', async t => {
await t.test('registers dependencies, routes, tools and skill', () => {
assert.deepEqual(host.inject, ['connection', 'tools', 'credentials', 'attachments', 'skills']);
assert.deepEqual([...routes.keys()].sort(), ['/api/imagegen/file', '/api/imagegen/generate', '/api/imagegen/models', '/api/imagegen/settings']);
assert.deepEqual([...tools.keys()].sort(), ['imagegen_generate', 'imagegen_status']);
assert.deepEqual([...routes.keys()].sort(), ['/api/imagegen/edit', '/api/imagegen/file', '/api/imagegen/generate', '/api/imagegen/models', '/api/imagegen/settings']);
assert.deepEqual([...tools.keys()].sort(), ['imagegen_edit', 'imagegen_generate', 'imagegen_status']);
assert.equal(skills.length, 1);
assert.equal(skills[0].name, 'dsh-imagegen');
assert.ok(skills[0].content.length > 0);
@@ -175,6 +179,53 @@ test('native host integration using only a local image API', async t => {
assert.deepEqual(await readFile(value.images[0].path), PNG);
assert.ok(!JSON.stringify(value).includes(key));
});
await t.test('edit tool uploads original bytes and saves separate output', async () => {
const inputPath = join(home, 'input.png');
await writeFile(inputPath, PNG);
const value = await tools.get('imagegen_edit').execute({ image_path: inputPath, model: 'test-image-model', prompt: 'Change background only' }, { signal: new AbortController().signal });
const request = requests.at(-1);
assert.equal(request.path, '/v1/images/edits');
assert.equal(request.authorization, `Bearer ${key}`);
assert.equal(request.body.get('model'), 'test-image-model');
assert.deepEqual(Buffer.from(await request.body.get('image').arrayBuffer()), PNG);
assert.deepEqual(await readFile(inputPath), PNG);
assert.notEqual(value.images[0].path, inputPath);
assert.deepEqual(await readFile(value.images[0].path), PNG);
assert.equal(tools.get('imagegen_edit').output.render({}, value)[1].type, 'image');
assert.ok(!JSON.stringify(value).includes(key));
const before = requests.length;
for (const image_path of ['relative.png', join(home, 'missing.png')]) {
await assert.rejects(tools.get('imagegen_edit').execute({ image_path, model: 'test-image-model', prompt: 'edit' }, { signal: new AbortController().signal }));
}
await assert.rejects(tools.get('imagegen_edit').execute({ image_path: inputPath, prompt: 'edit' }, { signal: new AbortController().signal }), /model/);
assert.equal(requests.length, before);
});
await t.test('multipart edit route accepts one file and rejects invalid uploads before API calls', async () => {
const send = form => routes.get('/api/imagegen/edit').fetch(new Request('http://localhost/api/imagegen/edit', { method: 'POST', body: form }));
const make = () => { const form = new FormData(); form.set('input', JSON.stringify({ model: 'test-image-model', prompt: 'Edit via browser' })); form.set('image', new Blob([PNG], { type: 'image/png' }), 'input.png'); return form; };
const result = await send(make());
assert.equal(result.status, 200);
assert.deepEqual(await readFile((await result.json()).images[0].path), PNG);
const before = requests.length;
const missing = make(); missing.delete('image');
const duplicate = make(); duplicate.append('image', new Blob([PNG]), 'second.png');
const invalid = make(); invalid.set('image', new Blob(['not an image']), 'fake.png');
const unsupported = make(); unsupported.set('mask', new Blob([PNG]), 'mask.png');
const invalidJson = make(); invalidJson.set('input', '{');
const noModel = make(); noModel.set('input', JSON.stringify({ prompt: 'edit' }));
for (const form of [missing, duplicate, invalid, unsupported, invalidJson, noModel]) assert.equal((await send(form)).status, 400);
assert.equal((await call('/api/imagegen/edit', 'POST', { image_path: join(home, 'input.png') })).status, 400);
const tooLarge = new Request('http://localhost/api/imagegen/edit', { method: 'POST', headers: { 'Content-Length': String(22 * 1024 * 1024) }, body: 'x' });
assert.equal((await routes.get('/api/imagegen/edit').fetch(tooLarge)).status, 400);
// Model bytes arriving from the browser transport; cancelling Node's own
// FormData encoder mid-Blob triggers an unrelated undici closed-stream bug.
const oversized = new Request('http://localhost/api/imagegen/edit', { method: 'POST', headers: { 'Content-Type': 'multipart/form-data; boundary=test' }, body: Buffer.alloc(22 * 1024 * 1024) });
assert.equal((await routes.get('/api/imagegen/edit').fetch(oversized)).status, 400, 'streamed body limit applies even without Content-Length');
const decode = ctx.attachments.validateImage;
ctx.attachments.validateImage = async () => { throw new Error('Image decode refused'); };
try { assert.equal((await send(make())).status, 400); } finally { ctx.attachments.validateImage = decode; }
assert.equal(requests.length, before);
});
await t.test('all digest prefixes are valid and clearing only removes the selected endpoint', async () => {
const originalRecords = new Map(records);
const prefixes = new Set();