Compare commits

..

4 Commits

Author SHA1 Message Date
Jim Liu 宝玉 fa155da15d fix: update version to 0.5.3 and enhance placeholder selection logic in publishArticle function 2026-01-17 01:29:10 -06:00
Jim Liu 宝玉 6194f71378 bump version 2026-01-16 21:44:14 -06:00
Jim Liu 宝玉 8e88cf4a8b feat: implement session management for image generation skills and add session handling functions 2026-01-16 20:20:07 -06:00
Jim Liu 宝玉 259baff413 update gemeni-web to support add image as reference 2026-01-16 19:06:14 -06:00
13 changed files with 757 additions and 153 deletions
+1 -1
View File
@@ -6,7 +6,7 @@
}, },
"metadata": { "metadata": {
"description": "Skills shared by Baoyu for improving daily work efficiency", "description": "Skills shared by Baoyu for improving daily work efficiency",
"version": "0.5.1" "version": "0.5.3"
}, },
"plugins": [ "plugins": [
{ {
+6
View File
@@ -104,6 +104,12 @@ For each page (cover + pages):
- If text-only: concatenate prompts into single text - If text-only: concatenate prompts into single text
- If multiple skills available, ask user preference - If multiple skills available, ask user preference
**Session Management**:
If the image generation skill supports `--sessionId`:
1. Generate a unique session ID at the start (e.g., `comic-{topic-slug}-{timestamp}`)
2. Use the same session ID for character sheet and all pages
3. This ensures visual consistency (character appearance, style) across all generated images
3. Report progress after each generation 3. Report progress after each generation
### Step 5: Completion Report ### Step 5: Completion Report
+58 -11
View File
@@ -5,6 +5,13 @@ description: Image generation skill using Gemini Web. Generates images from text
# Gemini Web Client # Gemini Web Client
Supports:
- Text generation
- Image generation (download + save)
- Reference image upload (attach images for vision tasks)
- Multi-turn conversations within the same executor instance (`keepSession`)
- Experimental video generation (`generateVideo`) — Gemini may return an async placeholder; download might require Gemini web UI
## Quick start ## Quick start
```bash ```bash
@@ -12,8 +19,25 @@ npx -y bun scripts/main.ts "Hello, Gemini"
npx -y bun scripts/main.ts --prompt "Explain quantum computing" npx -y bun scripts/main.ts --prompt "Explain quantum computing"
npx -y bun scripts/main.ts --prompt "A cute cat" --image cat.png npx -y bun scripts/main.ts --prompt "A cute cat" --image cat.png
npx -y bun scripts/main.ts --promptfiles system.md content.md --image out.png npx -y bun scripts/main.ts --promptfiles system.md content.md --image out.png
# Multi-turn conversation (agent generates unique sessionId)
npx -y bun scripts/main.ts "Remember this: 42" --sessionId my-unique-id-123
npx -y bun scripts/main.ts "What number?" --sessionId my-unique-id-123
``` ```
## Executor options (programmatic)
This skill is typically consumed via `createGeminiWebExecutor(geminiOptions)` (see `scripts/executor.ts`).
Key options in `GeminiWebOptions`:
- `referenceImages?: string | string[]` Upload local images as references (vision input).
- `keepSession?: boolean` Reuse Gemini `chatMetadata` to continue the same conversation across calls (required if you want reference images to persist across multiple messages).
- `generateVideo?: string` Generate a video and (best-effort) download to the given path. Gemini may return `video_gen_chip` (async); in that case you must open Gemini web UI to download the result.
Notes:
- `generateVideo` cannot be combined with `generateImage` / `editImage`.
- When `keepSession=true` and `referenceImages` is set, reference images are uploaded once per executor instance.
## Commands ## Commands
### Text generation ### Text generation
@@ -58,17 +82,21 @@ npx -y bun scripts/main.ts "Hello" --json
## Options ## Options
| Option | Short | Description | | Option | Description |
|--------|-------|-------------| |--------|-------------|
| `--prompt <text>` | `-p` | Prompt text | | `--prompt <text>`, `-p` | Prompt text |
| `--promptfiles <files...>` | | Read prompt from files (concatenated in order) | | `--promptfiles <files...>` | Read prompt from files (concatenated in order) |
| `--model <id>` | `-m` | Model: gemini-3-pro (default), gemini-2.5-pro, gemini-2.5-flash | | `--model <id>`, `-m` | Model: gemini-3-pro (default), gemini-2.5-pro, gemini-2.5-flash |
| `--image [path]` | | Generate image, save to path (default: generated.png) | | `--image [path]` | Generate image, save to path (default: generated.png) |
| `--json` | | Output as JSON | | `--sessionId <id>` | Session ID for multi-turn conversation (agent generates unique ID) |
| `--login` | | Refresh cookies only, then exit | | `--list-sessions` | List saved sessions (max 100, sorted by update time) |
| `--cookie-path <path>` | | Custom cookie file path | | `--json` | Output as JSON |
| `--profile-dir <path>` | | Chrome profile directory | | `--login` | Refresh cookies only, then exit |
| `--help` | `-h` | Show help | | `--cookie-path <path>` | Custom cookie file path |
| `--profile-dir <path>` | Chrome profile directory |
| `--help`, `-h` | Show help |
CLI note: `scripts/main.ts` supports text generation, image generation, and multi-turn conversations via `--sessionId`. Reference images and video generation are exposed via the executor API.
## Models ## Models
@@ -116,3 +144,22 @@ npx -y bun scripts/main.ts "Hello" --json | jq '.text'
# Concatenate system.md + content.md as prompt # Concatenate system.md + content.md as prompt
npx -y bun scripts/main.ts --promptfiles system.md content.md --image output.png npx -y bun scripts/main.ts --promptfiles system.md content.md --image output.png
``` ```
### Multi-turn conversation
```bash
# Start a session with unique ID (agent generates this)
npx -y bun scripts/main.ts "You are a helpful math tutor." --sessionId task-abc123
# Continue the conversation (remembers context)
npx -y bun scripts/main.ts "What is 2+2?" --sessionId task-abc123
npx -y bun scripts/main.ts "Now multiply that by 10" --sessionId task-abc123
# List recent sessions (max 100, sorted by update time)
npx -y bun scripts/main.ts --list-sessions
```
Session files are stored in `~/Library/Application Support/baoyu-skills/gemini-web/sessions/<id>.json` and contain:
- `id`: Session ID
- `metadata`: Gemini chat metadata for continuation
- `messages`: Array of `{role, content, timestamp, error?}`
- `createdAt`, `updatedAt`: Timestamps
@@ -216,6 +216,7 @@ class CdpConnection {
export async function getGeminiCookieMapViaChrome(options?: { export async function getGeminiCookieMapViaChrome(options?: {
timeoutMs?: number; timeoutMs?: number;
debugConnectTimeoutMs?: number; debugConnectTimeoutMs?: number;
tokenCheckTimeoutMs?: number;
pollIntervalMs?: number; pollIntervalMs?: number;
log?: GeminiWebLog; log?: GeminiWebLog;
userDataDir?: string; userDataDir?: string;
@@ -224,6 +225,7 @@ export async function getGeminiCookieMapViaChrome(options?: {
const log = options?.log; const log = options?.log;
const timeoutMs = options?.timeoutMs ?? 5 * 60_000; const timeoutMs = options?.timeoutMs ?? 5 * 60_000;
const debugConnectTimeoutMs = options?.debugConnectTimeoutMs ?? 30_000; const debugConnectTimeoutMs = options?.debugConnectTimeoutMs ?? 30_000;
const tokenCheckTimeoutMs = options?.tokenCheckTimeoutMs ?? 30_000;
const pollIntervalMs = options?.pollIntervalMs ?? 2_000; const pollIntervalMs = options?.pollIntervalMs ?? 2_000;
const userDataDir = options?.userDataDir ?? resolveGeminiWebChromeProfileDir(); const userDataDir = options?.userDataDir ?? resolveGeminiWebChromeProfileDir();
@@ -290,7 +292,7 @@ export async function getGeminiCookieMapViaChrome(options?: {
if (hasRequiredGeminiCookies(cookieMap)) { if (hasRequiredGeminiCookies(cookieMap)) {
try { try {
const controller = new AbortController(); const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 10_000); const timer = setTimeout(() => controller.abort(), tokenCheckTimeoutMs);
try { try {
await fetchGeminiAccessToken(cookieMap, controller.signal); await fetchGeminiAccessToken(cookieMap, controller.signal);
} finally { } finally {
+135 -21
View File
@@ -67,19 +67,69 @@ function buildCookieHeader(cookieMap: Record<string, string>): string {
.join('; '); .join('; ');
} }
function getSetCookieHeaders(res: Response): string[] {
const headers = res.headers as unknown as { getSetCookie?: () => string[] };
if (typeof headers.getSetCookie === 'function') {
try {
return headers.getSetCookie();
} catch {
return [];
}
}
const raw = res.headers.get('set-cookie');
return raw ? [raw] : [];
}
function applySetCookiesToMap(setCookies: string[], cookieMap: Record<string, string>): void {
for (const raw of setCookies) {
const first = raw.split(';')[0]?.trim();
if (!first) continue;
const idx = first.indexOf('=');
if (idx <= 0) continue;
const name = first.slice(0, idx).trim();
const value = first.slice(idx + 1).trim();
if (!name) continue;
cookieMap[name] = value;
}
}
async function fetchWithCookieJar(
url: string,
init: Omit<RequestInit, 'redirect' | 'headers'> & { headers?: Record<string, string> },
cookieMap: Record<string, string>,
signal?: AbortSignal,
maxRedirects = 20,
): Promise<Response> {
let current = url;
for (let i = 0; i <= maxRedirects; i += 1) {
const cookieHeader = buildCookieHeader(cookieMap);
const headers: Record<string, string> = {
...(init.headers ?? {}),
...(cookieHeader ? { cookie: cookieHeader } : {}),
'user-agent': USER_AGENT,
};
const res = await fetch(current, { ...init, redirect: 'manual', signal, headers });
applySetCookiesToMap(getSetCookieHeaders(res), cookieMap);
if (res.status >= 300 && res.status < 400) {
const location = res.headers.get('location');
if (!location) return res;
current = new URL(location, current).toString();
continue;
}
return res;
}
throw new Error(`Too many redirects while fetching ${url} (>${maxRedirects}).`);
}
export async function fetchGeminiAccessToken( export async function fetchGeminiAccessToken(
cookieMap: Record<string, string>, cookieMap: Record<string, string>,
signal?: AbortSignal, signal?: AbortSignal,
): Promise<string> { ): Promise<string> {
const cookieHeader = buildCookieHeader(cookieMap); const res = await fetchWithCookieJar(GEMINI_APP_URL, { method: 'GET' }, cookieMap, signal);
const res = await fetch(GEMINI_APP_URL, {
redirect: 'follow',
signal,
headers: {
cookie: cookieHeader,
'user-agent': USER_AGENT,
},
});
const html = await res.text(); const html = await res.text();
const tokens = ['SNlM0e', 'thykhd'] as const; const tokens = ['SNlM0e', 'thykhd'] as const;
@@ -107,7 +157,21 @@ function extractErrorCode(responseJson: unknown): number | undefined {
} }
function extractGgdlUrls(rawText: string): string[] { function extractGgdlUrls(rawText: string): string[] {
const matches = rawText.match(/https:\/\/lh3\.googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? []; const matches =
rawText.match(/https?:\/\/[^/\s"']*googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? [];
const seen = new Set<string>();
const urls: string[] = [];
for (const match of matches) {
if (seen.has(match)) continue;
seen.add(match);
urls.push(match);
}
return urls;
}
function extractImageGenerationContentUrls(rawText: string): string[] {
const matches =
rawText.match(/https?:\/\/googleusercontent\.com\/image_generation_content\/\d+/g) ?? [];
const seen = new Set<string>(); const seen = new Set<string>();
const urls: string[] = []; const urls: string[] = [];
for (const match of matches) { for (const match of matches) {
@@ -119,9 +183,17 @@ function extractGgdlUrls(rawText: string): string[] {
} }
function ensureFullSizeImageUrl(url: string): string { function ensureFullSizeImageUrl(url: string): string {
if (url.includes('=s2048')) return url; const trimmed = url.trim();
if (url.includes('=s')) return url; let normalized = trimmed;
return `${url}=s2048`; const backslashIndex = normalized.indexOf('\\');
if (backslashIndex >= 0) normalized = normalized.slice(0, backslashIndex);
// Some Gemini responses embed a size suffix as "/=s2048" which breaks downloads.
normalized = normalized.replace(/\/=s(?=\d+(?:$|[?#]))/, '=s');
normalized = normalized.replace(/\/=s(?=$|[?#])/, '=s');
if (normalized.endsWith('/')) normalized = normalized.slice(0, -1);
if (normalized.includes('=s2048')) return normalized;
if (normalized.includes('=s')) return normalized;
return `${normalized}=s2048`;
} }
async function fetchWithCookiePreservingRedirects( async function fetchWithCookiePreservingRedirects(
@@ -190,6 +262,29 @@ async function uploadGeminiFile(filePath: string, signal?: AbortSignal): Promise
return { id: text, name: fileName }; return { id: text, name: fileName };
} }
function guessMimeType(fileName: string): string {
const ext = path.extname(fileName).toLowerCase();
switch (ext) {
case '.png':
return 'image/png';
case '.jpg':
case '.jpeg':
return 'image/jpeg';
case '.webp':
return 'image/webp';
case '.gif':
return 'image/gif';
case '.mp4':
return 'video/mp4';
case '.mov':
return 'video/quicktime';
case '.webm':
return 'video/webm';
default:
return 'application/octet-stream';
}
}
function buildGeminiFReqPayload( function buildGeminiFReqPayload(
prompt: string, prompt: string,
uploaded: Array<{ id: string; name: string }>, uploaded: Array<{ id: string; name: string }>,
@@ -201,9 +296,8 @@ function buildGeminiFReqPayload(
prompt, prompt,
0, 0,
null, null,
// Matches gemini-webapi payload format: [[[fileId, 1]]] for a single attachment. // Matches gemini-web payload format: [[[fileId, 1, null, mimeType], fileName]] for an attachment.
// Keep it extensible for multiple uploads by emitting one [[id, 1]] entry per file. uploaded.map((file) => [[file.id, 1, null, guessMimeType(file.name)], file.name]),
uploaded.map((file) => [[file.id, 1]]),
] ]
: [prompt]; : [prompt];
@@ -248,7 +342,19 @@ export function parseGeminiStreamGenerateResponse(rawText: string): {
? (getNestedValue<string | null>(firstCandidate, [22, 0], null) ?? textRaw) ? (getNestedValue<string | null>(firstCandidate, [22, 0], null) ?? textRaw)
: textRaw; : textRaw;
const thoughts = getNestedValue<string | null>(firstCandidate, [37, 0, 0], null); const thoughts = getNestedValue<string | null>(firstCandidate, [37, 0, 0], null);
const metadata = getNestedValue<unknown>(body, [1], []); const conversationMeta = getNestedValue<unknown[]>(body, [1], []);
const conversationId =
typeof conversationMeta[0] === 'string' && conversationMeta[0].length > 0
? conversationMeta[0]
: null;
const responseId =
typeof conversationMeta[1] === 'string' && conversationMeta[1].length > 0
? conversationMeta[1]
: null;
const choiceIdRaw = getNestedValue<string | null>(firstCandidate, [0], null);
const choiceId = typeof choiceIdRaw === 'string' && choiceIdRaw.length > 0 ? choiceIdRaw : null;
const metadata =
conversationId && responseId && choiceId ? [conversationId, responseId, choiceId] : conversationMeta;
const images: GeminiWebCandidateImage[] = []; const images: GeminiWebCandidateImage[] = [];
@@ -305,8 +411,8 @@ export function isGeminiModelUnavailable(errorCode: number | undefined): boolean
} }
export async function runGeminiWebOnce(input: GeminiWebRunInput): Promise<GeminiWebRunOutput> { export async function runGeminiWebOnce(input: GeminiWebRunInput): Promise<GeminiWebRunOutput> {
const cookieHeader = buildCookieHeader(input.cookieMap);
const at = await fetchGeminiAccessToken(input.cookieMap, input.signal); const at = await fetchGeminiAccessToken(input.cookieMap, input.signal);
const cookieHeader = buildCookieHeader(input.cookieMap);
const uploaded: Array<{ id: string; name: string }> = []; const uploaded: Array<{ id: string; name: string }> = [];
for (const file of input.files ?? []) { for (const file of input.files ?? []) {
@@ -403,11 +509,19 @@ export async function saveFirstGeminiImageFromOutput(
return { saved: true, imageCount: output.images.length }; return { saved: true, imageCount: output.images.length };
} }
const ggdl = extractGgdlUrls(output.rawResponseText); const ggdl = extractGgdlUrls(`${output.text}\n${output.rawResponseText}`);
if (ggdl[0]) { const preferred = ggdl.length > 0 ? ggdl[ggdl.length - 1] : null;
await downloadGeminiImage(ggdl[0], cookieMap, outputPath, signal); if (preferred) {
await downloadGeminiImage(preferred, cookieMap, outputPath, signal);
return { saved: true, imageCount: ggdl.length }; return { saved: true, imageCount: ggdl.length };
} }
const imageGen = extractImageGenerationContentUrls(`${output.text}\n${output.rawResponseText}`);
const imageGenPreferred = imageGen.length > 0 ? imageGen[imageGen.length - 1] : null;
if (imageGenPreferred) {
await downloadGeminiImage(imageGenPreferred, cookieMap, outputPath, signal);
return { saved: true, imageCount: imageGen.length };
}
return { saved: false, imageCount: 0 }; return { saved: false, imageCount: 0 };
} }
+193 -10
View File
@@ -1,7 +1,8 @@
import { mkdir, writeFile } from 'node:fs/promises';
import path from 'node:path'; import path from 'node:path';
import type { BrowserRunOptions, BrowserRunResult, BrowserLogger, CookieParam } from '../browser/types.js'; import type { BrowserRunOptions, BrowserRunResult, BrowserLogger, CookieParam } from '../browser/types.js';
import { runGeminiWebWithFallback, saveFirstGeminiImageFromOutput } from './client.js'; import { runGeminiWebWithFallback, saveFirstGeminiImageFromOutput } from './client.js';
import type { GeminiWebModelId } from './client.js'; import type { GeminiWebModelId, GeminiWebRunOutput } from './client.js';
import { import {
buildGeminiCookieMap, buildGeminiCookieMap,
hasRequiredGeminiCookies, hasRequiredGeminiCookies,
@@ -11,6 +12,9 @@ import type { GeminiWebOptions, GeminiWebResponse } from './types.js';
export { hasRequiredGeminiCookies } from './cookie-store.js'; export { hasRequiredGeminiCookies } from './cookie-store.js';
const USER_AGENT =
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36';
function estimateTokenCount(text: string): number { function estimateTokenCount(text: string): number {
return Math.ceil(text.length / 4); return Math.ceil(text.length / 4);
} }
@@ -22,6 +26,115 @@ function resolveInvocationPath(value: string | undefined): string | undefined {
return path.isAbsolute(trimmed) ? trimmed : path.resolve(process.cwd(), trimmed); return path.isAbsolute(trimmed) ? trimmed : path.resolve(process.cwd(), trimmed);
} }
function normalizePathList(value: string | string[] | undefined): string[] {
if (!value) return [];
const raw = Array.isArray(value) ? value : [value];
const out: string[] = [];
for (const entry of raw) {
if (typeof entry !== 'string') continue;
const resolved = resolveInvocationPath(entry);
if (!resolved) continue;
out.push(resolved);
}
return out;
}
function dedupePaths(paths: string[]): string[] {
const seen = new Set<string>();
const out: string[] = [];
for (const item of paths) {
const trimmed = item.trim();
if (!trimmed || seen.has(trimmed)) continue;
seen.add(trimmed);
out.push(trimmed);
}
return out;
}
function buildCookieHeader(cookieMap: Record<string, string>): string {
return Object.entries(cookieMap)
.filter(([, value]) => typeof value === 'string' && value.length > 0)
.map(([name, value]) => `${name}=${value}`)
.join('; ');
}
async function fetchWithCookiePreservingRedirects(
url: string,
init: Omit<RequestInit, 'redirect'>,
signal?: AbortSignal,
maxRedirects = 10,
): Promise<Response> {
let current = url;
for (let i = 0; i <= maxRedirects; i += 1) {
const res = await fetch(current, { ...init, redirect: 'manual', signal });
if (res.status >= 300 && res.status < 400) {
const location = res.headers.get('location');
if (!location) return res;
current = new URL(location, current).toString();
continue;
}
return res;
}
throw new Error(`Too many redirects while downloading media (>${maxRedirects}).`);
}
async function downloadGeminiMedia(
url: string,
cookieMap: Record<string, string>,
outputPath: string,
signal?: AbortSignal,
): Promise<void> {
const cookieHeader = buildCookieHeader(cookieMap);
const res = await fetchWithCookiePreservingRedirects(
url,
{
headers: {
cookie: cookieHeader,
'user-agent': USER_AGENT,
},
},
signal,
);
if (!res.ok) {
throw new Error(`Failed to download media: ${res.status} ${res.statusText} (${res.url})`);
}
const data = new Uint8Array(await res.arrayBuffer());
await mkdir(path.dirname(outputPath), { recursive: true });
await writeFile(outputPath, data);
}
function extractGgdlUrls(rawText: string): string[] {
const matches =
rawText.match(/https?:\/\/[^/\s"']*googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? [];
const seen = new Set<string>();
const urls: string[] = [];
for (const match of matches) {
if (seen.has(match)) continue;
seen.add(match);
urls.push(match);
}
return urls;
}
async function saveFirstGeminiVideoFromOutput(
output: GeminiWebRunOutput,
cookieMap: Record<string, string>,
outputPath: string,
signal?: AbortSignal,
): Promise<{ saved: boolean; videoCount: number }> {
const ggdl = extractGgdlUrls(output.rawResponseText);
if (!ggdl[0]) return { saved: false, videoCount: 0 };
const videoCandidates = ggdl.filter((url) => /\.(mp4|webm|mov)(?:$|[?#])/i.test(url));
const preferred =
(videoCandidates.length > 0 ? videoCandidates[videoCandidates.length - 1] : null) ??
ggdl.find((url) => /video/i.test(url)) ??
ggdl[ggdl.length - 1];
await downloadGeminiMedia(preferred, cookieMap, outputPath, signal);
return { saved: true, videoCount: ggdl.length };
}
function resolveGeminiWebModel( function resolveGeminiWebModel(
desiredModel: string | null | undefined, desiredModel: string | null | undefined,
log?: BrowserLogger, log?: BrowserLogger,
@@ -115,6 +228,9 @@ export async function loadGeminiCookieMap(log?: BrowserLogger): Promise<Record<s
export function createGeminiWebExecutor( export function createGeminiWebExecutor(
geminiOptions: GeminiWebOptions, geminiOptions: GeminiWebOptions,
): (runOptions: BrowserRunOptions) => Promise<BrowserRunResult> { ): (runOptions: BrowserRunOptions) => Promise<BrowserRunResult> {
let persistedChatMetadata: unknown | null = null;
let referenceImagesUploaded = false;
return async (runOptions: BrowserRunOptions): Promise<BrowserRunResult> => { return async (runOptions: BrowserRunOptions): Promise<BrowserRunResult> => {
const startTime = Date.now(); const startTime = Date.now();
const log = runOptions.log; const log = runOptions.log;
@@ -133,23 +249,38 @@ export function createGeminiWebExecutor(
? Math.max(1_000, runOptions.config.timeoutMs) ? Math.max(1_000, runOptions.config.timeoutMs)
: null; : null;
const generateVideoPath = resolveInvocationPath(geminiOptions.generateVideo);
const defaultTimeoutMs = geminiOptions.youtube const defaultTimeoutMs = geminiOptions.youtube
? 240_000 ? 240_000
: geminiOptions.generateImage || geminiOptions.editImage : generateVideoPath
? 900_000
: geminiOptions.generateImage || geminiOptions.editImage
? 300_000 ? 300_000
: 120_000; : 120_000;
const timeoutMs = Math.min(configTimeout ?? defaultTimeoutMs, 600_000); const timeoutCapMs = generateVideoPath ? 1_800_000 : 600_000;
const timeoutMs = Math.min(configTimeout ?? defaultTimeoutMs, timeoutCapMs);
const controller = new AbortController(); const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs); const timeout = setTimeout(() => controller.abort(), timeoutMs);
const keepSession = geminiOptions.keepSession === true;
const generateImagePath = resolveInvocationPath(geminiOptions.generateImage); const generateImagePath = resolveInvocationPath(geminiOptions.generateImage);
const editImagePath = resolveInvocationPath(geminiOptions.editImage); const editImagePath = resolveInvocationPath(geminiOptions.editImage);
const outputPath = resolveInvocationPath(geminiOptions.outputPath); const outputPath = resolveInvocationPath(geminiOptions.outputPath);
const attachmentPaths = (runOptions.attachments ?? []).map((attachment) => attachment.path); const attachmentPaths = (runOptions.attachments ?? []).map((attachment) => attachment.path);
const referenceImagePaths = normalizePathList(geminiOptions.referenceImages);
const requestFilePaths = dedupePaths(
keepSession ? attachmentPaths : [...referenceImagePaths, ...attachmentPaths],
);
if (generateVideoPath && (generateImagePath || editImagePath)) {
throw new Error('Gemini web executor: generateVideo cannot be combined with generateImage/editImage options.');
}
let prompt = runOptions.prompt; let prompt = runOptions.prompt;
if (geminiOptions.aspectRatio && (generateImagePath || editImagePath)) { if (geminiOptions.aspectRatio && (generateImagePath || editImagePath || generateVideoPath)) {
prompt = `${prompt} (aspect ratio: ${geminiOptions.aspectRatio})`; prompt = `${prompt} (aspect ratio: ${geminiOptions.aspectRatio})`;
} }
if (geminiOptions.youtube) { if (geminiOptions.youtube) {
@@ -158,29 +289,50 @@ export function createGeminiWebExecutor(
if (generateImagePath && !editImagePath) { if (generateImagePath && !editImagePath) {
prompt = `Generate an image: ${prompt}`; prompt = `Generate an image: ${prompt}`;
} }
if (generateVideoPath) {
prompt = `Generate a video: ${prompt}`;
}
const model: GeminiWebModelId = resolveGeminiWebModel(runOptions.config?.desiredModel, log); const model: GeminiWebModelId = resolveGeminiWebModel(runOptions.config?.desiredModel, log);
let response: GeminiWebResponse; let response: GeminiWebResponse;
let videoSaveSummary: { saved: boolean; videoCount: number; outputPath: string } | null = null;
try { try {
let chatMetadata: unknown = keepSession ? persistedChatMetadata : null;
if (keepSession && referenceImagePaths.length > 0 && !referenceImagesUploaded) {
const intro = await runGeminiWebWithFallback({
prompt: 'Here are reference images for future messages.',
files: referenceImagePaths,
model,
cookieMap,
chatMetadata,
signal: controller.signal,
});
chatMetadata = intro.metadata;
persistedChatMetadata = intro.metadata;
referenceImagesUploaded = true;
}
if (editImagePath) { if (editImagePath) {
const intro = await runGeminiWebWithFallback({ const intro = await runGeminiWebWithFallback({
prompt: 'Here is an image to edit', prompt: 'Here is an image to edit',
files: [editImagePath], files: [editImagePath],
model, model,
cookieMap, cookieMap,
chatMetadata: null, chatMetadata,
signal: controller.signal, signal: controller.signal,
}); });
const editPrompt = `Use image generation tool to ${prompt}`; const editPrompt = `Use image generation tool to ${prompt}`;
const out = await runGeminiWebWithFallback({ const out = await runGeminiWebWithFallback({
prompt: editPrompt, prompt: editPrompt,
files: attachmentPaths, files: requestFilePaths,
model, model,
cookieMap, cookieMap,
chatMetadata: intro.metadata, chatMetadata: intro.metadata,
signal: controller.signal, signal: controller.signal,
}); });
if (keepSession) persistedChatMetadata = out.metadata;
response = { response = {
text: out.text ?? null, text: out.text ?? null,
thoughts: geminiOptions.showThoughts ? out.thoughts : null, thoughts: geminiOptions.showThoughts ? out.thoughts : null,
@@ -198,12 +350,13 @@ export function createGeminiWebExecutor(
} else if (generateImagePath) { } else if (generateImagePath) {
const out = await runGeminiWebWithFallback({ const out = await runGeminiWebWithFallback({
prompt, prompt,
files: attachmentPaths, files: requestFilePaths,
model, model,
cookieMap, cookieMap,
chatMetadata: null, chatMetadata,
signal: controller.signal, signal: controller.signal,
}); });
if (keepSession) persistedChatMetadata = out.metadata;
response = { response = {
text: out.text ?? null, text: out.text ?? null,
thoughts: geminiOptions.showThoughts ? out.thoughts : null, thoughts: geminiOptions.showThoughts ? out.thoughts : null,
@@ -216,15 +369,36 @@ export function createGeminiWebExecutor(
if (!imageSave.saved) { if (!imageSave.saved) {
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`); throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
} }
} else if (generateVideoPath) {
const out = await runGeminiWebWithFallback({
prompt,
files: requestFilePaths,
model,
cookieMap,
chatMetadata,
signal: controller.signal,
});
if (keepSession) persistedChatMetadata = out.metadata;
response = {
text: out.text ?? null,
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
has_images: false,
image_count: 0,
};
const resolvedOutputPath = generateVideoPath ?? outputPath ?? 'generated.mp4';
const save = await saveFirstGeminiVideoFromOutput(out, cookieMap, resolvedOutputPath, controller.signal);
videoSaveSummary = { ...save, outputPath: resolvedOutputPath };
} else { } else {
const out = await runGeminiWebWithFallback({ const out = await runGeminiWebWithFallback({
prompt, prompt,
files: attachmentPaths, files: requestFilePaths,
model, model,
cookieMap, cookieMap,
chatMetadata: null, chatMetadata,
signal: controller.signal, signal: controller.signal,
}); });
if (keepSession) persistedChatMetadata = out.metadata;
response = { response = {
text: out.text ?? null, text: out.text ?? null,
thoughts: geminiOptions.showThoughts ? out.thoughts : null, thoughts: geminiOptions.showThoughts ? out.thoughts : null,
@@ -247,6 +421,15 @@ export function createGeminiWebExecutor(
const imagePath = generateImagePath || outputPath || 'generated.png'; const imagePath = generateImagePath || outputPath || 'generated.png';
answerMarkdown += `\n\n*Generated ${response.image_count} image(s). Saved to: ${imagePath}*`; answerMarkdown += `\n\n*Generated ${response.image_count} image(s). Saved to: ${imagePath}*`;
} }
if (videoSaveSummary) {
if (videoSaveSummary.saved) {
answerMarkdown += `\n\n*Generated ${videoSaveSummary.videoCount || 1} video(s). Saved to: ${videoSaveSummary.outputPath}*`;
} else if (/video_gen_chip/.test(answerMarkdown) || /video_gen_chip/.test(response.text ?? '')) {
answerMarkdown += '\n\n*Video generation is asynchronous. Check Gemini web UI to download the result.*';
} else {
answerMarkdown += '\n\n*No downloadable video URL found in Gemini response.*';
}
}
const tookMs = Date.now() - startTime; const tookMs = Date.now() - startTime;
log?.(`[gemini-web] Completed in ${tookMs}ms`); log?.(`[gemini-web] Completed in ${tookMs}ms`);
+139 -65
View File
@@ -10,6 +10,7 @@ import {
writeGeminiCookieMapToDisk, writeGeminiCookieMapToDisk,
} from './cookie-store.js'; } from './cookie-store.js';
import { resolveGeminiWebChromeProfileDir, resolveGeminiWebCookiePath } from './paths.js'; import { resolveGeminiWebChromeProfileDir, resolveGeminiWebCookiePath } from './paths.js';
import { readSession, writeSession, listSessions } from './session-store.js';
function printUsage(exitCode = 0): never { function printUsage(exitCode = 0): never {
const cookiePath = resolveGeminiWebCookiePath(); const cookiePath = resolveGeminiWebCookiePath();
@@ -21,12 +22,19 @@ function printUsage(exitCode = 0): never {
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --prompt "A cute cat" --image generated.png npx -y bun skills/baoyu-gemini-web/scripts/main.ts --prompt "A cute cat" --image generated.png
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png npx -y bun skills/baoyu-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
Multi-turn conversation (agent generates unique sessionId):
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "Remember 42" --sessionId abc123
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "What number?" --sessionId abc123
Options: Options:
-p, --prompt <text> Prompt text -p, --prompt <text> Prompt text
--promptfiles <files...> Read prompt from one or more files (concatenated in order) --promptfiles <files...> Read prompt from one or more files (concatenated in order)
-m, --model <id> gemini-3-pro | gemini-2.5-pro | gemini-2.5-flash (default: gemini-3-pro) -m, --model <id> gemini-3-pro | gemini-2.5-pro | gemini-2.5-flash (default: gemini-3-pro)
--json Output JSON --json Output JSON
--image [path] Generate an image and save it (default: ./generated.png) --image [path] Generate an image and save it (default: ./generated.png)
--reference <files...> Reference images for vision input
--sessionId <id> Session ID for multi-turn conversation (agent should generate unique ID)
--list-sessions List saved sessions (max 100, sorted by update time)
--login Only refresh cookies, then exit --login Only refresh cookies, then exit
--cookie-path <path> Cookie file path (default: ${cookiePath}) --cookie-path <path> Cookie file path (default: ${cookiePath})
--profile-dir <path> Chrome profile dir (default: ${profileDir}) --profile-dir <path> Chrome profile dir (default: ${profileDir})
@@ -75,6 +83,9 @@ function parseArgs(argv: string[]): {
loginOnly?: boolean; loginOnly?: boolean;
cookiePath?: string; cookiePath?: string;
profileDir?: string; profileDir?: string;
referenceImages?: string[];
sessionId?: string;
listSessions?: boolean;
} { } {
const out: ReturnType<typeof parseArgs> = {}; const out: ReturnType<typeof parseArgs> = {};
const positional: string[] = []; const positional: string[] = [];
@@ -157,6 +168,32 @@ function parseArgs(argv: string[]): {
out.profileDir = arg.slice('--profile-dir='.length); out.profileDir = arg.slice('--profile-dir='.length);
continue; continue;
} }
if (arg === '--reference' || arg === '--ref') {
out.referenceImages = [];
while (i + 1 < argv.length) {
const next = argv[i + 1];
if (next && !next.startsWith('-')) {
out.referenceImages.push(next);
i += 1;
} else {
break;
}
}
continue;
}
if (arg === '--sessionId' || arg === '--session-id') {
out.sessionId = argv[i + 1] ?? '';
i += 1;
continue;
}
if (arg.startsWith('--sessionId=') || arg.startsWith('--session-id=')) {
out.sessionId = arg.split('=')[1] ?? '';
continue;
}
if (arg === '--list-sessions') {
out.listSessions = true;
continue;
}
if (arg.startsWith('-')) { if (arg.startsWith('-')) {
throw new Error(`Unknown option: ${arg}`); throw new Error(`Unknown option: ${arg}`);
@@ -178,6 +215,9 @@ function parseArgs(argv: string[]): {
if (out.cookiePath === '') delete out.cookiePath; if (out.cookiePath === '') delete out.cookiePath;
if (out.profileDir === '') delete out.profileDir; if (out.profileDir === '') delete out.profileDir;
if (out.promptFiles?.length === 0) delete out.promptFiles; if (out.promptFiles?.length === 0) delete out.promptFiles;
if (out.referenceImages?.length === 0) delete out.referenceImages;
if (out.sessionId != null) out.sessionId = out.sessionId.trim();
if (out.sessionId === '') delete out.sessionId;
return out; return out;
} }
@@ -186,7 +226,7 @@ async function isCookieMapValid(cookieMap: Record<string, string>): Promise<bool
if (!hasRequiredGeminiCookies(cookieMap)) return false; if (!hasRequiredGeminiCookies(cookieMap)) return false;
const controller = new AbortController(); const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 10_000); const timer = setTimeout(() => controller.abort(), 30_000);
try { try {
await fetchGeminiAccessToken(cookieMap, controller.signal); await fetchGeminiAccessToken(cookieMap, controller.signal);
return true; return true;
@@ -201,7 +241,7 @@ async function ensureGeminiCookieMap(options: {
cookiePath: string; cookiePath: string;
profileDir: string; profileDir: string;
}): Promise<Record<string, string>> { }): Promise<Record<string, string>> {
const log = (msg: string) => console.log(msg); const log = (msg: string) => console.error(msg);
let cookieMap = await readGeminiCookieMapFromDisk({ cookiePath: options.cookiePath, log }); let cookieMap = await readGeminiCookieMapFromDisk({ cookiePath: options.cookiePath, log });
if (await isCookieMapValid(cookieMap)) return cookieMap; if (await isCookieMapValid(cookieMap)) return cookieMap;
@@ -251,85 +291,63 @@ async function main(): Promise<void> {
const cookiePath = args.cookiePath ?? resolveGeminiWebCookiePath(); const cookiePath = args.cookiePath ?? resolveGeminiWebCookiePath();
const profileDir = args.profileDir ?? resolveGeminiWebChromeProfileDir(); const profileDir = args.profileDir ?? resolveGeminiWebChromeProfileDir();
if (args.listSessions) {
const sessions = await listSessions();
if (sessions.length === 0) {
console.log('No saved sessions.');
} else {
for (const { id, updatedAt } of sessions) {
console.log(`${id}\t${updatedAt}`);
}
}
return;
}
if (args.loginOnly) { if (args.loginOnly) {
await ensureGeminiCookieMap({ cookiePath, profileDir }); await ensureGeminiCookieMap({ cookiePath, profileDir });
return; return;
} }
const promptFromStdin = await readPromptFromStdin();
const promptFromFiles = args.promptFiles ? readPromptFiles(args.promptFiles) : null; const promptFromFiles = args.promptFiles ? readPromptFiles(args.promptFiles) : null;
const prompt = promptFromFiles || args.prompt || promptFromStdin; const promptFromArgs = promptFromFiles || args.prompt;
const prompt = promptFromArgs || (await readPromptFromStdin());
if (!prompt) printUsage(1); if (!prompt) printUsage(1);
const sessionData = args.sessionId ? await readSession(args.sessionId) : null;
const chatMetadata = sessionData?.metadata ?? null;
let cookieMap = await ensureGeminiCookieMap({ cookiePath, profileDir }); let cookieMap = await ensureGeminiCookieMap({ cookiePath, profileDir });
const desiredModel = resolveModel(args.model || 'gemini-3-pro'); const desiredModel = resolveModel(args.model || 'gemini-3-pro');
const imagePath = resolveImageOutputPath(args.imagePath); const imagePath = resolveImageOutputPath(args.imagePath);
const referenceImages = (args.referenceImages ?? []).map((p) =>
path.isAbsolute(p) ? p : path.resolve(process.cwd(), p),
);
try { try {
const effectivePrompt = imagePath ? `Generate an image: ${prompt}` : prompt; const controller = new AbortController();
const out = await runGeminiWebWithFallback({ const timeoutMs = imagePath ? 300_000 : 120_000;
prompt: effectivePrompt, const timeout = setTimeout(() => controller.abort(), timeoutMs);
files: [],
model: desiredModel,
cookieMap,
chatMetadata: null,
});
let imageSaved = false;
let imageCount = 0;
if (imagePath) {
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath);
imageSaved = save.saved;
imageCount = save.imageCount;
if (!imageSaved) {
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
}
}
if (args.json) {
process.stdout.write(
`${JSON.stringify(
imagePath ? { ...out, imageSaved, imageCount, imagePath } : out,
null,
2,
)}\n`,
);
if (out.errorMessage) process.exit(1);
return;
}
if (out.errorMessage) {
throw new Error(out.errorMessage);
}
process.stdout.write(out.text ?? '');
if (!out.text?.endsWith('\n')) process.stdout.write('\n');
if (imagePath) {
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
}
return;
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (message.includes('Unable to locate Gemini access token')) {
console.error('[gemini-web] Cookies may be expired. Re-opening browser to refresh cookies...');
await sleep(500);
cookieMap = await getGeminiCookieMapViaChrome({ userDataDir: profileDir, log: (m) => console.log(m) });
await writeGeminiCookieMapToDisk(cookieMap, { cookiePath, log: (m) => console.log(m) });
try {
const effectivePrompt = imagePath ? `Generate an image: ${prompt}` : prompt;
const out = await runGeminiWebWithFallback({ const out = await runGeminiWebWithFallback({
prompt: imagePath ? `Generate an image: ${prompt}` : prompt, prompt: effectivePrompt,
files: [], files: referenceImages,
model: desiredModel, model: desiredModel,
cookieMap, cookieMap,
chatMetadata: null, chatMetadata,
signal: controller.signal,
}); });
if (args.sessionId && out.metadata) {
await writeSession(args.sessionId, out.metadata, prompt, out.text ?? '', out.errorMessage);
}
let imageSaved = false; let imageSaved = false;
let imageCount = 0; let imageCount = 0;
if (imagePath) { if (imagePath) {
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath); const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath, controller.signal);
imageSaved = save.saved; imageSaved = save.saved;
imageCount = save.imageCount; imageCount = save.imageCount;
if (!imageSaved) { if (!imageSaved) {
@@ -338,13 +356,8 @@ async function main(): Promise<void> {
} }
if (args.json) { if (args.json) {
process.stdout.write( const jsonOut = { ...out, ...(imagePath && { imageSaved, imageCount, imagePath }), ...(args.sessionId && { sessionId: args.sessionId }) };
`${JSON.stringify( process.stdout.write(`${JSON.stringify(jsonOut, null, 2)}\n`);
imagePath ? { ...out, imageSaved, imageCount, imagePath } : out,
null,
2,
)}\n`,
);
if (out.errorMessage) process.exit(1); if (out.errorMessage) process.exit(1);
return; return;
} }
@@ -359,6 +372,67 @@ async function main(): Promise<void> {
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`); process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
} }
return; return;
} finally {
clearTimeout(timeout);
}
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (message.includes('Unable to locate Gemini access token')) {
console.error('[gemini-web] Cookies may be expired. Re-opening browser to refresh cookies...');
await sleep(500);
cookieMap = await getGeminiCookieMapViaChrome({ userDataDir: profileDir, log: (m) => console.error(m) });
await writeGeminiCookieMapToDisk(cookieMap, { cookiePath, log: (m) => console.error(m) });
const controller = new AbortController();
const timeoutMs = imagePath ? 300_000 : 120_000;
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const out = await runGeminiWebWithFallback({
prompt: imagePath ? `Generate an image: ${prompt}` : prompt,
files: referenceImages,
model: desiredModel,
cookieMap,
chatMetadata,
signal: controller.signal,
});
if (args.sessionId && out.metadata) {
await writeSession(args.sessionId, out.metadata, prompt, out.text ?? '', out.errorMessage);
}
let imageSaved = false;
let imageCount = 0;
if (imagePath) {
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath, controller.signal);
imageSaved = save.saved;
imageCount = save.imageCount;
if (!imageSaved) {
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
}
}
if (args.json) {
const jsonOut = { ...out, ...(imagePath && { imageSaved, imageCount, imagePath }), ...(args.sessionId && { sessionId: args.sessionId }) };
process.stdout.write(`${JSON.stringify(jsonOut, null, 2)}\n`);
if (out.errorMessage) process.exit(1);
return;
}
if (out.errorMessage) {
throw new Error(out.errorMessage);
}
process.stdout.write(out.text ?? '');
if (!out.text?.endsWith('\n')) process.stdout.write('\n');
if (imagePath) {
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
}
return;
} finally {
clearTimeout(timeout);
}
} }
throw error; throw error;
+9
View File
@@ -34,3 +34,12 @@ export function resolveGeminiWebChromeProfileDir(): string {
if (override) return path.resolve(override); if (override) return path.resolve(override);
return path.join(resolveGeminiWebDataDir(), PROFILE_DIR_NAME); return path.join(resolveGeminiWebDataDir(), PROFILE_DIR_NAME);
} }
export function resolveGeminiWebSessionsDir(): string {
return path.join(resolveGeminiWebDataDir(), 'sessions');
}
export function resolveGeminiWebSessionPath(name: string): string {
const sanitized = name.replace(/[^a-zA-Z0-9_-]/g, '_');
return path.join(resolveGeminiWebSessionsDir(), `${sanitized}.json`);
}
@@ -0,0 +1,90 @@
import { mkdir, readFile, writeFile, readdir, stat } from 'node:fs/promises';
import path from 'node:path';
import { resolveGeminiWebSessionsDir, resolveGeminiWebSessionPath } from './paths.js';
export interface SessionMessage {
role: 'user' | 'assistant';
content: string;
timestamp: string;
error?: string;
}
export interface SessionData {
id: string;
metadata: unknown;
messages: SessionMessage[];
createdAt: string;
updatedAt: string;
}
export interface SessionListItem {
id: string;
updatedAt: string;
}
export async function readSession(id: string): Promise<SessionData | null> {
const sessionPath = resolveGeminiWebSessionPath(id);
try {
const content = await readFile(sessionPath, 'utf8');
return JSON.parse(content) as SessionData;
} catch {
return null;
}
}
export async function writeSession(
id: string,
metadata: unknown,
userMessage: string,
assistantMessage: string,
error?: string,
): Promise<void> {
const sessionPath = resolveGeminiWebSessionPath(id);
const sessionsDir = resolveGeminiWebSessionsDir();
await mkdir(sessionsDir, { recursive: true });
const existing = await readSession(id);
const now = new Date().toISOString();
const newMessages: SessionMessage[] = [
{ role: 'user', content: userMessage, timestamp: now },
{ role: 'assistant', content: assistantMessage, timestamp: now, ...(error && { error }) },
];
const data: SessionData = {
id,
metadata,
messages: [...(existing?.messages ?? []), ...newMessages],
createdAt: existing?.createdAt ?? now,
updatedAt: now,
};
await writeFile(sessionPath, JSON.stringify(data, null, 2));
}
export async function listSessions(limit = 100): Promise<SessionListItem[]> {
const sessionsDir = resolveGeminiWebSessionsDir();
try {
const files = await readdir(sessionsDir);
const jsonFiles = files.filter((f) => f.endsWith('.json'));
const items: { id: string; updatedAt: string; mtime: number }[] = [];
for (const file of jsonFiles) {
const filePath = path.join(sessionsDir, file);
try {
const stats = await stat(filePath);
items.push({
id: file.slice(0, -5),
updatedAt: stats.mtime.toISOString(),
mtime: stats.mtime.getTime(),
});
} catch {
continue;
}
}
items.sort((a, b) => b.mtime - a.mtime);
return items.slice(0, limit).map(({ id, updatedAt }) => ({ id, updatedAt }));
} catch {
return [];
}
}
+9
View File
@@ -2,9 +2,18 @@ export interface GeminiWebOptions {
youtube?: string; youtube?: string;
generateImage?: string; generateImage?: string;
editImage?: string; editImage?: string;
generateVideo?: string;
outputPath?: string; outputPath?: string;
showThoughts?: boolean; showThoughts?: boolean;
aspectRatio?: string; aspectRatio?: string;
/**
* One or more local image paths to upload as persistent reference images.
* - If `keepSession` is enabled, they are uploaded once per executor session.
* - Otherwise, they are attached to each request.
*/
referenceImages?: string | string[];
/** Preserve Gemini chat metadata to continue multi-turn conversations within the same executor instance. */
keepSession?: boolean;
} }
export interface GeminiWebResponse { export interface GeminiWebResponse {
+98 -41
View File
@@ -481,52 +481,78 @@ export async function publishArticle(options: ArticleOptions): Promise<void> {
const img = sortedImages[i]!; const img = sortedImages[i]!;
console.log(`[x-article] [${i + 1}/${sortedImages.length}] Inserting image at placeholder: ${img.placeholder}`); console.log(`[x-article] [${i + 1}/${sortedImages.length}] Inserting image at placeholder: ${img.placeholder}`);
// Find, scroll to, and select the placeholder text in DraftEditor // Helper to select placeholder with retry
const placeholderFound = await cdp.send<{ result: { value: boolean } }>('Runtime.evaluate', { const selectPlaceholder = async (maxRetries = 3): Promise<boolean> => {
expression: `(() => { for (let attempt = 1; attempt <= maxRetries; attempt++) {
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]'); // Find, scroll to, and select the placeholder text in DraftEditor
if (!editor) return false; await cdp!.send('Runtime.evaluate', {
expression: `(() => {
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]');
if (!editor) return false;
const placeholder = ${JSON.stringify(img.placeholder)}; const placeholder = ${JSON.stringify(img.placeholder)};
// Search through all text nodes in the editor // Search through all text nodes in the editor
const walker = document.createTreeWalker(editor, NodeFilter.SHOW_TEXT, null, false); const walker = document.createTreeWalker(editor, NodeFilter.SHOW_TEXT, null, false);
let node; let node;
while ((node = walker.nextNode())) { while ((node = walker.nextNode())) {
const text = node.textContent || ''; const text = node.textContent || '';
const idx = text.indexOf(placeholder); const idx = text.indexOf(placeholder);
if (idx !== -1) { if (idx !== -1) {
// Found the placeholder - scroll to it first // Found the placeholder - scroll to it first
const parentElement = node.parentElement; const parentElement = node.parentElement;
if (parentElement) { if (parentElement) {
parentElement.scrollIntoView({ behavior: 'smooth', block: 'center' }); parentElement.scrollIntoView({ behavior: 'smooth', block: 'center' });
}
// Select it
const range = document.createRange();
range.setStart(node, idx);
range.setEnd(node, idx + placeholder.length);
const sel = window.getSelection();
sel.removeAllRanges();
sel.addRange(range);
return true;
}
} }
return false;
})()`,
}, { sessionId });
// Select it // Wait for scroll and selection to settle
const range = document.createRange(); await sleep(800);
range.setStart(node, idx);
range.setEnd(node, idx + placeholder.length); // Verify selection matches the placeholder
const sel = window.getSelection(); const selectionCheck = await cdp!.send<{ result: { value: string } }>('Runtime.evaluate', {
sel.removeAllRanges(); expression: `window.getSelection()?.toString() || ''`,
sel.addRange(range); returnByValue: true,
return true; }, { sessionId });
}
const selectedText = selectionCheck.result.value.trim();
if (selectedText === img.placeholder) {
console.log(`[x-article] Selection verified: "${selectedText}"`);
return true;
} }
return false;
})()`,
returnByValue: true,
}, { sessionId });
// Wait for scroll animation if (attempt < maxRetries) {
await sleep(500); console.log(`[x-article] Selection attempt ${attempt} got "${selectedText}", retrying...`);
await sleep(500);
} else {
console.warn(`[x-article] Selection failed after ${maxRetries} attempts, got: "${selectedText}"`);
}
}
return false;
};
if (!placeholderFound.result.value) { // Try to select the placeholder
console.warn(`[x-article] Placeholder not found in DOM: ${img.placeholder}`); const selected = await selectPlaceholder(3);
if (!selected) {
console.warn(`[x-article] Skipping image - could not select placeholder: ${img.placeholder}`);
continue; continue;
} }
console.log(`[x-article] Placeholder selected, copying image: ${path.basename(img.localPath)}`); console.log(`[x-article] Copying image: ${path.basename(img.localPath)}`);
// Copy image to clipboard // Copy image to clipboard
if (!copyImageToClipboard(img.localPath)) { if (!copyImageToClipboard(img.localPath)) {
@@ -535,17 +561,48 @@ export async function publishArticle(options: ArticleOptions): Promise<void> {
} }
// Wait for clipboard to be fully ready // Wait for clipboard to be fully ready
await sleep(800); await sleep(1000);
// Delete placeholder by pressing Enter (placeholder is already selected) // Delete placeholder by pressing Backspace (more reliable than Enter for replacing selection)
console.log(`[x-article] Deleting placeholder with Enter...`); console.log(`[x-article] Deleting placeholder...`);
await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13 }, { sessionId }); await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13 }, { sessionId }); await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
// Wait and verify placeholder is deleted
await sleep(500);
// Check that placeholder is no longer in editor
const afterDelete = await cdp.send<{ result: { value: boolean } }>('Runtime.evaluate', {
expression: `(() => {
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]');
if (!editor) return true;
return !editor.innerText.includes(${JSON.stringify(img.placeholder)});
})()`,
returnByValue: true,
}, { sessionId });
if (!afterDelete.result.value) {
console.warn(`[x-article] Placeholder may not have been deleted, trying again...`);
// Try selecting and deleting again
await selectPlaceholder(1);
await sleep(300);
await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
await sleep(500);
}
// Focus editor to ensure cursor is in position
await cdp.send('Runtime.evaluate', {
expression: `(() => {
const editor = document.querySelector('.DraftEditor-editorContainer [contenteditable="true"]');
if (editor) editor.focus();
})()`,
}, { sessionId });
await sleep(300); await sleep(300);
// Paste image using paste script (activates Chrome, sends real keystroke) // Paste image using paste script (activates Chrome, sends real keystroke)
console.log(`[x-article] Pasting image...`); console.log(`[x-article] Pasting image...`);
if (pasteFromClipboard('Google Chrome', 5, 800)) { if (pasteFromClipboard('Google Chrome', 5, 1000)) {
console.log(`[x-article] Image pasted: ${path.basename(img.localPath)}`); console.log(`[x-article] Image pasted: ${path.basename(img.localPath)}`);
} else { } else {
console.warn(`[x-article] Failed to paste image after retries`); console.warn(`[x-article] Failed to paste image after retries`);
+9 -2
View File
@@ -232,14 +232,21 @@ Style notes: [specific style characteristics to emphasize]
### Step 5: Generate Images ### Step 5: Generate Images
**Session Management**:
If the image generation skill supports `--sessionId`:
1. Generate a unique session ID at the start (e.g., `slides-{topic-slug}-{timestamp}`)
2. Use the same session ID for all slides
3. This ensures visual consistency (color scheme, style, typography) across all slides
For each slide, generate using: For each slide, generate using:
```bash ```bash
/baoyu-gemini-web --promptfiles [SKILL_ROOT]/skills/baoyu-slide-deck/prompts/system.md [TARGET_DIR]/prompts/01-cover.md --image [TARGET_DIR]/01-cover.png # With session support
/baoyu-gemini-web --promptfiles [SKILL_ROOT]/skills/baoyu-slide-deck/prompts/system.md [TARGET_DIR]/prompts/01-cover.md --image [TARGET_DIR]/01-cover.png --sessionId slides-topic-20260117
``` ```
Generation flow: Generation flow:
1. Generate images sequentially 1. Generate images sequentially with the same session ID
2. After each image, output progress: "Generated X/N" 2. After each image, output progress: "Generated X/N"
3. On failure, auto-retry once 3. On failure, auto-retry once
4. If retry fails, log reason, continue to next 4. If retry fails, log reason, continue to next
+7 -1
View File
@@ -261,8 +261,14 @@ Style notes: [style-specific characteristics]
1. Check available image generation skills 1. Check available image generation skills
2. If multiple skills available, ask user to choose 2. If multiple skills available, ask user to choose
**Session Management**:
If the image generation skill supports `--sessionId`:
1. Generate a unique session ID at the start (e.g., `xhs-{topic-slug}-{timestamp}`)
2. Use the same session ID for all images in the series
3. This ensures style consistency across all generated images
**Generation Flow**: **Generation Flow**:
1. Call selected image generation skill with prompt file and output path 1. Call selected image generation skill with prompt file, output path, and session ID
2. Confirm generation success 2. Confirm generation success
3. Report progress: "Generated X/N" 3. Report progress: "Generated X/N"
4. Continue to next 4. Continue to next