Compare commits

..

4 Commits

Author SHA1 Message Date
Jim Liu 宝玉 fa155da15d fix: update version to 0.5.3 and enhance placeholder selection logic in publishArticle function 2026-01-17 01:29:10 -06:00
Jim Liu 宝玉 6194f71378 bump version 2026-01-16 21:44:14 -06:00
Jim Liu 宝玉 8e88cf4a8b feat: implement session management for image generation skills and add session handling functions 2026-01-16 20:20:07 -06:00
Jim Liu 宝玉 259baff413 update gemeni-web to support add image as reference 2026-01-16 19:06:14 -06:00
13 changed files with 757 additions and 153 deletions
+1 -1
View File
@@ -6,7 +6,7 @@
},
"metadata": {
"description": "Skills shared by Baoyu for improving daily work efficiency",
"version": "0.5.1"
"version": "0.5.3"
},
"plugins": [
{
+6
View File
@@ -104,6 +104,12 @@ For each page (cover + pages):
- If text-only: concatenate prompts into single text
- If multiple skills available, ask user preference
**Session Management**:
If the image generation skill supports `--sessionId`:
1. Generate a unique session ID at the start (e.g., `comic-{topic-slug}-{timestamp}`)
2. Use the same session ID for character sheet and all pages
3. This ensures visual consistency (character appearance, style) across all generated images
3. Report progress after each generation
### Step 5: Completion Report
+58 -11
View File
@@ -5,6 +5,13 @@ description: Image generation skill using Gemini Web. Generates images from text
# Gemini Web Client
Supports:
- Text generation
- Image generation (download + save)
- Reference image upload (attach images for vision tasks)
- Multi-turn conversations within the same executor instance (`keepSession`)
- Experimental video generation (`generateVideo`) — Gemini may return an async placeholder; download might require Gemini web UI
## Quick start
```bash
@@ -12,8 +19,25 @@ npx -y bun scripts/main.ts "Hello, Gemini"
npx -y bun scripts/main.ts --prompt "Explain quantum computing"
npx -y bun scripts/main.ts --prompt "A cute cat" --image cat.png
npx -y bun scripts/main.ts --promptfiles system.md content.md --image out.png
# Multi-turn conversation (agent generates unique sessionId)
npx -y bun scripts/main.ts "Remember this: 42" --sessionId my-unique-id-123
npx -y bun scripts/main.ts "What number?" --sessionId my-unique-id-123
```
## Executor options (programmatic)
This skill is typically consumed via `createGeminiWebExecutor(geminiOptions)` (see `scripts/executor.ts`).
Key options in `GeminiWebOptions`:
- `referenceImages?: string | string[]` Upload local images as references (vision input).
- `keepSession?: boolean` Reuse Gemini `chatMetadata` to continue the same conversation across calls (required if you want reference images to persist across multiple messages).
- `generateVideo?: string` Generate a video and (best-effort) download to the given path. Gemini may return `video_gen_chip` (async); in that case you must open Gemini web UI to download the result.
Notes:
- `generateVideo` cannot be combined with `generateImage` / `editImage`.
- When `keepSession=true` and `referenceImages` is set, reference images are uploaded once per executor instance.
## Commands
### Text generation
@@ -58,17 +82,21 @@ npx -y bun scripts/main.ts "Hello" --json
## Options
| Option | Short | Description |
|--------|-------|-------------|
| `--prompt <text>` | `-p` | Prompt text |
| `--promptfiles <files...>` | | Read prompt from files (concatenated in order) |
| `--model <id>` | `-m` | Model: gemini-3-pro (default), gemini-2.5-pro, gemini-2.5-flash |
| `--image [path]` | | Generate image, save to path (default: generated.png) |
| `--json` | | Output as JSON |
| `--login` | | Refresh cookies only, then exit |
| `--cookie-path <path>` | | Custom cookie file path |
| `--profile-dir <path>` | | Chrome profile directory |
| `--help` | `-h` | Show help |
| Option | Description |
|--------|-------------|
| `--prompt <text>`, `-p` | Prompt text |
| `--promptfiles <files...>` | Read prompt from files (concatenated in order) |
| `--model <id>`, `-m` | Model: gemini-3-pro (default), gemini-2.5-pro, gemini-2.5-flash |
| `--image [path]` | Generate image, save to path (default: generated.png) |
| `--sessionId <id>` | Session ID for multi-turn conversation (agent generates unique ID) |
| `--list-sessions` | List saved sessions (max 100, sorted by update time) |
| `--json` | Output as JSON |
| `--login` | Refresh cookies only, then exit |
| `--cookie-path <path>` | Custom cookie file path |
| `--profile-dir <path>` | Chrome profile directory |
| `--help`, `-h` | Show help |
CLI note: `scripts/main.ts` supports text generation, image generation, and multi-turn conversations via `--sessionId`. Reference images and video generation are exposed via the executor API.
## Models
@@ -116,3 +144,22 @@ npx -y bun scripts/main.ts "Hello" --json | jq '.text'
# Concatenate system.md + content.md as prompt
npx -y bun scripts/main.ts --promptfiles system.md content.md --image output.png
```
### Multi-turn conversation
```bash
# Start a session with unique ID (agent generates this)
npx -y bun scripts/main.ts "You are a helpful math tutor." --sessionId task-abc123
# Continue the conversation (remembers context)
npx -y bun scripts/main.ts "What is 2+2?" --sessionId task-abc123
npx -y bun scripts/main.ts "Now multiply that by 10" --sessionId task-abc123
# List recent sessions (max 100, sorted by update time)
npx -y bun scripts/main.ts --list-sessions
```
Session files are stored in `~/Library/Application Support/baoyu-skills/gemini-web/sessions/<id>.json` and contain:
- `id`: Session ID
- `metadata`: Gemini chat metadata for continuation
- `messages`: Array of `{role, content, timestamp, error?}`
- `createdAt`, `updatedAt`: Timestamps
@@ -216,6 +216,7 @@ class CdpConnection {
export async function getGeminiCookieMapViaChrome(options?: {
timeoutMs?: number;
debugConnectTimeoutMs?: number;
tokenCheckTimeoutMs?: number;
pollIntervalMs?: number;
log?: GeminiWebLog;
userDataDir?: string;
@@ -224,6 +225,7 @@ export async function getGeminiCookieMapViaChrome(options?: {
const log = options?.log;
const timeoutMs = options?.timeoutMs ?? 5 * 60_000;
const debugConnectTimeoutMs = options?.debugConnectTimeoutMs ?? 30_000;
const tokenCheckTimeoutMs = options?.tokenCheckTimeoutMs ?? 30_000;
const pollIntervalMs = options?.pollIntervalMs ?? 2_000;
const userDataDir = options?.userDataDir ?? resolveGeminiWebChromeProfileDir();
@@ -290,7 +292,7 @@ export async function getGeminiCookieMapViaChrome(options?: {
if (hasRequiredGeminiCookies(cookieMap)) {
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 10_000);
const timer = setTimeout(() => controller.abort(), tokenCheckTimeoutMs);
try {
await fetchGeminiAccessToken(cookieMap, controller.signal);
} finally {
+135 -21
View File
@@ -67,19 +67,69 @@ function buildCookieHeader(cookieMap: Record<string, string>): string {
.join('; ');
}
function getSetCookieHeaders(res: Response): string[] {
const headers = res.headers as unknown as { getSetCookie?: () => string[] };
if (typeof headers.getSetCookie === 'function') {
try {
return headers.getSetCookie();
} catch {
return [];
}
}
const raw = res.headers.get('set-cookie');
return raw ? [raw] : [];
}
function applySetCookiesToMap(setCookies: string[], cookieMap: Record<string, string>): void {
for (const raw of setCookies) {
const first = raw.split(';')[0]?.trim();
if (!first) continue;
const idx = first.indexOf('=');
if (idx <= 0) continue;
const name = first.slice(0, idx).trim();
const value = first.slice(idx + 1).trim();
if (!name) continue;
cookieMap[name] = value;
}
}
async function fetchWithCookieJar(
url: string,
init: Omit<RequestInit, 'redirect' | 'headers'> & { headers?: Record<string, string> },
cookieMap: Record<string, string>,
signal?: AbortSignal,
maxRedirects = 20,
): Promise<Response> {
let current = url;
for (let i = 0; i <= maxRedirects; i += 1) {
const cookieHeader = buildCookieHeader(cookieMap);
const headers: Record<string, string> = {
...(init.headers ?? {}),
...(cookieHeader ? { cookie: cookieHeader } : {}),
'user-agent': USER_AGENT,
};
const res = await fetch(current, { ...init, redirect: 'manual', signal, headers });
applySetCookiesToMap(getSetCookieHeaders(res), cookieMap);
if (res.status >= 300 && res.status < 400) {
const location = res.headers.get('location');
if (!location) return res;
current = new URL(location, current).toString();
continue;
}
return res;
}
throw new Error(`Too many redirects while fetching ${url} (>${maxRedirects}).`);
}
export async function fetchGeminiAccessToken(
cookieMap: Record<string, string>,
signal?: AbortSignal,
): Promise<string> {
const cookieHeader = buildCookieHeader(cookieMap);
const res = await fetch(GEMINI_APP_URL, {
redirect: 'follow',
signal,
headers: {
cookie: cookieHeader,
'user-agent': USER_AGENT,
},
});
const res = await fetchWithCookieJar(GEMINI_APP_URL, { method: 'GET' }, cookieMap, signal);
const html = await res.text();
const tokens = ['SNlM0e', 'thykhd'] as const;
@@ -107,7 +157,21 @@ function extractErrorCode(responseJson: unknown): number | undefined {
}
function extractGgdlUrls(rawText: string): string[] {
const matches = rawText.match(/https:\/\/lh3\.googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? [];
const matches =
rawText.match(/https?:\/\/[^/\s"']*googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? [];
const seen = new Set<string>();
const urls: string[] = [];
for (const match of matches) {
if (seen.has(match)) continue;
seen.add(match);
urls.push(match);
}
return urls;
}
function extractImageGenerationContentUrls(rawText: string): string[] {
const matches =
rawText.match(/https?:\/\/googleusercontent\.com\/image_generation_content\/\d+/g) ?? [];
const seen = new Set<string>();
const urls: string[] = [];
for (const match of matches) {
@@ -119,9 +183,17 @@ function extractGgdlUrls(rawText: string): string[] {
}
function ensureFullSizeImageUrl(url: string): string {
if (url.includes('=s2048')) return url;
if (url.includes('=s')) return url;
return `${url}=s2048`;
const trimmed = url.trim();
let normalized = trimmed;
const backslashIndex = normalized.indexOf('\\');
if (backslashIndex >= 0) normalized = normalized.slice(0, backslashIndex);
// Some Gemini responses embed a size suffix as "/=s2048" which breaks downloads.
normalized = normalized.replace(/\/=s(?=\d+(?:$|[?#]))/, '=s');
normalized = normalized.replace(/\/=s(?=$|[?#])/, '=s');
if (normalized.endsWith('/')) normalized = normalized.slice(0, -1);
if (normalized.includes('=s2048')) return normalized;
if (normalized.includes('=s')) return normalized;
return `${normalized}=s2048`;
}
async function fetchWithCookiePreservingRedirects(
@@ -190,6 +262,29 @@ async function uploadGeminiFile(filePath: string, signal?: AbortSignal): Promise
return { id: text, name: fileName };
}
function guessMimeType(fileName: string): string {
const ext = path.extname(fileName).toLowerCase();
switch (ext) {
case '.png':
return 'image/png';
case '.jpg':
case '.jpeg':
return 'image/jpeg';
case '.webp':
return 'image/webp';
case '.gif':
return 'image/gif';
case '.mp4':
return 'video/mp4';
case '.mov':
return 'video/quicktime';
case '.webm':
return 'video/webm';
default:
return 'application/octet-stream';
}
}
function buildGeminiFReqPayload(
prompt: string,
uploaded: Array<{ id: string; name: string }>,
@@ -201,9 +296,8 @@ function buildGeminiFReqPayload(
prompt,
0,
null,
// Matches gemini-webapi payload format: [[[fileId, 1]]] for a single attachment.
// Keep it extensible for multiple uploads by emitting one [[id, 1]] entry per file.
uploaded.map((file) => [[file.id, 1]]),
// Matches gemini-web payload format: [[[fileId, 1, null, mimeType], fileName]] for an attachment.
uploaded.map((file) => [[file.id, 1, null, guessMimeType(file.name)], file.name]),
]
: [prompt];
@@ -248,7 +342,19 @@ export function parseGeminiStreamGenerateResponse(rawText: string): {
? (getNestedValue<string | null>(firstCandidate, [22, 0], null) ?? textRaw)
: textRaw;
const thoughts = getNestedValue<string | null>(firstCandidate, [37, 0, 0], null);
const metadata = getNestedValue<unknown>(body, [1], []);
const conversationMeta = getNestedValue<unknown[]>(body, [1], []);
const conversationId =
typeof conversationMeta[0] === 'string' && conversationMeta[0].length > 0
? conversationMeta[0]
: null;
const responseId =
typeof conversationMeta[1] === 'string' && conversationMeta[1].length > 0
? conversationMeta[1]
: null;
const choiceIdRaw = getNestedValue<string | null>(firstCandidate, [0], null);
const choiceId = typeof choiceIdRaw === 'string' && choiceIdRaw.length > 0 ? choiceIdRaw : null;
const metadata =
conversationId && responseId && choiceId ? [conversationId, responseId, choiceId] : conversationMeta;
const images: GeminiWebCandidateImage[] = [];
@@ -305,8 +411,8 @@ export function isGeminiModelUnavailable(errorCode: number | undefined): boolean
}
export async function runGeminiWebOnce(input: GeminiWebRunInput): Promise<GeminiWebRunOutput> {
const cookieHeader = buildCookieHeader(input.cookieMap);
const at = await fetchGeminiAccessToken(input.cookieMap, input.signal);
const cookieHeader = buildCookieHeader(input.cookieMap);
const uploaded: Array<{ id: string; name: string }> = [];
for (const file of input.files ?? []) {
@@ -403,11 +509,19 @@ export async function saveFirstGeminiImageFromOutput(
return { saved: true, imageCount: output.images.length };
}
const ggdl = extractGgdlUrls(output.rawResponseText);
if (ggdl[0]) {
await downloadGeminiImage(ggdl[0], cookieMap, outputPath, signal);
const ggdl = extractGgdlUrls(`${output.text}\n${output.rawResponseText}`);
const preferred = ggdl.length > 0 ? ggdl[ggdl.length - 1] : null;
if (preferred) {
await downloadGeminiImage(preferred, cookieMap, outputPath, signal);
return { saved: true, imageCount: ggdl.length };
}
const imageGen = extractImageGenerationContentUrls(`${output.text}\n${output.rawResponseText}`);
const imageGenPreferred = imageGen.length > 0 ? imageGen[imageGen.length - 1] : null;
if (imageGenPreferred) {
await downloadGeminiImage(imageGenPreferred, cookieMap, outputPath, signal);
return { saved: true, imageCount: imageGen.length };
}
return { saved: false, imageCount: 0 };
}
+193 -10
View File
@@ -1,7 +1,8 @@
import { mkdir, writeFile } from 'node:fs/promises';
import path from 'node:path';
import type { BrowserRunOptions, BrowserRunResult, BrowserLogger, CookieParam } from '../browser/types.js';
import { runGeminiWebWithFallback, saveFirstGeminiImageFromOutput } from './client.js';
import type { GeminiWebModelId } from './client.js';
import type { GeminiWebModelId, GeminiWebRunOutput } from './client.js';
import {
buildGeminiCookieMap,
hasRequiredGeminiCookies,
@@ -11,6 +12,9 @@ import type { GeminiWebOptions, GeminiWebResponse } from './types.js';
export { hasRequiredGeminiCookies } from './cookie-store.js';
const USER_AGENT =
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36';
function estimateTokenCount(text: string): number {
return Math.ceil(text.length / 4);
}
@@ -22,6 +26,115 @@ function resolveInvocationPath(value: string | undefined): string | undefined {
return path.isAbsolute(trimmed) ? trimmed : path.resolve(process.cwd(), trimmed);
}
function normalizePathList(value: string | string[] | undefined): string[] {
if (!value) return [];
const raw = Array.isArray(value) ? value : [value];
const out: string[] = [];
for (const entry of raw) {
if (typeof entry !== 'string') continue;
const resolved = resolveInvocationPath(entry);
if (!resolved) continue;
out.push(resolved);
}
return out;
}
function dedupePaths(paths: string[]): string[] {
const seen = new Set<string>();
const out: string[] = [];
for (const item of paths) {
const trimmed = item.trim();
if (!trimmed || seen.has(trimmed)) continue;
seen.add(trimmed);
out.push(trimmed);
}
return out;
}
function buildCookieHeader(cookieMap: Record<string, string>): string {
return Object.entries(cookieMap)
.filter(([, value]) => typeof value === 'string' && value.length > 0)
.map(([name, value]) => `${name}=${value}`)
.join('; ');
}
async function fetchWithCookiePreservingRedirects(
url: string,
init: Omit<RequestInit, 'redirect'>,
signal?: AbortSignal,
maxRedirects = 10,
): Promise<Response> {
let current = url;
for (let i = 0; i <= maxRedirects; i += 1) {
const res = await fetch(current, { ...init, redirect: 'manual', signal });
if (res.status >= 300 && res.status < 400) {
const location = res.headers.get('location');
if (!location) return res;
current = new URL(location, current).toString();
continue;
}
return res;
}
throw new Error(`Too many redirects while downloading media (>${maxRedirects}).`);
}
async function downloadGeminiMedia(
url: string,
cookieMap: Record<string, string>,
outputPath: string,
signal?: AbortSignal,
): Promise<void> {
const cookieHeader = buildCookieHeader(cookieMap);
const res = await fetchWithCookiePreservingRedirects(
url,
{
headers: {
cookie: cookieHeader,
'user-agent': USER_AGENT,
},
},
signal,
);
if (!res.ok) {
throw new Error(`Failed to download media: ${res.status} ${res.statusText} (${res.url})`);
}
const data = new Uint8Array(await res.arrayBuffer());
await mkdir(path.dirname(outputPath), { recursive: true });
await writeFile(outputPath, data);
}
function extractGgdlUrls(rawText: string): string[] {
const matches =
rawText.match(/https?:\/\/[^/\s"']*googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? [];
const seen = new Set<string>();
const urls: string[] = [];
for (const match of matches) {
if (seen.has(match)) continue;
seen.add(match);
urls.push(match);
}
return urls;
}
async function saveFirstGeminiVideoFromOutput(
output: GeminiWebRunOutput,
cookieMap: Record<string, string>,
outputPath: string,
signal?: AbortSignal,
): Promise<{ saved: boolean; videoCount: number }> {
const ggdl = extractGgdlUrls(output.rawResponseText);
if (!ggdl[0]) return { saved: false, videoCount: 0 };
const videoCandidates = ggdl.filter((url) => /\.(mp4|webm|mov)(?:$|[?#])/i.test(url));
const preferred =
(videoCandidates.length > 0 ? videoCandidates[videoCandidates.length - 1] : null) ??
ggdl.find((url) => /video/i.test(url)) ??
ggdl[ggdl.length - 1];
await downloadGeminiMedia(preferred, cookieMap, outputPath, signal);
return { saved: true, videoCount: ggdl.length };
}
function resolveGeminiWebModel(
desiredModel: string | null | undefined,
log?: BrowserLogger,
@@ -115,6 +228,9 @@ export async function loadGeminiCookieMap(log?: BrowserLogger): Promise<Record<s
export function createGeminiWebExecutor(
geminiOptions: GeminiWebOptions,
): (runOptions: BrowserRunOptions) => Promise<BrowserRunResult> {
let persistedChatMetadata: unknown | null = null;
let referenceImagesUploaded = false;
return async (runOptions: BrowserRunOptions): Promise<BrowserRunResult> => {
const startTime = Date.now();
const log = runOptions.log;
@@ -133,23 +249,38 @@ export function createGeminiWebExecutor(
? Math.max(1_000, runOptions.config.timeoutMs)
: null;
const generateVideoPath = resolveInvocationPath(geminiOptions.generateVideo);
const defaultTimeoutMs = geminiOptions.youtube
? 240_000
: geminiOptions.generateImage || geminiOptions.editImage
: generateVideoPath
? 900_000
: geminiOptions.generateImage || geminiOptions.editImage
? 300_000
: 120_000;
const timeoutMs = Math.min(configTimeout ?? defaultTimeoutMs, 600_000);
const timeoutCapMs = generateVideoPath ? 1_800_000 : 600_000;
const timeoutMs = Math.min(configTimeout ?? defaultTimeoutMs, timeoutCapMs);
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);
const keepSession = geminiOptions.keepSession === true;
const generateImagePath = resolveInvocationPath(geminiOptions.generateImage);
const editImagePath = resolveInvocationPath(geminiOptions.editImage);
const outputPath = resolveInvocationPath(geminiOptions.outputPath);
const attachmentPaths = (runOptions.attachments ?? []).map((attachment) => attachment.path);
const referenceImagePaths = normalizePathList(geminiOptions.referenceImages);
const requestFilePaths = dedupePaths(
keepSession ? attachmentPaths : [...referenceImagePaths, ...attachmentPaths],
);
if (generateVideoPath && (generateImagePath || editImagePath)) {
throw new Error('Gemini web executor: generateVideo cannot be combined with generateImage/editImage options.');
}
let prompt = runOptions.prompt;
if (geminiOptions.aspectRatio && (generateImagePath || editImagePath)) {
if (geminiOptions.aspectRatio && (generateImagePath || editImagePath || generateVideoPath)) {
prompt = `${prompt} (aspect ratio: ${geminiOptions.aspectRatio})`;
}
if (geminiOptions.youtube) {
@@ -158,29 +289,50 @@ export function createGeminiWebExecutor(
if (generateImagePath && !editImagePath) {
prompt = `Generate an image: ${prompt}`;
}
if (generateVideoPath) {
prompt = `Generate a video: ${prompt}`;
}
const model: GeminiWebModelId = resolveGeminiWebModel(runOptions.config?.desiredModel, log);
let response: GeminiWebResponse;
let videoSaveSummary: { saved: boolean; videoCount: number; outputPath: string } | null = null;
try {
let chatMetadata: unknown = keepSession ? persistedChatMetadata : null;
if (keepSession && referenceImagePaths.length > 0 && !referenceImagesUploaded) {
const intro = await runGeminiWebWithFallback({
prompt: 'Here are reference images for future messages.',
files: referenceImagePaths,
model,
cookieMap,
chatMetadata,
signal: controller.signal,
});
chatMetadata = intro.metadata;
persistedChatMetadata = intro.metadata;
referenceImagesUploaded = true;
}
if (editImagePath) {
const intro = await runGeminiWebWithFallback({
prompt: 'Here is an image to edit',
files: [editImagePath],
model,
cookieMap,
chatMetadata: null,
chatMetadata,
signal: controller.signal,
});
const editPrompt = `Use image generation tool to ${prompt}`;
const out = await runGeminiWebWithFallback({
prompt: editPrompt,
files: attachmentPaths,
files: requestFilePaths,
model,
cookieMap,
chatMetadata: intro.metadata,
signal: controller.signal,
});
if (keepSession) persistedChatMetadata = out.metadata;
response = {
text: out.text ?? null,
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
@@ -198,12 +350,13 @@ export function createGeminiWebExecutor(
} else if (generateImagePath) {
const out = await runGeminiWebWithFallback({
prompt,
files: attachmentPaths,
files: requestFilePaths,
model,
cookieMap,
chatMetadata: null,
chatMetadata,
signal: controller.signal,
});
if (keepSession) persistedChatMetadata = out.metadata;
response = {
text: out.text ?? null,
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
@@ -216,15 +369,36 @@ export function createGeminiWebExecutor(
if (!imageSave.saved) {
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
}
} else if (generateVideoPath) {
const out = await runGeminiWebWithFallback({
prompt,
files: requestFilePaths,
model,
cookieMap,
chatMetadata,
signal: controller.signal,
});
if (keepSession) persistedChatMetadata = out.metadata;
response = {
text: out.text ?? null,
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
has_images: false,
image_count: 0,
};
const resolvedOutputPath = generateVideoPath ?? outputPath ?? 'generated.mp4';
const save = await saveFirstGeminiVideoFromOutput(out, cookieMap, resolvedOutputPath, controller.signal);
videoSaveSummary = { ...save, outputPath: resolvedOutputPath };
} else {
const out = await runGeminiWebWithFallback({
prompt,
files: attachmentPaths,
files: requestFilePaths,
model,
cookieMap,
chatMetadata: null,
chatMetadata,
signal: controller.signal,
});
if (keepSession) persistedChatMetadata = out.metadata;
response = {
text: out.text ?? null,
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
@@ -247,6 +421,15 @@ export function createGeminiWebExecutor(
const imagePath = generateImagePath || outputPath || 'generated.png';
answerMarkdown += `\n\n*Generated ${response.image_count} image(s). Saved to: ${imagePath}*`;
}
if (videoSaveSummary) {
if (videoSaveSummary.saved) {
answerMarkdown += `\n\n*Generated ${videoSaveSummary.videoCount || 1} video(s). Saved to: ${videoSaveSummary.outputPath}*`;
} else if (/video_gen_chip/.test(answerMarkdown) || /video_gen_chip/.test(response.text ?? '')) {
answerMarkdown += '\n\n*Video generation is asynchronous. Check Gemini web UI to download the result.*';
} else {
answerMarkdown += '\n\n*No downloadable video URL found in Gemini response.*';
}
}
const tookMs = Date.now() - startTime;
log?.(`[gemini-web] Completed in ${tookMs}ms`);
+139 -65
View File
@@ -10,6 +10,7 @@ import {
writeGeminiCookieMapToDisk,
} from './cookie-store.js';
import { resolveGeminiWebChromeProfileDir, resolveGeminiWebCookiePath } from './paths.js';
import { readSession, writeSession, listSessions } from './session-store.js';
function printUsage(exitCode = 0): never {
const cookiePath = resolveGeminiWebCookiePath();
@@ -21,12 +22,19 @@ function printUsage(exitCode = 0): never {
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --prompt "A cute cat" --image generated.png
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
Multi-turn conversation (agent generates unique sessionId):
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "Remember 42" --sessionId abc123
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "What number?" --sessionId abc123
Options:
-p, --prompt <text> Prompt text
--promptfiles <files...> Read prompt from one or more files (concatenated in order)
-m, --model <id> gemini-3-pro | gemini-2.5-pro | gemini-2.5-flash (default: gemini-3-pro)
--json Output JSON
--image [path] Generate an image and save it (default: ./generated.png)
--reference <files...> Reference images for vision input
--sessionId <id> Session ID for multi-turn conversation (agent should generate unique ID)
--list-sessions List saved sessions (max 100, sorted by update time)
--login Only refresh cookies, then exit
--cookie-path <path> Cookie file path (default: ${cookiePath})
--profile-dir <path> Chrome profile dir (default: ${profileDir})
@@ -75,6 +83,9 @@ function parseArgs(argv: string[]): {
loginOnly?: boolean;
cookiePath?: string;
profileDir?: string;
referenceImages?: string[];
sessionId?: string;
listSessions?: boolean;
} {
const out: ReturnType<typeof parseArgs> = {};
const positional: string[] = [];
@@ -157,6 +168,32 @@ function parseArgs(argv: string[]): {
out.profileDir = arg.slice('--profile-dir='.length);
continue;
}
if (arg === '--reference' || arg === '--ref') {
out.referenceImages = [];
while (i + 1 < argv.length) {
const next = argv[i + 1];
if (next && !next.startsWith('-')) {
out.referenceImages.push(next);
i += 1;
} else {
break;
}
}
continue;
}
if (arg === '--sessionId' || arg === '--session-id') {
out.sessionId = argv[i + 1] ?? '';
i += 1;
continue;
}
if (arg.startsWith('--sessionId=') || arg.startsWith('--session-id=')) {
out.sessionId = arg.split('=')[1] ?? '';
continue;
}
if (arg === '--list-sessions') {
out.listSessions = true;
continue;
}
if (arg.startsWith('-')) {
throw new Error(`Unknown option: ${arg}`);
@@ -178,6 +215,9 @@ function parseArgs(argv: string[]): {
if (out.cookiePath === '') delete out.cookiePath;
if (out.profileDir === '') delete out.profileDir;
if (out.promptFiles?.length === 0) delete out.promptFiles;
if (out.referenceImages?.length === 0) delete out.referenceImages;
if (out.sessionId != null) out.sessionId = out.sessionId.trim();
if (out.sessionId === '') delete out.sessionId;
return out;
}
@@ -186,7 +226,7 @@ async function isCookieMapValid(cookieMap: Record<string, string>): Promise<bool
if (!hasRequiredGeminiCookies(cookieMap)) return false;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 10_000);
const timer = setTimeout(() => controller.abort(), 30_000);
try {
await fetchGeminiAccessToken(cookieMap, controller.signal);
return true;
@@ -201,7 +241,7 @@ async function ensureGeminiCookieMap(options: {
cookiePath: string;
profileDir: string;
}): Promise<Record<string, string>> {
const log = (msg: string) => console.log(msg);
const log = (msg: string) => console.error(msg);
let cookieMap = await readGeminiCookieMapFromDisk({ cookiePath: options.cookiePath, log });
if (await isCookieMapValid(cookieMap)) return cookieMap;
@@ -251,85 +291,63 @@ async function main(): Promise<void> {
const cookiePath = args.cookiePath ?? resolveGeminiWebCookiePath();
const profileDir = args.profileDir ?? resolveGeminiWebChromeProfileDir();
if (args.listSessions) {
const sessions = await listSessions();
if (sessions.length === 0) {
console.log('No saved sessions.');
} else {
for (const { id, updatedAt } of sessions) {
console.log(`${id}\t${updatedAt}`);
}
}
return;
}
if (args.loginOnly) {
await ensureGeminiCookieMap({ cookiePath, profileDir });
return;
}
const promptFromStdin = await readPromptFromStdin();
const promptFromFiles = args.promptFiles ? readPromptFiles(args.promptFiles) : null;
const prompt = promptFromFiles || args.prompt || promptFromStdin;
const promptFromArgs = promptFromFiles || args.prompt;
const prompt = promptFromArgs || (await readPromptFromStdin());
if (!prompt) printUsage(1);
const sessionData = args.sessionId ? await readSession(args.sessionId) : null;
const chatMetadata = sessionData?.metadata ?? null;
let cookieMap = await ensureGeminiCookieMap({ cookiePath, profileDir });
const desiredModel = resolveModel(args.model || 'gemini-3-pro');
const imagePath = resolveImageOutputPath(args.imagePath);
const referenceImages = (args.referenceImages ?? []).map((p) =>
path.isAbsolute(p) ? p : path.resolve(process.cwd(), p),
);
try {
const effectivePrompt = imagePath ? `Generate an image: ${prompt}` : prompt;
const out = await runGeminiWebWithFallback({
prompt: effectivePrompt,
files: [],
model: desiredModel,
cookieMap,
chatMetadata: null,
});
let imageSaved = false;
let imageCount = 0;
if (imagePath) {
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath);
imageSaved = save.saved;
imageCount = save.imageCount;
if (!imageSaved) {
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
}
}
if (args.json) {
process.stdout.write(
`${JSON.stringify(
imagePath ? { ...out, imageSaved, imageCount, imagePath } : out,
null,
2,
)}\n`,
);
if (out.errorMessage) process.exit(1);
return;
}
if (out.errorMessage) {
throw new Error(out.errorMessage);
}
process.stdout.write(out.text ?? '');
if (!out.text?.endsWith('\n')) process.stdout.write('\n');
if (imagePath) {
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
}
return;
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (message.includes('Unable to locate Gemini access token')) {
console.error('[gemini-web] Cookies may be expired. Re-opening browser to refresh cookies...');
await sleep(500);
cookieMap = await getGeminiCookieMapViaChrome({ userDataDir: profileDir, log: (m) => console.log(m) });
await writeGeminiCookieMapToDisk(cookieMap, { cookiePath, log: (m) => console.log(m) });
const controller = new AbortController();
const timeoutMs = imagePath ? 300_000 : 120_000;
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const effectivePrompt = imagePath ? `Generate an image: ${prompt}` : prompt;
const out = await runGeminiWebWithFallback({
prompt: imagePath ? `Generate an image: ${prompt}` : prompt,
files: [],
prompt: effectivePrompt,
files: referenceImages,
model: desiredModel,
cookieMap,
chatMetadata: null,
chatMetadata,
signal: controller.signal,
});
if (args.sessionId && out.metadata) {
await writeSession(args.sessionId, out.metadata, prompt, out.text ?? '', out.errorMessage);
}
let imageSaved = false;
let imageCount = 0;
if (imagePath) {
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath);
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath, controller.signal);
imageSaved = save.saved;
imageCount = save.imageCount;
if (!imageSaved) {
@@ -338,13 +356,8 @@ async function main(): Promise<void> {
}
if (args.json) {
process.stdout.write(
`${JSON.stringify(
imagePath ? { ...out, imageSaved, imageCount, imagePath } : out,
null,
2,
)}\n`,
);
const jsonOut = { ...out, ...(imagePath && { imageSaved, imageCount, imagePath }), ...(args.sessionId && { sessionId: args.sessionId }) };
process.stdout.write(`${JSON.stringify(jsonOut, null, 2)}\n`);
if (out.errorMessage) process.exit(1);
return;
}
@@ -359,6 +372,67 @@ async function main(): Promise<void> {
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
}
return;
} finally {
clearTimeout(timeout);
}
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (message.includes('Unable to locate Gemini access token')) {
console.error('[gemini-web] Cookies may be expired. Re-opening browser to refresh cookies...');
await sleep(500);
cookieMap = await getGeminiCookieMapViaChrome({ userDataDir: profileDir, log: (m) => console.error(m) });
await writeGeminiCookieMapToDisk(cookieMap, { cookiePath, log: (m) => console.error(m) });
const controller = new AbortController();
const timeoutMs = imagePath ? 300_000 : 120_000;
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const out = await runGeminiWebWithFallback({
prompt: imagePath ? `Generate an image: ${prompt}` : prompt,
files: referenceImages,
model: desiredModel,
cookieMap,
chatMetadata,
signal: controller.signal,
});
if (args.sessionId && out.metadata) {
await writeSession(args.sessionId, out.metadata, prompt, out.text ?? '', out.errorMessage);
}
let imageSaved = false;
let imageCount = 0;
if (imagePath) {
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath, controller.signal);
imageSaved = save.saved;
imageCount = save.imageCount;
if (!imageSaved) {
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
}
}
if (args.json) {
const jsonOut = { ...out, ...(imagePath && { imageSaved, imageCount, imagePath }), ...(args.sessionId && { sessionId: args.sessionId }) };
process.stdout.write(`${JSON.stringify(jsonOut, null, 2)}\n`);
if (out.errorMessage) process.exit(1);
return;
}
if (out.errorMessage) {
throw new Error(out.errorMessage);
}
process.stdout.write(out.text ?? '');
if (!out.text?.endsWith('\n')) process.stdout.write('\n');
if (imagePath) {
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
}
return;
} finally {
clearTimeout(timeout);
}
}
throw error;
+9
View File
@@ -34,3 +34,12 @@ export function resolveGeminiWebChromeProfileDir(): string {
if (override) return path.resolve(override);
return path.join(resolveGeminiWebDataDir(), PROFILE_DIR_NAME);
}
export function resolveGeminiWebSessionsDir(): string {
return path.join(resolveGeminiWebDataDir(), 'sessions');
}
export function resolveGeminiWebSessionPath(name: string): string {
const sanitized = name.replace(/[^a-zA-Z0-9_-]/g, '_');
return path.join(resolveGeminiWebSessionsDir(), `${sanitized}.json`);
}
@@ -0,0 +1,90 @@
import { mkdir, readFile, writeFile, readdir, stat } from 'node:fs/promises';
import path from 'node:path';
import { resolveGeminiWebSessionsDir, resolveGeminiWebSessionPath } from './paths.js';
export interface SessionMessage {
role: 'user' | 'assistant';
content: string;
timestamp: string;
error?: string;
}
export interface SessionData {
id: string;
metadata: unknown;
messages: SessionMessage[];
createdAt: string;
updatedAt: string;
}
export interface SessionListItem {
id: string;
updatedAt: string;
}
export async function readSession(id: string): Promise<SessionData | null> {
const sessionPath = resolveGeminiWebSessionPath(id);
try {
const content = await readFile(sessionPath, 'utf8');
return JSON.parse(content) as SessionData;
} catch {
return null;
}
}
export async function writeSession(
id: string,
metadata: unknown,
userMessage: string,
assistantMessage: string,
error?: string,
): Promise<void> {
const sessionPath = resolveGeminiWebSessionPath(id);
const sessionsDir = resolveGeminiWebSessionsDir();
await mkdir(sessionsDir, { recursive: true });
const existing = await readSession(id);
const now = new Date().toISOString();
const newMessages: SessionMessage[] = [
{ role: 'user', content: userMessage, timestamp: now },
{ role: 'assistant', content: assistantMessage, timestamp: now, ...(error && { error }) },
];
const data: SessionData = {
id,
metadata,
messages: [...(existing?.messages ?? []), ...newMessages],
createdAt: existing?.createdAt ?? now,
updatedAt: now,
};
await writeFile(sessionPath, JSON.stringify(data, null, 2));
}
export async function listSessions(limit = 100): Promise<SessionListItem[]> {
const sessionsDir = resolveGeminiWebSessionsDir();
try {
const files = await readdir(sessionsDir);
const jsonFiles = files.filter((f) => f.endsWith('.json'));
const items: { id: string; updatedAt: string; mtime: number }[] = [];
for (const file of jsonFiles) {
const filePath = path.join(sessionsDir, file);
try {
const stats = await stat(filePath);
items.push({
id: file.slice(0, -5),
updatedAt: stats.mtime.toISOString(),
mtime: stats.mtime.getTime(),
});
} catch {
continue;
}
}
items.sort((a, b) => b.mtime - a.mtime);
return items.slice(0, limit).map(({ id, updatedAt }) => ({ id, updatedAt }));
} catch {
return [];
}
}
+9
View File
@@ -2,9 +2,18 @@ export interface GeminiWebOptions {
youtube?: string;
generateImage?: string;
editImage?: string;
generateVideo?: string;
outputPath?: string;
showThoughts?: boolean;
aspectRatio?: string;
/**
* One or more local image paths to upload as persistent reference images.
* - If `keepSession` is enabled, they are uploaded once per executor session.
* - Otherwise, they are attached to each request.
*/
referenceImages?: string | string[];
/** Preserve Gemini chat metadata to continue multi-turn conversations within the same executor instance. */
keepSession?: boolean;
}
export interface GeminiWebResponse {
+98 -41
View File
@@ -481,52 +481,78 @@ export async function publishArticle(options: ArticleOptions): Promise<void> {
const img = sortedImages[i]!;
console.log(`[x-article] [${i + 1}/${sortedImages.length}] Inserting image at placeholder: ${img.placeholder}`);
// Find, scroll to, and select the placeholder text in DraftEditor
const placeholderFound = await cdp.send<{ result: { value: boolean } }>('Runtime.evaluate', {
expression: `(() => {
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]');
if (!editor) return false;
// Helper to select placeholder with retry
const selectPlaceholder = async (maxRetries = 3): Promise<boolean> => {
for (let attempt = 1; attempt <= maxRetries; attempt++) {
// Find, scroll to, and select the placeholder text in DraftEditor
await cdp!.send('Runtime.evaluate', {
expression: `(() => {
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]');
if (!editor) return false;
const placeholder = ${JSON.stringify(img.placeholder)};
const placeholder = ${JSON.stringify(img.placeholder)};
// Search through all text nodes in the editor
const walker = document.createTreeWalker(editor, NodeFilter.SHOW_TEXT, null, false);
let node;
// Search through all text nodes in the editor
const walker = document.createTreeWalker(editor, NodeFilter.SHOW_TEXT, null, false);
let node;
while ((node = walker.nextNode())) {
const text = node.textContent || '';
const idx = text.indexOf(placeholder);
if (idx !== -1) {
// Found the placeholder - scroll to it first
const parentElement = node.parentElement;
if (parentElement) {
parentElement.scrollIntoView({ behavior: 'smooth', block: 'center' });
while ((node = walker.nextNode())) {
const text = node.textContent || '';
const idx = text.indexOf(placeholder);
if (idx !== -1) {
// Found the placeholder - scroll to it first
const parentElement = node.parentElement;
if (parentElement) {
parentElement.scrollIntoView({ behavior: 'smooth', block: 'center' });
}
// Select it
const range = document.createRange();
range.setStart(node, idx);
range.setEnd(node, idx + placeholder.length);
const sel = window.getSelection();
sel.removeAllRanges();
sel.addRange(range);
return true;
}
}
return false;
})()`,
}, { sessionId });
// Select it
const range = document.createRange();
range.setStart(node, idx);
range.setEnd(node, idx + placeholder.length);
const sel = window.getSelection();
sel.removeAllRanges();
sel.addRange(range);
return true;
}
// Wait for scroll and selection to settle
await sleep(800);
// Verify selection matches the placeholder
const selectionCheck = await cdp!.send<{ result: { value: string } }>('Runtime.evaluate', {
expression: `window.getSelection()?.toString() || ''`,
returnByValue: true,
}, { sessionId });
const selectedText = selectionCheck.result.value.trim();
if (selectedText === img.placeholder) {
console.log(`[x-article] Selection verified: "${selectedText}"`);
return true;
}
return false;
})()`,
returnByValue: true,
}, { sessionId });
// Wait for scroll animation
await sleep(500);
if (attempt < maxRetries) {
console.log(`[x-article] Selection attempt ${attempt} got "${selectedText}", retrying...`);
await sleep(500);
} else {
console.warn(`[x-article] Selection failed after ${maxRetries} attempts, got: "${selectedText}"`);
}
}
return false;
};
if (!placeholderFound.result.value) {
console.warn(`[x-article] Placeholder not found in DOM: ${img.placeholder}`);
// Try to select the placeholder
const selected = await selectPlaceholder(3);
if (!selected) {
console.warn(`[x-article] Skipping image - could not select placeholder: ${img.placeholder}`);
continue;
}
console.log(`[x-article] Placeholder selected, copying image: ${path.basename(img.localPath)}`);
console.log(`[x-article] Copying image: ${path.basename(img.localPath)}`);
// Copy image to clipboard
if (!copyImageToClipboard(img.localPath)) {
@@ -535,17 +561,48 @@ export async function publishArticle(options: ArticleOptions): Promise<void> {
}
// Wait for clipboard to be fully ready
await sleep(800);
await sleep(1000);
// Delete placeholder by pressing Enter (placeholder is already selected)
console.log(`[x-article] Deleting placeholder with Enter...`);
await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13 }, { sessionId });
await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13 }, { sessionId });
// Delete placeholder by pressing Backspace (more reliable than Enter for replacing selection)
console.log(`[x-article] Deleting placeholder...`);
await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
// Wait and verify placeholder is deleted
await sleep(500);
// Check that placeholder is no longer in editor
const afterDelete = await cdp.send<{ result: { value: boolean } }>('Runtime.evaluate', {
expression: `(() => {
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]');
if (!editor) return true;
return !editor.innerText.includes(${JSON.stringify(img.placeholder)});
})()`,
returnByValue: true,
}, { sessionId });
if (!afterDelete.result.value) {
console.warn(`[x-article] Placeholder may not have been deleted, trying again...`);
// Try selecting and deleting again
await selectPlaceholder(1);
await sleep(300);
await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
await sleep(500);
}
// Focus editor to ensure cursor is in position
await cdp.send('Runtime.evaluate', {
expression: `(() => {
const editor = document.querySelector('.DraftEditor-editorContainer [contenteditable="true"]');
if (editor) editor.focus();
})()`,
}, { sessionId });
await sleep(300);
// Paste image using paste script (activates Chrome, sends real keystroke)
console.log(`[x-article] Pasting image...`);
if (pasteFromClipboard('Google Chrome', 5, 800)) {
if (pasteFromClipboard('Google Chrome', 5, 1000)) {
console.log(`[x-article] Image pasted: ${path.basename(img.localPath)}`);
} else {
console.warn(`[x-article] Failed to paste image after retries`);
+9 -2
View File
@@ -232,14 +232,21 @@ Style notes: [specific style characteristics to emphasize]
### Step 5: Generate Images
**Session Management**:
If the image generation skill supports `--sessionId`:
1. Generate a unique session ID at the start (e.g., `slides-{topic-slug}-{timestamp}`)
2. Use the same session ID for all slides
3. This ensures visual consistency (color scheme, style, typography) across all slides
For each slide, generate using:
```bash
/baoyu-gemini-web --promptfiles [SKILL_ROOT]/skills/baoyu-slide-deck/prompts/system.md [TARGET_DIR]/prompts/01-cover.md --image [TARGET_DIR]/01-cover.png
# With session support
/baoyu-gemini-web --promptfiles [SKILL_ROOT]/skills/baoyu-slide-deck/prompts/system.md [TARGET_DIR]/prompts/01-cover.md --image [TARGET_DIR]/01-cover.png --sessionId slides-topic-20260117
```
Generation flow:
1. Generate images sequentially
1. Generate images sequentially with the same session ID
2. After each image, output progress: "Generated X/N"
3. On failure, auto-retry once
4. If retry fails, log reason, continue to next
+7 -1
View File
@@ -261,8 +261,14 @@ Style notes: [style-specific characteristics]
1. Check available image generation skills
2. If multiple skills available, ask user to choose
**Session Management**:
If the image generation skill supports `--sessionId`:
1. Generate a unique session ID at the start (e.g., `xhs-{topic-slug}-{timestamp}`)
2. Use the same session ID for all images in the series
3. This ensures style consistency across all generated images
**Generation Flow**:
1. Call selected image generation skill with prompt file and output path
1. Call selected image generation skill with prompt file, output path, and session ID
2. Confirm generation success
3. Report progress: "Generated X/N"
4. Continue to next