mirror of
https://github.com/JimLiu/baoyu-skills.git
synced 2026-07-12 05:51:44 +08:00
Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| fa155da15d | |||
| 6194f71378 | |||
| 8e88cf4a8b | |||
| 259baff413 |
@@ -6,7 +6,7 @@
|
||||
},
|
||||
"metadata": {
|
||||
"description": "Skills shared by Baoyu for improving daily work efficiency",
|
||||
"version": "0.5.1"
|
||||
"version": "0.5.3"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
|
||||
@@ -104,6 +104,12 @@ For each page (cover + pages):
|
||||
- If text-only: concatenate prompts into single text
|
||||
- If multiple skills available, ask user preference
|
||||
|
||||
**Session Management**:
|
||||
If the image generation skill supports `--sessionId`:
|
||||
1. Generate a unique session ID at the start (e.g., `comic-{topic-slug}-{timestamp}`)
|
||||
2. Use the same session ID for character sheet and all pages
|
||||
3. This ensures visual consistency (character appearance, style) across all generated images
|
||||
|
||||
3. Report progress after each generation
|
||||
|
||||
### Step 5: Completion Report
|
||||
|
||||
@@ -5,6 +5,13 @@ description: Image generation skill using Gemini Web. Generates images from text
|
||||
|
||||
# Gemini Web Client
|
||||
|
||||
Supports:
|
||||
- Text generation
|
||||
- Image generation (download + save)
|
||||
- Reference image upload (attach images for vision tasks)
|
||||
- Multi-turn conversations within the same executor instance (`keepSession`)
|
||||
- Experimental video generation (`generateVideo`) — Gemini may return an async placeholder; download might require Gemini web UI
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
@@ -12,8 +19,25 @@ npx -y bun scripts/main.ts "Hello, Gemini"
|
||||
npx -y bun scripts/main.ts --prompt "Explain quantum computing"
|
||||
npx -y bun scripts/main.ts --prompt "A cute cat" --image cat.png
|
||||
npx -y bun scripts/main.ts --promptfiles system.md content.md --image out.png
|
||||
|
||||
# Multi-turn conversation (agent generates unique sessionId)
|
||||
npx -y bun scripts/main.ts "Remember this: 42" --sessionId my-unique-id-123
|
||||
npx -y bun scripts/main.ts "What number?" --sessionId my-unique-id-123
|
||||
```
|
||||
|
||||
## Executor options (programmatic)
|
||||
|
||||
This skill is typically consumed via `createGeminiWebExecutor(geminiOptions)` (see `scripts/executor.ts`).
|
||||
|
||||
Key options in `GeminiWebOptions`:
|
||||
- `referenceImages?: string | string[]` Upload local images as references (vision input).
|
||||
- `keepSession?: boolean` Reuse Gemini `chatMetadata` to continue the same conversation across calls (required if you want reference images to persist across multiple messages).
|
||||
- `generateVideo?: string` Generate a video and (best-effort) download to the given path. Gemini may return `video_gen_chip` (async); in that case you must open Gemini web UI to download the result.
|
||||
|
||||
Notes:
|
||||
- `generateVideo` cannot be combined with `generateImage` / `editImage`.
|
||||
- When `keepSession=true` and `referenceImages` is set, reference images are uploaded once per executor instance.
|
||||
|
||||
## Commands
|
||||
|
||||
### Text generation
|
||||
@@ -58,17 +82,21 @@ npx -y bun scripts/main.ts "Hello" --json
|
||||
|
||||
## Options
|
||||
|
||||
| Option | Short | Description |
|
||||
|--------|-------|-------------|
|
||||
| `--prompt <text>` | `-p` | Prompt text |
|
||||
| `--promptfiles <files...>` | | Read prompt from files (concatenated in order) |
|
||||
| `--model <id>` | `-m` | Model: gemini-3-pro (default), gemini-2.5-pro, gemini-2.5-flash |
|
||||
| `--image [path]` | | Generate image, save to path (default: generated.png) |
|
||||
| `--json` | | Output as JSON |
|
||||
| `--login` | | Refresh cookies only, then exit |
|
||||
| `--cookie-path <path>` | | Custom cookie file path |
|
||||
| `--profile-dir <path>` | | Chrome profile directory |
|
||||
| `--help` | `-h` | Show help |
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--prompt <text>`, `-p` | Prompt text |
|
||||
| `--promptfiles <files...>` | Read prompt from files (concatenated in order) |
|
||||
| `--model <id>`, `-m` | Model: gemini-3-pro (default), gemini-2.5-pro, gemini-2.5-flash |
|
||||
| `--image [path]` | Generate image, save to path (default: generated.png) |
|
||||
| `--sessionId <id>` | Session ID for multi-turn conversation (agent generates unique ID) |
|
||||
| `--list-sessions` | List saved sessions (max 100, sorted by update time) |
|
||||
| `--json` | Output as JSON |
|
||||
| `--login` | Refresh cookies only, then exit |
|
||||
| `--cookie-path <path>` | Custom cookie file path |
|
||||
| `--profile-dir <path>` | Chrome profile directory |
|
||||
| `--help`, `-h` | Show help |
|
||||
|
||||
CLI note: `scripts/main.ts` supports text generation, image generation, and multi-turn conversations via `--sessionId`. Reference images and video generation are exposed via the executor API.
|
||||
|
||||
## Models
|
||||
|
||||
@@ -116,3 +144,22 @@ npx -y bun scripts/main.ts "Hello" --json | jq '.text'
|
||||
# Concatenate system.md + content.md as prompt
|
||||
npx -y bun scripts/main.ts --promptfiles system.md content.md --image output.png
|
||||
```
|
||||
|
||||
### Multi-turn conversation
|
||||
```bash
|
||||
# Start a session with unique ID (agent generates this)
|
||||
npx -y bun scripts/main.ts "You are a helpful math tutor." --sessionId task-abc123
|
||||
|
||||
# Continue the conversation (remembers context)
|
||||
npx -y bun scripts/main.ts "What is 2+2?" --sessionId task-abc123
|
||||
npx -y bun scripts/main.ts "Now multiply that by 10" --sessionId task-abc123
|
||||
|
||||
# List recent sessions (max 100, sorted by update time)
|
||||
npx -y bun scripts/main.ts --list-sessions
|
||||
```
|
||||
|
||||
Session files are stored in `~/Library/Application Support/baoyu-skills/gemini-web/sessions/<id>.json` and contain:
|
||||
- `id`: Session ID
|
||||
- `metadata`: Gemini chat metadata for continuation
|
||||
- `messages`: Array of `{role, content, timestamp, error?}`
|
||||
- `createdAt`, `updatedAt`: Timestamps
|
||||
|
||||
@@ -216,6 +216,7 @@ class CdpConnection {
|
||||
export async function getGeminiCookieMapViaChrome(options?: {
|
||||
timeoutMs?: number;
|
||||
debugConnectTimeoutMs?: number;
|
||||
tokenCheckTimeoutMs?: number;
|
||||
pollIntervalMs?: number;
|
||||
log?: GeminiWebLog;
|
||||
userDataDir?: string;
|
||||
@@ -224,6 +225,7 @@ export async function getGeminiCookieMapViaChrome(options?: {
|
||||
const log = options?.log;
|
||||
const timeoutMs = options?.timeoutMs ?? 5 * 60_000;
|
||||
const debugConnectTimeoutMs = options?.debugConnectTimeoutMs ?? 30_000;
|
||||
const tokenCheckTimeoutMs = options?.tokenCheckTimeoutMs ?? 30_000;
|
||||
const pollIntervalMs = options?.pollIntervalMs ?? 2_000;
|
||||
const userDataDir = options?.userDataDir ?? resolveGeminiWebChromeProfileDir();
|
||||
|
||||
@@ -290,7 +292,7 @@ export async function getGeminiCookieMapViaChrome(options?: {
|
||||
if (hasRequiredGeminiCookies(cookieMap)) {
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), 10_000);
|
||||
const timer = setTimeout(() => controller.abort(), tokenCheckTimeoutMs);
|
||||
try {
|
||||
await fetchGeminiAccessToken(cookieMap, controller.signal);
|
||||
} finally {
|
||||
|
||||
@@ -67,19 +67,69 @@ function buildCookieHeader(cookieMap: Record<string, string>): string {
|
||||
.join('; ');
|
||||
}
|
||||
|
||||
function getSetCookieHeaders(res: Response): string[] {
|
||||
const headers = res.headers as unknown as { getSetCookie?: () => string[] };
|
||||
if (typeof headers.getSetCookie === 'function') {
|
||||
try {
|
||||
return headers.getSetCookie();
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
const raw = res.headers.get('set-cookie');
|
||||
return raw ? [raw] : [];
|
||||
}
|
||||
|
||||
function applySetCookiesToMap(setCookies: string[], cookieMap: Record<string, string>): void {
|
||||
for (const raw of setCookies) {
|
||||
const first = raw.split(';')[0]?.trim();
|
||||
if (!first) continue;
|
||||
const idx = first.indexOf('=');
|
||||
if (idx <= 0) continue;
|
||||
const name = first.slice(0, idx).trim();
|
||||
const value = first.slice(idx + 1).trim();
|
||||
if (!name) continue;
|
||||
cookieMap[name] = value;
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchWithCookieJar(
|
||||
url: string,
|
||||
init: Omit<RequestInit, 'redirect' | 'headers'> & { headers?: Record<string, string> },
|
||||
cookieMap: Record<string, string>,
|
||||
signal?: AbortSignal,
|
||||
maxRedirects = 20,
|
||||
): Promise<Response> {
|
||||
let current = url;
|
||||
for (let i = 0; i <= maxRedirects; i += 1) {
|
||||
const cookieHeader = buildCookieHeader(cookieMap);
|
||||
const headers: Record<string, string> = {
|
||||
...(init.headers ?? {}),
|
||||
...(cookieHeader ? { cookie: cookieHeader } : {}),
|
||||
'user-agent': USER_AGENT,
|
||||
};
|
||||
|
||||
const res = await fetch(current, { ...init, redirect: 'manual', signal, headers });
|
||||
applySetCookiesToMap(getSetCookieHeaders(res), cookieMap);
|
||||
|
||||
if (res.status >= 300 && res.status < 400) {
|
||||
const location = res.headers.get('location');
|
||||
if (!location) return res;
|
||||
current = new URL(location, current).toString();
|
||||
continue;
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
throw new Error(`Too many redirects while fetching ${url} (>${maxRedirects}).`);
|
||||
}
|
||||
|
||||
export async function fetchGeminiAccessToken(
|
||||
cookieMap: Record<string, string>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> {
|
||||
const cookieHeader = buildCookieHeader(cookieMap);
|
||||
const res = await fetch(GEMINI_APP_URL, {
|
||||
redirect: 'follow',
|
||||
signal,
|
||||
headers: {
|
||||
cookie: cookieHeader,
|
||||
'user-agent': USER_AGENT,
|
||||
},
|
||||
});
|
||||
const res = await fetchWithCookieJar(GEMINI_APP_URL, { method: 'GET' }, cookieMap, signal);
|
||||
const html = await res.text();
|
||||
|
||||
const tokens = ['SNlM0e', 'thykhd'] as const;
|
||||
@@ -107,7 +157,21 @@ function extractErrorCode(responseJson: unknown): number | undefined {
|
||||
}
|
||||
|
||||
function extractGgdlUrls(rawText: string): string[] {
|
||||
const matches = rawText.match(/https:\/\/lh3\.googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? [];
|
||||
const matches =
|
||||
rawText.match(/https?:\/\/[^/\s"']*googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? [];
|
||||
const seen = new Set<string>();
|
||||
const urls: string[] = [];
|
||||
for (const match of matches) {
|
||||
if (seen.has(match)) continue;
|
||||
seen.add(match);
|
||||
urls.push(match);
|
||||
}
|
||||
return urls;
|
||||
}
|
||||
|
||||
function extractImageGenerationContentUrls(rawText: string): string[] {
|
||||
const matches =
|
||||
rawText.match(/https?:\/\/googleusercontent\.com\/image_generation_content\/\d+/g) ?? [];
|
||||
const seen = new Set<string>();
|
||||
const urls: string[] = [];
|
||||
for (const match of matches) {
|
||||
@@ -119,9 +183,17 @@ function extractGgdlUrls(rawText: string): string[] {
|
||||
}
|
||||
|
||||
function ensureFullSizeImageUrl(url: string): string {
|
||||
if (url.includes('=s2048')) return url;
|
||||
if (url.includes('=s')) return url;
|
||||
return `${url}=s2048`;
|
||||
const trimmed = url.trim();
|
||||
let normalized = trimmed;
|
||||
const backslashIndex = normalized.indexOf('\\');
|
||||
if (backslashIndex >= 0) normalized = normalized.slice(0, backslashIndex);
|
||||
// Some Gemini responses embed a size suffix as "/=s2048" which breaks downloads.
|
||||
normalized = normalized.replace(/\/=s(?=\d+(?:$|[?#]))/, '=s');
|
||||
normalized = normalized.replace(/\/=s(?=$|[?#])/, '=s');
|
||||
if (normalized.endsWith('/')) normalized = normalized.slice(0, -1);
|
||||
if (normalized.includes('=s2048')) return normalized;
|
||||
if (normalized.includes('=s')) return normalized;
|
||||
return `${normalized}=s2048`;
|
||||
}
|
||||
|
||||
async function fetchWithCookiePreservingRedirects(
|
||||
@@ -190,6 +262,29 @@ async function uploadGeminiFile(filePath: string, signal?: AbortSignal): Promise
|
||||
return { id: text, name: fileName };
|
||||
}
|
||||
|
||||
function guessMimeType(fileName: string): string {
|
||||
const ext = path.extname(fileName).toLowerCase();
|
||||
switch (ext) {
|
||||
case '.png':
|
||||
return 'image/png';
|
||||
case '.jpg':
|
||||
case '.jpeg':
|
||||
return 'image/jpeg';
|
||||
case '.webp':
|
||||
return 'image/webp';
|
||||
case '.gif':
|
||||
return 'image/gif';
|
||||
case '.mp4':
|
||||
return 'video/mp4';
|
||||
case '.mov':
|
||||
return 'video/quicktime';
|
||||
case '.webm':
|
||||
return 'video/webm';
|
||||
default:
|
||||
return 'application/octet-stream';
|
||||
}
|
||||
}
|
||||
|
||||
function buildGeminiFReqPayload(
|
||||
prompt: string,
|
||||
uploaded: Array<{ id: string; name: string }>,
|
||||
@@ -201,9 +296,8 @@ function buildGeminiFReqPayload(
|
||||
prompt,
|
||||
0,
|
||||
null,
|
||||
// Matches gemini-webapi payload format: [[[fileId, 1]]] for a single attachment.
|
||||
// Keep it extensible for multiple uploads by emitting one [[id, 1]] entry per file.
|
||||
uploaded.map((file) => [[file.id, 1]]),
|
||||
// Matches gemini-web payload format: [[[fileId, 1, null, mimeType], fileName]] for an attachment.
|
||||
uploaded.map((file) => [[file.id, 1, null, guessMimeType(file.name)], file.name]),
|
||||
]
|
||||
: [prompt];
|
||||
|
||||
@@ -248,7 +342,19 @@ export function parseGeminiStreamGenerateResponse(rawText: string): {
|
||||
? (getNestedValue<string | null>(firstCandidate, [22, 0], null) ?? textRaw)
|
||||
: textRaw;
|
||||
const thoughts = getNestedValue<string | null>(firstCandidate, [37, 0, 0], null);
|
||||
const metadata = getNestedValue<unknown>(body, [1], []);
|
||||
const conversationMeta = getNestedValue<unknown[]>(body, [1], []);
|
||||
const conversationId =
|
||||
typeof conversationMeta[0] === 'string' && conversationMeta[0].length > 0
|
||||
? conversationMeta[0]
|
||||
: null;
|
||||
const responseId =
|
||||
typeof conversationMeta[1] === 'string' && conversationMeta[1].length > 0
|
||||
? conversationMeta[1]
|
||||
: null;
|
||||
const choiceIdRaw = getNestedValue<string | null>(firstCandidate, [0], null);
|
||||
const choiceId = typeof choiceIdRaw === 'string' && choiceIdRaw.length > 0 ? choiceIdRaw : null;
|
||||
const metadata =
|
||||
conversationId && responseId && choiceId ? [conversationId, responseId, choiceId] : conversationMeta;
|
||||
|
||||
const images: GeminiWebCandidateImage[] = [];
|
||||
|
||||
@@ -305,8 +411,8 @@ export function isGeminiModelUnavailable(errorCode: number | undefined): boolean
|
||||
}
|
||||
|
||||
export async function runGeminiWebOnce(input: GeminiWebRunInput): Promise<GeminiWebRunOutput> {
|
||||
const cookieHeader = buildCookieHeader(input.cookieMap);
|
||||
const at = await fetchGeminiAccessToken(input.cookieMap, input.signal);
|
||||
const cookieHeader = buildCookieHeader(input.cookieMap);
|
||||
|
||||
const uploaded: Array<{ id: string; name: string }> = [];
|
||||
for (const file of input.files ?? []) {
|
||||
@@ -403,11 +509,19 @@ export async function saveFirstGeminiImageFromOutput(
|
||||
return { saved: true, imageCount: output.images.length };
|
||||
}
|
||||
|
||||
const ggdl = extractGgdlUrls(output.rawResponseText);
|
||||
if (ggdl[0]) {
|
||||
await downloadGeminiImage(ggdl[0], cookieMap, outputPath, signal);
|
||||
const ggdl = extractGgdlUrls(`${output.text}\n${output.rawResponseText}`);
|
||||
const preferred = ggdl.length > 0 ? ggdl[ggdl.length - 1] : null;
|
||||
if (preferred) {
|
||||
await downloadGeminiImage(preferred, cookieMap, outputPath, signal);
|
||||
return { saved: true, imageCount: ggdl.length };
|
||||
}
|
||||
|
||||
const imageGen = extractImageGenerationContentUrls(`${output.text}\n${output.rawResponseText}`);
|
||||
const imageGenPreferred = imageGen.length > 0 ? imageGen[imageGen.length - 1] : null;
|
||||
if (imageGenPreferred) {
|
||||
await downloadGeminiImage(imageGenPreferred, cookieMap, outputPath, signal);
|
||||
return { saved: true, imageCount: imageGen.length };
|
||||
}
|
||||
|
||||
return { saved: false, imageCount: 0 };
|
||||
}
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import { mkdir, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import type { BrowserRunOptions, BrowserRunResult, BrowserLogger, CookieParam } from '../browser/types.js';
|
||||
import { runGeminiWebWithFallback, saveFirstGeminiImageFromOutput } from './client.js';
|
||||
import type { GeminiWebModelId } from './client.js';
|
||||
import type { GeminiWebModelId, GeminiWebRunOutput } from './client.js';
|
||||
import {
|
||||
buildGeminiCookieMap,
|
||||
hasRequiredGeminiCookies,
|
||||
@@ -11,6 +12,9 @@ import type { GeminiWebOptions, GeminiWebResponse } from './types.js';
|
||||
|
||||
export { hasRequiredGeminiCookies } from './cookie-store.js';
|
||||
|
||||
const USER_AGENT =
|
||||
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36';
|
||||
|
||||
function estimateTokenCount(text: string): number {
|
||||
return Math.ceil(text.length / 4);
|
||||
}
|
||||
@@ -22,6 +26,115 @@ function resolveInvocationPath(value: string | undefined): string | undefined {
|
||||
return path.isAbsolute(trimmed) ? trimmed : path.resolve(process.cwd(), trimmed);
|
||||
}
|
||||
|
||||
function normalizePathList(value: string | string[] | undefined): string[] {
|
||||
if (!value) return [];
|
||||
const raw = Array.isArray(value) ? value : [value];
|
||||
const out: string[] = [];
|
||||
for (const entry of raw) {
|
||||
if (typeof entry !== 'string') continue;
|
||||
const resolved = resolveInvocationPath(entry);
|
||||
if (!resolved) continue;
|
||||
out.push(resolved);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function dedupePaths(paths: string[]): string[] {
|
||||
const seen = new Set<string>();
|
||||
const out: string[] = [];
|
||||
for (const item of paths) {
|
||||
const trimmed = item.trim();
|
||||
if (!trimmed || seen.has(trimmed)) continue;
|
||||
seen.add(trimmed);
|
||||
out.push(trimmed);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function buildCookieHeader(cookieMap: Record<string, string>): string {
|
||||
return Object.entries(cookieMap)
|
||||
.filter(([, value]) => typeof value === 'string' && value.length > 0)
|
||||
.map(([name, value]) => `${name}=${value}`)
|
||||
.join('; ');
|
||||
}
|
||||
|
||||
async function fetchWithCookiePreservingRedirects(
|
||||
url: string,
|
||||
init: Omit<RequestInit, 'redirect'>,
|
||||
signal?: AbortSignal,
|
||||
maxRedirects = 10,
|
||||
): Promise<Response> {
|
||||
let current = url;
|
||||
for (let i = 0; i <= maxRedirects; i += 1) {
|
||||
const res = await fetch(current, { ...init, redirect: 'manual', signal });
|
||||
if (res.status >= 300 && res.status < 400) {
|
||||
const location = res.headers.get('location');
|
||||
if (!location) return res;
|
||||
current = new URL(location, current).toString();
|
||||
continue;
|
||||
}
|
||||
return res;
|
||||
}
|
||||
throw new Error(`Too many redirects while downloading media (>${maxRedirects}).`);
|
||||
}
|
||||
|
||||
async function downloadGeminiMedia(
|
||||
url: string,
|
||||
cookieMap: Record<string, string>,
|
||||
outputPath: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
const cookieHeader = buildCookieHeader(cookieMap);
|
||||
const res = await fetchWithCookiePreservingRedirects(
|
||||
url,
|
||||
{
|
||||
headers: {
|
||||
cookie: cookieHeader,
|
||||
'user-agent': USER_AGENT,
|
||||
},
|
||||
},
|
||||
signal,
|
||||
);
|
||||
if (!res.ok) {
|
||||
throw new Error(`Failed to download media: ${res.status} ${res.statusText} (${res.url})`);
|
||||
}
|
||||
|
||||
const data = new Uint8Array(await res.arrayBuffer());
|
||||
await mkdir(path.dirname(outputPath), { recursive: true });
|
||||
await writeFile(outputPath, data);
|
||||
}
|
||||
|
||||
function extractGgdlUrls(rawText: string): string[] {
|
||||
const matches =
|
||||
rawText.match(/https?:\/\/[^/\s"']*googleusercontent\.com\/gg-dl\/[^\s"']+/g) ?? [];
|
||||
const seen = new Set<string>();
|
||||
const urls: string[] = [];
|
||||
for (const match of matches) {
|
||||
if (seen.has(match)) continue;
|
||||
seen.add(match);
|
||||
urls.push(match);
|
||||
}
|
||||
return urls;
|
||||
}
|
||||
|
||||
async function saveFirstGeminiVideoFromOutput(
|
||||
output: GeminiWebRunOutput,
|
||||
cookieMap: Record<string, string>,
|
||||
outputPath: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<{ saved: boolean; videoCount: number }> {
|
||||
const ggdl = extractGgdlUrls(output.rawResponseText);
|
||||
if (!ggdl[0]) return { saved: false, videoCount: 0 };
|
||||
|
||||
const videoCandidates = ggdl.filter((url) => /\.(mp4|webm|mov)(?:$|[?#])/i.test(url));
|
||||
const preferred =
|
||||
(videoCandidates.length > 0 ? videoCandidates[videoCandidates.length - 1] : null) ??
|
||||
ggdl.find((url) => /video/i.test(url)) ??
|
||||
ggdl[ggdl.length - 1];
|
||||
await downloadGeminiMedia(preferred, cookieMap, outputPath, signal);
|
||||
return { saved: true, videoCount: ggdl.length };
|
||||
}
|
||||
|
||||
function resolveGeminiWebModel(
|
||||
desiredModel: string | null | undefined,
|
||||
log?: BrowserLogger,
|
||||
@@ -115,6 +228,9 @@ export async function loadGeminiCookieMap(log?: BrowserLogger): Promise<Record<s
|
||||
export function createGeminiWebExecutor(
|
||||
geminiOptions: GeminiWebOptions,
|
||||
): (runOptions: BrowserRunOptions) => Promise<BrowserRunResult> {
|
||||
let persistedChatMetadata: unknown | null = null;
|
||||
let referenceImagesUploaded = false;
|
||||
|
||||
return async (runOptions: BrowserRunOptions): Promise<BrowserRunResult> => {
|
||||
const startTime = Date.now();
|
||||
const log = runOptions.log;
|
||||
@@ -133,23 +249,38 @@ export function createGeminiWebExecutor(
|
||||
? Math.max(1_000, runOptions.config.timeoutMs)
|
||||
: null;
|
||||
|
||||
const generateVideoPath = resolveInvocationPath(geminiOptions.generateVideo);
|
||||
|
||||
const defaultTimeoutMs = geminiOptions.youtube
|
||||
? 240_000
|
||||
: geminiOptions.generateImage || geminiOptions.editImage
|
||||
: generateVideoPath
|
||||
? 900_000
|
||||
: geminiOptions.generateImage || geminiOptions.editImage
|
||||
? 300_000
|
||||
: 120_000;
|
||||
|
||||
const timeoutMs = Math.min(configTimeout ?? defaultTimeoutMs, 600_000);
|
||||
const timeoutCapMs = generateVideoPath ? 1_800_000 : 600_000;
|
||||
const timeoutMs = Math.min(configTimeout ?? defaultTimeoutMs, timeoutCapMs);
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
const keepSession = geminiOptions.keepSession === true;
|
||||
|
||||
const generateImagePath = resolveInvocationPath(geminiOptions.generateImage);
|
||||
const editImagePath = resolveInvocationPath(geminiOptions.editImage);
|
||||
const outputPath = resolveInvocationPath(geminiOptions.outputPath);
|
||||
const attachmentPaths = (runOptions.attachments ?? []).map((attachment) => attachment.path);
|
||||
const referenceImagePaths = normalizePathList(geminiOptions.referenceImages);
|
||||
const requestFilePaths = dedupePaths(
|
||||
keepSession ? attachmentPaths : [...referenceImagePaths, ...attachmentPaths],
|
||||
);
|
||||
|
||||
if (generateVideoPath && (generateImagePath || editImagePath)) {
|
||||
throw new Error('Gemini web executor: generateVideo cannot be combined with generateImage/editImage options.');
|
||||
}
|
||||
|
||||
let prompt = runOptions.prompt;
|
||||
if (geminiOptions.aspectRatio && (generateImagePath || editImagePath)) {
|
||||
if (geminiOptions.aspectRatio && (generateImagePath || editImagePath || generateVideoPath)) {
|
||||
prompt = `${prompt} (aspect ratio: ${geminiOptions.aspectRatio})`;
|
||||
}
|
||||
if (geminiOptions.youtube) {
|
||||
@@ -158,29 +289,50 @@ export function createGeminiWebExecutor(
|
||||
if (generateImagePath && !editImagePath) {
|
||||
prompt = `Generate an image: ${prompt}`;
|
||||
}
|
||||
if (generateVideoPath) {
|
||||
prompt = `Generate a video: ${prompt}`;
|
||||
}
|
||||
|
||||
const model: GeminiWebModelId = resolveGeminiWebModel(runOptions.config?.desiredModel, log);
|
||||
let response: GeminiWebResponse;
|
||||
let videoSaveSummary: { saved: boolean; videoCount: number; outputPath: string } | null = null;
|
||||
|
||||
try {
|
||||
let chatMetadata: unknown = keepSession ? persistedChatMetadata : null;
|
||||
|
||||
if (keepSession && referenceImagePaths.length > 0 && !referenceImagesUploaded) {
|
||||
const intro = await runGeminiWebWithFallback({
|
||||
prompt: 'Here are reference images for future messages.',
|
||||
files: referenceImagePaths,
|
||||
model,
|
||||
cookieMap,
|
||||
chatMetadata,
|
||||
signal: controller.signal,
|
||||
});
|
||||
chatMetadata = intro.metadata;
|
||||
persistedChatMetadata = intro.metadata;
|
||||
referenceImagesUploaded = true;
|
||||
}
|
||||
|
||||
if (editImagePath) {
|
||||
const intro = await runGeminiWebWithFallback({
|
||||
prompt: 'Here is an image to edit',
|
||||
files: [editImagePath],
|
||||
model,
|
||||
cookieMap,
|
||||
chatMetadata: null,
|
||||
chatMetadata,
|
||||
signal: controller.signal,
|
||||
});
|
||||
const editPrompt = `Use image generation tool to ${prompt}`;
|
||||
const out = await runGeminiWebWithFallback({
|
||||
prompt: editPrompt,
|
||||
files: attachmentPaths,
|
||||
files: requestFilePaths,
|
||||
model,
|
||||
cookieMap,
|
||||
chatMetadata: intro.metadata,
|
||||
signal: controller.signal,
|
||||
});
|
||||
if (keepSession) persistedChatMetadata = out.metadata;
|
||||
response = {
|
||||
text: out.text ?? null,
|
||||
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
|
||||
@@ -198,12 +350,13 @@ export function createGeminiWebExecutor(
|
||||
} else if (generateImagePath) {
|
||||
const out = await runGeminiWebWithFallback({
|
||||
prompt,
|
||||
files: attachmentPaths,
|
||||
files: requestFilePaths,
|
||||
model,
|
||||
cookieMap,
|
||||
chatMetadata: null,
|
||||
chatMetadata,
|
||||
signal: controller.signal,
|
||||
});
|
||||
if (keepSession) persistedChatMetadata = out.metadata;
|
||||
response = {
|
||||
text: out.text ?? null,
|
||||
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
|
||||
@@ -216,15 +369,36 @@ export function createGeminiWebExecutor(
|
||||
if (!imageSave.saved) {
|
||||
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
|
||||
}
|
||||
} else if (generateVideoPath) {
|
||||
const out = await runGeminiWebWithFallback({
|
||||
prompt,
|
||||
files: requestFilePaths,
|
||||
model,
|
||||
cookieMap,
|
||||
chatMetadata,
|
||||
signal: controller.signal,
|
||||
});
|
||||
if (keepSession) persistedChatMetadata = out.metadata;
|
||||
response = {
|
||||
text: out.text ?? null,
|
||||
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
|
||||
has_images: false,
|
||||
image_count: 0,
|
||||
};
|
||||
|
||||
const resolvedOutputPath = generateVideoPath ?? outputPath ?? 'generated.mp4';
|
||||
const save = await saveFirstGeminiVideoFromOutput(out, cookieMap, resolvedOutputPath, controller.signal);
|
||||
videoSaveSummary = { ...save, outputPath: resolvedOutputPath };
|
||||
} else {
|
||||
const out = await runGeminiWebWithFallback({
|
||||
prompt,
|
||||
files: attachmentPaths,
|
||||
files: requestFilePaths,
|
||||
model,
|
||||
cookieMap,
|
||||
chatMetadata: null,
|
||||
chatMetadata,
|
||||
signal: controller.signal,
|
||||
});
|
||||
if (keepSession) persistedChatMetadata = out.metadata;
|
||||
response = {
|
||||
text: out.text ?? null,
|
||||
thoughts: geminiOptions.showThoughts ? out.thoughts : null,
|
||||
@@ -247,6 +421,15 @@ export function createGeminiWebExecutor(
|
||||
const imagePath = generateImagePath || outputPath || 'generated.png';
|
||||
answerMarkdown += `\n\n*Generated ${response.image_count} image(s). Saved to: ${imagePath}*`;
|
||||
}
|
||||
if (videoSaveSummary) {
|
||||
if (videoSaveSummary.saved) {
|
||||
answerMarkdown += `\n\n*Generated ${videoSaveSummary.videoCount || 1} video(s). Saved to: ${videoSaveSummary.outputPath}*`;
|
||||
} else if (/video_gen_chip/.test(answerMarkdown) || /video_gen_chip/.test(response.text ?? '')) {
|
||||
answerMarkdown += '\n\n*Video generation is asynchronous. Check Gemini web UI to download the result.*';
|
||||
} else {
|
||||
answerMarkdown += '\n\n*No downloadable video URL found in Gemini response.*';
|
||||
}
|
||||
}
|
||||
|
||||
const tookMs = Date.now() - startTime;
|
||||
log?.(`[gemini-web] Completed in ${tookMs}ms`);
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
writeGeminiCookieMapToDisk,
|
||||
} from './cookie-store.js';
|
||||
import { resolveGeminiWebChromeProfileDir, resolveGeminiWebCookiePath } from './paths.js';
|
||||
import { readSession, writeSession, listSessions } from './session-store.js';
|
||||
|
||||
function printUsage(exitCode = 0): never {
|
||||
const cookiePath = resolveGeminiWebCookiePath();
|
||||
@@ -21,12 +22,19 @@ function printUsage(exitCode = 0): never {
|
||||
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --prompt "A cute cat" --image generated.png
|
||||
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
|
||||
|
||||
Multi-turn conversation (agent generates unique sessionId):
|
||||
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "Remember 42" --sessionId abc123
|
||||
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "What number?" --sessionId abc123
|
||||
|
||||
Options:
|
||||
-p, --prompt <text> Prompt text
|
||||
--promptfiles <files...> Read prompt from one or more files (concatenated in order)
|
||||
-m, --model <id> gemini-3-pro | gemini-2.5-pro | gemini-2.5-flash (default: gemini-3-pro)
|
||||
--json Output JSON
|
||||
--image [path] Generate an image and save it (default: ./generated.png)
|
||||
--reference <files...> Reference images for vision input
|
||||
--sessionId <id> Session ID for multi-turn conversation (agent should generate unique ID)
|
||||
--list-sessions List saved sessions (max 100, sorted by update time)
|
||||
--login Only refresh cookies, then exit
|
||||
--cookie-path <path> Cookie file path (default: ${cookiePath})
|
||||
--profile-dir <path> Chrome profile dir (default: ${profileDir})
|
||||
@@ -75,6 +83,9 @@ function parseArgs(argv: string[]): {
|
||||
loginOnly?: boolean;
|
||||
cookiePath?: string;
|
||||
profileDir?: string;
|
||||
referenceImages?: string[];
|
||||
sessionId?: string;
|
||||
listSessions?: boolean;
|
||||
} {
|
||||
const out: ReturnType<typeof parseArgs> = {};
|
||||
const positional: string[] = [];
|
||||
@@ -157,6 +168,32 @@ function parseArgs(argv: string[]): {
|
||||
out.profileDir = arg.slice('--profile-dir='.length);
|
||||
continue;
|
||||
}
|
||||
if (arg === '--reference' || arg === '--ref') {
|
||||
out.referenceImages = [];
|
||||
while (i + 1 < argv.length) {
|
||||
const next = argv[i + 1];
|
||||
if (next && !next.startsWith('-')) {
|
||||
out.referenceImages.push(next);
|
||||
i += 1;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (arg === '--sessionId' || arg === '--session-id') {
|
||||
out.sessionId = argv[i + 1] ?? '';
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
if (arg.startsWith('--sessionId=') || arg.startsWith('--session-id=')) {
|
||||
out.sessionId = arg.split('=')[1] ?? '';
|
||||
continue;
|
||||
}
|
||||
if (arg === '--list-sessions') {
|
||||
out.listSessions = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (arg.startsWith('-')) {
|
||||
throw new Error(`Unknown option: ${arg}`);
|
||||
@@ -178,6 +215,9 @@ function parseArgs(argv: string[]): {
|
||||
if (out.cookiePath === '') delete out.cookiePath;
|
||||
if (out.profileDir === '') delete out.profileDir;
|
||||
if (out.promptFiles?.length === 0) delete out.promptFiles;
|
||||
if (out.referenceImages?.length === 0) delete out.referenceImages;
|
||||
if (out.sessionId != null) out.sessionId = out.sessionId.trim();
|
||||
if (out.sessionId === '') delete out.sessionId;
|
||||
|
||||
return out;
|
||||
}
|
||||
@@ -186,7 +226,7 @@ async function isCookieMapValid(cookieMap: Record<string, string>): Promise<bool
|
||||
if (!hasRequiredGeminiCookies(cookieMap)) return false;
|
||||
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), 10_000);
|
||||
const timer = setTimeout(() => controller.abort(), 30_000);
|
||||
try {
|
||||
await fetchGeminiAccessToken(cookieMap, controller.signal);
|
||||
return true;
|
||||
@@ -201,7 +241,7 @@ async function ensureGeminiCookieMap(options: {
|
||||
cookiePath: string;
|
||||
profileDir: string;
|
||||
}): Promise<Record<string, string>> {
|
||||
const log = (msg: string) => console.log(msg);
|
||||
const log = (msg: string) => console.error(msg);
|
||||
|
||||
let cookieMap = await readGeminiCookieMapFromDisk({ cookiePath: options.cookiePath, log });
|
||||
if (await isCookieMapValid(cookieMap)) return cookieMap;
|
||||
@@ -251,85 +291,63 @@ async function main(): Promise<void> {
|
||||
const cookiePath = args.cookiePath ?? resolveGeminiWebCookiePath();
|
||||
const profileDir = args.profileDir ?? resolveGeminiWebChromeProfileDir();
|
||||
|
||||
if (args.listSessions) {
|
||||
const sessions = await listSessions();
|
||||
if (sessions.length === 0) {
|
||||
console.log('No saved sessions.');
|
||||
} else {
|
||||
for (const { id, updatedAt } of sessions) {
|
||||
console.log(`${id}\t${updatedAt}`);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (args.loginOnly) {
|
||||
await ensureGeminiCookieMap({ cookiePath, profileDir });
|
||||
return;
|
||||
}
|
||||
|
||||
const promptFromStdin = await readPromptFromStdin();
|
||||
const promptFromFiles = args.promptFiles ? readPromptFiles(args.promptFiles) : null;
|
||||
const prompt = promptFromFiles || args.prompt || promptFromStdin;
|
||||
const promptFromArgs = promptFromFiles || args.prompt;
|
||||
const prompt = promptFromArgs || (await readPromptFromStdin());
|
||||
if (!prompt) printUsage(1);
|
||||
|
||||
const sessionData = args.sessionId ? await readSession(args.sessionId) : null;
|
||||
const chatMetadata = sessionData?.metadata ?? null;
|
||||
|
||||
let cookieMap = await ensureGeminiCookieMap({ cookiePath, profileDir });
|
||||
|
||||
const desiredModel = resolveModel(args.model || 'gemini-3-pro');
|
||||
const imagePath = resolveImageOutputPath(args.imagePath);
|
||||
const referenceImages = (args.referenceImages ?? []).map((p) =>
|
||||
path.isAbsolute(p) ? p : path.resolve(process.cwd(), p),
|
||||
);
|
||||
|
||||
try {
|
||||
const effectivePrompt = imagePath ? `Generate an image: ${prompt}` : prompt;
|
||||
const out = await runGeminiWebWithFallback({
|
||||
prompt: effectivePrompt,
|
||||
files: [],
|
||||
model: desiredModel,
|
||||
cookieMap,
|
||||
chatMetadata: null,
|
||||
});
|
||||
|
||||
let imageSaved = false;
|
||||
let imageCount = 0;
|
||||
if (imagePath) {
|
||||
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath);
|
||||
imageSaved = save.saved;
|
||||
imageCount = save.imageCount;
|
||||
if (!imageSaved) {
|
||||
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (args.json) {
|
||||
process.stdout.write(
|
||||
`${JSON.stringify(
|
||||
imagePath ? { ...out, imageSaved, imageCount, imagePath } : out,
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
);
|
||||
if (out.errorMessage) process.exit(1);
|
||||
return;
|
||||
}
|
||||
|
||||
if (out.errorMessage) {
|
||||
throw new Error(out.errorMessage);
|
||||
}
|
||||
|
||||
process.stdout.write(out.text ?? '');
|
||||
if (!out.text?.endsWith('\n')) process.stdout.write('\n');
|
||||
if (imagePath) {
|
||||
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
|
||||
}
|
||||
return;
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
|
||||
if (message.includes('Unable to locate Gemini access token')) {
|
||||
console.error('[gemini-web] Cookies may be expired. Re-opening browser to refresh cookies...');
|
||||
await sleep(500);
|
||||
cookieMap = await getGeminiCookieMapViaChrome({ userDataDir: profileDir, log: (m) => console.log(m) });
|
||||
await writeGeminiCookieMapToDisk(cookieMap, { cookiePath, log: (m) => console.log(m) });
|
||||
const controller = new AbortController();
|
||||
const timeoutMs = imagePath ? 300_000 : 120_000;
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
try {
|
||||
const effectivePrompt = imagePath ? `Generate an image: ${prompt}` : prompt;
|
||||
const out = await runGeminiWebWithFallback({
|
||||
prompt: imagePath ? `Generate an image: ${prompt}` : prompt,
|
||||
files: [],
|
||||
prompt: effectivePrompt,
|
||||
files: referenceImages,
|
||||
model: desiredModel,
|
||||
cookieMap,
|
||||
chatMetadata: null,
|
||||
chatMetadata,
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (args.sessionId && out.metadata) {
|
||||
await writeSession(args.sessionId, out.metadata, prompt, out.text ?? '', out.errorMessage);
|
||||
}
|
||||
|
||||
let imageSaved = false;
|
||||
let imageCount = 0;
|
||||
if (imagePath) {
|
||||
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath);
|
||||
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath, controller.signal);
|
||||
imageSaved = save.saved;
|
||||
imageCount = save.imageCount;
|
||||
if (!imageSaved) {
|
||||
@@ -338,13 +356,8 @@ async function main(): Promise<void> {
|
||||
}
|
||||
|
||||
if (args.json) {
|
||||
process.stdout.write(
|
||||
`${JSON.stringify(
|
||||
imagePath ? { ...out, imageSaved, imageCount, imagePath } : out,
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
);
|
||||
const jsonOut = { ...out, ...(imagePath && { imageSaved, imageCount, imagePath }), ...(args.sessionId && { sessionId: args.sessionId }) };
|
||||
process.stdout.write(`${JSON.stringify(jsonOut, null, 2)}\n`);
|
||||
if (out.errorMessage) process.exit(1);
|
||||
return;
|
||||
}
|
||||
@@ -359,6 +372,67 @@ async function main(): Promise<void> {
|
||||
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
|
||||
}
|
||||
return;
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
|
||||
if (message.includes('Unable to locate Gemini access token')) {
|
||||
console.error('[gemini-web] Cookies may be expired. Re-opening browser to refresh cookies...');
|
||||
await sleep(500);
|
||||
cookieMap = await getGeminiCookieMapViaChrome({ userDataDir: profileDir, log: (m) => console.error(m) });
|
||||
await writeGeminiCookieMapToDisk(cookieMap, { cookiePath, log: (m) => console.error(m) });
|
||||
|
||||
const controller = new AbortController();
|
||||
const timeoutMs = imagePath ? 300_000 : 120_000;
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
try {
|
||||
const out = await runGeminiWebWithFallback({
|
||||
prompt: imagePath ? `Generate an image: ${prompt}` : prompt,
|
||||
files: referenceImages,
|
||||
model: desiredModel,
|
||||
cookieMap,
|
||||
chatMetadata,
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (args.sessionId && out.metadata) {
|
||||
await writeSession(args.sessionId, out.metadata, prompt, out.text ?? '', out.errorMessage);
|
||||
}
|
||||
|
||||
let imageSaved = false;
|
||||
let imageCount = 0;
|
||||
if (imagePath) {
|
||||
const save = await saveFirstGeminiImageFromOutput(out, cookieMap, imagePath, controller.signal);
|
||||
imageSaved = save.saved;
|
||||
imageCount = save.imageCount;
|
||||
if (!imageSaved) {
|
||||
throw new Error(`No images generated. Response text:\n${out.text || '(empty response)'}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (args.json) {
|
||||
const jsonOut = { ...out, ...(imagePath && { imageSaved, imageCount, imagePath }), ...(args.sessionId && { sessionId: args.sessionId }) };
|
||||
process.stdout.write(`${JSON.stringify(jsonOut, null, 2)}\n`);
|
||||
if (out.errorMessage) process.exit(1);
|
||||
return;
|
||||
}
|
||||
|
||||
if (out.errorMessage) {
|
||||
throw new Error(out.errorMessage);
|
||||
}
|
||||
|
||||
process.stdout.write(out.text ?? '');
|
||||
if (!out.text?.endsWith('\n')) process.stdout.write('\n');
|
||||
if (imagePath) {
|
||||
process.stdout.write(`Saved image (${imageCount || 1}) to: ${imagePath}\n`);
|
||||
}
|
||||
return;
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
throw error;
|
||||
|
||||
@@ -34,3 +34,12 @@ export function resolveGeminiWebChromeProfileDir(): string {
|
||||
if (override) return path.resolve(override);
|
||||
return path.join(resolveGeminiWebDataDir(), PROFILE_DIR_NAME);
|
||||
}
|
||||
|
||||
export function resolveGeminiWebSessionsDir(): string {
|
||||
return path.join(resolveGeminiWebDataDir(), 'sessions');
|
||||
}
|
||||
|
||||
export function resolveGeminiWebSessionPath(name: string): string {
|
||||
const sanitized = name.replace(/[^a-zA-Z0-9_-]/g, '_');
|
||||
return path.join(resolveGeminiWebSessionsDir(), `${sanitized}.json`);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
import { mkdir, readFile, writeFile, readdir, stat } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import { resolveGeminiWebSessionsDir, resolveGeminiWebSessionPath } from './paths.js';
|
||||
|
||||
export interface SessionMessage {
|
||||
role: 'user' | 'assistant';
|
||||
content: string;
|
||||
timestamp: string;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export interface SessionData {
|
||||
id: string;
|
||||
metadata: unknown;
|
||||
messages: SessionMessage[];
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
export interface SessionListItem {
|
||||
id: string;
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
export async function readSession(id: string): Promise<SessionData | null> {
|
||||
const sessionPath = resolveGeminiWebSessionPath(id);
|
||||
try {
|
||||
const content = await readFile(sessionPath, 'utf8');
|
||||
return JSON.parse(content) as SessionData;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function writeSession(
|
||||
id: string,
|
||||
metadata: unknown,
|
||||
userMessage: string,
|
||||
assistantMessage: string,
|
||||
error?: string,
|
||||
): Promise<void> {
|
||||
const sessionPath = resolveGeminiWebSessionPath(id);
|
||||
const sessionsDir = resolveGeminiWebSessionsDir();
|
||||
await mkdir(sessionsDir, { recursive: true });
|
||||
|
||||
const existing = await readSession(id);
|
||||
const now = new Date().toISOString();
|
||||
|
||||
const newMessages: SessionMessage[] = [
|
||||
{ role: 'user', content: userMessage, timestamp: now },
|
||||
{ role: 'assistant', content: assistantMessage, timestamp: now, ...(error && { error }) },
|
||||
];
|
||||
|
||||
const data: SessionData = {
|
||||
id,
|
||||
metadata,
|
||||
messages: [...(existing?.messages ?? []), ...newMessages],
|
||||
createdAt: existing?.createdAt ?? now,
|
||||
updatedAt: now,
|
||||
};
|
||||
await writeFile(sessionPath, JSON.stringify(data, null, 2));
|
||||
}
|
||||
|
||||
export async function listSessions(limit = 100): Promise<SessionListItem[]> {
|
||||
const sessionsDir = resolveGeminiWebSessionsDir();
|
||||
try {
|
||||
const files = await readdir(sessionsDir);
|
||||
const jsonFiles = files.filter((f) => f.endsWith('.json'));
|
||||
|
||||
const items: { id: string; updatedAt: string; mtime: number }[] = [];
|
||||
for (const file of jsonFiles) {
|
||||
const filePath = path.join(sessionsDir, file);
|
||||
try {
|
||||
const stats = await stat(filePath);
|
||||
items.push({
|
||||
id: file.slice(0, -5),
|
||||
updatedAt: stats.mtime.toISOString(),
|
||||
mtime: stats.mtime.getTime(),
|
||||
});
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
items.sort((a, b) => b.mtime - a.mtime);
|
||||
return items.slice(0, limit).map(({ id, updatedAt }) => ({ id, updatedAt }));
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
@@ -2,9 +2,18 @@ export interface GeminiWebOptions {
|
||||
youtube?: string;
|
||||
generateImage?: string;
|
||||
editImage?: string;
|
||||
generateVideo?: string;
|
||||
outputPath?: string;
|
||||
showThoughts?: boolean;
|
||||
aspectRatio?: string;
|
||||
/**
|
||||
* One or more local image paths to upload as persistent reference images.
|
||||
* - If `keepSession` is enabled, they are uploaded once per executor session.
|
||||
* - Otherwise, they are attached to each request.
|
||||
*/
|
||||
referenceImages?: string | string[];
|
||||
/** Preserve Gemini chat metadata to continue multi-turn conversations within the same executor instance. */
|
||||
keepSession?: boolean;
|
||||
}
|
||||
|
||||
export interface GeminiWebResponse {
|
||||
|
||||
@@ -481,52 +481,78 @@ export async function publishArticle(options: ArticleOptions): Promise<void> {
|
||||
const img = sortedImages[i]!;
|
||||
console.log(`[x-article] [${i + 1}/${sortedImages.length}] Inserting image at placeholder: ${img.placeholder}`);
|
||||
|
||||
// Find, scroll to, and select the placeholder text in DraftEditor
|
||||
const placeholderFound = await cdp.send<{ result: { value: boolean } }>('Runtime.evaluate', {
|
||||
expression: `(() => {
|
||||
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]');
|
||||
if (!editor) return false;
|
||||
// Helper to select placeholder with retry
|
||||
const selectPlaceholder = async (maxRetries = 3): Promise<boolean> => {
|
||||
for (let attempt = 1; attempt <= maxRetries; attempt++) {
|
||||
// Find, scroll to, and select the placeholder text in DraftEditor
|
||||
await cdp!.send('Runtime.evaluate', {
|
||||
expression: `(() => {
|
||||
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]');
|
||||
if (!editor) return false;
|
||||
|
||||
const placeholder = ${JSON.stringify(img.placeholder)};
|
||||
const placeholder = ${JSON.stringify(img.placeholder)};
|
||||
|
||||
// Search through all text nodes in the editor
|
||||
const walker = document.createTreeWalker(editor, NodeFilter.SHOW_TEXT, null, false);
|
||||
let node;
|
||||
// Search through all text nodes in the editor
|
||||
const walker = document.createTreeWalker(editor, NodeFilter.SHOW_TEXT, null, false);
|
||||
let node;
|
||||
|
||||
while ((node = walker.nextNode())) {
|
||||
const text = node.textContent || '';
|
||||
const idx = text.indexOf(placeholder);
|
||||
if (idx !== -1) {
|
||||
// Found the placeholder - scroll to it first
|
||||
const parentElement = node.parentElement;
|
||||
if (parentElement) {
|
||||
parentElement.scrollIntoView({ behavior: 'smooth', block: 'center' });
|
||||
while ((node = walker.nextNode())) {
|
||||
const text = node.textContent || '';
|
||||
const idx = text.indexOf(placeholder);
|
||||
if (idx !== -1) {
|
||||
// Found the placeholder - scroll to it first
|
||||
const parentElement = node.parentElement;
|
||||
if (parentElement) {
|
||||
parentElement.scrollIntoView({ behavior: 'smooth', block: 'center' });
|
||||
}
|
||||
|
||||
// Select it
|
||||
const range = document.createRange();
|
||||
range.setStart(node, idx);
|
||||
range.setEnd(node, idx + placeholder.length);
|
||||
const sel = window.getSelection();
|
||||
sel.removeAllRanges();
|
||||
sel.addRange(range);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
})()`,
|
||||
}, { sessionId });
|
||||
|
||||
// Select it
|
||||
const range = document.createRange();
|
||||
range.setStart(node, idx);
|
||||
range.setEnd(node, idx + placeholder.length);
|
||||
const sel = window.getSelection();
|
||||
sel.removeAllRanges();
|
||||
sel.addRange(range);
|
||||
return true;
|
||||
}
|
||||
// Wait for scroll and selection to settle
|
||||
await sleep(800);
|
||||
|
||||
// Verify selection matches the placeholder
|
||||
const selectionCheck = await cdp!.send<{ result: { value: string } }>('Runtime.evaluate', {
|
||||
expression: `window.getSelection()?.toString() || ''`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
|
||||
const selectedText = selectionCheck.result.value.trim();
|
||||
if (selectedText === img.placeholder) {
|
||||
console.log(`[x-article] Selection verified: "${selectedText}"`);
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
})()`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
|
||||
// Wait for scroll animation
|
||||
await sleep(500);
|
||||
if (attempt < maxRetries) {
|
||||
console.log(`[x-article] Selection attempt ${attempt} got "${selectedText}", retrying...`);
|
||||
await sleep(500);
|
||||
} else {
|
||||
console.warn(`[x-article] Selection failed after ${maxRetries} attempts, got: "${selectedText}"`);
|
||||
}
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
if (!placeholderFound.result.value) {
|
||||
console.warn(`[x-article] Placeholder not found in DOM: ${img.placeholder}`);
|
||||
// Try to select the placeholder
|
||||
const selected = await selectPlaceholder(3);
|
||||
if (!selected) {
|
||||
console.warn(`[x-article] Skipping image - could not select placeholder: ${img.placeholder}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
console.log(`[x-article] Placeholder selected, copying image: ${path.basename(img.localPath)}`);
|
||||
console.log(`[x-article] Copying image: ${path.basename(img.localPath)}`);
|
||||
|
||||
// Copy image to clipboard
|
||||
if (!copyImageToClipboard(img.localPath)) {
|
||||
@@ -535,17 +561,48 @@ export async function publishArticle(options: ArticleOptions): Promise<void> {
|
||||
}
|
||||
|
||||
// Wait for clipboard to be fully ready
|
||||
await sleep(800);
|
||||
await sleep(1000);
|
||||
|
||||
// Delete placeholder by pressing Enter (placeholder is already selected)
|
||||
console.log(`[x-article] Deleting placeholder with Enter...`);
|
||||
await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13 }, { sessionId });
|
||||
await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13 }, { sessionId });
|
||||
// Delete placeholder by pressing Backspace (more reliable than Enter for replacing selection)
|
||||
console.log(`[x-article] Deleting placeholder...`);
|
||||
await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
|
||||
await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
|
||||
|
||||
// Wait and verify placeholder is deleted
|
||||
await sleep(500);
|
||||
|
||||
// Check that placeholder is no longer in editor
|
||||
const afterDelete = await cdp.send<{ result: { value: boolean } }>('Runtime.evaluate', {
|
||||
expression: `(() => {
|
||||
const editor = document.querySelector('.DraftEditor-editorContainer [data-contents="true"]');
|
||||
if (!editor) return true;
|
||||
return !editor.innerText.includes(${JSON.stringify(img.placeholder)});
|
||||
})()`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
|
||||
if (!afterDelete.result.value) {
|
||||
console.warn(`[x-article] Placeholder may not have been deleted, trying again...`);
|
||||
// Try selecting and deleting again
|
||||
await selectPlaceholder(1);
|
||||
await sleep(300);
|
||||
await cdp.send('Input.dispatchKeyEvent', { type: 'keyDown', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
|
||||
await cdp.send('Input.dispatchKeyEvent', { type: 'keyUp', key: 'Backspace', code: 'Backspace', windowsVirtualKeyCode: 8 }, { sessionId });
|
||||
await sleep(500);
|
||||
}
|
||||
|
||||
// Focus editor to ensure cursor is in position
|
||||
await cdp.send('Runtime.evaluate', {
|
||||
expression: `(() => {
|
||||
const editor = document.querySelector('.DraftEditor-editorContainer [contenteditable="true"]');
|
||||
if (editor) editor.focus();
|
||||
})()`,
|
||||
}, { sessionId });
|
||||
await sleep(300);
|
||||
|
||||
// Paste image using paste script (activates Chrome, sends real keystroke)
|
||||
console.log(`[x-article] Pasting image...`);
|
||||
if (pasteFromClipboard('Google Chrome', 5, 800)) {
|
||||
if (pasteFromClipboard('Google Chrome', 5, 1000)) {
|
||||
console.log(`[x-article] Image pasted: ${path.basename(img.localPath)}`);
|
||||
} else {
|
||||
console.warn(`[x-article] Failed to paste image after retries`);
|
||||
|
||||
@@ -232,14 +232,21 @@ Style notes: [specific style characteristics to emphasize]
|
||||
|
||||
### Step 5: Generate Images
|
||||
|
||||
**Session Management**:
|
||||
If the image generation skill supports `--sessionId`:
|
||||
1. Generate a unique session ID at the start (e.g., `slides-{topic-slug}-{timestamp}`)
|
||||
2. Use the same session ID for all slides
|
||||
3. This ensures visual consistency (color scheme, style, typography) across all slides
|
||||
|
||||
For each slide, generate using:
|
||||
|
||||
```bash
|
||||
/baoyu-gemini-web --promptfiles [SKILL_ROOT]/skills/baoyu-slide-deck/prompts/system.md [TARGET_DIR]/prompts/01-cover.md --image [TARGET_DIR]/01-cover.png
|
||||
# With session support
|
||||
/baoyu-gemini-web --promptfiles [SKILL_ROOT]/skills/baoyu-slide-deck/prompts/system.md [TARGET_DIR]/prompts/01-cover.md --image [TARGET_DIR]/01-cover.png --sessionId slides-topic-20260117
|
||||
```
|
||||
|
||||
Generation flow:
|
||||
1. Generate images sequentially
|
||||
1. Generate images sequentially with the same session ID
|
||||
2. After each image, output progress: "Generated X/N"
|
||||
3. On failure, auto-retry once
|
||||
4. If retry fails, log reason, continue to next
|
||||
|
||||
@@ -261,8 +261,14 @@ Style notes: [style-specific characteristics]
|
||||
1. Check available image generation skills
|
||||
2. If multiple skills available, ask user to choose
|
||||
|
||||
**Session Management**:
|
||||
If the image generation skill supports `--sessionId`:
|
||||
1. Generate a unique session ID at the start (e.g., `xhs-{topic-slug}-{timestamp}`)
|
||||
2. Use the same session ID for all images in the series
|
||||
3. This ensures style consistency across all generated images
|
||||
|
||||
**Generation Flow**:
|
||||
1. Call selected image generation skill with prompt file and output path
|
||||
1. Call selected image generation skill with prompt file, output path, and session ID
|
||||
2. Confirm generation success
|
||||
3. Report progress: "Generated X/N"
|
||||
4. Continue to next
|
||||
|
||||
Reference in New Issue
Block a user