mirror of
https://github.com/JimLiu/baoyu-skills.git
synced 2026-07-12 22:09:48 +08:00
Compare commits
26 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 94eab2de63 | |||
| 126b72ed36 | |||
| 38f4f253df | |||
| 8bc01debac | |||
| c5c54e26da | |||
| 2a0bba6161 | |||
| c44a524fa6 | |||
| 826535abe4 | |||
| fc50f31694 | |||
| 204765a137 | |||
| 4874cd2dae | |||
| b791ee5dc7 | |||
| 450c76d955 | |||
| db33da26e7 | |||
| c7c98ba034 | |||
| 60ab574559 | |||
| 8e2967d4a2 | |||
| 3a8b0cc158 | |||
| 9e3d72cf42 | |||
| 5eeb1e6d8d | |||
| 0ee6dd4305 | |||
| 7891f3c3c0 | |||
| 74f4a48ca7 | |||
| 881c03262e | |||
| 6afcfa80cc | |||
| 9eb032a22f |
@@ -6,7 +6,7 @@
|
||||
},
|
||||
"metadata": {
|
||||
"description": "Skills shared by Baoyu for improving daily work efficiency",
|
||||
"version": "1.88.0"
|
||||
"version": "1.91.0"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
|
||||
@@ -2,6 +2,42 @@
|
||||
|
||||
English | [中文](./CHANGELOG.zh.md)
|
||||
|
||||
## 1.90.1 - 2026-04-05
|
||||
|
||||
### Fixes
|
||||
- `baoyu-post-to-wechat`: detect actual image format from buffer magic bytes to fix CDN content-type mismatches (e.g. WebP served for .png URLs); treat WebP as PNG-preferred for transparency handling
|
||||
|
||||
## 1.89.1 - 2026-04-01
|
||||
|
||||
### Features
|
||||
- `baoyu-chrome-cdp`: add `gracefulKillChrome` that waits for Chrome to exit and release its port; fix `killChrome` to use `exitCode`/`signalCode` instead of `.killed` for reliable process state detection
|
||||
- `baoyu-fetch`: auto-detect login state before extraction in interaction wait mode
|
||||
|
||||
### Maintenance
|
||||
- Sync vendor baoyu-chrome-cdp across CDP skills
|
||||
- `baoyu-url-to-markdown`: sync vendor baoyu-fetch with login auto-detect
|
||||
|
||||
## 1.89.0 - 2026-03-31
|
||||
|
||||
### Features
|
||||
- `baoyu-fetch`: add X session cookie sidecar to persist login across runs, graceful Chrome shutdown via Browser.close, and stale profile lock auto-recovery
|
||||
- `baoyu-article-illustrator`: add warm palette variant for vector-illustration style with new `warm-knowledge` preset
|
||||
- `baoyu-post-to-x`: add X session persistence after login, Chrome lock recovery, and graceful shutdown
|
||||
|
||||
### Documentation
|
||||
- `baoyu-post-to-weibo`: add post type auto-selection rules and safer CDP kill instructions
|
||||
|
||||
### Refactor
|
||||
- `baoyu-danger-gemini-web`: use graceful Chrome shutdown instead of hard kill
|
||||
- `baoyu-danger-x-to-markdown`: use graceful Chrome shutdown instead of hard kill
|
||||
|
||||
### Fixes
|
||||
- Sync npm lockfile and root node tests
|
||||
|
||||
### Maintenance
|
||||
- `baoyu-url-to-markdown`: sync vendor baoyu-fetch with session and lifecycle changes
|
||||
- Update bun.lock files
|
||||
|
||||
## 1.88.0 - 2026-03-27
|
||||
|
||||
### Features
|
||||
|
||||
@@ -2,6 +2,42 @@
|
||||
|
||||
[English](./CHANGELOG.md) | 中文
|
||||
|
||||
## 1.90.1 - 2026-04-05
|
||||
|
||||
### 修复
|
||||
- `baoyu-post-to-wechat`:通过 magic bytes 检测实际图片格式,修复 CDN 返回与 URL 扩展名不一致的 content-type 问题(如 .png URL 实际返回 WebP);WebP 格式按 PNG 策略处理以保留透明度
|
||||
|
||||
## 1.89.1 - 2026-04-01
|
||||
|
||||
### 新功能
|
||||
- `baoyu-chrome-cdp`:新增 `gracefulKillChrome`,等待 Chrome 进程退出并释放端口;修复 `killChrome` 使用 `exitCode`/`signalCode` 替代 `.killed` 以更可靠地检测进程状态
|
||||
- `baoyu-fetch`:在交互等待模式下自动检测登录状态,未登录时提示用户先登录再提取内容
|
||||
|
||||
### 维护
|
||||
- 同步 vendor baoyu-chrome-cdp 至所有 CDP 技能
|
||||
- `baoyu-url-to-markdown`:同步 vendor baoyu-fetch 的登录自动检测功能
|
||||
|
||||
## 1.89.0 - 2026-03-31
|
||||
|
||||
### 新功能
|
||||
- `baoyu-fetch`:新增 X 会话 Cookie 旁路文件,跨运行持久化登录状态;通过 Browser.close 优雅关闭 Chrome;自动检测并清理过期的 Chrome 配置锁文件
|
||||
- `baoyu-article-illustrator`:新增暖色调矢量插画配色方案,含 `warm-knowledge` 预设
|
||||
- `baoyu-post-to-x`:新增登录后 X 会话持久化、Chrome 锁文件恢复和优雅关闭
|
||||
|
||||
### 文档
|
||||
- `baoyu-post-to-weibo`:新增发帖类型自动选择规则,优化 CDP Chrome 终止指令
|
||||
|
||||
### 重构
|
||||
- `baoyu-danger-gemini-web`:使用优雅 Chrome 关闭替代强制终止
|
||||
- `baoyu-danger-x-to-markdown`:使用优雅 Chrome 关闭替代强制终止
|
||||
|
||||
### 修复
|
||||
- 同步 npm lockfile 及修复根目录 Node 测试
|
||||
|
||||
### 维护
|
||||
- `baoyu-url-to-markdown`:同步 vendor baoyu-fetch 的会话和生命周期改进
|
||||
- 更新 bun.lock 文件
|
||||
|
||||
## 1.88.0 - 2026-03-27
|
||||
|
||||
### 新功能
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# CLAUDE.md
|
||||
|
||||
Claude Code marketplace plugin providing AI-powered content generation skills. Version: **1.88.0**.
|
||||
Claude Code marketplace plugin providing AI-powered content generation skills. Version: **1.91.0**.
|
||||
|
||||
## Architecture
|
||||
|
||||
|
||||
@@ -116,6 +116,10 @@ Xiaohongshu (RedNote) infographic series generator. Breaks down content into 1-1
|
||||
|
||||
# Direct content input
|
||||
/baoyu-xhs-images 今日星座运势
|
||||
|
||||
# Non-interactive (skip all confirmations, for scheduled tasks)
|
||||
/baoyu-xhs-images posts/ai-future/article.md --yes
|
||||
/baoyu-xhs-images posts/ai-future/article.md --yes --preset knowledge-card
|
||||
```
|
||||
|
||||
**Styles** (visual aesthetics): `cute` (default), `fresh`, `warm`, `bold`, `minimal`, `retro`, `pop`, `notion`, `chalkboard`
|
||||
|
||||
@@ -116,6 +116,10 @@ clawhub install baoyu-markdown-to-html
|
||||
|
||||
# 直接输入内容
|
||||
/baoyu-xhs-images 今日星座运势
|
||||
|
||||
# 非交互模式(跳过所有确认,适用于定时任务)
|
||||
/baoyu-xhs-images posts/ai-future/article.md --yes
|
||||
/baoyu-xhs-images posts/ai-future/article.md --yes --preset knowledge-card
|
||||
```
|
||||
|
||||
**风格**(视觉美学):`cute`(默认)、`fresh`、`warm`、`bold`、`minimal`、`retro`、`pop`、`notion`、`chalkboard`
|
||||
|
||||
Generated
+2101
-9
File diff suppressed because it is too large
Load Diff
+5
-2
@@ -6,8 +6,8 @@
|
||||
"packages/*"
|
||||
],
|
||||
"scripts": {
|
||||
"test": "node --import tsx --test",
|
||||
"test:coverage": "node --import tsx --experimental-test-coverage --test"
|
||||
"test": "node ./scripts/run-node-tests.mjs",
|
||||
"test:coverage": "node ./scripts/run-node-tests.mjs --experimental-test-coverage"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@mozilla/readability": "^0.6.0",
|
||||
@@ -15,5 +15,8 @@
|
||||
"turndown": "^7.2.2",
|
||||
"turndown-plugin-gfm": "^1.0.2",
|
||||
"tsx": "^4.20.5"
|
||||
},
|
||||
"dependencies": {
|
||||
"pdf-lib": "^1.17.1"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
discoverRunningChromeDebugPort,
|
||||
findChromeExecutable,
|
||||
findExistingChromeDebugPort,
|
||||
gracefulKillChrome,
|
||||
getFreePort,
|
||||
openPageSession,
|
||||
resolveSharedChromeProfileDir,
|
||||
@@ -110,6 +111,44 @@ async function stopProcess(child: ChildProcess | null): Promise<void> {
|
||||
await new Promise((resolve) => child.once("exit", resolve));
|
||||
}
|
||||
|
||||
async function startPortHoldingProcess(port: number): Promise<ChildProcess> {
|
||||
const child = spawn(
|
||||
process.execPath,
|
||||
[
|
||||
"-e",
|
||||
`
|
||||
const http = require("node:http");
|
||||
const port = Number(process.argv[1]);
|
||||
const server = http.createServer((_req, res) => res.end("ok"));
|
||||
server.listen(port, "127.0.0.1", () => process.stdout.write("ready\\n"));
|
||||
setInterval(() => {}, 1000);
|
||||
`,
|
||||
String(port),
|
||||
],
|
||||
{
|
||||
stdio: ["ignore", "pipe", "ignore"],
|
||||
},
|
||||
);
|
||||
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const timer = setTimeout(() => reject(new Error("Timed out waiting for child server to start.")), 3_000);
|
||||
child.once("error", (error) => {
|
||||
clearTimeout(timer);
|
||||
reject(error);
|
||||
});
|
||||
child.stdout?.once("data", () => {
|
||||
clearTimeout(timer);
|
||||
resolve();
|
||||
});
|
||||
child.once("exit", () => {
|
||||
clearTimeout(timer);
|
||||
reject(new Error("Child server exited before becoming ready."));
|
||||
});
|
||||
});
|
||||
|
||||
return child;
|
||||
}
|
||||
|
||||
test("getFreePort honors a fixed environment override and otherwise allocates a TCP port", async (t) => {
|
||||
useEnv(t, { TEST_FIXED_PORT: "45678" });
|
||||
assert.equal(await getFreePort("TEST_FIXED_PORT"), 45678);
|
||||
@@ -305,3 +344,19 @@ test("waitForChromeDebugPort retries until the debug endpoint becomes available"
|
||||
|
||||
assert.equal(websocketUrl, `ws://127.0.0.1:${port}/devtools/browser/demo`);
|
||||
});
|
||||
|
||||
test("gracefulKillChrome waits for the Chrome process to exit and release its port", async (t) => {
|
||||
const port = await getFreePort();
|
||||
const child = await startPortHoldingProcess(port);
|
||||
t.after(async () => { await stopProcess(child); });
|
||||
|
||||
assert.equal(await waitForChromeDebugPort(port, 1_000).catch(() => null), null);
|
||||
|
||||
await gracefulKillChrome(child, port, 4_000);
|
||||
|
||||
assert.ok(child.exitCode !== null || child.signalCode !== null);
|
||||
assert.equal(
|
||||
await fetch(`http://127.0.0.1:${port}`).then(() => true).catch(() => false),
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -478,7 +478,7 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
chrome.kill("SIGTERM");
|
||||
} catch {}
|
||||
setTimeout(() => {
|
||||
if (!chrome.killed) {
|
||||
if (chrome.exitCode === null && chrome.signalCode === null) {
|
||||
try {
|
||||
chrome.kill("SIGKILL");
|
||||
} catch {}
|
||||
@@ -486,6 +486,37 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
}, 2_000).unref?.();
|
||||
}
|
||||
|
||||
export async function gracefulKillChrome(
|
||||
chrome: ChildProcess,
|
||||
port?: number,
|
||||
timeoutMs = 6_000,
|
||||
): Promise<void> {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
|
||||
const exitPromise = new Promise<void>((resolve) => {
|
||||
chrome.once("exit", () => resolve());
|
||||
});
|
||||
|
||||
killChrome(chrome);
|
||||
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
if (port !== undefined && !await isPortListening(port, 250)) return;
|
||||
|
||||
const exited = await Promise.race([
|
||||
exitPromise.then(() => true),
|
||||
sleep(100).then(() => false),
|
||||
]);
|
||||
if (exited) return;
|
||||
}
|
||||
|
||||
await Promise.race([
|
||||
exitPromise,
|
||||
sleep(250),
|
||||
]);
|
||||
}
|
||||
|
||||
export async function openPageSession(options: OpenPageSessionOptions): Promise<PageSession> {
|
||||
let targetId: string;
|
||||
let createdTarget = false;
|
||||
|
||||
@@ -2,7 +2,12 @@ import { afterEach, describe, expect, test } from "bun:test";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
import { ensureChromeProfileDir, resolveChromeProfileDir } from "../browser/profile";
|
||||
import {
|
||||
ensureChromeProfileDir,
|
||||
hasChromeLockArtifacts,
|
||||
resolveChromeProfileDir,
|
||||
shouldRetryChromeLaunchRecovery,
|
||||
} from "../browser/profile";
|
||||
|
||||
const originalProfile = process.env.BAOYU_CHROME_PROFILE_DIR;
|
||||
|
||||
@@ -47,3 +52,17 @@ describe("ensureChromeProfileDir", () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("stale lock recovery helpers", () => {
|
||||
test("detects Chrome singleton lock artifacts", () => {
|
||||
expect(hasChromeLockArtifacts(["Cookies", "SingletonLock"])).toBe(true);
|
||||
expect(hasChromeLockArtifacts(["chrome.pid"])).toBe(true);
|
||||
expect(hasChromeLockArtifacts(["Preferences", "Cookies"])).toBe(false);
|
||||
});
|
||||
|
||||
test("only retries stale-lock recovery when no live owner exists", () => {
|
||||
expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: true, hasLiveOwner: false })).toBe(true);
|
||||
expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: true, hasLiveOwner: true })).toBe(false);
|
||||
expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: false, hasLiveOwner: false })).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { shouldAutoContinueForceWait } from "../commands/convert";
|
||||
import { shouldAutoContinueForceWait, shouldKeepBrowserOpenAfterInteraction } from "../commands/convert";
|
||||
|
||||
describe("shouldAutoContinueForceWait", () => {
|
||||
test("continues when a challenge disappears", () => {
|
||||
@@ -9,28 +9,49 @@ describe("shouldAutoContinueForceWait", () => {
|
||||
url: "https://example.com/challenge",
|
||||
hasGate: true,
|
||||
loginState: "unknown",
|
||||
sessionReady: true,
|
||||
},
|
||||
{
|
||||
url: "https://example.com/article",
|
||||
hasGate: false,
|
||||
loginState: "unknown",
|
||||
sessionReady: true,
|
||||
},
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
test("continues when login state improves from logged out", () => {
|
||||
test("waits for X session cookies before continuing after login", () => {
|
||||
expect(
|
||||
shouldAutoContinueForceWait(
|
||||
{
|
||||
url: "https://x.com/i/flow/login",
|
||||
hasGate: false,
|
||||
loginState: "logged_out",
|
||||
sessionReady: false,
|
||||
},
|
||||
{
|
||||
url: "https://x.com/home",
|
||||
hasGate: false,
|
||||
loginState: "logged_in",
|
||||
sessionReady: false,
|
||||
},
|
||||
),
|
||||
).toBe(false);
|
||||
|
||||
expect(
|
||||
shouldAutoContinueForceWait(
|
||||
{
|
||||
url: "https://x.com/i/flow/login",
|
||||
hasGate: false,
|
||||
loginState: "logged_out",
|
||||
sessionReady: false,
|
||||
},
|
||||
{
|
||||
url: "https://x.com/home",
|
||||
hasGate: false,
|
||||
loginState: "logged_in",
|
||||
sessionReady: true,
|
||||
},
|
||||
),
|
||||
).toBe(true);
|
||||
@@ -43,13 +64,42 @@ describe("shouldAutoContinueForceWait", () => {
|
||||
url: "https://x.com/lennysan/status/2036483059407810640",
|
||||
hasGate: false,
|
||||
loginState: "unknown",
|
||||
sessionReady: false,
|
||||
},
|
||||
{
|
||||
url: "https://x.com/lennysan/status/2036483059407810640",
|
||||
hasGate: false,
|
||||
loginState: "unknown",
|
||||
sessionReady: false,
|
||||
},
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("shouldKeepBrowserOpenAfterInteraction", () => {
|
||||
test("keeps launched X login browsers open", () => {
|
||||
expect(
|
||||
shouldKeepBrowserOpenAfterInteraction({
|
||||
launched: true,
|
||||
interaction: { kind: "login", provider: "x" },
|
||||
}),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
test("does not keep reused or non-login browsers open", () => {
|
||||
expect(
|
||||
shouldKeepBrowserOpenAfterInteraction({
|
||||
launched: false,
|
||||
interaction: { kind: "login", provider: "x" },
|
||||
}),
|
||||
).toBe(false);
|
||||
|
||||
expect(
|
||||
shouldKeepBrowserOpenAfterInteraction({
|
||||
launched: true,
|
||||
interaction: { kind: "cloudflare", provider: "cloudflare" },
|
||||
}),
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { buildXSessionCookieMap, hasRequiredXSessionCookies } from "../adapters/x/session";
|
||||
|
||||
describe("X session helpers", () => {
|
||||
test("keeps non-empty X session cookies", () => {
|
||||
expect(
|
||||
buildXSessionCookieMap([
|
||||
{ name: "auth_token", value: "auth" },
|
||||
{ name: "ct0", value: "csrf" },
|
||||
{ name: "twid", value: "u=123" },
|
||||
{ name: "ct0", value: "" },
|
||||
{ name: "", value: "ignored" },
|
||||
{ name: "gt", value: undefined },
|
||||
]),
|
||||
).toEqual({
|
||||
auth_token: "auth",
|
||||
ct0: "csrf",
|
||||
twid: "u=123",
|
||||
});
|
||||
});
|
||||
|
||||
test("requires auth_token and ct0 for a ready X session", () => {
|
||||
expect(hasRequiredXSessionCookies({ auth_token: "auth" })).toBe(false);
|
||||
expect(hasRequiredXSessionCookies({ ct0: "csrf" })).toBe(false);
|
||||
expect(hasRequiredXSessionCookies({ auth_token: "auth", ct0: "csrf" })).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -64,6 +64,8 @@ export interface Adapter {
|
||||
name: string;
|
||||
match(input: AdapterInput): boolean;
|
||||
checkLogin?(context: AdapterContext): Promise<AdapterLoginInfo>;
|
||||
exportCookies?(context: AdapterContext, profileDir?: string): Promise<boolean>;
|
||||
restoreCookies?(context: AdapterContext, profileDir?: string): Promise<boolean>;
|
||||
downloadMedia?(request: MediaDownloadRequest): Promise<MediaDownloadResult>;
|
||||
process(context: AdapterContext): Promise<AdapterProcessResult>;
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { Adapter, AdapterLoginInfo } from "../types";
|
||||
import { exportCookies, restoreCookies, type CookieSidecarConfig } from "../../browser/cookie-sidecar";
|
||||
import { detectInteractionGate } from "../../browser/interaction-gates";
|
||||
import type { ExtractedDocument } from "../../extract/document";
|
||||
import { collectMediaFromDocument } from "../../media/markdown-media";
|
||||
@@ -10,6 +11,16 @@ import { extractSingleTweetDocumentFromPayload } from "./single";
|
||||
import { extractThreadDocumentFromPayloads } from "./thread";
|
||||
import { loadFullXThread } from "./thread-loader";
|
||||
|
||||
const cookieConfig: CookieSidecarConfig = {
|
||||
urls: ["https://x.com/", "https://twitter.com/"],
|
||||
filename: "x-session-cookies.json",
|
||||
requiredCookieNames: ["auth_token", "ct0"],
|
||||
filterCookie: (c) => {
|
||||
const d = c.domain ?? "";
|
||||
return d.endsWith("x.com") || d.endsWith("twitter.com");
|
||||
},
|
||||
};
|
||||
|
||||
function extractDocumentFromPayloads(
|
||||
payloads: unknown[],
|
||||
statusId: string,
|
||||
@@ -49,6 +60,12 @@ export const xAdapter: Adapter = {
|
||||
async checkLogin(context) {
|
||||
return detectXLogin(context);
|
||||
},
|
||||
async exportCookies(context, profileDir) {
|
||||
return exportCookies(context.browser.targetSession, cookieConfig, profileDir);
|
||||
},
|
||||
async restoreCookies(context, profileDir) {
|
||||
return restoreCookies(context.browser.targetSession, cookieConfig, profileDir);
|
||||
},
|
||||
async process(context) {
|
||||
const statusId = extractStatusId(context.input.url);
|
||||
if (!statusId) {
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
import type { AdapterContext } from "../types";
|
||||
|
||||
const X_SESSION_URLS = ["https://x.com/", "https://twitter.com/"] as const;
|
||||
const REQUIRED_X_SESSION_COOKIES = ["auth_token", "ct0"] as const;
|
||||
|
||||
interface CookieLike {
|
||||
name?: string;
|
||||
value?: string | null;
|
||||
}
|
||||
|
||||
interface NetworkGetCookiesResult {
|
||||
cookies?: CookieLike[];
|
||||
}
|
||||
|
||||
export function buildXSessionCookieMap(cookies: readonly CookieLike[]): Record<string, string> {
|
||||
const cookieMap: Record<string, string> = {};
|
||||
for (const cookie of cookies) {
|
||||
const name = cookie.name?.trim();
|
||||
const value = cookie.value?.trim();
|
||||
if (!name || !value) {
|
||||
continue;
|
||||
}
|
||||
cookieMap[name] = value;
|
||||
}
|
||||
return cookieMap;
|
||||
}
|
||||
|
||||
export function hasRequiredXSessionCookies(cookieMap: Record<string, string>): boolean {
|
||||
return REQUIRED_X_SESSION_COOKIES.every((name) => Boolean(cookieMap[name]));
|
||||
}
|
||||
|
||||
export async function readXSessionCookieMap(
|
||||
context: Pick<AdapterContext, "browser">,
|
||||
): Promise<Record<string, string>> {
|
||||
const { cookies } = await context.browser.targetSession.send<NetworkGetCookiesResult>(
|
||||
"Network.getCookies",
|
||||
{ urls: [...X_SESSION_URLS] },
|
||||
);
|
||||
return buildXSessionCookieMap(cookies ?? []);
|
||||
}
|
||||
|
||||
export async function isXSessionReady(
|
||||
context: Pick<AdapterContext, "browser">,
|
||||
): Promise<boolean> {
|
||||
const cookieMap = await readXSessionCookieMap(context);
|
||||
return hasRequiredXSessionCookies(cookieMap);
|
||||
}
|
||||
@@ -1,6 +1,16 @@
|
||||
import { launch, type LaunchedChrome } from "chrome-launcher";
|
||||
import WebSocket from "ws";
|
||||
import type { Logger } from "../utils/logger";
|
||||
import { ensureChromeProfileDir, findExistingChromeDebugPort, resolveChromeProfileDir } from "./profile";
|
||||
import {
|
||||
cleanChromeLockArtifacts,
|
||||
ensureChromeProfileDir,
|
||||
findChromeProcessUsingProfile,
|
||||
findExistingChromeDebugPort,
|
||||
hasChromeLockArtifacts,
|
||||
listChromeProfileEntries,
|
||||
resolveChromeProfileDir,
|
||||
shouldRetryChromeLaunchRecovery,
|
||||
} from "./profile";
|
||||
|
||||
interface ChromeVersionResponse {
|
||||
webSocketDebuggerUrl: string;
|
||||
@@ -65,6 +75,80 @@ async function tryReuseChrome(profileDir: string, logger?: Logger): Promise<Chro
|
||||
return null;
|
||||
}
|
||||
|
||||
async function launchFreshChrome(
|
||||
profileDir: string,
|
||||
options: Pick<ChromeConnectOptions, "browserPath" | "headless">,
|
||||
): Promise<ChromeConnection> {
|
||||
let launchedChrome: LaunchedChrome | null = null;
|
||||
try {
|
||||
launchedChrome = await launch({
|
||||
chromePath: options.browserPath,
|
||||
userDataDir: profileDir,
|
||||
chromeFlags: [
|
||||
"--disable-background-networking",
|
||||
"--disable-default-apps",
|
||||
"--disable-popup-blocking",
|
||||
"--disable-sync",
|
||||
"--no-first-run",
|
||||
"--no-default-browser-check",
|
||||
"--remote-allow-origins=*",
|
||||
...(!options.headless ? ["--no-startup-window"] : []),
|
||||
...(options.headless ? ["--headless=new"] : []),
|
||||
],
|
||||
});
|
||||
|
||||
const origin = `http://127.0.0.1:${launchedChrome.port}`;
|
||||
const version = await fetchJson<ChromeVersionResponse>(`${origin}/json/version`);
|
||||
|
||||
const chrome = launchedChrome;
|
||||
return {
|
||||
browserWsUrl: version.webSocketDebuggerUrl,
|
||||
origin,
|
||||
port: launchedChrome.port,
|
||||
profileDir,
|
||||
launched: true,
|
||||
async close() {
|
||||
if (!chrome) return;
|
||||
await gracefulCloseChrome(chrome, origin);
|
||||
},
|
||||
};
|
||||
} catch (error) {
|
||||
launchedChrome?.kill();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function gracefulCloseChrome(chrome: LaunchedChrome, origin: string): Promise<void> {
|
||||
try {
|
||||
const resp = await fetch(`${origin}/json/version`);
|
||||
const { webSocketDebuggerUrl } = (await resp.json()) as ChromeVersionResponse;
|
||||
if (webSocketDebuggerUrl) {
|
||||
const ws = await new Promise<WebSocket>((resolve, reject) => {
|
||||
const socket = new WebSocket(webSocketDebuggerUrl);
|
||||
socket.once("open", () => resolve(socket));
|
||||
socket.once("error", reject);
|
||||
});
|
||||
const id = 1;
|
||||
ws.send(JSON.stringify({ id, method: "Browser.close" }));
|
||||
await new Promise<void>((resolve) => {
|
||||
const timer = setTimeout(() => { ws.close(); resolve(); }, 5_000);
|
||||
ws.once("close", () => { clearTimeout(timer); resolve(); });
|
||||
});
|
||||
const exited = await new Promise<boolean>((resolve) => {
|
||||
if (chrome.pid && !isProcessAlive(chrome.pid)) { resolve(true); return; }
|
||||
const timer = setTimeout(() => resolve(false), 3_000);
|
||||
chrome.process.once("exit", () => { clearTimeout(timer); resolve(true); });
|
||||
});
|
||||
if (exited) return;
|
||||
}
|
||||
} catch {}
|
||||
chrome.kill();
|
||||
}
|
||||
|
||||
function isProcessAlive(pid: number): boolean {
|
||||
try { process.kill(pid, 0); return true; } catch { return false; }
|
||||
}
|
||||
|
||||
export async function connectChrome(options: ChromeConnectOptions): Promise<ChromeConnection> {
|
||||
if (options.cdpUrl) {
|
||||
if (options.cdpUrl.startsWith("ws://") || options.cdpUrl.startsWith("wss://")) {
|
||||
@@ -84,34 +168,20 @@ export async function connectChrome(options: ChromeConnectOptions): Promise<Chro
|
||||
}
|
||||
|
||||
options.logger?.warn(`No running Chrome debugger found for profile ${profileDir}. Launching Chrome with that profile.`);
|
||||
try {
|
||||
return await launchFreshChrome(profileDir, options);
|
||||
} catch (error) {
|
||||
const entries = await listChromeProfileEntries(profileDir);
|
||||
const shouldRetry = shouldRetryChromeLaunchRecovery({
|
||||
hasLockArtifacts: hasChromeLockArtifacts(entries),
|
||||
hasLiveOwner: findChromeProcessUsingProfile(profileDir),
|
||||
});
|
||||
if (!shouldRetry) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
const launchedChrome: LaunchedChrome = await launch({
|
||||
chromePath: options.browserPath,
|
||||
userDataDir: profileDir,
|
||||
chromeFlags: [
|
||||
"--disable-background-networking",
|
||||
"--disable-default-apps",
|
||||
"--disable-popup-blocking",
|
||||
"--disable-sync",
|
||||
"--no-first-run",
|
||||
"--no-default-browser-check",
|
||||
"--remote-allow-origins=*",
|
||||
...(!options.headless ? ["--no-startup-window"] : []),
|
||||
...(options.headless ? ["--headless=new"] : []),
|
||||
],
|
||||
});
|
||||
|
||||
const origin = `http://127.0.0.1:${launchedChrome.port}`;
|
||||
const version = await fetchJson<ChromeVersionResponse>(`${origin}/json/version`);
|
||||
|
||||
return {
|
||||
browserWsUrl: version.webSocketDebuggerUrl,
|
||||
origin,
|
||||
port: launchedChrome.port,
|
||||
profileDir,
|
||||
launched: true,
|
||||
async close() {
|
||||
launchedChrome.kill();
|
||||
},
|
||||
};
|
||||
options.logger?.warn(`Chrome launch failed with stale profile locks. Cleaning ${profileDir} and retrying once.`);
|
||||
cleanChromeLockArtifacts(profileDir);
|
||||
return await launchFreshChrome(profileDir, options);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
import { readFile, writeFile, mkdir } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { resolveChromeProfileDir } from "./profile";
|
||||
import type { TargetSession } from "./cdp-client";
|
||||
|
||||
export interface CdpCookie {
|
||||
name: string;
|
||||
value: string;
|
||||
domain: string;
|
||||
path: string;
|
||||
expires: number;
|
||||
size: number;
|
||||
httpOnly: boolean;
|
||||
secure: boolean;
|
||||
session: boolean;
|
||||
sameSite?: string;
|
||||
priority?: string;
|
||||
sameParty?: boolean;
|
||||
sourceScheme?: string;
|
||||
sourcePort?: number;
|
||||
partitionKey?: string;
|
||||
}
|
||||
|
||||
interface SidecarData {
|
||||
savedAt: string;
|
||||
cookies: CdpCookie[];
|
||||
}
|
||||
|
||||
export interface CookieSidecarConfig {
|
||||
urls: readonly string[];
|
||||
filename: string;
|
||||
requiredCookieNames: readonly string[];
|
||||
filterCookie?: (cookie: CdpCookie) => boolean;
|
||||
}
|
||||
|
||||
function sidecarPath(filename: string, profileDir?: string): string {
|
||||
return join(resolveChromeProfileDir(profileDir), filename);
|
||||
}
|
||||
|
||||
function hasRequired(cookies: CdpCookie[], names: readonly string[]): boolean {
|
||||
return names.every((name) =>
|
||||
cookies.some((c) => c.name === name && Boolean(c.value)),
|
||||
);
|
||||
}
|
||||
|
||||
async function getCookies(session: TargetSession, urls: readonly string[]): Promise<CdpCookie[]> {
|
||||
const { cookies } = await session.send<{ cookies: CdpCookie[] }>(
|
||||
"Network.getCookies",
|
||||
{ urls: [...urls] },
|
||||
);
|
||||
return cookies ?? [];
|
||||
}
|
||||
|
||||
export async function exportCookies(
|
||||
session: TargetSession,
|
||||
config: CookieSidecarConfig,
|
||||
profileDir?: string,
|
||||
): Promise<boolean> {
|
||||
const all = await getCookies(session, config.urls);
|
||||
const filtered = config.filterCookie ? all.filter(config.filterCookie) : all;
|
||||
if (!hasRequired(filtered, config.requiredCookieNames)) return false;
|
||||
|
||||
const filePath = sidecarPath(config.filename, profileDir);
|
||||
await mkdir(dirname(filePath), { recursive: true });
|
||||
const data: SidecarData = { savedAt: new Date().toISOString(), cookies: filtered };
|
||||
await writeFile(filePath, JSON.stringify(data, null, 2));
|
||||
return true;
|
||||
}
|
||||
|
||||
export async function restoreCookies(
|
||||
session: TargetSession,
|
||||
config: CookieSidecarConfig,
|
||||
profileDir?: string,
|
||||
): Promise<boolean> {
|
||||
const live = await getCookies(session, config.urls);
|
||||
if (hasRequired(live, config.requiredCookieNames)) return false;
|
||||
|
||||
const filePath = sidecarPath(config.filename, profileDir);
|
||||
const raw = await readFile(filePath, "utf8");
|
||||
const data = JSON.parse(raw) as SidecarData;
|
||||
if (!data.cookies?.length) return false;
|
||||
|
||||
const now = Date.now() / 1000;
|
||||
const valid = data.cookies.filter((c) => c.session || !c.expires || c.expires > now);
|
||||
if (!hasRequired(valid, config.requiredCookieNames)) return false;
|
||||
|
||||
await session.send("Network.setCookies", {
|
||||
cookies: valid.map((c) => ({
|
||||
name: c.name,
|
||||
value: c.value,
|
||||
domain: c.domain,
|
||||
path: c.path,
|
||||
httpOnly: c.httpOnly,
|
||||
secure: c.secure,
|
||||
sameSite: c.sameSite,
|
||||
expires: c.expires,
|
||||
})),
|
||||
});
|
||||
return true;
|
||||
}
|
||||
@@ -19,6 +19,8 @@ interface ChromeVersionResponse {
|
||||
webSocketDebuggerUrl?: string;
|
||||
}
|
||||
|
||||
const CHROME_LOCK_FILE_NAMES = ["SingletonLock", "SingletonSocket", "SingletonCookie", "chrome.pid"] as const;
|
||||
|
||||
function resolveDataBaseDir(): string {
|
||||
if (process.platform === "darwin") {
|
||||
return path.join(os.homedir(), "Library", "Application Support");
|
||||
@@ -61,6 +63,57 @@ export function ensureChromeProfileDir(profileDir: string): string {
|
||||
return profileDir;
|
||||
}
|
||||
|
||||
export function hasChromeLockArtifacts(entries: readonly string[]): boolean {
|
||||
return CHROME_LOCK_FILE_NAMES.some((name) => entries.includes(name));
|
||||
}
|
||||
|
||||
export function shouldRetryChromeLaunchRecovery(options: {
|
||||
hasLockArtifacts: boolean;
|
||||
hasLiveOwner: boolean;
|
||||
}): boolean {
|
||||
return options.hasLockArtifacts && !options.hasLiveOwner;
|
||||
}
|
||||
|
||||
export function findChromeProcessUsingProfile(profileDir: string): boolean {
|
||||
if (process.platform === "win32") {
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
const result = spawnSync("ps", ["aux"], {
|
||||
encoding: "utf8",
|
||||
timeout: 5_000,
|
||||
});
|
||||
if (result.status !== 0 || !result.stdout) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return result.stdout
|
||||
.split("\n")
|
||||
.some((line) => line.includes(`--user-data-dir=${profileDir}`));
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export function cleanChromeLockArtifacts(profileDir: string): void {
|
||||
for (const name of CHROME_LOCK_FILE_NAMES) {
|
||||
try {
|
||||
fs.unlinkSync(path.join(profileDir, name));
|
||||
} catch {
|
||||
// Ignore missing files and continue cleaning the remaining artifacts.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export async function listChromeProfileEntries(profileDir: string): Promise<string[]> {
|
||||
try {
|
||||
return await fs.promises.readdir(profileDir);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchWithTimeout(url: string, timeoutMs = 3_000): Promise<Response> {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
@@ -7,6 +7,7 @@ import { detectInteractionGate } from "../browser/interaction-gates";
|
||||
import { NetworkJournal } from "../browser/network-journal";
|
||||
import { BrowserSession } from "../browser/session";
|
||||
import { genericAdapter, resolveAdapter } from "../adapters";
|
||||
import { isXSessionReady } from "../adapters/x/session";
|
||||
import type { ExtractedDocument } from "../extract/document";
|
||||
import { renderMarkdown } from "../extract/markdown-renderer";
|
||||
import { downloadMediaAssets } from "../media/default-downloader";
|
||||
@@ -55,6 +56,7 @@ interface ForceWaitSnapshot {
|
||||
url: string;
|
||||
hasGate: boolean;
|
||||
loginState: LoginState | "unavailable";
|
||||
sessionReady: boolean;
|
||||
}
|
||||
|
||||
interface SuccessfulConvertOutput {
|
||||
@@ -78,6 +80,17 @@ function sleep(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
function isForceWaitSessionReady(snapshot: ForceWaitSnapshot): boolean {
|
||||
return snapshot.sessionReady;
|
||||
}
|
||||
|
||||
export function shouldKeepBrowserOpenAfterInteraction(options: {
|
||||
launched: boolean;
|
||||
interaction: Pick<WaitForInteractionRequest, "kind" | "provider">;
|
||||
}): boolean {
|
||||
return options.launched && options.interaction.kind === "login" && options.interaction.provider === "x";
|
||||
}
|
||||
|
||||
export function shouldAutoContinueForceWait(
|
||||
initial: ForceWaitSnapshot,
|
||||
current: ForceWaitSnapshot,
|
||||
@@ -86,15 +99,20 @@ export function shouldAutoContinueForceWait(
|
||||
return true;
|
||||
}
|
||||
|
||||
if (initial.loginState === "logged_out" && current.loginState !== "logged_out") {
|
||||
if (initial.loginState === "logged_out" && current.loginState !== "logged_out" && isForceWaitSessionReady(current)) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if (initial.loginState !== "logged_in" && current.loginState === "logged_in") {
|
||||
if (initial.loginState !== "logged_in" && current.loginState === "logged_in" && isForceWaitSessionReady(current)) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if (current.url !== initial.url && !current.hasGate && current.loginState !== "logged_out") {
|
||||
if (
|
||||
current.url !== initial.url &&
|
||||
!current.hasGate &&
|
||||
current.loginState !== "logged_out" &&
|
||||
isForceWaitSessionReady(current)
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -173,6 +191,16 @@ async function closeRuntime(runtime: RuntimeResources | null | undefined): Promi
|
||||
await runtime.chrome.close().catch(() => {});
|
||||
}
|
||||
|
||||
async function isInteractionSessionReady(
|
||||
context: AdapterContext,
|
||||
interaction: WaitForInteractionRequest,
|
||||
): Promise<boolean> {
|
||||
if (interaction.provider !== "x") {
|
||||
return true;
|
||||
}
|
||||
return await isXSessionReady(context).catch(() => false);
|
||||
}
|
||||
|
||||
async function reopenInteractiveRuntime(
|
||||
runtime: RuntimeResources,
|
||||
options: ConvertCommandOptions,
|
||||
@@ -203,6 +231,7 @@ async function captureForceWaitSnapshot(
|
||||
url,
|
||||
hasGate: Boolean(gate),
|
||||
loginState: login?.state ?? "unavailable",
|
||||
sessionReady: adapter.name === "x" ? await isXSessionReady(context).catch(() => false) : true,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -280,7 +309,7 @@ async function waitForInteraction(
|
||||
while (Date.now() - startedAt < timeoutMs) {
|
||||
if (interaction.kind === "login" && adapter.checkLogin) {
|
||||
lastLogin = await adapter.checkLogin(context);
|
||||
if (lastLogin.state === "logged_in") {
|
||||
if (lastLogin.state === "logged_in" && await isInteractionSessionReady(context, interaction)) {
|
||||
return lastLogin;
|
||||
}
|
||||
}
|
||||
@@ -303,7 +332,7 @@ async function waitForInteraction(
|
||||
}
|
||||
|
||||
lastLogin = await adapter.checkLogin(context);
|
||||
if (lastLogin.state !== "logged_out") {
|
||||
if (lastLogin.state !== "logged_out" && await isInteractionSessionReady(context, interaction)) {
|
||||
return lastLogin;
|
||||
}
|
||||
}
|
||||
@@ -347,10 +376,13 @@ export async function runConvertCommand(options: ConvertCommandOptions): Promise
|
||||
const url = normalizeUrl(options.url);
|
||||
let runtime = await openRuntime(options, options.waitMode !== "none", Boolean(options.debugDir));
|
||||
const logger = createLogger(Boolean(options.debugDir));
|
||||
let didLogin = false;
|
||||
let adapter: Adapter | null = null;
|
||||
let context: AdapterContext | null = null;
|
||||
|
||||
try {
|
||||
const adapter = resolveAdapter({ url }, options.adapter);
|
||||
let context: AdapterContext = {
|
||||
adapter = resolveAdapter({ url }, options.adapter);
|
||||
context = {
|
||||
input: { url },
|
||||
browser: runtime.browser,
|
||||
network: runtime.network,
|
||||
@@ -362,6 +394,27 @@ export async function runConvertCommand(options: ConvertCommandOptions): Promise
|
||||
downloadMedia: options.downloadMedia,
|
||||
};
|
||||
|
||||
if (adapter.restoreCookies) {
|
||||
const restored = await adapter.restoreCookies(context, runtime.chrome.profileDir).catch(() => false);
|
||||
if (restored) logger.info(`Restored ${adapter.name} session cookies from sidecar.`);
|
||||
}
|
||||
|
||||
if (options.waitMode === "interaction" && adapter.checkLogin) {
|
||||
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
|
||||
const preLogin = await adapter.checkLogin(context);
|
||||
if (preLogin.state !== "logged_in") {
|
||||
didLogin = true;
|
||||
await waitForInteraction(adapter, context, {
|
||||
type: "wait_for_interaction",
|
||||
kind: "login",
|
||||
provider: preLogin.provider ?? adapter.name,
|
||||
prompt: `Please sign in to ${adapter.name === "x" ? "X" : adapter.name} in the opened Chrome window. Extraction will continue automatically once login is detected.`,
|
||||
reason: preLogin.reason ?? `Not logged in to ${adapter.name}`,
|
||||
requiresVisibleBrowser: true,
|
||||
}, options);
|
||||
}
|
||||
}
|
||||
|
||||
if (options.waitMode === "force") {
|
||||
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
|
||||
await waitForForceResume(adapter, context, options);
|
||||
@@ -414,6 +467,9 @@ export async function runConvertCommand(options: ConvertCommandOptions): Promise
|
||||
};
|
||||
|
||||
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
|
||||
if (result.interaction.kind === "login") {
|
||||
didLogin = true;
|
||||
}
|
||||
await waitForInteraction(adapter, context, result.interaction, options);
|
||||
result = await adapter.process(context);
|
||||
|
||||
@@ -516,6 +572,9 @@ export async function runConvertCommand(options: ConvertCommandOptions): Promise
|
||||
|
||||
printOutput(markdown);
|
||||
} finally {
|
||||
if (adapter?.exportCookies && context) {
|
||||
await adapter.exportCookies(context, runtime.chrome.profileDir).catch(() => {});
|
||||
}
|
||||
await closeRuntime(runtime);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { readdir, readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import process from "node:process";
|
||||
|
||||
const ROOT_DIR = process.cwd();
|
||||
const TEST_FILE_PATTERN = /\.test\.(?:[cm]?[jt]s|tsx)$/;
|
||||
const SKIP_DIRECTORIES = new Set([".git", "node_modules"]);
|
||||
const BUN_TEST_IMPORT_PATTERN = /from\s+["']bun:test["']/;
|
||||
|
||||
async function collectTestFiles(directory) {
|
||||
const entries = await readdir(directory, { withFileTypes: true });
|
||||
const files = [];
|
||||
|
||||
for (const entry of entries) {
|
||||
const entryPath = path.join(directory, entry.name);
|
||||
|
||||
if (entry.isDirectory()) {
|
||||
if (SKIP_DIRECTORIES.has(entry.name)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
files.push(...(await collectTestFiles(entryPath)));
|
||||
continue;
|
||||
}
|
||||
|
||||
if (entry.isFile() && TEST_FILE_PATTERN.test(entry.name)) {
|
||||
files.push(entryPath);
|
||||
}
|
||||
}
|
||||
|
||||
return files;
|
||||
}
|
||||
|
||||
async function isNodeCompatibleTest(filePath) {
|
||||
const source = await readFile(filePath, "utf8");
|
||||
return !BUN_TEST_IMPORT_PATTERN.test(source);
|
||||
}
|
||||
|
||||
const allTestFiles = await collectTestFiles(ROOT_DIR);
|
||||
const runnableTestFiles = [];
|
||||
|
||||
for (const filePath of allTestFiles.sort()) {
|
||||
if (await isNodeCompatibleTest(filePath)) {
|
||||
runnableTestFiles.push(filePath);
|
||||
}
|
||||
}
|
||||
|
||||
if (runnableTestFiles.length === 0) {
|
||||
console.error("No Node-compatible test files found.");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const forwardedArgs = process.argv.slice(2);
|
||||
const nodeArgs = ["--import", "tsx", ...forwardedArgs, "--test", ...runnableTestFiles];
|
||||
const child = spawn(process.execPath, nodeArgs, { stdio: "inherit" });
|
||||
|
||||
child.on("exit", (code, signal) => {
|
||||
if (signal) {
|
||||
process.kill(process.pid, signal);
|
||||
return;
|
||||
}
|
||||
|
||||
process.exit(code ?? 1);
|
||||
});
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: baoyu-article-illustrator
|
||||
description: Analyzes article structure, identifies positions requiring visual aids, generates illustrations with Type × Style two-dimension approach. Use when user asks to "illustrate article", "add images", "generate images for article", or "为文章配图".
|
||||
version: 1.56.1
|
||||
version: 1.57.0
|
||||
metadata:
|
||||
openclaw:
|
||||
homepage: https://github.com/JimLiu/baoyu-skills#baoyu-article-illustrator
|
||||
|
||||
@@ -135,6 +135,16 @@ COLORS: Cream background (#F5F0E6), Coral Red (#E07A5F), Mint Green (#81B29A), M
|
||||
ELEMENTS: Geometric simplified icons, no gradients, playful decorative elements (dots, stars)
|
||||
```
|
||||
|
||||
**Infographic + vector-illustration (warm palette)**:
|
||||
```
|
||||
Flat vector illustration infographic. Clean black outlines on all elements.
|
||||
Warm-only color palette, no cool colors.
|
||||
COLORS: Soft Peach background (#FFECD2), Warm Orange (#ED8936),
|
||||
Terracotta (#C05621), Golden Yellow (#F6AD55), Deep Brown (#744210)
|
||||
ELEMENTS: Geometric simplified icons, no gradients, rounded corners,
|
||||
modular card layout, consistent icon style
|
||||
```
|
||||
|
||||
### Scene
|
||||
|
||||
```
|
||||
@@ -197,6 +207,15 @@ COLORS: Left side Coral (#E07A5F), Right side Mint (#81B29A), cream background
|
||||
ELEMENTS: Bold icons, black outlines, centered divider line
|
||||
```
|
||||
|
||||
**Comparison + vector-illustration (warm palette)**:
|
||||
```
|
||||
Flat vector comparison with split layout. Clear visual separation.
|
||||
Warm-only color palette, no cool colors.
|
||||
COLORS: Left side Warm Orange (#ED8936), Right side Terracotta (#C05621),
|
||||
Soft Peach background (#FFECD2), Deep Brown (#744210) accents
|
||||
ELEMENTS: Bold icons, black outlines, centered divider line
|
||||
```
|
||||
|
||||
### Framework
|
||||
|
||||
```
|
||||
@@ -220,6 +239,15 @@ COLORS: Cream background (#F5F0E6), nodes in Coral/Mint/Mustard/Blue, black outl
|
||||
ELEMENTS: Rounded rectangles or circles for nodes, thick connecting lines
|
||||
```
|
||||
|
||||
**Framework + vector-illustration (warm palette)**:
|
||||
```
|
||||
Flat vector framework diagram with geometric nodes and bold connectors.
|
||||
Warm-only color palette, no cool colors.
|
||||
COLORS: Soft Peach background (#FFECD2), nodes in Warm Orange (#ED8936),
|
||||
Terracotta (#C05621), Golden Yellow (#F6AD55), black outlines
|
||||
ELEMENTS: Rounded rectangles or circles for nodes, thick connecting lines
|
||||
```
|
||||
|
||||
### Timeline
|
||||
|
||||
```
|
||||
|
||||
@@ -21,6 +21,7 @@
|
||||
| `saas-guide` | `infographic` | `notion` | Product guides, SaaS docs, tool walkthroughs |
|
||||
| `tutorial` | `flowchart` | `vector-illustration` | Step-by-step tutorials, setup guides |
|
||||
| `process-flow` | `flowchart` | `notion` | Workflow documentation, onboarding flows |
|
||||
| `warm-knowledge` | `infographic` | `vector-illustration` (warm palette) | Product showcases, team intros, feature cards, brand content |
|
||||
|
||||
### Data & Analysis
|
||||
|
||||
@@ -62,7 +63,7 @@ Use this table during Step 3 to recommend presets based on Step 2 content analys
|
||||
| Opinion / Editorial | `opinion-piece` | `cinematic`, `editorial-poster` |
|
||||
| Historical / Timeline | `history` | `evolution` |
|
||||
| Academic / Research | `science-paper` | `tech-explainer`, `data-report` |
|
||||
| SaaS / Product | `saas-guide` | `knowledge-base`, `process-flow` |
|
||||
| SaaS / Product | `saas-guide` | `knowledge-base`, `process-flow`, `warm-knowledge` |
|
||||
|
||||
## Override Examples
|
||||
|
||||
|
||||
@@ -98,6 +98,12 @@ Full specifications: `references/styles/<style>.md`
|
||||
- Modern, professional, highly readable
|
||||
- Perfect for knowledge articles and tutorials
|
||||
|
||||
### infographic + vector-illustration (warm palette)
|
||||
- Same rendering: flat vector, black outlines, geometric simplification
|
||||
- Warm-only palette: terracotta, gold, brown tones on soft peach
|
||||
- Modern-retro feel, approachable and brand-friendly
|
||||
- Best for product showcases, team intros, feature grids, brand content
|
||||
|
||||
### flowchart + vector-illustration
|
||||
- Bold arrows and connectors
|
||||
- Distinct step containers with icons
|
||||
|
||||
@@ -24,6 +24,26 @@ Flat vector illustration with no gradients or 3D effects. Clear, uniform-thickne
|
||||
| Accent 2 | Rock Blue | #577590 | Cool balance |
|
||||
| Text | Black | #1A1A1A | Text elements |
|
||||
|
||||
## Palette Variants
|
||||
|
||||
### Warm Palette
|
||||
|
||||
Use with `--preset warm-knowledge` or specify in prompt. For product showcases, team introductions, feature grids, brand content.
|
||||
|
||||
| Role | Color | Hex | Usage |
|
||||
|------|-------|-----|-------|
|
||||
| Background | Soft Peach | #FFECD2 | Primary background |
|
||||
| Outlines | Deep Charcoal | #2D2D2D | All element outlines |
|
||||
| Primary | Warm Orange | #ED8936 | Main accent color |
|
||||
| Secondary | Terracotta | #C05621 | Warm depth |
|
||||
| Tertiary | Golden Yellow | #F6AD55 | Highlights, energy |
|
||||
| Accent | Deep Brown | #744210 | Grounding, anchoring |
|
||||
| Text | Warm Charcoal | #4A4A4A | Text elements |
|
||||
|
||||
Warm-only palette, no cool colors (no green, blue, purple). Modern-retro feel.
|
||||
|
||||
Default palette includes Mint Green (#81B29A) and Rock Blue (#577590) for colorful educational content. Warm palette uses exclusively warm tones for brand/product/team content.
|
||||
|
||||
## Visual Elements
|
||||
|
||||
- All objects have closed black outlines (coloring book style)
|
||||
|
||||
+47
-26
@@ -29,9 +29,9 @@ Create original knowledge comics with flexible art style × tone combinations.
|
||||
|
||||
| Option | Values | Description |
|
||||
|--------|--------|-------------|
|
||||
| `--art` | ligne-claire (default), manga, realistic, ink-brush, chalk | Art style / rendering technique |
|
||||
| `--art` | ligne-claire (default), manga, realistic, ink-brush, chalk, minimalist | Art style / rendering technique |
|
||||
| `--tone` | neutral (default), warm, dramatic, romantic, energetic, vintage, action | Mood / atmosphere |
|
||||
| `--layout` | standard (default), cinematic, dense, splash, mixed, webtoon | Panel arrangement |
|
||||
| `--layout` | standard (default), cinematic, dense, splash, mixed, webtoon, four-panel | Panel arrangement |
|
||||
| `--aspect` | 3:4 (default, portrait), 4:3 (landscape), 16:9 (widescreen) | Page aspect ratio |
|
||||
| `--lang` | auto (default), zh, en, ja, etc. | Output language |
|
||||
|
||||
@@ -55,6 +55,7 @@ Details: [references/partial-workflows.md](references/partial-workflows.md)
|
||||
| `realistic` | 写实 | Digital painting, realistic proportions, sophisticated |
|
||||
| `ink-brush` | 水墨 | Chinese brush strokes, ink wash effects |
|
||||
| `chalk` | 粉笔 | Chalkboard aesthetic, hand-drawn warmth |
|
||||
| `minimalist` | 极简 | Clean black line art, limited spot color, stick-figure characters |
|
||||
|
||||
### Tones (基调)
|
||||
|
||||
@@ -77,6 +78,8 @@ Presets with special rules beyond art+tone:
|
||||
| `--style ohmsha` | `--art manga --tone neutral` | Visual metaphors, NO talking heads, gadget reveals |
|
||||
| `--style wuxia` | `--art ink-brush --tone action` | Qi effects, combat visuals, atmospheric elements |
|
||||
| `--style shoujo` | `--art manga --tone romantic` | Decorative elements, eye details, romantic beats |
|
||||
| `--style concept-story` | `--art manga --tone warm` | Visual symbol system, growth arc, dialogue+action balance |
|
||||
| `--style four-panel` | `--art minimalist --tone neutral --layout four-panel` | 起承转合 4-panel structure, B&W + spot color, stick-figure characters |
|
||||
|
||||
### Compatibility Matrix
|
||||
|
||||
@@ -87,6 +90,7 @@ Presets with special rules beyond art+tone:
|
||||
| realistic | neutral, warm, dramatic, vintage | action | romantic, energetic |
|
||||
| ink-brush | neutral, dramatic, action, vintage | warm | romantic, energetic |
|
||||
| chalk | neutral, warm, energetic | vintage | dramatic, action, romantic |
|
||||
| minimalist | neutral | warm, energetic | dramatic, vintage, romantic, action |
|
||||
|
||||
Details: [references/auto-selection.md](references/auto-selection.md)
|
||||
|
||||
@@ -101,6 +105,8 @@ Content signals determine default art + tone + layout (or preset):
|
||||
| Personal story, mentor | ligne-claire + warm |
|
||||
| Martial arts, wuxia | **wuxia** preset |
|
||||
| Romance, school life | **shoujo** preset |
|
||||
| Psychology, motivation, business narrative | **concept-story** preset |
|
||||
| Business allegory, fable, parable, short insight, 四格 | **four-panel** preset |
|
||||
| Biography, balanced | ligne-claire + neutral |
|
||||
|
||||
**When preset is recommended**: Load `references/presets/{preset}.md` and apply all special rules.
|
||||
@@ -172,9 +178,9 @@ Comic Progress:
|
||||
- [ ] Step 4: Review outline (conditional)
|
||||
- [ ] Step 5: Generate prompts
|
||||
- [ ] Step 6: Review prompts (conditional)
|
||||
- [ ] Step 7: Generate images ⚠️ CHARACTER REF REQUIRED
|
||||
- [ ] 7.1 Generate character sheet FIRST → characters/characters.png
|
||||
- [ ] 7.2 Generate pages WITH --ref characters/characters.png
|
||||
- [ ] Step 7: Generate images
|
||||
- [ ] 7.1 Generate character sheet (if needed) → characters/characters.png
|
||||
- [ ] 7.2 Generate pages (with --ref if character sheet exists)
|
||||
- [ ] Step 8: Merge to PDF
|
||||
- [ ] Step 9: Completion report
|
||||
```
|
||||
@@ -205,16 +211,24 @@ Analyze → [Check Existing?] → [Confirm: Style + Reviews] → Storyboard →
|
||||
| 4 | Review outline (if requested) | User approval |
|
||||
| 5 | Generate prompts | `prompts/*.md` |
|
||||
| 6 | Review prompts (if requested) | User approval |
|
||||
| **7.1** | **Generate character sheet FIRST** | `characters/characters.png` |
|
||||
| **7.2** | Generate pages **with character ref** | `*.png` files |
|
||||
| 7.1 | Generate character sheet (if needed) | `characters/characters.png` |
|
||||
| 7.2 | Generate pages (with character ref if available) | `*.png` files |
|
||||
| 8 | Merge to PDF | `{slug}.pdf` |
|
||||
| 9 | Completion report | Summary |
|
||||
|
||||
### Step 7: Image Generation ⚠️ CRITICAL
|
||||
### Step 7: Image Generation
|
||||
|
||||
**Character reference is MANDATORY for visual consistency.**
|
||||
**7.1 Generate character sheet (conditional)**:
|
||||
|
||||
**7.1 Generate character sheet first**:
|
||||
Character sheet is recommended for multi-page comics with recurring characters, but **NOT required** for all presets:
|
||||
|
||||
| Condition | Action |
|
||||
|-----------|--------|
|
||||
| Multi-page comic with detailed characters | Generate character sheet (recommended) |
|
||||
| Preset with simplified characters (e.g., four-panel minimalist) | Skip — prompt descriptions are sufficient |
|
||||
| Single-page comic | Skip unless characters are complex |
|
||||
|
||||
**When generating character sheet**:
|
||||
- **Backup rule**: If `characters/characters.png` exists, rename to `characters/characters-backup-YYYYMMDD-HHMMSS.png`
|
||||
- Invoke an installed image generation skill such as `baoyu-imagine`
|
||||
- Read that skill's `SKILL.md` and follow its documented interface rather than calling its scripts directly
|
||||
@@ -222,18 +236,25 @@ Analyze → [Check Existing?] → [Confirm: Style + Reviews] → Storyboard →
|
||||
- Save output to `characters/characters.png`
|
||||
- Use aspect ratio `4:3`
|
||||
|
||||
**Compress character sheet** (recommended):
|
||||
Compress to reduce token usage when used as reference image:
|
||||
**Compress character sheet** (recommended when using as `--ref`):
|
||||
- Use available image compression skill (if any)
|
||||
- Or system tools: `pngquant`, `optipng`, `sips` (macOS)
|
||||
- **Keep PNG format**, lossless compression preferred
|
||||
- Or system tools: `sips -s format jpeg -s formatOptions 80 input.png --out output.jpg` (macOS)
|
||||
- Or: `pngquant --quality=65-80 input.png -o output.png`
|
||||
- Compression reduces API payload size and avoids `--ref` failures
|
||||
|
||||
**7.2 Generate each page WITH character reference**:
|
||||
**7.2 Generate each page**:
|
||||
|
||||
| Skill Capability | Strategy |
|
||||
|------------------|----------|
|
||||
| Supports `--ref` | Pass `characters/characters.png` with EVERY page |
|
||||
| No `--ref` support | Prepend character descriptions to EVERY prompt file |
|
||||
| Character Sheet | Skill Capability | Strategy |
|
||||
|-----------------|------------------|----------|
|
||||
| Exists | Supports `--ref` | Pass `characters/characters.png` with EVERY page |
|
||||
| Exists | No `--ref` support | Prepend character descriptions to EVERY prompt file |
|
||||
| Skipped | — | Prompt file contains all character descriptions inline |
|
||||
|
||||
**`--ref` failure recovery**: If generation fails with `--ref`:
|
||||
1. **Compress**: Convert reference image to JPEG with reduced quality:
|
||||
`sips -s format jpeg -s formatOptions 70 characters.png --out characters-compressed.jpg`
|
||||
2. **Retry** with compressed image as `--ref`
|
||||
3. **If still fails**: Fall back to generating WITHOUT `--ref` (prompt-only, character descriptions embedded in prompt text)
|
||||
|
||||
**Backup rules for page generation**:
|
||||
- If prompt file exists: rename to `prompts/NN-{cover|page}-[slug]-backup-YYYYMMDD-HHMMSS.md`
|
||||
@@ -241,8 +262,8 @@ Compress to reduce token usage when used as reference image:
|
||||
- Invoke the installed image generation skill for each page
|
||||
- Use `prompts/01-page-xxx.md` as the prompt-file input
|
||||
- Save output to `01-page-xxx.png`
|
||||
- Use aspect ratio `3:4`
|
||||
- If the chosen skill supports reference images, pass `characters/characters.png` as `--ref`
|
||||
- Use aspect ratio from storyboard (default `3:4`, preset may override)
|
||||
- If character sheet exists and skill supports reference images, pass as `--ref`
|
||||
|
||||
**Full workflow details**: [references/workflow.md](references/workflow.md)
|
||||
|
||||
@@ -273,10 +294,10 @@ Schema: [references/config/preferences-schema.md](references/config/preferences-
|
||||
- [ohmsha-guide.md](references/ohmsha-guide.md) - Ohmsha manga specifics
|
||||
|
||||
**Style Definitions**:
|
||||
- `references/art-styles/` - Art styles (ligne-claire, manga, realistic, ink-brush, chalk)
|
||||
- `references/art-styles/` - Art styles (ligne-claire, manga, realistic, ink-brush, chalk, minimalist)
|
||||
- `references/tones/` - Tones (neutral, warm, dramatic, romantic, energetic, vintage, action)
|
||||
- `references/presets/` - Presets with special rules (ohmsha, wuxia, shoujo)
|
||||
- `references/layouts/` - Layouts (standard, cinematic, dense, splash, mixed, webtoon)
|
||||
- `references/presets/` - Presets with special rules (ohmsha, wuxia, shoujo, concept-story, four-panel)
|
||||
- `references/layouts/` - Layouts (standard, cinematic, dense, splash, mixed, webtoon, four-panel)
|
||||
|
||||
**Workflow**:
|
||||
- [workflow.md](references/workflow.md) - Full workflow details
|
||||
@@ -306,6 +327,6 @@ Schema: [references/config/preferences-schema.md](references/config/preferences-
|
||||
- Maintain style consistency via session ID
|
||||
- **Step 2 confirmation required** - do not skip
|
||||
- **Steps 4/6 conditional** - only if user requested in Step 2
|
||||
- **Step 7.1 character sheet MUST be generated before pages** - ensures consistency
|
||||
- **Step 7.2 EVERY page MUST reference characters** - use `--ref` or embed descriptions
|
||||
- **Step 7.1 character sheet** - recommended for multi-page comics, optional for simple presets
|
||||
- **Step 7.2 character reference** - use `--ref` if sheet exists; compress/convert on failure; fall back to prompt-only
|
||||
- Watermark/language configured once in EXTEND.md
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
# minimalist
|
||||
|
||||
极简画风 - Clean black line art, limited spot color, simplified stick-figure characters
|
||||
|
||||
## Overview
|
||||
|
||||
Minimalist cartoon illustration characterized by clean black line art on white background with very limited spot color for emphasis. Characters are simplified to near-stick-figure abstraction, focusing on gesture and concept rather than anatomical detail. Designed for business allegory, quick-read educational content, and concept illustration.
|
||||
|
||||
## Line Work
|
||||
|
||||
- Clean, uniform black lines (1.5-2px)
|
||||
- No hatching, cross-hatching, or shading techniques
|
||||
- Minimal detail — every line serves a purpose
|
||||
- Bold outlines for characters, thinner lines for props/labels
|
||||
- No decorative flourishes or ornamental lines
|
||||
|
||||
## Character Design
|
||||
|
||||
- Highly simplified, stick-figure-like business characters
|
||||
- Circle or oval heads with minimal facial features (dot eyes, simple line mouth)
|
||||
- Body as simple geometric shapes or line constructions
|
||||
- Distinguishing features through props only (tie, hat, briefcase, glasses)
|
||||
- No anatomical detail — expressive through posture and gesture
|
||||
- 4-5 head height proportions (squat, iconic)
|
||||
|
||||
## Background Treatment
|
||||
|
||||
- Mostly blank/white — negative space is a design element
|
||||
- Minimal environmental cues (a line for ground, simple desk outline)
|
||||
- Concept labels and text annotations replace detailed environments
|
||||
- Icons and symbols over realistic rendering
|
||||
- No perspective or spatial depth
|
||||
|
||||
## Color Approach
|
||||
|
||||
- Primarily black and white (90%+ of the image)
|
||||
- 1-2 spot accent colors for emphasis on key concepts
|
||||
- Accent color used sparingly: highlighting key objects, text labels, concept indicators
|
||||
- No gradients, no shading, no color fills on backgrounds
|
||||
- Color draws the eye to the most important element in each panel
|
||||
|
||||
## Default Color Palette
|
||||
|
||||
| Role | Color | Hex |
|
||||
|------|-------|-----|
|
||||
| Primary | Black ink | `#1A1A1A` |
|
||||
| Background | Clean white | `#FFFFFF` |
|
||||
| Accent 1 | Spot orange | `#FF6B35` |
|
||||
| Accent 2 | Spot blue (optional) | `#3182CE` |
|
||||
| Text labels | Dark gray | `#4A4A4A` |
|
||||
| Panel border | Medium gray | `#666666` |
|
||||
|
||||
## Visual Elements
|
||||
|
||||
- Text labels with accent-color backgrounds or underlines for key terms
|
||||
- Simple icons: arrows, circles, checkmarks, crosses
|
||||
- Concept highlight boxes with spot color
|
||||
- Minimal speech bubbles (simple oval or rectangle, thin black outline)
|
||||
- No sound effects, no motion lines, no screen tones
|
||||
|
||||
## Quality Markers
|
||||
|
||||
- ✓ Clean, purposeful line work with no unnecessary detail
|
||||
- ✓ 90%+ black-and-white with strategic spot color
|
||||
- ✓ Simplified characters readable at small sizes
|
||||
- ✓ Text labels integrated naturally into panels
|
||||
- ✓ Strong negative space usage
|
||||
- ✓ Every element serves the narrative point
|
||||
|
||||
## Compatibility
|
||||
|
||||
| Tone | Fit | Notes |
|
||||
|------|-----|-------|
|
||||
| neutral | ✓✓ | Ideal for business/educational content |
|
||||
| warm | ✓ | Works for gentle stories, slight warmth in accent |
|
||||
| energetic | ✓ | Works for punchy, high-energy content |
|
||||
| dramatic | ✗ | Style too stripped down for dramatic intensity |
|
||||
| vintage | ✗ | Minimalist aesthetic conflicts with aged/textured look |
|
||||
| romantic | ✗ | No capacity for decorative/soft elements |
|
||||
| action | ✗ | No dynamic line capability for speed/impact |
|
||||
|
||||
## Best For
|
||||
|
||||
Business allegory, management fables, short concept illustration, four-panel comic strips, quick-insight education, social media content
|
||||
@@ -11,10 +11,13 @@ Content signals determine default art + tone + layout (or preset).
|
||||
| Technical explanation, educational | manga | neutral | webtoon | **ohmsha** |
|
||||
| Pre-1950, classical, ancient | realistic | vintage | cinematic | - |
|
||||
| Personal story, mentor | ligne-claire | warm | standard | - |
|
||||
| Psychology, motivation, self-help, coaching | manga | warm | standard | **concept-story** |
|
||||
| Business narrative, management, leadership | manga | warm | standard | **concept-story** |
|
||||
| Conflict, breakthrough | (inherit) | dramatic | splash | - |
|
||||
| Wine, food, business, lifestyle | realistic | neutral | cinematic | - |
|
||||
| Wine, food, lifestyle | realistic | neutral | cinematic | - |
|
||||
| Martial arts, wuxia, xianxia | ink-brush | action | splash | **wuxia** |
|
||||
| Romance, love, school life | manga | romantic | standard | **shoujo** |
|
||||
| Business allegory, fable, parable, short insight, 四格 | minimalist | neutral | four-panel | **four-panel** |
|
||||
| Biography, balanced | ligne-claire | neutral | mixed | - |
|
||||
|
||||
## Preset Recommendation Rules
|
||||
@@ -36,6 +39,16 @@ Content signals determine default art + tone + layout (or preset).
|
||||
- **Special rules**: Decorative elements, eye details, romantic beats
|
||||
- **Base**: manga + romantic + standard
|
||||
|
||||
### concept-story
|
||||
- **Triggers**: Psychology, motivation, self-help, business narrative, management, leadership, personal growth, coaching, soft skills, abstract concept through story
|
||||
- **Special rules**: Visual symbol system, growth arc, dialogue+action balance, original characters
|
||||
- **Base**: manga + warm + standard
|
||||
|
||||
### four-panel
|
||||
- **Triggers**: Business allegory, fable, parable, short insight, four-panel, 四格, 四格漫画, single-page comic, minimalist comic strip
|
||||
- **Special rules**: Strict 起承转合 4-panel structure, B&W + spot color, simplified stick-figure characters, single-page story
|
||||
- **Base**: minimalist + neutral + four-panel
|
||||
|
||||
## Compatibility Matrix
|
||||
|
||||
Art Style × Tone combinations work best when matched appropriately:
|
||||
@@ -47,6 +60,7 @@ Art Style × Tone combinations work best when matched appropriately:
|
||||
| realistic | neutral, warm, dramatic, vintage | action | romantic, energetic |
|
||||
| ink-brush | neutral, dramatic, action, vintage | warm | romantic, energetic |
|
||||
| chalk | neutral, warm, energetic | vintage | dramatic, action, romantic |
|
||||
| minimalist | neutral | warm, energetic | dramatic, vintage, romantic, action |
|
||||
|
||||
**Note**: Art Style × Tone × Layout can be freely combined. Incompatible combinations work but may produce unexpected results.
|
||||
|
||||
|
||||
@@ -16,9 +16,9 @@ watermark:
|
||||
content: ""
|
||||
position: bottom-right # bottom-right|bottom-left|bottom-center|top-right
|
||||
|
||||
preferred_art: null # ligne-claire|manga|realistic|ink-brush|chalk
|
||||
preferred_art: null # ligne-claire|manga|realistic|ink-brush|chalk|minimalist
|
||||
preferred_tone: null # neutral|warm|dramatic|romantic|energetic|vintage|action
|
||||
preferred_layout: null # standard|cinematic|dense|splash|mixed|webtoon
|
||||
preferred_layout: null # standard|cinematic|dense|splash|mixed|webtoon|four-panel
|
||||
preferred_aspect: null # 3:4|4:3|16:9
|
||||
|
||||
language: null # zh|en|ja|ko|auto
|
||||
@@ -41,7 +41,7 @@ character_presets:
|
||||
| `watermark.enabled` | bool | false | Enable watermark |
|
||||
| `watermark.content` | string | "" | Watermark text (@username or custom) |
|
||||
| `watermark.position` | enum | bottom-right | Position on image |
|
||||
| `preferred_art` | string | null | Art style (ligne-claire, manga, realistic, ink-brush, chalk) |
|
||||
| `preferred_art` | string | null | Art style (ligne-claire, manga, realistic, ink-brush, chalk, minimalist) |
|
||||
| `preferred_tone` | string | null | Tone (neutral, warm, dramatic, romantic, energetic, vintage, action) |
|
||||
| `preferred_layout` | string | null | Layout preference or null |
|
||||
| `preferred_aspect` | string | null | Aspect ratio (3:4, 4:3, 16:9) |
|
||||
@@ -57,6 +57,7 @@ character_presets:
|
||||
| `realistic` | 写实 | Digital painting, realistic proportions |
|
||||
| `ink-brush` | 水墨 | Chinese brush strokes, ink wash effects |
|
||||
| `chalk` | 粉笔 | Chalkboard aesthetic, hand-drawn warmth |
|
||||
| `minimalist` | 极简 | Clean black line art, limited spot color, stick-figure characters |
|
||||
|
||||
## Tone Options
|
||||
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
# four-panel
|
||||
|
||||
四格漫画 - Strict 2×2 grid, single-page story
|
||||
|
||||
## Panel Structure
|
||||
|
||||
- **Panels per page**: 4 (exactly, no variation)
|
||||
- **Structure**: Strict 2×2 equal grid
|
||||
- **Gutters**: Consistent white space (8-10px), uniform on all sides
|
||||
|
||||
## Grid Configuration
|
||||
|
||||
- 2 columns × 2 rows, all panels identical size
|
||||
- Panel sizes: Exactly equal (each panel = 25% of content area)
|
||||
- Reading flow: Z-pattern — Panel 1 (top-left) → Panel 2 (top-right) → Panel 3 (bottom-left) → Panel 4 (bottom-right)
|
||||
|
||||
## Narrative Structure
|
||||
|
||||
Each panel serves a specific narrative role (起承转合 / kishōtenketsu):
|
||||
|
||||
| Panel | Position | Role | Purpose |
|
||||
|-------|----------|------|---------|
|
||||
| 1 | Top-left | 起 Setup | Establish situation, introduce characters/problem |
|
||||
| 2 | Top-right | 承 Development | Build on setup, add complication or attempt |
|
||||
| 3 | Bottom-left | 转 Turn | Twist, key insight, or reversal — the pivotal moment |
|
||||
| 4 | Bottom-right | 合 Conclusion | Resolution, punchline, or takeaway |
|
||||
|
||||
## Aspect Ratio
|
||||
|
||||
- Recommended page aspect: **4:3** (landscape)
|
||||
- Landscape gives each panel a comfortable wide rectangle
|
||||
- Portrait (3:4) makes panels tall and narrow — avoid for this layout
|
||||
|
||||
## Best For
|
||||
|
||||
Business allegory, quick-insight education, social media comics, fables, parables, single-concept explanation
|
||||
|
||||
## Best Style Pairings
|
||||
|
||||
minimalist, ligne-claire, chalk
|
||||
@@ -0,0 +1,121 @@
|
||||
# concept-story
|
||||
|
||||
概念故事预设 - Narrative comics that visualize abstract concepts through character-driven stories
|
||||
|
||||
## Base Configuration
|
||||
|
||||
| Dimension | Value |
|
||||
|-----------|-------|
|
||||
| Art Style | manga |
|
||||
| Tone | warm |
|
||||
| Layout | standard (default) |
|
||||
|
||||
Equivalent to: `--art manga --tone warm`
|
||||
|
||||
## Unique Rules
|
||||
|
||||
This preset includes special rules beyond the art+tone combination. When `--style concept-story` is used, ALL rules below must be applied.
|
||||
|
||||
### Concept Visualization System (CRITICAL)
|
||||
|
||||
Each major abstract concept SHOULD have a recurring visual symbol/metaphor:
|
||||
|
||||
| Concept Type | Visualization Approach |
|
||||
|-------------|----------------------|
|
||||
| Psychological need | Tangible object character holds or discovers (e.g., glowing energy ball = competence) |
|
||||
| Management principle | Environmental metaphor character navigates (e.g., ship wheel = autonomy) |
|
||||
| Growth/development | Living organic symbol that transforms (e.g., seed → flowering plant = relatedness) |
|
||||
| Abstract framework | Spatial structure characters can enter or observe |
|
||||
| Emotional state | Color/lighting shift in the scene atmosphere |
|
||||
|
||||
**Unlike ohmsha**: Dialogue panels are allowed and expected. The goal is to COMBINE visual metaphors WITH dialogue, not replace dialogue entirely.
|
||||
|
||||
**Pattern**: "Dialogue introduces idea" → "Visual metaphor illustrates it" → "Character reacts/applies it"
|
||||
|
||||
### Visual Symbol Continuity
|
||||
|
||||
Symbols must persist across the story:
|
||||
|
||||
| Stage | Treatment |
|
||||
|-------|-----------|
|
||||
| Introduction | Symbol appears with soft glow effect when concept is first mentioned |
|
||||
| Recurrence | Same symbol reappears in background or character interaction when concept is referenced |
|
||||
| Resolution | ALL symbols gather in the final composition, showing integration of learned concepts |
|
||||
|
||||
**Storyboard requirement**: Include a Symbol Mapping Table defining concept → visual symbol before panel breakdown.
|
||||
|
||||
### Character Archetypes (Flexible)
|
||||
|
||||
Create original characters based on content domain. No fixed defaults:
|
||||
|
||||
| Role | Archetype | Visual Cues |
|
||||
|------|-----------|------------|
|
||||
| Protagonist | Learner/worker facing a challenge | Modern professional or student, relatable, starts with constrained posture |
|
||||
| Mentor | Experienced guide who teaches through experience | Slightly older, calm demeanor, warm color accents |
|
||||
| Catalyst | Person or event that triggers transformation | Can be a colleague, situation, challenge, or opportunity |
|
||||
|
||||
**IMPORTANT**: Characters are created fresh each time based on the source content's domain (business, psychology, education, etc.). No default character set.
|
||||
|
||||
### Narrative Arc Structure
|
||||
|
||||
Enforce a five-stage growth arc:
|
||||
|
||||
| Act | Structure | Visual Tone |
|
||||
|-----|-----------|------------|
|
||||
| Opening | Protagonist stuck in routine, faces frustration | Muted warm tones, tight framing, constrained compositions |
|
||||
| Inciting moment | Mentor appears or opportunity arrives | Brightness increases, panels open up |
|
||||
| Learning | Concepts introduced through visual metaphors | Rich warm palette, symbols introduced one by one |
|
||||
| Turning point | Protagonist applies knowledge, faces test | Contrast increases, dynamic compositions |
|
||||
| Transformation | Growth demonstrated, new understanding visible | Full warm palette, expansive composition, all symbols present |
|
||||
|
||||
### Dialogue + Action Balance
|
||||
|
||||
- Dialogue is encouraged and expected (unlike ohmsha's NO talking heads rule)
|
||||
- Every page should combine at least one dialogue panel with at least one visual/action panel
|
||||
- Avoid pure "lecture" pages where a character explains for 4+ panels straight
|
||||
- When a character explains a concept verbally, the NEXT panel should visualize it
|
||||
|
||||
**Wrong approach**: Four consecutive panels of mentor lecturing at protagonist
|
||||
**Right approach**: Mentor introduces concept → visual metaphor panel → protagonist reacts → applies understanding
|
||||
|
||||
### Scene Atmosphere Rules
|
||||
|
||||
| Scene Type | Atmosphere |
|
||||
|------------|-----------|
|
||||
| Problem/frustration | Cool muted tones over warm base, tight framing, cluttered environment |
|
||||
| Mentoring moment | Golden hour lighting, open composition, warm indoor glow |
|
||||
| Concept visualization | Soft glow effects, clean simplified backgrounds, symbol spotlight |
|
||||
| Growth/transformation | Warm light expanding outward, character posture opening up |
|
||||
| Resolution | Full warm palette, spacious composition, all visual symbols visible |
|
||||
|
||||
### Ending Requirements
|
||||
|
||||
Final page MUST include:
|
||||
|
||||
1. Protagonist demonstrating transformed understanding (not just being told)
|
||||
2. Visual callback showing contrast with opening state (e.g., wilted plant → thriving plant)
|
||||
3. All concept symbols visible together in the composition
|
||||
4. A forward-looking element suggesting ongoing growth (not a closed ending)
|
||||
|
||||
### Page Title Convention
|
||||
|
||||
Every page MUST have a narrative title:
|
||||
|
||||
**Wrong**: "Chapter 3: Self-Determination Theory"
|
||||
**Right**: "The Day Xiao Ming Found His Own Engine"
|
||||
|
||||
## Quality Markers
|
||||
|
||||
- ✓ Each major concept has a recurring visual symbol
|
||||
- ✓ Dialogue and visual metaphors work together (not one replacing the other)
|
||||
- ✓ Clear growth arc from problem to transformation
|
||||
- ✓ Original characters suited to the content domain
|
||||
- ✓ Warm, professional atmosphere throughout
|
||||
- ✓ Visual symbols recur and accumulate through the story
|
||||
- ✓ Final page integrates all concept symbols with transformation callback
|
||||
|
||||
## Best For
|
||||
|
||||
Psychology concepts, business/management principles, motivation theory, personal development,
|
||||
self-help content, leadership frameworks, coaching narratives, soft skill education,
|
||||
abstract concept explanation through character-driven stories
|
||||
@@ -0,0 +1,107 @@
|
||||
# four-panel
|
||||
|
||||
四格漫画预设 - Minimalist four-panel business allegory comics
|
||||
|
||||
## Base Configuration
|
||||
|
||||
| Dimension | Value |
|
||||
|-----------|-------|
|
||||
| Art Style | minimalist |
|
||||
| Tone | neutral |
|
||||
| Layout | four-panel (default) |
|
||||
| Aspect | 4:3 (landscape) |
|
||||
|
||||
Equivalent to: `--art minimalist --tone neutral --layout four-panel --aspect 4:3`
|
||||
|
||||
## Unique Rules
|
||||
|
||||
This preset includes special rules beyond the art+tone combination. When `--style four-panel` is used, ALL rules below must be applied.
|
||||
|
||||
### 起承转合 Narrative Structure (CRITICAL)
|
||||
|
||||
Every comic MUST follow the four-panel 起承转合 structure:
|
||||
|
||||
| Panel | Role | Requirements |
|
||||
|-------|------|-------------|
|
||||
| 1 (起 Setup) | Introduce the situation | Show character(s) in a recognizable context. Establish the "normal" state or problem |
|
||||
| 2 (承 Development) | Build on the setup | Add complication, show an attempt, or introduce the concept. Stakes become clearer |
|
||||
| 3 (转 Turn) | The twist or key insight | **Most important panel.** Show the unexpected reversal, contrast, or "aha" moment that makes the allegory work |
|
||||
| 4 (合 Conclusion) | Resolution and takeaway | Show the result, consequence, or lesson learned. Can be a visual punchline or summary |
|
||||
|
||||
**CRITICAL**: Do NOT deviate from exactly 4 panels. No 5th panel, no title panel, no footer panel within the image.
|
||||
|
||||
### Single-Page Story Rule (CRITICAL)
|
||||
|
||||
- The entire story is told in ONE page with exactly 4 panels
|
||||
- Page count: always 1 (plus optional cover)
|
||||
- No multi-page four-panel stories — if content requires more, create multiple separate four-panel comics
|
||||
- Storyboard structure: Cover (optional) + 1 page
|
||||
|
||||
### Accent Color System
|
||||
|
||||
- The image is primarily black-and-white line art
|
||||
- Use exactly 1-2 spot colors per strip (default: orange `#FF6B35`)
|
||||
- Rules:
|
||||
- Key concept label or object: filled with accent color or outlined in accent
|
||||
- Panel 3 (转 Turn) should have the strongest color emphasis
|
||||
- Characters remain B&W — color is for concepts/objects/labels only
|
||||
- Consistent accent color across all 4 panels (do not switch colors between panels)
|
||||
|
||||
### Character Design Rules
|
||||
|
||||
- Simplified stick-figure-like characters
|
||||
- Distinguish characters through simple props: ties, glasses, hats, briefcases, aprons
|
||||
- No detailed faces — dot eyes, line mouth at most
|
||||
- Characters should be generic enough to represent archetypes (the manager, the employee, the customer)
|
||||
- Maximum 2-3 characters per strip
|
||||
|
||||
### Text in Panels
|
||||
|
||||
- Chinese text for dialogue and labels (or match source language)
|
||||
- Keep text minimal — 1-2 short lines per panel maximum
|
||||
- Key concept terms can be highlighted with accent color background
|
||||
- No narrator boxes — dialogue and labels only
|
||||
- Speech bubbles: simple rectangles or ovals, thin black outline
|
||||
|
||||
### Optional Title & Caption
|
||||
|
||||
- A brief descriptive title above the 4 panels
|
||||
- An optional one-line caption/moral below the panels
|
||||
- These are part of the page composition, not separate panels
|
||||
|
||||
### Character Archetypes (Flexible)
|
||||
|
||||
Create simple stick-figure characters based on content. No fixed defaults:
|
||||
|
||||
| Role | Archetype | Visual Cues |
|
||||
|------|-----------|------------|
|
||||
| Protagonist | Worker/employee facing a situation | Simple figure, minimal distinguishing feature (glasses, tie) |
|
||||
| Authority | Boss/manager/expert | Slightly larger figure, or prop like pointer/clipboard |
|
||||
| Object | The concept itself | Labeled object, icon, or highlighted text with accent color |
|
||||
|
||||
### Prompt Template
|
||||
|
||||
When generating image prompts for four-panel comics, include these keywords:
|
||||
|
||||
> A minimalist, clean line art digital comic strip in a four-panel grid layout (2×2). The style is simplified cartoon illustration with clear black outlines and a minimal color palette of black, white, and specific spot [accent color] for key concepts.
|
||||
|
||||
Each panel description should specify:
|
||||
- Panel position (Top Left / Top Right / Bottom Left / Bottom Right)
|
||||
- Character poses and gestures (simple, stick-figure style)
|
||||
- Dialogue text in Chinese (hand-drawn style)
|
||||
- Any accent-colored elements (concept labels, key objects)
|
||||
|
||||
## Quality Markers
|
||||
|
||||
- ✓ Exactly 4 panels in strict 2×2 grid
|
||||
- ✓ 起承转合 narrative arc clearly present
|
||||
- ✓ 90%+ black-and-white with strategic spot color
|
||||
- ✓ Simplified stick-figure characters
|
||||
- ✓ Key concept visually highlighted with accent color
|
||||
- ✓ Text is minimal and in Chinese (or source language)
|
||||
- ✓ Single complete story in one page
|
||||
- ✓ Panel 3 delivers a clear "turn" or insight
|
||||
|
||||
## Best For
|
||||
|
||||
Business allegory, management fables, short insights, workplace parables, concept contrasts, social media educational content, quick-read comics
|
||||
@@ -18,6 +18,8 @@ Comic Progress:
|
||||
- [ ] Step 5: Generate prompts
|
||||
- [ ] Step 6: Review prompts (conditional)
|
||||
- [ ] Step 7: Generate images
|
||||
- [ ] 7.1 Character sheet (if needed)
|
||||
- [ ] 7.2 Generate pages
|
||||
- [ ] Step 8: Merge to PDF
|
||||
- [ ] Step 9: Completion report
|
||||
```
|
||||
@@ -410,28 +412,41 @@ options:
|
||||
|
||||
With confirmed prompts from Step 5/6:
|
||||
|
||||
### 7.1 Generate Character Reference Sheet (first)
|
||||
### 7.1 Generate Character Reference Sheet (conditional)
|
||||
|
||||
Character sheet is recommended for multi-page comics with recurring characters, but **NOT required** for all presets.
|
||||
|
||||
**When to generate**:
|
||||
|
||||
| Condition | Action |
|
||||
|-----------|--------|
|
||||
| Multi-page comic with detailed/recurring characters | Generate character sheet (recommended) |
|
||||
| Preset with simplified characters (e.g., four-panel minimalist) | Skip — prompt descriptions are sufficient |
|
||||
| Single-page comic | Skip unless characters are complex |
|
||||
|
||||
**When generating**:
|
||||
1. Use Reference Sheet Prompt from `characters/characters.md`
|
||||
2. **Backup rule**: If `characters/characters.png` exists, rename to `characters/characters-backup-YYYYMMDD-HHMMSS.png`
|
||||
3. Generate → `characters/characters.png`
|
||||
4. This ensures visual consistency for all subsequent pages
|
||||
4. **Compress** to reduce API payload size when used as `--ref`:
|
||||
- `sips -s format jpeg -s formatOptions 80 characters.png --out characters-compressed.jpg` (macOS)
|
||||
- Or: `pngquant --quality=65-80 characters.png -o characters-compressed.png`
|
||||
|
||||
### 7.2 Generate Comic Pages
|
||||
|
||||
**CRITICAL: Character Reference is MANDATORY** for visual consistency across all pages.
|
||||
|
||||
**Before generating any page**:
|
||||
1. Read the image generation skill's SKILL.md
|
||||
2. Check if it supports reference image input (`--ref`, `--reference`, etc.)
|
||||
3. Choose the appropriate strategy below
|
||||
3. Determine if character sheet exists
|
||||
4. Choose the appropriate strategy below
|
||||
|
||||
**Character Reference Strategy**:
|
||||
**Page Generation Strategy**:
|
||||
|
||||
| Skill Capability | Strategy | Action |
|
||||
|------------------|----------|--------|
|
||||
| Supports `--ref` | **Strategy A** | Pass `characters/characters.png` with EVERY page |
|
||||
| Does NOT support `--ref` | **Strategy B** | Prepend character descriptions to EVERY prompt |
|
||||
| Character Sheet | Skill Capability | Strategy |
|
||||
|-----------------|------------------|----------|
|
||||
| Exists | Supports `--ref` | **A**: Pass character sheet as `--ref` with every page |
|
||||
| Exists | No `--ref` support | **B**: Embed character descriptions in every prompt |
|
||||
| Skipped | — | **C**: Prompt file contains all descriptions inline |
|
||||
|
||||
**Strategy A: Using `--ref` parameter** (e.g., baoyu-imagine)
|
||||
|
||||
@@ -439,8 +454,16 @@ With confirmed prompts from Step 5/6:
|
||||
- Invoke that installed skill via its documented interface, not by calling its scripts directly
|
||||
- For every page, use `prompts/01-page-xxx.md` as the prompt-file input
|
||||
- Save output to `01-page-xxx.png`
|
||||
- Use aspect ratio `3:4`
|
||||
- Pass `characters/characters.png` as `--ref` on every page generation
|
||||
- Use aspect ratio from storyboard (default `3:4`, preset may override)
|
||||
- Pass `characters/characters.png` (or compressed version) as `--ref`
|
||||
|
||||
**`--ref` failure recovery**:
|
||||
If generation fails when using `--ref`:
|
||||
1. **Compress/convert** reference image:
|
||||
- `sips -s format jpeg -s formatOptions 70 characters.png --out characters-compressed.jpg`
|
||||
- Or reduce resolution: `sips -Z 1024 characters.png --out characters-small.png`
|
||||
2. **Retry** with compressed/converted image as `--ref`
|
||||
3. **If still fails**: Fall back to **Strategy C** — generate WITHOUT `--ref`, with character descriptions embedded in prompt text
|
||||
|
||||
**Strategy B: Embedding character descriptions in prompt**
|
||||
|
||||
@@ -458,10 +481,17 @@ When skill does NOT support reference images, create combined prompt files:
|
||||
[Original page prompt here]
|
||||
```
|
||||
|
||||
**Strategy C: Prompt-only (no character sheet)**
|
||||
|
||||
When character sheet was skipped or `--ref` failed:
|
||||
- Prompt file already contains all character descriptions inline
|
||||
- No `--ref` parameter needed
|
||||
- Rely on detailed text descriptions for character consistency
|
||||
|
||||
**For each page (cover + pages)**:
|
||||
1. Read prompt from `prompts/NN-{cover|page}-[slug].md`
|
||||
2. **Backup rule**: If image file exists, rename to `NN-{cover|page}-[slug]-backup-YYYYMMDD-HHMMSS.png`
|
||||
3. Generate image using Strategy A or B (based on skill capability)
|
||||
3. Generate image using Strategy A, B, or C
|
||||
4. Save to `NN-{cover|page}-[slug].png`
|
||||
5. Report progress after each generation: "Generated X/N: [page title]"
|
||||
|
||||
@@ -493,7 +523,7 @@ Title: [title] | Art: [art] | Tone: [tone] | Pages: [count] | Aspect: [ratio] |
|
||||
Watermark: [enabled/disabled]
|
||||
Location: [path]
|
||||
✓ analysis.md
|
||||
✓ characters.png
|
||||
✓ characters.png (if generated)
|
||||
✓ 00-cover-[slug].png ... NN-page-[slug].png
|
||||
✓ {topic-slug}.pdf
|
||||
```
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"lockfileVersion": 1,
|
||||
"configVersion": 0,
|
||||
"workspaces": {
|
||||
"": {
|
||||
"name": "baoyu-danger-gemini-web-scripts",
|
||||
|
||||
@@ -5,8 +5,8 @@ import {
|
||||
discoverRunningChromeDebugPort,
|
||||
findChromeExecutable as findChromeExecutableBase,
|
||||
findExistingChromeDebugPort,
|
||||
gracefulKillChrome,
|
||||
getFreePort,
|
||||
killChrome,
|
||||
launchChrome as launchChromeBase,
|
||||
openPageSession,
|
||||
sleep,
|
||||
@@ -241,15 +241,11 @@ async function fetch_google_cookies_via_cdp(
|
||||
try {
|
||||
await cdp.send('Target.closeTarget', { targetId }, { timeoutMs: 5_000 });
|
||||
} catch {}
|
||||
} else {
|
||||
try {
|
||||
await cdp.send('Browser.close', {}, { timeoutMs: 5_000 });
|
||||
} catch {}
|
||||
}
|
||||
cdp.close();
|
||||
}
|
||||
|
||||
if (chrome) killChrome(chrome);
|
||||
if (chrome) await gracefulKillChrome(chrome, port);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+32
-1
@@ -478,7 +478,7 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
chrome.kill("SIGTERM");
|
||||
} catch {}
|
||||
setTimeout(() => {
|
||||
if (!chrome.killed) {
|
||||
if (chrome.exitCode === null && chrome.signalCode === null) {
|
||||
try {
|
||||
chrome.kill("SIGKILL");
|
||||
} catch {}
|
||||
@@ -486,6 +486,37 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
}, 2_000).unref?.();
|
||||
}
|
||||
|
||||
export async function gracefulKillChrome(
|
||||
chrome: ChildProcess,
|
||||
port?: number,
|
||||
timeoutMs = 6_000,
|
||||
): Promise<void> {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
|
||||
const exitPromise = new Promise<void>((resolve) => {
|
||||
chrome.once("exit", () => resolve());
|
||||
});
|
||||
|
||||
killChrome(chrome);
|
||||
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
if (port !== undefined && !await isPortListening(port, 250)) return;
|
||||
|
||||
const exited = await Promise.race([
|
||||
exitPromise.then(() => true),
|
||||
sleep(100).then(() => false),
|
||||
]);
|
||||
if (exited) return;
|
||||
}
|
||||
|
||||
await Promise.race([
|
||||
exitPromise,
|
||||
sleep(250),
|
||||
]);
|
||||
}
|
||||
|
||||
export async function openPageSession(options: OpenPageSessionOptions): Promise<PageSession> {
|
||||
let targetId: string;
|
||||
let createdTarget = false;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"lockfileVersion": 1,
|
||||
"configVersion": 0,
|
||||
"workspaces": {
|
||||
"": {
|
||||
"name": "baoyu-danger-x-to-markdown-scripts",
|
||||
|
||||
@@ -2,8 +2,8 @@ import {
|
||||
CdpConnection,
|
||||
findChromeExecutable as findChromeExecutableBase,
|
||||
findExistingChromeDebugPort,
|
||||
gracefulKillChrome,
|
||||
getFreePort,
|
||||
killChrome,
|
||||
launchChrome as launchChromeBase,
|
||||
openPageSession,
|
||||
sleep,
|
||||
@@ -122,15 +122,11 @@ async function fetchXCookiesViaCdp(
|
||||
try {
|
||||
await cdp.send("Target.closeTarget", { targetId }, { timeoutMs: 5_000 });
|
||||
} catch {}
|
||||
} else {
|
||||
try {
|
||||
await cdp.send("Browser.close", {}, { timeoutMs: 5_000 });
|
||||
} catch {}
|
||||
}
|
||||
cdp.close();
|
||||
}
|
||||
|
||||
if (chrome) killChrome(chrome);
|
||||
if (chrome) await gracefulKillChrome(chrome, port);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+32
-1
@@ -478,7 +478,7 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
chrome.kill("SIGTERM");
|
||||
} catch {}
|
||||
setTimeout(() => {
|
||||
if (!chrome.killed) {
|
||||
if (chrome.exitCode === null && chrome.signalCode === null) {
|
||||
try {
|
||||
chrome.kill("SIGKILL");
|
||||
} catch {}
|
||||
@@ -486,6 +486,37 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
}, 2_000).unref?.();
|
||||
}
|
||||
|
||||
export async function gracefulKillChrome(
|
||||
chrome: ChildProcess,
|
||||
port?: number,
|
||||
timeoutMs = 6_000,
|
||||
): Promise<void> {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
|
||||
const exitPromise = new Promise<void>((resolve) => {
|
||||
chrome.once("exit", () => resolve());
|
||||
});
|
||||
|
||||
killChrome(chrome);
|
||||
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
if (port !== undefined && !await isPortListening(port, 250)) return;
|
||||
|
||||
const exited = await Promise.race([
|
||||
exitPromise.then(() => true),
|
||||
sleep(100).then(() => false),
|
||||
]);
|
||||
if (exited) return;
|
||||
}
|
||||
|
||||
await Promise.race([
|
||||
exitPromise,
|
||||
sleep(250),
|
||||
]);
|
||||
}
|
||||
|
||||
export async function openPageSession(options: OpenPageSessionOptions): Promise<PageSession> {
|
||||
let targetId: string;
|
||||
let createdTarget = false;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"lockfileVersion": 1,
|
||||
"configVersion": 0,
|
||||
"workspaces": {
|
||||
"": {
|
||||
"name": "baoyu-markdown-to-html-scripts",
|
||||
|
||||
@@ -352,7 +352,7 @@ WECHAT_APP_SECRET=<user_input>
|
||||
| Field | If Missing |
|
||||
|-------|------------|
|
||||
| Title | Prompt: "Enter title, or press Enter to auto-generate from content" |
|
||||
| Summary | Prompt: "Enter summary, or press Enter to auto-generate (recommended for SEO)" |
|
||||
| Summary | Use fallback chain: frontmatter `description` → frontmatter `summary` → prompt user or auto-generate |
|
||||
| Author | Use fallback chain: CLI `--author` → frontmatter `author` → EXTEND.md `default_author` |
|
||||
|
||||
**Auto-Generation Logic**:
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"lockfileVersion": 1,
|
||||
"configVersion": 0,
|
||||
"workspaces": {
|
||||
"": {
|
||||
"name": "baoyu-post-to-wechat-scripts",
|
||||
|
||||
@@ -478,7 +478,7 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
chrome.kill("SIGTERM");
|
||||
} catch {}
|
||||
setTimeout(() => {
|
||||
if (!chrome.killed) {
|
||||
if (chrome.exitCode === null && chrome.signalCode === null) {
|
||||
try {
|
||||
chrome.kill("SIGKILL");
|
||||
} catch {}
|
||||
@@ -486,6 +486,37 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
}, 2_000).unref?.();
|
||||
}
|
||||
|
||||
export async function gracefulKillChrome(
|
||||
chrome: ChildProcess,
|
||||
port?: number,
|
||||
timeoutMs = 6_000,
|
||||
): Promise<void> {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
|
||||
const exitPromise = new Promise<void>((resolve) => {
|
||||
chrome.once("exit", () => resolve());
|
||||
});
|
||||
|
||||
killChrome(chrome);
|
||||
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
if (port !== undefined && !await isPortListening(port, 250)) return;
|
||||
|
||||
const exited = await Promise.race([
|
||||
exitPromise.then(() => true),
|
||||
sleep(100).then(() => false),
|
||||
]);
|
||||
if (exited) return;
|
||||
}
|
||||
|
||||
await Promise.race([
|
||||
exitPromise,
|
||||
sleep(250),
|
||||
]);
|
||||
}
|
||||
|
||||
export async function openPageSession(options: OpenPageSessionOptions): Promise<PageSession> {
|
||||
let targetId: string;
|
||||
let createdTarget = false;
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
type WechatUploadAsset,
|
||||
prepareWechatBodyImageUpload,
|
||||
needsWechatBodyImageProcessing,
|
||||
detectImageFormatFromBuffer,
|
||||
} from "./wechat-image-processor.ts";
|
||||
|
||||
interface AccessTokenResponse {
|
||||
@@ -138,6 +139,16 @@ async function loadUploadAsset(
|
||||
contentType = mimeTypes[fileExt] || "image/jpeg";
|
||||
}
|
||||
|
||||
// Detect actual format from magic bytes to fix extension/content-type mismatches
|
||||
// (e.g. CDNs serving WebP for URLs with .png extension)
|
||||
const detected = detectImageFormatFromBuffer(fileBuffer);
|
||||
if (detected && detected.contentType !== contentType) {
|
||||
console.error(`[wechat-api] Format mismatch: ${filename} declared as ${contentType}, actual ${detected.contentType}`);
|
||||
contentType = detected.contentType;
|
||||
fileExt = detected.fileExt;
|
||||
filename = `${path.basename(filename, path.extname(filename))}${detected.fileExt}`;
|
||||
}
|
||||
|
||||
return {
|
||||
buffer: fileBuffer,
|
||||
filename,
|
||||
|
||||
@@ -52,6 +52,39 @@ const MIME_TO_EXT: Record<string, string> = {
|
||||
const JPEG_QUALITY_STEPS = [82, 74, 66, 58, 50, 42, 34];
|
||||
const MAX_WIDTH_STEPS = [2560, 2048, 1600, 1280, 1024, 800, 640, 480];
|
||||
|
||||
/**
|
||||
* Detect actual image format from buffer magic bytes.
|
||||
* Returns corrected { contentType, fileExt } or null if unknown.
|
||||
*/
|
||||
export function detectImageFormatFromBuffer(buffer: Buffer): { contentType: string; fileExt: string } | null {
|
||||
if (buffer.length < 12) return null;
|
||||
|
||||
// WebP: RIFF....WEBP
|
||||
if (
|
||||
buffer[0] === 0x52 && buffer[1] === 0x49 && buffer[2] === 0x46 && buffer[3] === 0x46 &&
|
||||
buffer[8] === 0x57 && buffer[9] === 0x45 && buffer[10] === 0x42 && buffer[11] === 0x50
|
||||
) {
|
||||
return { contentType: "image/webp", fileExt: ".webp" };
|
||||
}
|
||||
// PNG: 89 50 4E 47
|
||||
if (buffer[0] === 0x89 && buffer[1] === 0x50 && buffer[2] === 0x4e && buffer[3] === 0x47) {
|
||||
return { contentType: "image/png", fileExt: ".png" };
|
||||
}
|
||||
// JPEG: FF D8 FF
|
||||
if (buffer[0] === 0xff && buffer[1] === 0xd8 && buffer[2] === 0xff) {
|
||||
return { contentType: "image/jpeg", fileExt: ".jpg" };
|
||||
}
|
||||
// GIF: GIF8
|
||||
if (buffer[0] === 0x47 && buffer[1] === 0x49 && buffer[2] === 0x46 && buffer[3] === 0x38) {
|
||||
return { contentType: "image/gif", fileExt: ".gif" };
|
||||
}
|
||||
// BMP: BM
|
||||
if (buffer[0] === 0x42 && buffer[1] === 0x4d) {
|
||||
return { contentType: "image/bmp", fileExt: ".bmp" };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
let webpDecoderReady: Promise<void> | undefined;
|
||||
|
||||
type JimpImage = Awaited<ReturnType<typeof Jimp.read>>;
|
||||
@@ -209,7 +242,8 @@ export async function prepareWechatBodyImageUpload(
|
||||
|
||||
const image = await loadImageForProcessing(asset);
|
||||
const widths = buildCandidateWidths(image.bitmap.width);
|
||||
const preferPng = imageHasTransparency(image) || ensureFileExt(asset) === ".png";
|
||||
const ext = ensureFileExt(asset);
|
||||
const preferPng = imageHasTransparency(image) || ext === ".png" || ext === ".webp";
|
||||
const processingNotes = buildProcessingNotes(asset);
|
||||
|
||||
for (const width of widths) {
|
||||
|
||||
@@ -141,17 +141,25 @@ If the check fails (warnings in output), alert the user with the specific issues
|
||||
|
||||
---
|
||||
|
||||
## Post Type Selection
|
||||
|
||||
Unless the user explicitly specifies the post type:
|
||||
- **Markdown file** (`.md`) → **Headline Article** (头条文章)
|
||||
- **Plain text** / text with images → **Regular Post**
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Chrome debug port not ready
|
||||
|
||||
If a script fails with `Chrome debug port not ready` or `Unable to connect`, kill existing Chrome CDP instances first, then retry:
|
||||
If a script fails with `Chrome debug port not ready` or `Unable to connect`, kill only the CDP Chrome instances (those with `--remote-debugging-port` AND the baoyu-skills profile), then retry:
|
||||
|
||||
```bash
|
||||
pkill -f "Chrome.*remote-debugging-port" 2>/dev/null; pkill -f "Chromium.*remote-debugging-port" 2>/dev/null; sleep 2
|
||||
pkill -f "remote-debugging-port.*baoyu-skills/chrome-profile" 2>/dev/null; sleep 2
|
||||
```
|
||||
|
||||
**Important**: This should be done automatically -- when encountering this error, kill Chrome CDP instances and retry the command without asking the user.
|
||||
**CRITICAL**: Never kill all Chrome processes (`pkill -f "Google Chrome"`). Only kill Chrome instances launched by CDP with the baoyu-skills profile directory. The user may have regular Chrome windows open.
|
||||
|
||||
**Important**: This should be done automatically -- when encountering this error, kill the CDP Chrome instances and retry the command without asking the user.
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"lockfileVersion": 1,
|
||||
"configVersion": 0,
|
||||
"workspaces": {
|
||||
"": {
|
||||
"name": "baoyu-post-to-weibo-scripts",
|
||||
|
||||
@@ -478,7 +478,7 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
chrome.kill("SIGTERM");
|
||||
} catch {}
|
||||
setTimeout(() => {
|
||||
if (!chrome.killed) {
|
||||
if (chrome.exitCode === null && chrome.signalCode === null) {
|
||||
try {
|
||||
chrome.kill("SIGKILL");
|
||||
} catch {}
|
||||
@@ -486,6 +486,37 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
}, 2_000).unref?.();
|
||||
}
|
||||
|
||||
export async function gracefulKillChrome(
|
||||
chrome: ChildProcess,
|
||||
port?: number,
|
||||
timeoutMs = 6_000,
|
||||
): Promise<void> {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
|
||||
const exitPromise = new Promise<void>((resolve) => {
|
||||
chrome.once("exit", () => resolve());
|
||||
});
|
||||
|
||||
killChrome(chrome);
|
||||
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
if (port !== undefined && !await isPortListening(port, 250)) return;
|
||||
|
||||
const exited = await Promise.race([
|
||||
exitPromise.then(() => true),
|
||||
sleep(100).then(() => false),
|
||||
]);
|
||||
if (exited) return;
|
||||
}
|
||||
|
||||
await Promise.race([
|
||||
exitPromise,
|
||||
sleep(250),
|
||||
]);
|
||||
}
|
||||
|
||||
export async function openPageSession(options: OpenPageSessionOptions): Promise<PageSession> {
|
||||
let targetId: string;
|
||||
let createdTarget = false;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"lockfileVersion": 1,
|
||||
"configVersion": 0,
|
||||
"workspaces": {
|
||||
"": {
|
||||
"name": "baoyu-post-to-x-scripts",
|
||||
|
||||
@@ -478,7 +478,7 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
chrome.kill("SIGTERM");
|
||||
} catch {}
|
||||
setTimeout(() => {
|
||||
if (!chrome.killed) {
|
||||
if (chrome.exitCode === null && chrome.signalCode === null) {
|
||||
try {
|
||||
chrome.kill("SIGKILL");
|
||||
} catch {}
|
||||
@@ -486,6 +486,37 @@ export function killChrome(chrome: ChildProcess): void {
|
||||
}, 2_000).unref?.();
|
||||
}
|
||||
|
||||
export async function gracefulKillChrome(
|
||||
chrome: ChildProcess,
|
||||
port?: number,
|
||||
timeoutMs = 6_000,
|
||||
): Promise<void> {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
|
||||
const exitPromise = new Promise<void>((resolve) => {
|
||||
chrome.once("exit", () => resolve());
|
||||
});
|
||||
|
||||
killChrome(chrome);
|
||||
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
|
||||
if (port !== undefined && !await isPortListening(port, 250)) return;
|
||||
|
||||
const exited = await Promise.race([
|
||||
exitPromise.then(() => true),
|
||||
sleep(100).then(() => false),
|
||||
]);
|
||||
if (exited) return;
|
||||
}
|
||||
|
||||
await Promise.race([
|
||||
exitPromise,
|
||||
sleep(250),
|
||||
]);
|
||||
}
|
||||
|
||||
export async function openPageSession(options: OpenPageSessionOptions): Promise<PageSession> {
|
||||
let targetId: string;
|
||||
let createdTarget = false;
|
||||
|
||||
@@ -7,10 +7,12 @@ import {
|
||||
copyImageToClipboard,
|
||||
findExistingChromeDebugPort,
|
||||
getDefaultProfileDir,
|
||||
gracefulKillChrome,
|
||||
launchChrome,
|
||||
openPageSession,
|
||||
pasteFromClipboard,
|
||||
sleep,
|
||||
waitForXSessionPersistence,
|
||||
waitForChromeDebugPort,
|
||||
} from './x-utils.js';
|
||||
|
||||
@@ -44,6 +46,8 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
else console.log(`[x-browser] Launching Chrome (profile: ${profileDir})`);
|
||||
|
||||
let cdp: CdpConnection | null = null;
|
||||
let sessionId: string | null = null;
|
||||
let loggedInDuringRun = false;
|
||||
|
||||
try {
|
||||
const wsUrl = await waitForChromeDebugPort(port, 30_000, { includeLastError: true });
|
||||
@@ -56,9 +60,11 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
matchTarget: (target) => target.type === 'page' && target.url.includes('x.com'),
|
||||
enablePage: true,
|
||||
enableRuntime: true,
|
||||
enableNetwork: true,
|
||||
});
|
||||
const { sessionId } = page;
|
||||
await cdp.send('Input.setIgnoreInputEvents', { ignore: false }, { sessionId });
|
||||
const activeSessionId = page.sessionId;
|
||||
sessionId = activeSessionId;
|
||||
await cdp.send('Input.setIgnoreInputEvents', { ignore: false }, { sessionId: activeSessionId });
|
||||
|
||||
console.log('[x-browser] Waiting for X editor...');
|
||||
await sleep(3000);
|
||||
@@ -69,7 +75,7 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
const result = await cdp!.send<{ result: { value: boolean } }>('Runtime.evaluate', {
|
||||
expression: `!!document.querySelector('[data-testid="tweetTextarea_0"]')`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
if (result.result.value) return true;
|
||||
await sleep(1000);
|
||||
}
|
||||
@@ -82,6 +88,7 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
console.log('[x-browser] Waiting for login...');
|
||||
const loggedIn = await waitForEditor();
|
||||
if (!loggedIn) throw new Error('Timed out waiting for X editor. Please log in first.');
|
||||
loggedInDuringRun = true;
|
||||
}
|
||||
|
||||
if (text) {
|
||||
@@ -94,7 +101,7 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
document.execCommand('insertText', false, ${JSON.stringify(text)});
|
||||
}
|
||||
`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(500);
|
||||
}
|
||||
|
||||
@@ -115,7 +122,7 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
const imgCountBefore = await cdp.send<{ result: { value: number } }>('Runtime.evaluate', {
|
||||
expression: `document.querySelectorAll('img[src^="blob:"]').length`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
|
||||
// Wait for clipboard to be ready
|
||||
await sleep(500);
|
||||
@@ -123,7 +130,7 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
// Focus the editor
|
||||
await cdp.send('Runtime.evaluate', {
|
||||
expression: `document.querySelector('[data-testid="tweetTextarea_0"]')?.focus()`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(200);
|
||||
|
||||
// Use paste script (handles platform differences, activates Chrome)
|
||||
@@ -140,14 +147,14 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
code: 'KeyV',
|
||||
modifiers,
|
||||
windowsVirtualKeyCode: 86,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await cdp.send('Input.dispatchKeyEvent', {
|
||||
type: 'keyUp',
|
||||
key: 'v',
|
||||
code: 'KeyV',
|
||||
modifiers,
|
||||
windowsVirtualKeyCode: 86,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
}
|
||||
|
||||
console.log('[x-browser] Verifying image upload...');
|
||||
@@ -158,7 +165,7 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
const r = await cdp!.send<{ result: { value: number } }>('Runtime.evaluate', {
|
||||
expression: `document.querySelectorAll('img[src^="blob:"]').length`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
if (r.result.value >= expectedImgCount) {
|
||||
imgUploadOk = true;
|
||||
break;
|
||||
@@ -177,18 +184,32 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
|
||||
console.log('[x-browser] Submitting post...');
|
||||
await cdp.send('Runtime.evaluate', {
|
||||
expression: `document.querySelector('[data-testid="tweetButton"]')?.click()`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(2000);
|
||||
console.log('[x-browser] Post submitted!');
|
||||
} else {
|
||||
console.log('[x-browser] Post composed. Please review and click the publish button in the browser.');
|
||||
}
|
||||
} finally {
|
||||
let leaveChromeOpen = !submit;
|
||||
if (chrome && submit && loggedInDuringRun && cdp && sessionId) {
|
||||
console.log('[x-browser] Waiting for X session cookies to persist...');
|
||||
const sessionReady = await waitForXSessionPersistence({ cdp, sessionId });
|
||||
if (!sessionReady) {
|
||||
console.warn('[x-browser] X session cookies not observed yet. Leaving Chrome open so login can finish persisting.');
|
||||
leaveChromeOpen = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (cdp) {
|
||||
cdp.close();
|
||||
}
|
||||
if (chrome) {
|
||||
chrome.unref();
|
||||
if (leaveChromeOpen) {
|
||||
chrome.unref();
|
||||
} else {
|
||||
await gracefulKillChrome(chrome, port);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,10 +5,11 @@ import {
|
||||
CdpConnection,
|
||||
findExistingChromeDebugPort,
|
||||
getDefaultProfileDir,
|
||||
killChrome,
|
||||
gracefulKillChrome,
|
||||
launchChrome,
|
||||
openPageSession,
|
||||
sleep,
|
||||
waitForXSessionPersistence,
|
||||
waitForChromeDebugPort,
|
||||
} from './x-utils.js';
|
||||
|
||||
@@ -49,7 +50,9 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
else console.log(`[x-quote] Launching Chrome (profile: ${profileDir})`);
|
||||
|
||||
let cdp: CdpConnection | null = null;
|
||||
let sessionId: string | null = null;
|
||||
let targetId: string | null = null;
|
||||
let loggedInDuringRun = false;
|
||||
|
||||
try {
|
||||
const wsUrl = await waitForChromeDebugPort(port, 30_000, { includeLastError: true });
|
||||
@@ -62,8 +65,10 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
matchTarget: (target) => target.type === 'page' && target.url.includes('x.com'),
|
||||
enablePage: true,
|
||||
enableRuntime: true,
|
||||
enableNetwork: true,
|
||||
});
|
||||
const { sessionId } = page;
|
||||
const activeSessionId = page.sessionId;
|
||||
sessionId = activeSessionId;
|
||||
targetId = page.targetId;
|
||||
|
||||
console.log('[x-quote] Waiting for tweet to load...');
|
||||
@@ -76,7 +81,7 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
const result = await cdp!.send<{ result: { value: boolean } }>('Runtime.evaluate', {
|
||||
expression: `!!document.querySelector('[data-testid="retweet"]')`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
if (result.result.value) return true;
|
||||
await sleep(1000);
|
||||
}
|
||||
@@ -89,13 +94,14 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
console.log('[x-quote] Waiting for login...');
|
||||
const loggedIn = await waitForRetweetButton();
|
||||
if (!loggedIn) throw new Error('Timed out waiting for tweet. Please log in first or check the tweet URL.');
|
||||
loggedInDuringRun = true;
|
||||
}
|
||||
|
||||
// Click the retweet button
|
||||
console.log('[x-quote] Clicking retweet button...');
|
||||
await cdp.send('Runtime.evaluate', {
|
||||
expression: `document.querySelector('[data-testid="retweet"]')?.click()`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(1000);
|
||||
|
||||
// Wait for and click the "Quote" option in the menu
|
||||
@@ -106,7 +112,7 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
const result = await cdp!.send<{ result: { value: boolean } }>('Runtime.evaluate', {
|
||||
expression: `!!document.querySelector('[data-testid="Dropdown"] [role="menuitem"]:nth-child(2)')`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
if (result.result.value) return true;
|
||||
await sleep(200);
|
||||
}
|
||||
@@ -121,7 +127,7 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
// Click the quote option (second menu item)
|
||||
await cdp.send('Runtime.evaluate', {
|
||||
expression: `document.querySelector('[data-testid="Dropdown"] [role="menuitem"]:nth-child(2)')?.click()`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(2000);
|
||||
|
||||
// Wait for the quote compose dialog
|
||||
@@ -132,7 +138,7 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
const result = await cdp!.send<{ result: { value: boolean } }>('Runtime.evaluate', {
|
||||
expression: `!!document.querySelector('[data-testid="tweetTextarea_0"]')`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
if (result.result.value) return true;
|
||||
await sleep(200);
|
||||
}
|
||||
@@ -150,12 +156,12 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
// Use CDP Input.insertText for more reliable text insertion
|
||||
await cdp.send('Runtime.evaluate', {
|
||||
expression: `document.querySelector('[data-testid="tweetTextarea_0"]')?.focus()`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(200);
|
||||
|
||||
await cdp.send('Input.insertText', {
|
||||
text: comment,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(500);
|
||||
}
|
||||
|
||||
@@ -163,7 +169,7 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
console.log('[x-quote] Submitting quote post...');
|
||||
await cdp.send('Runtime.evaluate', {
|
||||
expression: `document.querySelector('[data-testid="tweetButton"]')?.click()`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(2000);
|
||||
console.log('[x-quote] Quote post submitted!');
|
||||
} else {
|
||||
@@ -172,15 +178,29 @@ export async function quotePost(options: QuoteOptions): Promise<void> {
|
||||
await sleep(30_000);
|
||||
}
|
||||
} finally {
|
||||
let leaveChromeOpen = false;
|
||||
if (chrome && loggedInDuringRun && cdp && sessionId) {
|
||||
console.log('[x-quote] Waiting for X session cookies to persist...');
|
||||
const sessionReady = await waitForXSessionPersistence({ cdp, sessionId });
|
||||
if (!sessionReady) {
|
||||
console.warn('[x-quote] X session cookies not observed yet. Leaving Chrome open so login can finish persisting.');
|
||||
leaveChromeOpen = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (cdp) {
|
||||
if (reusing && targetId) {
|
||||
try { await cdp.send('Target.closeTarget', { targetId }, { timeoutMs: 5_000 }); } catch {}
|
||||
} else if (!reusing) {
|
||||
try { await cdp.send('Browser.close', {}, { timeoutMs: 5_000 }); } catch {}
|
||||
}
|
||||
cdp.close();
|
||||
}
|
||||
if (chrome) killChrome(chrome);
|
||||
if (chrome) {
|
||||
if (leaveChromeOpen) {
|
||||
chrome.unref();
|
||||
} else {
|
||||
await gracefulKillChrome(chrome, port);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
|
||||
import {
|
||||
buildXSessionCookieMap,
|
||||
hasChromeLockArtifacts,
|
||||
hasRequiredXSessionCookies,
|
||||
shouldRetryChromeLaunch,
|
||||
} from "./x-utils.ts";
|
||||
|
||||
test("hasChromeLockArtifacts detects Chrome singleton artifacts", () => {
|
||||
assert.equal(hasChromeLockArtifacts(["SingletonSocket"]), true);
|
||||
assert.equal(hasChromeLockArtifacts(["chrome.pid"]), true);
|
||||
assert.equal(hasChromeLockArtifacts(["Cookies", "Preferences"]), false);
|
||||
});
|
||||
|
||||
test("shouldRetryChromeLaunch only retries when no live owner exists", () => {
|
||||
assert.equal(
|
||||
shouldRetryChromeLaunch({ lockArtifactsPresent: true, hasLiveOwner: false }),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
shouldRetryChromeLaunch({ lockArtifactsPresent: true, hasLiveOwner: true }),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
shouldRetryChromeLaunch({ lockArtifactsPresent: false, hasLiveOwner: false }),
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
test("buildXSessionCookieMap keeps only non-empty cookies", () => {
|
||||
assert.deepEqual(
|
||||
buildXSessionCookieMap([
|
||||
{ name: "auth_token", value: "auth" },
|
||||
{ name: "ct0", value: "csrf" },
|
||||
{ name: "twid", value: "u=123" },
|
||||
{ name: "ct0", value: "" },
|
||||
{ name: "", value: "ignored" },
|
||||
{ name: "gt", value: undefined },
|
||||
]),
|
||||
{
|
||||
auth_token: "auth",
|
||||
ct0: "csrf",
|
||||
twid: "u=123",
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test("hasRequiredXSessionCookies requires auth_token and ct0", () => {
|
||||
assert.equal(hasRequiredXSessionCookies({ auth_token: "auth" }), false);
|
||||
assert.equal(hasRequiredXSessionCookies({ ct0: "csrf" }), false);
|
||||
assert.equal(hasRequiredXSessionCookies({ auth_token: "auth", ct0: "csrf" }), true);
|
||||
});
|
||||
@@ -1,4 +1,5 @@
|
||||
import { execSync, spawnSync } from 'node:child_process';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import process from 'node:process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
@@ -8,6 +9,7 @@ import {
|
||||
findChromeExecutable as findChromeExecutableBase,
|
||||
findExistingChromeDebugPort as findExistingChromeDebugPortBase,
|
||||
getFreePort as getFreePortBase,
|
||||
gracefulKillChrome,
|
||||
killChrome,
|
||||
launchChrome as launchChromeBase,
|
||||
openPageSession,
|
||||
@@ -17,9 +19,21 @@ import {
|
||||
type PlatformCandidates,
|
||||
} from 'baoyu-chrome-cdp';
|
||||
|
||||
export { CdpConnection, killChrome, openPageSession, sleep, waitForChromeDebugPort };
|
||||
export { CdpConnection, gracefulKillChrome, killChrome, openPageSession, sleep, waitForChromeDebugPort };
|
||||
export type { PlatformCandidates } from 'baoyu-chrome-cdp';
|
||||
|
||||
const X_SESSION_URLS = ['https://x.com/', 'https://twitter.com/'] as const;
|
||||
const REQUIRED_X_SESSION_COOKIES = ['auth_token', 'ct0'] as const;
|
||||
|
||||
interface CookieLike {
|
||||
name?: string;
|
||||
value?: string | null;
|
||||
}
|
||||
|
||||
interface NetworkGetCookiesResult {
|
||||
cookies?: CookieLike[];
|
||||
}
|
||||
|
||||
export const CHROME_CANDIDATES_BASIC: PlatformCandidates = {
|
||||
darwin: [
|
||||
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
|
||||
@@ -94,15 +108,105 @@ export async function findExistingChromeDebugPort(profileDir: string): Promise<n
|
||||
return await findExistingChromeDebugPortBase({ profileDir });
|
||||
}
|
||||
|
||||
export async function launchChrome(
|
||||
const CHROME_LOCK_FILES = ['SingletonLock', 'SingletonSocket', 'SingletonCookie', 'chrome.pid'] as const;
|
||||
|
||||
export function hasChromeLockArtifacts(entries: readonly string[]): boolean {
|
||||
return CHROME_LOCK_FILES.some((name) => entries.includes(name));
|
||||
}
|
||||
|
||||
export function shouldRetryChromeLaunch(options: {
|
||||
lockArtifactsPresent: boolean;
|
||||
hasLiveOwner: boolean;
|
||||
}): boolean {
|
||||
return options.lockArtifactsPresent && !options.hasLiveOwner;
|
||||
}
|
||||
|
||||
export function buildXSessionCookieMap(cookies: readonly CookieLike[]): Record<string, string> {
|
||||
const cookieMap: Record<string, string> = {};
|
||||
for (const cookie of cookies) {
|
||||
const name = cookie.name?.trim();
|
||||
const value = cookie.value?.trim();
|
||||
if (!name || !value) {
|
||||
continue;
|
||||
}
|
||||
cookieMap[name] = value;
|
||||
}
|
||||
return cookieMap;
|
||||
}
|
||||
|
||||
export function hasRequiredXSessionCookies(cookieMap: Record<string, string>): boolean {
|
||||
return REQUIRED_X_SESSION_COOKIES.every((name) => Boolean(cookieMap[name]));
|
||||
}
|
||||
|
||||
export async function readXSessionCookieMap(
|
||||
cdp: CdpConnection,
|
||||
sessionId?: string,
|
||||
): Promise<Record<string, string>> {
|
||||
const { cookies } = await cdp.send<NetworkGetCookiesResult>(
|
||||
'Network.getCookies',
|
||||
{ urls: [...X_SESSION_URLS] },
|
||||
{
|
||||
sessionId,
|
||||
timeoutMs: 5_000,
|
||||
},
|
||||
);
|
||||
return buildXSessionCookieMap(cookies ?? []);
|
||||
}
|
||||
|
||||
export async function waitForXSessionPersistence(options: {
|
||||
cdp: CdpConnection;
|
||||
sessionId?: string;
|
||||
timeoutMs?: number;
|
||||
pollIntervalMs?: number;
|
||||
}): Promise<boolean> {
|
||||
const timeoutMs = options.timeoutMs ?? 15_000;
|
||||
const pollIntervalMs = options.pollIntervalMs ?? 1_000;
|
||||
const start = Date.now();
|
||||
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
const cookieMap = await readXSessionCookieMap(options.cdp, options.sessionId).catch(() => ({}));
|
||||
if (hasRequiredXSessionCookies(cookieMap)) {
|
||||
return true;
|
||||
}
|
||||
await sleep(pollIntervalMs);
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
function cleanStaleLockFiles(profileDir: string): void {
|
||||
for (const name of CHROME_LOCK_FILES) {
|
||||
try { fs.unlinkSync(path.join(profileDir, name)); } catch {}
|
||||
}
|
||||
}
|
||||
|
||||
function hasLiveChromeOwner(profileDir: string): boolean {
|
||||
if (process.platform === 'win32') return false;
|
||||
try {
|
||||
const result = spawnSync('ps', ['aux'], {
|
||||
encoding: 'utf8',
|
||||
timeout: 5000,
|
||||
});
|
||||
if (result.status !== 0 || !result.stdout) return false;
|
||||
return result.stdout.split('\n').some((line) => line.includes(`--user-data-dir=${profileDir}`));
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function listProfileDirEntries(profileDir: string): Promise<string[]> {
|
||||
try {
|
||||
return await fs.promises.readdir(profileDir);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
async function launchChromeOnce(
|
||||
url: string,
|
||||
profileDir: string,
|
||||
candidates: PlatformCandidates,
|
||||
chromePathOverride?: string,
|
||||
chromePath: string,
|
||||
): Promise<{ chrome: Awaited<ReturnType<typeof launchChromeBase>>; port: number }> {
|
||||
const chromePath = chromePathOverride?.trim() || findChromeExecutable(candidates);
|
||||
if (!chromePath) throw new Error('Chrome not found. Set X_BROWSER_CHROME_PATH env var.');
|
||||
|
||||
const port = await getFreePort();
|
||||
const chrome = await launchChromeBase({
|
||||
chromePath,
|
||||
@@ -112,7 +216,37 @@ export async function launchChrome(
|
||||
extraArgs: ['--start-maximized'],
|
||||
});
|
||||
|
||||
return { chrome, port };
|
||||
try {
|
||||
await waitForChromeDebugPort(port, 30_000, { includeLastError: true });
|
||||
return { chrome, port };
|
||||
} catch (error) {
|
||||
killChrome(chrome);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export async function launchChrome(
|
||||
url: string,
|
||||
profileDir: string,
|
||||
candidates: PlatformCandidates,
|
||||
chromePathOverride?: string,
|
||||
): Promise<{ chrome: Awaited<ReturnType<typeof launchChromeBase>>; port: number }> {
|
||||
const chromePath = chromePathOverride?.trim() || findChromeExecutable(candidates);
|
||||
if (!chromePath) throw new Error('Chrome not found. Set X_BROWSER_CHROME_PATH env var.');
|
||||
|
||||
try {
|
||||
return await launchChromeOnce(url, profileDir, chromePath);
|
||||
} catch (error) {
|
||||
const entries = await listProfileDirEntries(profileDir);
|
||||
const shouldRetry = shouldRetryChromeLaunch({
|
||||
lockArtifactsPresent: hasChromeLockArtifacts(entries),
|
||||
hasLiveOwner: hasLiveChromeOwner(profileDir),
|
||||
});
|
||||
if (!shouldRetry) throw error;
|
||||
|
||||
cleanStaleLockFiles(profileDir);
|
||||
return await launchChromeOnce(url, profileDir, chromePath);
|
||||
}
|
||||
}
|
||||
|
||||
export function getScriptDir(): string {
|
||||
|
||||
@@ -7,10 +7,11 @@ import {
|
||||
CdpConnection,
|
||||
findExistingChromeDebugPort,
|
||||
getDefaultProfileDir,
|
||||
killChrome,
|
||||
gracefulKillChrome,
|
||||
launchChrome,
|
||||
openPageSession,
|
||||
sleep,
|
||||
waitForXSessionPersistence,
|
||||
waitForChromeDebugPort,
|
||||
} from './x-utils.js';
|
||||
|
||||
@@ -49,7 +50,9 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
else console.log(`[x-video] Launching Chrome (profile: ${profileDir})`);
|
||||
|
||||
let cdp: CdpConnection | null = null;
|
||||
let sessionId: string | null = null;
|
||||
let targetId: string | null = null;
|
||||
let loggedInDuringRun = false;
|
||||
|
||||
try {
|
||||
const wsUrl = await waitForChromeDebugPort(port, 30_000, { includeLastError: true });
|
||||
@@ -63,10 +66,12 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
enablePage: true,
|
||||
enableRuntime: true,
|
||||
enableDom: true,
|
||||
enableNetwork: true,
|
||||
});
|
||||
const { sessionId } = page;
|
||||
const activeSessionId = page.sessionId;
|
||||
sessionId = activeSessionId;
|
||||
targetId = page.targetId;
|
||||
await cdp.send('Input.setIgnoreInputEvents', { ignore: false }, { sessionId });
|
||||
await cdp.send('Input.setIgnoreInputEvents', { ignore: false }, { sessionId: activeSessionId });
|
||||
|
||||
console.log('[x-video] Waiting for X editor...');
|
||||
await sleep(3000);
|
||||
@@ -77,7 +82,7 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
const result = await cdp!.send<{ result: { value: boolean } }>('Runtime.evaluate', {
|
||||
expression: `!!document.querySelector('[data-testid="tweetTextarea_0"]')`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
if (result.result.value) return true;
|
||||
await sleep(1000);
|
||||
}
|
||||
@@ -90,16 +95,17 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
console.log('[x-video] Waiting for login...');
|
||||
const loggedIn = await waitForEditor();
|
||||
if (!loggedIn) throw new Error('Timed out waiting for X editor. Please log in first.');
|
||||
loggedInDuringRun = true;
|
||||
}
|
||||
|
||||
// Upload video FIRST (before typing text to avoid text being cleared)
|
||||
console.log('[x-video] Uploading video...');
|
||||
|
||||
const { root } = await cdp.send<{ root: { nodeId: number } }>('DOM.getDocument', {}, { sessionId });
|
||||
const { root } = await cdp.send<{ root: { nodeId: number } }>('DOM.getDocument', {}, { sessionId: activeSessionId });
|
||||
const { nodeId } = await cdp.send<{ nodeId: number }>('DOM.querySelector', {
|
||||
nodeId: root.nodeId,
|
||||
selector: 'input[type="file"][accept*="video"], input[data-testid="fileInput"], input[type="file"]',
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
|
||||
if (!nodeId || nodeId === 0) {
|
||||
throw new Error('Could not find file input for video upload.');
|
||||
@@ -108,7 +114,7 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
await cdp.send('DOM.setFileInputFiles', {
|
||||
nodeId,
|
||||
files: [absVideoPath],
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
console.log('[x-video] Video file set, uploading in background...');
|
||||
|
||||
// Wait a moment for upload to start, then type text while video processes
|
||||
@@ -125,7 +131,7 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
document.execCommand('insertText', false, ${JSON.stringify(text)});
|
||||
}
|
||||
`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(500);
|
||||
}
|
||||
|
||||
@@ -143,7 +149,7 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
return { hasMedia, buttonEnabled };
|
||||
})()`,
|
||||
returnByValue: true,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
|
||||
const { hasMedia, buttonEnabled } = result.result.value;
|
||||
if (hasMedia && buttonEnabled) {
|
||||
@@ -171,7 +177,7 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
console.log('[x-video] Submitting post...');
|
||||
await cdp.send('Runtime.evaluate', {
|
||||
expression: `document.querySelector('[data-testid="tweetButton"]')?.click()`,
|
||||
}, { sessionId });
|
||||
}, { sessionId: activeSessionId });
|
||||
await sleep(5000);
|
||||
console.log('[x-video] Post submitted!');
|
||||
} else {
|
||||
@@ -179,13 +185,29 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
|
||||
console.log('[x-video] Browser stays open for review.');
|
||||
}
|
||||
} finally {
|
||||
let leaveChromeOpen = !submit;
|
||||
if (chrome && submit && loggedInDuringRun && cdp && sessionId) {
|
||||
console.log('[x-video] Waiting for X session cookies to persist...');
|
||||
const sessionReady = await waitForXSessionPersistence({ cdp, sessionId });
|
||||
if (!sessionReady) {
|
||||
console.warn('[x-video] X session cookies not observed yet. Leaving Chrome open so login can finish persisting.');
|
||||
leaveChromeOpen = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (cdp) {
|
||||
if (reusing && submit && targetId) {
|
||||
try { await cdp.send('Target.closeTarget', { targetId }, { timeoutMs: 5_000 }); } catch {}
|
||||
}
|
||||
cdp.close();
|
||||
}
|
||||
if (chrome && submit) killChrome(chrome);
|
||||
if (chrome && submit) {
|
||||
if (leaveChromeOpen) {
|
||||
chrome.unref();
|
||||
} else {
|
||||
await gracefulKillChrome(chrome, port);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"lockfileVersion": 1,
|
||||
"configVersion": 0,
|
||||
"workspaces": {
|
||||
"": {
|
||||
"name": "baoyu-url-to-markdown-scripts",
|
||||
|
||||
@@ -64,6 +64,8 @@ export interface Adapter {
|
||||
name: string;
|
||||
match(input: AdapterInput): boolean;
|
||||
checkLogin?(context: AdapterContext): Promise<AdapterLoginInfo>;
|
||||
exportCookies?(context: AdapterContext, profileDir?: string): Promise<boolean>;
|
||||
restoreCookies?(context: AdapterContext, profileDir?: string): Promise<boolean>;
|
||||
downloadMedia?(request: MediaDownloadRequest): Promise<MediaDownloadResult>;
|
||||
process(context: AdapterContext): Promise<AdapterProcessResult>;
|
||||
}
|
||||
|
||||
+17
@@ -1,4 +1,5 @@
|
||||
import type { Adapter, AdapterLoginInfo } from "../types";
|
||||
import { exportCookies, restoreCookies, type CookieSidecarConfig } from "../../browser/cookie-sidecar";
|
||||
import { detectInteractionGate } from "../../browser/interaction-gates";
|
||||
import type { ExtractedDocument } from "../../extract/document";
|
||||
import { collectMediaFromDocument } from "../../media/markdown-media";
|
||||
@@ -10,6 +11,16 @@ import { extractSingleTweetDocumentFromPayload } from "./single";
|
||||
import { extractThreadDocumentFromPayloads } from "./thread";
|
||||
import { loadFullXThread } from "./thread-loader";
|
||||
|
||||
const cookieConfig: CookieSidecarConfig = {
|
||||
urls: ["https://x.com/", "https://twitter.com/"],
|
||||
filename: "x-session-cookies.json",
|
||||
requiredCookieNames: ["auth_token", "ct0"],
|
||||
filterCookie: (c) => {
|
||||
const d = c.domain ?? "";
|
||||
return d.endsWith("x.com") || d.endsWith("twitter.com");
|
||||
},
|
||||
};
|
||||
|
||||
function extractDocumentFromPayloads(
|
||||
payloads: unknown[],
|
||||
statusId: string,
|
||||
@@ -49,6 +60,12 @@ export const xAdapter: Adapter = {
|
||||
async checkLogin(context) {
|
||||
return detectXLogin(context);
|
||||
},
|
||||
async exportCookies(context, profileDir) {
|
||||
return exportCookies(context.browser.targetSession, cookieConfig, profileDir);
|
||||
},
|
||||
async restoreCookies(context, profileDir) {
|
||||
return restoreCookies(context.browser.targetSession, cookieConfig, profileDir);
|
||||
},
|
||||
async process(context) {
|
||||
const statusId = extractStatusId(context.input.url);
|
||||
if (!statusId) {
|
||||
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
import type { AdapterContext } from "../types";
|
||||
|
||||
const X_SESSION_URLS = ["https://x.com/", "https://twitter.com/"] as const;
|
||||
const REQUIRED_X_SESSION_COOKIES = ["auth_token", "ct0"] as const;
|
||||
|
||||
interface CookieLike {
|
||||
name?: string;
|
||||
value?: string | null;
|
||||
}
|
||||
|
||||
interface NetworkGetCookiesResult {
|
||||
cookies?: CookieLike[];
|
||||
}
|
||||
|
||||
export function buildXSessionCookieMap(cookies: readonly CookieLike[]): Record<string, string> {
|
||||
const cookieMap: Record<string, string> = {};
|
||||
for (const cookie of cookies) {
|
||||
const name = cookie.name?.trim();
|
||||
const value = cookie.value?.trim();
|
||||
if (!name || !value) {
|
||||
continue;
|
||||
}
|
||||
cookieMap[name] = value;
|
||||
}
|
||||
return cookieMap;
|
||||
}
|
||||
|
||||
export function hasRequiredXSessionCookies(cookieMap: Record<string, string>): boolean {
|
||||
return REQUIRED_X_SESSION_COOKIES.every((name) => Boolean(cookieMap[name]));
|
||||
}
|
||||
|
||||
export async function readXSessionCookieMap(
|
||||
context: Pick<AdapterContext, "browser">,
|
||||
): Promise<Record<string, string>> {
|
||||
const { cookies } = await context.browser.targetSession.send<NetworkGetCookiesResult>(
|
||||
"Network.getCookies",
|
||||
{ urls: [...X_SESSION_URLS] },
|
||||
);
|
||||
return buildXSessionCookieMap(cookies ?? []);
|
||||
}
|
||||
|
||||
export async function isXSessionReady(
|
||||
context: Pick<AdapterContext, "browser">,
|
||||
): Promise<boolean> {
|
||||
const cookieMap = await readXSessionCookieMap(context);
|
||||
return hasRequiredXSessionCookies(cookieMap);
|
||||
}
|
||||
+100
-30
@@ -1,6 +1,16 @@
|
||||
import { launch, type LaunchedChrome } from "chrome-launcher";
|
||||
import WebSocket from "ws";
|
||||
import type { Logger } from "../utils/logger";
|
||||
import { ensureChromeProfileDir, findExistingChromeDebugPort, resolveChromeProfileDir } from "./profile";
|
||||
import {
|
||||
cleanChromeLockArtifacts,
|
||||
ensureChromeProfileDir,
|
||||
findChromeProcessUsingProfile,
|
||||
findExistingChromeDebugPort,
|
||||
hasChromeLockArtifacts,
|
||||
listChromeProfileEntries,
|
||||
resolveChromeProfileDir,
|
||||
shouldRetryChromeLaunchRecovery,
|
||||
} from "./profile";
|
||||
|
||||
interface ChromeVersionResponse {
|
||||
webSocketDebuggerUrl: string;
|
||||
@@ -65,6 +75,80 @@ async function tryReuseChrome(profileDir: string, logger?: Logger): Promise<Chro
|
||||
return null;
|
||||
}
|
||||
|
||||
async function launchFreshChrome(
|
||||
profileDir: string,
|
||||
options: Pick<ChromeConnectOptions, "browserPath" | "headless">,
|
||||
): Promise<ChromeConnection> {
|
||||
let launchedChrome: LaunchedChrome | null = null;
|
||||
try {
|
||||
launchedChrome = await launch({
|
||||
chromePath: options.browserPath,
|
||||
userDataDir: profileDir,
|
||||
chromeFlags: [
|
||||
"--disable-background-networking",
|
||||
"--disable-default-apps",
|
||||
"--disable-popup-blocking",
|
||||
"--disable-sync",
|
||||
"--no-first-run",
|
||||
"--no-default-browser-check",
|
||||
"--remote-allow-origins=*",
|
||||
...(!options.headless ? ["--no-startup-window"] : []),
|
||||
...(options.headless ? ["--headless=new"] : []),
|
||||
],
|
||||
});
|
||||
|
||||
const origin = `http://127.0.0.1:${launchedChrome.port}`;
|
||||
const version = await fetchJson<ChromeVersionResponse>(`${origin}/json/version`);
|
||||
|
||||
const chrome = launchedChrome;
|
||||
return {
|
||||
browserWsUrl: version.webSocketDebuggerUrl,
|
||||
origin,
|
||||
port: launchedChrome.port,
|
||||
profileDir,
|
||||
launched: true,
|
||||
async close() {
|
||||
if (!chrome) return;
|
||||
await gracefulCloseChrome(chrome, origin);
|
||||
},
|
||||
};
|
||||
} catch (error) {
|
||||
launchedChrome?.kill();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function gracefulCloseChrome(chrome: LaunchedChrome, origin: string): Promise<void> {
|
||||
try {
|
||||
const resp = await fetch(`${origin}/json/version`);
|
||||
const { webSocketDebuggerUrl } = (await resp.json()) as ChromeVersionResponse;
|
||||
if (webSocketDebuggerUrl) {
|
||||
const ws = await new Promise<WebSocket>((resolve, reject) => {
|
||||
const socket = new WebSocket(webSocketDebuggerUrl);
|
||||
socket.once("open", () => resolve(socket));
|
||||
socket.once("error", reject);
|
||||
});
|
||||
const id = 1;
|
||||
ws.send(JSON.stringify({ id, method: "Browser.close" }));
|
||||
await new Promise<void>((resolve) => {
|
||||
const timer = setTimeout(() => { ws.close(); resolve(); }, 5_000);
|
||||
ws.once("close", () => { clearTimeout(timer); resolve(); });
|
||||
});
|
||||
const exited = await new Promise<boolean>((resolve) => {
|
||||
if (chrome.pid && !isProcessAlive(chrome.pid)) { resolve(true); return; }
|
||||
const timer = setTimeout(() => resolve(false), 3_000);
|
||||
chrome.process.once("exit", () => { clearTimeout(timer); resolve(true); });
|
||||
});
|
||||
if (exited) return;
|
||||
}
|
||||
} catch {}
|
||||
chrome.kill();
|
||||
}
|
||||
|
||||
function isProcessAlive(pid: number): boolean {
|
||||
try { process.kill(pid, 0); return true; } catch { return false; }
|
||||
}
|
||||
|
||||
export async function connectChrome(options: ChromeConnectOptions): Promise<ChromeConnection> {
|
||||
if (options.cdpUrl) {
|
||||
if (options.cdpUrl.startsWith("ws://") || options.cdpUrl.startsWith("wss://")) {
|
||||
@@ -84,34 +168,20 @@ export async function connectChrome(options: ChromeConnectOptions): Promise<Chro
|
||||
}
|
||||
|
||||
options.logger?.warn(`No running Chrome debugger found for profile ${profileDir}. Launching Chrome with that profile.`);
|
||||
try {
|
||||
return await launchFreshChrome(profileDir, options);
|
||||
} catch (error) {
|
||||
const entries = await listChromeProfileEntries(profileDir);
|
||||
const shouldRetry = shouldRetryChromeLaunchRecovery({
|
||||
hasLockArtifacts: hasChromeLockArtifacts(entries),
|
||||
hasLiveOwner: findChromeProcessUsingProfile(profileDir),
|
||||
});
|
||||
if (!shouldRetry) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
const launchedChrome: LaunchedChrome = await launch({
|
||||
chromePath: options.browserPath,
|
||||
userDataDir: profileDir,
|
||||
chromeFlags: [
|
||||
"--disable-background-networking",
|
||||
"--disable-default-apps",
|
||||
"--disable-popup-blocking",
|
||||
"--disable-sync",
|
||||
"--no-first-run",
|
||||
"--no-default-browser-check",
|
||||
"--remote-allow-origins=*",
|
||||
...(!options.headless ? ["--no-startup-window"] : []),
|
||||
...(options.headless ? ["--headless=new"] : []),
|
||||
],
|
||||
});
|
||||
|
||||
const origin = `http://127.0.0.1:${launchedChrome.port}`;
|
||||
const version = await fetchJson<ChromeVersionResponse>(`${origin}/json/version`);
|
||||
|
||||
return {
|
||||
browserWsUrl: version.webSocketDebuggerUrl,
|
||||
origin,
|
||||
port: launchedChrome.port,
|
||||
profileDir,
|
||||
launched: true,
|
||||
async close() {
|
||||
launchedChrome.kill();
|
||||
},
|
||||
};
|
||||
options.logger?.warn(`Chrome launch failed with stale profile locks. Cleaning ${profileDir} and retrying once.`);
|
||||
cleanChromeLockArtifacts(profileDir);
|
||||
return await launchFreshChrome(profileDir, options);
|
||||
}
|
||||
}
|
||||
|
||||
+100
@@ -0,0 +1,100 @@
|
||||
import { readFile, writeFile, mkdir } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { resolveChromeProfileDir } from "./profile";
|
||||
import type { TargetSession } from "./cdp-client";
|
||||
|
||||
export interface CdpCookie {
|
||||
name: string;
|
||||
value: string;
|
||||
domain: string;
|
||||
path: string;
|
||||
expires: number;
|
||||
size: number;
|
||||
httpOnly: boolean;
|
||||
secure: boolean;
|
||||
session: boolean;
|
||||
sameSite?: string;
|
||||
priority?: string;
|
||||
sameParty?: boolean;
|
||||
sourceScheme?: string;
|
||||
sourcePort?: number;
|
||||
partitionKey?: string;
|
||||
}
|
||||
|
||||
interface SidecarData {
|
||||
savedAt: string;
|
||||
cookies: CdpCookie[];
|
||||
}
|
||||
|
||||
export interface CookieSidecarConfig {
|
||||
urls: readonly string[];
|
||||
filename: string;
|
||||
requiredCookieNames: readonly string[];
|
||||
filterCookie?: (cookie: CdpCookie) => boolean;
|
||||
}
|
||||
|
||||
function sidecarPath(filename: string, profileDir?: string): string {
|
||||
return join(resolveChromeProfileDir(profileDir), filename);
|
||||
}
|
||||
|
||||
function hasRequired(cookies: CdpCookie[], names: readonly string[]): boolean {
|
||||
return names.every((name) =>
|
||||
cookies.some((c) => c.name === name && Boolean(c.value)),
|
||||
);
|
||||
}
|
||||
|
||||
async function getCookies(session: TargetSession, urls: readonly string[]): Promise<CdpCookie[]> {
|
||||
const { cookies } = await session.send<{ cookies: CdpCookie[] }>(
|
||||
"Network.getCookies",
|
||||
{ urls: [...urls] },
|
||||
);
|
||||
return cookies ?? [];
|
||||
}
|
||||
|
||||
export async function exportCookies(
|
||||
session: TargetSession,
|
||||
config: CookieSidecarConfig,
|
||||
profileDir?: string,
|
||||
): Promise<boolean> {
|
||||
const all = await getCookies(session, config.urls);
|
||||
const filtered = config.filterCookie ? all.filter(config.filterCookie) : all;
|
||||
if (!hasRequired(filtered, config.requiredCookieNames)) return false;
|
||||
|
||||
const filePath = sidecarPath(config.filename, profileDir);
|
||||
await mkdir(dirname(filePath), { recursive: true });
|
||||
const data: SidecarData = { savedAt: new Date().toISOString(), cookies: filtered };
|
||||
await writeFile(filePath, JSON.stringify(data, null, 2));
|
||||
return true;
|
||||
}
|
||||
|
||||
export async function restoreCookies(
|
||||
session: TargetSession,
|
||||
config: CookieSidecarConfig,
|
||||
profileDir?: string,
|
||||
): Promise<boolean> {
|
||||
const live = await getCookies(session, config.urls);
|
||||
if (hasRequired(live, config.requiredCookieNames)) return false;
|
||||
|
||||
const filePath = sidecarPath(config.filename, profileDir);
|
||||
const raw = await readFile(filePath, "utf8");
|
||||
const data = JSON.parse(raw) as SidecarData;
|
||||
if (!data.cookies?.length) return false;
|
||||
|
||||
const now = Date.now() / 1000;
|
||||
const valid = data.cookies.filter((c) => c.session || !c.expires || c.expires > now);
|
||||
if (!hasRequired(valid, config.requiredCookieNames)) return false;
|
||||
|
||||
await session.send("Network.setCookies", {
|
||||
cookies: valid.map((c) => ({
|
||||
name: c.name,
|
||||
value: c.value,
|
||||
domain: c.domain,
|
||||
path: c.path,
|
||||
httpOnly: c.httpOnly,
|
||||
secure: c.secure,
|
||||
sameSite: c.sameSite,
|
||||
expires: c.expires,
|
||||
})),
|
||||
});
|
||||
return true;
|
||||
}
|
||||
+53
@@ -19,6 +19,8 @@ interface ChromeVersionResponse {
|
||||
webSocketDebuggerUrl?: string;
|
||||
}
|
||||
|
||||
const CHROME_LOCK_FILE_NAMES = ["SingletonLock", "SingletonSocket", "SingletonCookie", "chrome.pid"] as const;
|
||||
|
||||
function resolveDataBaseDir(): string {
|
||||
if (process.platform === "darwin") {
|
||||
return path.join(os.homedir(), "Library", "Application Support");
|
||||
@@ -61,6 +63,57 @@ export function ensureChromeProfileDir(profileDir: string): string {
|
||||
return profileDir;
|
||||
}
|
||||
|
||||
export function hasChromeLockArtifacts(entries: readonly string[]): boolean {
|
||||
return CHROME_LOCK_FILE_NAMES.some((name) => entries.includes(name));
|
||||
}
|
||||
|
||||
export function shouldRetryChromeLaunchRecovery(options: {
|
||||
hasLockArtifacts: boolean;
|
||||
hasLiveOwner: boolean;
|
||||
}): boolean {
|
||||
return options.hasLockArtifacts && !options.hasLiveOwner;
|
||||
}
|
||||
|
||||
export function findChromeProcessUsingProfile(profileDir: string): boolean {
|
||||
if (process.platform === "win32") {
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
const result = spawnSync("ps", ["aux"], {
|
||||
encoding: "utf8",
|
||||
timeout: 5_000,
|
||||
});
|
||||
if (result.status !== 0 || !result.stdout) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return result.stdout
|
||||
.split("\n")
|
||||
.some((line) => line.includes(`--user-data-dir=${profileDir}`));
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export function cleanChromeLockArtifacts(profileDir: string): void {
|
||||
for (const name of CHROME_LOCK_FILE_NAMES) {
|
||||
try {
|
||||
fs.unlinkSync(path.join(profileDir, name));
|
||||
} catch {
|
||||
// Ignore missing files and continue cleaning the remaining artifacts.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export async function listChromeProfileEntries(profileDir: string): Promise<string[]> {
|
||||
try {
|
||||
return await fs.promises.readdir(profileDir);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchWithTimeout(url: string, timeoutMs = 3_000): Promise<Response> {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
+66
-7
@@ -7,6 +7,7 @@ import { detectInteractionGate } from "../browser/interaction-gates";
|
||||
import { NetworkJournal } from "../browser/network-journal";
|
||||
import { BrowserSession } from "../browser/session";
|
||||
import { genericAdapter, resolveAdapter } from "../adapters";
|
||||
import { isXSessionReady } from "../adapters/x/session";
|
||||
import type { ExtractedDocument } from "../extract/document";
|
||||
import { renderMarkdown } from "../extract/markdown-renderer";
|
||||
import { downloadMediaAssets } from "../media/default-downloader";
|
||||
@@ -55,6 +56,7 @@ interface ForceWaitSnapshot {
|
||||
url: string;
|
||||
hasGate: boolean;
|
||||
loginState: LoginState | "unavailable";
|
||||
sessionReady: boolean;
|
||||
}
|
||||
|
||||
interface SuccessfulConvertOutput {
|
||||
@@ -78,6 +80,17 @@ function sleep(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
function isForceWaitSessionReady(snapshot: ForceWaitSnapshot): boolean {
|
||||
return snapshot.sessionReady;
|
||||
}
|
||||
|
||||
export function shouldKeepBrowserOpenAfterInteraction(options: {
|
||||
launched: boolean;
|
||||
interaction: Pick<WaitForInteractionRequest, "kind" | "provider">;
|
||||
}): boolean {
|
||||
return options.launched && options.interaction.kind === "login" && options.interaction.provider === "x";
|
||||
}
|
||||
|
||||
export function shouldAutoContinueForceWait(
|
||||
initial: ForceWaitSnapshot,
|
||||
current: ForceWaitSnapshot,
|
||||
@@ -86,15 +99,20 @@ export function shouldAutoContinueForceWait(
|
||||
return true;
|
||||
}
|
||||
|
||||
if (initial.loginState === "logged_out" && current.loginState !== "logged_out") {
|
||||
if (initial.loginState === "logged_out" && current.loginState !== "logged_out" && isForceWaitSessionReady(current)) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if (initial.loginState !== "logged_in" && current.loginState === "logged_in") {
|
||||
if (initial.loginState !== "logged_in" && current.loginState === "logged_in" && isForceWaitSessionReady(current)) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if (current.url !== initial.url && !current.hasGate && current.loginState !== "logged_out") {
|
||||
if (
|
||||
current.url !== initial.url &&
|
||||
!current.hasGate &&
|
||||
current.loginState !== "logged_out" &&
|
||||
isForceWaitSessionReady(current)
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -173,6 +191,16 @@ async function closeRuntime(runtime: RuntimeResources | null | undefined): Promi
|
||||
await runtime.chrome.close().catch(() => {});
|
||||
}
|
||||
|
||||
async function isInteractionSessionReady(
|
||||
context: AdapterContext,
|
||||
interaction: WaitForInteractionRequest,
|
||||
): Promise<boolean> {
|
||||
if (interaction.provider !== "x") {
|
||||
return true;
|
||||
}
|
||||
return await isXSessionReady(context).catch(() => false);
|
||||
}
|
||||
|
||||
async function reopenInteractiveRuntime(
|
||||
runtime: RuntimeResources,
|
||||
options: ConvertCommandOptions,
|
||||
@@ -203,6 +231,7 @@ async function captureForceWaitSnapshot(
|
||||
url,
|
||||
hasGate: Boolean(gate),
|
||||
loginState: login?.state ?? "unavailable",
|
||||
sessionReady: adapter.name === "x" ? await isXSessionReady(context).catch(() => false) : true,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -280,7 +309,7 @@ async function waitForInteraction(
|
||||
while (Date.now() - startedAt < timeoutMs) {
|
||||
if (interaction.kind === "login" && adapter.checkLogin) {
|
||||
lastLogin = await adapter.checkLogin(context);
|
||||
if (lastLogin.state === "logged_in") {
|
||||
if (lastLogin.state === "logged_in" && await isInteractionSessionReady(context, interaction)) {
|
||||
return lastLogin;
|
||||
}
|
||||
}
|
||||
@@ -303,7 +332,7 @@ async function waitForInteraction(
|
||||
}
|
||||
|
||||
lastLogin = await adapter.checkLogin(context);
|
||||
if (lastLogin.state !== "logged_out") {
|
||||
if (lastLogin.state !== "logged_out" && await isInteractionSessionReady(context, interaction)) {
|
||||
return lastLogin;
|
||||
}
|
||||
}
|
||||
@@ -347,10 +376,13 @@ export async function runConvertCommand(options: ConvertCommandOptions): Promise
|
||||
const url = normalizeUrl(options.url);
|
||||
let runtime = await openRuntime(options, options.waitMode !== "none", Boolean(options.debugDir));
|
||||
const logger = createLogger(Boolean(options.debugDir));
|
||||
let didLogin = false;
|
||||
let adapter: Adapter | null = null;
|
||||
let context: AdapterContext | null = null;
|
||||
|
||||
try {
|
||||
const adapter = resolveAdapter({ url }, options.adapter);
|
||||
let context: AdapterContext = {
|
||||
adapter = resolveAdapter({ url }, options.adapter);
|
||||
context = {
|
||||
input: { url },
|
||||
browser: runtime.browser,
|
||||
network: runtime.network,
|
||||
@@ -362,6 +394,27 @@ export async function runConvertCommand(options: ConvertCommandOptions): Promise
|
||||
downloadMedia: options.downloadMedia,
|
||||
};
|
||||
|
||||
if (adapter.restoreCookies) {
|
||||
const restored = await adapter.restoreCookies(context, runtime.chrome.profileDir).catch(() => false);
|
||||
if (restored) logger.info(`Restored ${adapter.name} session cookies from sidecar.`);
|
||||
}
|
||||
|
||||
if (options.waitMode === "interaction" && adapter.checkLogin) {
|
||||
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
|
||||
const preLogin = await adapter.checkLogin(context);
|
||||
if (preLogin.state !== "logged_in") {
|
||||
didLogin = true;
|
||||
await waitForInteraction(adapter, context, {
|
||||
type: "wait_for_interaction",
|
||||
kind: "login",
|
||||
provider: preLogin.provider ?? adapter.name,
|
||||
prompt: `Please sign in to ${adapter.name === "x" ? "X" : adapter.name} in the opened Chrome window. Extraction will continue automatically once login is detected.`,
|
||||
reason: preLogin.reason ?? `Not logged in to ${adapter.name}`,
|
||||
requiresVisibleBrowser: true,
|
||||
}, options);
|
||||
}
|
||||
}
|
||||
|
||||
if (options.waitMode === "force") {
|
||||
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
|
||||
await waitForForceResume(adapter, context, options);
|
||||
@@ -414,6 +467,9 @@ export async function runConvertCommand(options: ConvertCommandOptions): Promise
|
||||
};
|
||||
|
||||
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
|
||||
if (result.interaction.kind === "login") {
|
||||
didLogin = true;
|
||||
}
|
||||
await waitForInteraction(adapter, context, result.interaction, options);
|
||||
result = await adapter.process(context);
|
||||
|
||||
@@ -516,6 +572,9 @@ export async function runConvertCommand(options: ConvertCommandOptions): Promise
|
||||
|
||||
printOutput(markdown);
|
||||
} finally {
|
||||
if (adapter?.exportCookies && context) {
|
||||
await adapter.exportCookies(context, runtime.chrome.profileDir).catch(() => {});
|
||||
}
|
||||
await closeRuntime(runtime);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -39,6 +39,10 @@ Break down complex content into eye-catching infographic series for Xiaohongshu
|
||||
# Direct input with options
|
||||
/baoyu-xhs-images --style bold --layout comparison
|
||||
[paste content]
|
||||
|
||||
# Non-interactive (for scheduled tasks / automation)
|
||||
/baoyu-xhs-images posts/ai-future/article.md --yes
|
||||
/baoyu-xhs-images posts/ai-future/article.md --yes --preset knowledge-card
|
||||
```
|
||||
|
||||
## Options
|
||||
@@ -48,6 +52,7 @@ Break down complex content into eye-catching infographic series for Xiaohongshu
|
||||
| `--style <name>` | Visual style (see Style Gallery) |
|
||||
| `--layout <name>` | Information layout (see Layout Gallery) |
|
||||
| `--preset <name>` | Style + layout shorthand (see [Style Presets](references/style-presets.md)) |
|
||||
| `--yes` | Non-interactive mode: skip all confirmations. Uses EXTEND.md preferences if found, otherwise uses defaults (no watermark, auto style/layout). Auto-confirms recommended plan (Path A). Suitable for scheduled tasks and automation. |
|
||||
|
||||
## Two Dimensions
|
||||
|
||||
@@ -237,11 +242,11 @@ Copy and track progress:
|
||||
|
||||
```
|
||||
XHS Infographic Progress:
|
||||
- [ ] Step 0: Check preferences (EXTEND.md) ⛔ BLOCKING
|
||||
- [ ] Step 0: Check preferences (EXTEND.md) ⛔ BLOCKING (--yes: use defaults if not found)
|
||||
- [ ] Found → load preferences → continue
|
||||
- [ ] Not found → run first-time setup → MUST complete before Step 1
|
||||
- [ ] Not found → run first-time setup → MUST complete before Step 1 (--yes: skip setup, use defaults)
|
||||
- [ ] Step 1: Analyze content → analysis.md
|
||||
- [ ] Step 2: Smart Confirm ⚠️ REQUIRED
|
||||
- [ ] Step 2: Smart Confirm ⚠️ REQUIRED (--yes: auto-confirm Path A)
|
||||
- [ ] Path A: Quick confirm → generate recommended outline
|
||||
- [ ] Path B: Customize → adjust then generate outline
|
||||
- [ ] Path C: Detailed → 3 outlines → second confirm → generate outline
|
||||
@@ -252,26 +257,30 @@ XHS Infographic Progress:
|
||||
### Flow
|
||||
|
||||
```
|
||||
Input → [Step 0: Preferences] ─┬─ Found → Continue
|
||||
│
|
||||
└─ Not found → First-Time Setup ⛔ BLOCKING
|
||||
│
|
||||
└─ Complete setup → Save EXTEND.md → Continue
|
||||
│
|
||||
┌───────────────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
Analyze → [Smart Confirm] ─┬─ Quick: confirm recommended → outline.md → Generate → Complete
|
||||
│
|
||||
├─ Customize: adjust options → outline.md → Generate → Complete
|
||||
│
|
||||
└─ Detailed: 3 outlines → [Confirm 2] → outline.md → Generate → Complete
|
||||
Input → [--yes?] ─┬─ Yes → [Step 0: Load or defaults] → Analyze → Auto-confirm → Generate → Complete
|
||||
│
|
||||
└─ No → [Step 0: Preferences] ─┬─ Found → Continue
|
||||
│
|
||||
└─ Not found → First-Time Setup ⛔ BLOCKING
|
||||
│
|
||||
└─ Complete setup → Save EXTEND.md → Continue
|
||||
│
|
||||
┌─────────────────────────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
Analyze → [Smart Confirm] ─┬─ Quick: confirm recommended → outline.md → Generate → Complete
|
||||
│
|
||||
├─ Customize: adjust options → outline.md → Generate → Complete
|
||||
│
|
||||
└─ Detailed: 3 outlines → [Confirm 2] → outline.md → Generate → Complete
|
||||
```
|
||||
|
||||
### Step 0: Load Preferences (EXTEND.md) ⛔ BLOCKING
|
||||
|
||||
**Purpose**: Load user preferences or run first-time setup.
|
||||
|
||||
**CRITICAL**: If EXTEND.md not found, MUST complete first-time setup before ANY other questions or steps. Do NOT proceed to content analysis, do NOT ask about style, do NOT ask about layout — ONLY complete the preferences setup first.
|
||||
**`--yes` mode**: If EXTEND.md found → load it. If not found → use built-in defaults (no watermark, style/layout auto-select, language from content). Do NOT run first-time setup, do NOT create EXTEND.md, do NOT ask any questions. Proceed directly to Step 1.
|
||||
|
||||
**CRITICAL** (interactive mode only): If EXTEND.md not found, MUST complete first-time setup before ANY other questions or steps. Do NOT proceed to content analysis, do NOT ask about style, do NOT ask about layout — ONLY complete the preferences setup first.
|
||||
|
||||
Check EXTEND.md existence (priority order):
|
||||
|
||||
@@ -340,7 +349,11 @@ Read source content, save it if needed, and perform deep analysis.
|
||||
|
||||
### Step 2: Smart Confirm ⚠️
|
||||
|
||||
**Purpose**: Present auto-recommended plan, let user confirm or adjust. **Do NOT skip.**
|
||||
**Purpose**: Present auto-recommended plan, let user confirm or adjust.
|
||||
|
||||
**`--yes` mode**: Skip this entire step. Use auto-recommended strategy + style + layout from Step 1 analysis (or `--style`/`--layout`/`--preset` if provided). Generate outline directly using Path A logic → save to `outline.md` → proceed to Step 3. No AskUserQuestion calls.
|
||||
|
||||
**Interactive mode**: Do NOT skip.
|
||||
|
||||
**Auto-Recommendation Logic**:
|
||||
1. Use Auto Selection table to match content signals → best strategy + style + layout
|
||||
@@ -491,7 +504,7 @@ Reference: `references/config/watermark-guide.md`
|
||||
|
||||
**Image Generation Skill Selection**:
|
||||
- Check available image generation skills
|
||||
- If multiple skills available, ask user preference
|
||||
- If multiple skills available: ask user preference (interactive) or use first available skill (`--yes` mode)
|
||||
|
||||
**Session Management**:
|
||||
If image generation skill supports `--sessionId`:
|
||||
|
||||
@@ -2,7 +2,7 @@ import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { findTranscript, parseTranscriptJson3, parseWebVtt } from "./transcript.ts";
|
||||
import { buildTranscriptListFromYtDlp, resolveVideoSource, selectYtDlpTrack } from "./youtube.ts";
|
||||
import { buildTranscriptListFromYtDlp, fetchTranscriptWithFallback, resolveVideoSource, selectYtDlpTrack } from "./youtube.ts";
|
||||
|
||||
test("selectYtDlpTrack prefers json3 over xml and vtt", () => {
|
||||
const track = selectYtDlpTrack([
|
||||
@@ -123,3 +123,61 @@ test("resolveVideoSource falls back to yt-dlp only after fallback-eligible error
|
||||
assert.equal(fallbackCalled, true);
|
||||
assert.equal(source.transcripts[0].languageCode, "en");
|
||||
});
|
||||
|
||||
test("fetchTranscriptWithFallback retries with yt-dlp when InnerTube transcript payload is empty", async () => {
|
||||
const warnings: string[] = [];
|
||||
let fallbackCalled = false;
|
||||
const result = await fetchTranscriptWithFallback(
|
||||
"video12345ab",
|
||||
{
|
||||
kind: "innertube",
|
||||
data: { videoDetails: { title: "Primary" } },
|
||||
transcripts: [{
|
||||
language: "English",
|
||||
languageCode: "en",
|
||||
isGenerated: false,
|
||||
isTranslatable: false,
|
||||
baseUrl: "https://www.youtube.com/api/timedtext?v=video12345ab&lang=en&fmt=json3",
|
||||
translationLanguages: [],
|
||||
}],
|
||||
},
|
||||
{
|
||||
languages: ["en"],
|
||||
translate: "",
|
||||
excludeGenerated: false,
|
||||
excludeManual: false,
|
||||
},
|
||||
async (info) => {
|
||||
if (info.baseUrl.includes("youtube.com/api/timedtext")) {
|
||||
return { snippets: [], language: info.language, languageCode: info.languageCode };
|
||||
}
|
||||
return {
|
||||
snippets: [{ text: "Recovered subtitle", start: 0, duration: 2 }],
|
||||
language: info.language,
|
||||
languageCode: info.languageCode,
|
||||
};
|
||||
},
|
||||
async () => {
|
||||
fallbackCalled = true;
|
||||
return {
|
||||
kind: "yt-dlp",
|
||||
info: { title: "Fallback" },
|
||||
transcripts: [{
|
||||
language: "English",
|
||||
languageCode: "en",
|
||||
isGenerated: false,
|
||||
isTranslatable: false,
|
||||
baseUrl: "https://example.com/subtitles.en.json3",
|
||||
translationLanguages: [],
|
||||
}],
|
||||
};
|
||||
},
|
||||
(message) => warnings.push(message)
|
||||
);
|
||||
|
||||
assert.equal(fallbackCalled, true);
|
||||
assert.equal(result.source.kind, "yt-dlp");
|
||||
assert.equal(result.snippets.length, 1);
|
||||
assert.equal(result.snippets[0].text, "Recovered subtitle");
|
||||
assert.match(warnings[0] || "", /Retrying with yt-dlp fallback/);
|
||||
});
|
||||
|
||||
@@ -13,13 +13,13 @@ import {
|
||||
registerVideoDir,
|
||||
resolveBaseDir,
|
||||
} from "./storage.ts";
|
||||
import { findTranscript, formatListOutput, formatMarkdown, formatSrt, segmentIntoSentences } from "./transcript.ts";
|
||||
import { formatListOutput, formatMarkdown, formatSrt, segmentIntoSentences } from "./transcript.ts";
|
||||
import type { Options, Sentence, Snippet, VideoMeta, VideoResult } from "./types.ts";
|
||||
import {
|
||||
buildVideoMeta,
|
||||
buildVideoMetaFromYtDlp,
|
||||
downloadCoverImage,
|
||||
fetchTranscriptSnippets,
|
||||
fetchTranscriptWithFallback,
|
||||
fetchVideoSource,
|
||||
getThumbnailUrls,
|
||||
getYtDlpThumbnailUrls,
|
||||
@@ -31,10 +31,12 @@ async function fetchAndCache(
|
||||
baseDir: string,
|
||||
opts: Options
|
||||
): Promise<{ meta: VideoMeta; snippets: Snippet[]; sentences: Sentence[]; videoDir: string }> {
|
||||
const source = await fetchVideoSource(videoId);
|
||||
const requestedLanguages = source.kind === "yt-dlp" && opts.translate ? [opts.translate] : opts.languages;
|
||||
const transcript = findTranscript(source.transcripts, requestedLanguages, opts.excludeGenerated, opts.excludeManual);
|
||||
const result = await fetchTranscriptSnippets(transcript, source.kind === "yt-dlp" ? undefined : opts.translate || undefined);
|
||||
const initialSource = await fetchVideoSource(videoId);
|
||||
const { source, transcript, snippets, language, languageCode } = await fetchTranscriptWithFallback(
|
||||
videoId,
|
||||
initialSource,
|
||||
opts
|
||||
);
|
||||
const description = source.kind === "yt-dlp"
|
||||
? source.info.description || ""
|
||||
: source.data?.videoDetails?.shortDescription || "";
|
||||
@@ -42,21 +44,21 @@ async function fetchAndCache(
|
||||
? Number(source.info.duration || 0)
|
||||
: parseInt(source.data?.videoDetails?.lengthSeconds || "0");
|
||||
const chapters = parseChapters(description, duration);
|
||||
const language = {
|
||||
code: result.languageCode,
|
||||
name: result.language,
|
||||
const languageMeta = {
|
||||
code: languageCode,
|
||||
name: language,
|
||||
isGenerated: transcript.isGenerated,
|
||||
};
|
||||
const meta = source.kind === "yt-dlp"
|
||||
? buildVideoMetaFromYtDlp(source.info, videoId, language, chapters)
|
||||
: buildVideoMeta(source.data, videoId, language, chapters);
|
||||
? buildVideoMetaFromYtDlp(source.info, videoId, languageMeta, chapters)
|
||||
: buildVideoMeta(source.data, videoId, languageMeta, chapters);
|
||||
|
||||
const videoDir = registerVideoDir(videoId, slugify(meta.channel), slugify(meta.title), baseDir);
|
||||
ensureDir(join(videoDir, "meta.json"));
|
||||
|
||||
writeFileSync(join(videoDir, "transcript-raw.json"), JSON.stringify(result.snippets, null, 2));
|
||||
writeFileSync(join(videoDir, "transcript-raw.json"), JSON.stringify(snippets, null, 2));
|
||||
|
||||
const sentences = segmentIntoSentences(result.snippets);
|
||||
const sentences = segmentIntoSentences(snippets);
|
||||
writeFileSync(join(videoDir, "transcript-sentences.json"), JSON.stringify(sentences, null, 2));
|
||||
|
||||
const imagePath = join(videoDir, "imgs", "cover.jpg");
|
||||
@@ -69,7 +71,7 @@ async function fetchAndCache(
|
||||
|
||||
writeFileSync(join(videoDir, "meta.json"), JSON.stringify(meta, null, 2));
|
||||
|
||||
return { meta, snippets: result.snippets, sentences, videoDir };
|
||||
return { meta, snippets, sentences, videoDir };
|
||||
}
|
||||
|
||||
async function processVideo(videoId: string, opts: Options): Promise<VideoResult> {
|
||||
|
||||
@@ -2,12 +2,13 @@ import { spawnSync } from "child_process";
|
||||
import { writeFileSync } from "fs";
|
||||
|
||||
import { makeError, normalizeError, normalizePublishDate, shouldTryAlternateClient, shouldTryYtDlpFallback } from "./shared.ts";
|
||||
import { parseTranscriptPayload } from "./transcript.ts";
|
||||
import { findTranscript, parseTranscriptPayload } from "./transcript.ts";
|
||||
import type {
|
||||
Chapter,
|
||||
InnerTubeClient,
|
||||
InnerTubeSession,
|
||||
LanguageMeta,
|
||||
Options,
|
||||
Snippet,
|
||||
TranscriptInfo,
|
||||
VideoMeta,
|
||||
@@ -219,6 +220,68 @@ export async function fetchTranscriptSnippets(
|
||||
};
|
||||
}
|
||||
|
||||
function buildYtDlpVideoSource(videoId: string, info: YtDlpInfo): VideoSource {
|
||||
const transcripts = buildTranscriptListFromYtDlp(info);
|
||||
if (!transcripts.length) throw makeError(`Transcripts disabled for ${videoId}`, "TRANSCRIPTS_DISABLED");
|
||||
return { kind: "yt-dlp", info, transcripts };
|
||||
}
|
||||
|
||||
function getRequestedLanguages(
|
||||
source: VideoSource,
|
||||
opts: Pick<Options, "languages" | "translate">
|
||||
): string[] {
|
||||
return source.kind === "yt-dlp" && opts.translate ? [opts.translate] : opts.languages;
|
||||
}
|
||||
|
||||
export async function fetchTranscriptWithFallback(
|
||||
videoId: string,
|
||||
source: VideoSource,
|
||||
opts: Pick<Options, "languages" | "translate" | "excludeGenerated" | "excludeManual">,
|
||||
fetchSnippets: (
|
||||
info: TranscriptInfo,
|
||||
translateTo?: string
|
||||
) => Promise<{ snippets: Snippet[]; language: string; languageCode: string }> = fetchTranscriptSnippets,
|
||||
fetchFallbackSource: (videoId: string) => Promise<VideoSource> | VideoSource = (requestedVideoId) =>
|
||||
buildYtDlpVideoSource(requestedVideoId, fetchYtDlpInfo(requestedVideoId)),
|
||||
logWarning: (message: string) => void = (message) => console.error(message)
|
||||
): Promise<{
|
||||
source: VideoSource;
|
||||
transcript: TranscriptInfo;
|
||||
snippets: Snippet[];
|
||||
language: string;
|
||||
languageCode: string;
|
||||
}> {
|
||||
const transcript = findTranscript(
|
||||
source.transcripts,
|
||||
getRequestedLanguages(source, opts),
|
||||
opts.excludeGenerated,
|
||||
opts.excludeManual
|
||||
);
|
||||
const result = await fetchSnippets(transcript, source.kind === "yt-dlp" ? undefined : opts.translate || undefined);
|
||||
if (result.snippets.length > 0) return { source, transcript, ...result };
|
||||
|
||||
if (source.kind === "yt-dlp") {
|
||||
throw makeError(`Transcript fetch returned empty snippets for ${videoId}`, "EMPTY_TRANSCRIPT");
|
||||
}
|
||||
|
||||
logWarning(`Warning (${videoId}): Transcript fetch returned empty snippets. Retrying with yt-dlp fallback.`);
|
||||
const fallbackSource = await fetchFallbackSource(videoId);
|
||||
const fallbackTranscript = findTranscript(
|
||||
fallbackSource.transcripts,
|
||||
getRequestedLanguages(fallbackSource, opts),
|
||||
opts.excludeGenerated,
|
||||
opts.excludeManual
|
||||
);
|
||||
const fallbackResult = await fetchSnippets(
|
||||
fallbackTranscript,
|
||||
fallbackSource.kind === "yt-dlp" ? undefined : opts.translate || undefined
|
||||
);
|
||||
if (!fallbackResult.snippets.length) {
|
||||
throw makeError(`Transcript fetch returned empty snippets for ${videoId} after yt-dlp fallback`, "EMPTY_TRANSCRIPT");
|
||||
}
|
||||
return { source: fallbackSource, transcript: fallbackTranscript, ...fallbackResult };
|
||||
}
|
||||
|
||||
export function detectYtDlpCommand(): { command: string; args: string[]; label: string } | null {
|
||||
if (cachedYtDlpCommand !== undefined) return cachedYtDlpCommand;
|
||||
const candidates = [
|
||||
@@ -366,10 +429,7 @@ export async function resolveVideoSource(
|
||||
const normalized = normalizeError(error);
|
||||
if (!shouldTryYtDlpFallback(normalized)) throw normalized;
|
||||
logWarning(`Warning (${videoId}): ${normalized.message}. Retrying with yt-dlp fallback.`);
|
||||
const info = fetchFallback(videoId);
|
||||
const transcripts = buildTranscriptListFromYtDlp(info);
|
||||
if (!transcripts.length) throw makeError(`Transcripts disabled for ${videoId}`, "TRANSCRIPTS_DISABLED");
|
||||
return { kind: "yt-dlp", info, transcripts };
|
||||
return buildYtDlpVideoSource(videoId, fetchFallback(videoId));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user