Compare commits

...

9 Commits

Author SHA1 Message Date
Jim Liu 宝玉 eb92bdb9df chore: release v1.15.0 2026-01-22 11:47:56 -06:00
Jim Liu 宝玉 e07d33fed0 chore: release v1.14.0 2026-01-22 01:24:52 -06:00
Jim Liu 宝玉 e964feb5e5 Merge pull request #14 from fkysly/fix/video-ready-detection
fix(x-video): improve video ready detection
2026-01-21 21:42:55 -06:00
fkysly 7669556736 fix(x-video): improve video ready detection
- Type text while video uploads in background instead of waiting
- Fix video ready detection by checking tweet button state instead of
  progressbar presence (X always has progressbar elements for video
  playback, causing false negatives)
2026-01-22 09:56:02 +08:00
Jim Liu 宝玉 97da7ab4eb chore: release v1.13.0 2026-01-21 19:40:46 -06:00
Jim Liu 宝玉 235868343c chore: release v1.12.0 2026-01-21 11:28:41 -06:00
Jim Liu 宝玉 f43dc2be56 Merge pull request #13 from threehotpot-bot/feature/quote-tweet
feat: add quote tweet support
2026-01-21 10:39:22 -06:00
threenrm d793c19a72 fix: use CDP Input.insertText for reliable text insertion
The previous document.execCommand('insertText') caused text to be
duplicated multiple times. Switch to CDP's Input.insertText for
more reliable text input.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 18:21:36 +08:00
threenrm 8cc8d25ad1 feat: add quote tweet support
Add x-quote.ts script to quote existing tweets with comments.
This allows users to share content while giving credit to the original creator.

- New script: scripts/x-quote.ts for quote tweet functionality
- Uses CDP to navigate through X's native quote UI
- Supports preview mode and --submit flag
- Updated SKILL.md with documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 18:14:54 +08:00
28 changed files with 2239 additions and 571 deletions
+3 -2
View File
@@ -6,7 +6,7 @@
},
"metadata": {
"description": "Skills shared by Baoyu for improving daily work efficiency",
"version": "1.11.0"
"version": "1.15.0"
},
"plugins": [
{
@@ -42,7 +42,8 @@
"strict": false,
"skills": [
"./skills/baoyu-danger-x-to-markdown",
"./skills/baoyu-compress-image"
"./skills/baoyu-compress-image",
"./skills/baoyu-url-to-markdown"
]
}
]
+4
View File
@@ -6,6 +6,7 @@ yarn-debug.log*
yarn-error.log*
lerna-debug.log*
# Diagnostic reports (https://nodejs.org/api/report.html)
report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
@@ -145,3 +146,6 @@ tests-data/
.baoyu-skills/
x-to-markdown/
xhs-images/
url-to-markdown/
cover-image/
slide-deck/
+29
View File
@@ -2,6 +2,35 @@
English | [中文](./CHANGELOG.zh.md)
## 1.15.0 - 2026-01-22
### Features
- `baoyu-xhs-images`: adds user preferences support via EXTEND.md—configure watermark (content, position, opacity), preferred style, preferred layout, and custom styles. New Step 0 checks for preferences at project (`.baoyu-skills/`) or user (`~/.baoyu-skills/`) level with first-time setup flow.
### Documentation
- `baoyu-xhs-images`: adds three reference documents—`preferences-schema.md` (YAML schema), `watermark-guide.md` (position and opacity guide), `first-time-setup.md` (setup flow).
## 1.14.0 - 2026-01-22
### Fixes
- `baoyu-post-to-x`: improves video ready detection for more reliable video posting.
### Documentation
- `baoyu-slide-deck`: comprehensive SKILL.md enhancement—adds slide count guidance (recommended 8-25, max 30), audience guidelines table with audience-specific principles, style selection principles with content-type recommendations, layout selection tips with common mistakes to avoid, visual hierarchy principles, content density guidelines (McKinsey-style high-density principles), color selection guide, typography principles with font recommendations (English and Chinese fonts with multilingual pairing), and visual elements reference (backgrounds, typography treatments, geometric accents).
## 1.13.0 - 2026-01-21
### Features
- `baoyu-url-to-markdown`: new utility skill for fetching any URL via Chrome CDP and converting to clean markdown. Supports two capture modes—auto (immediate capture on page load) and wait (user-controlled capture for login-required pages).
### Improvements
- `baoyu-xhs-images`: updates style recommendations—replaces `tech` references with `notion` and `chalkboard` for technical and educational content.
## 1.12.0 - 2026-01-21
### Refactor
- `baoyu-post-to-x`: extracts shared utilities to `x-utils.ts`—consolidates Chrome detection, CDP connection, clipboard operations, and helper functions from `x-article.ts`, `x-browser.ts`, `x-quote.ts`, and `x-video.ts` into a single reusable module.
## 1.11.0 - 2026-01-21
### Features
+29
View File
@@ -2,6 +2,35 @@
[English](./CHANGELOG.md) | 中文
## 1.15.0 - 2026-01-22
### 新功能
- `baoyu-xhs-images`:新增用户偏好设置支持(通过 EXTEND.md 配置)——可设置水印(内容、位置、透明度)、首选风格、首选布局和自定义风格。新增 Step 0 检查项目级(`.baoyu-skills/`)或用户级(`~/.baoyu-skills/`)偏好设置,首次使用时引导设置。
### 文档
- `baoyu-xhs-images`:新增三个参考文档——`preferences-schema.md`YAML 配置模式)、`watermark-guide.md`(水印位置和透明度指南)、`first-time-setup.md`(首次设置流程)。
## 1.14.0 - 2026-01-22
### 修复
- `baoyu-post-to-x`:改进视频就绪检测,提升视频发布稳定性。
### 文档
- `baoyu-slide-deck`:SKILL.md 全面增强——新增幻灯片数量指南(推荐 8-25 张,最多 30 张)、受众指南表格及各受众特定原则、风格选择原则与内容类型推荐、布局选择技巧与常见错误提示、视觉层次原则、内容密度指南(麦肯锡风格高密度原则)、配色选择指南、字体排版原则与字体推荐(中英文字体及多语言搭配方案)、视觉元素参考(背景处理、字体处理、几何装饰)。
## 1.13.0 - 2026-01-21
### 新功能
- `baoyu-url-to-markdown`:新增 URL 转 Markdown 工具技能,通过 Chrome CDP 抓取任意网页并转换为干净的 Markdown 格式。支持两种抓取模式——自动模式(页面加载后立即抓取)和等待模式(用户控制抓取时机,适用于需要登录的页面)。
### 改进
- `baoyu-xhs-images`:更新风格推荐——将 `tech` 风格引用替换为 `notion``chalkboard`,用于技术和教育内容。
## 1.12.0 - 2026-01-21
### 重构
- `baoyu-post-to-x`:提取公共工具函数到 `x-utils.ts`——将 `x-article.ts``x-browser.ts``x-quote.ts``x-video.ts` 中重复的 Chrome 检测、CDP 连接、剪贴板操作等功能整合为统一的可复用模块。
## 1.11.0 - 2026-01-21
### 新功能
+30 -1
View File
@@ -55,7 +55,7 @@ Simply tell Claude Code:
|--------|-------------|--------|
| **content-skills** | Content generation and publishing | [xhs-images](#baoyu-xhs-images), [infographic](#baoyu-infographic), [cover-image](#baoyu-cover-image), [slide-deck](#baoyu-slide-deck), [comic](#baoyu-comic), [article-illustrator](#baoyu-article-illustrator), [post-to-x](#baoyu-post-to-x), [post-to-wechat](#baoyu-post-to-wechat) |
| **ai-generation-skills** | AI-powered generation backends | [image-gen](#baoyu-image-gen), [danger-gemini-web](#baoyu-danger-gemini-web) |
| **utility-skills** | Utility tools for content processing | [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image) |
| **utility-skills** | Utility tools for content processing | [url-to-markdown](#baoyu-url-to-markdown), [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image) |
## Update Skills
@@ -586,6 +586,35 @@ Interacts with Gemini Web to generate text and images.
Utility tools for content processing.
#### baoyu-url-to-markdown
Fetch any URL via Chrome CDP and convert to clean markdown. Supports two capture modes for different scenarios.
```bash
# Auto mode (default) - capture when page loads
/baoyu-url-to-markdown https://example.com/article
# Wait mode - for login-required pages
/baoyu-url-to-markdown https://example.com/private --wait
# Save to specific file
/baoyu-url-to-markdown https://example.com/article -o output.md
```
**Capture Modes**:
| Mode | Description | Best For |
|------|-------------|----------|
| Auto (default) | Captures immediately after page load | Public pages, static content |
| Wait (`--wait`) | Waits for user signal before capture | Login-required, dynamic content |
**Options**:
| Option | Description |
|--------|-------------|
| `<url>` | URL to fetch |
| `-o <path>` | Output file path |
| `--wait` | Wait for user signal before capturing |
| `--timeout <ms>` | Page load timeout (default: 30000) |
#### baoyu-danger-x-to-markdown
Converts X (Twitter) content to markdown format. Supports tweet threads and X Articles.
+30 -1
View File
@@ -55,7 +55,7 @@ npx skills add jimliu/baoyu-skills
|------|------|----------|
| **content-skills** | 内容生成和发布 | [xhs-images](#baoyu-xhs-images), [infographic](#baoyu-infographic), [cover-image](#baoyu-cover-image), [slide-deck](#baoyu-slide-deck), [comic](#baoyu-comic), [article-illustrator](#baoyu-article-illustrator), [post-to-x](#baoyu-post-to-x), [post-to-wechat](#baoyu-post-to-wechat) |
| **ai-generation-skills** | AI 生成后端 | [image-gen](#baoyu-image-gen), [danger-gemini-web](#baoyu-danger-gemini-web) |
| **utility-skills** | 内容处理工具 | [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image) |
| **utility-skills** | 内容处理工具 | [url-to-markdown](#baoyu-url-to-markdown), [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image) |
## 更新技能
@@ -586,6 +586,35 @@ AI 驱动的生成后端。
内容处理工具。
#### baoyu-url-to-markdown
通过 Chrome CDP 抓取任意 URL 并转换为干净的 Markdown。支持两种抓取模式,适应不同场景。
```bash
# 自动模式(默认)- 页面加载后立即抓取
/baoyu-url-to-markdown https://example.com/article
# 等待模式 - 适用于需要登录的页面
/baoyu-url-to-markdown https://example.com/private --wait
# 保存到指定文件
/baoyu-url-to-markdown https://example.com/article -o output.md
```
**抓取模式**
| 模式 | 说明 | 适用场景 |
|------|------|----------|
| 自动(默认) | 页面加载后立即抓取 | 公开页面、静态内容 |
| 等待(`--wait`) | 等待用户信号后抓取 | 需登录页面、动态内容 |
**选项**
| 选项 | 说明 |
|------|------|
| `<url>` | 要抓取的 URL |
| `-o <path>` | 输出文件路径 |
| `--wait` | 等待用户信号后抓取 |
| `--timeout <ms>` | 页面加载超时(默认:30000) |
#### baoyu-danger-x-to-markdown
将 X (Twitter) 内容转换为 markdown 格式。支持推文串和 X 文章。
+23
View File
@@ -21,6 +21,7 @@ Post content, images, videos, and long-form articles to X using real Chrome brow
|--------|---------|
| `scripts/x-browser.ts` | Regular posts (text + images) |
| `scripts/x-video.ts` | Video posts (text + video) |
| `scripts/x-quote.ts` | Quote tweet with comment |
| `scripts/x-article.ts` | Long-form article publishing (Markdown) |
| `scripts/md-to-html.ts` | Markdown → HTML conversion |
| `scripts/copy-to-clipboard.ts` | Copy content to clipboard |
@@ -91,6 +92,28 @@ npx -y bun ${SKILL_DIR}/scripts/x-video.ts "Amazing content" --video ./demo.mp4
---
## Quote Tweets
Quote an existing tweet with your comment - a way to share content while giving credit to the original creator.
```bash
# Preview mode (doesn't post)
npx -y bun ${SKILL_DIR}/scripts/x-quote.ts https://x.com/user/status/123456789 "Great insight!"
# Actually post
npx -y bun ${SKILL_DIR}/scripts/x-quote.ts https://x.com/user/status/123456789 "I agree!" --submit
```
**Parameters**:
| Parameter | Description |
|-----------|-------------|
| `<tweet-url>` | URL of the tweet to quote (positional argument) |
| `<comment>` | Your comment text (positional argument, optional) |
| `--submit` | Actually post (default: preview only) |
| `--profile <dir>` | Custom Chrome profile directory |
---
## X Articles
Long-form Markdown articles (requires X Premium).
+15 -161
View File
@@ -1,12 +1,23 @@
import { spawn, spawnSync } from 'node:child_process';
import { spawn } from 'node:child_process';
import fs from 'node:fs';
import { mkdir, writeFile } from 'node:fs/promises';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import process from 'node:process';
import { parseMarkdown } from './md-to-html.js';
import {
CHROME_CANDIDATES_BASIC,
CdpConnection,
copyHtmlToClipboard,
copyImageToClipboard,
findChromeExecutable,
getDefaultProfileDir,
getFreePort,
pasteFromClipboard,
sleep,
waitForChromeDebugPort,
} from './x-utils.js';
const X_ARTICLES_URL = 'https://x.com/compose/articles';
@@ -41,163 +52,6 @@ const I18N_SELECTORS = {
],
};
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function getFreePort(): Promise<number> {
return new Promise((resolve, reject) => {
const server = net.createServer();
server.unref();
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
if (!address || typeof address === 'string') {
server.close(() => reject(new Error('Unable to allocate port')));
return;
}
server.close((err) => (err ? reject(err) : resolve(address.port)));
});
});
}
function findChromeExecutable(): string | undefined {
const override = process.env.X_BROWSER_CHROME_PATH?.trim();
if (override && fs.existsSync(override)) return override;
const candidates: string[] = [];
switch (process.platform) {
case 'darwin':
candidates.push(
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
'/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary',
'/Applications/Chromium.app/Contents/MacOS/Chromium',
);
break;
case 'win32':
candidates.push(
'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
);
break;
default:
candidates.push('/usr/bin/google-chrome', '/usr/bin/chromium', '/usr/bin/chromium-browser');
break;
}
for (const p of candidates) if (fs.existsSync(p)) return p;
return undefined;
}
function getDefaultProfileDir(): string {
const base = process.env.XDG_DATA_HOME || path.join(os.homedir(), '.local', 'share');
return path.join(base, 'x-browser-profile');
}
async function fetchJson<T>(url: string): Promise<T> {
const res = await fetch(url, { redirect: 'follow' });
if (!res.ok) throw new Error(`Request failed: ${res.status}`);
return res.json() as Promise<T>;
}
async function waitForChromeDebugPort(port: number, timeoutMs: number): Promise<string> {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(`http://127.0.0.1:${port}/json/version`);
if (version.webSocketDebuggerUrl) return version.webSocketDebuggerUrl;
} catch {}
await sleep(200);
}
throw new Error('Chrome debug port not ready');
}
class CdpConnection {
private ws: WebSocket;
private nextId = 0;
private pending = new Map<number, { resolve: (v: unknown) => void; reject: (e: Error) => void; timer: ReturnType<typeof setTimeout> | null }>();
private constructor(ws: WebSocket) {
this.ws = ws;
this.ws.addEventListener('message', (event) => {
try {
const data = typeof event.data === 'string' ? event.data : new TextDecoder().decode(event.data as ArrayBuffer);
const msg = JSON.parse(data) as { id?: number; result?: unknown; error?: { message?: string } };
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer) clearTimeout(pending.timer);
if (msg.error?.message) pending.reject(new Error(msg.error.message));
else pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener('close', () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer) clearTimeout(pending.timer);
pending.reject(new Error('CDP connection closed'));
}
});
}
static async connect(url: string, timeoutMs: number): Promise<CdpConnection> {
const ws = new WebSocket(url);
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => reject(new Error('CDP connection timeout')), timeoutMs);
ws.addEventListener('open', () => { clearTimeout(timer); resolve(); });
ws.addEventListener('error', () => { clearTimeout(timer); reject(new Error('CDP connection failed')); });
});
return new CdpConnection(ws);
}
async send<T = unknown>(method: string, params?: Record<string, unknown>, options?: { sessionId?: string; timeoutMs?: number }): Promise<T> {
const id = ++this.nextId;
const message: Record<string, unknown> = { id, method };
if (params) message.params = params;
if (options?.sessionId) message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? 30_000;
return new Promise<T>((resolve, reject) => {
const timer = timeoutMs > 0 ? setTimeout(() => { this.pending.delete(id); reject(new Error(`CDP timeout: ${method}`)); }, timeoutMs) : null;
this.pending.set(id, { resolve: resolve as (v: unknown) => void, reject, timer });
this.ws.send(JSON.stringify(message));
});
}
close(): void {
try { this.ws.close(); } catch {}
}
}
function getScriptDir(): string {
return path.dirname(new URL(import.meta.url).pathname);
}
function copyImageToClipboard(imagePath: string): boolean {
const copyScript = path.join(getScriptDir(), 'copy-to-clipboard.ts');
const result = spawnSync('npx', ['-y', 'bun', copyScript, 'image', imagePath], { stdio: 'inherit' });
return result.status === 0;
}
function copyHtmlToClipboard(htmlPath: string): boolean {
const copyScript = path.join(getScriptDir(), 'copy-to-clipboard.ts');
const result = spawnSync('npx', ['-y', 'bun', copyScript, 'html', '--file', htmlPath], { stdio: 'inherit' });
return result.status === 0;
}
function pasteFromClipboard(targetApp?: string, retries = 3, delayMs = 500): boolean {
const pasteScript = path.join(getScriptDir(), 'paste-from-clipboard.ts');
const args = ['npx', '-y', 'bun', pasteScript, '--retries', String(retries), '--delay', String(delayMs)];
if (targetApp) {
args.push('--app', targetApp);
}
const result = spawnSync(args[0]!, args.slice(1), { stdio: 'inherit' });
return result.status === 0;
}
interface ArticleOptions {
markdownPath: string;
coverImage?: string;
@@ -225,7 +79,7 @@ export async function publishArticle(options: ArticleOptions): Promise<void> {
await writeFile(htmlPath, parsed.html, 'utf-8');
console.log(`[x-article] HTML saved to: ${htmlPath}`);
const chromePath = options.chromePath ?? findChromeExecutable();
const chromePath = options.chromePath ?? findChromeExecutable(CHROME_CANDIDATES_BASIC);
if (!chromePath) throw new Error('Chrome not found');
await mkdir(profileDir, { recursive: true });
@@ -246,7 +100,7 @@ export async function publishArticle(options: ArticleOptions): Promise<void> {
try {
const wsUrl = await waitForChromeDebugPort(port, 30_000);
cdp = await CdpConnection.connect(wsUrl, 30_000);
cdp = await CdpConnection.connect(wsUrl, 30_000, { defaultTimeoutMs: 30_000 });
// Get page target
const targets = await cdp.send<{ targetInfos: Array<{ targetId: string; url: string; type: string }> }>('Target.getTargets');
+15 -197
View File
@@ -1,203 +1,21 @@
import { spawn, spawnSync } from 'node:child_process';
import { spawn } from 'node:child_process';
import fs from 'node:fs';
import { mkdir } from 'node:fs/promises';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import process from 'node:process';
import {
CHROME_CANDIDATES_FULL,
CdpConnection,
copyImageToClipboard,
findChromeExecutable,
getDefaultProfileDir,
getFreePort,
pasteFromClipboard,
sleep,
waitForChromeDebugPort,
} from './x-utils.js';
const X_COMPOSE_URL = 'https://x.com/compose/post';
function getScriptDir(): string {
return path.dirname(new URL(import.meta.url).pathname);
}
function copyImageToClipboard(imagePath: string): boolean {
const copyScript = path.join(getScriptDir(), 'copy-to-clipboard.ts');
const result = spawnSync('npx', ['-y', 'bun', copyScript, 'image', imagePath], { stdio: 'inherit' });
return result.status === 0;
}
function pasteFromClipboard(targetApp?: string, retries = 3, delayMs = 500): boolean {
const pasteScript = path.join(getScriptDir(), 'paste-from-clipboard.ts');
const args = ['npx', '-y', 'bun', pasteScript, '--retries', String(retries), '--delay', String(delayMs)];
if (targetApp) {
args.push('--app', targetApp);
}
const result = spawnSync(args[0]!, args.slice(1), { stdio: 'inherit' });
return result.status === 0;
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function getFreePort(): Promise<number> {
return new Promise((resolve, reject) => {
const server = net.createServer();
server.unref();
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
if (!address || typeof address === 'string') {
server.close(() => reject(new Error('Unable to allocate a free TCP port.')));
return;
}
const port = address.port;
server.close((err) => {
if (err) reject(err);
else resolve(port);
});
});
});
}
function findChromeExecutable(): string | undefined {
const override = process.env.X_BROWSER_CHROME_PATH?.trim();
if (override && fs.existsSync(override)) return override;
const candidates: string[] = [];
switch (process.platform) {
case 'darwin':
candidates.push(
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
'/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary',
'/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta',
'/Applications/Chromium.app/Contents/MacOS/Chromium',
'/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge',
);
break;
case 'win32':
candidates.push(
'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
'C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe',
'C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe',
);
break;
default:
candidates.push(
'/usr/bin/google-chrome',
'/usr/bin/google-chrome-stable',
'/usr/bin/chromium',
'/usr/bin/chromium-browser',
'/snap/bin/chromium',
'/usr/bin/microsoft-edge',
);
break;
}
for (const p of candidates) {
if (fs.existsSync(p)) return p;
}
return undefined;
}
function getDefaultProfileDir(): string {
const base = process.env.XDG_DATA_HOME || path.join(os.homedir(), '.local', 'share');
return path.join(base, 'x-browser-profile');
}
async function fetchJson<T = unknown>(url: string): Promise<T> {
const res = await fetch(url, { redirect: 'follow' });
if (!res.ok) throw new Error(`Request failed: ${res.status} ${res.statusText}`);
return (await res.json()) as T;
}
async function waitForChromeDebugPort(port: number, timeoutMs: number): Promise<string> {
const start = Date.now();
let lastError: unknown = null;
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(`http://127.0.0.1:${port}/json/version`);
if (version.webSocketDebuggerUrl) return version.webSocketDebuggerUrl;
lastError = new Error('Missing webSocketDebuggerUrl');
} catch (error) {
lastError = error;
}
await sleep(200);
}
throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
}
class CdpConnection {
private ws: WebSocket;
private nextId = 0;
private pending = new Map<number, { resolve: (v: unknown) => void; reject: (e: Error) => void; timer: ReturnType<typeof setTimeout> | null }>();
private eventHandlers = new Map<string, Set<(params: unknown) => void>>();
private constructor(ws: WebSocket) {
this.ws = ws;
this.ws.addEventListener('message', (event) => {
try {
const data = typeof event.data === 'string' ? event.data : new TextDecoder().decode(event.data as ArrayBuffer);
const msg = JSON.parse(data) as { id?: number; method?: string; params?: unknown; result?: unknown; error?: { message?: string } };
if (msg.method) {
const handlers = this.eventHandlers.get(msg.method);
if (handlers) handlers.forEach((h) => h(msg.params));
}
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer) clearTimeout(pending.timer);
if (msg.error?.message) pending.reject(new Error(msg.error.message));
else pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener('close', () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer) clearTimeout(pending.timer);
pending.reject(new Error('CDP connection closed.'));
}
});
}
static async connect(url: string, timeoutMs: number): Promise<CdpConnection> {
const ws = new WebSocket(url);
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => reject(new Error('CDP connection timeout.')), timeoutMs);
ws.addEventListener('open', () => { clearTimeout(timer); resolve(); });
ws.addEventListener('error', () => { clearTimeout(timer); reject(new Error('CDP connection failed.')); });
});
return new CdpConnection(ws);
}
on(method: string, handler: (params: unknown) => void): void {
if (!this.eventHandlers.has(method)) this.eventHandlers.set(method, new Set());
this.eventHandlers.get(method)!.add(handler);
}
async send<T = unknown>(method: string, params?: Record<string, unknown>, options?: { sessionId?: string; timeoutMs?: number }): Promise<T> {
const id = ++this.nextId;
const message: Record<string, unknown> = { id, method };
if (params) message.params = params;
if (options?.sessionId) message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? 15_000;
const result = await new Promise<unknown>((resolve, reject) => {
const timer = timeoutMs > 0 ? setTimeout(() => { this.pending.delete(id); reject(new Error(`CDP timeout: ${method}`)); }, timeoutMs) : null;
this.pending.set(id, { resolve, reject, timer });
this.ws.send(JSON.stringify(message));
});
return result as T;
}
close(): void {
try { this.ws.close(); } catch {}
}
}
interface XBrowserOptions {
text?: string;
images?: string[];
@@ -210,7 +28,7 @@ interface XBrowserOptions {
export async function postToX(options: XBrowserOptions): Promise<void> {
const { text, images = [], submit = false, timeoutMs = 120_000, profileDir = getDefaultProfileDir() } = options;
const chromePath = options.chromePath ?? findChromeExecutable();
const chromePath = options.chromePath ?? findChromeExecutable(CHROME_CANDIDATES_FULL);
if (!chromePath) throw new Error('Chrome not found. Set X_BROWSER_CHROME_PATH env var.');
await mkdir(profileDir, { recursive: true });
@@ -231,8 +49,8 @@ export async function postToX(options: XBrowserOptions): Promise<void> {
let cdp: CdpConnection | null = null;
try {
const wsUrl = await waitForChromeDebugPort(port, 30_000);
cdp = await CdpConnection.connect(wsUrl, 30_000);
const wsUrl = await waitForChromeDebugPort(port, 30_000, { includeLastError: true });
cdp = await CdpConnection.connect(wsUrl, 30_000, { defaultTimeoutMs: 15_000 });
const targets = await cdp.send<{ targetInfos: Array<{ targetId: string; url: string; type: string }> }>('Target.getTargets');
let pageTarget = targets.targetInfos.find((t) => t.type === 'page' && t.url.includes('x.com'));
+247
View File
@@ -0,0 +1,247 @@
import { spawn } from 'node:child_process';
import { mkdir } from 'node:fs/promises';
import process from 'node:process';
import {
CHROME_CANDIDATES_FULL,
CdpConnection,
findChromeExecutable,
getDefaultProfileDir,
getFreePort,
sleep,
waitForChromeDebugPort,
} from './x-utils.js';
function extractTweetUrl(urlOrId: string): string | null {
// If it's already a full URL, normalize it
if (urlOrId.match(/(?:x\.com|twitter\.com)\/\w+\/status\/\d+/)) {
return urlOrId.replace(/twitter\.com/, 'x.com').split('?')[0];
}
return null;
}
interface QuoteOptions {
tweetUrl: string;
comment?: string;
submit?: boolean;
timeoutMs?: number;
profileDir?: string;
chromePath?: string;
}
export async function quotePost(options: QuoteOptions): Promise<void> {
const { tweetUrl, comment, submit = false, timeoutMs = 120_000, profileDir = getDefaultProfileDir() } = options;
const chromePath = options.chromePath ?? findChromeExecutable(CHROME_CANDIDATES_FULL);
if (!chromePath) throw new Error('Chrome not found. Set X_BROWSER_CHROME_PATH env var.');
await mkdir(profileDir, { recursive: true });
const port = await getFreePort();
console.log(`[x-quote] Launching Chrome (profile: ${profileDir})`);
console.log(`[x-quote] Opening tweet: ${tweetUrl}`);
const chrome = spawn(chromePath, [
`--remote-debugging-port=${port}`,
`--user-data-dir=${profileDir}`,
'--no-first-run',
'--no-default-browser-check',
'--disable-blink-features=AutomationControlled',
'--start-maximized',
tweetUrl,
], { stdio: 'ignore' });
let cdp: CdpConnection | null = null;
try {
const wsUrl = await waitForChromeDebugPort(port, 30_000, { includeLastError: true });
cdp = await CdpConnection.connect(wsUrl, 30_000, { defaultTimeoutMs: 15_000 });
const targets = await cdp.send<{ targetInfos: Array<{ targetId: string; url: string; type: string }> }>('Target.getTargets');
let pageTarget = targets.targetInfos.find((t) => t.type === 'page' && t.url.includes('x.com'));
if (!pageTarget) {
const { targetId } = await cdp.send<{ targetId: string }>('Target.createTarget', { url: tweetUrl });
pageTarget = { targetId, url: tweetUrl, type: 'page' };
}
const { sessionId } = await cdp.send<{ sessionId: string }>('Target.attachToTarget', { targetId: pageTarget.targetId, flatten: true });
await cdp.send('Page.enable', {}, { sessionId });
await cdp.send('Runtime.enable', {}, { sessionId });
console.log('[x-quote] Waiting for tweet to load...');
await sleep(3000);
// Wait for retweet button to appear (indicates tweet loaded and user logged in)
const waitForRetweetButton = async (): Promise<boolean> => {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const result = await cdp!.send<{ result: { value: boolean } }>('Runtime.evaluate', {
expression: `!!document.querySelector('[data-testid="retweet"]')`,
returnByValue: true,
}, { sessionId });
if (result.result.value) return true;
await sleep(1000);
}
return false;
};
const retweetFound = await waitForRetweetButton();
if (!retweetFound) {
console.log('[x-quote] Tweet not found or not logged in. Please log in to X in the browser window.');
console.log('[x-quote] Waiting for login...');
const loggedIn = await waitForRetweetButton();
if (!loggedIn) throw new Error('Timed out waiting for tweet. Please log in first or check the tweet URL.');
}
// Click the retweet button
console.log('[x-quote] Clicking retweet button...');
await cdp.send('Runtime.evaluate', {
expression: `document.querySelector('[data-testid="retweet"]')?.click()`,
}, { sessionId });
await sleep(1000);
// Wait for and click the "Quote" option in the menu
console.log('[x-quote] Selecting quote option...');
const waitForQuoteOption = async (): Promise<boolean> => {
const start = Date.now();
while (Date.now() - start < 10_000) {
const result = await cdp!.send<{ result: { value: boolean } }>('Runtime.evaluate', {
expression: `!!document.querySelector('[data-testid="Dropdown"] [role="menuitem"]:nth-child(2)')`,
returnByValue: true,
}, { sessionId });
if (result.result.value) return true;
await sleep(200);
}
return false;
};
const quoteOptionFound = await waitForQuoteOption();
if (!quoteOptionFound) {
throw new Error('Quote option not found. The menu may not have opened.');
}
// Click the quote option (second menu item)
await cdp.send('Runtime.evaluate', {
expression: `document.querySelector('[data-testid="Dropdown"] [role="menuitem"]:nth-child(2)')?.click()`,
}, { sessionId });
await sleep(2000);
// Wait for the quote compose dialog
console.log('[x-quote] Waiting for quote compose dialog...');
const waitForQuoteDialog = async (): Promise<boolean> => {
const start = Date.now();
while (Date.now() - start < 10_000) {
const result = await cdp!.send<{ result: { value: boolean } }>('Runtime.evaluate', {
expression: `!!document.querySelector('[data-testid="tweetTextarea_0"]')`,
returnByValue: true,
}, { sessionId });
if (result.result.value) return true;
await sleep(200);
}
return false;
};
const dialogFound = await waitForQuoteDialog();
if (!dialogFound) {
throw new Error('Quote compose dialog not found.');
}
// Type the comment if provided
if (comment) {
console.log('[x-quote] Typing comment...');
// Use CDP Input.insertText for more reliable text insertion
await cdp.send('Runtime.evaluate', {
expression: `document.querySelector('[data-testid="tweetTextarea_0"]')?.focus()`,
}, { sessionId });
await sleep(200);
await cdp.send('Input.insertText', {
text: comment,
}, { sessionId });
await sleep(500);
}
if (submit) {
console.log('[x-quote] Submitting quote post...');
await cdp.send('Runtime.evaluate', {
expression: `document.querySelector('[data-testid="tweetButton"]')?.click()`,
}, { sessionId });
await sleep(2000);
console.log('[x-quote] Quote post submitted!');
} else {
console.log('[x-quote] Quote composed (preview mode). Add --submit to post.');
console.log('[x-quote] Browser will stay open for 30 seconds for preview...');
await sleep(30_000);
}
} finally {
if (cdp) {
try { await cdp.send('Browser.close', {}, { timeoutMs: 5_000 }); } catch {}
cdp.close();
}
setTimeout(() => {
if (!chrome.killed) try { chrome.kill('SIGKILL'); } catch {}
}, 2_000).unref?.();
try { chrome.kill('SIGTERM'); } catch {}
}
}
function printUsage(): never {
console.log(`Quote a tweet on X (Twitter) using real Chrome browser
Usage:
npx -y bun x-quote.ts <tweet-url> [options] [comment]
Options:
--submit Actually post (default: preview only)
--profile <dir> Chrome profile directory
--help Show this help
Examples:
npx -y bun x-quote.ts https://x.com/user/status/123456789 "Great insight!"
npx -y bun x-quote.ts https://x.com/user/status/123456789 "I agree!" --submit
`);
process.exit(0);
}
async function main(): Promise<void> {
const args = process.argv.slice(2);
if (args.includes('--help') || args.includes('-h')) printUsage();
let tweetUrl: string | undefined;
let submit = false;
let profileDir: string | undefined;
const commentParts: string[] = [];
for (let i = 0; i < args.length; i++) {
const arg = args[i]!;
if (arg === '--submit') {
submit = true;
} else if (arg === '--profile' && args[i + 1]) {
profileDir = args[++i];
} else if (!arg.startsWith('-')) {
// First non-option argument is the tweet URL
if (!tweetUrl && arg.match(/(?:x\.com|twitter\.com)\/\w+\/status\/\d+/)) {
tweetUrl = extractTweetUrl(arg) ?? undefined;
} else {
commentParts.push(arg);
}
}
}
if (!tweetUrl) {
console.error('Error: Please provide a tweet URL.');
console.error('Example: npx -y bun x-quote.ts https://x.com/user/status/123456789 "Your comment"');
process.exit(1);
}
const comment = commentParts.join(' ').trim() || undefined;
await quotePost({ tweetUrl, comment, submit, profileDir });
}
await main().catch((err) => {
console.error(`Error: ${err instanceof Error ? err.message : String(err)}`);
process.exit(1);
});
+242
View File
@@ -0,0 +1,242 @@
import { spawnSync } from 'node:child_process';
import fs from 'node:fs';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import process from 'node:process';
export type PlatformCandidates = {
darwin?: string[];
win32?: string[];
default: string[];
};
export const CHROME_CANDIDATES_BASIC: PlatformCandidates = {
darwin: [
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
'/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary',
'/Applications/Chromium.app/Contents/MacOS/Chromium',
],
win32: [
'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
],
default: [
'/usr/bin/google-chrome',
'/usr/bin/chromium',
'/usr/bin/chromium-browser',
],
};
export const CHROME_CANDIDATES_FULL: PlatformCandidates = {
darwin: [
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
'/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary',
'/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta',
'/Applications/Chromium.app/Contents/MacOS/Chromium',
'/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge',
],
win32: [
'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
'C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe',
'C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe',
],
default: [
'/usr/bin/google-chrome',
'/usr/bin/google-chrome-stable',
'/usr/bin/chromium',
'/usr/bin/chromium-browser',
'/snap/bin/chromium',
'/usr/bin/microsoft-edge',
],
};
function getCandidatesForPlatform(candidates: PlatformCandidates): string[] {
if (process.platform === 'darwin' && candidates.darwin?.length) return candidates.darwin;
if (process.platform === 'win32' && candidates.win32?.length) return candidates.win32;
return candidates.default;
}
export function findChromeExecutable(candidates: PlatformCandidates): string | undefined {
const override = process.env.X_BROWSER_CHROME_PATH?.trim();
if (override && fs.existsSync(override)) return override;
for (const candidate of getCandidatesForPlatform(candidates)) {
if (fs.existsSync(candidate)) return candidate;
}
return undefined;
}
export function getDefaultProfileDir(): string {
const base = process.env.XDG_DATA_HOME || path.join(os.homedir(), '.local', 'share');
return path.join(base, 'x-browser-profile');
}
export function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
export async function getFreePort(): Promise<number> {
return new Promise((resolve, reject) => {
const server = net.createServer();
server.unref();
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
if (!address || typeof address === 'string') {
server.close(() => reject(new Error('Unable to allocate a free TCP port.')));
return;
}
const port = address.port;
server.close((err) => {
if (err) reject(err);
else resolve(port);
});
});
});
}
async function fetchJson<T = unknown>(url: string): Promise<T> {
const res = await fetch(url, { redirect: 'follow' });
if (!res.ok) throw new Error(`Request failed: ${res.status} ${res.statusText}`);
return (await res.json()) as T;
}
export async function waitForChromeDebugPort(
port: number,
timeoutMs: number,
options?: { includeLastError?: boolean },
): Promise<string> {
const start = Date.now();
let lastError: unknown = null;
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(`http://127.0.0.1:${port}/json/version`);
if (version.webSocketDebuggerUrl) return version.webSocketDebuggerUrl;
lastError = new Error('Missing webSocketDebuggerUrl');
} catch (error) {
lastError = error;
}
await sleep(200);
}
if (options?.includeLastError && lastError) {
throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
}
throw new Error('Chrome debug port not ready');
}
type PendingRequest = {
resolve: (value: unknown) => void;
reject: (error: Error) => void;
timer: ReturnType<typeof setTimeout> | null;
};
export class CdpConnection {
private ws: WebSocket;
private nextId = 0;
private pending = new Map<number, PendingRequest>();
private eventHandlers = new Map<string, Set<(params: unknown) => void>>();
private defaultTimeoutMs: number;
private constructor(ws: WebSocket, options?: { defaultTimeoutMs?: number }) {
this.ws = ws;
this.defaultTimeoutMs = options?.defaultTimeoutMs ?? 15_000;
this.ws.addEventListener('message', (event) => {
try {
const data = typeof event.data === 'string' ? event.data : new TextDecoder().decode(event.data as ArrayBuffer);
const msg = JSON.parse(data) as { id?: number; method?: string; params?: unknown; result?: unknown; error?: { message?: string } };
if (msg.method) {
const handlers = this.eventHandlers.get(msg.method);
if (handlers) handlers.forEach((h) => h(msg.params));
}
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer) clearTimeout(pending.timer);
if (msg.error?.message) pending.reject(new Error(msg.error.message));
else pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener('close', () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer) clearTimeout(pending.timer);
pending.reject(new Error('CDP connection closed.'));
}
});
}
static async connect(url: string, timeoutMs: number, options?: { defaultTimeoutMs?: number }): Promise<CdpConnection> {
const ws = new WebSocket(url);
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => reject(new Error('CDP connection timeout.')), timeoutMs);
ws.addEventListener('open', () => { clearTimeout(timer); resolve(); });
ws.addEventListener('error', () => { clearTimeout(timer); reject(new Error('CDP connection failed.')); });
});
return new CdpConnection(ws, options);
}
on(method: string, handler: (params: unknown) => void): void {
if (!this.eventHandlers.has(method)) this.eventHandlers.set(method, new Set());
this.eventHandlers.get(method)!.add(handler);
}
async send<T = unknown>(method: string, params?: Record<string, unknown>, options?: { sessionId?: string; timeoutMs?: number }): Promise<T> {
const id = ++this.nextId;
const message: Record<string, unknown> = { id, method };
if (params) message.params = params;
if (options?.sessionId) message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
const result = await new Promise<unknown>((resolve, reject) => {
const timer = timeoutMs > 0
? setTimeout(() => { this.pending.delete(id); reject(new Error(`CDP timeout: ${method}`)); }, timeoutMs)
: null;
this.pending.set(id, { resolve, reject, timer });
this.ws.send(JSON.stringify(message));
});
return result as T;
}
close(): void {
try { this.ws.close(); } catch {}
}
}
export function getScriptDir(): string {
return path.dirname(new URL(import.meta.url).pathname);
}
function runBunScript(scriptPath: string, args: string[]): boolean {
const result = spawnSync('npx', ['-y', 'bun', scriptPath, ...args], { stdio: 'inherit' });
return result.status === 0;
}
export function copyImageToClipboard(imagePath: string): boolean {
const copyScript = path.join(getScriptDir(), 'copy-to-clipboard.ts');
return runBunScript(copyScript, ['image', imagePath]);
}
export function copyHtmlToClipboard(htmlPath: string): boolean {
const copyScript = path.join(getScriptDir(), 'copy-to-clipboard.ts');
return runBunScript(copyScript, ['html', '--file', htmlPath]);
}
export function pasteFromClipboard(targetApp?: string, retries = 3, delayMs = 500): boolean {
const pasteScript = path.join(getScriptDir(), 'paste-from-clipboard.ts');
const args = ['--retries', String(retries), '--delay', String(delayMs)];
if (targetApp) args.push('--app', targetApp);
return runBunScript(pasteScript, args);
}
+39 -186
View File
@@ -1,172 +1,20 @@
import { spawn } from 'node:child_process';
import fs from 'node:fs';
import { mkdir } from 'node:fs/promises';
import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import process from 'node:process';
import {
CHROME_CANDIDATES_FULL,
CdpConnection,
findChromeExecutable,
getDefaultProfileDir,
getFreePort,
sleep,
waitForChromeDebugPort,
} from './x-utils.js';
const X_COMPOSE_URL = 'https://x.com/compose/post';
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function getFreePort(): Promise<number> {
return new Promise((resolve, reject) => {
const server = net.createServer();
server.unref();
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
if (!address || typeof address === 'string') {
server.close(() => reject(new Error('Unable to allocate a free TCP port.')));
return;
}
const port = address.port;
server.close((err) => {
if (err) reject(err);
else resolve(port);
});
});
});
}
function findChromeExecutable(): string | undefined {
const override = process.env.X_BROWSER_CHROME_PATH?.trim();
if (override && fs.existsSync(override)) return override;
const candidates: string[] = [];
switch (process.platform) {
case 'darwin':
candidates.push(
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
'/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary',
'/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta',
'/Applications/Chromium.app/Contents/MacOS/Chromium',
'/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge',
);
break;
case 'win32':
candidates.push(
'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
'C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe',
'C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe',
);
break;
default:
candidates.push(
'/usr/bin/google-chrome',
'/usr/bin/google-chrome-stable',
'/usr/bin/chromium',
'/usr/bin/chromium-browser',
'/snap/bin/chromium',
'/usr/bin/microsoft-edge',
);
break;
}
for (const p of candidates) {
if (fs.existsSync(p)) return p;
}
return undefined;
}
function getDefaultProfileDir(): string {
const base = process.env.XDG_DATA_HOME || path.join(os.homedir(), '.local', 'share');
return path.join(base, 'x-browser-profile');
}
async function fetchJson<T = unknown>(url: string): Promise<T> {
const res = await fetch(url, { redirect: 'follow' });
if (!res.ok) throw new Error(`Request failed: ${res.status} ${res.statusText}`);
return (await res.json()) as T;
}
async function waitForChromeDebugPort(port: number, timeoutMs: number): Promise<string> {
const start = Date.now();
let lastError: unknown = null;
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(`http://127.0.0.1:${port}/json/version`);
if (version.webSocketDebuggerUrl) return version.webSocketDebuggerUrl;
lastError = new Error('Missing webSocketDebuggerUrl');
} catch (error) {
lastError = error;
}
await sleep(200);
}
throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
}
class CdpConnection {
private ws: WebSocket;
private nextId = 0;
private pending = new Map<number, { resolve: (v: unknown) => void; reject: (e: Error) => void; timer: ReturnType<typeof setTimeout> | null }>();
private constructor(ws: WebSocket) {
this.ws = ws;
this.ws.addEventListener('message', (event) => {
try {
const data = typeof event.data === 'string' ? event.data : new TextDecoder().decode(event.data as ArrayBuffer);
const msg = JSON.parse(data) as { id?: number; result?: unknown; error?: { message?: string } };
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer) clearTimeout(pending.timer);
if (msg.error?.message) pending.reject(new Error(msg.error.message));
else pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener('close', () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer) clearTimeout(pending.timer);
pending.reject(new Error('CDP connection closed.'));
}
});
}
static async connect(url: string, timeoutMs: number): Promise<CdpConnection> {
const ws = new WebSocket(url);
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => reject(new Error('CDP connection timeout.')), timeoutMs);
ws.addEventListener('open', () => { clearTimeout(timer); resolve(); });
ws.addEventListener('error', () => { clearTimeout(timer); reject(new Error('CDP connection failed.')); });
});
return new CdpConnection(ws);
}
async send<T = unknown>(method: string, params?: Record<string, unknown>, options?: { sessionId?: string; timeoutMs?: number }): Promise<T> {
const id = ++this.nextId;
const message: Record<string, unknown> = { id, method };
if (params) message.params = params;
if (options?.sessionId) message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? 30_000;
const result = await new Promise<unknown>((resolve, reject) => {
const timer = timeoutMs > 0 ? setTimeout(() => { this.pending.delete(id); reject(new Error(`CDP timeout: ${method}`)); }, timeoutMs) : null;
this.pending.set(id, { resolve, reject, timer });
this.ws.send(JSON.stringify(message));
});
return result as T;
}
close(): void {
try { this.ws.close(); } catch {}
}
}
interface XVideoOptions {
text?: string;
videoPath: string;
@@ -179,7 +27,7 @@ interface XVideoOptions {
export async function postVideoToX(options: XVideoOptions): Promise<void> {
const { text, videoPath, submit = false, timeoutMs = 120_000, profileDir = getDefaultProfileDir() } = options;
const chromePath = options.chromePath ?? findChromeExecutable();
const chromePath = options.chromePath ?? findChromeExecutable(CHROME_CANDIDATES_FULL);
if (!chromePath) throw new Error('Chrome not found. Set X_BROWSER_CHROME_PATH env var.');
if (!fs.existsSync(videoPath)) throw new Error(`Video not found: ${videoPath}`);
@@ -205,8 +53,8 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
let cdp: CdpConnection | null = null;
try {
const wsUrl = await waitForChromeDebugPort(port, 30_000);
cdp = await CdpConnection.connect(wsUrl, 30_000);
const wsUrl = await waitForChromeDebugPort(port, 30_000, { includeLastError: true });
cdp = await CdpConnection.connect(wsUrl, 30_000, { defaultTimeoutMs: 30_000 });
const targets = await cdp.send<{ targetInfos: Array<{ targetId: string; url: string; type: string }> }>('Target.getTargets');
let pageTarget = targets.targetInfos.find((t) => t.type === 'page' && t.url.includes('x.com'));
@@ -264,24 +112,44 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
nodeId,
files: [absVideoPath],
}, { sessionId });
console.log('[x-video] Video file set, waiting for processing...');
console.log('[x-video] Video file set, uploading in background...');
// Wait for video to process
// Wait a moment for upload to start, then type text while video processes
await sleep(2000);
// Type text while video uploads in background
if (text) {
console.log('[x-video] Typing text...');
await cdp.send('Runtime.evaluate', {
expression: `
const editor = document.querySelector('[data-testid="tweetTextarea_0"]');
if (editor) {
editor.focus();
document.execCommand('insertText', false, ${JSON.stringify(text)});
}
`,
}, { sessionId });
await sleep(500);
}
// Wait for video to finish processing by checking if tweet button is enabled
console.log('[x-video] Waiting for video processing...');
const waitForVideoReady = async (maxWaitMs = 180_000): Promise<boolean> => {
const start = Date.now();
let dots = 0;
while (Date.now() - start < maxWaitMs) {
const result = await cdp!.send<{ result: { value: { hasMedia: boolean; isProcessing: boolean } } }>('Runtime.evaluate', {
const result = await cdp!.send<{ result: { value: { hasMedia: boolean; buttonEnabled: boolean } } }>('Runtime.evaluate', {
expression: `(() => {
const hasMedia = !!document.querySelector('[data-testid="attachments"] video, [data-testid="videoPlayer"], video');
const isProcessing = !!document.querySelector('[role="progressbar"], [data-testid="progressBar"]');
return { hasMedia, isProcessing };
const tweetBtn = document.querySelector('[data-testid="tweetButton"]');
const buttonEnabled = tweetBtn && !tweetBtn.disabled && tweetBtn.getAttribute('aria-disabled') !== 'true';
return { hasMedia, buttonEnabled };
})()`,
returnByValue: true,
}, { sessionId });
const { hasMedia, isProcessing } = result.result.value;
if (hasMedia && !isProcessing) {
const { hasMedia, buttonEnabled } = result.result.value;
if (hasMedia && buttonEnabled) {
console.log('');
return true;
}
@@ -302,21 +170,6 @@ export async function postVideoToX(options: XVideoOptions): Promise<void> {
console.log('[x-video] Video may still be processing. Please check browser window.');
}
// Type text AFTER video is uploaded
if (text) {
console.log('[x-video] Typing text...');
await cdp.send('Runtime.evaluate', {
expression: `
const editor = document.querySelector('[data-testid="tweetTextarea_0"]');
if (editor) {
editor.focus();
document.execCommand('insertText', false, ${JSON.stringify(text)});
}
`,
}, { sessionId });
await sleep(500);
}
if (submit) {
console.log('[x-video] Submitting post...');
await cdp.send('Runtime.evaluate', {
+226 -1
View File
@@ -41,11 +41,74 @@ Transform content into professional slide deck images with flexible style option
| `--style <name>` | Visual style (see Style Gallery) |
| `--audience <type>` | Target audience: beginners, intermediate, experts, executives, general |
| `--lang <code>` | Output language (en, zh, ja, etc.) |
| `--slides <number>` | Target slide count |
| `--slides <number>` | Target slide count (recommended: 8-25, max: 30) |
| `--outline-only` | Generate outline only, skip image generation |
**Slide Count Guidance**:
| Content Length | Recommended Slides |
|----------------|-------------------|
| Short (< 1000 words) | 5-10 |
| Medium (1000-3000 words) | 10-18 |
| Long (3000-5000 words) | 15-25 |
| Very Long (> 5000 words) | 20-30 (consider splitting) |
Maximum 30 slides per deck. For longer content, split into multiple decks.
## Audience Guidelines
Design decisions should adapt to target audience. Use `--audience` to set.
| Audience | Content Density | Visual Style | Terminology | Slides |
|----------|-----------------|--------------|-------------|--------|
| `beginners` | Low | Friendly, illustrative | Plain language | 8-15 |
| `intermediate` | Medium | Balanced, structured | Some jargon OK | 10-20 |
| `experts` | High | Data-rich, precise | Technical terms | 12-25 |
| `executives` | Medium-High | Clean, impactful | Business language | 8-12 |
| `general` | Medium | Accessible, engaging | Minimal jargon | 10-18 |
### Audience-Specific Principles
**Beginners**:
- One concept per slide
- Visual metaphors over abstract diagrams
- Step-by-step progression
- Generous whitespace
**Experts**:
- Multiple data points per slide acceptable
- Technical diagrams with precise labels
- Assume domain knowledge
- Dense but organized information
**Executives**:
- Lead with insights, not data
- "So what?" on every slide
- Decision-enabling content
- Bottom-line upfront (BLUF)
## Style Gallery
### Style Selection Principles
**Content-First Approach**:
1. Analyze content topic, mood, and industry before selecting
2. Consider target audience expectations
3. Match style to subject matter (not personal preference)
**Quick Reference by Content Type**:
| Content Type | Recommended Styles |
|--------------|-------------------|
| Technical/Architecture | `blueprint`, `intuition-machine` |
| Educational/Tutorials | `sketch-notes`, `chalkboard` |
| Corporate/Business | `corporate`, `minimal` |
| Creative/Artistic | `vector-illustration`, `watercolor` |
| Product/SaaS | `notion`, `bold-editorial` |
| Scientific/Research | `scientific`, `editorial-infographic` |
**Note**: Full style specifications in `references/styles/<style>.md`
### Available Styles
| Style | Description | Best For |
|-------|-------------|----------|
| `blueprint` (Default) | Technical schematics, grid texture | Architecture, system design |
@@ -127,6 +190,32 @@ Optional layout hints for individual slides. Specify in outline's `// LAYOUT` se
**Usage**: Add `Layout: <name>` in slide's `// LAYOUT` section to guide visual composition.
### Layout Selection Tips
**Match Layout to Content**:
| Content Type | Recommended Layouts |
|--------------|-------------------|
| Single narrative | `bullet-list`, `image-caption` |
| Two concepts | `split-screen`, `binary-comparison` |
| Three items | `three-columns`, `icon-grid` |
| Process/Steps | `linear-progression`, `winding-roadmap` |
| Data/Metrics | `dashboard`, `key-stat` |
| Relationships | `hub-spoke`, `venn-diagram` |
| Hierarchy | `hierarchical-layers`, `tree-branching` |
**Layout Flow Patterns**:
| Position | Recommended Layouts |
|----------|-------------------|
| Opening | `title-hero`, `agenda` |
| Middle | Content-specific layouts |
| Closing | `quote-callout`, `key-stat` |
**Common Mistakes to Avoid**:
- ✗ Using 3-column layout for 2 items (leaves columns empty)
- ✗ Stacking charts/tables below text (use side-by-side instead)
- ✗ Image layouts without actual images
- ✗ Quote layouts for emphasis (use only for real quotes with attribution)
## Design Philosophy
This deck is designed for **reading and sharing**, not live presentation:
@@ -135,6 +224,142 @@ This deck is designed for **reading and sharing**, not live presentation:
- Include **all necessary context** within each slide
- Optimize for **social media sharing** and offline reading
### Visual Hierarchy Principles
| Principle | Description |
|-----------|-------------|
| Focal Point | ONE dominant element per slide draws attention first |
| Rule of Thirds | Position key elements at grid intersections |
| Z-Pattern | Guide eye: top-left → top-right → bottom-left → bottom-right |
| Size Contrast | Headlines 2-3x larger than body text |
| Breathing Room | Minimum 10% margin from all edges |
### Content Density
Professional presentations balance information density with clarity.
| Level | Description | Use When |
|-------|-------------|----------|
| High | Multiple data points, detailed charts, dense text | Expert audience, technical reviews, data-driven decisions |
| Medium | Key points with supporting details, moderate visuals | General business, mixed audiences |
| Low | One main idea, large visuals, minimal text | Beginners, keynotes, emotional impact |
**High-Density Principles** (McKinsey-style):
- Every element earns its space
- Data speaks louder than decoration
- Annotations explain insights, not describe data
- White space is strategic, not filler
- Information hierarchy guides the eye
**Density by Slide Type**:
| Slide Type | Recommended Density |
|------------|-------------------|
| Cover/Title | Low |
| Agenda/Overview | Medium |
| Content/Analysis | Medium-High |
| Data/Metrics | High |
| Quote/Impact | Low |
| Summary/Takeaway | Medium |
### Color Selection
**Content-First Approach**:
1. Analyze content topic, mood, and industry
2. Consider target audience expectations
3. Match palette to subject matter (not defaults)
4. Ensure strong contrast for readability
**Quick Palette Guide**:
| Content Type | Recommended Palettes |
|--------------|---------------------|
| Technical/Architecture | Blues, grays, blueprint tones |
| Educational/Friendly | Warm colors, earth tones |
| Corporate/Professional | Navy, gold, structured palettes |
| Creative/Artistic | Bold colors, unexpected combinations |
| Scientific/Medical | Clean whites, precise color coding |
**Note**: Full color specs in `references/styles/<style>.md`
### Typography Principles
| Element | Treatment |
|---------|-----------|
| Headlines | Bold, 2-3x body size, narrative style |
| Body Text | Regular weight, readable size |
| Captions | Smaller, lighter weight |
| Data Labels | Monospace for technical content |
| Emphasis | Use bold or color, not underlines |
### Font Recommendations
Fonts for AI-generated slides. Specify in prompts for consistent rendering.
**English Fonts**:
| Font | Style | Best For |
|------|-------|----------|
| Liter | Sans-serif, geometric | Modern, clean, technical |
| HedvigLettersSans | Sans-serif, distinctive | Brand-forward, creative |
| Oranienbaum | High-contrast serif | Elegant, classical |
| SortsMillGoudy | Classical serif | Traditional, readable |
| Coda | Round sans-serif | Friendly, approachable |
**Chinese Fonts**:
| Font | Style | Best For |
|------|-------|----------|
| MiSans | Modern sans-serif | Clean, versatile, screen-optimized |
| Noto Sans SC | Neutral sans-serif | Standard, multilingual |
| siyuanSongti | Refined Song typeface | Elegant, editorial |
| alimamashuheiti | Geometric sans-serif | Commercial, structured |
| LXGW Bright | Song-Kai hybrid | Warm, readable |
**Multilingual Pairing**:
| Use Case | English | Chinese |
|----------|---------|---------|
| Technical | Liter | MiSans |
| Editorial | Oranienbaum | siyuanSongti |
| Friendly | Coda | LXGW Bright |
| Corporate | HedvigLettersSans | alimamashuheiti |
### Consistency Requirements
| Element | Guideline |
|---------|-----------|
| Spacing | Consistent margins and padding throughout |
| Colors | Maximum 3-4 colors per slide, palette consistent across deck |
| Typography | Same font families and sizes for same content types |
| Visual Language | Repeat patterns, shapes, and treatments |
## Visual Elements Reference
Quick reference for visual treatments. Full specs in style definitions.
### Background Treatments
| Treatment | Description | Best For |
|-----------|-------------|----------|
| Solid color | Single background color | Clean, minimal |
| Split background | Two colors, diagonal or vertical | Contrast, sections |
| Gradient | Subtle vertical or diagonal fade | Modern, dynamic |
| Textured | Pattern or texture overlay | Character, style |
### Typography Treatments
| Treatment | Description | Best For |
|-----------|-------------|----------|
| Size contrast | 3-4x difference headline vs body | Impact, hierarchy |
| All-caps headers | Uppercase with letter spacing | Authority, structure |
| Monospace data | Fixed-width for numbers/code | Technical, precision |
| Hand-drawn | Organic, imperfect letterforms | Friendly, approachable |
### Geometric Accents
| Element | Description | Best For |
|---------|-------------|----------|
| Diagonal dividers | Angled section separators | Energy, movement |
| Corner brackets | L-shaped frames | Focus, framing |
| Circles/hexagons | Shape frames for images | Modern, tech |
| Underline accents | Thick lines under headers | Emphasis, hierarchy |
## File Management
### Output Directory
+169
View File
@@ -0,0 +1,169 @@
---
name: baoyu-url-to-markdown
description: Fetch any URL and convert to markdown using Chrome CDP. Supports two modes - auto-capture on page load, or wait for user signal (for pages requiring login). Use when user wants to save a webpage as markdown.
---
# URL to Markdown
Fetches any URL via Chrome CDP and converts HTML to clean markdown.
## Script Directory
**Important**: All scripts are located in the `scripts/` subdirectory of this skill.
**Agent Execution Instructions**:
1. Determine this SKILL.md file's directory path as `SKILL_DIR`
2. Script path = `${SKILL_DIR}/scripts/<script-name>.ts`
3. Replace all `${SKILL_DIR}` in this document with the actual path
**Script Reference**:
| Script | Purpose |
|--------|---------|
| `scripts/main.ts` | CLI entry point for URL fetching |
## Features
- Chrome CDP for full JavaScript rendering
- Two capture modes: auto or wait-for-user
- Clean markdown output with metadata
- Handles login-required pages via wait mode
## Usage
```bash
# Auto mode (default) - capture when page loads
npx -y bun ${SKILL_DIR}/scripts/main.ts <url>
# Wait mode - wait for user signal before capture
npx -y bun ${SKILL_DIR}/scripts/main.ts <url> --wait
# Save to specific file
npx -y bun ${SKILL_DIR}/scripts/main.ts <url> -o output.md
```
## Options
| Option | Description |
|--------|-------------|
| `<url>` | URL to fetch |
| `-o <path>` | Output file path (default: auto-generated) |
| `--wait` | Wait for user signal before capturing |
| `--timeout <ms>` | Page load timeout (default: 30000) |
## Capture Modes
### Auto Mode (default)
Page loads → waits for network idle → captures immediately.
Best for:
- Public pages
- Static content
- No login required
### Wait Mode (`--wait`)
Page opens → user can interact (login, scroll, etc.) → user signals ready → captures.
Best for:
- Login-required pages
- Dynamic content needing interaction
- Pages with lazy loading
**Agent workflow for wait mode**:
1. Run script with `--wait` flag
2. Script outputs: `Page opened. Press Enter when ready to capture...`
3. Use `AskUserQuestion` to ask user if page is ready
4. When user confirms, send newline to stdin to trigger capture
## Output Format
```markdown
---
url: https://example.com/page
title: "Page Title"
description: "Meta description if available"
author: "Author if available"
published: "2024-01-01"
captured_at: "2024-01-15T10:30:00Z"
---
# Page Title
Converted markdown content...
```
## Mode Selection Guide
When user requests URL capture, help select appropriate mode:
**Suggest Auto Mode when**:
- URL is public (no login wall visible)
- Content appears static
- User doesn't mention login requirements
**Suggest Wait Mode when**:
- User mentions needing to log in
- Site known to require authentication
- User wants to scroll/interact before capture
- Content is behind paywall
**Ask user when unclear**:
```
The page may require login or interaction before capturing.
Which mode should I use?
1. Auto - Capture immediately when loaded
2. Wait - Wait for you to interact first
```
## Output Directory
Each capture creates a file organized by domain:
```
url-to-markdown/
└── <domain>/
└── <slug>.md
```
**Path Components**:
- `<domain>`: Site domain (e.g., `example.com`, `github.com`)
- `<slug>`: Generated from page title or URL path (kebab-case)
**Slug Generation**:
1. Extract from page title (preferred) or URL path
2. Convert to kebab-case, 2-6 words
3. Example: "Getting Started with React" → `getting-started-with-react`
**Conflict Resolution**:
If `url-to-markdown/<domain>/<slug>.md` already exists:
- Append timestamp: `<slug>-YYYYMMDD-HHMMSS.md`
- Example: `getting-started.md` exists → `getting-started-20260118-143052.md`
## Error Handling
| Error | Resolution |
|-------|------------|
| Chrome not found | Install Chrome or set `URL_CHROME_PATH` env |
| Page timeout | Increase `--timeout` value |
| Capture failed | Try wait mode for complex pages |
| Empty content | Page may need JS rendering time |
## Environment Variables
| Variable | Description |
|----------|-------------|
| `URL_CHROME_PATH` | Custom Chrome executable path |
| `URL_DATA_DIR` | Custom data directory |
| `URL_CHROME_PROFILE_DIR` | Custom Chrome profile directory |
## Extension Support
Custom configurations via EXTEND.md.
**Check paths** (priority order):
1. `.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` (project)
2. `~/.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` (user)
If found, load before workflow. Extension content overrides defaults.
+290
View File
@@ -0,0 +1,290 @@
import { spawn, type ChildProcess } from "node:child_process";
import fs from "node:fs";
import { mkdir } from "node:fs/promises";
import net from "node:net";
import process from "node:process";
import { resolveUrlToMarkdownChromeProfileDir } from "./paths.js";
import { CDP_CONNECT_TIMEOUT_MS, NETWORK_IDLE_TIMEOUT_MS } from "./constants.js";
type CdpSendOptions = { sessionId?: string; timeoutMs?: number };
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function fetchWithTimeout(url: string, init: RequestInit & { timeoutMs?: number } = {}): Promise<Response> {
const { timeoutMs, ...rest } = init;
if (!timeoutMs || timeoutMs <= 0) return fetch(url, rest);
const ctl = new AbortController();
const t = setTimeout(() => ctl.abort(), timeoutMs);
try {
return await fetch(url, { ...rest, signal: ctl.signal });
} finally {
clearTimeout(t);
}
}
export class CdpConnection {
private ws: WebSocket;
private nextId = 0;
private pending = new Map<number, { resolve: (v: unknown) => void; reject: (e: Error) => void; timer: ReturnType<typeof setTimeout> | null }>();
private eventHandlers = new Map<string, Set<(params: unknown) => void>>();
private constructor(ws: WebSocket) {
this.ws = ws;
this.ws.addEventListener("message", (event) => {
try {
const data = typeof event.data === "string" ? event.data : new TextDecoder().decode(event.data as ArrayBuffer);
const msg = JSON.parse(data) as { id?: number; method?: string; params?: unknown; result?: unknown; error?: { message?: string } };
if (msg.id) {
const p = this.pending.get(msg.id);
if (p) {
this.pending.delete(msg.id);
if (p.timer) clearTimeout(p.timer);
if (msg.error?.message) p.reject(new Error(msg.error.message));
else p.resolve(msg.result);
}
} else if (msg.method) {
const handlers = this.eventHandlers.get(msg.method);
if (handlers) {
for (const h of handlers) h(msg.params);
}
}
} catch {}
});
this.ws.addEventListener("close", () => {
for (const [id, p] of this.pending.entries()) {
this.pending.delete(id);
if (p.timer) clearTimeout(p.timer);
p.reject(new Error("CDP connection closed."));
}
});
}
static async connect(url: string, timeoutMs: number): Promise<CdpConnection> {
const ws = new WebSocket(url);
await new Promise<void>((resolve, reject) => {
const t = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
ws.addEventListener("open", () => { clearTimeout(t); resolve(); });
ws.addEventListener("error", () => { clearTimeout(t); reject(new Error("CDP connection failed.")); });
});
return new CdpConnection(ws);
}
on(event: string, handler: (params: unknown) => void): void {
let handlers = this.eventHandlers.get(event);
if (!handlers) {
handlers = new Set();
this.eventHandlers.set(event, handlers);
}
handlers.add(handler);
}
off(event: string, handler: (params: unknown) => void): void {
this.eventHandlers.get(event)?.delete(handler);
}
async send<T = unknown>(method: string, params?: Record<string, unknown>, opts?: CdpSendOptions): Promise<T> {
const id = ++this.nextId;
const msg: Record<string, unknown> = { id, method };
if (params) msg.params = params;
if (opts?.sessionId) msg.sessionId = opts.sessionId;
const timeoutMs = opts?.timeoutMs ?? 15_000;
const out = await new Promise<unknown>((resolve, reject) => {
const t = timeoutMs > 0 ? setTimeout(() => { this.pending.delete(id); reject(new Error(`CDP timeout: ${method}`)); }, timeoutMs) : null;
this.pending.set(id, { resolve, reject, timer: t });
this.ws.send(JSON.stringify(msg));
});
return out as T;
}
close(): void {
try { this.ws.close(); } catch {}
}
}
export async function getFreePort(): Promise<number> {
return await new Promise((resolve, reject) => {
const srv = net.createServer();
srv.unref();
srv.on("error", reject);
srv.listen(0, "127.0.0.1", () => {
const addr = srv.address();
if (!addr || typeof addr === "string") {
srv.close(() => reject(new Error("Unable to allocate a free TCP port.")));
return;
}
const port = addr.port;
srv.close((err) => (err ? reject(err) : resolve(port)));
});
});
}
export function findChromeExecutable(): string | null {
const override = process.env.URL_CHROME_PATH?.trim();
if (override && fs.existsSync(override)) return override;
const candidates: string[] = [];
switch (process.platform) {
case "darwin":
candidates.push(
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary",
"/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"
);
break;
case "win32":
candidates.push(
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe",
"C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe"
);
break;
default:
candidates.push(
"/usr/bin/google-chrome",
"/usr/bin/google-chrome-stable",
"/usr/bin/chromium",
"/usr/bin/chromium-browser",
"/snap/bin/chromium",
"/usr/bin/microsoft-edge"
);
break;
}
for (const p of candidates) {
if (fs.existsSync(p)) return p;
}
return null;
}
export async function waitForChromeDebugPort(port: number, timeoutMs: number): Promise<string> {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
try {
const res = await fetchWithTimeout(`http://127.0.0.1:${port}/json/version`, { timeoutMs: 5_000 });
if (!res.ok) throw new Error(`status=${res.status}`);
const j = (await res.json()) as { webSocketDebuggerUrl?: string };
if (j.webSocketDebuggerUrl) return j.webSocketDebuggerUrl;
} catch {}
await sleep(200);
}
throw new Error("Chrome debug port not ready");
}
export async function launchChrome(url: string, port: number, headless: boolean = false): Promise<ChildProcess> {
const chrome = findChromeExecutable();
if (!chrome) throw new Error("Chrome executable not found. Install Chrome or set URL_CHROME_PATH env.");
const profileDir = resolveUrlToMarkdownChromeProfileDir();
await mkdir(profileDir, { recursive: true });
const args = [
`--remote-debugging-port=${port}`,
`--user-data-dir=${profileDir}`,
"--no-first-run",
"--no-default-browser-check",
"--disable-popup-blocking",
];
if (headless) args.push("--headless=new");
args.push(url);
return spawn(chrome, args, { stdio: "ignore" });
}
export async function waitForNetworkIdle(cdp: CdpConnection, sessionId: string, timeoutMs: number = NETWORK_IDLE_TIMEOUT_MS): Promise<void> {
return new Promise((resolve) => {
let timer: ReturnType<typeof setTimeout> | null = null;
let pending = 0;
const cleanup = () => {
if (timer) clearTimeout(timer);
cdp.off("Network.requestWillBeSent", onRequest);
cdp.off("Network.loadingFinished", onFinish);
cdp.off("Network.loadingFailed", onFinish);
};
const done = () => { cleanup(); resolve(); };
const resetTimer = () => {
if (timer) clearTimeout(timer);
timer = setTimeout(done, timeoutMs);
};
const onRequest = () => { pending++; resetTimer(); };
const onFinish = () => { pending = Math.max(0, pending - 1); if (pending <= 2) resetTimer(); };
cdp.on("Network.requestWillBeSent", onRequest);
cdp.on("Network.loadingFinished", onFinish);
cdp.on("Network.loadingFailed", onFinish);
resetTimer();
});
}
export async function waitForPageLoad(cdp: CdpConnection, sessionId: string, timeoutMs: number = 30_000): Promise<void> {
return new Promise((resolve, reject) => {
const timer = setTimeout(() => {
cdp.off("Page.loadEventFired", handler);
resolve();
}, timeoutMs);
const handler = () => {
clearTimeout(timer);
cdp.off("Page.loadEventFired", handler);
resolve();
};
cdp.on("Page.loadEventFired", handler);
});
}
export async function createTargetAndAttach(cdp: CdpConnection, url: string): Promise<{ targetId: string; sessionId: string }> {
const { targetId } = await cdp.send<{ targetId: string }>("Target.createTarget", { url });
const { sessionId } = await cdp.send<{ sessionId: string }>("Target.attachToTarget", { targetId, flatten: true });
await cdp.send("Network.enable", {}, { sessionId });
await cdp.send("Page.enable", {}, { sessionId });
return { targetId, sessionId };
}
export async function navigateAndWait(cdp: CdpConnection, sessionId: string, url: string, timeoutMs: number): Promise<void> {
const loadPromise = new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => reject(new Error("Page load timeout")), timeoutMs);
const handler = (params: unknown) => {
const p = params as { name?: string };
if (p.name === "load" || p.name === "DOMContentLoaded") {
clearTimeout(timer);
cdp.off("Page.lifecycleEvent", handler);
resolve();
}
};
cdp.on("Page.lifecycleEvent", handler);
});
await cdp.send("Page.navigate", { url }, { sessionId });
await loadPromise;
}
export async function evaluateScript<T>(cdp: CdpConnection, sessionId: string, expression: string, timeoutMs: number = 30_000): Promise<T> {
const result = await cdp.send<{ result: { value?: T; type?: string; description?: string } }>(
"Runtime.evaluate",
{ expression, returnByValue: true, awaitPromise: true },
{ sessionId, timeoutMs }
);
return result.result.value as T;
}
export async function autoScroll(cdp: CdpConnection, sessionId: string, steps: number = 8, waitMs: number = 600): Promise<void> {
let lastHeight = await evaluateScript<number>(cdp, sessionId, "document.body.scrollHeight");
for (let i = 0; i < steps; i++) {
await evaluateScript<void>(cdp, sessionId, "window.scrollTo(0, document.body.scrollHeight)");
await sleep(waitMs);
const newHeight = await evaluateScript<number>(cdp, sessionId, "document.body.scrollHeight");
if (newHeight === lastHeight) break;
lastHeight = newHeight;
}
await evaluateScript<void>(cdp, sessionId, "window.scrollTo(0, 0)");
}
export function killChrome(chrome: ChildProcess): void {
try { chrome.kill("SIGTERM"); } catch {}
setTimeout(() => {
if (!chrome.killed) {
try { chrome.kill("SIGKILL"); } catch {}
}
}, 2_000).unref?.();
}
@@ -0,0 +1,13 @@
import { resolveUrlToMarkdownChromeProfileDir } from "./paths.js";
export const DEFAULT_USER_AGENT =
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36";
export const USER_DATA_DIR = resolveUrlToMarkdownChromeProfileDir();
export const DEFAULT_TIMEOUT_MS = 30_000;
export const CDP_CONNECT_TIMEOUT_MS = 15_000;
export const NETWORK_IDLE_TIMEOUT_MS = 1_500;
export const POST_LOAD_DELAY_MS = 800;
export const SCROLL_STEP_WAIT_MS = 600;
export const SCROLL_MAX_STEPS = 8;
@@ -0,0 +1,223 @@
export interface PageMetadata {
url: string;
title: string;
description?: string;
author?: string;
published?: string;
captured_at: string;
}
export interface ConversionResult {
metadata: PageMetadata;
markdown: string;
}
export const cleanupAndExtractScript = `
(function() {
const removeSelectors = [
'script', 'style', 'noscript', 'iframe', 'svg', 'canvas',
'header nav', 'footer', '.sidebar', '.nav', '.navigation',
'.advertisement', '.ad', '.ads', '.cookie-banner', '.popup',
'[role="banner"]', '[role="navigation"]', '[role="complementary"]'
];
for (const sel of removeSelectors) {
try {
document.querySelectorAll(sel).forEach(el => el.remove());
} catch {}
}
document.querySelectorAll('*').forEach(el => {
el.removeAttribute('style');
el.removeAttribute('onclick');
el.removeAttribute('onload');
el.removeAttribute('onerror');
});
const baseUrl = document.baseURI || location.href;
document.querySelectorAll('a[href]').forEach(a => {
try {
const href = a.getAttribute('href');
if (href && !href.startsWith('#') && !href.startsWith('javascript:')) {
a.setAttribute('href', new URL(href, baseUrl).href);
}
} catch {}
});
document.querySelectorAll('img[src]').forEach(img => {
try {
const src = img.getAttribute('src');
if (src) img.setAttribute('src', new URL(src, baseUrl).href);
} catch {}
});
const getMeta = (names) => {
for (const name of names) {
const el = document.querySelector(\`meta[name="\${name}"], meta[property="\${name}"]\`);
if (el) {
const content = el.getAttribute('content');
if (content) return content.trim();
}
}
return undefined;
};
const getTitle = () => {
const ogTitle = getMeta(['og:title', 'twitter:title']);
if (ogTitle) return ogTitle;
const h1 = document.querySelector('h1');
if (h1) return h1.textContent?.trim();
return document.title?.trim() || '';
};
const getPublished = () => {
const timeEl = document.querySelector('time[datetime]');
if (timeEl) return timeEl.getAttribute('datetime');
return getMeta(['article:published_time', 'datePublished', 'date']);
};
const main = document.querySelector('main, article, [role="main"], .main-content, .post-content, .article-content, .content');
const html = main ? main.innerHTML : document.body.innerHTML;
return {
title: getTitle(),
description: getMeta(['description', 'og:description', 'twitter:description']),
author: getMeta(['author', 'article:author', 'twitter:creator']),
published: getPublished(),
html: html
};
})()
`;
function decodeHtmlEntities(text: string): string {
return text
.replace(/&nbsp;/g, ' ')
.replace(/&amp;/g, '&')
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&#x27;/g, "'")
.replace(/&#(\d+);/g, (_, n) => String.fromCharCode(parseInt(n, 10)))
.replace(/&#x([0-9a-fA-F]+);/g, (_, n) => String.fromCharCode(parseInt(n, 16)));
}
function stripTags(html: string): string {
return html.replace(/<[^>]+>/g, '');
}
function normalizeWhitespace(text: string): string {
return text.replace(/[ \t]+/g, ' ').replace(/\n{3,}/g, '\n\n').trim();
}
export function htmlToMarkdown(html: string): string {
let md = html;
md = md.replace(/<br\s*\/?>/gi, '\n');
md = md.replace(/<hr\s*\/?>/gi, '\n\n---\n\n');
md = md.replace(/<h1[^>]*>([\s\S]*?)<\/h1>/gi, (_, c) => `\n\n# ${stripTags(c).trim()}\n\n`);
md = md.replace(/<h2[^>]*>([\s\S]*?)<\/h2>/gi, (_, c) => `\n\n## ${stripTags(c).trim()}\n\n`);
md = md.replace(/<h3[^>]*>([\s\S]*?)<\/h3>/gi, (_, c) => `\n\n### ${stripTags(c).trim()}\n\n`);
md = md.replace(/<h4[^>]*>([\s\S]*?)<\/h4>/gi, (_, c) => `\n\n#### ${stripTags(c).trim()}\n\n`);
md = md.replace(/<h5[^>]*>([\s\S]*?)<\/h5>/gi, (_, c) => `\n\n##### ${stripTags(c).trim()}\n\n`);
md = md.replace(/<h6[^>]*>([\s\S]*?)<\/h6>/gi, (_, c) => `\n\n###### ${stripTags(c).trim()}\n\n`);
md = md.replace(/<strong[^>]*>([\s\S]*?)<\/strong>/gi, (_, c) => `**${stripTags(c).trim()}**`);
md = md.replace(/<b[^>]*>([\s\S]*?)<\/b>/gi, (_, c) => `**${stripTags(c).trim()}**`);
md = md.replace(/<em[^>]*>([\s\S]*?)<\/em>/gi, (_, c) => `*${stripTags(c).trim()}*`);
md = md.replace(/<i[^>]*>([\s\S]*?)<\/i>/gi, (_, c) => `*${stripTags(c).trim()}*`);
md = md.replace(/<del[^>]*>([\s\S]*?)<\/del>/gi, (_, c) => `~~${stripTags(c).trim()}~~`);
md = md.replace(/<s[^>]*>([\s\S]*?)<\/s>/gi, (_, c) => `~~${stripTags(c).trim()}~~`);
md = md.replace(/<mark[^>]*>([\s\S]*?)<\/mark>/gi, (_, c) => `==${stripTags(c).trim()}==`);
md = md.replace(/<a[^>]*href=["']([^"']+)["'][^>]*>([\s\S]*?)<\/a>/gi, (_, href, text) => {
const t = stripTags(text).trim();
if (!t || href.startsWith('javascript:')) return t;
return `[${t}](${href})`;
});
md = md.replace(/<img[^>]*src=["']([^"']+)["'][^>]*alt=["']([^"']*)["'][^>]*\/?>/gi, (_, src, alt) => `![${alt}](${src})`);
md = md.replace(/<img[^>]*alt=["']([^"']*)["'][^>]*src=["']([^"']+)["'][^>]*\/?>/gi, (_, alt, src) => `![${alt}](${src})`);
md = md.replace(/<img[^>]*src=["']([^"']+)["'][^>]*\/?>/gi, (_, src) => `![](${src})`);
md = md.replace(/<pre[^>]*><code[^>]*>([\s\S]*?)<\/code><\/pre>/gi, (_, code) => `\n\n\`\`\`\n${decodeHtmlEntities(stripTags(code)).trim()}\n\`\`\`\n\n`);
md = md.replace(/<pre[^>]*>([\s\S]*?)<\/pre>/gi, (_, code) => `\n\n\`\`\`\n${decodeHtmlEntities(stripTags(code)).trim()}\n\`\`\`\n\n`);
md = md.replace(/<code[^>]*>([\s\S]*?)<\/code>/gi, (_, code) => `\`${decodeHtmlEntities(stripTags(code)).trim()}\``);
md = md.replace(/<blockquote[^>]*>([\s\S]*?)<\/blockquote>/gi, (_, content) => {
const lines = stripTags(content).trim().split('\n');
return '\n\n' + lines.map(l => `> ${l.trim()}`).join('\n') + '\n\n';
});
md = md.replace(/<ul[^>]*>([\s\S]*?)<\/ul>/gi, (_, items) => {
const lis = items.match(/<li[^>]*>([\s\S]*?)<\/li>/gi) || [];
const lines = lis.map(li => {
const content = li.replace(/<li[^>]*>([\s\S]*?)<\/li>/i, '$1');
return `- ${stripTags(content).trim()}`;
});
return '\n\n' + lines.join('\n') + '\n\n';
});
md = md.replace(/<ol[^>]*>([\s\S]*?)<\/ol>/gi, (_, items) => {
const lis = items.match(/<li[^>]*>([\s\S]*?)<\/li>/gi) || [];
const lines = lis.map((li, i) => {
const content = li.replace(/<li[^>]*>([\s\S]*?)<\/li>/i, '$1');
return `${i + 1}. ${stripTags(content).trim()}`;
});
return '\n\n' + lines.join('\n') + '\n\n';
});
md = md.replace(/<table[^>]*>([\s\S]*?)<\/table>/gi, (_, table) => {
const rows: string[][] = [];
const trMatches = table.match(/<tr[^>]*>([\s\S]*?)<\/tr>/gi) || [];
for (const tr of trMatches) {
const cells: string[] = [];
const cellMatches = tr.match(/<t[hd][^>]*>([\s\S]*?)<\/t[hd]>/gi) || [];
for (const cell of cellMatches) {
const content = cell.replace(/<t[hd][^>]*>([\s\S]*?)<\/t[hd]>/i, '$1');
cells.push(stripTags(content).trim().replace(/\|/g, '\\|'));
}
if (cells.length > 0) rows.push(cells);
}
if (rows.length === 0) return '';
const colCount = Math.max(...rows.map(r => r.length));
const normalizedRows = rows.map(r => {
while (r.length < colCount) r.push('');
return r;
});
const header = `| ${normalizedRows[0].join(' | ')} |`;
const sep = `| ${normalizedRows[0].map(() => '---').join(' | ')} |`;
const body = normalizedRows.slice(1).map(r => `| ${r.join(' | ')} |`).join('\n');
return '\n\n' + header + '\n' + sep + (body ? '\n' + body : '') + '\n\n';
});
md = md.replace(/<p[^>]*>([\s\S]*?)<\/p>/gi, (_, c) => `\n\n${stripTags(c).trim()}\n\n`);
md = md.replace(/<div[^>]*>([\s\S]*?)<\/div>/gi, (_, c) => `\n${stripTags(c).trim()}\n`);
md = md.replace(/<span[^>]*>([\s\S]*?)<\/span>/gi, (_, c) => stripTags(c));
md = stripTags(md);
md = decodeHtmlEntities(md);
md = normalizeWhitespace(md);
return md;
}
export function formatMetadataYaml(meta: PageMetadata): string {
const lines = ['---'];
lines.push(`url: ${meta.url}`);
lines.push(`title: "${meta.title.replace(/"/g, '\\"')}"`);
if (meta.description) lines.push(`description: "${meta.description.replace(/"/g, '\\"')}"`);
if (meta.author) lines.push(`author: "${meta.author.replace(/"/g, '\\"')}"`);
if (meta.published) lines.push(`published: "${meta.published}"`);
lines.push(`captured_at: "${meta.captured_at}"`);
lines.push('---');
return lines.join('\n');
}
export function createMarkdownDocument(result: ConversionResult): string {
const yaml = formatMetadataYaml(result.metadata);
const titleRegex = new RegExp(`^#\\s+${result.metadata.title.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\s*\\n`, 'i');
const hasTitle = titleRegex.test(result.markdown);
const title = result.metadata.title && !hasTitle ? `\n\n# ${result.metadata.title}\n\n` : '\n\n';
return yaml + title + result.markdown;
}
@@ -0,0 +1,176 @@
import { createInterface } from "node:readline";
import { writeFile, mkdir, access } from "node:fs/promises";
import path from "node:path";
import process from "node:process";
import { CdpConnection, getFreePort, launchChrome, waitForChromeDebugPort, waitForNetworkIdle, waitForPageLoad, autoScroll, evaluateScript, killChrome } from "./cdp.js";
import { cleanupAndExtractScript, htmlToMarkdown, createMarkdownDocument, type PageMetadata, type ConversionResult } from "./html-to-markdown.js";
import { resolveUrlToMarkdownDataDir } from "./paths.js";
import { DEFAULT_TIMEOUT_MS, CDP_CONNECT_TIMEOUT_MS, NETWORK_IDLE_TIMEOUT_MS, POST_LOAD_DELAY_MS, SCROLL_STEP_WAIT_MS, SCROLL_MAX_STEPS } from "./constants.js";
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function fileExists(filePath: string): Promise<boolean> {
try {
await access(filePath);
return true;
} catch {
return false;
}
}
interface Args {
url: string;
output?: string;
wait: boolean;
timeout: number;
}
function parseArgs(argv: string[]): Args {
const args: Args = { url: "", wait: false, timeout: DEFAULT_TIMEOUT_MS };
for (let i = 2; i < argv.length; i++) {
const arg = argv[i];
if (arg === "--wait" || arg === "-w") {
args.wait = true;
} else if (arg === "-o" || arg === "--output") {
args.output = argv[++i];
} else if (arg === "--timeout" || arg === "-t") {
args.timeout = parseInt(argv[++i], 10) || DEFAULT_TIMEOUT_MS;
} else if (!arg.startsWith("-") && !args.url) {
args.url = arg;
}
}
return args;
}
function generateSlug(title: string, url: string): string {
const text = title || new URL(url).pathname.replace(/\//g, "-");
return text
.toLowerCase()
.replace(/[^\w\s-]/g, "")
.replace(/\s+/g, "-")
.replace(/-+/g, "-")
.replace(/^-|-$/g, "")
.slice(0, 50) || "page";
}
function formatTimestamp(): string {
const now = new Date();
const pad = (n: number) => n.toString().padStart(2, "0");
return `${now.getFullYear()}${pad(now.getMonth() + 1)}${pad(now.getDate())}-${pad(now.getHours())}${pad(now.getMinutes())}${pad(now.getSeconds())}`;
}
async function generateOutputPath(url: string, title: string): Promise<string> {
const domain = new URL(url).hostname.replace(/^www\./, "");
const slug = generateSlug(title, url);
const dataDir = resolveUrlToMarkdownDataDir();
const basePath = path.join(dataDir, domain, `${slug}.md`);
if (!(await fileExists(basePath))) {
return basePath;
}
const timestampSlug = `${slug}-${formatTimestamp()}`;
return path.join(dataDir, domain, `${timestampSlug}.md`);
}
async function waitForUserSignal(): Promise<void> {
console.log("Page opened. Press Enter when ready to capture...");
const rl = createInterface({ input: process.stdin, output: process.stdout });
await new Promise<void>((resolve) => {
rl.once("line", () => { rl.close(); resolve(); });
});
}
async function captureUrl(args: Args): Promise<ConversionResult> {
const port = await getFreePort();
const chrome = await launchChrome(args.url, port, false);
let cdp: CdpConnection | null = null;
try {
const wsUrl = await waitForChromeDebugPort(port, 30_000);
cdp = await CdpConnection.connect(wsUrl, CDP_CONNECT_TIMEOUT_MS);
const targets = await cdp.send<{ targetInfos: Array<{ targetId: string; type: string; url: string }> }>("Target.getTargets");
const pageTarget = targets.targetInfos.find(t => t.type === "page" && t.url.startsWith("http"));
if (!pageTarget) throw new Error("No page target found");
const { sessionId } = await cdp.send<{ sessionId: string }>("Target.attachToTarget", { targetId: pageTarget.targetId, flatten: true });
await cdp.send("Network.enable", {}, { sessionId });
await cdp.send("Page.enable", {}, { sessionId });
if (args.wait) {
await waitForUserSignal();
} else {
console.log("Waiting for page to load...");
await Promise.race([
waitForPageLoad(cdp, sessionId, 15_000),
sleep(8_000)
]);
await waitForNetworkIdle(cdp, sessionId, NETWORK_IDLE_TIMEOUT_MS);
await sleep(POST_LOAD_DELAY_MS);
console.log("Scrolling to trigger lazy load...");
await autoScroll(cdp, sessionId, SCROLL_MAX_STEPS, SCROLL_STEP_WAIT_MS);
await sleep(POST_LOAD_DELAY_MS);
}
console.log("Capturing page content...");
const extracted = await evaluateScript<{ title: string; description?: string; author?: string; published?: string; html: string }>(
cdp, sessionId, cleanupAndExtractScript, args.timeout
);
const metadata: PageMetadata = {
url: args.url,
title: extracted.title || "",
description: extracted.description,
author: extracted.author,
published: extracted.published,
captured_at: new Date().toISOString()
};
const markdown = htmlToMarkdown(extracted.html);
return { metadata, markdown };
} finally {
if (cdp) {
try { await cdp.send("Browser.close", {}, { timeoutMs: 5_000 }); } catch {}
cdp.close();
}
killChrome(chrome);
}
}
async function main(): Promise<void> {
const args = parseArgs(process.argv);
if (!args.url) {
console.error("Usage: bun main.ts <url> [-o output.md] [--wait] [--timeout ms]");
process.exit(1);
}
try {
new URL(args.url);
} catch {
console.error(`Invalid URL: ${args.url}`);
process.exit(1);
}
console.log(`Fetching: ${args.url}`);
console.log(`Mode: ${args.wait ? "wait" : "auto"}`);
const result = await captureUrl(args);
const outputPath = args.output || await generateOutputPath(args.url, result.metadata.title);
const outputDir = path.dirname(outputPath);
await mkdir(outputDir, { recursive: true });
const document = createMarkdownDocument(result);
await writeFile(outputPath, document, "utf-8");
console.log(`Saved: ${outputPath}`);
console.log(`Title: ${result.metadata.title || "(no title)"}`);
}
main().catch((err) => {
console.error("Error:", err instanceof Error ? err.message : String(err));
process.exit(1);
});
@@ -0,0 +1,29 @@
import os from "node:os";
import path from "node:path";
import process from "node:process";
const APP_DATA_DIR = "baoyu-skills";
const URL_TO_MARKDOWN_DATA_DIR = "url-to-markdown";
const PROFILE_DIR_NAME = "chrome-profile";
export function resolveUserDataRoot(): string {
if (process.platform === "win32") {
return process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming");
}
if (process.platform === "darwin") {
return path.join(os.homedir(), "Library", "Application Support");
}
return process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share");
}
export function resolveUrlToMarkdownDataDir(): string {
const override = process.env.URL_DATA_DIR?.trim();
if (override) return path.resolve(override);
return path.join(process.cwd(), URL_TO_MARKDOWN_DATA_DIR);
}
export function resolveUrlToMarkdownChromeProfileDir(): string {
const override = process.env.URL_CHROME_PROFILE_DIR?.trim();
if (override) return path.resolve(override);
return path.join(resolveUserDataRoot(), APP_DATA_DIR, URL_TO_MARKDOWN_DATA_DIR, PROFILE_DIR_NAME);
}
+61 -10
View File
@@ -20,7 +20,7 @@ Break down complex content into eye-catching infographic series for Xiaohongshu
/baoyu-xhs-images posts/ai-future/article.md --layout dense
# Combine style and layout
/baoyu-xhs-images posts/ai-future/article.md --style tech --layout list
/baoyu-xhs-images posts/ai-future/article.md --style notion --layout list
# Direct content input
/baoyu-xhs-images
@@ -98,7 +98,7 @@ Each session creates an independent directory named by content slug:
xhs-images/{topic-slug}/
├── source-{slug}.{ext} # Source files (text, images, etc.)
├── analysis.md # Deep analysis results
├── outline-style-[slug].md # Variant A (e.g., outline-style-tech.md)
├── outline-style-[slug].md # Variant A (e.g., outline-style-notion.md)
├── outline-style-[slug].md # Variant B (e.g., outline-style-notion.md)
├── outline-style-[slug].md # Variant C (e.g., outline-style-minimal.md)
├── outline.md # Final selected
@@ -127,6 +127,31 @@ Copy all sources with naming `source-{slug}.{ext}`:
## Workflow
### Step 0: Check Preferences
**Check paths** (priority order):
1. `.baoyu-skills/baoyu-xhs-images/EXTEND.md` (project)
2. `~/.baoyu-skills/baoyu-xhs-images/EXTEND.md` (user)
**If preferences found**:
1. Parse YAML frontmatter
2. Display current preferences summary:
```
Loaded preferences from [path]:
- Watermark: [enabled/disabled] "[content]" at [position]
- Style: [name] - [description]
- Layout: [layout]
- Language: [lang]
```
3. Continue to Step 1
**If NO preferences found**:
1. Ask user with AskUserQuestion (see `references/first-time-setup.md`)
2. Create EXTEND.md with user choices
3. Continue to Step 1
Schema reference: `references/preferences-schema.md`
### Step 1: Analyze Content → `analysis.md`
Read source content, save it if needed, and perform deep analysis.
@@ -162,7 +187,7 @@ Based on analysis, create three distinct style variants.
| Variant | Selection Logic | Example Filename |
|---------|-----------------|------------------|
| A | Primary recommendation | `outline-style-tech.md` |
| A | Primary recommendation | `outline-style-notion.md` |
| B | Alternative style | `outline-style-notion.md` |
| C | Different audience/mood | `outline-style-minimal.md` |
@@ -172,6 +197,10 @@ Based on analysis, create three distinct style variants.
**IMPORTANT**: Present ALL options in a single confirmation step using AskUserQuestion. Do NOT interrupt workflow with multiple separate confirmations.
**Prioritize user preferences** (from Step 0):
- If user has `preferred_style`: Show as first option with "(Your preference)"
- If user has `preferred_layout`: Show as first option with "(Your preference)"
**Determine which questions to ask**:
| Question | When to Ask |
@@ -188,14 +217,15 @@ Based on analysis, create three distinct style variants.
```
Question 1 (Style): Which style variant?
- A: tech + dense (Recommended) - 专业科技感,适合干货
- B: notion + list - 清爽知识卡片
- C: minimal + balanced - 简约高端风格
- User preference: notion + dense (Your preference) - 您的默认设置
- A: notion + list - AI推荐: 清爽知识卡片
- B: minimal + balanced - AI推荐: 简约高端风格
- Custom: 自定义风格描述
Question 2 (Layout) - only if relevant:
- Your preference: dense (Your preference)
- Keep variant default (Recommended)
- sparse / balanced / dense / list / comparison / flow
- sparse / balanced / list / comparison / flow
Question 3 (Language) - only if mismatch:
- 中文 (匹配原文)
@@ -217,6 +247,15 @@ With confirmed outline + style + layout:
2. Generate image using confirmed style and layout
3. Report progress after each generation
**Watermark Application** (if enabled in preferences):
Add to each image generation prompt:
```
Include a subtle watermark "[content]" positioned at [position]
with approximately [opacity*100]% visibility. The watermark should
be legible but not distracting from the main content.
```
Reference: `references/watermark-guide.md`
**Image Generation Skill Selection**:
- Check available image generation skills
- If multiple skills available, ask user preference
@@ -239,10 +278,10 @@ Location: [directory path]
Images: N total
✓ analysis.md
✓ outline-style-tech.md
✓ outline-style-notion.md
✓ outline-style-chalkboard.md
✓ outline-style-minimal.md
✓ outline.md (selected: tech + dense)
✓ outline.md (selected: notion + dense)
Files:
- 01-cover-[slug].png ✓ Cover (sparse)
@@ -301,6 +340,9 @@ Detailed templates and guidelines in `references/` directory:
- `styles/<style>.md` - Detailed style definitions
- `layouts/<layout>.md` - Detailed layout definitions
- `base-prompt.md` - Base prompt template
- `preferences-schema.md` - EXTEND.md YAML schema
- `watermark-guide.md` - Watermark configuration guide
- `first-time-setup.md` - First-time setup flow
## Notes
@@ -318,4 +360,13 @@ Custom styles and configurations via EXTEND.md.
1. `.baoyu-skills/baoyu-xhs-images/EXTEND.md` (project)
2. `~/.baoyu-skills/baoyu-xhs-images/EXTEND.md` (user)
If found, load before Step 1. Extension content overrides defaults.
If found, load in Step 0. Extension content overrides defaults.
**Supported preferences**:
- Watermark settings (content, position, opacity)
- Preferred style with custom description
- Preferred layout
- Custom style definitions
**Schema**: `references/preferences-schema.md`
**First-time setup**: `references/first-time-setup.md`
@@ -25,9 +25,9 @@ Unlike other platforms, Xiaohongshu content must prioritize:
| Type | Characteristics | Best Style | Best Layout |
|------|----------------|------------|-------------|
| 种草/安利 | Product recommendation, benefits focus | cute/fresh | balanced/list |
| 干货分享 | Knowledge, tips, how-to | notion/tech | dense/list |
| 干货分享 | Knowledge, tips, how-to | notion | dense/list |
| 个人故事 | Personal experience, emotional | warm | balanced |
| 测评对比 | Review, comparison, pros/cons | tech/bold | comparison |
| 测评对比 | Review, comparison, pros/cons | bold/notion | comparison |
| 教程步骤 | Step-by-step guide | fresh/notion | flow/list |
| 避坑指南 | Warnings, mistakes to avoid | bold | list/comparison |
| 清单合集 | Collections, recommendations | cute/minimal | list/dense |
@@ -55,10 +55,10 @@ Evaluate title/hook potential using these patterns:
| Audience | Interests | Preferred Style | Content Focus |
|----------|-----------|-----------------|---------------|
| 学生党 | 省钱、学习、校园 | cute/fresh | 平价、教程、学习方法 |
| 打工人 | 效率、职场、减压 | minimal/tech | 工具、技巧、摸鱼 |
| 打工人 | 效率、职场、减压 | minimal/notion | 工具、技巧、摸鱼 |
| 宝妈 | 育儿、家居、省心 | warm/fresh | 实用、安全、经验 |
| 精致女孩 | 美妆、穿搭、仪式感 | cute/retro | 好看、氛围、品质 |
| 技术宅 | 工具、效率、极客 | tech/notion | 深度、专业、新奇 |
| 技术宅 | 工具、效率、极客 | notion/chalkboard | 深度、专业、新奇 |
| 美食爱好者 | 探店、食谱、测评 | warm/pop | 好吃、简单、颜值 |
| 旅行达人 | 攻略、打卡、小众 | fresh/retro | 省钱、避坑、拍照 |
@@ -165,7 +165,7 @@ recommended_image_count: 6
## Content Signals
- "AI工具" tech + dense
- "AI工具" notion + dense
- "效率" → notion + list
- "干货" → minimal + dense
@@ -180,7 +180,7 @@ recommended_image_count: 6
## Recommended Approaches
1. **Tech + Dense** - 专业科技感,适合干货分享 (recommended)
1. **Notion + Dense** - 知识卡片风格,适合干货分享 (recommended)
2. **Notion + List** - 清爽知识卡片风格
3. **Minimal + Balanced** - 简约高端,适合职场人群
```
@@ -0,0 +1,140 @@
---
name: first-time-setup
description: First-time setup flow for baoyu-xhs-images preferences
---
# First-Time Setup
## Overview
When no EXTEND.md is found, guide user through preference setup.
## Setup Flow
```
No EXTEND.md found
┌─────────────────────┐
│ AskUserQuestion │
│ (all questions) │
└─────────────────────┘
┌─────────────────────┐
│ Create EXTEND.md │
└─────────────────────┘
Continue to Step 1
```
## Questions
Use single AskUserQuestion with multiple questions (AskUserQuestion auto-adds "Other" option):
### Question 1: Watermark
```
header: "Watermark"
question: "Enable watermark on generated images?"
options:
- label: "Enabled"
description: "Add watermark to all images"
- label: "Disabled"
description: "No watermark (can enable later)"
```
### Question 2: Watermark Content (if enabled)
```
header: "Content"
question: "What text for your watermark?"
options:
- label: "@username"
description: "Your XHS handle (replace 'username')"
- label: "Custom text"
description: "Enter your own text"
```
### Question 3: Watermark Position (if enabled)
```
header: "Position"
question: "Where to place watermark?"
options:
- label: "bottom-right"
description: "Lower right corner (most common)"
- label: "bottom-left"
description: "Lower left corner"
- label: "bottom-center"
description: "Bottom center"
- label: "top-right"
description: "Upper right corner"
```
### Question 4: Preferred Style
```
header: "Style"
question: "Default visual style preference?"
options:
- label: "cute"
description: "Sweet, adorable - classic XHS aesthetic"
- label: "notion"
description: "Minimalist hand-drawn, intellectual"
- label: "None"
description: "Auto-select based on content"
```
### Question 5: Save Location
```
header: "Save"
question: "Where to save preferences?"
options:
- label: "Project"
description: ".baoyu-skills/ (this project only)"
- label: "User"
description: "~/.baoyu-skills/ (all projects)"
```
## Save Locations
| Choice | Path | Scope |
|--------|------|-------|
| Project | `.baoyu-skills/baoyu-xhs-images/EXTEND.md` | Current project |
| User | `~/.baoyu-skills/baoyu-xhs-images/EXTEND.md` | All projects |
## After Setup
1. Create directory if needed
2. Write EXTEND.md with frontmatter
3. Confirm: "Preferences saved to [path]"
4. Continue to Step 1
## EXTEND.md Template
```yaml
---
version: 1
watermark:
enabled: [true/false]
content: "[user input]"
position: [selected position]
opacity: 0.7
preferred_style:
name: [selected style or null]
description: ""
preferred_layout: null
language: null
custom_styles: []
---
```
## Modifying Preferences Later
Users can edit EXTEND.md directly or run setup again:
- Delete EXTEND.md to trigger setup
- Edit YAML frontmatter for quick changes
- Full schema: `references/preferences-schema.md`
@@ -28,4 +28,4 @@ Comparisons, transformations, decision helpers, 对比图
## Best Style Pairings
bold (dramatic contrast), tech (data comparison), warm (before/after stories)
bold (dramatic contrast), notion (data comparison), warm (before/after stories)
@@ -28,4 +28,4 @@ Summary cards, cheat sheets, comprehensive guides, 干货总结
## Best Style Pairings
tech, notion, minimal (clean styles prevent visual overload)
notion, minimal, chalkboard (clean styles prevent visual overload)
@@ -27,4 +27,4 @@ Processes, timelines, cause-effect chains, workflows
## Best Style Pairings
tech (process diagrams), notion (simple flows), fresh (organic flows)
notion (process diagrams), chalkboard (educational flows), fresh (organic flows)
@@ -5,8 +5,8 @@ Template for generating infographic series outlines.
## File Naming
Outline files use style slug in the name:
- `outline-style-tech.md` - Tech style variant
- `outline-style-notion.md` - Notion style variant
- `outline-style-chalkboard.md` - Chalkboard style variant
- `outline-style-minimal.md` - Minimal style variant
- `outline.md` - Final selected (copied from chosen variant)
@@ -43,7 +43,7 @@ NN-{type}-[slug].md (in prompts/)
# Xiaohongshu Infographic Series Outline
---
style: tech
style: notion
default_layout: dense
image_count: 6
generated: YYYY-MM-DD HH:mm
@@ -223,6 +223,6 @@ Three variants should differ meaningfully:
| Audience | Primary target | Secondary target | Broader appeal |
**Example for "AI工具推荐"**:
- `outline-style-tech.md`: Tech + Dense - 专业极客
- `outline-style-notion.md`: Notion + Dense - 知识卡片
- `outline-style-notion.md`: Notion + List - 清爽知识卡片
- `outline-style-cute.md`: Cute + Balanced - 可爱易读风
@@ -0,0 +1,121 @@
---
name: preferences-schema
description: EXTEND.md YAML schema for baoyu-xhs-images user preferences
---
# Preferences Schema
## Full Schema
```yaml
---
version: 1
watermark:
enabled: false
content: ""
position: bottom-right # bottom-right|bottom-left|bottom-center|top-right
opacity: 0.7 # 0.1-1.0
preferred_style:
name: null # Built-in or custom style name
description: "" # Override/notes
preferred_layout: null # sparse|balanced|dense|list|comparison|flow
language: null # zh|en|ja|ko|auto
custom_styles:
- name: my-style
description: "Style description"
color_palette:
primary: ["#FED7E2", "#FEEBC8"]
background: "#FFFAF0"
accents: ["#FF69B4", "#FF6B6B"]
visual_elements: "Hearts, stars, sparkles"
typography: "Rounded, bubbly hand lettering"
best_for: "Lifestyle, beauty"
---
```
## Field Reference
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `version` | int | 1 | Schema version |
| `watermark.enabled` | bool | false | Enable watermark |
| `watermark.content` | string | "" | Watermark text (@username or custom) |
| `watermark.position` | enum | bottom-right | Position on image |
| `watermark.opacity` | float | 0.7 | Transparency (0.1-1.0) |
| `preferred_style.name` | string | null | Style name or null |
| `preferred_style.description` | string | "" | Custom notes/override |
| `preferred_layout` | string | null | Layout preference or null |
| `language` | string | null | Output language (null = auto-detect) |
| `custom_styles` | array | [] | User-defined styles |
## Position Options
| Value | Description |
|-------|-------------|
| `bottom-right` | Lower right corner (default, most common) |
| `bottom-left` | Lower left corner |
| `bottom-center` | Bottom center |
| `top-right` | Upper right corner |
## Custom Style Fields
| Field | Required | Description |
|-------|----------|-------------|
| `name` | Yes | Unique style identifier (kebab-case) |
| `description` | Yes | What the style conveys |
| `color_palette.primary` | No | Main colors (array) |
| `color_palette.background` | No | Background color |
| `color_palette.accents` | No | Accent colors (array) |
| `visual_elements` | No | Decorative elements |
| `typography` | No | Font/lettering style |
| `best_for` | No | Recommended content types |
## Example: Minimal Preferences
```yaml
---
version: 1
watermark:
enabled: true
content: "@myusername"
preferred_style:
name: notion
---
```
## Example: Full Preferences
```yaml
---
version: 1
watermark:
enabled: true
content: "@myxhsaccount"
position: bottom-right
opacity: 0.5
preferred_style:
name: notion
description: "Clean knowledge cards for tech content"
preferred_layout: dense
language: zh
custom_styles:
- name: corporate
description: "Professional B2B style"
color_palette:
primary: ["#1E3A5F", "#4A90D9"]
background: "#F5F7FA"
accents: ["#00B4D8", "#48CAE4"]
visual_elements: "Clean lines, subtle gradients, geometric shapes"
typography: "Modern sans-serif, professional"
best_for: "Business, SaaS, enterprise"
---
```
@@ -0,0 +1,73 @@
---
name: watermark-guide
description: Watermark configuration guide for baoyu-xhs-images
---
# Watermark Guide
## Position Diagram
```
┌─────────────────────────────┐
│ [top-right]│
│ │
│ │
│ IMAGE CONTENT │
│ │
│ │
│[bottom-left][bottom-center][bottom-right]│
└─────────────────────────────┘
```
## Position Recommendations
| Position | Best For | Avoid When |
|----------|----------|------------|
| `bottom-right` | Default choice, most common | Key info in bottom-right |
| `bottom-left` | Right-heavy layouts | Key info in bottom-left |
| `bottom-center` | Centered designs | Text-heavy bottom area |
| `top-right` | Bottom-heavy content | Title/header in top-right |
## Opacity Examples
| Opacity | Visual Effect | Use Case |
|---------|---------------|----------|
| 0.3 | Very subtle, barely visible | Clean aesthetic priority |
| 0.5 | Balanced, noticeable but not distracting | Default recommendation |
| 0.7 | Clearly visible | Brand recognition priority |
| 0.9 | Strong presence | Anti-copy protection |
## Content Format
| Format | Example | Style |
|--------|---------|-------|
| Handle | `@username` | Most common for XHS |
| Text | `MyBrand` | Simple branding |
| Chinese | `小红书:用户名` | Platform specific |
| URL | `myblog.com` | Cross-platform |
## Best Practices
1. **Consistency**: Use same watermark across all images in series
2. **Legibility**: Ensure watermark readable on both light/dark areas
3. **Size**: Keep subtle - should not distract from content
4. **Contrast**: Opacity may need adjustment per image background
## Prompt Integration
When watermark is enabled, add to image generation prompt:
```
Include a subtle watermark "[content]" positioned at [position]
with approximately [opacity*100]% visibility. The watermark should
be legible but not distracting from the main content.
```
## Common Issues
| Issue | Solution |
|-------|----------|
| Watermark invisible | Increase opacity or adjust position |
| Watermark too prominent | Decrease opacity (0.3-0.5) |
| Watermark overlaps content | Change position |
| Inconsistent across images | Use session ID for consistency |