Compare commits

...

10 Commits

Author SHA1 Message Date
Jim Liu 宝玉 7a0ffd9533 chore: release v1.84.0 2026-03-25 16:29:22 -05:00
Jim Liu 宝玉 69355b4ee1 feat(baoyu-imagine): rename baoyu-image-gen to baoyu-imagine 2026-03-25 16:28:06 -05:00
Jim Liu 宝玉 23b7487321 chore: release v1.83.0 2026-03-25 15:40:22 -05:00
Jim Liu 宝玉 ad8781c1c5 feat(baoyu-image-gen): add MiniMax provider with subject reference and custom sizes 2026-03-25 15:39:40 -05:00
Jim Liu 宝玉 86a3d6521b chore: release v1.82.0 2026-03-24 22:40:23 -05:00
Jim Liu 宝玉 e99ce744cd feat(baoyu-url-to-markdown): add browser fallback strategy, content cleaner, and data URI support
- Browser strategy: headless first with automatic retry in visible Chrome on failure
- New --browser auto|headless|headed flag with --headless/--headed shortcuts
- Content cleaner module for HTML preprocessing (remove ads, base64 images, scripts)
- Media localizer now handles base64 data URIs alongside remote URLs
- Capture finalUrl from browser to track redirects for output path
- Agent quality gate documentation for post-capture validation
- Upgrade defuddle ^0.12.0 → ^0.14.0
- Add unit tests for content-cleaner, html-to-markdown, legacy-converter, media-localizer
2026-03-24 22:39:17 -05:00
Jim Liu 宝玉 40f9f05c22 chore: release v1.81.0 2026-03-24 20:59:56 -05:00
Jim Liu 宝玉 09ce80357f feat(baoyu-youtube-transcript): add yt-dlp fallback and modularize codebase
Retry with alternate InnerTube client identities when YouTube returns
anti-bot responses, then fall back to yt-dlp when available. Split
monolithic main.ts into typed modules (youtube, transcript, storage,
shared, types) and add unit tests.
2026-03-24 20:59:04 -05:00
Jim Liu 宝玉 7c995fcc24 chore: release v1.80.1 2026-03-24 20:06:02 -05:00
Jim Liu 宝玉 151f1ec2a8 fix(baoyu-image-gen): use correct prompt field name for Jimeng API 2026-03-24 20:04:21 -05:00
56 changed files with 3189 additions and 869 deletions
+2 -2
View File
@@ -6,7 +6,7 @@
},
"metadata": {
"description": "Skills shared by Baoyu for improving daily work efficiency",
"version": "1.80.0"
"version": "1.84.0"
},
"plugins": [
{
@@ -22,7 +22,7 @@
"./skills/baoyu-danger-gemini-web",
"./skills/baoyu-danger-x-to-markdown",
"./skills/baoyu-format-markdown",
"./skills/baoyu-image-gen",
"./skills/baoyu-imagine",
"./skills/baoyu-infographic",
"./skills/baoyu-markdown-to-html",
"./skills/baoyu-post-to-weibo",
+38
View File
@@ -2,6 +2,44 @@
English | [中文](./CHANGELOG.zh.md)
## 1.84.0 - 2026-03-25
### Features
- Rename `baoyu-image-gen` skill to `baoyu-imagine` — shorter command name, all references updated across docs, configs, and dependent skills
## 1.83.0 - 2026-03-25
### Features
- `baoyu-image-gen`: add MiniMax provider (`image-01` / `image-01-live`) with subject_reference for character/portrait consistency, custom sizes, and aspect ratio support
## 1.82.0 - 2026-03-24
### Features
- `baoyu-url-to-markdown`: add browser fallback strategy — headless first, automatic retry in visible Chrome on technical failure; new `--browser auto|headless|headed` flag with `--headless`/`--headed` shortcuts
- `baoyu-url-to-markdown`: add content cleaner module for HTML preprocessing before extraction (remove ads, base64 images, scripts, styles)
- `baoyu-url-to-markdown`: support base64 data URI images in media localizer alongside remote URLs
- `baoyu-url-to-markdown`: capture final URL from browser to track redirects for output path generation
- `baoyu-url-to-markdown`: add agent quality gate documentation for post-capture content validation
### Dependencies
- `baoyu-url-to-markdown`: upgrade defuddle ^0.12.0 → ^0.14.0
### Tests
- `baoyu-url-to-markdown`: add unit tests for content-cleaner, html-to-markdown, legacy-converter, media-localizer
## 1.81.0 - 2026-03-24
### Features
- `baoyu-youtube-transcript`: add yt-dlp fallback when YouTube blocks direct InnerTube API, with alternate client identity retry and cookie support via `YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER` env var
### Refactor
- `baoyu-youtube-transcript`: split monolithic script into typed modules (youtube, transcript, storage, shared, types) and add unit tests
## 1.80.1 - 2026-03-24
### Fixes
- `baoyu-image-gen`: use correct `prompt` field name for Jimeng API request
## 1.80.0 - 2026-03-24
### Features
+38
View File
@@ -2,6 +2,44 @@
[English](./CHANGELOG.md) | 中文
## 1.84.0 - 2026-03-25
### 新功能
-`baoyu-image-gen` 技能重命名为 `baoyu-imagine` — 更简短的命令名,所有文档、配置和依赖技能中的引用已同步更新
## 1.83.0 - 2026-03-25
### 新功能
- `baoyu-image-gen`:新增 MiniMax 服务商(`image-01` / `image-01-live`),支持 subject_reference 角色/肖像一致性、自定义尺寸和宽高比
## 1.82.0 - 2026-03-24
### 新功能
- `baoyu-url-to-markdown`:新增浏览器回退策略 — 默认无头模式优先,技术故障时自动重试有头 Chrome;新增 `--browser auto|headless|headed` 参数及 `--headless`/`--headed` 快捷方式
- `baoyu-url-to-markdown`:新增内容清理模块,提取前预处理 HTML(移除广告、base64 图片、脚本、样式)
- `baoyu-url-to-markdown`:媒体本地化支持 base64 data URI 图片
- `baoyu-url-to-markdown`:从浏览器捕获最终 URL 以跟踪重定向,用于输出路径生成
- `baoyu-url-to-markdown`:新增 Agent 质量门控文档,规范捕获后的内容验证流程
### 依赖
- `baoyu-url-to-markdown`:升级 defuddle ^0.12.0 → ^0.14.0
### 测试
- `baoyu-url-to-markdown`:新增 content-cleaner、html-to-markdown、legacy-converter、media-localizer 单元测试
## 1.81.0 - 2026-03-24
### 新功能
- `baoyu-youtube-transcript`YouTube 封锁直连 InnerTube API 时自动回退到 yt-dlp,支持备用客户端身份重试及通过 `YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER` 环境变量传递浏览器 Cookie
### 重构
- `baoyu-youtube-transcript`:将单体脚本拆分为类型化模块(youtube、transcript、storage、shared、types)并添加单元测试
## 1.80.1 - 2026-03-24
### 修复
- `baoyu-image-gen`:修正即梦 API 请求中的 `prompt` 字段名
## 1.80.0 - 2026-03-24
### 新功能
+3 -3
View File
@@ -1,6 +1,6 @@
# CLAUDE.md
Claude Code marketplace plugin providing AI-powered content generation skills. Version: **1.80.0**.
Claude Code marketplace plugin providing AI-powered content generation skills. Version: **1.84.0**.
## Architecture
@@ -31,7 +31,7 @@ Execute: `${BUN_X} skills/<skill>/scripts/main.ts [options]`
- **Bun**: TypeScript runtime (`bun` preferred, fallback `npx -y bun`)
- **Chrome**: Required for CDP-based skills (gemini-web, post-to-x/wechat/weibo, url-to-markdown). All CDP skills share a single profile, override via `BAOYU_CHROME_PROFILE_DIR` env var. Platform paths: [docs/chrome-profile.md](docs/chrome-profile.md)
- **Image generation APIs**: `baoyu-image-gen` requires API key (OpenAI, Azure OpenAI, Google, OpenRouter, DashScope, or Replicate) configured in EXTEND.md
- **Image generation APIs**: `baoyu-imagine` requires API key (OpenAI, Azure OpenAI, Google, OpenRouter, DashScope, or Replicate) configured in EXTEND.md
- **Gemini Web auth**: Browser cookies (first run opens Chrome for login, `--login` to refresh)
## Security
@@ -46,7 +46,7 @@ Execute: `${BUN_X} skills/<skill>/scripts/main.ts [options]`
| Rule | Description |
|------|-------------|
| **Load project skills first** | Project skills override system/user-level skills with same name |
| **Default image generation** | Use `skills/baoyu-image-gen/SKILL.md` unless user specifies otherwise |
| **Default image generation** | Use `skills/baoyu-imagine/SKILL.md` unless user specifies otherwise |
Priority: project `skills/``$HOME/.baoyu-skills/` → system-level.
+80 -22
View File
@@ -32,7 +32,7 @@ This repository now supports publishing each `skills/baoyu-*` directory as an in
ClawHub installs skills individually, not as one marketplace bundle. After publishing, users can install specific skills such as:
```bash
clawhub install baoyu-image-gen
clawhub install baoyu-imagine
clawhub install baoyu-markdown-to-html
```
@@ -661,43 +661,58 @@ Post content to Weibo (微博). Supports regular posts with text, images, and vi
AI-powered generation backends.
#### baoyu-image-gen
#### baoyu-imagine
AI SDK-based image generation using OpenAI, Azure OpenAI, Google, OpenRouter, DashScope (Aliyun Tongyi Wanxiang), Jimeng (即梦), Seedream (豆包), and Replicate APIs. Supports text-to-image, reference images, aspect ratios, and quality presets.
AI SDK-based image generation using OpenAI, Azure OpenAI, Google, OpenRouter, DashScope (Aliyun Tongyi Wanxiang), MiniMax, Jimeng (即梦), Seedream (豆包), and Replicate APIs. Supports text-to-image, reference images, aspect ratios, custom sizes, batch generation, and quality presets.
```bash
# Basic generation (auto-detect provider)
/baoyu-image-gen --prompt "A cute cat" --image cat.png
/baoyu-imagine --prompt "A cute cat" --image cat.png
# With aspect ratio
/baoyu-image-gen --prompt "A landscape" --image landscape.png --ar 16:9
/baoyu-imagine --prompt "A landscape" --image landscape.png --ar 16:9
# High quality (2k)
/baoyu-image-gen --prompt "A banner" --image banner.png --quality 2k
/baoyu-imagine --prompt "A banner" --image banner.png --quality 2k
# Specific provider
/baoyu-image-gen --prompt "A cat" --image cat.png --provider openai
/baoyu-imagine --prompt "A cat" --image cat.png --provider openai
# Azure OpenAI (model = deployment name)
/baoyu-image-gen --prompt "A cat" --image cat.png --provider azure --model gpt-image-1.5
/baoyu-imagine --prompt "A cat" --image cat.png --provider azure --model gpt-image-1.5
# OpenRouter
/baoyu-image-gen --prompt "A cat" --image cat.png --provider openrouter
/baoyu-imagine --prompt "A cat" --image cat.png --provider openrouter
# OpenRouter with reference images
/baoyu-imagine --prompt "Make it blue" --image out.png --provider openrouter --model google/gemini-3.1-flash-image-preview --ref source.png
# DashScope (Aliyun Tongyi Wanxiang)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider dashscope
/baoyu-imagine --prompt "一只可爱的猫" --image cat.png --provider dashscope
# DashScope with custom size
/baoyu-imagine --prompt "为咖啡品牌设计一张 21:9 横幅海报,包含清晰中文标题" --image banner.png --provider dashscope --model qwen-image-2.0-pro --size 2048x872
# MiniMax
/baoyu-imagine --prompt "A fashion editorial portrait by a bright studio window" --image out.jpg --provider minimax
# MiniMax with subject reference
/baoyu-imagine --prompt "A girl stands by the library window, cinematic lighting" --image out.jpg --provider minimax --model image-01 --ref portrait.png --ar 16:9
# Replicate
/baoyu-image-gen --prompt "A cat" --image cat.png --provider replicate
/baoyu-imagine --prompt "A cat" --image cat.png --provider replicate
# Jimeng (即梦)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider jimeng
/baoyu-imagine --prompt "一只可爱的猫" --image cat.png --provider jimeng
# Seedream (豆包)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider seedream
/baoyu-imagine --prompt "一只可爱的猫" --image cat.png --provider seedream
# With reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate, or Seedream 5.0/4.5/4.0)
/baoyu-image-gen --prompt "Make it blue" --image out.png --ref source.png
# With reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate, MiniMax, or Seedream 5.0/4.5/4.0)
/baoyu-imagine --prompt "Make it blue" --image out.png --ref source.png
# Batch mode
/baoyu-imagine --batchfile batch.json --jobs 4 --json
```
**Options**:
@@ -706,44 +721,73 @@ AI SDK-based image generation using OpenAI, Azure OpenAI, Google, OpenRouter, Da
| `--prompt`, `-p` | Prompt text |
| `--promptfiles` | Read prompt from files (concatenated) |
| `--image` | Output image path (required) |
| `--provider` | `google`, `openai`, `openrouter`, `dashscope`, `jimeng`, `seedream` or `replicate` (default: auto-detect; prefers google) |
| `--model`, `-m` | Model ID |
| `--batchfile` | JSON batch file for multi-image generation |
| `--jobs` | Worker count for batch mode |
| `--provider` | `google`, `openai`, `azure`, `openrouter`, `dashscope`, `minimax`, `jimeng`, `seedream`, or `replicate` |
| `--model`, `-m` | Model ID or deployment name. Azure uses deployment name; OpenRouter uses full model IDs; MiniMax uses `image-01` / `image-01-live` |
| `--ar` | Aspect ratio (e.g., `16:9`, `1:1`, `4:3`) |
| `--size` | Size (e.g., `1024x1024`) |
| `--quality` | `normal` or `2k` (default: `2k`) |
| `--ref` | Reference images (Google, OpenAI, OpenRouter, Replicate, or Seedream 5.0/4.5/4.0) |
| `--imageSize` | `1K`, `2K`, or `4K` for Google/OpenRouter |
| `--ref` | Reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate, MiniMax, or Seedream 5.0/4.5/4.0) |
| `--n` | Number of images per request |
| `--json` | JSON output |
**Environment Variables** (see [Environment Configuration](#environment-configuration) for setup):
| Variable | Description | Default |
|----------|-------------|---------|
| `OPENAI_API_KEY` | OpenAI API key | - |
| `AZURE_OPENAI_API_KEY` | Azure OpenAI API key | - |
| `OPENROUTER_API_KEY` | OpenRouter API key | - |
| `GOOGLE_API_KEY` | Google API key | - |
| `GEMINI_API_KEY` | Alias for `GOOGLE_API_KEY` | - |
| `DASHSCOPE_API_KEY` | DashScope API key (Aliyun) | - |
| `MINIMAX_API_KEY` | MiniMax API key | - |
| `REPLICATE_API_TOKEN` | Replicate API token | - |
| `JIMENG_ACCESS_KEY_ID` | Jimeng Volcengine access key | - |
| `JIMENG_SECRET_ACCESS_KEY` | Jimeng Volcengine secret key | - |
| `ARK_API_KEY` | Seedream Volcengine ARK API key | - |
| `OPENAI_IMAGE_MODEL` | OpenAI model | `gpt-image-1.5` |
| `AZURE_OPENAI_DEPLOYMENT` | Azure default deployment name | - |
| `AZURE_OPENAI_IMAGE_MODEL` | Backward-compatible Azure deployment/model alias | `gpt-image-1.5` |
| `OPENROUTER_IMAGE_MODEL` | OpenRouter model | `google/gemini-3.1-flash-image-preview` |
| `GOOGLE_IMAGE_MODEL` | Google model | `gemini-3-pro-image-preview` |
| `DASHSCOPE_IMAGE_MODEL` | DashScope model | `qwen-image-2.0-pro` |
| `MINIMAX_IMAGE_MODEL` | MiniMax model | `image-01` |
| `REPLICATE_IMAGE_MODEL` | Replicate model | `google/nano-banana-pro` |
| `JIMENG_IMAGE_MODEL` | Jimeng model | `jimeng_t2i_v40` |
| `SEEDREAM_IMAGE_MODEL` | Seedream model | `doubao-seedream-5-0-260128` |
| `OPENAI_BASE_URL` | Custom OpenAI endpoint | - |
| `OPENAI_IMAGE_USE_CHAT` | Use `/chat/completions` for OpenAI image generation | `false` |
| `AZURE_OPENAI_BASE_URL` | Azure resource or deployment endpoint | - |
| `AZURE_API_VERSION` | Azure image API version | `2025-04-01-preview` |
| `OPENROUTER_BASE_URL` | Custom OpenRouter endpoint | `https://openrouter.ai/api/v1` |
| `OPENROUTER_HTTP_REFERER` | Optional app/site URL for OpenRouter attribution | - |
| `OPENROUTER_TITLE` | Optional app name for OpenRouter attribution | - |
| `GOOGLE_BASE_URL` | Custom Google endpoint | - |
| `DASHSCOPE_BASE_URL` | Custom DashScope endpoint | - |
| `MINIMAX_BASE_URL` | Custom MiniMax endpoint | `https://api.minimax.io` |
| `REPLICATE_BASE_URL` | Custom Replicate endpoint | - |
| `JIMENG_BASE_URL` | Custom Jimeng endpoint | `https://visual.volcengineapi.com` |
| `JIMENG_REGION` | Jimeng region | `cn-north-1` |
| `SEEDREAM_BASE_URL` | Custom Seedream endpoint | `https://ark.cn-beijing.volces.com/api/v3` |
| `BAOYU_IMAGE_GEN_MAX_WORKERS` | Override batch worker cap | `10` |
| `BAOYU_IMAGE_GEN_<PROVIDER>_CONCURRENCY` | Override provider concurrency | provider-specific |
| `BAOYU_IMAGE_GEN_<PROVIDER>_START_INTERVAL_MS` | Override provider request start gap | provider-specific |
**Provider Notes**:
- Azure OpenAI: `--model` means Azure deployment name, not the underlying model family.
- DashScope: `qwen-image-2.0-pro` is the recommended default for custom `--size`, `21:9`, and strong Chinese/English text rendering.
- MiniMax: `image-01` supports documented custom `width` / `height`; `image-01-live` is lower latency and works best with `--ar`.
- MiniMax reference images are sent as `subject_reference`; the current API is specialized toward character / portrait consistency.
- Jimeng does not support reference images.
- Seedream reference images are supported by Seedream 5.0 / 4.5 / 4.0, not Seedream 3.0.
**Provider Auto-Selection**:
1. If `--provider` specified → use it
2. If only one API key available → use that provider
3. If multiple available → default to Google
1. If `--provider` is specified → use it
2. If `--ref` is provided and no provider is specified → try Google, then OpenAI, Azure, OpenRouter, Replicate, Seedream, and finally MiniMax
3. If only one API key is available → use that provider
4. If multiple providers are available → default to Google
#### baoyu-danger-gemini-web
@@ -1001,7 +1045,7 @@ Custom style descriptions are also accepted, e.g., `--style "poetic and lyrical"
Some skills require API keys or custom configuration. Environment variables can be set in `.env` files:
**Load Priority** (higher priority overrides lower):
1. CLI environment variables (e.g., `OPENAI_API_KEY=xxx /baoyu-image-gen ...`)
1. CLI environment variables (e.g., `OPENAI_API_KEY=xxx /baoyu-imagine ...`)
2. `process.env` (system environment)
3. `<cwd>/.baoyu-skills/.env` (project-level)
4. `~/.baoyu-skills/.env` (user-level)
@@ -1018,11 +1062,20 @@ cat > ~/.baoyu-skills/.env << 'EOF'
OPENAI_API_KEY=sk-xxx
OPENAI_IMAGE_MODEL=gpt-image-1.5
# OPENAI_BASE_URL=https://api.openai.com/v1
# OPENAI_IMAGE_USE_CHAT=false
# Azure OpenAI
AZURE_OPENAI_API_KEY=xxx
AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
AZURE_OPENAI_DEPLOYMENT=gpt-image-1.5
# AZURE_API_VERSION=2025-04-01-preview
# OpenRouter
OPENROUTER_API_KEY=sk-or-xxx
OPENROUTER_IMAGE_MODEL=google/gemini-3.1-flash-image-preview
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# OPENROUTER_HTTP_REFERER=https://your-app.example.com
# OPENROUTER_TITLE=Your App Name
# Google
GOOGLE_API_KEY=xxx
@@ -1034,6 +1087,11 @@ DASHSCOPE_API_KEY=sk-xxx
DASHSCOPE_IMAGE_MODEL=qwen-image-2.0-pro
# DASHSCOPE_BASE_URL=https://dashscope.aliyuncs.com/api/v1
# MiniMax
MINIMAX_API_KEY=xxx
MINIMAX_IMAGE_MODEL=image-01
# MINIMAX_BASE_URL=https://api.minimax.io
# Replicate
REPLICATE_API_TOKEN=r8_xxx
REPLICATE_IMAGE_MODEL=google/nano-banana-pro
+79 -21
View File
@@ -32,7 +32,7 @@ npx skills add jimliu/baoyu-skills
ClawHub 按“单个 skill”安装,不是把整个 marketplace 一次性装进去。发布后,用户可以按需安装:
```bash
clawhub install baoyu-image-gen
clawhub install baoyu-imagine
clawhub install baoyu-markdown-to-html
```
@@ -661,43 +661,58 @@ accounts:
AI 驱动的生成后端。
#### baoyu-image-gen
#### baoyu-imagine
基于 AI SDK 的图像生成,支持 OpenAI、Azure OpenAI、Google、OpenRouter、DashScope(阿里通义万相)、即梦(Jimeng)、豆包(Seedream)和 Replicate API。支持文生图、参考图、宽高比和质量预设。
基于 AI SDK 的图像生成,支持 OpenAI、Azure OpenAI、Google、OpenRouter、DashScope(阿里通义万相)、MiniMax、即梦(Jimeng)、豆包(Seedream)和 Replicate API。支持文生图、参考图、宽高比、自定义尺寸、批量生成和质量预设。
```bash
# 基础生成(自动检测服务商)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png
/baoyu-imagine --prompt "一只可爱的猫" --image cat.png
# 指定宽高比
/baoyu-image-gen --prompt "风景图" --image landscape.png --ar 16:9
/baoyu-imagine --prompt "风景图" --image landscape.png --ar 16:9
# 高质量(2k 分辨率)
/baoyu-image-gen --prompt "横幅图" --image banner.png --quality 2k
/baoyu-imagine --prompt "横幅图" --image banner.png --quality 2k
# 指定服务商
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider openai
/baoyu-imagine --prompt "一只猫" --image cat.png --provider openai
# Azure OpenAImodel 为部署名称)
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider azure --model gpt-image-1.5
/baoyu-imagine --prompt "一只猫" --image cat.png --provider azure --model gpt-image-1.5
# OpenRouter
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider openrouter
/baoyu-imagine --prompt "一只猫" --image cat.png --provider openrouter
# OpenRouter + 参考图
/baoyu-imagine --prompt "把它变成蓝色" --image out.png --provider openrouter --model google/gemini-3.1-flash-image-preview --ref source.png
# DashScope(阿里通义万相)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider dashscope
/baoyu-imagine --prompt "一只可爱的猫" --image cat.png --provider dashscope
# DashScope 自定义尺寸
/baoyu-imagine --prompt "为咖啡品牌设计一张 21:9 横幅海报,包含清晰中文标题" --image banner.png --provider dashscope --model qwen-image-2.0-pro --size 2048x872
# MiniMax
/baoyu-imagine --prompt "A fashion editorial portrait by a bright studio window" --image out.jpg --provider minimax
# MiniMax + 角色参考图
/baoyu-imagine --prompt "A girl stands by the library window, cinematic lighting" --image out.jpg --provider minimax --model image-01 --ref portrait.png --ar 16:9
# Replicate
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider replicate
/baoyu-imagine --prompt "一只猫" --image cat.png --provider replicate
# 即梦(Jimeng
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider jimeng
/baoyu-imagine --prompt "一只可爱的猫" --image cat.png --provider jimeng
# 豆包(Seedream
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider seedream
/baoyu-imagine --prompt "一只可爱的猫" --image cat.png --provider seedream
# 带参考图(Google、OpenAI、Azure OpenAI、OpenRouter、Replicate 或 Seedream 5.0/4.5/4.0
/baoyu-image-gen --prompt "把它变成蓝色" --image out.png --ref source.png
# 带参考图(Google、OpenAI、Azure OpenAI、OpenRouter、Replicate、MiniMax 或 Seedream 5.0/4.5/4.0
/baoyu-imagine --prompt "把它变成蓝色" --image out.png --ref source.png
# 批量模式
/baoyu-imagine --batchfile batch.json --jobs 4 --json
```
**选项**
@@ -706,44 +721,73 @@ AI 驱动的生成后端。
| `--prompt`, `-p` | 提示词文本 |
| `--promptfiles` | 从文件读取提示词(多文件拼接) |
| `--image` | 输出图片路径(必需) |
| `--provider` | `google``openai``openrouter``dashscope``jimeng``seedream``replicate`(默认:自动检测,优先 google |
| `--model`, `-m` | 模型 ID |
| `--batchfile` | 多图批量生成的 JSON 文件 |
| `--jobs` | 批量模式的并发 worker 数 |
| `--provider` | `google``openai``azure``openrouter``dashscope``minimax``jimeng``seedream``replicate` |
| `--model`, `-m` | 模型 ID 或部署名。Azure 使用部署名;OpenRouter 使用完整模型 IDMiniMax 使用 `image-01` / `image-01-live` |
| `--ar` | 宽高比(如 `16:9``1:1``4:3` |
| `--size` | 尺寸(如 `1024x1024` |
| `--quality` | `normal``2k`(默认:`2k` |
| `--ref` | 参考图片(Google、OpenAI、OpenRouter、Replicate 或 Seedream 5.0/4.5/4.0 |
| `--imageSize` | Google/OpenRouter 使用的 `1K``2K``4K` |
| `--ref` | 参考图片(Google、OpenAI、Azure OpenAI、OpenRouter、Replicate、MiniMax 或 Seedream 5.0/4.5/4.0 |
| `--n` | 单次请求生成图片数量 |
| `--json` | 输出 JSON 结果 |
**环境变量**(配置方法见[环境配置](#环境配置)):
| 变量 | 说明 | 默认值 |
|------|------|--------|
| `OPENAI_API_KEY` | OpenAI API 密钥 | - |
| `AZURE_OPENAI_API_KEY` | Azure OpenAI API 密钥 | - |
| `OPENROUTER_API_KEY` | OpenRouter API 密钥 | - |
| `GOOGLE_API_KEY` | Google API 密钥 | - |
| `GEMINI_API_KEY` | `GOOGLE_API_KEY` 的别名 | - |
| `DASHSCOPE_API_KEY` | DashScope API 密钥(阿里云) | - |
| `MINIMAX_API_KEY` | MiniMax API 密钥 | - |
| `REPLICATE_API_TOKEN` | Replicate API Token | - |
| `JIMENG_ACCESS_KEY_ID` | 即梦火山引擎 Access Key | - |
| `JIMENG_SECRET_ACCESS_KEY` | 即梦火山引擎 Secret Key | - |
| `ARK_API_KEY` | 豆包火山引擎 ARK API 密钥 | - |
| `OPENAI_IMAGE_MODEL` | OpenAI 模型 | `gpt-image-1.5` |
| `AZURE_OPENAI_DEPLOYMENT` | Azure 默认部署名 | - |
| `AZURE_OPENAI_IMAGE_MODEL` | 兼容旧配置的 Azure 部署/模型别名 | `gpt-image-1.5` |
| `OPENROUTER_IMAGE_MODEL` | OpenRouter 模型 | `google/gemini-3.1-flash-image-preview` |
| `GOOGLE_IMAGE_MODEL` | Google 模型 | `gemini-3-pro-image-preview` |
| `DASHSCOPE_IMAGE_MODEL` | DashScope 模型 | `qwen-image-2.0-pro` |
| `MINIMAX_IMAGE_MODEL` | MiniMax 模型 | `image-01` |
| `REPLICATE_IMAGE_MODEL` | Replicate 模型 | `google/nano-banana-pro` |
| `JIMENG_IMAGE_MODEL` | 即梦模型 | `jimeng_t2i_v40` |
| `SEEDREAM_IMAGE_MODEL` | 豆包模型 | `doubao-seedream-5-0-260128` |
| `OPENAI_BASE_URL` | 自定义 OpenAI 端点 | - |
| `OPENAI_IMAGE_USE_CHAT` | OpenAI 改走 `/chat/completions` | `false` |
| `AZURE_OPENAI_BASE_URL` | Azure 资源或部署端点 | - |
| `AZURE_API_VERSION` | Azure 图像 API 版本 | `2025-04-01-preview` |
| `OPENROUTER_BASE_URL` | 自定义 OpenRouter 端点 | `https://openrouter.ai/api/v1` |
| `OPENROUTER_HTTP_REFERER` | OpenRouter 归因用站点 URL | - |
| `OPENROUTER_TITLE` | OpenRouter 归因用应用名 | - |
| `GOOGLE_BASE_URL` | 自定义 Google 端点 | - |
| `DASHSCOPE_BASE_URL` | 自定义 DashScope 端点 | - |
| `MINIMAX_BASE_URL` | 自定义 MiniMax 端点 | `https://api.minimax.io` |
| `REPLICATE_BASE_URL` | 自定义 Replicate 端点 | - |
| `JIMENG_BASE_URL` | 自定义即梦端点 | `https://visual.volcengineapi.com` |
| `JIMENG_REGION` | 即梦区域 | `cn-north-1` |
| `SEEDREAM_BASE_URL` | 自定义豆包端点 | `https://ark.cn-beijing.volces.com/api/v3` |
| `BAOYU_IMAGE_GEN_MAX_WORKERS` | 批量模式最大 worker 数 | `10` |
| `BAOYU_IMAGE_GEN_<PROVIDER>_CONCURRENCY` | 覆盖 provider 并发数 | provider 默认值 |
| `BAOYU_IMAGE_GEN_<PROVIDER>_START_INTERVAL_MS` | 覆盖 provider 请求启动间隔 | provider 默认值 |
**Provider 说明**
- Azure OpenAI`--model` 表示 Azure deployment name,不是底层模型家族名。
- DashScope`qwen-image-2.0-pro` 是自定义 `--size``21:9` 和中英文排版的推荐默认模型。
- MiniMax`image-01` 支持官方文档里的自定义 `width` / `height``image-01-live` 更偏低延迟,适合配合 `--ar` 使用。
- MiniMax 参考图会走 `subject_reference`,当前能力更偏角色 / 人像一致性。
- 即梦不支持参考图。
- 豆包参考图能力仅适用于 Seedream 5.0 / 4.5 / 4.0,不适用于 Seedream 3.0。
**服务商自动选择**
1. 如果指定了 `--provider` → 使用指定的
2. 如果只有一个 API 密钥 → 使用对应服务商
3. 如果多个可用 → 默认使用 Google
2. 如果传了 `--ref` 且未指定 provider → 依次尝试 Google、OpenAI、Azure、OpenRouter、Replicate、Seedream,最后是 MiniMax
3. 如果只有一个 API 密钥 → 使用对应服务商
4. 如果多个可用 → 默认使用 Google
#### baoyu-danger-gemini-web
@@ -1001,7 +1045,7 @@ AI 驱动的生成后端。
部分技能需要 API 密钥或自定义配置。环境变量可以在 `.env` 文件中设置:
**加载优先级**(高优先级覆盖低优先级):
1. 命令行环境变量(如 `OPENAI_API_KEY=xxx /baoyu-image-gen ...`
1. 命令行环境变量(如 `OPENAI_API_KEY=xxx /baoyu-imagine ...`
2. `process.env`(系统环境变量)
3. `<cwd>/.baoyu-skills/.env`(项目级)
4. `~/.baoyu-skills/.env`(用户级)
@@ -1018,11 +1062,20 @@ cat > ~/.baoyu-skills/.env << 'EOF'
OPENAI_API_KEY=sk-xxx
OPENAI_IMAGE_MODEL=gpt-image-1.5
# OPENAI_BASE_URL=https://api.openai.com/v1
# OPENAI_IMAGE_USE_CHAT=false
# Azure OpenAI
AZURE_OPENAI_API_KEY=xxx
AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
AZURE_OPENAI_DEPLOYMENT=gpt-image-1.5
# AZURE_API_VERSION=2025-04-01-preview
# OpenRouter
OPENROUTER_API_KEY=sk-or-xxx
OPENROUTER_IMAGE_MODEL=google/gemini-3.1-flash-image-preview
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# OPENROUTER_HTTP_REFERER=https://your-app.example.com
# OPENROUTER_TITLE=你的应用名
# Google
GOOGLE_API_KEY=xxx
@@ -1034,6 +1087,11 @@ DASHSCOPE_API_KEY=sk-xxx
DASHSCOPE_IMAGE_MODEL=qwen-image-2.0-pro
# DASHSCOPE_BASE_URL=https://dashscope.aliyuncs.com/api/v1
# MiniMax
MINIMAX_API_KEY=xxx
MINIMAX_IMAGE_MODEL=image-01
# MINIMAX_BASE_URL=https://api.minimax.io
# Replicate
REPLICATE_API_TOKEN=r8_xxx
REPLICATE_IMAGE_MODEL=google/nano-banana-pro
+3 -3
View File
@@ -4,7 +4,7 @@ Skills that require image generation MUST delegate to available image generation
## Skill Selection
**Default**: `skills/baoyu-image-gen/SKILL.md` (unless user specifies otherwise).
**Default**: `skills/baoyu-imagine/SKILL.md` (unless user specifies otherwise).
1. Read skill's SKILL.md for parameters and capabilities
2. If user requests different skill, check `skills/` for alternatives
@@ -16,7 +16,7 @@ Skills that require image generation MUST delegate to available image generation
### Step N: Generate Images
**Skill Selection**:
1. Check available skills (`baoyu-image-gen` default, or `baoyu-danger-gemini-web`)
1. Check available skills (`baoyu-imagine` default, or `baoyu-danger-gemini-web`)
2. Read selected skill's SKILL.md for parameters
3. If multiple skills available, ask user to choose
@@ -27,7 +27,7 @@ Skills that require image generation MUST delegate to available image generation
4. On failure, auto-retry once before reporting error
```
**Batch Parallel** (`baoyu-image-gen` only): concurrent workers with per-provider throttling via `batch.max_workers` in EXTEND.md.
**Batch Parallel** (`baoyu-imagine` only): concurrent workers with per-provider throttling via `batch.max_workers` in EXTEND.md.
## Output Path Convention
+1 -1
View File
@@ -118,7 +118,7 @@ Full template: [references/workflow.md](references/workflow.md#step-4-generate-o
**BLOCKING: Prompt files MUST be saved before ANY image generation.**
**Execution strategy**: When multiple illustrations have saved prompt files and the task is now plain generation, prefer `baoyu-image-gen` batch mode (`build-batch.ts``--batchfile`) over spawning subagents. Use subagents only when each image still needs separate prompt iteration or creative exploration.
**Execution strategy**: When multiple illustrations have saved prompt files and the task is now plain generation, prefer `baoyu-imagine` batch mode (`build-batch.ts``--batchfile`) over spawning subagents. Use subagents only when each image still needs separate prompt iteration or creative exploration.
1. For each illustration, create a prompt file per [references/prompt-construction.md](references/prompt-construction.md)
2. Save to `prompts/NN-{type}-{slug}.md` with YAML frontmatter
@@ -316,7 +316,7 @@ Prompt Files:
**DO NOT** pass ad-hoc inline text to `--prompt` without first saving prompt files. The generation command should either use `--promptfiles prompts/NN-{type}-{slug}.md` or read the saved file content for `--prompt`.
**Execution choice**:
- If multiple illustrations already have saved prompt files and the task is now plain generation, prefer `baoyu-image-gen` batch mode (`build-batch.ts` -> `main.ts --batchfile`)
- If multiple illustrations already have saved prompt files and the task is now plain generation, prefer `baoyu-imagine` batch mode (`build-batch.ts` -> `main.ts --batchfile`)
- Use subagents only when each illustration still needs separate prompt rewriting, style exploration, or other per-image reasoning before generation
**CRITICAL - References in Frontmatter**:
@@ -352,7 +352,7 @@ Check available skills. If multiple, ask user.
| Skill Supports `--ref` | Action |
|------------------------|--------|
| Yes (e.g., baoyu-image-gen with Google) | Pass reference images via `--ref` |
| Yes (e.g., baoyu-imagine with Google) | Pass reference images via `--ref` |
| No | Convert to text description, append to prompt |
**Verification**: Before generating, confirm reference processing:
@@ -29,8 +29,8 @@ Options:
--prompts <path> Path to prompts directory
--output <path> Path to output batch.json
--images-dir <path> Directory for generated images
--provider <name> Provider for baoyu-image-gen batch tasks (default: replicate)
--model <id> Model for baoyu-image-gen batch tasks (default: google/nano-banana-pro)
--provider <name> Provider for baoyu-imagine batch tasks (default: replicate)
--model <id> Model for baoyu-imagine batch tasks (default: google/nano-banana-pro)
--ar <ratio> Aspect ratio for all tasks (default: 16:9)
--quality <level> Quality for all tasks (default: 2k)
--jobs <count> Recommended worker count metadata (optional)
+1 -1
View File
@@ -216,7 +216,7 @@ Analyze → [Check Existing?] → [Confirm: Style + Reviews] → Storyboard →
**7.1 Generate character sheet first**:
- **Backup rule**: If `characters/characters.png` exists, rename to `characters/characters-backup-YYYYMMDD-HHMMSS.png`
- Invoke an installed image generation skill such as `baoyu-image-gen`
- Invoke an installed image generation skill such as `baoyu-imagine`
- Read that skill's `SKILL.md` and follow its documented interface rather than calling its scripts directly
- Use `characters/characters.md` as the prompt-file input
- Save output to `characters/characters.png`
+1 -1
View File
@@ -433,7 +433,7 @@ With confirmed prompts from Step 5/6:
| Supports `--ref` | **Strategy A** | Pass `characters/characters.png` with EVERY page |
| Does NOT support `--ref` | **Strategy B** | Prepend character descriptions to EVERY prompt |
**Strategy A: Using `--ref` parameter** (e.g., baoyu-image-gen)
**Strategy A: Using `--ref` parameter** (e.g., baoyu-imagine)
- Read the chosen image generation skill's `SKILL.md`
- Invoke that installed skill via its documented interface, not by calling its scripts directly
@@ -1,10 +1,10 @@
---
name: baoyu-image-gen
description: AI image generation with OpenAI, Azure OpenAI, Google, OpenRouter, DashScope, Jimeng, Seedream and Replicate APIs. Supports text-to-image, reference images, aspect ratios, and batch generation from saved prompt files. Sequential by default; use batch parallel generation when the user already has multiple prompts or wants stable multi-image throughput. Use when user asks to generate, create, or draw images.
version: 1.56.3
name: baoyu-imagine
description: AI image generation with OpenAI, Azure OpenAI, Google, OpenRouter, DashScope, MiniMax, Jimeng, Seedream and Replicate APIs. Supports text-to-image, reference images, aspect ratios, and batch generation from saved prompt files. Sequential by default; use batch parallel generation when the user already has multiple prompts or wants stable multi-image throughput. Use when user asks to generate, create, or draw images.
version: 1.56.4
metadata:
openclaw:
homepage: https://github.com/JimLiu/baoyu-skills#baoyu-image-gen
homepage: https://github.com/JimLiu/baoyu-skills#baoyu-imagine
requires:
anyBins:
- bun
@@ -13,7 +13,7 @@ metadata:
# Image Generation (AI SDK)
Official API-based image generation. Supports OpenAI, Azure OpenAI, Google, OpenRouter, DashScope (阿里通义万象), Jimeng (即梦), Seedream (豆包) and Replicate providers.
Official API-based image generation. Supports OpenAI, Azure OpenAI, Google, OpenRouter, DashScope (阿里通义万象), MiniMax, Jimeng (即梦), Seedream (豆包) and Replicate providers.
## Script Directory
@@ -30,17 +30,17 @@ Check EXTEND.md existence (priority: project → user):
```bash
# macOS, Linux, WSL, Git Bash
test -f .baoyu-skills/baoyu-image-gen/EXTEND.md && echo "project"
test -f "${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/baoyu-image-gen/EXTEND.md" && echo "xdg"
test -f "$HOME/.baoyu-skills/baoyu-image-gen/EXTEND.md" && echo "user"
test -f .baoyu-skills/baoyu-imagine/EXTEND.md && echo "project"
test -f "${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/baoyu-imagine/EXTEND.md" && echo "xdg"
test -f "$HOME/.baoyu-skills/baoyu-imagine/EXTEND.md" && echo "user"
```
```powershell
# PowerShell (Windows)
if (Test-Path .baoyu-skills/baoyu-image-gen/EXTEND.md) { "project" }
if (Test-Path .baoyu-skills/baoyu-imagine/EXTEND.md) { "project" }
$xdg = if ($env:XDG_CONFIG_HOME) { $env:XDG_CONFIG_HOME } else { "$HOME/.config" }
if (Test-Path "$xdg/baoyu-skills/baoyu-image-gen/EXTEND.md") { "xdg" }
if (Test-Path "$HOME/.baoyu-skills/baoyu-image-gen/EXTEND.md") { "user" }
if (Test-Path "$xdg/baoyu-skills/baoyu-imagine/EXTEND.md") { "xdg" }
if (Test-Path "$HOME/.baoyu-skills/baoyu-imagine/EXTEND.md") { "user" }
```
| Result | Action |
@@ -52,8 +52,8 @@ if (Test-Path "$HOME/.baoyu-skills/baoyu-image-gen/EXTEND.md") { "user" }
| Path | Location |
|------|----------|
| `.baoyu-skills/baoyu-image-gen/EXTEND.md` | Project directory |
| `$HOME/.baoyu-skills/baoyu-image-gen/EXTEND.md` | User home |
| `.baoyu-skills/baoyu-imagine/EXTEND.md` | Project directory |
| `$HOME/.baoyu-skills/baoyu-imagine/EXTEND.md` | User home |
**EXTEND.md Supports**: Default provider | Default quality | Default aspect ratio | Default image size | Default models | Batch worker cap | Provider-specific batch limits
@@ -74,7 +74,7 @@ ${BUN_X} {baseDir}/scripts/main.ts --prompt "A cat" --image out.png --quality 2k
# From prompt files
${BUN_X} {baseDir}/scripts/main.ts --promptfiles system.md content.md --image out.png
# With reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate, or Seedream 4.0/4.5/5.0)
# With reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate, MiniMax, or Seedream 4.0/4.5/5.0)
${BUN_X} {baseDir}/scripts/main.ts --prompt "Make blue" --image out.png --ref source.png
# With reference images (explicit provider/model)
@@ -101,6 +101,15 @@ ${BUN_X} {baseDir}/scripts/main.ts --prompt "为咖啡品牌设计一张 21:9
# DashScope legacy Qwen fixed-size model
${BUN_X} {baseDir}/scripts/main.ts --prompt "一张电影感海报" --image out.png --provider dashscope --model qwen-image-max --size 1664x928
# MiniMax
${BUN_X} {baseDir}/scripts/main.ts --prompt "A fashion editorial portrait by a bright studio window" --image out.jpg --provider minimax
# MiniMax with subject reference (best for character/portrait consistency)
${BUN_X} {baseDir}/scripts/main.ts --prompt "A girl stands by the library window, cinematic lighting" --image out.jpg --provider minimax --model image-01 --ref portrait.png --ar 16:9
# MiniMax with custom size (documented for image-01)
${BUN_X} {baseDir}/scripts/main.ts --prompt "A cinematic poster" --image out.jpg --provider minimax --model image-01 --size 1536x1024
# Replicate (google/nano-banana-pro)
${BUN_X} {baseDir}/scripts/main.ts --prompt "A cat" --image out.png --provider replicate
@@ -150,13 +159,13 @@ Paths in `promptFiles`, `image`, and `ref` are resolved relative to the batch fi
| `--image <path>` | Output image path (required in single-image mode) |
| `--batchfile <path>` | JSON batch file for multi-image generation |
| `--jobs <count>` | Worker count for batch mode (default: auto, max from config, built-in default 10) |
| `--provider google\|openai\|azure\|openrouter\|dashscope\|jimeng\|seedream\|replicate` | Force provider (default: auto-detect) |
| `--model <id>`, `-m` | Model ID (Google: `gemini-3-pro-image-preview`; OpenAI: `gpt-image-1.5`; Azure: deployment name such as `gpt-image-1.5` or `image-prod`; OpenRouter: `google/gemini-3.1-flash-image-preview`; DashScope: `qwen-image-2.0-pro`) |
| `--provider google\|openai\|azure\|openrouter\|dashscope\|minimax\|jimeng\|seedream\|replicate` | Force provider (default: auto-detect) |
| `--model <id>`, `-m` | Model ID (Google: `gemini-3-pro-image-preview`; OpenAI: `gpt-image-1.5`; Azure: deployment name such as `gpt-image-1.5` or `image-prod`; OpenRouter: `google/gemini-3.1-flash-image-preview`; DashScope: `qwen-image-2.0-pro`; MiniMax: `image-01`) |
| `--ar <ratio>` | Aspect ratio (e.g., `16:9`, `1:1`, `4:3`) |
| `--size <WxH>` | Size (e.g., `1024x1024`) |
| `--quality normal\|2k` | Quality preset (default: `2k`) |
| `--imageSize 1K\|2K\|4K` | Image size for Google/OpenRouter (default: from quality) |
| `--ref <files...>` | Reference images. Supported by Google multimodal, OpenAI GPT Image edits, Azure OpenAI edits (PNG/JPG only), OpenRouter multimodal models, Replicate, and Seedream 5.0/4.5/4.0. Not supported by Jimeng, Seedream 3.0, or removed SeedEdit 3.0 |
| `--ref <files...>` | Reference images. Supported by Google multimodal, OpenAI GPT Image edits, Azure OpenAI edits (PNG/JPG only), OpenRouter multimodal models, Replicate, MiniMax subject-reference, and Seedream 5.0/4.5/4.0. Not supported by Jimeng, Seedream 3.0, or removed SeedEdit 3.0 |
| `--n <count>` | Number of images |
| `--json` | JSON output |
@@ -169,6 +178,7 @@ Paths in `promptFiles`, `image`, and `ref` are resolved relative to the batch fi
| `OPENROUTER_API_KEY` | OpenRouter API key |
| `GOOGLE_API_KEY` | Google API key |
| `DASHSCOPE_API_KEY` | DashScope API key (阿里云) |
| `MINIMAX_API_KEY` | MiniMax API key |
| `REPLICATE_API_TOKEN` | Replicate API token |
| `JIMENG_ACCESS_KEY_ID` | Jimeng (即梦) Volcengine access key |
| `JIMENG_SECRET_ACCESS_KEY` | Jimeng (即梦) Volcengine secret key |
@@ -179,6 +189,7 @@ Paths in `promptFiles`, `image`, and `ref` are resolved relative to the batch fi
| `OPENROUTER_IMAGE_MODEL` | OpenRouter model override (default: `google/gemini-3.1-flash-image-preview`) |
| `GOOGLE_IMAGE_MODEL` | Google model override |
| `DASHSCOPE_IMAGE_MODEL` | DashScope model override (default: `qwen-image-2.0-pro`) |
| `MINIMAX_IMAGE_MODEL` | MiniMax model override (default: `image-01`) |
| `REPLICATE_IMAGE_MODEL` | Replicate model override (default: google/nano-banana-pro) |
| `JIMENG_IMAGE_MODEL` | Jimeng model override (default: jimeng_t2i_v40) |
| `SEEDREAM_IMAGE_MODEL` | Seedream model override (default: doubao-seedream-5-0-260128) |
@@ -190,6 +201,7 @@ Paths in `promptFiles`, `image`, and `ref` are resolved relative to the batch fi
| `OPENROUTER_TITLE` | Optional app name for OpenRouter attribution |
| `GOOGLE_BASE_URL` | Custom Google endpoint |
| `DASHSCOPE_BASE_URL` | Custom DashScope endpoint |
| `MINIMAX_BASE_URL` | Custom MiniMax endpoint (default: `https://api.minimax.io`) |
| `REPLICATE_BASE_URL` | Custom Replicate endpoint |
| `JIMENG_BASE_URL` | Custom Jimeng endpoint (default: `https://visual.volcengineapi.com`) |
| `JIMENG_REGION` | Jimeng region (default: `cn-north-1`) |
@@ -240,7 +252,7 @@ When translating CLI args into DashScope behavior:
- `--size` wins over `--ar`
- For `qwen-image-2.0*`, prefer explicit `--size`; otherwise infer from `--ar` and use the official recommended resolutions below
- For `qwen-image-max/plus/image`, only use the five official fixed sizes; if the requested ratio is not covered, switch to `qwen-image-2.0-pro`
- `--quality` is a baoyu-image-gen compatibility preset, not a native DashScope API field. Mapping `normal` / `2k` onto the `qwen-image-2.0*` table below is an implementation inference, not an official API guarantee
- `--quality` is a baoyu-imagine compatibility preset, not a native DashScope API field. Mapping `normal` / `2k` onto the `qwen-image-2.0*` table below is an implementation inference, not an official API guarantee
Recommended `qwen-image-2.0*` sizes for common aspect ratios:
@@ -255,7 +267,7 @@ Recommended `qwen-image-2.0*` sizes for common aspect ratios:
| `16:9` | `1280*720` | `1920*1080` |
| `21:9` | `1344*576` | `2048*872` |
DashScope official APIs also expose `negative_prompt`, `prompt_extend`, and `watermark`, but `baoyu-image-gen` does not expose them as dedicated CLI flags today.
DashScope official APIs also expose `negative_prompt`, `prompt_extend`, and `watermark`, but `baoyu-imagine` does not expose them as dedicated CLI flags today.
Official references:
@@ -263,6 +275,34 @@ Official references:
- [Text-to-image guide](https://help.aliyun.com/zh/model-studio/text-to-image)
- [Qwen-Image Edit API](https://help.aliyun.com/zh/model-studio/qwen-image-edit-api)
### MiniMax Models
Use `--model image-01` or set `default_model.minimax` / `MINIMAX_IMAGE_MODEL` when the user wants MiniMax image generation.
Official MiniMax image model options currently documented in the API reference:
- `image-01` (recommended default)
- Supports text-to-image and subject-reference image generation
- Supports official `aspect_ratio` values: `1:1`, `16:9`, `4:3`, `3:2`, `2:3`, `3:4`, `9:16`, `21:9`
- Supports documented custom `width` / `height` output sizes when using `--size <WxH>`
- `width` and `height` must both be between `512` and `2048`, and both must be divisible by `8`
- `image-01-live`
- Lower-latency variant
- Use `--ar` for sizing; MiniMax documents custom `width` / `height` as only effective for `image-01`
MiniMax subject reference notes:
- `--ref` files are sent as MiniMax `subject_reference`
- MiniMax docs currently describe `subject_reference[].type` as `character`
- Official docs say `image_file` supports public URLs or Base64 Data URLs; `baoyu-imagine` sends local refs as Data URLs
- Official docs recommend front-facing portrait references in JPG/JPEG/PNG under 10MB
Official references:
- [MiniMax Image Generation Guide](https://platform.minimax.io/docs/guides/image-generation)
- [MiniMax Text-to-Image API](https://platform.minimax.io/docs/api-reference/image-generation-t2i)
- [MiniMax Image-to-Image API](https://platform.minimax.io/docs/api-reference/image-generation-i2i)
### OpenRouter Models
Use full OpenRouter model IDs, e.g.:
@@ -297,8 +337,8 @@ ${BUN_X} {baseDir}/scripts/main.ts --prompt "A cat" --image out.png --provider r
## Provider Selection
1. `--ref` provided + no `--provider` → auto-select Google first, then OpenAI, then OpenRouter, then Replicate (Jimeng and Seedream do not support reference images)
2. `--provider` specified → use it (if `--ref`, must be `google`, `openai`, `openrouter`, or `replicate`)
1. `--ref` provided + no `--provider` → auto-select Google first, then OpenAI, then Azure, then OpenRouter, then Replicate, then Seedream, then MiniMax (MiniMax subject reference is more specialized toward character/portrait consistency)
2. `--provider` specified → use it (if `--ref`, must be `google`, `openai`, `azure`, `openrouter`, `replicate`, `seedream`, or `minimax`)
3. Only one API key available → use that provider
4. Multiple available → default to Google
@@ -319,6 +359,7 @@ Supported: `1:1`, `16:9`, `9:16`, `4:3`, `3:4`, `2.35:1`
- OpenAI: maps to closest supported size
- OpenRouter: sends `imageGenerationOptions.aspect_ratio`; if only `--size <WxH>` is given, aspect ratio is inferred automatically
- Replicate: passes `aspect_ratio` to model; when `--ref` is provided without `--ar`, defaults to `match_input_image`
- MiniMax: sends official `aspect_ratio` values directly; if `--size <WxH>` is given without `--ar`, `width` / `height` are sent for `image-01`
## Generation Mode
@@ -1,6 +1,6 @@
---
name: first-time-setup
description: First-time setup and default model selection flow for baoyu-image-gen
description: First-time setup and default model selection flow for baoyu-imagine
---
# First-Time Setup
@@ -53,6 +53,8 @@ options:
description: "Router for Gemini/FLUX/OpenAI-compatible image models"
- label: "DashScope"
description: "Alibaba Cloud - Qwen-Image, strong Chinese/English text rendering"
- label: "MiniMax"
description: "MiniMax image generation with subject-reference character workflows"
- label: "Replicate"
description: "Community models - nano-banana-pro, flexible model selection"
```
@@ -103,6 +105,20 @@ options:
description: "Previous GPT Image deployment name"
```
### Question 2d: Default MiniMax Model
Only show if user selected MiniMax.
```yaml
header: "MiniMax Model"
question: "Default MiniMax image generation model?"
options:
- label: "image-01 (Recommended)"
description: "Best default, supports aspect ratios and custom width/height"
- label: "image-01-live"
description: "Faster variant, use aspect ratio instead of custom size"
```
### Question 3: Default Quality
```yaml
@@ -131,8 +147,8 @@ options:
| Choice | Path | Scope |
|--------|------|-------|
| Project | `.baoyu-skills/baoyu-image-gen/EXTEND.md` | Current project |
| User | `$HOME/.baoyu-skills/baoyu-image-gen/EXTEND.md` | All projects |
| Project | `.baoyu-skills/baoyu-imagine/EXTEND.md` | Current project |
| User | `$HOME/.baoyu-skills/baoyu-imagine/EXTEND.md` | All projects |
### EXTEND.md Template
@@ -149,6 +165,7 @@ default_model:
azure: [selected azure deployment or null]
openrouter: [selected openrouter model or null]
dashscope: null
minimax: [selected minimax model or null]
replicate: null
---
```
@@ -197,7 +214,7 @@ options:
Notes for Azure setup:
- In `baoyu-image-gen`, Azure `--model` / `default_model.azure` should be the Azure deployment name, not just the underlying model family.
- In `baoyu-imagine`, Azure `--model` / `default_model.azure` should be the Azure deployment name, not just the underlying model family.
- If the deployment name is custom, save that exact deployment name in `default_model.azure`.
### OpenRouter Model Selection
@@ -238,7 +255,7 @@ Notes for DashScope setup:
- Prefer `qwen-image-2.0-pro` when the user needs custom `--size`, uncommon ratios like `21:9`, or strong Chinese/English text rendering.
- `qwen-image-max` / `qwen-image-plus` / `qwen-image` only support five fixed sizes: `1664*928`, `1472*1104`, `1328*1328`, `1104*1472`, `928*1664`.
- In `baoyu-image-gen`, `quality` is a compatibility preset. It is not a native DashScope parameter.
- In `baoyu-imagine`, `quality` is a compatibility preset. It is not a native DashScope parameter.
### Replicate Model Selection
@@ -252,6 +269,24 @@ options:
description: "Google's base image model on Replicate"
```
### MiniMax Model Selection
```yaml
header: "MiniMax Model"
question: "Choose a default MiniMax image generation model?"
options:
- label: "image-01 (Recommended)"
description: "Best general-purpose MiniMax image model with custom width/height support"
- label: "image-01-live"
description: "Lower-latency MiniMax image model using aspect ratios"
```
Notes for MiniMax setup:
- `image-01` is the safest default. It supports official `aspect_ratio` values and documented custom `width` / `height` output sizes.
- `image-01-live` is useful when the user prefers faster generation and can work with aspect-ratio-based sizing.
- MiniMax subject reference currently uses `subject_reference[].type = character`; docs recommend front-facing portrait references in JPG/JPEG/PNG under 10MB.
### Update EXTEND.md
After user selects a model:
@@ -267,6 +302,7 @@ default_model:
azure: [value or null]
openrouter: [value or null]
dashscope: [value or null]
minimax: [value or null]
replicate: [value or null]
```
@@ -1,6 +1,6 @@
---
name: preferences-schema
description: EXTEND.md YAML schema for baoyu-image-gen user preferences
description: EXTEND.md YAML schema for baoyu-imagine user preferences
---
# Preferences Schema
@@ -11,7 +11,7 @@ description: EXTEND.md YAML schema for baoyu-image-gen user preferences
---
version: 1
default_provider: null # google|openai|azure|openrouter|dashscope|replicate|null (null = auto-detect)
default_provider: null # google|openai|azure|openrouter|dashscope|minimax|replicate|null (null = auto-detect)
default_quality: null # normal|2k|null (null = use default: 2k)
@@ -25,6 +25,7 @@ default_model:
azure: null # Azure deployment name, e.g., "gpt-image-1.5" or "image-prod"
openrouter: null # e.g., "google/gemini-3.1-flash-image-preview"
dashscope: null # e.g., "qwen-image-2.0-pro"
minimax: null # e.g., "image-01"
replicate: null # e.g., "google/nano-banana-pro"
batch:
@@ -48,6 +49,9 @@ batch:
dashscope:
concurrency: 3
start_interval_ms: 1100
minimax:
concurrency: 3
start_interval_ms: 1100
---
```
@@ -65,6 +69,7 @@ batch:
| `default_model.azure` | string\|null | null | Azure default deployment name |
| `default_model.openrouter` | string\|null | null | OpenRouter default model |
| `default_model.dashscope` | string\|null | null | DashScope default model |
| `default_model.minimax` | string\|null | null | MiniMax default model |
| `default_model.replicate` | string\|null | null | Replicate default model |
| `batch.max_workers` | int\|null | 10 | Batch worker cap |
| `batch.provider_limits.<provider>.concurrency` | int\|null | provider default | Max simultaneous requests per provider |
@@ -95,6 +100,7 @@ default_model:
azure: "gpt-image-1.5"
openrouter: "google/gemini-3.1-flash-image-preview"
dashscope: "qwen-image-2.0-pro"
minimax: "image-01"
replicate: "google/nano-banana-pro"
batch:
max_workers: 10
@@ -108,5 +114,8 @@ batch:
openrouter:
concurrency: 3
start_interval_ms: 1100
minimax:
concurrency: 3
start_interval_ms: 1100
---
```
@@ -69,7 +69,7 @@ async function makeTempDir(prefix: string): Promise<string> {
return fs.mkdtemp(path.join(os.tmpdir(), prefix));
}
test("parseArgs parses the main image-gen CLI flags", () => {
test("parseArgs parses the main baoyu-imagine CLI flags", () => {
const args = parseArgs([
"--promptfiles",
"prompts/system.md",
@@ -124,6 +124,7 @@ default_model:
google: gemini-3-pro-image-preview
openai: gpt-image-1.5
azure: image-prod
minimax: image-01
batch:
max_workers: 8
provider_limits:
@@ -132,6 +133,9 @@ batch:
start_interval_ms: 900
openai:
concurrency: 4
minimax:
concurrency: 2
start_interval_ms: 1400
azure:
concurrency: 1
start_interval_ms: 1500
@@ -147,6 +151,7 @@ batch:
assert.equal(config.default_model?.google, "gemini-3-pro-image-preview");
assert.equal(config.default_model?.openai, "gpt-image-1.5");
assert.equal(config.default_model?.azure, "image-prod");
assert.equal(config.default_model?.minimax, "image-01");
assert.equal(config.batch?.max_workers, 8);
assert.deepEqual(config.batch?.provider_limits?.google, {
concurrency: 2,
@@ -155,6 +160,10 @@ batch:
assert.deepEqual(config.batch?.provider_limits?.openai, {
concurrency: 4,
});
assert.deepEqual(config.batch?.provider_limits?.minimax, {
concurrency: 2,
start_interval_ms: 1400,
});
assert.deepEqual(config.batch?.provider_limits?.azure, {
concurrency: 1,
start_interval_ms: 1500,
@@ -200,6 +209,7 @@ test("detectProvider rejects non-ref-capable providers and prefers Google first
OPENAI_API_KEY: "openai-key",
OPENROUTER_API_KEY: null,
DASHSCOPE_API_KEY: null,
MINIMAX_API_KEY: null,
REPLICATE_API_TOKEN: null,
JIMENG_ACCESS_KEY_ID: null,
JIMENG_SECRET_ACCESS_KEY: null,
@@ -216,6 +226,7 @@ test("detectProvider selects an available ref-capable provider for reference-ima
AZURE_OPENAI_BASE_URL: null,
OPENROUTER_API_KEY: null,
DASHSCOPE_API_KEY: null,
MINIMAX_API_KEY: null,
REPLICATE_API_TOKEN: null,
JIMENG_ACCESS_KEY_ID: null,
JIMENG_SECRET_ACCESS_KEY: null,
@@ -235,6 +246,7 @@ test("detectProvider selects Azure when only Azure credentials are configured",
AZURE_OPENAI_BASE_URL: "https://example.openai.azure.com",
OPENROUTER_API_KEY: null,
DASHSCOPE_API_KEY: null,
MINIMAX_API_KEY: null,
REPLICATE_API_TOKEN: null,
JIMENG_ACCESS_KEY_ID: null,
JIMENG_SECRET_ACCESS_KEY: null,
@@ -254,6 +266,7 @@ test("detectProvider infers Seedream from model id and allows Seedream reference
OPENAI_API_KEY: null,
OPENROUTER_API_KEY: null,
DASHSCOPE_API_KEY: null,
MINIMAX_API_KEY: null,
REPLICATE_API_TOKEN: null,
JIMENG_ACCESS_KEY_ID: null,
JIMENG_SECRET_ACCESS_KEY: null,
@@ -281,6 +294,26 @@ test("detectProvider infers Seedream from model id and allows Seedream reference
);
});
test("detectProvider selects MiniMax when only MiniMax credentials are configured or the model id matches", (t) => {
useEnv(t, {
GOOGLE_API_KEY: null,
OPENAI_API_KEY: null,
AZURE_OPENAI_API_KEY: null,
AZURE_OPENAI_BASE_URL: null,
OPENROUTER_API_KEY: null,
DASHSCOPE_API_KEY: null,
MINIMAX_API_KEY: "minimax-key",
REPLICATE_API_TOKEN: null,
JIMENG_ACCESS_KEY_ID: null,
JIMENG_SECRET_ACCESS_KEY: null,
ARK_API_KEY: null,
});
assert.equal(detectProvider(makeArgs()), "minimax");
assert.equal(detectProvider(makeArgs({ referenceImages: ["ref.png"] })), "minimax");
assert.equal(detectProvider(makeArgs({ model: "image-01-live" })), "minimax");
});
test("batch worker and provider-rate-limit configuration prefer env over EXTEND config", (t) => {
useEnv(t, {
BAOYU_IMAGE_GEN_MAX_WORKERS: "12",
@@ -296,6 +329,10 @@ test("batch worker and provider-rate-limit configuration prefer env over EXTEND
concurrency: 2,
start_interval_ms: 900,
},
minimax: {
concurrency: 1,
start_interval_ms: 1500,
},
},
},
};
@@ -305,10 +342,14 @@ test("batch worker and provider-rate-limit configuration prefer env over EXTEND
concurrency: 5,
startIntervalMs: 450,
});
assert.deepEqual(getConfiguredProviderRateLimits(extendConfig).minimax, {
concurrency: 1,
startIntervalMs: 1500,
});
});
test("loadBatchTasks and createTaskArgs resolve batch-relative paths", async (t) => {
const root = await makeTempDir("baoyu-image-gen-batch-");
const root = await makeTempDir("baoyu-imagine-batch-");
t.after(() => fs.rm(root, { recursive: true, force: true }));
const batchFile = path.join(root, "jobs", "batch.json");
@@ -58,6 +58,7 @@ const DEFAULT_PROVIDER_RATE_LIMITS: Record<Provider, ProviderRateLimit> = {
openai: { concurrency: 3, startIntervalMs: 1100 },
openrouter: { concurrency: 3, startIntervalMs: 1100 },
dashscope: { concurrency: 3, startIntervalMs: 1100 },
minimax: { concurrency: 3, startIntervalMs: 1100 },
jimeng: { concurrency: 3, startIntervalMs: 1100 },
seedream: { concurrency: 3, startIntervalMs: 1100 },
azure: { concurrency: 3, startIntervalMs: 1100 },
@@ -75,13 +76,13 @@ Options:
--image <path> Output image path (required in single-image mode)
--batchfile <path> JSON batch file for multi-image generation
--jobs <count> Worker count for batch mode (default: auto, max from config, built-in default 10)
--provider google|openai|openrouter|dashscope|replicate|jimeng|seedream|azure Force provider (auto-detect by default)
--provider google|openai|openrouter|dashscope|minimax|replicate|jimeng|seedream|azure Force provider (auto-detect by default)
-m, --model <id> Model ID
--ar <ratio> Aspect ratio (e.g., 16:9, 1:1, 4:3)
--size <WxH> Size (e.g., 1024x1024)
--quality normal|2k Quality preset (default: 2k)
--imageSize 1K|2K|4K Image size for Google/OpenRouter (default: from quality)
--ref <files...> Reference images (Google, OpenAI, Azure, OpenRouter, Replicate, or Seedream 4.0/4.5/5.0)
--ref <files...> Reference images (Google, OpenAI, Azure, OpenRouter, Replicate, MiniMax, or Seedream 4.0/4.5/5.0)
--n <count> Number of images for the current task (default: 1)
--json JSON output
-h, --help Show help
@@ -112,6 +113,7 @@ Environment variables:
GOOGLE_API_KEY Google API key
GEMINI_API_KEY Gemini API key (alias for GOOGLE_API_KEY)
DASHSCOPE_API_KEY DashScope API key
MINIMAX_API_KEY MiniMax API key
REPLICATE_API_TOKEN Replicate API token
JIMENG_ACCESS_KEY_ID Jimeng Access Key ID
JIMENG_SECRET_ACCESS_KEY Jimeng Secret Access Key
@@ -120,6 +122,7 @@ Environment variables:
OPENROUTER_IMAGE_MODEL Default OpenRouter model (google/gemini-3.1-flash-image-preview)
GOOGLE_IMAGE_MODEL Default Google model (gemini-3-pro-image-preview)
DASHSCOPE_IMAGE_MODEL Default DashScope model (qwen-image-2.0-pro)
MINIMAX_IMAGE_MODEL Default MiniMax model (image-01)
REPLICATE_IMAGE_MODEL Default Replicate model (google/nano-banana-pro)
JIMENG_IMAGE_MODEL Default Jimeng model (jimeng_t2i_v40)
SEEDREAM_IMAGE_MODEL Default Seedream model (doubao-seedream-5-0-260128)
@@ -130,6 +133,7 @@ Environment variables:
OPENROUTER_TITLE Optional app name for OpenRouter attribution
GOOGLE_BASE_URL Custom Google endpoint
DASHSCOPE_BASE_URL Custom DashScope endpoint
MINIMAX_BASE_URL Custom MiniMax endpoint
REPLICATE_BASE_URL Custom Replicate endpoint
JIMENG_BASE_URL Custom Jimeng endpoint
AZURE_OPENAI_API_KEY Azure OpenAI API key
@@ -235,6 +239,7 @@ export function parseArgs(argv: string[]): CliArgs {
v !== "openai" &&
v !== "openrouter" &&
v !== "dashscope" &&
v !== "minimax" &&
v !== "replicate" &&
v !== "jimeng" &&
v !== "seedream" &&
@@ -390,6 +395,7 @@ export function parseSimpleYaml(yaml: string): Partial<ExtendConfig> {
openai: null,
openrouter: null,
dashscope: null,
minimax: null,
replicate: null,
jimeng: null,
seedream: null,
@@ -417,6 +423,7 @@ export function parseSimpleYaml(yaml: string): Partial<ExtendConfig> {
key === "openai" ||
key === "openrouter" ||
key === "dashscope" ||
key === "minimax" ||
key === "replicate" ||
key === "jimeng" ||
key === "seedream" ||
@@ -434,6 +441,7 @@ export function parseSimpleYaml(yaml: string): Partial<ExtendConfig> {
key === "openai" ||
key === "openrouter" ||
key === "dashscope" ||
key === "minimax" ||
key === "replicate" ||
key === "jimeng" ||
key === "seedream" ||
@@ -468,8 +476,8 @@ async function loadExtendConfig(): Promise<Partial<ExtendConfig>> {
const cwd = process.cwd();
const paths = [
path.join(cwd, ".baoyu-skills", "baoyu-image-gen", "EXTEND.md"),
path.join(home, ".baoyu-skills", "baoyu-image-gen", "EXTEND.md"),
path.join(cwd, ".baoyu-skills", "baoyu-imagine", "EXTEND.md"),
path.join(home, ".baoyu-skills", "baoyu-imagine", "EXTEND.md"),
];
for (const p of paths) {
@@ -528,12 +536,13 @@ export function getConfiguredProviderRateLimits(
openai: { ...DEFAULT_PROVIDER_RATE_LIMITS.openai },
openrouter: { ...DEFAULT_PROVIDER_RATE_LIMITS.openrouter },
dashscope: { ...DEFAULT_PROVIDER_RATE_LIMITS.dashscope },
minimax: { ...DEFAULT_PROVIDER_RATE_LIMITS.minimax },
jimeng: { ...DEFAULT_PROVIDER_RATE_LIMITS.jimeng },
seedream: { ...DEFAULT_PROVIDER_RATE_LIMITS.seedream },
azure: { ...DEFAULT_PROVIDER_RATE_LIMITS.azure },
};
for (const provider of ["replicate", "google", "openai", "openrouter", "dashscope", "jimeng", "seedream", "azure"] as Provider[]) {
for (const provider of ["replicate", "google", "openai", "openrouter", "dashscope", "minimax", "jimeng", "seedream", "azure"] as Provider[]) {
const envPrefix = `BAOYU_IMAGE_GEN_${provider.toUpperCase()}`;
const extendLimit = extendConfig.batch?.provider_limits?.[provider];
configured[provider] = {
@@ -582,7 +591,9 @@ export function normalizeOutputImagePath(p: string, defaultExtension = ".png"):
function inferProviderFromModel(model: string | null): Provider | null {
if (!model) return null;
if (model.includes("seedream") || model.includes("seededit")) return "seedream";
const normalized = model.trim();
if (normalized.includes("seedream") || normalized.includes("seededit")) return "seedream";
if (normalized === "image-01" || normalized === "image-01-live") return "minimax";
return null;
}
@@ -595,10 +606,11 @@ export function detectProvider(args: CliArgs): Provider {
args.provider !== "azure" &&
args.provider !== "openrouter" &&
args.provider !== "replicate" &&
args.provider !== "seedream"
args.provider !== "seedream" &&
args.provider !== "minimax"
) {
throw new Error(
"Reference images require a ref-capable provider. Use --provider google (Gemini multimodal), --provider openai (GPT Image edits), --provider azure (Azure OpenAI), --provider openrouter (OpenRouter multimodal), --provider replicate, or --provider seedream for supported Seedream models."
"Reference images require a ref-capable provider. Use --provider google (Gemini multimodal), --provider openai (GPT Image edits), --provider azure (Azure OpenAI), --provider openrouter (OpenRouter multimodal), --provider replicate, --provider seedream for supported Seedream models, or --provider minimax for MiniMax subject-reference workflows."
);
}
@@ -609,6 +621,7 @@ export function detectProvider(args: CliArgs): Provider {
const hasOpenai = !!process.env.OPENAI_API_KEY;
const hasOpenrouter = !!process.env.OPENROUTER_API_KEY;
const hasDashscope = !!process.env.DASHSCOPE_API_KEY;
const hasMinimax = !!process.env.MINIMAX_API_KEY;
const hasReplicate = !!process.env.REPLICATE_API_TOKEN;
const hasJimeng = !!(process.env.JIMENG_ACCESS_KEY_ID && process.env.JIMENG_SECRET_ACCESS_KEY);
const hasSeedream = !!process.env.ARK_API_KEY;
@@ -621,6 +634,13 @@ export function detectProvider(args: CliArgs): Provider {
return "seedream";
}
if (modelProvider === "minimax") {
if (!hasMinimax) {
throw new Error("Model looks like a MiniMax image model, but MINIMAX_API_KEY is not set.");
}
return "minimax";
}
if (args.referenceImages.length > 0) {
if (hasGoogle) return "google";
if (hasOpenai) return "openai";
@@ -628,8 +648,9 @@ export function detectProvider(args: CliArgs): Provider {
if (hasOpenrouter) return "openrouter";
if (hasReplicate) return "replicate";
if (hasSeedream) return "seedream";
if (hasMinimax) return "minimax";
throw new Error(
"Reference images require Google, OpenAI, Azure, OpenRouter, Replicate, or supported Seedream models. Set GOOGLE_API_KEY/GEMINI_API_KEY, OPENAI_API_KEY, AZURE_OPENAI_API_KEY+AZURE_OPENAI_BASE_URL, OPENROUTER_API_KEY, REPLICATE_API_TOKEN, or ARK_API_KEY, or remove --ref."
"Reference images require Google, OpenAI, Azure, OpenRouter, Replicate, supported Seedream models, or MiniMax. Set GOOGLE_API_KEY/GEMINI_API_KEY, OPENAI_API_KEY, AZURE_OPENAI_API_KEY+AZURE_OPENAI_BASE_URL, OPENROUTER_API_KEY, REPLICATE_API_TOKEN, ARK_API_KEY, or MINIMAX_API_KEY, or remove --ref."
);
}
@@ -639,6 +660,7 @@ export function detectProvider(args: CliArgs): Provider {
hasAzure && "azure",
hasOpenrouter && "openrouter",
hasDashscope && "dashscope",
hasMinimax && "minimax",
hasReplicate && "replicate",
hasJimeng && "jimeng",
hasSeedream && "seedream",
@@ -648,7 +670,7 @@ export function detectProvider(args: CliArgs): Provider {
if (available.length > 1) return available[0]!;
throw new Error(
"No API key found. Set GOOGLE_API_KEY, GEMINI_API_KEY, OPENAI_API_KEY, AZURE_OPENAI_API_KEY+AZURE_OPENAI_BASE_URL, OPENROUTER_API_KEY, DASHSCOPE_API_KEY, REPLICATE_API_TOKEN, JIMENG keys, or ARK_API_KEY.\n" +
"No API key found. Set GOOGLE_API_KEY, GEMINI_API_KEY, OPENAI_API_KEY, AZURE_OPENAI_API_KEY+AZURE_OPENAI_BASE_URL, OPENROUTER_API_KEY, DASHSCOPE_API_KEY, MINIMAX_API_KEY, REPLICATE_API_TOKEN, JIMENG keys, or ARK_API_KEY.\n" +
"Create ~/.baoyu-skills/.env or <cwd>/.baoyu-skills/.env with your keys."
);
}
@@ -687,6 +709,7 @@ export function isRetryableGenerationError(error: unknown): boolean {
async function loadProviderModule(provider: Provider): Promise<ProviderModule> {
if (provider === "google") return (await import("./providers/google")) as ProviderModule;
if (provider === "dashscope") return (await import("./providers/dashscope")) as ProviderModule;
if (provider === "minimax") return (await import("./providers/minimax")) as ProviderModule;
if (provider === "replicate") return (await import("./providers/replicate")) as ProviderModule;
if (provider === "openrouter") return (await import("./providers/openrouter")) as ProviderModule;
if (provider === "jimeng") return (await import("./providers/jimeng")) as ProviderModule;
@@ -717,6 +740,7 @@ function getModelForProvider(
return extendConfig.default_model.openrouter;
}
if (provider === "dashscope" && extendConfig.default_model.dashscope) return extendConfig.default_model.dashscope;
if (provider === "minimax" && extendConfig.default_model.minimax) return extendConfig.default_model.minimax;
if (provider === "replicate" && extendConfig.default_model.replicate) return extendConfig.default_model.replicate;
if (provider === "jimeng" && extendConfig.default_model.jimeng) return extendConfig.default_model.jimeng;
if (provider === "seedream" && extendConfig.default_model.seedream) return extendConfig.default_model.seedream;
@@ -136,7 +136,7 @@ test("Azure image generation routes model to deployment and sends mapped quality
});
test("Azure image edits include quality in multipart requests", async (t) => {
const root = await makeTempDir("baoyu-image-gen-azure-");
const root = await makeTempDir("baoyu-imagine-azure-");
t.after(() => fs.rm(root, { recursive: true, force: true }));
const pngPath = path.join(root, "ref.png");
@@ -421,7 +421,7 @@ export async function generateImage(
if (args.referenceImages.length > 0) {
throw new Error(
"Reference images are not supported with DashScope provider in baoyu-image-gen. Use --provider google with a Gemini multimodal model."
"Reference images are not supported with DashScope provider in baoyu-imagine. Use --provider google with a Gemini multimodal model."
);
}
@@ -0,0 +1,114 @@
import assert from "node:assert/strict";
import test, { type TestContext } from "node:test";
import type { CliArgs } from "../types.ts";
import { generateImage } from "./jimeng.ts";
function makeArgs(overrides: Partial<CliArgs> = {}): CliArgs {
return {
prompt: null,
promptFiles: [],
imagePath: null,
provider: null,
model: null,
aspectRatio: null,
size: null,
quality: null,
imageSize: null,
referenceImages: [],
n: 1,
batchFile: null,
jobs: null,
json: false,
help: false,
...overrides,
};
}
function useEnv(
t: TestContext,
values: Record<string, string | null>,
): void {
const previous = new Map<string, string | undefined>();
for (const [key, value] of Object.entries(values)) {
previous.set(key, process.env[key]);
if (value == null) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
t.after(() => {
for (const [key, value] of previous.entries()) {
if (value == null) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
});
}
test("Jimeng submit request uses prompt field expected by current API", async (t) => {
useEnv(t, {
JIMENG_ACCESS_KEY_ID: "test-access-key",
JIMENG_SECRET_ACCESS_KEY: "test-secret-key",
JIMENG_BASE_URL: null,
JIMENG_REGION: null,
});
const originalFetch = globalThis.fetch;
t.after(() => {
globalThis.fetch = originalFetch;
});
const calls: Array<{
input: string;
init?: RequestInit;
}> = [];
globalThis.fetch = async (input, init) => {
calls.push({
input: String(input),
init,
});
if (calls.length === 1) {
return Response.json({
code: 10000,
data: {
task_id: "task-123",
},
});
}
return Response.json({
code: 10000,
data: {
status: "done",
binary_data_base64: [Buffer.from("jimeng-image").toString("base64")],
},
});
};
const image = await generateImage(
"A quiet bamboo forest",
"jimeng_t2i_v40",
makeArgs({ quality: "normal" }),
);
assert.equal(Buffer.from(image).toString("utf8"), "jimeng-image");
assert.equal(calls.length, 2);
assert.equal(
calls[0]?.input,
"https://visual.volcengineapi.com/?Action=CVSync2AsyncSubmitTask&Version=2022-08-31",
);
const submitBody = JSON.parse(String(calls[0]?.init?.body)) as Record<string, unknown>;
assert.equal(submitBody.req_key, "jimeng_t2i_v40");
assert.equal(submitBody.prompt, "A quiet bamboo forest");
assert.ok(!("prompt_text" in submitBody));
assert.equal(submitBody.width, 1024);
assert.equal(submitBody.height, 1024);
});
@@ -246,7 +246,7 @@ async function submitTask(
const [width, height] = size.split("x").map(Number);
const bodyObj = {
req_key: model,
prompt_text: prompt,
prompt,
// Use separate width and height parameters instead of size string
width: width,
height: height,
@@ -0,0 +1,171 @@
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import test, { type TestContext } from "node:test";
import type { CliArgs } from "../types.ts";
import {
buildMinimaxUrl,
buildRequestBody,
buildSubjectReference,
extractImageFromResponse,
parsePixelSize,
validateArgs,
} from "./minimax.ts";
function useEnv(
t: TestContext,
values: Record<string, string | null>,
): void {
const previous = new Map<string, string | undefined>();
for (const [key, value] of Object.entries(values)) {
previous.set(key, process.env[key]);
if (value == null) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
t.after(() => {
for (const [key, value] of previous.entries()) {
if (value == null) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
});
}
function makeArgs(overrides: Partial<CliArgs> = {}): CliArgs {
return {
prompt: null,
promptFiles: [],
imagePath: null,
provider: null,
model: null,
aspectRatio: null,
size: null,
quality: null,
imageSize: null,
referenceImages: [],
n: 1,
batchFile: null,
jobs: null,
json: false,
help: false,
...overrides,
};
}
test("MiniMax URL builder normalizes /v1 suffixes", (t) => {
useEnv(t, { MINIMAX_BASE_URL: "https://api.minimax.io" });
assert.equal(buildMinimaxUrl(), "https://api.minimax.io/v1/image_generation");
process.env.MINIMAX_BASE_URL = "https://proxy.example.com/custom/v1/";
assert.equal(buildMinimaxUrl(), "https://proxy.example.com/custom/v1/image_generation");
});
test("MiniMax size parsing and validation follow documented constraints", () => {
assert.deepEqual(parsePixelSize("1536x1024"), { width: 1536, height: 1024 });
assert.deepEqual(parsePixelSize("1536*1024"), { width: 1536, height: 1024 });
assert.equal(parsePixelSize("wide"), null);
validateArgs("image-01", makeArgs({ size: "1536x1024", n: 9 }));
assert.throws(
() => validateArgs("image-01-live", makeArgs({ size: "1536x1024" })),
/only supported with model image-01/,
);
assert.throws(
() => validateArgs("image-01", makeArgs({ size: "1537x1024" })),
/divisible by 8/,
);
assert.throws(
() => validateArgs("image-01", makeArgs({ aspectRatio: "2.35:1" })),
/aspect_ratio must be one of/,
);
assert.throws(
() => validateArgs("image-01", makeArgs({ n: 10 })),
/at most 9 images/,
);
});
test("MiniMax request body maps aspect ratio, size, n, and subject references", async (t) => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "minimax-test-"));
t.after(() => fs.rm(dir, { recursive: true, force: true }));
const refPath = path.join(dir, "portrait.png");
await fs.writeFile(refPath, Buffer.from("portrait"));
const ratioBody = await buildRequestBody(
"A portrait by the window",
"image-01",
makeArgs({ aspectRatio: "16:9", n: 2, referenceImages: [refPath] }),
);
assert.equal(ratioBody.aspect_ratio, "16:9");
assert.equal(ratioBody.n, 2);
assert.equal(ratioBody.response_format, "base64");
assert.match(ratioBody.subject_reference?.[0]?.image_file || "", /^data:image\/png;base64,/);
const sizeBody = await buildRequestBody(
"A portrait by the window",
"image-01",
makeArgs({ size: "1536x1024" }),
);
assert.equal(sizeBody.width, 1536);
assert.equal(sizeBody.height, 1024);
assert.equal(sizeBody.aspect_ratio, undefined);
});
test("MiniMax subject references require supported file types", async (t) => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "minimax-ref-"));
t.after(() => fs.rm(dir, { recursive: true, force: true }));
const good = path.join(dir, "portrait.jpg");
const bad = path.join(dir, "portrait.webp");
await fs.writeFile(good, Buffer.from("portrait"));
await fs.writeFile(bad, Buffer.from("portrait"));
const subjectReference = await buildSubjectReference([good]);
assert.equal(subjectReference?.[0]?.type, "character");
await assert.rejects(
() => buildSubjectReference([bad]),
/only supports JPG, JPEG, or PNG/,
);
});
test("MiniMax response extraction supports base64 and URL payloads", async (t) => {
const originalFetch = globalThis.fetch;
t.after(() => {
globalThis.fetch = originalFetch;
});
const fromBase64 = await extractImageFromResponse({
data: {
image_base64: [Buffer.from("hello").toString("base64")],
},
});
assert.equal(Buffer.from(fromBase64).toString("utf8"), "hello");
globalThis.fetch = async () =>
new Response(Uint8Array.from([1, 2, 3]), {
status: 200,
headers: { "Content-Type": "image/jpeg" },
});
const fromUrl = await extractImageFromResponse({
data: {
image_urls: ["https://example.com/output.jpg"],
},
});
assert.deepEqual([...fromUrl], [1, 2, 3]);
await assert.rejects(
() => extractImageFromResponse({ base_resp: { status_code: 1001, status_msg: "blocked" } }),
/blocked/,
);
});
@@ -0,0 +1,220 @@
import path from "node:path";
import { readFile } from "node:fs/promises";
import type { CliArgs } from "../types";
const DEFAULT_MODEL = "image-01";
const MAX_REFERENCE_IMAGE_BYTES = 10 * 1024 * 1024;
const SUPPORTED_ASPECT_RATIOS = new Set(["1:1", "16:9", "4:3", "3:2", "2:3", "3:4", "9:16", "21:9"]);
type MinimaxSubjectReference = {
type: "character";
image_file: string;
};
type MinimaxRequestBody = {
model: string;
prompt: string;
response_format: "base64";
aspect_ratio?: string;
width?: number;
height?: number;
n?: number;
subject_reference?: MinimaxSubjectReference[];
};
type MinimaxResponse = {
id?: string;
data?: {
image_urls?: string[];
image_base64?: string[];
};
base_resp?: {
status_code?: number;
status_msg?: string;
};
};
export function getDefaultModel(): string {
return process.env.MINIMAX_IMAGE_MODEL || DEFAULT_MODEL;
}
function getApiKey(): string | null {
return process.env.MINIMAX_API_KEY || null;
}
export function buildMinimaxUrl(): string {
const base = (process.env.MINIMAX_BASE_URL || "https://api.minimax.io").replace(/\/+$/g, "");
return base.endsWith("/v1") ? `${base}/image_generation` : `${base}/v1/image_generation`;
}
function getMimeType(filename: string): "image/jpeg" | "image/png" {
const ext = path.extname(filename).toLowerCase();
if (ext === ".jpg" || ext === ".jpeg") return "image/jpeg";
if (ext === ".png") return "image/png";
throw new Error(
`MiniMax subject_reference only supports JPG, JPEG, or PNG files: ${filename}`
);
}
export function parsePixelSize(size: string): { width: number; height: number } | null {
const match = size.trim().match(/^(\d+)\s*[xX*]\s*(\d+)$/);
if (!match) return null;
const width = parseInt(match[1]!, 10);
const height = parseInt(match[2]!, 10);
if (!Number.isFinite(width) || !Number.isFinite(height) || width <= 0 || height <= 0) {
return null;
}
return { width, height };
}
function validatePixelSize(width: number, height: number): void {
if (width < 512 || width > 2048 || height < 512 || height > 2048) {
throw new Error("MiniMax custom size must keep width and height between 512 and 2048.");
}
if (width % 8 !== 0 || height % 8 !== 0) {
throw new Error("MiniMax custom size requires width and height divisible by 8.");
}
}
export function validateArgs(model: string, args: CliArgs): void {
if (args.n > 9) {
throw new Error("MiniMax supports at most 9 images per request.");
}
if (args.aspectRatio && !SUPPORTED_ASPECT_RATIOS.has(args.aspectRatio)) {
throw new Error(
`MiniMax aspect_ratio must be one of: ${Array.from(SUPPORTED_ASPECT_RATIOS).join(", ")}.`
);
}
if (args.size && !args.aspectRatio) {
if (model !== "image-01") {
throw new Error("MiniMax custom --size is only supported with model image-01. Use --model image-01 or pass --ar instead.");
}
const parsed = parsePixelSize(args.size);
if (!parsed) {
throw new Error("MiniMax --size must be in WxH format, for example 1536x1024.");
}
validatePixelSize(parsed.width, parsed.height);
}
}
export async function buildSubjectReference(
referenceImages: string[],
): Promise<MinimaxSubjectReference[] | undefined> {
if (referenceImages.length === 0) return undefined;
const subjectReference: MinimaxSubjectReference[] = [];
for (const refPath of referenceImages) {
const bytes = await readFile(refPath);
if (bytes.length > MAX_REFERENCE_IMAGE_BYTES) {
throw new Error(`MiniMax subject_reference images must be smaller than 10MB: ${refPath}`);
}
subjectReference.push({
type: "character",
image_file: `data:${getMimeType(refPath)};base64,${bytes.toString("base64")}`,
});
}
return subjectReference;
}
export async function buildRequestBody(
prompt: string,
model: string,
args: CliArgs,
): Promise<MinimaxRequestBody> {
validateArgs(model, args);
const body: MinimaxRequestBody = {
model,
prompt,
response_format: "base64",
};
if (args.aspectRatio) {
body.aspect_ratio = args.aspectRatio;
} else if (args.size) {
const parsed = parsePixelSize(args.size);
if (!parsed) {
throw new Error("MiniMax --size must be in WxH format, for example 1536x1024.");
}
body.width = parsed.width;
body.height = parsed.height;
}
if (args.n > 1) {
body.n = args.n;
}
const subjectReference = await buildSubjectReference(args.referenceImages);
if (subjectReference) {
body.subject_reference = subjectReference;
}
return body;
}
async function downloadImage(url: string): Promise<Uint8Array> {
const response = await fetch(url);
if (!response.ok) {
throw new Error(`Failed to download image from MiniMax: ${response.status}`);
}
return new Uint8Array(await response.arrayBuffer());
}
export async function extractImageFromResponse(result: MinimaxResponse): Promise<Uint8Array> {
const baseResp = result.base_resp;
if (baseResp && baseResp.status_code !== undefined && baseResp.status_code !== 0) {
throw new Error(baseResp.status_msg || `MiniMax API returned status_code=${baseResp.status_code}`);
}
const base64Image = result.data?.image_base64?.[0];
if (base64Image) {
return Uint8Array.from(Buffer.from(base64Image, "base64"));
}
const url = result.data?.image_urls?.[0];
if (url) {
return downloadImage(url);
}
throw new Error("No image data in MiniMax response");
}
export function getDefaultOutputExtension(): ".jpg" {
return ".jpg";
}
export async function generateImage(
prompt: string,
model: string,
args: CliArgs
): Promise<Uint8Array> {
const apiKey = getApiKey();
if (!apiKey) {
throw new Error("MINIMAX_API_KEY is required. Get one from https://platform.minimax.io/");
}
const body = await buildRequestBody(prompt, model, args);
const response = await fetch(buildMinimaxUrl(), {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify(body),
});
if (!response.ok) {
const err = await response.text();
throw new Error(`MiniMax API error (${response.status}): ${err}`);
}
const result = (await response.json()) as MinimaxResponse;
return extractImageFromResponse(result);
}
@@ -1,4 +1,13 @@
export type Provider = "google" | "openai" | "openrouter" | "dashscope" | "replicate" | "jimeng" | "seedream" | "azure";
export type Provider =
| "google"
| "openai"
| "openrouter"
| "dashscope"
| "minimax"
| "replicate"
| "jimeng"
| "seedream"
| "azure";
export type Quality = "normal" | "2k";
export type CliArgs = {
@@ -52,6 +61,7 @@ export type ExtendConfig = {
openai: string | null;
openrouter: string | null;
dashscope: string | null;
minimax: string | null;
replicate: string | null;
jimeng: string | null;
seedream: string | null;
+51 -3
View File
@@ -118,6 +118,7 @@ Full reference: [references/config/first-time-setup.md](references/config/first-
## Features
- Chrome CDP for full JavaScript rendering
- Browser strategy fallback: default headless first, then visible Chrome on technical failure
- URL-specific parser layer for sites that need custom HTML rules before generic extraction
- Two capture modes: auto or wait-for-user
- Save rendered HTML as a sibling `-captured.html` file
@@ -137,6 +138,12 @@ Full reference: [references/config/first-time-setup.md](references/config/first-
# Auto mode (default) - capture when page loads
${BUN_X} {baseDir}/scripts/main.ts <url>
# Force headless only
${BUN_X} {baseDir}/scripts/main.ts <url> --browser headless
# Force visible browser
${BUN_X} {baseDir}/scripts/main.ts <url> --browser headed
# Wait mode - wait for user signal before capture
${BUN_X} {baseDir}/scripts/main.ts <url> --wait
@@ -158,6 +165,9 @@ ${BUN_X} {baseDir}/scripts/main.ts <url> --download-media
| `-o <path>` | Output file path — must be a **file** path, not directory (default: auto-generated) |
| `--output-dir <dir>` | Base output directory — auto-generates `{dir}/{domain}/{slug}.md` (default: `./url-to-markdown/`) |
| `--wait` | Wait for user signal before capturing |
| `--browser <mode>` | Browser strategy: `auto` (default), `headless`, or `headed` |
| `--headless` | Shortcut for `--browser headless` |
| `--headed` | Shortcut for `--browser headed` |
| `--timeout <ms>` | Page load timeout (default: 30000) |
| `--download-media` | Download image/video assets to local `imgs/` and `videos/`, and rewrite markdown links to local relative paths |
@@ -165,7 +175,7 @@ ${BUN_X} {baseDir}/scripts/main.ts <url> --download-media
| Mode | Behavior | Use When |
|------|----------|----------|
| Auto (default) | Capture on network idle | Public pages, static content |
| Auto (default) | Try headless first, then retry in visible Chrome if needed | Public pages, static content, unknown pages |
| Wait (`--wait`) | User signals when ready | Login-required, lazy loading, paywalls |
**Wait mode workflow**:
@@ -173,6 +183,43 @@ ${BUN_X} {baseDir}/scripts/main.ts <url> --download-media
2. Ask user to confirm page is ready
3. Send newline to stdin to trigger capture
**Default browser fallback**:
1. Auto mode starts with headless Chrome and captures on network idle
2. If headless capture fails technically, retry with visible Chrome
3. If a shared Chrome session for this profile already exists, reuse it instead of launching a new browser
4. The script does not hard-code login or paywall detection; the agent must inspect the captured markdown or HTML and decide whether to rerun with `--browser headed --wait`
## Agent Quality Gate
**CRITICAL**: The agent must treat headless capture as provisional. Some sites render differently in headless mode and can silently return an error shell, partially hydrated page, or low-quality extraction **without** causing the CLI to fail.
After every run that used `--browser auto` or `--browser headless`, the agent **MUST** inspect the saved markdown first, and inspect the saved `-captured.html` when the markdown looks suspicious.
### Quality checks the agent must perform
1. Confirm the markdown title matches the target page, not a generic site shell
2. Confirm the body contains the expected article or page content, not just navigation, footer, or a generic error
3. Watch for obvious failure signs such as:
- `Application error`
- `This page could not be found`
- login, signup, subscribe, or verification shells
- extremely short markdown for a page that should be long-form
- raw framework payloads or mostly boilerplate content
4. If the result is low quality, incomplete, or clearly wrong, do **not** accept the run as successful just because the CLI exited with code 0
### Recovery workflow the agent must follow
1. First run with default `auto` unless there is already a clear reason to use wait mode
2. Review markdown quality immediately after the run
3. If the content is low quality, rerun locally with visible Chrome:
- `--browser headed` for ordinary rendering issues
- `--browser headed --wait` when the page may need login, anti-bot interaction, cookie acceptance, or extra hydration time
4. If `--wait` is used, tell the user exactly what to do:
- if login is required, ask them to sign in
- if the page needs time to hydrate, ask them to wait until the full content is visible
- once ready, ask them to press Enter so capture can continue
5. Only fall back to hosted `defuddle.md` after the local browser strategies have failed or are clearly lower fidelity
## Output Format
Each run saves two files side by side:
@@ -211,8 +258,9 @@ Conversion order:
2. If no specialized parser matches, try Defuddle
3. For rich pages such as YouTube, prefer Defuddle's extractor-specific output (including transcripts when available) instead of replacing it with the legacy pipeline
4. If Defuddle throws, cannot load, returns obviously incomplete markdown, or captures lower-quality content than the legacy pipeline, automatically fall back to the pre-Defuddle extractor
5. If the entire local browser capture flow fails before markdown can be produced, try the hosted `https://defuddle.md/<url>` API and save its markdown output directly
6. The legacy fallback path uses the older Readability/selector/Next.js-data based HTML-to-Markdown implementation recovered from git history
5. If the agent determines the captured result is a login screen, verification screen, or paywall shell, rerun locally with `--browser headed --wait` and ask the user to complete access before capture
6. If the entire local browser capture flow still fails before markdown can be produced, try the hosted `https://defuddle.md/<url>` API and save its markdown output directly
7. The legacy fallback path uses the older Readability/selector/Next.js-data based HTML-to-Markdown implementation recovered from git history
CLI output will show:
@@ -6,7 +6,7 @@
"dependencies": {
"@mozilla/readability": "^0.6.0",
"baoyu-chrome-cdp": "file:./vendor/baoyu-chrome-cdp",
"defuddle": "^0.12.0",
"defuddle": "^0.14.0",
"jsdom": "^24.1.3",
"linkedom": "^0.18.12",
"turndown": "^7.2.2",
@@ -61,7 +61,7 @@
"decimal.js": ["decimal.js@10.6.0", "", {}, "sha512-YpgQiITW3JXGntzdUmyUR1V812Hn8T1YVXhCu+wO3OpS4eU9l4YdD3qjyiKdV6mvV29zapkMeD390UVEf2lkUg=="],
"defuddle": ["defuddle@0.12.0", "", { "dependencies": { "commander": "^12.1.0" }, "optionalDependencies": { "mathml-to-latex": "^1.5.0", "temml": "^0.13.1", "turndown": "^7.2.0" }, "peerDependencies": { "jsdom": "^24.0.0" }, "bin": { "defuddle": "dist/cli.js" } }, "sha512-Y/WgyGKBxwxFir+hWNth4nmWDDDb8BzQi3qASS2NWYPXsKU42Ku49/3M5yFYefnRef9prynnmasfnXjk99EWgA=="],
"defuddle": ["defuddle@0.14.0", "", { "dependencies": { "commander": "^12.1.0" }, "optionalDependencies": { "linkedom": "^0.18.12", "mathml-to-latex": "^1.5.0", "temml": "^0.13.1", "turndown": "^7.2.0" }, "bin": { "defuddle": "dist/cli.js" } }, "sha512-btavZGd1WgiVqrVM62WGRXMUi/aU7ckTZiq0xXWLZMHvzIqNZjwIFQEDRx8MarD7fIgsB90NXZ9xHJkKtapt2Q=="],
"delayed-stream": ["delayed-stream@1.0.0", "", {}, "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ=="],
@@ -0,0 +1,55 @@
import assert from "node:assert/strict";
import test from "node:test";
import { cleanContent } from "./content-cleaner.js";
const SAMPLE_HTML = `<!doctype html>
<html>
<head>
<title>Example Story</title>
<style>.cookie-banner { position: fixed; }</style>
<script>window.__noise = true;</script>
</head>
<body>
<!-- comment that should be removed -->
<header>
<nav>
<a href="/home">Home</a>
<a href="/topics">Topics</a>
</nav>
</header>
<div class="cookie-banner">Accept cookies</div>
<aside>Sidebar links</aside>
<main>
<article class="content">
<h1>Actual Story Title</h1>
<p>
This is the first paragraph of the real story body, and it is intentionally long enough
to survive the cleaner's main-content heuristics without being mistaken for navigation.
</p>
<p>
This is the second paragraph with more useful detail, a
<a href="/read-more">supporting link</a>, and a normal image.
</p>
<img src="/images/cover.jpg" alt="Cover">
<img src="data:image/png;base64,AAAA" alt="Inline data">
</article>
</main>
<footer>Footer boilerplate</footer>
</body>
</html>`;
test("cleanContent keeps the article body and removes obvious boilerplate", () => {
const cleaned = cleanContent(SAMPLE_HTML, "https://example.com/posts/story");
assert.match(cleaned, /Actual Story Title/);
assert.match(cleaned, /https:\/\/example\.com\/read-more/);
assert.match(cleaned, /https:\/\/example\.com\/images\/cover\.jpg/);
assert.doesNotMatch(cleaned, /Accept cookies/);
assert.doesNotMatch(cleaned, /Sidebar links/);
assert.doesNotMatch(cleaned, /Footer boilerplate/);
assert.doesNotMatch(cleaned, /window\.__noise/);
assert.doesNotMatch(cleaned, /comment that should be removed/);
assert.doesNotMatch(cleaned, /data:image\/png;base64/);
});
@@ -0,0 +1,432 @@
import { parseHTML } from "linkedom";
export interface CleaningOptions {
removeAds?: boolean;
removeBase64Images?: boolean;
onlyMainContent?: boolean;
includeTags?: string[];
excludeTags?: string[];
}
const ALWAYS_REMOVE_SELECTORS = [
"script",
"style",
"noscript",
"link[rel='stylesheet']",
"[hidden]",
"[aria-hidden='true']",
"[style*='display: none']",
"[style*='display:none']",
"[style*='visibility: hidden']",
"[style*='visibility:hidden']",
"svg[aria-hidden='true']",
"svg.icon",
"svg[class*='icon']",
"template",
"meta",
"iframe",
"canvas",
"object",
"embed",
"form",
"input",
"select",
"textarea",
"button",
];
const OVERLAY_SELECTORS = [
"[class*='modal']",
"[class*='popup']",
"[class*='overlay']",
"[class*='dialog']",
"[role='dialog']",
"[role='alertdialog']",
"[class*='cookie']",
"[class*='consent']",
"[class*='gdpr']",
"[class*='privacy-banner']",
"[class*='notification-bar']",
"[id*='cookie']",
"[id*='consent']",
"[id*='gdpr']",
"[style*='position: fixed']",
"[style*='position:fixed']",
"[style*='position: sticky']",
"[style*='position:sticky']",
];
const NAVIGATION_SELECTORS = [
"header",
"footer",
"nav",
"aside",
".header",
".top",
".navbar",
"#header",
".footer",
".bottom",
"#footer",
".sidebar",
".side",
".aside",
"#sidebar",
".modal",
".popup",
"#modal",
".overlay",
".ad",
".ads",
".advert",
"#ad",
".lang-selector",
".language",
"#language-selector",
".social",
".social-media",
".social-links",
"#social",
".menu",
".navigation",
"#nav",
".breadcrumbs",
"#breadcrumbs",
".share",
"#share",
".widget",
"#widget",
".cookie",
"#cookie",
];
const FORCE_INCLUDE_SELECTORS = [
"#main",
"#content",
"#main-content",
"#article",
"#post",
"#page-content",
"main",
"article",
"[role='main']",
".main-content",
".content",
".post-content",
".article-content",
".entry-content",
".page-content",
".article-body",
".post-body",
".story-content",
".blog-content",
];
const AD_SELECTORS = [
"ins.adsbygoogle",
".google-ad",
".adsense",
"[data-ad]",
"[data-ads]",
"[data-ad-slot]",
"[data-ad-client]",
".ad-container",
".ad-wrapper",
".advertisement",
".sponsored-content",
"img[width='1'][height='1']",
"img[src*='pixel']",
"img[src*='tracking']",
"img[src*='analytics']",
];
function getLinkDensity(element: Element): number {
const text = element.textContent || "";
const textLength = text.trim().length;
if (textLength === 0) return 1;
let linkLength = 0;
element.querySelectorAll("a").forEach((link: Element) => {
linkLength += (link.textContent || "").trim().length;
});
return linkLength / textLength;
}
function getContentScore(element: Element): number {
let score = 0;
const text = element.textContent || "";
const textLength = text.trim().length;
score += Math.min(textLength / 100, 50);
score += element.querySelectorAll("p").length * 3;
score += element.querySelectorAll("h1, h2, h3, h4, h5, h6").length * 2;
score += element.querySelectorAll("img").length;
score -= element.querySelectorAll("a").length * 0.5;
score -= element.querySelectorAll("li").length * 0.2;
const linkDensity = getLinkDensity(element);
if (linkDensity > 0.5) score -= 30;
else if (linkDensity > 0.3) score -= 15;
const className = typeof element.className === "string" ? element.className : "";
const classAndId = `${className} ${element.id || ""}`;
if (/article|content|post|body|main|entry/i.test(classAndId)) score += 25;
if (/comment|sidebar|footer|nav|menu|header|widget|ad/i.test(classAndId)) score -= 25;
return score;
}
function looksLikeNavigation(element: Element): boolean {
const linkDensity = getLinkDensity(element);
if (linkDensity > 0.5) return true;
const listItems = element.querySelectorAll("li");
const links = element.querySelectorAll("a");
return listItems.length > 5 && links.length > listItems.length * 0.8;
}
function removeElements(document: Document, selectors: string[]): void {
for (const selector of selectors) {
try {
document.querySelectorAll(selector).forEach((element: Element) => element.remove());
} catch {
// Ignore unsupported selectors from linkedom/jsdom differences.
}
}
}
function removeWithProtection(
document: Document,
selectorsToRemove: string[],
protectedSelectors: string[]
): void {
for (const selector of selectorsToRemove) {
try {
document.querySelectorAll(selector).forEach((element: Element) => {
const isProtected = protectedSelectors.some((protectedSelector) => {
try {
return element.matches(protectedSelector);
} catch {
return false;
}
});
if (isProtected) return;
const containsProtected = protectedSelectors.some((protectedSelector) => {
try {
return element.querySelector(protectedSelector) !== null;
} catch {
return false;
}
});
if (containsProtected) return;
element.remove();
});
} catch {
// Ignore unsupported selectors from linkedom/jsdom differences.
}
}
}
function findMainContent(document: Document): Element | null {
const isValidContent = (element: Element | null): element is Element => {
if (!element) return false;
const text = element.textContent || "";
if (text.trim().length < 100) return false;
return !looksLikeNavigation(element);
};
const main = document.querySelector("main");
if (isValidContent(main) && getLinkDensity(main) < 0.4) return main;
const roleMain = document.querySelector('[role="main"]');
if (isValidContent(roleMain) && getLinkDensity(roleMain) < 0.4) return roleMain;
const articles = document.querySelectorAll("article");
if (articles.length === 1 && isValidContent(articles[0] ?? null)) {
return articles[0] ?? null;
}
const contentSelectors = [
"#content",
"#main-content",
"#main",
".content",
".main-content",
".post-content",
".article-content",
".entry-content",
".page-content",
".article-body",
".post-body",
".story-content",
".blog-content",
];
for (const selector of contentSelectors) {
try {
const element = document.querySelector(selector);
if (isValidContent(element) && getLinkDensity(element) < 0.4) {
return element;
}
} catch {
// Ignore invalid selectors.
}
}
const candidates: Array<{ element: Element; score: number }> = [];
const containers = document.querySelectorAll("div, section, article");
containers.forEach((element: Element) => {
const text = element.textContent || "";
if (text.trim().length < 200) return;
const score = getContentScore(element);
if (score > 0) {
candidates.push({ element, score });
}
});
candidates.sort((left, right) => right.score - left.score);
if ((candidates[0]?.score ?? 0) > 20) {
return candidates[0]?.element ?? null;
}
return null;
}
function removeBase64ImagesFromDocument(document: Document): void {
document.querySelectorAll("img[src^='data:']").forEach((element: Element) => {
element.remove();
});
document.querySelectorAll("[style*='data:image']").forEach((element: Element) => {
const style = element.getAttribute("style");
if (!style) return;
const cleanedStyle = style.replace(
/background(-image)?:\s*url\([^)]*data:image[^)]*\)[^;]*;?/gi,
""
);
if (cleanedStyle.trim()) {
element.setAttribute("style", cleanedStyle);
} else {
element.removeAttribute("style");
}
});
document.querySelectorAll("source[src^='data:'], source[srcset*='data:']").forEach((element: Element) => {
element.remove();
});
}
function makeAbsoluteUrl(value: string, baseUrl: string): string | null {
try {
return new URL(value, baseUrl).toString();
} catch {
return null;
}
}
function convertRelativeUrls(document: Document, baseUrl: string): void {
document.querySelectorAll("[src]").forEach((element: Element) => {
const src = element.getAttribute("src");
if (!src || src.startsWith("http") || src.startsWith("//") || src.startsWith("data:")) return;
const absolute = makeAbsoluteUrl(src, baseUrl);
if (absolute) element.setAttribute("src", absolute);
});
document.querySelectorAll("[href]").forEach((element: Element) => {
const href = element.getAttribute("href");
if (
!href ||
href.startsWith("http") ||
href.startsWith("//") ||
href.startsWith("#") ||
href.startsWith("mailto:") ||
href.startsWith("tel:") ||
href.startsWith("javascript:")
) {
return;
}
const absolute = makeAbsoluteUrl(href, baseUrl);
if (absolute) element.setAttribute("href", absolute);
});
}
export function cleanHtml(html: string, baseUrl: string, options: CleaningOptions = {}): string {
const {
removeAds = true,
removeBase64Images = true,
onlyMainContent = true,
includeTags,
excludeTags,
} = options;
const { document } = parseHTML(html);
removeElements(document, ALWAYS_REMOVE_SELECTORS);
removeElements(document, OVERLAY_SELECTORS);
if (removeAds) {
removeElements(document, AD_SELECTORS);
}
if (excludeTags?.length) {
removeElements(document, excludeTags);
}
if (onlyMainContent) {
removeWithProtection(document, NAVIGATION_SELECTORS, FORCE_INCLUDE_SELECTORS);
const mainContent = findMainContent(document);
if (mainContent && document.body) {
const clone = mainContent.cloneNode(true) as Element;
document.body.innerHTML = "";
document.body.appendChild(clone);
}
}
if (includeTags?.length && document.body) {
const matchedElements: Element[] = [];
for (const selector of includeTags) {
try {
document.querySelectorAll(selector).forEach((element: Element) => {
matchedElements.push(element.cloneNode(true) as Element);
});
} catch {
// Ignore invalid selectors.
}
}
if (matchedElements.length > 0) {
document.body.innerHTML = "";
matchedElements.forEach((element) => document.body?.appendChild(element));
}
}
if (removeBase64Images) {
removeBase64ImagesFromDocument(document);
}
const walker = document.createTreeWalker(document, 128);
const comments: Node[] = [];
while (walker.nextNode()) {
comments.push(walker.currentNode);
}
comments.forEach((comment) => comment.parentNode?.removeChild(comment));
convertRelativeUrls(document, baseUrl);
return document.documentElement?.outerHTML || html;
}
export function cleanContent(html: string, baseUrl: string, options: CleaningOptions = {}): string {
return cleanHtml(html, baseUrl, options);
}
@@ -0,0 +1,28 @@
import assert from "node:assert/strict";
import test from "node:test";
import { extractContent } from "./html-to-markdown.js";
const EMBEDDED_IMAGE_HTML = `<!doctype html>
<html>
<body>
<main>
<article>
<h1>Embedded Image Story</h1>
<p>
This paragraph is intentionally long enough to satisfy the extractor thresholds so the
resulting markdown keeps the main article body and the embedded image reference.
</p>
<img src="data:image/png;base64,AAAA" alt="inline">
</article>
</main>
</body>
</html>`;
test("extractContent preserves base64 images when requested for media download", async () => {
const result = await extractContent(EMBEDDED_IMAGE_HTML, "https://example.com/embedded", {
preserveBase64Images: true,
});
assert.match(result.markdown, /!\[inline\]\(data:image\/png;base64,AAAA\)/);
});
@@ -13,10 +13,15 @@ import {
shouldCompareWithLegacy,
} from "./legacy-converter.js";
import { tryUrlRuleParsers } from "./parsers/index.js";
import { cleanContent } from "./content-cleaner.js";
export type { ConversionResult, PageMetadata };
export { createMarkdownDocument, formatMetadataYaml };
export interface ExtractContentOptions {
preserveBase64Images?: boolean;
}
export const absolutizeUrlsScript = String.raw`
(function() {
const baseUrl = document.baseURI || location.href;
@@ -85,7 +90,10 @@ export const absolutizeUrlsScript = String.raw`
absAttr(htmlClone, "video[poster]", "poster");
absSrcset(htmlClone, "img[srcset], source[srcset]");
return { html: "<!doctype html>\n" + htmlClone.outerHTML };
return {
html: "<!doctype html>\n" + htmlClone.outerHTML,
finalUrl: location.href,
};
})()
`;
@@ -102,7 +110,11 @@ function shouldPreferDefuddle(result: ConversionResult): boolean {
return /^##?\s+transcript\b/im.test(result.markdown);
}
export async function extractContent(html: string, url: string): Promise<ConversionResult> {
export async function extractContent(
html: string,
url: string,
options: ExtractContentOptions = {}
): Promise<ConversionResult> {
const capturedAt = new Date().toISOString();
const baseMetadata = extractMetadataFromHtml(html, url, capturedAt);
@@ -111,14 +123,23 @@ export async function extractContent(html: string, url: string): Promise<Convers
return specializedResult;
}
const defuddleResult = await tryDefuddleConversion(html, url, baseMetadata);
let cleanedHtml = html;
try {
cleanedHtml = cleanContent(html, url, {
removeBase64Images: !options.preserveBase64Images,
});
} catch {
cleanedHtml = html;
}
const defuddleResult = await tryDefuddleConversion(cleanedHtml, url, baseMetadata);
if (defuddleResult.ok) {
if (shouldPreferDefuddle(defuddleResult.result)) {
return defuddleResult.result;
return { ...defuddleResult.result, rawHtml: html };
}
if (shouldCompareWithLegacy(defuddleResult.result.markdown)) {
const legacyResult = convertWithLegacyExtractor(html, baseMetadata);
const legacyResult = convertWithLegacyExtractor(html, baseMetadata, cleanedHtml);
const legacyScore = scoreMarkdownQuality(legacyResult.markdown);
const defuddleScore = scoreMarkdownQuality(defuddleResult.result.markdown);
@@ -130,10 +151,10 @@ export async function extractContent(html: string, url: string): Promise<Convers
}
}
return defuddleResult.result;
return { ...defuddleResult.result, rawHtml: html };
}
const fallbackResult = convertWithLegacyExtractor(html, baseMetadata);
const fallbackResult = convertWithLegacyExtractor(html, baseMetadata, cleanedHtml);
return {
...fallbackResult,
fallbackReason: defuddleResult.reason,
@@ -0,0 +1,48 @@
import assert from "node:assert/strict";
import test from "node:test";
import { cleanContent } from "./content-cleaner.js";
import { convertWithLegacyExtractor } from "./legacy-converter.js";
import { extractMetadataFromHtml } from "./markdown-conversion-shared.js";
const CAPTURED_AT = "2026-03-24T03:00:00.000Z";
const NEXT_DATA_HTML = `<!doctype html>
<html>
<head>
<title>Hydrated Story</title>
</head>
<body>
<div class="cookie-banner">Accept cookies</div>
<main>
<p>Short teaser text that should not win over the structured article payload.</p>
</main>
<script id="__NEXT_DATA__" type="application/json">
{
"props": {
"pageProps": {
"article": {
"title": "Hydrated Story",
"description": "A structured article payload from Next.js",
"body": "<p>The full article lives in __NEXT_DATA__ and should still be extracted even when the cleaned HTML removes scripts before the selector and readability passes run.</p><p>A second paragraph keeps the content comfortably above the minimum extraction threshold and proves the legacy extractor still has access to the original structured payload.</p>"
}
}
}
}
</script>
</body>
</html>`;
test("legacy extractor still uses original __NEXT_DATA__ after HTML cleaning", () => {
const url = "https://example.com/posts/hydrated-story";
const baseMetadata = extractMetadataFromHtml(NEXT_DATA_HTML, url, CAPTURED_AT);
const cleanedHtml = cleanContent(NEXT_DATA_HTML, url);
const result = convertWithLegacyExtractor(NEXT_DATA_HTML, baseMetadata, cleanedHtml);
assert.equal(result.conversionMethod, "legacy:next-data");
assert.match(result.markdown, /The full article lives in .*NEXT.*DATA/);
assert.match(result.markdown, /A second paragraph keeps the content comfortably above the minimum extraction threshold/);
assert.doesNotMatch(result.markdown, /Short teaser text that should not win/);
assert.equal(result.rawHtml, NEXT_DATA_HTML);
});
@@ -336,29 +336,32 @@ function tryNextDataExtraction(document: Document): ExtractionCandidate | null {
function buildReadabilityCandidate(
article: ReturnType<Readability["parse"]>,
document: Document,
referenceDocument: Document,
method: string
): ExtractionCandidate | null {
const textContent = article?.textContent?.trim() ?? "";
if (textContent.length < MIN_CONTENT_LENGTH) return null;
return {
title: pickString(article?.title, extractTitle(document)),
title: pickString(article?.title, extractTitle(referenceDocument)),
byline: pickString((article as { byline?: string } | null)?.byline),
excerpt: pickString(article?.excerpt, generateExcerpt(null, textContent)),
published: pickString((article as { publishedTime?: string } | null)?.publishedTime, extractPublishedTime(document)),
published: pickString(
(article as { publishedTime?: string } | null)?.publishedTime,
extractPublishedTime(referenceDocument)
),
html: article?.content ? sanitizeHtml(article.content) : null,
textContent,
method,
};
}
function tryReadability(document: Document): ExtractionCandidate | null {
function tryReadability(document: Document, referenceDocument: Document = document): ExtractionCandidate | null {
try {
const strictClone = document.cloneNode(true) as Document;
const strictResult = buildReadabilityCandidate(
new Readability(strictClone).parse(),
document,
referenceDocument,
"readability"
);
if (strictResult) return strictResult;
@@ -366,7 +369,7 @@ function tryReadability(document: Document): ExtractionCandidate | null {
const relaxedClone = document.cloneNode(true) as Document;
return buildReadabilityCandidate(
new Readability(relaxedClone, { charThreshold: 120 }).parse(),
document,
referenceDocument,
"readability-relaxed"
);
} catch {
@@ -471,14 +474,15 @@ function pickBestCandidate(candidates: ExtractionCandidate[]): ExtractionCandida
return ranked[0];
}
function extractFromHtml(html: string): ExtractionCandidate | null {
const document = parseDocument(html);
function extractFromHtml(html: string, cleanedHtml: string = html): ExtractionCandidate | null {
const originalDocument = parseDocument(html);
const cleanedDocument = parseDocument(cleanedHtml);
const readabilityCandidate = tryReadability(document);
const nextDataCandidate = tryNextDataExtraction(document);
const jsonLdCandidate = tryJsonLdExtraction(document);
const selectorCandidate = trySelectorExtraction(document);
const bodyCandidate = tryBodyExtraction(document);
const readabilityCandidate = tryReadability(cleanedDocument, originalDocument);
const nextDataCandidate = tryNextDataExtraction(originalDocument);
const jsonLdCandidate = tryJsonLdExtraction(originalDocument);
const selectorCandidate = trySelectorExtraction(cleanedDocument);
const bodyCandidate = tryBodyExtraction(cleanedDocument);
const candidates = [
readabilityCandidate,
@@ -493,8 +497,8 @@ function extractFromHtml(html: string): ExtractionCandidate | null {
return {
...winner,
title: winner.title ?? extractTitle(document),
published: winner.published ?? extractPublishedTime(document),
title: winner.title ?? extractTitle(originalDocument),
published: winner.published ?? extractPublishedTime(originalDocument),
excerpt: winner.excerpt ?? generateExcerpt(null, winner.textContent),
};
}
@@ -610,12 +614,16 @@ export function shouldCompareWithLegacy(markdown: string): boolean {
);
}
export function convertWithLegacyExtractor(html: string, baseMetadata: PageMetadata): ConversionResult {
const extracted = extractFromHtml(html);
export function convertWithLegacyExtractor(
html: string,
baseMetadata: PageMetadata,
cleanedHtml: string = html
): ConversionResult {
const extracted = extractFromHtml(html, cleanedHtml);
let markdown = extracted?.html ? convertHtmlFragmentToMarkdown(extracted.html) : "";
if (!markdown.trim()) {
markdown = extracted?.textContent?.trim() || fallbackPlainText(html);
markdown = extracted?.textContent?.trim() || fallbackPlainText(cleanedHtml);
}
return {
+121 -16
View File
@@ -29,10 +29,33 @@ interface Args {
wait: boolean;
timeout: number;
downloadMedia: boolean;
browserMode: BrowserMode;
}
type BrowserMode = "auto" | "headless" | "headed";
interface CaptureAttemptOptions {
headless: boolean;
wait: boolean;
existingPort?: number;
waitPrompt?: string;
}
interface CaptureSnapshot {
html: string;
finalUrl: string;
}
const BROWSER_MODES = new Set<BrowserMode>(["auto", "headless", "headed"]);
function parseArgs(argv: string[]): Args {
const args: Args = { url: "", wait: false, timeout: DEFAULT_TIMEOUT_MS, downloadMedia: false };
const args: Args = {
url: "",
wait: false,
timeout: DEFAULT_TIMEOUT_MS,
downloadMedia: false,
browserMode: "auto",
};
for (let i = 2; i < argv.length; i++) {
const arg = argv[i];
if (arg === "--wait" || arg === "-w") {
@@ -45,6 +68,12 @@ function parseArgs(argv: string[]): Args {
args.outputDir = argv[++i];
} else if (arg === "--download-media") {
args.downloadMedia = true;
} else if (arg === "--browser") {
args.browserMode = (argv[++i] as BrowserMode | undefined) ?? "auto";
} else if (arg === "--headless") {
args.browserMode = "headless";
} else if (arg === "--headed" || arg === "--noheadless" || arg === "--no-headless") {
args.browserMode = "headed";
} else if (!arg.startsWith("-") && !args.url) {
args.url = arg;
}
@@ -194,21 +223,28 @@ async function generateOutputPath(url: string, title: string, outputDir?: string
return path.join(dataDir, domain, timestampSlug, `${timestampSlug}.md`);
}
async function waitForUserSignal(): Promise<void> {
console.log("Page opened. Press Enter when ready to capture...");
function defaultWaitPrompt(): string {
return "A browser window has been opened. If the page requires login or verification, complete it first, then press Enter to capture.";
}
async function waitForUserSignal(prompt: string): Promise<void> {
console.log(prompt);
const rl = createInterface({ input: process.stdin, output: process.stdout });
await new Promise<void>((resolve) => {
rl.once("line", () => { rl.close(); resolve(); });
});
}
async function captureUrl(args: Args): Promise<ConversionResult> {
const existingPort = await findExistingChromePort();
const reusing = existingPort !== null;
const port = existingPort ?? await getFreePort();
const chrome = reusing ? null : await launchChrome(args.url, port, false);
async function captureUrlOnce(args: Args, options: CaptureAttemptOptions): Promise<ConversionResult> {
const reusing = options.existingPort !== undefined;
const port = options.existingPort ?? await getFreePort();
const chrome = reusing ? null : await launchChrome(args.url, port, options.headless);
if (reusing) console.log(`Reusing existing Chrome on port ${port}`);
if (reusing) {
console.log(`Reusing existing Chrome on port ${port}`);
} else {
console.log(`Launching Chrome (${options.headless ? "headless" : "headed"})...`);
}
let cdp: CdpConnection | null = null;
let targetId: string | null = null;
@@ -235,8 +271,8 @@ async function captureUrl(args: Args): Promise<ConversionResult> {
await cdp.send("Page.enable", {}, { sessionId });
}
if (args.wait) {
await waitForUserSignal();
if (options.wait) {
await waitForUserSignal(options.waitPrompt ?? defaultWaitPrompt());
} else {
console.log("Waiting for page to load...");
await Promise.race([
@@ -251,11 +287,12 @@ async function captureUrl(args: Args): Promise<ConversionResult> {
}
console.log("Capturing page content...");
const { html } = await evaluateScript<{ html: string }>(
const snapshot = await evaluateScript<CaptureSnapshot>(
cdp, sessionId, absolutizeUrlsScript, args.timeout
);
return await extractContent(html, args.url);
return await extractContent(snapshot.html, snapshot.finalUrl || args.url, {
preserveBase64Images: args.downloadMedia,
});
} finally {
if (reusing) {
if (cdp && targetId) {
@@ -272,10 +309,67 @@ async function captureUrl(args: Args): Promise<ConversionResult> {
}
}
async function runHeadedFlow(
args: Args,
options: { existingPort?: number; wait: boolean; waitPrompt?: string }
): Promise<ConversionResult> {
return await captureUrlOnce(args, {
headless: false,
wait: options.wait,
existingPort: options.existingPort,
waitPrompt: options.waitPrompt,
});
}
async function captureUrl(args: Args): Promise<ConversionResult> {
const existingPort = await findExistingChromePort();
if (existingPort !== null) {
console.log("Found an existing Chrome session for this profile. Reusing it instead of launching a new browser.");
return await runHeadedFlow(args, {
existingPort,
wait: args.wait,
waitPrompt: args.wait ? defaultWaitPrompt() : undefined,
});
}
if (args.browserMode === "headless") {
return await captureUrlOnce(args, { headless: true, wait: false });
}
if (args.browserMode === "headed") {
return await runHeadedFlow(args, {
wait: args.wait,
waitPrompt: args.wait ? defaultWaitPrompt() : undefined,
});
}
if (args.wait) {
return await runHeadedFlow(args, {
wait: true,
waitPrompt: defaultWaitPrompt(),
});
}
try {
return await captureUrlOnce(args, { headless: true, wait: false });
} catch (error) {
const headlessMessage = error instanceof Error ? error.message : String(error);
console.warn(`Headless capture failed: ${headlessMessage}`);
console.log("Retrying with a visible browser window...");
try {
return await runHeadedFlow(args, { wait: false });
} catch (headedError) {
const headedMessage = headedError instanceof Error ? headedError.message : String(headedError);
throw new Error(`Headless capture failed (${headlessMessage}); headed retry failed (${headedMessage})`);
}
}
}
async function main(): Promise<void> {
const args = parseArgs(process.argv);
if (!args.url) {
console.error("Usage: bun main.ts <url> [-o output.md] [--output-dir dir] [--wait] [--timeout ms] [--download-media]");
console.error("Usage: bun main.ts <url> [-o output.md] [--output-dir dir] [--wait] [--browser auto|headless|headed] [--timeout ms] [--download-media]");
process.exit(1);
}
@@ -286,6 +380,16 @@ async function main(): Promise<void> {
process.exit(1);
}
if (!BROWSER_MODES.has(args.browserMode)) {
console.error(`Invalid --browser mode: ${args.browserMode}. Expected auto, headless, or headed.`);
process.exit(1);
}
if (args.wait && args.browserMode === "headless") {
console.error("Error: --wait requires a visible browser. Use --browser auto or --browser headed.");
process.exit(1);
}
if (args.output) {
const stat = await import("node:fs").then(fs => fs.statSync(args.output!, { throwIfNoEntry: false }));
if (stat?.isDirectory()) {
@@ -296,6 +400,7 @@ async function main(): Promise<void> {
console.log(`Fetching: ${args.url}`);
console.log(`Mode: ${args.wait ? "wait" : "auto"}`);
console.log(`Browser: ${args.browserMode}`);
let outputPath: string;
let htmlSnapshotPath: string | null = null;
@@ -306,7 +411,7 @@ async function main(): Promise<void> {
try {
const result = await captureUrl(args);
document = createMarkdownDocument(result);
outputPath = args.output || await generateOutputPath(args.url, result.metadata.title, args.outputDir, document);
outputPath = args.output || await generateOutputPath(result.metadata.url || args.url, result.metadata.title, args.outputDir, document);
const outputDir = path.dirname(outputPath);
htmlSnapshotPath = deriveHtmlSnapshotPath(outputPath);
await mkdir(outputDir, { recursive: true });
@@ -0,0 +1,40 @@
import assert from "node:assert/strict";
import { mkdtemp, readFile, readdir } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import test from "node:test";
import { localizeMarkdownMedia } from "./media-localizer.js";
const PNG_1X1_BASE64 =
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO7Z0ioAAAAASUVORK5CYII=";
test("localizeMarkdownMedia saves embedded base64 images into imgs directory", async () => {
const tempDir = await mkdtemp(path.join(os.tmpdir(), "url-to-markdown-media-"));
const dataUri = `data:image/png;base64,${PNG_1X1_BASE64}`;
const markdown = [
"---",
`coverImage: "${dataUri}"`,
"---",
"",
"# Embedded Image",
"",
`![inline](${dataUri})`,
"",
].join("\n");
const result = await localizeMarkdownMedia(markdown, {
markdownPath: path.join(tempDir, "post.md"),
});
assert.equal(result.downloadedImages, 1);
assert.equal(result.downloadedVideos, 0);
assert.match(result.markdown, /coverImage: "imgs\/img-001\.png"/);
assert.match(result.markdown, /!\[inline\]\(imgs\/img-001\.png\)/);
const files = await readdir(path.join(tempDir, "imgs"));
assert.deepEqual(files, ["img-001.png"]);
const bytes = await readFile(path.join(tempDir, "imgs", "img-001.png"));
assert.equal(bytes.length, Buffer.from(PNG_1X1_BASE64, "base64").length);
});
@@ -3,10 +3,12 @@ import { mkdir, writeFile } from "node:fs/promises";
type MediaKind = "image" | "video";
type MediaHint = "image" | "unknown";
type MediaSource = "remote" | "data";
type MarkdownLinkCandidate = {
url: string;
hint: MediaHint;
source: MediaSource;
};
export type LocalizeMarkdownMediaOptions = {
@@ -22,8 +24,9 @@ export type LocalizeMarkdownMediaResult = {
videoDir: string | null;
};
const MARKDOWN_LINK_RE = /(!?\[[^\]\n]*\])\((<)?(https?:\/\/[^)\s>]+)(>)?\)/g;
const FRONTMATTER_COVER_RE = /^(coverImage:\s*")(https?:\/\/[^"]+)(")/m;
const MARKDOWN_LINK_RE =
/(!?\[[^\]\n]*\])\((<)?((?:https?:\/\/[^)\s>]+)|(?:data:[^)>\s]+))(>)?\)/g;
const FRONTMATTER_COVER_RE = /^(coverImage:\s*")((?:https?:\/\/[^"]+)|(?:data:[^"]+))(")/m;
const IMAGE_EXTENSIONS = new Set([
"jpg",
@@ -86,6 +89,10 @@ function resolveExtensionFromUrl(rawUrl: string): string | undefined {
return undefined;
}
function resolveExtensionFromContentType(contentType: string): string | undefined {
return normalizeExtension(MIME_EXTENSION_MAP[contentType]);
}
function resolveKindFromContentType(contentType: string): MediaKind | undefined {
if (!contentType) return undefined;
if (contentType.startsWith("image/")) return "image";
@@ -124,7 +131,7 @@ function resolveOutputExtension(
extension: string | undefined,
kind: MediaKind
): string {
const extFromMime = normalizeExtension(MIME_EXTENSION_MAP[contentType]);
const extFromMime = resolveExtensionFromContentType(contentType);
if (extFromMime) return extFromMime;
const normalizedExt = normalizeExtension(extension);
@@ -150,6 +157,10 @@ function sanitizeFileSegment(input: string): string {
}
function resolveFileStem(rawUrl: string, extension: string): string {
if (isDataUri(rawUrl)) {
return "";
}
try {
const parsed = new URL(rawUrl);
const base = path.posix.basename(parsed.pathname);
@@ -172,6 +183,26 @@ function buildFileName(kind: MediaKind, index: number, sourceUrl: string, extens
return `${prefix}-${serial}${suffix}.${extension}`;
}
function isDataUri(value: string): boolean {
return value.startsWith("data:");
}
function parseBase64DataUri(rawUrl: string): { contentType: string; bytes: Buffer } | null {
const match = rawUrl.match(/^data:([^;,]+);base64,([A-Za-z0-9+/=\s]+)$/i);
if (!match?.[1] || !match[2]) return null;
const contentType = normalizeContentType(match[1]);
if (!contentType) return null;
try {
const bytes = Buffer.from(match[2].replace(/\s+/g, ""), "base64");
if (bytes.length === 0) return null;
return { contentType, bytes };
} catch {
return null;
}
}
function collectMarkdownLinkCandidates(markdown: string): MarkdownLinkCandidate[] {
const candidates: MarkdownLinkCandidate[] = [];
const seen = new Set<string>();
@@ -181,7 +212,11 @@ function collectMarkdownLinkCandidates(markdown: string): MarkdownLinkCandidate[
const coverMatch = fmMatch[1]?.match(FRONTMATTER_COVER_RE);
if (coverMatch?.[2] && !seen.has(coverMatch[2])) {
seen.add(coverMatch[2]);
candidates.push({ url: coverMatch[2], hint: "image" });
candidates.push({
url: coverMatch[2],
hint: "image",
source: isDataUri(coverMatch[2]) ? "data" : "remote",
});
}
}
@@ -195,6 +230,7 @@ function collectMarkdownLinkCandidates(markdown: string): MarkdownLinkCandidate[
candidates.push({
url: rawUrl,
hint: label.startsWith("![") ? "image" : "unknown",
source: isDataUri(rawUrl) ? "data" : "remote",
});
}
@@ -244,24 +280,45 @@ export async function localizeMarkdownMedia(
for (const candidate of candidates) {
try {
const response = await fetch(candidate.url, {
method: "GET",
redirect: "follow",
headers: {
"user-agent": DOWNLOAD_USER_AGENT,
},
});
let sourceUrl = candidate.url;
let contentType = "";
let extension: string | undefined;
let kind: MediaKind | undefined;
let bytes: Buffer | null = null;
if (!response.ok) {
log(`[url-to-markdown] Skip media (${response.status}): ${candidate.url}`);
continue;
if (candidate.source === "data") {
const parsed = parseBase64DataUri(candidate.url);
if (!parsed) {
log("[url-to-markdown] Skip embedded media: unsupported or invalid data URI");
continue;
}
contentType = parsed.contentType;
extension = resolveExtensionFromContentType(contentType);
kind = resolveMediaKind(sourceUrl, contentType, extension, candidate.hint);
bytes = parsed.bytes;
} else {
const response = await fetch(candidate.url, {
method: "GET",
redirect: "follow",
headers: {
"user-agent": DOWNLOAD_USER_AGENT,
},
});
if (!response.ok) {
log(`[url-to-markdown] Skip media (${response.status}): ${candidate.url}`);
continue;
}
sourceUrl = response.url || candidate.url;
contentType = normalizeContentType(response.headers.get("content-type"));
extension = resolveExtensionFromUrl(sourceUrl) ?? resolveExtensionFromUrl(candidate.url);
kind = resolveMediaKind(sourceUrl, contentType, extension, candidate.hint);
bytes = Buffer.from(await response.arrayBuffer());
}
const sourceUrl = response.url || candidate.url;
const contentType = normalizeContentType(response.headers.get("content-type"));
const extension = resolveExtensionFromUrl(sourceUrl) ?? resolveExtensionFromUrl(candidate.url);
const kind = resolveMediaKind(sourceUrl, contentType, extension, candidate.hint);
if (!kind) {
if (!kind || !bytes) {
continue;
}
@@ -274,7 +331,6 @@ export async function localizeMarkdownMedia(
const fileName = buildFileName(kind, nextIndex, sourceUrl, outputExtension);
const absolutePath = path.join(targetDir, fileName);
const relativePath = path.posix.join(dirName, fileName);
const bytes = Buffer.from(await response.arrayBuffer());
await writeFile(absolutePath, bytes);
replacements.set(candidate.url, relativePath);
@@ -305,6 +361,7 @@ export function countRemoteMedia(markdown: string): { images: number; videos: nu
let images = 0;
let videos = 0;
for (const c of candidates) {
if (c.source !== "remote") continue;
const ext = resolveExtensionFromUrl(c.url);
const kind = resolveKindFromExtension(ext);
if (kind === "video") {
@@ -5,7 +5,7 @@
"dependencies": {
"@mozilla/readability": "^0.6.0",
"baoyu-chrome-cdp": "file:./vendor/baoyu-chrome-cdp",
"defuddle": "^0.12.0",
"defuddle": "^0.14.0",
"jsdom": "^24.1.3",
"linkedom": "^0.18.12",
"turndown": "^7.2.2",
+10 -1
View File
@@ -13,7 +13,7 @@ metadata:
# YouTube Transcript
Downloads transcripts (subtitles/captions) from YouTube videos. Works with both manually created and auto-generated transcripts. No API key or browser required — uses YouTube's InnerTube API directly.
Downloads transcripts (subtitles/captions) from YouTube videos. Works with both manually created and auto-generated transcripts. No API key or browser required — uses YouTube's InnerTube API directly and automatically falls back to `yt-dlp` when YouTube blocks the direct API path.
Fetches video metadata and cover image on first run, caches raw data for fast re-formatting.
@@ -75,6 +75,12 @@ ${BUN_X} {baseDir}/scripts/main.ts <url> --refresh
| `-o, --output <path>` | Save to specific file path | auto-generated |
| `--output-dir <dir>` | Base output directory | `youtube-transcript` |
## Optional Environment Variables
| Variable | Description |
|----------|-------------|
| `YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER` | Passed to `yt-dlp --cookies-from-browser` during fallback, e.g. `chrome`, `safari`, `firefox`, or `chrome:Profile 1` |
## Input Formats
Accepts any of these as video input:
@@ -121,6 +127,8 @@ On first fetch, the script saves:
Subsequent runs for the same video use cached data (no network calls). Use `--refresh` to force re-fetch. If a different language is requested, the cache is automatically refreshed.
When YouTube returns anti-bot / blocked responses on the direct InnerTube path, the script retries with alternate client identities and then falls back to `yt-dlp` if available. If fallback is needed but `yt-dlp` is unavailable, the agent should decide how to make `yt-dlp` available and continue rather than pushing the installation decision to the user.
SRT output (`--format srt`) is generated from `transcript-raw.json`. Text/markdown output uses `transcript-sentences.json` for natural sentence boundaries.
## Workflow
@@ -175,3 +183,4 @@ When `--speakers` is used, `--chapters` is implied — the processed output alwa
| Video unavailable | Video deleted, private, or region-locked |
| IP blocked | Too many requests, try again later |
| Age restricted | Video requires login for age verification |
| bot detected | The script retries alternate clients and then `yt-dlp`; if fallback tooling is missing, the agent should resolve that itself, otherwise if it still fails try `YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER=safari` (or your browser) |
@@ -0,0 +1,125 @@
import test from "node:test";
import assert from "node:assert/strict";
import { findTranscript, parseTranscriptJson3, parseWebVtt } from "./transcript.ts";
import { buildTranscriptListFromYtDlp, resolveVideoSource, selectYtDlpTrack } from "./youtube.ts";
test("selectYtDlpTrack prefers json3 over xml and vtt", () => {
const track = selectYtDlpTrack([
{ ext: "vtt", url: "https://example.com/subs.vtt" },
{ ext: "srv3", url: "https://example.com/subs.srv3" },
{ ext: "json3", url: "https://example.com/subs.json3" },
]);
assert.equal(track?.ext, "json3");
});
test("buildTranscriptListFromYtDlp keeps manual and generated tracks separate", () => {
const transcripts = buildTranscriptListFromYtDlp({
subtitles: {
en: [
{ ext: "json3", url: "https://example.com/en.json3", name: "English" },
],
},
automatic_captions: {
"zh-Hans": [
{ ext: "json3", url: "https://example.com/zh.json3", name: "Chinese (Simplified)" },
],
},
});
assert.equal(transcripts.length, 2);
assert.equal(transcripts[0].isGenerated, false);
assert.equal(transcripts[1].isGenerated, true);
assert.equal(transcripts[0].translationLanguages[0]?.languageCode, "zh-Hans");
const translated = findTranscript(transcripts, ["zh-Hans"], false, false);
assert.equal(translated.languageCode, "zh-Hans");
assert.equal(translated.isGenerated, true);
});
test("parseTranscriptJson3 reads youtube timedtext json3 payloads", () => {
const snippets = parseTranscriptJson3(JSON.stringify({
events: [
{
tStartMs: 80,
dDurationMs: 3120,
segs: [{ utf8: "hello\nworld" }],
},
{
tStartMs: 4000,
dDurationMs: 1800,
segs: [{ utf8: "again" }],
},
],
}));
assert.deepEqual(snippets, [
{ text: "hello world", start: 0.08, duration: 3.12 },
{ text: "again", start: 4, duration: 1.8 },
]);
});
test("parseWebVtt strips tags and cue settings", () => {
const snippets = parseWebVtt(`WEBVTT
00:00:00.080 --> 00:00:03.200 align:start position:0%
<c.colorE5E5E5>Hello</c> world
00:00:04.000 --> 00:00:05.800
Again
`);
assert.equal(snippets.length, 2);
assert.equal(snippets[0].text, "Hello world");
assert.equal(snippets[0].start, 0.08);
assert.equal(snippets[0].duration, 3.12);
assert.equal(snippets[1].text, "Again");
assert.equal(snippets[1].start, 4);
assert.equal(Number(snippets[1].duration.toFixed(1)), 1.8);
});
test("resolveVideoSource prefers primary InnerTube result before fallback", async () => {
let fallbackCalled = false;
const source = await resolveVideoSource(
"video12345ab",
async () => ({ kind: "innertube", data: { videoDetails: { title: "Primary" } }, transcripts: [] }),
() => {
fallbackCalled = true;
return {
subtitles: {
en: [{ ext: "json3", url: "https://example.com/en.json3", name: "English" }],
},
};
},
() => {}
);
assert.equal(source.kind, "innertube");
assert.equal(fallbackCalled, false);
});
test("resolveVideoSource falls back to yt-dlp only after fallback-eligible errors", async () => {
let fallbackCalled = false;
const source = await resolveVideoSource(
"video12345ab",
async () => {
const error = new Error("Request blocked for video12345ab: bot detected");
(error as Error & { code?: string }).code = "BOT_DETECTED";
throw error;
},
() => {
fallbackCalled = true;
return {
automatic_captions: {
en: [{ ext: "json3", url: "https://example.com/en.json3", name: "English (auto-generated)" }],
},
};
},
() => {}
);
assert.equal(source.kind, "yt-dlp");
assert.equal(fallbackCalled, true);
assert.equal(source.transcripts[0].languageCode, "en");
});
+93 -699
View File
@@ -1,659 +1,55 @@
#!/usr/bin/env bun
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "fs";
import { dirname, join, resolve } from "path";
type Format = "text" | "srt";
interface Options {
videoIds: string[];
languages: string[];
format: Format;
translate: string;
list: boolean;
excludeGenerated: boolean;
excludeManual: boolean;
output: string;
outputDir: string;
timestamps: boolean;
chapters: boolean;
speakers: boolean;
refresh: boolean;
}
interface Snippet {
text: string;
start: number;
duration: number;
}
interface Sentence {
text: string;
start: string;
end: string;
}
interface TranscriptInfo {
language: string;
languageCode: string;
isGenerated: boolean;
isTranslatable: boolean;
baseUrl: string;
translationLanguages: { language: string; languageCode: string }[];
}
interface Chapter {
title: string;
start: number;
end: number;
}
interface VideoMeta {
videoId: string;
title: string;
channel: string;
channelId: string;
description: string;
duration: number;
publishDate: string;
url: string;
coverImage: string;
thumbnailUrl: string;
language: { code: string; name: string; isGenerated: boolean };
chapters: Chapter[];
}
interface VideoResult {
videoId: string;
title?: string;
filePath?: string;
content?: string;
error?: string;
}
const WATCH_URL = "https://www.youtube.com/watch?v=";
const INNERTUBE_URL = "https://www.youtube.com/youtubei/v1/player";
const INNERTUBE_CTX = { client: { clientName: "ANDROID", clientVersion: "20.10.38" } };
function extractVideoId(input: string): string {
input = input.replace(/\\/g, "").trim();
const patterns = [
/(?:youtube\.com\/watch\?.*v=|youtu\.be\/|youtube\.com\/embed\/|youtube\.com\/v\/|youtube\.com\/shorts\/)([a-zA-Z0-9_-]{11})/,
/^([a-zA-Z0-9_-]{11})$/,
];
for (const p of patterns) {
const m = input.match(p);
if (m) return m[1];
}
return input;
}
function slugify(s: string): string {
return s
.toLowerCase()
.replace(/[^\w\s-]/g, "")
.replace(/\s+/g, "-")
.replace(/-+/g, "-")
.replace(/^-|-$/g, "") || "untitled";
}
function htmlUnescape(s: string): string {
return s
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&#x27;/g, "'")
.replace(/&#x2F;/g, "/")
.replace(/&apos;/g, "'")
.replace(/&#(\d+);/g, (_, n) => String.fromCharCode(parseInt(n)))
.replace(/&#x([0-9a-fA-F]+);/g, (_, n) => String.fromCharCode(parseInt(n, 16)));
}
function stripTags(s: string): string {
return s.replace(/<[^>]*>/g, "");
}
function parseTranscriptXml(xml: string): Snippet[] {
const snippets: Snippet[] = [];
const re = /<text\s+start="([^"]*)"(?:\s+dur="([^"]*)")?[^>]*>([\s\S]*?)<\/text>/g;
let m: RegExpExecArray | null;
while ((m = re.exec(xml)) !== null) {
const raw = m[3];
if (!raw) continue;
snippets.push({
text: htmlUnescape(stripTags(raw)),
start: parseFloat(m[1]),
duration: parseFloat(m[2] || "0"),
});
}
return snippets;
}
// --- YouTube API ---
async function fetchHtml(videoId: string): Promise<string> {
const r = await fetch(WATCH_URL + videoId, {
headers: { "Accept-Language": "en-US", "User-Agent": "Mozilla/5.0" },
});
if (!r.ok) throw new Error(`HTTP ${r.status} fetching video page`);
let html = await r.text();
if (html.includes('action="https://consent.youtube.com/s"')) {
const cv = html.match(/name="v" value="(.*?)"/);
if (!cv) throw new Error("Failed to create consent cookie");
const r2 = await fetch(WATCH_URL + videoId, {
headers: {
"Accept-Language": "en-US",
"User-Agent": "Mozilla/5.0",
Cookie: `CONSENT=YES+${cv[1]}`,
},
});
if (!r2.ok) throw new Error(`HTTP ${r2.status} fetching video page (consent)`);
html = await r2.text();
}
return html;
}
function extractApiKey(html: string, videoId: string): string {
const m = html.match(/"INNERTUBE_API_KEY":\s*"([a-zA-Z0-9_-]+)"/);
if (!m) {
if (html.includes('class="g-recaptcha"')) throw new Error(`IP blocked for ${videoId} (reCAPTCHA)`);
throw new Error(`Cannot extract API key for ${videoId}`);
}
return m[1];
}
async function fetchInnertubeData(videoId: string, apiKey: string): Promise<any> {
const r = await fetch(`${INNERTUBE_URL}?key=${apiKey}`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ context: INNERTUBE_CTX, videoId }),
});
if (r.status === 429) throw new Error(`IP blocked for ${videoId} (429)`);
if (!r.ok) throw new Error(`HTTP ${r.status} from InnerTube API`);
return r.json();
}
function assertPlayability(data: any, videoId: string) {
const ps = data?.playabilityStatus;
if (!ps) return;
const status = ps.status;
if (status === "OK" || !status) return;
const reason = ps.reason || "";
if (status === "LOGIN_REQUIRED") {
if (reason.includes("bot")) throw new Error(`Request blocked for ${videoId}: bot detected`);
if (reason.includes("inappropriate")) throw new Error(`Age restricted: ${videoId}`);
}
if (status === "ERROR" && reason.includes("unavailable")) {
if (videoId.startsWith("http")) throw new Error(`Invalid video ID: pass the ID, not the URL`);
throw new Error(`Video unavailable: ${videoId}`);
}
const subreasons = ps.errorScreen?.playerErrorMessageRenderer?.subreason?.runs?.map((r: any) => r.text).join("") || "";
throw new Error(`Video unplayable (${videoId}): ${reason} ${subreasons}`.trim());
}
function extractCaptionsJson(data: any, videoId: string): any {
assertPlayability(data, videoId);
const cj = data?.captions?.playerCaptionsTracklistRenderer;
if (!cj || !cj.captionTracks) throw new Error(`Transcripts disabled for ${videoId}`);
return cj;
}
function buildTranscriptList(captionsJson: any): TranscriptInfo[] {
const tlLangs = (captionsJson.translationLanguages || []).map((tl: any) => ({
language: tl.languageName?.runs?.[0]?.text || tl.languageName?.simpleText || "",
languageCode: tl.languageCode,
}));
return (captionsJson.captionTracks || []).map((t: any) => ({
language: t.name?.runs?.[0]?.text || t.name?.simpleText || "",
languageCode: t.languageCode,
isGenerated: t.kind === "asr",
isTranslatable: !!t.isTranslatable,
baseUrl: (t.baseUrl || "").replace(/&fmt=srv3/g, ""),
translationLanguages: t.isTranslatable ? tlLangs : [],
}));
}
function findTranscript(
transcripts: TranscriptInfo[],
languages: string[],
excludeGenerated: boolean,
excludeManual: boolean
): TranscriptInfo {
let filtered = transcripts;
if (excludeGenerated) filtered = filtered.filter((t) => !t.isGenerated);
if (excludeManual) filtered = filtered.filter((t) => t.isGenerated);
for (const lang of languages) {
const found = filtered.find((t) => t.languageCode === lang);
if (found) return found;
}
const available = filtered.map((t) => `${t.languageCode} ("${t.language}")`).join(", ");
throw new Error(`No transcript found for languages [${languages.join(", ")}]. Available: ${available || "none"}`);
}
async function fetchTranscriptSnippets(info: TranscriptInfo, translateTo?: string): Promise<{ snippets: Snippet[]; language: string; languageCode: string }> {
let url = info.baseUrl;
let lang = info.language;
let langCode = info.languageCode;
if (translateTo) {
if (!info.isTranslatable) throw new Error(`Transcript ${info.languageCode} is not translatable`);
const tl = info.translationLanguages.find((t) => t.languageCode === translateTo);
if (!tl) throw new Error(`Translation language ${translateTo} not available`);
url += `&tlang=${translateTo}`;
lang = tl.language;
langCode = translateTo;
}
const r = await fetch(url, { headers: { "Accept-Language": "en-US" } });
if (!r.ok) throw new Error(`HTTP ${r.status} fetching transcript`);
return { snippets: parseTranscriptXml(await r.text()), language: lang, languageCode: langCode };
}
// --- Metadata & chapters ---
function parseChapters(description: string, duration: number = 0): Chapter[] {
const raw: { title: string; start: number }[] = [];
for (const line of description.split("\n")) {
const m = line.trim().match(/^(?:(\d{1,2}):)?(\d{1,2}):(\d{2})\s+(.+)$/);
if (m) {
const h = m[1] ? parseInt(m[1]) : 0;
raw.push({ title: m[4].trim(), start: h * 3600 + parseInt(m[2]) * 60 + parseInt(m[3]) });
}
}
if (raw.length < 2) return [];
return raw.map((ch, i) => ({
title: ch.title,
start: ch.start,
end: i < raw.length - 1 ? raw[i + 1].start : Math.max(duration, ch.start),
}));
}
function getThumbnailUrls(videoId: string, data: any): string[] {
const urls = [
`https://i.ytimg.com/vi/${videoId}/maxresdefault.jpg`,
`https://i.ytimg.com/vi/${videoId}/hqdefault.jpg`,
];
const thumbnails = data?.videoDetails?.thumbnail?.thumbnails ||
data?.microformat?.playerMicroformatRenderer?.thumbnail?.thumbnails || [];
if (thumbnails.length) {
const sorted = [...thumbnails].sort((a: any, b: any) => (b.width || 0) - (a.width || 0));
for (const t of sorted) if (t.url && !urls.includes(t.url)) urls.push(t.url);
}
return urls;
}
function buildVideoMeta(data: any, videoId: string, langInfo: { code: string; name: string; isGenerated: boolean }, chapters: Chapter[]): VideoMeta {
const vd = data?.videoDetails || {};
const mf = data?.microformat?.playerMicroformatRenderer || {};
return {
videoId,
title: vd.title || mf.title?.simpleText || "",
channel: vd.author || mf.ownerChannelName || "",
channelId: vd.channelId || mf.externalChannelId || "",
description: vd.shortDescription || mf.description?.simpleText || "",
duration: parseInt(vd.lengthSeconds || "0"),
publishDate: mf.publishDate || mf.uploadDate || "",
url: `https://www.youtube.com/watch?v=${videoId}`,
coverImage: "",
thumbnailUrl: getThumbnailUrls(videoId, data)[0],
language: langInfo,
chapters,
};
}
async function downloadCoverImage(urls: string[], outputPath: string): Promise<boolean> {
for (const u of urls) {
try {
const r = await fetch(u);
if (r.ok) {
writeFileSync(outputPath, Buffer.from(await r.arrayBuffer()));
return true;
}
} catch {}
}
return false;
}
function parseSrt(srt: string): Snippet[] {
const blocks = srt.trim().split(/\n\n+/);
const snippets: Snippet[] = [];
for (const block of blocks) {
const lines = block.split("\n");
if (lines.length < 3) continue;
const m = lines[1].match(/(\d{2}):(\d{2}):(\d{2}),(\d{3})\s*-->\s*(\d{2}):(\d{2}):(\d{2}),(\d{3})/);
if (!m) continue;
const start = parseInt(m[1]) * 3600 + parseInt(m[2]) * 60 + parseInt(m[3]) + parseInt(m[4]) / 1000;
const end = parseInt(m[5]) * 3600 + parseInt(m[6]) * 60 + parseInt(m[7]) + parseInt(m[8]) / 1000;
snippets.push({ text: lines.slice(2).join(" "), start, duration: end - start });
}
return snippets;
}
// --- Timestamp formatting ---
function ts(t: number): string {
const h = Math.floor(t / 3600);
const m = Math.floor((t % 3600) / 60);
const s = Math.floor(t % 60);
return `${String(h).padStart(2, "0")}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}`;
}
function tsMs(t: number, sep: string): string {
const h = Math.floor(t / 3600);
const m = Math.floor((t % 3600) / 60);
const s = Math.floor(t % 60);
const ms = Math.round((t - Math.floor(t)) * 1000);
return `${String(h).padStart(2, "0")}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}${sep}${String(ms).padStart(3, "0")}`;
}
// --- Paragraph grouping ---
interface Paragraph {
text: string;
start: number;
end: number;
}
function groupIntoParagraphs(snippets: Snippet[]): Paragraph[] {
if (!snippets.length) return [];
const paras: Paragraph[] = [];
let buf: Snippet[] = [];
for (let i = 0; i < snippets.length; i++) {
buf.push(snippets[i]);
const last = i === snippets.length - 1;
const gap = !last && snippets[i + 1].start - (snippets[i].start + snippets[i].duration) > 1.5;
if (last || gap || buf.length >= 8) {
const lastS = buf[buf.length - 1];
paras.push({ text: buf.map(s => s.text).join(" "), start: buf[0].start, end: lastS.start + lastS.duration });
buf = [];
}
}
return paras;
}
// --- Sentence segmentation ---
const SENTENCE_END_RE = /[.?!…。?!⁈⁇‼‽.]/;
function isCJK(ch: string): boolean {
const code = ch.charCodeAt(0);
return (code >= 0x4E00 && code <= 0x9FFF) ||
(code >= 0x3040 && code <= 0x309F) ||
(code >= 0x30A0 && code <= 0x30FF) ||
(code >= 0xAC00 && code <= 0xD7AF) ||
(code >= 0x3400 && code <= 0x4DBF) ||
(code >= 0xF900 && code <= 0xFAFF);
}
function splitSnippetAtPunctuation(s: Snippet): { text: string; start: number; end: number }[] {
const { text, start, duration } = s;
const end = start + duration;
if (!text.length) return [];
const splitPoints: number[] = [];
for (let i = 0; i < text.length; i++) {
if (SENTENCE_END_RE.test(text[i])) {
while (i + 1 < text.length && SENTENCE_END_RE.test(text[i + 1])) i++;
if (i < text.length - 1) splitPoints.push(i);
}
}
if (!splitPoints.length) return [{ text, start, end }];
const parts: { text: string; start: number; end: number }[] = [];
let prev = 0;
for (const pos of splitPoints) {
const partText = text.slice(prev, pos + 1).trim();
if (partText) {
parts.push({
text: partText,
start: start + (prev / text.length) * duration,
end: start + ((pos + 1) / text.length) * duration,
});
}
prev = pos + 1;
}
const remaining = text.slice(prev).trim();
if (remaining) {
parts.push({ text: remaining, start: start + (prev / text.length) * duration, end });
}
return parts;
}
function mergeTexts(texts: string[]): string {
if (!texts.length) return "";
let result = texts[0];
for (let i = 1; i < texts.length; i++) {
const next = texts[i];
if (!next) continue;
const lastChar = result[result.length - 1];
const firstChar = next[0];
if (isCJK(lastChar) || isCJK(firstChar)) {
result += next;
} else {
result = result.trimEnd() + " " + next.trimStart();
}
}
return result.replace(/ {2,}/g, " ");
}
function segmentIntoSentences(snippets: Snippet[]): Sentence[] {
const parts: { text: string; start: number; end: number }[] = [];
for (const s of snippets) parts.push(...splitSnippetAtPunctuation(s));
const sentences: Sentence[] = [];
let buf: { text: string; start: number; end: number }[] = [];
for (const part of parts) {
buf.push(part);
if (SENTENCE_END_RE.test(part.text[part.text.length - 1])) {
sentences.push({
text: mergeTexts(buf.map(b => b.text)),
start: ts(buf[0].start),
end: ts(buf[buf.length - 1].end),
});
buf = [];
}
}
if (buf.length) {
sentences.push({
text: mergeTexts(buf.map(b => b.text)),
start: ts(buf[0].start),
end: ts(buf[buf.length - 1].end),
});
}
return sentences;
}
function parseTs(t: string): number {
const [h, m, s] = t.split(":").map(Number);
return h * 3600 + m * 60 + s;
}
function groupSentenceParas(sentences: Sentence[]): Paragraph[] {
if (!sentences.length) return [];
const paras: Paragraph[] = [];
let buf: Sentence[] = [];
for (let i = 0; i < sentences.length; i++) {
buf.push(sentences[i]);
const last = i === sentences.length - 1;
const gap = !last && parseTs(sentences[i + 1].start) - parseTs(sentences[i].end) > 2;
if (last || gap || buf.length >= 5) {
paras.push({
text: mergeTexts(buf.map(s => s.text)),
start: parseTs(buf[0].start),
end: parseTs(buf[buf.length - 1].end),
});
buf = [];
}
}
return paras;
}
// --- Format functions ---
function formatSrt(snippets: Snippet[]): string {
return snippets
.map((s, i) => {
const end = i < snippets.length - 1 && snippets[i + 1].start < s.start + s.duration
? snippets[i + 1].start
: s.start + s.duration;
return `${i + 1}\n${tsMs(s.start, ",")} --> ${tsMs(end, ",")}\n${s.text}`;
})
.join("\n\n") + "\n";
}
function yamlEscape(s: string): string {
if (/[:"'{}\[\]#&*!|>%@`\n]/.test(s) || s.trim() !== s) return `"${s.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"`;
return s;
}
function extractSummary(description: string): string {
if (!description) return "";
const firstPara = description.split(/\n\s*\n/)[0].trim();
const lines = firstPara.split("\n").filter(l => !/^\s*(https?:\/\/|#|@|\d+:\d+)/.test(l) && l.trim());
return lines.join(" ").slice(0, 300).trim();
}
function formatMarkdown(sentences: Sentence[], meta: VideoMeta, opts: { timestamps: boolean; chapters: boolean; speakers: boolean }, snippets?: Snippet[]): string {
const summary = extractSummary(meta.description);
let md = "---\n";
md += `title: ${yamlEscape(meta.title)}\n`;
md += `channel: ${yamlEscape(meta.channel)}\n`;
if (meta.publishDate) md += `date: ${meta.publishDate}\n`;
md += `url: ${yamlEscape(meta.url)}\n`;
if (meta.coverImage) md += `cover: ${meta.coverImage}\n`;
if (summary) md += `description: ${yamlEscape(summary)}\n`;
if (meta.language) md += `language: ${meta.language.code}\n`;
md += "---\n\n";
if (opts.speakers) {
md += `# ${meta.title}\n\n`;
if (summary) md += `${summary}\n\n`;
if (meta.description) md += "# Description\n\n" + meta.description.trim() + "\n\n";
if (meta.chapters.length) {
md += "# Chapters\n\n";
for (const ch of meta.chapters) md += `* [${ts(ch.start)}] ${ch.title}\n`;
md += "\n";
}
md += "# Transcript\n\n";
md += snippets ? formatSrt(snippets) : "";
return md;
}
md += `# ${meta.title}\n\n`;
if (summary) md += `${summary}\n\n`;
const chapters = opts.chapters ? meta.chapters : [];
if (chapters.length) {
md += "## Table of Contents\n\n";
for (const ch of chapters) md += opts.timestamps ? `* [${ts(ch.start)}] ${ch.title}\n` : `* ${ch.title}\n`;
md += "\n";
if (meta.coverImage) md += `\n![cover](${meta.coverImage})\n`;
md += "\n";
for (let i = 0; i < chapters.length; i++) {
const nextStart = i < chapters.length - 1 ? chapters[i + 1].start : Infinity;
const chSentences = sentences.filter(s => parseTs(s.start) >= chapters[i].start && parseTs(s.start) < nextStart);
const paras = groupSentenceParas(chSentences);
md += opts.timestamps
? `## [${ts(chapters[i].start)}] ${chapters[i].title}\n\n`
: `## ${chapters[i].title}\n\n`;
for (const p of paras) md += opts.timestamps ? `${p.text} [${ts(p.start)}${ts(p.end)}]\n\n` : `${p.text}\n\n`;
md += "\n";
}
} else {
const paras = groupSentenceParas(sentences);
for (const p of paras) md += opts.timestamps ? `${p.text} [${ts(p.start)}${ts(p.end)}]\n\n` : `${p.text}\n\n`;
}
return md.trimEnd() + "\n";
}
function formatListOutput(videoId: string, title: string, transcripts: TranscriptInfo[]): string {
const manual = transcripts.filter((t) => !t.isGenerated);
const generated = transcripts.filter((t) => t.isGenerated);
const tlLangs = transcripts.find((t) => t.translationLanguages.length > 0)?.translationLanguages || [];
const fmtList = (list: TranscriptInfo[]) =>
list.length ? list.map((t) => ` - ${t.languageCode} ("${t.language}")${t.isTranslatable ? " [TRANSLATABLE]" : ""}`).join("\n") : "None";
const fmtTl = tlLangs.length
? tlLangs.map((t) => ` - ${t.languageCode} ("${t.language}")`).join("\n")
: "None";
return `Transcripts for ${videoId}${title ? ` (${title})` : ""}:\n\n(MANUALLY CREATED)\n${fmtList(manual)}\n\n(GENERATED)\n${fmtList(generated)}\n\n(TRANSLATION LANGUAGES)\n${fmtTl}`;
}
// --- File helpers ---
function ensureDir(p: string) {
const dir = dirname(p);
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
}
function resolveBaseDir(outputDir: string): string {
return resolve(outputDir || "youtube-transcript");
}
function loadIndex(baseDir: string): Record<string, string> {
try { return JSON.parse(readFileSync(join(baseDir, ".index.json"), "utf-8")); } catch { return {}; }
}
function saveIndex(baseDir: string, index: Record<string, string>) {
const p = join(baseDir, ".index.json");
ensureDir(p);
writeFileSync(p, JSON.stringify(index, null, 2));
}
function lookupVideoDir(videoId: string, baseDir: string): string | null {
const rel = loadIndex(baseDir)[videoId];
if (rel) {
const dir = resolve(baseDir, rel);
if (existsSync(dir)) return dir;
}
return null;
}
function registerVideoDir(videoId: string, channelSlug: string, titleSlug: string, baseDir: string): string {
const rel = join(channelSlug, titleSlug);
const index = loadIndex(baseDir);
index[videoId] = rel;
saveIndex(baseDir, index);
return resolve(baseDir, rel);
}
function hasCachedData(videoDir: string): boolean {
return existsSync(join(videoDir, "meta.json")) && existsSync(join(videoDir, "transcript-raw.json"));
}
function loadMeta(videoDir: string): VideoMeta {
return JSON.parse(readFileSync(join(videoDir, "meta.json"), "utf-8"));
}
function loadSnippets(videoDir: string): Snippet[] {
return JSON.parse(readFileSync(join(videoDir, "transcript-raw.json"), "utf-8"));
}
function loadSentences(videoDir: string): Sentence[] {
return JSON.parse(readFileSync(join(videoDir, "transcript-sentences.json"), "utf-8"));
}
// --- Main processing ---
async function fetchAndCache(videoId: string, baseDir: string, opts: Options): Promise<{ meta: VideoMeta; snippets: Snippet[]; sentences: Sentence[]; videoDir: string }> {
const html = await fetchHtml(videoId);
const apiKey = extractApiKey(html, videoId);
const data = await fetchInnertubeData(videoId, apiKey);
const captionsJson = extractCaptionsJson(data, videoId);
const transcripts = buildTranscriptList(captionsJson);
const info = findTranscript(transcripts, opts.languages, opts.excludeGenerated, opts.excludeManual);
const result = await fetchTranscriptSnippets(info, opts.translate || undefined);
const description = data?.videoDetails?.shortDescription || "";
const duration = parseInt(data?.videoDetails?.lengthSeconds || "0");
import { writeFileSync } from "fs";
import { join, resolve } from "path";
import { extractVideoId, slugify } from "./shared.ts";
import {
ensureDir,
hasCachedData,
loadMeta,
loadSentences,
loadSnippets,
lookupVideoDir,
registerVideoDir,
resolveBaseDir,
} from "./storage.ts";
import { findTranscript, formatListOutput, formatMarkdown, formatSrt, segmentIntoSentences } from "./transcript.ts";
import type { Options, Sentence, Snippet, VideoMeta, VideoResult } from "./types.ts";
import {
buildVideoMeta,
buildVideoMetaFromYtDlp,
downloadCoverImage,
fetchTranscriptSnippets,
fetchVideoSource,
getThumbnailUrls,
getYtDlpThumbnailUrls,
parseChapters,
} from "./youtube.ts";
async function fetchAndCache(
videoId: string,
baseDir: string,
opts: Options
): Promise<{ meta: VideoMeta; snippets: Snippet[]; sentences: Sentence[]; videoDir: string }> {
const source = await fetchVideoSource(videoId);
const requestedLanguages = source.kind === "yt-dlp" && opts.translate ? [opts.translate] : opts.languages;
const transcript = findTranscript(source.transcripts, requestedLanguages, opts.excludeGenerated, opts.excludeManual);
const result = await fetchTranscriptSnippets(transcript, source.kind === "yt-dlp" ? undefined : opts.translate || undefined);
const description = source.kind === "yt-dlp"
? source.info.description || ""
: source.data?.videoDetails?.shortDescription || "";
const duration = source.kind === "yt-dlp"
? Number(source.info.duration || 0)
: parseInt(source.data?.videoDetails?.lengthSeconds || "0");
const chapters = parseChapters(description, duration);
const langInfo = { code: result.languageCode, name: result.language, isGenerated: info.isGenerated };
const meta = buildVideoMeta(data, videoId, langInfo, chapters);
const language = {
code: result.languageCode,
name: result.language,
isGenerated: transcript.isGenerated,
};
const meta = source.kind === "yt-dlp"
? buildVideoMetaFromYtDlp(source.info, videoId, language, chapters)
: buildVideoMeta(source.data, videoId, language, chapters);
const videoDir = registerVideoDir(videoId, slugify(meta.channel), slugify(meta.title), baseDir);
ensureDir(join(videoDir, "meta.json"));
@@ -663,9 +59,12 @@ async function fetchAndCache(videoId: string, baseDir: string, opts: Options): P
const sentences = segmentIntoSentences(result.snippets);
writeFileSync(join(videoDir, "transcript-sentences.json"), JSON.stringify(sentences, null, 2));
const imgPath = join(videoDir, "imgs", "cover.jpg");
ensureDir(imgPath);
const downloaded = await downloadCoverImage(getThumbnailUrls(videoId, data), imgPath);
const imagePath = join(videoDir, "imgs", "cover.jpg");
ensureDir(imagePath);
const downloaded = await downloadCoverImage(
source.kind === "yt-dlp" ? getYtDlpThumbnailUrls(videoId, source.info) : getThumbnailUrls(videoId, source.data),
imagePath
);
meta.coverImage = downloaded ? "imgs/cover.jpg" : "";
writeFileSync(join(videoDir, "meta.json"), JSON.stringify(meta, null, 2));
@@ -676,15 +75,10 @@ async function fetchAndCache(videoId: string, baseDir: string, opts: Options): P
async function processVideo(videoId: string, opts: Options): Promise<VideoResult> {
const baseDir = resolveBaseDir(opts.outputDir);
// --list: always fetch fresh
if (opts.list) {
const html = await fetchHtml(videoId);
const apiKey = extractApiKey(html, videoId);
const data = await fetchInnertubeData(videoId, apiKey);
const title = data?.videoDetails?.title || "";
const captionsJson = extractCaptionsJson(data, videoId);
const transcripts = buildTranscriptList(captionsJson);
return { videoId, title, content: formatListOutput(videoId, title, transcripts) };
const source = await fetchVideoSource(videoId);
const title = source.kind === "yt-dlp" ? source.info.title || "" : source.data?.videoDetails?.title || "";
return { videoId, title, content: formatListOutput(videoId, title, source.transcripts) };
}
let videoDir = lookupVideoDir(videoId, baseDir);
@@ -697,16 +91,17 @@ async function processVideo(videoId: string, opts: Options): Promise<VideoResult
meta = loadMeta(videoDir);
snippets = loadSnippets(videoDir);
sentences = loadSentences(videoDir);
const wantLangs = opts.translate ? [opts.translate] : opts.languages;
if (!wantLangs.includes(meta.language.code)) needsFetch = true;
// Backfill chapter end times for caches created before this field existed
if (!needsFetch && meta.chapters.length > 0 && meta.chapters.some((ch: any) => ch.end === undefined)) {
const wantedLanguages = opts.translate ? [opts.translate] : opts.languages;
if (!wantedLanguages.includes(meta.language.code)) needsFetch = true;
if (!needsFetch && meta.chapters.length > 0 && meta.chapters.some((chapter: any) => chapter.end === undefined)) {
for (let i = 0; i < meta.chapters.length; i++) {
meta.chapters[i].end = i < meta.chapters.length - 1
? meta.chapters[i + 1].start
: Math.max(meta.duration, meta.chapters[i].start);
}
try { writeFileSync(join(videoDir, "meta.json"), JSON.stringify(meta, null, 2)); } catch {}
try {
writeFileSync(join(videoDir, "meta.json"), JSON.stringify(meta, null, 2));
} catch {}
}
}
@@ -722,21 +117,19 @@ async function processVideo(videoId: string, opts: Options): Promise<VideoResult
sentences = sentences!;
}
let content: string;
let ext: string;
if (opts.format === "srt") {
content = formatSrt(snippets);
ext = "srt";
} else {
content = formatMarkdown(sentences, meta, {
timestamps: opts.timestamps,
chapters: opts.chapters,
speakers: opts.speakers,
}, snippets);
ext = "md";
}
const content = opts.format === "srt"
? formatSrt(snippets)
: formatMarkdown(
sentences,
meta,
{
timestamps: opts.timestamps,
chapters: opts.chapters,
speakers: opts.speakers,
},
snippets
);
const ext = opts.format === "srt" ? "srt" : "md";
const filePath = opts.output ? resolve(opts.output) : join(videoDir!, `transcript.${ext}`);
ensureDir(filePath);
writeFileSync(filePath, content);
@@ -744,8 +137,6 @@ async function processVideo(videoId: string, opts: Options): Promise<VideoResult
return { videoId, title: meta.title, filePath };
}
// --- CLI ---
function printHelp() {
console.log(`Usage: bun main.ts <video-url-or-id> [options]
@@ -789,13 +180,13 @@ function parseArgs(argv: string[]): Options | null {
printHelp();
process.exit(0);
} else if (arg === "--languages") {
const v = argv[++i];
if (v) opts.languages = v.split(",").map((s) => s.trim());
const value = argv[++i];
if (value) opts.languages = value.split(",").map((entry) => entry.trim());
} else if (arg === "--format") {
const v = argv[++i]?.toLowerCase();
if (v === "text" || v === "srt") opts.format = v;
const value = argv[++i]?.toLowerCase();
if (value === "text" || value === "srt") opts.format = value;
else {
console.error(`Invalid format: ${v}. Use: text, srt`);
console.error(`Invalid format: ${value}. Use: text, srt`);
return null;
}
} else if (arg === "--translate") {
@@ -830,6 +221,7 @@ function parseArgs(argv: string[]): Options | null {
printHelp();
return null;
}
return opts;
}
@@ -844,14 +236,16 @@ async function main() {
for (const videoId of opts.videoIds) {
try {
const r = await processVideo(videoId, opts);
if (r.error) console.error(`Error (${r.videoId}): ${r.error}`);
else if (r.filePath) console.log(r.filePath);
else if (r.content) console.log(r.content);
} catch (e) {
console.error(`Error (${videoId}): ${(e as Error).message}`);
const result = await processVideo(videoId, opts);
if (result.error) console.error(`Error (${result.videoId}): ${result.error}`);
else if (result.filePath) console.log(result.filePath);
else if (result.content) console.log(result.content);
} catch (error) {
console.error(`Error (${videoId}): ${(error as Error).message}`);
}
}
}
main();
if (import.meta.main) {
main();
}
@@ -0,0 +1,83 @@
import type { TranscriptError } from "./types.ts";
export function extractVideoId(input: string): string {
input = input.replace(/\\/g, "").trim();
const patterns = [
/(?:youtube\.com\/watch\?.*v=|youtu\.be\/|youtube\.com\/embed\/|youtube\.com\/v\/|youtube\.com\/shorts\/)([a-zA-Z0-9_-]{11})/,
/^([a-zA-Z0-9_-]{11})$/,
];
for (const pattern of patterns) {
const match = input.match(pattern);
if (match) return match[1];
}
return input;
}
export function slugify(value: string): string {
return value
.toLowerCase()
.replace(/[^\w\s-]/g, "")
.replace(/\s+/g, "-")
.replace(/-+/g, "-")
.replace(/^-|-$/g, "") || "untitled";
}
export function htmlUnescape(value: string): string {
return value
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&#x27;/g, "'")
.replace(/&#x2F;/g, "/")
.replace(/&apos;/g, "'")
.replace(/&#(\d+);/g, (_, n) => String.fromCharCode(parseInt(n)))
.replace(/&#x([0-9a-fA-F]+);/g, (_, n) => String.fromCharCode(parseInt(n, 16)));
}
export function stripTags(value: string): string {
return value.replace(/<[^>]*>/g, "");
}
export function makeError(message: string, code?: string): Error {
const error = new Error(message) as TranscriptError;
if (code) error.code = code;
return error;
}
export function normalizeError(error: unknown): TranscriptError {
if (error instanceof Error) {
const known = error as TranscriptError;
if (known.code) return known;
const message = known.message || String(error);
const lower = message.toLowerCase();
if (lower.includes("bot detected")) known.code = "BOT_DETECTED";
else if (lower.includes("age restricted")) known.code = "AGE_RESTRICTED";
else if (lower.includes("video unavailable")) known.code = "VIDEO_UNAVAILABLE";
else if (lower.includes("transcripts disabled")) known.code = "TRANSCRIPTS_DISABLED";
else if (lower.includes("no transcript found")) known.code = "NO_TRANSCRIPT";
else if (lower.includes("invalid video id")) known.code = "INVALID_VIDEO_ID";
else if (lower.includes("ip blocked") || lower.includes("recaptcha") || lower.includes("http 429")) known.code = "IP_BLOCKED";
else if (lower.includes("cannot extract api key")) known.code = "PAGE_FETCH_FAILED";
else if (lower.includes("innertube api") || lower.includes("http 403")) known.code = "INNERTUBE_REJECTED";
else if (lower.includes("yt-dlp fallback failed")) known.code = "YT_DLP_FAILED";
return known;
}
return makeError(String(error), "UNKNOWN") as TranscriptError;
}
export function shouldTryAlternateClient(error: unknown): boolean {
const code = normalizeError(error).code;
return code === "BOT_DETECTED" || code === "IP_BLOCKED" || code === "INNERTUBE_REJECTED" || code === "AGE_RESTRICTED" || code === "VIDEO_UNAVAILABLE";
}
export function shouldTryYtDlpFallback(error: unknown): boolean {
const code = normalizeError(error).code;
return code === "BOT_DETECTED" || code === "IP_BLOCKED" || code === "INNERTUBE_REJECTED" || code === "PAGE_FETCH_FAILED" || code === "AGE_RESTRICTED" || code === "VIDEO_UNAVAILABLE";
}
export function normalizePublishDate(uploadDate?: string): string {
if (!uploadDate || !/^\d{8}$/.test(uploadDate)) return uploadDate || "";
return `${uploadDate.slice(0, 4)}-${uploadDate.slice(4, 6)}-${uploadDate.slice(6, 8)}`;
}
@@ -0,0 +1,60 @@
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "fs";
import { dirname, join, resolve } from "path";
import type { Sentence, Snippet, VideoMeta } from "./types.ts";
export function ensureDir(path: string) {
const dir = dirname(path);
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
}
export function resolveBaseDir(outputDir: string): string {
return resolve(outputDir || "youtube-transcript");
}
function loadIndex(baseDir: string): Record<string, string> {
try {
return JSON.parse(readFileSync(join(baseDir, ".index.json"), "utf-8"));
} catch {
return {};
}
}
function saveIndex(baseDir: string, index: Record<string, string>) {
const path = join(baseDir, ".index.json");
ensureDir(path);
writeFileSync(path, JSON.stringify(index, null, 2));
}
export function lookupVideoDir(videoId: string, baseDir: string): string | null {
const rel = loadIndex(baseDir)[videoId];
if (rel) {
const dir = resolve(baseDir, rel);
if (existsSync(dir)) return dir;
}
return null;
}
export function registerVideoDir(videoId: string, channelSlug: string, titleSlug: string, baseDir: string): string {
const rel = join(channelSlug, titleSlug);
const index = loadIndex(baseDir);
index[videoId] = rel;
saveIndex(baseDir, index);
return resolve(baseDir, rel);
}
export function hasCachedData(videoDir: string): boolean {
return existsSync(join(videoDir, "meta.json")) && existsSync(join(videoDir, "transcript-raw.json"));
}
export function loadMeta(videoDir: string): VideoMeta {
return JSON.parse(readFileSync(join(videoDir, "meta.json"), "utf-8"));
}
export function loadSnippets(videoDir: string): Snippet[] {
return JSON.parse(readFileSync(join(videoDir, "transcript-raw.json"), "utf-8"));
}
export function loadSentences(videoDir: string): Sentence[] {
return JSON.parse(readFileSync(join(videoDir, "transcript-sentences.json"), "utf-8"));
}
@@ -0,0 +1,349 @@
import { htmlUnescape, makeError, stripTags } from "./shared.ts";
import type { Sentence, Snippet, TranscriptInfo, VideoMeta } from "./types.ts";
interface Paragraph {
text: string;
start: number;
end: number;
}
const SENTENCE_END_RE = /[.?!…。?!⁈⁇‼‽.]/;
export function parseTranscriptXml(xml: string): Snippet[] {
const snippets: Snippet[] = [];
const pattern = /<text\s+start="([^"]*)"(?:\s+dur="([^"]*)")?[^>]*>([\s\S]*?)<\/text>/g;
let match: RegExpExecArray | null;
while ((match = pattern.exec(xml)) !== null) {
const raw = match[3];
if (!raw) continue;
snippets.push({
text: htmlUnescape(stripTags(raw)),
start: parseFloat(match[1]),
duration: parseFloat(match[2] || "0"),
});
}
return snippets;
}
export function parseTranscriptJson3(text: string): Snippet[] {
const data = JSON.parse(text);
const events = Array.isArray(data?.events) ? data.events : [];
const snippets: Snippet[] = [];
for (const event of events) {
const segs = Array.isArray(event?.segs) ? event.segs : [];
const textParts = segs
.map((seg: any) => htmlUnescape(String(seg?.utf8 || "").replace(/\n+/g, " ").trim()))
.filter(Boolean);
const merged = mergeTexts(textParts).trim();
if (!merged) continue;
snippets.push({
text: merged,
start: Number(event?.tStartMs || 0) / 1000,
duration: Number(event?.dDurationMs || 0) / 1000,
});
}
return snippets;
}
function parseSrt(srt: string): Snippet[] {
const blocks = srt.trim().split(/\n\n+/);
const snippets: Snippet[] = [];
for (const block of blocks) {
const lines = block.split("\n");
if (lines.length < 3) continue;
const match = lines[1].match(/(\d{2}):(\d{2}):(\d{2}),(\d{3})\s*-->\s*(\d{2}):(\d{2}):(\d{2}),(\d{3})/);
if (!match) continue;
const start = parseInt(match[1]) * 3600 + parseInt(match[2]) * 60 + parseInt(match[3]) + parseInt(match[4]) / 1000;
const end = parseInt(match[5]) * 3600 + parseInt(match[6]) * 60 + parseInt(match[7]) + parseInt(match[8]) / 1000;
snippets.push({ text: lines.slice(2).join(" "), start, duration: end - start });
}
return snippets;
}
export function parseWebVtt(vtt: string): Snippet[] {
const blocks = vtt
.replace(/^WEBVTT\s*/m, "")
.trim()
.split(/\n\n+/);
const snippets: Snippet[] = [];
for (const block of blocks) {
const lines = block.split("\n").map((line) => line.trim()).filter(Boolean);
const tsLine = lines.find((line) => line.includes("-->"));
if (!tsLine) continue;
const match = tsLine.match(
/(?:(\d{2}):)?(\d{2}):(\d{2})\.(\d{3})\s*-->\s*(?:(\d{2}):)?(\d{2}):(\d{2})\.(\d{3})/
);
if (!match) continue;
const start =
(match[1] ? parseInt(match[1]) : 0) * 3600 +
parseInt(match[2]) * 60 +
parseInt(match[3]) +
parseInt(match[4]) / 1000;
const end =
(match[5] ? parseInt(match[5]) : 0) * 3600 +
parseInt(match[6]) * 60 +
parseInt(match[7]) +
parseInt(match[8]) / 1000;
const text = htmlUnescape(stripTags(lines.slice(lines.indexOf(tsLine) + 1).join(" ").replace(/\s+/g, " ").trim()));
if (!text) continue;
snippets.push({ text, start, duration: end - start });
}
return snippets;
}
export function parseTranscriptPayload(payload: string, url: string): Snippet[] {
const normalized = payload.trimStart();
if (url.includes("fmt=json3") || normalized.startsWith("{")) return parseTranscriptJson3(payload);
if (normalized.startsWith("WEBVTT")) return parseWebVtt(payload);
if (/^\d+\s*\n\d{2}:\d{2}:\d{2},\d{3}\s*-->/.test(normalized)) return parseSrt(payload);
return parseTranscriptXml(payload);
}
function isCJK(ch: string): boolean {
const code = ch.charCodeAt(0);
return (code >= 0x4E00 && code <= 0x9FFF) ||
(code >= 0x3040 && code <= 0x309F) ||
(code >= 0x30A0 && code <= 0x30FF) ||
(code >= 0xAC00 && code <= 0xD7AF) ||
(code >= 0x3400 && code <= 0x4DBF) ||
(code >= 0xF900 && code <= 0xFAFF);
}
function splitSnippetAtPunctuation(snippet: Snippet): { text: string; start: number; end: number }[] {
const { text, start, duration } = snippet;
const end = start + duration;
if (!text.length) return [];
const splitPoints: number[] = [];
for (let i = 0; i < text.length; i++) {
if (SENTENCE_END_RE.test(text[i])) {
while (i + 1 < text.length && SENTENCE_END_RE.test(text[i + 1])) i++;
if (i < text.length - 1) splitPoints.push(i);
}
}
if (!splitPoints.length) return [{ text, start, end }];
const parts: { text: string; start: number; end: number }[] = [];
let prev = 0;
for (const pos of splitPoints) {
const partText = text.slice(prev, pos + 1).trim();
if (partText) {
parts.push({
text: partText,
start: start + (prev / text.length) * duration,
end: start + ((pos + 1) / text.length) * duration,
});
}
prev = pos + 1;
}
const remaining = text.slice(prev).trim();
if (remaining) parts.push({ text: remaining, start: start + (prev / text.length) * duration, end });
return parts;
}
function mergeTexts(texts: string[]): string {
if (!texts.length) return "";
let result = texts[0];
for (let i = 1; i < texts.length; i++) {
const next = texts[i];
if (!next) continue;
const lastChar = result[result.length - 1];
const firstChar = next[0];
if (isCJK(lastChar) || isCJK(firstChar)) {
result += next;
} else {
result = result.trimEnd() + " " + next.trimStart();
}
}
return result.replace(/ {2,}/g, " ");
}
export function ts(time: number): string {
const h = Math.floor(time / 3600);
const m = Math.floor((time % 3600) / 60);
const s = Math.floor(time % 60);
return `${String(h).padStart(2, "0")}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}`;
}
function tsMs(time: number, sep: string): string {
const h = Math.floor(time / 3600);
const m = Math.floor((time % 3600) / 60);
const s = Math.floor(time % 60);
const ms = Math.round((time - Math.floor(time)) * 1000);
return `${String(h).padStart(2, "0")}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}${sep}${String(ms).padStart(3, "0")}`;
}
function parseTs(time: string): number {
const [h, m, s] = time.split(":").map(Number);
return h * 3600 + m * 60 + s;
}
export function segmentIntoSentences(snippets: Snippet[]): Sentence[] {
const parts: { text: string; start: number; end: number }[] = [];
for (const snippet of snippets) parts.push(...splitSnippetAtPunctuation(snippet));
const sentences: Sentence[] = [];
let buffer: { text: string; start: number; end: number }[] = [];
for (const part of parts) {
buffer.push(part);
if (SENTENCE_END_RE.test(part.text[part.text.length - 1])) {
sentences.push({
text: mergeTexts(buffer.map((entry) => entry.text)),
start: ts(buffer[0].start),
end: ts(buffer[buffer.length - 1].end),
});
buffer = [];
}
}
if (buffer.length) {
sentences.push({
text: mergeTexts(buffer.map((entry) => entry.text)),
start: ts(buffer[0].start),
end: ts(buffer[buffer.length - 1].end),
});
}
return sentences;
}
function groupSentenceParas(sentences: Sentence[]): Paragraph[] {
if (!sentences.length) return [];
const paragraphs: Paragraph[] = [];
let buffer: Sentence[] = [];
for (let i = 0; i < sentences.length; i++) {
buffer.push(sentences[i]);
const last = i === sentences.length - 1;
const gap = !last && parseTs(sentences[i + 1].start) - parseTs(sentences[i].end) > 2;
if (last || gap || buffer.length >= 5) {
paragraphs.push({
text: mergeTexts(buffer.map((sentence) => sentence.text)),
start: parseTs(buffer[0].start),
end: parseTs(buffer[buffer.length - 1].end),
});
buffer = [];
}
}
return paragraphs;
}
export function formatSrt(snippets: Snippet[]): string {
return snippets
.map((snippet, index) => {
const end = index < snippets.length - 1 && snippets[index + 1].start < snippet.start + snippet.duration
? snippets[index + 1].start
: snippet.start + snippet.duration;
return `${index + 1}\n${tsMs(snippet.start, ",")} --> ${tsMs(end, ",")}\n${snippet.text}`;
})
.join("\n\n") + "\n";
}
function yamlEscape(value: string): string {
if (/[:"'{}\[\]#&*!|>%@`\n]/.test(value) || value.trim() !== value) {
return `"${value.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"`;
}
return value;
}
function extractSummary(description: string): string {
if (!description) return "";
const firstPara = description.split(/\n\s*\n/)[0].trim();
const lines = firstPara.split("\n").filter((line) => !/^\s*(https?:\/\/|#|@|\d+:\d+)/.test(line) && line.trim());
return lines.join(" ").slice(0, 300).trim();
}
export function formatMarkdown(
sentences: Sentence[],
meta: VideoMeta,
opts: { timestamps: boolean; chapters: boolean; speakers: boolean },
snippets?: Snippet[]
): string {
const summary = extractSummary(meta.description);
let md = "---\n";
md += `title: ${yamlEscape(meta.title)}\n`;
md += `channel: ${yamlEscape(meta.channel)}\n`;
if (meta.publishDate) md += `date: ${meta.publishDate}\n`;
md += `url: ${yamlEscape(meta.url)}\n`;
if (meta.coverImage) md += `cover: ${meta.coverImage}\n`;
if (summary) md += `description: ${yamlEscape(summary)}\n`;
if (meta.language) md += `language: ${meta.language.code}\n`;
md += "---\n\n";
if (opts.speakers) {
md += `# ${meta.title}\n\n`;
if (summary) md += `${summary}\n\n`;
if (meta.description) md += `# Description\n\n${meta.description.trim()}\n\n`;
if (meta.chapters.length) {
md += "# Chapters\n\n";
for (const chapter of meta.chapters) md += `* [${ts(chapter.start)}] ${chapter.title}\n`;
md += "\n";
}
md += "# Transcript\n\n";
md += snippets ? formatSrt(snippets) : "";
return md;
}
md += `# ${meta.title}\n\n`;
if (summary) md += `${summary}\n\n`;
const chapters = opts.chapters ? meta.chapters : [];
if (chapters.length) {
md += "## Table of Contents\n\n";
for (const chapter of chapters) md += opts.timestamps ? `* [${ts(chapter.start)}] ${chapter.title}\n` : `* ${chapter.title}\n`;
md += "\n";
if (meta.coverImage) md += `\n![cover](${meta.coverImage})\n`;
md += "\n";
for (let i = 0; i < chapters.length; i++) {
const nextStart = i < chapters.length - 1 ? chapters[i + 1].start : Infinity;
const chapterSentences = sentences.filter((sentence) => parseTs(sentence.start) >= chapters[i].start && parseTs(sentence.start) < nextStart);
const paragraphs = groupSentenceParas(chapterSentences);
md += opts.timestamps ? `## [${ts(chapters[i].start)}] ${chapters[i].title}\n\n` : `## ${chapters[i].title}\n\n`;
for (const paragraph of paragraphs) {
md += opts.timestamps ? `${paragraph.text} [${ts(paragraph.start)}${ts(paragraph.end)}]\n\n` : `${paragraph.text}\n\n`;
}
md += "\n";
}
} else {
const paragraphs = groupSentenceParas(sentences);
for (const paragraph of paragraphs) {
md += opts.timestamps ? `${paragraph.text} [${ts(paragraph.start)}${ts(paragraph.end)}]\n\n` : `${paragraph.text}\n\n`;
}
}
return md.trimEnd() + "\n";
}
export function formatListOutput(videoId: string, title: string, transcripts: TranscriptInfo[]): string {
const manual = transcripts.filter((transcript) => !transcript.isGenerated);
const generated = transcripts.filter((transcript) => transcript.isGenerated);
const translationLanguages = transcripts.find((transcript) => transcript.translationLanguages.length > 0)?.translationLanguages || [];
const formatList = (list: TranscriptInfo[]) =>
list.length
? list.map((transcript) => ` - ${transcript.languageCode} ("${transcript.language}")${transcript.isTranslatable ? " [TRANSLATABLE]" : ""}`).join("\n")
: "None";
const formatTranslations = translationLanguages.length
? translationLanguages.map((language) => ` - ${language.languageCode} ("${language.language}")`).join("\n")
: "None";
return `Transcripts for ${videoId}${title ? ` (${title})` : ""}:\n\n(MANUALLY CREATED)\n${formatList(manual)}\n\n(GENERATED)\n${formatList(generated)}\n\n(TRANSLATION LANGUAGES)\n${formatTranslations}`;
}
export function findTranscript(
transcripts: TranscriptInfo[],
languages: string[],
excludeGenerated: boolean,
excludeManual: boolean
): TranscriptInfo {
let filtered = transcripts;
if (excludeGenerated) filtered = filtered.filter((transcript) => !transcript.isGenerated);
if (excludeManual) filtered = filtered.filter((transcript) => transcript.isGenerated);
for (const language of languages) {
const found = filtered.find((transcript) => transcript.languageCode === language);
if (found) return found;
}
const available = filtered.map((transcript) => `${transcript.languageCode} ("${transcript.language}")`).join(", ");
throw makeError(`No transcript found for languages [${languages.join(", ")}]. Available: ${available || "none"}`, "NO_TRANSCRIPT");
}
@@ -0,0 +1,123 @@
export type Format = "text" | "srt";
export interface Options {
videoIds: string[];
languages: string[];
format: Format;
translate: string;
list: boolean;
excludeGenerated: boolean;
excludeManual: boolean;
output: string;
outputDir: string;
timestamps: boolean;
chapters: boolean;
speakers: boolean;
refresh: boolean;
}
export interface Snippet {
text: string;
start: number;
duration: number;
}
export interface Sentence {
text: string;
start: string;
end: string;
}
export interface TranscriptLanguage {
language: string;
languageCode: string;
}
export interface TranscriptInfo {
language: string;
languageCode: string;
isGenerated: boolean;
isTranslatable: boolean;
baseUrl: string;
translationLanguages: TranscriptLanguage[];
}
export interface Chapter {
title: string;
start: number;
end: number;
}
export interface LanguageMeta {
code: string;
name: string;
isGenerated: boolean;
}
export interface VideoMeta {
videoId: string;
title: string;
channel: string;
channelId: string;
description: string;
duration: number;
publishDate: string;
url: string;
coverImage: string;
thumbnailUrl: string;
language: LanguageMeta;
chapters: Chapter[];
}
export interface VideoResult {
videoId: string;
title?: string;
filePath?: string;
content?: string;
error?: string;
}
export interface InnerTubeSession {
apiKey: string;
webClientVersion: string;
visitorData: string;
}
export interface InnerTubeClient {
id: string;
clientName: string;
clientVersion?: string;
clientHeaderName?: string;
userAgent: string;
extraContext?: Record<string, any>;
}
export interface TranscriptError extends Error {
code?: string;
}
export interface YtDlpTrack {
ext?: string;
url?: string;
name?: string;
}
export interface YtDlpInfo {
title?: string;
channel?: string;
channel_id?: string;
uploader?: string;
uploader_id?: string;
description?: string;
duration?: number;
upload_date?: string;
webpage_url?: string;
thumbnail?: string;
thumbnails?: { url?: string; width?: number; height?: number }[];
subtitles?: Record<string, YtDlpTrack[]>;
automatic_captions?: Record<string, YtDlpTrack[]>;
}
export type VideoSource =
| { kind: "innertube"; data: any; transcripts: TranscriptInfo[] }
| { kind: "yt-dlp"; info: YtDlpInfo; transcripts: TranscriptInfo[] };
@@ -0,0 +1,477 @@
import { spawnSync } from "child_process";
import { writeFileSync } from "fs";
import { makeError, normalizeError, normalizePublishDate, shouldTryAlternateClient, shouldTryYtDlpFallback } from "./shared.ts";
import { parseTranscriptPayload } from "./transcript.ts";
import type {
Chapter,
InnerTubeClient,
InnerTubeSession,
LanguageMeta,
Snippet,
TranscriptInfo,
VideoMeta,
VideoSource,
YtDlpInfo,
YtDlpTrack,
} from "./types.ts";
const WATCH_URL = "https://www.youtube.com/watch?v=";
const INNERTUBE_URL = "https://www.youtube.com/youtubei/v1/player";
const WATCH_PAGE_USER_AGENT =
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36";
const DEFAULT_WEB_CLIENT_VERSION = "2.20260320.08.00";
const YT_DLP_MAX_BUFFER = 32 * 1024 * 1024;
let cachedYtDlpCommand: { command: string; args: string[]; label: string } | null | undefined;
const INNER_TUBE_CLIENTS: InnerTubeClient[] = [
{
id: "android",
clientName: "ANDROID",
clientHeaderName: "3",
clientVersion: "20.10.38",
userAgent:
"com.google.android.youtube/20.10.38 (Linux; U; Android 14; en_US; Pixel 8 Pro; Build/AP1A.240405.002)",
extraContext: {
clientFormFactor: "SMALL_FORM_FACTOR",
androidSdkVersion: 34,
osName: "Android",
osVersion: "14",
platform: "MOBILE",
},
},
{
id: "web",
clientName: "WEB",
clientHeaderName: "1",
userAgent: WATCH_PAGE_USER_AGENT,
},
{
id: "ios",
clientName: "IOS",
clientHeaderName: "5",
clientVersion: "20.10.4",
userAgent:
"com.google.ios.youtube/20.10.4 (iPhone16,2; U; CPU iOS 18_3 like Mac OS X; en_US)",
extraContext: {
deviceMake: "Apple",
deviceModel: "iPhone16,2",
osName: "iPhone",
osVersion: "18.3.0.22D5054f",
platform: "MOBILE",
},
},
];
async function fetchHtml(videoId: string): Promise<string> {
const watchUrl = `${WATCH_URL}${videoId}&hl=en&persist_hl=1&has_verified=1&bpctr=9999999999`;
const baseHeaders = {
Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Cache-Control": "no-cache",
Pragma: "no-cache",
"User-Agent": WATCH_PAGE_USER_AGENT,
};
const response = await fetch(watchUrl, { headers: baseHeaders });
if (!response.ok) throw new Error(`HTTP ${response.status} fetching video page`);
let html = await response.text();
if (html.includes('action="https://consent.youtube.com/s"')) {
const consentValue = html.match(/name="v" value="(.*?)"/);
if (!consentValue) throw new Error("Failed to create consent cookie");
const consentResponse = await fetch(watchUrl, {
headers: {
...baseHeaders,
Cookie: `CONSENT=YES+${consentValue[1]}`,
},
});
if (!consentResponse.ok) throw new Error(`HTTP ${consentResponse.status} fetching video page (consent)`);
html = await consentResponse.text();
}
return html;
}
function extractSession(html: string, videoId: string): InnerTubeSession {
const apiKey = html.match(/"INNERTUBE_API_KEY":\s*"([a-zA-Z0-9_-]+)"/)?.[1];
if (!apiKey) {
if (html.includes('class="g-recaptcha"')) throw new Error(`IP blocked for ${videoId} (reCAPTCHA)`);
throw new Error(`Cannot extract API key for ${videoId}`);
}
const webClientVersion =
html.match(/"INNERTUBE_CLIENT_VERSION":\s*"([^"]+)"/)?.[1] ||
html.match(/"clientVersion":"([^"]+)"/)?.[1] ||
DEFAULT_WEB_CLIENT_VERSION;
const visitorData =
html.match(/"VISITOR_DATA":"([^"]+)"/)?.[1] ||
html.match(/"visitorData":"([^"]+)"/)?.[1] ||
"";
return { apiKey, webClientVersion, visitorData };
}
function buildInnerTubeContext(client: InnerTubeClient, session: InnerTubeSession, videoId: string) {
return {
context: {
client: {
hl: "en",
gl: "US",
utcOffsetMinutes: 0,
visitorData: session.visitorData,
clientName: client.clientName,
clientVersion: client.clientVersion || session.webClientVersion,
...client.extraContext,
},
request: { useSsl: true },
},
videoId,
};
}
async function fetchInnertubeData(videoId: string, session: InnerTubeSession, client: InnerTubeClient): Promise<any> {
const clientVersion = client.clientVersion || session.webClientVersion;
const headers: Record<string, string> = {
Accept: "application/json",
"Accept-Language": "en-US,en;q=0.9",
"Content-Type": "application/json",
Origin: "https://www.youtube.com",
Referer: `${WATCH_URL}${videoId}`,
"User-Agent": client.userAgent,
"X-YouTube-Client-Name": client.clientHeaderName || "1",
"X-YouTube-Client-Version": clientVersion,
};
if (session.visitorData) headers["X-Goog-Visitor-Id"] = session.visitorData;
const response = await fetch(`${INNERTUBE_URL}?key=${session.apiKey}&prettyPrint=false`, {
method: "POST",
headers,
body: JSON.stringify(buildInnerTubeContext(client, session, videoId)),
});
if (response.status === 429) throw new Error(`IP blocked for ${videoId} (429)`);
if (!response.ok) throw new Error(`HTTP ${response.status} from InnerTube API`);
return response.json();
}
function assertPlayability(data: any, videoId: string) {
const playabilityStatus = data?.playabilityStatus;
if (!playabilityStatus) return;
const status = playabilityStatus.status;
if (status === "OK" || !status) return;
const reason = playabilityStatus.reason || "";
const reasonLower = reason.toLowerCase();
if (status === "LOGIN_REQUIRED") {
if (reasonLower.includes("bot")) throw makeError(`Request blocked for ${videoId}: bot detected`, "BOT_DETECTED");
if (reasonLower.includes("inappropriate")) throw makeError(`Age restricted: ${videoId}`, "AGE_RESTRICTED");
}
if (status === "ERROR" && reasonLower.includes("unavailable")) {
if (videoId.startsWith("http")) throw makeError("Invalid video ID: pass the ID, not the URL", "INVALID_VIDEO_ID");
throw makeError(`Video unavailable: ${videoId}`, "VIDEO_UNAVAILABLE");
}
const subreasons = playabilityStatus.errorScreen?.playerErrorMessageRenderer?.subreason?.runs?.map((run: any) => run.text).join("") || "";
throw new Error(`Video unplayable (${videoId}): ${reason} ${subreasons}`.trim());
}
function extractCaptionsJson(data: any, videoId: string): any {
assertPlayability(data, videoId);
const captionsJson = data?.captions?.playerCaptionsTracklistRenderer;
if (!captionsJson || !captionsJson.captionTracks) throw makeError(`Transcripts disabled for ${videoId}`, "TRANSCRIPTS_DISABLED");
return captionsJson;
}
function buildTranscriptList(captionsJson: any): TranscriptInfo[] {
const translationLanguages = (captionsJson.translationLanguages || []).map((language: any) => ({
language: language.languageName?.runs?.[0]?.text || language.languageName?.simpleText || "",
languageCode: language.languageCode,
}));
return (captionsJson.captionTracks || []).map((track: any) => ({
language: track.name?.runs?.[0]?.text || track.name?.simpleText || "",
languageCode: track.languageCode,
isGenerated: track.kind === "asr",
isTranslatable: !!track.isTranslatable,
baseUrl: track.baseUrl || "",
translationLanguages: track.isTranslatable ? translationLanguages : [],
}));
}
export async function fetchTranscriptSnippets(
info: TranscriptInfo,
translateTo?: string
): Promise<{ snippets: Snippet[]; language: string; languageCode: string }> {
let url = info.baseUrl;
let language = info.language;
let languageCode = info.languageCode;
if (translateTo) {
if (!info.isTranslatable) throw new Error(`Transcript ${info.languageCode} is not translatable`);
const translatedLanguage = info.translationLanguages.find((entry) => entry.languageCode === translateTo);
if (!translatedLanguage) throw new Error(`Translation language ${translateTo} not available`);
url += `&tlang=${translateTo}`;
language = translatedLanguage.language;
languageCode = translateTo;
}
const response = await fetch(url, {
headers: {
"Accept-Language": "en-US,en;q=0.9",
"User-Agent": WATCH_PAGE_USER_AGENT,
},
});
if (!response.ok) throw new Error(`HTTP ${response.status} fetching transcript`);
return {
snippets: parseTranscriptPayload(await response.text(), url),
language,
languageCode,
};
}
export function detectYtDlpCommand(): { command: string; args: string[]; label: string } | null {
if (cachedYtDlpCommand !== undefined) return cachedYtDlpCommand;
const candidates = [
{ command: "yt-dlp", args: [], label: "yt-dlp" },
{ command: "uvx", args: ["--from", "yt-dlp", "yt-dlp"], label: "uvx --from yt-dlp yt-dlp" },
{ command: "python3", args: ["-m", "yt_dlp"], label: "python3 -m yt_dlp" },
];
for (const candidate of candidates) {
const probe = spawnSync(candidate.command, [...candidate.args, "--version"], {
encoding: "utf8",
maxBuffer: 1024 * 1024,
});
if (probe.status !== 0) continue;
const helpProbe = spawnSync(candidate.command, [...candidate.args, "--help"], {
encoding: "utf8",
maxBuffer: 2 * 1024 * 1024,
});
const helpText = `${helpProbe.stdout || ""}\n${helpProbe.stderr || ""}`;
const supportsRequiredFlags =
helpProbe.status === 0 &&
helpText.includes("--js-runtimes") &&
helpText.includes("--remote-components");
if (supportsRequiredFlags) {
cachedYtDlpCommand = candidate;
return candidate;
}
}
cachedYtDlpCommand = null;
return cachedYtDlpCommand;
}
export function selectYtDlpTrack(entries: YtDlpTrack[]): YtDlpTrack | null {
const preferredExts = ["json3", "srv3", "srv2", "srv1", "ttml", "vtt"];
for (const ext of preferredExts) {
const match = entries.find((entry) => entry.url && entry.ext === ext);
if (match) return match;
}
return entries.find((entry) => !!entry.url) || null;
}
export function buildTranscriptListFromYtDlp(info: YtDlpInfo): TranscriptInfo[] {
const translationLanguages = Object.entries(info.automatic_captions || {}).map(([languageCode, entries]) => ({
language: entries.find((entry) => entry.name)?.name || languageCode,
languageCode,
}));
const manual = Object.entries(info.subtitles || {}).flatMap(([languageCode, entries]) => {
const selected = selectYtDlpTrack(entries);
if (!selected?.url) return [];
return [{
language: selected.name || languageCode,
languageCode,
isGenerated: false,
isTranslatable: translationLanguages.length > 0,
baseUrl: selected.url,
translationLanguages,
}];
});
const generated = Object.entries(info.automatic_captions || {}).flatMap(([languageCode, entries]) => {
const selected = selectYtDlpTrack(entries);
if (!selected?.url) return [];
return [{
language: selected.name || languageCode,
languageCode,
isGenerated: true,
isTranslatable: translationLanguages.length > 0,
baseUrl: selected.url,
translationLanguages,
}];
});
return [...manual, ...generated];
}
function fetchYtDlpInfo(videoId: string): YtDlpInfo {
const command = detectYtDlpCommand();
if (!command) {
throw makeError(
`Request blocked for ${videoId}: bot detected. yt-dlp fallback unavailable (install yt-dlp or uv).`,
"YT_DLP_UNAVAILABLE"
);
}
const args = [
...command.args,
"-J",
"--skip-download",
"--js-runtimes",
"bun",
"--remote-components",
"ejs:github",
];
const cookiesFromBrowser = process.env.YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER?.trim();
if (cookiesFromBrowser) args.push("--cookies-from-browser", cookiesFromBrowser);
args.push(`${WATCH_URL}${videoId}`);
const result = spawnSync(command.command, args, {
encoding: "utf8",
maxBuffer: YT_DLP_MAX_BUFFER,
});
if (result.status !== 0) {
const stderr = (result.stderr || "").trim();
const stdout = (result.stdout || "").trim();
const detail = stderr || stdout || `exit ${result.status ?? "unknown"}`;
throw makeError(`yt-dlp fallback failed for ${videoId} (${command.label}): ${detail}`, "YT_DLP_FAILED");
}
return JSON.parse(result.stdout);
}
async function fetchInnertubeSource(videoId: string): Promise<VideoSource> {
const html = await fetchHtml(videoId);
const session = extractSession(html, videoId);
const attempts: string[] = [];
let lastError: Error | null = null;
for (const client of INNER_TUBE_CLIENTS) {
try {
const data = await fetchInnertubeData(videoId, session, client);
const captionsJson = extractCaptionsJson(data, videoId);
return { kind: "innertube", data, transcripts: buildTranscriptList(captionsJson) };
} catch (error) {
const normalized = normalizeError(error);
attempts.push(`${client.id}: ${normalized.message}`);
lastError = normalized;
if (!shouldTryAlternateClient(normalized)) break;
}
}
if (!lastError) throw makeError(`Unable to fetch transcript metadata for ${videoId}`, "UNKNOWN");
if (attempts.length > 1) {
throw makeError(`${lastError.message}. Tried clients: ${attempts.join("; ")}`, normalizeError(lastError).code);
}
throw lastError;
}
export async function resolveVideoSource(
videoId: string,
fetchPrimary: (videoId: string) => Promise<VideoSource>,
fetchFallback: (videoId: string) => YtDlpInfo,
logWarning: (message: string) => void = (message) => console.error(message)
): Promise<VideoSource> {
try {
return await fetchPrimary(videoId);
} catch (error) {
const normalized = normalizeError(error);
if (!shouldTryYtDlpFallback(normalized)) throw normalized;
logWarning(`Warning (${videoId}): ${normalized.message}. Retrying with yt-dlp fallback.`);
const info = fetchFallback(videoId);
const transcripts = buildTranscriptListFromYtDlp(info);
if (!transcripts.length) throw makeError(`Transcripts disabled for ${videoId}`, "TRANSCRIPTS_DISABLED");
return { kind: "yt-dlp", info, transcripts };
}
}
export async function fetchVideoSource(videoId: string): Promise<VideoSource> {
return resolveVideoSource(videoId, fetchInnertubeSource, fetchYtDlpInfo);
}
export function parseChapters(description: string, duration: number = 0): Chapter[] {
const raw: { title: string; start: number }[] = [];
for (const line of description.split("\n")) {
const match = line.trim().match(/^(?:(\d{1,2}):)?(\d{1,2}):(\d{2})\s+(.+)$/);
if (match) {
const hours = match[1] ? parseInt(match[1]) : 0;
raw.push({ title: match[4].trim(), start: hours * 3600 + parseInt(match[2]) * 60 + parseInt(match[3]) });
}
}
if (raw.length < 2) return [];
return raw.map((chapter, index) => ({
title: chapter.title,
start: chapter.start,
end: index < raw.length - 1 ? raw[index + 1].start : Math.max(duration, chapter.start),
}));
}
export function getThumbnailUrls(videoId: string, data: any): string[] {
const urls = [
`https://i.ytimg.com/vi/${videoId}/maxresdefault.jpg`,
`https://i.ytimg.com/vi/${videoId}/hqdefault.jpg`,
];
const thumbnails = data?.videoDetails?.thumbnail?.thumbnails ||
data?.microformat?.playerMicroformatRenderer?.thumbnail?.thumbnails ||
[];
if (thumbnails.length) {
const sorted = [...thumbnails].sort((a: any, b: any) => (b.width || 0) - (a.width || 0));
for (const thumbnail of sorted) {
if (thumbnail.url && !urls.includes(thumbnail.url)) urls.push(thumbnail.url);
}
}
return urls;
}
export function getYtDlpThumbnailUrls(videoId: string, info: YtDlpInfo): string[] {
const urls = getThumbnailUrls(videoId, null);
const thumbnails = Array.isArray(info.thumbnails) ? info.thumbnails : [];
const sorted = [...thumbnails].sort((a, b) => (b?.width || 0) - (a?.width || 0));
for (const thumbnail of sorted) {
if (thumbnail?.url && !urls.includes(thumbnail.url)) urls.push(thumbnail.url);
}
if (info.thumbnail && !urls.includes(info.thumbnail)) urls.push(info.thumbnail);
return urls;
}
export function buildVideoMeta(data: any, videoId: string, language: LanguageMeta, chapters: Chapter[]): VideoMeta {
const videoDetails = data?.videoDetails || {};
const microformat = data?.microformat?.playerMicroformatRenderer || {};
return {
videoId,
title: videoDetails.title || microformat.title?.simpleText || "",
channel: videoDetails.author || microformat.ownerChannelName || "",
channelId: videoDetails.channelId || microformat.externalChannelId || "",
description: videoDetails.shortDescription || microformat.description?.simpleText || "",
duration: parseInt(videoDetails.lengthSeconds || "0"),
publishDate: microformat.publishDate || microformat.uploadDate || "",
url: `${WATCH_URL}${videoId}`,
coverImage: "",
thumbnailUrl: getThumbnailUrls(videoId, data)[0],
language,
chapters,
};
}
export function buildVideoMetaFromYtDlp(
info: YtDlpInfo,
videoId: string,
language: LanguageMeta,
chapters: Chapter[]
): VideoMeta {
return {
videoId,
title: info.title || "",
channel: info.channel || info.uploader || "",
channelId: info.channel_id || info.uploader_id || "",
description: info.description || "",
duration: Number(info.duration || 0),
publishDate: normalizePublishDate(info.upload_date),
url: info.webpage_url || `${WATCH_URL}${videoId}`,
coverImage: "",
thumbnailUrl: getYtDlpThumbnailUrls(videoId, info)[0] || "",
language,
chapters,
};
}
export async function downloadCoverImage(urls: string[], outputPath: string): Promise<boolean> {
for (const url of urls) {
try {
const response = await fetch(url);
if (response.ok) {
writeFileSync(outputPath, Buffer.from(await response.arrayBuffer()));
return true;
}
} catch {}
}
return false;
}