chore: release v1.81.0

feat(baoyu-youtube-transcript): add yt-dlp fallback and modularize codebase
Retry with alternate InnerTube client identities when YouTube returns anti-bot responses, then fall back to yt-dlp when available. Split monolithic main.ts into typed modules (youtube, transcript, storage, shared, types) and add unit tests.
2026-07-15 23:19:48 +08:00 · 2026-03-24 20:59:56 -05:00 · 2026-03-24 20:59:04 -05:00 · 2026-03-24 20:06:02 -05:00 · 2026-03-24 20:04:21 -05:00 · 2026-03-24 19:27:57 -05:00
50 changed files with 3493 additions and 868 deletions
@@ -6,48 +6,32 @@
  },
  "metadata": {
    "description": "Skills shared by Baoyu for improving daily work efficiency",
-    "version": "1.76.0"
+    "version": "1.81.0"
  },
  "plugins": [
    {
-      "name": "content-skills",
+      "name": "baoyu-skills",
-      "description": "Content generation and publishing skills",
+      "description": "Content generation, AI backends, and utility tools for daily work efficiency",
      "source": "./",
      "strict": true,
      "skills": [
        "./skills/baoyu-xhs-images",
        "./skills/baoyu-post-to-x",
        "./skills/baoyu-post-to-wechat",
        "./skills/baoyu-post-to-weibo",
        "./skills/baoyu-article-illustrator",
        "./skills/baoyu-cover-image",
        "./skills/baoyu-slide-deck",
        "./skills/baoyu-comic",
        "./skills/baoyu-infographic"
      ]
    },
    {
      "name": "ai-generation-skills",
      "description": "AI-powered generation backends",
      "source": "./",
      "strict": true,
      "skills": [
        "./skills/baoyu-danger-gemini-web",
        "./skills/baoyu-image-gen"
      ]
    },
    {
      "name": "utility-skills",
      "description": "Utility tools for content processing",
      "source": "./",
      "strict": true,
      "skills": [
        "./skills/baoyu-danger-x-to-markdown",
        "./skills/baoyu-compress-image",
-        "./skills/baoyu-url-to-markdown",
+        "./skills/baoyu-cover-image",
        "./skills/baoyu-danger-gemini-web",
        "./skills/baoyu-danger-x-to-markdown",
        "./skills/baoyu-format-markdown",
        "./skills/baoyu-image-gen",
        "./skills/baoyu-infographic",
        "./skills/baoyu-markdown-to-html",
        "./skills/baoyu-post-to-weibo",
        "./skills/baoyu-post-to-wechat",
        "./skills/baoyu-post-to-x",
        "./skills/baoyu-slide-deck",
        "./skills/baoyu-translate",
        "./skills/baoyu-url-to-markdown",
        "./skills/baoyu-xhs-images",
        "./skills/baoyu-youtube-transcript"
      ]
    }
@@ -2,6 +2,72 @@
 English | [中文](./CHANGELOG.zh.md)
 ## 1.81.0 - 2026-03-24
 ### Features
 - `baoyu-youtube-transcript`: add yt-dlp fallback when YouTube blocks direct InnerTube API, with alternate client identity retry and cookie support via `YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER` env var
 ### Refactor
 - `baoyu-youtube-transcript`: split monolithic script into typed modules (youtube, transcript, storage, shared, types) and add unit tests
 ## 1.80.1 - 2026-03-24
 ### Fixes
 - `baoyu-image-gen`: use correct `prompt` field name for Jimeng API request
 ## 1.80.0 - 2026-03-24
 ### Features
 - `baoyu-image-gen`: add Azure OpenAI as independent image generation provider with flexible endpoint parsing, deployment-name resolution, quality mapping, and reference image validation
 ## 1.79.2 - 2026-03-23
 ### Fixes
 - `baoyu-cover-image`: simplify reference image handling — use `--ref` when model supports it, only create description files for models without reference image support
 - `baoyu-post-to-weibo`: add no-theme rule for article markdown-to-HTML conversion
 ### Tests
 - Fix Node-compatible parser tests and add parser test dependencies
 ## 1.79.1 - 2026-03-23
 ### Fixes
 - Consolidate to single plugin to prevent duplicate skill registration (by @TyrealQ)
 - `baoyu-article-illustrator`: remove opacity parameter from watermark prompt
 - `baoyu-comic`: fix Doraemon naming spacing and remove opacity from watermark prompt
 - `baoyu-xhs-images`: remove opacity from watermark prompt and fix CJK spacing
 ### Documentation
 - Update project documentation to reflect single-plugin architecture
 ## 1.79.0 - 2026-03-22
 ### Features
 - `baoyu-post-to-wechat`: improve credential loading with multi-source resolution, priority ordering, and diagnostics for skipped incomplete sources
 ## 1.78.0 - 2026-03-22
 ### Features
 - `baoyu-url-to-markdown`: add URL-specific parser layer for X/Twitter and archive.ph sites
 - `baoyu-url-to-markdown`: improved slug generation with stop words removal and subdirectory output structure
 ### Fixes
 - `baoyu-url-to-markdown`: preserve anchor elements containing media in legacy converter
 - `baoyu-url-to-markdown`: smarter title deduplication to avoid redundant headings
 ## 1.77.0 - 2026-03-22
 ### Features
 - `baoyu-youtube-transcript`: add end times to chapter data (by @jzOcb)
 ### Fixes
 - `sync-clawhub`: skip failed skills instead of aborting
 ## 1.76.1 - 2026-03-21
 ### Documentation
 - `baoyu-youtube-transcript`: fix zsh glob issue — always single-quote YouTube URLs when running the script
 ## 1.76.0 - 2026-03-21
 ### Features
@@ -2,6 +2,72 @@
 [English](./CHANGELOG.md) | 中文
 ## 1.81.0 - 2026-03-24
 ### 新功能
 - `baoyu-youtube-transcript`：YouTube 封锁直连 InnerTube API 时自动回退到 yt-dlp，支持备用客户端身份重试及通过 `YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER` 环境变量传递浏览器 Cookie
 ### 重构
 - `baoyu-youtube-transcript`：将单体脚本拆分为类型化模块（youtube、transcript、storage、shared、types）并添加单元测试
 ## 1.80.1 - 2026-03-24
 ### 修复
 - `baoyu-image-gen`：修正即梦 API 请求中的 `prompt` 字段名
 ## 1.80.0 - 2026-03-24
 ### 新功能
 - `baoyu-image-gen`：新增 Azure OpenAI 作为独立图像生成服务商，支持灵活的端点解析、部署名称推断、质量映射及参考图片格式校验
 ## 1.79.2 - 2026-03-23
 ### 修复
 - `baoyu-cover-image`：简化参考图片处理流程 — 模型支持 `--ref` 时直接传递，仅在模型不支持参考图时创建描述文件
 - `baoyu-post-to-weibo`：文章 Markdown 转 HTML 时不传递 --theme 参数
 ### 测试
 - 修复 Node 兼容的解析器测试，添加解析器测试依赖
 ## 1.79.1 - 2026-03-23
 ### 修复
 - 合并为单一插件，防止 skill 重复注册 (by @TyrealQ)
 - `baoyu-article-illustrator`：移除水印提示词中的不透明度参数
 - `baoyu-comic`：修正哆啦 A 梦命名间距，移除水印不透明度参数
 - `baoyu-xhs-images`：移除水印不透明度参数，修正中英文间距
 ### 文档
 - 更新项目文档以反映单一插件架构
 ## 1.79.0 - 2026-03-22
 ### 新功能
 - `baoyu-post-to-wechat`：改进凭据加载机制，支持多来源优先级解析，并提供不完整凭据来源的诊断信息
 ## 1.78.0 - 2026-03-22
 ### 新功能
 - `baoyu-url-to-markdown`：新增 URL 专用解析层，支持 X/Twitter 和 archive.ph 站点的定制化 HTML 提取
 - `baoyu-url-to-markdown`：改进 slug 生成算法，去除停用词并采用子目录输出结构
 ### 修复
 - `baoyu-url-to-markdown`：旧版转换器保留包含媒体元素的锚标签
 - `baoyu-url-to-markdown`：更智能的标题去重，避免重复添加标题
 ## 1.77.0 - 2026-03-22
 ### 新功能
 - `baoyu-youtube-transcript`：为章节数据添加结束时间 (by @jzOcb)
 ### 修复
 - `sync-clawhub`：跳过失败的技能而不是中止同步
 ## 1.76.1 - 2026-03-21
 ### 文档
 - `baoyu-youtube-transcript`：修复 zsh glob 问题 — 运行脚本时始终对 YouTube URL 使用单引号
 ## 1.76.0 - 2026-03-21
 ### 新功能
@@ -1,16 +1,16 @@
 # CLAUDE.md
-Claude Code marketplace plugin providing AI-powered content generation skills. Version: **1.76.0**.
+Claude Code marketplace plugin providing AI-powered content generation skills. Version: **1.81.0**.
 ## Architecture
-Skills organized into three categories in `.claude-plugin/marketplace.json` (defines plugin metadata, version, and skill paths):
+Skills are exposed through the single `baoyu-skills` plugin in `.claude-plugin/marketplace.json` (which defines plugin metadata, version, and skill paths). The repo docs still group them into three logical areas:
-| Category | Description |
+| Group | Description |
-|----------|-------------|
+|-------|-------------|
-| `content-skills` | Generate or publish content (images, slides, comics, posts) |
+| Content Skills | Generate or publish content (images, slides, comics, posts) |
-| `ai-generation-skills` | AI generation backends |
+| AI Generation Skills | AI generation backends |
-| `utility-skills` | Content processing (conversion, compression, translation) |
+| Utility Skills | Content processing (conversion, compression, translation) |
 Each skill contains `SKILL.md` (YAML front matter + docs), optional `scripts/`, `references/`, `prompts/`.
@@ -31,7 +31,7 @@ Execute: `${BUN_X} skills/<skill>/scripts/main.ts [options]`
 - **Bun**: TypeScript runtime (`bun` preferred, fallback `npx -y bun`)
 - **Chrome**: Required for CDP-based skills (gemini-web, post-to-x/wechat/weibo, url-to-markdown). All CDP skills share a single profile, override via `BAOYU_CHROME_PROFILE_DIR` env var. Platform paths: [docs/chrome-profile.md](docs/chrome-profile.md)
- **Image generation APIs**: `baoyu-image-gen` requires API key (OpenAI, Google, OpenRouter, DashScope, or Replicate) configured in EXTEND.md
+- **Image generation APIs**: `baoyu-image-gen` requires API key (OpenAI, Azure OpenAI, Google, OpenRouter, DashScope, or Replicate) configured in EXTEND.md
 - **Gemini Web auth**: Browser cookies (first run opens Chrome for login, `--login` to refresh)
 ## Security
@@ -52,16 +52,14 @@ Run the following command in Claude Code:
 1. Select **Browse and install plugins**
 2. Select **baoyu-skills**
-3. Select the plugin(s) you want to install
+3. Select the **baoyu-skills** plugin
 4. Select **Install now**
 **Option 2: Direct Install**
 ```bash
-# Install specific plugin
+# Install the marketplace's single plugin
-/plugin install content-skills@baoyu-skills
+/plugin install baoyu-skills@baoyu-skills
 /plugin install ai-generation-skills@baoyu-skills
 /plugin install utility-skills@baoyu-skills
 ```
 **Option 3: Ask the Agent**
@@ -70,13 +68,13 @@ Simply tell Claude Code:
 > Please install Skills from github.com/JimLiu/baoyu-skills
-### Available Plugins
+### Available Plugin
-| Plugin | Description | Skills |
+The marketplace now exposes a single plugin so each skill is registered exactly once.
-|--------|-------------|--------|
+
-| **content-skills** | Content generation and publishing | [xhs-images](#baoyu-xhs-images), [infographic](#baoyu-infographic), [cover-image](#baoyu-cover-image), [slide-deck](#baoyu-slide-deck), [comic](#baoyu-comic), [article-illustrator](#baoyu-article-illustrator), [post-to-x](#baoyu-post-to-x), [post-to-wechat](#baoyu-post-to-wechat), [post-to-weibo](#baoyu-post-to-weibo) |
+| Plugin | Description | Includes |
-| **ai-generation-skills** | AI-powered generation backends | [image-gen](#baoyu-image-gen), [danger-gemini-web](#baoyu-danger-gemini-web) |
+|--------|-------------|----------|
-| **utility-skills** | Utility tools for content processing | [youtube-transcript](#baoyu-youtube-transcript), [url-to-markdown](#baoyu-url-to-markdown), [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image), [format-markdown](#baoyu-format-markdown), [markdown-to-html](#baoyu-markdown-to-html), [translate](#baoyu-translate) |
+| **baoyu-skills** | Content generation, AI backends, and utility tools for daily work efficiency | All skills in this repository, organized below as Content Skills, AI Generation Skills, and Utility Skills |
 ## Update Skills
@@ -665,7 +663,7 @@ AI-powered generation backends.
 #### baoyu-image-gen
-AI SDK-based image generation using OpenAI, Google, OpenRouter, DashScope (Aliyun Tongyi Wanxiang), Jimeng (即梦), Seedream (豆包), and Replicate APIs. Supports text-to-image, reference images, aspect ratios, and quality presets.
+AI SDK-based image generation using OpenAI, Azure OpenAI, Google, OpenRouter, DashScope (Aliyun Tongyi Wanxiang), Jimeng (即梦), Seedream (豆包), and Replicate APIs. Supports text-to-image, reference images, aspect ratios, and quality presets.
 ```bash
 # Basic generation (auto-detect provider)
@@ -680,6 +678,9 @@ AI SDK-based image generation using OpenAI, Google, OpenRouter, DashScope (Aliyu
 # Specific provider
 /baoyu-image-gen --prompt "A cat" --image cat.png --provider openai
 # Azure OpenAI (model = deployment name)
 /baoyu-image-gen --prompt "A cat" --image cat.png --provider azure --model gpt-image-1.5
 # OpenRouter
 /baoyu-image-gen --prompt "A cat" --image cat.png --provider openrouter
@@ -695,7 +696,7 @@ AI SDK-based image generation using OpenAI, Google, OpenRouter, DashScope (Aliyu
 # Seedream (豆包)
 /baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider seedream
-# With reference images (Google, OpenAI, OpenRouter, Replicate, or Seedream 5.0/4.5/4.0)
+# With reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate, or Seedream 5.0/4.5/4.0)
 /baoyu-image-gen --prompt "Make it blue" --image out.png --ref source.png
 ```
@@ -52,16 +52,14 @@ clawhub install baoyu-markdown-to-html
 1. 选择 **Browse and install plugins**
 2. 选择 **baoyu-skills**
-3. 选择要安装的插件
+3. 选择 **baoyu-skills** 插件
 4. 选择 **Install now**
 **方式二：直接安装**
 ```bash
-# 安装指定插件
+# 安装 marketplace 中唯一的插件
-/plugin install content-skills@baoyu-skills
+/plugin install baoyu-skills@baoyu-skills
 /plugin install ai-generation-skills@baoyu-skills
 /plugin install utility-skills@baoyu-skills
 ```
 **方式三：告诉 Agent**
@@ -72,11 +70,11 @@ clawhub install baoyu-markdown-to-html
 ### 可用插件
-| 插件 | 说明 | 包含技能 |
+现在 marketplace 只暴露一个插件，这样每个 skill 只会注册一次。
 | 插件 | 说明 | 包含内容 |
 |------|------|----------|
-| **content-skills** | 内容生成和发布 | [xhs-images](#baoyu-xhs-images), [infographic](#baoyu-infographic), [cover-image](#baoyu-cover-image), [slide-deck](#baoyu-slide-deck), [comic](#baoyu-comic), [article-illustrator](#baoyu-article-illustrator), [post-to-x](#baoyu-post-to-x), [post-to-wechat](#baoyu-post-to-wechat), [post-to-weibo](#baoyu-post-to-weibo) |
+| **baoyu-skills** | 提供内容生成、AI 后端和日常效率工具技能 | 仓库中的全部 skills，仍按下方的内容技能、AI 生成技能、工具技能三个分类展示 |
 | **ai-generation-skills** | AI 生成后端 | [image-gen](#baoyu-image-gen), [danger-gemini-web](#baoyu-danger-gemini-web) |
 | **utility-skills** | 内容处理工具 | [youtube-transcript](#baoyu-youtube-transcript), [url-to-markdown](#baoyu-url-to-markdown), [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image), [format-markdown](#baoyu-format-markdown), [markdown-to-html](#baoyu-markdown-to-html), [translate](#baoyu-translate) |
 ## 更新技能
@@ -665,7 +663,7 @@ AI 驱动的生成后端。
 #### baoyu-image-gen
-基于 AI SDK 的图像生成，支持 OpenAI、Google、OpenRouter、DashScope（阿里通义万相）、即梦（Jimeng）、豆包（Seedream）和 Replicate API。支持文生图、参考图、宽高比和质量预设。
+基于 AI SDK 的图像生成，支持 OpenAI、Azure OpenAI、Google、OpenRouter、DashScope（阿里通义万相）、即梦（Jimeng）、豆包（Seedream）和 Replicate API。支持文生图、参考图、宽高比和质量预设。
 ```bash
 # 基础生成（自动检测服务商）
@@ -680,6 +678,9 @@ AI 驱动的生成后端。
 # 指定服务商
 /baoyu-image-gen --prompt "一只猫" --image cat.png --provider openai
 # Azure OpenAI（model 为部署名称）
 /baoyu-image-gen --prompt "一只猫" --image cat.png --provider azure --model gpt-image-1.5
 # OpenRouter
 /baoyu-image-gen --prompt "一只猫" --image cat.png --provider openrouter
@@ -695,7 +696,7 @@ AI 驱动的生成后端。
 # 豆包（Seedream）
 /baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider seedream
-# 带参考图（Google、OpenAI、OpenRouter、Replicate 或 Seedream 5.0/4.5/4.0）
+# 带参考图（Google、OpenAI、Azure OpenAI、OpenRouter、Replicate 或 Seedream 5.0/4.5/4.0）
 /baoyu-image-gen --prompt "把它变成蓝色" --image out.png --ref source.png
 ```
@@ -34,20 +34,22 @@ metadata:
 1. Create `skills/baoyu-<name>/SKILL.md` with YAML front matter
 2. Add TypeScript in `skills/baoyu-<name>/scripts/` (if applicable)
 3. Add prompt templates in `skills/baoyu-<name>/prompts/` if needed
-4. Register in `marketplace.json` under appropriate category
+4. Register the skill in `.claude-plugin/marketplace.json` under the `baoyu-skills` plugin entry
 5. Add Script Directory section to SKILL.md if skill has scripts
 6. Add openclaw metadata to frontmatter
-## Category Selection
+## Skill Grouping
-| If your skill... | Use category |
+All skills are registered under the single `baoyu-skills` plugin. Use these logical groups when deciding where the skill should appear in the docs:
 |------------------|--------------|
 | Generates visual content (images, slides, comics) | `content-skills` |
 | Publishes to platforms (X, WeChat, Weibo) | `content-skills` |
 | Provides AI generation backend | `ai-generation-skills` |
 | Converts or processes content | `utility-skills` |
-New category: add plugin object to `marketplace.json` with `name`, `description`, `skills[]`.
+| If your skill... | Use group |
 |------------------|-----------|
 | Generates visual content (images, slides, comics) | Content Skills |
 | Publishes to platforms (X, WeChat, Weibo) | Content Skills |
 | Provides AI generation backend | AI Generation Skills |
 | Converts or processes content | Utility Skills |
 If you add a new logical group, update the docs that present grouped skills, but keep the skill registered under the single `baoyu-skills` plugin entry.
 ## Writing Descriptions
@@ -9,7 +9,11 @@
        "packages/*"
      ],
      "devDependencies": {
-        "tsx": "^4.20.5"
+        "@mozilla/readability": "^0.6.0",
        "linkedom": "^0.18.12",
        "tsx": "^4.20.5",
        "turndown": "^7.2.2",
        "turndown-plugin-gfm": "^1.0.2"
      }
    },
    "node_modules/@esbuild/aix-ppc64": {
@@ -454,6 +458,23 @@
        "node": ">=18"
      }
    },
    "node_modules/@mixmark-io/domino": {
      "version": "2.2.0",
      "resolved": "https://registry.npmjs.org/@mixmark-io/domino/-/domino-2.2.0.tgz",
      "integrity": "sha512-Y28PR25bHXUg88kCV7nivXrP2Nj2RueZ3/l/jdx6J9f8J4nsEGcgX0Qe6lt7Pa+J79+kPiJU3LguR6O/6zrLOw==",
      "dev": true,
      "license": "BSD-2-Clause"
    },
    "node_modules/@mozilla/readability": {
      "version": "0.6.0",
      "resolved": "https://registry.npmjs.org/@mozilla/readability/-/readability-0.6.0.tgz",
      "integrity": "sha512-juG5VWh4qAivzTAeMzvY9xs9HY5rAcr2E4I7tiSSCokRFi7XIZCAu92ZkSTsIj1OPceCifL3cpfteP3pDT9/QQ==",
      "dev": true,
      "license": "Apache-2.0",
      "engines": {
        "node": ">=14.0.0"
      }
    },
    "node_modules/@types/debug": {
      "version": "4.1.12",
      "resolved": "https://registry.npmjs.org/@types/debug/-/debug-4.1.12.tgz",
@@ -615,6 +636,13 @@
        "url": "https://github.com/sponsors/fb55"
      }
    },
    "node_modules/cssom": {
      "version": "0.5.0",
      "resolved": "https://registry.npmjs.org/cssom/-/cssom-0.5.0.tgz",
      "integrity": "sha512-iKuQcq+NdHqlAcwUY0o/HL69XQrUaQdMjmStJ8JFmUaiiQErlhrmuigkg/CU4E2J0IyUKUrMAgl36TvN67MqTw==",
      "dev": true,
      "license": "MIT"
    },
    "node_modules/debug": {
      "version": "4.4.3",
      "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
@@ -896,6 +924,13 @@
        "node": ">=12.0.0"
      }
    },
    "node_modules/html-escaper": {
      "version": "3.0.3",
      "resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-3.0.3.tgz",
      "integrity": "sha512-RuMffC89BOWQoY0WKGpIhn5gX3iI54O6nRA0yC124NYVtzjmFWBIiFd8M0x+ZdX0P9R4lADg1mgP8C7PxGOWuQ==",
      "dev": true,
      "license": "MIT"
    },
    "node_modules/htmlparser2": {
      "version": "9.1.0",
      "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-9.1.0.tgz",
@@ -984,6 +1019,51 @@
        "node": ">=18.17"
      }
    },
    "node_modules/linkedom": {
      "version": "0.18.12",
      "resolved": "https://registry.npmjs.org/linkedom/-/linkedom-0.18.12.tgz",
      "integrity": "sha512-jalJsOwIKuQJSeTvsgzPe9iJzyfVaEJiEXl+25EkKevsULHvMJzpNqwvj1jOESWdmgKDiXObyjOYwlUqG7wo1Q==",
      "dev": true,
      "license": "ISC",
      "dependencies": {
        "css-select": "^5.1.0",
        "cssom": "^0.5.0",
        "html-escaper": "^3.0.3",
        "htmlparser2": "^10.0.0",
        "uhyphen": "^0.2.0"
      },
      "engines": {
        "node": ">=16"
      },
      "peerDependencies": {
        "canvas": ">= 2"
      },
      "peerDependenciesMeta": {
        "canvas": {
          "optional": true
        }
      }
    },
    "node_modules/linkedom/node_modules/htmlparser2": {
      "version": "10.1.0",
      "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-10.1.0.tgz",
      "integrity": "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==",
      "dev": true,
      "funding": [
        "https://github.com/fb55/htmlparser2?sponsor=1",
        {
          "type": "github",
          "url": "https://github.com/sponsors/fb55"
        }
      ],
      "license": "MIT",
      "dependencies": {
        "domelementtype": "^2.3.0",
        "domhandler": "^5.0.3",
        "domutils": "^3.2.2",
        "entities": "^7.0.1"
      }
    },
    "node_modules/longest-streak": {
      "version": "3.1.0",
      "resolved": "https://registry.npmjs.org/longest-streak/-/longest-streak-3.1.0.tgz",
@@ -1768,6 +1848,30 @@
        "fsevents": "~2.3.3"
      }
    },
    "node_modules/turndown": {
      "version": "7.2.2",
      "resolved": "https://registry.npmjs.org/turndown/-/turndown-7.2.2.tgz",
      "integrity": "sha512-1F7db8BiExOKxjSMU2b7if62D/XOyQyZbPKq/nUwopfgnHlqXHqQ0lvfUTeUIr1lZJzOPFn43dODyMSIfvWRKQ==",
      "dev": true,
      "license": "MIT",
      "dependencies": {
        "@mixmark-io/domino": "^2.2.0"
      }
    },
    "node_modules/turndown-plugin-gfm": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/turndown-plugin-gfm/-/turndown-plugin-gfm-1.0.2.tgz",
      "integrity": "sha512-vwz9tfvF7XN/jE0dGoBei3FXWuvll78ohzCZQuOb+ZjWrs3a0XhQVomJEb2Qh4VHTPNRO4GPZh0V7VRbiWwkRg==",
      "dev": true,
      "license": "MIT"
    },
    "node_modules/uhyphen": {
      "version": "0.2.0",
      "resolved": "https://registry.npmjs.org/uhyphen/-/uhyphen-0.2.0.tgz",
      "integrity": "sha512-qz3o9CHXmJJPGBdqzab7qAYuW8kQGKNEuoHFYrBwV6hWIMcpAmxDLXojcHfFr9US1Pe6zUswEIJIbLI610fuqA==",
      "dev": true,
      "license": "ISC"
    },
    "node_modules/undici": {
      "version": "6.24.0",
      "resolved": "https://registry.npmjs.org/undici/-/undici-6.24.0.tgz",
@@ -10,6 +10,10 @@
    "test:coverage": "node --import tsx --experimental-test-coverage --test"
  },
  "devDependencies": {
    "@mozilla/readability": "^0.6.0",
    "linkedom": "^0.18.12",
    "turndown": "^7.2.2",
    "turndown-plugin-gfm": "^1.0.2",
    "tsx": "^4.20.5"
  }
 }
@@ -151,6 +151,9 @@ async function main() {
    .map((tag) => tag.trim())
    .filter(Boolean);
  let succeeded = 0;
  const failed = [];
  for (const candidate of actionable) {
    const version =
      candidate.status === "new"
@@ -158,20 +161,30 @@ async function main() {
        : bumpSemver(candidate.latestVersion, options.bump);
    console.log(`Publishing ${candidate.slug}@${version}`);
-    const files = await listTextFiles(candidate.folder);
+    try {
-    await publishSkill({
+      const files = await listTextFiles(candidate.folder);
-      registry,
+      await publishSkill({
-      token: config.token,
+        registry,
-      skill: candidate,
+        token: config.token,
-      files,
+        skill: candidate,
-      version,
+        files,
-      changelog: options.changelog,
+        version,
-      tags,
+        changelog: options.changelog,
-    });
+        tags,
      });
      succeeded++;
    } catch (err) {
      const msg = err instanceof Error ? err.message : String(err);
      console.error(`SKIPPED ${candidate.slug}: ${msg}`);
      failed.push(candidate.slug);
    }
  }
  console.log("");
-  console.log(`Uploaded ${actionable.length} skill(s).`);
+  console.log(`Uploaded ${succeeded}/${actionable.length} skill(s).`);
  if (failed.length > 0) {
    console.log(`Failed (${failed.length}): ${failed.join(", ")}`);
  }
 }
 function parseArgs(argv) {
@@ -280,5 +280,5 @@ TEXTURE: Halftone transitions between sides
 If watermark enabled in preferences, append:
 ```
-Include a subtle watermark "[content]" positioned at [position] with approximately [opacity*100]% visibility.
+Include a subtle watermark "[content]" positioned at [position].
 ```
@@ -278,7 +278,7 @@ Create storyboard and character definitions using the confirmed style from Step
 | Role | Character | Visual Description |
 |------|-----------|-------------------|
 | Student | 大雄 (Nobita) | Japanese boy, 10yo, round glasses, black hair parted in middle, yellow shirt, navy shorts |
-| Mentor | 哆啦A梦 (Doraemon) | Round blue robot cat, big white eyes, red nose, whiskers, white belly with 4D pocket, golden bell, no ears |
+| Mentor | 哆啦 A 梦 (Doraemon) | Round blue robot cat, big white eyes, red nose, whiskers, white belly with 4D pocket, golden bell, no ears |
 | Challenge | 胖虎 (Gian) | Stocky boy, rough features, small eyes, orange shirt |
 | Support | 静香 (Shizuka) | Cute girl, black short hair, pink dress, gentle expression |
@@ -359,8 +359,7 @@ Art: [art style] | Tone: [tone] | Layout: [layout type]
 **Watermark Application** (if enabled in preferences):
 Add to each prompt:
 ```
-Include a subtle watermark "[content]" positioned at [position]
+Include a subtle watermark "[content]" positioned at [position]. The watermark should
 with approximately [opacity*100]% visibility. The watermark should
 be legible but not distracting from the comic panels and storytelling.
 Ensure watermark does not overlap speech bubbles or key action.
 ```
@@ -452,8 +451,8 @@ When skill does NOT support reference images, create combined prompt files:
 ## Character Reference (maintain consistency)
 [Copy relevant sections from characters/characters.md here]
- 大雄: Japanese boy, round glasses, yellow shirt, navy shorts...
+- 大雄：Japanese boy, round glasses, yellow shirt, navy shorts...
- 哆啦A梦: Round blue robot cat, white belly, red nose, golden bell...
+- 哆啦 A 梦：Round blue robot cat, white belly, red nose, golden bell...
 ## Page Content
 [Original page prompt here]
@@ -162,15 +162,14 @@ if (Test-Path "$HOME/.baoyu-skills/baoyu-cover-image/EXTEND.md") { "user" }
 5. **Detect language**: Compare source, user input, EXTEND.md preference
 6. **Determine output directory**: Per File Structure rules
-**⚠️ People in Reference Images — MUST follow all 3 rules:**
+**⚠️ People in Reference Images:**
 If reference images contain **people** who should appear in the cover:
-1. **`usage: direct`** — MUST set in refs description file. NEVER use `style` or `palette` when people need to appear
+- **Model supports `--ref`** (default): Copy image to `refs/`, pass via `--ref` at generation. No description file needed — the model sees the face directly.
-2. **Per-character description** — MUST describe each person's distinctive features (hair, glasses, skin tone, clothing) in `refs/ref-NN-{slug}.md`. Vague descriptions like "a man" will fail
+- **Model does NOT support `--ref`** (Jimeng, Seedream 3.0): Create `refs/ref-NN-{slug}.md` with per-character description (hair, glasses, skin tone, clothing). Embed as MUST/REQUIRED instructions in prompt text.
 3. **`--ref` flag** — MUST pass reference image via `--ref` in Step 4 so the model sees actual faces
-See [reference-images.md § Character Analysis](references/workflow/reference-images.md) for description format.
+See [reference-images.md](references/workflow/reference-images.md) for full decision table.
 ### Step 2: Confirm Options ⚠️
@@ -16,17 +16,24 @@ Guide for processing user-provided reference images in cover generation.
 **If user provides file path**:
 1. Copy to `refs/ref-NN-{slug}.{ext}` (NN = 01, 02, ...)
-2. Create description: `refs/ref-NN-{slug}.md`
+2. **Only** create description file `refs/ref-NN-{slug}.md` when model does NOT support `--ref` (see below)
-3. Verify files exist before proceeding
+3. Verify image file exists before proceeding
-**Description File Format**:
+**When to create description file**:
 | Situation | Action |
 |-----------|--------|
 | Model supports `--ref` (Google, OpenAI, OpenRouter, Replicate, Seedream 4.0+) | Copy image only. **No description file needed.** Pass via `--ref` at generation. |
 | Model does NOT support `--ref` (Jimeng, Seedream 3.0) | Copy image + create description file. Embed description in prompt text. |
 **Description File Format** (only when needed):
 ```yaml
 ---
 ref_id: NN
 filename: ref-NN-{slug}.{ext}
 usage: direct | style | palette
 ---
-[User's description or auto-generated description]
+[Character or style description to embed in prompt]
 ```
 | Usage | When to Use |
@@ -1,6 +1,6 @@
 ---
 name: baoyu-image-gen
-description: AI image generation with OpenAI, Google, OpenRouter, DashScope, Jimeng, Seedream and Replicate APIs. Supports text-to-image, reference images, aspect ratios, and batch generation from saved prompt files. Sequential by default; use batch parallel generation when the user already has multiple prompts or wants stable multi-image throughput. Use when user asks to generate, create, or draw images.
+description: AI image generation with OpenAI, Azure OpenAI, Google, OpenRouter, DashScope, Jimeng, Seedream and Replicate APIs. Supports text-to-image, reference images, aspect ratios, and batch generation from saved prompt files. Sequential by default; use batch parallel generation when the user already has multiple prompts or wants stable multi-image throughput. Use when user asks to generate, create, or draw images.
 version: 1.56.3
 metadata:
  openclaw:
@@ -13,7 +13,7 @@ metadata:
 # Image Generation (AI SDK)
-Official API-based image generation. Supports OpenAI, Google, OpenRouter, DashScope (阿里通义万象), Jimeng (即梦), Seedream (豆包) and Replicate providers.
+Official API-based image generation. Supports OpenAI, Azure OpenAI, Google, OpenRouter, DashScope (阿里通义万象), Jimeng (即梦), Seedream (豆包) and Replicate providers.
 ## Script Directory
@@ -74,12 +74,15 @@ ${BUN_X} {baseDir}/scripts/main.ts --prompt "A cat" --image out.png --quality 2k
 # From prompt files
 ${BUN_X} {baseDir}/scripts/main.ts --promptfiles system.md content.md --image out.png
-# With reference images (Google, OpenAI, OpenRouter, Replicate, or Seedream 4.0/4.5/5.0)
+# With reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate, or Seedream 4.0/4.5/5.0)
 ${BUN_X} {baseDir}/scripts/main.ts --prompt "Make blue" --image out.png --ref source.png
 # With reference images (explicit provider/model)
 ${BUN_X} {baseDir}/scripts/main.ts --prompt "Make blue" --image out.png --provider google --model gemini-3-pro-image-preview --ref source.png
 # Azure OpenAI (model means deployment name)
 ${BUN_X} {baseDir}/scripts/main.ts --prompt "A cat" --image out.png --provider azure --model gpt-image-1.5
 # OpenRouter (recommended default model)
 ${BUN_X} {baseDir}/scripts/main.ts --prompt "A cat" --image out.png --provider openrouter
@@ -147,13 +150,13 @@ Paths in `promptFiles`, `image`, and `ref` are resolved relative to the batch fi
 | `--image <path>` | Output image path (required in single-image mode) |
 | `--batchfile <path>` | JSON batch file for multi-image generation |
 | `--jobs <count>` | Worker count for batch mode (default: auto, max from config, built-in default 10) |
-| `--provider google\|openai\|openrouter\|dashscope\|jimeng\|seedream\|replicate` | Force provider (default: auto-detect) |
+| `--provider google\|openai\|azure\|openrouter\|dashscope\|jimeng\|seedream\|replicate` | Force provider (default: auto-detect) |
-| `--model <id>`, `-m` | Model ID (Google: `gemini-3-pro-image-preview`; OpenAI: `gpt-image-1.5`; OpenRouter: `google/gemini-3.1-flash-image-preview`; DashScope: `qwen-image-2.0-pro`) |
+| `--model <id>`, `-m` | Model ID (Google: `gemini-3-pro-image-preview`; OpenAI: `gpt-image-1.5`; Azure: deployment name such as `gpt-image-1.5` or `image-prod`; OpenRouter: `google/gemini-3.1-flash-image-preview`; DashScope: `qwen-image-2.0-pro`) |
 | `--ar <ratio>` | Aspect ratio (e.g., `16:9`, `1:1`, `4:3`) |
 | `--size <WxH>` | Size (e.g., `1024x1024`) |
 | `--quality normal\|2k` | Quality preset (default: `2k`) |
 | `--imageSize 1K\|2K\|4K` | Image size for Google/OpenRouter (default: from quality) |
-| `--ref <files...>` | Reference images. Supported by Google multimodal, OpenAI GPT Image edits, OpenRouter multimodal models, Replicate, and Seedream 5.0/4.5/4.0. Not supported by Jimeng, Seedream 3.0, or removed SeedEdit 3.0 |
+| `--ref <files...>` | Reference images. Supported by Google multimodal, OpenAI GPT Image edits, Azure OpenAI edits (PNG/JPG only), OpenRouter multimodal models, Replicate, and Seedream 5.0/4.5/4.0. Not supported by Jimeng, Seedream 3.0, or removed SeedEdit 3.0 |
 | `--n <count>` | Number of images |
 | `--json` | JSON output |
@@ -162,6 +165,7 @@ Paths in `promptFiles`, `image`, and `ref` are resolved relative to the batch fi
 | Variable | Description |
 |----------|-------------|
 | `OPENAI_API_KEY` | OpenAI API key |
 | `AZURE_OPENAI_API_KEY` | Azure OpenAI API key |
 | `OPENROUTER_API_KEY` | OpenRouter API key |
 | `GOOGLE_API_KEY` | Google API key |
 | `DASHSCOPE_API_KEY` | DashScope API key (阿里云) |
@@ -170,6 +174,8 @@ Paths in `promptFiles`, `image`, and `ref` are resolved relative to the batch fi
 | `JIMENG_SECRET_ACCESS_KEY` | Jimeng (即梦) Volcengine secret key |
 | `ARK_API_KEY` | Seedream (豆包) Volcengine ARK API key |
 | `OPENAI_IMAGE_MODEL` | OpenAI model override |
 | `AZURE_OPENAI_DEPLOYMENT` | Azure default deployment name |
 | `AZURE_OPENAI_IMAGE_MODEL` | Backward-compatible alias for Azure default deployment/model name |
 | `OPENROUTER_IMAGE_MODEL` | OpenRouter model override (default: `google/gemini-3.1-flash-image-preview`) |
 | `GOOGLE_IMAGE_MODEL` | Google model override |
 | `DASHSCOPE_IMAGE_MODEL` | DashScope model override (default: `qwen-image-2.0-pro`) |
@@ -177,6 +183,8 @@ Paths in `promptFiles`, `image`, and `ref` are resolved relative to the batch fi
 | `JIMENG_IMAGE_MODEL` | Jimeng model override (default: jimeng_t2i_v40) |
 | `SEEDREAM_IMAGE_MODEL` | Seedream model override (default: doubao-seedream-5-0-260128) |
 | `OPENAI_BASE_URL` | Custom OpenAI endpoint |
 | `AZURE_OPENAI_BASE_URL` | Azure resource endpoint or deployment endpoint |
 | `AZURE_API_VERSION` | Azure image API version (default: `2025-04-01-preview`) |
 | `OPENROUTER_BASE_URL` | Custom OpenRouter endpoint (default: `https://openrouter.ai/api/v1`) |
 | `OPENROUTER_HTTP_REFERER` | Optional app/site URL for OpenRouter attribution |
 | `OPENROUTER_TITLE` | Optional app name for OpenRouter attribution |
@@ -201,6 +209,8 @@ Model priority (highest → lowest), applies to all providers:
 3. Env var: `<PROVIDER>_IMAGE_MODEL` (e.g., `GOOGLE_IMAGE_MODEL`)
 4. Built-in default
 For Azure, `--model` / `default_model.azure` should be the Azure deployment name. `AZURE_OPENAI_DEPLOYMENT` is the preferred env var, and `AZURE_OPENAI_IMAGE_MODEL` remains as a backward-compatible alias.
 **EXTEND.md overrides env vars**. If both EXTEND.md `default_model.google: "gemini-3-pro-image-preview"` and env var `GOOGLE_IMAGE_MODEL=gemini-3.1-flash-image-preview` exist, EXTEND.md wins.
 **Agent MUST display model info** before each generation:
@@ -47,6 +47,8 @@ options:
    description: "Gemini multimodal - high quality, reference images, flexible sizes"
  - label: "OpenAI"
    description: "GPT Image - consistent quality, reliable output"
  - label: "Azure OpenAI"
    description: "Azure-hosted GPT Image deployments with resource-specific routing"
  - label: "OpenRouter"
    description: "Router for Gemini/FLUX/OpenAI-compatible image models"
  - label: "DashScope"
@@ -87,6 +89,20 @@ options:
    description: "Strong text-to-image quality through OpenRouter"
 ```
 ### Question 2c: Default Azure Deployment
 Only show if user selected Azure OpenAI.
 ```yaml
 header: "Azure Deploy"
 question: "Default Azure image deployment name?"
 options:
  - label: "gpt-image-1.5 (Recommended)"
    description: "Best default if your Azure deployment uses the same name"
  - label: "gpt-image-1"
    description: "Previous GPT Image deployment name"
 ```
 ### Question 3: Default Quality
 ```yaml
@@ -130,6 +146,7 @@ default_image_size: null
 default_model:
  google: [selected google model or null]
  openai: null
  azure: [selected azure deployment or null]
  openrouter: [selected openrouter model or null]
  dashscope: null
  replicate: null
@@ -166,6 +183,23 @@ options:
    description: "Previous generation GPT Image model"
 ```
 ### Azure Deployment Selection
 ```yaml
 header: "Azure Deploy"
 question: "Choose a default Azure image deployment name?"
 options:
  - label: "gpt-image-1.5 (Recommended)"
    description: "Use when your Azure deployment name matches the GPT-image-1.5 model"
  - label: "gpt-image-1"
    description: "Use when your Azure deployment name matches GPT-image-1"
 ```
 Notes for Azure setup:
 - In `baoyu-image-gen`, Azure `--model` / `default_model.azure` should be the Azure deployment name, not just the underlying model family.
 - If the deployment name is custom, save that exact deployment name in `default_model.azure`.
 ### OpenRouter Model Selection
 ```yaml
@@ -230,6 +264,7 @@ After user selects a model:
 default_model:
  google: [value or null]
  openai: [value or null]
  azure: [value or null]
  openrouter: [value or null]
  dashscope: [value or null]
  replicate: [value or null]
@@ -11,7 +11,7 @@ description: EXTEND.md YAML schema for baoyu-image-gen user preferences
 ---
 version: 1
-default_provider: null      # google|openai|openrouter|dashscope|replicate|null (null = auto-detect)
+default_provider: null      # google|openai|azure|openrouter|dashscope|replicate|null (null = auto-detect)
 default_quality: null       # normal|2k|null (null = use default: 2k)
@@ -22,6 +22,7 @@ default_image_size: null    # 1K|2K|4K|null (Google/OpenRouter, overrides qualit
 default_model:
  google: null              # e.g., "gemini-3-pro-image-preview", "gemini-3.1-flash-image-preview"
  openai: null              # e.g., "gpt-image-1.5", "gpt-image-1"
  azure: null               # Azure deployment name, e.g., "gpt-image-1.5" or "image-prod"
  openrouter: null          # e.g., "google/gemini-3.1-flash-image-preview"
  dashscope: null           # e.g., "qwen-image-2.0-pro"
  replicate: null           # e.g., "google/nano-banana-pro"
@@ -38,6 +39,9 @@ batch:
    openai:
      concurrency: 3
      start_interval_ms: 1100
    azure:
      concurrency: 3
      start_interval_ms: 1100
    openrouter:
      concurrency: 3
      start_interval_ms: 1100
@@ -58,6 +62,7 @@ batch:
 | `default_image_size` | string\|null | null | Google/OpenRouter image size (overrides quality) |
 | `default_model.google` | string\|null | null | Google default model |
 | `default_model.openai` | string\|null | null | OpenAI default model |
 | `default_model.azure` | string\|null | null | Azure default deployment name |
 | `default_model.openrouter` | string\|null | null | OpenRouter default model |
 | `default_model.dashscope` | string\|null | null | DashScope default model |
 | `default_model.replicate` | string\|null | null | Replicate default model |
@@ -87,6 +92,7 @@ default_image_size: 2K
 default_model:
  google: "gemini-3-pro-image-preview"
  openai: "gpt-image-1.5"
  azure: "gpt-image-1.5"
  openrouter: "google/gemini-3.1-flash-image-preview"
  dashscope: "qwen-image-2.0-pro"
  replicate: "google/nano-banana-pro"
@@ -96,6 +102,9 @@ batch:
    replicate:
      concurrency: 5
      start_interval_ms: 700
    azure:
      concurrency: 3
      start_interval_ms: 1100
    openrouter:
      concurrency: 3
      start_interval_ms: 1100
@@ -123,6 +123,7 @@ default_image_size: 2K
 default_model:
  google: gemini-3-pro-image-preview
  openai: gpt-image-1.5
  azure: image-prod
 batch:
  max_workers: 8
  provider_limits:
@@ -131,6 +132,9 @@ batch:
      start_interval_ms: 900
    openai:
      concurrency: 4
    azure:
      concurrency: 1
      start_interval_ms: 1500
 `;
  const config = parseSimpleYaml(yaml);
@@ -142,6 +146,7 @@ batch:
  assert.equal(config.default_image_size, "2K");
  assert.equal(config.default_model?.google, "gemini-3-pro-image-preview");
  assert.equal(config.default_model?.openai, "gpt-image-1.5");
  assert.equal(config.default_model?.azure, "image-prod");
  assert.equal(config.batch?.max_workers, 8);
  assert.deepEqual(config.batch?.provider_limits?.google, {
    concurrency: 2,
@@ -150,6 +155,10 @@ batch:
  assert.deepEqual(config.batch?.provider_limits?.openai, {
    concurrency: 4,
  });
  assert.deepEqual(config.batch?.provider_limits?.azure, {
    concurrency: 1,
    start_interval_ms: 1500,
  });
 });
 test("mergeConfig only fills values missing from CLI args", () => {
@@ -203,6 +212,8 @@ test("detectProvider selects an available ref-capable provider for reference-ima
  useEnv(t, {
    GOOGLE_API_KEY: null,
    OPENAI_API_KEY: "openai-key",
    AZURE_OPENAI_API_KEY: null,
    AZURE_OPENAI_BASE_URL: null,
    OPENROUTER_API_KEY: null,
    DASHSCOPE_API_KEY: null,
    REPLICATE_API_TOKEN: null,
@@ -216,6 +227,27 @@ test("detectProvider selects an available ref-capable provider for reference-ima
  );
 });
 test("detectProvider selects Azure when only Azure credentials are configured", (t) => {
  useEnv(t, {
    GOOGLE_API_KEY: null,
    OPENAI_API_KEY: null,
    AZURE_OPENAI_API_KEY: "azure-key",
    AZURE_OPENAI_BASE_URL: "https://example.openai.azure.com",
    OPENROUTER_API_KEY: null,
    DASHSCOPE_API_KEY: null,
    REPLICATE_API_TOKEN: null,
    JIMENG_ACCESS_KEY_ID: null,
    JIMENG_SECRET_ACCESS_KEY: null,
    ARK_API_KEY: null,
  });
  assert.equal(detectProvider(makeArgs()), "azure");
  assert.equal(
    detectProvider(makeArgs({ referenceImages: ["ref.png"] })),
    "azure",
  );
 });
 test("detectProvider infers Seedream from model id and allows Seedream reference-image workflows", (t) => {
  useEnv(t, {
    GOOGLE_API_KEY: null,
@@ -60,6 +60,7 @@ const DEFAULT_PROVIDER_RATE_LIMITS: Record<Provider, ProviderRateLimit> = {
  dashscope: { concurrency: 3, startIntervalMs: 1100 },
  jimeng: { concurrency: 3, startIntervalMs: 1100 },
  seedream: { concurrency: 3, startIntervalMs: 1100 },
  azure: { concurrency: 3, startIntervalMs: 1100 },
 };
 function printUsage(): void {
@@ -74,13 +75,13 @@ Options:
  --image <path>            Output image path (required in single-image mode)
  --batchfile <path>        JSON batch file for multi-image generation
  --jobs <count>            Worker count for batch mode (default: auto, max from config, built-in default 10)
-  --provider google|openai|openrouter|dashscope|replicate|jimeng|seedream  Force provider (auto-detect by default)
+  --provider google|openai|openrouter|dashscope|replicate|jimeng|seedream|azure  Force provider (auto-detect by default)
  -m, --model <id>          Model ID
  --ar <ratio>              Aspect ratio (e.g., 16:9, 1:1, 4:3)
  --size <WxH>              Size (e.g., 1024x1024)
  --quality normal|2k       Quality preset (default: 2k)
  --imageSize 1K|2K|4K      Image size for Google/OpenRouter (default: from quality)
-  --ref <files...>          Reference images (Google, OpenAI, OpenRouter, Replicate, or Seedream 4.0/4.5/5.0)
+  --ref <files...>          Reference images (Google, OpenAI, Azure, OpenRouter, Replicate, or Seedream 4.0/4.5/5.0)
  --n <count>               Number of images for the current task (default: 1)
  --json                    JSON output
  -h, --help                Show help
@@ -131,6 +132,11 @@ Environment variables:
  DASHSCOPE_BASE_URL        Custom DashScope endpoint
  REPLICATE_BASE_URL        Custom Replicate endpoint
  JIMENG_BASE_URL           Custom Jimeng endpoint
  AZURE_OPENAI_API_KEY      Azure OpenAI API key
  AZURE_OPENAI_BASE_URL     Azure OpenAI resource or deployment endpoint
  AZURE_OPENAI_DEPLOYMENT   Default Azure deployment name
  AZURE_API_VERSION         Azure API version (default: 2025-04-01-preview)
  AZURE_OPENAI_IMAGE_MODEL  Backward-compatible Azure deployment/model alias (defaults to gpt-image-1.5)
  SEEDREAM_BASE_URL         Custom Seedream endpoint
  BAOYU_IMAGE_GEN_MAX_WORKERS  Override batch worker cap
  BAOYU_IMAGE_GEN_<PROVIDER>_CONCURRENCY  Override provider concurrency
@@ -231,7 +237,8 @@ export function parseArgs(argv: string[]): CliArgs {
        v !== "dashscope" &&
        v !== "replicate" &&
        v !== "jimeng" &&
-        v !== "seedream"
+        v !== "seedream" &&
        v !== "azure"
      ) {
        throw new Error(`Invalid provider: ${v}`);
      }
@@ -386,6 +393,7 @@ export function parseSimpleYaml(yaml: string): Partial<ExtendConfig> {
          replicate: null,
          jimeng: null,
          seedream: null,
          azure: null,
        };
        currentKey = "default_model";
        currentProvider = null;
@@ -411,7 +419,8 @@ export function parseSimpleYaml(yaml: string): Partial<ExtendConfig> {
          key === "dashscope" ||
          key === "replicate" ||
          key === "jimeng" ||
-          key === "seedream"
+          key === "seedream" ||
          key === "azure"
        )
      ) {
        config.batch ??= {};
@@ -427,7 +436,8 @@ export function parseSimpleYaml(yaml: string): Partial<ExtendConfig> {
          key === "dashscope" ||
          key === "replicate" ||
          key === "jimeng" ||
-          key === "seedream"
+          key === "seedream" ||
          key === "azure"
        )
      ) {
        const cleaned = value.replace(/['"]/g, "");
@@ -520,9 +530,10 @@ export function getConfiguredProviderRateLimits(
    dashscope: { ...DEFAULT_PROVIDER_RATE_LIMITS.dashscope },
    jimeng: { ...DEFAULT_PROVIDER_RATE_LIMITS.jimeng },
    seedream: { ...DEFAULT_PROVIDER_RATE_LIMITS.seedream },
    azure: { ...DEFAULT_PROVIDER_RATE_LIMITS.azure },
  };
-  for (const provider of ["replicate", "google", "openai", "openrouter", "dashscope", "jimeng", "seedream"] as Provider[]) {
+  for (const provider of ["replicate", "google", "openai", "openrouter", "dashscope", "jimeng", "seedream", "azure"] as Provider[]) {
    const envPrefix = `BAOYU_IMAGE_GEN_${provider.toUpperCase()}`;
    const extendLimit = extendConfig.batch?.provider_limits?.[provider];
    configured[provider] = {
@@ -581,18 +592,20 @@ export function detectProvider(args: CliArgs): Provider {
    args.provider &&
    args.provider !== "google" &&
    args.provider !== "openai" &&
    args.provider !== "azure" &&
    args.provider !== "openrouter" &&
    args.provider !== "replicate" &&
    args.provider !== "seedream"
  ) {
    throw new Error(
-      "Reference images require a ref-capable provider. Use --provider google (Gemini multimodal), --provider openai (GPT Image edits), --provider openrouter (OpenRouter multimodal), --provider replicate, or --provider seedream for supported Seedream models."
+      "Reference images require a ref-capable provider. Use --provider google (Gemini multimodal), --provider openai (GPT Image edits), --provider azure (Azure OpenAI), --provider openrouter (OpenRouter multimodal), --provider replicate, or --provider seedream for supported Seedream models."
    );
  }
  if (args.provider) return args.provider;
  const hasGoogle = !!(process.env.GOOGLE_API_KEY || process.env.GEMINI_API_KEY);
  const hasAzure = !!(process.env.AZURE_OPENAI_API_KEY && process.env.AZURE_OPENAI_BASE_URL);
  const hasOpenai = !!process.env.OPENAI_API_KEY;
  const hasOpenrouter = !!process.env.OPENROUTER_API_KEY;
  const hasDashscope = !!process.env.DASHSCOPE_API_KEY;
@@ -611,17 +624,19 @@ export function detectProvider(args: CliArgs): Provider {
  if (args.referenceImages.length > 0) {
    if (hasGoogle) return "google";
    if (hasOpenai) return "openai";
    if (hasAzure) return "azure";
    if (hasOpenrouter) return "openrouter";
    if (hasReplicate) return "replicate";
    if (hasSeedream) return "seedream";
    throw new Error(
-      "Reference images require Google, OpenAI, OpenRouter, Replicate, or supported Seedream models. Set GOOGLE_API_KEY/GEMINI_API_KEY, OPENAI_API_KEY, OPENROUTER_API_KEY, REPLICATE_API_TOKEN, or ARK_API_KEY, or remove --ref."
+      "Reference images require Google, OpenAI, Azure, OpenRouter, Replicate, or supported Seedream models. Set GOOGLE_API_KEY/GEMINI_API_KEY, OPENAI_API_KEY, AZURE_OPENAI_API_KEY+AZURE_OPENAI_BASE_URL, OPENROUTER_API_KEY, REPLICATE_API_TOKEN, or ARK_API_KEY, or remove --ref."
    );
  }
  const available = [
    hasGoogle && "google",
    hasOpenai && "openai",
    hasAzure && "azure",
    hasOpenrouter && "openrouter",
    hasDashscope && "dashscope",
    hasReplicate && "replicate",
@@ -633,7 +648,7 @@ export function detectProvider(args: CliArgs): Provider {
  if (available.length > 1) return available[0]!;
  throw new Error(
-    "No API key found. Set GOOGLE_API_KEY, GEMINI_API_KEY, OPENAI_API_KEY, OPENROUTER_API_KEY, DASHSCOPE_API_KEY, REPLICATE_API_TOKEN, JIMENG keys, or ARK_API_KEY.\n" +
+    "No API key found. Set GOOGLE_API_KEY, GEMINI_API_KEY, OPENAI_API_KEY, AZURE_OPENAI_API_KEY+AZURE_OPENAI_BASE_URL, OPENROUTER_API_KEY, DASHSCOPE_API_KEY, REPLICATE_API_TOKEN, JIMENG keys, or ARK_API_KEY.\n" +
      "Create ~/.baoyu-skills/.env or <cwd>/.baoyu-skills/.env with your keys."
  );
 }
@@ -676,6 +691,7 @@ async function loadProviderModule(provider: Provider): Promise<ProviderModule> {
  if (provider === "openrouter") return (await import("./providers/openrouter")) as ProviderModule;
  if (provider === "jimeng") return (await import("./providers/jimeng")) as ProviderModule;
  if (provider === "seedream") return (await import("./providers/seedream")) as ProviderModule;
  if (provider === "azure") return (await import("./providers/azure")) as ProviderModule;
  return (await import("./providers/openai")) as ProviderModule;
 }
@@ -704,6 +720,7 @@ function getModelForProvider(
    if (provider === "replicate" && extendConfig.default_model.replicate) return extendConfig.default_model.replicate;
    if (provider === "jimeng" && extendConfig.default_model.jimeng) return extendConfig.default_model.jimeng;
    if (provider === "seedream" && extendConfig.default_model.seedream) return extendConfig.default_model.seedream;
    if (provider === "azure" && extendConfig.default_model.azure) return extendConfig.default_model.azure;
  }
  return providerModule.getDefaultModel();
 }
@@ -923,7 +940,7 @@ async function runBatchTasks(
  const acquireProvider = createProviderGate(providerRateLimits);
  const workerCount = getWorkerCount(tasks.length, jobs, maxWorkers);
  console.error(`Batch mode: ${tasks.length} tasks, ${workerCount} workers, parallel mode enabled.`);
-  for (const provider of ["replicate", "google", "openai", "openrouter", "dashscope", "jimeng", "seedream"] as Provider[]) {
+  for (const provider of ["replicate", "google", "openai", "openrouter", "dashscope", "jimeng", "seedream", "azure"] as Provider[]) {
    const limit = providerRateLimits[provider];
    console.error(`- ${provider}: concurrency=${limit.concurrency}, startIntervalMs=${limit.startIntervalMs}`);
  }
@@ -0,0 +1,188 @@
 import assert from "node:assert/strict";
 import fs from "node:fs/promises";
 import os from "node:os";
 import path from "node:path";
 import test, { type TestContext } from "node:test";
 import type { CliArgs } from "../types.ts";
 import {
  generateImage,
  getDefaultModel,
  parseAzureBaseURL,
  validateArgs,
 } from "./azure.ts";
 function useEnv(
  t: TestContext,
  values: Record<string, string | null>,
 ): void {
  const previous = new Map<string, string | undefined>();
  for (const [key, value] of Object.entries(values)) {
    previous.set(key, process.env[key]);
    if (value == null) {
      delete process.env[key];
    } else {
      process.env[key] = value;
    }
  }
  t.after(() => {
    for (const [key, value] of previous.entries()) {
      if (value == null) {
        delete process.env[key];
      } else {
        process.env[key] = value;
      }
    }
  });
 }
 function makeArgs(overrides: Partial<CliArgs> = {}): CliArgs {
  return {
    prompt: null,
    promptFiles: [],
    imagePath: null,
    provider: null,
    model: null,
    aspectRatio: null,
    size: null,
    quality: null,
    imageSize: null,
    referenceImages: [],
    n: 1,
    batchFile: null,
    jobs: null,
    json: false,
    help: false,
    ...overrides,
  };
 }
 async function makeTempDir(prefix: string): Promise<string> {
  return fs.mkdtemp(path.join(os.tmpdir(), prefix));
 }
 test("Azure endpoint parsing and default deployment selection follow env precedence", (t) => {
  assert.deepEqual(parseAzureBaseURL("https://example.openai.azure.com"), {
    resourceBaseURL: "https://example.openai.azure.com/openai",
    deployment: null,
  });
  assert.deepEqual(
    parseAzureBaseURL("https://example.openai.azure.com/openai/deployments/from-url"),
    {
      resourceBaseURL: "https://example.openai.azure.com/openai",
      deployment: "from-url",
    },
  );
  useEnv(t, {
    AZURE_OPENAI_BASE_URL: "https://example.openai.azure.com/openai/deployments/from-url",
    AZURE_OPENAI_DEPLOYMENT: "explicit-deploy",
    AZURE_OPENAI_IMAGE_MODEL: "env-fallback",
  });
  assert.equal(getDefaultModel(), "explicit-deploy");
 });
 test("Azure validateArgs rejects unsupported edit input formats before the API call", () => {
  assert.doesNotThrow(() =>
    validateArgs("demo-deployment", makeArgs({ referenceImages: ["hero.png", "photo.jpeg"] })),
  );
  assert.throws(
    () => validateArgs("demo-deployment", makeArgs({ referenceImages: ["hero.webp"] })),
    /PNG or JPG\/JPEG/,
  );
 });
 test("Azure image generation routes model to deployment and sends mapped quality", async (t) => {
  useEnv(t, {
    AZURE_OPENAI_API_KEY: "azure-key",
    AZURE_OPENAI_BASE_URL: "https://example.openai.azure.com/openai/deployments/default-deploy",
    AZURE_API_VERSION: null,
    AZURE_OPENAI_DEPLOYMENT: null,
    AZURE_OPENAI_IMAGE_MODEL: null,
  });
  const originalFetch = globalThis.fetch;
  t.after(() => {
    globalThis.fetch = originalFetch;
  });
  const calls: Array<{ url: string; body: string }> = [];
  globalThis.fetch = async (input, init) => {
    calls.push({
      url: String(input),
      body: String(init?.body ?? ""),
    });
    return Response.json({
      data: [{ b64_json: Buffer.from("azure-image").toString("base64") }],
    });
  };
  const bytes = await generateImage(
    "A calm lake at sunset",
    "custom-deploy",
    makeArgs({ quality: "normal" }),
  );
  assert.equal(Buffer.from(bytes).toString("utf8"), "azure-image");
  assert.equal(
    calls[0]?.url,
    "https://example.openai.azure.com/openai/deployments/custom-deploy/images/generations?api-version=2025-04-01-preview",
  );
  const body = JSON.parse(calls[0]!.body) as Record<string, string>;
  assert.equal(body.quality, "medium");
  assert.equal(body.size, "1024x1024");
 });
 test("Azure image edits include quality in multipart requests", async (t) => {
  const root = await makeTempDir("baoyu-image-gen-azure-");
  t.after(() => fs.rm(root, { recursive: true, force: true }));
  const pngPath = path.join(root, "ref.png");
  const jpgPath = path.join(root, "ref.jpg");
  await fs.writeFile(pngPath, "png-bytes");
  await fs.writeFile(jpgPath, "jpg-bytes");
  useEnv(t, {
    AZURE_OPENAI_API_KEY: "azure-key",
    AZURE_OPENAI_BASE_URL: "https://example.openai.azure.com",
    AZURE_API_VERSION: "2025-04-01-preview",
    AZURE_OPENAI_DEPLOYMENT: null,
    AZURE_OPENAI_IMAGE_MODEL: null,
  });
  const originalFetch = globalThis.fetch;
  t.after(() => {
    globalThis.fetch = originalFetch;
  });
  const calls: Array<{ url: string; form: FormData }> = [];
  globalThis.fetch = async (input, init) => {
    calls.push({
      url: String(input),
      form: init?.body as FormData,
    });
    return Response.json({
      data: [{ b64_json: Buffer.from("edited-image").toString("base64") }],
    });
  };
  const bytes = await generateImage(
    "Add warm lighting",
    "edit-deploy",
    makeArgs({
      quality: "2k",
      referenceImages: [pngPath, jpgPath],
    }),
  );
  assert.equal(Buffer.from(bytes).toString("utf8"), "edited-image");
  assert.equal(
    calls[0]?.url,
    "https://example.openai.azure.com/openai/deployments/edit-deploy/images/edits?api-version=2025-04-01-preview",
  );
  assert.equal(calls[0]?.form.get("quality"), "high");
  assert.equal(calls[0]?.form.get("size"), "1024x1024");
  assert.equal(calls[0]?.form.getAll("image[]").length, 2);
 });
@@ -0,0 +1,192 @@
 import path from "node:path";
 import { readFile } from "node:fs/promises";
 import type { CliArgs } from "../types";
 import { getOpenAISize, extractImageFromResponse } from "./openai.ts";
 type OpenAIImageResponse = { data: Array<{ url?: string; b64_json?: string }> };
 type AzureEndpoint = {
  resourceBaseURL: string;
  deployment: string | null;
 };
 const DEFAULT_AZURE_API_VERSION = "2025-04-01-preview";
 const AZURE_EDIT_IMAGE_EXTENSIONS = new Set([".png", ".jpg", ".jpeg"]);
 export function parseAzureBaseURL(url: string): AzureEndpoint {
  const parsed = new URL(url);
  const trimmedPath = parsed.pathname.replace(/\/+$/, "");
  const deploymentMatch = trimmedPath.match(/^(.*?)(?:\/openai)?\/deployments\/([^/]+)$/);
  if (deploymentMatch) {
    parsed.pathname = `${deploymentMatch[1] || ""}/openai`;
    return {
      resourceBaseURL: parsed.toString().replace(/\/+$/, ""),
      deployment: decodeURIComponent(deploymentMatch[2]!),
    };
  }
  parsed.pathname = trimmedPath.endsWith("/openai") ? trimmedPath : `${trimmedPath}/openai`;
  return {
    resourceBaseURL: parsed.toString().replace(/\/+$/, ""),
    deployment: null,
  };
 }
 export function getDefaultModel(): string {
  const explicitDeployment = process.env.AZURE_OPENAI_DEPLOYMENT?.trim();
  if (explicitDeployment) return explicitDeployment;
  const baseURL = process.env.AZURE_OPENAI_BASE_URL;
  if (baseURL) {
    try {
      const { deployment } = parseAzureBaseURL(baseURL);
      if (deployment) return deployment;
    } catch {
      // Ignore invalid URLs here so the required-env check can raise the user-facing error later.
    }
  }
  return process.env.AZURE_OPENAI_IMAGE_MODEL || "gpt-image-1.5";
 }
 function getEndpoint(): AzureEndpoint {
  const url = process.env.AZURE_OPENAI_BASE_URL;
  if (!url) {
    throw new Error(
      "AZURE_OPENAI_BASE_URL is required. Set it to your Azure resource or deployment endpoint, e.g.: https://your-resource.openai.azure.com or https://your-resource.openai.azure.com/openai/deployments/your-deployment"
    );
  }
  return parseAzureBaseURL(url);
 }
 function getApiKey(): string {
  const key = process.env.AZURE_OPENAI_API_KEY;
  if (!key) {
    throw new Error(
      "AZURE_OPENAI_API_KEY is required. Get it from Azure Portal → your OpenAI resource → Keys and Endpoint."
    );
  }
  return key;
 }
 function getApiVersion(): string {
  return process.env.AZURE_API_VERSION || DEFAULT_AZURE_API_VERSION;
 }
 function getDeployment(model: string): string {
  const deployment = model.trim();
  if (!deployment) {
    throw new Error(
      "Azure deployment name is required. Use --model <deployment>, AZURE_OPENAI_DEPLOYMENT, AZURE_OPENAI_IMAGE_MODEL, or embed the deployment in AZURE_OPENAI_BASE_URL."
    );
  }
  return deployment;
 }
 function buildURL(deployment: string, pathSuffix: string): string {
  const { resourceBaseURL } = getEndpoint();
  return `${resourceBaseURL}/deployments/${encodeURIComponent(deployment)}${pathSuffix}?api-version=${getApiVersion()}`;
 }
 function authHeaders(): Record<string, string> {
  return { "api-key": getApiKey() };
 }
 function getAzureQuality(quality: CliArgs["quality"]): "medium" | "high" {
  return quality === "2k" ? "high" : "medium";
 }
 export function validateArgs(_model: string, args: CliArgs): void {
  for (const refPath of args.referenceImages) {
    const ext = path.extname(refPath).toLowerCase();
    if (!AZURE_EDIT_IMAGE_EXTENSIONS.has(ext)) {
      throw new Error(
        `Azure OpenAI reference images must be PNG or JPG/JPEG. Unsupported file: ${refPath}`
      );
    }
  }
 }
 export async function generateImage(
  prompt: string,
  model: string,
  args: CliArgs
 ): Promise<Uint8Array> {
  const deployment = getDeployment(model);
  const size = args.size || getOpenAISize(model, args.aspectRatio, args.quality);
  if (args.referenceImages.length > 0) {
    return generateWithAzureEdits(prompt, deployment, size, args.referenceImages, args.quality);
  }
  return generateWithAzureGenerations(prompt, deployment, size, args.quality);
 }
 async function generateWithAzureGenerations(
  prompt: string,
  deployment: string,
  size: string,
  quality: CliArgs["quality"]
 ): Promise<Uint8Array> {
  const body: Record<string, any> = {
    prompt,
    size,
    n: 1,
    quality: getAzureQuality(quality),
  };
  const res = await fetch(buildURL(deployment, "/images/generations"), {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      ...authHeaders(),
    },
    body: JSON.stringify(body),
  });
  if (!res.ok) {
    const err = await res.text();
    throw new Error(`Azure OpenAI API error: ${err}`);
  }
  const result = (await res.json()) as OpenAIImageResponse;
  return extractImageFromResponse(result);
 }
 async function generateWithAzureEdits(
  prompt: string,
  deployment: string,
  size: string,
  referenceImages: string[],
  quality: CliArgs["quality"]
 ): Promise<Uint8Array> {
  const form = new FormData();
  form.append("prompt", prompt);
  form.append("size", size);
  form.append("n", "1");
  form.append("quality", getAzureQuality(quality));
  for (const refPath of referenceImages) {
    const bytes = await readFile(refPath);
    const filename = path.basename(refPath);
    const mimeType = path.extname(filename).toLowerCase() === ".png" ? "image/png" : "image/jpeg";
    const blob = new Blob([bytes], { type: mimeType });
    form.append("image[]", blob, filename);
  }
  const res = await fetch(buildURL(deployment, "/images/edits"), {
    method: "POST",
    headers: {
      ...authHeaders(),
    },
    body: form,
  });
  if (!res.ok) {
    const err = await res.text();
    throw new Error(`Azure OpenAI edits API error: ${err}`);
  }
  const result = (await res.json()) as OpenAIImageResponse;
  return extractImageFromResponse(result);
 }
@@ -0,0 +1,114 @@
 import assert from "node:assert/strict";
 import test, { type TestContext } from "node:test";
 import type { CliArgs } from "../types.ts";
 import { generateImage } from "./jimeng.ts";
 function makeArgs(overrides: Partial<CliArgs> = {}): CliArgs {
  return {
    prompt: null,
    promptFiles: [],
    imagePath: null,
    provider: null,
    model: null,
    aspectRatio: null,
    size: null,
    quality: null,
    imageSize: null,
    referenceImages: [],
    n: 1,
    batchFile: null,
    jobs: null,
    json: false,
    help: false,
    ...overrides,
  };
 }
 function useEnv(
  t: TestContext,
  values: Record<string, string | null>,
 ): void {
  const previous = new Map<string, string | undefined>();
  for (const [key, value] of Object.entries(values)) {
    previous.set(key, process.env[key]);
    if (value == null) {
      delete process.env[key];
    } else {
      process.env[key] = value;
    }
  }
  t.after(() => {
    for (const [key, value] of previous.entries()) {
      if (value == null) {
        delete process.env[key];
      } else {
        process.env[key] = value;
      }
    }
  });
 }
 test("Jimeng submit request uses prompt field expected by current API", async (t) => {
  useEnv(t, {
    JIMENG_ACCESS_KEY_ID: "test-access-key",
    JIMENG_SECRET_ACCESS_KEY: "test-secret-key",
    JIMENG_BASE_URL: null,
    JIMENG_REGION: null,
  });
  const originalFetch = globalThis.fetch;
  t.after(() => {
    globalThis.fetch = originalFetch;
  });
  const calls: Array<{
    input: string;
    init?: RequestInit;
  }> = [];
  globalThis.fetch = async (input, init) => {
    calls.push({
      input: String(input),
      init,
    });
    if (calls.length === 1) {
      return Response.json({
        code: 10000,
        data: {
          task_id: "task-123",
        },
      });
    }
    return Response.json({
      code: 10000,
      data: {
        status: "done",
        binary_data_base64: [Buffer.from("jimeng-image").toString("base64")],
      },
    });
  };
  const image = await generateImage(
    "A quiet bamboo forest",
    "jimeng_t2i_v40",
    makeArgs({ quality: "normal" }),
  );
  assert.equal(Buffer.from(image).toString("utf8"), "jimeng-image");
  assert.equal(calls.length, 2);
  assert.equal(
    calls[0]?.input,
    "https://visual.volcengineapi.com/?Action=CVSync2AsyncSubmitTask&Version=2022-08-31",
  );
  const submitBody = JSON.parse(String(calls[0]?.init?.body)) as Record<string, unknown>;
  assert.equal(submitBody.req_key, "jimeng_t2i_v40");
  assert.equal(submitBody.prompt, "A quiet bamboo forest");
  assert.ok(!("prompt_text" in submitBody));
  assert.equal(submitBody.width, 1024);
  assert.equal(submitBody.height, 1024);
 });
@@ -246,7 +246,7 @@ async function submitTask(
  const [width, height] = size.split("x").map(Number);
  const bodyObj = {
    req_key: model,
-    prompt_text: prompt,
+    prompt,
    // Use separate width and height parameters instead of size string
    width: width,
    height: height,
@@ -1,4 +1,4 @@
-export type Provider = "google" | "openai" | "openrouter" | "dashscope" | "replicate" | "jimeng" | "seedream";
+export type Provider = "google" | "openai" | "openrouter" | "dashscope" | "replicate" | "jimeng" | "seedream" | "azure";
 export type Quality = "normal" | "2k";
 export type CliArgs = {
@@ -55,6 +55,7 @@ export type ExtendConfig = {
    replicate: string | null;
    jimeng: string | null;
    seedream: string | null;
    azure: string | null;
  };
  batch?: {
    max_workers?: number | null;
@@ -694,6 +694,10 @@ async function main(): Promise<void> {
  }
  const creds = loadCredentials(resolved);
  for (const skippedSource of creds.skippedSources) {
    console.error(`[wechat-api] Skipped incomplete credential source: ${skippedSource}`);
  }
  console.error(`[wechat-api] Credentials source: ${creds.source}`);
  console.error("[wechat-api] Fetching access token...");
  const accessToken = await fetchAccessToken(creds.appId, creds.appSecret);
@@ -0,0 +1,139 @@
 import assert from "node:assert/strict";
 import fs from "node:fs/promises";
 import os from "node:os";
 import path from "node:path";
 import process from "node:process";
 import test, { type TestContext } from "node:test";
 import { loadCredentials } from "./wechat-extend-config.ts";
 function useCwd(t: TestContext, cwd: string): void {
  const previous = process.cwd();
  process.chdir(cwd);
  t.after(() => {
    process.chdir(previous);
  });
 }
 function useHome(t: TestContext, home: string): void {
  const previous = process.env.HOME;
  process.env.HOME = home;
  t.after(() => {
    if (previous === undefined) {
      delete process.env.HOME;
      return;
    }
    process.env.HOME = previous;
  });
 }
 function useWechatEnv(
  t: TestContext,
  values: Partial<Record<"WECHAT_APP_ID" | "WECHAT_APP_SECRET", string | undefined>>,
 ): void {
  const previous = {
    WECHAT_APP_ID: process.env.WECHAT_APP_ID,
    WECHAT_APP_SECRET: process.env.WECHAT_APP_SECRET,
  };
  if (values.WECHAT_APP_ID === undefined) {
    delete process.env.WECHAT_APP_ID;
  } else {
    process.env.WECHAT_APP_ID = values.WECHAT_APP_ID;
  }
  if (values.WECHAT_APP_SECRET === undefined) {
    delete process.env.WECHAT_APP_SECRET;
  } else {
    process.env.WECHAT_APP_SECRET = values.WECHAT_APP_SECRET;
  }
  t.after(() => {
    if (previous.WECHAT_APP_ID === undefined) {
      delete process.env.WECHAT_APP_ID;
    } else {
      process.env.WECHAT_APP_ID = previous.WECHAT_APP_ID;
    }
    if (previous.WECHAT_APP_SECRET === undefined) {
      delete process.env.WECHAT_APP_SECRET;
    } else {
      process.env.WECHAT_APP_SECRET = previous.WECHAT_APP_SECRET;
    }
  });
 }
 async function makeTempDir(prefix: string): Promise<string> {
  return fs.mkdtemp(path.join(os.tmpdir(), prefix));
 }
 async function writeEnvFile(root: string, content: string): Promise<void> {
  const envPath = path.join(root, ".baoyu-skills", ".env");
  await fs.mkdir(path.dirname(envPath), { recursive: true });
  await fs.writeFile(envPath, content);
 }
 test("loadCredentials selects the first complete source without mixing values across sources", async (t) => {
  const cwdRoot = await makeTempDir("wechat-creds-cwd-");
  const homeRoot = await makeTempDir("wechat-creds-home-");
  useCwd(t, cwdRoot);
  useHome(t, homeRoot);
  useWechatEnv(t, {
    WECHAT_APP_ID: undefined,
    WECHAT_APP_SECRET: "stale-secret-from-process-env",
  });
  await writeEnvFile(cwdRoot, "WECHAT_APP_ID=cwd-app-id\nWECHAT_APP_SECRET=cwd-app-secret\n");
  await writeEnvFile(homeRoot, "WECHAT_APP_ID=home-app-id\nWECHAT_APP_SECRET=home-app-secret\n");
  const credentials = loadCredentials();
  assert.equal(credentials.appId, "cwd-app-id");
  assert.equal(credentials.appSecret, "cwd-app-secret");
  assert.equal(credentials.source, "<cwd>/.baoyu-skills/.env");
  assert.deepEqual(credentials.skippedSources, [
    "process.env missing WECHAT_APP_ID",
  ]);
 });
 test("loadCredentials prefers a complete process.env pair over lower-priority files", async (t) => {
  const cwdRoot = await makeTempDir("wechat-creds-cwd-");
  const homeRoot = await makeTempDir("wechat-creds-home-");
  useCwd(t, cwdRoot);
  useHome(t, homeRoot);
  useWechatEnv(t, {
    WECHAT_APP_ID: "env-app-id",
    WECHAT_APP_SECRET: "env-app-secret",
  });
  await writeEnvFile(cwdRoot, "WECHAT_APP_ID=cwd-app-id\nWECHAT_APP_SECRET=cwd-app-secret\n");
  await writeEnvFile(homeRoot, "WECHAT_APP_ID=home-app-id\nWECHAT_APP_SECRET=home-app-secret\n");
  const credentials = loadCredentials();
  assert.equal(credentials.appId, "env-app-id");
  assert.equal(credentials.appSecret, "env-app-secret");
  assert.equal(credentials.source, "process.env");
  assert.deepEqual(credentials.skippedSources, []);
 });
 test("loadCredentials reports skipped incomplete sources when no complete pair exists", async (t) => {
  const cwdRoot = await makeTempDir("wechat-creds-cwd-");
  const homeRoot = await makeTempDir("wechat-creds-home-");
  useCwd(t, cwdRoot);
  useHome(t, homeRoot);
  useWechatEnv(t, {
    WECHAT_APP_ID: "env-app-id",
    WECHAT_APP_SECRET: undefined,
  });
  await writeEnvFile(cwdRoot, "WECHAT_APP_SECRET=cwd-app-secret\n");
  assert.throws(
    () => loadCredentials(),
    /Incomplete credential sources skipped:\n- process\.env missing WECHAT_APP_SECRET\n- <cwd>\/\.baoyu-skills\/\.env missing WECHAT_APP_ID/,
  );
 });
@@ -196,48 +196,116 @@ function aliasToEnvKey(alias: string): string {
  return alias.toUpperCase().replace(/-/g, "_");
 }
-export function loadCredentials(account?: ResolvedAccount): { appId: string; appSecret: string } {
+interface CredentialSource {
-  if (account?.app_id && account?.app_secret) {
+  name: string;
-    return { appId: account.app_id, appSecret: account.app_secret };
+  appIdKey: string;
  appSecretKey: string;
  appId?: string;
  appSecret?: string;
 }
 export interface LoadedCredentials {
  appId: string;
  appSecret: string;
  source: string;
  skippedSources: string[];
 }
 function normalizeCredentialValue(value?: string): string | undefined {
  const trimmed = value?.trim();
  return trimmed ? trimmed : undefined;
 }
 function describeMissingKeys(source: CredentialSource): string {
  const missingKeys: string[] = [];
  if (!source.appId) missingKeys.push(source.appIdKey);
  if (!source.appSecret) missingKeys.push(source.appSecretKey);
  return `${source.name} missing ${missingKeys.join(" and ")}`;
 }
 function buildCredentialSource(
  name: string,
  values: Record<string, string | undefined>,
  appIdKey: string,
  appSecretKey: string,
 ): CredentialSource {
  return {
    name,
    appIdKey,
    appSecretKey,
    appId: normalizeCredentialValue(values[appIdKey]),
    appSecret: normalizeCredentialValue(values[appSecretKey]),
  };
 }
 function resolveCredentialSource(
  sources: CredentialSource[],
  account?: ResolvedAccount,
 ): LoadedCredentials {
  const skippedSources: string[] = [];
  for (const source of sources) {
    if (source.appId && source.appSecret) {
      return {
        appId: source.appId,
        appSecret: source.appSecret,
        source: source.name,
        skippedSources,
      };
    }
    if (source.appId || source.appSecret) {
      skippedSources.push(describeMissingKeys(source));
    }
  }
  const hint = account?.alias ? ` (account: ${account.alias})` : "";
  const partialHint = skippedSources.length > 0
    ? `\nIncomplete credential sources skipped:\n- ${skippedSources.join("\n- ")}`
    : "";
  throw new Error(
    `Missing WECHAT_APP_ID or WECHAT_APP_SECRET${hint}.\n` +
    "Set via EXTEND.md account config, environment variables, or .baoyu-skills/.env file." +
    partialHint
  );
 }
 export function loadCredentials(account?: ResolvedAccount): LoadedCredentials {
  const cwdEnvPath = path.join(process.cwd(), ".baoyu-skills", ".env");
  const homeEnvPath = path.join(os.homedir(), ".baoyu-skills", ".env");
  const cwdEnv = loadEnvFile(cwdEnvPath);
  const homeEnv = loadEnvFile(homeEnvPath);
  const sources: CredentialSource[] = [];
  if (account?.app_id || account?.app_secret) {
    sources.push({
      name: account.alias ? `EXTEND.md account "${account.alias}"` : "EXTEND.md account config",
      appIdKey: "app_id",
      appSecretKey: "app_secret",
      appId: normalizeCredentialValue(account.app_id),
      appSecret: normalizeCredentialValue(account.app_secret),
    });
  }
  const prefix = account?.alias ? `WECHAT_${aliasToEnvKey(account.alias)}_` : "";
  let appId = "";
  let appSecret = "";
  if (prefix) {
-    appId = process.env[`${prefix}APP_ID`]
+    const prefixedKeyLabel = `${prefix}APP_ID/${prefix}APP_SECRET`;
-      || cwdEnv[`${prefix}APP_ID`]
+    sources.push(
-      || homeEnv[`${prefix}APP_ID`]
+      buildCredentialSource(`process.env (${prefixedKeyLabel})`, process.env, `${prefix}APP_ID`, `${prefix}APP_SECRET`),
-      || "";
+      buildCredentialSource(`<cwd>/.baoyu-skills/.env (${prefixedKeyLabel})`, cwdEnv, `${prefix}APP_ID`, `${prefix}APP_SECRET`),
-    appSecret = process.env[`${prefix}APP_SECRET`]
+      buildCredentialSource(`~/.baoyu-skills/.env (${prefixedKeyLabel})`, homeEnv, `${prefix}APP_ID`, `${prefix}APP_SECRET`),
      || cwdEnv[`${prefix}APP_SECRET`]
      || homeEnv[`${prefix}APP_SECRET`]
      || "";
  }
  if (!appId) {
    appId = process.env.WECHAT_APP_ID || cwdEnv.WECHAT_APP_ID || homeEnv.WECHAT_APP_ID || "";
  }
  if (!appSecret) {
    appSecret = process.env.WECHAT_APP_SECRET || cwdEnv.WECHAT_APP_SECRET || homeEnv.WECHAT_APP_SECRET || "";
  }
  if (!appId || !appSecret) {
    const hint = account?.alias ? ` (account: ${account.alias})` : "";
    throw new Error(
      `Missing WECHAT_APP_ID or WECHAT_APP_SECRET${hint}.\n` +
      "Set via EXTEND.md account config, environment variables, or .baoyu-skills/.env file."
    );
  }
-  return { appId, appSecret };
+  sources.push(
    buildCredentialSource("process.env", process.env, "WECHAT_APP_ID", "WECHAT_APP_SECRET"),
    buildCredentialSource("<cwd>/.baoyu-skills/.env", cwdEnv, "WECHAT_APP_ID", "WECHAT_APP_SECRET"),
    buildCredentialSource("~/.baoyu-skills/.env", homeEnv, "WECHAT_APP_ID", "WECHAT_APP_SECRET"),
  );
  return resolveCredentialSource(sources, account);
 }
 export function listAccounts(config: WechatExtendConfig): string[] {
@@ -123,6 +123,8 @@ ${BUN_X} {baseDir}/scripts/weibo-article.ts article.md --cover ./cover.jpg
 - Title: 32 characters max (truncated with warning if longer)
 - Summary/导语: 44 characters max (auto-regenerated from content if longer)
 **Markdown-to-HTML**: Do NOT pass any `--theme` parameter when converting markdown to HTML. Use the default theme (no theme argument).
 **Article Workflow**:
 1. Opens `https://card.weibo.com/article/v3/editor`
 2. Clicks "写文章" button, waits for editor to become editable
@@ -1,7 +1,7 @@
 ---
 name: baoyu-url-to-markdown
 description: Fetch any URL and convert to markdown using Chrome CDP. Saves the rendered HTML snapshot alongside the markdown, uses an upgraded Defuddle pipeline with better web-component handling and YouTube transcript extraction, and automatically falls back to the pre-Defuddle HTML-to-Markdown pipeline when needed. If local browser capture fails entirely, it can fall back to the hosted defuddle.md API. Supports two modes - auto-capture on page load, or wait for user signal (for pages requiring login). Use when user wants to save a webpage as markdown.
-version: 1.58.1
+version: 1.59.0
 metadata:
  openclaw:
    homepage: https://github.com/JimLiu/baoyu-skills#baoyu-url-to-markdown
@@ -30,6 +30,9 @@ Fetches any URL via Chrome CDP, saves the rendered HTML snapshot, and converts i
 |--------|---------|
 | `scripts/main.ts` | CLI entry point for URL fetching |
 | `scripts/html-to-markdown.ts` | Markdown conversion entry point and converter selection |
 | `scripts/parsers/index.ts` | Unified parser entry: dispatches URL-specific rules before generic converters |
 | `scripts/parsers/types.ts` | Unified parser interface shared by all rule files |
 | `scripts/parsers/rules/*.ts` | One file per URL rule, for example X status and X article |
 | `scripts/defuddle-converter.ts` | Defuddle-based conversion |
 | `scripts/legacy-converter.ts` | Pre-Defuddle legacy extraction and markdown conversion |
 | `scripts/markdown-conversion-shared.ts` | Shared metadata parsing and markdown document helpers |
@@ -115,10 +118,13 @@ Full reference: [references/config/first-time-setup.md](references/config/first-
 ## Features
 - Chrome CDP for full JavaScript rendering
 - URL-specific parser layer for sites that need custom HTML rules before generic extraction
 - Two capture modes: auto or wait-for-user
 - Save rendered HTML as a sibling `-captured.html` file
 - Clean markdown output with metadata
 - Upgraded Defuddle-first markdown conversion with automatic fallback to the pre-Defuddle extractor from git history
 - X/Twitter pages can use HTML-specific parsing for Tweets and Articles, which improves title/body/media extraction on `x.com` / `twitter.com`
 - `archive.ph` / related archive mirrors can restore the original URL from `input[name=q]` and prefer `#CONTENT` before falling back to the page body
 - Materializes shadow DOM content before conversion so web-component pages survive serialization better
 - YouTube pages can include transcript/caption text in the markdown when YouTube exposes a caption track
 - If local browser capture fails completely, can fall back to `defuddle.md/<url>` and still save markdown
@@ -201,14 +207,16 @@ When `--download-media` is enabled:
 Conversion order:
-1. Try Defuddle first
+1. Try the URL-specific parser layer first when a site rule matches
-2. For rich pages such as YouTube, prefer Defuddle's extractor-specific output (including transcripts when available) instead of replacing it with the legacy pipeline
+2. If no specialized parser matches, try Defuddle
-3. If Defuddle throws, cannot load, returns obviously incomplete markdown, or captures lower-quality content than the legacy pipeline, automatically fall back to the pre-Defuddle extractor
+3. For rich pages such as YouTube, prefer Defuddle's extractor-specific output (including transcripts when available) instead of replacing it with the legacy pipeline
-4. If the entire local browser capture flow fails before markdown can be produced, try the hosted `https://defuddle.md/<url>` API and save its markdown output directly
+4. If Defuddle throws, cannot load, returns obviously incomplete markdown, or captures lower-quality content than the legacy pipeline, automatically fall back to the pre-Defuddle extractor
-5. The legacy fallback path uses the older Readability/selector/Next.js-data based HTML-to-Markdown implementation recovered from git history
+5. If the entire local browser capture flow fails before markdown can be produced, try the hosted `https://defuddle.md/<url>` API and save its markdown output directly
 6. The legacy fallback path uses the older Readability/selector/Next.js-data based HTML-to-Markdown implementation recovered from git history
 CLI output will show:
 - `Converter: parser:...` when a URL-specific parser succeeded
 - `Converter: defuddle` when Defuddle succeeds
 - `Converter: legacy:...` plus `Fallback used: ...` when fallback was needed
 - `Converter: defuddle-api` when local browser capture failed and the hosted API was used instead
@@ -12,6 +12,7 @@ import {
  scoreMarkdownQuality,
  shouldCompareWithLegacy,
 } from "./legacy-converter.js";
 import { tryUrlRuleParsers } from "./parsers/index.js";
 export type { ConversionResult, PageMetadata };
 export { createMarkdownDocument, formatMetadataYaml };
@@ -105,6 +106,11 @@ export async function extractContent(html: string, url: string): Promise<Convers
  const capturedAt = new Date().toISOString();
  const baseMetadata = extractMetadataFromHtml(html, url, capturedAt);
  const specializedResult = tryUrlRuleParsers(html, url, baseMetadata);
  if (specializedResult) {
    return specializedResult;
  }
  const defuddleResult = await tryDefuddleConversion(html, url, baseMetadata);
  if (defuddleResult.ok) {
    if (shouldPreferDefuddle(defuddleResult.result)) {
@@ -521,14 +521,18 @@ turndown.addRule("collapseFigure", {
 turndown.addRule("dropInvisibleAnchors", {
  filter(node) {
-    return node.nodeName === "A" && !(node as Element).textContent?.trim();
+    return (
      node.nodeName === "A" &&
      !(node as Element).textContent?.trim() &&
      !(node as Element).querySelector("img, video, picture, source")
    );
  },
  replacement() {
    return "";
  },
 });
-function convertHtmlToMarkdown(html: string): string {
+export function convertHtmlFragmentToMarkdown(html: string): string {
  if (!html || !html.trim()) return "";
  try {
@@ -609,7 +613,7 @@ export function shouldCompareWithLegacy(markdown: string): boolean {
 export function convertWithLegacyExtractor(html: string, baseMetadata: PageMetadata): ConversionResult {
  const extracted = extractFromHtml(html);
-  let markdown = extracted?.html ? convertHtmlToMarkdown(extracted.html) : "";
+  let markdown = extracted?.html ? convertHtmlFragmentToMarkdown(extracted.html) : "";
  if (!markdown.trim()) {
    markdown = extracted?.textContent?.trim() || fallbackPlainText(html);
  }
@@ -52,15 +52,71 @@ function parseArgs(argv: string[]): Args {
  return args;
 }
-function generateSlug(title: string, url: string): string {
+const SLUG_STOP_WORDS = new Set([
-  const text = title || new URL(url).pathname.replace(/\//g, "-");
+  "the", "a", "an", "is", "are", "was", "were", "be", "been", "being",
-  return text
+  "have", "has", "had", "do", "does", "did", "will", "would", "shall",
  "should", "may", "might", "must", "can", "could", "to", "of", "in",
  "for", "on", "with", "at", "by", "from", "as", "into", "through",
  "during", "before", "after", "above", "below", "between", "out",
  "off", "over", "under", "again", "further", "then", "once", "here",
  "there", "when", "where", "why", "how", "all", "both", "each",
  "few", "more", "most", "other", "some", "such", "no", "nor", "not",
  "only", "own", "same", "so", "than", "too", "very", "just", "but",
  "and", "or", "if", "this", "that", "these", "those", "it", "its",
  "http", "https", "www", "com", "org", "net", "post", "article",
 ]);
 function extractSlugFromContent(content: string): string | null {
  const body = content.replace(/^---\n[\s\S]*?\n---\n?/, "").slice(0, 1000);
  const words = body
    .replace(/[^\w\s-]/g, " ")
    .split(/\s+/)
    .filter((w) => /^[a-zA-Z]/.test(w) && w.length >= 2 && !SLUG_STOP_WORDS.has(w.toLowerCase()))
    .map((w) => w.toLowerCase());
  const unique: string[] = [];
  const seen = new Set<string>();
  for (const w of words) {
    if (!seen.has(w)) {
      seen.add(w);
      unique.push(w);
      if (unique.length >= 6) break;
    }
  }
  return unique.length >= 2 ? unique.join("-").slice(0, 50) : null;
 }
 function generateSlug(title: string, url: string, content?: string): string {
  const asciiWords = title
    .replace(/[^\w\s]/g, " ")
    .split(/\s+/)
    .filter((w) => /[a-zA-Z]/.test(w) && w.length >= 2 && !SLUG_STOP_WORDS.has(w.toLowerCase()))
    .map((w) => w.toLowerCase());
  if (asciiWords.length >= 2) {
    return asciiWords.slice(0, 6).join("-").slice(0, 50);
  }
  if (content) {
    const contentSlug = extractSlugFromContent(content);
    if (contentSlug) return contentSlug;
  }
  const GENERIC_PATH_SEGMENTS = new Set(["status", "article", "post", "posts", "p", "blog", "news", "articles"]);
  const parsed = new URL(url);
  const pathSlug = parsed.pathname
    .split("/")
    .filter((s) => s.length > 0 && !/^\d{10,}$/.test(s) && !GENERIC_PATH_SEGMENTS.has(s.toLowerCase()))
    .join("-")
    .toLowerCase()
-    .replace(/[^\w\s-]/g, "")
+    .replace(/[^\w-]/g, "-")
    .replace(/\s+/g, "-")
    .replace(/-+/g, "-")
    .replace(/^-|-$/g, "")
-    .slice(0, 50) || "page";
+    .slice(0, 40);
  const prefix = asciiWords.slice(0, 2).join("-");
  const combined = prefix ? `${prefix}-${pathSlug}` : pathSlug;
  return combined.slice(0, 50) || "page";
 }
 function formatTimestamp(): string {
@@ -124,18 +180,18 @@ async function fetchDefuddleApiMarkdown(targetUrl: string): Promise<{ markdown:
  };
 }
-async function generateOutputPath(url: string, title: string, outputDir?: string): Promise<string> {
+async function generateOutputPath(url: string, title: string, outputDir?: string, content?: string): Promise<string> {
  const domain = new URL(url).hostname.replace(/^www\./, "");
-  const slug = generateSlug(title, url);
+  const slug = generateSlug(title, url, content);
  const dataDir = outputDir ? path.resolve(outputDir) : resolveUrlToMarkdownDataDir();
-  const basePath = path.join(dataDir, domain, `${slug}.md`);
+  const basePath = path.join(dataDir, domain, slug, `${slug}.md`);
  if (!(await fileExists(basePath))) {
    return basePath;
  }
  const timestampSlug = `${slug}-${formatTimestamp()}`;
-  return path.join(dataDir, domain, `${timestampSlug}.md`);
+  return path.join(dataDir, domain, timestampSlug, `${timestampSlug}.md`);
 }
 async function waitForUserSignal(): Promise<void> {
@@ -249,13 +305,12 @@ async function main(): Promise<void> {
  try {
    const result = await captureUrl(args);
-    outputPath = args.output || await generateOutputPath(args.url, result.metadata.title, args.outputDir);
+    document = createMarkdownDocument(result);
    outputPath = args.output || await generateOutputPath(args.url, result.metadata.title, args.outputDir, document);
    const outputDir = path.dirname(outputPath);
    htmlSnapshotPath = deriveHtmlSnapshotPath(outputPath);
    await mkdir(outputDir, { recursive: true });
    await writeFile(htmlSnapshotPath, result.rawHtml, "utf-8");
    document = createMarkdownDocument(result);
    conversionMethod = result.conversionMethod;
    fallbackReason = result.fallbackReason;
  } catch (error) {
@@ -265,10 +320,9 @@ async function main(): Promise<void> {
    try {
      const remoteResult = await fetchDefuddleApiMarkdown(args.url);
      outputPath = args.output || await generateOutputPath(args.url, remoteResult.title, args.outputDir);
      await mkdir(path.dirname(outputPath), { recursive: true });
      document = remoteResult.markdown;
      outputPath = args.output || await generateOutputPath(args.url, remoteResult.title, args.outputDir, document);
      await mkdir(path.dirname(outputPath), { recursive: true });
      conversionMethod = "defuddle-api";
      fallbackReason = `Local browser capture failed: ${primaryError}`;
    } catch (remoteError) {
@@ -300,6 +300,24 @@ export function createMarkdownDocument(result: ConversionResult): string {
  const escapedTitle = result.metadata.title.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
  const titleRegex = new RegExp(`^#\\s+${escapedTitle}\\s*(\\n|$)`, "i");
  const hasTitle = titleRegex.test(result.markdown.trimStart());
-  const title = result.metadata.title && !hasTitle ? `\n\n# ${result.metadata.title}\n\n` : "\n\n";
+  const firstMeaningfulLine = result.markdown
    .replace(/\r\n/g, "\n")
    .split("\n")
    .map((line) => line.trim())
    .find((line) => line && !/^!?\[[^\]]*\]\([^)]+\)$/.test(line))
    ?.replace(/^>\s*/, "")
    ?.replace(/^#+\s+/, "")
    ?.trim();
  const comparableTitle = result.metadata.title.toLowerCase().replace(/(?:\.{3}|…)\s*$/, "");
  const comparableFirstLine = firstMeaningfulLine?.toLowerCase() ?? "";
  const titleRepeatsContent =
    comparableTitle !== "" &&
    comparableFirstLine !== "" &&
    (comparableFirstLine === comparableTitle ||
      comparableFirstLine.startsWith(comparableTitle) ||
      comparableTitle.startsWith(comparableFirstLine));
  const title = result.metadata.title && !hasTitle && !titleRepeatsContent
    ? `\n\n# ${result.metadata.title}\n\n`
    : "\n\n";
  return yaml + title + result.markdown;
 }
@@ -0,0 +1,206 @@
 import assert from "node:assert/strict";
 import test from "node:test";
 import {
  createMarkdownDocument,
  extractMetadataFromHtml,
 } from "../markdown-conversion-shared.js";
 import { tryUrlRuleParsers } from "./index.js";
 const CAPTURED_AT = "2026-03-22T06:00:00.000Z";
 const ARTICLE_HTML = `<!doctype html>
 <html lang="zh-CN">
  <body>
    <div data-testid="twitterArticleReadView">
      <a href="/dotey/article/2035141635713941927/media/1">
        <div data-testid="tweetPhoto">
          <img src="https://pbs.twimg.com/media/article-cover.jpg" alt="Image">
        </div>
      </a>
      <div data-testid="twitter-article-title">Karpathy："写代码"已经不是对的动词了</div>
      <div data-testid="User-Name">
        <a href="/dotey">宝玉 Verified account</a>
        <a href="/dotey">@dotey</a>
        <time datetime="2026-03-20T23:49:11.000Z">Mar 20</time>
      </div>
      <div data-testid="twitterArticleRichTextView">
        <p>Andrej Karpathy 说他从 2024 年 12 月起就基本没手写过一行代码。</p>
        <a href="/dotey/article/2035141635713941927/media/2">
          <div>
            <div>
              <div data-testid="tweetPhoto">
                <img src="https://pbs.twimg.com/media/article-inline.jpg" alt="Image">
              </div>
            </div>
          </div>
        </a>
        <h2>要点速览</h2>
        <ul>
          <li>核心焦虑从 GPU 利用率转向 Token 吞吐量</li>
        </ul>
        <blockquote>
          <p>写代码已经不是对的动词了。</p>
        </blockquote>
      </div>
    </div>
  </body>
 </html>`;
 const STATUS_HTML = `<!doctype html>
 <html lang="en">
  <body>
    <article data-testid="tweet">
      <div data-testid="User-Name">
        <a href="/dotey">宝玉 Verified account</a>
        <a href="/dotey">@dotey</a>
        <time datetime="2026-03-22T05:33:00.000Z">Mar 22</time>
      </div>
      <div data-testid="tweetText">
        <span>转译：把下面这段加到你的 Codex 自定义指令里，体验会好太多：</span>
      </div>
      <div data-testid="tweetPhoto">
        <img src="https://pbs.twimg.com/media/tweet-main.jpg" alt="Image">
      </div>
      <div data-testid="User-Name">
        <a href="/mattshumer_">Matt Shumer Verified account</a>
        <a href="/mattshumer_">@mattshumer_</a>
        <time datetime="2026-03-17T00:00:00.000Z">Mar 17</time>
      </div>
      <div data-testid="tweetText">
        <span>Add this to your Codex custom instructions for a way better experience.</span>
      </div>
    </article>
  </body>
 </html>`;
 const ARCHIVE_HTML = `<!doctype html>
 <html>
  <head>
    <title>archive.ph</title>
  </head>
  <body>
    <form>
      <input
        type="text"
        name="q"
        value="https://www.newscientist.com/article/2520204-major-leap-towards-reanimation-after-death-as-mammals-brain-preserved/"
      >
    </form>
    <div id="HEADER">
      Archive shell text that should be ignored when CONTENT exists.
    </div>
    <div id="CONTENT">
      <h1>Major leap towards reanimation after death as mammal brain preserved</h1>
      <p>
        Researchers say the preserved structure and activity markers suggest a significant step
        forward in keeping delicate brain tissue viable after clinical death.
      </p>
      <p>
        The archive wrapper should not take precedence over the actual article body when the
        CONTENT container is available for parsing.
      </p>
      <img src="https://cdn.example.com/brain.jpg" alt="Brain tissue">
    </div>
  </body>
 </html>`;
 const ARCHIVE_FALLBACK_HTML = `<!doctype html>
 <html>
  <head>
    <title>archive.ph</title>
  </head>
  <body>
    <input type="text" name="q" value="https://example.com/fallback-story">
    <main>
      <h1>Fallback body parsing still works</h1>
      <p>
        When CONTENT is absent, the parser should fall back to the body content instead of
        returning null or keeping the archive wrapper as the final URL.
      </p>
      <p>
        This ensures archived pages with slightly different layouts still produce usable markdown.
      </p>
    </main>
  </body>
 </html>`;
 function parse(html: string, url: string) {
  const baseMetadata = extractMetadataFromHtml(html, url, CAPTURED_AT);
  return tryUrlRuleParsers(html, url, baseMetadata);
 }
 test("parses archive.ph pages from CONTENT and restores the original URL", () => {
  const result = parse(ARCHIVE_HTML, "https://archive.ph/SMcX5");
  assert.ok(result);
  assert.equal(result.conversionMethod, "parser:archive-ph");
  assert.equal(
    result.metadata.url,
    "https://www.newscientist.com/article/2520204-major-leap-towards-reanimation-after-death-as-mammals-brain-preserved/"
  );
  assert.equal(
    result.metadata.title,
    "Major leap towards reanimation after death as mammal brain preserved"
  );
  assert.equal(result.metadata.coverImage, "https://cdn.example.com/brain.jpg");
  assert.ok(result.markdown.includes("Researchers say the preserved structure"));
  assert.ok(result.markdown.includes("![Brain tissue](https://cdn.example.com/brain.jpg)"));
  assert.ok(!result.markdown.includes("Archive shell text that should be ignored"));
 });
 test("falls back to body when archive.ph CONTENT is missing", () => {
  const result = parse(ARCHIVE_FALLBACK_HTML, "https://archive.ph/fallback");
  assert.ok(result);
  assert.equal(result.conversionMethod, "parser:archive-ph");
  assert.equal(result.metadata.url, "https://example.com/fallback-story");
  assert.equal(result.metadata.title, "Fallback body parsing still works");
  assert.ok(result.markdown.includes("When CONTENT is absent"));
 });
 test("parses X article pages from HTML", () => {
  const result = parse(
    ARTICLE_HTML,
    "https://x.com/dotey/article/2035141635713941927"
  );
  assert.ok(result);
  assert.equal(result.conversionMethod, "parser:x-article");
  assert.equal(result.metadata.title, "Karpathy：\"写代码\"已经不是对的动词了");
  assert.equal(result.metadata.author, "宝玉 (@dotey)");
  assert.equal(result.metadata.coverImage, "https://pbs.twimg.com/media/article-cover.jpg");
  assert.equal(result.metadata.published, "2026-03-20T23:49:11.000Z");
  assert.equal(result.metadata.language, "zh");
  assert.ok(result.markdown.includes("## 要点速览"));
  assert.ok(
    result.markdown.includes(
      "[![](https://pbs.twimg.com/media/article-inline.jpg)](/dotey/article/2035141635713941927/media/2)"
    )
  );
  assert.ok(result.markdown.includes("写代码已经不是对的动词了。"));
  const document = createMarkdownDocument(result);
  assert.ok(document.includes("# Karpathy：\"写代码\"已经不是对的动词了"));
 });
 test("parses X status pages from HTML without duplicating the title heading", () => {
  const result = parse(
    STATUS_HTML,
    "https://x.com/dotey/status/2035590649081196710"
  );
  assert.ok(result);
  assert.equal(result.conversionMethod, "parser:x-status");
  assert.equal(result.metadata.author, "宝玉 (@dotey)");
  assert.equal(result.metadata.coverImage, "https://pbs.twimg.com/media/tweet-main.jpg");
  assert.equal(result.metadata.language, "zh");
  assert.ok(result.markdown.includes("转译：把下面这段加到你的 Codex 自定义指令里"));
  assert.ok(result.markdown.includes("> Quote from Matt Shumer (@mattshumer_)"));
  assert.ok(result.markdown.includes("!["));
  const document = createMarkdownDocument(result);
  assert.ok(
    !document.includes("\n\n# 转译：把下面这段加到你的 Codex 自定义指令里，体验会好太多：\n\n")
  );
 });
@@ -0,0 +1,47 @@
 import {
  isMarkdownUsable,
  normalizeMarkdown,
  parseDocument,
  type ConversionResult,
  type PageMetadata,
 } from "../markdown-conversion-shared.js";
 import { URL_RULE_PARSERS } from "./rules/index.js";
 import type { UrlRuleParserContext } from "./types.js";
 export type { UrlRuleParser, UrlRuleParserContext } from "./types.js";
 export function tryUrlRuleParsers(
  html: string,
  url: string,
  baseMetadata: PageMetadata
 ): ConversionResult | null {
  const document = parseDocument(html);
  const context: UrlRuleParserContext = {
    html,
    url,
    document,
    baseMetadata,
  };
  for (const parser of URL_RULE_PARSERS) {
    if (!parser.supports(context)) continue;
    try {
      const result = parser.parse(context);
      if (!result) continue;
      const markdown = normalizeMarkdown(result.markdown);
      if (!isMarkdownUsable(markdown, html)) continue;
      return {
        ...result,
        markdown,
      };
    } catch (error) {
      const message = error instanceof Error ? error.message : String(error);
      console.warn(`[url-to-markdown] parser ${parser.id} failed: ${message}`);
    }
  }
  return null;
 }
@@ -0,0 +1,97 @@
 import { convertHtmlFragmentToMarkdown } from "../../legacy-converter.js";
 import {
  normalizeMarkdown,
  pickString,
  type ConversionResult,
 } from "../../markdown-conversion-shared.js";
 import type { UrlRuleParser, UrlRuleParserContext } from "../types.js";
 const ARCHIVE_HOSTS = new Set([
  "archive.ph",
  "archive.is",
  "archive.today",
  "archive.md",
  "archive.vn",
  "archive.li",
  "archive.fo",
 ]);
 function isArchiveHost(url: string): boolean {
  try {
    return ARCHIVE_HOSTS.has(new URL(url).hostname.toLowerCase());
  } catch {
    return false;
  }
 }
 function readOriginalUrl(document: Document): string | undefined {
  const value = document.querySelector("input[name='q']")?.getAttribute("value")?.trim();
  if (!value) return undefined;
  try {
    return new URL(value).href;
  } catch {
    return undefined;
  }
 }
 function summarize(text: string, maxLength: number): string | undefined {
  const normalized = text.replace(/\s+/g, " ").trim();
  if (!normalized) return undefined;
  if (normalized.length <= maxLength) return normalized;
  return `${normalized.slice(0, Math.max(0, maxLength - 1)).trimEnd()}…`;
 }
 function pickContentRoot(document: Document): Element | null {
  return (
    document.querySelector("#CONTENT") ??
    document.querySelector("#content") ??
    document.body
  );
 }
 function pickContentTitle(root: Element, fallbackTitle: string): string {
  const contentTitle = pickString(
    root.querySelector("h1")?.textContent,
    root.querySelector("[itemprop='headline']")?.textContent,
    root.querySelector("article h2")?.textContent
  );
  if (contentTitle) return contentTitle;
  if (fallbackTitle && !/^archive\./i.test(fallbackTitle.trim())) return fallbackTitle;
  return "";
 }
 function parseArchivePage(context: UrlRuleParserContext): ConversionResult | null {
  const root = pickContentRoot(context.document);
  if (!root) return null;
  const markdown = normalizeMarkdown(convertHtmlFragmentToMarkdown(root.innerHTML));
  if (!markdown) return null;
  const originalUrl = readOriginalUrl(context.document) ?? context.baseMetadata.url;
  const bodyText = root.textContent?.replace(/\s+/g, " ").trim() ?? "";
  const published = root.querySelector("time[datetime]")?.getAttribute("datetime") ?? undefined;
  const coverImage = root.querySelector("img[src]")?.getAttribute("src") ?? undefined;
  return {
    metadata: {
      ...context.baseMetadata,
      url: originalUrl,
      title: pickContentTitle(root, context.baseMetadata.title),
      description: summarize(bodyText, 220) ?? context.baseMetadata.description,
      published: pickString(published, context.baseMetadata.published) ?? undefined,
      coverImage: pickString(coverImage, context.baseMetadata.coverImage) ?? undefined,
    },
    markdown,
    rawHtml: context.html,
    conversionMethod: "parser:archive-ph",
  };
 }
 export const archivePhRuleParser: UrlRuleParser = {
  id: "archive-ph",
  supports(context) {
    return isArchiveHost(context.url);
  },
  parse: parseArchivePage,
 };
@@ -0,0 +1,10 @@
 import { archivePhRuleParser } from "./archive-ph.js";
 import { xArticleRuleParser } from "./x-article.js";
 import { xStatusRuleParser } from "./x-status.js";
 import type { UrlRuleParser } from "../types.js";
 export const URL_RULE_PARSERS: UrlRuleParser[] = [
  archivePhRuleParser,
  xArticleRuleParser,
  xStatusRuleParser,
 ];
@@ -0,0 +1,137 @@
 import {
  normalizeMarkdown,
  pickString,
  type ConversionResult,
 } from "../../markdown-conversion-shared.js";
 import type { UrlRuleParser, UrlRuleParserContext } from "../types.js";
 import {
  cleanText,
  collectMediaMarkdown,
  convertXRichTextElementToMarkdown,
  extractPublishedForCurrentUrl,
  inferLanguage,
  isXArticlePath,
  isXHost,
  normalizeXMarkdown,
  parseUrl,
  pickFirstValidLinkText,
  sanitizeCoverImage,
  summarizeText,
 } from "./x-shared.js";
 function collectArticleMarkdown(root: Element): { markdown: string; mediaUrls: string[] } {
  const parts: string[] = [];
  const seenMedia = new Set<string>();
  const mediaUrls: string[] = [];
  function pushPart(value: string): void {
    const normalized = normalizeMarkdown(value);
    if (!normalized) return;
    parts.push(normalized);
  }
  function walk(node: Element): void {
    const testId = node.getAttribute("data-testid");
    if (testId === "twitterArticleRichTextView" || testId === "longformRichTextComponent") {
      const bodyMedia = collectMediaMarkdown(node, seenMedia);
      mediaUrls.push(...bodyMedia.urls.filter((url) => !mediaUrls.includes(url)));
      pushPart(convertXRichTextElementToMarkdown(node));
      return;
    }
    if (testId === "tweetPhoto") {
      const media = collectMediaMarkdown(node, seenMedia);
      mediaUrls.push(...media.urls.filter((url) => !mediaUrls.includes(url)));
      for (const line of media.lines) pushPart(line);
      return;
    }
    if (
      testId === "twitter-article-title" ||
      testId === "User-Name" ||
      testId === "Tweet-User-Avatar" ||
      testId === "reply" ||
      testId === "retweet" ||
      testId === "like" ||
      testId === "bookmark" ||
      testId === "caret" ||
      testId === "app-text-transition-container"
    ) {
      return;
    }
    if (node.tagName === "TIME" || node.tagName === "BUTTON") {
      return;
    }
    for (const child of Array.from(node.children)) {
      walk(child);
    }
  }
  for (const child of Array.from(root.children)) {
    walk(child);
  }
  return {
    markdown: normalizeXMarkdown(parts.join("\n\n")),
    mediaUrls,
  };
 }
 function parseXArticle(context: UrlRuleParserContext): ConversionResult | null {
  const articleRoot = context.document.querySelector("[data-testid='twitterArticleReadView']") as Element | null;
  if (!articleRoot) return null;
  const title = cleanText(
    context.document.querySelector("[data-testid='twitter-article-title']")?.textContent
  );
  const identity = pickFirstValidLinkText(
    context.document.querySelector("[data-testid='User-Name']")
  );
  const published = extractPublishedForCurrentUrl(articleRoot, context.url);
  const { markdown, mediaUrls } = collectArticleMarkdown(articleRoot);
  if (!markdown) return null;
  const bodyText = cleanText(
    context.document.querySelector("[data-testid='twitterArticleRichTextView']")?.textContent ??
    context.document.querySelector("[data-testid='longformRichTextComponent']")?.textContent
  );
  return {
    metadata: {
      ...context.baseMetadata,
      title: pickString(title, context.baseMetadata.title) ?? "",
      description: summarizeText(bodyText, 220) ?? context.baseMetadata.description,
      author: pickString(identity.author, context.baseMetadata.author) ?? undefined,
      published: pickString(published, context.baseMetadata.published) ?? undefined,
      coverImage: sanitizeCoverImage(mediaUrls[0], context.baseMetadata.coverImage),
      language: inferLanguage(bodyText, context.baseMetadata.language),
    },
    markdown,
    rawHtml: context.html,
    conversionMethod: "parser:x-article",
  };
 }
 export const xArticleRuleParser: UrlRuleParser = {
  id: "x-article",
  supports(context) {
    const parsed = parseUrl(context.url);
    if (!parsed || !isXHost(parsed.hostname)) {
      return false;
    }
    return (
      isXArticlePath(parsed.pathname) ||
      Boolean(
        context.document.querySelector("[data-testid='twitterArticleReadView']") ||
        context.document.querySelector("[data-testid='twitterArticleRichTextView']")
      )
    );
  },
  parse(context) {
    return parseXArticle(context);
  },
 };
@@ -0,0 +1,249 @@
 import { convertHtmlFragmentToMarkdown } from "../../legacy-converter.js";
 import { normalizeMarkdown } from "../../markdown-conversion-shared.js";
 export const DEFAULT_X_OG_IMAGE = "https://abs.twimg.com/rweb/ssr/default/v2/og/image.png";
 export type MediaResult = {
  lines: string[];
  urls: string[];
 };
 export function isXHost(hostname: string): boolean {
  const normalized = hostname.toLowerCase();
  return (
    normalized === "x.com" ||
    normalized === "twitter.com" ||
    normalized.endsWith(".x.com") ||
    normalized.endsWith(".twitter.com")
  );
 }
 export function parseUrl(input: string): URL | null {
  try {
    return new URL(input);
  } catch {
    return null;
  }
 }
 export function isXStatusPath(pathname: string): boolean {
  return /^\/[^/]+\/status(?:es)?\/\d+$/i.test(pathname) || /^\/i\/web\/status\/\d+$/i.test(pathname);
 }
 export function isXArticlePath(pathname: string): boolean {
  return /^\/[^/]+\/article\/\d+$/i.test(pathname) || /^\/(?:i\/)?article\/\d+$/i.test(pathname);
 }
 export function cleanText(value: string | null | undefined): string {
  return (value ?? "").replace(/\s+/g, " ").trim();
 }
 export function cleanUserLabel(value: string | null | undefined): string {
  return cleanText(value).replace(/\bVerified account\b/gi, "").replace(/\s{2,}/g, " ").trim();
 }
 export function escapeMarkdownAlt(text: string): string {
  return text.replace(/[\[\]]/g, "\\$&");
 }
 export function normalizeAlt(text: string | null | undefined): string {
  const cleaned = cleanText(text);
  if (!cleaned || /^(image|photo)$/i.test(cleaned)) return "";
  return escapeMarkdownAlt(cleaned);
 }
 export function summarizeText(text: string, maxLength: number): string | undefined {
  const normalized = cleanText(text);
  if (!normalized) return undefined;
  return normalized.length > maxLength
    ? `${normalized.slice(0, maxLength - 3)}...`
    : normalized;
 }
 export function buildTweetTitle(text: string, fallback: string): string {
  return summarizeText(text, 80) ?? fallback;
 }
 export function normalizeXMarkdown(markdown: string): string {
  return normalizeMarkdown(markdown.replace(/^(#{1,6})\s*\n+([^\n])/gm, "$1 $2"));
 }
 export function inferLanguage(text: string, fallback?: string): string | undefined {
  const normalized = cleanText(text);
  if (!normalized) return fallback;
  const han = (normalized.match(/\p{Script=Han}/gu) || []).length;
  const hiragana = (normalized.match(/\p{Script=Hiragana}/gu) || []).length;
  const katakana = (normalized.match(/\p{Script=Katakana}/gu) || []).length;
  const hangul = (normalized.match(/\p{Script=Hangul}/gu) || []).length;
  if (hangul >= 8) return "ko";
  if (hiragana + katakana >= 8) return "ja";
  if (han >= 16) return "zh";
  return fallback;
 }
 export function buildQuoteMarkdown(markdown: string, author?: string): string {
  const normalized = normalizeMarkdown(markdown);
  if (!normalized) return "";
  const lines = normalized.split("\n");
  const prefixed = lines.map((line) => (line ? `> ${line}` : ">")).join("\n");
  const header = author ? `> Quote from ${author}` : "> Quote";
  return `${header}\n${prefixed}`;
 }
 export function pickFirstValidLinkText(userNameEl: Element | null | undefined): {
  name?: string;
  username?: string;
  author?: string;
 } {
  if (!userNameEl) return {};
  const linkTexts = Array.from(userNameEl.querySelectorAll("a[href]"))
    .map((link) => cleanUserLabel(link.textContent))
    .filter(Boolean);
  let username = linkTexts.find((text) => text.startsWith("@"));
  let name = linkTexts.find((text) => !text.startsWith("@") && !/^(promote|more)$/i.test(text));
  if (!username || !name) {
    const text = cleanUserLabel(userNameEl.textContent);
    const fallbackMatch = text.match(/^(.*?)\s*(@[A-Za-z0-9_]+)(?:\s*·.*)?$/);
    if (fallbackMatch) {
      name = name ?? cleanText(fallbackMatch[1]);
      username = username ?? cleanText(fallbackMatch[2]);
    }
  }
  const author = name && username ? `${name} (${username})` : username ?? name;
  return { name, username, author };
 }
 export function extractPublishedForCurrentUrl(root: ParentNode, url: string): string | undefined {
  const parsed = parseUrl(url);
  if (!parsed) return undefined;
  const currentPath = parsed.pathname.toLowerCase();
  for (const timeElement of root.querySelectorAll("a[href] time[datetime]")) {
    const href = timeElement.closest("a")?.getAttribute("href");
    const hrefUrl = href ? parseUrl(href.startsWith("http") ? href : `${parsed.origin}${href}`) : null;
    if (hrefUrl?.pathname.toLowerCase() === currentPath) {
      return timeElement.getAttribute("datetime") ?? undefined;
    }
  }
  return root.querySelector("time[datetime]")?.getAttribute("datetime") ?? undefined;
 }
 export function collectMediaMarkdown(root: ParentNode, seen: Set<string>): MediaResult {
  const lines: string[] = [];
  const urls: string[] = [];
  const rootElement = root as Element & {
    getAttribute?: (name: string) => string | null;
  };
  const photoNodes = [
    ...(typeof rootElement.getAttribute === "function" &&
    rootElement.getAttribute("data-testid") === "tweetPhoto"
      ? [rootElement]
      : []),
    ...Array.from(root.querySelectorAll("[data-testid='tweetPhoto']")),
  ];
  for (const node of photoNodes) {
    const img = node.querySelector("img");
    const imageUrl = img?.getAttribute("src");
    if (imageUrl && !seen.has(imageUrl)) {
      seen.add(imageUrl);
      urls.push(imageUrl);
      lines.push(`![${normalizeAlt(img?.getAttribute("alt"))}](${imageUrl})`);
    }
    const video = node.querySelector("video");
    const posterUrl = video?.getAttribute("poster");
    if (posterUrl && !seen.has(posterUrl)) {
      seen.add(posterUrl);
      urls.push(posterUrl);
      lines.push(`![video](${posterUrl})`);
    }
    const videoUrl = video?.getAttribute("src") ?? video?.querySelector("source")?.getAttribute("src");
    if (videoUrl && !seen.has(videoUrl)) {
      seen.add(videoUrl);
      urls.push(videoUrl);
      lines.push(`[video](${videoUrl})`);
    }
  }
  return { lines, urls };
 }
 export function materializeTweetPhotoNodes(root: Element): void {
  for (const photo of Array.from(root.querySelectorAll("[data-testid='tweetPhoto']"))) {
    const document = photo.ownerDocument;
    const container = document.createElement("span");
    const img = photo.querySelector("img");
    const imageUrl = img?.getAttribute("src");
    if (imageUrl) {
      const image = document.createElement("img");
      image.setAttribute("src", imageUrl);
      const alt = normalizeAlt(img?.getAttribute("alt"));
      if (alt) {
        image.setAttribute("alt", alt);
      }
      container.appendChild(image);
    }
    const video = photo.querySelector("video");
    const posterUrl = video?.getAttribute("poster");
    if (posterUrl) {
      const poster = document.createElement("img");
      poster.setAttribute("src", posterUrl);
      poster.setAttribute("alt", "video");
      container.appendChild(poster);
    }
    const videoUrl = video?.getAttribute("src") ?? video?.querySelector("source")?.getAttribute("src");
    if (videoUrl) {
      if (container.childNodes.length > 0) {
        container.appendChild(document.createTextNode(" "));
      }
      const link = document.createElement("a");
      link.setAttribute("href", videoUrl);
      link.textContent = "video";
      container.appendChild(link);
    }
    if (container.childNodes.length === 0) {
      photo.remove();
      continue;
    }
    photo.replaceWith(container);
  }
 }
 function collapseLinkedMediaContainers(root: Element): void {
  for (const anchor of Array.from(root.querySelectorAll("a[href]"))) {
    const images = Array.from(anchor.querySelectorAll("img"));
    if (images.length !== 1) continue;
    if (cleanText(anchor.textContent)) continue;
    const image = images[0].cloneNode(true);
    anchor.replaceChildren(image);
  }
 }
 export function convertXRichTextElementToMarkdown(node: Element): string {
  const clone = node.cloneNode(true) as Element;
  materializeTweetPhotoNodes(clone);
  collapseLinkedMediaContainers(clone);
  return normalizeXMarkdown(convertHtmlFragmentToMarkdown(clone.innerHTML));
 }
 export function sanitizeCoverImage(primary?: string, fallback?: string): string | undefined {
  if (primary) return primary;
  if (!fallback || fallback === DEFAULT_X_OG_IMAGE) return undefined;
  return fallback;
 }
@@ -0,0 +1,82 @@
 import type { ConversionResult } from "../../markdown-conversion-shared.js";
 import type { UrlRuleParser, UrlRuleParserContext } from "../types.js";
 import {
  buildQuoteMarkdown,
  buildTweetTitle,
  cleanText,
  collectMediaMarkdown,
  convertXRichTextElementToMarkdown,
  extractPublishedForCurrentUrl,
  inferLanguage,
  isXHost,
  isXStatusPath,
  normalizeXMarkdown,
  parseUrl,
  pickFirstValidLinkText,
  sanitizeCoverImage,
  summarizeText,
 } from "./x-shared.js";
 function parseXStatus(context: UrlRuleParserContext): ConversionResult | null {
  const article = context.document.querySelector("article[data-testid='tweet'], article") as Element | null;
  if (!article) return null;
  const tweetTextElements = Array.from(article.querySelectorAll("[data-testid='tweetText']")) as Element[];
  if (tweetTextElements.length === 0) return null;
  const userNameElements = Array.from(article.querySelectorAll("[data-testid='User-Name']")) as Element[];
  const mainTextElement = tweetTextElements[0];
  const mainIdentity = pickFirstValidLinkText(userNameElements[0]);
  const published = extractPublishedForCurrentUrl(article, context.url);
  const mainMarkdown = normalizeXMarkdown(convertXRichTextElementToMarkdown(mainTextElement));
  if (!mainMarkdown) return null;
  const parts = [mainMarkdown];
  const quotedTextElements = tweetTextElements.slice(1);
  const quotedUserNameElements = userNameElements.slice(1);
  quotedTextElements.forEach((element, index) => {
    const quoteMarkdown = normalizeXMarkdown(convertXRichTextElementToMarkdown(element));
    if (!quoteMarkdown) return;
    const quoteIdentity = pickFirstValidLinkText(quotedUserNameElements[index]);
    parts.push(buildQuoteMarkdown(quoteMarkdown, quoteIdentity.author));
  });
  const media = collectMediaMarkdown(article, new Set<string>());
  if (media.lines.length > 0) {
    parts.push(media.lines.join("\n\n"));
  }
  const mainText = cleanText(mainTextElement.textContent);
  const markdown = normalizeXMarkdown(parts.join("\n\n"));
  return {
    metadata: {
      ...context.baseMetadata,
      title: buildTweetTitle(mainText, context.baseMetadata.title),
      description: summarizeText(mainText, 220) ?? context.baseMetadata.description,
      author: mainIdentity.author ?? context.baseMetadata.author,
      published: published ?? context.baseMetadata.published,
      coverImage: sanitizeCoverImage(media.urls[0], context.baseMetadata.coverImage),
      language: inferLanguage(mainText, context.baseMetadata.language),
    },
    markdown,
    rawHtml: context.html,
    conversionMethod: "parser:x-status",
  };
 }
 export const xStatusRuleParser: UrlRuleParser = {
  id: "x-status",
  supports(context) {
    const parsed = parseUrl(context.url);
    if (!parsed || !isXHost(parsed.hostname)) {
      return false;
    }
    return isXStatusPath(parsed.pathname) && Boolean(context.document.querySelector("[data-testid='tweetText']"));
  },
  parse(context): ConversionResult | null {
    return parseXStatus(context);
  },
 };
@@ -0,0 +1,14 @@
 import type { ConversionResult, PageMetadata } from "../markdown-conversion-shared.js";
 export interface UrlRuleParserContext {
  html: string;
  url: string;
  document: Document;
  baseMetadata: PageMetadata;
 }
 export interface UrlRuleParser {
  id: string;
  supports(context: UrlRuleParserContext): boolean;
  parse(context: UrlRuleParserContext): ConversionResult | null;
 }
@@ -151,8 +151,7 @@ From outline entry:
 ```markdown
 ## Watermark
-Include a subtle watermark "{content}" positioned at {position}
+Include a subtle watermark "{content}" positioned at {position}. The watermark should
 with approximately {opacity*100}% visibility. The watermark should
 be legible but not distracting from the main content.
 ```
@@ -295,19 +294,19 @@ Create a Xiaohongshu (Little Red Book) style infographic following these guideli
 ## Content
 **Position**: Content (Page 3 of 6)
-**Core Message**: ChatGPT使用技巧
+**Core Message**: ChatGPT 使用技巧
 **Text Content**:
 - Title: 「ChatGPT」
- Subtitle: 最强AI助手
+- Subtitle: 最强 AI 助手
 - Points:
  - 写文案：给出框架，秒出初稿
  - 改文章：润色、翻译、总结
-  - 编程：写代码、找bug
+  - 编程：写代码、找 bug
  - 学习：解释概念、出题练习
 **Visual Concept**:
-ChatGPT logo居中，四周放射状展示功能点
+ChatGPT logo 居中，四周放射状展示功能点
 深色科技背景，霓虹绿点缀
 ---
@@ -13,7 +13,7 @@ metadata:
 # YouTube Transcript
-Downloads transcripts (subtitles/captions) from YouTube videos. Works with both manually created and auto-generated transcripts. No API key or browser required — uses YouTube's InnerTube API directly.
+Downloads transcripts (subtitles/captions) from YouTube videos. Works with both manually created and auto-generated transcripts. No API key or browser required — uses YouTube's InnerTube API directly and automatically falls back to `yt-dlp` when YouTube blocks the direct API path.
 Fetches video metadata and cover image on first run, caches raw data for fast re-formatting.
@@ -75,6 +75,12 @@ ${BUN_X} {baseDir}/scripts/main.ts <url> --refresh
 | `-o, --output <path>` | Save to specific file path | auto-generated |
 | `--output-dir <dir>` | Base output directory | `youtube-transcript` |
 ## Optional Environment Variables
 | Variable | Description |
 |----------|-------------|
 | `YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER` | Passed to `yt-dlp --cookies-from-browser` during fallback, e.g. `chrome`, `safari`, `firefox`, or `chrome:Profile 1` |
 ## Input Formats
 Accepts any of these as video input:
@@ -121,6 +127,8 @@ On first fetch, the script saves:
 Subsequent runs for the same video use cached data (no network calls). Use `--refresh` to force re-fetch. If a different language is requested, the cache is automatically refreshed.
 When YouTube returns anti-bot / blocked responses on the direct InnerTube path, the script retries with alternate client identities and then falls back to `yt-dlp` if available. If fallback is needed but `yt-dlp` is unavailable, the agent should decide how to make `yt-dlp` available and continue rather than pushing the installation decision to the user.
 SRT output (`--format srt`) is generated from `transcript-raw.json`. Text/markdown output uses `transcript-sentences.json` for natural sentence boundaries.
 ## Workflow
@@ -128,7 +136,8 @@ SRT output (`--format srt`) is generated from `transcript-raw.json`. Text/markdo
 When user provides a YouTube URL and wants the transcript:
 1. Run with `--list` first if the user hasn't specified a language, to show available options
-2. Default: run with `--chapters --speakers` for the richest output (chapters + speaker identification)
+2. **Always single-quote the URL** when running the script — zsh treats `?` as a glob wildcard, so an unquoted YouTube URL causes "no matches found": use `'https://www.youtube.com/watch?v=ID'`
 3. Default: run with `--chapters --speakers` for the richest output (chapters + speaker identification)
 3. The script auto-saves cached data + output file and prints the file path
 4. For `--speakers` mode: after the script saves the raw file, follow the speaker identification workflow below to post-process with speaker labels
@@ -174,3 +183,4 @@ When `--speakers` is used, `--chapters` is implied — the processed output alwa
 | Video unavailable | Video deleted, private, or region-locked |
 | IP blocked | Too many requests, try again later |
 | Age restricted | Video requires login for age verification |
 | bot detected | The script retries alternate clients and then `yt-dlp`; if fallback tooling is missing, the agent should resolve that itself, otherwise if it still fails try `YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER=safari` (or your browser) |
@@ -0,0 +1,125 @@
 import test from "node:test";
 import assert from "node:assert/strict";
 import { findTranscript, parseTranscriptJson3, parseWebVtt } from "./transcript.ts";
 import { buildTranscriptListFromYtDlp, resolveVideoSource, selectYtDlpTrack } from "./youtube.ts";
 test("selectYtDlpTrack prefers json3 over xml and vtt", () => {
  const track = selectYtDlpTrack([
    { ext: "vtt", url: "https://example.com/subs.vtt" },
    { ext: "srv3", url: "https://example.com/subs.srv3" },
    { ext: "json3", url: "https://example.com/subs.json3" },
  ]);
  assert.equal(track?.ext, "json3");
 });
 test("buildTranscriptListFromYtDlp keeps manual and generated tracks separate", () => {
  const transcripts = buildTranscriptListFromYtDlp({
    subtitles: {
      en: [
        { ext: "json3", url: "https://example.com/en.json3", name: "English" },
      ],
    },
    automatic_captions: {
      "zh-Hans": [
        { ext: "json3", url: "https://example.com/zh.json3", name: "Chinese (Simplified)" },
      ],
    },
  });
  assert.equal(transcripts.length, 2);
  assert.equal(transcripts[0].isGenerated, false);
  assert.equal(transcripts[1].isGenerated, true);
  assert.equal(transcripts[0].translationLanguages[0]?.languageCode, "zh-Hans");
  const translated = findTranscript(transcripts, ["zh-Hans"], false, false);
  assert.equal(translated.languageCode, "zh-Hans");
  assert.equal(translated.isGenerated, true);
 });
 test("parseTranscriptJson3 reads youtube timedtext json3 payloads", () => {
  const snippets = parseTranscriptJson3(JSON.stringify({
    events: [
      {
        tStartMs: 80,
        dDurationMs: 3120,
        segs: [{ utf8: "hello\nworld" }],
      },
      {
        tStartMs: 4000,
        dDurationMs: 1800,
        segs: [{ utf8: "again" }],
      },
    ],
  }));
  assert.deepEqual(snippets, [
    { text: "hello world", start: 0.08, duration: 3.12 },
    { text: "again", start: 4, duration: 1.8 },
  ]);
 });
 test("parseWebVtt strips tags and cue settings", () => {
  const snippets = parseWebVtt(`WEBVTT
 00:00:00.080 --> 00:00:03.200 align:start position:0%
 <c.colorE5E5E5>Hello</c> world
 00:00:04.000 --> 00:00:05.800
 Again
 `);
  assert.equal(snippets.length, 2);
  assert.equal(snippets[0].text, "Hello world");
  assert.equal(snippets[0].start, 0.08);
  assert.equal(snippets[0].duration, 3.12);
  assert.equal(snippets[1].text, "Again");
  assert.equal(snippets[1].start, 4);
  assert.equal(Number(snippets[1].duration.toFixed(1)), 1.8);
 });
 test("resolveVideoSource prefers primary InnerTube result before fallback", async () => {
  let fallbackCalled = false;
  const source = await resolveVideoSource(
    "video12345ab",
    async () => ({ kind: "innertube", data: { videoDetails: { title: "Primary" } }, transcripts: [] }),
    () => {
      fallbackCalled = true;
      return {
        subtitles: {
          en: [{ ext: "json3", url: "https://example.com/en.json3", name: "English" }],
        },
      };
    },
    () => {}
  );
  assert.equal(source.kind, "innertube");
  assert.equal(fallbackCalled, false);
 });
 test("resolveVideoSource falls back to yt-dlp only after fallback-eligible errors", async () => {
  let fallbackCalled = false;
  const source = await resolveVideoSource(
    "video12345ab",
    async () => {
      const error = new Error("Request blocked for video12345ab: bot detected");
      (error as Error & { code?: string }).code = "BOT_DETECTED";
      throw error;
    },
    () => {
      fallbackCalled = true;
      return {
        automatic_captions: {
          en: [{ ext: "json3", url: "https://example.com/en.json3", name: "English (auto-generated)" }],
        },
      };
    },
    () => {}
  );
  assert.equal(source.kind, "yt-dlp");
  assert.equal(fallbackCalled, true);
  assert.equal(source.transcripts[0].languageCode, "en");
 });
@@ -1,652 +1,55 @@
 #!/usr/bin/env bun
-import { existsSync, mkdirSync, readFileSync, writeFileSync } from "fs";
+import { writeFileSync } from "fs";
-import { dirname, join, resolve } from "path";
+import { join, resolve } from "path";
-type Format = "text" | "srt";
+import { extractVideoId, slugify } from "./shared.ts";
 import {
  ensureDir,
  hasCachedData,
  loadMeta,
  loadSentences,
  loadSnippets,
  lookupVideoDir,
  registerVideoDir,
  resolveBaseDir,
 } from "./storage.ts";
 import { findTranscript, formatListOutput, formatMarkdown, formatSrt, segmentIntoSentences } from "./transcript.ts";
 import type { Options, Sentence, Snippet, VideoMeta, VideoResult } from "./types.ts";
 import {
  buildVideoMeta,
  buildVideoMetaFromYtDlp,
  downloadCoverImage,
  fetchTranscriptSnippets,
  fetchVideoSource,
  getThumbnailUrls,
  getYtDlpThumbnailUrls,
  parseChapters,
 } from "./youtube.ts";
-interface Options {
+async function fetchAndCache(
-  videoIds: string[];
+  videoId: string,
-  languages: string[];
+  baseDir: string,
-  format: Format;
+  opts: Options
-  translate: string;
+): Promise<{ meta: VideoMeta; snippets: Snippet[]; sentences: Sentence[]; videoDir: string }> {
-  list: boolean;
+  const source = await fetchVideoSource(videoId);
-  excludeGenerated: boolean;
+  const requestedLanguages = source.kind === "yt-dlp" && opts.translate ? [opts.translate] : opts.languages;
-  excludeManual: boolean;
+  const transcript = findTranscript(source.transcripts, requestedLanguages, opts.excludeGenerated, opts.excludeManual);
-  output: string;
+  const result = await fetchTranscriptSnippets(transcript, source.kind === "yt-dlp" ? undefined : opts.translate || undefined);
-  outputDir: string;
+  const description = source.kind === "yt-dlp"
-  timestamps: boolean;
+    ? source.info.description || ""
-  chapters: boolean;
+    : source.data?.videoDetails?.shortDescription || "";
-  speakers: boolean;
+  const duration = source.kind === "yt-dlp"
-  refresh: boolean;
+    ? Number(source.info.duration || 0)
-}
+    : parseInt(source.data?.videoDetails?.lengthSeconds || "0");
-
+  const chapters = parseChapters(description, duration);
-interface Snippet {
+  const language = {
-  text: string;
+    code: result.languageCode,
-  start: number;
+    name: result.language,
-  duration: number;
+    isGenerated: transcript.isGenerated,
 }
 interface Sentence {
  text: string;
  start: string;
  end: string;
 }
 interface TranscriptInfo {
  language: string;
  languageCode: string;
  isGenerated: boolean;
  isTranslatable: boolean;
  baseUrl: string;
  translationLanguages: { language: string; languageCode: string }[];
 }
 interface Chapter {
  title: string;
  start: number;
 }
 interface VideoMeta {
  videoId: string;
  title: string;
  channel: string;
  channelId: string;
  description: string;
  duration: number;
  publishDate: string;
  url: string;
  coverImage: string;
  thumbnailUrl: string;
  language: { code: string; name: string; isGenerated: boolean };
  chapters: Chapter[];
 }
 interface VideoResult {
  videoId: string;
  title?: string;
  filePath?: string;
  content?: string;
  error?: string;
 }
 const WATCH_URL = "https://www.youtube.com/watch?v=";
 const INNERTUBE_URL = "https://www.youtube.com/youtubei/v1/player";
 const INNERTUBE_CTX = { client: { clientName: "ANDROID", clientVersion: "20.10.38" } };
 function extractVideoId(input: string): string {
  input = input.replace(/\\/g, "").trim();
  const patterns = [
    /(?:youtube\.com\/watch\?.*v=|youtu\.be\/|youtube\.com\/embed\/|youtube\.com\/v\/|youtube\.com\/shorts\/)([a-zA-Z0-9_-]{11})/,
    /^([a-zA-Z0-9_-]{11})$/,
  ];
  for (const p of patterns) {
    const m = input.match(p);
    if (m) return m[1];
  }
  return input;
 }
 function slugify(s: string): string {
  return s
    .toLowerCase()
    .replace(/[^\w\s-]/g, "")
    .replace(/\s+/g, "-")
    .replace(/-+/g, "-")
    .replace(/^-|-$/g, "") || "untitled";
 }
 function htmlUnescape(s: string): string {
  return s
    .replace(/&amp;/g, "&")
    .replace(/&lt;/g, "<")
    .replace(/&gt;/g, ">")
    .replace(/&quot;/g, '"')
    .replace(/&#39;/g, "'")
    .replace(/&#x27;/g, "'")
    .replace(/&#x2F;/g, "/")
    .replace(/&apos;/g, "'")
    .replace(/&#(\d+);/g, (_, n) => String.fromCharCode(parseInt(n)))
    .replace(/&#x([0-9a-fA-F]+);/g, (_, n) => String.fromCharCode(parseInt(n, 16)));
 }
 function stripTags(s: string): string {
  return s.replace(/<[^>]*>/g, "");
 }
 function parseTranscriptXml(xml: string): Snippet[] {
  const snippets: Snippet[] = [];
  const re = /<text\s+start="([^"]*)"(?:\s+dur="([^"]*)")?[^>]*>([\s\S]*?)<\/text>/g;
  let m: RegExpExecArray | null;
  while ((m = re.exec(xml)) !== null) {
    const raw = m[3];
    if (!raw) continue;
    snippets.push({
      text: htmlUnescape(stripTags(raw)),
      start: parseFloat(m[1]),
      duration: parseFloat(m[2] || "0"),
    });
  }
  return snippets;
 }
 // --- YouTube API ---
 async function fetchHtml(videoId: string): Promise<string> {
  const r = await fetch(WATCH_URL + videoId, {
    headers: { "Accept-Language": "en-US", "User-Agent": "Mozilla/5.0" },
  });
  if (!r.ok) throw new Error(`HTTP ${r.status} fetching video page`);
  let html = await r.text();
  if (html.includes('action="https://consent.youtube.com/s"')) {
    const cv = html.match(/name="v" value="(.*?)"/);
    if (!cv) throw new Error("Failed to create consent cookie");
    const r2 = await fetch(WATCH_URL + videoId, {
      headers: {
        "Accept-Language": "en-US",
        "User-Agent": "Mozilla/5.0",
        Cookie: `CONSENT=YES+${cv[1]}`,
      },
    });
    if (!r2.ok) throw new Error(`HTTP ${r2.status} fetching video page (consent)`);
    html = await r2.text();
  }
  return html;
 }
 function extractApiKey(html: string, videoId: string): string {
  const m = html.match(/"INNERTUBE_API_KEY":\s*"([a-zA-Z0-9_-]+)"/);
  if (!m) {
    if (html.includes('class="g-recaptcha"')) throw new Error(`IP blocked for ${videoId} (reCAPTCHA)`);
    throw new Error(`Cannot extract API key for ${videoId}`);
  }
  return m[1];
 }
 async function fetchInnertubeData(videoId: string, apiKey: string): Promise<any> {
  const r = await fetch(`${INNERTUBE_URL}?key=${apiKey}`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ context: INNERTUBE_CTX, videoId }),
  });
  if (r.status === 429) throw new Error(`IP blocked for ${videoId} (429)`);
  if (!r.ok) throw new Error(`HTTP ${r.status} from InnerTube API`);
  return r.json();
 }
 function assertPlayability(data: any, videoId: string) {
  const ps = data?.playabilityStatus;
  if (!ps) return;
  const status = ps.status;
  if (status === "OK" || !status) return;
  const reason = ps.reason || "";
  if (status === "LOGIN_REQUIRED") {
    if (reason.includes("bot")) throw new Error(`Request blocked for ${videoId}: bot detected`);
    if (reason.includes("inappropriate")) throw new Error(`Age restricted: ${videoId}`);
  }
  if (status === "ERROR" && reason.includes("unavailable")) {
    if (videoId.startsWith("http")) throw new Error(`Invalid video ID: pass the ID, not the URL`);
    throw new Error(`Video unavailable: ${videoId}`);
  }
  const subreasons = ps.errorScreen?.playerErrorMessageRenderer?.subreason?.runs?.map((r: any) => r.text).join("") || "";
  throw new Error(`Video unplayable (${videoId}): ${reason} ${subreasons}`.trim());
 }
 function extractCaptionsJson(data: any, videoId: string): any {
  assertPlayability(data, videoId);
  const cj = data?.captions?.playerCaptionsTracklistRenderer;
  if (!cj || !cj.captionTracks) throw new Error(`Transcripts disabled for ${videoId}`);
  return cj;
 }
 function buildTranscriptList(captionsJson: any): TranscriptInfo[] {
  const tlLangs = (captionsJson.translationLanguages || []).map((tl: any) => ({
    language: tl.languageName?.runs?.[0]?.text || tl.languageName?.simpleText || "",
    languageCode: tl.languageCode,
  }));
  return (captionsJson.captionTracks || []).map((t: any) => ({
    language: t.name?.runs?.[0]?.text || t.name?.simpleText || "",
    languageCode: t.languageCode,
    isGenerated: t.kind === "asr",
    isTranslatable: !!t.isTranslatable,
    baseUrl: (t.baseUrl || "").replace(/&fmt=srv3/g, ""),
    translationLanguages: t.isTranslatable ? tlLangs : [],
  }));
 }
 function findTranscript(
  transcripts: TranscriptInfo[],
  languages: string[],
  excludeGenerated: boolean,
  excludeManual: boolean
 ): TranscriptInfo {
  let filtered = transcripts;
  if (excludeGenerated) filtered = filtered.filter((t) => !t.isGenerated);
  if (excludeManual) filtered = filtered.filter((t) => t.isGenerated);
  for (const lang of languages) {
    const found = filtered.find((t) => t.languageCode === lang);
    if (found) return found;
  }
  const available = filtered.map((t) => `${t.languageCode} ("${t.language}")`).join(", ");
  throw new Error(`No transcript found for languages [${languages.join(", ")}]. Available: ${available || "none"}`);
 }
 async function fetchTranscriptSnippets(info: TranscriptInfo, translateTo?: string): Promise<{ snippets: Snippet[]; language: string; languageCode: string }> {
  let url = info.baseUrl;
  let lang = info.language;
  let langCode = info.languageCode;
  if (translateTo) {
    if (!info.isTranslatable) throw new Error(`Transcript ${info.languageCode} is not translatable`);
    const tl = info.translationLanguages.find((t) => t.languageCode === translateTo);
    if (!tl) throw new Error(`Translation language ${translateTo} not available`);
    url += `&tlang=${translateTo}`;
    lang = tl.language;
    langCode = translateTo;
  }
  const r = await fetch(url, { headers: { "Accept-Language": "en-US" } });
  if (!r.ok) throw new Error(`HTTP ${r.status} fetching transcript`);
  return { snippets: parseTranscriptXml(await r.text()), language: lang, languageCode: langCode };
 }
 // --- Metadata & chapters ---
 function parseChapters(description: string): Chapter[] {
  const chapters: Chapter[] = [];
  for (const line of description.split("\n")) {
    const m = line.trim().match(/^(?:(\d{1,2}):)?(\d{1,2}):(\d{2})\s+(.+)$/);
    if (m) {
      const h = m[1] ? parseInt(m[1]) : 0;
      chapters.push({ title: m[4].trim(), start: h * 3600 + parseInt(m[2]) * 60 + parseInt(m[3]) });
    }
  }
  return chapters.length >= 2 ? chapters : [];
 }
 function getThumbnailUrls(videoId: string, data: any): string[] {
  const urls = [
    `https://i.ytimg.com/vi/${videoId}/maxresdefault.jpg`,
    `https://i.ytimg.com/vi/${videoId}/hqdefault.jpg`,
  ];
  const thumbnails = data?.videoDetails?.thumbnail?.thumbnails ||
    data?.microformat?.playerMicroformatRenderer?.thumbnail?.thumbnails || [];
  if (thumbnails.length) {
    const sorted = [...thumbnails].sort((a: any, b: any) => (b.width || 0) - (a.width || 0));
    for (const t of sorted) if (t.url && !urls.includes(t.url)) urls.push(t.url);
  }
  return urls;
 }
 function buildVideoMeta(data: any, videoId: string, langInfo: { code: string; name: string; isGenerated: boolean }, chapters: Chapter[]): VideoMeta {
  const vd = data?.videoDetails || {};
  const mf = data?.microformat?.playerMicroformatRenderer || {};
  return {
    videoId,
    title: vd.title || mf.title?.simpleText || "",
    channel: vd.author || mf.ownerChannelName || "",
    channelId: vd.channelId || mf.externalChannelId || "",
    description: vd.shortDescription || mf.description?.simpleText || "",
    duration: parseInt(vd.lengthSeconds || "0"),
    publishDate: mf.publishDate || mf.uploadDate || "",
    url: `https://www.youtube.com/watch?v=${videoId}`,
    coverImage: "",
    thumbnailUrl: getThumbnailUrls(videoId, data)[0],
    language: langInfo,
    chapters,
  };
-}
+  const meta = source.kind === "yt-dlp"
-
+    ? buildVideoMetaFromYtDlp(source.info, videoId, language, chapters)
-async function downloadCoverImage(urls: string[], outputPath: string): Promise<boolean> {
+    : buildVideoMeta(source.data, videoId, language, chapters);
  for (const u of urls) {
    try {
      const r = await fetch(u);
      if (r.ok) {
        writeFileSync(outputPath, Buffer.from(await r.arrayBuffer()));
        return true;
      }
    } catch {}
  }
  return false;
 }
 function parseSrt(srt: string): Snippet[] {
  const blocks = srt.trim().split(/\n\n+/);
  const snippets: Snippet[] = [];
  for (const block of blocks) {
    const lines = block.split("\n");
    if (lines.length < 3) continue;
    const m = lines[1].match(/(\d{2}):(\d{2}):(\d{2}),(\d{3})\s*-->\s*(\d{2}):(\d{2}):(\d{2}),(\d{3})/);
    if (!m) continue;
    const start = parseInt(m[1]) * 3600 + parseInt(m[2]) * 60 + parseInt(m[3]) + parseInt(m[4]) / 1000;
    const end = parseInt(m[5]) * 3600 + parseInt(m[6]) * 60 + parseInt(m[7]) + parseInt(m[8]) / 1000;
    snippets.push({ text: lines.slice(2).join(" "), start, duration: end - start });
  }
  return snippets;
 }
 // --- Timestamp formatting ---
 function ts(t: number): string {
  const h = Math.floor(t / 3600);
  const m = Math.floor((t % 3600) / 60);
  const s = Math.floor(t % 60);
  return `${String(h).padStart(2, "0")}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}`;
 }
 function tsMs(t: number, sep: string): string {
  const h = Math.floor(t / 3600);
  const m = Math.floor((t % 3600) / 60);
  const s = Math.floor(t % 60);
  const ms = Math.round((t - Math.floor(t)) * 1000);
  return `${String(h).padStart(2, "0")}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}${sep}${String(ms).padStart(3, "0")}`;
 }
 // --- Paragraph grouping ---
 interface Paragraph {
  text: string;
  start: number;
  end: number;
 }
 function groupIntoParagraphs(snippets: Snippet[]): Paragraph[] {
  if (!snippets.length) return [];
  const paras: Paragraph[] = [];
  let buf: Snippet[] = [];
  for (let i = 0; i < snippets.length; i++) {
    buf.push(snippets[i]);
    const last = i === snippets.length - 1;
    const gap = !last && snippets[i + 1].start - (snippets[i].start + snippets[i].duration) > 1.5;
    if (last || gap || buf.length >= 8) {
      const lastS = buf[buf.length - 1];
      paras.push({ text: buf.map(s => s.text).join(" "), start: buf[0].start, end: lastS.start + lastS.duration });
      buf = [];
    }
  }
  return paras;
 }
 // --- Sentence segmentation ---
 const SENTENCE_END_RE = /[.?!…。？！⁈⁇‼‽．]/;
 function isCJK(ch: string): boolean {
  const code = ch.charCodeAt(0);
  return (code >= 0x4E00 && code <= 0x9FFF) ||
    (code >= 0x3040 && code <= 0x309F) ||
    (code >= 0x30A0 && code <= 0x30FF) ||
    (code >= 0xAC00 && code <= 0xD7AF) ||
    (code >= 0x3400 && code <= 0x4DBF) ||
    (code >= 0xF900 && code <= 0xFAFF);
 }
 function splitSnippetAtPunctuation(s: Snippet): { text: string; start: number; end: number }[] {
  const { text, start, duration } = s;
  const end = start + duration;
  if (!text.length) return [];
  const splitPoints: number[] = [];
  for (let i = 0; i < text.length; i++) {
    if (SENTENCE_END_RE.test(text[i])) {
      while (i + 1 < text.length && SENTENCE_END_RE.test(text[i + 1])) i++;
      if (i < text.length - 1) splitPoints.push(i);
    }
  }
  if (!splitPoints.length) return [{ text, start, end }];
  const parts: { text: string; start: number; end: number }[] = [];
  let prev = 0;
  for (const pos of splitPoints) {
    const partText = text.slice(prev, pos + 1).trim();
    if (partText) {
      parts.push({
        text: partText,
        start: start + (prev / text.length) * duration,
        end: start + ((pos + 1) / text.length) * duration,
      });
    }
    prev = pos + 1;
  }
  const remaining = text.slice(prev).trim();
  if (remaining) {
    parts.push({ text: remaining, start: start + (prev / text.length) * duration, end });
  }
  return parts;
 }
 function mergeTexts(texts: string[]): string {
  if (!texts.length) return "";
  let result = texts[0];
  for (let i = 1; i < texts.length; i++) {
    const next = texts[i];
    if (!next) continue;
    const lastChar = result[result.length - 1];
    const firstChar = next[0];
    if (isCJK(lastChar) || isCJK(firstChar)) {
      result += next;
    } else {
      result = result.trimEnd() + " " + next.trimStart();
    }
  }
  return result.replace(/ {2,}/g, " ");
 }
 function segmentIntoSentences(snippets: Snippet[]): Sentence[] {
  const parts: { text: string; start: number; end: number }[] = [];
  for (const s of snippets) parts.push(...splitSnippetAtPunctuation(s));
  const sentences: Sentence[] = [];
  let buf: { text: string; start: number; end: number }[] = [];
  for (const part of parts) {
    buf.push(part);
    if (SENTENCE_END_RE.test(part.text[part.text.length - 1])) {
      sentences.push({
        text: mergeTexts(buf.map(b => b.text)),
        start: ts(buf[0].start),
        end: ts(buf[buf.length - 1].end),
      });
      buf = [];
    }
  }
  if (buf.length) {
    sentences.push({
      text: mergeTexts(buf.map(b => b.text)),
      start: ts(buf[0].start),
      end: ts(buf[buf.length - 1].end),
    });
  }
  return sentences;
 }
 function parseTs(t: string): number {
  const [h, m, s] = t.split(":").map(Number);
  return h * 3600 + m * 60 + s;
 }
 function groupSentenceParas(sentences: Sentence[]): Paragraph[] {
  if (!sentences.length) return [];
  const paras: Paragraph[] = [];
  let buf: Sentence[] = [];
  for (let i = 0; i < sentences.length; i++) {
    buf.push(sentences[i]);
    const last = i === sentences.length - 1;
    const gap = !last && parseTs(sentences[i + 1].start) - parseTs(sentences[i].end) > 2;
    if (last || gap || buf.length >= 5) {
      paras.push({
        text: mergeTexts(buf.map(s => s.text)),
        start: parseTs(buf[0].start),
        end: parseTs(buf[buf.length - 1].end),
      });
      buf = [];
    }
  }
  return paras;
 }
 // --- Format functions ---
 function formatSrt(snippets: Snippet[]): string {
  return snippets
    .map((s, i) => {
      const end = i < snippets.length - 1 && snippets[i + 1].start < s.start + s.duration
        ? snippets[i + 1].start
        : s.start + s.duration;
      return `${i + 1}\n${tsMs(s.start, ",")} --> ${tsMs(end, ",")}\n${s.text}`;
    })
    .join("\n\n") + "\n";
 }
 function yamlEscape(s: string): string {
  if (/[:"'{}\[\]#&*!|>%@`\n]/.test(s) || s.trim() !== s) return `"${s.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"`;
  return s;
 }
 function extractSummary(description: string): string {
  if (!description) return "";
  const firstPara = description.split(/\n\s*\n/)[0].trim();
  const lines = firstPara.split("\n").filter(l => !/^\s*(https?:\/\/|#|@|\d+:\d+)/.test(l) && l.trim());
  return lines.join(" ").slice(0, 300).trim();
 }
 function formatMarkdown(sentences: Sentence[], meta: VideoMeta, opts: { timestamps: boolean; chapters: boolean; speakers: boolean }, snippets?: Snippet[]): string {
  const summary = extractSummary(meta.description);
  let md = "---\n";
  md += `title: ${yamlEscape(meta.title)}\n`;
  md += `channel: ${yamlEscape(meta.channel)}\n`;
  if (meta.publishDate) md += `date: ${meta.publishDate}\n`;
  md += `url: ${yamlEscape(meta.url)}\n`;
  if (meta.coverImage) md += `cover: ${meta.coverImage}\n`;
  if (summary) md += `description: ${yamlEscape(summary)}\n`;
  if (meta.language) md += `language: ${meta.language.code}\n`;
  md += "---\n\n";
  if (opts.speakers) {
    md += `# ${meta.title}\n\n`;
    if (summary) md += `${summary}\n\n`;
    if (meta.description) md += "# Description\n\n" + meta.description.trim() + "\n\n";
    if (meta.chapters.length) {
      md += "# Chapters\n\n";
      for (const ch of meta.chapters) md += `* [${ts(ch.start)}] ${ch.title}\n`;
      md += "\n";
    }
    md += "# Transcript\n\n";
    md += snippets ? formatSrt(snippets) : "";
    return md;
  }
  md += `# ${meta.title}\n\n`;
  if (summary) md += `${summary}\n\n`;
  const chapters = opts.chapters ? meta.chapters : [];
  if (chapters.length) {
    md += "## Table of Contents\n\n";
    for (const ch of chapters) md += opts.timestamps ? `* [${ts(ch.start)}] ${ch.title}\n` : `* ${ch.title}\n`;
    md += "\n";
    if (meta.coverImage) md += `\n![cover](${meta.coverImage})\n`;
    md += "\n";
    for (let i = 0; i < chapters.length; i++) {
      const nextStart = i < chapters.length - 1 ? chapters[i + 1].start : Infinity;
      const chSentences = sentences.filter(s => parseTs(s.start) >= chapters[i].start && parseTs(s.start) < nextStart);
      const paras = groupSentenceParas(chSentences);
      md += opts.timestamps
        ? `## [${ts(chapters[i].start)}] ${chapters[i].title}\n\n`
        : `## ${chapters[i].title}\n\n`;
      for (const p of paras) md += opts.timestamps ? `${p.text} [${ts(p.start)} → ${ts(p.end)}]\n\n` : `${p.text}\n\n`;
      md += "\n";
    }
  } else {
    const paras = groupSentenceParas(sentences);
    for (const p of paras) md += opts.timestamps ? `${p.text} [${ts(p.start)} → ${ts(p.end)}]\n\n` : `${p.text}\n\n`;
  }
  return md.trimEnd() + "\n";
 }
 function formatListOutput(videoId: string, title: string, transcripts: TranscriptInfo[]): string {
  const manual = transcripts.filter((t) => !t.isGenerated);
  const generated = transcripts.filter((t) => t.isGenerated);
  const tlLangs = transcripts.find((t) => t.translationLanguages.length > 0)?.translationLanguages || [];
  const fmtList = (list: TranscriptInfo[]) =>
    list.length ? list.map((t) => ` - ${t.languageCode} ("${t.language}")${t.isTranslatable ? " [TRANSLATABLE]" : ""}`).join("\n") : "None";
  const fmtTl = tlLangs.length
    ? tlLangs.map((t) => ` - ${t.languageCode} ("${t.language}")`).join("\n")
    : "None";
  return `Transcripts for ${videoId}${title ? ` (${title})` : ""}:\n\n(MANUALLY CREATED)\n${fmtList(manual)}\n\n(GENERATED)\n${fmtList(generated)}\n\n(TRANSLATION LANGUAGES)\n${fmtTl}`;
 }
 // --- File helpers ---
 function ensureDir(p: string) {
  const dir = dirname(p);
  if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
 }
 function resolveBaseDir(outputDir: string): string {
  return resolve(outputDir || "youtube-transcript");
 }
 function loadIndex(baseDir: string): Record<string, string> {
  try { return JSON.parse(readFileSync(join(baseDir, ".index.json"), "utf-8")); } catch { return {}; }
 }
 function saveIndex(baseDir: string, index: Record<string, string>) {
  const p = join(baseDir, ".index.json");
  ensureDir(p);
  writeFileSync(p, JSON.stringify(index, null, 2));
 }
 function lookupVideoDir(videoId: string, baseDir: string): string | null {
  const rel = loadIndex(baseDir)[videoId];
  if (rel) {
    const dir = resolve(baseDir, rel);
    if (existsSync(dir)) return dir;
  }
  return null;
 }
 function registerVideoDir(videoId: string, channelSlug: string, titleSlug: string, baseDir: string): string {
  const rel = join(channelSlug, titleSlug);
  const index = loadIndex(baseDir);
  index[videoId] = rel;
  saveIndex(baseDir, index);
  return resolve(baseDir, rel);
 }
 function hasCachedData(videoDir: string): boolean {
  return existsSync(join(videoDir, "meta.json")) && existsSync(join(videoDir, "transcript-raw.json"));
 }
 function loadMeta(videoDir: string): VideoMeta {
  return JSON.parse(readFileSync(join(videoDir, "meta.json"), "utf-8"));
 }
 function loadSnippets(videoDir: string): Snippet[] {
  return JSON.parse(readFileSync(join(videoDir, "transcript-raw.json"), "utf-8"));
 }
 function loadSentences(videoDir: string): Sentence[] {
  return JSON.parse(readFileSync(join(videoDir, "transcript-sentences.json"), "utf-8"));
 }
 // --- Main processing ---
 async function fetchAndCache(videoId: string, baseDir: string, opts: Options): Promise<{ meta: VideoMeta; snippets: Snippet[]; sentences: Sentence[]; videoDir: string }> {
  const html = await fetchHtml(videoId);
  const apiKey = extractApiKey(html, videoId);
  const data = await fetchInnertubeData(videoId, apiKey);
  const captionsJson = extractCaptionsJson(data, videoId);
  const transcripts = buildTranscriptList(captionsJson);
  const info = findTranscript(transcripts, opts.languages, opts.excludeGenerated, opts.excludeManual);
  const result = await fetchTranscriptSnippets(info, opts.translate || undefined);
  const description = data?.videoDetails?.shortDescription || "";
  const chapters = parseChapters(description);
  const langInfo = { code: result.languageCode, name: result.language, isGenerated: info.isGenerated };
  const meta = buildVideoMeta(data, videoId, langInfo, chapters);
  const videoDir = registerVideoDir(videoId, slugify(meta.channel), slugify(meta.title), baseDir);
  ensureDir(join(videoDir, "meta.json"));
@@ -656,9 +59,12 @@ async function fetchAndCache(videoId: string, baseDir: string, opts: Options): P
  const sentences = segmentIntoSentences(result.snippets);
  writeFileSync(join(videoDir, "transcript-sentences.json"), JSON.stringify(sentences, null, 2));
-  const imgPath = join(videoDir, "imgs", "cover.jpg");
+  const imagePath = join(videoDir, "imgs", "cover.jpg");
-  ensureDir(imgPath);
+  ensureDir(imagePath);
-  const downloaded = await downloadCoverImage(getThumbnailUrls(videoId, data), imgPath);
+  const downloaded = await downloadCoverImage(
    source.kind === "yt-dlp" ? getYtDlpThumbnailUrls(videoId, source.info) : getThumbnailUrls(videoId, source.data),
    imagePath
  );
  meta.coverImage = downloaded ? "imgs/cover.jpg" : "";
  writeFileSync(join(videoDir, "meta.json"), JSON.stringify(meta, null, 2));
@@ -669,15 +75,10 @@ async function fetchAndCache(videoId: string, baseDir: string, opts: Options): P
 async function processVideo(videoId: string, opts: Options): Promise<VideoResult> {
  const baseDir = resolveBaseDir(opts.outputDir);
  // --list: always fetch fresh
  if (opts.list) {
-    const html = await fetchHtml(videoId);
+    const source = await fetchVideoSource(videoId);
-    const apiKey = extractApiKey(html, videoId);
+    const title = source.kind === "yt-dlp" ? source.info.title || "" : source.data?.videoDetails?.title || "";
-    const data = await fetchInnertubeData(videoId, apiKey);
+    return { videoId, title, content: formatListOutput(videoId, title, source.transcripts) };
    const title = data?.videoDetails?.title || "";
    const captionsJson = extractCaptionsJson(data, videoId);
    const transcripts = buildTranscriptList(captionsJson);
    return { videoId, title, content: formatListOutput(videoId, title, transcripts) };
  }
  let videoDir = lookupVideoDir(videoId, baseDir);
@@ -690,8 +91,18 @@ async function processVideo(videoId: string, opts: Options): Promise<VideoResult
    meta = loadMeta(videoDir);
    snippets = loadSnippets(videoDir);
    sentences = loadSentences(videoDir);
-    const wantLangs = opts.translate ? [opts.translate] : opts.languages;
+    const wantedLanguages = opts.translate ? [opts.translate] : opts.languages;
-    if (!wantLangs.includes(meta.language.code)) needsFetch = true;
+    if (!wantedLanguages.includes(meta.language.code)) needsFetch = true;
    if (!needsFetch && meta.chapters.length > 0 && meta.chapters.some((chapter: any) => chapter.end === undefined)) {
      for (let i = 0; i < meta.chapters.length; i++) {
        meta.chapters[i].end = i < meta.chapters.length - 1
          ? meta.chapters[i + 1].start
          : Math.max(meta.duration, meta.chapters[i].start);
      }
      try {
        writeFileSync(join(videoDir, "meta.json"), JSON.stringify(meta, null, 2));
      } catch {}
    }
  }
  if (needsFetch) {
@@ -706,21 +117,19 @@ async function processVideo(videoId: string, opts: Options): Promise<VideoResult
    sentences = sentences!;
  }
-  let content: string;
+  const content = opts.format === "srt"
-  let ext: string;
+    ? formatSrt(snippets)
-
+    : formatMarkdown(
-  if (opts.format === "srt") {
+      sentences,
-    content = formatSrt(snippets);
+      meta,
-    ext = "srt";
+      {
-  } else {
+        timestamps: opts.timestamps,
-    content = formatMarkdown(sentences, meta, {
+        chapters: opts.chapters,
-      timestamps: opts.timestamps,
+        speakers: opts.speakers,
-      chapters: opts.chapters,
+      },
-      speakers: opts.speakers,
+      snippets
-    }, snippets);
+    );
-    ext = "md";
+  const ext = opts.format === "srt" ? "srt" : "md";
  }
  const filePath = opts.output ? resolve(opts.output) : join(videoDir!, `transcript.${ext}`);
  ensureDir(filePath);
  writeFileSync(filePath, content);
@@ -728,8 +137,6 @@ async function processVideo(videoId: string, opts: Options): Promise<VideoResult
  return { videoId, title: meta.title, filePath };
 }
 // --- CLI ---
 function printHelp() {
  console.log(`Usage: bun main.ts <video-url-or-id> [options]
@@ -773,13 +180,13 @@ function parseArgs(argv: string[]): Options | null {
      printHelp();
      process.exit(0);
    } else if (arg === "--languages") {
-      const v = argv[++i];
+      const value = argv[++i];
-      if (v) opts.languages = v.split(",").map((s) => s.trim());
+      if (value) opts.languages = value.split(",").map((entry) => entry.trim());
    } else if (arg === "--format") {
-      const v = argv[++i]?.toLowerCase();
+      const value = argv[++i]?.toLowerCase();
-      if (v === "text" || v === "srt") opts.format = v;
+      if (value === "text" || value === "srt") opts.format = value;
      else {
-        console.error(`Invalid format: ${v}. Use: text, srt`);
+        console.error(`Invalid format: ${value}. Use: text, srt`);
        return null;
      }
    } else if (arg === "--translate") {
@@ -814,6 +221,7 @@ function parseArgs(argv: string[]): Options | null {
    printHelp();
    return null;
  }
  return opts;
 }
@@ -828,14 +236,16 @@ async function main() {
  for (const videoId of opts.videoIds) {
    try {
-      const r = await processVideo(videoId, opts);
+      const result = await processVideo(videoId, opts);
-      if (r.error) console.error(`Error (${r.videoId}): ${r.error}`);
+      if (result.error) console.error(`Error (${result.videoId}): ${result.error}`);
-      else if (r.filePath) console.log(r.filePath);
+      else if (result.filePath) console.log(result.filePath);
-      else if (r.content) console.log(r.content);
+      else if (result.content) console.log(result.content);
-    } catch (e) {
+    } catch (error) {
-      console.error(`Error (${videoId}): ${(e as Error).message}`);
+      console.error(`Error (${videoId}): ${(error as Error).message}`);
    }
  }
 }
-main();
+if (import.meta.main) {
  main();
 }
@@ -0,0 +1,83 @@
 import type { TranscriptError } from "./types.ts";
 export function extractVideoId(input: string): string {
  input = input.replace(/\\/g, "").trim();
  const patterns = [
    /(?:youtube\.com\/watch\?.*v=|youtu\.be\/|youtube\.com\/embed\/|youtube\.com\/v\/|youtube\.com\/shorts\/)([a-zA-Z0-9_-]{11})/,
    /^([a-zA-Z0-9_-]{11})$/,
  ];
  for (const pattern of patterns) {
    const match = input.match(pattern);
    if (match) return match[1];
  }
  return input;
 }
 export function slugify(value: string): string {
  return value
    .toLowerCase()
    .replace(/[^\w\s-]/g, "")
    .replace(/\s+/g, "-")
    .replace(/-+/g, "-")
    .replace(/^-|-$/g, "") || "untitled";
 }
 export function htmlUnescape(value: string): string {
  return value
    .replace(/&amp;/g, "&")
    .replace(/&lt;/g, "<")
    .replace(/&gt;/g, ">")
    .replace(/&quot;/g, '"')
    .replace(/&#39;/g, "'")
    .replace(/&#x27;/g, "'")
    .replace(/&#x2F;/g, "/")
    .replace(/&apos;/g, "'")
    .replace(/&#(\d+);/g, (_, n) => String.fromCharCode(parseInt(n)))
    .replace(/&#x([0-9a-fA-F]+);/g, (_, n) => String.fromCharCode(parseInt(n, 16)));
 }
 export function stripTags(value: string): string {
  return value.replace(/<[^>]*>/g, "");
 }
 export function makeError(message: string, code?: string): Error {
  const error = new Error(message) as TranscriptError;
  if (code) error.code = code;
  return error;
 }
 export function normalizeError(error: unknown): TranscriptError {
  if (error instanceof Error) {
    const known = error as TranscriptError;
    if (known.code) return known;
    const message = known.message || String(error);
    const lower = message.toLowerCase();
    if (lower.includes("bot detected")) known.code = "BOT_DETECTED";
    else if (lower.includes("age restricted")) known.code = "AGE_RESTRICTED";
    else if (lower.includes("video unavailable")) known.code = "VIDEO_UNAVAILABLE";
    else if (lower.includes("transcripts disabled")) known.code = "TRANSCRIPTS_DISABLED";
    else if (lower.includes("no transcript found")) known.code = "NO_TRANSCRIPT";
    else if (lower.includes("invalid video id")) known.code = "INVALID_VIDEO_ID";
    else if (lower.includes("ip blocked") || lower.includes("recaptcha") || lower.includes("http 429")) known.code = "IP_BLOCKED";
    else if (lower.includes("cannot extract api key")) known.code = "PAGE_FETCH_FAILED";
    else if (lower.includes("innertube api") || lower.includes("http 403")) known.code = "INNERTUBE_REJECTED";
    else if (lower.includes("yt-dlp fallback failed")) known.code = "YT_DLP_FAILED";
    return known;
  }
  return makeError(String(error), "UNKNOWN") as TranscriptError;
 }
 export function shouldTryAlternateClient(error: unknown): boolean {
  const code = normalizeError(error).code;
  return code === "BOT_DETECTED" || code === "IP_BLOCKED" || code === "INNERTUBE_REJECTED" || code === "AGE_RESTRICTED" || code === "VIDEO_UNAVAILABLE";
 }
 export function shouldTryYtDlpFallback(error: unknown): boolean {
  const code = normalizeError(error).code;
  return code === "BOT_DETECTED" || code === "IP_BLOCKED" || code === "INNERTUBE_REJECTED" || code === "PAGE_FETCH_FAILED" || code === "AGE_RESTRICTED" || code === "VIDEO_UNAVAILABLE";
 }
 export function normalizePublishDate(uploadDate?: string): string {
  if (!uploadDate || !/^\d{8}$/.test(uploadDate)) return uploadDate || "";
  return `${uploadDate.slice(0, 4)}-${uploadDate.slice(4, 6)}-${uploadDate.slice(6, 8)}`;
 }
@@ -0,0 +1,60 @@
 import { existsSync, mkdirSync, readFileSync, writeFileSync } from "fs";
 import { dirname, join, resolve } from "path";
 import type { Sentence, Snippet, VideoMeta } from "./types.ts";
 export function ensureDir(path: string) {
  const dir = dirname(path);
  if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
 }
 export function resolveBaseDir(outputDir: string): string {
  return resolve(outputDir || "youtube-transcript");
 }
 function loadIndex(baseDir: string): Record<string, string> {
  try {
    return JSON.parse(readFileSync(join(baseDir, ".index.json"), "utf-8"));
  } catch {
    return {};
  }
 }
 function saveIndex(baseDir: string, index: Record<string, string>) {
  const path = join(baseDir, ".index.json");
  ensureDir(path);
  writeFileSync(path, JSON.stringify(index, null, 2));
 }
 export function lookupVideoDir(videoId: string, baseDir: string): string | null {
  const rel = loadIndex(baseDir)[videoId];
  if (rel) {
    const dir = resolve(baseDir, rel);
    if (existsSync(dir)) return dir;
  }
  return null;
 }
 export function registerVideoDir(videoId: string, channelSlug: string, titleSlug: string, baseDir: string): string {
  const rel = join(channelSlug, titleSlug);
  const index = loadIndex(baseDir);
  index[videoId] = rel;
  saveIndex(baseDir, index);
  return resolve(baseDir, rel);
 }
 export function hasCachedData(videoDir: string): boolean {
  return existsSync(join(videoDir, "meta.json")) && existsSync(join(videoDir, "transcript-raw.json"));
 }
 export function loadMeta(videoDir: string): VideoMeta {
  return JSON.parse(readFileSync(join(videoDir, "meta.json"), "utf-8"));
 }
 export function loadSnippets(videoDir: string): Snippet[] {
  return JSON.parse(readFileSync(join(videoDir, "transcript-raw.json"), "utf-8"));
 }
 export function loadSentences(videoDir: string): Sentence[] {
  return JSON.parse(readFileSync(join(videoDir, "transcript-sentences.json"), "utf-8"));
 }
@@ -0,0 +1,349 @@
 import { htmlUnescape, makeError, stripTags } from "./shared.ts";
 import type { Sentence, Snippet, TranscriptInfo, VideoMeta } from "./types.ts";
 interface Paragraph {
  text: string;
  start: number;
  end: number;
 }
 const SENTENCE_END_RE = /[.?!…。？！⁈⁇‼‽．]/;
 export function parseTranscriptXml(xml: string): Snippet[] {
  const snippets: Snippet[] = [];
  const pattern = /<text\s+start="([^"]*)"(?:\s+dur="([^"]*)")?[^>]*>([\s\S]*?)<\/text>/g;
  let match: RegExpExecArray | null;
  while ((match = pattern.exec(xml)) !== null) {
    const raw = match[3];
    if (!raw) continue;
    snippets.push({
      text: htmlUnescape(stripTags(raw)),
      start: parseFloat(match[1]),
      duration: parseFloat(match[2] || "0"),
    });
  }
  return snippets;
 }
 export function parseTranscriptJson3(text: string): Snippet[] {
  const data = JSON.parse(text);
  const events = Array.isArray(data?.events) ? data.events : [];
  const snippets: Snippet[] = [];
  for (const event of events) {
    const segs = Array.isArray(event?.segs) ? event.segs : [];
    const textParts = segs
      .map((seg: any) => htmlUnescape(String(seg?.utf8 || "").replace(/\n+/g, " ").trim()))
      .filter(Boolean);
    const merged = mergeTexts(textParts).trim();
    if (!merged) continue;
    snippets.push({
      text: merged,
      start: Number(event?.tStartMs || 0) / 1000,
      duration: Number(event?.dDurationMs || 0) / 1000,
    });
  }
  return snippets;
 }
 function parseSrt(srt: string): Snippet[] {
  const blocks = srt.trim().split(/\n\n+/);
  const snippets: Snippet[] = [];
  for (const block of blocks) {
    const lines = block.split("\n");
    if (lines.length < 3) continue;
    const match = lines[1].match(/(\d{2}):(\d{2}):(\d{2}),(\d{3})\s*-->\s*(\d{2}):(\d{2}):(\d{2}),(\d{3})/);
    if (!match) continue;
    const start = parseInt(match[1]) * 3600 + parseInt(match[2]) * 60 + parseInt(match[3]) + parseInt(match[4]) / 1000;
    const end = parseInt(match[5]) * 3600 + parseInt(match[6]) * 60 + parseInt(match[7]) + parseInt(match[8]) / 1000;
    snippets.push({ text: lines.slice(2).join(" "), start, duration: end - start });
  }
  return snippets;
 }
 export function parseWebVtt(vtt: string): Snippet[] {
  const blocks = vtt
    .replace(/^WEBVTT\s*/m, "")
    .trim()
    .split(/\n\n+/);
  const snippets: Snippet[] = [];
  for (const block of blocks) {
    const lines = block.split("\n").map((line) => line.trim()).filter(Boolean);
    const tsLine = lines.find((line) => line.includes("-->"));
    if (!tsLine) continue;
    const match = tsLine.match(
      /(?:(\d{2}):)?(\d{2}):(\d{2})\.(\d{3})\s*-->\s*(?:(\d{2}):)?(\d{2}):(\d{2})\.(\d{3})/
    );
    if (!match) continue;
    const start =
      (match[1] ? parseInt(match[1]) : 0) * 3600 +
      parseInt(match[2]) * 60 +
      parseInt(match[3]) +
      parseInt(match[4]) / 1000;
    const end =
      (match[5] ? parseInt(match[5]) : 0) * 3600 +
      parseInt(match[6]) * 60 +
      parseInt(match[7]) +
      parseInt(match[8]) / 1000;
    const text = htmlUnescape(stripTags(lines.slice(lines.indexOf(tsLine) + 1).join(" ").replace(/\s+/g, " ").trim()));
    if (!text) continue;
    snippets.push({ text, start, duration: end - start });
  }
  return snippets;
 }
 export function parseTranscriptPayload(payload: string, url: string): Snippet[] {
  const normalized = payload.trimStart();
  if (url.includes("fmt=json3") || normalized.startsWith("{")) return parseTranscriptJson3(payload);
  if (normalized.startsWith("WEBVTT")) return parseWebVtt(payload);
  if (/^\d+\s*\n\d{2}:\d{2}:\d{2},\d{3}\s*-->/.test(normalized)) return parseSrt(payload);
  return parseTranscriptXml(payload);
 }
 function isCJK(ch: string): boolean {
  const code = ch.charCodeAt(0);
  return (code >= 0x4E00 && code <= 0x9FFF) ||
    (code >= 0x3040 && code <= 0x309F) ||
    (code >= 0x30A0 && code <= 0x30FF) ||
    (code >= 0xAC00 && code <= 0xD7AF) ||
    (code >= 0x3400 && code <= 0x4DBF) ||
    (code >= 0xF900 && code <= 0xFAFF);
 }
 function splitSnippetAtPunctuation(snippet: Snippet): { text: string; start: number; end: number }[] {
  const { text, start, duration } = snippet;
  const end = start + duration;
  if (!text.length) return [];
  const splitPoints: number[] = [];
  for (let i = 0; i < text.length; i++) {
    if (SENTENCE_END_RE.test(text[i])) {
      while (i + 1 < text.length && SENTENCE_END_RE.test(text[i + 1])) i++;
      if (i < text.length - 1) splitPoints.push(i);
    }
  }
  if (!splitPoints.length) return [{ text, start, end }];
  const parts: { text: string; start: number; end: number }[] = [];
  let prev = 0;
  for (const pos of splitPoints) {
    const partText = text.slice(prev, pos + 1).trim();
    if (partText) {
      parts.push({
        text: partText,
        start: start + (prev / text.length) * duration,
        end: start + ((pos + 1) / text.length) * duration,
      });
    }
    prev = pos + 1;
  }
  const remaining = text.slice(prev).trim();
  if (remaining) parts.push({ text: remaining, start: start + (prev / text.length) * duration, end });
  return parts;
 }
 function mergeTexts(texts: string[]): string {
  if (!texts.length) return "";
  let result = texts[0];
  for (let i = 1; i < texts.length; i++) {
    const next = texts[i];
    if (!next) continue;
    const lastChar = result[result.length - 1];
    const firstChar = next[0];
    if (isCJK(lastChar) || isCJK(firstChar)) {
      result += next;
    } else {
      result = result.trimEnd() + " " + next.trimStart();
    }
  }
  return result.replace(/ {2,}/g, " ");
 }
 export function ts(time: number): string {
  const h = Math.floor(time / 3600);
  const m = Math.floor((time % 3600) / 60);
  const s = Math.floor(time % 60);
  return `${String(h).padStart(2, "0")}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}`;
 }
 function tsMs(time: number, sep: string): string {
  const h = Math.floor(time / 3600);
  const m = Math.floor((time % 3600) / 60);
  const s = Math.floor(time % 60);
  const ms = Math.round((time - Math.floor(time)) * 1000);
  return `${String(h).padStart(2, "0")}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}${sep}${String(ms).padStart(3, "0")}`;
 }
 function parseTs(time: string): number {
  const [h, m, s] = time.split(":").map(Number);
  return h * 3600 + m * 60 + s;
 }
 export function segmentIntoSentences(snippets: Snippet[]): Sentence[] {
  const parts: { text: string; start: number; end: number }[] = [];
  for (const snippet of snippets) parts.push(...splitSnippetAtPunctuation(snippet));
  const sentences: Sentence[] = [];
  let buffer: { text: string; start: number; end: number }[] = [];
  for (const part of parts) {
    buffer.push(part);
    if (SENTENCE_END_RE.test(part.text[part.text.length - 1])) {
      sentences.push({
        text: mergeTexts(buffer.map((entry) => entry.text)),
        start: ts(buffer[0].start),
        end: ts(buffer[buffer.length - 1].end),
      });
      buffer = [];
    }
  }
  if (buffer.length) {
    sentences.push({
      text: mergeTexts(buffer.map((entry) => entry.text)),
      start: ts(buffer[0].start),
      end: ts(buffer[buffer.length - 1].end),
    });
  }
  return sentences;
 }
 function groupSentenceParas(sentences: Sentence[]): Paragraph[] {
  if (!sentences.length) return [];
  const paragraphs: Paragraph[] = [];
  let buffer: Sentence[] = [];
  for (let i = 0; i < sentences.length; i++) {
    buffer.push(sentences[i]);
    const last = i === sentences.length - 1;
    const gap = !last && parseTs(sentences[i + 1].start) - parseTs(sentences[i].end) > 2;
    if (last || gap || buffer.length >= 5) {
      paragraphs.push({
        text: mergeTexts(buffer.map((sentence) => sentence.text)),
        start: parseTs(buffer[0].start),
        end: parseTs(buffer[buffer.length - 1].end),
      });
      buffer = [];
    }
  }
  return paragraphs;
 }
 export function formatSrt(snippets: Snippet[]): string {
  return snippets
    .map((snippet, index) => {
      const end = index < snippets.length - 1 && snippets[index + 1].start < snippet.start + snippet.duration
        ? snippets[index + 1].start
        : snippet.start + snippet.duration;
      return `${index + 1}\n${tsMs(snippet.start, ",")} --> ${tsMs(end, ",")}\n${snippet.text}`;
    })
    .join("\n\n") + "\n";
 }
 function yamlEscape(value: string): string {
  if (/[:"'{}\[\]#&*!|>%@`\n]/.test(value) || value.trim() !== value) {
    return `"${value.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"`;
  }
  return value;
 }
 function extractSummary(description: string): string {
  if (!description) return "";
  const firstPara = description.split(/\n\s*\n/)[0].trim();
  const lines = firstPara.split("\n").filter((line) => !/^\s*(https?:\/\/|#|@|\d+:\d+)/.test(line) && line.trim());
  return lines.join(" ").slice(0, 300).trim();
 }
 export function formatMarkdown(
  sentences: Sentence[],
  meta: VideoMeta,
  opts: { timestamps: boolean; chapters: boolean; speakers: boolean },
  snippets?: Snippet[]
 ): string {
  const summary = extractSummary(meta.description);
  let md = "---\n";
  md += `title: ${yamlEscape(meta.title)}\n`;
  md += `channel: ${yamlEscape(meta.channel)}\n`;
  if (meta.publishDate) md += `date: ${meta.publishDate}\n`;
  md += `url: ${yamlEscape(meta.url)}\n`;
  if (meta.coverImage) md += `cover: ${meta.coverImage}\n`;
  if (summary) md += `description: ${yamlEscape(summary)}\n`;
  if (meta.language) md += `language: ${meta.language.code}\n`;
  md += "---\n\n";
  if (opts.speakers) {
    md += `# ${meta.title}\n\n`;
    if (summary) md += `${summary}\n\n`;
    if (meta.description) md += `# Description\n\n${meta.description.trim()}\n\n`;
    if (meta.chapters.length) {
      md += "# Chapters\n\n";
      for (const chapter of meta.chapters) md += `* [${ts(chapter.start)}] ${chapter.title}\n`;
      md += "\n";
    }
    md += "# Transcript\n\n";
    md += snippets ? formatSrt(snippets) : "";
    return md;
  }
  md += `# ${meta.title}\n\n`;
  if (summary) md += `${summary}\n\n`;
  const chapters = opts.chapters ? meta.chapters : [];
  if (chapters.length) {
    md += "## Table of Contents\n\n";
    for (const chapter of chapters) md += opts.timestamps ? `* [${ts(chapter.start)}] ${chapter.title}\n` : `* ${chapter.title}\n`;
    md += "\n";
    if (meta.coverImage) md += `\n![cover](${meta.coverImage})\n`;
    md += "\n";
    for (let i = 0; i < chapters.length; i++) {
      const nextStart = i < chapters.length - 1 ? chapters[i + 1].start : Infinity;
      const chapterSentences = sentences.filter((sentence) => parseTs(sentence.start) >= chapters[i].start && parseTs(sentence.start) < nextStart);
      const paragraphs = groupSentenceParas(chapterSentences);
      md += opts.timestamps ? `## [${ts(chapters[i].start)}] ${chapters[i].title}\n\n` : `## ${chapters[i].title}\n\n`;
      for (const paragraph of paragraphs) {
        md += opts.timestamps ? `${paragraph.text} [${ts(paragraph.start)} → ${ts(paragraph.end)}]\n\n` : `${paragraph.text}\n\n`;
      }
      md += "\n";
    }
  } else {
    const paragraphs = groupSentenceParas(sentences);
    for (const paragraph of paragraphs) {
      md += opts.timestamps ? `${paragraph.text} [${ts(paragraph.start)} → ${ts(paragraph.end)}]\n\n` : `${paragraph.text}\n\n`;
    }
  }
  return md.trimEnd() + "\n";
 }
 export function formatListOutput(videoId: string, title: string, transcripts: TranscriptInfo[]): string {
  const manual = transcripts.filter((transcript) => !transcript.isGenerated);
  const generated = transcripts.filter((transcript) => transcript.isGenerated);
  const translationLanguages = transcripts.find((transcript) => transcript.translationLanguages.length > 0)?.translationLanguages || [];
  const formatList = (list: TranscriptInfo[]) =>
    list.length
      ? list.map((transcript) => ` - ${transcript.languageCode} ("${transcript.language}")${transcript.isTranslatable ? " [TRANSLATABLE]" : ""}`).join("\n")
      : "None";
  const formatTranslations = translationLanguages.length
    ? translationLanguages.map((language) => ` - ${language.languageCode} ("${language.language}")`).join("\n")
    : "None";
  return `Transcripts for ${videoId}${title ? ` (${title})` : ""}:\n\n(MANUALLY CREATED)\n${formatList(manual)}\n\n(GENERATED)\n${formatList(generated)}\n\n(TRANSLATION LANGUAGES)\n${formatTranslations}`;
 }
 export function findTranscript(
  transcripts: TranscriptInfo[],
  languages: string[],
  excludeGenerated: boolean,
  excludeManual: boolean
 ): TranscriptInfo {
  let filtered = transcripts;
  if (excludeGenerated) filtered = filtered.filter((transcript) => !transcript.isGenerated);
  if (excludeManual) filtered = filtered.filter((transcript) => transcript.isGenerated);
  for (const language of languages) {
    const found = filtered.find((transcript) => transcript.languageCode === language);
    if (found) return found;
  }
  const available = filtered.map((transcript) => `${transcript.languageCode} ("${transcript.language}")`).join(", ");
  throw makeError(`No transcript found for languages [${languages.join(", ")}]. Available: ${available || "none"}`, "NO_TRANSCRIPT");
 }
@@ -0,0 +1,123 @@
 export type Format = "text" | "srt";
 export interface Options {
  videoIds: string[];
  languages: string[];
  format: Format;
  translate: string;
  list: boolean;
  excludeGenerated: boolean;
  excludeManual: boolean;
  output: string;
  outputDir: string;
  timestamps: boolean;
  chapters: boolean;
  speakers: boolean;
  refresh: boolean;
 }
 export interface Snippet {
  text: string;
  start: number;
  duration: number;
 }
 export interface Sentence {
  text: string;
  start: string;
  end: string;
 }
 export interface TranscriptLanguage {
  language: string;
  languageCode: string;
 }
 export interface TranscriptInfo {
  language: string;
  languageCode: string;
  isGenerated: boolean;
  isTranslatable: boolean;
  baseUrl: string;
  translationLanguages: TranscriptLanguage[];
 }
 export interface Chapter {
  title: string;
  start: number;
  end: number;
 }
 export interface LanguageMeta {
  code: string;
  name: string;
  isGenerated: boolean;
 }
 export interface VideoMeta {
  videoId: string;
  title: string;
  channel: string;
  channelId: string;
  description: string;
  duration: number;
  publishDate: string;
  url: string;
  coverImage: string;
  thumbnailUrl: string;
  language: LanguageMeta;
  chapters: Chapter[];
 }
 export interface VideoResult {
  videoId: string;
  title?: string;
  filePath?: string;
  content?: string;
  error?: string;
 }
 export interface InnerTubeSession {
  apiKey: string;
  webClientVersion: string;
  visitorData: string;
 }
 export interface InnerTubeClient {
  id: string;
  clientName: string;
  clientVersion?: string;
  clientHeaderName?: string;
  userAgent: string;
  extraContext?: Record<string, any>;
 }
 export interface TranscriptError extends Error {
  code?: string;
 }
 export interface YtDlpTrack {
  ext?: string;
  url?: string;
  name?: string;
 }
 export interface YtDlpInfo {
  title?: string;
  channel?: string;
  channel_id?: string;
  uploader?: string;
  uploader_id?: string;
  description?: string;
  duration?: number;
  upload_date?: string;
  webpage_url?: string;
  thumbnail?: string;
  thumbnails?: { url?: string; width?: number; height?: number }[];
  subtitles?: Record<string, YtDlpTrack[]>;
  automatic_captions?: Record<string, YtDlpTrack[]>;
 }
 export type VideoSource =
  | { kind: "innertube"; data: any; transcripts: TranscriptInfo[] }
  | { kind: "yt-dlp"; info: YtDlpInfo; transcripts: TranscriptInfo[] };
@@ -0,0 +1,477 @@
 import { spawnSync } from "child_process";
 import { writeFileSync } from "fs";
 import { makeError, normalizeError, normalizePublishDate, shouldTryAlternateClient, shouldTryYtDlpFallback } from "./shared.ts";
 import { parseTranscriptPayload } from "./transcript.ts";
 import type {
  Chapter,
  InnerTubeClient,
  InnerTubeSession,
  LanguageMeta,
  Snippet,
  TranscriptInfo,
  VideoMeta,
  VideoSource,
  YtDlpInfo,
  YtDlpTrack,
 } from "./types.ts";
 const WATCH_URL = "https://www.youtube.com/watch?v=";
 const INNERTUBE_URL = "https://www.youtube.com/youtubei/v1/player";
 const WATCH_PAGE_USER_AGENT =
  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36";
 const DEFAULT_WEB_CLIENT_VERSION = "2.20260320.08.00";
 const YT_DLP_MAX_BUFFER = 32 * 1024 * 1024;
 let cachedYtDlpCommand: { command: string; args: string[]; label: string } | null | undefined;
 const INNER_TUBE_CLIENTS: InnerTubeClient[] = [
  {
    id: "android",
    clientName: "ANDROID",
    clientHeaderName: "3",
    clientVersion: "20.10.38",
    userAgent:
      "com.google.android.youtube/20.10.38 (Linux; U; Android 14; en_US; Pixel 8 Pro; Build/AP1A.240405.002)",
    extraContext: {
      clientFormFactor: "SMALL_FORM_FACTOR",
      androidSdkVersion: 34,
      osName: "Android",
      osVersion: "14",
      platform: "MOBILE",
    },
  },
  {
    id: "web",
    clientName: "WEB",
    clientHeaderName: "1",
    userAgent: WATCH_PAGE_USER_AGENT,
  },
  {
    id: "ios",
    clientName: "IOS",
    clientHeaderName: "5",
    clientVersion: "20.10.4",
    userAgent:
      "com.google.ios.youtube/20.10.4 (iPhone16,2; U; CPU iOS 18_3 like Mac OS X; en_US)",
    extraContext: {
      deviceMake: "Apple",
      deviceModel: "iPhone16,2",
      osName: "iPhone",
      osVersion: "18.3.0.22D5054f",
      platform: "MOBILE",
    },
  },
 ];
 async function fetchHtml(videoId: string): Promise<string> {
  const watchUrl = `${WATCH_URL}${videoId}&hl=en&persist_hl=1&has_verified=1&bpctr=9999999999`;
  const baseHeaders = {
    Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
    "Accept-Language": "en-US,en;q=0.9",
    "Cache-Control": "no-cache",
    Pragma: "no-cache",
    "User-Agent": WATCH_PAGE_USER_AGENT,
  };
  const response = await fetch(watchUrl, { headers: baseHeaders });
  if (!response.ok) throw new Error(`HTTP ${response.status} fetching video page`);
  let html = await response.text();
  if (html.includes('action="https://consent.youtube.com/s"')) {
    const consentValue = html.match(/name="v" value="(.*?)"/);
    if (!consentValue) throw new Error("Failed to create consent cookie");
    const consentResponse = await fetch(watchUrl, {
      headers: {
        ...baseHeaders,
        Cookie: `CONSENT=YES+${consentValue[1]}`,
      },
    });
    if (!consentResponse.ok) throw new Error(`HTTP ${consentResponse.status} fetching video page (consent)`);
    html = await consentResponse.text();
  }
  return html;
 }
 function extractSession(html: string, videoId: string): InnerTubeSession {
  const apiKey = html.match(/"INNERTUBE_API_KEY":\s*"([a-zA-Z0-9_-]+)"/)?.[1];
  if (!apiKey) {
    if (html.includes('class="g-recaptcha"')) throw new Error(`IP blocked for ${videoId} (reCAPTCHA)`);
    throw new Error(`Cannot extract API key for ${videoId}`);
  }
  const webClientVersion =
    html.match(/"INNERTUBE_CLIENT_VERSION":\s*"([^"]+)"/)?.[1] ||
    html.match(/"clientVersion":"([^"]+)"/)?.[1] ||
    DEFAULT_WEB_CLIENT_VERSION;
  const visitorData =
    html.match(/"VISITOR_DATA":"([^"]+)"/)?.[1] ||
    html.match(/"visitorData":"([^"]+)"/)?.[1] ||
    "";
  return { apiKey, webClientVersion, visitorData };
 }
 function buildInnerTubeContext(client: InnerTubeClient, session: InnerTubeSession, videoId: string) {
  return {
    context: {
      client: {
        hl: "en",
        gl: "US",
        utcOffsetMinutes: 0,
        visitorData: session.visitorData,
        clientName: client.clientName,
        clientVersion: client.clientVersion || session.webClientVersion,
        ...client.extraContext,
      },
      request: { useSsl: true },
    },
    videoId,
  };
 }
 async function fetchInnertubeData(videoId: string, session: InnerTubeSession, client: InnerTubeClient): Promise<any> {
  const clientVersion = client.clientVersion || session.webClientVersion;
  const headers: Record<string, string> = {
    Accept: "application/json",
    "Accept-Language": "en-US,en;q=0.9",
    "Content-Type": "application/json",
    Origin: "https://www.youtube.com",
    Referer: `${WATCH_URL}${videoId}`,
    "User-Agent": client.userAgent,
    "X-YouTube-Client-Name": client.clientHeaderName || "1",
    "X-YouTube-Client-Version": clientVersion,
  };
  if (session.visitorData) headers["X-Goog-Visitor-Id"] = session.visitorData;
  const response = await fetch(`${INNERTUBE_URL}?key=${session.apiKey}&prettyPrint=false`, {
    method: "POST",
    headers,
    body: JSON.stringify(buildInnerTubeContext(client, session, videoId)),
  });
  if (response.status === 429) throw new Error(`IP blocked for ${videoId} (429)`);
  if (!response.ok) throw new Error(`HTTP ${response.status} from InnerTube API`);
  return response.json();
 }
 function assertPlayability(data: any, videoId: string) {
  const playabilityStatus = data?.playabilityStatus;
  if (!playabilityStatus) return;
  const status = playabilityStatus.status;
  if (status === "OK" || !status) return;
  const reason = playabilityStatus.reason || "";
  const reasonLower = reason.toLowerCase();
  if (status === "LOGIN_REQUIRED") {
    if (reasonLower.includes("bot")) throw makeError(`Request blocked for ${videoId}: bot detected`, "BOT_DETECTED");
    if (reasonLower.includes("inappropriate")) throw makeError(`Age restricted: ${videoId}`, "AGE_RESTRICTED");
  }
  if (status === "ERROR" && reasonLower.includes("unavailable")) {
    if (videoId.startsWith("http")) throw makeError("Invalid video ID: pass the ID, not the URL", "INVALID_VIDEO_ID");
    throw makeError(`Video unavailable: ${videoId}`, "VIDEO_UNAVAILABLE");
  }
  const subreasons = playabilityStatus.errorScreen?.playerErrorMessageRenderer?.subreason?.runs?.map((run: any) => run.text).join("") || "";
  throw new Error(`Video unplayable (${videoId}): ${reason} ${subreasons}`.trim());
 }
 function extractCaptionsJson(data: any, videoId: string): any {
  assertPlayability(data, videoId);
  const captionsJson = data?.captions?.playerCaptionsTracklistRenderer;
  if (!captionsJson || !captionsJson.captionTracks) throw makeError(`Transcripts disabled for ${videoId}`, "TRANSCRIPTS_DISABLED");
  return captionsJson;
 }
 function buildTranscriptList(captionsJson: any): TranscriptInfo[] {
  const translationLanguages = (captionsJson.translationLanguages || []).map((language: any) => ({
    language: language.languageName?.runs?.[0]?.text || language.languageName?.simpleText || "",
    languageCode: language.languageCode,
  }));
  return (captionsJson.captionTracks || []).map((track: any) => ({
    language: track.name?.runs?.[0]?.text || track.name?.simpleText || "",
    languageCode: track.languageCode,
    isGenerated: track.kind === "asr",
    isTranslatable: !!track.isTranslatable,
    baseUrl: track.baseUrl || "",
    translationLanguages: track.isTranslatable ? translationLanguages : [],
  }));
 }
 export async function fetchTranscriptSnippets(
  info: TranscriptInfo,
  translateTo?: string
 ): Promise<{ snippets: Snippet[]; language: string; languageCode: string }> {
  let url = info.baseUrl;
  let language = info.language;
  let languageCode = info.languageCode;
  if (translateTo) {
    if (!info.isTranslatable) throw new Error(`Transcript ${info.languageCode} is not translatable`);
    const translatedLanguage = info.translationLanguages.find((entry) => entry.languageCode === translateTo);
    if (!translatedLanguage) throw new Error(`Translation language ${translateTo} not available`);
    url += `&tlang=${translateTo}`;
    language = translatedLanguage.language;
    languageCode = translateTo;
  }
  const response = await fetch(url, {
    headers: {
      "Accept-Language": "en-US,en;q=0.9",
      "User-Agent": WATCH_PAGE_USER_AGENT,
    },
  });
  if (!response.ok) throw new Error(`HTTP ${response.status} fetching transcript`);
  return {
    snippets: parseTranscriptPayload(await response.text(), url),
    language,
    languageCode,
  };
 }
 export function detectYtDlpCommand(): { command: string; args: string[]; label: string } | null {
  if (cachedYtDlpCommand !== undefined) return cachedYtDlpCommand;
  const candidates = [
    { command: "yt-dlp", args: [], label: "yt-dlp" },
    { command: "uvx", args: ["--from", "yt-dlp", "yt-dlp"], label: "uvx --from yt-dlp yt-dlp" },
    { command: "python3", args: ["-m", "yt_dlp"], label: "python3 -m yt_dlp" },
  ];
  for (const candidate of candidates) {
    const probe = spawnSync(candidate.command, [...candidate.args, "--version"], {
      encoding: "utf8",
      maxBuffer: 1024 * 1024,
    });
    if (probe.status !== 0) continue;
    const helpProbe = spawnSync(candidate.command, [...candidate.args, "--help"], {
      encoding: "utf8",
      maxBuffer: 2 * 1024 * 1024,
    });
    const helpText = `${helpProbe.stdout || ""}\n${helpProbe.stderr || ""}`;
    const supportsRequiredFlags =
      helpProbe.status === 0 &&
      helpText.includes("--js-runtimes") &&
      helpText.includes("--remote-components");
    if (supportsRequiredFlags) {
      cachedYtDlpCommand = candidate;
      return candidate;
    }
  }
  cachedYtDlpCommand = null;
  return cachedYtDlpCommand;
 }
 export function selectYtDlpTrack(entries: YtDlpTrack[]): YtDlpTrack | null {
  const preferredExts = ["json3", "srv3", "srv2", "srv1", "ttml", "vtt"];
  for (const ext of preferredExts) {
    const match = entries.find((entry) => entry.url && entry.ext === ext);
    if (match) return match;
  }
  return entries.find((entry) => !!entry.url) || null;
 }
 export function buildTranscriptListFromYtDlp(info: YtDlpInfo): TranscriptInfo[] {
  const translationLanguages = Object.entries(info.automatic_captions || {}).map(([languageCode, entries]) => ({
    language: entries.find((entry) => entry.name)?.name || languageCode,
    languageCode,
  }));
  const manual = Object.entries(info.subtitles || {}).flatMap(([languageCode, entries]) => {
    const selected = selectYtDlpTrack(entries);
    if (!selected?.url) return [];
    return [{
      language: selected.name || languageCode,
      languageCode,
      isGenerated: false,
      isTranslatable: translationLanguages.length > 0,
      baseUrl: selected.url,
      translationLanguages,
    }];
  });
  const generated = Object.entries(info.automatic_captions || {}).flatMap(([languageCode, entries]) => {
    const selected = selectYtDlpTrack(entries);
    if (!selected?.url) return [];
    return [{
      language: selected.name || languageCode,
      languageCode,
      isGenerated: true,
      isTranslatable: translationLanguages.length > 0,
      baseUrl: selected.url,
      translationLanguages,
    }];
  });
  return [...manual, ...generated];
 }
 function fetchYtDlpInfo(videoId: string): YtDlpInfo {
  const command = detectYtDlpCommand();
  if (!command) {
    throw makeError(
      `Request blocked for ${videoId}: bot detected. yt-dlp fallback unavailable (install yt-dlp or uv).`,
      "YT_DLP_UNAVAILABLE"
    );
  }
  const args = [
    ...command.args,
    "-J",
    "--skip-download",
    "--js-runtimes",
    "bun",
    "--remote-components",
    "ejs:github",
  ];
  const cookiesFromBrowser = process.env.YOUTUBE_TRANSCRIPT_COOKIES_FROM_BROWSER?.trim();
  if (cookiesFromBrowser) args.push("--cookies-from-browser", cookiesFromBrowser);
  args.push(`${WATCH_URL}${videoId}`);
  const result = spawnSync(command.command, args, {
    encoding: "utf8",
    maxBuffer: YT_DLP_MAX_BUFFER,
  });
  if (result.status !== 0) {
    const stderr = (result.stderr || "").trim();
    const stdout = (result.stdout || "").trim();
    const detail = stderr || stdout || `exit ${result.status ?? "unknown"}`;
    throw makeError(`yt-dlp fallback failed for ${videoId} (${command.label}): ${detail}`, "YT_DLP_FAILED");
  }
  return JSON.parse(result.stdout);
 }
 async function fetchInnertubeSource(videoId: string): Promise<VideoSource> {
  const html = await fetchHtml(videoId);
  const session = extractSession(html, videoId);
  const attempts: string[] = [];
  let lastError: Error | null = null;
  for (const client of INNER_TUBE_CLIENTS) {
    try {
      const data = await fetchInnertubeData(videoId, session, client);
      const captionsJson = extractCaptionsJson(data, videoId);
      return { kind: "innertube", data, transcripts: buildTranscriptList(captionsJson) };
    } catch (error) {
      const normalized = normalizeError(error);
      attempts.push(`${client.id}: ${normalized.message}`);
      lastError = normalized;
      if (!shouldTryAlternateClient(normalized)) break;
    }
  }
  if (!lastError) throw makeError(`Unable to fetch transcript metadata for ${videoId}`, "UNKNOWN");
  if (attempts.length > 1) {
    throw makeError(`${lastError.message}. Tried clients: ${attempts.join("; ")}`, normalizeError(lastError).code);
  }
  throw lastError;
 }
 export async function resolveVideoSource(
  videoId: string,
  fetchPrimary: (videoId: string) => Promise<VideoSource>,
  fetchFallback: (videoId: string) => YtDlpInfo,
  logWarning: (message: string) => void = (message) => console.error(message)
 ): Promise<VideoSource> {
  try {
    return await fetchPrimary(videoId);
  } catch (error) {
    const normalized = normalizeError(error);
    if (!shouldTryYtDlpFallback(normalized)) throw normalized;
    logWarning(`Warning (${videoId}): ${normalized.message}. Retrying with yt-dlp fallback.`);
    const info = fetchFallback(videoId);
    const transcripts = buildTranscriptListFromYtDlp(info);
    if (!transcripts.length) throw makeError(`Transcripts disabled for ${videoId}`, "TRANSCRIPTS_DISABLED");
    return { kind: "yt-dlp", info, transcripts };
  }
 }
 export async function fetchVideoSource(videoId: string): Promise<VideoSource> {
  return resolveVideoSource(videoId, fetchInnertubeSource, fetchYtDlpInfo);
 }
 export function parseChapters(description: string, duration: number = 0): Chapter[] {
  const raw: { title: string; start: number }[] = [];
  for (const line of description.split("\n")) {
    const match = line.trim().match(/^(?:(\d{1,2}):)?(\d{1,2}):(\d{2})\s+(.+)$/);
    if (match) {
      const hours = match[1] ? parseInt(match[1]) : 0;
      raw.push({ title: match[4].trim(), start: hours * 3600 + parseInt(match[2]) * 60 + parseInt(match[3]) });
    }
  }
  if (raw.length < 2) return [];
  return raw.map((chapter, index) => ({
    title: chapter.title,
    start: chapter.start,
    end: index < raw.length - 1 ? raw[index + 1].start : Math.max(duration, chapter.start),
  }));
 }
 export function getThumbnailUrls(videoId: string, data: any): string[] {
  const urls = [
    `https://i.ytimg.com/vi/${videoId}/maxresdefault.jpg`,
    `https://i.ytimg.com/vi/${videoId}/hqdefault.jpg`,
  ];
  const thumbnails = data?.videoDetails?.thumbnail?.thumbnails ||
    data?.microformat?.playerMicroformatRenderer?.thumbnail?.thumbnails ||
    [];
  if (thumbnails.length) {
    const sorted = [...thumbnails].sort((a: any, b: any) => (b.width || 0) - (a.width || 0));
    for (const thumbnail of sorted) {
      if (thumbnail.url && !urls.includes(thumbnail.url)) urls.push(thumbnail.url);
    }
  }
  return urls;
 }
 export function getYtDlpThumbnailUrls(videoId: string, info: YtDlpInfo): string[] {
  const urls = getThumbnailUrls(videoId, null);
  const thumbnails = Array.isArray(info.thumbnails) ? info.thumbnails : [];
  const sorted = [...thumbnails].sort((a, b) => (b?.width || 0) - (a?.width || 0));
  for (const thumbnail of sorted) {
    if (thumbnail?.url && !urls.includes(thumbnail.url)) urls.push(thumbnail.url);
  }
  if (info.thumbnail && !urls.includes(info.thumbnail)) urls.push(info.thumbnail);
  return urls;
 }
 export function buildVideoMeta(data: any, videoId: string, language: LanguageMeta, chapters: Chapter[]): VideoMeta {
  const videoDetails = data?.videoDetails || {};
  const microformat = data?.microformat?.playerMicroformatRenderer || {};
  return {
    videoId,
    title: videoDetails.title || microformat.title?.simpleText || "",
    channel: videoDetails.author || microformat.ownerChannelName || "",
    channelId: videoDetails.channelId || microformat.externalChannelId || "",
    description: videoDetails.shortDescription || microformat.description?.simpleText || "",
    duration: parseInt(videoDetails.lengthSeconds || "0"),
    publishDate: microformat.publishDate || microformat.uploadDate || "",
    url: `${WATCH_URL}${videoId}`,
    coverImage: "",
    thumbnailUrl: getThumbnailUrls(videoId, data)[0],
    language,
    chapters,
  };
 }
 export function buildVideoMetaFromYtDlp(
  info: YtDlpInfo,
  videoId: string,
  language: LanguageMeta,
  chapters: Chapter[]
 ): VideoMeta {
  return {
    videoId,
    title: info.title || "",
    channel: info.channel || info.uploader || "",
    channelId: info.channel_id || info.uploader_id || "",
    description: info.description || "",
    duration: Number(info.duration || 0),
    publishDate: normalizePublishDate(info.upload_date),
    url: info.webpage_url || `${WATCH_URL}${videoId}`,
    coverImage: "",
    thumbnailUrl: getYtDlpThumbnailUrls(videoId, info)[0] || "",
    language,
    chapters,
  };
 }
 export async function downloadCoverImage(urls: string[], outputPath: string): Promise<boolean> {
  for (const url of urls) {
    try {
      const response = await fetch(url);
      if (response.ok) {
        writeFileSync(outputPath, Buffer.from(await response.arrayBuffer()));
        return true;
      }
    } catch {}
  }
  return false;
 }
Author	SHA1	Message	Date
Jim Liu 宝玉	40f9f05c22	chore: release v1.81.0	2026-03-24 20:59:56 -05:00
Jim Liu 宝玉	09ce80357f	feat(baoyu-youtube-transcript): add yt-dlp fallback and modularize codebase Retry with alternate InnerTube client identities when YouTube returns anti-bot responses, then fall back to yt-dlp when available. Split monolithic main.ts into typed modules (youtube, transcript, storage, shared, types) and add unit tests.	2026-03-24 20:59:04 -05:00
Jim Liu 宝玉	7c995fcc24	chore: release v1.80.1	2026-03-24 20:06:02 -05:00
Jim Liu 宝玉	151f1ec2a8	fix(baoyu-image-gen): use correct prompt field name for Jimeng API	2026-03-24 20:04:21 -05:00
Jim Liu 宝玉	12e207dc3f	chore: release v1.80.0	2026-03-24 19:27:57 -05:00
Jim Liu 宝玉	00e74ab071	feat(baoyu-image-gen): improve Azure OpenAI provider with flexible endpoint parsing and deployment resolution	2026-03-24 19:19:49 -05:00
优弧	1653b8544b	feat(baoyu-image-gen): add Azure OpenAI as independent image generation provider (#111 ) Azure OpenAI differs from standard OpenAI in two ways: 1. Auth via api-key header instead of Authorization: Bearer 2. URL requires ?api-version query param with deployment path Changes: - New file: scripts/providers/azure.ts (generations + edits, reuses openai utilities) - types.ts: add "azure" to Provider type and default_model - main.ts: register azure across rate limits, CLI args, auto-detection, provider loading, model resolution, help text, ref validation, EXTEND.md parsing, and batch logging Env vars: AZURE_OPENAI_API_KEY, AZURE_OPENAI_BASE_URL (required), AZURE_API_VERSION, AZURE_OPENAI_IMAGE_MODEL (optional) Co-authored-by: CatFly <zw.catfly@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-24 19:04:34 -05:00
Jim Liu 宝玉	dad8f3a800	chore: release v1.79.2	2026-03-23 22:39:14 -05:00
Jim Liu 宝玉	35298d7c9d	fix(baoyu-post-to-weibo): add no-theme rule for article markdown-to-HTML conversion	2026-03-23 22:38:30 -05:00
Jim Liu 宝玉	f22374ab62	fix(baoyu-cover-image): simplify reference image handling based on model capability	2026-03-23 22:38:27 -05:00
Jim Liu 宝玉	d4e80b1bc3	Fix Node-compatible parser tests (#107 ) * Fix Node-compatible parser tests * Add parser test dependencies to root test env	2026-03-23 15:30:42 -05:00
Jim Liu 宝玉	a5761dc71a	chore: release v1.79.1	2026-03-23 12:02:44 -05:00
Jim Liu 宝玉	a5189dff37	fix(baoyu-xhs-images): remove opacity from watermark prompt and fix CJK spacing	2026-03-23 12:01:03 -05:00
Jim Liu 宝玉	39fe872bf3	fix(baoyu-comic): fix Doraemon naming spacing and remove opacity from watermark prompt	2026-03-23 12:01:00 -05:00
Jim Liu 宝玉	52813504f8	fix(baoyu-article-illustrator): remove opacity parameter from watermark prompt	2026-03-23 12:00:53 -05:00
Jim Liu 宝玉	a4d4108cd1	docs(project): update documentation to reflect single-plugin architecture	2026-03-23 12:00:38 -05:00
Yizhou Qian 钱亦舟	d7e763f1f5	fix: consolidate to single plugin to prevent duplicate skill registration (#106 ) Merge the three plugins (content-skills, ai-generation-skills, utility-skills) into one plugin entry. Since all three shared the same source ("./"), Claude Code cached every skill three times. A single plugin with one source keeps the flat skills/ layout while ensuring each skill is registered exactly once.	2026-03-23 09:46:50 -05:00
Jim Liu 宝玉	097c09c59b	chore: release v1.79.0	2026-03-22 15:42:33 -05:00
Jim Liu 宝玉	e4cd8bfefc	feat(baoyu-post-to-wechat): improve credential loading with multi-source resolution and diagnostics	2026-03-22 15:42:08 -05:00
Jim Liu 宝玉	3dc5f2e06f	chore: release v1.78.0	2026-03-22 15:19:29 -05:00
Jim Liu 宝玉	e5d6c8ec68	feat(baoyu-url-to-markdown): add URL-specific parser layer for X/Twitter and archive.ph - New parsers/ module with pluggable rule system for site-specific HTML extraction - X status parser: extract tweet text, media, quotes, author from data-testid elements - X article parser: extract long-form article content with inline media - archive.ph parser: restore original URL and prefer #CONTENT container - Improved slug generation with stop words and content-aware slugs - Output path uses subdirectory structure (domain/slug/slug.md) - Fix: preserve anchor elements containing media in legacy converter - Fix: smarter title deduplication in markdown document builder	2026-03-22 15:18:46 -05:00
Jim Liu 宝玉	6a4b312146	chore: release v1.77.0	2026-03-22 15:17:12 -05:00
Jim Liu 宝玉	2d6fe533eb	Merge pull request #105 from jzOcb/feat/chapter-end-times feat(youtube-transcript): add end times to chapter data	2026-03-22 15:13:44 -05:00
jzocb	f53af25e65	fix: address review feedback on chapter end times - Guard last chapter end against duration=0: use Math.max(duration, ch.start) - Remove unnecessary 'as any' cast in backfill - Check all chapters for missing end (not just first) via .some() - Skip backfill when needsFetch is true (about to refetch anyway) - Wrap backfill writeFileSync in try/catch (best-effort persistence)	2026-03-22 16:07:05 -04:00
jzocb	c7e32b4590	fix: backfill chapter end times for cached videos Videos cached before the chapter end-time change would silently lack the 'end' field when loaded from cache. This adds a migration that detects missing 'end' fields on cache hit, computes them from adjacent chapters, and persists the updated meta.json. This ensures consistent output regardless of whether the data was freshly fetched or loaded from cache.	2026-03-22 15:58:13 -04:00
jzocb	8d973f2bc5	feat(youtube-transcript): add end times to chapter data Add 'end' field to Chapter interface and parseChapters output. Each chapter's end is derived from the next chapter's start time, with the last chapter ending at the video's total duration. This makes chapter data complete and ready for downstream consumers (e.g. video clipping with ffmpeg) without requiring them to compute end times from adjacent chapters. Before: { title: 'Overview', start: 0 } After: { title: 'Overview', start: 0, end: 21 }	2026-03-22 15:52:30 -04:00
Jim Liu 宝玉	ba20cf89f2	fix(sync-clawhub): skip failed skills instead of aborting	2026-03-21 23:25:45 -05:00
Jim Liu 宝玉	1827be9234	chore: release v1.76.1	2026-03-21 23:17:04 -05:00
Jim Liu 宝玉	93c98dfc3c	docs(baoyu-youtube-transcript): fix zsh glob issue for YouTube URLs	2026-03-21 23:16:51 -05:00