chore: release v1.63.0

feat(baoyu-url-to-markdown): add defuddle.md API fallback, YouTube transcripts, and modular converter architecture
docs: fix Claude Code marketplace repo casing
2026-07-12 05:51:44 +08:00 · 2026-03-13 00:24:32 -05:00 · 2026-03-13 00:22:03 -05:00 · 2026-03-12 12:46:25 -05:00 · 2026-03-12 01:51:18 -05:00 · 2026-03-12 01:50:36 -05:00
24 changed files with 1832 additions and 994 deletions
@@ -6,14 +6,14 @@
  },
  "metadata": {
    "description": "Skills shared by Baoyu for improving daily work efficiency",
-    "version": "1.60.0"
+    "version": "1.63.0"
  },
  "plugins": [
    {
      "name": "content-skills",
      "description": "Content generation and publishing skills",
      "source": "./",
-      "strict": false,
+      "strict": true,
      "skills": [
        "./skills/baoyu-xhs-images",
        "./skills/baoyu-post-to-x",
@@ -30,7 +30,7 @@
      "name": "ai-generation-skills",
      "description": "AI-powered generation backends",
      "source": "./",
-      "strict": false,
+      "strict": true,
      "skills": [
        "./skills/baoyu-danger-gemini-web",
        "./skills/baoyu-image-gen"
@@ -40,7 +40,7 @@
      "name": "utility-skills",
      "description": "Utility tools for content processing",
      "source": "./",
-      "strict": false,
+      "strict": true,
      "skills": [
        "./skills/baoyu-danger-x-to-markdown",
        "./skills/baoyu-compress-image",
@@ -2,6 +2,40 @@

 English | [中文](./CHANGELOG.zh.md)

+## 1.63.0 - 2026-03-13
+
+### Features
+- `baoyu-url-to-markdown`: add hosted `defuddle.md` API fallback when local browser capture fails
+- `baoyu-url-to-markdown`: extract YouTube transcript/caption text into markdown output
+- `baoyu-url-to-markdown`: materialize shadow DOM content for better web-component page conversion
+- `baoyu-url-to-markdown`: include language hint in markdown front matter when available
+
+### Refactor
+- `baoyu-url-to-markdown`: split monolithic converter into defuddle, legacy, and shared modules
+
+### Documentation
+- Fix Claude Code marketplace repo casing in READMEs
+
+## 1.62.0 - 2026-03-12
+
+### Features
+- `baoyu-infographic`: support flexible aspect ratios with custom W:H values (e.g., 3:4, 4:3, 2.35:1) in addition to named presets
+
+### Fixes
+- Set strict mode on plugins to prevent duplicated slash commands
+
+### Documentation
+- `baoyu-post-to-wechat`: replace credential-like placeholders
+
+## 1.61.0 - 2026-03-11
+
+### Features
+- `baoyu-post-to-wechat`: add multi-account support with `--account` CLI arg, EXTEND.md accounts block, isolated Chrome profiles, and credential resolution chain
+
+### Fixes
+- Exclude `out/dist/build` dirs and `bun.lockb` from skill release files
+- Use proper MIME types in skill publish to fix ClawhHub rejection
+
 ## 1.60.0 - 2026-03-11

 ### Features
@@ -2,6 +2,40 @@

 [English](./CHANGELOG.md) | 中文

+## 1.63.0 - 2026-03-13
+
+### 新功能
+- `baoyu-url-to-markdown`：本地浏览器抓取失败时自动回退到 `defuddle.md` 托管 API
+- `baoyu-url-to-markdown`：将 YouTube 字幕/文字记录提取到 Markdown 输出中
+- `baoyu-url-to-markdown`：转换前展开 Shadow DOM 内容，提升 Web Component 页面的转换质量
+- `baoyu-url-to-markdown`：Markdown front matter 中包含语言标识（如有）
+
+### 重构
+- `baoyu-url-to-markdown`：将单体转换器拆分为 defuddle、legacy 和 shared 三个模块
+
+### 文档
+- 修复 README 中 Claude Code marketplace 仓库名大小写
+
+## 1.62.0 - 2026-03-12
+
+### 新功能
+- `baoyu-infographic`：支持灵活宽高比，可使用自定义 W:H 值（如 3:4、4:3、2.35:1），同时保留预设名称
+
+### 修复
+- 设置插件严格模式，防止重复注册斜杠命令
+
+### 文档
+- `baoyu-post-to-wechat`：替换类似凭证的占位符
+
+## 1.61.0 - 2026-03-11
+
+### 新功能
+- `baoyu-post-to-wechat`：新增多账号支持，通过 `--account` 参数选择账号，EXTEND.md 支持 accounts 配置块，每个账号独立 Chrome 配置目录和凭证解析链
+
+### 修复
+- 排除 `out/dist/build` 目录和 `bun.lockb` 文件，避免打包到技能发布文件中
+- 修复技能发布时 MIME 类型不正确导致 ClawhHub 拒绝的问题
+
 ## 1.60.0 - 2026-03-11

 ### 新功能
@@ -1,6 +1,6 @@
 # CLAUDE.md

-Claude Code marketplace plugin providing AI-powered content generation skills. Version: **1.60.0**.
+Claude Code marketplace plugin providing AI-powered content generation skills. Version: **1.63.0**.

 ## Architecture

@@ -43,7 +43,7 @@ Publishing to ClawHub releases the published skill under `MIT-0`, per ClawHub's
 Run the following command in Claude Code:

 ```bash
-/plugin marketplace add jimliu/baoyu-skills
+/plugin marketplace add JimLiu/baoyu-skills
 ```

 ### Install Skills
@@ -169,8 +169,9 @@ Generate professional infographics with 20 layout types and 17 visual styles. An
 # Specify both
 /baoyu-infographic path/to/content.md --layout funnel --style corporate-memphis

-# With aspect ratio
+# With aspect ratio (named preset or custom W:H)
 /baoyu-infographic path/to/content.md --aspect portrait
+/baoyu-infographic path/to/content.md --aspect 3:4
 ```

 **Options**:
@@ -178,7 +179,7 @@ Generate professional infographics with 20 layout types and 17 visual styles. An
 |--------|-------------|
 | `--layout <name>` | Information layout (20 options) |
 | `--style <name>` | Visual style (17 options, default: craft-handmade) |
-| `--aspect <ratio>` | landscape (16:9), portrait (9:16), square (1:1) |
+| `--aspect <ratio>` | Named: landscape (16:9), portrait (9:16), square (1:1). Custom: any W:H ratio (e.g., 3:4, 4:3, 2.35:1) |
 | `--lang <code>` | Output language (en, zh, ja, etc.) |

 **Layouts** (information structure):
@@ -581,6 +582,47 @@ To obtain credentials:

 **Browser Method** (no API setup needed): Requires Google Chrome. First run opens browser for QR code login (session preserved).

+**Multi-Account Support**: Manage multiple WeChat Official Accounts via `EXTEND.md`:
+
+```bash
+mkdir -p .baoyu-skills/baoyu-post-to-wechat
+```
+
+Create `.baoyu-skills/baoyu-post-to-wechat/EXTEND.md`:
+
+```yaml
+# Global settings (shared across all accounts)
+default_theme: default
+default_color: blue
+
+# Account list
+accounts:
+  - name: My Tech Blog
+    alias: tech-blog
+    default: false
+    default_publish_method: api
+    default_author: Author Name
+    need_open_comment: 1
+    only_fans_can_comment: 0
+    app_id: your_wechat_app_id
+    app_secret: your_wechat_app_secret
+  - name: AI Newsletter
+    alias: ai-news
+    default_publish_method: browser
+    default_author: AI Newsletter
+    need_open_comment: 1
+    only_fans_can_comment: 0
+```
+
+| Accounts configured | Behavior |
+|---------------------|----------|
+| No `accounts` block | Single-account mode (backward compatible) |
+| 1 account | Auto-select, no prompt |
+| 2+ accounts | Prompt to select, or use `--account <alias>` |
+| 1 account has `default: true` | Pre-selected as default |
+
+Each account gets an isolated Chrome profile for independent login sessions (browser method). API credentials can be set inline in EXTEND.md or via `.env` with alias-prefixed keys (e.g., `WECHAT_TECH_BLOG_APP_ID`).
+
 #### baoyu-post-to-weibo

 Post content to Weibo (微博). Supports regular posts with text, images, and videos, and headline articles (头条文章) with Markdown input. Uses real Chrome with CDP to bypass anti-automation.
@@ -43,7 +43,7 @@ clawhub install baoyu-markdown-to-html
 在 Claude Code 中运行：

 ```bash
-/plugin marketplace add jimliu/baoyu-skills
+/plugin marketplace add JimLiu/baoyu-skills
 ```

 ### 安装技能
@@ -169,8 +169,9 @@ clawhub install baoyu-markdown-to-html
 # 同时指定布局和风格
 /baoyu-infographic path/to/content.md --layout funnel --style corporate-memphis

-# 指定比例
+# 指定比例（预设名称或自定义 W:H）
 /baoyu-infographic path/to/content.md --aspect portrait
+/baoyu-infographic path/to/content.md --aspect 3:4
 ```

 **选项**：
@@ -178,7 +179,7 @@ clawhub install baoyu-markdown-to-html
 |------|------|
 | `--layout <name>` | 信息布局（20 种选项） |
 | `--style <name>` | 视觉风格（17 种选项，默认：craft-handmade） |
-| `--aspect <ratio>` | landscape (16:9)、portrait (9:16)、square (1:1) |
+| `--aspect <ratio>` | 预设：landscape (16:9)、portrait (9:16)、square (1:1)。自定义：任意 W:H 比例（如 3:4、4:3、2.35:1） |
 | `--lang <code>` | 输出语言（en、zh、ja 等） |

 **布局**（信息结构）：
@@ -581,6 +582,47 @@ WECHAT_APP_SECRET=你的AppSecret

 **浏览器方式**（无需 API 配置）：需已安装 Google Chrome，首次运行需扫码登录（登录状态会保存）

+**多账号支持**：通过 `EXTEND.md` 管理多个微信公众号：
+
+```bash
+mkdir -p .baoyu-skills/baoyu-post-to-wechat
+```
+
+创建 `.baoyu-skills/baoyu-post-to-wechat/EXTEND.md`：
+
+```yaml
+# 全局设置（所有账号共享）
+default_theme: default
+default_color: blue
+
+# 账号列表
+accounts:
+  - name: 宝玉的技术分享
+    alias: baoyu
+    default: false
+    default_publish_method: api
+    default_author: 宝玉
+    need_open_comment: 1
+    only_fans_can_comment: 0
+    app_id: 你的微信AppID
+    app_secret: 你的微信AppSecret
+  - name: AI 工具集
+    alias: ai-tools
+    default_publish_method: browser
+    default_author: AI 工具集
+    need_open_comment: 1
+    only_fans_can_comment: 0
+```
+
+| 账号配置情况 | 行为 |
+|-------------|------|
+| 无 `accounts` 块 | 单账号模式（向后兼容） |
+| 1 个账号 | 自动选择，无需提示 |
+| 2+ 个账号 | 提示选择，或使用 `--account <别名>` |
+| 某账号设置 `default: true` | 预选为默认账号 |
+
+每个账号拥有独立的 Chrome 配置目录，保证浏览器方式下的登录会话互不干扰。API 凭证可在 EXTEND.md 中直接配置，也可通过 `.env` 文件使用别名前缀的环境变量（如 `WECHAT_BAOYU_APP_ID`）。
+
 #### baoyu-post-to-weibo

 发布内容到微博。支持文字、图片、视频发布和头条文章（长篇 Markdown）。使用真实 Chrome + CDP 绕过反自动化检测。
@@ -8,8 +8,8 @@ const PACKAGE_DEPENDENCY_SECTIONS = [
  "devDependencies",
 ];

-const SKIPPED_DIRS = new Set([".git", ".clawhub", ".clawdhub", "node_modules"]);
-const SKIPPED_FILES = new Set([".DS_Store"]);
+const SKIPPED_DIRS = new Set([".git", ".clawhub", ".clawdhub", "node_modules", "out", "dist", "build"]);
+const SKIPPED_FILES = new Set([".DS_Store", "bun.lockb"]);

 export async function listReleaseFiles(root) {
  const resolvedRoot = path.resolve(root);
@@ -220,7 +220,7 @@ async function publishSkill({ registry, token, skill, files, version, changelog,
  );

  for (const file of files) {
-    form.append("files", new Blob([file.bytes], { type: "application/octet-stream" }), file.relPath);
+    form.append("files", new Blob([file.bytes], { type: mimeType(file.relPath) }), file.relPath);
  }

  const response = await fetch(`${registry}/api/v1/skills`, {
@@ -281,6 +281,26 @@ function titleCase(value) {
    .replace(/\b\w/g, (char) => char.toUpperCase());
 }

+const MIME_MAP = {
+  ".md": "text/markdown",
+  ".ts": "text/plain",
+  ".js": "text/javascript",
+  ".mjs": "text/javascript",
+  ".json": "application/json",
+  ".yml": "text/yaml",
+  ".yaml": "text/yaml",
+  ".txt": "text/plain",
+  ".html": "text/html",
+  ".css": "text/css",
+  ".xml": "text/xml",
+  ".svg": "image/svg+xml",
+};
+
+function mimeType(relPath) {
+  const ext = path.extname(relPath).toLowerCase();
+  return MIME_MAP[ext] || "text/plain";
+}
+
 function parseBoolean(value) {
  return ["1", "true", "yes", "on"].includes(String(value).trim().toLowerCase());
 }
@@ -17,6 +17,7 @@ Two dimensions: **layout** (information structure) × **style** (visual aestheti
 /baoyu-infographic path/to/content.md
 /baoyu-infographic path/to/content.md --layout hierarchical-layers --style technical-schematic
 /baoyu-infographic path/to/content.md --aspect portrait --lang zh
+/baoyu-infographic path/to/content.md --aspect 3:4
 /baoyu-infographic  # then paste content
 ```

@@ -26,7 +27,7 @@ Two dimensions: **layout** (information structure) × **style** (visual aestheti
 |--------|--------|
 | `--layout` | 21 options (see Layout Gallery), default: bento-grid |
 | `--style` | 20 options (see Style Gallery), default: craft-handmade |
-| `--aspect` | landscape (16:9), portrait (9:16), square (1:1) |
+| `--aspect` | Named: landscape (16:9), portrait (9:16), square (1:1). Custom: any W:H ratio (e.g., 3:4, 4:3, 2.35:1) |
 | `--lang` | en, zh, ja, etc. |

 ## Layout Gallery
@@ -220,7 +221,7 @@ Use **single AskUserQuestion call** with multiple questions to confirm all optio
 | Question | When | Options |
 |----------|------|---------|
 | **Combination** | Always | 3+ layout×style combos with rationale |
-| **Aspect** | Always | landscape (16:9), portrait (9:16), square (1:1) |
+| **Aspect** | Always | Named presets (landscape/portrait/square) or custom W:H ratio (e.g., 3:4, 4:3, 2.35:1) |
 | **Language** | Only if source ≠ user language | Language for text content |

 **Important**: Do NOT split into separate AskUserQuestion calls. Combine all applicable questions into one call.
@@ -236,6 +237,10 @@ Combine:
 4. Structured content from Step 2
 5. All text in confirmed language

+**Aspect ratio resolution** for `{{ASPECT_RATIO}}`:
+- Named presets → ratio string: landscape→`16:9`, portrait→`9:16`, square→`1:1`
+- Custom W:H ratios → use as-is (e.g., `3:4`, `4:3`, `2.35:1`)
+
 ### Step 6: Generate Image

 1. Select available image generation skill (ask user if multiple)
@@ -95,9 +95,115 @@ chrome_profile_path: /path/to/chrome/profile
 **Value priority**:
 1. CLI arguments
 2. Frontmatter
-3. EXTEND.md
+3. EXTEND.md (account-level → global-level)
 4. Skill defaults

+## Multi-Account Support
+
+EXTEND.md supports managing multiple WeChat Official Accounts. When `accounts:` block is present, each account can have its own credentials, Chrome profile, and default settings.
+
+**Compatibility rules**:
+
+| Condition | Mode | Behavior |
+|-----------|------|----------|
+| No `accounts` block | Single-account | Current behavior, unchanged |
+| `accounts` with 1 entry | Single-account | Auto-select, no prompt |
+| `accounts` with 2+ entries | Multi-account | Prompt to select before publishing |
+| `accounts` with `default: true` | Multi-account | Pre-select default, user can switch |
+
+**Multi-account EXTEND.md example**:
+
+```md
+default_theme: default
+default_color: blue
+
+accounts:
+  - name: 宝玉的技术分享
+    alias: baoyu
+    default: true
+    default_publish_method: api
+    default_author: 宝玉
+    need_open_comment: 1
+    only_fans_can_comment: 0
+    app_id: your_wechat_app_id
+    app_secret: your_wechat_app_secret
+  - name: AI工具集
+    alias: ai-tools
+    default_publish_method: browser
+    default_author: AI工具集
+    need_open_comment: 1
+    only_fans_can_comment: 0
+```
+
+**Per-account keys** (can be set per-account or globally as fallback):
+`default_publish_method`, `default_author`, `need_open_comment`, `only_fans_can_comment`, `app_id`, `app_secret`, `chrome_profile_path`
+
+**Global-only keys** (always shared across accounts):
+`default_theme`, `default_color`
+
+### Account Selection (Step 0.5)
+
+Insert between Step 0 and Step 1 in the Article Posting Workflow:
+
+```
+if no accounts block:
+    → single-account mode (current behavior)
+elif accounts.length == 1:
+    → auto-select the only account
+elif --account <alias> CLI arg:
+    → select matching account
+elif one account has default: true:
+    → pre-select, show: "Using account: <name> (--account to switch)"
+else:
+    → prompt user:
+      "Multiple WeChat accounts configured:
+       1) <name1> (<alias1>)
+       2) <name2> (<alias2>)
+       Select account [1-N]:"
+```
+
+### Credential Resolution (API Method)
+
+For a selected account with alias `{alias}`:
+
+1. `app_id` / `app_secret` inline in EXTEND.md account block
+2. Env var `WECHAT_{ALIAS}_APP_ID` / `WECHAT_{ALIAS}_APP_SECRET` (alias uppercased, hyphens → underscores)
+3. `.baoyu-skills/.env` with prefixed key `WECHAT_{ALIAS}_APP_ID`
+4. `~/.baoyu-skills/.env` with prefixed key
+5. Fallback to unprefixed `WECHAT_APP_ID` / `WECHAT_APP_SECRET`
+
+**.env multi-account example**:
+
+```bash
+# Account: baoyu
+WECHAT_BAOYU_APP_ID=your_wechat_app_id
+WECHAT_BAOYU_APP_SECRET=your_wechat_app_secret
+
+# Account: ai-tools
+WECHAT_AI_TOOLS_APP_ID=your_ai_tools_wechat_app_id
+WECHAT_AI_TOOLS_APP_SECRET=your_ai_tools_wechat_app_secret
+```
+
+### Chrome Profile (Browser Method)
+
+Each account uses an isolated Chrome profile for independent login sessions:
+
+| Source | Path |
+|--------|------|
+| Account `chrome_profile_path` in EXTEND.md | Use as-is |
+| Auto-generated from alias | `{shared_profile_parent}/wechat-{alias}/` |
+| Single-account fallback | Shared default profile (current behavior) |
+
+### CLI `--account` Argument
+
+All publishing scripts accept `--account <alias>`:
+
+```bash
+${BUN_X} {baseDir}/scripts/wechat-api.ts <file> --theme default --account ai-tools
+${BUN_X} {baseDir}/scripts/wechat-article.ts --markdown <file> --theme default --account baoyu
+${BUN_X} {baseDir}/scripts/wechat-browser.ts --markdown <file> --images ./photos/ --account baoyu
+```
+
 ## Pre-flight Check (Optional)

 Before first use, suggest running the environment check. User can skip if they prefer.
@@ -139,6 +245,7 @@ Copy this checklist and check off items as you complete them:
 ```
 Publishing Progress:
 - [ ] Step 0: Load preferences (EXTEND.md)
+- [ ] Step 0.5: Resolve account (multi-account only)
 - [ ] Step 1: Determine input type
 - [ ] Step 2: Select method and configure credentials
 - [ ] Step 3: Resolve theme/color and validate metadata
@@ -152,6 +152,8 @@ options:

 ## EXTEND.md Template

+### Single Account (Default)
+
 ```md
 default_theme: [default/grace/simple/modern]
 default_color: [preset name, hex, or empty for theme default]
@@ -162,6 +164,40 @@ only_fans_can_comment: [1/0]
 chrome_profile_path:
 ```

+### Multi-Account
+
+```md
+default_theme: [default/grace/simple/modern]
+default_color: [preset name, hex, or empty for theme default]
+
+accounts:
+  - name: [display name]
+    alias: [short key, e.g. "baoyu"]
+    default: true
+    default_publish_method: [api/browser]
+    default_author: [author name]
+    need_open_comment: [1/0]
+    only_fans_can_comment: [1/0]
+    app_id: [WeChat App ID, optional]
+    app_secret: [WeChat App Secret, optional]
+  - name: [second account name]
+    alias: [short key, e.g. "ai-tools"]
+    default_publish_method: [api/browser]
+    default_author: [author name]
+    need_open_comment: [1/0]
+    only_fans_can_comment: [1/0]
+```
+
+## Adding More Accounts Later
+
+After initial setup, users can add accounts by editing EXTEND.md:
+
+1. Add an `accounts:` block with list items
+2. Move per-account settings (author, publish method, comments) into each account entry
+3. Keep global settings (theme, color) at the top level
+4. Each account needs a unique `alias` (used for CLI `--account` arg and Chrome profile naming)
+5. Set `default: true` on the primary account
+
 ## Modifying Preferences Later

 Users can edit EXTEND.md directly or delete it to trigger setup again.
@@ -1,4 +1,5 @@
 import { execSync, type ChildProcess } from 'node:child_process';
+import path from 'node:path';
 import process from 'node:process';

 import {
@@ -80,6 +81,11 @@ export function getDefaultProfileDir(): string {
  });
 }

+export function getAccountProfileDir(alias: string): string {
+  const base = getDefaultProfileDir();
+  return path.join(path.dirname(base), `wechat-${alias}`);
+}
+
 export interface ChromeSession {
  cdp: CdpConnection;
  sessionId: string;
@@ -1,13 +1,8 @@
 import fs from "node:fs";
 import path from "node:path";
-import os from "node:os";
 import { spawnSync } from "node:child_process";
 import { fileURLToPath } from "node:url";
-
-interface WechatConfig {
-  appId: string;
-  appSecret: string;
-}
+import { loadWechatExtendConfig, resolveAccount, loadCredentials } from "./wechat-extend-config.ts";

 interface AccessTokenResponse {
  access_token?: string;
@@ -38,53 +33,14 @@ interface ArticleOptions {
  thumbMediaId: string;
  articleType: ArticleType;
  imageMediaIds?: string[];
+  needOpenComment?: number;
+  onlyFansCanComment?: number;
 }

 const TOKEN_URL = "https://api.weixin.qq.com/cgi-bin/token";
 const UPLOAD_URL = "https://api.weixin.qq.com/cgi-bin/material/add_material";
 const DRAFT_URL = "https://api.weixin.qq.com/cgi-bin/draft/add";

-function loadEnvFile(envPath: string): Record<string, string> {
-  const env: Record<string, string> = {};
-  if (!fs.existsSync(envPath)) return env;
-
-  const content = fs.readFileSync(envPath, "utf-8");
-  for (const line of content.split("\n")) {
-    const trimmed = line.trim();
-    if (!trimmed || trimmed.startsWith("#")) continue;
-    const eqIdx = trimmed.indexOf("=");
-    if (eqIdx > 0) {
-      const key = trimmed.slice(0, eqIdx).trim();
-      let value = trimmed.slice(eqIdx + 1).trim();
-      if ((value.startsWith('"') && value.endsWith('"')) ||
-          (value.startsWith("'") && value.endsWith("'"))) {
-        value = value.slice(1, -1);
-      }
-      env[key] = value;
-    }
-  }
-  return env;
-}
-
-function loadConfig(): WechatConfig {
-  const cwdEnvPath = path.join(process.cwd(), ".baoyu-skills", ".env");
-  const homeEnvPath = path.join(os.homedir(), ".baoyu-skills", ".env");
-
-  const cwdEnv = loadEnvFile(cwdEnvPath);
-  const homeEnv = loadEnvFile(homeEnvPath);
-
-  const appId = process.env.WECHAT_APP_ID || cwdEnv.WECHAT_APP_ID || homeEnv.WECHAT_APP_ID;
-  const appSecret = process.env.WECHAT_APP_SECRET || cwdEnv.WECHAT_APP_SECRET || homeEnv.WECHAT_APP_SECRET;
-
-  if (!appId || !appSecret) {
-    throw new Error(
-      "Missing WECHAT_APP_ID or WECHAT_APP_SECRET.\n" +
-      "Set via environment variables or in .baoyu-skills/.env file."
-    );
-  }
-
-  return { appId, appSecret };
-}

 async function fetchAccessToken(appId: string, appSecret: string): Promise<string> {
  const url = `${TOKEN_URL}?grant_type=client_credential&appid=${appId}&secret=${appSecret}`;
@@ -241,6 +197,9 @@ async function publishToDraft(

  let article: Record<string, unknown>;

+  const noc = options.needOpenComment ?? 1;
+  const ofcc = options.onlyFansCanComment ?? 0;
+
  if (options.articleType === "newspic") {
    if (!options.imageMediaIds || options.imageMediaIds.length === 0) {
      throw new Error("newspic requires at least one image");
@@ -249,8 +208,8 @@ async function publishToDraft(
      article_type: "newspic",
      title: options.title,
      content: options.content,
-      need_open_comment: 1,
-      only_fans_can_comment: 0,
+      need_open_comment: noc,
+      only_fans_can_comment: ofcc,
      image_info: {
        image_list: options.imageMediaIds.map(id => ({ image_media_id: id })),
      },
@@ -262,8 +221,8 @@ async function publishToDraft(
      title: options.title,
      content: options.content,
      thumb_media_id: options.thumbMediaId,
-      need_open_comment: 1,
-      only_fans_can_comment: 0,
+      need_open_comment: noc,
+      only_fans_can_comment: ofcc,
    };
    if (options.author) article.author = options.author;
    if (options.digest) article.digest = options.digest;
@@ -368,6 +327,7 @@ Options:
  --theme <name>      Theme name for markdown (default, grace, simple, modern). Default: default
  --color <name|hex>  Primary color (blue, green, vermilion, etc. or hex)
  --cover <path>      Cover image path (local or URL)
+  --account <alias>   Select account by alias (for multi-account setups)
  --no-cite           Disable bottom citations for ordinary external links in markdown mode
  --dry-run           Parse and render only, don't publish
  --help              Show this help
@@ -412,6 +372,7 @@ interface CliArgs {
  theme: string;
  color?: string;
  cover?: string;
+  account?: string;
  citeStatus: boolean;
  dryRun: boolean;
 }
@@ -449,6 +410,8 @@ function parseArgs(argv: string[]): CliArgs {
      args.color = argv[++i];
    } else if (arg === "--cover" && argv[i + 1]) {
      args.cover = argv[++i];
+    } else if (arg === "--account" && argv[i + 1]) {
+      args.account = argv[++i];
    } else if (arg === "--cite") {
      args.citeStatus = true;
    } else if (arg === "--no-cite") {
@@ -550,6 +513,12 @@ async function main(): Promise<void> {
  if (digest) console.error(`[wechat-api] Digest: ${digest.slice(0, 50)}...`);
  console.error(`[wechat-api] Type: ${args.articleType}`);

+  const extConfig = loadWechatExtendConfig();
+  const resolved = resolveAccount(extConfig, args.account);
+  if (resolved.name) console.error(`[wechat-api] Account: ${resolved.name} (${resolved.alias})`);
+
+  if (!author && resolved.default_author) author = resolved.default_author;
+
  if (args.dryRun) {
    console.log(JSON.stringify({
      articleType: args.articleType,
@@ -558,13 +527,14 @@ async function main(): Promise<void> {
      digest: digest || undefined,
      htmlPath,
      contentLength: htmlContent.length,
+      account: resolved.alias || undefined,
    }, null, 2));
    return;
  }

-  const config = loadConfig();
+  const creds = loadCredentials(resolved);
  console.error("[wechat-api] Fetching access token...");
-  const accessToken = await fetchAccessToken(config.appId, config.appSecret);
+  const accessToken = await fetchAccessToken(creds.appId, creds.appSecret);

  console.error("[wechat-api] Uploading images...");
  const { html: processedHtml, firstMediaId, allMediaIds } = await uploadImagesInHtml(
@@ -617,6 +587,8 @@ async function main(): Promise<void> {
    thumbMediaId,
    articleType: args.articleType,
    imageMediaIds: args.articleType === "newspic" ? allMediaIds : undefined,
+    needOpenComment: resolved.need_open_comment,
+    onlyFansCanComment: resolved.only_fans_can_comment,
  }, accessToken);

  console.log(JSON.stringify({
@@ -3,7 +3,8 @@ import path from 'node:path';
 import { spawnSync } from 'node:child_process';
 import process from 'node:process';
 import { fileURLToPath } from 'node:url';
-import { launchChrome, tryConnectExisting, findExistingChromeDebugPort, getPageSession, waitForNewTab, clickElement, typeText, evaluate, sleep, type ChromeSession, type CdpConnection } from './cdp.ts';
+import { launchChrome, tryConnectExisting, findExistingChromeDebugPort, getPageSession, waitForNewTab, clickElement, typeText, evaluate, sleep, getAccountProfileDir, type ChromeSession, type CdpConnection } from './cdp.ts';
+import { loadWechatExtendConfig, resolveAccount } from './wechat-extend-config.ts';

 const WECHAT_URL = 'https://mp.weixin.qq.com/';

@@ -690,6 +691,7 @@ Options:
  --image <path>     Content image, can repeat (only with --content)
  --submit           Save as draft
  --profile <dir>    Chrome profile directory
+  --account <alias>  Select account by alias (for multi-account setups)
  --cdp-port <port>  Connect to existing Chrome debug port instead of launching new instance

 Examples:
@@ -725,6 +727,7 @@ async function main(): Promise<void> {
  let submit = false;
  let profileDir: string | undefined;
  let cdpPort: number | undefined;
+  let accountAlias: string | undefined;

  for (let i = 0; i < args.length; i++) {
    const arg = args[i]!;
@@ -741,9 +744,20 @@ async function main(): Promise<void> {
    else if (arg === '--image' && args[i + 1]) images.push(args[++i]!);
    else if (arg === '--submit') submit = true;
    else if (arg === '--profile' && args[i + 1]) profileDir = args[++i];
+    else if (arg === '--account' && args[i + 1]) accountAlias = args[++i];
    else if (arg === '--cdp-port' && args[i + 1]) cdpPort = parseInt(args[++i]!, 10);
  }

+  const extConfig = loadWechatExtendConfig();
+  const resolved = resolveAccount(extConfig, accountAlias);
+  if (resolved.name) console.log(`[wechat] Account: ${resolved.name} (${resolved.alias})`);
+
+  if (!author && resolved.default_author) author = resolved.default_author;
+
+  if (!profileDir && resolved.alias) {
+    profileDir = resolved.chrome_profile_path || getAccountProfileDir(resolved.alias);
+  }
+
  if (!markdownFile && !htmlFile && !title) { console.error('Error: --title is required (or use --markdown/--html)'); process.exit(1); }
  if (!markdownFile && !htmlFile && !content) { console.error('Error: --content, --html, or --markdown is required'); process.exit(1); }

@@ -7,9 +7,11 @@ import {
  CdpConnection,
  findChromeExecutable,
  getDefaultProfileDir,
+  getAccountProfileDir,
  launchChrome,
  sleep,
 } from './cdp.ts';
+import { loadWechatExtendConfig, resolveAccount } from './wechat-extend-config.ts';

 const WECHAT_URL = 'https://mp.weixin.qq.com/';

@@ -664,6 +666,7 @@ Options:
  --image <path>     Add image (can be repeated)
  --submit           Save as draft (default: preview only)
  --profile <dir>    Chrome profile directory
+  --account <alias>  Select account by alias (for multi-account setups)
  --help             Show this help

 Examples:
@@ -685,6 +688,7 @@ async function main(): Promise<void> {
  let content: string | undefined;
  let markdownFile: string | undefined;
  let imagesDir: string | undefined;
+  let accountAlias: string | undefined;

  for (let i = 0; i < args.length; i++) {
    const arg = args[i]!;
@@ -702,9 +706,19 @@ async function main(): Promise<void> {
      submit = true;
    } else if (arg === '--profile' && args[i + 1]) {
      profileDir = args[++i];
+    } else if (arg === '--account' && args[i + 1]) {
+      accountAlias = args[++i];
    }
  }

+  const extConfig = loadWechatExtendConfig();
+  const resolved = resolveAccount(extConfig, accountAlias);
+  if (resolved.name) console.log(`[wechat-browser] Account: ${resolved.name} (${resolved.alias})`);
+
+  if (!profileDir && resolved.alias) {
+    profileDir = resolved.chrome_profile_path || getAccountProfileDir(resolved.alias);
+  }
+
  if (!markdownFile && !title) {
    console.error('Error: --title or --markdown is required');
    process.exit(1);
@@ -0,0 +1,245 @@
+import fs from "node:fs";
+import path from "node:path";
+import os from "node:os";
+
+export interface WechatAccount {
+  name: string;
+  alias: string;
+  default?: boolean;
+  default_publish_method?: string;
+  default_author?: string;
+  need_open_comment?: number;
+  only_fans_can_comment?: number;
+  app_id?: string;
+  app_secret?: string;
+  chrome_profile_path?: string;
+}
+
+export interface WechatExtendConfig {
+  default_theme?: string;
+  default_color?: string;
+  default_publish_method?: string;
+  default_author?: string;
+  need_open_comment?: number;
+  only_fans_can_comment?: number;
+  chrome_profile_path?: string;
+  accounts?: WechatAccount[];
+}
+
+export interface ResolvedAccount {
+  name?: string;
+  alias?: string;
+  default_publish_method?: string;
+  default_author?: string;
+  need_open_comment: number;
+  only_fans_can_comment: number;
+  app_id?: string;
+  app_secret?: string;
+  chrome_profile_path?: string;
+}
+
+function stripQuotes(s: string): string {
+  return s.replace(/^['"]|['"]$/g, "");
+}
+
+function toBool01(v: string): number {
+  return v === "1" || v === "true" ? 1 : 0;
+}
+
+function parseWechatExtend(content: string): WechatExtendConfig {
+  const config: WechatExtendConfig = {};
+  const lines = content.split("\n");
+  let inAccounts = false;
+  let current: Record<string, string> | null = null;
+  const rawAccounts: Record<string, string>[] = [];
+
+  for (const raw of lines) {
+    const trimmed = raw.trim();
+    if (!trimmed || trimmed.startsWith("#")) continue;
+
+    if (trimmed === "accounts:") {
+      inAccounts = true;
+      continue;
+    }
+
+    if (inAccounts) {
+      const listMatch = raw.match(/^\s+-\s+(.+)$/);
+      if (listMatch) {
+        if (current) rawAccounts.push(current);
+        current = {};
+        const kv = listMatch[1]!;
+        const ci = kv.indexOf(":");
+        if (ci > 0) {
+          current[kv.slice(0, ci).trim()] = stripQuotes(kv.slice(ci + 1).trim());
+        }
+        continue;
+      }
+
+      if (current && /^\s{2,}/.test(raw) && !trimmed.startsWith("-")) {
+        const ci = trimmed.indexOf(":");
+        if (ci > 0) {
+          current[trimmed.slice(0, ci).trim()] = stripQuotes(trimmed.slice(ci + 1).trim());
+        }
+        continue;
+      }
+
+      if (!/^\s/.test(raw)) {
+        if (current) rawAccounts.push(current);
+        current = null;
+        inAccounts = false;
+      } else {
+        continue;
+      }
+    }
+
+    const ci = trimmed.indexOf(":");
+    if (ci < 0) continue;
+    const key = trimmed.slice(0, ci).trim();
+    const val = stripQuotes(trimmed.slice(ci + 1).trim());
+    if (val === "null" || val === "") continue;
+
+    switch (key) {
+      case "default_theme": config.default_theme = val; break;
+      case "default_color": config.default_color = val; break;
+      case "default_publish_method": config.default_publish_method = val; break;
+      case "default_author": config.default_author = val; break;
+      case "need_open_comment": config.need_open_comment = toBool01(val); break;
+      case "only_fans_can_comment": config.only_fans_can_comment = toBool01(val); break;
+      case "chrome_profile_path": config.chrome_profile_path = val; break;
+    }
+  }
+
+  if (current) rawAccounts.push(current);
+
+  if (rawAccounts.length > 0) {
+    config.accounts = rawAccounts.map(a => ({
+      name: a.name || "",
+      alias: a.alias || "",
+      default: a.default === "true" || a.default === "1",
+      default_publish_method: a.default_publish_method || undefined,
+      default_author: a.default_author || undefined,
+      need_open_comment: a.need_open_comment ? toBool01(a.need_open_comment) : undefined,
+      only_fans_can_comment: a.only_fans_can_comment ? toBool01(a.only_fans_can_comment) : undefined,
+      app_id: a.app_id || undefined,
+      app_secret: a.app_secret || undefined,
+      chrome_profile_path: a.chrome_profile_path || undefined,
+    }));
+  }
+
+  return config;
+}
+
+export function loadWechatExtendConfig(): WechatExtendConfig {
+  const paths = [
+    path.join(process.cwd(), ".baoyu-skills", "baoyu-post-to-wechat", "EXTEND.md"),
+    path.join(
+      process.env.XDG_CONFIG_HOME || path.join(os.homedir(), ".config"),
+      "baoyu-skills", "baoyu-post-to-wechat", "EXTEND.md"
+    ),
+    path.join(os.homedir(), ".baoyu-skills", "baoyu-post-to-wechat", "EXTEND.md"),
+  ];
+  for (const p of paths) {
+    try {
+      const content = fs.readFileSync(p, "utf-8");
+      return parseWechatExtend(content);
+    } catch {
+      continue;
+    }
+  }
+  return {};
+}
+
+function selectAccount(config: WechatExtendConfig, alias?: string): WechatAccount | undefined {
+  if (!config.accounts || config.accounts.length === 0) return undefined;
+  if (alias) return config.accounts.find(a => a.alias === alias);
+  if (config.accounts.length === 1) return config.accounts[0];
+  return config.accounts.find(a => a.default);
+}
+
+export function resolveAccount(config: WechatExtendConfig, alias?: string): ResolvedAccount {
+  const acct = selectAccount(config, alias);
+  return {
+    name: acct?.name,
+    alias: acct?.alias,
+    default_publish_method: acct?.default_publish_method ?? config.default_publish_method,
+    default_author: acct?.default_author ?? config.default_author,
+    need_open_comment: acct?.need_open_comment ?? config.need_open_comment ?? 1,
+    only_fans_can_comment: acct?.only_fans_can_comment ?? config.only_fans_can_comment ?? 0,
+    app_id: acct?.app_id,
+    app_secret: acct?.app_secret,
+    chrome_profile_path: acct?.chrome_profile_path ?? config.chrome_profile_path,
+  };
+}
+
+function loadEnvFile(envPath: string): Record<string, string> {
+  const env: Record<string, string> = {};
+  if (!fs.existsSync(envPath)) return env;
+  const content = fs.readFileSync(envPath, "utf-8");
+  for (const line of content.split("\n")) {
+    const trimmed = line.trim();
+    if (!trimmed || trimmed.startsWith("#")) continue;
+    const eqIdx = trimmed.indexOf("=");
+    if (eqIdx > 0) {
+      const key = trimmed.slice(0, eqIdx).trim();
+      let value = trimmed.slice(eqIdx + 1).trim();
+      if ((value.startsWith('"') && value.endsWith('"')) ||
+          (value.startsWith("'") && value.endsWith("'"))) {
+        value = value.slice(1, -1);
+      }
+      env[key] = value;
+    }
+  }
+  return env;
+}
+
+function aliasToEnvKey(alias: string): string {
+  return alias.toUpperCase().replace(/-/g, "_");
+}
+
+export function loadCredentials(account?: ResolvedAccount): { appId: string; appSecret: string } {
+  if (account?.app_id && account?.app_secret) {
+    return { appId: account.app_id, appSecret: account.app_secret };
+  }
+
+  const cwdEnvPath = path.join(process.cwd(), ".baoyu-skills", ".env");
+  const homeEnvPath = path.join(os.homedir(), ".baoyu-skills", ".env");
+  const cwdEnv = loadEnvFile(cwdEnvPath);
+  const homeEnv = loadEnvFile(homeEnvPath);
+
+  const prefix = account?.alias ? `WECHAT_${aliasToEnvKey(account.alias)}_` : "";
+
+  let appId = "";
+  let appSecret = "";
+
+  if (prefix) {
+    appId = process.env[`${prefix}APP_ID`]
+      || cwdEnv[`${prefix}APP_ID`]
+      || homeEnv[`${prefix}APP_ID`]
+      || "";
+    appSecret = process.env[`${prefix}APP_SECRET`]
+      || cwdEnv[`${prefix}APP_SECRET`]
+      || homeEnv[`${prefix}APP_SECRET`]
+      || "";
+  }
+
+  if (!appId) {
+    appId = process.env.WECHAT_APP_ID || cwdEnv.WECHAT_APP_ID || homeEnv.WECHAT_APP_ID || "";
+  }
+  if (!appSecret) {
+    appSecret = process.env.WECHAT_APP_SECRET || cwdEnv.WECHAT_APP_SECRET || homeEnv.WECHAT_APP_SECRET || "";
+  }
+
+  if (!appId || !appSecret) {
+    const hint = account?.alias ? ` (account: ${account.alias})` : "";
+    throw new Error(
+      `Missing WECHAT_APP_ID or WECHAT_APP_SECRET${hint}.\n` +
+      "Set via EXTEND.md account config, environment variables, or .baoyu-skills/.env file."
+    );
+  }
+
+  return { appId, appSecret };
+}
+
+export function listAccounts(config: WechatExtendConfig): string[] {
+  return (config.accounts || []).map(a => a.alias);
+}
@@ -1,7 +1,7 @@
 ---
 name: baoyu-url-to-markdown
-description: Fetch any URL and convert to markdown using Chrome CDP. Saves the rendered HTML snapshot alongside the markdown, and automatically falls back to the pre-Defuddle HTML-to-Markdown pipeline when Defuddle fails. Supports two modes - auto-capture on page load, or wait for user signal (for pages requiring login). Use when user wants to save a webpage as markdown.
-version: 1.56.1
+description: Fetch any URL and convert to markdown using Chrome CDP. Saves the rendered HTML snapshot alongside the markdown, uses an upgraded Defuddle pipeline with better web-component handling and YouTube transcript extraction, and automatically falls back to the pre-Defuddle HTML-to-Markdown pipeline when needed. If local browser capture fails entirely, it can fall back to the hosted defuddle.md API. Supports two modes - auto-capture on page load, or wait for user signal (for pages requiring login). Use when user wants to save a webpage as markdown.
+version: 1.58.1
 metadata:
  openclaw:
    homepage: https://github.com/JimLiu/baoyu-skills#baoyu-url-to-markdown
@@ -29,7 +29,10 @@ Fetches any URL via Chrome CDP, saves the rendered HTML snapshot, and converts i
 | Script | Purpose |
 |--------|---------|
 | `scripts/main.ts` | CLI entry point for URL fetching |
-| `scripts/html-to-markdown.ts` | Defuddle-first conversion with automatic legacy fallback |
+| `scripts/html-to-markdown.ts` | Markdown conversion entry point and converter selection |
+| `scripts/defuddle-converter.ts` | Defuddle-based conversion |
+| `scripts/legacy-converter.ts` | Pre-Defuddle legacy extraction and markdown conversion |
+| `scripts/markdown-conversion-shared.ts` | Shared metadata parsing and markdown document helpers |

 ## Preferences (EXTEND.md)

@@ -115,7 +118,10 @@ Full reference: [references/config/first-time-setup.md](references/config/first-
 - Two capture modes: auto or wait-for-user
 - Save rendered HTML as a sibling `-captured.html` file
 - Clean markdown output with metadata
- Defuddle-first markdown conversion with automatic fallback to the pre-Defuddle extractor from git history
+- Upgraded Defuddle-first markdown conversion with automatic fallback to the pre-Defuddle extractor from git history
+- Materializes shadow DOM content before conversion so web-component pages survive serialization better
+- YouTube pages can include transcript/caption text in the markdown when YouTube exposes a caption track
+- If local browser capture fails completely, can fall back to `defuddle.md/<url>` and still save markdown
 - Handles login-required pages via wait mode
 - Download images and videos to local directories

@@ -168,7 +174,10 @@ Each run saves two files side by side:
 - Markdown: YAML front matter with `url`, `title`, `description`, `author`, `published`, optional `coverImage`, and `captured_at`, followed by converted markdown content
 - HTML snapshot: `*-captured.html`, containing the rendered page HTML captured from Chrome

+When Defuddle or page metadata provides a language hint, the markdown front matter also includes `language`.
+
 The HTML snapshot is saved before any markdown media localization, so it stays a faithful capture of the page DOM used for conversion.
+If the hosted `defuddle.md` API fallback is used, markdown is still saved, but there is no local `-captured.html` snapshot for that run.

 ## Output Directory

@@ -193,13 +202,16 @@ When `--download-media` is enabled:
 Conversion order:

 1. Try Defuddle first
-2. If Defuddle throws, cannot load, returns obviously incomplete markdown, or captures lower-quality content than the legacy pipeline, automatically fall back to the pre-Defuddle extractor
-3. The fallback path uses the older Readability/selector/Next.js-data based HTML-to-Markdown implementation recovered from git history
+2. For rich pages such as YouTube, prefer Defuddle's extractor-specific output (including transcripts when available) instead of replacing it with the legacy pipeline
+3. If Defuddle throws, cannot load, returns obviously incomplete markdown, or captures lower-quality content than the legacy pipeline, automatically fall back to the pre-Defuddle extractor
+4. If the entire local browser capture flow fails before markdown can be produced, try the hosted `https://defuddle.md/<url>` API and save its markdown output directly
+5. The legacy fallback path uses the older Readability/selector/Next.js-data based HTML-to-Markdown implementation recovered from git history

 CLI output will show:

 - `Converter: defuddle` when Defuddle succeeds
 - `Converter: legacy:...` plus `Fallback used: ...` when fallback was needed
+- `Converter: defuddle-api` when local browser capture failed and the hosted API was used instead

 ## Media Download Workflow

@@ -232,6 +244,18 @@ Based on `download_media` setting in EXTEND.md:

 **Troubleshooting**: Chrome not found → set `URL_CHROME_PATH`. Timeout → increase `--timeout`. Complex pages → try `--wait` mode. If markdown quality is poor, inspect the saved `-captured.html` and check whether the run logged a legacy fallback.

+### YouTube Notes
+
+- The upgraded Defuddle path uses async extractors, so YouTube pages can include transcript text directly in the markdown body.
+- Transcript availability depends on YouTube exposing a caption track. Videos with captions disabled, restricted playback, or blocked regional access may still produce description-only output.
+- If the page needs time to finish loading descriptions, chapters, or player metadata, prefer `--wait` and capture after the watch page is fully hydrated.
+
+### Hosted API Fallback
+
+- The hosted fallback endpoint is `https://defuddle.md/<url>`. In shell form: `curl https://defuddle.md/stephango.com`
+- Use it only when the local Chrome/CDP capture path fails outright. The local path still has higher fidelity because it can save the captured HTML and handle authenticated pages.
+- The hosted API already returns Markdown with YAML frontmatter, so save that response as-is and then apply the normal media-localization step if requested.
+
 ## Extension Support

 Custom configurations via EXTEND.md. See **Preferences** section for paths and supported options.
@@ -6,7 +6,7 @@
      "dependencies": {
        "@mozilla/readability": "^0.6.0",
        "baoyu-chrome-cdp": "file:./vendor/baoyu-chrome-cdp",
-        "defuddle": "^0.10.0",
+        "defuddle": "^0.12.0",
        "jsdom": "^24.1.3",
        "linkedom": "^0.18.12",
        "turndown": "^7.2.2",
@@ -61,7 +61,7 @@

    "decimal.js": ["decimal.js@10.6.0", "", {}, "sha512-YpgQiITW3JXGntzdUmyUR1V812Hn8T1YVXhCu+wO3OpS4eU9l4YdD3qjyiKdV6mvV29zapkMeD390UVEf2lkUg=="],

-    "defuddle": ["defuddle@0.10.0", "", { "dependencies": { "commander": "^12.1.0" }, "optionalDependencies": { "mathml-to-latex": "^1.5.0", "temml": "^0.13.1", "turndown": "^7.2.0" }, "peerDependencies": { "jsdom": "^24.0.0" }, "bin": { "defuddle": "dist/cli.js" } }, "sha512-a43juTtHv6Vs4+sxvahVLM5NxoyDsarO1Ag3UxLORI4Fo/nsNFwzDxuQBvosKVGTIRxCwN/mfnWAzNXmQfieqw=="],
+    "defuddle": ["defuddle@0.12.0", "", { "dependencies": { "commander": "^12.1.0" }, "optionalDependencies": { "mathml-to-latex": "^1.5.0", "temml": "^0.13.1", "turndown": "^7.2.0" }, "peerDependencies": { "jsdom": "^24.0.0" }, "bin": { "defuddle": "dist/cli.js" } }, "sha512-Y/WgyGKBxwxFir+hWNth4nmWDDDb8BzQi3qASS2NWYPXsKU42Ku49/3M5yFYefnRef9prynnmasfnXjk99EWgA=="],

    "delayed-stream": ["delayed-stream@1.0.0", "", {}, "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ=="],

@@ -0,0 +1,58 @@
+import { JSDOM, VirtualConsole } from "jsdom";
+import { Defuddle } from "defuddle/node";
+
+import {
+  type ConversionResult,
+  type PageMetadata,
+  isMarkdownUsable,
+  normalizeMarkdown,
+  pickString,
+} from "./markdown-conversion-shared.js";
+
+export async function tryDefuddleConversion(
+  html: string,
+  url: string,
+  baseMetadata: PageMetadata
+): Promise<{ ok: true; result: ConversionResult } | { ok: false; reason: string }> {
+  try {
+    const virtualConsole = new VirtualConsole();
+    virtualConsole.on("jsdomError", (error: Error & { type?: string }) => {
+      if (error.type === "css parsing" || /Could not parse CSS stylesheet/i.test(error.message)) {
+        return;
+      }
+      console.warn(`[url-to-markdown] jsdom: ${error.message}`);
+    });
+
+    const dom = new JSDOM(html, { url, virtualConsole });
+    const result = await Defuddle(dom, url, { markdown: true });
+    const markdown = normalizeMarkdown(result.content || "");
+
+    if (!isMarkdownUsable(markdown, html)) {
+      return { ok: false, reason: "Defuddle returned empty or incomplete markdown" };
+    }
+
+    return {
+      ok: true,
+      result: {
+        metadata: {
+          ...baseMetadata,
+          title: pickString(result.title, baseMetadata.title) ?? "",
+          description: pickString(result.description, baseMetadata.description) ?? undefined,
+          author: pickString(result.author, baseMetadata.author) ?? undefined,
+          published: pickString(result.published, baseMetadata.published) ?? undefined,
+          coverImage: pickString(result.image, baseMetadata.coverImage) ?? undefined,
+          language: pickString(result.language, baseMetadata.language) ?? undefined,
+        },
+        markdown,
+        rawHtml: html,
+        conversionMethod: "defuddle",
+        variables: result.variables,
+      },
+    };
+  } catch (error) {
+    return {
+      ok: false,
+      reason: error instanceof Error ? error.message : String(error),
+    };
+  }
+}
@@ -0,0 +1,629 @@
+import { Readability } from "@mozilla/readability";
+import TurndownService from "turndown";
+import { gfm } from "turndown-plugin-gfm";
+
+import {
+  type AnyRecord,
+  type ConversionResult,
+  type PageMetadata,
+  GOOD_CONTENT_LENGTH,
+  MIN_CONTENT_LENGTH,
+  extractPublishedTime,
+  extractTextFromHtml,
+  extractTitle,
+  normalizeMarkdown,
+  parseDocument,
+  pickString,
+  sanitizeHtml,
+} from "./markdown-conversion-shared.js";
+
+interface ExtractionCandidate {
+  title: string | null;
+  byline: string | null;
+  excerpt: string | null;
+  published: string | null;
+  html: string | null;
+  textContent: string;
+  method: string;
+}
+
+const CONTENT_SELECTORS = [
+  "article",
+  "main article",
+  "[role='main'] article",
+  "[itemprop='articleBody']",
+  ".article-content",
+  ".article-body",
+  ".post-content",
+  ".entry-content",
+  ".story-body",
+  "main",
+  "[role='main']",
+  "#content",
+  ".content",
+];
+
+const REMOVE_SELECTORS = [
+  "script",
+  "style",
+  "noscript",
+  "template",
+  "iframe",
+  "svg",
+  "path",
+  "nav",
+  "aside",
+  "footer",
+  "header",
+  "form",
+  ".advertisement",
+  ".ads",
+  ".social-share",
+  ".related-articles",
+  ".comments",
+  ".newsletter",
+  ".cookie-banner",
+  ".cookie-consent",
+  "[role='navigation']",
+  "[aria-label*='cookie' i]",
+];
+
+const NEXT_DATA_CONTENT_PATHS = [
+  "props.pageProps.content.body",
+  "props.pageProps.article.body",
+  "props.pageProps.article.content",
+  "props.pageProps.post.body",
+  "props.pageProps.post.content",
+  "props.pageProps.data.body",
+  "props.pageProps.story.body.content",
+];
+
+const LOW_QUALITY_MARKERS = [
+  /Join The Conversation/i,
+  /One Community\. Many Voices/i,
+  /Read our community guidelines/i,
+  /Create a free account to share your thoughts/i,
+  /Become a Forbes Member/i,
+  /Subscribe to trusted journalism/i,
+  /\bComments\b/i,
+];
+
+function generateExcerpt(excerpt: string | null, textContent: string | null): string | null {
+  if (excerpt) return excerpt;
+  if (!textContent) return null;
+  const trimmed = textContent.trim();
+  if (!trimmed) return null;
+  return trimmed.length > 200 ? `${trimmed.slice(0, 200)}...` : trimmed;
+}
+
+function parseJsonLdItem(item: AnyRecord): ExtractionCandidate | null {
+  const type = Array.isArray(item["@type"]) ? item["@type"][0] : item["@type"];
+  if (typeof type !== "string" || !["Article", "NewsArticle", "BlogPosting", "WebPage", "ReportageNewsArticle"].includes(type)) {
+    return null;
+  }
+
+  const rawContent =
+    (typeof item.articleBody === "string" && item.articleBody) ||
+    (typeof item.text === "string" && item.text) ||
+    (typeof item.description === "string" && item.description) ||
+    null;
+
+  if (!rawContent) return null;
+
+  const content = rawContent.trim();
+  const htmlLike = /<\/?[a-z][\s\S]*>/i.test(content);
+  const textContent = htmlLike ? extractTextFromHtml(content) : content;
+
+  if (textContent.length < MIN_CONTENT_LENGTH) return null;
+
+  return {
+    title: pickString(item.headline, item.name),
+    byline: extractAuthorFromJsonLd(item.author),
+    excerpt: pickString(item.description),
+    published: pickString(item.datePublished, item.dateCreated),
+    html: htmlLike ? content : null,
+    textContent,
+    method: "json-ld",
+  };
+}
+
+function extractAuthorFromJsonLd(authorData: unknown): string | null {
+  if (typeof authorData === "string") return authorData;
+  if (!authorData || typeof authorData !== "object") return null;
+
+  if (Array.isArray(authorData)) {
+    const names = authorData
+      .map((author) => extractAuthorFromJsonLd(author))
+      .filter((name): name is string => Boolean(name));
+    return names.length > 0 ? names.join(", ") : null;
+  }
+
+  const author = authorData as AnyRecord;
+  return typeof author.name === "string" ? author.name : null;
+}
+
+function flattenJsonLdItems(data: unknown): AnyRecord[] {
+  if (!data || typeof data !== "object") return [];
+  if (Array.isArray(data)) return data.flatMap(flattenJsonLdItems);
+
+  const item = data as AnyRecord;
+  if (Array.isArray(item["@graph"])) {
+    return (item["@graph"] as unknown[]).flatMap(flattenJsonLdItems);
+  }
+
+  return [item];
+}
+
+function tryJsonLdExtraction(document: Document): ExtractionCandidate | null {
+  const scripts = document.querySelectorAll("script[type='application/ld+json']");
+
+  for (const script of scripts) {
+    try {
+      const data = JSON.parse(script.textContent ?? "");
+      for (const item of flattenJsonLdItems(data)) {
+        const extracted = parseJsonLdItem(item);
+        if (extracted) return extracted;
+      }
+    } catch {
+      // Ignore malformed blocks.
+    }
+  }
+
+  return null;
+}
+
+function getByPath(value: unknown, path: string): unknown {
+  let current = value;
+  for (const part of path.split(".")) {
+    if (!current || typeof current !== "object") return undefined;
+    current = (current as AnyRecord)[part];
+  }
+  return current;
+}
+
+function isContentBlockArray(value: unknown): value is AnyRecord[] {
+  if (!Array.isArray(value) || value.length === 0) return false;
+  return value.slice(0, 5).some((item) => {
+    if (!item || typeof item !== "object") return false;
+    const obj = item as AnyRecord;
+    return "type" in obj || "text" in obj || "textHtml" in obj || "content" in obj;
+  });
+}
+
+function extractTextFromContentBlocks(blocks: AnyRecord[]): string {
+  const parts: string[] = [];
+
+  function pushParagraph(text: string): void {
+    const trimmed = text.trim();
+    if (!trimmed) return;
+    parts.push(trimmed, "\n\n");
+  }
+
+  function walk(node: unknown): void {
+    if (!node || typeof node !== "object") return;
+    const block = node as AnyRecord;
+
+    if (typeof block.text === "string") {
+      pushParagraph(block.text);
+      return;
+    }
+
+    if (typeof block.textHtml === "string") {
+      pushParagraph(extractTextFromHtml(block.textHtml));
+      return;
+    }
+
+    if (Array.isArray(block.items)) {
+      for (const item of block.items) {
+        if (item && typeof item === "object") {
+          const text = pickString((item as AnyRecord).text);
+          if (text) parts.push(`- ${text}\n`);
+        }
+      }
+      parts.push("\n");
+    }
+
+    if (Array.isArray(block.components)) {
+      for (const component of block.components) {
+        walk(component);
+      }
+    }
+
+    if (Array.isArray(block.content)) {
+      for (const child of block.content) {
+        walk(child);
+      }
+    }
+  }
+
+  for (const block of blocks) {
+    walk(block);
+  }
+
+  return parts.join("").replace(/\n{3,}/g, "\n\n").trim();
+}
+
+function tryStringBodyExtraction(
+  content: string,
+  meta: AnyRecord,
+  document: Document,
+  method: string
+): ExtractionCandidate | null {
+  if (!content || content.length < MIN_CONTENT_LENGTH) return null;
+
+  const isHtml = /<\/?[a-z][\s\S]*>/i.test(content);
+  const html = isHtml ? sanitizeHtml(content) : null;
+  const textContent = isHtml ? extractTextFromHtml(html) : content.trim();
+
+  if (textContent.length < MIN_CONTENT_LENGTH) return null;
+
+  return {
+    title: pickString(meta.headline, meta.title, extractTitle(document)),
+    byline: pickString(meta.byline, meta.author),
+    excerpt: pickString(meta.description, meta.excerpt, generateExcerpt(null, textContent)),
+    published: pickString(meta.datePublished, meta.publishedAt, extractPublishedTime(document)),
+    html,
+    textContent,
+    method,
+  };
+}
+
+function tryNextDataExtraction(document: Document): ExtractionCandidate | null {
+  try {
+    const script = document.querySelector("script#__NEXT_DATA__");
+    if (!script?.textContent) return null;
+
+    const data = JSON.parse(script.textContent) as AnyRecord;
+    const pageProps = (getByPath(data, "props.pageProps") ?? {}) as AnyRecord;
+
+    for (const path of NEXT_DATA_CONTENT_PATHS) {
+      const value = getByPath(data, path);
+
+      if (typeof value === "string") {
+        const parentPath = path.split(".").slice(0, -1).join(".");
+        const parent = (getByPath(data, parentPath) ?? {}) as AnyRecord;
+        const meta = {
+          ...pageProps,
+          ...parent,
+          title: parent.title ?? (pageProps.title as string | undefined),
+        };
+
+        const candidate = tryStringBodyExtraction(value, meta, document, "next-data");
+        if (candidate) return candidate;
+      }
+
+      if (isContentBlockArray(value)) {
+        const textContent = extractTextFromContentBlocks(value);
+        if (textContent.length < MIN_CONTENT_LENGTH) continue;
+
+        return {
+          title: pickString(
+            getByPath(data, "props.pageProps.content.headline"),
+            getByPath(data, "props.pageProps.article.headline"),
+            getByPath(data, "props.pageProps.article.title"),
+            getByPath(data, "props.pageProps.post.title"),
+            pageProps.title,
+            extractTitle(document)
+          ),
+          byline: pickString(
+            getByPath(data, "props.pageProps.author.name"),
+            getByPath(data, "props.pageProps.article.author.name")
+          ),
+          excerpt: pickString(
+            getByPath(data, "props.pageProps.content.description"),
+            getByPath(data, "props.pageProps.article.description"),
+            pageProps.description,
+            generateExcerpt(null, textContent)
+          ),
+          published: pickString(
+            getByPath(data, "props.pageProps.content.datePublished"),
+            getByPath(data, "props.pageProps.article.datePublished"),
+            getByPath(data, "props.pageProps.publishedAt"),
+            extractPublishedTime(document)
+          ),
+          html: null,
+          textContent,
+          method: "next-data",
+        };
+      }
+    }
+  } catch {
+    return null;
+  }
+
+  return null;
+}
+
+function buildReadabilityCandidate(
+  article: ReturnType<Readability["parse"]>,
+  document: Document,
+  method: string
+): ExtractionCandidate | null {
+  const textContent = article?.textContent?.trim() ?? "";
+  if (textContent.length < MIN_CONTENT_LENGTH) return null;
+
+  return {
+    title: pickString(article?.title, extractTitle(document)),
+    byline: pickString((article as { byline?: string } | null)?.byline),
+    excerpt: pickString(article?.excerpt, generateExcerpt(null, textContent)),
+    published: pickString((article as { publishedTime?: string } | null)?.publishedTime, extractPublishedTime(document)),
+    html: article?.content ? sanitizeHtml(article.content) : null,
+    textContent,
+    method,
+  };
+}
+
+function tryReadability(document: Document): ExtractionCandidate | null {
+  try {
+    const strictClone = document.cloneNode(true) as Document;
+    const strictResult = buildReadabilityCandidate(
+      new Readability(strictClone).parse(),
+      document,
+      "readability"
+    );
+    if (strictResult) return strictResult;
+
+    const relaxedClone = document.cloneNode(true) as Document;
+    return buildReadabilityCandidate(
+      new Readability(relaxedClone, { charThreshold: 120 }).parse(),
+      document,
+      "readability-relaxed"
+    );
+  } catch {
+    return null;
+  }
+}
+
+function trySelectorExtraction(document: Document): ExtractionCandidate | null {
+  for (const selector of CONTENT_SELECTORS) {
+    const element = document.querySelector(selector);
+    if (!element) continue;
+
+    const clone = element.cloneNode(true) as Element;
+    for (const removeSelector of REMOVE_SELECTORS) {
+      for (const node of clone.querySelectorAll(removeSelector)) {
+        node.remove();
+      }
+    }
+
+    const html = sanitizeHtml(clone.innerHTML);
+    const textContent = extractTextFromHtml(html);
+    if (textContent.length < MIN_CONTENT_LENGTH) continue;
+
+    return {
+      title: extractTitle(document),
+      byline: null,
+      excerpt: generateExcerpt(null, textContent),
+      published: extractPublishedTime(document),
+      html,
+      textContent,
+      method: `selector:${selector}`,
+    };
+  }
+
+  return null;
+}
+
+function tryBodyExtraction(document: Document): ExtractionCandidate | null {
+  const body = document.body;
+  if (!body) return null;
+
+  const clone = body.cloneNode(true) as Element;
+  for (const removeSelector of REMOVE_SELECTORS) {
+    for (const node of clone.querySelectorAll(removeSelector)) {
+      node.remove();
+    }
+  }
+
+  const html = sanitizeHtml(clone.innerHTML);
+  const textContent = extractTextFromHtml(html);
+  if (!textContent) return null;
+
+  return {
+    title: extractTitle(document),
+    byline: null,
+    excerpt: generateExcerpt(null, textContent),
+    published: extractPublishedTime(document),
+    html,
+    textContent,
+    method: "body-fallback",
+  };
+}
+
+function pickBestCandidate(candidates: ExtractionCandidate[]): ExtractionCandidate | null {
+  if (candidates.length === 0) return null;
+
+  const methodOrder = [
+    "readability",
+    "readability-relaxed",
+    "next-data",
+    "json-ld",
+    "selector:",
+    "body-fallback",
+  ];
+
+  function methodRank(method: string): number {
+    const idx = methodOrder.findIndex((entry) =>
+      entry.endsWith(":") ? method.startsWith(entry) : method === entry
+    );
+    return idx === -1 ? methodOrder.length : idx;
+  }
+
+  const ranked = [...candidates].sort((a, b) => {
+    const rankA = methodRank(a.method);
+    const rankB = methodRank(b.method);
+    if (rankA !== rankB) return rankA - rankB;
+    return (b.textContent.length ?? 0) - (a.textContent.length ?? 0);
+  });
+
+  for (const candidate of ranked) {
+    if (candidate.textContent.length >= GOOD_CONTENT_LENGTH) {
+      return candidate;
+    }
+  }
+
+  for (const candidate of ranked) {
+    if (candidate.textContent.length >= MIN_CONTENT_LENGTH) {
+      return candidate;
+    }
+  }
+
+  return ranked[0];
+}
+
+function extractFromHtml(html: string): ExtractionCandidate | null {
+  const document = parseDocument(html);
+
+  const readabilityCandidate = tryReadability(document);
+  const nextDataCandidate = tryNextDataExtraction(document);
+  const jsonLdCandidate = tryJsonLdExtraction(document);
+  const selectorCandidate = trySelectorExtraction(document);
+  const bodyCandidate = tryBodyExtraction(document);
+
+  const candidates = [
+    readabilityCandidate,
+    nextDataCandidate,
+    jsonLdCandidate,
+    selectorCandidate,
+    bodyCandidate,
+  ].filter((candidate): candidate is ExtractionCandidate => Boolean(candidate));
+
+  const winner = pickBestCandidate(candidates);
+  if (!winner) return null;
+
+  return {
+    ...winner,
+    title: winner.title ?? extractTitle(document),
+    published: winner.published ?? extractPublishedTime(document),
+    excerpt: winner.excerpt ?? generateExcerpt(null, winner.textContent),
+  };
+}
+
+const turndown = new TurndownService({
+  headingStyle: "atx",
+  hr: "---",
+  bulletListMarker: "-",
+  codeBlockStyle: "fenced",
+  emDelimiter: "*",
+  strongDelimiter: "**",
+  linkStyle: "inlined",
+});
+
+turndown.use(gfm);
+turndown.remove(["script", "style", "iframe", "noscript", "template", "svg", "path"]);
+
+turndown.addRule("collapseFigure", {
+  filter: "figure",
+  replacement(content) {
+    return `\n\n${content.trim()}\n\n`;
+  },
+});
+
+turndown.addRule("dropInvisibleAnchors", {
+  filter(node) {
+    return node.nodeName === "A" && !(node as Element).textContent?.trim();
+  },
+  replacement() {
+    return "";
+  },
+});
+
+function convertHtmlToMarkdown(html: string): string {
+  if (!html || !html.trim()) return "";
+
+  try {
+    const sanitized = sanitizeHtml(html);
+    return turndown.turndown(sanitized);
+  } catch {
+    return "";
+  }
+}
+
+function fallbackPlainText(html: string): string {
+  const document = parseDocument(html);
+  for (const selector of ["script", "style", "noscript", "template", "iframe", "svg", "path"]) {
+    for (const el of document.querySelectorAll(selector)) {
+      el.remove();
+    }
+  }
+  const text = document.body?.textContent ?? document.documentElement?.textContent ?? "";
+  return normalizeMarkdown(text.replace(/\s+/g, " "));
+}
+
+function countBylines(markdown: string): number {
+  return (markdown.match(/(^|\n)By\s+/g) || []).length;
+}
+
+function countUsefulParagraphs(markdown: string): number {
+  const paragraphs = normalizeMarkdown(markdown).split(/\n{2,}/);
+  let count = 0;
+
+  for (const paragraph of paragraphs) {
+    const trimmed = paragraph.trim();
+    if (!trimmed) continue;
+    if (/^!?\[[^\]]*\]\([^)]+\)$/.test(trimmed)) continue;
+    if (/^#{1,6}\s+/.test(trimmed)) continue;
+    if ((trimmed.match(/\b[\p{L}\p{N}']+\b/gu) || []).length < 8) continue;
+    count++;
+  }
+
+  return count;
+}
+
+function countMarkerHits(markdown: string, markers: RegExp[]): number {
+  let hits = 0;
+  for (const marker of markers) {
+    if (marker.test(markdown)) hits++;
+  }
+  return hits;
+}
+
+export function scoreMarkdownQuality(markdown: string): number {
+  const normalized = normalizeMarkdown(markdown);
+  const wordCount = (normalized.match(/\b[\p{L}\p{N}']+\b/gu) || []).length;
+  const usefulParagraphs = countUsefulParagraphs(normalized);
+  const headingCount = (normalized.match(/^#{1,6}\s+/gm) || []).length;
+  const markerHits = countMarkerHits(normalized, LOW_QUALITY_MARKERS);
+  const bylineCount = countBylines(normalized);
+  const staffCount = (normalized.match(/\bForbes Staff\b/gi) || []).length;
+
+  return (
+    Math.min(wordCount, 4000) +
+    usefulParagraphs * 40 +
+    headingCount * 10 -
+    markerHits * 180 -
+    Math.max(0, bylineCount - 1) * 120 -
+    Math.max(0, staffCount - 1) * 80
+  );
+}
+
+export function shouldCompareWithLegacy(markdown: string): boolean {
+  const normalized = normalizeMarkdown(markdown);
+  return (
+    countMarkerHits(normalized, LOW_QUALITY_MARKERS) > 0 ||
+    countBylines(normalized) > 1 ||
+    countUsefulParagraphs(normalized) < 6
+  );
+}
+
+export function convertWithLegacyExtractor(html: string, baseMetadata: PageMetadata): ConversionResult {
+  const extracted = extractFromHtml(html);
+
+  let markdown = extracted?.html ? convertHtmlToMarkdown(extracted.html) : "";
+  if (!markdown.trim()) {
+    markdown = extracted?.textContent?.trim() || fallbackPlainText(html);
+  }
+
+  return {
+    metadata: {
+      ...baseMetadata,
+      title: pickString(extracted?.title, baseMetadata.title) ?? "",
+      description: pickString(extracted?.excerpt, baseMetadata.description) ?? undefined,
+      author: pickString(extracted?.byline, baseMetadata.author) ?? undefined,
+      published: pickString(extracted?.published, baseMetadata.published) ?? undefined,
+    },
+    markdown: normalizeMarkdown(markdown),
+    rawHtml: html,
+    conversionMethod: extracted ? `legacy:${extracted.method}` : "legacy:plain-text",
+  };
+}
@@ -75,6 +75,55 @@ function deriveHtmlSnapshotPath(markdownPath: string): string {
  return path.join(parsed.dir, `${basename}-captured.html`);
 }

+function extractTitleFromMarkdownDocument(document: string): string {
+  const normalized = document.replace(/\r\n/g, "\n");
+  const frontmatterMatch = normalized.match(/^---\n([\s\S]*?)\n---\n?/);
+  if (frontmatterMatch) {
+    const titleLine = frontmatterMatch[1]
+      .split("\n")
+      .find((line) => /^title:\s*/i.test(line));
+
+    if (titleLine) {
+      const rawValue = titleLine.replace(/^title:\s*/i, "").trim();
+      const unquoted = rawValue
+        .replace(/^"(.*)"$/, "$1")
+        .replace(/^'(.*)'$/, "$1")
+        .replace(/\\"/g, '"');
+      if (unquoted) return unquoted;
+    }
+  }
+
+  const headingMatch = normalized.match(/^#\s+(.+)$/m);
+  return headingMatch?.[1]?.trim() ?? "";
+}
+
+function buildDefuddleApiUrl(targetUrl: string): string {
+  return `https://defuddle.md/${encodeURIComponent(targetUrl)}`;
+}
+
+async function fetchDefuddleApiMarkdown(targetUrl: string): Promise<{ markdown: string; title: string }> {
+  const apiUrl = buildDefuddleApiUrl(targetUrl);
+  const response = await fetch(apiUrl, {
+    headers: {
+      accept: "text/markdown,text/plain;q=0.9,*/*;q=0.1",
+    },
+  });
+
+  if (!response.ok) {
+    throw new Error(`defuddle.md returned ${response.status} ${response.statusText}`);
+  }
+
+  const markdown = (await response.text()).replace(/\r\n/g, "\n").trim();
+  if (!markdown) {
+    throw new Error("defuddle.md returned empty markdown");
+  }
+
+  return {
+    markdown,
+    title: extractTitleFromMarkdownDocument(markdown),
+  };
+}
+
 async function generateOutputPath(url: string, title: string, outputDir?: string): Promise<string> {
  const domain = new URL(url).hostname.replace(/^www\./, "");
  const slug = generateSlug(title, url);
@@ -192,14 +241,41 @@ async function main(): Promise<void> {
  console.log(`Fetching: ${args.url}`);
  console.log(`Mode: ${args.wait ? "wait" : "auto"}`);

-  const result = await captureUrl(args);
-  const outputPath = args.output || await generateOutputPath(args.url, result.metadata.title, args.outputDir);
-  const outputDir = path.dirname(outputPath);
-  const htmlSnapshotPath = deriveHtmlSnapshotPath(outputPath);
-  await mkdir(outputDir, { recursive: true });
-  await writeFile(htmlSnapshotPath, result.rawHtml, "utf-8");
+  let outputPath: string;
+  let htmlSnapshotPath: string | null = null;
+  let document: string;
+  let conversionMethod: string;
+  let fallbackReason: string | undefined;

-  let document = createMarkdownDocument(result);
+  try {
+    const result = await captureUrl(args);
+    outputPath = args.output || await generateOutputPath(args.url, result.metadata.title, args.outputDir);
+    const outputDir = path.dirname(outputPath);
+    htmlSnapshotPath = deriveHtmlSnapshotPath(outputPath);
+    await mkdir(outputDir, { recursive: true });
+    await writeFile(htmlSnapshotPath, result.rawHtml, "utf-8");
+
+    document = createMarkdownDocument(result);
+    conversionMethod = result.conversionMethod;
+    fallbackReason = result.fallbackReason;
+  } catch (error) {
+    const primaryError = error instanceof Error ? error.message : String(error);
+    console.warn(`Primary capture failed: ${primaryError}`);
+    console.warn("Trying defuddle.md API fallback...");
+
+    try {
+      const remoteResult = await fetchDefuddleApiMarkdown(args.url);
+      outputPath = args.output || await generateOutputPath(args.url, remoteResult.title, args.outputDir);
+      await mkdir(path.dirname(outputPath), { recursive: true });
+
+      document = remoteResult.markdown;
+      conversionMethod = "defuddle-api";
+      fallbackReason = `Local browser capture failed: ${primaryError}`;
+    } catch (remoteError) {
+      const remoteMessage = remoteError instanceof Error ? remoteError.message : String(remoteError);
+      throw new Error(`Local browser capture failed (${primaryError}); defuddle.md fallback failed (${remoteMessage})`);
+    }
+  }

  if (args.downloadMedia) {
    const mediaResult = await localizeMarkdownMedia(document, {
@@ -220,11 +296,15 @@ async function main(): Promise<void> {
  await writeFile(outputPath, document, "utf-8");

  console.log(`Saved: ${outputPath}`);
-  console.log(`Saved HTML: ${htmlSnapshotPath}`);
-  console.log(`Title: ${result.metadata.title || "(no title)"}`);
-  console.log(`Converter: ${result.conversionMethod}`);
-  if (result.fallbackReason) {
-    console.warn(`Fallback used: ${result.fallbackReason}`);
+  if (htmlSnapshotPath) {
+    console.log(`Saved HTML: ${htmlSnapshotPath}`);
+  } else {
+    console.log("Saved HTML: unavailable (defuddle.md fallback)");
+  }
+  console.log(`Title: ${extractTitleFromMarkdownDocument(document) || "(no title)"}`);
+  console.log(`Converter: ${conversionMethod}`);
+  if (fallbackReason) {
+    console.warn(`Fallback used: ${fallbackReason}`);
  }
 }

@@ -0,0 +1,305 @@
+import { parseHTML } from "linkedom";
+
+export interface PageMetadata {
+  url: string;
+  title: string;
+  description?: string;
+  author?: string;
+  published?: string;
+  coverImage?: string;
+  language?: string;
+  captured_at: string;
+}
+
+export interface ConversionResult {
+  metadata: PageMetadata;
+  markdown: string;
+  rawHtml: string;
+  conversionMethod: string;
+  fallbackReason?: string;
+  variables?: Record<string, string>;
+}
+
+export type AnyRecord = Record<string, unknown>;
+
+export const MIN_CONTENT_LENGTH = 120;
+export const GOOD_CONTENT_LENGTH = 900;
+
+const PUBLISHED_TIME_SELECTORS = [
+  "meta[property='article:published_time']",
+  "meta[name='pubdate']",
+  "meta[name='publishdate']",
+  "meta[name='date']",
+  "time[datetime]",
+];
+
+const ARTICLE_TYPES = new Set([
+  "Article",
+  "NewsArticle",
+  "BlogPosting",
+  "WebPage",
+  "ReportageNewsArticle",
+]);
+
+export function pickString(...values: unknown[]): string | null {
+  for (const value of values) {
+    if (typeof value === "string") {
+      const trimmed = value.trim();
+      if (trimmed) return trimmed;
+    }
+  }
+  return null;
+}
+
+export function normalizeMarkdown(markdown: string): string {
+  return markdown
+    .replace(/\r\n/g, "\n")
+    .replace(/[ \t]+\n/g, "\n")
+    .replace(/\n{3,}/g, "\n\n")
+    .trim();
+}
+
+export function parseDocument(html: string): Document {
+  const normalized = /<\s*html[\s>]/i.test(html)
+    ? html
+    : `<!doctype html><html><body>${html}</body></html>`;
+  return parseHTML(normalized).document as unknown as Document;
+}
+
+export function sanitizeHtml(html: string): string {
+  const { document } = parseHTML(`<div id="__root">${html}</div>`);
+  const root = document.querySelector("#__root");
+  if (!root) return html;
+
+  for (const selector of ["script", "style", "iframe", "noscript", "template", "svg", "path"]) {
+    for (const el of root.querySelectorAll(selector)) {
+      el.remove();
+    }
+  }
+
+  return root.innerHTML;
+}
+
+export function extractTextFromHtml(html: string): string {
+  const { document } = parseHTML(`<!doctype html><html><body>${html}</body></html>`);
+  for (const selector of ["script", "style", "noscript", "template", "iframe", "svg", "path"]) {
+    for (const el of document.querySelectorAll(selector)) {
+      el.remove();
+    }
+  }
+  return document.body?.textContent?.replace(/\s+/g, " ").trim() ?? "";
+}
+
+export function getMetaContent(document: Document, names: string[]): string | null {
+  for (const name of names) {
+    const element =
+      document.querySelector(`meta[name="${name}"]`) ??
+      document.querySelector(`meta[property="${name}"]`);
+    const content = element?.getAttribute("content");
+    if (content && content.trim()) return content.trim();
+  }
+  return null;
+}
+
+function normalizeLanguageTag(value: string | null): string | null {
+  if (!value) return null;
+
+  const trimmed = value.trim();
+  if (!trimmed) return null;
+
+  const primary = trimmed.split(/[,\s;]/, 1)[0]?.trim();
+  if (!primary) return null;
+
+  return primary.replace(/_/g, "-");
+}
+
+function flattenJsonLdItems(data: unknown): AnyRecord[] {
+  if (!data || typeof data !== "object") return [];
+  if (Array.isArray(data)) return data.flatMap(flattenJsonLdItems);
+
+  const item = data as AnyRecord;
+  if (Array.isArray(item["@graph"])) {
+    return (item["@graph"] as unknown[]).flatMap(flattenJsonLdItems);
+  }
+
+  return [item];
+}
+
+function parseJsonLdScripts(document: Document): AnyRecord[] {
+  const results: AnyRecord[] = [];
+  const scripts = document.querySelectorAll("script[type='application/ld+json']");
+
+  for (const script of scripts) {
+    try {
+      const data = JSON.parse(script.textContent ?? "");
+      results.push(...flattenJsonLdItems(data));
+    } catch {
+      // Ignore malformed blocks.
+    }
+  }
+
+  return results;
+}
+
+function isArticleType(item: AnyRecord): boolean {
+  const value = Array.isArray(item["@type"]) ? item["@type"][0] : item["@type"];
+  return typeof value === "string" && ARTICLE_TYPES.has(value);
+}
+
+function extractAuthorFromJsonLd(authorData: unknown): string | null {
+  if (typeof authorData === "string") return authorData;
+  if (!authorData || typeof authorData !== "object") return null;
+
+  if (Array.isArray(authorData)) {
+    const names = authorData
+      .map((author) => extractAuthorFromJsonLd(author))
+      .filter((name): name is string => Boolean(name));
+    return names.length > 0 ? names.join(", ") : null;
+  }
+
+  const author = authorData as AnyRecord;
+  return typeof author.name === "string" ? author.name : null;
+}
+
+function extractPrimaryJsonLdMeta(document: Document): Partial<PageMetadata> {
+  for (const item of parseJsonLdScripts(document)) {
+    if (!isArticleType(item)) continue;
+
+    return {
+      title: pickString(item.headline, item.name) ?? undefined,
+      description: pickString(item.description) ?? undefined,
+      author: extractAuthorFromJsonLd(item.author) ?? undefined,
+      published: pickString(item.datePublished, item.dateCreated) ?? undefined,
+      coverImage:
+        pickString(
+          item.image,
+          (item.image as AnyRecord | undefined)?.url,
+          (Array.isArray(item.image) ? item.image[0] : undefined) as unknown
+        ) ?? undefined,
+    };
+  }
+
+  return {};
+}
+
+export function extractPublishedTime(document: Document): string | null {
+  for (const selector of PUBLISHED_TIME_SELECTORS) {
+    const el = document.querySelector(selector);
+    if (!el) continue;
+    const value = el.getAttribute("content") ?? el.getAttribute("datetime");
+    if (value && value.trim()) return value.trim();
+  }
+  return null;
+}
+
+export function extractTitle(document: Document): string | null {
+  const ogTitle = document.querySelector("meta[property='og:title']")?.getAttribute("content");
+  if (ogTitle && ogTitle.trim()) return ogTitle.trim();
+
+  const twitterTitle = document.querySelector("meta[name='twitter:title']")?.getAttribute("content");
+  if (twitterTitle && twitterTitle.trim()) return twitterTitle.trim();
+
+  const title = document.querySelector("title")?.textContent?.trim();
+  if (title) {
+    const cleaned = title.split(/\s*[-|–—]\s*/)[0]?.trim();
+    if (cleaned) return cleaned;
+  }
+
+  const h1 = document.querySelector("h1")?.textContent?.trim();
+  return h1 || null;
+}
+
+export function extractMetadataFromHtml(html: string, url: string, capturedAt: string): PageMetadata {
+  const document = parseDocument(html);
+  const jsonLd = extractPrimaryJsonLdMeta(document);
+  const timeEl = document.querySelector("time[datetime]");
+  const htmlLang = normalizeLanguageTag(document.documentElement?.getAttribute("lang"));
+  const metaLanguage = normalizeLanguageTag(
+    pickString(
+      getMetaContent(document, ["language", "content-language", "og:locale"]),
+      document.querySelector("meta[http-equiv='content-language']")?.getAttribute("content")
+    )
+  );
+
+  return {
+    url,
+    title:
+      pickString(
+        getMetaContent(document, ["og:title", "twitter:title"]),
+        jsonLd.title,
+        document.querySelector("h1")?.textContent,
+        document.title
+      ) ?? "",
+    description:
+      pickString(
+        getMetaContent(document, ["description", "og:description", "twitter:description"]),
+        jsonLd.description
+      ) ?? undefined,
+    author:
+      pickString(
+        getMetaContent(document, ["author", "article:author", "twitter:creator"]),
+        jsonLd.author
+      ) ?? undefined,
+    published:
+      pickString(
+        timeEl?.getAttribute("datetime"),
+        getMetaContent(document, ["article:published_time", "datePublished", "publishdate", "date"]),
+        jsonLd.published,
+        extractPublishedTime(document)
+      ) ?? undefined,
+    coverImage:
+      pickString(
+        getMetaContent(document, ["og:image", "twitter:image", "twitter:image:src"]),
+        jsonLd.coverImage
+      ) ?? undefined,
+    language: pickString(htmlLang, metaLanguage) ?? undefined,
+    captured_at: capturedAt,
+  };
+}
+
+export function isMarkdownUsable(markdown: string, html: string): boolean {
+  const normalized = normalizeMarkdown(markdown);
+  if (!normalized) return false;
+
+  const htmlTextLength = extractTextFromHtml(html).length;
+  if (htmlTextLength < MIN_CONTENT_LENGTH) return true;
+
+  if (normalized.length >= 80) return true;
+  return normalized.length >= Math.min(200, Math.floor(htmlTextLength * 0.2));
+}
+
+export function isYouTubeUrl(url: string): boolean {
+  try {
+    const hostname = new URL(url).hostname.toLowerCase();
+    return hostname === "youtu.be" || hostname.endsWith(".youtube.com") || hostname === "youtube.com";
+  } catch {
+    return false;
+  }
+}
+
+function escapeYamlValue(value: string): string {
+  return value.replace(/\\/g, "\\\\").replace(/"/g, '\\"').replace(/\r?\n/g, "\\n");
+}
+
+export function formatMetadataYaml(meta: PageMetadata): string {
+  const lines = ["---"];
+  lines.push(`url: ${meta.url}`);
+  lines.push(`title: "${escapeYamlValue(meta.title)}"`);
+  if (meta.description) lines.push(`description: "${escapeYamlValue(meta.description)}"`);
+  if (meta.author) lines.push(`author: "${escapeYamlValue(meta.author)}"`);
+  if (meta.published) lines.push(`published: "${escapeYamlValue(meta.published)}"`);
+  if (meta.coverImage) lines.push(`coverImage: "${escapeYamlValue(meta.coverImage)}"`);
+  if (meta.language) lines.push(`language: "${escapeYamlValue(meta.language)}"`);
+  lines.push(`captured_at: "${escapeYamlValue(meta.captured_at)}"`);
+  lines.push("---");
+  return lines.join("\n");
+}
+
+export function createMarkdownDocument(result: ConversionResult): string {
+  const yaml = formatMetadataYaml(result.metadata);
+  const escapedTitle = result.metadata.title.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+  const titleRegex = new RegExp(`^#\\s+${escapedTitle}\\s*(\\n|$)`, "i");
+  const hasTitle = titleRegex.test(result.markdown.trimStart());
+  const title = result.metadata.title && !hasTitle ? `\n\n# ${result.metadata.title}\n\n` : "\n\n";
+  return yaml + title + result.markdown;
+}
@@ -5,7 +5,7 @@
  "dependencies": {
    "@mozilla/readability": "^0.6.0",
    "baoyu-chrome-cdp": "file:./vendor/baoyu-chrome-cdp",
-    "defuddle": "^0.10.0",
+    "defuddle": "^0.12.0",
    "jsdom": "^24.1.3",
    "linkedom": "^0.18.12",
    "turndown": "^7.2.2",
Author	SHA1	Message	Date
Jim Liu 宝玉	c1e1526c84	chore: release v1.63.0	2026-03-13 00:24:32 -05:00
Jim Liu 宝玉	0279fa403d	feat(baoyu-url-to-markdown): add defuddle.md API fallback, YouTube transcripts, and modular converter architecture	2026-03-13 00:22:03 -05:00
Jim Liu 宝玉	994e47d1be	docs: fix Claude Code marketplace repo casing	2026-03-12 12:46:25 -05:00
Jim Liu 宝玉	6a71d5080d	chore: release v1.62.0	2026-03-12 01:51:18 -05:00
Jim Liu 宝玉	1e6e6637ac	feat(baoyu-infographic): support flexible aspect ratios Accept custom W:H ratios (e.g., 3:4, 4:3, 2.35:1) in addition to named presets (landscape/portrait/square).	2026-03-12 01:50:36 -05:00
Jim Liu 宝玉	873b60aee5	docs(wechat): replace credential-like placeholders	2026-03-12 00:53:33 -05:00
Jim Liu 宝玉	b88ac59133	fix: set strict mode on plugins to prevent duplicated slash commands Closes #79	2026-03-11 23:29:27 -05:00
Jim Liu 宝玉	d889c04084	chore: release v1.61.0	2026-03-11 23:24:43 -05:00
Jim Liu 宝玉	5276fae6bd	feat(baoyu-post-to-wechat): add multi-account support Support multiple WeChat Official Accounts via EXTEND.md accounts block. Each account gets isolated Chrome profile, credential resolution chain (inline → prefixed env → unprefixed env), and --account CLI arg.	2026-03-11 23:24:02 -05:00
Jim Liu 宝玉	c0941f8089	fix: exclude out/dist/build dirs and bun.lockb from skill release files	2026-03-11 22:55:47 -05:00
Jim Liu 宝玉	42b8b1fc99	fix: use proper MIME types in skill publish to fix ClawhHub rejection	2026-03-11 22:46:06 -05:00