feat\!: rename baoyu-imagine→baoyu-image-gen, baoyu-image-cards→baoyu-xhs-images (v2.0.0)

BREAKING CHANGE: removed `baoyu-imagine` and `baoyu-image-cards`. All
functionality now lives under `baoyu-image-gen` and `baoyu-xhs-images`
respectively. Cross-skill `## Image Generation Tools` examples updated
across baoyu-article-illustrator, baoyu-comic, baoyu-cover-image,
baoyu-infographic, and baoyu-slide-deck.

Migration: existing `~/.baoyu-skills/baoyu-imagine/EXTEND.md` configs are
auto-renamed to `…/baoyu-image-gen/EXTEND.md` on first run via the
legacy-path resolver in `scripts/main.ts`.

.claude-plugin/marketplace.json

@@ -6,23 +6,36 @@
   },
   "metadata": {
     "description": "Skills shared by Baoyu for improving daily work efficiency",
-    "version": "0.5.3"
+    "version": "2.0.0"
   },
   "plugins": [
     {
-      "name": "content-skills",
-      "description": "Skills shared by Baoyu for improving daily work efficiency",
+      "name": "baoyu-skills",
+      "description": "Content generation, AI backends, and utility tools for daily work efficiency",
       "source": "./",
       "strict": false,
       "skills": [
-        "./skills/baoyu-gemini-web",
-        "./skills/baoyu-xhs-images",
-        "./skills/baoyu-post-to-x",
-        "./skills/baoyu-post-to-wechat",
         "./skills/baoyu-article-illustrator",
+        "./skills/baoyu-comic",
+        "./skills/baoyu-compress-image",
         "./skills/baoyu-cover-image",
+        "./skills/baoyu-danger-gemini-web",
+        "./skills/baoyu-danger-x-to-markdown",
+        "./skills/baoyu-diagram",
+        "./skills/baoyu-electron-extract",
+        "./skills/baoyu-format-markdown",
+        "./skills/baoyu-image-gen",
+        "./skills/baoyu-infographic",
+        "./skills/baoyu-markdown-to-html",
+        "./skills/baoyu-post-to-weibo",
+        "./skills/baoyu-post-to-wechat",
+        "./skills/baoyu-post-to-x",
         "./skills/baoyu-slide-deck",
-        "./skills/baoyu-comic"
+        "./skills/baoyu-translate",
+        "./skills/baoyu-url-to-markdown",
+        "./skills/baoyu-wechat-summary",
+        "./skills/baoyu-xhs-images",
+        "./skills/baoyu-youtube-transcript"
       ]
     }
   ]

.claude/skills/release-skills/SKILL.md

@@ -0,0 +1,578 @@
+---
+name: release-skills
+description: Universal release workflow. Auto-detects version files and changelogs. Supports Node.js, Python, Rust, Claude Plugin, GitHub Releases, annotated tags, historical release backfill, and generic projects. Use when user says "release", "发布", "new version", "bump version", "push", "推送", "release notes", "GitHub Release", or "回填 Release".
+---
+
+# Release Skills
+
+Universal release workflow supporting any project type with multi-language changelog.
+
+## User Input Tools
+
+When this skill prompts the user, follow this tool-selection rule (priority order):
+
+1. **Prefer built-in user-input tools** exposed by the current agent runtime — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.
+2. **Fallback**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
+3. **Batching**: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.
+
+Concrete `AskUserQuestion` references below are examples — substitute the local equivalent in other runtimes.
+
+## Quick Start
+
+Just run `/release-skills` - auto-detects your project configuration.
+
+## Supported Projects
+
+| Project Type | Version File | Auto-Detected |
+|--------------|--------------|---------------|
+| Node.js | package.json | ✓ |
+| Python | pyproject.toml | ✓ |
+| Rust | Cargo.toml | ✓ |
+| Claude Plugin | marketplace.json | ✓ |
+| Generic | VERSION / version.txt | ✓ |
+
+## Options
+
+| Flag | Description |
+|------|-------------|
+| `--dry-run` | Preview changes without executing |
+| `--major` | Force major version bump |
+| `--minor` | Force minor version bump |
+| `--patch` | Force patch version bump |
+| `--backfill-releases` | Create missing GitHub Releases for existing tags from changelog sections |
+
+## Workflow
+
+### Step 1: Detect Project Configuration
+
+1. Check for `.releaserc.yml` (optional config override)
+   - If present, inspect whether it defines release hooks
+2. Auto-detect version file by scanning (priority order):
+   - `package.json` (Node.js)
+   - `pyproject.toml` (Python)
+   - `Cargo.toml` (Rust)
+   - `marketplace.json` or `.claude-plugin/marketplace.json` (Claude Plugin)
+   - `VERSION` or `version.txt` (Generic)
+3. Scan for changelog files using glob patterns:
+   - `CHANGELOG*.md`
+   - `HISTORY*.md`
+   - `CHANGES*.md`
+4. Identify language of each changelog by filename suffix
+5. Detect GitHub release support:
+   - Check whether `origin` points to GitHub
+   - Check whether `gh` is installed and authenticated
+   - Check existing releases with `gh release list --limit 5` when available
+6. Display detected configuration
+
+**Project Hook Contract**:
+
+If `.releaserc.yml` defines `release.hooks`, keep the release workflow generic and delegate project-specific packaging/publishing to those hooks.
+
+Supported hooks:
+
+| Hook | Purpose | Expected Responsibility |
+|------|---------|-------------------------|
+| `prepare_artifact` | Make one target releasable | Validate the target is self-contained, sync/embed local dependencies, optionally stage extra files |
+| `publish_artifact` | Publish one releasable target | Upload the prepared target (or a staged directory if the project uses one), attach version/changelog/tags |
+
+Supported placeholders:
+
+| Placeholder | Meaning |
+|-------------|---------|
+| `{project_root}` | Absolute path to repository root |
+| `{target}` | Absolute path to the module/skill being released |
+| `{artifact_dir}` | Absolute path to a temporary staging directory for this target, when the project uses one |
+| `{version}` | Version selected by the release workflow |
+| `{dry_run}` | `true` or `false` |
+| `{release_notes_file}` | Absolute path to a UTF-8 file containing release notes/changelog text |
+
+Execution rules:
+- Keep the skill generic: do not hardcode registry/package-manager/project layout details into this SKILL.
+- If `prepare_artifact` exists, run it once per target before publish-related checks that need the final releasable target state.
+- Write release notes to a temp file and pass that file path to `publish_artifact`; do not inline multiline changelog text into shell commands.
+- If hooks are absent, fall back to the default project-agnostic release workflow.
+
+**Language Detection Rules**:
+
+Changelog files follow the pattern `CHANGELOG_{LANG}.md` or `CHANGELOG.{lang}.md`, where `{lang}` / `{LANG}` is a language or region code.
+
+| Pattern | Example | Language |
+|---------|---------|----------|
+| No suffix | `CHANGELOG.md` | en (default) |
+| `_{LANG}` (uppercase) | `CHANGELOG_CN.md`, `CHANGELOG_JP.md` | Corresponding language |
+| `.{lang}` (lowercase) | `CHANGELOG.zh.md`, `CHANGELOG.ja.md` | Corresponding language |
+| `.{lang-region}` | `CHANGELOG.zh-CN.md` | Corresponding region variant |
+
+Common language codes: `zh` (Chinese), `ja` (Japanese), `ko` (Korean), `de` (German), `fr` (French), `es` (Spanish).
+
+**Output Example**:
+```
+Project detected:
+  Version file: package.json (1.2.3)
+  Changelogs:
+    - CHANGELOG.md (en)
+    - CHANGELOG.zh.md (zh)
+    - CHANGELOG.ja.md (ja)
+```
+
+### Step 2: Analyze Changes Since Last Tag
+
+```bash
+LAST_TAG=$(git tag --sort=-v:refname | head -1)
+git log ${LAST_TAG}..HEAD --oneline
+git diff ${LAST_TAG}..HEAD --stat
+```
+
+Categorize by conventional commit types:
+
+| Type | Description |
+|------|-------------|
+| feat | New features |
+| fix | Bug fixes |
+| docs | Documentation |
+| refactor | Code refactoring |
+| perf | Performance improvements |
+| test | Test changes |
+| style | Formatting, styling |
+| chore | Maintenance (skip in changelog) |
+
+**Breaking Change Detection**:
+- Commit message starts with `BREAKING CHANGE`
+- Commit body/footer contains `BREAKING CHANGE:`
+- Removed public APIs, renamed exports, changed interfaces
+
+If breaking changes detected, warn user: "Breaking changes detected. Consider major version bump (--major flag)."
+
+### Step 3: Determine Version Bump
+
+Rules (in priority order):
+1. User flag `--major/--minor/--patch` → Use specified
+2. BREAKING CHANGE detected → Major bump (1.x.x → 2.0.0)
+3. `feat:` commits present → Minor bump (1.2.x → 1.3.0)
+4. Otherwise → Patch bump (1.2.3 → 1.2.4)
+
+Display version change: `1.2.3 → 1.3.0`
+
+### Step 4: Generate Multi-language Changelogs
+
+For each detected changelog file:
+
+1. **Identify language** from filename suffix
+2. **Detect third-party contributors**:
+   - Check merge commits: `git log ${LAST_TAG}..HEAD --merges --pretty=format:"%H %s"`
+   - For each merged PR, identify the PR author via `gh pr view <number> --json author --jq '.author.login'`
+   - Compare against repo owner (`gh repo view --json owner --jq '.owner.login'`)
+   - If PR author ≠ repo owner → third-party contributor
+3. **Generate content in that language**:
+   - Section titles in target language
+   - Change descriptions written naturally in target language (not translated)
+   - Date format: YYYY-MM-DD (universal)
+   - **Third-party contributions**: Append contributor attribution `(by @username)` to the changelog entry
+4. **Insert at file head** (preserve existing content)
+
+**Section Title Translations** (built-in):
+
+| Type | en | zh | ja | ko | de | fr | es |
+|------|----|----|----|----|----|----|-----|
+| feat | Features | 新功能 | 新機能 | 새로운 기능 | Funktionen | Fonctionnalités | Características |
+| fix | Fixes | 修复 | 修正 | 수정 | Fehlerbehebungen | Corrections | Correcciones |
+| docs | Documentation | 文档 | ドキュメント | 문서 | Dokumentation | Documentation | Documentación |
+| refactor | Refactor | 重构 | リファクタリング | 리팩토링 | Refactoring | Refactorisation | Refactorización |
+| perf | Performance | 性能优化 | パフォーマンス | 성능 | Leistung | Performance | Rendimiento |
+| breaking | Breaking Changes | 破坏性变更 | 破壊的変更 | 주요 변경사항 | Breaking Changes | Changements majeurs | Cambios importantes |
+
+**Changelog Format**:
+
+```markdown
+## {VERSION} - {YYYY-MM-DD}
+
+### Features
+- Description of new feature
+- Description of third-party contribution (by @username)
+
+### Fixes
+- Description of fix
+
+### Documentation
+- Description of docs changes
+```
+
+Only include sections that have changes. Omit empty sections.
+
+**Third-Party Attribution Rules**:
+- Only add `(by @username)` for contributors who are NOT the repo owner
+- Use GitHub username with `@` prefix
+- Place at the end of the changelog entry line
+- Apply to all languages consistently (always use `(by @username)` format, not translated)
+
+**Multi-language Example**:
+
+English (CHANGELOG.md):
+```markdown
+## 1.3.0 - 2026-01-22
+
+### Features
+- Add user authentication module (by @contributor1)
+- Support OAuth2 login
+
+### Fixes
+- Fix memory leak in connection pool
+```
+
+Chinese (CHANGELOG.zh.md):
+```markdown
+## 1.3.0 - 2026-01-22
+
+### 新功能
+- 新增用户认证模块 (by @contributor1)
+- 支持 OAuth2 登录
+
+### 修复
+- 修复连接池内存泄漏问题
+```
+
+Japanese (CHANGELOG.ja.md):
+```markdown
+## 1.3.0 - 2026-01-22
+
+### 新機能
+- ユーザー認証モジュールを追加 (by @contributor1)
+- OAuth2 ログインをサポート
+
+### 修正
+- コネクションプールのメモリリークを修正
+```
+
+### Step 5: Group Changes by Skill/Module
+
+Analyze commits since last tag and group by affected skill/module:
+
+1. **Identify changed files** per commit
+2. **Group by skill/module**:
+   - `skills/<skill-name>/*` → Group under that skill
+   - Root files (CLAUDE.md, etc.) → Group as "project"
+   - Multiple skills in one commit → Split into multiple groups
+3. **For each group**, identify related README updates needed
+
+**Example Grouping**:
+```
+baoyu-cover-image:
+  - feat: add new style options
+  - fix: handle transparent backgrounds
+  → README updates: options table
+
+baoyu-comic:
+  - refactor: improve panel layout algorithm
+  → No README updates needed
+
+project:
+  - docs: update CLAUDE.md architecture section
+```
+
+### Step 6: Commit Each Skill/Module Separately
+
+For each skill/module group (in order of changes):
+
+1. **Check README updates needed**:
+   - Scan `README*.md` for mentions of this skill/module
+   - Verify options/flags documented correctly
+   - Update usage examples if syntax changed
+   - Update feature descriptions if behavior changed
+
+2. **Stage and commit**:
+   ```bash
+   git add skills/<skill-name>/*
+   git add README.md README.zh.md  # If updated for this skill
+   git commit -m "<type>(<skill-name>): <meaningful description>"
+   ```
+
+3. **Commit message format**:
+   - Use conventional commit format: `<type>(<scope>): <description>`
+   - `<type>`: feat, fix, refactor, docs, perf, etc.
+   - `<scope>`: skill name or "project"
+   - `<description>`: Clear, meaningful description of changes
+
+**Example Commits**:
+```bash
+git commit -m "feat(baoyu-cover-image): add watercolor and minimalist styles"
+git commit -m "fix(baoyu-comic): improve panel layout for long dialogues"
+git commit -m "docs(project): update architecture documentation"
+```
+
+**Common README Updates Needed**:
+| Change Type | README Section to Check |
+|-------------|------------------------|
+| New options/flags | Options table, usage examples |
+| Renamed options | Options table, usage examples |
+| New features | Feature description, examples |
+| Breaking changes | Migration notes, deprecation warnings |
+| Restructured internals | Architecture section (if exposed to users) |
+
+### Step 7: Generate Changelog and Update Version
+
+1. **Generate multi-language changelogs** (as described in Step 4)
+2. **Update version file**:
+   - Read version file (JSON/TOML/text)
+   - Update version number
+   - Write back (preserve formatting)
+3. **Create release notes file**:
+   - Prefer the new version section from `CHANGELOG.md`
+   - If no English/default changelog exists, use the first detected changelog
+   - Extract only the exact `## {VERSION} - {YYYY-MM-DD}` section through the next `##`
+   - Match both plain version and tag-prefixed headings when needed, e.g. `1.2.3` and `v1.2.3`
+   - Keep breaking changes near the top; if needed, add a short highlight before other sections
+   - Write notes to a UTF-8 temp file and reuse it for annotated tag messages, GitHub Releases, and `publish_artifact`
+   - In normal mode, stop rather than creating an empty tag or GitHub Release when notes cannot be found
+
+**Version Paths by File Type**:
+
+| File | Path |
+|------|------|
+| package.json | `$.version` |
+| pyproject.toml | `project.version` |
+| Cargo.toml | `package.version` |
+| marketplace.json | `$.metadata.version` |
+| VERSION / version.txt | Direct content |
+
+### Step 8: User Confirmation
+
+Before creating the release commit, ask user to confirm:
+
+**Use AskUserQuestion with three questions**:
+
+1. **Version bump** (single select):
+   - Show recommended version based on Step 3 analysis
+   - Options: recommended (with label), other semver options
+   - Example: `1.2.3 → 1.3.0 (Recommended)`, `1.2.3 → 1.2.4`, `1.2.3 → 2.0.0`
+
+2. **Push to remote** (single select):
+   - Options: "Yes, push after commit", "No, keep local only"
+
+3. **Publish GitHub Release** (single select):
+   - Offer this only when GitHub release support is available
+   - Default to "Yes, publish after tag push" when the user also chose push
+   - If the user keeps the release local, do not create or edit a GitHub Release
+
+**Example Output Before Confirmation**:
+```
+Commits created:
+  1. feat(baoyu-cover-image): add watercolor and minimalist styles
+  2. fix(baoyu-comic): improve panel layout for long dialogues
+  3. docs(project): update architecture documentation
+
+Changelog preview (en):
+  ## 1.3.0 - 2026-01-22
+  ### Features
+  - Add watercolor and minimalist styles to cover-image
+  ### Fixes
+  - Improve panel layout for long dialogues in comic
+
+Release notes source: CHANGELOG.md#1.3.0
+Ready to create release commit, annotated tag, and GitHub Release.
+```
+
+### Step 9: Create Release Commit and Annotated Tag
+
+After user confirmation:
+
+1. **Stage version and changelog files**:
+   ```bash
+   git add <version-file>
+   git add CHANGELOG*.md
+   ```
+
+2. **Create release commit**:
+   ```bash
+   git commit -m "chore: release v{VERSION}"
+   ```
+
+3. **Create annotated tag**:
+   ```bash
+   git tag -a v{VERSION} -F <release-notes-file>
+   ```
+   If `.releaserc.yml` sets `tag.sign: true`, use `git tag -s` with the same notes file.
+
+4. **Push if user confirmed** (Step 8):
+   ```bash
+   git push origin main
+   git push origin v{VERSION}
+   ```
+
+**Note**: Do NOT add Co-Authored-By line. This is a release commit, not a code contribution.
+
+### Step 10: Publish Release Artifacts and GitHub Release
+
+Project artifact publishing and GitHub Releases are separate outputs:
+
+1. **Project artifacts**:
+   - If `release.hooks.publish_artifact` exists, run it once per prepared target
+   - Pass the same `{release_notes_file}` used for the tag and GitHub Release
+   - In dry-run mode, pass `{dry_run}=true` and report what would be published
+
+2. **GitHub Release**:
+   - Run only if the user confirmed remote publishing and GitHub support is available
+   - Ensure the tag exists on the remote before creating the release
+   - Create or update using the extracted notes:
+     ```bash
+     if gh release view v{VERSION} >/dev/null 2>&1; then
+       gh release edit v{VERSION} --title "v{VERSION}" --notes-file <release-notes-file>
+     else
+       gh release create v{VERSION} --title "v{VERSION}" --notes-file <release-notes-file> --verify-tag
+     fi
+     ```
+   - Never inline multiline release notes into shell commands
+
+**Post-Release Output**:
+```
+Release v1.3.0 created.
+
+Commits:
+  1. feat(baoyu-cover-image): add watercolor and minimalist styles
+  2. fix(baoyu-comic): improve panel layout for long dialogues
+  3. docs(project): update architecture documentation
+  4. chore: release v1.3.0
+
+Tag: v1.3.0
+Tag type: annotated
+GitHub Release: published  # or "skipped/local only"
+Status: Pushed to origin  # or "Local only - run git push when ready"
+```
+
+## Backfill Existing GitHub Releases
+
+Use this mode when the user asks to backfill historical releases or passes `--backfill-releases`.
+
+1. Do not bump versions, edit changelogs, or create release commits.
+2. List existing tags in version order and detect missing releases:
+   ```bash
+   git tag --sort=v:refname
+   gh release view <tag>
+   ```
+3. For each tag without a GitHub Release:
+   - Normalize the changelog lookup by stripping the configured tag prefix, e.g. `v1.2.3` -> `1.2.3`
+   - Extract the matching section from `CHANGELOG.md`; fall back to the first matching changelog file
+   - Skip or ask before publishing if no matching changelog section exists
+   - Create the release with:
+     ```bash
+     gh release create <tag> --title "<tag>" --notes-file <release-notes-file> --verify-tag
+     ```
+4. Detect lightweight tags with `git cat-file -t <tag>` (`commit` means lightweight, `tag` means annotated).
+5. Do not rewrite public lightweight tags by default. Converting an existing remote tag to an annotated tag requires explicit user confirmation because it rewrites a published reference.
+
+## Configuration (.releaserc.yml)
+
+Optional config file in project root to override defaults:
+
+```yaml
+# .releaserc.yml - Optional configuration
+
+# Version file (auto-detected if not specified)
+version:
+  file: package.json
+  path: $.version  # JSONPath for JSON, dotted path for TOML
+
+# Changelog files (auto-detected if not specified)
+changelog:
+  files:
+    - path: CHANGELOG.md
+      lang: en
+    - path: CHANGELOG.zh.md
+      lang: zh
+    - path: CHANGELOG.ja.md
+      lang: ja
+
+  # Section mapping (conventional commit type → changelog section)
+  # Use null to skip a type in changelog
+  sections:
+    feat: Features
+    fix: Fixes
+    docs: Documentation
+    refactor: Refactor
+    perf: Performance
+    test: Tests
+    chore: null
+
+# Commit message format
+commit:
+  message: "chore: release v{version}"
+
+# Tag format
+tag:
+  prefix: v  # Results in v1.0.0
+  sign: false
+
+# Additional files to include in release commit
+include:
+  - README.md
+  - package.json
+```
+
+## Dry-Run Mode
+
+When `--dry-run` is specified:
+
+```
+=== DRY RUN MODE ===
+
+Project detected:
+  Version file: package.json (1.2.3)
+  Changelogs: CHANGELOG.md (en), CHANGELOG.zh.md (zh)
+
+Last tag: v1.2.3
+Proposed version: v1.3.0
+
+Changes grouped by skill/module:
+  baoyu-cover-image:
+    - feat: add watercolor style
+    - feat: add minimalist style
+    → Commit: feat(baoyu-cover-image): add watercolor and minimalist styles
+    → README updates: options table
+
+  baoyu-comic:
+    - fix: panel layout for long dialogues
+    → Commit: fix(baoyu-comic): improve panel layout for long dialogues
+    → No README updates
+
+Changelog preview (en):
+  ## 1.3.0 - 2026-01-22
+  ### Features
+  - Add watercolor and minimalist styles to cover-image
+  ### Fixes
+  - Improve panel layout for long dialogues in comic
+
+Changelog preview (zh):
+  ## 1.3.0 - 2026-01-22
+  ### 新功能
+  - 为 cover-image 添加水彩和极简风格
+  ### 修复
+  - 改进 comic 长对话的面板布局
+
+Commits to create:
+  1. feat(baoyu-cover-image): add watercolor and minimalist styles
+  2. fix(baoyu-comic): improve panel layout for long dialogues
+  3. chore: release v1.3.0
+
+No changes made. Run without --dry-run to execute.
+```
+
+## Example Usage
+
+```
+/release-skills              # Auto-detect version bump
+/release-skills --dry-run    # Preview only
+/release-skills --minor      # Force minor bump
+/release-skills --patch      # Force patch bump
+/release-skills --major      # Force major bump (with confirmation)
+/release-skills --backfill-releases  # Create missing GitHub Releases for existing tags
+```
+
+## When to Use
+
+Trigger this skill when user requests:
+- "release", "发布", "create release", "new version", "新版本"
+- "bump version", "update version", "更新版本"
+- "prepare release"
+- "release notes", "GitHub Release", "回填 Release"
+- "push to remote" (with uncommitted changes)
+
+**Important**: If user says "just push" or "直接 push" with uncommitted changes, STILL follow all steps above first.

.githooks/pre-push

@@ -0,0 +1,7 @@
+#!/bin/sh
+set -eu
+
+REPO_ROOT=$(git rev-parse --show-toplevel)
+cd "$REPO_ROOT"
+
+node scripts/verify-shared-package-deps.mjs --repo-root "$REPO_ROOT"

.github/workflows/codex-imagegen-tests.yml

@@ -0,0 +1,40 @@
+name: codex-imagegen tests
+
+on:
+  push:
+    paths:
+      - 'scripts/codex-imagegen/**'
+      - 'scripts/codex-imagegen.sh'
+      - '.github/workflows/codex-imagegen-tests.yml'
+  pull_request:
+    paths:
+      - 'scripts/codex-imagegen/**'
+      - 'scripts/codex-imagegen.sh'
+      - '.github/workflows/codex-imagegen-tests.yml'
+  workflow_dispatch:
+
+jobs:
+  unit-tests:
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Bun
+        uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+
+      - name: Show Bun version
+        run: bun --version
+
+      - name: Run unit tests
+        working-directory: scripts/codex-imagegen
+        run: bun test
+
+      - name: Bundle smoke test (catches import/syntax errors)
+        run: bun build --target=node scripts/codex-imagegen/main.ts --outfile /tmp/main-build.js
+
+      - name: Help output smoke test
+        run: bun scripts/codex-imagegen/main.ts --help

.github/workflows/test.yml

@@ -0,0 +1,35 @@
+name: Test
+
+on:
+  push:
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  node-tests:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+
+      - name: Verify skill release commits
+        run: npm run verify:skill-release-commits
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Install baoyu-post-to-wechat script dependencies
+        run: |
+          cd skills/baoyu-post-to-wechat/scripts
+          npm install --no-audit --no-fund --ignore-scripts
+
+      - name: Run tests
+        run: npm test

.gitignore

@@ -6,6 +6,7 @@ yarn-debug.log*
 yarn-error.log*
 lerna-debug.log*
 
+
 # Diagnostic reports (https://nodejs.org/api/report.html)
 report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
 
@@ -82,6 +83,10 @@ out
 .nuxt
 dist
 
+# Shared package builds are published to npm.
+!packages/*/dist/
+!packages/*/dist/**
+
 # Gatsby files
 .cache/
 # Comment in the public line in if your project uses Gatsby and not Next.js
@@ -140,3 +145,34 @@ vite.config.ts.timestamp-*
 tests-data/
 
 .DS_Store
+
+# Skill extensions (user customization)
+.baoyu-skills/
+x-to-markdown/
+xhs-images/
+url-to-markdown/
+cover-image/
+slide-deck/
+infographic/
+illustrations/
+comic/
+translate/
+posts/
+diagram/
+### IntelliJ IDEA ###
+.idea
+*.iws
+*.iml
+*.ipr
+
+.claude/skills/baoyu-skill-evolution
+# ClawHub local state (current and legacy directory names from the official CLI)
+.clawhub/
+.clawdhub/
+.release-artifacts/
+.worktrees/
+youtube-transcript/
+.omx/
+.codex-tmp/
+outputs/
+wechat/

.releaserc.yml

@@ -0,0 +1,6 @@
+release:
+  target_globs:
+    - skills/*
+  hooks:
+    prepare_artifact: node scripts/verify-shared-package-deps.mjs --repo-root "{project_root}" --target "{target}"
+    publish_artifact: node scripts/publish-skill.mjs --skill-dir "{target}" --version "{version}" --changelog-file "{release_notes_file}" --dry-run "{dry_run}"

CLAUDE.md

@@ -1,80 +1,110 @@
 # CLAUDE.md
 
-This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
-
-## Project Overview
-
-Claude Code marketplace plugin providing AI-powered content generation skills. Skills use Gemini Web API (reverse-engineered) for text/image generation and Chrome CDP for browser automation.
+Claude Code marketplace plugin providing AI-powered content generation skills. Version: **2.0.0**.
 
 ## Architecture
 
-```
-skills/
-├── baoyu-gemini-web/          # Core: Gemini API wrapper (text + image gen)
-├── baoyu-xhs-images/          # Xiaohongshu infographic series (1-10 images)
-├── baoyu-cover-image/         # Article cover images (2.35:1 aspect)
-├── baoyu-slide-deck/          # Presentation slides with outlines
-├── baoyu-article-illustrator/ # Smart illustration placement
-├── baoyu-post-to-x/           # X/Twitter posting automation
-└── baoyu-post-to-wechat/      # WeChat Official Account posting
-```
+Skills are exposed through the single `baoyu-skills` plugin in `.claude-plugin/marketplace.json` (which defines plugin metadata, version, and skill paths). The repo docs still group them into three logical areas:
 
-Each skill contains:
-- `SKILL.md` - YAML front matter (name, description) + documentation
-- `scripts/` - TypeScript implementations
-- `prompts/system.md` - AI generation guidelines (optional)
+| Group | Description |
+|-------|-------------|
+| Content Skills | Generate or publish content (images, slides, comics, posts) |
+| AI Generation Skills | AI generation backends |
+| Utility Skills | Content processing (conversion, compression, translation) |
+
+Each skill contains `SKILL.md` (YAML front matter + docs), optional `scripts/`, `references/`, `prompts/`.
+
+Top-level `scripts/` contains repo maintenance utilities (sync, hooks, publish).
 
 ## Running Skills
 
-All scripts run via Bun (no build step):
-
+TypeScript via Bun (no build step). Detect runtime once per session:
 ```bash
-npx -y bun skills/<skill>/scripts/main.ts [options]
+if command -v bun &>/dev/null; then BUN_X="bun"
+elif command -v npx &>/dev/null; then BUN_X="npx -y bun"
+else echo "Error: install bun: brew install oven-sh/bun/bun or npm install -g bun"; exit 1; fi
 ```
 
-Examples:
-```bash
-# Text generation
-npx -y bun skills/baoyu-gemini-web/scripts/main.ts "Hello"
-
-# Image generation
-npx -y bun skills/baoyu-gemini-web/scripts/main.ts --prompt "A cat" --image cat.png
-
-# From prompt files
-npx -y bun skills/baoyu-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
-```
+Execute: `${BUN_X} skills/<skill>/scripts/main.ts [options]`
 
 ## Key Dependencies
 
-- **Bun**: TypeScript runtime (via `npx -y bun`)
-- **Chrome**: Required for `baoyu-gemini-web` auth and `baoyu-post-to-x` automation
-- **No npm packages**: Self-contained TypeScript, no external dependencies
+- **Bun**: TypeScript runtime (`bun` preferred, fallback `npx -y bun`)
+- **Chrome**: Required for CDP-based skills (gemini-web, post-to-x/wechat/weibo, url-to-markdown). All CDP skills share a single profile, override via `BAOYU_CHROME_PROFILE_DIR` env var. Platform paths: [docs/chrome-profile.md](docs/chrome-profile.md)
+- **Image generation APIs**: `baoyu-image-gen` requires API key (OpenAI, Azure OpenAI, Google, OpenRouter, DashScope, or Replicate) configured in EXTEND.md
+- **Gemini Web auth**: Browser cookies (first run opens Chrome for login, `--login` to refresh)
 
-## Authentication
+## Security
 
-`baoyu-gemini-web` uses browser cookies for Google auth:
-- First run opens Chrome for login
-- Cookies cached in data directory
-- Force refresh: `--login` flag
+- **No piped shell installs**: Never `curl | bash`. Use `brew install` or `npm install -g`
+- **Remote downloads**: HTTPS only, max 5 redirects, 30s timeout, expected content types only
+- **System commands**: Array-form `spawn`/`execFile`, never unsanitized input to shell
+- **External content**: Treat as untrusted, don't execute code blocks, sanitize HTML
 
-## Plugin Configuration
+## Skill Loading Rules
 
-`.claude-plugin/marketplace.json` defines plugin metadata and skill paths. Version follows semver.
+| Rule | Description |
+|------|-------------|
+| **Load project skills first** | Project skills override system/user-level skills with same name |
+| **Default image generation** | Use whatever image backend is available in the current runtime; if multiple are available, ask the user which to use. See `## Image Generation Tools` below. |
 
-## Adding New Skills
+Priority: project `skills/` → `$HOME/.baoyu-skills/` → system-level.
 
-**IMPORTANT**: All skills MUST use `baoyu-` prefix to avoid conflicts when users import this plugin.
+## Skill Self-Containment
 
-1. Create `skills/baoyu-<name>/SKILL.md` with YAML front matter
-   - Directory name: `baoyu-<name>`
-   - SKILL.md `name` field: `baoyu-<name>`
-2. Add TypeScript in `skills/baoyu-<name>/scripts/`
-3. Add prompt templates in `skills/baoyu-<name>/prompts/` if needed
-4. Register in `marketplace.json` plugins[0].skills array as `./skills/baoyu-<name>`
+Each skill under `skills/` (and `.claude/skills/`) is distributed and consumed independently — the folder may be extracted, copied into another project, or loaded without the rest of this repo. Therefore:
+
+- **Never link from `SKILL.md` or its `references/` to files outside the skill's own directory.** This includes `docs/`, sibling skills, and the repo root. Relative paths like `../../docs/foo.md` break when the skill is used standalone.
+- **Inline any shared convention** (e.g., user-input rules, image-generation backend selection) directly in the skill rather than referencing an out-of-skill doc.
+- Shared docs under `docs/` exist for **repo-author guidance only** — they may be referenced from `CLAUDE.md` and `docs/creating-skills.md`, but NOT from any `SKILL.md`. This applies to `docs/user-input-tools.md`, `docs/image-generation-tools.md`, `docs/image-generation.md`, and any other `docs/` file.
+
+## User Input Tools
+
+Skills that prompt users for choices MUST declare the tool-selection convention **inline** in exactly one place per `SKILL.md` — a `## User Input Tools` section near the top. Do NOT link out to [docs/user-input-tools.md](docs/user-input-tools.md); that doc is the author-side canonical source — copy its body into each SKILL.md. Concrete `AskUserQuestion` mentions elsewhere in a skill are treated as examples — other runtimes substitute their local equivalent under the rule.
+
+## Image Generation Tools
+
+Skills that render images MUST declare the backend-selection convention **inline** in exactly one place per `SKILL.md` — a `## Image Generation Tools` section near the top (after `## User Input Tools`). Do NOT link out to [docs/image-generation-tools.md](docs/image-generation-tools.md); that doc is the author-side canonical source — copy its body into each SKILL.md. Concrete tool names (`imagegen`, `image_generate`, `baoyu-image-gen`) elsewhere in a skill are treated as examples — other runtimes substitute their local equivalent under the rule. The rule is stateless: use whatever backend is available; if multiple, ask the user once; if none, ask how to proceed. Every rendered image's full prompt must be written to a standalone `prompts/NN-*.md` file before any backend is invoked. Backend skills (`baoyu-image-gen`, `baoyu-danger-gemini-web`) are exempt — they render directly rather than selecting a backend.
+
+### `codex-imagegen` Backend
+
+A backend for non-Codex runtimes (e.g., Claude Code) that generates images by spawning `codex exec --json --sandbox danger-full-access` and delegating to Codex CLI's built-in `image_gen` tool. Uses the user's Codex subscription — no `OPENAI_API_KEY` required.
+
+Invoke via:
+
+```bash
+./scripts/codex-imagegen.sh \
+  --image <output.png> \
+  --prompt-file prompts/01-cover.md \
+  --aspect 16:9 \
+  --cache-dir ~/.cache/baoyu-codex-imagegen
+```
+
+Stdout emits a single JSON line: `{"status":"ok","path":...,"bytes":N,...}`. On failure, `{"status":"error","error_kind":...}`. Skills route here by setting `preferred_image_backend: codex-imagegen` in EXTEND.md. Full reference: [docs/codex-imagegen-backend.md](docs/codex-imagegen-backend.md).
+
+## Release Process
+
+Use `/release-skills` workflow. Never skip:
+1. `CHANGELOG.md` + `CHANGELOG.zh.md`
+2. `marketplace.json` version bump
+3. `README.md` + `README.zh.md` if applicable
+4. All files committed together before tag
 
 ## Code Style
 
-- TypeScript throughout, no comments
-- Async/await patterns
-- Short variable names
-- Type-safe interfaces
+TypeScript, no comments, async/await, short variable names, type-safe interfaces.
+
+## Adding New Skills
+
+All skills MUST use `baoyu-` prefix. Details: [docs/creating-skills.md](docs/creating-skills.md)
+
+## Reference Docs
+
+| Topic | File |
+|-------|------|
+| Image generation output guidelines | [docs/image-generation.md](docs/image-generation.md) |
+| Image generation backend selection | [docs/image-generation-tools.md](docs/image-generation-tools.md) |
+| User input tool convention | [docs/user-input-tools.md](docs/user-input-tools.md) |
+| Chrome profile platform paths | [docs/chrome-profile.md](docs/chrome-profile.md) |
+| Comic style maintenance | [docs/comic-style-maintenance.md](docs/comic-style-maintenance.md) |
+| ClawHub/OpenClaw publishing | [docs/publishing.md](docs/publishing.md) |

docs/chrome-profile.md

@@ -0,0 +1,27 @@
+# Chrome Profile
+
+All CDP skills share a single profile directory. Do NOT create per-skill profiles.
+
+Override: `BAOYU_CHROME_PROFILE_DIR` env var (takes priority over all defaults).
+
+| Platform | Default Path |
+|----------|-------------|
+| macOS | `~/Library/Application Support/baoyu-skills/chrome-profile` |
+| Linux | `$XDG_DATA_HOME/baoyu-skills/chrome-profile` (fallback `~/.local/share/`) |
+| Windows | `%APPDATA%/baoyu-skills/chrome-profile` |
+| WSL | Windows home `/.local/share/baoyu-skills/chrome-profile` |
+
+New skills: use `BAOYU_CHROME_PROFILE_DIR` only (not per-skill env vars like `X_BROWSER_PROFILE_DIR`).
+
+## Implementation Pattern
+
+```typescript
+function getDefaultProfileDir(): string {
+  const override = process.env.BAOYU_CHROME_PROFILE_DIR?.trim();
+  if (override) return path.resolve(override);
+  const base = process.platform === 'darwin'
+    ? path.join(os.homedir(), 'Library', 'Application Support')
+    : process.env.XDG_DATA_HOME || path.join(os.homedir(), '.local', 'share');
+  return path.join(base, 'baoyu-skills', 'chrome-profile');
+}
+```

docs/codex-imagegen-backend.md

@@ -0,0 +1,256 @@
+# `codex-imagegen` Backend
+
+Generate images via Codex CLI's built-in `image_gen` tool from non-Codex runtimes (e.g., Claude Code). The wrapper spawns `codex exec --json` and lets the user's existing Codex subscription drive image generation — **no `OPENAI_API_KEY` required**.
+
+This backend implements the `preferred_image_backend: codex-imagegen` config key already referenced in several `SKILL.md` files across this repo.
+
+## Features
+
+| Feature | Status |
+|---------|--------|
+| **Reliability**: retry + exponential backoff | Default 2 retries |
+| **Verification**: confirms `image_gen` was actually invoked (not bypassed) | Checks `$CODEX_HOME/generated_images/{thread_id}/` |
+| **Verification**: PNG magic-byte sanity check | ✓ |
+| **Idempotency cache**: reuses output for same prompt+aspect+refs | `--cache-dir` |
+| **Concurrency control**: file lock prevents parallel `codex exec` collisions | Built-in |
+| **Structured logging**: JSONL log file | `--log-file` |
+| **Token usage returned** | Embedded in result JSON |
+| **`--ref` reference images** | Repeatable |
+| **Unit tests** | 16 tests (parser / cache / validator) |
+| **Error classification**: retryable vs non-retryable | 9 `error_kind` values |
+
+## Why this backend
+
+| Scenario | Conventional backend | This backend |
+|----------|---------------------|--------------|
+| You have a Codex subscription | OpenAI Images API costs add up per image | Subscription already covers it — zero marginal API cost |
+| No `OPENAI_API_KEY` available | `baoyu-image-gen` needs an API key | `codex login` is enough |
+| Want to use GPT Image 2 | Only via OpenAI API | Codex's `image_gen` *is* GPT Image 2 |
+
+## Prerequisites
+
+```bash
+npm install -g @openai/codex
+codex login            # signs in with your OpenAI account (subscription)
+codex --version        # confirm >= 0.130
+```
+
+`bun` is preferred for running the wrapper. On macOS:
+
+```bash
+brew install oven-sh/bun/bun
+```
+
+If `bun` is missing, the shell entrypoint falls back to `npx -y bun`.
+
+## Usage
+
+### Direct CLI
+
+```bash
+# Inline prompt
+./scripts/codex-imagegen.sh \
+  --image /tmp/cat.png \
+  --prompt "A friendly orange cat, watercolor"
+
+# Prompt from file
+./scripts/codex-imagegen.sh \
+  --image cover.png \
+  --prompt-file prompts/01-cover.md \
+  --aspect 16:9
+
+# Verbose mode for debugging
+./scripts/codex-imagegen.sh -v --image dog.png --prompt "A corgi" --aspect 1:1
+```
+
+On success, stdout emits a single JSON line:
+
+```json
+{"status":"ok","path":"/tmp/cat.png","bytes":2567101,"elapsed_seconds":53}
+```
+
+On failure, exit code is non-zero and stderr contains the error message.
+
+### Enabling within image skills
+
+Image-generating skills (e.g., `baoyu-cover-image`, `baoyu-article-illustrator`) already support a `preferred_image_backend` preference. To route them through this backend, set the following in the corresponding `EXTEND.md`:
+
+```yaml
+# ~/.baoyu-skills/baoyu-cover-image/EXTEND.md
+preferred_image_backend: codex-imagegen
+```
+
+When the LLM runs the skill, it reads the preference and — guided by the `### codex-imagegen Backend` section in `CLAUDE.md` — invokes `scripts/codex-imagegen.sh`.
+
+> **Note**: The integration is mediated by the LLM reading `CLAUDE.md`. It is not a hard binding. If a skill does not route to the backend automatically, mentioning it explicitly in the prompt works.
+
+## Parameters
+
+| Flag | Required | Description |
+|------|----------|-------------|
+| `--image <path>` | ✓ | Output PNG path (absolute recommended; relative paths are resolved against cwd) |
+| `--prompt <text>` | one of | Prompt string (mutually exclusive with `--prompt-file`) |
+| `--prompt-file <path>` | one of | Read prompt from file (mutually exclusive with `--prompt`) |
+| `--aspect <ratio>` | | Aspect ratio. Default `1:1`. Common: `16:9`, `9:16`, `4:3`, `2.35:1` |
+| `--ref <file>` | | Reference image path (repeatable) |
+| `--timeout <ms>` | | `codex exec` timeout in ms. Default `300000` |
+| `--retries <n>` | | Retry count on retryable errors. Default `2` (total attempts = retries + 1) |
+| `--retry-delay <ms>` | | Base delay between retries (exponential backoff). Default `1500` |
+| `--cache-dir <path>` | | Enable idempotency cache (reuses output for same prompt+aspect+refs) |
+| `--log-file <path>` | | Structured JSONL log path (appended) |
+| `-v` / `--verbose` | | Mirror log entries to stderr |
+| `-h` / `--help` | | Show usage |
+
+## Structured Output
+
+On success, stdout contains a single JSON line:
+
+```json
+{
+  "status": "ok",
+  "path": "/tmp/owl.png",
+  "bytes": 1693831,
+  "elapsed_seconds": 87,
+  "thread_id": "019e40e8-daef-7c60-943d-5e7bb3f6cb3d",
+  "attempts": 1,
+  "cached": false,
+  "usage": {
+    "input": 110899,
+    "cached_input": 83456,
+    "output": 457,
+    "reasoning": 47
+  },
+  "tool_calls": [
+    {"tool": "shell", "status": "completed"},
+    {"tool": "agent_message", "status": "completed"}
+  ]
+}
+```
+
+Cache hits return with `elapsed_seconds: 0`, `cached: true`, `attempts: 0`.
+
+On failure, exit code is `1` and the JSON contains `error` and `error_kind`:
+
+```json
+{
+  "status": "error",
+  "error": "image_gen was not invoked: no PNG in ...",
+  "error_kind": "no_image_gen_tool_use"
+}
+```
+
+## Error Kinds
+
+| `error_kind` | Retryable | Meaning |
+|--------------|-----------|---------|
+| `codex_not_installed` | ✗ | `codex` CLI not found |
+| `invalid_args` | ✗ | Argument parsing error |
+| `prompt_file_missing` | ✗ | `--prompt-file` path does not exist |
+| `spawn_failed` | ✓ | `codex exec` exited non-zero |
+| `timeout` | ✓ | Exceeded `--timeout` |
+| `no_image_gen_tool_use` | ✓ | Agent did not invoke `image_gen` (it took another path) |
+| `output_missing` | ✓ | Output file not created |
+| `invalid_png` | ✓ | Output is not a valid PNG |
+| `agent_refused` | ✓ | No `thread_id` in event stream (Codex refused to respond) |
+| `lock_busy` | ✗ | Concurrency lock acquisition timed out |
+
+## Measured Performance
+
+| Metric | Value |
+|--------|-------|
+| First-run latency | 50–90 s |
+| Cache-hit latency | < 0.3 s |
+| Output dimensions | 1024×1024, 1672×941 (16:9), etc. — chosen by `image_gen` |
+| Output format | PNG (RGB, 8-bit) |
+| Token usage per call | ~110k input (~80k cached) + ~500 output |
+| Quota source | Codex subscription (does not consume OpenAI API quota) |
+| Default timeout | 300 s (5 min) |
+
+## Limitations & Risks
+
+1. **5–10× slower than direct API**. `codex exec` cold-starts the agent, loads the built-in `image_gen` SKILL.md, and runs reasoning before invoking the tool. Cache hits avoid this for repeated prompts.
+2. **ToS gray area**. Codex's `image_gen` tool is designed for interactive use. Invoking it programmatically via `codex exec` from an external agent is not explicitly addressed by current OpenAI policies. Suggested guardrails:
+   - Personal, low-volume use is reasonable.
+   - Not recommended for production automation or high-volume batch jobs.
+   - Users are responsible for ensuring their usage complies with applicable terms of service.
+3. **Sandbox permissions**. The wrapper passes `--sandbox danger-full-access` so the spawned agent can move the rendered PNG out of `$CODEX_HOME/generated_images/`. This is necessary because the agent must `cp`/`mv` the file to the user-specified output path.
+4. **Concurrency = 1**. The file lock serializes concurrent invocations to avoid `codex exec` collisions. Parallel calls queue.
+
+## Troubleshooting
+
+| Symptom | `error_kind` | Resolution |
+|---------|--------------|------------|
+| `command not found: codex` | `codex_not_installed` | `npm install -g @openai/codex` |
+| `codex exec` fails | `spawn_failed` | Check `codex login` status; inspect `raw_log` path |
+| Timeout | `timeout` | Pass `--timeout 600000` (10 min) for slow networks |
+| Agent skipped `image_gen` | `no_image_gen_tool_use` | Auto-retries; consider sharpening the prompt — abstract prompts let the agent wander |
+| Output missing | `output_missing` | Agent did not `cp` to the target path; check `raw_log` for the actual save location under `generated_images/` |
+| Lock held | `lock_busy` | Wait for the in-flight request to finish; or `rm ~/.cache/baoyu-codex-imagegen/codex-exec.lock` |
+| Low image quality | — | Sharpen the prompt, try a different aspect, or supply `--ref` |
+
+## Architecture
+
+```
+scripts/codex-imagegen.sh        # thin bash entrypoint
+scripts/codex-imagegen/
+├── main.ts        # parseArgs → cache → lock → retry loop → emit JSON
+├── types.ts       # CliOptions, GenerateResult, GenError, ErrorKind
+├── spawn.ts       # spawn codex exec --json --sandbox danger-full-access
+├── parser.ts      # parse JSONL event stream → toolCalls, usage, thread_id
+├── validator.ts   # verify image_gen invocation + PNG magic + file size
+├── cache.ts       # cacheKey(sha256), FileLock, lookup/store
+├── logger.ts      # JsonLogger (verbose stderr + JSONL file)
+├── parser.test.ts
+├── cache.test.ts
+└── validator.test.ts
+```
+
+Run tests:
+
+```bash
+cd scripts/codex-imagegen && bun test
+```
+
+## Internal Flow
+
+```mermaid
+flowchart LR
+    CC[Claude Code / any caller]
+    WRAPPER[scripts/codex-imagegen.sh]
+    CODEX["codex exec --json<br/>--sandbox danger-full-access"]
+    AGENT[Codex agent]
+    TOOL[image_gen built-in tool]
+    DEFAULT["$CODEX_HOME/<br/>generated_images/{thread_id}/"]
+    OUT[/specified OUTPUT path/]
+
+    CC -->|exec wrapper| WRAPPER
+    WRAPPER -->|stdin: instruction| CODEX
+    CODEX --> AGENT
+    AGENT -->|tool call| TOOL
+    TOOL -->|writes file| DEFAULT
+    AGENT -->|agent cp/mv| OUT
+    WRAPPER -->|verify + parse| CC
+
+    classDef cc fill:#1e40af,color:#fff,stroke:#93c5fd
+    classDef cdx fill:#7c2d12,color:#fff,stroke:#fdba74
+    class CC,WRAPPER cc
+    class CODEX,AGENT,TOOL cdx
+```
+
+## Design Decisions
+
+1. **Bash entrypoint + TypeScript implementation** — the shell wrapper picks the runtime (`bun` preferred, falling back to `npx -y bun`); TypeScript handles the orchestration, parsing, retry, cache, and logging. This mirrors the project's existing `scripts/*.mjs` and `skills/<skill>/scripts/main.ts` pattern.
+2. **`--sandbox danger-full-access`** — necessary so the spawned agent can `cp`/`mv` the rendered PNG out of `$CODEX_HOME/generated_images/` to the user-specified path. Standard sandboxes block this.
+3. **Parse the JSONL event stream** — the final `agent_message` and intermediate `command_execution` events let the wrapper verify what actually happened (was `image_gen` called? did `cp` reach the right destination?), which is far more reliable than scraping freeform stdout.
+4. **Infrastructure, not a skill** — this backend is a CLI utility that skills route to via `preferred_image_backend`. It belongs in `scripts/`, not `skills/`, because it has no `SKILL.md` and is never loaded directly by an agent.
+5. **File lock instead of internal queue** — keeps the implementation small and works across multiple shell sessions or processes invoking the same wrapper concurrently.
+
+## Related Files
+
+| File | Role |
+|------|------|
+| `scripts/codex-imagegen.sh` | CLI entrypoint |
+| `scripts/codex-imagegen/` | TypeScript implementation |
+| `docs/codex-imagegen-backend.md` | This document |
+| `CLAUDE.md` | Tells LLMs how to invoke this backend |
+| `.github/workflows/codex-imagegen-tests.yml` | CI unit tests |

docs/comic-style-maintenance.md

@@ -0,0 +1,35 @@
+# Style Maintenance (baoyu-comic)
+
+## Adding a New Style
+
+1. Create style definition: `skills/baoyu-comic/references/styles/<style-name>.md`
+2. Update SKILL.md: add to `--style` options table + auto-selection entry
+3. Generate showcase image:
+   ```bash
+   ${BUN_X} skills/baoyu-danger-gemini-web/scripts/main.ts \
+     --prompt "A single comic book page in <style-name> style showing [scene]. Features: [characteristics]. 3:4 portrait aspect ratio comic page." \
+     --image screenshots/comic-styles/<style-name>.png
+   ```
+4. Compress: `${BUN_X} skills/baoyu-compress-image/scripts/main.ts screenshots/comic-styles/<style-name>.png`
+5. Update both READMEs (`README.md` + `README.zh.md`): add style to options, description table, preview grid
+
+## Updating an Existing Style
+
+1. Update style definition in `references/styles/`
+2. Regenerate showcase image if visual characteristics changed (steps 3-4 above)
+3. Update READMEs if description changed
+
+## Deleting a Style
+
+1. Delete style definition + showcase image (`.webp`)
+2. Remove from SKILL.md `--style` options + auto-selection
+3. Remove from both READMEs (options, description table, preview grid)
+
+## Style Preview Grid Format
+
+```markdown
+| | | |
+|:---:|:---:|:---:|
+| ![style1](./screenshots/comic-styles/style1.webp) | ![style2](./screenshots/comic-styles/style2.webp) | ![style3](./screenshots/comic-styles/style3.webp) |
+| style1 | style2 | style3 |
+```

docs/creating-skills.md

@@ -0,0 +1,175 @@
+# Creating New Skills
+
+**REQUIRED READING**: [Skill authoring best practices](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices)
+
+## Key Requirements
+
+| Requirement | Details |
+|-------------|---------|
+| **Prefix** | All skills MUST use `baoyu-` prefix |
+| **name field** | Max 64 chars, lowercase letters/numbers/hyphens only, no "anthropic"/"claude" |
+| **description** | Max 1024 chars, third person, include what + when to use |
+| **SKILL.md body** | Keep under 500 lines; use `references/` for additional content |
+| **References** | One level deep from SKILL.md; avoid nested references |
+
+## SKILL.md Frontmatter Template
+
+```yaml
+---
+name: baoyu-<name>
+description: <Third-person description. What it does + when to use it.>
+version: <semver matching marketplace.json>
+metadata:
+  openclaw:
+    homepage: https://github.com/JimLiu/baoyu-skills#baoyu-<name>
+    requires:          # include only if skill has scripts
+      anyBins:
+        - bun
+        - npx
+---
+```
+
+## Steps
+
+1. Create `skills/baoyu-<name>/SKILL.md` with YAML front matter
+2. Add TypeScript in `skills/baoyu-<name>/scripts/` (if applicable)
+3. Add prompt templates in `skills/baoyu-<name>/prompts/` if needed
+4. Register the skill in `.claude-plugin/marketplace.json` under the `baoyu-skills` plugin entry
+5. Add Script Directory section to SKILL.md if skill has scripts
+6. Add openclaw metadata to frontmatter
+
+## Skill Grouping
+
+All skills are registered under the single `baoyu-skills` plugin. Use these logical groups when deciding where the skill should appear in the docs:
+
+| If your skill... | Use group |
+|------------------|-----------|
+| Generates visual content (images, slides, comics) | Content Skills |
+| Publishes to platforms (X, WeChat, Weibo) | Content Skills |
+| Provides AI generation backend | AI Generation Skills |
+| Converts or processes content | Utility Skills |
+
+If you add a new logical group, update the docs that present grouped skills, but keep the skill registered under the single `baoyu-skills` plugin entry.
+
+## Writing Descriptions
+
+**MUST write in third person**:
+
+```yaml
+# Good
+description: Generates Xiaohongshu infographic series from content. Use when user asks for "小红书图片", "XHS images".
+
+# Bad
+description: I can help you create Xiaohongshu images
+```
+
+## Script Directory Template
+
+Every SKILL.md with scripts MUST include:
+
+```markdown
+## Script Directory
+
+**Important**: All scripts are located in the `scripts/` subdirectory of this skill.
+
+**Agent Execution Instructions**:
+1. Determine this SKILL.md file's directory path as `{baseDir}`
+2. Script path = `{baseDir}/scripts/<script-name>.ts`
+3. Resolve `${BUN_X}` runtime: if `bun` installed → `bun`; if `npx` available → `npx -y bun`; else suggest installing bun
+4. Replace all `{baseDir}` and `${BUN_X}` in this document with actual values
+
+**Script Reference**:
+| Script | Purpose |
+|--------|---------|
+| `scripts/main.ts` | Main entry point |
+```
+
+## Progressive Disclosure
+
+For skills with extensive content:
+
+```
+skills/baoyu-example/
+├── SKILL.md              # Main instructions (<500 lines)
+├── references/
+│   ├── styles.md         # Loaded as needed
+│   └── examples.md       # Loaded as needed
+└── scripts/
+    └── main.ts
+```
+
+Link from SKILL.md (one level deep only):
+```markdown
+**Available styles**: See [references/styles.md](references/styles.md)
+```
+
+## Extension Support (EXTEND.md)
+
+Every SKILL.md MUST include EXTEND.md loading. Add as Step 1.1 (workflow skills) or "Preferences" section (utility skills):
+
+```markdown
+**1.1 Load Preferences (EXTEND.md)**
+
+Check EXTEND.md existence (priority order):
+
+\`\`\`bash
+test -f .baoyu-skills/<skill-name>/EXTEND.md && echo "project"
+test -f "${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/<skill-name>/EXTEND.md" && echo "xdg"
+test -f "$HOME/.baoyu-skills/<skill-name>/EXTEND.md" && echo "user"
+\`\`\`
+
+| Path | Location |
+|------|----------|
+| `.baoyu-skills/<skill-name>/EXTEND.md` | Project directory |
+| `$XDG_CONFIG_HOME/baoyu-skills/<skill-name>/EXTEND.md` | XDG config (~/.config) |
+| `$HOME/.baoyu-skills/<skill-name>/EXTEND.md` | User home (legacy) |
+
+| Result | Action |
+|--------|--------|
+| Found | Read, parse, display summary |
+| Not found | Ask user via the runtime's user-input tool (see [user-input-tools.md](user-input-tools.md)) |
+```
+
+End of SKILL.md should include:
+```markdown
+## Extension Support
+Custom configurations via EXTEND.md. See **Step 1.1** for paths and supported options.
+```
+
+## User Input Tools Section (Required)
+
+Every SKILL.md that prompts the user for choices MUST include exactly one `## User Input Tools` section near the top (right after the intro, before the main workflow). The rule must be **inlined** — do NOT link to `docs/user-input-tools.md` (skills are self-contained; see [CLAUDE.md → Skill Self-Containment](../CLAUDE.md)). The author-side canonical reference lives at [user-input-tools.md](user-input-tools.md); copy its body into each new SKILL.md.
+
+Standard snippet (copy verbatim):
+
+```markdown
+## User Input Tools
+
+When this skill prompts the user, follow this tool-selection rule (priority order):
+
+1. **Prefer built-in user-input tools** exposed by the current agent runtime — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.
+2. **Fallback**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
+3. **Batching**: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.
+
+Concrete `AskUserQuestion` references below are examples — substitute the local equivalent in other runtimes.
+```
+
+## Image Generation Tools Section (Required for image-gen skills)
+
+Every SKILL.md that renders images — whether by calling an image-generation API directly or by delegating to another skill — MUST include exactly one `## Image Generation Tools` section near the top (after `## User Input Tools`, before the main workflow). The rule must be **inlined** — do NOT link to `docs/image-generation-tools.md` (skills are self-contained; see [CLAUDE.md → Skill Self-Containment](../CLAUDE.md)). The author-side canonical reference lives at [image-generation-tools.md](image-generation-tools.md); copy its body into each new SKILL.md.
+
+Standard snippet (copy verbatim):
+
+```markdown
+## Image Generation Tools
+
+When this skill needs to render an image:
+
+- **Use whatever image-generation tool or skill is available** in the current runtime — e.g., Codex `imagegen`, Hermes `image_generate`, `baoyu-image-gen`, or any equivalent the user has installed.
+- **If multiple are available**, ask the user **once** at the start which to use (batch with any other initial questions).
+- **If none are available**, tell the user and ask how to proceed.
+
+**Prompt file requirement (hard)**: write each image's full, final prompt to a standalone file under `prompts/` (naming: `NN-{type}-[slug].md`) BEFORE invoking any backend. The backend receives the prompt file (or its content); the file is the reproducibility record and lets you switch backends without regenerating prompts.
+
+Concrete tool names (`imagegen`, `image_generate`, `baoyu-image-gen`) above are examples — substitute the local equivalents under the same rule.
+```

docs/image-generation-tools.md

@@ -0,0 +1,48 @@
+# Image Generation Tools
+
+Skills in this repo are loaded by multiple agent runtimes (Claude Code, Codex, Hermes, other agents, bare CLI). Each runtime exposes a different image-generation capability — some have a runtime-native tool (Codex `imagegen`, Hermes `image_generate`), others rely on an installed skill (`baoyu-image-gen`, or user-defined). This document defines the canonical **backend-selection rule** every skill that renders images follows so skills stay portable.
+
+## The Rule
+
+When a skill needs to render an image, resolve the backend in this order:
+
+1. **Current-request override** — if the user names a specific backend in the current message, use it.
+2. **Saved preference** — if the skill's `EXTEND.md` sets `preferred_image_backend` to a backend available right now, use it.
+3. **Auto-select** (when the preference is `auto`, unset, or the pinned backend isn't available):
+   - **Codex (`imagegen`)** — first, inspect your available-skills / tool inventory. If a skill named `imagegen` is listed, you are running inside Codex and MUST use it: invoke via the `Skill` tool with `skill: "imagegen"`, passing the saved prompt file's content (plus output path and aspect ratio per Codex `imagegen`'s own args). Codex `imagegen` is the official raster backend in that runtime and outranks any non-native skill (e.g., `baoyu-image-gen`) unless the user has explicitly pinned a different `preferred_image_backend`.
+   - **Other runtime-native tools** — if the runtime exposes a different native image tool (e.g., Hermes `image_generate`), use it the same way.
+   - Otherwise, if exactly one non-native backend is installed (e.g., `baoyu-image-gen`), use it.
+   - Otherwise (multiple non-native backends with no runtime-native tool), ask the user once — batch with any other initial questions.
+4. **If none are available**, tell the user and ask how to proceed.
+
+**⛔ Never substitute SVG, HTML, canvas, or other code-based rendering for raster image generation.** Codex `imagegen`'s own description says it should be used "when the output should be a bitmap asset rather than repo-native code or vector." If you cannot resolve a raster backend via step 3, fall through to step 4 and ask the user — do **not** silently emit SVG, write inline `<svg>` markup, or produce HTML/CSS art as a substitute. This applies even if the article/section seems "diagram-like": the consumer skill calling this rule has already decided that a raster image is what it needs.
+
+Setting `preferred_image_backend: ask` forces the step-3 prompt every run regardless of available backends.
+
+## The Preference Field
+
+Each image-consuming skill's `EXTEND.md` carries a single `preferred_image_backend` field:
+
+| Value | Meaning |
+|---|---|
+| `auto` (default) | Apply the auto-select rule — runtime-native preferred, fall back to only installed backend, ask if multiple non-native. |
+| `ask` | Always confirm the backend on every run, even when a runtime-native tool exists. |
+| `<backend-id>` (e.g., `codex-imagegen`, `baoyu-image-gen`, `image_generate`) | Pin this backend when available; fall back to `auto` if it isn't. |
+
+The field is **absent-equals-auto**: older `EXTEND.md` files without this field behave exactly as if `preferred_image_backend: auto` were set. No schema version bump is needed to introduce it.
+
+## Prompt File Requirement (hard)
+
+Regardless of which backend is chosen, every skill that renders images MUST write each image's full, final prompt to a standalone file under `prompts/` (naming: `NN-{type}-[slug].md`) BEFORE invoking any backend. The backend receives the prompt file (or its content); the file is the reproducibility record and allows switching backends without regenerating prompts.
+
+## How Skills Declare This
+
+Each `SKILL.md` that renders images includes **exactly one** `## Image Generation Tools` section (near the top, after `## User Input Tools` and before the main workflow) that **inlines** this rule. Skills are self-contained and cannot link to `docs/` — each skill folder must ship the rule inside its own `SKILL.md`. See [CLAUDE.md → Skill Self-Containment](../CLAUDE.md).
+
+Each skill's `references/config/preferences-schema.md` (and its `EXTEND.md` template in `first-time-setup.md`) lists `preferred_image_backend` alongside other preference fields. First-time setup does NOT ask the user about the backend — `auto` is set silently. Users who want to pin a specific backend edit `EXTEND.md` later, and each skill's `## Changing Preferences` section documents the common one-line edits.
+
+Concrete tool names (`imagegen`, `image_generate`, `baoyu-image-gen`) in this document and in SKILL.md are **examples** — agents in other runtimes apply the rule above and substitute the local equivalent. Skill-specific parameters for these backends are illustrative; runtimes without those knobs can omit them.
+
+## Backend Skills Are Exempt
+
+Skills that **are themselves** image-generation backends — currently `baoyu-image-gen`, `baoyu-image-gen` (deprecated), and `baoyu-danger-gemini-web` — do NOT include a `## Image Generation Tools` section. They render directly via their own provider integrations and have no need to "select a backend." The rule applies only to consumer skills that delegate rendering to whatever backend the runtime exposes.

docs/image-generation.md

@@ -0,0 +1,54 @@
+# Image Generation Guidelines
+
+Skills that require image generation MUST delegate to available image generation tools (runtime-native tools or installed skills).
+
+**Backend selection convention**: see [image-generation-tools.md](image-generation-tools.md) for the runtime-neutral rule. Short version: use whatever backend is available; if multiple, ask the user once; if none, ask how to proceed. This document covers output conventions (naming, paths) that apply regardless of which backend is selected.
+
+## Skill Selection
+
+1. Follow the rule in [image-generation-tools.md](image-generation-tools.md): use whatever backend is available; ask only on ambiguity.
+2. Read the chosen backend's documentation for parameters and capabilities.
+3. If user requests a specific backend, honor it.
+
+## Generation Flow Template
+
+```markdown
+### Step N: Generate Images
+
+**Backend Selection**:
+1. Detect available image-generation tools/skills (runtime-native + installed)
+2. If one available → use it. If multiple → ask user once. If none → ask how to proceed.
+3. Read the chosen backend's docs for parameters
+
+**Generation Flow**:
+1. Write the full prompt to `prompts/NN-{type}-[slug].md` BEFORE invoking the backend
+2. Call backend with the prompt (or prompt file), output path, and parameters
+3. Generate sequentially by default (batch parallel only when backend supports it and user has multiple prompts)
+4. Output progress: "Generated X/N"
+5. On failure, auto-retry once before reporting error
+```
+
+**Batch Parallel** (`baoyu-image-gen` only): concurrent workers with per-provider throttling via `batch.max_workers` in EXTEND.md.
+
+## Output Path Convention
+
+**Output Directory**: `<skill-suffix>/<topic-slug>/`
+- `<skill-suffix>`: e.g., `xhs-images`, `cover-image`, `slide-deck`, `comic`
+- `<topic-slug>`: 2-4 words, kebab-case from content topic
+- Conflict: append timestamp `<topic-slug>-YYYYMMDD-HHMMSS`
+
+**Source Files**: Copy to output dir as `source-{slug}.{ext}`
+
+## Image Naming Convention
+
+**Format**: `NN-{type}-[slug].png`
+- `NN`: Two-digit sequence (01, 02, ...)
+- `{type}`: cover, content, page, slide, illustration, etc.
+- `[slug]`: 2-5 word kebab-case descriptor, unique within directory
+
+Examples:
+```
+01-cover-ai-future.png
+02-content-key-benefits.png
+03-slide-architecture-overview.png
+```

docs/publishing.md

@@ -0,0 +1,48 @@
+# ClawHub / OpenClaw Publishing
+
+## OpenClaw Metadata
+
+Skills include `metadata.openclaw` in YAML front matter:
+
+```yaml
+metadata:
+  openclaw:
+    homepage: https://github.com/JimLiu/baoyu-skills#<skill-name>
+    requires:          # only for skills with scripts
+      anyBins:
+        - bun
+        - npx
+```
+
+## Publishing Commands
+
+```bash
+bash scripts/sync-clawhub.sh           # sync all skills
+bash scripts/sync-clawhub.sh <skill>   # sync one skill
+```
+
+Release hooks are configured via `.releaserc.yml`. This repo does not stage a separate release directory: publish reads the skill directory directly and validates that local package references and CLI bin targets are self-contained.
+
+Every skill release must keep the `version:` in that skill's `SKILL.md` aligned with the version being published. `publish-skill.mjs` and `sync-clawhub.mjs` both reject mismatches so a registry payload cannot ship with stale skill metadata.
+
+Commits that touch `skills/<name>/**` must use Conventional Commit subjects, for example `fix(baoyu-post-to-wechat): handle WeChat editor focus`. CI runs `npm run verify:skill-release-commits` against the pushed or PR commit range so bare subjects like `Fix WeChat browser article publishing` cannot bypass per-skill release versioning silently.
+
+## Shared Workspace Packages
+
+`packages/` is the source of truth for shared runtime code. Most skills consume shared packages from npm with semver ranges. `baoyu-url-to-markdown` is the exception: it vendors the `baoyu-fetch` runtime into `skills/baoyu-url-to-markdown/scripts/lib/` so the published skill is self-contained and does not depend on the `baoyu-fetch` npm package.
+
+Current packages:
+- `baoyu-chrome-cdp` (Chrome CDP utilities), consumed by 5 skills (`baoyu-danger-gemini-web`, `baoyu-danger-x-to-markdown`, `baoyu-post-to-wechat`, `baoyu-post-to-weibo`, `baoyu-post-to-x`)
+- `baoyu-md` (shared Markdown rendering and placeholder pipeline), consumed by 3 skills (`baoyu-markdown-to-html`, `baoyu-post-to-wechat`, `baoyu-post-to-weibo`)
+- `baoyu-fetch` (URL-to-Markdown CLI), vendored into 1 skill (`baoyu-url-to-markdown`)
+
+**How it works**: npm packages are built from `packages/` and published to the public npm registry. Skills normally depend on those packages with `^<version>` specs. Release prep runs `node scripts/verify-shared-package-deps.mjs` so accidental `file:` dependencies cannot slip back in. For vendored skill runtimes, keep the copied code under the skill directory and run `node scripts/publish-skill.mjs --skill-dir <skill> --version <version> --dry-run` before publishing.
+
+**Update workflow**:
+1. Edit package under `packages/`
+2. Run the package build, e.g. `bun run --cwd packages/baoyu-md build`
+3. Publish the changed npm package with `npm publish --access public`
+4. Update consuming skill `package.json` semver ranges if the package version changed
+5. Run `node scripts/verify-shared-package-deps.mjs`
+
+**Git hook**: Run `node scripts/install-git-hooks.mjs` once to enable the `pre-push` hook. It blocks pushes when a skill uses a local `file:` dependency or a vendored workspace package.

docs/testing.md

@@ -0,0 +1,73 @@
+# Testing Strategy
+
+This repository has many scripts, but they do not share a single runtime or dependency graph. The lowest-risk testing strategy is to start from stable Node-based library code, then expand outward to CLI and skill-specific smoke tests.
+
+## Current Baseline
+
+- Root test runner: `node:test`
+- Entry point: `npm test`
+- Coverage command: `npm run test:coverage`
+- CI trigger: GitHub Actions on `push`, `pull_request`, and manual dispatch
+
+This avoids introducing Jest/Vitest across a repo that already mixes plain Node scripts, Bun-based skill packages, npm-published shared packages, and browser automation.
+
+## Rollout Plan
+
+### Phase 1: Stable library coverage
+
+Focus on pure functions under `scripts/lib/` first.
+
+- `scripts/lib/release-files.mjs`
+- `scripts/verify-shared-package-deps.mjs`
+
+Goals:
+
+- Validate file filtering and release packaging rules
+- Catch regressions that reintroduce local `file:` dependencies or vendored workspace packages
+- Keep tests deterministic and free of network, Bun, or browser requirements
+
+### Phase 2: Root CLI integration tests
+
+Add temp-directory integration tests for root CLIs that already support dry-run or local-only flows.
+
+- `scripts/verify-shared-package-deps.mjs`
+- `scripts/publish-skill.mjs --dry-run`
+- `scripts/sync-clawhub.mjs` argument handling and local skill discovery
+
+Goals:
+
+- Assert exit codes and stdout for common flows
+- Cover CLI argument parsing without hitting external services
+
+### Phase 3: Skill script smoke tests
+
+Add opt-in smoke tests for selected `skills/*/scripts/` packages, starting with those that:
+
+- accept local input files
+- have deterministic output
+- do not require authenticated browser sessions
+
+Examples:
+
+- markdown transforms
+- file conversion helpers
+- local content analyzers
+
+Keep browser automation, login flows, and live API publishing scripts outside the default CI path unless they are explicitly mocked.
+
+### Phase 4: Coverage gates
+
+After the stable Node path has enough breadth, add coverage thresholds in CI for the tested root modules.
+
+Recommended order:
+
+1. Start with reporting only
+2. Add line/function thresholds for `scripts/lib/**`
+3. Expand include patterns once skill-level smoke tests are reliable
+
+## Conventions For New Tests
+
+- Prefer temp directories over committed fixtures unless the fixture is reused heavily
+- Test exported functions before testing CLI wrappers
+- Avoid network, browser, and credential dependencies in default CI
+- Keep tests isolated so they can run with plain `node --test`

docs/user-input-tools.md

@@ -0,0 +1,17 @@
+# User Input Tools
+
+Skills in this repo are loaded by multiple agent runtimes (Claude Code, other agents, bare CLI). Each runtime exposes a different API for asking the user questions. This document defines the canonical **tool-selection rule** every skill follows so skills stay portable.
+
+## Tool Selection (priority order)
+
+1. **Prefer built-in user-input tools** if the current agent runtime exposes one — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.
+2. **Fallback to plain text**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
+3. **Batching rule**:
+   - If the tool supports **multiple questions per call** (e.g., `AskUserQuestion`): **Combine all applicable questions into a single call. Do NOT split into separate calls.**
+   - If the tool supports **only one question per call** (e.g., single-prompt `clarify`): ask **one question per call, in priority order**.
+
+## How Skills Declare This
+
+Each `SKILL.md` that uses interactive user input includes **exactly one** `## User Input Tools` section (typically near the top, right after the intro) that **inlines** this rule. Do NOT link here from a SKILL.md — skills are self-contained (see [CLAUDE.md → Skill Self-Containment](../CLAUDE.md)). This document is the author-side canonical source; copy its body into each SKILL.md. The rule then governs every user-input interaction in that skill and its `references/` files.
+
+Specific mentions of a concrete tool (e.g., `AskUserQuestion`) elsewhere in a skill are **concrete examples** — agents in other runtimes apply the rule above and substitute the local equivalent. Tool-specific parameters (e.g., `header:`, `multiSelect:`) are illustrative; runtimes without those knobs can omit them.

package.json

@@ -0,0 +1,28 @@
+{
+  "name": "baoyu-skills",
+  "private": true,
+  "type": "module",
+  "workspaces": [
+    "packages/*"
+  ],
+  "scripts": {
+    "test": "node ./scripts/run-node-tests.mjs",
+    "test:coverage": "node ./scripts/run-node-tests.mjs --experimental-test-coverage",
+    "verify:skill-release-commits": "node ./scripts/verify-skill-release-commits.mjs"
+  },
+  "devDependencies": {
+    "@mozilla/readability": "^0.6.0",
+    "linkedom": "^0.18.12",
+    "turndown": "^7.2.2",
+    "turndown-plugin-gfm": "^1.0.2",
+    "tsx": "^4.20.5"
+  },
+  "dependencies": {
+    "pdf-lib": "^1.17.1",
+    "pptxgenjs": "^4.0.1",
+    "sharp": "^0.34.5"
+  },
+  "overrides": {
+    "@xmldom/xmldom": "0.8.13"
+  }
+}

packages/baoyu-chrome-cdp/dist/index.cjs

@@ -0,0 +1,479 @@
+var __create = Object.create;
+var __getProtoOf = Object.getPrototypeOf;
+var __defProp = Object.defineProperty;
+var __getOwnPropNames = Object.getOwnPropertyNames;
+var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
+var __hasOwnProp = Object.prototype.hasOwnProperty;
+function __accessProp(key) {
+  return this[key];
+}
+var __toESMCache_node;
+var __toESMCache_esm;
+var __toESM = (mod, isNodeMode, target) => {
+  var canCache = mod != null && typeof mod === "object";
+  if (canCache) {
+    var cache = isNodeMode ? __toESMCache_node ??= new WeakMap : __toESMCache_esm ??= new WeakMap;
+    var cached = cache.get(mod);
+    if (cached)
+      return cached;
+  }
+  target = mod != null ? __create(__getProtoOf(mod)) : {};
+  const to = isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target;
+  for (let key of __getOwnPropNames(mod))
+    if (!__hasOwnProp.call(to, key))
+      __defProp(to, key, {
+        get: __accessProp.bind(mod, key),
+        enumerable: true
+      });
+  if (canCache)
+    cache.set(mod, to);
+  return to;
+};
+var __toCommonJS = (from) => {
+  var entry = (__moduleCache ??= new WeakMap).get(from), desc;
+  if (entry)
+    return entry;
+  entry = __defProp({}, "__esModule", { value: true });
+  if (from && typeof from === "object" || typeof from === "function") {
+    for (var key of __getOwnPropNames(from))
+      if (!__hasOwnProp.call(entry, key))
+        __defProp(entry, key, {
+          get: __accessProp.bind(from, key),
+          enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable
+        });
+  }
+  __moduleCache.set(from, entry);
+  return entry;
+};
+var __moduleCache;
+var __returnValue = (v) => v;
+function __exportSetter(name, newValue) {
+  this[name] = __returnValue.bind(null, newValue);
+}
+var __export = (target, all) => {
+  for (var name in all)
+    __defProp(target, name, {
+      get: all[name],
+      enumerable: true,
+      configurable: true,
+      set: __exportSetter.bind(all, name)
+    });
+};
+
+// src/index.ts
+var exports_src = {};
+__export(exports_src, {
+  waitForChromeDebugPort: () => waitForChromeDebugPort,
+  sleep: () => sleep,
+  resolveSharedChromeProfileDir: () => resolveSharedChromeProfileDir,
+  openPageSession: () => openPageSession,
+  launchChrome: () => launchChrome,
+  killChrome: () => killChrome,
+  gracefulKillChrome: () => gracefulKillChrome,
+  getFreePort: () => getFreePort,
+  getDefaultChromeUserDataDirs: () => getDefaultChromeUserDataDirs,
+  findExistingChromeDebugPort: () => findExistingChromeDebugPort,
+  findChromeExecutable: () => findChromeExecutable,
+  discoverRunningChromeDebugPort: () => discoverRunningChromeDebugPort,
+  CdpConnection: () => CdpConnection
+});
+module.exports = __toCommonJS(exports_src);
+var import_node_child_process = require("node:child_process");
+var import_node_fs = __toESM(require("node:fs"));
+var import_node_net = __toESM(require("node:net"));
+var import_node_os = __toESM(require("node:os"));
+var import_node_path = __toESM(require("node:path"));
+var import_node_process = __toESM(require("node:process"));
+function sleep(ms) {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+async function getFreePort(fixedEnvName) {
+  const fixed = fixedEnvName ? Number.parseInt(import_node_process.default.env[fixedEnvName] ?? "", 10) : NaN;
+  if (Number.isInteger(fixed) && fixed > 0)
+    return fixed;
+  return await new Promise((resolve, reject) => {
+    const server = import_node_net.default.createServer();
+    server.unref();
+    server.on("error", reject);
+    server.listen(0, "127.0.0.1", () => {
+      const address = server.address();
+      if (!address || typeof address === "string") {
+        server.close(() => reject(new Error("Unable to allocate a free TCP port.")));
+        return;
+      }
+      const port = address.port;
+      server.close((err) => {
+        if (err)
+          reject(err);
+        else
+          resolve(port);
+      });
+    });
+  });
+}
+function findChromeExecutable(options) {
+  for (const envName of options.envNames ?? []) {
+    const override = import_node_process.default.env[envName]?.trim();
+    if (override && import_node_fs.default.existsSync(override))
+      return override;
+  }
+  const candidates = import_node_process.default.platform === "darwin" ? options.candidates.darwin ?? options.candidates.default : import_node_process.default.platform === "win32" ? options.candidates.win32 ?? options.candidates.default : options.candidates.default;
+  for (const candidate of candidates) {
+    if (import_node_fs.default.existsSync(candidate))
+      return candidate;
+  }
+  return;
+}
+function resolveSharedChromeProfileDir(options = {}) {
+  for (const envName of options.envNames ?? []) {
+    const override = import_node_process.default.env[envName]?.trim();
+    if (override)
+      return import_node_path.default.resolve(override);
+  }
+  const appDataDirName = options.appDataDirName ?? "baoyu-skills";
+  const profileDirName = options.profileDirName ?? "chrome-profile";
+  if (options.wslWindowsHome) {
+    return import_node_path.default.join(options.wslWindowsHome, ".local", "share", appDataDirName, profileDirName);
+  }
+  const base = import_node_process.default.platform === "darwin" ? import_node_path.default.join(import_node_os.default.homedir(), "Library", "Application Support") : import_node_process.default.platform === "win32" ? import_node_process.default.env.APPDATA ?? import_node_path.default.join(import_node_os.default.homedir(), "AppData", "Roaming") : import_node_process.default.env.XDG_DATA_HOME ?? import_node_path.default.join(import_node_os.default.homedir(), ".local", "share");
+  return import_node_path.default.join(base, appDataDirName, profileDirName);
+}
+async function fetchWithTimeout(url, timeoutMs) {
+  if (!timeoutMs || timeoutMs <= 0)
+    return await fetch(url, { redirect: "follow" });
+  const ctl = new AbortController;
+  const timer = setTimeout(() => ctl.abort(), timeoutMs);
+  try {
+    return await fetch(url, { redirect: "follow", signal: ctl.signal });
+  } finally {
+    clearTimeout(timer);
+  }
+}
+async function fetchJson(url, options = {}) {
+  const response = await fetchWithTimeout(url, options.timeoutMs);
+  if (!response.ok) {
+    throw new Error(`Request failed: ${response.status} ${response.statusText}`);
+  }
+  return await response.json();
+}
+async function isDebugPortReady(port, timeoutMs = 3000) {
+  try {
+    const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
+    return !!version.webSocketDebuggerUrl;
+  } catch {
+    return false;
+  }
+}
+function isPortListening(port, timeoutMs = 3000) {
+  return new Promise((resolve) => {
+    const socket = new import_node_net.default.Socket;
+    const timer = setTimeout(() => {
+      socket.destroy();
+      resolve(false);
+    }, timeoutMs);
+    socket.once("connect", () => {
+      clearTimeout(timer);
+      socket.destroy();
+      resolve(true);
+    });
+    socket.once("error", () => {
+      clearTimeout(timer);
+      resolve(false);
+    });
+    socket.connect(port, "127.0.0.1");
+  });
+}
+function parseDevToolsActivePort(filePath) {
+  try {
+    const content = import_node_fs.default.readFileSync(filePath, "utf-8");
+    const lines = content.split(/\r?\n/);
+    const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
+    const wsPath = lines[1]?.trim();
+    if (port > 0 && wsPath)
+      return { port, wsPath };
+  } catch {}
+  return null;
+}
+async function findExistingChromeDebugPort(options) {
+  const timeoutMs = options.timeoutMs ?? 3000;
+  const parsed = parseDevToolsActivePort(import_node_path.default.join(options.profileDir, "DevToolsActivePort"));
+  if (parsed && parsed.port > 0 && await isDebugPortReady(parsed.port, timeoutMs))
+    return parsed.port;
+  if (import_node_process.default.platform === "win32")
+    return null;
+  try {
+    const result = import_node_child_process.spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
+    if (result.status !== 0 || !result.stdout)
+      return null;
+    const lines = result.stdout.split(`
+`).filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
+    for (const line of lines) {
+      const portMatch = line.match(/--remote-debugging-port=(\d+)/);
+      const port = Number.parseInt(portMatch?.[1] ?? "", 10);
+      if (port > 0 && await isDebugPortReady(port, timeoutMs))
+        return port;
+    }
+  } catch {}
+  return null;
+}
+function getDefaultChromeUserDataDirs(channels = ["stable"]) {
+  const home = import_node_os.default.homedir();
+  const dirs = [];
+  const channelDirs = {
+    stable: {
+      darwin: import_node_path.default.join(home, "Library", "Application Support", "Google", "Chrome"),
+      linux: import_node_path.default.join(home, ".config", "google-chrome"),
+      win32: import_node_path.default.join(import_node_process.default.env.LOCALAPPDATA ?? import_node_path.default.join(home, "AppData", "Local"), "Google", "Chrome", "User Data")
+    },
+    beta: {
+      darwin: import_node_path.default.join(home, "Library", "Application Support", "Google", "Chrome Beta"),
+      linux: import_node_path.default.join(home, ".config", "google-chrome-beta"),
+      win32: import_node_path.default.join(import_node_process.default.env.LOCALAPPDATA ?? import_node_path.default.join(home, "AppData", "Local"), "Google", "Chrome Beta", "User Data")
+    },
+    canary: {
+      darwin: import_node_path.default.join(home, "Library", "Application Support", "Google", "Chrome Canary"),
+      linux: import_node_path.default.join(home, ".config", "google-chrome-canary"),
+      win32: import_node_path.default.join(import_node_process.default.env.LOCALAPPDATA ?? import_node_path.default.join(home, "AppData", "Local"), "Google", "Chrome SxS", "User Data")
+    },
+    dev: {
+      darwin: import_node_path.default.join(home, "Library", "Application Support", "Google", "Chrome Dev"),
+      linux: import_node_path.default.join(home, ".config", "google-chrome-dev"),
+      win32: import_node_path.default.join(import_node_process.default.env.LOCALAPPDATA ?? import_node_path.default.join(home, "AppData", "Local"), "Google", "Chrome Dev", "User Data")
+    }
+  };
+  const platform = import_node_process.default.platform === "darwin" ? "darwin" : import_node_process.default.platform === "win32" ? "win32" : "linux";
+  for (const ch of channels) {
+    const entry = channelDirs[ch];
+    if (entry)
+      dirs.push(entry[platform]);
+  }
+  return dirs;
+}
+async function discoverRunningChromeDebugPort(options = {}) {
+  const channels = options.channels ?? ["stable", "beta", "canary", "dev"];
+  const timeoutMs = options.timeoutMs ?? 3000;
+  const userDataDirs = (options.userDataDirs ?? getDefaultChromeUserDataDirs(channels)).map((dir) => import_node_path.default.resolve(dir));
+  for (const dir of userDataDirs) {
+    const parsed = parseDevToolsActivePort(import_node_path.default.join(dir, "DevToolsActivePort"));
+    if (!parsed)
+      continue;
+    if (await isPortListening(parsed.port, timeoutMs)) {
+      return { port: parsed.port, wsUrl: `ws://127.0.0.1:${parsed.port}${parsed.wsPath}` };
+    }
+  }
+  if (import_node_process.default.platform !== "win32") {
+    try {
+      const result = import_node_child_process.spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
+      if (result.status === 0 && result.stdout) {
+        const lines = result.stdout.split(`
+`).filter((line) => line.includes("--remote-debugging-port=") && userDataDirs.some((dir) => line.includes(dir)));
+        for (const line of lines) {
+          const portMatch = line.match(/--remote-debugging-port=(\d+)/);
+          const port = Number.parseInt(portMatch?.[1] ?? "", 10);
+          if (port > 0 && await isDebugPortReady(port, timeoutMs)) {
+            try {
+              const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
+              if (version.webSocketDebuggerUrl)
+                return { port, wsUrl: version.webSocketDebuggerUrl };
+            } catch {}
+          }
+        }
+      }
+    } catch {}
+  }
+  return null;
+}
+async function waitForChromeDebugPort(port, timeoutMs, options) {
+  const start = Date.now();
+  let lastError = null;
+  while (Date.now() - start < timeoutMs) {
+    try {
+      const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs: 5000 });
+      if (version.webSocketDebuggerUrl)
+        return version.webSocketDebuggerUrl;
+      lastError = new Error("Missing webSocketDebuggerUrl");
+    } catch (error) {
+      lastError = error;
+    }
+    await sleep(200);
+  }
+  if (options?.includeLastError && lastError) {
+    throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
+  }
+  throw new Error("Chrome debug port not ready");
+}
+
+class CdpConnection {
+  ws;
+  nextId = 0;
+  pending = new Map;
+  eventHandlers = new Map;
+  defaultTimeoutMs;
+  constructor(ws, defaultTimeoutMs = 15000) {
+    this.ws = ws;
+    this.defaultTimeoutMs = defaultTimeoutMs;
+    this.ws.addEventListener("message", (event) => {
+      try {
+        const data = typeof event.data === "string" ? event.data : new TextDecoder().decode(event.data);
+        const msg = JSON.parse(data);
+        if (msg.method) {
+          const handlers = this.eventHandlers.get(msg.method);
+          if (handlers) {
+            handlers.forEach((handler) => handler(msg.params));
+          }
+        }
+        if (msg.id) {
+          const pending = this.pending.get(msg.id);
+          if (pending) {
+            this.pending.delete(msg.id);
+            if (pending.timer)
+              clearTimeout(pending.timer);
+            if (msg.error?.message)
+              pending.reject(new Error(msg.error.message));
+            else
+              pending.resolve(msg.result);
+          }
+        }
+      } catch {}
+    });
+    this.ws.addEventListener("close", () => {
+      for (const [id, pending] of this.pending.entries()) {
+        this.pending.delete(id);
+        if (pending.timer)
+          clearTimeout(pending.timer);
+        pending.reject(new Error("CDP connection closed."));
+      }
+    });
+  }
+  static async connect(url, timeoutMs, options) {
+    const ws = new WebSocket(url);
+    await new Promise((resolve, reject) => {
+      const timer = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
+      ws.addEventListener("open", () => {
+        clearTimeout(timer);
+        resolve();
+      });
+      ws.addEventListener("error", () => {
+        clearTimeout(timer);
+        reject(new Error("CDP connection failed."));
+      });
+    });
+    return new CdpConnection(ws, options?.defaultTimeoutMs ?? 15000);
+  }
+  on(method, handler) {
+    if (!this.eventHandlers.has(method)) {
+      this.eventHandlers.set(method, new Set);
+    }
+    this.eventHandlers.get(method)?.add(handler);
+  }
+  off(method, handler) {
+    this.eventHandlers.get(method)?.delete(handler);
+  }
+  async send(method, params, options) {
+    const id = ++this.nextId;
+    const message = { id, method };
+    if (params)
+      message.params = params;
+    if (options?.sessionId)
+      message.sessionId = options.sessionId;
+    const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
+    const result = await new Promise((resolve, reject) => {
+      const timer = timeoutMs > 0 ? setTimeout(() => {
+        this.pending.delete(id);
+        reject(new Error(`CDP timeout: ${method}`));
+      }, timeoutMs) : null;
+      this.pending.set(id, { resolve, reject, timer });
+      this.ws.send(JSON.stringify(message));
+    });
+    return result;
+  }
+  close() {
+    try {
+      this.ws.close();
+    } catch {}
+  }
+}
+async function launchChrome(options) {
+  await import_node_fs.default.promises.mkdir(options.profileDir, { recursive: true });
+  const args = [
+    `--remote-debugging-port=${options.port}`,
+    `--user-data-dir=${options.profileDir}`,
+    "--no-first-run",
+    "--no-default-browser-check",
+    ...options.extraArgs ?? []
+  ];
+  if (options.headless)
+    args.push("--headless=new");
+  if (options.url)
+    args.push(options.url);
+  return import_node_child_process.spawn(options.chromePath, args, { stdio: "ignore" });
+}
+function killChrome(chrome) {
+  try {
+    chrome.kill("SIGTERM");
+  } catch {}
+  setTimeout(() => {
+    if (chrome.exitCode === null && chrome.signalCode === null) {
+      try {
+        chrome.kill("SIGKILL");
+      } catch {}
+    }
+  }, 2000).unref?.();
+}
+async function gracefulKillChrome(chrome, port, timeoutMs = 6000) {
+  if (chrome.exitCode !== null || chrome.signalCode !== null)
+    return;
+  const exitPromise = new Promise((resolve) => {
+    chrome.once("exit", () => resolve());
+  });
+  killChrome(chrome);
+  const start = Date.now();
+  while (Date.now() - start < timeoutMs) {
+    if (chrome.exitCode !== null || chrome.signalCode !== null)
+      return;
+    if (port !== undefined && !await isPortListening(port, 250))
+      return;
+    const exited = await Promise.race([
+      exitPromise.then(() => true),
+      sleep(100).then(() => false)
+    ]);
+    if (exited)
+      return;
+  }
+  await Promise.race([
+    exitPromise,
+    sleep(250)
+  ]);
+}
+async function openPageSession(options) {
+  let targetId;
+  let createdTarget = false;
+  if (options.reusing) {
+    const created = await options.cdp.send("Target.createTarget", { url: options.url });
+    targetId = created.targetId;
+    createdTarget = true;
+  } else {
+    const targets = await options.cdp.send("Target.getTargets");
+    const existing = targets.targetInfos.find(options.matchTarget);
+    if (existing) {
+      targetId = existing.targetId;
+    } else {
+      const created = await options.cdp.send("Target.createTarget", { url: options.url });
+      targetId = created.targetId;
+      createdTarget = true;
+    }
+  }
+  const { sessionId } = await options.cdp.send("Target.attachToTarget", { targetId, flatten: true });
+  if (options.activateTarget ?? true) {
+    await options.cdp.send("Target.activateTarget", { targetId });
+  }
+  if (options.enablePage)
+    await options.cdp.send("Page.enable", {}, { sessionId });
+  if (options.enableRuntime)
+    await options.cdp.send("Runtime.enable", {}, { sessionId });
+  if (options.enableDom)
+    await options.cdp.send("DOM.enable", {}, { sessionId });
+  if (options.enableNetwork)
+    await options.cdp.send("Network.enable", {}, { sessionId });
+  return { sessionId, targetId, createdTarget };
+}

packages/baoyu-chrome-cdp/dist/index.js

@@ -0,0 +1,415 @@
+// src/index.ts
+import { spawn, spawnSync } from "node:child_process";
+import fs from "node:fs";
+import net from "node:net";
+import os from "node:os";
+import path from "node:path";
+import process from "node:process";
+function sleep(ms) {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+async function getFreePort(fixedEnvName) {
+  const fixed = fixedEnvName ? Number.parseInt(process.env[fixedEnvName] ?? "", 10) : NaN;
+  if (Number.isInteger(fixed) && fixed > 0)
+    return fixed;
+  return await new Promise((resolve, reject) => {
+    const server = net.createServer();
+    server.unref();
+    server.on("error", reject);
+    server.listen(0, "127.0.0.1", () => {
+      const address = server.address();
+      if (!address || typeof address === "string") {
+        server.close(() => reject(new Error("Unable to allocate a free TCP port.")));
+        return;
+      }
+      const port = address.port;
+      server.close((err) => {
+        if (err)
+          reject(err);
+        else
+          resolve(port);
+      });
+    });
+  });
+}
+function findChromeExecutable(options) {
+  for (const envName of options.envNames ?? []) {
+    const override = process.env[envName]?.trim();
+    if (override && fs.existsSync(override))
+      return override;
+  }
+  const candidates = process.platform === "darwin" ? options.candidates.darwin ?? options.candidates.default : process.platform === "win32" ? options.candidates.win32 ?? options.candidates.default : options.candidates.default;
+  for (const candidate of candidates) {
+    if (fs.existsSync(candidate))
+      return candidate;
+  }
+  return;
+}
+function resolveSharedChromeProfileDir(options = {}) {
+  for (const envName of options.envNames ?? []) {
+    const override = process.env[envName]?.trim();
+    if (override)
+      return path.resolve(override);
+  }
+  const appDataDirName = options.appDataDirName ?? "baoyu-skills";
+  const profileDirName = options.profileDirName ?? "chrome-profile";
+  if (options.wslWindowsHome) {
+    return path.join(options.wslWindowsHome, ".local", "share", appDataDirName, profileDirName);
+  }
+  const base = process.platform === "darwin" ? path.join(os.homedir(), "Library", "Application Support") : process.platform === "win32" ? process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming") : process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share");
+  return path.join(base, appDataDirName, profileDirName);
+}
+async function fetchWithTimeout(url, timeoutMs) {
+  if (!timeoutMs || timeoutMs <= 0)
+    return await fetch(url, { redirect: "follow" });
+  const ctl = new AbortController;
+  const timer = setTimeout(() => ctl.abort(), timeoutMs);
+  try {
+    return await fetch(url, { redirect: "follow", signal: ctl.signal });
+  } finally {
+    clearTimeout(timer);
+  }
+}
+async function fetchJson(url, options = {}) {
+  const response = await fetchWithTimeout(url, options.timeoutMs);
+  if (!response.ok) {
+    throw new Error(`Request failed: ${response.status} ${response.statusText}`);
+  }
+  return await response.json();
+}
+async function isDebugPortReady(port, timeoutMs = 3000) {
+  try {
+    const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
+    return !!version.webSocketDebuggerUrl;
+  } catch {
+    return false;
+  }
+}
+function isPortListening(port, timeoutMs = 3000) {
+  return new Promise((resolve) => {
+    const socket = new net.Socket;
+    const timer = setTimeout(() => {
+      socket.destroy();
+      resolve(false);
+    }, timeoutMs);
+    socket.once("connect", () => {
+      clearTimeout(timer);
+      socket.destroy();
+      resolve(true);
+    });
+    socket.once("error", () => {
+      clearTimeout(timer);
+      resolve(false);
+    });
+    socket.connect(port, "127.0.0.1");
+  });
+}
+function parseDevToolsActivePort(filePath) {
+  try {
+    const content = fs.readFileSync(filePath, "utf-8");
+    const lines = content.split(/\r?\n/);
+    const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
+    const wsPath = lines[1]?.trim();
+    if (port > 0 && wsPath)
+      return { port, wsPath };
+  } catch {}
+  return null;
+}
+async function findExistingChromeDebugPort(options) {
+  const timeoutMs = options.timeoutMs ?? 3000;
+  const parsed = parseDevToolsActivePort(path.join(options.profileDir, "DevToolsActivePort"));
+  if (parsed && parsed.port > 0 && await isDebugPortReady(parsed.port, timeoutMs))
+    return parsed.port;
+  if (process.platform === "win32")
+    return null;
+  try {
+    const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
+    if (result.status !== 0 || !result.stdout)
+      return null;
+    const lines = result.stdout.split(`
+`).filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
+    for (const line of lines) {
+      const portMatch = line.match(/--remote-debugging-port=(\d+)/);
+      const port = Number.parseInt(portMatch?.[1] ?? "", 10);
+      if (port > 0 && await isDebugPortReady(port, timeoutMs))
+        return port;
+    }
+  } catch {}
+  return null;
+}
+function getDefaultChromeUserDataDirs(channels = ["stable"]) {
+  const home = os.homedir();
+  const dirs = [];
+  const channelDirs = {
+    stable: {
+      darwin: path.join(home, "Library", "Application Support", "Google", "Chrome"),
+      linux: path.join(home, ".config", "google-chrome"),
+      win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome", "User Data")
+    },
+    beta: {
+      darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Beta"),
+      linux: path.join(home, ".config", "google-chrome-beta"),
+      win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome Beta", "User Data")
+    },
+    canary: {
+      darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Canary"),
+      linux: path.join(home, ".config", "google-chrome-canary"),
+      win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome SxS", "User Data")
+    },
+    dev: {
+      darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Dev"),
+      linux: path.join(home, ".config", "google-chrome-dev"),
+      win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome Dev", "User Data")
+    }
+  };
+  const platform = process.platform === "darwin" ? "darwin" : process.platform === "win32" ? "win32" : "linux";
+  for (const ch of channels) {
+    const entry = channelDirs[ch];
+    if (entry)
+      dirs.push(entry[platform]);
+  }
+  return dirs;
+}
+async function discoverRunningChromeDebugPort(options = {}) {
+  const channels = options.channels ?? ["stable", "beta", "canary", "dev"];
+  const timeoutMs = options.timeoutMs ?? 3000;
+  const userDataDirs = (options.userDataDirs ?? getDefaultChromeUserDataDirs(channels)).map((dir) => path.resolve(dir));
+  for (const dir of userDataDirs) {
+    const parsed = parseDevToolsActivePort(path.join(dir, "DevToolsActivePort"));
+    if (!parsed)
+      continue;
+    if (await isPortListening(parsed.port, timeoutMs)) {
+      return { port: parsed.port, wsUrl: `ws://127.0.0.1:${parsed.port}${parsed.wsPath}` };
+    }
+  }
+  if (process.platform !== "win32") {
+    try {
+      const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
+      if (result.status === 0 && result.stdout) {
+        const lines = result.stdout.split(`
+`).filter((line) => line.includes("--remote-debugging-port=") && userDataDirs.some((dir) => line.includes(dir)));
+        for (const line of lines) {
+          const portMatch = line.match(/--remote-debugging-port=(\d+)/);
+          const port = Number.parseInt(portMatch?.[1] ?? "", 10);
+          if (port > 0 && await isDebugPortReady(port, timeoutMs)) {
+            try {
+              const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
+              if (version.webSocketDebuggerUrl)
+                return { port, wsUrl: version.webSocketDebuggerUrl };
+            } catch {}
+          }
+        }
+      }
+    } catch {}
+  }
+  return null;
+}
+async function waitForChromeDebugPort(port, timeoutMs, options) {
+  const start = Date.now();
+  let lastError = null;
+  while (Date.now() - start < timeoutMs) {
+    try {
+      const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs: 5000 });
+      if (version.webSocketDebuggerUrl)
+        return version.webSocketDebuggerUrl;
+      lastError = new Error("Missing webSocketDebuggerUrl");
+    } catch (error) {
+      lastError = error;
+    }
+    await sleep(200);
+  }
+  if (options?.includeLastError && lastError) {
+    throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
+  }
+  throw new Error("Chrome debug port not ready");
+}
+
+class CdpConnection {
+  ws;
+  nextId = 0;
+  pending = new Map;
+  eventHandlers = new Map;
+  defaultTimeoutMs;
+  constructor(ws, defaultTimeoutMs = 15000) {
+    this.ws = ws;
+    this.defaultTimeoutMs = defaultTimeoutMs;
+    this.ws.addEventListener("message", (event) => {
+      try {
+        const data = typeof event.data === "string" ? event.data : new TextDecoder().decode(event.data);
+        const msg = JSON.parse(data);
+        if (msg.method) {
+          const handlers = this.eventHandlers.get(msg.method);
+          if (handlers) {
+            handlers.forEach((handler) => handler(msg.params));
+          }
+        }
+        if (msg.id) {
+          const pending = this.pending.get(msg.id);
+          if (pending) {
+            this.pending.delete(msg.id);
+            if (pending.timer)
+              clearTimeout(pending.timer);
+            if (msg.error?.message)
+              pending.reject(new Error(msg.error.message));
+            else
+              pending.resolve(msg.result);
+          }
+        }
+      } catch {}
+    });
+    this.ws.addEventListener("close", () => {
+      for (const [id, pending] of this.pending.entries()) {
+        this.pending.delete(id);
+        if (pending.timer)
+          clearTimeout(pending.timer);
+        pending.reject(new Error("CDP connection closed."));
+      }
+    });
+  }
+  static async connect(url, timeoutMs, options) {
+    const ws = new WebSocket(url);
+    await new Promise((resolve, reject) => {
+      const timer = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
+      ws.addEventListener("open", () => {
+        clearTimeout(timer);
+        resolve();
+      });
+      ws.addEventListener("error", () => {
+        clearTimeout(timer);
+        reject(new Error("CDP connection failed."));
+      });
+    });
+    return new CdpConnection(ws, options?.defaultTimeoutMs ?? 15000);
+  }
+  on(method, handler) {
+    if (!this.eventHandlers.has(method)) {
+      this.eventHandlers.set(method, new Set);
+    }
+    this.eventHandlers.get(method)?.add(handler);
+  }
+  off(method, handler) {
+    this.eventHandlers.get(method)?.delete(handler);
+  }
+  async send(method, params, options) {
+    const id = ++this.nextId;
+    const message = { id, method };
+    if (params)
+      message.params = params;
+    if (options?.sessionId)
+      message.sessionId = options.sessionId;
+    const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
+    const result = await new Promise((resolve, reject) => {
+      const timer = timeoutMs > 0 ? setTimeout(() => {
+        this.pending.delete(id);
+        reject(new Error(`CDP timeout: ${method}`));
+      }, timeoutMs) : null;
+      this.pending.set(id, { resolve, reject, timer });
+      this.ws.send(JSON.stringify(message));
+    });
+    return result;
+  }
+  close() {
+    try {
+      this.ws.close();
+    } catch {}
+  }
+}
+async function launchChrome(options) {
+  await fs.promises.mkdir(options.profileDir, { recursive: true });
+  const args = [
+    `--remote-debugging-port=${options.port}`,
+    `--user-data-dir=${options.profileDir}`,
+    "--no-first-run",
+    "--no-default-browser-check",
+    ...options.extraArgs ?? []
+  ];
+  if (options.headless)
+    args.push("--headless=new");
+  if (options.url)
+    args.push(options.url);
+  return spawn(options.chromePath, args, { stdio: "ignore" });
+}
+function killChrome(chrome) {
+  try {
+    chrome.kill("SIGTERM");
+  } catch {}
+  setTimeout(() => {
+    if (chrome.exitCode === null && chrome.signalCode === null) {
+      try {
+        chrome.kill("SIGKILL");
+      } catch {}
+    }
+  }, 2000).unref?.();
+}
+async function gracefulKillChrome(chrome, port, timeoutMs = 6000) {
+  if (chrome.exitCode !== null || chrome.signalCode !== null)
+    return;
+  const exitPromise = new Promise((resolve) => {
+    chrome.once("exit", () => resolve());
+  });
+  killChrome(chrome);
+  const start = Date.now();
+  while (Date.now() - start < timeoutMs) {
+    if (chrome.exitCode !== null || chrome.signalCode !== null)
+      return;
+    if (port !== undefined && !await isPortListening(port, 250))
+      return;
+    const exited = await Promise.race([
+      exitPromise.then(() => true),
+      sleep(100).then(() => false)
+    ]);
+    if (exited)
+      return;
+  }
+  await Promise.race([
+    exitPromise,
+    sleep(250)
+  ]);
+}
+async function openPageSession(options) {
+  let targetId;
+  let createdTarget = false;
+  if (options.reusing) {
+    const created = await options.cdp.send("Target.createTarget", { url: options.url });
+    targetId = created.targetId;
+    createdTarget = true;
+  } else {
+    const targets = await options.cdp.send("Target.getTargets");
+    const existing = targets.targetInfos.find(options.matchTarget);
+    if (existing) {
+      targetId = existing.targetId;
+    } else {
+      const created = await options.cdp.send("Target.createTarget", { url: options.url });
+      targetId = created.targetId;
+      createdTarget = true;
+    }
+  }
+  const { sessionId } = await options.cdp.send("Target.attachToTarget", { targetId, flatten: true });
+  if (options.activateTarget ?? true) {
+    await options.cdp.send("Target.activateTarget", { targetId });
+  }
+  if (options.enablePage)
+    await options.cdp.send("Page.enable", {}, { sessionId });
+  if (options.enableRuntime)
+    await options.cdp.send("Runtime.enable", {}, { sessionId });
+  if (options.enableDom)
+    await options.cdp.send("DOM.enable", {}, { sessionId });
+  if (options.enableNetwork)
+    await options.cdp.send("Network.enable", {}, { sessionId });
+  return { sessionId, targetId, createdTarget };
+}
+export {
+  waitForChromeDebugPort,
+  sleep,
+  resolveSharedChromeProfileDir,
+  openPageSession,
+  launchChrome,
+  killChrome,
+  gracefulKillChrome,
+  getFreePort,
+  getDefaultChromeUserDataDirs,
+  findExistingChromeDebugPort,
+  findChromeExecutable,
+  discoverRunningChromeDebugPort,
+  CdpConnection
+};

packages/baoyu-chrome-cdp/package.json

@@ -0,0 +1,42 @@
+{
+  "name": "baoyu-chrome-cdp",
+  "version": "0.1.0",
+  "type": "module",
+  "files": [
+    "dist",
+    "src/**/*.ts",
+    "!src/**/*.test.ts"
+  ],
+  "exports": {
+    ".": {
+      "types": "./src/index.ts",
+      "import": "./dist/index.js",
+      "require": "./dist/index.cjs",
+      "default": "./dist/index.js"
+    },
+    "./src/*": "./src/*"
+  },
+  "description": "Shared Chrome DevTools Protocol utilities for baoyu skills.",
+  "main": "./dist/index.cjs",
+  "module": "./dist/index.js",
+  "types": "./src/index.ts",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/JimLiu/baoyu-skills.git",
+    "directory": "packages/baoyu-chrome-cdp"
+  },
+  "bugs": {
+    "url": "https://github.com/JimLiu/baoyu-skills/issues"
+  },
+  "homepage": "https://github.com/JimLiu/baoyu-skills/tree/main/packages/baoyu-chrome-cdp#readme",
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "build": "node ../../scripts/build-shared-package.mjs",
+    "prepack": "bun run build"
+  },
+  "engines": {
+    "bun": ">=1.2.0"
+  }
+}

packages/baoyu-chrome-cdp/src/index.test.ts

@@ -0,0 +1,362 @@
+import assert from "node:assert/strict";
+import { spawn, type ChildProcess } from "node:child_process";
+import fs from "node:fs/promises";
+import http from "node:http";
+import os from "node:os";
+import path from "node:path";
+import process from "node:process";
+import test, { type TestContext } from "node:test";
+
+import {
+  discoverRunningChromeDebugPort,
+  findChromeExecutable,
+  findExistingChromeDebugPort,
+  gracefulKillChrome,
+  getFreePort,
+  openPageSession,
+  resolveSharedChromeProfileDir,
+  waitForChromeDebugPort,
+} from "./index.ts";
+
+function useEnv(
+  t: TestContext,
+  values: Record<string, string | null>,
+): void {
+  const previous = new Map<string, string | undefined>();
+  for (const [key, value] of Object.entries(values)) {
+    previous.set(key, process.env[key]);
+    if (value == null) {
+      delete process.env[key];
+    } else {
+      process.env[key] = value;
+    }
+  }
+
+  t.after(() => {
+    for (const [key, value] of previous.entries()) {
+      if (value == null) {
+        delete process.env[key];
+      } else {
+        process.env[key] = value;
+      }
+    }
+  });
+}
+
+async function makeTempDir(prefix: string): Promise<string> {
+  return fs.mkdtemp(path.join(os.tmpdir(), prefix));
+}
+
+async function startDebugServer(port: number): Promise<http.Server> {
+  const server = http.createServer((req, res) => {
+    if (req.url === "/json/version") {
+      res.writeHead(200, { "Content-Type": "application/json" });
+      res.end(JSON.stringify({
+        webSocketDebuggerUrl: `ws://127.0.0.1:${port}/devtools/browser/demo`,
+      }));
+      return;
+    }
+
+    res.writeHead(404);
+    res.end();
+  });
+
+  await new Promise<void>((resolve, reject) => {
+    server.once("error", reject);
+    server.listen(port, "127.0.0.1", () => resolve());
+  });
+
+  return server;
+}
+
+async function closeServer(server: http.Server): Promise<void> {
+  await new Promise<void>((resolve, reject) => {
+    server.close((error) => {
+      if (error) reject(error);
+      else resolve();
+    });
+  });
+}
+
+function shellPathForPlatform(): string | null {
+  if (process.platform === "win32") return null;
+  return "/bin/bash";
+}
+
+async function startFakeChromiumProcess(port: number): Promise<ChildProcess | null> {
+  const shell = shellPathForPlatform();
+  if (!shell) return null;
+
+  const child = spawn(
+    shell,
+    [
+      "-lc",
+      `exec -a chromium-mock ${JSON.stringify(process.execPath)} -e 'setInterval(() => {}, 1000)' -- --remote-debugging-port=${port}`,
+    ],
+    { stdio: "ignore" },
+  );
+
+  await new Promise((resolve) => setTimeout(resolve, 250));
+  return child;
+}
+
+async function stopProcess(child: ChildProcess | null): Promise<void> {
+  if (!child) return;
+  if (child.exitCode !== null || child.signalCode !== null) return;
+
+  child.kill("SIGTERM");
+  await new Promise((resolve) => setTimeout(resolve, 100));
+  if (child.exitCode === null && child.signalCode === null) child.kill("SIGKILL");
+  if (child.exitCode !== null || child.signalCode !== null) return;
+  await new Promise((resolve) => child.once("exit", resolve));
+}
+
+async function startPortHoldingProcess(port: number): Promise<ChildProcess> {
+  const child = spawn(
+    process.execPath,
+    [
+      "-e",
+      `
+        const http = require("node:http");
+        const port = Number(process.argv[1]);
+        const server = http.createServer((_req, res) => res.end("ok"));
+        server.listen(port, "127.0.0.1", () => process.stdout.write("ready\\n"));
+        setInterval(() => {}, 1000);
+      `,
+      String(port),
+    ],
+    {
+      stdio: ["ignore", "pipe", "ignore"],
+    },
+  );
+
+  await new Promise<void>((resolve, reject) => {
+    const timer = setTimeout(() => reject(new Error("Timed out waiting for child server to start.")), 3_000);
+    child.once("error", (error) => {
+      clearTimeout(timer);
+      reject(error);
+    });
+    child.stdout?.once("data", () => {
+      clearTimeout(timer);
+      resolve();
+    });
+    child.once("exit", () => {
+      clearTimeout(timer);
+      reject(new Error("Child server exited before becoming ready."));
+    });
+  });
+
+  return child;
+}
+
+test("getFreePort honors a fixed environment override and otherwise allocates a TCP port", async (t) => {
+  useEnv(t, { TEST_FIXED_PORT: "45678" });
+  assert.equal(await getFreePort("TEST_FIXED_PORT"), 45678);
+
+  const dynamicPort = await getFreePort();
+  assert.ok(Number.isInteger(dynamicPort));
+  assert.ok(dynamicPort > 0);
+});
+
+test("findChromeExecutable prefers env overrides and falls back to candidate paths", async (t) => {
+  const root = await makeTempDir("baoyu-chrome-bin-");
+  t.after(() => fs.rm(root, { recursive: true, force: true }));
+
+  const envChrome = path.join(root, "env-chrome");
+  const fallbackChrome = path.join(root, "fallback-chrome");
+  await fs.writeFile(envChrome, "");
+  await fs.writeFile(fallbackChrome, "");
+
+  useEnv(t, { BAOYU_CHROME_PATH: envChrome });
+  assert.equal(
+    findChromeExecutable({
+      envNames: ["BAOYU_CHROME_PATH"],
+      candidates: { default: [fallbackChrome] },
+    }),
+    envChrome,
+  );
+
+  useEnv(t, { BAOYU_CHROME_PATH: null });
+  assert.equal(
+    findChromeExecutable({
+      envNames: ["BAOYU_CHROME_PATH"],
+      candidates: { default: [fallbackChrome] },
+    }),
+    fallbackChrome,
+  );
+});
+
+test("resolveSharedChromeProfileDir supports env overrides, WSL paths, and default suffixes", (t) => {
+  useEnv(t, { BAOYU_SHARED_PROFILE: "/tmp/custom-profile" });
+  assert.equal(
+    resolveSharedChromeProfileDir({
+      envNames: ["BAOYU_SHARED_PROFILE"],
+      appDataDirName: "demo-app",
+      profileDirName: "demo-profile",
+    }),
+    path.resolve("/tmp/custom-profile"),
+  );
+
+  useEnv(t, { BAOYU_SHARED_PROFILE: null });
+  assert.equal(
+    resolveSharedChromeProfileDir({
+      wslWindowsHome: "/mnt/c/Users/demo",
+      appDataDirName: "demo-app",
+      profileDirName: "demo-profile",
+    }),
+    path.join("/mnt/c/Users/demo", ".local", "share", "demo-app", "demo-profile"),
+  );
+
+  const fallback = resolveSharedChromeProfileDir({
+    appDataDirName: "demo-app",
+    profileDirName: "demo-profile",
+  });
+  assert.match(fallback, /demo-app[\\/]demo-profile$/);
+});
+
+test("findExistingChromeDebugPort reads DevToolsActivePort and validates it against a live endpoint", async (t) => {
+  const root = await makeTempDir("baoyu-cdp-profile-");
+  t.after(() => fs.rm(root, { recursive: true, force: true }));
+
+  const port = await getFreePort();
+  const server = await startDebugServer(port);
+  t.after(() => closeServer(server));
+
+  await fs.writeFile(path.join(root, "DevToolsActivePort"), `${port}\n/devtools/browser/demo\n`);
+
+  const found = await findExistingChromeDebugPort({ profileDir: root, timeoutMs: 1000 });
+  assert.equal(found, port);
+});
+
+test("discoverRunningChromeDebugPort reads DevToolsActivePort from the provided user-data dir", async (t) => {
+  const root = await makeTempDir("baoyu-cdp-user-data-");
+  t.after(() => fs.rm(root, { recursive: true, force: true }));
+
+  const port = await getFreePort();
+  const server = await startDebugServer(port);
+  t.after(() => closeServer(server));
+
+  await fs.writeFile(path.join(root, "DevToolsActivePort"), `${port}\n/devtools/browser/demo\n`);
+
+  const found = await discoverRunningChromeDebugPort({
+    userDataDirs: [root],
+    timeoutMs: 1000,
+  });
+  assert.deepEqual(found, {
+    port,
+    wsUrl: `ws://127.0.0.1:${port}/devtools/browser/demo`,
+  });
+});
+
+test("discoverRunningChromeDebugPort ignores unrelated debugging processes", async (t) => {
+  if (process.platform === "win32") {
+    t.skip("Process discovery fallback is not used on Windows.");
+    return;
+  }
+
+  const root = await makeTempDir("baoyu-cdp-user-data-");
+  t.after(() => fs.rm(root, { recursive: true, force: true }));
+
+  const port = await getFreePort();
+  const server = await startDebugServer(port);
+  t.after(() => closeServer(server));
+
+  const fakeChromium = await startFakeChromiumProcess(port);
+  t.after(async () => { await stopProcess(fakeChromium); });
+
+  const found = await discoverRunningChromeDebugPort({
+    userDataDirs: [root],
+    timeoutMs: 1000,
+  });
+  assert.equal(found, null);
+});
+
+test("openPageSession reports whether it created a new target", async () => {
+  const calls: string[] = [];
+  const cdpExisting = {
+    send: async <T>(method: string): Promise<T> => {
+      calls.push(method);
+      if (method === "Target.getTargets") {
+        return {
+          targetInfos: [{ targetId: "existing-target", type: "page", url: "https://gemini.google.com/app" }],
+        } as T;
+      }
+      if (method === "Target.attachToTarget") return { sessionId: "session-existing" } as T;
+      throw new Error(`Unexpected method: ${method}`);
+    },
+  };
+
+  const existing = await openPageSession({
+    cdp: cdpExisting as never,
+    reusing: false,
+    url: "https://gemini.google.com/app",
+    matchTarget: (target) => target.url.includes("gemini.google.com"),
+    activateTarget: false,
+  });
+
+  assert.deepEqual(existing, {
+    sessionId: "session-existing",
+    targetId: "existing-target",
+    createdTarget: false,
+  });
+  assert.deepEqual(calls, ["Target.getTargets", "Target.attachToTarget"]);
+
+  const createCalls: string[] = [];
+  const cdpCreated = {
+    send: async <T>(method: string): Promise<T> => {
+      createCalls.push(method);
+      if (method === "Target.getTargets") return { targetInfos: [] } as T;
+      if (method === "Target.createTarget") return { targetId: "created-target" } as T;
+      if (method === "Target.attachToTarget") return { sessionId: "session-created" } as T;
+      throw new Error(`Unexpected method: ${method}`);
+    },
+  };
+
+  const created = await openPageSession({
+    cdp: cdpCreated as never,
+    reusing: false,
+    url: "https://gemini.google.com/app",
+    matchTarget: (target) => target.url.includes("gemini.google.com"),
+    activateTarget: false,
+  });
+
+  assert.deepEqual(created, {
+    sessionId: "session-created",
+    targetId: "created-target",
+    createdTarget: true,
+  });
+  assert.deepEqual(createCalls, ["Target.getTargets", "Target.createTarget", "Target.attachToTarget"]);
+});
+
+test("waitForChromeDebugPort retries until the debug endpoint becomes available", async (t) => {
+  const port = await getFreePort();
+
+  const serverPromise = (async () => {
+    await new Promise((resolve) => setTimeout(resolve, 200));
+    const server = await startDebugServer(port);
+    t.after(() => closeServer(server));
+  })();
+
+  const websocketUrl = await waitForChromeDebugPort(port, 4000, {
+    includeLastError: true,
+  });
+  await serverPromise;
+
+  assert.equal(websocketUrl, `ws://127.0.0.1:${port}/devtools/browser/demo`);
+});
+
+test("gracefulKillChrome waits for the Chrome process to exit and release its port", async (t) => {
+  const port = await getFreePort();
+  const child = await startPortHoldingProcess(port);
+  t.after(async () => { await stopProcess(child); });
+
+  assert.equal(await waitForChromeDebugPort(port, 1_000).catch(() => null), null);
+
+  await gracefulKillChrome(child, port, 4_000);
+
+  assert.ok(child.exitCode !== null || child.signalCode !== null);
+  assert.equal(
+    await fetch(`http://127.0.0.1:${port}`).then(() => true).catch(() => false),
+    false,
+  );
+});

packages/baoyu-chrome-cdp/src/index.ts

@@ -0,0 +1,554 @@
+import { spawn, spawnSync, type ChildProcess } from "node:child_process";
+import fs from "node:fs";
+import net from "node:net";
+import os from "node:os";
+import path from "node:path";
+import process from "node:process";
+
+export type PlatformCandidates = {
+  darwin?: string[];
+  win32?: string[];
+  default: string[];
+};
+
+type PendingRequest = {
+  resolve: (value: unknown) => void;
+  reject: (error: Error) => void;
+  timer: ReturnType<typeof setTimeout> | null;
+};
+
+type CdpSendOptions = {
+  sessionId?: string;
+  timeoutMs?: number;
+};
+
+type FetchJsonOptions = {
+  timeoutMs?: number;
+};
+
+type FindChromeExecutableOptions = {
+  candidates: PlatformCandidates;
+  envNames?: string[];
+};
+
+type ResolveSharedChromeProfileDirOptions = {
+  envNames?: string[];
+  appDataDirName?: string;
+  profileDirName?: string;
+  wslWindowsHome?: string | null;
+};
+
+type FindExistingChromeDebugPortOptions = {
+  profileDir: string;
+  timeoutMs?: number;
+};
+
+export type ChromeChannel = "stable" | "beta" | "canary" | "dev";
+
+export type DiscoveredChrome = {
+  port: number;
+  wsUrl: string;
+};
+
+type DiscoverRunningChromeOptions = {
+  channels?: ChromeChannel[];
+  userDataDirs?: string[];
+  timeoutMs?: number;
+};
+
+type LaunchChromeOptions = {
+  chromePath: string;
+  profileDir: string;
+  port: number;
+  url?: string;
+  headless?: boolean;
+  extraArgs?: string[];
+};
+
+type ChromeTargetInfo = {
+  targetId: string;
+  url: string;
+  type: string;
+};
+
+type OpenPageSessionOptions = {
+  cdp: CdpConnection;
+  reusing: boolean;
+  url: string;
+  matchTarget: (target: ChromeTargetInfo) => boolean;
+  enablePage?: boolean;
+  enableRuntime?: boolean;
+  enableDom?: boolean;
+  enableNetwork?: boolean;
+  activateTarget?: boolean;
+};
+
+export type PageSession = {
+  sessionId: string;
+  targetId: string;
+  createdTarget: boolean;
+};
+
+export function sleep(ms: number): Promise<void> {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+export async function getFreePort(fixedEnvName?: string): Promise<number> {
+  const fixed = fixedEnvName ? Number.parseInt(process.env[fixedEnvName] ?? "", 10) : NaN;
+  if (Number.isInteger(fixed) && fixed > 0) return fixed;
+
+  return await new Promise((resolve, reject) => {
+    const server = net.createServer();
+    server.unref();
+    server.on("error", reject);
+    server.listen(0, "127.0.0.1", () => {
+      const address = server.address();
+      if (!address || typeof address === "string") {
+        server.close(() => reject(new Error("Unable to allocate a free TCP port.")));
+        return;
+      }
+      const port = address.port;
+      server.close((err) => {
+        if (err) reject(err);
+        else resolve(port);
+      });
+    });
+  });
+}
+
+export function findChromeExecutable(options: FindChromeExecutableOptions): string | undefined {
+  for (const envName of options.envNames ?? []) {
+    const override = process.env[envName]?.trim();
+    if (override && fs.existsSync(override)) return override;
+  }
+
+  const candidates = process.platform === "darwin"
+    ? options.candidates.darwin ?? options.candidates.default
+    : process.platform === "win32"
+      ? options.candidates.win32 ?? options.candidates.default
+      : options.candidates.default;
+
+  for (const candidate of candidates) {
+    if (fs.existsSync(candidate)) return candidate;
+  }
+  return undefined;
+}
+
+export function resolveSharedChromeProfileDir(options: ResolveSharedChromeProfileDirOptions = {}): string {
+  for (const envName of options.envNames ?? []) {
+    const override = process.env[envName]?.trim();
+    if (override) return path.resolve(override);
+  }
+
+  const appDataDirName = options.appDataDirName ?? "baoyu-skills";
+  const profileDirName = options.profileDirName ?? "chrome-profile";
+
+  if (options.wslWindowsHome) {
+    return path.join(options.wslWindowsHome, ".local", "share", appDataDirName, profileDirName);
+  }
+
+  const base = process.platform === "darwin"
+    ? path.join(os.homedir(), "Library", "Application Support")
+    : process.platform === "win32"
+      ? (process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming"))
+      : (process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share"));
+  return path.join(base, appDataDirName, profileDirName);
+}
+
+async function fetchWithTimeout(url: string, timeoutMs?: number): Promise<Response> {
+  if (!timeoutMs || timeoutMs <= 0) return await fetch(url, { redirect: "follow" });
+
+  const ctl = new AbortController();
+  const timer = setTimeout(() => ctl.abort(), timeoutMs);
+  try {
+    return await fetch(url, { redirect: "follow", signal: ctl.signal });
+  } finally {
+    clearTimeout(timer);
+  }
+}
+
+async function fetchJson<T = unknown>(url: string, options: FetchJsonOptions = {}): Promise<T> {
+  const response = await fetchWithTimeout(url, options.timeoutMs);
+  if (!response.ok) {
+    throw new Error(`Request failed: ${response.status} ${response.statusText}`);
+  }
+  return await response.json() as T;
+}
+
+async function isDebugPortReady(port: number, timeoutMs = 3_000): Promise<boolean> {
+  try {
+    const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(
+      `http://127.0.0.1:${port}/json/version`,
+      { timeoutMs }
+    );
+    return !!version.webSocketDebuggerUrl;
+  } catch {
+    return false;
+  }
+}
+
+function isPortListening(port: number, timeoutMs = 3_000): Promise<boolean> {
+  return new Promise((resolve) => {
+    const socket = new net.Socket();
+    const timer = setTimeout(() => { socket.destroy(); resolve(false); }, timeoutMs);
+    socket.once("connect", () => { clearTimeout(timer); socket.destroy(); resolve(true); });
+    socket.once("error", () => { clearTimeout(timer); resolve(false); });
+    socket.connect(port, "127.0.0.1");
+  });
+}
+
+function parseDevToolsActivePort(filePath: string): { port: number; wsPath: string } | null {
+  try {
+    const content = fs.readFileSync(filePath, "utf-8");
+    const lines = content.split(/\r?\n/);
+    const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
+    const wsPath = lines[1]?.trim();
+    if (port > 0 && wsPath) return { port, wsPath };
+  } catch {}
+  return null;
+}
+
+export async function findExistingChromeDebugPort(options: FindExistingChromeDebugPortOptions): Promise<number | null> {
+  const timeoutMs = options.timeoutMs ?? 3_000;
+  const parsed = parseDevToolsActivePort(path.join(options.profileDir, "DevToolsActivePort"));
+
+  if (parsed && parsed.port > 0 && await isDebugPortReady(parsed.port, timeoutMs)) return parsed.port;
+
+  if (process.platform === "win32") return null;
+
+  try {
+    const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5_000 });
+    if (result.status !== 0 || !result.stdout) return null;
+
+    const lines = result.stdout
+      .split("\n")
+      .filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
+
+    for (const line of lines) {
+      const portMatch = line.match(/--remote-debugging-port=(\d+)/);
+      const port = Number.parseInt(portMatch?.[1] ?? "", 10);
+      if (port > 0 && await isDebugPortReady(port, timeoutMs)) return port;
+    }
+  } catch {}
+
+  return null;
+}
+
+export function getDefaultChromeUserDataDirs(channels: ChromeChannel[] = ["stable"]): string[] {
+  const home = os.homedir();
+  const dirs: string[] = [];
+
+  const channelDirs: Record<string, { darwin: string; linux: string; win32: string }> = {
+    stable: {
+      darwin: path.join(home, "Library", "Application Support", "Google", "Chrome"),
+      linux: path.join(home, ".config", "google-chrome"),
+      win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome", "User Data"),
+    },
+    beta: {
+      darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Beta"),
+      linux: path.join(home, ".config", "google-chrome-beta"),
+      win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome Beta", "User Data"),
+    },
+    canary: {
+      darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Canary"),
+      linux: path.join(home, ".config", "google-chrome-canary"),
+      win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome SxS", "User Data"),
+    },
+    dev: {
+      darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Dev"),
+      linux: path.join(home, ".config", "google-chrome-dev"),
+      win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome Dev", "User Data"),
+    },
+  };
+
+  const platform = process.platform === "darwin" ? "darwin" : process.platform === "win32" ? "win32" : "linux";
+
+  for (const ch of channels) {
+    const entry = channelDirs[ch];
+    if (entry) dirs.push(entry[platform]);
+  }
+
+  return dirs;
+}
+
+// Best-effort reuse of an already-running local CDP session discovered from
+// known Chrome user-data dirs. This is distinct from Chrome DevTools MCP's
+// prompt-based --autoConnect flow.
+export async function discoverRunningChromeDebugPort(options: DiscoverRunningChromeOptions = {}): Promise<DiscoveredChrome | null> {
+  const channels = options.channels ?? ["stable", "beta", "canary", "dev"];
+  const timeoutMs = options.timeoutMs ?? 3_000;
+
+  const userDataDirs = (options.userDataDirs ?? getDefaultChromeUserDataDirs(channels))
+    .map((dir) => path.resolve(dir));
+  for (const dir of userDataDirs) {
+    const parsed = parseDevToolsActivePort(path.join(dir, "DevToolsActivePort"));
+    if (!parsed) continue;
+    if (await isPortListening(parsed.port, timeoutMs)) {
+      return { port: parsed.port, wsUrl: `ws://127.0.0.1:${parsed.port}${parsed.wsPath}` };
+    }
+  }
+
+  if (process.platform !== "win32") {
+    try {
+      const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5_000 });
+      if (result.status === 0 && result.stdout) {
+        const lines = result.stdout
+          .split("\n")
+          .filter((line) =>
+            line.includes("--remote-debugging-port=") &&
+            userDataDirs.some((dir) => line.includes(dir))
+          );
+
+        for (const line of lines) {
+          const portMatch = line.match(/--remote-debugging-port=(\d+)/);
+          const port = Number.parseInt(portMatch?.[1] ?? "", 10);
+          if (port > 0 && await isDebugPortReady(port, timeoutMs)) {
+            try {
+              const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
+              if (version.webSocketDebuggerUrl) return { port, wsUrl: version.webSocketDebuggerUrl };
+            } catch {}
+          }
+        }
+      }
+    } catch {}
+  }
+
+  return null;
+}
+
+export async function waitForChromeDebugPort(
+  port: number,
+  timeoutMs: number,
+  options?: { includeLastError?: boolean }
+): Promise<string> {
+  const start = Date.now();
+  let lastError: unknown = null;
+
+  while (Date.now() - start < timeoutMs) {
+    try {
+      const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(
+        `http://127.0.0.1:${port}/json/version`,
+        { timeoutMs: 5_000 }
+      );
+      if (version.webSocketDebuggerUrl) return version.webSocketDebuggerUrl;
+      lastError = new Error("Missing webSocketDebuggerUrl");
+    } catch (error) {
+      lastError = error;
+    }
+    await sleep(200);
+  }
+
+  if (options?.includeLastError && lastError) {
+    throw new Error(
+      `Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`
+    );
+  }
+  throw new Error("Chrome debug port not ready");
+}
+
+export class CdpConnection {
+  private ws: WebSocket;
+  private nextId = 0;
+  private pending = new Map<number, PendingRequest>();
+  private eventHandlers = new Map<string, Set<(params: unknown) => void>>();
+  private defaultTimeoutMs: number;
+
+  private constructor(ws: WebSocket, defaultTimeoutMs = 15_000) {
+    this.ws = ws;
+    this.defaultTimeoutMs = defaultTimeoutMs;
+
+    this.ws.addEventListener("message", (event) => {
+      try {
+        const data = typeof event.data === "string"
+          ? event.data
+          : new TextDecoder().decode(event.data as ArrayBuffer);
+        const msg = JSON.parse(data) as {
+          id?: number;
+          method?: string;
+          params?: unknown;
+          result?: unknown;
+          error?: { message?: string };
+        };
+
+        if (msg.method) {
+          const handlers = this.eventHandlers.get(msg.method);
+          if (handlers) {
+            handlers.forEach((handler) => handler(msg.params));
+          }
+        }
+
+        if (msg.id) {
+          const pending = this.pending.get(msg.id);
+          if (pending) {
+            this.pending.delete(msg.id);
+            if (pending.timer) clearTimeout(pending.timer);
+            if (msg.error?.message) pending.reject(new Error(msg.error.message));
+            else pending.resolve(msg.result);
+          }
+        }
+      } catch {}
+    });
+
+    this.ws.addEventListener("close", () => {
+      for (const [id, pending] of this.pending.entries()) {
+        this.pending.delete(id);
+        if (pending.timer) clearTimeout(pending.timer);
+        pending.reject(new Error("CDP connection closed."));
+      }
+    });
+  }
+
+  static async connect(
+    url: string,
+    timeoutMs: number,
+    options?: { defaultTimeoutMs?: number }
+  ): Promise<CdpConnection> {
+    const ws = new WebSocket(url);
+    await new Promise<void>((resolve, reject) => {
+      const timer = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
+      ws.addEventListener("open", () => {
+        clearTimeout(timer);
+        resolve();
+      });
+      ws.addEventListener("error", () => {
+        clearTimeout(timer);
+        reject(new Error("CDP connection failed."));
+      });
+    });
+    return new CdpConnection(ws, options?.defaultTimeoutMs ?? 15_000);
+  }
+
+  on(method: string, handler: (params: unknown) => void): void {
+    if (!this.eventHandlers.has(method)) {
+      this.eventHandlers.set(method, new Set());
+    }
+    this.eventHandlers.get(method)?.add(handler);
+  }
+
+  off(method: string, handler: (params: unknown) => void): void {
+    this.eventHandlers.get(method)?.delete(handler);
+  }
+
+  async send<T = unknown>(method: string, params?: Record<string, unknown>, options?: CdpSendOptions): Promise<T> {
+    const id = ++this.nextId;
+    const message: Record<string, unknown> = { id, method };
+    if (params) message.params = params;
+    if (options?.sessionId) message.sessionId = options.sessionId;
+
+    const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
+    const result = await new Promise<unknown>((resolve, reject) => {
+      const timer = timeoutMs > 0
+        ? setTimeout(() => {
+          this.pending.delete(id);
+          reject(new Error(`CDP timeout: ${method}`));
+        }, timeoutMs)
+        : null;
+      this.pending.set(id, { resolve, reject, timer });
+      this.ws.send(JSON.stringify(message));
+    });
+
+    return result as T;
+  }
+
+  close(): void {
+    try {
+      this.ws.close();
+    } catch {}
+  }
+}
+
+export async function launchChrome(options: LaunchChromeOptions): Promise<ChildProcess> {
+  await fs.promises.mkdir(options.profileDir, { recursive: true });
+
+  const args = [
+    `--remote-debugging-port=${options.port}`,
+    `--user-data-dir=${options.profileDir}`,
+    "--no-first-run",
+    "--no-default-browser-check",
+    ...(options.extraArgs ?? []),
+  ];
+  if (options.headless) args.push("--headless=new");
+  if (options.url) args.push(options.url);
+
+  return spawn(options.chromePath, args, { stdio: "ignore" });
+}
+
+export function killChrome(chrome: ChildProcess): void {
+  try {
+    chrome.kill("SIGTERM");
+  } catch {}
+  setTimeout(() => {
+    if (chrome.exitCode === null && chrome.signalCode === null) {
+      try {
+        chrome.kill("SIGKILL");
+      } catch {}
+    }
+  }, 2_000).unref?.();
+}
+
+export async function gracefulKillChrome(
+  chrome: ChildProcess,
+  port?: number,
+  timeoutMs = 6_000,
+): Promise<void> {
+  if (chrome.exitCode !== null || chrome.signalCode !== null) return;
+
+  const exitPromise = new Promise<void>((resolve) => {
+    chrome.once("exit", () => resolve());
+  });
+
+  killChrome(chrome);
+
+  const start = Date.now();
+  while (Date.now() - start < timeoutMs) {
+    if (chrome.exitCode !== null || chrome.signalCode !== null) return;
+    if (port !== undefined && !await isPortListening(port, 250)) return;
+
+    const exited = await Promise.race([
+      exitPromise.then(() => true),
+      sleep(100).then(() => false),
+    ]);
+    if (exited) return;
+  }
+
+  await Promise.race([
+    exitPromise,
+    sleep(250),
+  ]);
+}
+
+export async function openPageSession(options: OpenPageSessionOptions): Promise<PageSession> {
+  let targetId: string;
+  let createdTarget = false;
+
+  if (options.reusing) {
+    const created = await options.cdp.send<{ targetId: string }>("Target.createTarget", { url: options.url });
+    targetId = created.targetId;
+    createdTarget = true;
+  } else {
+    const targets = await options.cdp.send<{ targetInfos: ChromeTargetInfo[] }>("Target.getTargets");
+    const existing = targets.targetInfos.find(options.matchTarget);
+    if (existing) {
+      targetId = existing.targetId;
+    } else {
+      const created = await options.cdp.send<{ targetId: string }>("Target.createTarget", { url: options.url });
+      targetId = created.targetId;
+      createdTarget = true;
+    }
+  }
+
+  const { sessionId } = await options.cdp.send<{ sessionId: string }>(
+    "Target.attachToTarget",
+    { targetId, flatten: true }
+  );
+
+  if (options.activateTarget ?? true) {
+    await options.cdp.send("Target.activateTarget", { targetId });
+  }
+  if (options.enablePage) await options.cdp.send("Page.enable", {}, { sessionId });
+  if (options.enableRuntime) await options.cdp.send("Runtime.enable", {}, { sessionId });
+  if (options.enableDom) await options.cdp.send("DOM.enable", {}, { sessionId });
+  if (options.enableNetwork) await options.cdp.send("Network.enable", {}, { sessionId });
+
+  return { sessionId, targetId, createdTarget };
+}

packages/baoyu-fetch/.changeset/README.md

@@ -0,0 +1,11 @@
+# Changesets
+
+This folder stores release notes for version bumps managed by Changesets.
+
+Create a new changeset before merging a user-facing change:
+
+```bash
+bunx changeset
+```
+
+After the changeset lands on `main`, GitHub Actions will open or update the release PR automatically. Merging that release PR publishes the next npm version.

packages/baoyu-fetch/.changeset/config.json

@@ -0,0 +1,11 @@
+{
+  "$schema": "https://unpkg.com/@changesets/config@3.1.3/schema.json",
+  "changelog": "@changesets/cli/changelog",
+  "commit": false,
+  "fixed": [],
+  "linked": [],
+  "access": "public",
+  "baseBranch": "main",
+  "updateInternalDependencies": "patch",
+  "ignore": []
+}

packages/baoyu-fetch/.changeset/remote-defuddle-fallback.md

@@ -0,0 +1,5 @@
+---
+"baoyu-reader": patch
+---
+
+Add a `defuddle.md` markdown fallback for generic extraction when the CLI is asked to produce markdown output.

packages/baoyu-fetch/.github/workflows/release.yml

@@ -0,0 +1,58 @@
+name: Release
+
+on:
+  push:
+    branches:
+      - main
+  workflow_dispatch:
+
+concurrency: ${{ github.workflow }}-${{ github.ref }}
+
+permissions:
+  contents: write
+  pull-requests: write
+  id-token: write
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v5
+        with:
+          fetch-depth: 0
+
+      - name: Setup Bun
+        uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: 1.2.23
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v6
+        with:
+          node-version: 24
+          registry-url: https://registry.npmjs.org
+
+      - name: Install dependencies
+        run: bun install --frozen-lockfile
+
+      - name: Type check
+        run: bun run check
+
+      - name: Test
+        run: bun test
+
+      - name: Verify npm package contents
+        run: npm pack --dry-run
+
+      - name: Create release PR or publish to npm
+        id: changesets
+        uses: changesets/action@v1
+        with:
+          version: bun run version-packages
+          publish: bun run release
+          commit: "chore: release"
+          title: "chore: release"
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

packages/baoyu-fetch/.gitignore

@@ -0,0 +1,6 @@
+node_modules
+dist
+.DS_Store
+.debug
+coverage
+

packages/baoyu-fetch/CHANGELOG.md

@@ -0,0 +1,65 @@
+# Changelog
+
+English | [简体中文](./CHANGELOG.zh-CN.md)
+
+The format is inspired by Keep a Changelog, and the project follows Semantic
+Versioning.
+
+## [0.1.2] - 2026-04-21
+
+#### Changed
+
+- Upgraded Defuddle to 0.17.0 and jsdom to 29.0.2 for generic extraction.
+- Added an `@xmldom/xmldom` override to keep the optional Defuddle MathML
+  dependency chain on a non-vulnerable release.
+
+#### Fixed
+
+- Fixed X/Twitter video extraction for single posts and X Articles by selecting
+  the highest-bitrate MP4 variant instead of the preview image URL.
+- Fixed X Article media rendering so video entities are emitted as
+  `[video](...)` links instead of image embeds.
+
+## [0.1.1] - 2026-03-27
+
+#### Added
+
+- Added the `hn` adapter for Hacker News stories and comment threads.
+- Added `--download-media` and `--media-dir` to download extracted media and
+  rewrite Markdown links.
+- Added Defuddle as the first generic extraction pass, with Readability +
+  HTML-to-Markdown as fallback.
+- Added interactive wait modes for login and verification flows, including
+  manual verification handoff and force-wait resume behavior.
+- Added `--format markdown|json` while keeping `--json` as a compatibility
+  alias.
+- Added Changesets-based release automation for npm publishing.
+
+#### Changed
+
+- Renamed the package and CLI from `baoyu-markdown` to `baoyu-fetch`.
+- Changed the published package to run `src/cli.ts` directly with Bun instead of
+  shipping a prebuilt `dist`.
+- Improved X extraction for threads, articles, note tweets, embeds, image URLs,
+  login state handling, and media metadata.
+- Improved YouTube transcript extraction and normalized Markdown image output.
+
+#### Fixed
+
+- Fixed X note tweet URL expansion.
+- Fixed media URL normalization before download, including Substack media links.
+- Fixed foreground behavior for interactive flows so manual steps are easier to
+  complete.
+
+## [0.1.0] - 2026-03-25
+
+#### Added
+
+- Initial public release as `baoyu-markdown`.
+- Added Chrome CDP session management, controlled tabs, and network journaling.
+- Added built-in adapters for `x`, `youtube`, and the generic fallback.
+- Added X article parsing, X single/tweet extraction, and YouTube transcript
+  extraction.
+- Added Markdown rendering, document metadata output, and CLI support for file
+  output, JSON output, debug exports, custom Chrome connection settings,
+  headless mode, and timeout control.

packages/baoyu-fetch/CHANGELOG.zh-CN.md

@@ -0,0 +1,55 @@
+# 更新日志
+
+[English](./CHANGELOG.md) | 简体中文
+
+格式参考 Keep a Changelog，版本号遵循 Semantic Versioning。
+
+## [0.1.2] - 2026-04-21
+
+### 变更
+
+- 将 Defuddle 升级到 0.17.0、jsdom 升级到 29.0.2，用于通用页面提取。
+- 新增 `@xmldom/xmldom` override，使 Defuddle 的可选 MathML 依赖链保持在无漏洞版本。
+
+### 修复
+
+- 修复 X/Twitter 单条内容和 X Article 的视频提取逻辑，改为选择最高码率 MP4 变体，而不是预览图 URL。
+- 修复 X Article 媒体渲染，视频实体现在输出为 `[video](...)` 链接，而不是图片嵌入。
+
+## [0.1.1] - 2026-03-27
+
+### 新增
+
+- 新增 `hn` adapter，可提取 Hacker News 帖子与评论串。
+- 新增 `--download-media` 和 `--media-dir`，可下载提取出的媒体文件并重写
+  Markdown 链接。
+- 通用提取链路新增 Defuddle 首选路径，并保留 Readability + HTML to Markdown
+  作为回退方案。
+- 新增登录/验证场景的交互等待模式，支持手动验证接管和 force wait 自动恢复。
+- 新增 `--format markdown|json`，同时保留 `--json` 作为兼容别名。
+- 新增基于 Changesets 的 npm 发版自动化流程。
+
+### 变更
+
+- 将包名和 CLI 名称从 `baoyu-markdown` 更名为 `baoyu-fetch`。
+- npm 发布物改为直接以 Bun 执行 `src/cli.ts`，不再附带预构建的 `dist`。
+- 强化 X 提取链路，覆盖 thread、article、note tweet、embed、图片 URL、
+  登录态判断与媒体元数据。
+- 增强 YouTube transcript 提取，并规范化 Markdown 图片输出。
+
+### 修复
+
+- 修复 X note tweet 的 URL 展开问题。
+- 修复媒体下载前的 URL 规范化问题，包括 Substack 媒体链接。
+- 修复交互模式的前台行为，使手动登录/验证流程更稳定。
+
+## [0.1.0] - 2026-03-25
+
+### 新增
+
+- `baoyu-markdown` 的首个公开版本。
+- 新增 Chrome CDP 会话管理、受控 tab 与网络日志采集能力。
+- 新增内置 `x`、`youtube` 与通用 fallback adapters。
+- 新增 X article 解析、X 单条内容提取，以及 YouTube transcript 提取。
+- 新增 Markdown 渲染与文档元数据输出，并提供文件输出、JSON 输出、调试导出、
+  自定义 Chrome 连接、headless 模式和超时控制等 CLI 能力。

packages/baoyu-fetch/README.md

@@ -0,0 +1,124 @@
+# baoyu-fetch
+
+English | [简体中文](./README.zh-CN.md) | [Changelog](./CHANGELOG.md) | [中文更新日志](./CHANGELOG.zh-CN.md)
+
+`baoyu-fetch` is a Bun CLI built on Chrome CDP. Give it a URL and it returns
+high-quality `markdown` or `json`. When a site adapter matches, it prefers API
+responses or structured page data; otherwise it falls back to generic HTML
+extraction.
+
+## Features
+
+- Capture rendered page content through Chrome CDP
+- Observe network requests and responses, and fetch bodies when needed
+- Adapter registry that auto-selects a handler from the URL
+- Built-in adapters for `x`, `youtube`, and `hn`
+- Generic fallback: Defuddle first, then Readability + HTML-to-Markdown; when `--format markdown` is requested, it can also fall back to `defuddle.md`
+- Print `markdown` / `json` to stdout or save with `--output`
+- Optionally download extracted images or videos and rewrite Markdown links
+- Optional wait modes for login and verification flows
+- Chrome profile defaults to `baoyu-skills/chrome-profile`
+
+## Installation
+
+```bash
+bun install
+```
+
+For package usage, the quickest option is:
+
+```bash
+bunx baoyu-fetch https://example.com
+```
+
+You can also install it globally:
+
+```bash
+npm install -g baoyu-fetch
+```
+
+The npm package ships TypeScript source entrypoints instead of a prebuilt
+`dist`, so Bun is required at runtime.
+
+## Usage
+
+```bash
+bun run src/cli.ts https://example.com
+bunx baoyu-fetch https://example.com
+baoyu-fetch https://example.com
+baoyu-fetch https://example.com --format markdown --output article.md
+baoyu-fetch https://example.com --format markdown --output article.md --download-media
+baoyu-fetch https://x.com/jack/status/20 --format json --output article.json
+baoyu-fetch https://x.com/jack/status/20 --json
+baoyu-fetch https://x.com/jack/status/20 --wait-for interaction
+baoyu-fetch https://x.com/jack/status/20 --wait-for force
+baoyu-fetch https://x.com/jack/status/20 --chrome-profile-dir ~/Library/Application\\ Support/baoyu-skills/chrome-profile
+```
+
+## Options
+
+```bash
+baoyu-fetch <url> [options]
+
+Options:
+  --output <file>       Save output to file
+  --format <type>       Output format: markdown | json
+  --json                Alias for --format json
+  --adapter <name>      Force an adapter (for example x / hn / generic)
+  --download-media      Download adapter-reported media into ./imgs and ./videos, then rewrite markdown links
+  --media-dir <dir>     Base directory for downloaded media. Defaults to the output directory
+  --debug-dir <dir>     Write debug artifacts (html, document.json, network.json)
+  --cdp-url <url>       Reuse an existing Chrome DevTools endpoint
+  --browser-path <path> Explicit Chrome binary path
+  --chrome-profile-dir <path>
+                        Chrome user data dir. Defaults to BAOYU_CHROME_PROFILE_DIR
+                        or baoyu-skills/chrome-profile
+  --headless            Launch a temporary headless Chrome if needed
+  --wait-for <mode>     Wait mode: interaction | force
+  --wait-for-interaction
+                        Alias for --wait-for interaction
+  --wait-for-login      Alias for --wait-for interaction
+  --interaction-timeout <ms>
+                        Manual interaction timeout. Default: 600000
+  --interaction-poll-interval <ms>
+                        Poll interval while waiting. Default: 1500
+  --login-timeout <ms>  Alias for --interaction-timeout
+  --login-poll-interval <ms>
+                        Alias for --interaction-poll-interval
+  --timeout <ms>        Page load timeout. Default: 30000
+  --help                Show help
+```
+
+## How It Works
+
+1. The CLI parses the target URL and options.
+2. It opens or connects to a Chrome CDP session and creates a controlled tab.
+3. `NetworkJournal` records requests and responses.
+4. The adapter registry resolves a site-specific adapter when possible.
+5. The adapter returns a structured `ExtractedDocument`.
+6. If nothing matches, generic HTML extraction runs instead.
+7. The result is rendered as Markdown, or returned as JSON with both
+   `document` and `markdown`.
+
+## Development
+
+```bash
+bun run check
+bun run test
+bun run build
+```
+
+## Release
+
+When you make a user-visible change, add a changeset first:
+
+```bash
+bunx changeset
+```
+
+After the generated `.changeset/*.md` file lands on `main`, GitHub Actions will
+open or update the release PR. Merging that release PR publishes the package to
+npm.
+
+The publish flow does not build `dist`; it publishes `src/*.ts` for Bun
+execution directly.

packages/baoyu-fetch/README.zh-CN.md

@@ -0,0 +1,122 @@
+# baoyu-fetch
+
+[English](./README.md) | 简体中文 | [更新日志](./CHANGELOG.zh-CN.md) | [English Changelog](./CHANGELOG.md)
+
+`baoyu-fetch` 是一个基于 Chrome CDP 的 Bun CLI。输入 URL，它会输出高质量
+`markdown` 或 `json`；命中站点 adapter 时优先消费 API 返回或页面内结构化
+数据，未命中时回退到通用 HTML 提取。
+
+## 当前能力
+
+- 通过 Chrome CDP 抓取渲染后的页面内容
+- 监听网络请求与响应，按需拉取响应体
+- adapter registry，支持按 URL 自动命中站点处理器
+- 内置 `x`、`youtube`、`hn` adapters
+- 通用 fallback：Defuddle 优先，Readability + HTML to Markdown 回退；`--format markdown` 时会再尝试 `defuddle.md` 兜底
+- `stdout` 或 `--output` 输出 `markdown` / `json`
+- 可选下载提取出的图片/视频并重写 Markdown 链接
+- 提供登录/验证场景下的交互等待模式
+- Chrome profile 默认对齐 `baoyu-skills/chrome-profile`
+
+## 安装
+
+```bash
+bun install
+```
+
+作为包使用时，推荐直接这样运行：
+
+```bash
+bunx baoyu-fetch https://example.com
+```
+
+也可以全局安装：
+
+```bash
+npm install -g baoyu-fetch
+```
+
+npm 包发布的是 TypeScript 源码入口，不包含预编译的 `dist`，所以运行时需要
+Bun。
+
+## 用法
+
+```bash
+bun run src/cli.ts https://example.com
+bunx baoyu-fetch https://example.com
+baoyu-fetch https://example.com
+baoyu-fetch https://example.com --format markdown --output article.md
+baoyu-fetch https://example.com --format markdown --output article.md --download-media
+baoyu-fetch https://x.com/jack/status/20 --format json --output article.json
+baoyu-fetch https://x.com/jack/status/20 --json
+baoyu-fetch https://x.com/jack/status/20 --wait-for interaction
+baoyu-fetch https://x.com/jack/status/20 --wait-for force
+baoyu-fetch https://x.com/jack/status/20 --chrome-profile-dir ~/Library/Application\\ Support/baoyu-skills/chrome-profile
+```
+
+## 主要参数
+
+```bash
+baoyu-fetch <url> [options]
+
+Options:
+  --output <file>       保存输出内容到文件
+  --format <type>       输出格式：markdown | json
+  --json                `--format json` 的兼容别名
+  --adapter <name>      强制使用指定 adapter（如 x / hn / generic）
+  --download-media      下载 adapter 返回的媒体到 ./imgs 和 ./videos，并重写 markdown 链接
+  --media-dir <dir>     指定媒体下载根目录；默认使用输出文件所在目录
+  --debug-dir <dir>     导出调试信息（html、document.json、network.json）
+  --cdp-url <url>       连接现有 Chrome 调试地址
+  --browser-path <path> 指定 Chrome 可执行文件
+  --chrome-profile-dir <path>
+                        指定 Chrome profile 目录。默认使用 BAOYU_CHROME_PROFILE_DIR，
+                        否则回退到 baoyu-skills/chrome-profile
+  --headless            启动临时 headless Chrome（未连现有实例时）
+  --wait-for <mode>     等待模式：interaction | force
+  --wait-for-interaction
+                        `--wait-for interaction` 的别名
+  --wait-for-login      `--wait-for interaction` 的别名
+  --interaction-timeout <ms>
+                        手动交互等待超时，默认 600000
+  --interaction-poll-interval <ms>
+                        等待期间的轮询间隔，默认 1500
+  --login-timeout <ms>  `--interaction-timeout` 的别名
+  --login-poll-interval <ms>
+                        `--interaction-poll-interval` 的别名
+  --timeout <ms>        页面加载超时，默认 30000
+  --help                显示帮助
+```
+
+## 设计
+
+核心链路：
+
+1. CLI 解析 URL 和选项
+2. 建立 CDP 会话并创建受控 tab
+3. 启动 `NetworkJournal` 收集所有请求/响应
+4. 由 adapter registry 匹配站点 adapter
+5. adapter 返回结构化 `ExtractedDocument`
+6. 没命中则走通用 HTML 提取
+7. 按请求输出 Markdown，或输出包含 `document` 和 `markdown` 的 JSON
+
+## 开发
+
+```bash
+bun run check
+bun run test
+bun run build
+```
+
+## 发版
+
+新增用户可见改动后，先添加一个 changeset：
+
+```bash
+bunx changeset
+```
+
+把生成的 `.changeset/*.md` 一起合并到 `main` 后，GitHub Actions 会自动创建或
+更新 release PR；合并 release PR 之后，会自动发布到 npm。
+
+发布流程不会编译 `dist`，而是直接把 `src/*.ts` 发布出去供 Bun 执行。

packages/baoyu-fetch/bun.lock

@@ -0,0 +1,530 @@
+{
+  "lockfileVersion": 1,
+  "configVersion": 0,
+  "workspaces": {
+    "": {
+      "name": "baoyu-fetch",
+      "dependencies": {
+        "@mozilla/readability": "^0.6.0",
+        "chrome-launcher": "^1.2.1",
+        "defuddle": "^0.17.0",
+        "jsdom": "^29.0.2",
+        "remark-gfm": "^4.0.1",
+        "remark-parse": "^11.0.0",
+        "turndown": "^7.2.0",
+        "turndown-plugin-gfm": "^1.0.2",
+        "unified": "^11.0.5",
+        "ws": "^8.18.3",
+      },
+      "devDependencies": {
+        "@changesets/cli": "^2.30.0",
+        "@types/bun": "^1.2.23",
+        "@types/jsdom": "^21.1.7",
+        "@types/ws": "^8.18.1",
+        "typescript": "^5.9.2",
+      },
+    },
+  },
+  "overrides": {
+    "@xmldom/xmldom": "0.8.13",
+  },
+  "packages": {
+    "@asamuzakjp/css-color": ["@asamuzakjp/css-color@5.1.11", "", { "dependencies": { "@asamuzakjp/generational-cache": "^1.0.1", "@csstools/css-calc": "^3.2.0", "@csstools/css-color-parser": "^4.1.0", "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-KVw6qIiCTUQhByfTd78h2yD1/00waTmm9uy/R7Ck/ctUyAPj+AEDLkQIdJW0T8+qGgj3j5bpNKK7Q3G+LedJWg=="],
+
+    "@asamuzakjp/dom-selector": ["@asamuzakjp/dom-selector@7.1.1", "", { "dependencies": { "@asamuzakjp/generational-cache": "^1.0.1", "@asamuzakjp/nwsapi": "^2.3.9", "bidi-js": "^1.0.3", "css-tree": "^3.2.1", "is-potential-custom-element-name": "^1.0.1" } }, "sha512-67RZDnYRc8H/8MLDgQCDE//zoqVFwajkepHZgmXrbwybzXOEwOWGPYGmALYl9J2DOLfFPPs6kKCqmbzV895hTQ=="],
+
+    "@asamuzakjp/generational-cache": ["@asamuzakjp/generational-cache@1.0.1", "", {}, "sha512-wajfB8KqzMCN2KGNFdLkReeHncd0AslUSrvHVvvYWuU8ghncRJoA50kT3zP9MVL0+9g4/67H+cdvBskj9THPzg=="],
+
+    "@asamuzakjp/nwsapi": ["@asamuzakjp/nwsapi@2.3.9", "", {}, "sha512-n8GuYSrI9bF7FFZ/SjhwevlHc8xaVlb/7HmHelnc/PZXBD2ZR49NnN9sMMuDdEGPeeRQ5d0hqlSlEpgCX3Wl0Q=="],
+
+    "@babel/runtime": ["@babel/runtime@7.29.2", "", {}, "sha512-JiDShH45zKHWyGe4ZNVRrCjBz8Nh9TMmZG1kh4QTK8hCBTWBi8Da+i7s1fJw7/lYpM4ccepSNfqzZ/QvABBi5g=="],
+
+    "@bramus/specificity": ["@bramus/specificity@2.4.2", "", { "dependencies": { "css-tree": "^3.0.0" }, "bin": { "specificity": "bin/cli.js" } }, "sha512-ctxtJ/eA+t+6q2++vj5j7FYX3nRu311q1wfYH3xjlLOsczhlhxAg2FWNUXhpGvAw3BWo1xBcvOV6/YLc2r5FJw=="],
+
+    "@changesets/apply-release-plan": ["@changesets/apply-release-plan@7.1.0", "", { "dependencies": { "@changesets/config": "^3.1.3", "@changesets/get-version-range-type": "^0.4.0", "@changesets/git": "^3.0.4", "@changesets/should-skip-package": "^0.1.2", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "detect-indent": "^6.0.0", "fs-extra": "^7.0.1", "lodash.startcase": "^4.4.0", "outdent": "^0.5.0", "prettier": "^2.7.1", "resolve-from": "^5.0.0", "semver": "^7.5.3" } }, "sha512-yq8ML3YS7koKQ/9bk1PqO0HMzApIFNwjlwCnwFEXMzNe8NpzeeYYKCmnhWJGkN8g7E51MnWaSbqRcTcdIxUgnQ=="],
+
+    "@changesets/assemble-release-plan": ["@changesets/assemble-release-plan@6.0.9", "", { "dependencies": { "@changesets/errors": "^0.2.0", "@changesets/get-dependents-graph": "^2.1.3", "@changesets/should-skip-package": "^0.1.2", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "semver": "^7.5.3" } }, "sha512-tPgeeqCHIwNo8sypKlS3gOPmsS3wP0zHt67JDuL20P4QcXiw/O4Hl7oXiuLnP9yg+rXLQ2sScdV1Kkzde61iSQ=="],
+
+    "@changesets/changelog-git": ["@changesets/changelog-git@0.2.1", "", { "dependencies": { "@changesets/types": "^6.1.0" } }, "sha512-x/xEleCFLH28c3bQeQIyeZf8lFXyDFVn1SgcBiR2Tw/r4IAWlk1fzxCEZ6NxQAjF2Nwtczoen3OA2qR+UawQ8Q=="],
+
+    "@changesets/cli": ["@changesets/cli@2.30.0", "", { "dependencies": { "@changesets/apply-release-plan": "^7.1.0", "@changesets/assemble-release-plan": "^6.0.9", "@changesets/changelog-git": "^0.2.1", "@changesets/config": "^3.1.3", "@changesets/errors": "^0.2.0", "@changesets/get-dependents-graph": "^2.1.3", "@changesets/get-release-plan": "^4.0.15", "@changesets/git": "^3.0.4", "@changesets/logger": "^0.1.1", "@changesets/pre": "^2.0.2", "@changesets/read": "^0.6.7", "@changesets/should-skip-package": "^0.1.2", "@changesets/types": "^6.1.0", "@changesets/write": "^0.4.0", "@inquirer/external-editor": "^1.0.2", "@manypkg/get-packages": "^1.1.3", "ansi-colors": "^4.1.3", "enquirer": "^2.4.1", "fs-extra": "^7.0.1", "mri": "^1.2.0", "package-manager-detector": "^0.2.0", "picocolors": "^1.1.0", "resolve-from": "^5.0.0", "semver": "^7.5.3", "spawndamnit": "^3.0.1", "term-size": "^2.1.0" }, "bin": { "changeset": "bin.js" } }, "sha512-5D3Nk2JPqMI1wK25pEymeWRSlSMdo5QOGlyfrKg0AOufrUcjEE3RQgaCpHoBiM31CSNrtSgdJ0U6zL1rLDDfBA=="],
+
+    "@changesets/config": ["@changesets/config@3.1.3", "", { "dependencies": { "@changesets/errors": "^0.2.0", "@changesets/get-dependents-graph": "^2.1.3", "@changesets/logger": "^0.1.1", "@changesets/should-skip-package": "^0.1.2", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "fs-extra": "^7.0.1", "micromatch": "^4.0.8" } }, "sha512-vnXjcey8YgBn2L1OPWd3ORs0bGC4LoYcK/ubpgvzNVr53JXV5GiTVj7fWdMRsoKUH7hhhMAQnsJUqLr21EncNw=="],
+
+    "@changesets/errors": ["@changesets/errors@0.2.0", "", { "dependencies": { "extendable-error": "^0.1.5" } }, "sha512-6BLOQUscTpZeGljvyQXlWOItQyU71kCdGz7Pi8H8zdw6BI0g3m43iL4xKUVPWtG+qrrL9DTjpdn8eYuCQSRpow=="],
+
+    "@changesets/get-dependents-graph": ["@changesets/get-dependents-graph@2.1.3", "", { "dependencies": { "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "picocolors": "^1.1.0", "semver": "^7.5.3" } }, "sha512-gphr+v0mv2I3Oxt19VdWRRUxq3sseyUpX9DaHpTUmLj92Y10AGy+XOtV+kbM6L/fDcpx7/ISDFK6T8A/P3lOdQ=="],
+
+    "@changesets/get-release-plan": ["@changesets/get-release-plan@4.0.15", "", { "dependencies": { "@changesets/assemble-release-plan": "^6.0.9", "@changesets/config": "^3.1.3", "@changesets/pre": "^2.0.2", "@changesets/read": "^0.6.7", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3" } }, "sha512-Q04ZaRPuEVZtA+auOYgFaVQQSA98dXiVe/yFaZfY7hoSmQICHGvP0TF4u3EDNHWmmCS4ekA/XSpKlSM2PyTS2g=="],
+
+    "@changesets/get-version-range-type": ["@changesets/get-version-range-type@0.4.0", "", {}, "sha512-hwawtob9DryoGTpixy1D3ZXbGgJu1Rhr+ySH2PvTLHvkZuQ7sRT4oQwMh0hbqZH1weAooedEjRsbrWcGLCeyVQ=="],
+
+    "@changesets/git": ["@changesets/git@3.0.4", "", { "dependencies": { "@changesets/errors": "^0.2.0", "@manypkg/get-packages": "^1.1.3", "is-subdir": "^1.1.1", "micromatch": "^4.0.8", "spawndamnit": "^3.0.1" } }, "sha512-BXANzRFkX+XcC1q/d27NKvlJ1yf7PSAgi8JG6dt8EfbHFHi4neau7mufcSca5zRhwOL8j9s6EqsxmT+s+/E6Sw=="],
+
+    "@changesets/logger": ["@changesets/logger@0.1.1", "", { "dependencies": { "picocolors": "^1.1.0" } }, "sha512-OQtR36ZlnuTxKqoW4Sv6x5YIhOmClRd5pWsjZsddYxpWs517R0HkyiefQPIytCVh4ZcC5x9XaG8KTdd5iRQUfg=="],
+
+    "@changesets/parse": ["@changesets/parse@0.4.3", "", { "dependencies": { "@changesets/types": "^6.1.0", "js-yaml": "^4.1.1" } }, "sha512-ZDmNc53+dXdWEv7fqIUSgRQOLYoUom5Z40gmLgmATmYR9NbL6FJJHwakcCpzaeCy+1D0m0n7mT4jj2B/MQPl7A=="],
+
+    "@changesets/pre": ["@changesets/pre@2.0.2", "", { "dependencies": { "@changesets/errors": "^0.2.0", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "fs-extra": "^7.0.1" } }, "sha512-HaL/gEyFVvkf9KFg6484wR9s0qjAXlZ8qWPDkTyKF6+zqjBe/I2mygg3MbpZ++hdi0ToqNUF8cjj7fBy0dg8Ug=="],
+
+    "@changesets/read": ["@changesets/read@0.6.7", "", { "dependencies": { "@changesets/git": "^3.0.4", "@changesets/logger": "^0.1.1", "@changesets/parse": "^0.4.3", "@changesets/types": "^6.1.0", "fs-extra": "^7.0.1", "p-filter": "^2.1.0", "picocolors": "^1.1.0" } }, "sha512-D1G4AUYGrBEk8vj8MGwf75k9GpN6XL3wg8i42P2jZZwFLXnlr2Pn7r9yuQNbaMCarP7ZQWNJbV6XLeysAIMhTA=="],
+
+    "@changesets/should-skip-package": ["@changesets/should-skip-package@0.1.2", "", { "dependencies": { "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3" } }, "sha512-qAK/WrqWLNCP22UDdBTMPH5f41elVDlsNyat180A33dWxuUDyNpg6fPi/FyTZwRriVjg0L8gnjJn2F9XAoF0qw=="],
+
+    "@changesets/types": ["@changesets/types@6.1.0", "", {}, "sha512-rKQcJ+o1nKNgeoYRHKOS07tAMNd3YSN0uHaJOZYjBAgxfV7TUE7JE+z4BzZdQwb5hKaYbayKN5KrYV7ODb2rAA=="],
+
+    "@changesets/write": ["@changesets/write@0.4.0", "", { "dependencies": { "@changesets/types": "^6.1.0", "fs-extra": "^7.0.1", "human-id": "^4.1.1", "prettier": "^2.7.1" } }, "sha512-CdTLvIOPiCNuH71pyDu3rA+Q0n65cmAbXnwWH84rKGiFumFzkmHNT8KHTMEchcxN+Kl8I54xGUhJ7l3E7X396Q=="],
+
+    "@csstools/color-helpers": ["@csstools/color-helpers@6.0.2", "", {}, "sha512-LMGQLS9EuADloEFkcTBR3BwV/CGHV7zyDxVRtVDTwdI2Ca4it0CCVTT9wCkxSgokjE5Ho41hEPgb8OEUwoXr6Q=="],
+
+    "@csstools/css-calc": ["@csstools/css-calc@3.2.0", "", { "peerDependencies": { "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-bR9e6o2BDB12jzN/gIbjHa5wLJ4UjD1CB9pM7ehlc0ddk6EBz+yYS1EV2MF55/HUxrHcB/hehAyt5vhsA3hx7w=="],
+
+    "@csstools/css-color-parser": ["@csstools/css-color-parser@4.1.0", "", { "dependencies": { "@csstools/color-helpers": "^6.0.2", "@csstools/css-calc": "^3.2.0" }, "peerDependencies": { "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-U0KhLYmy2GVj6q4T3WaAe6NPuFYCPQoE3b0dRGxejWDgcPp8TP7S5rVdM5ZrFaqu4N67X8YaPBw14dQSYx3IyQ=="],
+
+    "@csstools/css-parser-algorithms": ["@csstools/css-parser-algorithms@4.0.0", "", { "peerDependencies": { "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-+B87qS7fIG3L5h3qwJ/IFbjoVoOe/bpOdh9hAjXbvx0o8ImEmUsGXN0inFOnk2ChCFgqkkGFQ+TpM5rbhkKe4w=="],
+
+    "@csstools/css-syntax-patches-for-csstree": ["@csstools/css-syntax-patches-for-csstree@1.1.3", "", { "peerDependencies": { "css-tree": "^3.2.1" }, "optionalPeers": ["css-tree"] }, "sha512-SH60bMfrRCJF3morcdk57WklujF4Jr/EsQUzqkarfHXEFcAR1gg7fS/chAE922Sehgzc1/+Tz5H3Ypa1HiEKrg=="],
+
+    "@csstools/css-tokenizer": ["@csstools/css-tokenizer@4.0.0", "", {}, "sha512-QxULHAm7cNu72w97JUNCBFODFaXpbDg+dP8b/oWFAZ2MTRppA3U00Y2L1HqaS4J6yBqxwa/Y3nMBaxVKbB/NsA=="],
+
+    "@exodus/bytes": ["@exodus/bytes@1.15.0", "", { "peerDependencies": { "@noble/hashes": "^1.8.0 || ^2.0.0" }, "optionalPeers": ["@noble/hashes"] }, "sha512-UY0nlA+feH81UGSHv92sLEPLCeZFjXOuHhrIo0HQydScuQc8s0A7kL/UdgwgDq8g8ilksmuoF35YVTNphV2aBQ=="],
+
+    "@inquirer/external-editor": ["@inquirer/external-editor@1.0.3", "", { "dependencies": { "chardet": "^2.1.1", "iconv-lite": "^0.7.0" }, "peerDependencies": { "@types/node": ">=18" }, "optionalPeers": ["@types/node"] }, "sha512-RWbSrDiYmO4LbejWY7ttpxczuwQyZLBUyygsA9Nsv95hpzUWwnNTVQmAq3xuh7vNwCp07UTmE5i11XAEExx4RA=="],
+
+    "@manypkg/find-root": ["@manypkg/find-root@1.1.0", "", { "dependencies": { "@babel/runtime": "^7.5.5", "@types/node": "^12.7.1", "find-up": "^4.1.0", "fs-extra": "^8.1.0" } }, "sha512-mki5uBvhHzO8kYYix/WRy2WX8S3B5wdVSc9D6KcU5lQNglP2yt58/VfLuAK49glRXChosY8ap2oJ1qgma3GUVA=="],
+
+    "@manypkg/get-packages": ["@manypkg/get-packages@1.1.3", "", { "dependencies": { "@babel/runtime": "^7.5.5", "@changesets/types": "^4.0.1", "@manypkg/find-root": "^1.1.0", "fs-extra": "^8.1.0", "globby": "^11.0.0", "read-yaml-file": "^1.1.0" } }, "sha512-fo+QhuU3qE/2TQMQmbVMqaQ6EWbMhi4ABWP+O4AM1NqPBuy0OrApV5LO6BrrgnhtAHS2NH6RrVk9OL181tTi8A=="],
+
+    "@mixmark-io/domino": ["@mixmark-io/domino@2.2.0", "", {}, "sha512-Y28PR25bHXUg88kCV7nivXrP2Nj2RueZ3/l/jdx6J9f8J4nsEGcgX0Qe6lt7Pa+J79+kPiJU3LguR6O/6zrLOw=="],
+
+    "@mozilla/readability": ["@mozilla/readability@0.6.0", "", {}, "sha512-juG5VWh4qAivzTAeMzvY9xs9HY5rAcr2E4I7tiSSCokRFi7XIZCAu92ZkSTsIj1OPceCifL3cpfteP3pDT9/QQ=="],
+
+    "@nodelib/fs.scandir": ["@nodelib/fs.scandir@2.1.5", "", { "dependencies": { "@nodelib/fs.stat": "2.0.5", "run-parallel": "^1.1.9" } }, "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g=="],
+
+    "@nodelib/fs.stat": ["@nodelib/fs.stat@2.0.5", "", {}, "sha512-RkhPPp2zrqDAQA/2jNhnztcPAlv64XdhIp7a7454A5ovI7Bukxgt7MX7udwAu3zg1DcpPU0rz3VV1SeaqvY4+A=="],
+
+    "@nodelib/fs.walk": ["@nodelib/fs.walk@1.2.8", "", { "dependencies": { "@nodelib/fs.scandir": "2.1.5", "fastq": "^1.6.0" } }, "sha512-oGB+UxlgWcgQkgwo8GcEGwemoTFt3FIO9ababBmaGwXIoBKZ+GTy0pP185beGg7Llih/NSHSV2XAs1lnznocSg=="],
+
+    "@types/bun": ["@types/bun@1.3.11", "", { "dependencies": { "bun-types": "1.3.11" } }, "sha512-5vPne5QvtpjGpsGYXiFyycfpDF2ECyPcTSsFBMa0fraoxiQyMJ3SmuQIGhzPg2WJuWxVBoxWJ2kClYTcw/4fAg=="],
+
+    "@types/debug": ["@types/debug@4.1.13", "", { "dependencies": { "@types/ms": "*" } }, "sha512-KSVgmQmzMwPlmtljOomayoR89W4FynCAi3E8PPs7vmDVPe84hT+vGPKkJfThkmXs0x0jAaa9U8uW8bbfyS2fWw=="],
+
+    "@types/jsdom": ["@types/jsdom@21.1.7", "", { "dependencies": { "@types/node": "*", "@types/tough-cookie": "*", "parse5": "^7.0.0" } }, "sha512-yOriVnggzrnQ3a9OKOCxaVuSug3w3/SbOj5i7VwXWZEyUNl3bLF9V3MfxGbZKuwqJOQyRfqXyROBB1CoZLFWzA=="],
+
+    "@types/mdast": ["@types/mdast@4.0.4", "", { "dependencies": { "@types/unist": "*" } }, "sha512-kGaNbPh1k7AFzgpud/gMdvIm5xuECykRR+JnWKQno9TAXVa6WIVCGTPvYGekIDL4uwCZQSYbUxNBSb1aUo79oA=="],
+
+    "@types/ms": ["@types/ms@2.1.0", "", {}, "sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA=="],
+
+    "@types/node": ["@types/node@25.5.0", "", { "dependencies": { "undici-types": "~7.18.0" } }, "sha512-jp2P3tQMSxWugkCUKLRPVUpGaL5MVFwF8RDuSRztfwgN1wmqJeMSbKlnEtQqU8UrhTmzEmZdu2I6v2dpp7XIxw=="],
+
+    "@types/tough-cookie": ["@types/tough-cookie@4.0.5", "", {}, "sha512-/Ad8+nIOV7Rl++6f1BdKxFSMgmoqEoYbHRpPcx3JEfv8VRsQe9Z4mCXeJBzxs7mbHY/XOZZuXlRNfhpVPbs6ZA=="],
+
+    "@types/unist": ["@types/unist@3.0.3", "", {}, "sha512-ko/gIFJRv177XgZsZcBwnqJN5x/Gien8qNOn0D5bQU/zAzVf9Zt3BlcUiLqhV9y4ARk0GbT3tnUiPNgnTXzc/Q=="],
+
+    "@types/ws": ["@types/ws@8.18.1", "", { "dependencies": { "@types/node": "*" } }, "sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg=="],
+
+    "@xmldom/xmldom": ["@xmldom/xmldom@0.8.13", "", {}, "sha512-KRYzxepc14G/CEpEGc3Yn+JKaAeT63smlDr+vjB8jRfgTBBI9wRj/nkQEO+ucV8p8I9bfKLWp37uHgFrbntPvw=="],
+
+    "ansi-colors": ["ansi-colors@4.1.3", "", {}, "sha512-/6w/C21Pm1A7aZitlI5Ni/2J6FFQN8i1Cvz3kHABAAbw93v/NlvKdVOqz7CCWz/3iv/JplRSEEZ83XION15ovw=="],
+
+    "ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="],
+
+    "argparse": ["argparse@2.0.1", "", {}, "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q=="],
+
+    "array-union": ["array-union@2.1.0", "", {}, "sha512-HGyxoOTYUyCM6stUe6EJgnd4EoewAI7zMdfqO+kGjnlZmBDz/cR5pf8r/cR4Wq60sL/p0IkcjUEEPwS3GFrIyw=="],
+
+    "bail": ["bail@2.0.2", "", {}, "sha512-0xO6mYd7JB2YesxDKplafRpsiOzPt9V02ddPCLbY1xYGPOX24NTyN50qnUxgCPcSoYMhKpAuBTjQoRZCAkUDRw=="],
+
+    "better-path-resolve": ["better-path-resolve@1.0.0", "", { "dependencies": { "is-windows": "^1.0.0" } }, "sha512-pbnl5XzGBdrFU/wT4jqmJVPn2B6UHPBOhzMQkY/SPUPB6QtUXtmBHBIwCbXJol93mOpGMnQyP/+BB19q04xj7g=="],
+
+    "bidi-js": ["bidi-js@1.0.3", "", { "dependencies": { "require-from-string": "^2.0.2" } }, "sha512-RKshQI1R3YQ+n9YJz2QQ147P66ELpa1FQEg20Dk8oW9t2KgLbpDLLp9aGZ7y8WHSshDknG0bknqGw5/tyCs5tw=="],
+
+    "boolbase": ["boolbase@1.0.0", "", {}, "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww=="],
+
+    "braces": ["braces@3.0.3", "", { "dependencies": { "fill-range": "^7.1.1" } }, "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA=="],
+
+    "bun-types": ["bun-types@1.3.11", "", { "dependencies": { "@types/node": "*" } }, "sha512-1KGPpoxQWl9f6wcZh57LvrPIInQMn2TQ7jsgxqpRzg+l0QPOFvJVH7HmvHo/AiPgwXy+/Thf6Ov3EdVn1vOabg=="],
+
+    "ccount": ["ccount@2.0.1", "", {}, "sha512-eyrF0jiFpY+3drT6383f1qhkbGsLSifNAjA61IUjZjmLCWjItY6LB9ft9YhoDgwfmclB2zhu51Lc7+95b8NRAg=="],
+
+    "character-entities": ["character-entities@2.0.2", "", {}, "sha512-shx7oQ0Awen/BRIdkjkvz54PnEEI/EjwXDSIZp86/KKdbafHh1Df/RYGBhn4hbe2+uKC9FnT5UCEdyPz3ai9hQ=="],
+
+    "chardet": ["chardet@2.1.1", "", {}, "sha512-PsezH1rqdV9VvyNhxxOW32/d75r01NY7TQCmOqomRo15ZSOKbpTFVsfjghxo6JloQUCGnH4k1LGu0R4yCLlWQQ=="],
+
+    "chrome-launcher": ["chrome-launcher@1.2.1", "", { "dependencies": { "@types/node": "*", "escape-string-regexp": "^4.0.0", "is-wsl": "^2.2.0", "lighthouse-logger": "^2.0.1" }, "bin": { "print-chrome-path": "bin/print-chrome-path.cjs" } }, "sha512-qmFR5PLMzHyuNJHwOloHPAHhbaNglkfeV/xDtt5b7xiFFyU1I+AZZX0PYseMuhenJSSirgxELYIbswcoc+5H4A=="],
+
+    "commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="],
+
+    "cross-spawn": ["cross-spawn@7.0.6", "", { "dependencies": { "path-key": "^3.1.0", "shebang-command": "^2.0.0", "which": "^2.0.1" } }, "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA=="],
+
+    "css-select": ["css-select@5.2.2", "", { "dependencies": { "boolbase": "^1.0.0", "css-what": "^6.1.0", "domhandler": "^5.0.2", "domutils": "^3.0.1", "nth-check": "^2.0.1" } }, "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw=="],
+
+    "css-tree": ["css-tree@3.2.1", "", { "dependencies": { "mdn-data": "2.27.1", "source-map-js": "^1.2.1" } }, "sha512-X7sjQzceUhu1u7Y/ylrRZFU2FS6LRiFVp6rKLPg23y3x3c3DOKAwuXGDp+PAGjh6CSnCjYeAul8pcT8bAl+lSA=="],
+
+    "css-what": ["css-what@6.2.2", "", {}, "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA=="],
+
+    "cssom": ["cssom@0.5.0", "", {}, "sha512-iKuQcq+NdHqlAcwUY0o/HL69XQrUaQdMjmStJ8JFmUaiiQErlhrmuigkg/CU4E2J0IyUKUrMAgl36TvN67MqTw=="],
+
+    "data-urls": ["data-urls@7.0.0", "", { "dependencies": { "whatwg-mimetype": "^5.0.0", "whatwg-url": "^16.0.0" } }, "sha512-23XHcCF+coGYevirZceTVD7NdJOqVn+49IHyxgszm+JIiHLoB2TkmPtsYkNWT1pvRSGkc35L6NHs0yHkN2SumA=="],
+
+    "debug": ["debug@4.4.3", "", { "dependencies": { "ms": "^2.1.3" } }, "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA=="],
+
+    "decimal.js": ["decimal.js@10.6.0", "", {}, "sha512-YpgQiITW3JXGntzdUmyUR1V812Hn8T1YVXhCu+wO3OpS4eU9l4YdD3qjyiKdV6mvV29zapkMeD390UVEf2lkUg=="],
+
+    "decode-named-character-reference": ["decode-named-character-reference@1.3.0", "", { "dependencies": { "character-entities": "^2.0.0" } }, "sha512-GtpQYB283KrPp6nRw50q3U9/VfOutZOe103qlN7BPP6Ad27xYnOIWv4lPzo8HCAL+mMZofJ9KEy30fq6MfaK6Q=="],
+
+    "defuddle": ["defuddle@0.17.0", "", { "dependencies": { "commander": "^12.1.0" }, "optionalDependencies": { "linkedom": "^0.18.12", "mathml-to-latex": "^1.5.0", "temml": "^0.13.1", "turndown": "^7.2.0" }, "bin": { "defuddle": "dist/cli.js" } }, "sha512-rjRb9kxgrJLFhFJjXTSAitbmmSyvjEqGCi5JiyM6ImOf4leC/O+f7TPVPAymk/gnENFTrn9j7T/LTfW9IVwiLw=="],
+
+    "dequal": ["dequal@2.0.3", "", {}, "sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA=="],
+
+    "detect-indent": ["detect-indent@6.1.0", "", {}, "sha512-reYkTUJAZb9gUuZ2RvVCNhVHdg62RHnJ7WJl8ftMi4diZ6NWlciOzQN88pUhSELEwflJht4oQDv0F0BMlwaYtA=="],
+
+    "devlop": ["devlop@1.1.0", "", { "dependencies": { "dequal": "^2.0.0" } }, "sha512-RWmIqhcFf1lRYBvNmr7qTNuyCt/7/ns2jbpp1+PalgE/rDQcBT0fioSMUpJ93irlUhC5hrg4cYqe6U+0ImW0rA=="],
+
+    "dir-glob": ["dir-glob@3.0.1", "", { "dependencies": { "path-type": "^4.0.0" } }, "sha512-WkrWp9GR4KXfKGYzOLmTuGVi1UWFfws377n9cc55/tb6DuqyF6pcQ5AbiHEshaDpY9v6oaSr2XCDidGmMwdzIA=="],
+
+    "dom-serializer": ["dom-serializer@2.0.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.2", "entities": "^4.2.0" } }, "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg=="],
+
+    "domelementtype": ["domelementtype@2.3.0", "", {}, "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw=="],
+
+    "domhandler": ["domhandler@5.0.3", "", { "dependencies": { "domelementtype": "^2.3.0" } }, "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w=="],
+
+    "domutils": ["domutils@3.2.2", "", { "dependencies": { "dom-serializer": "^2.0.0", "domelementtype": "^2.3.0", "domhandler": "^5.0.3" } }, "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw=="],
+
+    "enquirer": ["enquirer@2.4.1", "", { "dependencies": { "ansi-colors": "^4.1.1", "strip-ansi": "^6.0.1" } }, "sha512-rRqJg/6gd538VHvR3PSrdRBb/1Vy2YfzHqzvbhGIQpDRKIa4FgV/54b5Q1xYSxOOwKvjXweS26E0Q+nAMwp2pQ=="],
+
+    "entities": ["entities@6.0.1", "", {}, "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g=="],
+
+    "escape-string-regexp": ["escape-string-regexp@4.0.0", "", {}, "sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA=="],
+
+    "esprima": ["esprima@4.0.1", "", { "bin": { "esparse": "./bin/esparse.js", "esvalidate": "./bin/esvalidate.js" } }, "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A=="],
+
+    "extend": ["extend@3.0.2", "", {}, "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g=="],
+
+    "extendable-error": ["extendable-error@0.1.7", "", {}, "sha512-UOiS2in6/Q0FK0R0q6UY9vYpQ21mr/Qn1KOnte7vsACuNJf514WvCCUHSRCPcgjPT2bAhNIJdlE6bVap1GKmeg=="],
+
+    "fast-glob": ["fast-glob@3.3.3", "", { "dependencies": { "@nodelib/fs.stat": "^2.0.2", "@nodelib/fs.walk": "^1.2.3", "glob-parent": "^5.1.2", "merge2": "^1.3.0", "micromatch": "^4.0.8" } }, "sha512-7MptL8U0cqcFdzIzwOTHoilX9x5BrNqye7Z/LuC7kCMRio1EMSyqRK3BEAUD7sXRq4iT4AzTVuZdhgQ2TCvYLg=="],
+
+    "fastq": ["fastq@1.20.1", "", { "dependencies": { "reusify": "^1.0.4" } }, "sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw=="],
+
+    "fill-range": ["fill-range@7.1.1", "", { "dependencies": { "to-regex-range": "^5.0.1" } }, "sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg=="],
+
+    "find-up": ["find-up@4.1.0", "", { "dependencies": { "locate-path": "^5.0.0", "path-exists": "^4.0.0" } }, "sha512-PpOwAdQ/YlXQ2vj8a3h8IipDuYRi3wceVQQGYWxNINccq40Anw7BlsEXCMbt1Zt+OLA6Fq9suIpIWD0OsnISlw=="],
+
+    "fs-extra": ["fs-extra@7.0.1", "", { "dependencies": { "graceful-fs": "^4.1.2", "jsonfile": "^4.0.0", "universalify": "^0.1.0" } }, "sha512-YJDaCJZEnBmcbw13fvdAM9AwNOJwOzrE4pqMqBq5nFiEqXUqHwlK4B+3pUw6JNvfSPtX05xFHtYy/1ni01eGCw=="],
+
+    "glob-parent": ["glob-parent@5.1.2", "", { "dependencies": { "is-glob": "^4.0.1" } }, "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow=="],
+
+    "globby": ["globby@11.1.0", "", { "dependencies": { "array-union": "^2.1.0", "dir-glob": "^3.0.1", "fast-glob": "^3.2.9", "ignore": "^5.2.0", "merge2": "^1.4.1", "slash": "^3.0.0" } }, "sha512-jhIXaOzy1sb8IyocaruWSn1TjmnBVs8Ayhcy83rmxNJ8q2uWKCAj3CnJY+KpGSXCueAPc0i05kVvVKtP1t9S3g=="],
+
+    "graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="],
+
+    "html-encoding-sniffer": ["html-encoding-sniffer@6.0.0", "", { "dependencies": { "@exodus/bytes": "^1.6.0" } }, "sha512-CV9TW3Y3f8/wT0BRFc1/KAVQ3TUHiXmaAb6VW9vtiMFf7SLoMd1PdAc4W3KFOFETBJUb90KatHqlsZMWV+R9Gg=="],
+
+    "html-escaper": ["html-escaper@3.0.3", "", {}, "sha512-RuMffC89BOWQoY0WKGpIhn5gX3iI54O6nRA0yC124NYVtzjmFWBIiFd8M0x+ZdX0P9R4lADg1mgP8C7PxGOWuQ=="],
+
+    "htmlparser2": ["htmlparser2@10.1.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.3", "domutils": "^3.2.2", "entities": "^7.0.1" } }, "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ=="],
+
+    "human-id": ["human-id@4.1.3", "", { "bin": { "human-id": "dist/cli.js" } }, "sha512-tsYlhAYpjCKa//8rXZ9DqKEawhPoSytweBC2eNvcaDK+57RZLHGqNs3PZTQO6yekLFSuvA6AlnAfrw1uBvtb+Q=="],
+
+    "iconv-lite": ["iconv-lite@0.7.2", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw=="],
+
+    "ignore": ["ignore@5.3.2", "", {}, "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g=="],
+
+    "is-docker": ["is-docker@2.2.1", "", { "bin": { "is-docker": "cli.js" } }, "sha512-F+i2BKsFrH66iaUFc0woD8sLy8getkwTwtOBjvs56Cx4CgJDeKQeqfz8wAYiSb8JOprWhHH5p77PbmYCvvUuXQ=="],
+
+    "is-extglob": ["is-extglob@2.1.1", "", {}, "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ=="],
+
+    "is-glob": ["is-glob@4.0.3", "", { "dependencies": { "is-extglob": "^2.1.1" } }, "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg=="],
+
+    "is-number": ["is-number@7.0.0", "", {}, "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng=="],
+
+    "is-plain-obj": ["is-plain-obj@4.1.0", "", {}, "sha512-+Pgi+vMuUNkJyExiMBt5IlFoMyKnr5zhJ4Uspz58WOhBF5QoIZkFyNHIbBAtHwzVAgk5RtndVNsDRN61/mmDqg=="],
+
+    "is-potential-custom-element-name": ["is-potential-custom-element-name@1.0.1", "", {}, "sha512-bCYeRA2rVibKZd+s2625gGnGF/t7DSqDs4dP7CrLA1m7jKWz6pps0LpYLJN8Q64HtmPKJ1hrN3nzPNKFEKOUiQ=="],
+
+    "is-subdir": ["is-subdir@1.2.0", "", { "dependencies": { "better-path-resolve": "1.0.0" } }, "sha512-2AT6j+gXe/1ueqbW6fLZJiIw3F8iXGJtt0yDrZaBhAZEG1raiTxKWU+IPqMCzQAXOUCKdA4UDMgacKH25XG2Cw=="],
+
+    "is-windows": ["is-windows@1.0.2", "", {}, "sha512-eXK1UInq2bPmjyX6e3VHIzMLobc4J94i4AWn+Hpq3OU5KkrRC96OAcR3PRJ/pGu6m8TRnBHP9dkXQVsT/COVIA=="],
+
+    "is-wsl": ["is-wsl@2.2.0", "", { "dependencies": { "is-docker": "^2.0.0" } }, "sha512-fKzAra0rGJUUBwGBgNkHZuToZcn+TtXHpeCgmkMJMMYx1sQDYaCSyjJBSCa2nH1DGm7s3n1oBnohoVTBaN7Lww=="],
+
+    "isexe": ["isexe@2.0.0", "", {}, "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw=="],
+
+    "js-yaml": ["js-yaml@4.1.1", "", { "dependencies": { "argparse": "^2.0.1" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA=="],
+
+    "jsdom": ["jsdom@29.0.2", "", { "dependencies": { "@asamuzakjp/css-color": "^5.1.5", "@asamuzakjp/dom-selector": "^7.0.6", "@bramus/specificity": "^2.4.2", "@csstools/css-syntax-patches-for-csstree": "^1.1.1", "@exodus/bytes": "^1.15.0", "css-tree": "^3.2.1", "data-urls": "^7.0.0", "decimal.js": "^10.6.0", "html-encoding-sniffer": "^6.0.0", "is-potential-custom-element-name": "^1.0.1", "lru-cache": "^11.2.7", "parse5": "^8.0.0", "saxes": "^6.0.0", "symbol-tree": "^3.2.4", "tough-cookie": "^6.0.1", "undici": "^7.24.5", "w3c-xmlserializer": "^5.0.0", "webidl-conversions": "^8.0.1", "whatwg-mimetype": "^5.0.0", "whatwg-url": "^16.0.1", "xml-name-validator": "^5.0.0" }, "peerDependencies": { "canvas": "^3.0.0" }, "optionalPeers": ["canvas"] }, "sha512-9VnGEBosc/ZpwyOsJBCQ/3I5p7Q5ngOY14a9bf5btenAORmZfDse1ZEheMiWcJ3h81+Fv7HmJFdS0szo/waF2w=="],
+
+    "jsonfile": ["jsonfile@4.0.0", "", { "optionalDependencies": { "graceful-fs": "^4.1.6" } }, "sha512-m6F1R3z8jjlf2imQHS2Qez5sjKWQzbuuhuJ/FKYFRZvPE3PuHcSMVZzfsLhGVOkfd20obL5SWEBew5ShlquNxg=="],
+
+    "lighthouse-logger": ["lighthouse-logger@2.0.2", "", { "dependencies": { "debug": "^4.4.1", "marky": "^1.2.2" } }, "sha512-vWl2+u5jgOQuZR55Z1WM0XDdrJT6mzMP8zHUct7xTlWhuQs+eV0g+QL0RQdFjT54zVmbhLCP8vIVpy1wGn/gCg=="],
+
+    "linkedom": ["linkedom@0.18.12", "", { "dependencies": { "css-select": "^5.1.0", "cssom": "^0.5.0", "html-escaper": "^3.0.3", "htmlparser2": "^10.0.0", "uhyphen": "^0.2.0" }, "peerDependencies": { "canvas": ">= 2" }, "optionalPeers": ["canvas"] }, "sha512-jalJsOwIKuQJSeTvsgzPe9iJzyfVaEJiEXl+25EkKevsULHvMJzpNqwvj1jOESWdmgKDiXObyjOYwlUqG7wo1Q=="],
+
+    "locate-path": ["locate-path@5.0.0", "", { "dependencies": { "p-locate": "^4.1.0" } }, "sha512-t7hw9pI+WvuwNJXwk5zVHpyhIqzg2qTlklJOf0mVxGSbe3Fp2VieZcduNYjaLDoy6p9uGpQEGWG87WpMKlNq8g=="],
+
+    "lodash.startcase": ["lodash.startcase@4.4.0", "", {}, "sha512-+WKqsK294HMSc2jEbNgpHpd0JfIBhp7rEV4aqXWqFr6AlXov+SlcgB1Fv01y2kGe3Gc8nMW7VA0SrGuSkRfIEg=="],
+
+    "longest-streak": ["longest-streak@3.1.0", "", {}, "sha512-9Ri+o0JYgehTaVBBDoMqIl8GXtbWg711O3srftcHhZ0dqnETqLaoIK0x17fUw9rFSlK/0NlsKe0Ahhyl5pXE2g=="],
+
+    "lru-cache": ["lru-cache@11.3.5", "", {}, "sha512-NxVFwLAnrd9i7KUBxC4DrUhmgjzOs+1Qm50D3oF1/oL+r1NpZ4gA7xvG0/zJ8evR7zIKn4vLf7qTNduWFtCrRw=="],
+
+    "markdown-table": ["markdown-table@3.0.4", "", {}, "sha512-wiYz4+JrLyb/DqW2hkFJxP7Vd7JuTDm77fvbM8VfEQdmSMqcImWeeRbHwZjBjIFki/VaMK2BhFi7oUUZeM5bqw=="],
+
+    "marky": ["marky@1.3.0", "", {}, "sha512-ocnPZQLNpvbedwTy9kNrQEsknEfgvcLMvOtz3sFeWApDq1MXH1TqkCIx58xlpESsfwQOnuBO9beyQuNGzVvuhQ=="],
+
+    "mathml-to-latex": ["mathml-to-latex@1.5.0", "", { "dependencies": { "@xmldom/xmldom": "^0.8.10" } }, "sha512-rrWn0eEvcEcdMM4xfHcSGIy+i01DX9byOdXTLWg+w1iJ6O6ohP5UXY1dVzNUZLhzfl3EGcRekWLhY7JT5Omaew=="],
+
+    "mdast-util-find-and-replace": ["mdast-util-find-and-replace@3.0.2", "", { "dependencies": { "@types/mdast": "^4.0.0", "escape-string-regexp": "^5.0.0", "unist-util-is": "^6.0.0", "unist-util-visit-parents": "^6.0.0" } }, "sha512-Tmd1Vg/m3Xz43afeNxDIhWRtFZgM2VLyaf4vSTYwudTyeuTneoL3qtWMA5jeLyz/O1vDJmmV4QuScFCA2tBPwg=="],
+
+    "mdast-util-from-markdown": ["mdast-util-from-markdown@2.0.3", "", { "dependencies": { "@types/mdast": "^4.0.0", "@types/unist": "^3.0.0", "decode-named-character-reference": "^1.0.0", "devlop": "^1.0.0", "mdast-util-to-string": "^4.0.0", "micromark": "^4.0.0", "micromark-util-decode-numeric-character-reference": "^2.0.0", "micromark-util-decode-string": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0", "unist-util-stringify-position": "^4.0.0" } }, "sha512-W4mAWTvSlKvf8L6J+VN9yLSqQ9AOAAvHuoDAmPkz4dHf553m5gVj2ejadHJhoJmcmxEnOv6Pa8XJhpxE93kb8Q=="],
+
+    "mdast-util-gfm": ["mdast-util-gfm@3.1.0", "", { "dependencies": { "mdast-util-from-markdown": "^2.0.0", "mdast-util-gfm-autolink-literal": "^2.0.0", "mdast-util-gfm-footnote": "^2.0.0", "mdast-util-gfm-strikethrough": "^2.0.0", "mdast-util-gfm-table": "^2.0.0", "mdast-util-gfm-task-list-item": "^2.0.0", "mdast-util-to-markdown": "^2.0.0" } }, "sha512-0ulfdQOM3ysHhCJ1p06l0b0VKlhU0wuQs3thxZQagjcjPrlFRqY215uZGHHJan9GEAXd9MbfPjFJz+qMkVR6zQ=="],
+
+    "mdast-util-gfm-autolink-literal": ["mdast-util-gfm-autolink-literal@2.0.1", "", { "dependencies": { "@types/mdast": "^4.0.0", "ccount": "^2.0.0", "devlop": "^1.0.0", "mdast-util-find-and-replace": "^3.0.0", "micromark-util-character": "^2.0.0" } }, "sha512-5HVP2MKaP6L+G6YaxPNjuL0BPrq9orG3TsrZ9YXbA3vDw/ACI4MEsnoDpn6ZNm7GnZgtAcONJyPhOP8tNJQavQ=="],
+
+    "mdast-util-gfm-footnote": ["mdast-util-gfm-footnote@2.1.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "devlop": "^1.1.0", "mdast-util-from-markdown": "^2.0.0", "mdast-util-to-markdown": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0" } }, "sha512-sqpDWlsHn7Ac9GNZQMeUzPQSMzR6Wv0WKRNvQRg0KqHh02fpTz69Qc1QSseNX29bhz1ROIyNyxExfawVKTm1GQ=="],
+
+    "mdast-util-gfm-strikethrough": ["mdast-util-gfm-strikethrough@2.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "mdast-util-from-markdown": "^2.0.0", "mdast-util-to-markdown": "^2.0.0" } }, "sha512-mKKb915TF+OC5ptj5bJ7WFRPdYtuHv0yTRxK2tJvi+BDqbkiG7h7u/9SI89nRAYcmap2xHQL9D+QG/6wSrTtXg=="],
+
+    "mdast-util-gfm-table": ["mdast-util-gfm-table@2.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "devlop": "^1.0.0", "markdown-table": "^3.0.0", "mdast-util-from-markdown": "^2.0.0", "mdast-util-to-markdown": "^2.0.0" } }, "sha512-78UEvebzz/rJIxLvE7ZtDd/vIQ0RHv+3Mh5DR96p7cS7HsBhYIICDBCu8csTNWNO6tBWfqXPWekRuj2FNOGOZg=="],
+
+    "mdast-util-gfm-task-list-item": ["mdast-util-gfm-task-list-item@2.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "devlop": "^1.0.0", "mdast-util-from-markdown": "^2.0.0", "mdast-util-to-markdown": "^2.0.0" } }, "sha512-IrtvNvjxC1o06taBAVJznEnkiHxLFTzgonUdy8hzFVeDun0uTjxxrRGVaNFqkU1wJR3RBPEfsxmU6jDWPofrTQ=="],
+
+    "mdast-util-phrasing": ["mdast-util-phrasing@4.1.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "unist-util-is": "^6.0.0" } }, "sha512-TqICwyvJJpBwvGAMZjj4J2n0X8QWp21b9l0o7eXyVJ25YNWYbJDVIyD1bZXE6WtV6RmKJVYmQAKWa0zWOABz2w=="],
+
+    "mdast-util-to-markdown": ["mdast-util-to-markdown@2.1.2", "", { "dependencies": { "@types/mdast": "^4.0.0", "@types/unist": "^3.0.0", "longest-streak": "^3.0.0", "mdast-util-phrasing": "^4.0.0", "mdast-util-to-string": "^4.0.0", "micromark-util-classify-character": "^2.0.0", "micromark-util-decode-string": "^2.0.0", "unist-util-visit": "^5.0.0", "zwitch": "^2.0.0" } }, "sha512-xj68wMTvGXVOKonmog6LwyJKrYXZPvlwabaryTjLh9LuvovB/KAH+kvi8Gjj+7rJjsFi23nkUxRQv1KqSroMqA=="],
+
+    "mdast-util-to-string": ["mdast-util-to-string@4.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0" } }, "sha512-0H44vDimn51F0YwvxSJSm0eCDOJTRlmN0R1yBh4HLj9wiV1Dn0QoXGbvFAWj2hSItVTlCmBF1hqKlIyUBVFLPg=="],
+
+    "mdn-data": ["mdn-data@2.27.1", "", {}, "sha512-9Yubnt3e8A0OKwxYSXyhLymGW4sCufcLG6VdiDdUGVkPhpqLxlvP5vl1983gQjJl3tqbrM731mjaZaP68AgosQ=="],
+
+    "merge2": ["merge2@1.4.1", "", {}, "sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg=="],
+
+    "micromark": ["micromark@4.0.2", "", { "dependencies": { "@types/debug": "^4.0.0", "debug": "^4.0.0", "decode-named-character-reference": "^1.0.0", "devlop": "^1.0.0", "micromark-core-commonmark": "^2.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-combine-extensions": "^2.0.0", "micromark-util-decode-numeric-character-reference": "^2.0.0", "micromark-util-encode": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-resolve-all": "^2.0.0", "micromark-util-sanitize-uri": "^2.0.0", "micromark-util-subtokenize": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-zpe98Q6kvavpCr1NPVSCMebCKfD7CA2NqZ+rykeNhONIJBpc1tFKt9hucLGwha3jNTNI8lHpctWJWoimVF4PfA=="],
+
+    "micromark-core-commonmark": ["micromark-core-commonmark@2.0.3", "", { "dependencies": { "decode-named-character-reference": "^1.0.0", "devlop": "^1.0.0", "micromark-factory-destination": "^2.0.0", "micromark-factory-label": "^2.0.0", "micromark-factory-space": "^2.0.0", "micromark-factory-title": "^2.0.0", "micromark-factory-whitespace": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-classify-character": "^2.0.0", "micromark-util-html-tag-name": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-resolve-all": "^2.0.0", "micromark-util-subtokenize": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-RDBrHEMSxVFLg6xvnXmb1Ayr2WzLAWjeSATAoxwKYJV94TeNavgoIdA0a9ytzDSVzBy2YKFK+emCPOEibLeCrg=="],
+
+    "micromark-extension-gfm": ["micromark-extension-gfm@3.0.0", "", { "dependencies": { "micromark-extension-gfm-autolink-literal": "^2.0.0", "micromark-extension-gfm-footnote": "^2.0.0", "micromark-extension-gfm-strikethrough": "^2.0.0", "micromark-extension-gfm-table": "^2.0.0", "micromark-extension-gfm-tagfilter": "^2.0.0", "micromark-extension-gfm-task-list-item": "^2.0.0", "micromark-util-combine-extensions": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-vsKArQsicm7t0z2GugkCKtZehqUm31oeGBV/KVSorWSy8ZlNAv7ytjFhvaryUiCUJYqs+NoE6AFhpQvBTM6Q4w=="],
+
+    "micromark-extension-gfm-autolink-literal": ["micromark-extension-gfm-autolink-literal@2.1.0", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-sanitize-uri": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-oOg7knzhicgQ3t4QCjCWgTmfNhvQbDDnJeVu9v81r7NltNCVmhPy1fJRX27pISafdjL+SVc4d3l48Gb6pbRypw=="],
+
+    "micromark-extension-gfm-footnote": ["micromark-extension-gfm-footnote@2.1.0", "", { "dependencies": { "devlop": "^1.0.0", "micromark-core-commonmark": "^2.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-sanitize-uri": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-/yPhxI1ntnDNsiHtzLKYnE3vf9JZ6cAisqVDauhp4CEHxlb4uoOTxOCJ+9s51bIB8U1N1FJ1RXOKTIlD5B/gqw=="],
+
+    "micromark-extension-gfm-strikethrough": ["micromark-extension-gfm-strikethrough@2.1.0", "", { "dependencies": { "devlop": "^1.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-classify-character": "^2.0.0", "micromark-util-resolve-all": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-ADVjpOOkjz1hhkZLlBiYA9cR2Anf8F4HqZUO6e5eDcPQd0Txw5fxLzzxnEkSkfnD0wziSGiv7sYhk/ktvbf1uw=="],
+
+    "micromark-extension-gfm-table": ["micromark-extension-gfm-table@2.1.1", "", { "dependencies": { "devlop": "^1.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-t2OU/dXXioARrC6yWfJ4hqB7rct14e8f7m0cbI5hUmDyyIlwv5vEtooptH8INkbLzOatzKuVbQmAYcbWoyz6Dg=="],
+
+    "micromark-extension-gfm-tagfilter": ["micromark-extension-gfm-tagfilter@2.0.0", "", { "dependencies": { "micromark-util-types": "^2.0.0" } }, "sha512-xHlTOmuCSotIA8TW1mDIM6X2O1SiX5P9IuDtqGonFhEK0qgRI4yeC6vMxEV2dgyr2TiD+2PQ10o+cOhdVAcwfg=="],
+
+    "micromark-extension-gfm-task-list-item": ["micromark-extension-gfm-task-list-item@2.1.0", "", { "dependencies": { "devlop": "^1.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-qIBZhqxqI6fjLDYFTBIa4eivDMnP+OZqsNwmQ3xNLE4Cxwc+zfQEfbs6tzAo2Hjq+bh6q5F+Z8/cksrLFYWQQw=="],
+
+    "micromark-factory-destination": ["micromark-factory-destination@2.0.1", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-Xe6rDdJlkmbFRExpTOmRj9N3MaWmbAgdpSrBQvCFqhezUn4AHqJHbaEnfbVYYiexVSs//tqOdY/DxhjdCiJnIA=="],
+
+    "micromark-factory-label": ["micromark-factory-label@2.0.1", "", { "dependencies": { "devlop": "^1.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-VFMekyQExqIW7xIChcXn4ok29YE3rnuyveW3wZQWWqF4Nv9Wk5rgJ99KzPvHjkmPXF93FXIbBp6YdW3t71/7Vg=="],
+
+    "micromark-factory-space": ["micromark-factory-space@2.0.1", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-zRkxjtBxxLd2Sc0d+fbnEunsTj46SWXgXciZmHq0kDYGnck/ZSGj9/wULTV95uoeYiK5hRXP2mJ98Uo4cq/LQg=="],
+
+    "micromark-factory-title": ["micromark-factory-title@2.0.1", "", { "dependencies": { "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-5bZ+3CjhAd9eChYTHsjy6TGxpOFSKgKKJPJxr293jTbfry2KDoWkhBb6TcPVB4NmzaPhMs1Frm9AZH7OD4Cjzw=="],
+
+    "micromark-factory-whitespace": ["micromark-factory-whitespace@2.0.1", "", { "dependencies": { "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-Ob0nuZ3PKt/n0hORHyvoD9uZhr+Za8sFoP+OnMcnWK5lngSzALgQYKMr9RJVOWLqQYuyn6ulqGWSXdwf6F80lQ=="],
+
+    "micromark-util-character": ["micromark-util-character@2.1.1", "", { "dependencies": { "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-wv8tdUTJ3thSFFFJKtpYKOYiGP2+v96Hvk4Tu8KpCAsTMs6yi+nVmGh1syvSCsaxz45J6Jbw+9DD6g97+NV67Q=="],
+
+    "micromark-util-chunked": ["micromark-util-chunked@2.0.1", "", { "dependencies": { "micromark-util-symbol": "^2.0.0" } }, "sha512-QUNFEOPELfmvv+4xiNg2sRYeS/P84pTW0TCgP5zc9FpXetHY0ab7SxKyAQCNCc1eK0459uoLI1y5oO5Vc1dbhA=="],
+
+    "micromark-util-classify-character": ["micromark-util-classify-character@2.0.1", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-K0kHzM6afW/MbeWYWLjoHQv1sgg2Q9EccHEDzSkxiP/EaagNzCm7T/WMKZ3rjMbvIpvBiZgwR3dKMygtA4mG1Q=="],
+
+    "micromark-util-combine-extensions": ["micromark-util-combine-extensions@2.0.1", "", { "dependencies": { "micromark-util-chunked": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-OnAnH8Ujmy59JcyZw8JSbK9cGpdVY44NKgSM7E9Eh7DiLS2E9RNQf0dONaGDzEG9yjEl5hcqeIsj4hfRkLH/Bg=="],
+
+    "micromark-util-decode-numeric-character-reference": ["micromark-util-decode-numeric-character-reference@2.0.2", "", { "dependencies": { "micromark-util-symbol": "^2.0.0" } }, "sha512-ccUbYk6CwVdkmCQMyr64dXz42EfHGkPQlBj5p7YVGzq8I7CtjXZJrubAYezf7Rp+bjPseiROqe7G6foFd+lEuw=="],
+
+    "micromark-util-decode-string": ["micromark-util-decode-string@2.0.1", "", { "dependencies": { "decode-named-character-reference": "^1.0.0", "micromark-util-character": "^2.0.0", "micromark-util-decode-numeric-character-reference": "^2.0.0", "micromark-util-symbol": "^2.0.0" } }, "sha512-nDV/77Fj6eH1ynwscYTOsbK7rR//Uj0bZXBwJZRfaLEJ1iGBR6kIfNmlNqaqJf649EP0F3NWNdeJi03elllNUQ=="],
+
+    "micromark-util-encode": ["micromark-util-encode@2.0.1", "", {}, "sha512-c3cVx2y4KqUnwopcO9b/SCdo2O67LwJJ/UyqGfbigahfegL9myoEFoDYZgkT7f36T0bLrM9hZTAaAyH+PCAXjw=="],
+
+    "micromark-util-html-tag-name": ["micromark-util-html-tag-name@2.0.1", "", {}, "sha512-2cNEiYDhCWKI+Gs9T0Tiysk136SnR13hhO8yW6BGNyhOC4qYFnwF1nKfD3HFAIXA5c45RrIG1ub11GiXeYd1xA=="],
+
+    "micromark-util-normalize-identifier": ["micromark-util-normalize-identifier@2.0.1", "", { "dependencies": { "micromark-util-symbol": "^2.0.0" } }, "sha512-sxPqmo70LyARJs0w2UclACPUUEqltCkJ6PhKdMIDuJ3gSf/Q+/GIe3WKl0Ijb/GyH9lOpUkRAO2wp0GVkLvS9Q=="],
+
+    "micromark-util-resolve-all": ["micromark-util-resolve-all@2.0.1", "", { "dependencies": { "micromark-util-types": "^2.0.0" } }, "sha512-VdQyxFWFT2/FGJgwQnJYbe1jjQoNTS4RjglmSjTUlpUMa95Htx9NHeYW4rGDJzbjvCsl9eLjMQwGeElsqmzcHg=="],
+
+    "micromark-util-sanitize-uri": ["micromark-util-sanitize-uri@2.0.1", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-encode": "^2.0.0", "micromark-util-symbol": "^2.0.0" } }, "sha512-9N9IomZ/YuGGZZmQec1MbgxtlgougxTodVwDzzEouPKo3qFWvymFHWcnDi2vzV1ff6kas9ucW+o3yzJK9YB1AQ=="],
+
+    "micromark-util-subtokenize": ["micromark-util-subtokenize@2.1.0", "", { "dependencies": { "devlop": "^1.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-XQLu552iSctvnEcgXw6+Sx75GflAPNED1qx7eBJ+wydBb2KCbRZe+NwvIEEMM83uml1+2WSXpBAcp9IUCgCYWA=="],
+
+    "micromark-util-symbol": ["micromark-util-symbol@2.0.1", "", {}, "sha512-vs5t8Apaud9N28kgCrRUdEed4UJ+wWNvicHLPxCa9ENlYuAY31M0ETy5y1vA33YoNPDFTghEbnh6efaE8h4x0Q=="],
+
+    "micromark-util-types": ["micromark-util-types@2.0.2", "", {}, "sha512-Yw0ECSpJoViF1qTU4DC6NwtC4aWGt1EkzaQB8KPPyCRR8z9TWeV0HbEFGTO+ZY1wB22zmxnJqhPyTpOVCpeHTA=="],
+
+    "micromatch": ["micromatch@4.0.8", "", { "dependencies": { "braces": "^3.0.3", "picomatch": "^2.3.1" } }, "sha512-PXwfBhYu0hBCPw8Dn0E+WDYb7af3dSLVWKi3HGv84IdF4TyFoC0ysxFd0Goxw7nSv4T/PzEJQxsYsEiFCKo2BA=="],
+
+    "mri": ["mri@1.2.0", "", {}, "sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA=="],
+
+    "ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="],
+
+    "nth-check": ["nth-check@2.1.1", "", { "dependencies": { "boolbase": "^1.0.0" } }, "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w=="],
+
+    "outdent": ["outdent@0.5.0", "", {}, "sha512-/jHxFIzoMXdqPzTaCpFzAAWhpkSjZPF4Vsn6jAfNpmbH/ymsmd7Qc6VE9BGn0L6YMj6uwpQLxCECpus4ukKS9Q=="],
+
+    "p-filter": ["p-filter@2.1.0", "", { "dependencies": { "p-map": "^2.0.0" } }, "sha512-ZBxxZ5sL2HghephhpGAQdoskxplTwr7ICaehZwLIlfL6acuVgZPm8yBNuRAFBGEqtD/hmUeq9eqLg2ys9Xr/yw=="],
+
+    "p-limit": ["p-limit@2.3.0", "", { "dependencies": { "p-try": "^2.0.0" } }, "sha512-//88mFWSJx8lxCzwdAABTJL2MyWB12+eIY7MDL2SqLmAkeKU9qxRvWuSyTjm3FUmpBEMuFfckAIqEaVGUDxb6w=="],
+
+    "p-locate": ["p-locate@4.1.0", "", { "dependencies": { "p-limit": "^2.2.0" } }, "sha512-R79ZZ/0wAxKGu3oYMlz8jy/kbhsNrS7SKZ7PxEHBgJ5+F2mtFW2fK2cOtBh1cHYkQsbzFV7I+EoRKe6Yt0oK7A=="],
+
+    "p-map": ["p-map@2.1.0", "", {}, "sha512-y3b8Kpd8OAN444hxfBbFfj1FY/RjtTd8tzYwhUqNYXx0fXx2iX4maP4Qr6qhIKbQXI02wTLAda4fYUbDagTUFw=="],
+
+    "p-try": ["p-try@2.2.0", "", {}, "sha512-R4nPAVTAU0B9D35/Gk3uJf/7XYbQcyohSKdvAxIRSNghFl4e71hVoGnBNQz9cWaXxO2I10KTC+3jMdvvoKw6dQ=="],
+
+    "package-manager-detector": ["package-manager-detector@0.2.11", "", { "dependencies": { "quansync": "^0.2.7" } }, "sha512-BEnLolu+yuz22S56CU1SUKq3XC3PkwD5wv4ikR4MfGvnRVcmzXR9DwSlW2fEamyTPyXHomBJRzgapeuBvRNzJQ=="],
+
+    "parse5": ["parse5@7.3.0", "", { "dependencies": { "entities": "^6.0.0" } }, "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw=="],
+
+    "path-exists": ["path-exists@4.0.0", "", {}, "sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w=="],
+
+    "path-key": ["path-key@3.1.1", "", {}, "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q=="],
+
+    "path-type": ["path-type@4.0.0", "", {}, "sha512-gDKb8aZMDeD/tZWs9P6+q0J9Mwkdl6xMV8TjnGP3qJVJ06bdMgkbBlLU8IdfOsIsFz2BW1rNVT3XuNEl8zPAvw=="],
+
+    "picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="],
+
+    "picomatch": ["picomatch@2.3.2", "", {}, "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA=="],
+
+    "pify": ["pify@4.0.1", "", {}, "sha512-uB80kBFb/tfd68bVleG9T5GGsGPjJrLAUpR5PZIrhBnIaRTQRjqdJSsIKkOP6OAIFbj7GOrcudc5pNjZ+geV2g=="],
+
+    "prettier": ["prettier@2.8.8", "", { "bin": { "prettier": "bin-prettier.js" } }, "sha512-tdN8qQGvNjw4CHbY+XXk0JgCXn9QiF21a55rBe5LJAU+kDyC4WQn4+awm2Xfk2lQMk5fKup9XgzTZtGkjBdP9Q=="],
+
+    "punycode": ["punycode@2.3.1", "", {}, "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg=="],
+
+    "quansync": ["quansync@0.2.11", "", {}, "sha512-AifT7QEbW9Nri4tAwR5M/uzpBuqfZf+zwaEM/QkzEjj7NBuFD2rBuy0K3dE+8wltbezDV7JMA0WfnCPYRSYbXA=="],
+
+    "queue-microtask": ["queue-microtask@1.2.3", "", {}, "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A=="],
+
+    "read-yaml-file": ["read-yaml-file@1.1.0", "", { "dependencies": { "graceful-fs": "^4.1.5", "js-yaml": "^3.6.1", "pify": "^4.0.1", "strip-bom": "^3.0.0" } }, "sha512-VIMnQi/Z4HT2Fxuwg5KrY174U1VdUIASQVWXXyqtNRtxSr9IYkn1rsI6Tb6HsrHCmB7gVpNwX6JxPTHcH6IoTA=="],
+
+    "remark-gfm": ["remark-gfm@4.0.1", "", { "dependencies": { "@types/mdast": "^4.0.0", "mdast-util-gfm": "^3.0.0", "micromark-extension-gfm": "^3.0.0", "remark-parse": "^11.0.0", "remark-stringify": "^11.0.0", "unified": "^11.0.0" } }, "sha512-1quofZ2RQ9EWdeN34S79+KExV1764+wCUGop5CPL1WGdD0ocPpu91lzPGbwWMECpEpd42kJGQwzRfyov9j4yNg=="],
+
+    "remark-parse": ["remark-parse@11.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "mdast-util-from-markdown": "^2.0.0", "micromark-util-types": "^2.0.0", "unified": "^11.0.0" } }, "sha512-FCxlKLNGknS5ba/1lmpYijMUzX2esxW5xQqjWxw2eHFfS2MSdaHVINFmhjo+qN1WhZhNimq0dZATN9pH0IDrpA=="],
+
+    "remark-stringify": ["remark-stringify@11.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "mdast-util-to-markdown": "^2.0.0", "unified": "^11.0.0" } }, "sha512-1OSmLd3awB/t8qdoEOMazZkNsfVTeY4fTsgzcQFdXNq8ToTN4ZGwrMnlda4K6smTFKD+GRV6O48i6Z4iKgPPpw=="],
+
+    "require-from-string": ["require-from-string@2.0.2", "", {}, "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw=="],
+
+    "resolve-from": ["resolve-from@5.0.0", "", {}, "sha512-qYg9KP24dD5qka9J47d0aVky0N+b4fTU89LN9iDnjB5waksiC49rvMB0PrUJQGoTmH50XPiqOvAjDfaijGxYZw=="],
+
+    "reusify": ["reusify@1.1.0", "", {}, "sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw=="],
+
+    "run-parallel": ["run-parallel@1.2.0", "", { "dependencies": { "queue-microtask": "^1.2.2" } }, "sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA=="],
+
+    "safer-buffer": ["safer-buffer@2.1.2", "", {}, "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg=="],
+
+    "saxes": ["saxes@6.0.0", "", { "dependencies": { "xmlchars": "^2.2.0" } }, "sha512-xAg7SOnEhrm5zI3puOOKyy1OMcMlIJZYNJY7xLBwSze0UjhPLnWfj2GF2EpT0jmzaJKIWKHLsaSSajf35bcYnA=="],
+
+    "semver": ["semver@7.7.4", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-vFKC2IEtQnVhpT78h1Yp8wzwrf8CM+MzKMHGJZfBtzhZNycRFnXsHk6E5TxIkkMsgNS7mdX3AGB7x2QM2di4lA=="],
+
+    "shebang-command": ["shebang-command@2.0.0", "", { "dependencies": { "shebang-regex": "^3.0.0" } }, "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA=="],
+
+    "shebang-regex": ["shebang-regex@3.0.0", "", {}, "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A=="],
+
+    "signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
+
+    "slash": ["slash@3.0.0", "", {}, "sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q=="],
+
+    "source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="],
+
+    "spawndamnit": ["spawndamnit@3.0.1", "", { "dependencies": { "cross-spawn": "^7.0.5", "signal-exit": "^4.0.1" } }, "sha512-MmnduQUuHCoFckZoWnXsTg7JaiLBJrKFj9UI2MbRPGaJeVpsLcVBu6P/IGZovziM/YBsellCmsprgNA+w0CzVg=="],
+
+    "sprintf-js": ["sprintf-js@1.0.3", "", {}, "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g=="],
+
+    "strip-ansi": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="],
+
+    "strip-bom": ["strip-bom@3.0.0", "", {}, "sha512-vavAMRXOgBVNF6nyEEmL3DBK19iRpDcoIwW+swQ+CbGiu7lju6t+JklA1MHweoWtadgt4ISVUsXLyDq34ddcwA=="],
+
+    "symbol-tree": ["symbol-tree@3.2.4", "", {}, "sha512-9QNk5KwDF+Bvz+PyObkmSYjI5ksVUYtjW7AU22r2NKcfLJcXp96hkDWU3+XndOsUb+AQ9QhfzfCT2O+CNWT5Tw=="],
+
+    "temml": ["temml@0.13.2", "", {}, "sha512-n8fDRSsLscq9nh9j6z+FgkCvFMT0IJm6GCgwfzh+7AHT3Sfb4jFTQlsA6hVcF2dYYr3b66oDBVES95RfoukyrA=="],
+
+    "term-size": ["term-size@2.2.1", "", {}, "sha512-wK0Ri4fOGjv/XPy8SBHZChl8CM7uMc5VML7SqiQ0zG7+J5Vr+RMQDoHa2CNT6KHUnTGIXH34UDMkPzAUyapBZg=="],
+
+    "tldts": ["tldts@7.0.28", "", { "dependencies": { "tldts-core": "^7.0.28" }, "bin": { "tldts": "bin/cli.js" } }, "sha512-+Zg3vWhRUv8B1maGSTFdev9mjoo8Etn2Ayfs4cnjlD3CsGkxXX4QyW3j2WJ0wdjYcYmy7Lx2RDsZMhgCWafKIw=="],
+
+    "tldts-core": ["tldts-core@7.0.28", "", {}, "sha512-7W5Efjhsc3chVdFhqtaU0KtK32J37Zcr9RKtID54nG+tIpcY79CQK/veYPODxtD/LJ4Lue66jvrQzIX2Z2/pUQ=="],
+
+    "to-regex-range": ["to-regex-range@5.0.1", "", { "dependencies": { "is-number": "^7.0.0" } }, "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ=="],
+
+    "tough-cookie": ["tough-cookie@6.0.1", "", { "dependencies": { "tldts": "^7.0.5" } }, "sha512-LktZQb3IeoUWB9lqR5EWTHgW/VTITCXg4D21M+lvybRVdylLrRMnqaIONLVb5mav8vM19m44HIcGq4qASeu2Qw=="],
+
+    "tr46": ["tr46@6.0.0", "", { "dependencies": { "punycode": "^2.3.1" } }, "sha512-bLVMLPtstlZ4iMQHpFHTR7GAGj2jxi8Dg0s2h2MafAE4uSWF98FC/3MomU51iQAMf8/qDUbKWf5GxuvvVcXEhw=="],
+
+    "trough": ["trough@2.2.0", "", {}, "sha512-tmMpK00BjZiUyVyvrBK7knerNgmgvcV/KLVyuma/SC+TQN167GrMRciANTz09+k3zW8L8t60jWO1GpfkZdjTaw=="],
+
+    "turndown": ["turndown@7.2.2", "", { "dependencies": { "@mixmark-io/domino": "^2.2.0" } }, "sha512-1F7db8BiExOKxjSMU2b7if62D/XOyQyZbPKq/nUwopfgnHlqXHqQ0lvfUTeUIr1lZJzOPFn43dODyMSIfvWRKQ=="],
+
+    "turndown-plugin-gfm": ["turndown-plugin-gfm@1.0.2", "", {}, "sha512-vwz9tfvF7XN/jE0dGoBei3FXWuvll78ohzCZQuOb+ZjWrs3a0XhQVomJEb2Qh4VHTPNRO4GPZh0V7VRbiWwkRg=="],
+
+    "typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="],
+
+    "uhyphen": ["uhyphen@0.2.0", "", {}, "sha512-qz3o9CHXmJJPGBdqzab7qAYuW8kQGKNEuoHFYrBwV6hWIMcpAmxDLXojcHfFr9US1Pe6zUswEIJIbLI610fuqA=="],
+
+    "undici": ["undici@7.25.0", "", {}, "sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ=="],
+
+    "undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
+
+    "unified": ["unified@11.0.5", "", { "dependencies": { "@types/unist": "^3.0.0", "bail": "^2.0.0", "devlop": "^1.0.0", "extend": "^3.0.0", "is-plain-obj": "^4.0.0", "trough": "^2.0.0", "vfile": "^6.0.0" } }, "sha512-xKvGhPWw3k84Qjh8bI3ZeJjqnyadK+GEFtazSfZv/rKeTkTjOJho6mFqh2SM96iIcZokxiOpg78GazTSg8+KHA=="],
+
+    "unist-util-is": ["unist-util-is@6.0.1", "", { "dependencies": { "@types/unist": "^3.0.0" } }, "sha512-LsiILbtBETkDz8I9p1dQ0uyRUWuaQzd/cuEeS1hoRSyW5E5XGmTzlwY1OrNzzakGowI9Dr/I8HVaw4hTtnxy8g=="],
+
+    "unist-util-stringify-position": ["unist-util-stringify-position@4.0.0", "", { "dependencies": { "@types/unist": "^3.0.0" } }, "sha512-0ASV06AAoKCDkS2+xw5RXJywruurpbC4JZSm7nr7MOt1ojAzvyyaO+UxZf18j8FCF6kmzCZKcAgN/yu2gm2XgQ=="],
+
+    "unist-util-visit": ["unist-util-visit@5.1.0", "", { "dependencies": { "@types/unist": "^3.0.0", "unist-util-is": "^6.0.0", "unist-util-visit-parents": "^6.0.0" } }, "sha512-m+vIdyeCOpdr/QeQCu2EzxX/ohgS8KbnPDgFni4dQsfSCtpz8UqDyY5GjRru8PDKuYn7Fq19j1CQ+nJSsGKOzg=="],
+
+    "unist-util-visit-parents": ["unist-util-visit-parents@6.0.2", "", { "dependencies": { "@types/unist": "^3.0.0", "unist-util-is": "^6.0.0" } }, "sha512-goh1s1TBrqSqukSc8wrjwWhL0hiJxgA8m4kFxGlQ+8FYQ3C/m11FcTs4YYem7V664AhHVvgoQLk890Ssdsr2IQ=="],
+
+    "universalify": ["universalify@0.1.2", "", {}, "sha512-rBJeI5CXAlmy1pV+617WB9J63U6XcazHHF2f2dbJix4XzpUF0RS3Zbj0FGIOCAva5P/d/GBOYaACQ1w+0azUkg=="],
+
+    "vfile": ["vfile@6.0.3", "", { "dependencies": { "@types/unist": "^3.0.0", "vfile-message": "^4.0.0" } }, "sha512-KzIbH/9tXat2u30jf+smMwFCsno4wHVdNmzFyL+T/L3UGqqk6JKfVqOFOZEpZSHADH1k40ab6NUIXZq422ov3Q=="],
+
+    "vfile-message": ["vfile-message@4.0.3", "", { "dependencies": { "@types/unist": "^3.0.0", "unist-util-stringify-position": "^4.0.0" } }, "sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw=="],
+
+    "w3c-xmlserializer": ["w3c-xmlserializer@5.0.0", "", { "dependencies": { "xml-name-validator": "^5.0.0" } }, "sha512-o8qghlI8NZHU1lLPrpi2+Uq7abh4GGPpYANlalzWxyWteJOCsr/P+oPBA49TOLu5FTZO4d3F9MnWJfiMo4BkmA=="],
+
+    "webidl-conversions": ["webidl-conversions@8.0.1", "", {}, "sha512-BMhLD/Sw+GbJC21C/UgyaZX41nPt8bUTg+jWyDeg7e7YN4xOM05YPSIXceACnXVtqyEw/LMClUQMtMZ+PGGpqQ=="],
+
+    "whatwg-mimetype": ["whatwg-mimetype@5.0.0", "", {}, "sha512-sXcNcHOC51uPGF0P/D4NVtrkjSU2fNsm9iog4ZvZJsL3rjoDAzXZhkm2MWt1y+PUdggKAYVoMAIYcs78wJ51Cw=="],
+
+    "whatwg-url": ["whatwg-url@16.0.1", "", { "dependencies": { "@exodus/bytes": "^1.11.0", "tr46": "^6.0.0", "webidl-conversions": "^8.0.1" } }, "sha512-1to4zXBxmXHV3IiSSEInrreIlu02vUOvrhxJJH5vcxYTBDAx51cqZiKdyTxlecdKNSjj8EcxGBxNf6Vg+945gw=="],
+
+    "which": ["which@2.0.2", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="],
+
+    "ws": ["ws@8.20.0", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-sAt8BhgNbzCtgGbt2OxmpuryO63ZoDk/sqaB/znQm94T4fCEsy/yV+7CdC1kJhOU9lboAEU7R3kquuycDoibVA=="],
+
+    "xml-name-validator": ["xml-name-validator@5.0.0", "", {}, "sha512-EvGK8EJ3DhaHfbRlETOWAS5pO9MZITeauHKJyb8wyajUfQUenkIg2MvLDTZ4T/TgIcm3HU0TFBgWWboAZ30UHg=="],
+
+    "xmlchars": ["xmlchars@2.2.0", "", {}, "sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw=="],
+
+    "zwitch": ["zwitch@2.0.4", "", {}, "sha512-bXE4cR/kVZhKZX/RjPEflHaKVhUVl85noU3v6b8apfQEc1x4A+zBxjZ4lN8LqGd6WZ3dl98pY4o717VFmoPp+A=="],
+
+    "@manypkg/find-root/@types/node": ["@types/node@12.20.55", "", {}, "sha512-J8xLz7q2OFulZ2cyGTLE1TbbZcjpno7FaN6zdJNrgAdrJ+DZzh/uFR6YrTb4C+nXakvud8Q4+rbhoIWlYQbUFQ=="],
+
+    "@manypkg/find-root/fs-extra": ["fs-extra@8.1.0", "", { "dependencies": { "graceful-fs": "^4.2.0", "jsonfile": "^4.0.0", "universalify": "^0.1.0" } }, "sha512-yhlQgA6mnOJUKOsRUFsgJdQCvkKhcz8tlZG5HBQfReYZy46OwLcY+Zia0mtdHsOo9y/hP+CxMN0TU9QxoOtG4g=="],
+
+    "@manypkg/get-packages/@changesets/types": ["@changesets/types@4.1.0", "", {}, "sha512-LDQvVDv5Kb50ny2s25Fhm3d9QSZimsoUGBsUioj6MC3qbMUCuC8GPIvk/M6IvXx3lYhAs0lwWUQLb+VIEUCECw=="],
+
+    "@manypkg/get-packages/fs-extra": ["fs-extra@8.1.0", "", { "dependencies": { "graceful-fs": "^4.2.0", "jsonfile": "^4.0.0", "universalify": "^0.1.0" } }, "sha512-yhlQgA6mnOJUKOsRUFsgJdQCvkKhcz8tlZG5HBQfReYZy46OwLcY+Zia0mtdHsOo9y/hP+CxMN0TU9QxoOtG4g=="],
+
+    "dom-serializer/entities": ["entities@4.5.0", "", {}, "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw=="],
+
+    "htmlparser2/entities": ["entities@7.0.1", "", {}, "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA=="],
+
+    "jsdom/parse5": ["parse5@8.0.1", "", { "dependencies": { "entities": "^8.0.0" } }, "sha512-z1e/HMG90obSGeidlli3hj7cbocou0/wa5HacvI3ASx34PecNjNQeaHNo5WIZpWofN9kgkqV1q5YvXe3F0FoPw=="],
+
+    "mdast-util-find-and-replace/escape-string-regexp": ["escape-string-regexp@5.0.0", "", {}, "sha512-/veY75JbMK4j1yjvuUxuVsiS/hr/4iHs9FTT6cgTexxdE0Ly/glccBAkloH/DofkjRbZU3bnoj38mOmhkZ0lHw=="],
+
+    "read-yaml-file/js-yaml": ["js-yaml@3.14.2", "", { "dependencies": { "argparse": "^1.0.7", "esprima": "^4.0.0" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-PMSmkqxr106Xa156c2M265Z+FTrPl+oxd/rgOQy2tijQeK5TxQ43psO1ZCwhVOSdnn+RzkzlRz/eY4BgJBYVpg=="],
+
+    "jsdom/parse5/entities": ["entities@8.0.0", "", {}, "sha512-zwfzJecQ/Uej6tusMqwAqU/6KL2XaB2VZ2Jg54Je6ahNBGNH6Ek6g3jjNCF0fG9EWQKGZNddNjU5F1ZQn/sBnA=="],
+
+    "read-yaml-file/js-yaml/argparse": ["argparse@1.0.10", "", { "dependencies": { "sprintf-js": "~1.0.2" } }, "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg=="],
+  }
+}

packages/baoyu-fetch/package.json

@@ -0,0 +1,68 @@
+{
+  "name": "baoyu-fetch",
+  "version": "0.1.2",
+  "description": "Read URLs into high-quality Markdown or JSON with Chrome CDP and site adapters.",
+  "type": "module",
+  "bin": {
+    "baoyu-fetch": "dist/cli.js"
+  },
+  "files": [
+    "dist",
+    "README.zh-CN.md",
+    "src/adapters",
+    "src/browser",
+    "src/cli.ts",
+    "src/commands",
+    "src/extract",
+    "src/media",
+    "src/types",
+    "src/utils",
+    "README.md"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/JimLiu/baoyu-skills.git",
+    "directory": "packages/baoyu-fetch"
+  },
+  "bugs": {
+    "url": "https://github.com/JimLiu/baoyu-skills/issues"
+  },
+  "homepage": "https://github.com/JimLiu/baoyu-skills/tree/main/packages/baoyu-fetch#readme",
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "build": "rm -rf dist && bun build ./src/cli.ts --target bun --outfile ./dist/cli.js && chmod +x ./dist/cli.js",
+    "check": "tsc --noEmit",
+    "dev": "bun run ./src/cli.ts",
+    "release": "changeset publish",
+    "test": "bun test",
+    "version-packages": "changeset version",
+    "prepack": "bun run build"
+  },
+  "engines": {
+    "bun": ">=1.2.0"
+  },
+  "dependencies": {
+    "@mozilla/readability": "^0.6.0",
+    "chrome-launcher": "^1.2.1",
+    "defuddle": "^0.17.0",
+    "jsdom": "^29.0.2",
+    "remark-gfm": "^4.0.1",
+    "remark-parse": "^11.0.0",
+    "turndown": "^7.2.0",
+    "turndown-plugin-gfm": "^1.0.2",
+    "unified": "^11.0.5",
+    "ws": "^8.18.3"
+  },
+  "devDependencies": {
+    "@changesets/cli": "^2.30.0",
+    "@types/bun": "^1.2.23",
+    "@types/jsdom": "^21.1.7",
+    "@types/ws": "^8.18.1",
+    "typescript": "^5.9.2"
+  },
+  "overrides": {
+    "@xmldom/xmldom": "0.8.13"
+  }
+}

packages/baoyu-fetch/src/__tests__/adapters.test.ts

@@ -0,0 +1,25 @@
+import { describe, expect, test } from "bun:test";
+import { resolveAdapter } from "../adapters";
+
+describe("adapter registry", () => {
+  test("matches x adapter for x.com status URLs", () => {
+    const adapter = resolveAdapter({
+      url: new URL("https://x.com/openai/status/1234567890"),
+    });
+    expect(adapter.name).toBe("x");
+  });
+
+  test("matches hn adapter for item URLs", () => {
+    const adapter = resolveAdapter({
+      url: new URL("https://news.ycombinator.com/item?id=47534848"),
+    });
+    expect(adapter.name).toBe("hn");
+  });
+
+  test("falls back to generic adapter", () => {
+    const adapter = resolveAdapter({
+      url: new URL("https://example.com/post"),
+    });
+    expect(adapter.name).toBe("generic");
+  });
+});

packages/baoyu-fetch/src/__tests__/cli.test.ts

@@ -0,0 +1,69 @@
+import { describe, expect, test } from "bun:test";
+import { HELP_TEXT, parseArgs } from "../cli";
+
+describe("parseArgs", () => {
+  test("defaults to markdown output", () => {
+    const options = parseArgs(["bun", "src/cli.ts", "https://example.com"]);
+    expect(options.format).toBe("markdown");
+  });
+
+  test("parses explicit json output format", () => {
+    const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--format", "json"]);
+    expect(options.format).toBe("json");
+  });
+
+  test("maps --json to json output format", () => {
+    const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--json"]);
+    expect(options.format).toBe("json");
+  });
+
+  test("parses --wait-for interaction", () => {
+    const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--wait-for", "interaction"]);
+    expect(options.waitMode).toBe("interaction");
+  });
+
+  test("parses --wait-for force", () => {
+    const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--wait-for", "force"]);
+    expect(options.waitMode).toBe("force");
+  });
+
+  test("maps legacy wait flags to interaction mode", () => {
+    const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--wait-for-interaction"]);
+    expect(options.waitMode).toBe("interaction");
+  });
+
+  test("parses media download options", () => {
+    const options = parseArgs([
+      "bun",
+      "src/cli.ts",
+      "https://example.com",
+      "--download-media",
+      "--media-dir",
+      "./assets",
+    ]);
+
+    expect(options.downloadMedia).toBe(true);
+    expect(options.mediaDir).toBe("./assets");
+  });
+
+  test("rejects invalid wait modes", () => {
+    expect(() =>
+      parseArgs(["bun", "src/cli.ts", "https://example.com", "--wait-for", "unknown"]),
+    ).toThrow("Invalid wait mode");
+  });
+
+  test("rejects invalid output formats", () => {
+    expect(() =>
+      parseArgs(["bun", "src/cli.ts", "https://example.com", "--format", "xml"]),
+    ).toThrow("Invalid output format");
+  });
+
+  test("documents wait modes in help text", () => {
+    expect(HELP_TEXT).toContain("baoyu-fetch");
+    expect(HELP_TEXT).toContain("--format <type>");
+    expect(HELP_TEXT).toContain("--wait-for <mode>");
+    expect(HELP_TEXT).toContain("--download-media");
+    expect(HELP_TEXT).toContain("force: start visible Chrome, then auto-continue");
+    expect(HELP_TEXT).toContain("or continue immediately when you press Enter");
+  });
+});

packages/baoyu-fetch/src/__tests__/convert-output.test.ts

@@ -0,0 +1,55 @@
+import { describe, expect, test } from "bun:test";
+import { formatOutputContent } from "../commands/convert";
+
+describe("formatOutputContent", () => {
+  test("returns raw markdown for markdown output", () => {
+    expect(
+      formatOutputContent("markdown", {
+        adapter: "generic",
+        status: "ok",
+        media: [],
+        downloads: null,
+        document: {
+          url: "https://example.com",
+          content: [],
+        },
+        markdown: "# Example",
+      }),
+    ).toBe("# Example");
+  });
+
+  test("returns structured json for json output", () => {
+    const parsed = JSON.parse(
+      formatOutputContent("json", {
+        adapter: "generic",
+        status: "ok",
+        media: [],
+        downloads: null,
+        document: {
+          url: "https://example.com",
+          content: [],
+        },
+        markdown: "# Example",
+      }),
+    );
+
+    expect(parsed.status).toBe("ok");
+    expect(parsed.markdown).toBe("# Example");
+    expect(parsed.document.url).toBe("https://example.com");
+  });
+
+  test("rejects markdown output for interaction-required payloads", () => {
+    expect(() =>
+      formatOutputContent("markdown", {
+        adapter: "x",
+        status: "needs_interaction",
+        interaction: {
+          type: "wait_for_interaction",
+          kind: "login",
+          provider: "x",
+          prompt: "Login required",
+        },
+      }),
+    ).toThrow("Markdown output is only available");
+  });
+});

packages/baoyu-fetch/src/__tests__/hn-adapter.test.ts

@@ -0,0 +1,167 @@
+import { describe, expect, test } from "bun:test";
+import { renderMarkdown } from "../extract/markdown-renderer";
+import {
+  buildHnDocument,
+  buildHnThreadMarkdown,
+  extractHnThreadFromHtml,
+  parseHnItemId,
+  type HnCommentNode,
+  type HnItem,
+} from "../adapters/hn";
+
+describe("hn adapter helpers", () => {
+  test("parses item id from hn item url", () => {
+    expect(parseHnItemId(new URL("https://news.ycombinator.com/item?id=47534848"))).toBe(47534848);
+    expect(parseHnItemId(new URL("https://news.ycombinator.com/newest"))).toBeNull();
+  });
+
+  test("renders threaded comments with author, time, and nested indentation", () => {
+    const story: HnItem = {
+      id: 47534848,
+      type: "story",
+      by: "mmcclure",
+      time: 1774554485,
+      title: "Example & Title",
+      url: "https://example.com/post",
+      score: 257,
+      descendants: 2,
+    };
+
+    const comments: HnCommentNode[] = [
+      {
+        item: {
+          id: 47535377,
+          type: "comment",
+          by: "jackfruitpeel",
+          time: 1774557334,
+          text: "Root comment<p>With two paragraphs.",
+        },
+        children: [
+          {
+            item: {
+              id: 47535469,
+              type: "comment",
+              by: "__MatrixMan__",
+              time: 1774557848,
+              text: "Nested reply with a <a href=\"item?id=1\">relative link</a>.",
+            },
+            children: [],
+          },
+        ],
+      },
+    ];
+
+    const body = buildHnThreadMarkdown(story, comments, "https://news.ycombinator.com/item?id=47534848");
+    expect(body).toContain("Source: [https://example.com/post](https://example.com/post)");
+    expect(body).toContain("Submitted by mmcclure at 2026-03-26 19:48:05 UTC");
+    expect(body).toContain("- jackfruitpeel · [2026-03-26 20:35:34 UTC](https://news.ycombinator.com/item?id=47534848#47535377)");
+    expect(body).toContain("    Root comment");
+    expect(body).toContain("    With two paragraphs.");
+    expect(body).toContain("    - __MatrixMan__ · [2026-03-26 20:44:08 UTC](https://news.ycombinator.com/item?id=47534848#47535469)");
+    expect(body).toContain("        Nested reply with a [relative link](https://news.ycombinator.com/item?id=1).");
+  });
+
+  test("extracts story metadata and nested comments from hn html", () => {
+    const parsed = extractHnThreadFromHtml(
+      `
+      <html>
+        <body>
+          <table class="fatitem">
+            <tr class="athing submission" id="47534848">
+              <td class="title">
+                <span class="titleline">
+                  <a href="https://example.com/post">Example story</a>
+                </span>
+              </td>
+            </tr>
+            <tr>
+              <td class="subtext">
+                <span class="subline">
+                  <span class="score">257 points</span>
+                  by <a href="user?id=mmcclure" class="hnuser">mmcclure</a>
+                  <span class="age" title="2026-03-26T19:48:05 1774554485">
+                    <a href="item?id=47534848">1 hour ago</a>
+                  </span>
+                  <a href="item?id=47534848">152 comments</a>
+                </span>
+              </td>
+            </tr>
+            <tr>
+              <td><div class="toptext">Story <p>body</p></div></td>
+            </tr>
+          </table>
+          <table class="comment-tree">
+            <tr class="athing comtr" id="47535377">
+              <td class="ind" indent="0"></td>
+              <td class="default">
+                <span class="comhead">
+                  <a href="user?id=jackfruitpeel" class="hnuser">jackfruitpeel</a>
+                  <span class="age" title="2026-03-26T20:35:34 1774557334">
+                    <a href="item?id=47535377">36 minutes ago</a>
+                  </span>
+                </span>
+                <div class="comment"><div class="commtext c00">Root</div></div>
+              </td>
+            </tr>
+            <tr class="athing comtr" id="47535469">
+              <td class="ind" indent="1"></td>
+              <td class="default">
+                <span class="comhead">
+                  <a href="user?id=willio58" class="hnuser">willio58</a>
+                  <span class="age" title="2026-03-26T20:44:08 1774557848">
+                    <a href="item?id=47535469">27 minutes ago</a>
+                  </span>
+                </span>
+                <div class="comment"><div class="commtext c00">Child</div></div>
+              </td>
+            </tr>
+          </table>
+        </body>
+      </html>
+      `,
+      "https://news.ycombinator.com/item?id=47534848",
+    );
+
+    expect(parsed).not.toBeNull();
+    expect(parsed?.story.title).toBe("Example story");
+    expect(parsed?.story.url).toBe("https://example.com/post");
+    expect(parsed?.story.by).toBe("mmcclure");
+    expect(parsed?.story.time).toBe(1774554485);
+    expect(parsed?.story.score).toBe(257);
+    expect(parsed?.story.descendants).toBe(152);
+    expect(parsed?.story.text).toContain("Story");
+    expect(parsed?.comments).toHaveLength(1);
+    expect(parsed?.comments[0]?.item.by).toBe("jackfruitpeel");
+    expect(parsed?.comments[0]?.children).toHaveLength(1);
+    expect(parsed?.comments[0]?.children[0]?.item.by).toBe("willio58");
+  });
+
+  test("builds hn document with metadata and markdown body", () => {
+    const document = buildHnDocument(
+      {
+        id: 123,
+        type: "story",
+        by: "pg",
+        time: 1175714200,
+        title: "Ask HN: Example",
+        text: "What are you working on?",
+        score: 111,
+        descendants: 0,
+      },
+      [],
+      "https://news.ycombinator.com/item?id=123",
+    );
+
+    const markdown = renderMarkdown(document);
+    expect(document.adapter).toBe("hn");
+    expect(document.siteName).toBe("Hacker News");
+    expect(document.publishedAt).toBe("2007-04-04T19:16:40.000Z");
+    expect(markdown).toContain('adapter: "hn"');
+    expect(markdown).toContain('siteName: "Hacker News"');
+    expect(markdown).toContain("# Ask HN: Example");
+    expect(markdown).toContain("## Post");
+    expect(markdown).toContain("What are you working on?");
+    expect(markdown).toContain("## Comments");
+    expect(markdown).toContain("No comments.");
+  });
+});

packages/baoyu-fetch/src/__tests__/html-to-markdown.test.ts

@@ -0,0 +1,105 @@
+import { afterEach, describe, expect, test } from "bun:test";
+import {
+  convertHtmlToMarkdown,
+  extractTitleFromMarkdownDocument,
+} from "../extract/html-to-markdown";
+
+const originalFetch = globalThis.fetch;
+
+afterEach(() => {
+  globalThis.fetch = originalFetch;
+});
+
+describe("extractTitleFromMarkdownDocument", () => {
+  test("prefers frontmatter title when present", () => {
+    const title = extractTitleFromMarkdownDocument(`---
+title: "Frontmatter Title"
+---
+
+# Heading Title
+`);
+
+    expect(title).toBe("Frontmatter Title");
+  });
+
+  test("falls back to the first markdown heading", () => {
+    const title = extractTitleFromMarkdownDocument(`
+Intro text
+
+# Heading Title
+
+Body text.
+`);
+
+    expect(title).toBe("Heading Title");
+  });
+});
+
+describe("convertHtmlToMarkdown remote fallback", () => {
+  test("does not call defuddle.md when the remote fallback option is disabled", async () => {
+    let fetchCalls = 0;
+    globalThis.fetch = Object.assign(
+      async () => {
+        fetchCalls += 1;
+        return new Response("# Remote Title\n\nRemote body.", {
+          headers: {
+            "content-type": "text/markdown",
+          },
+        });
+      },
+      {
+        preconnect: originalFetch.preconnect,
+      },
+    ) as typeof fetch;
+
+    const result = await convertHtmlToMarkdown(
+      "<!doctype html><html><head><title>Local Title</title></head><body></body></html>",
+      "https://example.com/post",
+    );
+
+    expect(fetchCalls).toBe(0);
+    expect(result.conversionMethod).not.toBe("defuddle-api");
+  });
+
+  test("uses defuddle.md markdown when local extraction is empty", async () => {
+    const fetchCalls: Array<{ input: RequestInfo | URL; init?: RequestInit }> = [];
+    globalThis.fetch = Object.assign(
+      async (input: RequestInfo | URL, init?: RequestInit) => {
+        fetchCalls.push({ input, init });
+        return new Response(`---
+title: "Remote Title"
+---
+
+# Remote Title
+
+Remote body.
+`, {
+          headers: {
+            "content-type": "text/markdown",
+          },
+        });
+      },
+      {
+        preconnect: originalFetch.preconnect,
+      },
+    ) as typeof fetch;
+
+    const result = await convertHtmlToMarkdown(
+      "<!doctype html><html><head><title>Local Title</title></head><body></body></html>",
+      "https://example.com/post",
+      { enableRemoteMarkdownFallback: true },
+    );
+
+    expect(fetchCalls).toHaveLength(1);
+    expect(String(fetchCalls[0]?.input)).toBe(
+      "https://defuddle.md/https%3A%2F%2Fexample.com%2Fpost",
+    );
+    expect(fetchCalls[0]?.init?.headers).toEqual({
+      accept: "text/markdown,text/plain;q=0.9,*/*;q=0.1",
+    });
+    expect(result.conversionMethod).toBe("defuddle-api");
+    expect(result.metadata.title).toBe("Remote Title");
+    expect(result.markdown).toBe("# Remote Title\n\nRemote body.");
+    expect(result.fallbackReason).toContain("defuddle.md");
+  });
+});

packages/baoyu-fetch/src/__tests__/interaction-gates.test.ts

@@ -0,0 +1,54 @@
+import { describe, expect, test } from "bun:test";
+import { detectInteractionGateFromSnapshot } from "../browser/interaction-gates";
+
+describe("detectInteractionGateFromSnapshot", () => {
+  test("detects cloudflare challenge", () => {
+    const gate = detectInteractionGateFromSnapshot({
+      title: "Just a moment...",
+      currentUrl: "https://example.com/cdn-cgi/challenge-platform/h/b",
+      bodyText: "Checking your browser before accessing example.com",
+      hasCloudflareTurnstile: true,
+      hasCloudflareChallenge: true,
+      hasRecaptcha: false,
+      hasRecaptchaIframe: false,
+      hasHcaptcha: false,
+      hasHcaptchaIframe: false,
+    });
+
+    expect(gate?.kind).toBe("cloudflare");
+    expect(gate?.provider).toBe("cloudflare");
+  });
+
+  test("detects google recaptcha", () => {
+    const gate = detectInteractionGateFromSnapshot({
+      title: "Protected page",
+      currentUrl: "https://example.com/form",
+      bodyText: "Please verify that you're not a robot via reCAPTCHA",
+      hasCloudflareTurnstile: false,
+      hasCloudflareChallenge: false,
+      hasRecaptcha: true,
+      hasRecaptchaIframe: true,
+      hasHcaptcha: false,
+      hasHcaptchaIframe: false,
+    });
+
+    expect(gate?.kind).toBe("recaptcha");
+    expect(gate?.provider).toBe("google_recaptcha");
+  });
+
+  test("returns null when no challenge is present", () => {
+    const gate = detectInteractionGateFromSnapshot({
+      title: "Example",
+      currentUrl: "https://example.com/article",
+      bodyText: "Normal article body",
+      hasCloudflareTurnstile: false,
+      hasCloudflareChallenge: false,
+      hasRecaptcha: false,
+      hasRecaptchaIframe: false,
+      hasHcaptcha: false,
+      hasHcaptchaIframe: false,
+    });
+
+    expect(gate).toBeNull();
+  });
+});

packages/baoyu-fetch/src/__tests__/markdown-media.test.ts

@@ -0,0 +1,138 @@
+import { describe, expect, test } from "bun:test";
+import {
+  collectMediaFromDocument,
+  collectMediaFromMarkdown,
+  normalizeMarkdownMediaLinks,
+  rewriteMarkdownMediaLinks,
+} from "../media/markdown-media";
+
+describe("markdown media helpers", () => {
+  test("collects cover, image markdown, and plain media urls from a document", () => {
+    const media = collectMediaFromDocument({
+      url: "https://example.com/post",
+      metadata: {
+        coverImage: "https://cdn.example.com/cover.jpg",
+      },
+      content: [
+        { type: "paragraph", text: "Poster: https://cdn.example.com/poster.png" },
+        { type: "markdown", markdown: "![inline](https://cdn.example.com/body.webp)\n\n[video](https://cdn.example.com/clip.mp4)" },
+      ],
+    });
+
+    expect(media).toEqual([
+      { url: "https://cdn.example.com/cover.jpg", kind: "image", role: "cover" },
+      { url: "https://cdn.example.com/poster.png", kind: "image", role: "inline" },
+      { url: "https://cdn.example.com/body.webp", kind: "image", role: "inline" },
+      { url: "https://cdn.example.com/clip.mp4", kind: "video", role: "inline" },
+    ]);
+  });
+
+  test("rewrites markdown links, frontmatter cover images, and plain url mentions", () => {
+    const markdown = `---
+coverImage: "https://cdn.example.com/cover.jpg"
+---
+
+![inline](https://cdn.example.com/body.webp)
+
+Poster: https://cdn.example.com/poster.png
+`;
+
+    const rewritten = rewriteMarkdownMediaLinks(markdown, [
+      {
+        url: "https://cdn.example.com/cover.jpg",
+        localPath: "imgs/img-001-cover.jpg",
+        absolutePath: "/tmp/imgs/img-001-cover.jpg",
+        kind: "image",
+      },
+      {
+        url: "https://cdn.example.com/body.webp",
+        localPath: "imgs/img-002-body.webp",
+        absolutePath: "/tmp/imgs/img-002-body.webp",
+        kind: "image",
+      },
+      {
+        url: "https://cdn.example.com/poster.png",
+        localPath: "imgs/img-003-poster.png",
+        absolutePath: "/tmp/imgs/img-003-poster.png",
+        kind: "image",
+      },
+    ]);
+
+    expect(rewritten).toContain('coverImage: "imgs/img-001-cover.jpg"');
+    expect(rewritten).toContain("![inline](imgs/img-002-body.webp)");
+    expect(rewritten).toContain("Poster: imgs/img-003-poster.png");
+  });
+
+  test("normalizes and dedupes linked Substack CDN image variants", () => {
+    const resizedUrl =
+      "https://substackcdn.com/image/fetch/$s_!wORh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+    const linkedUrl =
+      "https://substackcdn.com/image/fetch/$s_!wORh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+    const canonicalUrl =
+      "https://substack-post-media.s3.amazonaws.com/public/images/b83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+    const markdown = `[![](${resizedUrl})](${linkedUrl})`;
+
+    expect(normalizeMarkdownMediaLinks(markdown)).toBe(`![](${canonicalUrl})`);
+    expect(collectMediaFromMarkdown(markdown)).toEqual([
+      {
+        url: canonicalUrl,
+        kind: "image",
+        role: "inline",
+      },
+    ]);
+  });
+
+  test("collapses linked images when href equals image url after normalization", () => {
+    const resizedUrl =
+      "https://substackcdn.com/image/fetch/$s_!wORh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+    const linkedUrl =
+      "https://substackcdn.com/image/fetch/$s_!wORh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+    const canonicalUrl =
+      "https://substack-post-media.s3.amazonaws.com/public/images/b83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+    const markdown = `[
+
+![](${resizedUrl})
+
+](${linkedUrl})`;
+
+    expect(normalizeMarkdownMediaLinks(markdown)).toBe(`![](${canonicalUrl})`);
+  });
+
+  test("compacts linked images when href differs from the image url", () => {
+    const markdown = `[
+
+![diagram](https://cdn.example.com/body.webp)
+
+](https://example.com/source)`;
+
+    expect(normalizeMarkdownMediaLinks(markdown)).toBe(
+      "[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)",
+    );
+  });
+
+  test("keeps single-line linked images on one line after parser-based normalization", () => {
+    const markdown = `[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)`;
+
+    expect(normalizeMarkdownMediaLinks(markdown)).toBe(
+      "[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)",
+    );
+  });
+
+  test("repairs broken linked image blocks without disturbing surrounding paragraphs", () => {
+    const markdown = `Before
+
+[
+
+![diagram](https://cdn.example.com/body.webp)
+
+](https://example.com/source)
+
+After`;
+
+    expect(normalizeMarkdownMediaLinks(markdown)).toBe(`Before
+
+[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)
+
+After`);
+  });
+});

packages/baoyu-fetch/src/__tests__/markdown-renderer.test.ts

@@ -0,0 +1,99 @@
+import { describe, expect, test } from "bun:test";
+import { renderMarkdown } from "../extract/markdown-renderer";
+
+describe("renderMarkdown", () => {
+  test("renders frontmatter and content blocks", () => {
+    const markdown = renderMarkdown({
+      url: "https://example.com/post",
+      requestedUrl: "https://example.com/post?ref=test",
+      title: "Example Title",
+      author: "Alice",
+      siteName: "Example",
+      publishedAt: "2026-03-25",
+      adapter: "generic",
+      metadata: {
+        authorName: "Alice Example",
+        authorUsername: "alice",
+        authorUrl: "https://example.com/@alice",
+        kind: "generic/article",
+      },
+      content: [
+        { type: "paragraph", text: "First paragraph." },
+        { type: "list", ordered: false, items: ["One", "Two"] },
+      ],
+    });
+
+    expect(markdown).toContain("---");
+    expect(markdown).toContain('title: "Example Title"');
+    expect(markdown).toContain('url: "https://example.com/post"');
+    expect(markdown).toContain('requestedUrl: "https://example.com/post?ref=test"');
+    expect(markdown).toContain('author: "Alice"');
+    expect(markdown).toContain('authorName: "Alice Example"');
+    expect(markdown).toContain('authorUsername: "alice"');
+    expect(markdown).toContain('authorUrl: "https://example.com/@alice"');
+    expect(markdown).toContain("# Example Title");
+    expect(markdown).toContain("First paragraph.");
+    expect(markdown).toContain("- One");
+  });
+
+  test("avoids duplicating the title when body already starts with it", () => {
+    const markdown = renderMarkdown({
+      url: "https://example.com/post",
+      title: "Example Title",
+      content: [{ type: "markdown", markdown: "# Example Title\n\nBody text." }],
+    });
+
+    expect(markdown.match(/# Example Title/g)?.length).toBe(1);
+    expect(markdown).toContain("Body text.");
+  });
+
+  test("normalizes Substack CDN image links in rendered markdown", () => {
+    const resizedUrl =
+      "https://substackcdn.com/image/fetch/$s_!wORh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+    const linkedUrl =
+      "https://substackcdn.com/image/fetch/$s_!wORh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+    const canonicalUrl =
+      "https://substack-post-media.s3.amazonaws.com/public/images/b83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
+
+    const markdown = renderMarkdown({
+      url: "https://example.com/post",
+      metadata: {
+        coverImage: resizedUrl,
+      },
+      content: [
+        {
+          type: "markdown",
+          markdown: `[
+
+![](${resizedUrl})
+
+](${linkedUrl})`,
+        },
+      ],
+    });
+
+    expect(markdown).toContain(`coverImage: "${canonicalUrl}"`);
+    expect(markdown).toContain(`![](${canonicalUrl})`);
+    expect(markdown).not.toContain(`[![](${canonicalUrl})](${canonicalUrl})`);
+    expect(markdown).not.toContain("substackcdn.com/image/fetch");
+  });
+
+  test("renders linked images on a single line when href differs from the image url", () => {
+    const markdown = renderMarkdown({
+      url: "https://example.com/post",
+      content: [
+        {
+          type: "markdown",
+          markdown: `[
+
+![diagram](https://cdn.example.com/body.webp)
+
+](https://example.com/source)`,
+        },
+      ],
+    });
+
+    expect(markdown).toContain("[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)");
+    expect(markdown).not.toContain("](https://example.com/source)\n");
+  });
+});

packages/baoyu-fetch/src/__tests__/profile.test.ts

@@ -0,0 +1,68 @@
+import { afterEach, describe, expect, test } from "bun:test";
+import fs from "node:fs";
+import path from "node:path";
+import os from "node:os";
+import {
+  ensureChromeProfileDir,
+  hasChromeLockArtifacts,
+  resolveChromeProfileDir,
+  shouldRetryChromeLaunchRecovery,
+} from "../browser/profile";
+
+const originalProfile = process.env.BAOYU_CHROME_PROFILE_DIR;
+
+afterEach(() => {
+  if (originalProfile === undefined) {
+    delete process.env.BAOYU_CHROME_PROFILE_DIR;
+  } else {
+    process.env.BAOYU_CHROME_PROFILE_DIR = originalProfile;
+  }
+});
+
+describe("resolveChromeProfileDir", () => {
+  test("uses BAOYU_CHROME_PROFILE_DIR when set", () => {
+    process.env.BAOYU_CHROME_PROFILE_DIR = "/tmp/baoyu-profile";
+    expect(resolveChromeProfileDir()).toBe("/tmp/baoyu-profile");
+  });
+
+  test("falls back to shared baoyu-skills profile path", () => {
+    delete process.env.BAOYU_CHROME_PROFILE_DIR;
+    const resolved = resolveChromeProfileDir();
+    if (process.platform === "darwin") {
+      expect(resolved).toBe(path.join(os.homedir(), "Library", "Application Support", "baoyu-skills", "chrome-profile"));
+    } else if (process.platform === "win32") {
+      expect(resolved.endsWith(path.join("baoyu-skills", "chrome-profile"))).toBe(true);
+    } else {
+      expect(resolved.endsWith(path.join("baoyu-skills", "chrome-profile"))).toBe(true);
+    }
+  });
+});
+
+describe("ensureChromeProfileDir", () => {
+  test("creates the profile directory when missing", () => {
+    const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "baoyu-fetch-profile-"));
+    const profileDir = path.join(tempRoot, "nested", "chrome-profile");
+
+    try {
+      expect(fs.existsSync(profileDir)).toBe(false);
+      expect(ensureChromeProfileDir(profileDir)).toBe(profileDir);
+      expect(fs.statSync(profileDir).isDirectory()).toBe(true);
+    } finally {
+      fs.rmSync(tempRoot, { force: true, recursive: true });
+    }
+  });
+});
+
+describe("stale lock recovery helpers", () => {
+  test("detects Chrome singleton lock artifacts", () => {
+    expect(hasChromeLockArtifacts(["Cookies", "SingletonLock"])).toBe(true);
+    expect(hasChromeLockArtifacts(["chrome.pid"])).toBe(true);
+    expect(hasChromeLockArtifacts(["Preferences", "Cookies"])).toBe(false);
+  });
+
+  test("only retries stale-lock recovery when no live owner exists", () => {
+    expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: true, hasLiveOwner: false })).toBe(true);
+    expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: true, hasLiveOwner: true })).toBe(false);
+    expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: false, hasLiveOwner: false })).toBe(false);
+  });
+});

packages/baoyu-fetch/src/__tests__/wait-mode.test.ts

@@ -0,0 +1,105 @@
+import { describe, expect, test } from "bun:test";
+import { shouldAutoContinueForceWait, shouldKeepBrowserOpenAfterInteraction } from "../commands/convert";
+
+describe("shouldAutoContinueForceWait", () => {
+  test("continues when a challenge disappears", () => {
+    expect(
+      shouldAutoContinueForceWait(
+        {
+          url: "https://example.com/challenge",
+          hasGate: true,
+          loginState: "unknown",
+          sessionReady: true,
+        },
+        {
+          url: "https://example.com/article",
+          hasGate: false,
+          loginState: "unknown",
+          sessionReady: true,
+        },
+      ),
+    ).toBe(true);
+  });
+
+  test("waits for X session cookies before continuing after login", () => {
+    expect(
+      shouldAutoContinueForceWait(
+        {
+          url: "https://x.com/i/flow/login",
+          hasGate: false,
+          loginState: "logged_out",
+          sessionReady: false,
+        },
+        {
+          url: "https://x.com/home",
+          hasGate: false,
+          loginState: "logged_in",
+          sessionReady: false,
+        },
+      ),
+    ).toBe(false);
+
+    expect(
+      shouldAutoContinueForceWait(
+        {
+          url: "https://x.com/i/flow/login",
+          hasGate: false,
+          loginState: "logged_out",
+          sessionReady: false,
+        },
+        {
+          url: "https://x.com/home",
+          hasGate: false,
+          loginState: "logged_in",
+          sessionReady: true,
+        },
+      ),
+    ).toBe(true);
+  });
+
+  test("does not continue when nothing changed yet", () => {
+    expect(
+      shouldAutoContinueForceWait(
+        {
+          url: "https://x.com/lennysan/status/2036483059407810640",
+          hasGate: false,
+          loginState: "unknown",
+          sessionReady: false,
+        },
+        {
+          url: "https://x.com/lennysan/status/2036483059407810640",
+          hasGate: false,
+          loginState: "unknown",
+          sessionReady: false,
+        },
+      ),
+    ).toBe(false);
+  });
+});
+
+describe("shouldKeepBrowserOpenAfterInteraction", () => {
+  test("keeps launched X login browsers open", () => {
+    expect(
+      shouldKeepBrowserOpenAfterInteraction({
+        launched: true,
+        interaction: { kind: "login", provider: "x" },
+      }),
+    ).toBe(true);
+  });
+
+  test("does not keep reused or non-login browsers open", () => {
+    expect(
+      shouldKeepBrowserOpenAfterInteraction({
+        launched: false,
+        interaction: { kind: "login", provider: "x" },
+      }),
+    ).toBe(false);
+
+    expect(
+      shouldKeepBrowserOpenAfterInteraction({
+        launched: true,
+        interaction: { kind: "cloudflare", provider: "cloudflare" },
+      }),
+    ).toBe(false);
+  });
+});

packages/baoyu-fetch/src/__tests__/x-article.test.ts

@@ -0,0 +1,443 @@
+import { describe, expect, test } from "bun:test";
+import { extractArticleDocumentFromPayload } from "../adapters/x/article";
+
+describe("x article extraction", () => {
+  test("renders markdown entities referenced by atomic blocks", () => {
+    const payload = {
+      data: {
+        tweetResult: {
+          result: {
+            rest_id: "2036762680401223946",
+            legacy: {
+              full_text: "Fallback text",
+              favorite_count: 12,
+              retweet_count: 3,
+              reply_count: 1,
+              created_at: "Wed Mar 25 11:10:38 +0000 2026",
+            },
+            core: {
+              user_results: {
+                result: {
+                  legacy: {
+                    name: "Eric Zakariasson",
+                    screen_name: "ericzakariasson",
+                  },
+                },
+              },
+            },
+            article: {
+              article_results: {
+                result: {
+                  title: "Building CLIs for agents",
+                  content_state: {
+                    blocks: [
+                      {
+                        type: "unstyled",
+                        text: "Make it non-interactive.",
+                        data: {},
+                        entityRanges: [],
+                        inlineStyleRanges: [],
+                      },
+                      {
+                        type: "atomic",
+                        text: " ",
+                        data: {},
+                        entityRanges: [{ key: 0, length: 1, offset: 0 }],
+                        inlineStyleRanges: [],
+                      },
+                      {
+                        type: "unstyled",
+                        text: "Return data on success.",
+                        data: {},
+                        entityRanges: [],
+                        inlineStyleRanges: [],
+                      },
+                    ],
+                    entityMap: [
+                      {
+                        key: "0",
+                        value: {
+                          type: "MARKDOWN",
+                          mutability: "Mutable",
+                          data: {
+                            markdown: "```bash\n$ mycli deploy --env production --dry-run\n```",
+                          },
+                        },
+                      },
+                    ],
+                  },
+                },
+              },
+            },
+          },
+        },
+      },
+    };
+
+    const document = extractArticleDocumentFromPayload(
+      payload,
+      "2036762680401223946",
+      "https://x.com/ericzakariasson/status/2036762680401223946",
+    );
+
+    expect(document).not.toBeNull();
+    expect(document?.metadata?.kind).toBe("x/article");
+
+    const content = document?.content[0];
+    expect(content?.type).toBe("markdown");
+    if (!content || content.type !== "markdown") {
+      throw new Error("Expected markdown content");
+    }
+
+    expect(content.markdown).toContain("```bash");
+    expect(content.markdown).toContain("$ mycli deploy --env production --dry-run");
+    expect(content.markdown).toContain("Make it non-interactive.");
+    expect(content.markdown).toContain("Return data on success.");
+  });
+
+  test("renders media, embedded tweets, and cover image from article entities", () => {
+    const embeddedTweetPayload = {
+      data: {
+        tweetResult: {
+          result: {
+            rest_id: "999",
+            legacy: {
+              full_text: "Embedded tweet text",
+              favorite_count: 4,
+              retweet_count: 2,
+              reply_count: 1,
+              created_at: "Wed Mar 25 11:10:38 +0000 2026",
+              extended_entities: {
+                media: [
+                  {
+                    type: "photo",
+                    media_url_https: "https://pbs.twimg.com/media/embedded.jpg",
+                  },
+                ],
+              },
+            },
+            core: {
+              user_results: {
+                result: {
+                  core: {
+                    name: "Embedded Author",
+                    screen_name: "embedded_author",
+                  },
+                  legacy: {},
+                },
+              },
+            },
+          },
+        },
+      },
+    };
+
+    const articlePayload = {
+      data: {
+        tweetResult: {
+          result: {
+            rest_id: "2036670816344064290",
+            legacy: {
+              full_text: "Fallback text",
+              favorite_count: 12,
+              retweet_count: 3,
+              reply_count: 1,
+              created_at: "Wed Mar 25 11:10:38 +0000 2026",
+            },
+            core: {
+              user_results: {
+                result: {
+                  legacy: {
+                    name: "Eric Zakariasson",
+                    screen_name: "ericzakariasson",
+                  },
+                },
+              },
+            },
+            article: {
+              article_results: {
+                result: {
+                  title: "Article with media",
+                  cover_media: {
+                    media_info: {
+                      original_img_url: "https://pbs.twimg.com/media/cover?format=jpeg&name=small",
+                    },
+                  },
+                  media_entities: [
+                    {
+                      media_id: "42",
+                      media_info: {
+                        original_img_url: "https://pbs.twimg.com/media/body.jpg",
+                      },
+                    },
+                  ],
+                  content_state: {
+                    blocks: [
+                      {
+                        type: "unstyled",
+                        text: "Read more: https://t.co/example",
+                        data: {},
+                        entityRanges: [{ key: 2, length: 20, offset: 11 }],
+                        inlineStyleRanges: [],
+                      },
+                      {
+                        type: "atomic",
+                        text: " ",
+                        data: {},
+                        entityRanges: [{ key: 0, length: 1, offset: 0 }],
+                        inlineStyleRanges: [],
+                      },
+                      {
+                        type: "atomic",
+                        text: " ",
+                        data: {},
+                        entityRanges: [{ key: 1, length: 1, offset: 0 }],
+                        inlineStyleRanges: [],
+                      },
+                    ],
+                    entityMap: [
+                      {
+                        key: "0",
+                        value: {
+                          type: "MEDIA",
+                          mutability: "Immutable",
+                          data: {
+                            mediaItems: [{ mediaId: "42" }],
+                          },
+                        },
+                      },
+                      {
+                        key: "1",
+                        value: {
+                          type: "TWEET",
+                          mutability: "Immutable",
+                          data: {
+                            tweetId: "999",
+                          },
+                        },
+                      },
+                      {
+                        key: "2",
+                        value: {
+                          type: "LINK",
+                          mutability: "Mutable",
+                          data: {
+                            url: "https://example.com/report",
+                          },
+                        },
+                      },
+                    ],
+                  },
+                },
+              },
+            },
+          },
+        },
+      },
+    };
+
+    const document = extractArticleDocumentFromPayload(
+      articlePayload,
+      "2036670816344064290",
+      "https://x.com/ericzakariasson/status/2036670816344064290",
+      [articlePayload, embeddedTweetPayload],
+    );
+
+    expect(document).not.toBeNull();
+    expect(document?.metadata?.coverImage).toBe(
+      "https://pbs.twimg.com/media/cover?format=jpg&name=4096x4096",
+    );
+
+    const content = document?.content[0];
+    expect(content?.type).toBe("markdown");
+    if (!content || content.type !== "markdown") {
+      throw new Error("Expected markdown content");
+    }
+
+    expect(content.markdown).toContain("https://example.com/report");
+    expect(content.markdown).toContain("![](https://pbs.twimg.com/media/body?format=jpg&name=4096x4096)");
+    expect(content.markdown).toContain("> Embedded Author (@embedded_author)");
+    expect(content.markdown).toContain("> Embedded tweet text");
+    expect(content.markdown).toContain(
+      "> ![](https://pbs.twimg.com/media/embedded?format=jpg&name=4096x4096)",
+    );
+  });
+
+  test("prefers expanded link entity urls in article blocks", () => {
+    const payload = {
+      data: {
+        tweetResult: {
+          result: {
+            rest_id: "2036670816344064290",
+            legacy: {
+              full_text: "Fallback text",
+              favorite_count: 12,
+              retweet_count: 3,
+              reply_count: 1,
+              created_at: "Wed Mar 25 11:10:38 +0000 2026",
+            },
+            core: {
+              user_results: {
+                result: {
+                  legacy: {
+                    name: "Eric Zakariasson",
+                    screen_name: "ericzakariasson",
+                  },
+                },
+              },
+            },
+            article: {
+              article_results: {
+                result: {
+                  title: "Article with expanded links",
+                  content_state: {
+                    blocks: [
+                      {
+                        type: "unstyled",
+                        text: "Read more: https://t.co/example",
+                        data: {},
+                        entityRanges: [{ key: 0, length: 20, offset: 11 }],
+                        inlineStyleRanges: [],
+                      },
+                    ],
+                    entityMap: [
+                      {
+                        key: "0",
+                        value: {
+                          type: "LINK",
+                          mutability: "Mutable",
+                          data: {
+                            expanded_url: "https://example.com/report",
+                            url: "https://t.co/example",
+                          },
+                        },
+                      },
+                    ],
+                  },
+                },
+              },
+            },
+          },
+        },
+      },
+    };
+
+    const document = extractArticleDocumentFromPayload(
+      payload,
+      "2036670816344064290",
+      "https://x.com/ericzakariasson/status/2036670816344064290",
+    );
+
+    expect(document).not.toBeNull();
+
+    const content = document?.content[0];
+    expect(content?.type).toBe("markdown");
+    if (!content || content.type !== "markdown") {
+      throw new Error("Expected markdown content");
+    }
+
+    expect(content.markdown).toContain("https://example.com/report");
+    expect(content.markdown).not.toContain("https://t.co/example");
+  });
+
+  test("renders article video media as the highest bitrate mp4 link", () => {
+    const payload = {
+      data: {
+        tweetResult: {
+          result: {
+            rest_id: "2046628728210350366",
+            legacy: {
+              full_text: "Fallback text",
+              favorite_count: 12,
+              retweet_count: 3,
+              reply_count: 1,
+              created_at: "Tue Apr 21 16:34:47 +0000 2026",
+            },
+            core: {
+              user_results: {
+                result: {
+                  legacy: {
+                    name: "Google AI Studio",
+                    screen_name: "GoogleAIStudio",
+                  },
+                },
+              },
+            },
+            article: {
+              article_results: {
+                result: {
+                  title: "Article with video",
+                  media_entities: [
+                    {
+                      media_id: "2046627051822530560",
+                      media_info: {
+                        __typename: "ApiVideo",
+                        variants: [
+                          {
+                            bit_rate: 2176000,
+                            content_type: "video/mp4",
+                            url: "https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1280x720/medium.mp4",
+                          },
+                          {
+                            content_type: "application/x-mpegURL",
+                            url: "https://video.twimg.com/amplify_video/2046627051822530560/pl/playlist.m3u8",
+                          },
+                          {
+                            bit_rate: 10368000,
+                            content_type: "video/mp4",
+                            url: "https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1920x1080/high.mp4",
+                          },
+                        ],
+                      },
+                    },
+                  ],
+                  content_state: {
+                    blocks: [
+                      {
+                        type: "atomic",
+                        text: " ",
+                        data: {},
+                        entityRanges: [{ key: 0, length: 1, offset: 0 }],
+                        inlineStyleRanges: [],
+                      },
+                    ],
+                    entityMap: [
+                      {
+                        key: "0",
+                        value: {
+                          type: "MEDIA",
+                          mutability: "Immutable",
+                          data: {
+                            mediaItems: [{ mediaId: "2046627051822530560" }],
+                          },
+                        },
+                      },
+                    ],
+                  },
+                },
+              },
+            },
+          },
+        },
+      },
+    };
+
+    const document = extractArticleDocumentFromPayload(
+      payload,
+      "2046628728210350366",
+      "https://x.com/GoogleAIStudio/status/2046628728210350366",
+    );
+
+    expect(document).not.toBeNull();
+
+    const content = document?.content[0];
+    expect(content?.type).toBe("markdown");
+    if (!content || content.type !== "markdown") {
+      throw new Error("Expected markdown content");
+    }
+
+    expect(content.markdown).toBe(
+      "[video](https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1920x1080/high.mp4)",
+    );
+  });
+});

packages/baoyu-fetch/src/__tests__/x-session.test.ts

@@ -0,0 +1,28 @@
+import { describe, expect, test } from "bun:test";
+
+import { buildXSessionCookieMap, hasRequiredXSessionCookies } from "../adapters/x/session";
+
+describe("X session helpers", () => {
+  test("keeps non-empty X session cookies", () => {
+    expect(
+      buildXSessionCookieMap([
+        { name: "auth_token", value: "auth" },
+        { name: "ct0", value: "csrf" },
+        { name: "twid", value: "u=123" },
+        { name: "ct0", value: "" },
+        { name: "", value: "ignored" },
+        { name: "gt", value: undefined },
+      ]),
+    ).toEqual({
+      auth_token: "auth",
+      ct0: "csrf",
+      twid: "u=123",
+    });
+  });
+
+  test("requires auth_token and ct0 for a ready X session", () => {
+    expect(hasRequiredXSessionCookies({ auth_token: "auth" })).toBe(false);
+    expect(hasRequiredXSessionCookies({ ct0: "csrf" })).toBe(false);
+    expect(hasRequiredXSessionCookies({ auth_token: "auth", ct0: "csrf" })).toBe(true);
+  });
+});

packages/baoyu-fetch/src/__tests__/x-single.test.ts

@@ -0,0 +1,260 @@
+import { describe, expect, test } from "bun:test";
+import { extractSingleTweetDocumentFromPayload } from "../adapters/x/single";
+
+describe("x single tweet extraction", () => {
+  test("replaces t.co links in note tweets with expanded urls", () => {
+    const payload = {
+      data: {
+        tweetResult: {
+          result: {
+            rest_id: "2036483061635039711",
+            legacy: {
+              full_text:
+                "First, some context:\n\n1. This analysis is based on data from @trueupio, one of my favorite collaborators and sources of data. They track job openings at tech companies and top startups around the world (over 9,000 companies) and make it easy to browse open gigs. Their data looks",
+              favorite_count: 43,
+              retweet_count: 1,
+              reply_count: 1,
+              created_at: "Tue Mar 24 16:39:32 +0000 2026",
+              entities: {
+                hashtags: [],
+                symbols: [],
+                timestamps: [],
+                urls: [],
+                user_mentions: [
+                  {
+                    id_str: "1407256023547613193",
+                    indices: [61, 70],
+                    name: "TrueUp",
+                    screen_name: "trueupio",
+                  },
+                ],
+              },
+            },
+            note_tweet: {
+              note_tweet_results: {
+                result: {
+                  text:
+                    "First, some context:\n\n1. This analysis is based on data from @trueupio, one of my favorite collaborators and sources of data. They track job openings at tech companies and top startups around the world (over 9,000 companies) and make it easy to browse open gigs. Their data looks at roles at tech companies—the most sought-after and lucrative jobs. (It doesn’t include roles at non-tech companies and consulting agencies.) Browse open roles here: https://t.co/x7ff2NjpP1\n\n2. Keep reading for highlights, or jump straight to the full report: https://t.co/AbqPp2TEde",
+                  entity_set: {
+                    hashtags: [],
+                    symbols: [],
+                    urls: [
+                      {
+                        display_url: "trueup.io/jobs",
+                        expanded_url: "https://trueup.io/jobs",
+                        indices: [447, 470],
+                        url: "https://t.co/x7ff2NjpP1",
+                      },
+                      {
+                        display_url: "lennysnewsletter.com/i/191595250/if…",
+                        expanded_url:
+                          "https://www.lennysnewsletter.com/i/191595250/if-youre-having-trouble-finding-a-job",
+                        indices: [541, 564],
+                        url: "https://t.co/AbqPp2TEde",
+                      },
+                    ],
+                    user_mentions: [
+                      {
+                        id_str: "1407256023547613193",
+                        indices: [61, 70],
+                        name: "TrueUp",
+                        screen_name: "trueupio",
+                      },
+                    ],
+                  },
+                },
+              },
+            },
+            core: {
+              user_results: {
+                result: {
+                  legacy: {
+                    name: "Lenny Rachitsky",
+                    screen_name: "lennysan",
+                  },
+                },
+              },
+            },
+          },
+        },
+      },
+    };
+
+    const document = extractSingleTweetDocumentFromPayload(
+      payload,
+      "2036483061635039711",
+      "https://x.com/lennysan/status/2036483061635039711",
+    );
+
+    expect(document).not.toBeNull();
+
+    const paragraphBlock = document?.content.find((block) => block.type === "paragraph");
+    expect(paragraphBlock).toEqual({
+      type: "paragraph",
+      text:
+        "First, some context:\n\n1. This analysis is based on data from @trueupio, one of my favorite collaborators and sources of data. They track job openings at tech companies and top startups around the world (over 9,000 companies) and make it easy to browse open gigs. Their data looks at roles at tech companies—the most sought-after and lucrative jobs. (It doesn’t include roles at non-tech companies and consulting agencies.) Browse open roles here: https://trueup.io/jobs\n\n2. Keep reading for highlights, or jump straight to the full report: https://www.lennysnewsletter.com/i/191595250/if-youre-having-trouble-finding-a-job",
+    });
+  });
+
+  test("upgrades image urls to high resolution for tweet and quoted tweet media", () => {
+    const payload = {
+      data: {
+        tweetResult: {
+          result: {
+            rest_id: "2036762680401223946",
+            legacy: {
+              full_text: "Main tweet text https://t.co/media",
+              favorite_count: 12,
+              retweet_count: 3,
+              reply_count: 1,
+              created_at: "Wed Mar 25 11:10:38 +0000 2026",
+              extended_entities: {
+                media: [
+                  {
+                    type: "photo",
+                    media_url_https: "https://pbs.twimg.com/media/main-image.png",
+                    url: "https://t.co/media",
+                  },
+                ],
+              },
+            },
+            core: {
+              user_results: {
+                result: {
+                  legacy: {
+                    name: "Eric Zakariasson",
+                    screen_name: "ericzakariasson",
+                  },
+                },
+              },
+            },
+            quoted_status_result: {
+              result: {
+                rest_id: "999",
+                legacy: {
+                  full_text: "Quoted tweet text",
+                  favorite_count: 4,
+                  retweet_count: 2,
+                  reply_count: 1,
+                  created_at: "Wed Mar 25 10:10:38 +0000 2026",
+                  extended_entities: {
+                    media: [
+                      {
+                        type: "photo",
+                        media_url_https: "https://pbs.twimg.com/media/quoted?format=jpeg&name=small",
+                      },
+                    ],
+                  },
+                },
+                core: {
+                  user_results: {
+                    result: {
+                      legacy: {
+                        name: "Quoted Author",
+                        screen_name: "quoted_author",
+                      },
+                    },
+                  },
+                },
+              },
+            },
+          },
+        },
+      },
+    };
+
+    const document = extractSingleTweetDocumentFromPayload(
+      payload,
+      "2036762680401223946",
+      "https://x.com/ericzakariasson/status/2036762680401223946",
+    );
+
+    expect(document).not.toBeNull();
+
+    const imageBlock = document?.content.find((block) => block.type === "image");
+    expect(imageBlock).toEqual({
+      type: "image",
+      url: "https://pbs.twimg.com/media/main-image?format=png&name=4096x4096",
+    });
+
+    const quoteBlock = document?.content.find((block) => block.type === "quote");
+    expect(quoteBlock).toEqual({
+      type: "quote",
+      text:
+        "Quoted Author (@quoted_author)\n\nQuoted tweet text\n\nphoto: https://pbs.twimg.com/media/quoted?format=jpg&name=4096x4096",
+    });
+  });
+
+  test("uses the highest bitrate mp4 variant for tweet video media", () => {
+    const payload = {
+      data: {
+        tweetResult: {
+          result: {
+            rest_id: "2046628728210350366",
+            legacy: {
+              full_text: "Video post https://t.co/video",
+              favorite_count: 12,
+              retweet_count: 3,
+              reply_count: 1,
+              created_at: "Tue Apr 21 16:34:47 +0000 2026",
+              extended_entities: {
+                media: [
+                  {
+                    type: "video",
+                    media_url_https: "https://pbs.twimg.com/amplify_video_thumb/2046627051822530560/img/poster.jpg",
+                    url: "https://t.co/video",
+                    video_info: {
+                      variants: [
+                        {
+                          content_type: "application/x-mpegURL",
+                          url: "https://video.twimg.com/amplify_video/2046627051822530560/pl/playlist.m3u8",
+                        },
+                        {
+                          bitrate: 256000,
+                          content_type: "video/mp4",
+                          url: "https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/480x270/low.mp4",
+                        },
+                        {
+                          bitrate: 10368000,
+                          content_type: "video/mp4",
+                          url: "https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1920x1080/high.mp4",
+                        },
+                      ],
+                    },
+                  },
+                ],
+              },
+            },
+            core: {
+              user_results: {
+                result: {
+                  legacy: {
+                    name: "Google AI Studio",
+                    screen_name: "GoogleAIStudio",
+                  },
+                },
+              },
+            },
+          },
+        },
+      },
+    };
+
+    const document = extractSingleTweetDocumentFromPayload(
+      payload,
+      "2046628728210350366",
+      "https://x.com/GoogleAIStudio/status/2046628728210350366",
+    );
+
+    expect(document).not.toBeNull();
+
+    const listBlock = document?.content.find((block) => block.type === "list");
+    expect(listBlock).toEqual({
+      type: "list",
+      ordered: false,
+      items: [
+        "video: https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1920x1080/high.mp4",
+      ],
+    });
+  });
+});

packages/baoyu-fetch/src/__tests__/x-thread.test.ts

@@ -0,0 +1,305 @@
+import { describe, expect, test } from "bun:test";
+import { extractThreadDocumentFromPayloads, extractThreadTweetsFromPayloads } from "../adapters/x/thread";
+
+function buildTweet(options: {
+  id: string;
+  text: string;
+  createdAt: string;
+  userId?: string;
+  screenName?: string;
+  name?: string;
+  conversationId?: string;
+  inReplyToStatusId?: string;
+  inReplyToUserId?: string;
+  quotedTweet?: unknown;
+}) {
+  const userId = options.userId ?? "3178231";
+  const screenName = options.screenName ?? "dotey";
+  const name = options.name ?? "宝玉";
+
+  return {
+    __typename: "Tweet",
+    rest_id: options.id,
+    legacy: {
+      id_str: options.id,
+      full_text: options.text,
+      favorite_count: 0,
+      retweet_count: 0,
+      reply_count: 0,
+      created_at: options.createdAt,
+      user_id_str: userId,
+      conversation_id_str: options.conversationId ?? options.id,
+      in_reply_to_status_id_str: options.inReplyToStatusId,
+      in_reply_to_user_id_str: options.inReplyToUserId,
+    },
+    core: {
+      user_results: {
+        result: {
+          core: {
+            name,
+            screen_name: screenName,
+          },
+          legacy: {},
+        },
+      },
+    },
+    quoted_status_result: options.quotedTweet
+      ? {
+          result: options.quotedTweet,
+        }
+      : undefined,
+  };
+}
+
+function tweetEntry(tweet: unknown) {
+  return {
+    content: {
+      itemContent: {
+        tweet_results: {
+          result: tweet,
+        },
+      },
+    },
+  };
+}
+
+function moduleTweetItem(tweet: unknown) {
+  return {
+    item: {
+      itemContent: {
+        tweet_results: {
+          result: tweet,
+        },
+      },
+    },
+  };
+}
+
+describe("x thread extraction", () => {
+  test("keeps only the continuous same-author reply chain", () => {
+    const rootId = "1996285439867556304";
+    const reply1Id = "1996285442275340783";
+    const reply2Id = "1996285444582146559";
+    const quotedId = "1993729800922341810";
+    const quotedInsideThreadId = "1993729800922341811";
+    const otherAuthorReplyId = "2000000000000000001";
+    const sameAuthorAfterOtherId = "2000000000000000002";
+
+    const root = buildTweet({
+      id: rootId,
+      text: "A thread for my nana banana pro prompts 🧵",
+      createdAt: "Wed Dec 03 18:28:32 +0000 2025",
+      conversationId: rootId,
+    });
+    const reply1 = buildTweet({
+      id: reply1Id,
+      text: "Prompt 1",
+      createdAt: "Wed Dec 03 18:28:33 +0000 2025",
+      conversationId: rootId,
+      inReplyToStatusId: rootId,
+      inReplyToUserId: "3178231",
+      quotedTweet: buildTweet({
+        id: quotedInsideThreadId,
+        text: "Quoted inside the thread body",
+        createdAt: "Tue Nov 25 18:28:35 +0000 2025",
+        screenName: "quoted_author",
+        name: "Quoted Author",
+      }),
+    });
+    const reply2 = buildTweet({
+      id: reply2Id,
+      text: "Prompt 2",
+      createdAt: "Wed Dec 03 18:28:34 +0000 2025",
+      conversationId: rootId,
+      inReplyToStatusId: reply1Id,
+      inReplyToUserId: "3178231",
+    });
+    const quotedSameAuthor = buildTweet({
+      id: quotedId,
+      text: "Quoted standalone tweet",
+      createdAt: "Tue Nov 25 18:28:34 +0000 2025",
+    });
+    const otherAuthorReply = buildTweet({
+      id: otherAuthorReplyId,
+      text: "Another author joined the conversation",
+      createdAt: "Wed Dec 03 18:28:35 +0000 2025",
+      userId: "42",
+      screenName: "someone_else",
+      name: "Someone Else",
+      conversationId: rootId,
+      inReplyToStatusId: reply2Id,
+      inReplyToUserId: "3178231",
+    });
+    const sameAuthorAfterOther = buildTweet({
+      id: sameAuthorAfterOtherId,
+      text: "This should not be part of the continuous author chain",
+      createdAt: "Wed Dec 03 18:28:36 +0000 2025",
+      conversationId: rootId,
+      inReplyToStatusId: otherAuthorReplyId,
+      inReplyToUserId: "42",
+    });
+
+    const payloads = [
+      {
+        data: {
+          threaded_conversation_with_injections_v2: {
+            instructions: [
+              {
+                type: "TimelineAddEntries",
+                entries: [
+                  tweetEntry(root),
+                  tweetEntry(reply1),
+                  tweetEntry(quotedSameAuthor),
+                  {
+                    content: {
+                      items: [moduleTweetItem(reply2)],
+                    },
+                  },
+                ],
+              },
+              {
+                type: "TimelineAddToModule",
+                moduleItems: [moduleTweetItem(otherAuthorReply), moduleTweetItem(sameAuthorAfterOther)],
+              },
+            ],
+          },
+        },
+      },
+    ];
+
+    const tweets = extractThreadTweetsFromPayloads(
+      payloads,
+      rootId,
+      "https://x.com/dotey/status/1996285439867556304",
+    );
+
+    expect(tweets.map((tweet) => tweet.id)).toEqual([rootId, reply1Id, reply2Id]);
+
+    const document = extractThreadDocumentFromPayloads(
+      payloads,
+      rootId,
+      "https://x.com/dotey/status/1996285439867556304",
+    );
+
+    expect(document).not.toBeNull();
+    expect(document?.metadata?.tweetCount).toBe(3);
+    expect(document?.metadata?.lastTweetId).toBe(reply2Id);
+
+    const content = document?.content[0];
+    expect(content?.type).toBe("markdown");
+    if (!content || content.type !== "markdown") {
+      throw new Error("Expected markdown content");
+    }
+
+    expect(content.markdown).toContain("Prompt 1");
+    expect(content.markdown).toContain("Prompt 2");
+    expect(content.markdown).toContain("Quoted inside the thread body");
+    expect(content.markdown).not.toContain("Quoted standalone tweet");
+    expect(content.markdown).not.toContain("This should not be part of the continuous author chain");
+  });
+
+  test("returns null when there is no same-author reply chain", () => {
+    const rootId = "1996285439867556304";
+    const root = buildTweet({
+      id: rootId,
+      text: "Root tweet",
+      createdAt: "Wed Dec 03 18:28:32 +0000 2025",
+      conversationId: rootId,
+    });
+    const quotedSameAuthor = buildTweet({
+      id: "1993729800922341810",
+      text: "Quoted standalone tweet",
+      createdAt: "Tue Nov 25 18:28:34 +0000 2025",
+    });
+
+    const payloads = [
+      {
+        data: {
+          threaded_conversation_with_injections_v2: {
+            instructions: [
+              {
+                type: "TimelineAddEntries",
+                entries: [tweetEntry(root), tweetEntry(quotedSameAuthor)],
+              },
+            ],
+          },
+        },
+      },
+    ];
+
+    expect(
+      extractThreadDocumentFromPayloads(
+        payloads,
+        rootId,
+        "https://x.com/dotey/status/1996285439867556304",
+      ),
+    ).toBeNull();
+  });
+
+  test("restores ancestors when the requested tweet is in the middle of a thread", () => {
+    const rootId = "1996285439867556304";
+    const reply1Id = "1996285442275340783";
+    const reply2Id = "1996285444582146559";
+
+    const root = buildTweet({
+      id: rootId,
+      text: "Root tweet",
+      createdAt: "Wed Dec 03 18:28:32 +0000 2025",
+      conversationId: rootId,
+    });
+    const reply1 = buildTweet({
+      id: reply1Id,
+      text: "Middle tweet",
+      createdAt: "Wed Dec 03 18:28:33 +0000 2025",
+      conversationId: rootId,
+      inReplyToStatusId: rootId,
+      inReplyToUserId: "3178231",
+    });
+    const reply2 = buildTweet({
+      id: reply2Id,
+      text: "Last tweet",
+      createdAt: "Wed Dec 03 18:28:34 +0000 2025",
+      conversationId: rootId,
+      inReplyToStatusId: reply1Id,
+      inReplyToUserId: "3178231",
+    });
+
+    const payloads = [
+      {
+        data: {
+          threaded_conversation_with_injections_v2: {
+            instructions: [
+              {
+                type: "TimelineAddEntries",
+                entries: [
+                  tweetEntry(root),
+                  tweetEntry(reply1),
+                  tweetEntry(reply2),
+                ],
+              },
+            ],
+          },
+        },
+      },
+    ];
+
+    const tweets = extractThreadTweetsFromPayloads(
+      payloads,
+      reply1Id,
+      "https://x.com/dotey/status/1996285442275340783",
+    );
+
+    expect(tweets.map((tweet) => tweet.id)).toEqual([rootId, reply1Id, reply2Id]);
+
+    const document = extractThreadDocumentFromPayloads(
+      payloads,
+      reply1Id,
+      "https://x.com/dotey/status/1996285442275340783",
+    );
+
+    expect(document).not.toBeNull();
+    expect(document?.metadata?.tweetId).toBe(rootId);
+    expect(document?.metadata?.lastTweetId).toBe(reply2Id);
+    expect(document?.metadata?.tweetCount).toBe(3);
+  });
+});

packages/baoyu-fetch/src/__tests__/youtube-utils.test.ts

@@ -0,0 +1,97 @@
+import { describe, expect, test } from "bun:test";
+import {
+  buildYouTubeThumbnailCandidates,
+  formatTimestampRange,
+  parseYouTubeDescriptionChapters,
+  parseYouTubeVideoId,
+  renderYouTubeTranscriptMarkdown,
+} from "../adapters/youtube/utils";
+
+describe("parseYouTubeVideoId", () => {
+  test("parses watch URLs", () => {
+    expect(parseYouTubeVideoId(new URL("https://www.youtube.com/watch?v=abc123"))).toBe("abc123");
+  });
+
+  test("parses youtu.be URLs", () => {
+    expect(parseYouTubeVideoId(new URL("https://youtu.be/abc123"))).toBe("abc123");
+  });
+
+  test("parses shorts URLs", () => {
+    expect(parseYouTubeVideoId(new URL("https://www.youtube.com/shorts/abc123"))).toBe("abc123");
+  });
+});
+
+describe("parseYouTubeDescriptionChapters", () => {
+  test("extracts chapter timestamps from description lines", () => {
+    expect(
+      parseYouTubeDescriptionChapters(`0:00 Intro
+2:15 What is a product engineer?
+10:05 Career paths`),
+    ).toEqual([
+      { title: "Intro", time: 0 },
+      { title: "What is a product engineer?", time: 135 },
+      { title: "Career paths", time: 605 },
+    ]);
+  });
+
+  test("ignores isolated timestamps that do not look like chapters", () => {
+    expect(parseYouTubeDescriptionChapters("Published on 2026-03-26\nSee you at 1:23")).toEqual([]);
+  });
+});
+
+describe("renderYouTubeTranscriptMarkdown", () => {
+  test("renders description before chapters and keeps every segment on its own line", () => {
+    const markdown = renderYouTubeTranscriptMarkdown({
+      description: "Line one\nLine two",
+      chapters: [
+        { title: "Intro", time: 0 },
+        { title: "Deep Dive", time: 4 },
+      ],
+      segments: [
+        { start: 0, end: 2, text: "Hello everyone." },
+        { start: 2, end: 4, text: "Welcome back." },
+        { start: 4, end: 7, text: "Now the details." },
+      ],
+    });
+
+    expect(markdown).toContain("## Description");
+    expect(markdown).toContain("Line one  \nLine two");
+    expect(markdown).toContain("## Chapters");
+    expect(markdown).toContain("### Intro [0:00 -> 0:04]");
+    expect(markdown).toContain("[0:00 -> 0:02] Hello everyone.");
+    expect(markdown).toContain("[0:02 -> 0:04] Welcome back.");
+    expect(markdown).toContain("### Deep Dive [0:04 -> 0:07]");
+    expect(markdown).toContain("[0:04 -> 0:07] Now the details.");
+  });
+
+  test("falls back to a transcript section when chapters are unavailable", () => {
+    const markdown = renderYouTubeTranscriptMarkdown({
+      segments: [{ start: 65, end: 70, text: "Single line." }],
+      chapters: [],
+    });
+
+    expect(markdown).toContain("## Transcript");
+    expect(markdown).toContain("[1:05 -> 1:10] Single line.");
+  });
+});
+
+describe("thumbnail helpers", () => {
+  test("prefers max resolution thumbnail candidates before listed fallbacks", () => {
+    expect(
+      buildYouTubeThumbnailCandidates("abc123", [
+        "https://i.ytimg.com/vi/abc123/hqdefault.jpg",
+        "https://i.ytimg.com/vi/abc123/mqdefault.jpg?foo=bar",
+      ]),
+    ).toEqual([
+      "https://i.ytimg.com/vi/abc123/maxresdefault.jpg",
+      "https://i.ytimg.com/vi/abc123/sddefault.jpg",
+      "https://i.ytimg.com/vi/abc123/hqdefault.jpg",
+      "https://i.ytimg.com/vi/abc123/mqdefault.jpg",
+      "https://i.ytimg.com/vi/abc123/default.jpg",
+    ]);
+  });
+
+  test("renders timestamp ranges with start and end values", () => {
+    expect(formatTimestampRange(3661, 3675)).toBe("[1:01:01 -> 1:01:15]");
+  });
+});

packages/baoyu-fetch/src/adapters/generic/index.ts

@@ -0,0 +1,74 @@
+import type { Adapter } from "../types";
+import { detectInteractionGate } from "../../browser/interaction-gates";
+import { captureNormalizedPageSnapshot } from "../../browser/page-snapshot";
+import { convertHtmlToMarkdown } from "../../extract/html-to-markdown";
+
+export const genericAdapter: Adapter = {
+  name: "generic",
+  match() {
+    return true;
+  },
+  async process(context) {
+    context.log.info(`Loading ${context.input.url.toString()} with generic adapter`);
+    await context.browser.goto(context.input.url.toString(), context.timeoutMs);
+
+    try {
+      await context.network.waitForIdle({
+        idleMs: 1_200,
+        timeoutMs: Math.min(context.timeoutMs, 15_000),
+      });
+    } catch {
+      context.log.debug("Network idle timed out on initial load; continuing.");
+    }
+
+    await context.browser.scrollToEnd({ maxSteps: 4, delayMs: 300 });
+
+    try {
+      await context.network.waitForIdle({
+        idleMs: 900,
+        timeoutMs: Math.min(context.timeoutMs, 10_000),
+      });
+    } catch {
+      context.log.debug("Network idle timed out after scrolling; continuing.");
+    }
+
+    const interaction = await detectInteractionGate(context.browser);
+    if (interaction) {
+      return {
+        status: "needs_interaction",
+        interaction,
+      };
+    }
+
+    const snapshot = await captureNormalizedPageSnapshot(context.browser);
+    const converted = await convertHtmlToMarkdown(snapshot.html, snapshot.finalUrl, {
+      enableRemoteMarkdownFallback: context.outputFormat === "markdown",
+      preserveBase64Images: context.downloadMedia,
+    });
+    const document = {
+      url: snapshot.finalUrl,
+      canonicalUrl: converted.metadata.canonicalUrl,
+      title: converted.metadata.title,
+      author: converted.metadata.author,
+      siteName: converted.metadata.siteName,
+      publishedAt: converted.metadata.publishedAt,
+      summary: converted.metadata.summary,
+      adapter: "generic",
+      metadata: {
+        coverImage: converted.metadata.coverImage,
+        language: converted.metadata.language,
+        capturedAt: converted.metadata.capturedAt,
+        conversionMethod: converted.conversionMethod,
+        fallbackReason: converted.fallbackReason,
+        kind: "generic/article",
+      },
+      content: converted.markdown ? [{ type: "markdown" as const, markdown: converted.markdown }] : [],
+    };
+
+    return {
+      status: "ok",
+      document,
+      media: converted.media,
+    };
+  },
+};

packages/baoyu-fetch/src/adapters/hn/index.ts

@@ -0,0 +1,391 @@
+import { JSDOM } from "jsdom";
+import TurndownService from "turndown";
+import { gfm } from "turndown-plugin-gfm";
+import type { Adapter } from "../types";
+import type { ExtractedDocument } from "../../extract/document";
+import { collectMediaFromDocument } from "../../media/markdown-media";
+
+const HN_BASE_URL = "https://news.ycombinator.com";
+
+const turndown = new TurndownService({
+  headingStyle: "atx",
+  bulletListMarker: "-",
+  codeBlockStyle: "fenced",
+});
+
+turndown.use(gfm);
+
+export interface HnItem {
+  id: number;
+  type: "story" | "comment" | "job" | "poll" | "pollopt" | string;
+  by?: string;
+  time?: number;
+  text?: string;
+  title?: string;
+  url?: string;
+  score?: number;
+  descendants?: number;
+  kids?: number[];
+  parent?: number;
+  deleted?: boolean;
+  dead?: boolean;
+}
+
+export interface HnCommentNode {
+  item: HnItem;
+  children: HnCommentNode[];
+}
+
+interface ParsedHnThread {
+  story: HnItem;
+  comments: HnCommentNode[];
+}
+
+function decodeHtmlText(value: string | undefined): string | undefined {
+  if (!value) {
+    return undefined;
+  }
+
+  const dom = new JSDOM(`<!doctype html><html><body>${value}</body></html>`);
+  return dom.window.document.body.textContent?.trim() || undefined;
+}
+
+function normalizeMarkdown(markdown: string): string {
+  return markdown
+    .replace(/\r\n/g, "\n")
+    .replace(/[ \t]+\n/g, "\n")
+    .replace(/\n{3,}/g, "\n\n")
+    .trim();
+}
+
+function convertHnHtmlToMarkdown(html: string | undefined, baseUrl: string): string {
+  if (!html?.trim()) {
+    return "";
+  }
+
+  const dom = new JSDOM(`<div id="__root">${html}</div>`, { url: baseUrl });
+  const root = dom.window.document.querySelector("#__root");
+  if (!root) {
+    return "";
+  }
+
+  root.querySelectorAll("a[href]").forEach((element) => {
+    const href = element.getAttribute("href");
+    if (!href) {
+      return;
+    }
+
+    try {
+      element.setAttribute("href", new URL(href, baseUrl).toString());
+    } catch {
+      // Ignore malformed URLs and keep the original href.
+    }
+  });
+
+  return normalizeMarkdown(turndown.turndown(root.innerHTML));
+}
+
+function formatIsoTimestamp(unixSeconds: number | undefined): string | undefined {
+  if (!unixSeconds || !Number.isFinite(unixSeconds)) {
+    return undefined;
+  }
+
+  return new Date(unixSeconds * 1_000).toISOString();
+}
+
+function formatDisplayTimestamp(unixSeconds: number | undefined): string {
+  const iso = formatIsoTimestamp(unixSeconds);
+  if (!iso) {
+    return "unknown time";
+  }
+
+  return iso.replace("T", " ").replace(".000Z", " UTC");
+}
+
+function indentMarkdown(markdown: string, spaces: number): string {
+  const prefix = " ".repeat(spaces);
+  return markdown
+    .split("\n")
+    .map((line) => (line ? `${prefix}${line}` : prefix))
+    .join("\n");
+}
+
+function renderCommentHeader(item: HnItem, pageUrl: string): string {
+  const author = item.by ?? "[deleted]";
+  const time = item.id
+    ? `[${formatDisplayTimestamp(item.time)}](${pageUrl}#${item.id})`
+    : formatDisplayTimestamp(item.time);
+  return `${author} · ${time}`;
+}
+
+function renderCommentNode(node: HnCommentNode, pageUrl: string, depth = 0): string {
+  const baseIndent = " ".repeat(depth * 4);
+  const lines = [`${baseIndent}- ${renderCommentHeader(node.item, pageUrl)}`];
+  const body = convertHnHtmlToMarkdown(node.item.text, pageUrl);
+
+  if (body) {
+    lines.push("");
+    lines.push(indentMarkdown(body, depth * 4 + 4));
+  } else if (node.item.deleted || node.item.dead) {
+    lines.push("");
+    lines.push(`${baseIndent}    [comment unavailable]`);
+  }
+
+  for (const child of node.children) {
+    lines.push("");
+    lines.push(renderCommentNode(child, pageUrl, depth + 1));
+  }
+
+  return lines.join("\n");
+}
+
+export function buildHnThreadMarkdown(
+  story: HnItem,
+  comments: HnCommentNode[],
+  pageUrl: string,
+): string {
+  const lines: string[] = [];
+  const storyUrl = story.url ? new URL(story.url, pageUrl).toString() : undefined;
+  const storyText = convertHnHtmlToMarkdown(story.text, pageUrl);
+
+  if (storyUrl && storyUrl !== pageUrl) {
+    lines.push(`Source: [${storyUrl}](${storyUrl})`);
+  }
+  lines.push(`HN Item: [${story.id}](${pageUrl})`);
+
+  const submittedBy = story.by ? ` by ${story.by}` : "";
+  const submittedAt = formatDisplayTimestamp(story.time);
+  lines.push(`Submitted${submittedBy} at ${submittedAt}`);
+
+  const stats: string[] = [];
+  if (typeof story.score === "number") {
+    stats.push(`${story.score} points`);
+  }
+  if (typeof story.descendants === "number") {
+    stats.push(`${story.descendants} comments`);
+  }
+  if (stats.length > 0) {
+    lines.push(stats.join(" | "));
+  }
+
+  if (storyText) {
+    lines.push("");
+    lines.push("## Post");
+    lines.push("");
+    lines.push(storyText);
+  }
+
+  lines.push("");
+  lines.push("## Comments");
+  lines.push("");
+
+  if (comments.length === 0) {
+    lines.push("No comments.");
+  } else {
+    lines.push(comments.map((comment) => renderCommentNode(comment, pageUrl)).join("\n\n"));
+  }
+
+  return normalizeMarkdown(lines.join("\n"));
+}
+
+export function buildHnDocument(
+  story: HnItem,
+  comments: HnCommentNode[],
+  pageUrl: string,
+): ExtractedDocument {
+  const decodedTitle = decodeHtmlText(story.title) ?? `HN Item ${story.id}`;
+
+  return {
+    url: pageUrl,
+    canonicalUrl: pageUrl,
+    title: decodedTitle,
+    author: story.by,
+    siteName: "Hacker News",
+    publishedAt: formatIsoTimestamp(story.time),
+    adapter: "hn",
+    metadata: {
+      kind: "hn/story",
+      storyId: story.id,
+      storyUrl: story.url ? new URL(story.url, pageUrl).toString() : undefined,
+      points: story.score,
+      commentCount: story.descendants,
+    },
+    content: [
+      {
+        type: "markdown",
+        markdown: buildHnThreadMarkdown(story, comments, pageUrl),
+      },
+    ],
+  };
+}
+
+export function parseHnItemId(url: URL): number | null {
+  if (url.hostname !== "news.ycombinator.com") {
+    return null;
+  }
+
+  if (url.pathname !== "/item") {
+    return null;
+  }
+
+  const value = url.searchParams.get("id");
+  if (!value || !/^\d+$/.test(value)) {
+    return null;
+  }
+
+  return Number(value);
+}
+
+function extractUnixSecondsFromAge(element: Element | null): number | undefined {
+  const title = element?.getAttribute("title")?.trim();
+  if (!title) {
+    return undefined;
+  }
+
+  const match = title.match(/(\d{9,})$/);
+  return match ? Number(match[1]) : undefined;
+}
+
+function extractScore(text: string | null | undefined): number | undefined {
+  if (!text) {
+    return undefined;
+  }
+
+  const match = text.match(/(\d+)/);
+  return match ? Number(match[1]) : undefined;
+}
+
+function extractCommentCount(container: ParentNode): number | undefined {
+  const anchors = Array.from(container.querySelectorAll("a"));
+  for (const anchor of anchors) {
+    const match = anchor.textContent?.trim().match(/(\d+)\s+comments?/i);
+    if (match) {
+      return Number(match[1]);
+    }
+  }
+  return undefined;
+}
+
+function normalizeStoryUrl(storyId: number, href: string | null | undefined, pageUrl: string): string | undefined {
+  if (!href) {
+    return undefined;
+  }
+
+  try {
+    const resolved = new URL(href, pageUrl).toString();
+    if (resolved === pageUrl || resolved === `${HN_BASE_URL}/item?id=${storyId}`) {
+      return undefined;
+    }
+    return resolved;
+  } catch {
+    return undefined;
+  }
+}
+
+export function extractHnThreadFromHtml(html: string, pageUrl: string): ParsedHnThread | null {
+  const dom = new JSDOM(html, { url: pageUrl });
+  const { document } = dom.window;
+  const storyRow = document.querySelector("table.fatitem tr.athing.submission");
+  if (!storyRow) {
+    return null;
+  }
+
+  const storyId = Number(storyRow.getAttribute("id"));
+  if (!Number.isFinite(storyId)) {
+    return null;
+  }
+
+  const titleLink = storyRow.querySelector(".titleline > a");
+  const subline = document.querySelector("table.fatitem .subline");
+  const topText = document.querySelector("table.fatitem .toptext");
+
+  const story: HnItem = {
+    id: storyId,
+    type: "story",
+    by: subline?.querySelector(".hnuser")?.textContent?.trim() || undefined,
+    time: extractUnixSecondsFromAge(subline?.querySelector(".age") ?? null),
+    title: titleLink?.innerHTML?.trim() || undefined,
+    url: normalizeStoryUrl(storyId, titleLink?.getAttribute("href"), pageUrl),
+    text: topText?.innerHTML?.trim() || undefined,
+    score: extractScore(subline?.querySelector(".score")?.textContent),
+    descendants: extractCommentCount(subline ?? document),
+  };
+
+  const roots: HnCommentNode[] = [];
+  const stack: HnCommentNode[] = [];
+
+  document.querySelectorAll("tr.athing.comtr").forEach((row) => {
+    const commentId = Number(row.getAttribute("id"));
+    if (!Number.isFinite(commentId)) {
+      return;
+    }
+
+    const indentRaw = row.querySelector("td.ind")?.getAttribute("indent");
+    const depth = indentRaw && /^\d+$/.test(indentRaw) ? Number(indentRaw) : 0;
+    const comhead = row.querySelector(".comhead");
+    const item: HnItem = {
+      id: commentId,
+      type: "comment",
+      by: comhead?.querySelector(".hnuser")?.textContent?.trim() || undefined,
+      time: extractUnixSecondsFromAge(comhead?.querySelector(".age") ?? null),
+      text: row.querySelector(".comment > .commtext")?.innerHTML?.trim() || undefined,
+      deleted: row.querySelector(".comment > .commtext") === null,
+    };
+
+    const node: HnCommentNode = {
+      item,
+      children: [],
+    };
+
+    while (stack.length > depth) {
+      stack.pop();
+    }
+
+    const parent = stack[stack.length - 1];
+    if (parent) {
+      parent.children.push(node);
+    } else {
+      roots.push(node);
+    }
+
+    stack.push(node);
+  });
+
+  return {
+    story,
+    comments: roots,
+  };
+}
+
+export const hnAdapter: Adapter = {
+  name: "hn",
+  match(input) {
+    return parseHnItemId(input.url) !== null;
+  },
+  async process(context) {
+    const itemId = parseHnItemId(context.input.url);
+    if (!itemId) {
+      return {
+        status: "no_document",
+      };
+    }
+
+    const pageUrl = context.input.url.toString();
+    context.log.info(`Loading ${pageUrl} with hn adapter`);
+    await context.browser.goto(pageUrl, context.timeoutMs);
+    const html = await context.browser.getHTML();
+    const thread = extractHnThreadFromHtml(html, pageUrl);
+    if (!thread) {
+      return {
+        status: "no_document",
+      };
+    }
+
+    const document = buildHnDocument(thread.story, thread.comments, pageUrl);
+    return {
+      status: "ok",
+      document,
+      media: collectMediaFromDocument(document),
+    };
+  },
+};

packages/baoyu-fetch/src/adapters/index.ts

@@ -0,0 +1,29 @@
+import type { Adapter, AdapterInput } from "./types";
+import { genericAdapter } from "./generic";
+import { hnAdapter } from "./hn";
+import { xAdapter } from "./x";
+import { youtubeAdapter } from "./youtube";
+
+const adapters: Adapter[] = [xAdapter, youtubeAdapter, hnAdapter, genericAdapter];
+
+export function listAdapters(): Adapter[] {
+  return adapters;
+}
+
+export function resolveAdapter(input: AdapterInput, forcedName?: string): Adapter {
+  if (forcedName) {
+    const forced = adapters.find((adapter) => adapter.name === forcedName);
+    if (!forced) {
+      throw new Error(`Unknown adapter: ${forcedName}`);
+    }
+    return forced;
+  }
+
+  const matched = adapters.find((adapter) => adapter.match(input));
+  if (!matched) {
+    throw new Error("No adapter matched the URL");
+  }
+  return matched;
+}
+
+export { genericAdapter };

packages/baoyu-fetch/src/adapters/types.ts

@@ -0,0 +1,73 @@
+import type { BrowserSession } from "../browser/session";
+import type { CdpClient } from "../browser/cdp-client";
+import type { NetworkJournal } from "../browser/network-journal";
+import type { ExtractedDocument } from "../extract/document";
+import type { MediaDownloadRequest, MediaDownloadResult, MediaAsset } from "../media/types";
+import type { Logger } from "../utils/logger";
+
+export interface AdapterInput {
+  url: URL;
+}
+
+export type LoginState = "logged_in" | "logged_out" | "unknown";
+export type InteractionKind = "login" | "cloudflare" | "recaptcha" | "hcaptcha" | "captcha" | "challenge";
+
+export interface AdapterLoginInfo {
+  provider: string;
+  state: LoginState;
+  required?: boolean;
+  username?: string;
+  reason?: string;
+}
+
+export interface WaitForInteractionRequest {
+  type: "wait_for_interaction";
+  kind: InteractionKind;
+  provider: string;
+  prompt: string;
+  reason?: string;
+  timeoutMs?: number;
+  pollIntervalMs?: number;
+  requiresVisibleBrowser?: boolean;
+}
+
+export type AdapterProcessResult =
+  | {
+      status: "ok";
+      document: ExtractedDocument;
+      media?: MediaAsset[];
+      login?: AdapterLoginInfo;
+    }
+  | {
+      status: "needs_interaction";
+      interaction: WaitForInteractionRequest;
+      login?: AdapterLoginInfo;
+    }
+  | {
+      status: "no_document";
+      login?: AdapterLoginInfo;
+    };
+
+export interface AdapterContext {
+  input: AdapterInput;
+  browser: BrowserSession;
+  network: NetworkJournal;
+  cdp: CdpClient;
+  log: Logger;
+  outputFormat: "markdown" | "json";
+  timeoutMs: number;
+  interactive: boolean;
+  downloadMedia: boolean;
+}
+
+export interface Adapter {
+  name: string;
+  match(input: AdapterInput): boolean;
+  checkLogin?(context: AdapterContext): Promise<AdapterLoginInfo>;
+  exportCookies?(context: AdapterContext, profileDir?: string): Promise<boolean>;
+  restoreCookies?(context: AdapterContext, profileDir?: string): Promise<boolean>;
+  downloadMedia?(request: MediaDownloadRequest): Promise<MediaDownloadResult>;
+  process(context: AdapterContext): Promise<AdapterProcessResult>;
+}
+
+export type { MediaAsset };

packages/baoyu-fetch/src/adapters/x/article.ts

@@ -0,0 +1,461 @@
+import type { ExtractedDocument } from "../../extract/document";
+import {
+  findTweetNode,
+  findTweetNodeById,
+  formatMediaList,
+  formatTweetAuthor,
+  getTweetAuthorMetadata,
+  getTweetText,
+  getUser,
+  isRecord,
+  normalizeTitle,
+  resolveBestXVideoVariantUrl,
+  toHighResXImageUrl,
+  toXTweet,
+} from "./shared";
+import type { JsonObject } from "./types";
+
+interface ArticleMedia {
+  kind: "image" | "video";
+  url: string;
+}
+
+function resolveArticleMedia(mediaInfo: JsonObject): ArticleMedia | null {
+  const videoUrl = resolveBestXVideoVariantUrl(mediaInfo);
+  if (videoUrl) {
+    return {
+      kind: "video",
+      url: videoUrl,
+    };
+  }
+
+  const rawUrl =
+    (typeof mediaInfo.original_img_url === "string" && mediaInfo.original_img_url) ||
+    (typeof mediaInfo.url === "string" && mediaInfo.url) ||
+    "";
+
+  if (!rawUrl) {
+    return null;
+  }
+
+  return {
+    kind: "image",
+    url: toHighResXImageUrl(rawUrl),
+  };
+}
+
+function resolveArticleMediaUrl(mediaInfo: JsonObject): string {
+  return resolveArticleMedia(mediaInfo)?.url ?? "";
+}
+
+function normalizeEntityMap(entityMap: unknown): Map<string, JsonObject> {
+  const normalized = new Map<string, JsonObject>();
+
+  if (Array.isArray(entityMap)) {
+    for (const entry of entityMap) {
+      if (!isRecord(entry)) {
+        continue;
+      }
+
+      const key =
+        typeof entry.key === "string" || typeof entry.key === "number"
+          ? String(entry.key)
+          : undefined;
+      const value = isRecord(entry.value) ? entry.value : undefined;
+      if (!key || !value) {
+        continue;
+      }
+      normalized.set(key, value);
+    }
+
+    return normalized;
+  }
+
+  if (!isRecord(entityMap)) {
+    return normalized;
+  }
+
+  for (const [key, value] of Object.entries(entityMap)) {
+    if (!isRecord(value)) {
+      continue;
+    }
+    normalized.set(key, value);
+  }
+
+  return normalized;
+}
+
+function getEntityMarkdown(entityMap: Map<string, JsonObject>, entityKey: unknown): string | null {
+  const key =
+    typeof entityKey === "string" || typeof entityKey === "number"
+      ? String(entityKey)
+      : undefined;
+  if (!key) {
+    return null;
+  }
+
+  const entity = entityMap.get(key);
+  if (!entity || entity.type !== "MARKDOWN") {
+    return null;
+  }
+
+  const data = isRecord(entity.data) ? entity.data : {};
+  if (typeof data.markdown !== "string") {
+    return null;
+  }
+
+  const markdown = data.markdown.trim();
+  return markdown || null;
+}
+
+function getLinkUrl(entityMap: Map<string, JsonObject>, entityKey: unknown): string | null {
+  const key =
+    typeof entityKey === "string" || typeof entityKey === "number"
+      ? String(entityKey)
+      : undefined;
+  if (!key) {
+    return null;
+  }
+
+  const entity = entityMap.get(key);
+  if (!entity || entity.type !== "LINK") {
+    return null;
+  }
+
+  const data = isRecord(entity.data) ? entity.data : {};
+  const candidates = [
+    data.expanded_url,
+    data.expandedUrl,
+    data.original_url,
+    data.originalUrl,
+    data.url,
+    data.display_url,
+    data.displayUrl,
+  ];
+
+  for (const candidate of candidates) {
+    if (typeof candidate === "string" && candidate.trim()) {
+      return candidate.trim();
+    }
+  }
+
+  return null;
+}
+
+function getTweetId(entityMap: Map<string, JsonObject>, entityKey: unknown): string | null {
+  const key =
+    typeof entityKey === "string" || typeof entityKey === "number"
+      ? String(entityKey)
+      : undefined;
+  if (!key) {
+    return null;
+  }
+
+  const entity = entityMap.get(key);
+  if (!entity || entity.type !== "TWEET") {
+    return null;
+  }
+
+  const data = isRecord(entity.data) ? entity.data : {};
+  if (typeof data.tweetId !== "string") {
+    return null;
+  }
+
+  return data.tweetId;
+}
+
+function buildMediaMap(articleResult: JsonObject): Map<string, ArticleMedia> {
+  const mediaMap = new Map<string, ArticleMedia>();
+  const mediaEntities = Array.isArray(articleResult.media_entities) ? articleResult.media_entities : [];
+
+  for (const entity of mediaEntities) {
+    if (!isRecord(entity) || typeof entity.media_id !== "string" || !isRecord(entity.media_info)) {
+      continue;
+    }
+
+    const media = resolveArticleMedia(entity.media_info);
+    if (media) {
+      mediaMap.set(entity.media_id, media);
+    }
+  }
+
+  const coverMedia = isRecord(articleResult.cover_media) ? articleResult.cover_media : null;
+  if (coverMedia && typeof coverMedia.media_id === "string" && isRecord(coverMedia.media_info)) {
+    const media = resolveArticleMedia(coverMedia.media_info);
+    if (media) {
+      mediaMap.set(coverMedia.media_id, media);
+    }
+  }
+
+  return mediaMap;
+}
+
+function getMediaMarkdown(
+  entityMap: Map<string, JsonObject>,
+  entityKey: unknown,
+  mediaMap: Map<string, ArticleMedia>,
+): string[] {
+  const key =
+    typeof entityKey === "string" || typeof entityKey === "number"
+      ? String(entityKey)
+      : undefined;
+  if (!key) {
+    return [];
+  }
+
+  const entity = entityMap.get(key);
+  if (!entity || entity.type !== "MEDIA") {
+    return [];
+  }
+
+  const data = isRecord(entity.data) ? entity.data : {};
+  const mediaItems = Array.isArray(data.mediaItems) ? data.mediaItems : [];
+  const media: ArticleMedia[] = [];
+
+  for (const item of mediaItems) {
+    if (!isRecord(item) || typeof item.mediaId !== "string") {
+      continue;
+    }
+    const mediaItem = mediaMap.get(item.mediaId);
+    if (mediaItem && !media.some((value) => value.url === mediaItem.url)) {
+      media.push(mediaItem);
+    }
+  }
+
+  return media.map((item) => item.kind === "image" ? `![](${item.url})` : `[video](${item.url})`);
+}
+
+function resolveTweetMarkdown(payloads: unknown[], tweetId: string, pageUrl: string): string | null {
+  for (const payload of payloads) {
+    const tweet = findTweetNodeById(payload, tweetId);
+    if (!tweet) {
+      continue;
+    }
+
+    const xTweet = toXTweet(tweet, pageUrl);
+    const author = formatTweetAuthor(xTweet) ?? xTweet.url;
+    const lines = [`> ${author}`, ...xTweet.text.split("\n").map((line) => `> ${line}`)];
+
+    const media = formatMediaList(xTweet.media).map((line) =>
+      line.startsWith("photo: ") ? `> ![](${line.slice("photo: ".length)})` : `> - ${line}`,
+    );
+
+    const parts = [lines.join("\n")];
+    if (media.length > 0) {
+      parts.push([">", ...media].join("\n"));
+    }
+    parts.push(`> ${xTweet.url}`);
+
+    return parts.join("\n").trim();
+  }
+
+  return `> Embedded tweet: https://x.com/i/status/${tweetId}`;
+}
+
+function replaceLinkEntities(text: string, block: JsonObject, entityMap: Map<string, JsonObject>): string {
+  const entityRanges = Array.isArray(block.entityRanges) ? block.entityRanges : [];
+  const replacements = entityRanges
+    .filter((range): range is JsonObject => isRecord(range))
+    .map((range) => {
+      const offset = typeof range.offset === "number" ? range.offset : -1;
+      const length = typeof range.length === "number" ? range.length : -1;
+      const url = getLinkUrl(entityMap, range.key);
+      return { offset, length, url };
+    })
+    .filter((range) => range.offset >= 0 && range.length > 0 && range.url)
+    .sort((left, right) => right.offset - left.offset);
+
+  let next = text;
+  for (const replacement of replacements) {
+    next =
+      next.slice(0, replacement.offset) +
+      replacement.url +
+      next.slice(replacement.offset + replacement.length);
+  }
+  return next;
+}
+
+function renderAtomicBlock(
+  block: JsonObject,
+  entityMap: Map<string, JsonObject>,
+  mediaMap: Map<string, ArticleMedia>,
+  payloads: unknown[],
+  pageUrl: string,
+): string | null {
+  const entityRanges = Array.isArray(block.entityRanges) ? block.entityRanges : [];
+  const parts: string[] = [];
+
+  for (const range of entityRanges) {
+    if (!isRecord(range)) {
+      continue;
+    }
+
+    const markdown = getEntityMarkdown(entityMap, range.key);
+    if (markdown) {
+      parts.push(markdown);
+      continue;
+    }
+
+    const mediaMarkdown = getMediaMarkdown(entityMap, range.key, mediaMap);
+    if (mediaMarkdown.length > 0) {
+      parts.push(mediaMarkdown.join("\n\n"));
+      continue;
+    }
+
+    const tweetId = getTweetId(entityMap, range.key);
+    if (tweetId) {
+      const tweetMarkdown = resolveTweetMarkdown(payloads, tweetId, pageUrl);
+      if (tweetMarkdown) {
+        parts.push(tweetMarkdown);
+      }
+    }
+  }
+
+  if (parts.length === 0) {
+    return null;
+  }
+
+  return parts.join("\n\n");
+}
+
+function renderArticleBlocks(
+  blocks: unknown[],
+  entityMap: Map<string, JsonObject>,
+  mediaMap: Map<string, ArticleMedia>,
+  payloads: unknown[],
+  pageUrl: string,
+): string {
+  const parts: string[] = [];
+  let orderedCounter = 0;
+
+  for (const block of blocks) {
+    if (!isRecord(block)) {
+      continue;
+    }
+
+    const blockType = typeof block.type === "string" ? block.type : "unstyled";
+    const rawText = typeof block.text === "string" ? block.text : "";
+    const text = replaceLinkEntities(rawText, block, entityMap).trim();
+    if (!text && blockType !== "atomic") {
+      continue;
+    }
+
+    if (blockType !== "ordered-list-item") {
+      orderedCounter = 0;
+    }
+
+    switch (blockType) {
+      case "header-one":
+        parts.push(`# ${text}`);
+        break;
+      case "header-two":
+        parts.push(`## ${text}`);
+        break;
+      case "header-three":
+        parts.push(`### ${text}`);
+        break;
+      case "blockquote":
+        parts.push(`> ${text}`);
+        break;
+      case "unordered-list-item":
+        parts.push(`- ${text}`);
+        break;
+      case "ordered-list-item":
+        orderedCounter += 1;
+        parts.push(`${orderedCounter}. ${text}`);
+        break;
+      case "code-block":
+        parts.push(`\`\`\`\n${text}\n\`\`\``);
+        break;
+      case "atomic": {
+        const markdown = renderAtomicBlock(block, entityMap, mediaMap, payloads, pageUrl);
+        if (markdown) {
+          parts.push(markdown);
+        }
+        break;
+      }
+      default:
+        parts.push(text);
+        break;
+    }
+  }
+
+  return parts.join("\n\n").trim();
+}
+
+function getArticleResult(tweet: JsonObject): JsonObject | null {
+  if (
+    isRecord(tweet.article) &&
+    isRecord(tweet.article.article_results) &&
+    isRecord(tweet.article.article_results.result)
+  ) {
+    return tweet.article.article_results.result as JsonObject;
+  }
+  return null;
+}
+
+function extractSummary(markdown: string): string | undefined {
+  const segments = markdown
+    .split(/\n\n+/)
+    .map((segment) => segment.trim())
+    .filter(Boolean);
+
+  const preferred = segments.find((segment) => !/^(#|>|- |\d+\. |\`\`\`)/.test(segment));
+  return preferred?.slice(0, 220);
+}
+
+export function extractArticleDocumentFromPayload(
+  payload: unknown,
+  statusId: string,
+  pageUrl: string,
+  payloads: unknown[] = [payload],
+): ExtractedDocument | null {
+  const tweet = findTweetNode(payload, statusId);
+  if (!tweet) {
+    return null;
+  }
+
+  const articleResult = getArticleResult(tweet);
+  if (!articleResult) {
+    return null;
+  }
+
+  const title = typeof articleResult.title === "string" ? articleResult.title.trim() : undefined;
+  const contentState = isRecord(articleResult.content_state) ? articleResult.content_state : {};
+  const blocks = Array.isArray(contentState.blocks) ? contentState.blocks : [];
+  const entityMap = normalizeEntityMap(contentState.entityMap);
+  const mediaMap = buildMediaMap(articleResult);
+  const richMarkdown = renderArticleBlocks(blocks, entityMap, mediaMap, payloads, pageUrl);
+  const plainText = typeof articleResult.plain_text === "string" ? articleResult.plain_text.trim() : "";
+  const markdown = richMarkdown || plainText || getTweetText(tweet);
+  if (!markdown) {
+    return null;
+  }
+
+  const xTweet = toXTweet(tweet, pageUrl);
+  const user = getUser(tweet);
+  const coverMedia = isRecord(articleResult.cover_media) ? articleResult.cover_media : null;
+  const coverMediaInfo = coverMedia && isRecord(coverMedia.media_info) ? coverMedia.media_info : null;
+  const coverImage = coverMediaInfo ? resolveArticleMediaUrl(coverMediaInfo) || undefined : undefined;
+
+  return {
+    url: pageUrl,
+    canonicalUrl: xTweet.url,
+    title: title || normalizeTitle(xTweet.text, "X Article"),
+    author: formatTweetAuthor(xTweet),
+    siteName: "X",
+    publishedAt: xTweet.createdAt,
+    summary: extractSummary(markdown) || xTweet.text.slice(0, 200) || undefined,
+    adapter: "x",
+    metadata: {
+      kind: "x/article",
+      tweetId: xTweet.id,
+      coverImage,
+      authorName: xTweet.authorName ?? user.name,
+      authorUsername: xTweet.author ?? user.screenName,
+      authorUrl: (xTweet.author ?? user.screenName) ? `https://x.com/${xTweet.author ?? user.screenName}` : undefined,
+      ...getTweetAuthorMetadata(xTweet),
+    },
+    content: [{ type: "markdown", markdown }],
+  };
+}

packages/baoyu-fetch/src/adapters/x/index.ts

@@ -0,0 +1,134 @@
+import type { Adapter, AdapterLoginInfo } from "../types";
+import { exportCookies, restoreCookies, type CookieSidecarConfig } from "../../browser/cookie-sidecar";
+import { detectInteractionGate } from "../../browser/interaction-gates";
+import type { ExtractedDocument } from "../../extract/document";
+import { collectMediaFromDocument } from "../../media/markdown-media";
+import { extractArticleDocumentFromPayload } from "./article";
+import { buildNeedsLoginResult, detectXLogin } from "./login";
+import { extractStatusId, isXHost } from "./match";
+import { collectXJsonPayloads, waitForInitialXPayload } from "./payloads";
+import { extractSingleTweetDocumentFromPayload } from "./single";
+import { extractThreadDocumentFromPayloads } from "./thread";
+import { loadFullXThread } from "./thread-loader";
+
+const cookieConfig: CookieSidecarConfig = {
+  urls: ["https://x.com/", "https://twitter.com/"],
+  filename: "x-session-cookies.json",
+  requiredCookieNames: ["auth_token", "ct0"],
+  filterCookie: (c) => {
+    const d = c.domain ?? "";
+    return d.endsWith("x.com") || d.endsWith("twitter.com");
+  },
+};
+
+function extractDocumentFromPayloads(
+  payloads: unknown[],
+  statusId: string,
+  pageUrl: string,
+): ExtractedDocument | null {
+  for (const payload of payloads) {
+    const articleDocument = extractArticleDocumentFromPayload(payload, statusId, pageUrl, payloads);
+    if (articleDocument) {
+      return articleDocument;
+    }
+  }
+
+  const threadDocument = extractThreadDocumentFromPayloads(payloads, statusId, pageUrl);
+  if (threadDocument) {
+    return threadDocument;
+  }
+
+  for (const payload of payloads) {
+    const singleDocument = extractSingleTweetDocumentFromPayload(payload, statusId, pageUrl);
+    if (singleDocument) {
+      return singleDocument;
+    }
+  }
+
+  return null;
+}
+
+async function ensureXLoginState(context: Parameters<Adapter["process"]>[0]): Promise<AdapterLoginInfo> {
+  return detectXLogin(context);
+}
+
+export const xAdapter: Adapter = {
+  name: "x",
+  match(input) {
+    return isXHost(input.url.hostname);
+  },
+  async checkLogin(context) {
+    return detectXLogin(context);
+  },
+  async exportCookies(context, profileDir) {
+    return exportCookies(context.browser.targetSession, cookieConfig, profileDir);
+  },
+  async restoreCookies(context, profileDir) {
+    return restoreCookies(context.browser.targetSession, cookieConfig, profileDir);
+  },
+  async process(context) {
+    const statusId = extractStatusId(context.input.url);
+    if (!statusId) {
+      return {
+        status: "no_document",
+      };
+    }
+
+    context.log.info(`Loading ${context.input.url.toString()} with x adapter`);
+    await context.browser.goto(context.input.url.toString(), context.timeoutMs);
+
+    const interaction = await detectInteractionGate(context.browser);
+    if (interaction) {
+      return {
+        status: "needs_interaction",
+        interaction,
+      };
+    }
+
+    let login = await ensureXLoginState(context);
+    if (login.state === "logged_out") {
+      return buildNeedsLoginResult(login);
+    }
+
+    await waitForInitialXPayload(context);
+    await loadFullXThread(context, statusId);
+
+    const pageUrl = await context.browser.getURL();
+    const postLoadInteraction = await detectInteractionGate(context.browser);
+    if (postLoadInteraction) {
+      return {
+        status: "needs_interaction",
+        interaction: postLoadInteraction,
+        login,
+      };
+    }
+
+    login = await ensureXLoginState(context).catch(() => login);
+    if (login.state === "logged_out") {
+      return buildNeedsLoginResult(login);
+    }
+
+    const payloads = await collectXJsonPayloads(context);
+    if (payloads.length === 0) {
+      return {
+        status: "no_document",
+        login,
+      };
+    }
+
+    const document = extractDocumentFromPayloads(payloads, statusId, pageUrl);
+    if (document) {
+      return {
+        status: "ok",
+        document,
+        media: collectMediaFromDocument(document),
+        login,
+      };
+    }
+
+    return {
+      status: "no_document",
+      login,
+    };
+  },
+};

packages/baoyu-fetch/src/adapters/x/login.ts

@@ -0,0 +1,80 @@
+import type { AdapterContext, AdapterLoginInfo, AdapterProcessResult } from "../types";
+
+interface XLoginSnapshot {
+  currentUrl: string;
+  hasAccountMenu: boolean;
+  hasLoginInputs: boolean;
+  bodyText: string;
+}
+
+export async function detectXLogin(context: AdapterContext): Promise<AdapterLoginInfo> {
+  const snapshot = await context.browser.evaluate<XLoginSnapshot>(`
+    (() => {
+      const bodyText = (document.body?.innerText ?? "").slice(0, 2500);
+      return {
+        currentUrl: window.location.href,
+        hasAccountMenu: Boolean(
+          document.querySelector(
+            '[data-testid="SideNav_AccountSwitcher_Button"], [data-testid="AppTabBar_Profile_Link"], [aria-label="Account menu"]'
+          )
+        ),
+        hasLoginInputs: Boolean(
+          document.querySelector(
+            'input[name="text"], input[name="password"], input[autocomplete="username"], input[autocomplete="current-password"]'
+          )
+        ),
+        bodyText,
+      };
+    })()
+  `).catch(async () => ({
+    currentUrl: await context.browser.getURL().catch(() => context.input.url.toString()),
+    hasAccountMenu: false,
+    hasLoginInputs: false,
+    bodyText: "",
+  }));
+
+  if (
+    /\/i\/flow\/login|\/login/i.test(snapshot.currentUrl) ||
+    snapshot.hasLoginInputs ||
+    /sign in to x|join x today|登录 x|注册 x|登录到 x/i.test(snapshot.bodyText)
+  ) {
+    return {
+      provider: "x",
+      state: "logged_out",
+      required: true,
+      reason: "X login page detected",
+    };
+  }
+
+  if (snapshot.hasAccountMenu) {
+    return {
+      provider: "x",
+      state: "logged_in",
+    };
+  }
+
+  return {
+    provider: "x",
+    state: "unknown",
+  };
+}
+
+export function buildNeedsLoginResult(login: AdapterLoginInfo): AdapterProcessResult {
+  return {
+    status: "needs_interaction",
+    login: {
+      ...login,
+      provider: "x",
+      state: login.state === "logged_in" ? "unknown" : login.state,
+      required: true,
+    },
+    interaction: {
+      type: "wait_for_interaction",
+      kind: "login",
+      provider: "x",
+      reason: login.reason,
+      prompt: "Please sign in to X in the opened Chrome window. Extraction will continue automatically once login is detected.",
+      requiresVisibleBrowser: true,
+    },
+  };
+}

packages/baoyu-fetch/src/adapters/x/match.ts

@@ -0,0 +1,9 @@
+export function isXHost(hostname: string): boolean {
+  return ["x.com", "www.x.com", "twitter.com", "www.twitter.com"].includes(hostname);
+}
+
+export function extractStatusId(url: URL): string | undefined {
+  const match = url.pathname.match(/\/(?:status|article)\/(\d+)/);
+  return match?.[1];
+}
+

packages/baoyu-fetch/src/adapters/x/payloads.ts

@@ -0,0 +1,50 @@
+import type { AdapterContext } from "../types";
+import { filterXGraphQlEntries } from "./shared";
+
+export function getRelevantXThreadEntries(context: AdapterContext) {
+  return filterXGraphQlEntries(context.network.getEntries()).filter(
+    (entry) =>
+      entry.method === "GET" &&
+      entry.finished &&
+      (
+        entry.url.includes("TweetDetail") ||
+        entry.url.includes("TweetResultByRestId") ||
+        entry.url.includes("TweetResultsByRestIds")
+      ),
+  );
+}
+
+export async function prefetchRelevantXThreadBodies(context: AdapterContext): Promise<void> {
+  const entries = getRelevantXThreadEntries(context).filter((entry) => entry.body === undefined && !entry.bodyError);
+  for (const entry of entries) {
+    await context.network.ensureBody(entry);
+  }
+}
+
+export async function collectXJsonPayloads(context: AdapterContext): Promise<unknown[]> {
+  await prefetchRelevantXThreadBodies(context);
+  const entries = getRelevantXThreadEntries(context);
+
+  const payloads: unknown[] = [];
+  for (const entry of entries) {
+    const payload = await context.network.getJsonBody(entry);
+    if (payload) {
+      payloads.push(payload);
+    }
+  }
+  return payloads;
+}
+
+export async function waitForInitialXPayload(context: AdapterContext): Promise<void> {
+  try {
+    await context.network.waitForResponse(
+      (entry) =>
+        entry.url.includes("/graphql/") &&
+        (entry.url.includes("TweetDetail") || entry.url.includes("TweetResultByRestId")),
+      { timeoutMs: Math.min(context.timeoutMs, 15_000) },
+    );
+    await prefetchRelevantXThreadBodies(context);
+  } catch {
+    context.log.debug("No tweet GraphQL response observed before timeout.");
+  }
+}

packages/baoyu-fetch/src/adapters/x/session.ts

@@ -0,0 +1,47 @@
+import type { AdapterContext } from "../types";
+
+const X_SESSION_URLS = ["https://x.com/", "https://twitter.com/"] as const;
+const REQUIRED_X_SESSION_COOKIES = ["auth_token", "ct0"] as const;
+
+interface CookieLike {
+  name?: string;
+  value?: string | null;
+}
+
+interface NetworkGetCookiesResult {
+  cookies?: CookieLike[];
+}
+
+export function buildXSessionCookieMap(cookies: readonly CookieLike[]): Record<string, string> {
+  const cookieMap: Record<string, string> = {};
+  for (const cookie of cookies) {
+    const name = cookie.name?.trim();
+    const value = cookie.value?.trim();
+    if (!name || !value) {
+      continue;
+    }
+    cookieMap[name] = value;
+  }
+  return cookieMap;
+}
+
+export function hasRequiredXSessionCookies(cookieMap: Record<string, string>): boolean {
+  return REQUIRED_X_SESSION_COOKIES.every((name) => Boolean(cookieMap[name]));
+}
+
+export async function readXSessionCookieMap(
+  context: Pick<AdapterContext, "browser">,
+): Promise<Record<string, string>> {
+  const { cookies } = await context.browser.targetSession.send<NetworkGetCookiesResult>(
+    "Network.getCookies",
+    { urls: [...X_SESSION_URLS] },
+  );
+  return buildXSessionCookieMap(cookies ?? []);
+}
+
+export async function isXSessionReady(
+  context: Pick<AdapterContext, "browser">,
+): Promise<boolean> {
+  const cookieMap = await readXSessionCookieMap(context);
+  return hasRequiredXSessionCookies(cookieMap);
+}

packages/baoyu-fetch/src/adapters/x/shared.ts

@@ -0,0 +1,423 @@
+import path from "node:path";
+import type { NetworkEntry } from "../../browser/network-journal";
+import type { XMedia, XQuotedTweet, XTweet, XUser, JsonObject } from "./types";
+
+const X_IMAGE_EXTENSIONS = new Set(["jpg", "jpeg", "png", "webp", "gif", "bmp", "avif"]);
+
+function emptyObject(): JsonObject {
+  return {};
+}
+
+export function isRecord(value: unknown): value is JsonObject {
+  return Boolean(value) && typeof value === "object" && !Array.isArray(value);
+}
+
+export function walk(value: unknown, visitor: (node: unknown) => boolean | void): boolean {
+  if (visitor(value)) {
+    return true;
+  }
+
+  if (Array.isArray(value)) {
+    for (const item of value) {
+      if (walk(item, visitor)) {
+        return true;
+      }
+    }
+    return false;
+  }
+
+  if (isRecord(value)) {
+    for (const child of Object.values(value)) {
+      if (walk(child, visitor)) {
+        return true;
+      }
+    }
+  }
+
+  return false;
+}
+
+function hasTweetText(node: JsonObject): boolean {
+  const legacy = isRecord(node.legacy) ? node.legacy : emptyObject();
+  return (
+    typeof legacy.full_text === "string" ||
+    typeof getNoteTweetText(node) === "string"
+  );
+}
+
+export function findTweetNodeById(payload: unknown, tweetId: string): JsonObject | null {
+  let match: JsonObject | null = null;
+
+  walk(payload, (node) => {
+    if (!isRecord(node) || typeof node.rest_id !== "string" || !isRecord(node.legacy)) {
+      return false;
+    }
+
+    if (!hasTweetText(node)) {
+      return false;
+    }
+
+    if (node.rest_id === tweetId) {
+      match = node;
+      return true;
+    }
+
+    return false;
+  });
+
+  return match;
+}
+
+export function findTweetNode(payload: unknown, statusId: string): JsonObject | null {
+  let firstMatch: JsonObject | null = null;
+  const exactMatch = findTweetNodeById(payload, statusId);
+  if (exactMatch) {
+    return exactMatch;
+  }
+
+  walk(payload, (node) => {
+    if (!isRecord(node) || typeof node.rest_id !== "string" || !isRecord(node.legacy)) {
+      return false;
+    }
+    if (!hasTweetText(node)) {
+      return false;
+    }
+    if (!firstMatch) {
+      firstMatch = node;
+    }
+    return false;
+  });
+
+  return firstMatch;
+}
+
+export function getLegacy(tweet: JsonObject): JsonObject {
+  return isRecord(tweet.legacy) ? tweet.legacy : emptyObject();
+}
+
+export function unwrapTweetResult(node: unknown): JsonObject | null {
+  if (!isRecord(node)) {
+    return null;
+  }
+
+  if (node.__typename === "TweetWithVisibilityResults" && isRecord(node.tweet)) {
+    return unwrapTweetResult(node.tweet);
+  }
+
+  const tweet = isRecord(node.tweet) ? (node.tweet as JsonObject) : node;
+  if (typeof tweet.rest_id !== "string" || !isRecord(tweet.legacy)) {
+    return null;
+  }
+
+  return tweet;
+}
+
+export function getUser(tweet: JsonObject): XUser {
+  const result =
+    isRecord(tweet.core) &&
+    isRecord(tweet.core.user_results) &&
+    isRecord(tweet.core.user_results.result)
+      ? (tweet.core.user_results.result as JsonObject)
+      : emptyObject();
+  const legacy = isRecord(result.legacy) ? result.legacy : emptyObject();
+  const core = isRecord(result.core) ? result.core : emptyObject();
+  return {
+    name:
+      (typeof legacy.name === "string" ? legacy.name : undefined) ??
+      (typeof core.name === "string" ? core.name : undefined),
+    screenName:
+      (typeof legacy.screen_name === "string" ? legacy.screen_name : undefined) ??
+      (typeof core.screen_name === "string" ? core.screen_name : undefined),
+  };
+}
+
+function getNoteTweetResult(tweet: JsonObject): JsonObject | null {
+  if (
+    !isRecord(tweet.note_tweet) ||
+    !isRecord(tweet.note_tweet.note_tweet_results) ||
+    !isRecord(tweet.note_tweet.note_tweet_results.result)
+  ) {
+    return null;
+  }
+
+  return tweet.note_tweet.note_tweet_results.result as JsonObject;
+}
+
+function getNoteTweetText(tweet: JsonObject): string | undefined {
+  const noteTweet = getNoteTweetResult(tweet);
+  return typeof noteTweet?.text === "string" ? noteTweet.text : undefined;
+}
+
+interface TweetUrlEntity {
+  url: string;
+  expandedUrl?: string;
+  displayUrl?: string;
+}
+
+function collectTweetUrlEntities(values: unknown[]): TweetUrlEntity[] {
+  return values.reduce<TweetUrlEntity[]>((entities, value) => {
+      if (!isRecord(value) || typeof value.url !== "string" || !value.url) {
+        return entities;
+      }
+
+      entities.push({
+        url: value.url,
+        expandedUrl: typeof value.expanded_url === "string" ? value.expanded_url : undefined,
+        displayUrl: typeof value.display_url === "string" ? value.display_url : undefined,
+      });
+
+      return entities;
+    }, []);
+}
+
+function getTweetUrlEntities(tweet: JsonObject): TweetUrlEntity[] {
+  const noteTweet = getNoteTweetResult(tweet);
+  const noteTweetEntitySet = noteTweet && isRecord(noteTweet.entity_set) ? noteTweet.entity_set : emptyObject();
+  const noteTweetUrls = collectTweetUrlEntities(Array.isArray(noteTweetEntitySet.urls) ? noteTweetEntitySet.urls : []);
+
+  const legacy = getLegacy(tweet);
+  const legacyEntities = isRecord(legacy.entities) ? legacy.entities : emptyObject();
+  const legacyUrls = collectTweetUrlEntities(Array.isArray(legacyEntities.urls) ? legacyEntities.urls : []);
+
+  const seen = new Set<string>();
+  return [...noteTweetUrls, ...legacyUrls].filter((value) => {
+    if (seen.has(value.url)) {
+      return false;
+    }
+    seen.add(value.url);
+    return true;
+  });
+}
+
+export function getTweetText(tweet: JsonObject): string {
+  const legacy = getLegacy(tweet);
+  let text =
+    getNoteTweetText(tweet) ?? (typeof legacy.full_text === "string" ? legacy.full_text : "");
+
+  for (const value of getTweetUrlEntities(tweet)) {
+    const replacement =
+      (typeof value.expandedUrl === "string" && value.expandedUrl) ||
+      (typeof value.displayUrl === "string" && value.displayUrl) ||
+      value.url;
+    text = text.replaceAll(value.url, replacement);
+  }
+
+  const extendedEntities = isRecord(legacy.extended_entities) ? legacy.extended_entities : emptyObject();
+  const media = Array.isArray(extendedEntities.media) ? extendedEntities.media : [];
+  for (const value of media) {
+    if (isRecord(value) && typeof value.url === "string") {
+      text = text.replaceAll(value.url, "").trim();
+    }
+  }
+
+  return text.replace(/\n{3,}/g, "\n\n").trim();
+}
+
+function normalizeXImageExtension(raw: string | undefined | null): string | undefined {
+  if (!raw) {
+    return undefined;
+  }
+
+  const normalized = raw.replace(/^\./, "").trim().toLowerCase();
+  if (!normalized) {
+    return undefined;
+  }
+
+  return normalized === "jpeg" ? "jpg" : normalized;
+}
+
+export function toHighResXImageUrl(rawUrl: string): string {
+  try {
+    const parsed = new URL(rawUrl);
+    if (parsed.hostname.toLowerCase() !== "pbs.twimg.com") {
+      return rawUrl;
+    }
+
+    const pathExtension = normalizeXImageExtension(path.posix.extname(parsed.pathname));
+    const format = normalizeXImageExtension(parsed.searchParams.get("format")) ?? pathExtension;
+    if (!format || !X_IMAGE_EXTENSIONS.has(format)) {
+      return rawUrl;
+    }
+
+    if (pathExtension) {
+      parsed.pathname = parsed.pathname.replace(new RegExp(`\\.${pathExtension}$`, "i"), "");
+    }
+
+    parsed.searchParams.set("format", format);
+    parsed.searchParams.set("name", "4096x4096");
+    return parsed.toString();
+  } catch {
+    return rawUrl;
+  }
+}
+
+function getVideoVariantBitrate(variant: JsonObject): number {
+  const value = variant.bitrate ?? variant.bit_rate;
+  return typeof value === "number" && Number.isFinite(value) ? value : 0;
+}
+
+function getVideoVariantContentType(variant: JsonObject): string {
+  const value = variant.content_type ?? variant.contentType;
+  return typeof value === "string" ? value.toLowerCase() : "";
+}
+
+export function resolveBestXVideoVariantUrl(mediaInfo: unknown): string | undefined {
+  if (!isRecord(mediaInfo)) {
+    return undefined;
+  }
+
+  const variantsSource =
+    Array.isArray(mediaInfo.variants)
+      ? mediaInfo.variants
+      : isRecord(mediaInfo.video_info) && Array.isArray(mediaInfo.video_info.variants)
+        ? mediaInfo.video_info.variants
+        : [];
+
+  const variants = variantsSource
+    .filter(
+      (variant): variant is JsonObject =>
+        isRecord(variant) && typeof variant.url === "string" && variant.url.length > 0,
+    )
+    .filter((variant) => getVideoVariantContentType(variant) === "video/mp4")
+    .sort((left, right) => getVideoVariantBitrate(right) - getVideoVariantBitrate(left));
+
+  return typeof variants[0]?.url === "string" ? variants[0].url : undefined;
+}
+
+export function getTweetMedia(tweet: JsonObject): XMedia[] {
+  const legacy = getLegacy(tweet);
+  const extendedEntities = isRecord(legacy.extended_entities) ? legacy.extended_entities : emptyObject();
+  const media = Array.isArray(extendedEntities.media) ? extendedEntities.media : [];
+
+  return media
+    .map((value) => {
+      if (!isRecord(value) || typeof value.type !== "string") {
+        return null;
+      }
+      if (value.type === "photo" && typeof value.media_url_https === "string") {
+        return {
+          type: value.type,
+          url: toHighResXImageUrl(value.media_url_https),
+          alt: typeof value.ext_alt_text === "string" ? value.ext_alt_text : undefined,
+        };
+      }
+      if (value.type === "video" || value.type === "animated_gif") {
+        const videoUrl = resolveBestXVideoVariantUrl(value);
+        if (!videoUrl) {
+          return null;
+        }
+        return {
+          type: value.type,
+          url: videoUrl,
+        };
+      }
+      return null;
+    })
+    .filter((value): value is XMedia => value !== null);
+}
+
+export function getTweetUrl(tweet: JsonObject, fallbackUrl: string): string {
+  const user = getUser(tweet);
+  const fallbackScreenName = extractScreenNameFromUrl(fallbackUrl);
+  const id = typeof tweet.rest_id === "string" ? tweet.rest_id : "";
+  const screenName = user.screenName ?? fallbackScreenName;
+  if (screenName && id) {
+    return `https://x.com/${screenName}/status/${id}`;
+  }
+  return fallbackUrl;
+}
+
+export function getQuotedTweet(tweet: JsonObject, fallbackUrl: string): XQuotedTweet | undefined {
+  const quoted = unwrapTweetResult(
+    isRecord(tweet.quoted_status_result) ? tweet.quoted_status_result.result : null,
+  );
+  if (!quoted) {
+    return undefined;
+  }
+
+  const user = getUser(quoted);
+  return {
+    id: typeof quoted.rest_id === "string" ? quoted.rest_id : "",
+    author: user.screenName,
+    authorName: user.name,
+    text: getTweetText(quoted),
+    url: getTweetUrl(quoted, fallbackUrl),
+    media: getTweetMedia(quoted),
+  };
+}
+
+export function extractScreenNameFromUrl(url: string): string | undefined {
+  try {
+    const parsed = new URL(url);
+    const match = parsed.pathname.match(/^\/([^/]+)\/(?:status|article)\//);
+    if (!match) {
+      return undefined;
+    }
+    if (match[1] === "i") {
+      return undefined;
+    }
+    return match[1];
+  } catch {
+    return undefined;
+  }
+}
+
+export function toXTweet(tweet: JsonObject, fallbackUrl: string): XTweet {
+  const legacy = getLegacy(tweet);
+  const user = getUser(tweet);
+  const fallbackScreenName = extractScreenNameFromUrl(fallbackUrl);
+  const screenName = user.screenName ?? fallbackScreenName;
+  return {
+    id: typeof tweet.rest_id === "string" ? tweet.rest_id : "",
+    author: screenName,
+    authorName: user.name,
+    text: getTweetText(tweet),
+    likes: typeof legacy.favorite_count === "number" ? legacy.favorite_count : 0,
+    retweets: typeof legacy.retweet_count === "number" ? legacy.retweet_count : 0,
+    replies: typeof legacy.reply_count === "number" ? legacy.reply_count : 0,
+    createdAt: typeof legacy.created_at === "string" ? legacy.created_at : undefined,
+    inReplyTo: typeof legacy.in_reply_to_status_id_str === "string" ? legacy.in_reply_to_status_id_str : undefined,
+    url: getTweetUrl(tweet, fallbackUrl),
+    media: getTweetMedia(tweet),
+    quotedTweet: getQuotedTweet(tweet, fallbackUrl),
+  };
+}
+
+export function normalizeTitle(text: string, fallback: string): string {
+  const firstLine = text.split("\n")[0]?.trim();
+  if (!firstLine) {
+    return fallback;
+  }
+  return firstLine.slice(0, 120);
+}
+
+export function formatTweetAuthor(tweet: XTweet): string | undefined {
+  if (tweet.author && tweet.authorName) {
+    return `${tweet.authorName} (@${tweet.author})`;
+  }
+  if (tweet.author) {
+    return `@${tweet.author}`;
+  }
+  return tweet.authorName;
+}
+
+export function getTweetAuthorMetadata(tweet: XTweet): Record<string, unknown> {
+  return {
+    authorName: tweet.authorName,
+    authorUsername: tweet.author,
+    authorUrl: tweet.author ? `https://x.com/${tweet.author}` : undefined,
+  };
+}
+
+export function formatMediaList(media: XMedia[]): string[] {
+  return media.map((item) => {
+    if (item.type === "photo") {
+      return `photo: ${item.url}`;
+    }
+    return `${item.type}: ${item.url}`;
+  });
+}
+
+export function filterXGraphQlEntries(entries: NetworkEntry[]): NetworkEntry[] {
+  return entries.filter((entry) => entry.url.includes("/graphql/"));
+}

packages/baoyu-fetch/src/adapters/x/single.ts

@@ -0,0 +1,87 @@
+import type { ExtractedDocument, ContentBlock } from "../../extract/document";
+import { findTweetNode, formatMediaList, formatTweetAuthor, getTweetAuthorMetadata, normalizeTitle, toXTweet } from "./shared";
+
+export function extractSingleTweetDocumentFromPayload(
+  payload: unknown,
+  statusId: string,
+  pageUrl: string,
+): ExtractedDocument | null {
+  const tweet = findTweetNode(payload, statusId);
+  if (!tweet) {
+    return null;
+  }
+
+  const xTweet = toXTweet(tweet, pageUrl);
+  const content: ContentBlock[] = [];
+
+  if (xTweet.text) {
+    content.push({ type: "paragraph", text: xTweet.text });
+  }
+
+  for (const mediaLine of formatMediaList(xTweet.media)) {
+    if (mediaLine.startsWith("photo: ")) {
+      content.push({
+        type: "image",
+        url: mediaLine.slice("photo: ".length),
+      });
+    } else {
+      content.push({
+        type: "list",
+        ordered: false,
+        items: [mediaLine],
+      });
+    }
+  }
+
+  if (xTweet.quotedTweet) {
+    const quotedLines: string[] = [];
+    const quotedAuthor =
+      xTweet.quotedTweet.author && xTweet.quotedTweet.authorName
+        ? `${xTweet.quotedTweet.authorName} (@${xTweet.quotedTweet.author})`
+        : xTweet.quotedTweet.author
+          ? `@${xTweet.quotedTweet.author}`
+          : xTweet.quotedTweet.authorName;
+
+    if (quotedAuthor) {
+      quotedLines.push(quotedAuthor);
+    }
+    if (xTweet.quotedTweet.text) {
+      quotedLines.push(xTweet.quotedTweet.text);
+    }
+    quotedLines.push(...formatMediaList(xTweet.quotedTweet.media));
+
+    if (quotedLines.length > 0) {
+      content.push({ type: "heading", depth: 2, text: "Quoted Tweet" });
+      content.push({ type: "quote", text: quotedLines.join("\n\n") });
+    }
+  }
+
+  return {
+    url: pageUrl,
+    canonicalUrl: xTweet.url,
+    title: normalizeTitle(
+      xTweet.author ? `@${xTweet.author}: ${xTweet.text}` : xTweet.text,
+      "Tweet",
+    ),
+    author: formatTweetAuthor(xTweet),
+    siteName: "X",
+    publishedAt: xTweet.createdAt,
+    summary: xTweet.text.slice(0, 200) || undefined,
+    adapter: "x",
+    metadata: {
+      kind: "x/post",
+      tweetId: xTweet.id,
+      ...getTweetAuthorMetadata(xTweet),
+      conversationId:
+        typeof tweet.legacy === "object" &&
+        tweet.legacy !== null &&
+        typeof (tweet.legacy as Record<string, unknown>).conversation_id_str === "string"
+          ? (tweet.legacy as Record<string, unknown>).conversation_id_str
+          : undefined,
+      favoriteCount: xTweet.likes,
+      replyCount: xTweet.replies,
+      retweetCount: xTweet.retweets,
+    },
+    content,
+  };
+}

packages/baoyu-fetch/src/adapters/x/thread-loader.ts

@@ -0,0 +1,286 @@
+import type { AdapterContext } from "../types";
+import { extractThreadTweetsFromPayloads } from "./thread";
+import { collectXJsonPayloads, getRelevantXThreadEntries, prefetchRelevantXThreadBodies } from "./payloads";
+
+interface ClickTextResult {
+  clicked: boolean;
+  text?: string;
+}
+
+interface ScrollStepResult {
+  moved: boolean;
+  atTop: boolean;
+  atBottom: boolean;
+}
+
+interface ThreadProgress {
+  tweetCount: number;
+  firstTweetId?: string;
+  lastTweetId?: string;
+  requestCount: number;
+  tweetDetailCount: number;
+}
+
+interface TopProbeState {
+  requestCount: number;
+  tweetDetailCount: number;
+  scrollHeight: number;
+}
+
+function sleep(ms: number): Promise<void> {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+async function waitForXNetworkSettle(context: AdapterContext, reason: string): Promise<void> {
+  try {
+    await context.network.waitForIdle({
+      idleMs: 650,
+      timeoutMs: Math.min(context.timeoutMs, 5_000),
+    });
+  } catch {
+    context.log.debug(`Network idle timed out after ${reason}.`);
+  }
+}
+
+async function captureTopProbeState(context: AdapterContext): Promise<TopProbeState> {
+  const entries = getRelevantXThreadEntries(context);
+  const scrollHeight = await context.browser.evaluate<number>(`
+    (() => {
+      const scrollRoot = document.scrollingElement ?? document.documentElement ?? document.body;
+      return scrollRoot.scrollHeight;
+    })()
+  `);
+
+  return {
+    requestCount: entries.length,
+    tweetDetailCount: entries.filter((entry) => entry.url.includes("TweetDetail")).length,
+    scrollHeight,
+  };
+}
+
+async function waitForTopProbe(context: AdapterContext): Promise<boolean> {
+  const initial = await captureTopProbeState(context);
+  const deadline = Date.now() + 1_200;
+
+  while (Date.now() < deadline) {
+    try {
+      await context.network.waitForIdle({
+        idleMs: 250,
+        timeoutMs: 350,
+      });
+    } catch {
+      // Keep polling until the shorter top-probe budget expires.
+    }
+
+    await prefetchRelevantXThreadBodies(context);
+    const next = await captureTopProbeState(context);
+    if (
+      next.requestCount > initial.requestCount ||
+      next.tweetDetailCount > initial.tweetDetailCount ||
+      next.scrollHeight > initial.scrollHeight + 4
+    ) {
+      context.log.debug("Observed additional X thread activity while probing the page top.");
+      return true;
+    }
+
+    await sleep(120);
+  }
+
+  return false;
+}
+
+async function scrollThreadToTop(context: AdapterContext): Promise<void> {
+  let settledTopChecks = 0;
+
+  while (settledTopChecks < 2) {
+    const scroll = await context.browser.evaluate<ScrollStepResult>(`
+      (() => {
+        const scrollRoot = document.scrollingElement ?? document.documentElement ?? document.body;
+        const before = window.scrollY;
+        window.scrollTo({ top: 0, left: 0, behavior: "instant" });
+        const after = window.scrollY;
+        return {
+          moved: after !== before,
+          atTop: after <= 4,
+          atBottom: window.innerHeight + after >= scrollRoot.scrollHeight - 4,
+        };
+      })()
+    `);
+    await sleep(140);
+    await waitForXNetworkSettle(context, "scrolling X thread to top");
+    await prefetchRelevantXThreadBodies(context);
+
+    if (scroll.moved) {
+      settledTopChecks = 0;
+      continue;
+    }
+
+    const observedTopActivity = await waitForTopProbe(context);
+    if (observedTopActivity) {
+      settledTopChecks = 0;
+      continue;
+    }
+
+    settledTopChecks += 1;
+  }
+}
+
+async function clickVisibleShowReplies(context: AdapterContext): Promise<ClickTextResult> {
+  return context.browser.evaluate<ClickTextResult>(`
+    (() => {
+      const normalize = (value) => value.replace(/\\s+/g, " ").trim();
+      const matches = [
+        /^Show replies$/i,
+        /^Show more replies$/i,
+        /^Show additional replies$/i,
+        /^显示回复$/,
+        /^展开回复$/,
+      ];
+      const isVisible = (element) => {
+        if (!(element instanceof HTMLElement)) {
+          return false;
+        }
+        const rect = element.getBoundingClientRect();
+        const style = window.getComputedStyle(element);
+        return (
+          rect.width > 0 &&
+          rect.height > 0 &&
+          style.visibility !== "hidden" &&
+          style.display !== "none"
+        );
+      };
+
+      const selectors = [
+        "a",
+        "button",
+        '[role="button"]',
+        '[role="link"]',
+      ];
+
+      for (const element of document.querySelectorAll(selectors.join(","))) {
+        if (!isVisible(element)) {
+          continue;
+        }
+        const text = normalize(element.textContent ?? "");
+        if (!text || !matches.some((pattern) => pattern.test(text))) {
+          continue;
+        }
+        element.scrollIntoView({ block: "center", inline: "nearest" });
+        if (element instanceof HTMLElement) {
+          element.click();
+          return { clicked: true, text };
+        }
+      }
+
+      return { clicked: false };
+    })()
+  `);
+}
+
+async function expandVisibleShowReplies(context: AdapterContext): Promise<number> {
+  let clickCount = 0;
+
+  while (clickCount < 8) {
+    const result = await clickVisibleShowReplies(context).catch<ClickTextResult>(() => ({ clicked: false }));
+    if (!result.clicked) {
+      break;
+    }
+
+    clickCount += 1;
+    context.log.debug(`Expanded X thread replies via "${result.text ?? "Show replies"}".`);
+    await sleep(250);
+    await waitForXNetworkSettle(context, "expanding Show replies");
+    await prefetchRelevantXThreadBodies(context);
+  }
+
+  return clickCount;
+}
+
+async function scrollThreadBy(context: AdapterContext, stepPx: number): Promise<ScrollStepResult> {
+  const result = await context.browser.evaluate<ScrollStepResult>(`
+    (() => {
+      const scrollRoot = document.scrollingElement ?? document.documentElement ?? document.body;
+      const before = window.scrollY;
+      window.scrollBy({ top: ${stepPx}, left: 0, behavior: "instant" });
+      const after = window.scrollY;
+      return {
+        moved: after !== before,
+        atTop: after <= 4,
+        atBottom: window.innerHeight + after >= scrollRoot.scrollHeight - 4,
+      };
+    })()
+  `);
+
+  await sleep(140);
+  await waitForXNetworkSettle(context, "scrolling X thread");
+  await prefetchRelevantXThreadBodies(context);
+  return result;
+}
+
+async function captureThreadProgress(context: AdapterContext, statusId: string): Promise<ThreadProgress> {
+  const entries = getRelevantXThreadEntries(context);
+  const payloads = await collectXJsonPayloads(context);
+  const tweets = extractThreadTweetsFromPayloads(payloads, statusId, context.input.url.toString());
+  return {
+    tweetCount: tweets.length,
+    firstTweetId: tweets[0]?.id,
+    lastTweetId: tweets[tweets.length - 1]?.id,
+    requestCount: entries.length,
+    tweetDetailCount: entries.filter((entry) => entry.url.includes("TweetDetail")).length,
+  };
+}
+
+export async function loadFullXThread(context: AdapterContext, statusId: string): Promise<void> {
+  await scrollThreadToTop(context);
+
+  let progress = await captureThreadProgress(context, statusId);
+  let stagnantRounds = 0;
+  let roundsWithoutMovement = 0;
+  let distanceWithoutThreadActivityPx = 0;
+
+  for (let round = 0; ; round += 1) {
+    const stepPx = round < 12 ? 1_200 : 1_600;
+    let expandedCount = await expandVisibleShowReplies(context);
+    const scroll = await scrollThreadBy(context, stepPx);
+    expandedCount += await expandVisibleShowReplies(context);
+    const nextProgress = await captureThreadProgress(context, statusId);
+    const grew =
+      nextProgress.tweetCount > progress.tweetCount ||
+      nextProgress.firstTweetId !== progress.firstTweetId ||
+      nextProgress.lastTweetId !== progress.lastTweetId ||
+      nextProgress.requestCount > progress.requestCount ||
+      nextProgress.tweetDetailCount > progress.tweetDetailCount;
+
+    if (grew) {
+      context.log.debug(
+        `X thread progress: ${nextProgress.tweetCount} tweets (${nextProgress.firstTweetId ?? "unknown"} -> ${nextProgress.lastTweetId ?? "unknown"}), ${nextProgress.requestCount} requests, ${nextProgress.tweetDetailCount} TweetDetail.`,
+      );
+      stagnantRounds = 0;
+      distanceWithoutThreadActivityPx = 0;
+    } else if (expandedCount > 0) {
+      stagnantRounds = 0;
+      distanceWithoutThreadActivityPx = 0;
+    } else {
+      stagnantRounds += 1;
+      distanceWithoutThreadActivityPx += stepPx;
+    }
+
+    roundsWithoutMovement = scroll.moved ? 0 : roundsWithoutMovement + 1;
+    progress = nextProgress;
+
+    if (scroll.atBottom && stagnantRounds >= 6) {
+      context.log.debug("Stopping X thread scroll after reaching page bottom with no further thread progress.");
+      break;
+    }
+
+    if (roundsWithoutMovement >= 2 && stagnantRounds >= 4) {
+      context.log.debug("Stopping X thread scroll after repeated downward scrolls no longer move the page.");
+      break;
+    }
+
+    if (distanceWithoutThreadActivityPx >= 24_000 && stagnantRounds >= 12) {
+      context.log.debug("Stopping X thread scroll after a long stretch with no thread-related progress.");
+      break;
+    }
+  }
+}

packages/baoyu-fetch/src/adapters/x/thread.ts

@@ -0,0 +1,316 @@
+import type { ExtractedDocument } from "../../extract/document";
+import {
+  formatMediaList,
+  formatTweetAuthor,
+  getLegacy,
+  getTweetAuthorMetadata,
+  isRecord,
+  normalizeTitle,
+  toXTweet,
+  unwrapTweetResult,
+} from "./shared";
+import type { JsonObject, XQuotedTweet, XTweet } from "./types";
+
+interface ParsedThreadTweet extends XTweet {
+  userId?: string;
+  conversationId?: string;
+  inReplyToUserId?: string;
+  sortTimestamp: number;
+}
+
+function compareTweetIds(left: string, right: string): number {
+  try {
+    const leftId = BigInt(left);
+    const rightId = BigInt(right);
+    if (leftId === rightId) {
+      return 0;
+    }
+    return leftId < rightId ? -1 : 1;
+  } catch {
+    return left.localeCompare(right);
+  }
+}
+
+function toTimestamp(value: string | undefined): number {
+  if (!value) {
+    return 0;
+  }
+  const parsed = Date.parse(value);
+  return Number.isNaN(parsed) ? 0 : parsed;
+}
+
+function scoreParsedTweet(tweet: ParsedThreadTweet): number {
+  return (
+    (tweet.text ? 4 : 0) +
+    (tweet.author ? 2 : 0) +
+    (tweet.authorName ? 2 : 0) +
+    (tweet.media.length > 0 ? 1 : 0)
+  );
+}
+
+function toParsedThreadTweet(tweet: JsonObject, pageUrl: string): ParsedThreadTweet {
+  const legacy = getLegacy(tweet);
+  const xTweet = toXTweet(tweet, pageUrl);
+
+  return {
+    ...xTweet,
+    userId: typeof legacy.user_id_str === "string" ? legacy.user_id_str : undefined,
+    conversationId: typeof legacy.conversation_id_str === "string" ? legacy.conversation_id_str : undefined,
+    inReplyToUserId: typeof legacy.in_reply_to_user_id_str === "string" ? legacy.in_reply_to_user_id_str : undefined,
+    sortTimestamp: toTimestamp(xTweet.createdAt),
+  };
+}
+
+function collectTweetFromItemContent(
+  itemContent: unknown,
+  pageUrl: string,
+  tweets: Map<string, ParsedThreadTweet>,
+): void {
+  if (!isRecord(itemContent)) {
+    return;
+  }
+
+  const tweet = unwrapTweetResult(
+    isRecord(itemContent.tweet_results) ? itemContent.tweet_results.result : null,
+  );
+  if (!tweet || typeof tweet.rest_id !== "string") {
+    return;
+  }
+
+  const parsed = toParsedThreadTweet(tweet, pageUrl);
+  const existing = tweets.get(parsed.id);
+  if (!existing || scoreParsedTweet(parsed) >= scoreParsedTweet(existing)) {
+    tweets.set(parsed.id, parsed);
+  }
+}
+
+function collectTweetsFromItems(
+  items: unknown,
+  pageUrl: string,
+  tweets: Map<string, ParsedThreadTweet>,
+): void {
+  if (!Array.isArray(items)) {
+    return;
+  }
+
+  for (const item of items) {
+    if (!isRecord(item)) {
+      continue;
+    }
+
+    if (isRecord(item.item) && isRecord(item.item.itemContent)) {
+      collectTweetFromItemContent(item.item.itemContent, pageUrl, tweets);
+      continue;
+    }
+
+    if (isRecord(item.itemContent)) {
+      collectTweetFromItemContent(item.itemContent, pageUrl, tweets);
+    }
+  }
+}
+
+function getInstructions(payload: unknown): unknown[] {
+  if (!isRecord(payload) || !isRecord(payload.data)) {
+    return [];
+  }
+
+  const { data } = payload;
+  return (
+    (isRecord(data.threaded_conversation_with_injections_v2) &&
+    Array.isArray(data.threaded_conversation_with_injections_v2.instructions)
+      ? data.threaded_conversation_with_injections_v2.instructions
+      : undefined) ??
+    (isRecord(data.threaded_conversation_with_injections) &&
+    Array.isArray(data.threaded_conversation_with_injections.instructions)
+      ? data.threaded_conversation_with_injections.instructions
+      : undefined) ??
+    (isRecord(data.tweetResult) &&
+    isRecord(data.tweetResult.result) &&
+    isRecord(data.tweetResult.result.timeline) &&
+    Array.isArray(data.tweetResult.result.timeline.instructions)
+      ? data.tweetResult.result.timeline.instructions
+      : [])
+  );
+}
+
+function parseTweetDetailPayload(payload: unknown, pageUrl: string): ParsedThreadTweet[] {
+  const tweets = new Map<string, ParsedThreadTweet>();
+
+  const instructions = getInstructions(payload);
+  for (const instruction of instructions) {
+    if (!isRecord(instruction)) {
+      continue;
+    }
+
+    collectTweetsFromItems(instruction.moduleItems, pageUrl, tweets);
+
+    if (!Array.isArray(instruction.entries)) {
+      continue;
+    }
+
+    for (const entry of instruction.entries) {
+      if (!isRecord(entry)) {
+        continue;
+      }
+
+      const content = isRecord(entry.content) ? entry.content : {};
+      collectTweetFromItemContent(content.itemContent, pageUrl, tweets);
+      collectTweetsFromItems(content.items, pageUrl, tweets);
+    }
+  }
+
+  return Array.from(tweets.values());
+}
+
+function buildContinuousThread(tweets: ParsedThreadTweet[], statusId: string): ParsedThreadTweet[] {
+  const byId = new Map<string, ParsedThreadTweet>();
+  for (const tweet of tweets) {
+    const existing = byId.get(tweet.id);
+    if (!existing || scoreParsedTweet(tweet) >= scoreParsedTweet(existing)) {
+      byId.set(tweet.id, tweet);
+    }
+  }
+
+  const rootTweet = byId.get(statusId);
+  if (!rootTweet?.userId || !rootTweet.conversationId) {
+    return [];
+  }
+
+  const candidates = Array.from(byId.values()).filter(
+    (tweet) =>
+      tweet.id === statusId ||
+      (tweet.userId === rootTweet.userId && tweet.conversationId === rootTweet.conversationId),
+  );
+
+  const repliesByParent = new Map<string, ParsedThreadTweet[]>();
+  for (const tweet of candidates) {
+    if (!tweet.inReplyTo || tweet.id === statusId) {
+      continue;
+    }
+    const bucket = repliesByParent.get(tweet.inReplyTo) ?? [];
+    bucket.push(tweet);
+    bucket.sort((left, right) => {
+      if (left.sortTimestamp !== right.sortTimestamp) {
+        return left.sortTimestamp - right.sortTimestamp;
+      }
+      return compareTweetIds(left.id, right.id);
+    });
+    repliesByParent.set(tweet.inReplyTo, bucket);
+  }
+
+  const ancestorPath: ParsedThreadTweet[] = [rootTweet];
+  const ancestorSeen = new Set<string>([rootTweet.id]);
+  let currentAncestor = rootTweet;
+
+  while (currentAncestor.inReplyTo) {
+    const parent = byId.get(currentAncestor.inReplyTo);
+    if (!parent || ancestorSeen.has(parent.id)) {
+      break;
+    }
+    ancestorPath.unshift(parent);
+    ancestorSeen.add(parent.id);
+    currentAncestor = parent;
+  }
+
+  const chain = ancestorPath.slice();
+  const seen = new Set<string>(chain.map((tweet) => tweet.id));
+  let currentId = rootTweet.id;
+
+  while (true) {
+    const next = (repliesByParent.get(currentId) ?? []).find((tweet) => !seen.has(tweet.id));
+    if (!next) {
+      break;
+    }
+    chain.push(next);
+    seen.add(next.id);
+    currentId = next.id;
+  }
+
+  return chain;
+}
+
+export function extractThreadTweetsFromPayloads(
+  payloads: unknown[],
+  statusId: string,
+  pageUrl: string,
+): XTweet[] {
+  const parsedTweets: ParsedThreadTweet[] = [];
+
+  for (const payload of payloads) {
+    parsedTweets.push(...parseTweetDetailPayload(payload, pageUrl));
+  }
+
+  return buildContinuousThread(parsedTweets, statusId).map(({ sortTimestamp: _sortTimestamp, ...tweet }) => tweet);
+}
+
+function buildQuotedTweetMarkdown(quotedTweet: XQuotedTweet): string {
+  const author = quotedTweet.author ? `@${quotedTweet.author}` : "Unknown";
+  const name = quotedTweet.authorName ? `${quotedTweet.authorName} ` : "";
+  const lines: string[] = [`Quoted Tweet${quotedTweet.author || quotedTweet.authorName ? `: ${name}${author}`.trim() : ""}`];
+
+  if (quotedTweet.text) {
+    lines.push(...quotedTweet.text.split("\n"));
+  }
+
+  for (const mediaLine of formatMediaList(quotedTweet.media)) {
+    lines.push(mediaLine);
+  }
+
+  return lines.map((line) => (line ? `> ${line}` : ">")).join("\n");
+}
+
+function buildThreadMarkdown(tweets: XTweet[]): string {
+  return tweets
+    .map((tweet, index) => {
+      const lines: string[] = [];
+      const author = tweet.author ? `@${tweet.author}` : "Unknown";
+      const name = tweet.authorName ? `${tweet.authorName} ` : "";
+      lines.push(`## ${index + 1}. ${name}${author}`.trim());
+      if (tweet.createdAt) {
+        lines.push(`_Published: ${tweet.createdAt}_`);
+      }
+      lines.push(tweet.text || "(No text)");
+      const mediaLines = formatMediaList(tweet.media);
+      if (mediaLines.length > 0) {
+        lines.push(mediaLines.map((line) => `- ${line}`).join("\n"));
+      }
+      if (tweet.quotedTweet) {
+        lines.push(buildQuotedTweetMarkdown(tweet.quotedTweet));
+      }
+      return lines.join("\n\n");
+    })
+    .join("\n\n");
+}
+
+export function extractThreadDocumentFromPayloads(
+  payloads: unknown[],
+  statusId: string,
+  pageUrl: string,
+): ExtractedDocument | null {
+  const tweets = extractThreadTweetsFromPayloads(payloads, statusId, pageUrl);
+  if (tweets.length <= 1) {
+    return null;
+  }
+
+  const rootTweet = tweets[0];
+  const rootAuthor = formatTweetAuthor(rootTweet);
+
+  return {
+    url: pageUrl,
+    canonicalUrl: rootTweet.url,
+    title: normalizeTitle(rootTweet.text, "X Thread"),
+    author: rootAuthor,
+    siteName: "X",
+    publishedAt: rootTweet.createdAt,
+    summary: rootTweet.text.slice(0, 200) || undefined,
+    adapter: "x",
+    metadata: {
+      kind: "x/thread",
+      tweetId: rootTweet.id,
+      tweetCount: tweets.length,
+      lastTweetId: tweets[tweets.length - 1]?.id,
+      ...getTweetAuthorMetadata(rootTweet),
+    },
+    content: [{ type: "markdown", markdown: buildThreadMarkdown(tweets) }],
+  };
+}

packages/baoyu-fetch/src/adapters/x/types.ts

@@ -0,0 +1,36 @@
+export type JsonObject = Record<string, unknown>;
+
+export interface XUser {
+  name?: string;
+  screenName?: string;
+}
+
+export interface XMedia {
+  type: string;
+  url: string;
+  alt?: string;
+}
+
+export interface XQuotedTweet {
+  id: string;
+  author?: string;
+  authorName?: string;
+  text: string;
+  url: string;
+  media: XMedia[];
+}
+
+export interface XTweet {
+  id: string;
+  author?: string;
+  authorName?: string;
+  text: string;
+  likes: number;
+  retweets: number;
+  replies: number;
+  createdAt?: string;
+  inReplyTo?: string;
+  url: string;
+  media: XMedia[];
+  quotedTweet?: XQuotedTweet;
+}

packages/baoyu-fetch/src/adapters/youtube/index.ts

@@ -0,0 +1,33 @@
+import type { Adapter } from "../types";
+import { collectMediaFromDocument } from "../../media/markdown-media";
+import { extractYouTubeTranscriptDocument } from "./transcript";
+import { isYouTubeHost, parseYouTubeVideoId } from "./utils";
+
+export const youtubeAdapter: Adapter = {
+  name: "youtube",
+  match(input) {
+    return isYouTubeHost(input.url.hostname);
+  },
+  async process(context) {
+    const videoId = parseYouTubeVideoId(context.input.url);
+    if (!videoId) {
+      return {
+        status: "no_document",
+      };
+    }
+
+    context.log.info(`Loading ${context.input.url.toString()} with youtube adapter`);
+    const document = await extractYouTubeTranscriptDocument(context, videoId);
+    if (!document) {
+      return {
+        status: "no_document",
+      };
+    }
+
+    return {
+      status: "ok",
+      document,
+      media: collectMediaFromDocument(document),
+    };
+  },
+};

packages/baoyu-fetch/src/adapters/youtube/transcript.ts

@@ -0,0 +1,392 @@
+import type { ExtractedDocument } from "../../extract/document";
+import { detectInteractionGate } from "../../browser/interaction-gates";
+import {
+  buildYouTubeThumbnailCandidates,
+  parseYouTubeDescriptionChapters,
+  renderYouTubeTranscriptMarkdown,
+  type YouTubeChapter,
+  type YouTubeTranscriptSegment,
+} from "./utils";
+
+interface CaptionInfo {
+  captionUrl: string;
+  language: string;
+  kind: string;
+  available: string[];
+  title?: string;
+  author?: string;
+  authorUrl?: string;
+  channelId?: string;
+  description?: string;
+  publishedAt?: string;
+  viewCount?: number;
+  durationSeconds?: number;
+  keywords: string[];
+  category?: string;
+  isLiveContent?: boolean;
+  coverImages: string[];
+}
+
+function normalizeUrl(url: string | undefined): string | undefined {
+  if (!url) {
+    return undefined;
+  }
+
+  try {
+    const parsed = new URL(url);
+    if (parsed.protocol === "http:") {
+      parsed.protocol = "https:";
+    }
+    return parsed.toString();
+  } catch {
+    return url;
+  }
+}
+
+function buildSummary(description: string | undefined, segments: YouTubeTranscriptSegment[]): string | undefined {
+  const descriptionSummary = description
+    ?.replace(/\r\n/g, "\n")
+    .split("\n")
+    .map((line) => line.trim())
+    .find((line) => line && !/^https?:\/\//i.test(line));
+
+  if (descriptionSummary) {
+    return descriptionSummary.slice(0, 240);
+  }
+
+  const transcriptSummary = segments
+    .slice(0, 8)
+    .map((segment) => segment.text)
+    .join(" ")
+    .slice(0, 240)
+    .trim();
+
+  return transcriptSummary || undefined;
+}
+
+async function canFetchThumbnail(url: string): Promise<boolean> {
+  try {
+    const response = await fetch(url, { method: "HEAD", redirect: "follow" });
+    if (response.ok) {
+      return true;
+    }
+
+    if (response.status === 405) {
+      const fallbackResponse = await fetch(url, {
+        method: "GET",
+        headers: { Range: "bytes=0-0" },
+        redirect: "follow",
+      });
+      return fallbackResponse.ok;
+    }
+  } catch {
+    return false;
+  }
+
+  return false;
+}
+
+async function resolveBestCoverImage(videoId: string, coverImages: string[]): Promise<string | undefined> {
+  const candidates = buildYouTubeThumbnailCandidates(videoId, coverImages);
+
+  for (const candidate of candidates) {
+    if (await canFetchThumbnail(candidate)) {
+      return candidate;
+    }
+  }
+
+  return candidates[0];
+}
+
+export async function extractYouTubeTranscriptDocument(
+  context: Parameters<import("../types").Adapter["process"]>[0],
+  videoId: string,
+): Promise<ExtractedDocument | null> {
+  const videoUrl = `https://www.youtube.com/watch?v=${videoId}`;
+  await context.browser.goto(videoUrl, context.timeoutMs);
+
+  const interaction = await detectInteractionGate(context.browser);
+  if (interaction) {
+    context.log.debug(`Interaction gate detected on YouTube: ${interaction.provider}`);
+    return null;
+  }
+
+  try {
+    await context.network.waitForIdle({
+      idleMs: 1_000,
+      timeoutMs: Math.min(context.timeoutMs, 8_000),
+    });
+  } catch {
+    context.log.debug("Network idle timed out on YouTube load.");
+  }
+
+  const captionInfo = await context.browser.evaluate<CaptionInfo | { error: string }>(`
+    (async () => {
+      function readText(value) {
+        if (!value) return undefined;
+        if (typeof value === 'string') {
+          const text = value.trim();
+          return text || undefined;
+        }
+        if (typeof value.simpleText === 'string') {
+          const text = value.simpleText.trim();
+          return text || undefined;
+        }
+        if (Array.isArray(value.runs)) {
+          const text = value.runs
+            .map((run) => typeof run?.text === 'string' ? run.text : '')
+            .join('')
+            .trim();
+          return text || undefined;
+        }
+        return undefined;
+      }
+
+      function parsePositiveInteger(value) {
+        if (typeof value === 'number' && Number.isFinite(value) && value >= 0) {
+          return Math.floor(value);
+        }
+        if (typeof value !== 'string') {
+          return undefined;
+        }
+        const normalized = value.replace(/[^\\d]/g, '');
+        if (!normalized) {
+          return undefined;
+        }
+        const parsed = Number.parseInt(normalized, 10);
+        return Number.isFinite(parsed) ? parsed : undefined;
+      }
+
+      const apiKey = window.ytcfg?.data_?.INNERTUBE_API_KEY;
+      const playerResponse = window.ytInitialPlayerResponse;
+      const videoDetails = playerResponse?.videoDetails || {};
+      const microformat = playerResponse?.microformat?.playerMicroformatRenderer || {};
+      const title =
+        videoDetails.title ||
+        readText(microformat.title) ||
+        document.title.replace(/ - YouTube$/, '').trim();
+      const author =
+        videoDetails.author ||
+        microformat.ownerChannelName ||
+        document.querySelector('link[itemprop="name"]')?.getAttribute('content') ||
+        undefined;
+      const authorUrl =
+        microformat.ownerProfileUrl ||
+        (typeof videoDetails.channelId === 'string' && videoDetails.channelId
+          ? 'https://www.youtube.com/channel/' + videoDetails.channelId
+          : undefined);
+      const description =
+        readText(microformat.description) ||
+        (typeof videoDetails.shortDescription === 'string' ? videoDetails.shortDescription.trim() : undefined);
+      const keywords = Array.isArray(videoDetails.keywords)
+        ? videoDetails.keywords.filter((keyword) => typeof keyword === 'string' && keyword.trim())
+        : [];
+      const thumbnails = [
+        ...(Array.isArray(videoDetails.thumbnail?.thumbnails) ? videoDetails.thumbnail.thumbnails : []),
+        ...(Array.isArray(microformat.thumbnail?.thumbnails) ? microformat.thumbnail.thumbnails : []),
+      ]
+        .filter((thumbnail) => typeof thumbnail?.url === 'string' && thumbnail.url)
+        .sort((left, right) => ((right?.width || 0) * (right?.height || 0)) - ((left?.width || 0) * (left?.height || 0)))
+        .map((thumbnail) => thumbnail.url);
+
+      if (!apiKey) {
+        return { error: 'INNERTUBE_API_KEY not found on page' };
+      }
+
+      const response = await fetch('/youtubei/v1/player?key=' + apiKey + '&prettyPrint=false', {
+        method: 'POST',
+        credentials: 'include',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({
+          context: { client: { clientName: 'ANDROID', clientVersion: '20.10.38' } },
+          videoId: ${JSON.stringify(videoId)}
+        })
+      });
+
+      if (!response.ok) {
+        return { error: 'InnerTube player API returned HTTP ' + response.status };
+      }
+
+      const data = await response.json();
+      const renderer = data.captions?.playerCaptionsTracklistRenderer;
+      if (!renderer?.captionTracks?.length) {
+        return { error: 'No captions available for this video' };
+      }
+
+      const tracks = renderer.captionTracks;
+      const track = tracks.find((item) => item.kind !== 'asr') || tracks[0];
+
+      return {
+        captionUrl: track.baseUrl,
+        language: track.languageCode,
+        kind: track.kind || 'manual',
+        available: tracks.map((item) => {
+          const languageLabel = readText(item.name) || item.languageCode;
+          return item.kind === 'asr'
+            ? languageLabel + ' [' + item.languageCode + ', auto]'
+            : languageLabel + ' [' + item.languageCode + ']';
+        }),
+        title,
+        author,
+        authorUrl,
+        channelId: typeof videoDetails.channelId === 'string' ? videoDetails.channelId : undefined,
+        description,
+        publishedAt:
+          (typeof microformat.publishDate === 'string' && microformat.publishDate) ||
+          (typeof microformat.uploadDate === 'string' && microformat.uploadDate) ||
+          document.querySelector('meta[itemprop="datePublished"]')?.getAttribute('content') ||
+          undefined,
+        viewCount: parsePositiveInteger(videoDetails.viewCount) ?? parsePositiveInteger(microformat.viewCount),
+        durationSeconds: parsePositiveInteger(videoDetails.lengthSeconds),
+        keywords,
+        category: typeof microformat.category === 'string' ? microformat.category : undefined,
+        isLiveContent: Boolean(videoDetails.isLiveContent || microformat.isLiveContent),
+        coverImages: thumbnails,
+      };
+    })()
+  `);
+
+  if ("error" in captionInfo) {
+    context.log.debug(`YouTube transcript unavailable: ${captionInfo.error}`);
+    return null;
+  }
+
+  const segments = await context.browser.evaluate<YouTubeTranscriptSegment[] | { error: string }>(`
+    (async () => {
+      const response = await fetch(${JSON.stringify(captionInfo.captionUrl)});
+      const xml = await response.text();
+      if (!xml) {
+        return { error: 'Caption XML is empty' };
+      }
+
+      function getAttr(tag, name) {
+        const needle = name + '="';
+        const index = tag.indexOf(needle);
+        if (index === -1) return '';
+        const valueStart = index + needle.length;
+        const valueEnd = tag.indexOf('"', valueStart);
+        if (valueEnd === -1) return '';
+        return tag.substring(valueStart, valueEnd);
+      }
+
+      function decodeEntities(value) {
+        return value
+          .replaceAll('&amp;', '&')
+          .replaceAll('&lt;', '<')
+          .replaceAll('&gt;', '>')
+          .replaceAll('&quot;', '"')
+          .replaceAll('&#39;', "'");
+      }
+
+      const marker = xml.includes('<p t="') ? '<p ' : '<text ';
+      const endMarker = marker === '<p ' ? '</p>' : '</text>';
+      const results = [];
+      let position = 0;
+
+      while (true) {
+        const tagStart = xml.indexOf(marker, position);
+        if (tagStart === -1) break;
+        let contentStart = xml.indexOf('>', tagStart);
+        if (contentStart === -1) break;
+        contentStart += 1;
+        const tagEnd = xml.indexOf(endMarker, contentStart);
+        if (tagEnd === -1) break;
+
+        const attrString = xml.substring(tagStart + marker.length, contentStart - 1);
+        const content = xml.substring(contentStart, tagEnd);
+        const start = marker === '<p '
+          ? (parseFloat(getAttr(attrString, 't')) || 0) / 1000
+          : (parseFloat(getAttr(attrString, 'start')) || 0);
+        const duration = marker === '<p '
+          ? (parseFloat(getAttr(attrString, 'd')) || 0) / 1000
+          : (parseFloat(getAttr(attrString, 'dur')) || 0);
+        const text = decodeEntities(content.replace(/<[^>]+>/g, '')).split('\\n').join(' ').trim();
+        if (text) {
+          results.push({ start, end: start + duration, text });
+        }
+
+        position = tagEnd + endMarker.length;
+      }
+
+      if (results.length === 0) {
+        return { error: 'Parsed 0 transcript segments' };
+      }
+      return results;
+    })()
+  `);
+
+  if (!Array.isArray(segments) || segments.length === 0) {
+    context.log.debug("Parsed no YouTube transcript segments.");
+    return null;
+  }
+
+  const extractedChapters = await context.browser.evaluate<YouTubeChapter[]>(`
+    (() => {
+      const data = window.ytInitialData;
+      const markers = data?.playerOverlays?.playerOverlayRenderer
+        ?.decoratedPlayerBarRenderer?.decoratedPlayerBarRenderer
+        ?.playerBar?.multiMarkersPlayerBarRenderer?.markersMap || [];
+      const results = [];
+
+      for (const marker of markers) {
+        const chapters = marker?.value?.chapters;
+        if (!Array.isArray(chapters)) continue;
+        for (const chapter of chapters) {
+          const renderer = chapter?.chapterRenderer;
+          const title = renderer?.title?.simpleText;
+          const timeRangeStartMillis = renderer?.timeRangeStartMillis;
+          if (title && typeof timeRangeStartMillis === 'number') {
+            results.push({ title, time: Math.floor(timeRangeStartMillis / 1000) });
+          }
+        }
+      }
+
+      return results;
+    })()
+  `).catch(() => []);
+
+  const descriptionChapters = parseYouTubeDescriptionChapters(captionInfo.description);
+  const chapters = extractedChapters.length > 0 ? extractedChapters : descriptionChapters;
+  const markdown = renderYouTubeTranscriptMarkdown({
+    description: captionInfo.description,
+    segments,
+    chapters,
+  });
+
+  if (!markdown) {
+    return null;
+  }
+
+  const pageUrl = await context.browser.getURL();
+  const coverImage = await resolveBestCoverImage(videoId, captionInfo.coverImages);
+  const summary = buildSummary(captionInfo.description, segments);
+
+  return {
+    url: pageUrl,
+    canonicalUrl: pageUrl,
+    title: captionInfo.title || "YouTube Transcript",
+    author: captionInfo.author,
+    publishedAt: captionInfo.publishedAt,
+    siteName: "YouTube",
+    summary,
+    adapter: "youtube",
+    metadata: {
+      kind: "youtube/transcript",
+      videoId,
+      authorUrl: normalizeUrl(captionInfo.authorUrl),
+      channelId: captionInfo.channelId,
+      coverImage,
+      description: captionInfo.description,
+      durationSeconds: captionInfo.durationSeconds,
+      language: captionInfo.language,
+      captionKind: captionInfo.kind,
+      availableLanguages: captionInfo.available,
+      viewCount: captionInfo.viewCount,
+      keywords: captionInfo.keywords,
+      category: captionInfo.category,
+      isLiveContent: captionInfo.isLiveContent,
+      chapterCount: chapters.length,
+    },
+    content: [{ type: "markdown", markdown }],
+  };
+}

packages/baoyu-fetch/src/adapters/youtube/utils.ts

@@ -0,0 +1,253 @@
+export interface YouTubeTranscriptSegment {
+  start: number;
+  end: number;
+  text: string;
+}
+
+export interface YouTubeChapter {
+  title: string;
+  time: number;
+}
+
+interface RenderYouTubeTranscriptMarkdownInput {
+  description?: string;
+  segments: YouTubeTranscriptSegment[];
+  chapters: YouTubeChapter[];
+}
+
+const DESCRIPTION_CHAPTER_RE = /^((?:\d{1,2}:)?\d{1,2}:\d{2})(?:\s+[-|:]\s+|\s+)(.+)$/;
+const YOUTUBE_THUMBNAIL_VARIANTS = [
+  "maxresdefault.jpg",
+  "sddefault.jpg",
+  "hqdefault.jpg",
+  "mqdefault.jpg",
+  "default.jpg",
+];
+
+export function isYouTubeHost(hostname: string): boolean {
+  return [
+    "youtube.com",
+    "www.youtube.com",
+    "m.youtube.com",
+    "youtu.be",
+  ].includes(hostname);
+}
+
+export function parseYouTubeVideoId(url: URL): string | null {
+  if (url.hostname === "youtu.be") {
+    return url.pathname.split("/").filter(Boolean)[0] ?? null;
+  }
+
+  if (url.pathname === "/watch") {
+    return url.searchParams.get("v");
+  }
+
+  const shortsMatch = url.pathname.match(/^\/shorts\/([^/?#]+)/);
+  if (shortsMatch) {
+    return shortsMatch[1];
+  }
+
+  const liveMatch = url.pathname.match(/^\/live\/([^/?#]+)/);
+  if (liveMatch) {
+    return liveMatch[1];
+  }
+
+  return null;
+}
+
+function parseTimestampValue(raw: string): number | null {
+  const parts = raw
+    .split(":")
+    .map((part) => Number.parseInt(part, 10))
+    .filter((part) => Number.isFinite(part));
+
+  if (parts.length < 2 || parts.length > 3) {
+    return null;
+  }
+
+  if (parts.some((part) => part < 0)) {
+    return null;
+  }
+
+  if (parts.length === 2) {
+    const [minutes, seconds] = parts;
+    return minutes * 60 + seconds;
+  }
+
+  const [hours, minutes, seconds] = parts;
+  return hours * 3600 + minutes * 60 + seconds;
+}
+
+export function formatTimestamp(totalSeconds: number): string {
+  const rounded = Math.max(0, Math.floor(totalSeconds));
+  const hours = Math.floor(rounded / 3600);
+  const minutes = Math.floor((rounded % 3600) / 60);
+  const seconds = rounded % 60;
+
+  if (hours > 0) {
+    return `${hours}:${String(minutes).padStart(2, "0")}:${String(seconds).padStart(2, "0")}`;
+  }
+  return `${minutes}:${String(seconds).padStart(2, "0")}`;
+}
+
+export function formatTimestampRange(start: number, end: number): string {
+  const safeStart = Math.max(0, start);
+  const safeEnd = Math.max(safeStart, end);
+  return `[${formatTimestamp(safeStart)} -> ${formatTimestamp(safeEnd)}]`;
+}
+
+export function normalizeYouTubeChapters(chapters: YouTubeChapter[]): YouTubeChapter[] {
+  const seenTimes = new Set<number>();
+
+  return chapters
+    .map((chapter) => ({
+      title: chapter.title.trim(),
+      time: Math.max(0, Math.floor(chapter.time)),
+    }))
+    .filter((chapter) => chapter.title)
+    .sort((left, right) => left.time - right.time)
+    .filter((chapter) => {
+      if (seenTimes.has(chapter.time)) {
+        return false;
+      }
+      seenTimes.add(chapter.time);
+      return true;
+    });
+}
+
+export function parseYouTubeDescriptionChapters(description?: string | null): YouTubeChapter[] {
+  if (!description) {
+    return [];
+  }
+
+  const chapters: YouTubeChapter[] = [];
+  const seen = new Set<string>();
+
+  for (const rawLine of description.replace(/\r\n/g, "\n").split("\n")) {
+    const line = rawLine.trim();
+    if (!line) {
+      continue;
+    }
+
+    const match = line.match(DESCRIPTION_CHAPTER_RE);
+    if (!match) {
+      continue;
+    }
+
+    const time = parseTimestampValue(match[1]);
+    const title = match[2]?.trim();
+    if (time === null || !title) {
+      continue;
+    }
+
+    const key = `${time}:${title.toLowerCase()}`;
+    if (seen.has(key)) {
+      continue;
+    }
+
+    seen.add(key);
+    chapters.push({ title, time });
+  }
+
+  const normalized = normalizeYouTubeChapters(chapters);
+  if (normalized.length >= 2) {
+    return normalized;
+  }
+
+  if (normalized.length === 1 && normalized[0]?.time === 0) {
+    return normalized;
+  }
+
+  return [];
+}
+
+function renderDescriptionMarkdown(description: string): string {
+  return description
+    .replace(/\r\n/g, "\n")
+    .trim()
+    .split(/\n{2,}/)
+    .map((block) => block.split("\n").map((line) => line.trimEnd()).join("  \n"))
+    .join("\n\n")
+    .trim();
+}
+
+function renderSegmentLine(segment: YouTubeTranscriptSegment): string {
+  return `${formatTimestampRange(segment.start, segment.end)} ${segment.text}`;
+}
+
+export function renderYouTubeTranscriptMarkdown({
+  description,
+  segments,
+  chapters,
+}: RenderYouTubeTranscriptMarkdownInput): string {
+  if (segments.length === 0) {
+    return "";
+  }
+
+  const parts: string[] = [];
+  const normalizedDescription = description?.trim();
+  const transcriptEnd = segments.reduce((maxEnd, segment) => Math.max(maxEnd, segment.end, segment.start), 0);
+  const normalizedChapters = normalizeYouTubeChapters(chapters).filter(
+    (chapter) => transcriptEnd <= 0 || chapter.time < transcriptEnd,
+  );
+
+  if (normalizedDescription) {
+    parts.push("## Description");
+    parts.push(renderDescriptionMarkdown(normalizedDescription));
+  }
+
+  if (normalizedChapters.length > 0) {
+    parts.push("## Chapters");
+
+    for (let index = 0; index < normalizedChapters.length; index += 1) {
+      const chapter = normalizedChapters[index];
+      const nextChapter = normalizedChapters[index + 1];
+      const chapterEnd = nextChapter ? nextChapter.time : transcriptEnd;
+      const chapterSegments = segments.filter(
+        (segment) => segment.start >= chapter.time && segment.start < chapterEnd,
+      );
+
+      parts.push(`### ${chapter.title} ${formatTimestampRange(chapter.time, chapterEnd)}`);
+
+      if (chapterSegments.length > 0) {
+        parts.push(chapterSegments.map(renderSegmentLine).join("\n"));
+      }
+    }
+  } else {
+    parts.push("## Transcript");
+    parts.push(segments.map(renderSegmentLine).join("\n"));
+  }
+
+  return parts.filter(Boolean).join("\n\n").trim();
+}
+
+function normalizeThumbnailKey(url: string): string {
+  try {
+    const parsed = new URL(url);
+    return `${parsed.origin}${parsed.pathname}`;
+  } catch {
+    return url;
+  }
+}
+
+export function buildYouTubeThumbnailCandidates(videoId: string, listedUrls: string[]): string[] {
+  const candidates = [
+    ...YOUTUBE_THUMBNAIL_VARIANTS.map((variant) => `https://i.ytimg.com/vi/${videoId}/${variant}`),
+    ...listedUrls,
+  ];
+
+  const seen = new Set<string>();
+  return candidates.filter((candidate) => {
+    if (!candidate) {
+      return false;
+    }
+
+    const key = normalizeThumbnailKey(candidate);
+    if (seen.has(key)) {
+      return false;
+    }
+
+    seen.add(key);
+    return true;
+  });
+}

packages/baoyu-fetch/src/browser/cdp-client.ts

@@ -0,0 +1,258 @@
+import { EventEmitter } from "node:events";
+import WebSocket from "ws";
+
+type JsonObject = Record<string, unknown>;
+
+interface CdpPendingCommand {
+  resolve(value: unknown): void;
+  reject(error: unknown): void;
+  method: string;
+}
+
+interface CdpErrorShape {
+  message?: string;
+}
+
+interface CdpCommandResult<T> {
+  result?: T;
+  error?: CdpErrorShape;
+}
+
+interface CreatePageSessionOptions {
+  initialUrl?: string;
+  visible?: boolean;
+}
+
+export class TargetSession extends EventEmitter {
+  constructor(
+    private readonly client: CdpClient,
+    public readonly targetId: string,
+    public readonly sessionId: string,
+  ) {
+    super();
+  }
+
+  async send<T>(method: string, params: JsonObject = {}): Promise<T> {
+    return this.client.sendSessionCommand<T>(this.sessionId, method, params);
+  }
+
+  handleEvent(method: string, params: JsonObject): void {
+    this.emit(method, params);
+    this.emit("event", { method, params });
+  }
+
+  async waitForEvent<T extends JsonObject>(
+    method: string,
+    predicate?: (params: T) => boolean,
+    timeoutMs = 30_000,
+  ): Promise<T> {
+    return new Promise<T>((resolve, reject) => {
+      const timeout = setTimeout(() => {
+        this.off(method, listener);
+        reject(new Error(`Timed out waiting for ${method}`));
+      }, timeoutMs);
+
+      const listener = (params: T): void => {
+        if (predicate && !predicate(params)) {
+          return;
+        }
+        clearTimeout(timeout);
+        this.off(method, listener);
+        resolve(params);
+      };
+
+      this.on(method, listener);
+    });
+  }
+}
+
+export class CdpClient {
+  private readonly ws: WebSocket;
+  private readonly pending = new Map<number, CdpPendingCommand>();
+  private readonly sessions = new Map<string, TargetSession>();
+  private nextId = 1;
+
+  private constructor(ws: WebSocket) {
+    this.ws = ws;
+    this.ws.on("message", (raw) => {
+      this.handleMessage(raw.toString());
+    });
+  }
+
+  static async connect(browserWsUrl: string): Promise<CdpClient> {
+    const ws = await new Promise<WebSocket>((resolve, reject) => {
+      const socket = new WebSocket(browserWsUrl);
+      socket.once("open", () => resolve(socket));
+      socket.once("error", (error) => reject(error));
+    });
+
+    return new CdpClient(ws);
+  }
+
+  private handleMessage(rawMessage: string): void {
+    const message = JSON.parse(rawMessage) as {
+      id?: number;
+      sessionId?: string;
+      method?: string;
+      params?: JsonObject;
+      result?: unknown;
+      error?: CdpErrorShape;
+    };
+
+    if (typeof message.id === "number") {
+      const pending = this.pending.get(message.id);
+      if (!pending) {
+        return;
+      }
+      this.pending.delete(message.id);
+      if (message.error) {
+        pending.reject(new Error(`${pending.method}: ${message.error.message ?? "Unknown CDP error"}`));
+        return;
+      }
+      pending.resolve(message.result);
+      return;
+    }
+
+    if (typeof message.sessionId === "string" && typeof message.method === "string") {
+      const session = this.sessions.get(message.sessionId);
+      if (session) {
+        session.handleEvent(message.method, (message.params ?? {}) as JsonObject);
+      }
+    }
+  }
+
+  private async sendCommand<T>(
+    method: string,
+    params: JsonObject = {},
+    sessionId?: string,
+  ): Promise<T> {
+    const id = this.nextId;
+    this.nextId += 1;
+
+    const payload = sessionId ? { id, method, params, sessionId } : { id, method, params };
+
+    const result = new Promise<T>((resolve, reject) => {
+      this.pending.set(id, {
+        resolve: (value) => resolve(value as T),
+        reject,
+        method,
+      });
+    });
+
+    this.ws.send(JSON.stringify(payload));
+    return result;
+  }
+
+  async sendBrowserCommand<T>(method: string, params: JsonObject = {}): Promise<T> {
+    return this.sendCommand<T>(method, params);
+  }
+
+  async sendSessionCommand<T>(sessionId: string, method: string, params: JsonObject = {}): Promise<T> {
+    return this.sendCommand<T>(method, params, sessionId);
+  }
+
+  private async createPageTarget(initialUrl: string, visible = false): Promise<{ targetId: string }> {
+    const attempts: JsonObject[] = visible
+      ? [
+          {
+            url: initialUrl,
+            newWindow: true,
+            focus: true,
+          },
+          {
+            url: initialUrl,
+            focus: true,
+          },
+          {
+            url: initialUrl,
+          },
+        ]
+      : [
+          {
+            url: initialUrl,
+            hidden: true,
+          },
+          {
+            url: initialUrl,
+            background: true,
+            focus: false,
+          },
+          {
+            url: initialUrl,
+          },
+        ];
+
+    let lastError: unknown;
+
+    for (const params of attempts) {
+      try {
+        return await this.sendBrowserCommand<{ targetId: string }>("Target.createTarget", params);
+      } catch (error) {
+        lastError = error;
+      }
+    }
+
+    throw lastError instanceof Error ? lastError : new Error("Target.createTarget failed");
+  }
+
+  async createPageSession(options: CreatePageSessionOptions = {}): Promise<TargetSession> {
+    const initialUrl = options.initialUrl ?? "about:blank";
+    const created = await this.createPageTarget(initialUrl, Boolean(options.visible));
+    const attached = await this.sendBrowserCommand<{ sessionId: string }>("Target.attachToTarget", {
+      targetId: created.targetId,
+      flatten: true,
+    });
+
+    const session = new TargetSession(this, created.targetId, attached.sessionId);
+    this.sessions.set(attached.sessionId, session);
+
+    if (options.visible) {
+      await this.sendBrowserCommand("Target.activateTarget", {
+        targetId: created.targetId,
+      }).catch(() => {});
+    }
+
+    await session.send("Page.enable");
+    await session.send("Runtime.enable");
+    await session.send("DOM.enable");
+
+    if (options.visible) {
+      await session.send("Page.bringToFront").catch(() => {});
+    }
+
+    return session;
+  }
+
+  async closeTarget(targetId: string): Promise<void> {
+    try {
+      await this.sendBrowserCommand("Target.closeTarget", { targetId });
+    } catch {
+      // Target may already be gone.
+    }
+  }
+
+  async close(): Promise<void> {
+    await new Promise<void>((resolve) => {
+      if (this.ws.readyState === WebSocket.CLOSED) {
+        resolve();
+        return;
+      }
+      this.ws.once("close", () => resolve());
+      this.ws.close();
+    });
+  }
+}
+
+export async function evaluateRuntime<T>(session: TargetSession, expression: string): Promise<T> {
+  const response = await session.send<CdpCommandResult<{ value?: T; description?: string }>>("Runtime.evaluate", {
+    expression,
+    awaitPromise: true,
+    returnByValue: true,
+  });
+
+  if (response.error) {
+    throw new Error(response.error.message ?? "Runtime.evaluate failed");
+  }
+
+  return (response.result?.value as T | undefined) ?? (undefined as T);
+}

packages/baoyu-fetch/src/browser/chrome-launcher.ts

@@ -0,0 +1,187 @@
+import { launch, type LaunchedChrome } from "chrome-launcher";
+import WebSocket from "ws";
+import type { Logger } from "../utils/logger";
+import {
+  cleanChromeLockArtifacts,
+  ensureChromeProfileDir,
+  findChromeProcessUsingProfile,
+  findExistingChromeDebugPort,
+  hasChromeLockArtifacts,
+  listChromeProfileEntries,
+  resolveChromeProfileDir,
+  shouldRetryChromeLaunchRecovery,
+} from "./profile";
+
+interface ChromeVersionResponse {
+  webSocketDebuggerUrl: string;
+}
+
+export interface ChromeConnectOptions {
+  cdpUrl?: string;
+  browserPath?: string;
+  headless?: boolean;
+  logger?: Logger;
+  profileDir?: string;
+}
+
+export interface ChromeConnection {
+  browserWsUrl: string;
+  origin?: string;
+  port?: number;
+  profileDir?: string;
+  launched: boolean;
+  close(): Promise<void>;
+}
+
+async function fetchJson<T>(url: string): Promise<T> {
+  const response = await fetch(url);
+  if (!response.ok) {
+    throw new Error(`Failed to fetch ${url}: HTTP ${response.status}`);
+  }
+  return (await response.json()) as T;
+}
+
+async function connectToHttpEndpoint(origin: string): Promise<ChromeConnection> {
+  const normalizedOrigin = origin.replace(/\/$/, "");
+  const version = await fetchJson<ChromeVersionResponse>(`${normalizedOrigin}/json/version`);
+  return {
+    browserWsUrl: version.webSocketDebuggerUrl,
+    origin: normalizedOrigin,
+    port: Number(new URL(normalizedOrigin).port || 80),
+    launched: false,
+    async close() {
+      // Reused external Chrome, nothing to close here.
+    },
+  };
+}
+
+async function tryReuseChrome(profileDir: string, logger?: Logger): Promise<ChromeConnection | null> {
+  const port = await findExistingChromeDebugPort({ profileDir });
+  if (!port) {
+    return null;
+  }
+
+  const origin = `http://127.0.0.1:${port}`;
+  try {
+    const connection = await connectToHttpEndpoint(origin);
+    logger?.info(`Reusing Chrome debugger at ${origin} for profile ${profileDir}`);
+    return {
+      ...connection,
+      profileDir,
+    };
+  } catch {
+    // Debugger disappeared between detection and connect.
+  }
+  return null;
+}
+
+async function launchFreshChrome(
+  profileDir: string,
+  options: Pick<ChromeConnectOptions, "browserPath" | "headless">,
+): Promise<ChromeConnection> {
+  let launchedChrome: LaunchedChrome | null = null;
+  try {
+    launchedChrome = await launch({
+      chromePath: options.browserPath,
+      userDataDir: profileDir,
+      chromeFlags: [
+        "--disable-background-networking",
+        "--disable-default-apps",
+        "--disable-popup-blocking",
+        "--disable-sync",
+        "--no-first-run",
+        "--no-default-browser-check",
+        "--remote-allow-origins=*",
+        ...(!options.headless ? ["--no-startup-window"] : []),
+        ...(options.headless ? ["--headless=new"] : []),
+      ],
+    });
+
+    const origin = `http://127.0.0.1:${launchedChrome.port}`;
+    const version = await fetchJson<ChromeVersionResponse>(`${origin}/json/version`);
+
+    const chrome = launchedChrome;
+    return {
+      browserWsUrl: version.webSocketDebuggerUrl,
+      origin,
+      port: launchedChrome.port,
+      profileDir,
+      launched: true,
+      async close() {
+        if (!chrome) return;
+        await gracefulCloseChrome(chrome, origin);
+      },
+    };
+  } catch (error) {
+    launchedChrome?.kill();
+    throw error;
+  }
+}
+
+async function gracefulCloseChrome(chrome: LaunchedChrome, origin: string): Promise<void> {
+  try {
+    const resp = await fetch(`${origin}/json/version`);
+    const { webSocketDebuggerUrl } = (await resp.json()) as ChromeVersionResponse;
+    if (webSocketDebuggerUrl) {
+      const ws = await new Promise<WebSocket>((resolve, reject) => {
+        const socket = new WebSocket(webSocketDebuggerUrl);
+        socket.once("open", () => resolve(socket));
+        socket.once("error", reject);
+      });
+      const id = 1;
+      ws.send(JSON.stringify({ id, method: "Browser.close" }));
+      await new Promise<void>((resolve) => {
+        const timer = setTimeout(() => { ws.close(); resolve(); }, 5_000);
+        ws.once("close", () => { clearTimeout(timer); resolve(); });
+      });
+      const exited = await new Promise<boolean>((resolve) => {
+        if (chrome.pid && !isProcessAlive(chrome.pid)) { resolve(true); return; }
+        const timer = setTimeout(() => resolve(false), 3_000);
+        chrome.process.once("exit", () => { clearTimeout(timer); resolve(true); });
+      });
+      if (exited) return;
+    }
+  } catch {}
+  chrome.kill();
+}
+
+function isProcessAlive(pid: number): boolean {
+  try { process.kill(pid, 0); return true; } catch { return false; }
+}
+
+export async function connectChrome(options: ChromeConnectOptions): Promise<ChromeConnection> {
+  if (options.cdpUrl) {
+    if (options.cdpUrl.startsWith("ws://") || options.cdpUrl.startsWith("wss://")) {
+      return {
+        browserWsUrl: options.cdpUrl,
+        launched: false,
+        async close() {},
+      };
+    }
+    return connectToHttpEndpoint(options.cdpUrl);
+  }
+
+  const profileDir = ensureChromeProfileDir(resolveChromeProfileDir(options.profileDir));
+  const reused = await tryReuseChrome(profileDir, options.logger);
+  if (reused) {
+    return reused;
+  }
+
+  options.logger?.warn(`No running Chrome debugger found for profile ${profileDir}. Launching Chrome with that profile.`);
+  try {
+    return await launchFreshChrome(profileDir, options);
+  } catch (error) {
+    const entries = await listChromeProfileEntries(profileDir);
+    const shouldRetry = shouldRetryChromeLaunchRecovery({
+      hasLockArtifacts: hasChromeLockArtifacts(entries),
+      hasLiveOwner: findChromeProcessUsingProfile(profileDir),
+    });
+    if (!shouldRetry) {
+      throw error;
+    }
+
+    options.logger?.warn(`Chrome launch failed with stale profile locks. Cleaning ${profileDir} and retrying once.`);
+    cleanChromeLockArtifacts(profileDir);
+    return await launchFreshChrome(profileDir, options);
+  }
+}

packages/baoyu-fetch/src/browser/cookie-sidecar.ts

@@ -0,0 +1,100 @@
+import { readFile, writeFile, mkdir } from "node:fs/promises";
+import { dirname, join } from "node:path";
+import { resolveChromeProfileDir } from "./profile";
+import type { TargetSession } from "./cdp-client";
+
+export interface CdpCookie {
+  name: string;
+  value: string;
+  domain: string;
+  path: string;
+  expires: number;
+  size: number;
+  httpOnly: boolean;
+  secure: boolean;
+  session: boolean;
+  sameSite?: string;
+  priority?: string;
+  sameParty?: boolean;
+  sourceScheme?: string;
+  sourcePort?: number;
+  partitionKey?: string;
+}
+
+interface SidecarData {
+  savedAt: string;
+  cookies: CdpCookie[];
+}
+
+export interface CookieSidecarConfig {
+  urls: readonly string[];
+  filename: string;
+  requiredCookieNames: readonly string[];
+  filterCookie?: (cookie: CdpCookie) => boolean;
+}
+
+function sidecarPath(filename: string, profileDir?: string): string {
+  return join(resolveChromeProfileDir(profileDir), filename);
+}
+
+function hasRequired(cookies: CdpCookie[], names: readonly string[]): boolean {
+  return names.every((name) =>
+    cookies.some((c) => c.name === name && Boolean(c.value)),
+  );
+}
+
+async function getCookies(session: TargetSession, urls: readonly string[]): Promise<CdpCookie[]> {
+  const { cookies } = await session.send<{ cookies: CdpCookie[] }>(
+    "Network.getCookies",
+    { urls: [...urls] },
+  );
+  return cookies ?? [];
+}
+
+export async function exportCookies(
+  session: TargetSession,
+  config: CookieSidecarConfig,
+  profileDir?: string,
+): Promise<boolean> {
+  const all = await getCookies(session, config.urls);
+  const filtered = config.filterCookie ? all.filter(config.filterCookie) : all;
+  if (!hasRequired(filtered, config.requiredCookieNames)) return false;
+
+  const filePath = sidecarPath(config.filename, profileDir);
+  await mkdir(dirname(filePath), { recursive: true });
+  const data: SidecarData = { savedAt: new Date().toISOString(), cookies: filtered };
+  await writeFile(filePath, JSON.stringify(data, null, 2));
+  return true;
+}
+
+export async function restoreCookies(
+  session: TargetSession,
+  config: CookieSidecarConfig,
+  profileDir?: string,
+): Promise<boolean> {
+  const live = await getCookies(session, config.urls);
+  if (hasRequired(live, config.requiredCookieNames)) return false;
+
+  const filePath = sidecarPath(config.filename, profileDir);
+  const raw = await readFile(filePath, "utf8");
+  const data = JSON.parse(raw) as SidecarData;
+  if (!data.cookies?.length) return false;
+
+  const now = Date.now() / 1000;
+  const valid = data.cookies.filter((c) => c.session || !c.expires || c.expires > now);
+  if (!hasRequired(valid, config.requiredCookieNames)) return false;
+
+  await session.send("Network.setCookies", {
+    cookies: valid.map((c) => ({
+      name: c.name,
+      value: c.value,
+      domain: c.domain,
+      path: c.path,
+      httpOnly: c.httpOnly,
+      secure: c.secure,
+      sameSite: c.sameSite,
+      expires: c.expires,
+    })),
+  });
+  return true;
+}

packages/baoyu-fetch/src/browser/interaction-gates.ts

@@ -0,0 +1,123 @@
+import type { WaitForInteractionRequest } from "../adapters/types";
+import type { BrowserSession } from "./session";
+
+interface GateSnapshot {
+  title: string;
+  currentUrl: string;
+  bodyText: string;
+  hasCloudflareTurnstile: boolean;
+  hasCloudflareChallenge: boolean;
+  hasRecaptcha: boolean;
+  hasRecaptchaIframe: boolean;
+  hasHcaptcha: boolean;
+  hasHcaptchaIframe: boolean;
+}
+
+export function detectInteractionGateFromSnapshot(snapshot: GateSnapshot): WaitForInteractionRequest | null {
+  const text = snapshot.bodyText.toLowerCase();
+  const title = snapshot.title.toLowerCase();
+  const url = snapshot.currentUrl.toLowerCase();
+
+  if (
+    snapshot.hasCloudflareTurnstile ||
+    snapshot.hasCloudflareChallenge ||
+    title.includes("just a moment") ||
+    text.includes("verify you are human") ||
+    text.includes("checking your browser before accessing") ||
+    text.includes("enable javascript and cookies to continue") ||
+    url.includes("/cdn-cgi/challenge-platform/")
+  ) {
+    return {
+      type: "wait_for_interaction",
+      kind: "cloudflare",
+      provider: "cloudflare",
+      reason: "Cloudflare human verification detected",
+      prompt: "Please complete the Cloudflare verification in the opened Chrome window. Extraction will continue automatically once the challenge disappears.",
+      requiresVisibleBrowser: true,
+    };
+  }
+
+  if (
+    snapshot.hasRecaptcha ||
+    snapshot.hasRecaptchaIframe ||
+    text.includes("i'm not a robot") ||
+    text.includes("recaptcha")
+  ) {
+    return {
+      type: "wait_for_interaction",
+      kind: "recaptcha",
+      provider: "google_recaptcha",
+      reason: "Google reCAPTCHA detected",
+      prompt: "Please complete the reCAPTCHA verification in the opened Chrome window. Extraction will continue automatically once the challenge disappears.",
+      requiresVisibleBrowser: true,
+    };
+  }
+
+  if (
+    snapshot.hasHcaptcha ||
+    snapshot.hasHcaptchaIframe ||
+    text.includes("hcaptcha")
+  ) {
+    return {
+      type: "wait_for_interaction",
+      kind: "hcaptcha",
+      provider: "hcaptcha",
+      reason: "hCaptcha verification detected",
+      prompt: "Please complete the hCaptcha verification in the opened Chrome window. Extraction will continue automatically once the challenge disappears.",
+      requiresVisibleBrowser: true,
+    };
+  }
+
+  return null;
+}
+
+export async function detectInteractionGate(browser: BrowserSession): Promise<WaitForInteractionRequest | null> {
+  const snapshot = await browser.evaluate<GateSnapshot>(`
+    (() => {
+      const bodyText = (document.body?.innerText ?? "").slice(0, 4000);
+      return {
+        title: document.title ?? "",
+        currentUrl: window.location.href,
+        bodyText,
+        hasCloudflareTurnstile: Boolean(
+          document.querySelector(
+            '.cf-turnstile, [name="cf-turnstile-response"], iframe[src*="challenges.cloudflare.com"]'
+          )
+        ),
+        hasCloudflareChallenge: Boolean(
+          document.querySelector(
+            '#challenge-running, #cf-challenge-running, .challenge-platform, [data-ray], [data-translate="checking_browser"]'
+          )
+        ),
+        hasRecaptcha: Boolean(
+          document.querySelector(
+            '.g-recaptcha, textarea[name="g-recaptcha-response"], iframe[title*="reCAPTCHA"]'
+          )
+        ),
+        hasRecaptchaIframe: Boolean(
+          document.querySelector('iframe[src*="google.com/recaptcha"], iframe[src*="recaptcha/api2"]')
+        ),
+        hasHcaptcha: Boolean(
+          document.querySelector(
+            '.h-captcha, textarea[name="h-captcha-response"], iframe[title*="hCaptcha"]'
+          )
+        ),
+        hasHcaptchaIframe: Boolean(
+          document.querySelector('iframe[src*="hcaptcha.com"]')
+        ),
+      };
+    })()
+  `).catch(() => ({
+    title: "",
+    currentUrl: "",
+    bodyText: "",
+    hasCloudflareTurnstile: false,
+    hasCloudflareChallenge: false,
+    hasRecaptcha: false,
+    hasRecaptchaIframe: false,
+    hasHcaptcha: false,
+    hasHcaptchaIframe: false,
+  }));
+
+  return detectInteractionGateFromSnapshot(snapshot);
+}

packages/baoyu-fetch/src/browser/network-journal.ts

@@ -0,0 +1,235 @@
+import type { TargetSession } from "./cdp-client";
+import type { Logger } from "../utils/logger";
+
+type JsonObject = Record<string, unknown>;
+
+export interface NetworkEntry {
+  requestId: string;
+  url: string;
+  method: string;
+  resourceType: string;
+  timestamp: number;
+  requestHeaders?: Record<string, string>;
+  requestBody?: string;
+  status?: number;
+  statusText?: string;
+  responseHeaders?: Record<string, string>;
+  mimeType?: string;
+  body?: string;
+  bodyBase64?: boolean;
+  bodyError?: string;
+  failed?: boolean;
+  failureReason?: string;
+  finished: boolean;
+}
+
+function normalizeHeaders(headers: unknown): Record<string, string> | undefined {
+  if (!headers || typeof headers !== "object") {
+    return undefined;
+  }
+  return Object.fromEntries(
+    Object.entries(headers as Record<string, unknown>).map(([key, value]) => [key, String(value)]),
+  );
+}
+
+function sleep(ms: number): Promise<void> {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+export class NetworkJournal {
+  private readonly entries = new Map<string, NetworkEntry>();
+  private lastActivityAt = Date.now();
+  private started = false;
+
+  constructor(
+    private readonly session: TargetSession,
+    private readonly log: Logger,
+  ) {}
+
+  async start(): Promise<void> {
+    if (this.started) {
+      return;
+    }
+
+    this.started = true;
+    this.session.on("Network.requestWillBeSent", this.handleRequestWillBeSent);
+    this.session.on("Network.responseReceived", this.handleResponseReceived);
+    this.session.on("Network.loadingFinished", this.handleLoadingFinished);
+    this.session.on("Network.loadingFailed", this.handleLoadingFailed);
+    await this.session.send("Network.enable");
+  }
+
+  stop(): void {
+    if (!this.started) {
+      return;
+    }
+    this.session.off("Network.requestWillBeSent", this.handleRequestWillBeSent);
+    this.session.off("Network.responseReceived", this.handleResponseReceived);
+    this.session.off("Network.loadingFinished", this.handleLoadingFinished);
+    this.session.off("Network.loadingFailed", this.handleLoadingFailed);
+    this.started = false;
+  }
+
+  private touch(): void {
+    this.lastActivityAt = Date.now();
+  }
+
+  private readonly handleRequestWillBeSent = (params: JsonObject): void => {
+    const requestId = typeof params.requestId === "string" ? params.requestId : undefined;
+    const request = params.request as JsonObject | undefined;
+    if (!requestId || !request) {
+      return;
+    }
+
+    this.touch();
+    this.entries.set(requestId, {
+      requestId,
+      url: String(request.url ?? ""),
+      method: String(request.method ?? "GET"),
+      resourceType: String(params.type ?? "Other"),
+      timestamp: Date.now(),
+      requestHeaders: normalizeHeaders(request.headers),
+      requestBody: typeof request.postData === "string" ? request.postData : undefined,
+      finished: false,
+    });
+  };
+
+  private readonly handleResponseReceived = (params: JsonObject): void => {
+    const requestId = typeof params.requestId === "string" ? params.requestId : undefined;
+    const response = params.response as JsonObject | undefined;
+    if (!requestId || !response) {
+      return;
+    }
+
+    this.touch();
+    const existing = this.entries.get(requestId);
+    if (!existing) {
+      return;
+    }
+
+    existing.status = typeof response.status === "number" ? response.status : undefined;
+    existing.statusText = typeof response.statusText === "string" ? response.statusText : undefined;
+    existing.responseHeaders = normalizeHeaders(response.headers);
+    existing.mimeType = typeof response.mimeType === "string" ? response.mimeType : undefined;
+    this.entries.set(requestId, existing);
+  };
+
+  private readonly handleLoadingFinished = (params: JsonObject): void => {
+    const requestId = typeof params.requestId === "string" ? params.requestId : undefined;
+    if (!requestId) {
+      return;
+    }
+
+    this.touch();
+    const existing = this.entries.get(requestId);
+    if (!existing) {
+      return;
+    }
+    existing.finished = true;
+    this.entries.set(requestId, existing);
+  };
+
+  private readonly handleLoadingFailed = (params: JsonObject): void => {
+    const requestId = typeof params.requestId === "string" ? params.requestId : undefined;
+    if (!requestId) {
+      return;
+    }
+
+    this.touch();
+    const existing = this.entries.get(requestId);
+    if (!existing) {
+      return;
+    }
+    existing.finished = true;
+    existing.failed = true;
+    existing.failureReason = typeof params.errorText === "string" ? params.errorText : "Unknown error";
+    this.entries.set(requestId, existing);
+  };
+
+  getEntries(): NetworkEntry[] {
+    return Array.from(this.entries.values());
+  }
+
+  findEntries(predicate: (entry: NetworkEntry) => boolean): NetworkEntry[] {
+    return this.getEntries().filter(predicate);
+  }
+
+  async waitForIdle(options: { idleMs?: number; timeoutMs?: number } = {}): Promise<void> {
+    const idleMs = options.idleMs ?? 1_200;
+    const timeoutMs = options.timeoutMs ?? 15_000;
+    const startedAt = Date.now();
+
+    while (Date.now() - startedAt < timeoutMs) {
+      if (Date.now() - this.lastActivityAt >= idleMs) {
+        return;
+      }
+      await sleep(Math.min(150, idleMs));
+    }
+
+    throw new Error("Timed out waiting for network idle");
+  }
+
+  async waitForResponse(
+    predicate: (entry: NetworkEntry) => boolean,
+    options: { timeoutMs?: number } = {},
+  ): Promise<NetworkEntry> {
+    const timeoutMs = options.timeoutMs ?? 10_000;
+    const startedAt = Date.now();
+
+    while (Date.now() - startedAt < timeoutMs) {
+      const matched = this.getEntries().find((entry) => entry.finished && predicate(entry));
+      if (matched) {
+        return matched;
+      }
+      await sleep(150);
+    }
+
+    throw new Error("Timed out waiting for matching network response");
+  }
+
+  async ensureBody(entry: NetworkEntry): Promise<string | undefined> {
+    if (entry.body !== undefined) {
+      return entry.body;
+    }
+    if (entry.bodyError || entry.failed || !entry.finished) {
+      return undefined;
+    }
+
+    try {
+      const result = await this.session.send<{ body: string; base64Encoded: boolean }>("Network.getResponseBody", {
+        requestId: entry.requestId,
+      });
+      entry.bodyBase64 = result.base64Encoded;
+      entry.body = result.base64Encoded ? Buffer.from(result.body, "base64").toString("utf8") : result.body;
+      return entry.body;
+    } catch (error) {
+      entry.bodyError = error instanceof Error ? error.message : String(error);
+      this.log.debug(`Failed to fetch response body for ${entry.url}: ${entry.bodyError}`);
+      return undefined;
+    }
+  }
+
+  async getJsonBody(entry: NetworkEntry): Promise<unknown | null> {
+    const body = await this.ensureBody(entry);
+    if (!body) {
+      return null;
+    }
+
+    try {
+      return JSON.parse(body);
+    } catch {
+      return null;
+    }
+  }
+
+  async toJSON(options: { includeBodies?: boolean } = {}): Promise<NetworkEntry[]> {
+    const entries = this.getEntries();
+    if (!options.includeBodies) {
+      return entries;
+    }
+
+    await Promise.all(entries.map((entry) => this.ensureBody(entry)));
+    return entries;
+  }
+}
+

packages/baoyu-fetch/src/browser/page-snapshot.ts

@@ -0,0 +1,105 @@
+import type { BrowserSession } from "./session";
+
+export interface CapturedPageSnapshot {
+  html: string;
+  finalUrl: string;
+}
+
+export const CAPTURE_NORMALIZED_PAGE_SCRIPT = String.raw`
+(() => {
+  const baseUrl = document.baseURI || location.href;
+  const htmlClone = document.documentElement.cloneNode(true);
+
+  function materializeShadowDom(sourceRoot, cloneRoot) {
+    const sourceElements = Array.from(sourceRoot.querySelectorAll("*"));
+    const cloneElements = Array.from(cloneRoot.querySelectorAll("*"));
+
+    for (let index = sourceElements.length - 1; index >= 0; index -= 1) {
+      const sourceElement = sourceElements[index];
+      const cloneElement = cloneElements[index];
+      const shadowRoot = sourceElement && sourceElement.shadowRoot;
+      if (!shadowRoot || !cloneElement || !shadowRoot.innerHTML) {
+        continue;
+      }
+
+      if (cloneElement.tagName && cloneElement.tagName.includes("-")) {
+        const wrapper = document.createElement("div");
+        wrapper.setAttribute("data-shadow-host", cloneElement.tagName.toLowerCase());
+        wrapper.innerHTML = shadowRoot.innerHTML;
+        cloneElement.replaceWith(wrapper);
+      } else {
+        cloneElement.innerHTML = shadowRoot.innerHTML;
+      }
+    }
+  }
+
+  function toAbsolute(url) {
+    if (!url) return url;
+    try {
+      return new URL(url, baseUrl).href;
+    } catch {
+      return url;
+    }
+  }
+
+  function absolutizeAttribute(root, selector, attribute) {
+    root.querySelectorAll(selector).forEach((element) => {
+      const value = element.getAttribute(attribute);
+      if (!value) return;
+      const absolute = toAbsolute(value);
+      if (absolute) {
+        element.setAttribute(attribute, absolute);
+      }
+    });
+  }
+
+  function absolutizeSrcset(root, selector) {
+    root.querySelectorAll(selector).forEach((element) => {
+      const srcset = element.getAttribute("srcset");
+      if (!srcset) return;
+      element.setAttribute(
+        "srcset",
+        srcset
+          .split(",")
+          .map((part) => {
+            const trimmed = part.trim();
+            if (!trimmed) return "";
+            const [url, ...descriptor] = trimmed.split(/\s+/);
+            const absolute = toAbsolute(url);
+            return descriptor.length > 0 ? absolute + " " + descriptor.join(" ") : absolute;
+          })
+          .filter(Boolean)
+          .join(", "),
+      );
+    });
+  }
+
+  materializeShadowDom(document.documentElement, htmlClone);
+
+  htmlClone
+    .querySelectorAll("img[data-src], video[data-src], audio[data-src], source[data-src]")
+    .forEach((element) => {
+      const dataSource = element.getAttribute("data-src");
+      const current = element.getAttribute("src");
+      if (dataSource && (!current || current === "" || current.startsWith("data:"))) {
+        element.setAttribute("src", dataSource);
+      }
+    });
+
+  absolutizeAttribute(htmlClone, "a[href]", "href");
+  absolutizeAttribute(htmlClone, "img[src], video[src], audio[src], source[src], iframe[src]", "src");
+  absolutizeAttribute(htmlClone, "video[poster]", "poster");
+  absolutizeSrcset(htmlClone, "img[srcset], source[srcset]");
+
+  return {
+    html: "<!doctype html>\n" + htmlClone.outerHTML,
+    finalUrl: location.href,
+  };
+})()
+`;
+
+export async function captureNormalizedPageSnapshot(
+  browser: BrowserSession,
+): Promise<CapturedPageSnapshot> {
+  return browser.evaluate<CapturedPageSnapshot>(CAPTURE_NORMALIZED_PAGE_SCRIPT);
+}

packages/baoyu-fetch/src/browser/profile.ts

@@ -0,0 +1,201 @@
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import process from "node:process";
+import { spawnSync } from "node:child_process";
+
+export interface ResolveSharedChromeProfileDirOptions {
+  envNames?: string[];
+  appDataDirName?: string;
+  profileDirName?: string;
+}
+
+export interface FindExistingChromeDebugPortOptions {
+  profileDir: string;
+  timeoutMs?: number;
+}
+
+interface ChromeVersionResponse {
+  webSocketDebuggerUrl?: string;
+}
+
+const CHROME_LOCK_FILE_NAMES = ["SingletonLock", "SingletonSocket", "SingletonCookie", "chrome.pid"] as const;
+
+function resolveDataBaseDir(): string {
+  if (process.platform === "darwin") {
+    return path.join(os.homedir(), "Library", "Application Support");
+  }
+  if (process.platform === "win32") {
+    return process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming");
+  }
+  return process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share");
+}
+
+export function resolveSharedChromeProfileDir(
+  options: ResolveSharedChromeProfileDirOptions = {},
+): string {
+  for (const envName of options.envNames ?? []) {
+    const override = process.env[envName]?.trim();
+    if (override) {
+      return path.resolve(override);
+    }
+  }
+
+  const appDataDirName = options.appDataDirName ?? "baoyu-skills";
+  const profileDirName = options.profileDirName ?? "chrome-profile";
+  return path.join(resolveDataBaseDir(), appDataDirName, profileDirName);
+}
+
+export function resolveChromeProfileDir(profileDir?: string): string {
+  if (profileDir?.trim()) {
+    return path.resolve(profileDir.trim());
+  }
+
+  return resolveSharedChromeProfileDir({
+    envNames: ["BAOYU_CHROME_PROFILE_DIR"],
+    appDataDirName: "baoyu-skills",
+    profileDirName: "chrome-profile",
+  });
+}
+
+export function ensureChromeProfileDir(profileDir: string): string {
+  fs.mkdirSync(profileDir, { recursive: true });
+  return profileDir;
+}
+
+export function hasChromeLockArtifacts(entries: readonly string[]): boolean {
+  return CHROME_LOCK_FILE_NAMES.some((name) => entries.includes(name));
+}
+
+export function shouldRetryChromeLaunchRecovery(options: {
+  hasLockArtifacts: boolean;
+  hasLiveOwner: boolean;
+}): boolean {
+  return options.hasLockArtifacts && !options.hasLiveOwner;
+}
+
+export function findChromeProcessUsingProfile(profileDir: string): boolean {
+  if (process.platform === "win32") {
+    return false;
+  }
+
+  try {
+    const result = spawnSync("ps", ["aux"], {
+      encoding: "utf8",
+      timeout: 5_000,
+    });
+    if (result.status !== 0 || !result.stdout) {
+      return false;
+    }
+
+    return result.stdout
+      .split("\n")
+      .some((line) => line.includes(`--user-data-dir=${profileDir}`));
+  } catch {
+    return false;
+  }
+}
+
+export function cleanChromeLockArtifacts(profileDir: string): void {
+  for (const name of CHROME_LOCK_FILE_NAMES) {
+    try {
+      fs.unlinkSync(path.join(profileDir, name));
+    } catch {
+      // Ignore missing files and continue cleaning the remaining artifacts.
+    }
+  }
+}
+
+export async function listChromeProfileEntries(profileDir: string): Promise<string[]> {
+  try {
+    return await fs.promises.readdir(profileDir);
+  } catch {
+    return [];
+  }
+}
+
+async function fetchWithTimeout(url: string, timeoutMs = 3_000): Promise<Response> {
+  const controller = new AbortController();
+  const timer = setTimeout(() => controller.abort(), timeoutMs);
+  try {
+    return await fetch(url, {
+      redirect: "follow",
+      signal: controller.signal,
+    });
+  } finally {
+    clearTimeout(timer);
+  }
+}
+
+async function fetchJson<T>(url: string, timeoutMs = 3_000): Promise<T> {
+  const response = await fetchWithTimeout(url, timeoutMs);
+  if (!response.ok) {
+    throw new Error(`Request failed: ${response.status} ${response.statusText}`);
+  }
+  return (await response.json()) as T;
+}
+
+async function isDebugPortReady(port: number, timeoutMs = 3_000): Promise<boolean> {
+  try {
+    const version = await fetchJson<ChromeVersionResponse>(`http://127.0.0.1:${port}/json/version`, timeoutMs);
+    return Boolean(version.webSocketDebuggerUrl);
+  } catch {
+    return false;
+  }
+}
+
+function parseDevToolsActivePort(filePath: string): { port: number; wsPath: string } | null {
+  try {
+    const content = fs.readFileSync(filePath, "utf8");
+    const lines = content.split(/\r?\n/);
+    const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
+    const wsPath = lines[1]?.trim() ?? "";
+    if (port > 0 && wsPath) {
+      return { port, wsPath };
+    }
+  } catch {
+    // Ignore and fall back to process inspection.
+  }
+
+  return null;
+}
+
+export async function findExistingChromeDebugPort(
+  options: FindExistingChromeDebugPortOptions,
+): Promise<number | null> {
+  const timeoutMs = options.timeoutMs ?? 3_000;
+  const activePort = parseDevToolsActivePort(path.join(options.profileDir, "DevToolsActivePort"));
+  if (activePort && await isDebugPortReady(activePort.port, timeoutMs)) {
+    return activePort.port;
+  }
+
+  if (process.platform === "win32") {
+    return null;
+  }
+
+  try {
+    const result = spawnSync("ps", ["aux"], {
+      encoding: "utf8",
+      timeout: 5_000,
+    });
+    if (result.status !== 0 || !result.stdout) {
+      return null;
+    }
+
+    const lines = result.stdout
+      .split("\n")
+      .filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
+
+    for (const line of lines) {
+      const match = line.match(/--remote-debugging-port=(\d+)/);
+      const port = Number.parseInt(match?.[1] ?? "", 10);
+      if (port > 0 && await isDebugPortReady(port, timeoutMs)) {
+        return port;
+      }
+    }
+  } catch {
+    // Ignore and report no reusable debugger.
+  }
+
+  return null;
+}

packages/baoyu-fetch/src/browser/session.ts

@@ -0,0 +1,155 @@
+import { execFile } from "node:child_process";
+import { promisify } from "node:util";
+import { CdpClient, TargetSession, evaluateRuntime } from "./cdp-client";
+
+interface NavigationResult {
+  errorText?: string;
+}
+
+const execFileAsync = promisify(execFile);
+const MACOS_BROWSER_APP_IDS = [
+  "com.google.Chrome",
+  "org.chromium.Chromium",
+  "com.brave.Browser",
+  "com.microsoft.edgemac",
+];
+
+function sleep(ms: number): Promise<void> {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+async function activateBrowserApp(): Promise<void> {
+  if (process.platform !== "darwin") {
+    return;
+  }
+
+  for (const appId of MACOS_BROWSER_APP_IDS) {
+    try {
+      await execFileAsync("osascript", ["-e", `tell application id "${appId}" to activate`]);
+      return;
+    } catch {
+      // Try the next installed browser bundle id.
+    }
+  }
+}
+
+export class BrowserSession {
+  private constructor(
+    private readonly cdp: CdpClient,
+    public readonly targetSession: TargetSession,
+    public readonly interactive: boolean,
+  ) {}
+
+  static async open(
+    cdp: CdpClient,
+    options: {
+      initialUrl?: string;
+      interactive?: boolean;
+    } = {},
+  ): Promise<BrowserSession> {
+    const targetSession = await cdp.createPageSession({
+      initialUrl: options.initialUrl,
+      visible: options.interactive,
+    });
+    const browser = new BrowserSession(cdp, targetSession, Boolean(options.interactive));
+    if (browser.interactive) {
+      await browser.bringToFront().catch(() => {});
+    }
+    return browser;
+  }
+
+  async goto(url: string, timeoutMs = 30_000): Promise<void> {
+    const loadPromise = this.targetSession.waitForEvent("Page.loadEventFired", undefined, timeoutMs).catch(() => null);
+    const result = await this.targetSession.send<NavigationResult>("Page.navigate", { url });
+    if (result.errorText) {
+      throw new Error(`Navigation failed: ${result.errorText}`);
+    }
+    await loadPromise;
+    await this.waitForReadyState(timeoutMs);
+  }
+
+  async waitForReadyState(timeoutMs = 30_000): Promise<void> {
+    const startedAt = Date.now();
+    while (Date.now() - startedAt < timeoutMs) {
+      const state = await this.evaluate<string>("document.readyState");
+      if (state === "interactive" || state === "complete") {
+        return;
+      }
+      await sleep(150);
+    }
+    throw new Error("Timed out waiting for document.readyState");
+  }
+
+  async evaluate<T>(expression: string): Promise<T> {
+    return evaluateRuntime<T>(this.targetSession, expression);
+  }
+
+  async getHTML(): Promise<string> {
+    return this.evaluate<string>("document.documentElement.outerHTML");
+  }
+
+  async getTitle(): Promise<string> {
+    return this.evaluate<string>("document.title");
+  }
+
+  async getURL(): Promise<string> {
+    return this.evaluate<string>("window.location.href");
+  }
+
+  async bringToFront(): Promise<void> {
+    await this.targetSession.send("Page.bringToFront").catch(async () => {
+      await this.cdp.sendBrowserCommand("Target.activateTarget", {
+        targetId: this.targetSession.targetId,
+      });
+    });
+    if (this.interactive) {
+      await activateBrowserApp().catch(() => {});
+    }
+  }
+
+  async click(selector: string): Promise<void> {
+    const result = await this.evaluate<{ ok: boolean; error?: string }>(`
+      (() => {
+        const element = document.querySelector(${JSON.stringify(selector)});
+        if (!element) {
+          return { ok: false, error: "Element not found" };
+        }
+        element.scrollIntoView({ block: "center", inline: "center" });
+        if (element instanceof HTMLElement) {
+          element.click();
+          return { ok: true };
+        }
+        return { ok: false, error: "Element is not clickable" };
+      })()
+    `);
+
+    if (!result.ok) {
+      throw new Error(result.error ?? `Failed to click ${selector}`);
+    }
+  }
+
+  async scrollToEnd(options: { stepPx?: number; delayMs?: number; maxSteps?: number } = {}): Promise<void> {
+    const stepPx = options.stepPx ?? 1_400;
+    const delayMs = options.delayMs ?? 250;
+    const maxSteps = options.maxSteps ?? 6;
+
+    for (let step = 0; step < maxSteps; step += 1) {
+      const done = await this.evaluate<boolean>(`
+        (() => {
+          const before = window.scrollY;
+          window.scrollBy(0, ${stepPx});
+          const atBottom = window.innerHeight + window.scrollY >= document.body.scrollHeight - 4;
+          return atBottom || window.scrollY === before;
+        })()
+      `);
+      if (done) {
+        break;
+      }
+      await sleep(delayMs);
+    }
+  }
+
+  async close(): Promise<void> {
+    await this.cdp.closeTarget(this.targetSession.targetId);
+  }
+}

packages/baoyu-fetch/src/cli.ts

@@ -0,0 +1,227 @@
+#!/usr/bin/env bun
+
+import {
+  runConvertCommand,
+  type ConvertCommandOptions,
+  type OutputFormat,
+  type WaitMode,
+} from "./commands/convert";
+
+export const HELP_TEXT = `
+baoyu-fetch - Read a URL into Markdown or JSON with Chrome CDP
+
+Usage:
+  baoyu-fetch <url> [options]
+
+Options:
+  --output <file>       Save output to file
+  --format <type>       Output format: markdown | json
+  --json                Alias for --format json
+  --adapter <name>      Force an adapter (e.g. x, generic)
+  --download-media      Download adapter-reported media into ./imgs and ./videos, then rewrite markdown links
+  --media-dir <dir>     Base directory for downloaded media. Defaults to the output directory
+  --debug-dir <dir>     Write debug artifacts
+  --cdp-url <url>       Reuse an existing Chrome DevTools endpoint
+  --browser-path <path> Explicit Chrome binary path
+  --chrome-profile-dir <path>
+                        Chrome user data dir. Defaults to BAOYU_CHROME_PROFILE_DIR
+                        or baoyu-skills/chrome-profile.
+  --headless            Launch a temporary headless Chrome if needed
+  --wait-for <mode>     Wait mode: interaction | force
+                        interaction: start visible Chrome and auto-wait only when login or verification is required
+                        force: start visible Chrome, then auto-continue after it detects login/challenge progress
+                               or continue immediately when you press Enter
+  --wait-for-interaction
+                        Alias for --wait-for interaction
+  --wait-for-login      Alias for --wait-for interaction
+  --interaction-timeout <ms>
+                        How long to wait for manual interaction before failing (default: 600000)
+  --interaction-poll-interval <ms>
+                        How often to poll interaction state while waiting (default: 1500)
+  --login-timeout <ms>  Alias for --interaction-timeout
+  --login-poll-interval <ms>
+                        Alias for --interaction-poll-interval
+  --timeout <ms>        Page timeout in milliseconds (default: 30000)
+  --help                Show help
+
+Examples:
+  baoyu-fetch https://example.com
+  baoyu-fetch https://example.com --format markdown --output article.md --download-media
+  baoyu-fetch https://example.com --format json --output article.json
+  baoyu-fetch https://x.com/lennysan/status/2036483059407810640 --wait-for interaction
+  baoyu-fetch https://x.com/lennysan/status/2036483059407810640 --wait-for force
+`.trim();
+
+interface CliOptions extends ConvertCommandOptions {
+  url?: string;
+  help: boolean;
+}
+
+function normalizeWaitMode(raw: string): WaitMode {
+  const value = raw.toLowerCase();
+  if (value === "interaction" || value === "auto") {
+    return "interaction";
+  }
+  if (value === "force" || value === "manual" || value === "always") {
+    return "force";
+  }
+  throw new Error(`Invalid wait mode: ${raw}. Expected interaction or force.`);
+}
+
+function normalizeOutputFormat(raw: string): OutputFormat {
+  const value = raw.toLowerCase();
+  if (value === "markdown" || value === "json") {
+    return value;
+  }
+
+  throw new Error(`Invalid output format: ${raw}. Expected markdown or json.`);
+}
+
+export function parseArgs(argv: string[]): CliOptions {
+  const options: CliOptions = {
+    format: "markdown",
+    headless: false,
+    downloadMedia: false,
+    waitMode: "none",
+    interactionTimeoutMs: 600_000,
+    interactionPollIntervalMs: 1_500,
+    timeoutMs: 30_000,
+    help: false,
+  };
+
+  const args = argv.slice(2);
+  for (let index = 0; index < args.length; index += 1) {
+    const value = args[index];
+
+    if (value === "--help" || value === "-h") {
+      options.help = true;
+      continue;
+    }
+    if (value === "--format") {
+      const format = args[index + 1];
+      if (!format) {
+        throw new Error("--format requires a value");
+      }
+      options.format = normalizeOutputFormat(format);
+      index += 1;
+      continue;
+    }
+    if (value === "--json") {
+      options.format = "json";
+      continue;
+    }
+    if (value === "--download-media") {
+      options.downloadMedia = true;
+      continue;
+    }
+    if (value === "--headless") {
+      options.headless = true;
+      continue;
+    }
+    if (value === "--wait-for") {
+      const mode = args[index + 1];
+      if (!mode) {
+        throw new Error("--wait-for requires a mode");
+      }
+      options.waitMode = normalizeWaitMode(mode);
+      index += 1;
+      continue;
+    }
+    if (value === "--wait-for-interaction" || value === "--wait-for-login") {
+      options.waitMode = "interaction";
+      continue;
+    }
+    if (value === "--output") {
+      options.output = args[index + 1];
+      index += 1;
+      continue;
+    }
+    if (value === "--adapter") {
+      options.adapter = args[index + 1];
+      index += 1;
+      continue;
+    }
+    if (value === "--debug-dir") {
+      options.debugDir = args[index + 1];
+      index += 1;
+      continue;
+    }
+    if (value === "--media-dir") {
+      options.mediaDir = args[index + 1];
+      index += 1;
+      continue;
+    }
+    if (value === "--cdp-url") {
+      options.cdpUrl = args[index + 1];
+      index += 1;
+      continue;
+    }
+    if (value === "--browser-path") {
+      options.browserPath = args[index + 1];
+      index += 1;
+      continue;
+    }
+    if (value === "--chrome-profile-dir") {
+      options.chromeProfileDir = args[index + 1];
+      index += 1;
+      continue;
+    }
+    if (value === "--timeout") {
+      const parsed = Number(args[index + 1]);
+      if (!Number.isFinite(parsed) || parsed <= 0) {
+        throw new Error(`Invalid timeout: ${args[index + 1]}`);
+      }
+      options.timeoutMs = parsed;
+      index += 1;
+      continue;
+    }
+    if (value === "--interaction-timeout" || value === "--login-timeout") {
+      const parsed = Number(args[index + 1]);
+      if (!Number.isFinite(parsed) || parsed <= 0) {
+        throw new Error(`Invalid interaction timeout: ${args[index + 1]}`);
+      }
+      options.interactionTimeoutMs = parsed;
+      index += 1;
+      continue;
+    }
+    if (value === "--interaction-poll-interval" || value === "--login-poll-interval") {
+      const parsed = Number(args[index + 1]);
+      if (!Number.isFinite(parsed) || parsed <= 0) {
+        throw new Error(`Invalid interaction poll interval: ${args[index + 1]}`);
+      }
+      options.interactionPollIntervalMs = parsed;
+      index += 1;
+      continue;
+    }
+    if (value.startsWith("-")) {
+      throw new Error(`Unknown option: ${value}`);
+    }
+    if (!options.url) {
+      options.url = value;
+      continue;
+    }
+    throw new Error(`Unexpected argument: ${value}`);
+  }
+
+  return options;
+}
+
+async function main(): Promise<void> {
+  try {
+    const options = parseArgs(process.argv);
+    if (options.help || !options.url) {
+      console.log(HELP_TEXT);
+      return;
+    }
+
+    await runConvertCommand(options);
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    console.error(message);
+    process.exitCode = 1;
+  }
+}
+
+if (import.meta.main) {
+  void main();
+}

packages/baoyu-fetch/src/commands/convert.ts

@@ -0,0 +1,580 @@
+import { mkdir, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+import { createInterface } from "node:readline";
+import { connectChrome, type ChromeConnection } from "../browser/chrome-launcher";
+import { CdpClient } from "../browser/cdp-client";
+import { detectInteractionGate } from "../browser/interaction-gates";
+import { NetworkJournal } from "../browser/network-journal";
+import { BrowserSession } from "../browser/session";
+import { genericAdapter, resolveAdapter } from "../adapters";
+import { isXSessionReady } from "../adapters/x/session";
+import type { ExtractedDocument } from "../extract/document";
+import { renderMarkdown } from "../extract/markdown-renderer";
+import { downloadMediaAssets } from "../media/default-downloader";
+import { rewriteMarkdownMediaLinks } from "../media/markdown-media";
+import { createLogger } from "../utils/logger";
+import { normalizeUrl } from "../utils/url";
+import type {
+  Adapter,
+  AdapterContext,
+  AdapterLoginInfo,
+  LoginState,
+  MediaAsset,
+  WaitForInteractionRequest,
+} from "../adapters/types";
+
+export type WaitMode = "none" | "interaction" | "force";
+export type OutputFormat = "markdown" | "json";
+
+export interface ConvertCommandOptions {
+  url?: string;
+  output?: string;
+  format: OutputFormat;
+  adapter?: string;
+  debugDir?: string;
+  cdpUrl?: string;
+  browserPath?: string;
+  chromeProfileDir?: string;
+  headless: boolean;
+  downloadMedia: boolean;
+  mediaDir?: string;
+  waitMode: WaitMode;
+  interactionTimeoutMs: number;
+  interactionPollIntervalMs: number;
+  timeoutMs: number;
+}
+
+interface RuntimeResources {
+  chrome: ChromeConnection;
+  cdp: CdpClient;
+  browser: BrowserSession;
+  network: NetworkJournal;
+  interactive: boolean;
+}
+
+interface ForceWaitSnapshot {
+  url: string;
+  hasGate: boolean;
+  loginState: LoginState | "unavailable";
+  sessionReady: boolean;
+}
+
+interface SuccessfulConvertOutput {
+  adapter: string;
+  status: "ok";
+  login?: AdapterLoginInfo;
+  media: MediaAsset[];
+  downloads: Awaited<ReturnType<typeof downloadMediaAssets>> | null;
+  document: ExtractedDocument;
+  markdown: string;
+}
+
+interface InteractionRequiredOutput {
+  adapter: string;
+  status: "needs_interaction";
+  login?: AdapterLoginInfo;
+  interaction: WaitForInteractionRequest;
+}
+
+function sleep(ms: number): Promise<void> {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+function isForceWaitSessionReady(snapshot: ForceWaitSnapshot): boolean {
+  return snapshot.sessionReady;
+}
+
+export function shouldKeepBrowserOpenAfterInteraction(options: {
+  launched: boolean;
+  interaction: Pick<WaitForInteractionRequest, "kind" | "provider">;
+}): boolean {
+  return options.launched && options.interaction.kind === "login" && options.interaction.provider === "x";
+}
+
+export function shouldAutoContinueForceWait(
+  initial: ForceWaitSnapshot,
+  current: ForceWaitSnapshot,
+): boolean {
+  if (initial.hasGate && !current.hasGate) {
+    return true;
+  }
+
+  if (initial.loginState === "logged_out" && current.loginState !== "logged_out" && isForceWaitSessionReady(current)) {
+    return true;
+  }
+
+  if (initial.loginState !== "logged_in" && current.loginState === "logged_in" && isForceWaitSessionReady(current)) {
+    return true;
+  }
+
+  if (
+    current.url !== initial.url &&
+    !current.hasGate &&
+    current.loginState !== "logged_out" &&
+    isForceWaitSessionReady(current)
+  ) {
+    return true;
+  }
+
+  return false;
+}
+
+async function writeOutput(path: string, content: string): Promise<void> {
+  const directory = path.includes("/") ? path.slice(0, path.lastIndexOf("/")) : "";
+  if (directory) {
+    await mkdir(directory, { recursive: true });
+  }
+  await writeFile(path, content, "utf8");
+}
+
+async function writeDebugArtifacts(
+  debugDir: string,
+  document: ExtractedDocument,
+  markdown: string,
+  browser: BrowserSession,
+  network: NetworkJournal,
+): Promise<void> {
+  await mkdir(debugDir, { recursive: true });
+
+  const html = await browser.getHTML().catch(() => "");
+  const networkDump = await network.toJSON({ includeBodies: true });
+
+  await Promise.all([
+    writeFile(join(debugDir, "document.json"), JSON.stringify(document, null, 2), "utf8"),
+    writeFile(join(debugDir, "markdown.md"), markdown, "utf8"),
+    writeFile(join(debugDir, "page.html"), html, "utf8"),
+    writeFile(join(debugDir, "network.json"), JSON.stringify(networkDump, null, 2), "utf8"),
+  ]);
+}
+
+async function openRuntime(
+  options: ConvertCommandOptions,
+  interactive: boolean,
+  debugEnabled: boolean,
+): Promise<RuntimeResources> {
+  const logger = createLogger(debugEnabled);
+  if (interactive) {
+    logger.info("Opening Chrome in interactive mode.");
+  }
+  const chrome = await connectChrome({
+    cdpUrl: options.cdpUrl,
+    browserPath: options.browserPath,
+    profileDir: options.chromeProfileDir,
+    headless: interactive ? false : options.headless,
+    logger,
+  });
+
+  const cdp = await CdpClient.connect(chrome.browserWsUrl);
+  const browser = await BrowserSession.open(cdp, { interactive });
+  if (interactive) {
+    await browser.bringToFront().catch(() => {});
+  }
+  const network = new NetworkJournal(browser.targetSession, logger);
+  await network.start();
+
+  return {
+    chrome,
+    cdp,
+    browser,
+    network,
+    interactive,
+  };
+}
+
+async function closeRuntime(runtime: RuntimeResources | null | undefined): Promise<void> {
+  if (!runtime) {
+    return;
+  }
+  runtime.network.stop();
+  await runtime.browser.close().catch(() => {});
+  await runtime.cdp.close().catch(() => {});
+  await runtime.chrome.close().catch(() => {});
+}
+
+async function isInteractionSessionReady(
+  context: AdapterContext,
+  interaction: WaitForInteractionRequest,
+): Promise<boolean> {
+  if (interaction.provider !== "x") {
+    return true;
+  }
+  return await isXSessionReady(context).catch(() => false);
+}
+
+async function reopenInteractiveRuntime(
+  runtime: RuntimeResources,
+  options: ConvertCommandOptions,
+  debugEnabled: boolean,
+): Promise<RuntimeResources> {
+  if (runtime.interactive) {
+    return runtime;
+  }
+
+  await closeRuntime(runtime);
+  return openRuntime(options, true, debugEnabled);
+}
+
+async function captureForceWaitSnapshot(
+  adapter: Adapter,
+  context: AdapterContext,
+): Promise<ForceWaitSnapshot> {
+  const [gate, url, login] = await Promise.all([
+    detectInteractionGate(context.browser).catch(() => null),
+    context.browser.getURL().catch(() => context.input.url.toString()),
+    adapter.checkLogin?.(context).catch(() => ({
+      provider: adapter.name,
+      state: "unknown" as const,
+    })),
+  ]);
+
+  return {
+    url,
+    hasGate: Boolean(gate),
+    loginState: login?.state ?? "unavailable",
+    sessionReady: adapter.name === "x" ? await isXSessionReady(context).catch(() => false) : true,
+  };
+}
+
+async function waitForForceResume(
+  adapter: Adapter,
+  context: AdapterContext,
+  options: ConvertCommandOptions,
+): Promise<void> {
+  if (context.interactive) {
+    await context.browser.bringToFront().catch(() => {});
+  }
+
+  const prompt =
+    "Chrome is ready. Complete any manual login or verification. Extraction will continue automatically after it detects progress, or press Enter to continue immediately.";
+  context.log.info(prompt);
+
+  const rl = createInterface({
+    input: process.stdin,
+    output: process.stderr,
+  });
+
+  let manualContinue = false;
+  let closed = false;
+  const closeReadline = (): void => {
+    if (!closed) {
+      closed = true;
+      rl.close();
+    }
+  };
+
+  rl.once("line", () => {
+    manualContinue = true;
+    closeReadline();
+  });
+
+  const initial = await captureForceWaitSnapshot(adapter, context);
+  const startedAt = Date.now();
+
+  try {
+    while (Date.now() - startedAt < options.interactionTimeoutMs) {
+      if (manualContinue) {
+        return;
+      }
+
+      const current = await captureForceWaitSnapshot(adapter, context);
+      if (shouldAutoContinueForceWait(initial, current)) {
+        return;
+      }
+
+      await sleep(options.interactionPollIntervalMs);
+    }
+  } finally {
+    closeReadline();
+  }
+
+  throw new Error("Timed out waiting for force-mode interaction to complete");
+}
+
+async function waitForInteraction(
+  adapter: Adapter,
+  context: AdapterContext,
+  interaction: WaitForInteractionRequest,
+  options: ConvertCommandOptions,
+): Promise<AdapterLoginInfo> {
+  const timeoutMs = interaction.timeoutMs ?? options.interactionTimeoutMs;
+  const pollIntervalMs = interaction.pollIntervalMs ?? options.interactionPollIntervalMs;
+  if (context.interactive) {
+    await context.browser.bringToFront().catch(() => {});
+  }
+  context.log.info(interaction.prompt);
+
+  const startedAt = Date.now();
+  let lastLogin: AdapterLoginInfo | null = null;
+
+  while (Date.now() - startedAt < timeoutMs) {
+    if (interaction.kind === "login" && adapter.checkLogin) {
+      lastLogin = await adapter.checkLogin(context);
+      if (lastLogin.state === "logged_in" && await isInteractionSessionReady(context, interaction)) {
+        return lastLogin;
+      }
+    }
+
+    const gate = await detectInteractionGate(context.browser);
+    if (!gate) {
+      if (interaction.kind !== "login") {
+        return lastLogin ?? {
+          provider: interaction.provider,
+          state: "unknown",
+          reason: `${interaction.provider} challenge cleared`,
+        };
+      }
+
+      if (!adapter.checkLogin) {
+        return {
+          provider: interaction.provider,
+          state: "unknown",
+        };
+      }
+
+      lastLogin = await adapter.checkLogin(context);
+      if (lastLogin.state !== "logged_out" && await isInteractionSessionReady(context, interaction)) {
+        return lastLogin;
+      }
+    }
+    await sleep(pollIntervalMs);
+  }
+
+  const reason = lastLogin?.reason ? ` (${lastLogin.reason})` : "";
+  throw new Error(`Timed out waiting for ${interaction.provider} interaction${reason}`);
+}
+
+export function formatOutputContent(
+  format: OutputFormat,
+  payload: SuccessfulConvertOutput | InteractionRequiredOutput,
+): string {
+  if (format === "json") {
+    return JSON.stringify(payload, null, 2);
+  }
+
+  if (payload.status !== "ok") {
+    throw new Error("Markdown output is only available for successful extraction results");
+  }
+
+  return payload.markdown;
+}
+
+function printOutput(content: string): void {
+  process.stdout.write(content);
+  if (!content.endsWith("\n")) {
+    process.stdout.write("\n");
+  }
+}
+
+export async function runConvertCommand(options: ConvertCommandOptions): Promise<void> {
+  if (!options.url) {
+    throw new Error("URL is required");
+  }
+  if (options.downloadMedia && !options.output) {
+    throw new Error("--download-media requires --output so media paths can be rewritten relative to the saved output file");
+  }
+
+  const url = normalizeUrl(options.url);
+  let runtime = await openRuntime(options, options.waitMode !== "none", Boolean(options.debugDir));
+  const logger = createLogger(Boolean(options.debugDir));
+  let didLogin = false;
+  let adapter: Adapter | null = null;
+  let context: AdapterContext | null = null;
+
+  try {
+    adapter = resolveAdapter({ url }, options.adapter);
+    context = {
+      input: { url },
+      browser: runtime.browser,
+      network: runtime.network,
+      cdp: runtime.cdp,
+      log: logger,
+      outputFormat: options.format,
+      timeoutMs: options.timeoutMs,
+      interactive: runtime.interactive,
+      downloadMedia: options.downloadMedia,
+    };
+
+    if (adapter.restoreCookies) {
+      const restored = await adapter.restoreCookies(context, runtime.chrome.profileDir).catch(() => false);
+      if (restored) logger.info(`Restored ${adapter.name} session cookies from sidecar.`);
+    }
+
+    if (options.waitMode === "interaction" && adapter.checkLogin) {
+      await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
+      const preLogin = await adapter.checkLogin(context);
+      if (preLogin.state !== "logged_in") {
+        didLogin = true;
+        await waitForInteraction(adapter, context, {
+          type: "wait_for_interaction",
+          kind: "login",
+          provider: preLogin.provider ?? adapter.name,
+          prompt: `Please sign in to ${adapter.name === "x" ? "X" : adapter.name} in the opened Chrome window. Extraction will continue automatically once login is detected.`,
+          reason: preLogin.reason ?? `Not logged in to ${adapter.name}`,
+          requiresVisibleBrowser: true,
+        }, options);
+      }
+    }
+
+    if (options.waitMode === "force") {
+      await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
+      await waitForForceResume(adapter, context, options);
+    }
+
+    let result = await adapter.process(context);
+
+    if (result.status === "no_document") {
+      const interaction = await detectInteractionGate(context.browser);
+      if (interaction) {
+        result = {
+          status: "needs_interaction",
+          interaction,
+          login: result.login,
+        };
+      }
+    }
+
+    while (result.status === "needs_interaction") {
+      if (options.waitMode === "none") {
+        if (options.format === "json") {
+          printOutput(
+            formatOutputContent(options.format, {
+            adapter: adapter.name,
+            status: result.status,
+            login: result.login,
+            interaction: result.interaction,
+            }),
+          );
+          return;
+        }
+
+        throw new Error(`${adapter.name} requires manual interaction. Re-run with --wait-for interaction to continue after completing it.`);
+      }
+
+      if (result.interaction.requiresVisibleBrowser !== false) {
+        runtime = await reopenInteractiveRuntime(runtime, options, Boolean(options.debugDir));
+      }
+
+      context = {
+        input: { url },
+        browser: runtime.browser,
+        network: runtime.network,
+        cdp: runtime.cdp,
+        log: logger,
+        outputFormat: options.format,
+        timeoutMs: options.timeoutMs,
+        interactive: runtime.interactive,
+        downloadMedia: options.downloadMedia,
+      };
+
+      await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
+      if (result.interaction.kind === "login") {
+        didLogin = true;
+      }
+      await waitForInteraction(adapter, context, result.interaction, options);
+      result = await adapter.process(context);
+
+      if (result.status === "no_document") {
+        const interaction = await detectInteractionGate(context.browser);
+        if (interaction) {
+          result = {
+            status: "needs_interaction",
+            interaction,
+            login: result.login,
+          };
+        }
+      }
+    }
+
+    let document: ExtractedDocument | null = result.status === "ok" ? result.document : null;
+    let media: MediaAsset[] = result.status === "ok" ? (result.media ?? []) : [];
+    let login = result.login;
+    let mediaAdapter = adapter;
+
+    if (!document && adapter.name !== genericAdapter.name && result.status === "no_document") {
+      logger.info(`Adapter ${adapter.name} returned no structured document; falling back to generic extraction`);
+      const fallback = await genericAdapter.process(context);
+      if (fallback.status === "ok") {
+        document = fallback.document;
+        media = fallback.media ?? [];
+        mediaAdapter = genericAdapter;
+      }
+    }
+
+    if (!document) {
+      throw new Error("Failed to extract a document from the target URL");
+    }
+
+    document.requestedUrl ??= url.toString();
+
+    let markdown = renderMarkdown(document);
+    let downloadResult:
+      | Awaited<ReturnType<typeof downloadMediaAssets>>
+      | null = null;
+
+    if (options.downloadMedia && options.output) {
+      downloadResult = mediaAdapter.downloadMedia
+        ? await mediaAdapter.downloadMedia({
+            media,
+            outputPath: options.output,
+            mediaDir: options.mediaDir,
+            log: logger,
+          })
+        : await downloadMediaAssets({
+            media,
+            outputPath: options.output,
+            mediaDir: options.mediaDir,
+            log: logger,
+          });
+
+      markdown = rewriteMarkdownMediaLinks(markdown, downloadResult.replacements);
+      if (downloadResult.downloadedImages > 0 || downloadResult.downloadedVideos > 0) {
+        logger.info(
+          `Downloaded ${downloadResult.downloadedImages} images and ${downloadResult.downloadedVideos} videos`,
+        );
+      }
+    }
+
+    if (options.output) {
+      await writeOutput(
+        options.output,
+        formatOutputContent(options.format, {
+          adapter: document.adapter ?? adapter.name,
+          status: "ok",
+          login,
+          media,
+          downloads: downloadResult,
+          document,
+          markdown,
+        }),
+      );
+      logger.info(`Saved ${options.format} to ${options.output}`);
+    }
+
+    if (options.debugDir) {
+      await writeDebugArtifacts(options.debugDir, document, markdown, runtime.browser, runtime.network);
+      logger.info(`Wrote debug artifacts to ${options.debugDir}`);
+    }
+
+    if (options.format === "json") {
+      printOutput(
+        formatOutputContent(options.format, {
+        adapter: document.adapter ?? adapter.name,
+        status: "ok",
+        login,
+        media,
+        downloads: downloadResult,
+        document,
+        markdown,
+        }),
+      );
+      return;
+    }
+
+    printOutput(markdown);
+  } finally {
+    if (adapter?.exportCookies && context) {
+      await adapter.exportCookies(context, runtime.chrome.profileDir).catch(() => {});
+    }
+    await closeRuntime(runtime);
+  }
+}

packages/baoyu-fetch/src/extract/document.ts

@@ -0,0 +1,51 @@
+export type ContentBlock =
+  | {
+      type: "paragraph";
+      text: string;
+    }
+  | {
+      type: "heading";
+      depth: number;
+      text: string;
+    }
+  | {
+      type: "list";
+      ordered: boolean;
+      items: string[];
+    }
+  | {
+      type: "quote";
+      text: string;
+    }
+  | {
+      type: "code";
+      code: string;
+      language?: string;
+    }
+  | {
+      type: "image";
+      url: string;
+      alt?: string;
+    }
+  | {
+      type: "html";
+      html: string;
+    }
+  | {
+      type: "markdown";
+      markdown: string;
+    };
+
+export interface ExtractedDocument {
+  url: string;
+  requestedUrl?: string;
+  canonicalUrl?: string;
+  title?: string;
+  author?: string;
+  siteName?: string;
+  publishedAt?: string;
+  summary?: string;
+  content: ContentBlock[];
+  metadata?: Record<string, unknown>;
+  adapter?: string;
+}

packages/baoyu-fetch/src/extract/html-cleaner.ts

@@ -0,0 +1,467 @@
+import { JSDOM } from "jsdom";
+
+export interface CleanHtmlOptions {
+  removeAds?: boolean;
+  removeBase64Images?: boolean;
+  onlyMainContent?: boolean;
+  includeSelectors?: string[];
+  excludeSelectors?: string[];
+}
+
+const ALWAYS_REMOVE_SELECTORS = [
+  "script",
+  "style",
+  "noscript",
+  "link[rel='stylesheet']",
+  "[hidden]",
+  "[aria-hidden='true']",
+  "[style*='display: none']",
+  "[style*='display:none']",
+  "[style*='visibility: hidden']",
+  "[style*='visibility:hidden']",
+  "svg[aria-hidden='true']",
+  "svg.icon",
+  "svg[class*='icon']",
+  "template",
+  "meta",
+  "iframe",
+  "canvas",
+  "object",
+  "embed",
+  "form",
+  "input",
+  "select",
+  "textarea",
+  "button",
+];
+
+const OVERLAY_SELECTORS = [
+  "[class*='modal']",
+  "[class*='popup']",
+  "[class*='overlay']",
+  "[class*='dialog']",
+  "[role='dialog']",
+  "[role='alertdialog']",
+  "[class*='cookie']",
+  "[class*='consent']",
+  "[class*='gdpr']",
+  "[class*='privacy-banner']",
+  "[class*='notification-bar']",
+  "[id*='cookie']",
+  "[id*='consent']",
+  "[id*='gdpr']",
+  "[style*='position: fixed']",
+  "[style*='position:fixed']",
+  "[style*='position: sticky']",
+  "[style*='position:sticky']",
+];
+
+const NAVIGATION_SELECTORS = [
+  "header",
+  "footer",
+  "nav",
+  "aside",
+  ".header",
+  ".top",
+  ".navbar",
+  "#header",
+  ".footer",
+  ".bottom",
+  "#footer",
+  ".sidebar",
+  ".side",
+  ".aside",
+  "#sidebar",
+  ".modal",
+  ".popup",
+  "#modal",
+  ".overlay",
+  ".ad",
+  ".ads",
+  ".advert",
+  "#ad",
+  ".lang-selector",
+  ".language",
+  "#language-selector",
+  ".social",
+  ".social-media",
+  ".social-links",
+  "#social",
+  ".menu",
+  ".navigation",
+  "#nav",
+  ".breadcrumbs",
+  "#breadcrumbs",
+  ".share",
+  "#share",
+  ".widget",
+  "#widget",
+  ".cookie",
+  "#cookie",
+];
+
+const FORCE_INCLUDE_SELECTORS = [
+  "#main",
+  "#content",
+  "#main-content",
+  "#article",
+  "#post",
+  "#page-content",
+  "main",
+  "article",
+  "[role='main']",
+  ".main-content",
+  ".content",
+  ".post-content",
+  ".article-content",
+  ".entry-content",
+  ".page-content",
+  ".article-body",
+  ".post-body",
+  ".story-content",
+  ".blog-content",
+];
+
+const AD_SELECTORS = [
+  "ins.adsbygoogle",
+  ".google-ad",
+  ".adsense",
+  "[data-ad]",
+  "[data-ads]",
+  "[data-ad-slot]",
+  "[data-ad-client]",
+  ".ad-container",
+  ".ad-wrapper",
+  ".advertisement",
+  ".sponsored-content",
+  "img[width='1'][height='1']",
+  "img[src*='pixel']",
+  "img[src*='tracking']",
+  "img[src*='analytics']",
+];
+
+function getLinkDensity(element: Element): number {
+  const text = element.textContent || "";
+  const textLength = text.trim().length;
+  if (textLength === 0) {
+    return 1;
+  }
+
+  let linkLength = 0;
+  element.querySelectorAll("a").forEach((link) => {
+    linkLength += (link.textContent || "").trim().length;
+  });
+
+  return linkLength / textLength;
+}
+
+function getContentScore(element: Element): number {
+  let score = 0;
+  const text = element.textContent || "";
+  const textLength = text.trim().length;
+
+  score += Math.min(textLength / 100, 50);
+  score += element.querySelectorAll("p").length * 3;
+  score += element.querySelectorAll("h1, h2, h3, h4, h5, h6").length * 2;
+  score += element.querySelectorAll("img").length;
+
+  score -= element.querySelectorAll("a").length * 0.5;
+  score -= element.querySelectorAll("li").length * 0.2;
+
+  const linkDensity = getLinkDensity(element);
+  if (linkDensity > 0.5) {
+    score -= 30;
+  } else if (linkDensity > 0.3) {
+    score -= 15;
+  }
+
+  const className = typeof element.className === "string" ? element.className : "";
+  const classAndId = `${className} ${element.id || ""}`;
+  if (/article|content|post|body|main|entry/i.test(classAndId)) {
+    score += 25;
+  }
+  if (/comment|sidebar|footer|nav|menu|header|widget|ad/i.test(classAndId)) {
+    score -= 25;
+  }
+
+  return score;
+}
+
+function looksLikeNavigation(element: Element): boolean {
+  const linkDensity = getLinkDensity(element);
+  if (linkDensity > 0.5) {
+    return true;
+  }
+
+  const listItems = element.querySelectorAll("li");
+  const links = element.querySelectorAll("a");
+  return listItems.length > 5 && links.length > listItems.length * 0.8;
+}
+
+function removeElements(document: Document, selectors: string[]): void {
+  for (const selector of selectors) {
+    try {
+      document.querySelectorAll(selector).forEach((element) => element.remove());
+    } catch {
+      // Ignore unsupported selectors.
+    }
+  }
+}
+
+function removeWithProtection(
+  document: Document,
+  selectorsToRemove: string[],
+  protectedSelectors: string[],
+): void {
+  for (const selector of selectorsToRemove) {
+    try {
+      document.querySelectorAll(selector).forEach((element) => {
+        const isProtected = protectedSelectors.some((protectedSelector) => {
+          try {
+            return element.matches(protectedSelector);
+          } catch {
+            return false;
+          }
+        });
+
+        if (isProtected) {
+          return;
+        }
+
+        const containsProtected = protectedSelectors.some((protectedSelector) => {
+          try {
+            return element.querySelector(protectedSelector) !== null;
+          } catch {
+            return false;
+          }
+        });
+
+        if (containsProtected) {
+          return;
+        }
+
+        element.remove();
+      });
+    } catch {
+      // Ignore unsupported selectors.
+    }
+  }
+}
+
+function isValidContent(element: Element | null): element is Element {
+  if (!element) {
+    return false;
+  }
+  const text = element.textContent || "";
+  if (text.trim().length < 100) {
+    return false;
+  }
+  return !looksLikeNavigation(element);
+}
+
+function findMainContent(document: Document): Element | null {
+  const main = document.querySelector("main");
+  if (isValidContent(main) && getLinkDensity(main) < 0.4) {
+    return main;
+  }
+
+  const roleMain = document.querySelector('[role="main"]');
+  if (isValidContent(roleMain) && getLinkDensity(roleMain) < 0.4) {
+    return roleMain;
+  }
+
+  const articles = document.querySelectorAll("article");
+  if (articles.length === 1 && isValidContent(articles[0] ?? null)) {
+    return articles[0] ?? null;
+  }
+
+  const contentSelectors = [
+    "#content",
+    "#main-content",
+    "#main",
+    ".content",
+    ".main-content",
+    ".post-content",
+    ".article-content",
+    ".entry-content",
+    ".page-content",
+    ".article-body",
+    ".post-body",
+    ".story-content",
+    ".blog-content",
+  ];
+
+  for (const selector of contentSelectors) {
+    try {
+      const element = document.querySelector(selector);
+      if (isValidContent(element) && getLinkDensity(element) < 0.4) {
+        return element;
+      }
+    } catch {
+      // Ignore invalid selectors.
+    }
+  }
+
+  const candidates: Array<{ element: Element; score: number }> = [];
+  document.querySelectorAll("div, section, article").forEach((element) => {
+    const text = element.textContent || "";
+    if (text.trim().length < 200) {
+      return;
+    }
+
+    const score = getContentScore(element);
+    if (score > 0) {
+      candidates.push({ element, score });
+    }
+  });
+
+  candidates.sort((left, right) => right.score - left.score);
+  if ((candidates[0]?.score ?? 0) > 20) {
+    return candidates[0]?.element ?? null;
+  }
+
+  return null;
+}
+
+function removeBase64ImagesFromDocument(document: Document): void {
+  document.querySelectorAll("img[src^='data:']").forEach((element) => element.remove());
+
+  document.querySelectorAll("[style*='data:image']").forEach((element) => {
+    const style = element.getAttribute("style");
+    if (!style) {
+      return;
+    }
+
+    const cleanedStyle = style.replace(
+      /background(-image)?:\s*url\([^)]*data:image[^)]*\)[^;]*;?/gi,
+      "",
+    );
+
+    if (cleanedStyle.trim()) {
+      element.setAttribute("style", cleanedStyle);
+    } else {
+      element.removeAttribute("style");
+    }
+  });
+
+  document
+    .querySelectorAll("source[src^='data:'], source[srcset*='data:']")
+    .forEach((element) => element.remove());
+}
+
+function makeAbsoluteUrl(value: string, baseUrl: string): string | null {
+  try {
+    return new URL(value, baseUrl).toString();
+  } catch {
+    return null;
+  }
+}
+
+function convertRelativeUrls(document: Document, baseUrl: string): void {
+  document.querySelectorAll("[src]").forEach((element) => {
+    const src = element.getAttribute("src");
+    if (!src || src.startsWith("http") || src.startsWith("//") || src.startsWith("data:")) {
+      return;
+    }
+
+    const absolute = makeAbsoluteUrl(src, baseUrl);
+    if (absolute) {
+      element.setAttribute("src", absolute);
+    }
+  });
+
+  document.querySelectorAll("[href]").forEach((element) => {
+    const href = element.getAttribute("href");
+    if (
+      !href ||
+      href.startsWith("http") ||
+      href.startsWith("//") ||
+      href.startsWith("#") ||
+      href.startsWith("mailto:") ||
+      href.startsWith("tel:") ||
+      href.startsWith("javascript:")
+    ) {
+      return;
+    }
+
+    const absolute = makeAbsoluteUrl(href, baseUrl);
+    if (absolute) {
+      element.setAttribute("href", absolute);
+    }
+  });
+}
+
+function removeComments(document: Document): void {
+  const walker = document.createTreeWalker(document, document.defaultView?.NodeFilter.SHOW_COMMENT ?? 128);
+  const comments: Comment[] = [];
+  while (walker.nextNode()) {
+    comments.push(walker.currentNode as Comment);
+  }
+  comments.forEach((comment) => comment.parentNode?.removeChild(comment));
+}
+
+export function cleanHtml(
+  html: string,
+  baseUrl: string,
+  options: CleanHtmlOptions = {},
+): string {
+  const {
+    removeAds = true,
+    removeBase64Images = true,
+    onlyMainContent = true,
+    includeSelectors,
+    excludeSelectors,
+  } = options;
+
+  const dom = new JSDOM(html, { url: baseUrl });
+  const { document } = dom.window;
+
+  removeElements(document, ALWAYS_REMOVE_SELECTORS);
+  removeElements(document, OVERLAY_SELECTORS);
+
+  if (removeAds) {
+    removeElements(document, AD_SELECTORS);
+  }
+
+  if (excludeSelectors?.length) {
+    removeElements(document, excludeSelectors);
+  }
+
+  if (onlyMainContent) {
+    removeWithProtection(document, NAVIGATION_SELECTORS, FORCE_INCLUDE_SELECTORS);
+
+    const mainContent = findMainContent(document);
+    if (mainContent && document.body) {
+      const clone = mainContent.cloneNode(true);
+      document.body.innerHTML = "";
+      document.body.appendChild(clone);
+    }
+  }
+
+  if (includeSelectors?.length && document.body) {
+    const matchedElements: Element[] = [];
+    for (const selector of includeSelectors) {
+      try {
+        document.querySelectorAll(selector).forEach((element) => {
+          matchedElements.push(element.cloneNode(true) as Element);
+        });
+      } catch {
+        // Ignore invalid selectors.
+      }
+    }
+
+    if (matchedElements.length > 0) {
+      document.body.innerHTML = "";
+      matchedElements.forEach((element) => document.body?.appendChild(element));
+    }
+  }
+
+  if (removeBase64Images) {
+    removeBase64ImagesFromDocument(document);
+  }
+
+  removeComments(document);
+  convertRelativeUrls(document, baseUrl);
+
+  return document.documentElement.outerHTML || html;
+}

packages/baoyu-fetch/src/extract/html-extractor.ts

@@ -0,0 +1,83 @@
+import { Readability } from "@mozilla/readability";
+import { JSDOM } from "jsdom";
+import type { ExtractedDocument } from "./document";
+
+function getMetaContent(document: Document, selectors: string[]): string | undefined {
+  for (const selector of selectors) {
+    const value = document.querySelector(selector)?.getAttribute("content")?.trim();
+    if (value) {
+      return value;
+    }
+  }
+  return undefined;
+}
+
+export function extractDocumentFromHtml(input: {
+  url: string;
+  html: string;
+  adapter?: string;
+}): ExtractedDocument {
+  const dom = new JSDOM(input.html, { url: input.url });
+  const document = dom.window.document;
+
+  const canonicalUrl =
+    document.querySelector('link[rel="canonical"]')?.getAttribute("href")?.trim() ??
+    getMetaContent(document, ['meta[property="og:url"]']);
+
+  const siteName = getMetaContent(document, [
+    'meta[property="og:site_name"]',
+    'meta[name="application-name"]',
+  ]);
+
+  const metadataAuthor = getMetaContent(document, [
+    'meta[name="author"]',
+    'meta[property="article:author"]',
+    'meta[name="twitter:creator"]',
+  ]);
+
+  const publishedAt = getMetaContent(document, [
+    'meta[property="article:published_time"]',
+    'meta[name="pubdate"]',
+    'meta[name="date"]',
+    'meta[itemprop="datePublished"]',
+  ]);
+
+  const article = new Readability(document).parse();
+  const title =
+    article?.title?.trim() ||
+    getMetaContent(document, ['meta[property="og:title"]']) ||
+    document.title.trim() ||
+    undefined;
+
+  const summary =
+    article?.excerpt?.trim() ||
+    getMetaContent(document, [
+      'meta[name="description"]',
+      'meta[property="og:description"]',
+      'meta[name="twitter:description"]',
+    ]);
+
+  const contentHtml =
+    article?.content?.trim() ||
+    document.querySelector("main")?.innerHTML?.trim() ||
+    document.body?.innerHTML?.trim() ||
+    "";
+
+  const author = article?.byline?.trim() || metadataAuthor;
+
+  return {
+    url: input.url,
+    canonicalUrl,
+    title,
+    author,
+    siteName,
+    publishedAt,
+    summary,
+    adapter: input.adapter ?? "generic",
+    metadata: {
+      language: document.documentElement.lang || undefined,
+    },
+    content: contentHtml ? [{ type: "html", html: contentHtml }] : [],
+  };
+}
+

packages/baoyu-fetch/src/extract/html-to-markdown.ts

@@ -0,0 +1,758 @@
+import { Readability } from "@mozilla/readability";
+import { Defuddle } from "defuddle/node";
+import { JSDOM, VirtualConsole } from "jsdom";
+import TurndownService from "turndown";
+import { gfm } from "turndown-plugin-gfm";
+import { collectMediaFromMarkdown } from "../media/markdown-media";
+import type { MediaAsset } from "../media/types";
+import { cleanHtml } from "./html-cleaner";
+
+export interface HtmlConversionMetadata {
+  url: string;
+  canonicalUrl?: string;
+  siteName?: string;
+  title?: string;
+  summary?: string;
+  author?: string;
+  publishedAt?: string;
+  coverImage?: string;
+  language?: string;
+  capturedAt: string;
+}
+
+export interface ConvertHtmlToMarkdownOptions {
+  enableRemoteMarkdownFallback?: boolean;
+  preserveBase64Images?: boolean;
+}
+
+export interface HtmlToMarkdownResult {
+  metadata: HtmlConversionMetadata;
+  markdown: string;
+  rawHtml: string;
+  cleanedHtml: string;
+  media: MediaAsset[];
+  conversionMethod: string;
+  fallbackReason?: string;
+}
+
+type JsonObject = Record<string, unknown>;
+
+const MIN_CONTENT_LENGTH = 120;
+const DEFUDDLE_API_ORIGIN = "https://defuddle.md";
+const LOCAL_FALLBACK_SCORE_DELTA = 120;
+const REMOTE_FALLBACK_SCORE_DELTA = 20;
+const LOW_QUALITY_MARKERS = [
+  /Join The Conversation/i,
+  /One Community\. Many Voices/i,
+  /Read our community guidelines/i,
+  /Create a free account to share your thoughts/i,
+  /Become a Forbes Member/i,
+  /Subscribe to trusted journalism/i,
+  /\bComments\b/i,
+];
+
+const ARTICLE_TYPES = new Set([
+  "Article",
+  "NewsArticle",
+  "BlogPosting",
+  "WebPage",
+  "ReportageNewsArticle",
+]);
+
+const turndown = new TurndownService({
+  headingStyle: "atx",
+  bulletListMarker: "-",
+  codeBlockStyle: "fenced",
+}) as TurndownService & {
+  remove(selectors: string[]): void;
+  addRule(
+    key: string,
+    rule: {
+      filter: string | ((node: Node) => boolean);
+      replacement: (content: string) => string;
+    },
+  ): void;
+};
+
+turndown.use(gfm);
+turndown.remove(["script", "style", "iframe", "noscript", "template", "svg", "path"]);
+turndown.addRule("collapseFigure", {
+  filter: "figure",
+  replacement(content: string) {
+    return `\n\n${content.trim()}\n\n`;
+  },
+});
+turndown.addRule("dropInvisibleAnchors", {
+  filter(node: Node) {
+    return (
+      node.nodeName === "A" &&
+      !(node as Element).textContent?.trim() &&
+      !(node as Element).querySelector("img, video, picture, source")
+    );
+  },
+  replacement() {
+    return "";
+  },
+});
+
+function pickString(...values: unknown[]): string | undefined {
+  for (const value of values) {
+    if (typeof value !== "string") {
+      continue;
+    }
+    const trimmed = value.trim();
+    if (trimmed) {
+      return trimmed;
+    }
+  }
+  return undefined;
+}
+
+function normalizeMarkdown(markdown: string): string {
+  return markdown
+    .replace(/\r\n/g, "\n")
+    .replace(/[ \t]+\n/g, "\n")
+    .replace(/\n{3,}/g, "\n\n")
+    .trim();
+}
+
+function stripWrappingQuotes(value: string): string {
+  const trimmed = value.trim();
+  if (
+    (trimmed.startsWith('"') && trimmed.endsWith('"')) ||
+    (trimmed.startsWith("'") && trimmed.endsWith("'"))
+  ) {
+    return trimmed.slice(1, -1).trim();
+  }
+  return trimmed;
+}
+
+function stripMarkdownFrontmatter(markdown: string): string {
+  return markdown.replace(/^\uFEFF?---\n[\s\S]*?\n---(?:\n|$)/, "").trim();
+}
+
+function cleanMarkdownTitle(value: string): string | undefined {
+  const cleaned = stripWrappingQuotes(
+    value
+      .replace(/\s+#+\s*$/, "")
+      .replace(/!\[[^\]]*\]\([^)]+\)/g, "")
+      .replace(/\[([^\]]+)\]\([^)]+\)/g, "$1")
+      .replace(/[*_`~]/g, "")
+      .trim(),
+  );
+
+  return cleaned || undefined;
+}
+
+export function extractTitleFromMarkdownDocument(markdown: string): string | undefined {
+  const normalized = markdown.replace(/\r\n/g, "\n").trim();
+  if (!normalized) {
+    return undefined;
+  }
+
+  const frontmatterMatch = normalized.match(/^\uFEFF?---\n([\s\S]*?)\n---(?:\n|$)/);
+  if (frontmatterMatch) {
+    for (const line of frontmatterMatch[1].split("\n")) {
+      const match = line.match(/^title:\s*(.+?)\s*$/i);
+      if (!match) {
+        continue;
+      }
+
+      const title = cleanMarkdownTitle(match[1]);
+      if (title) {
+        return title;
+      }
+    }
+  }
+
+  const body = stripMarkdownFrontmatter(normalized);
+  const headingMatch = body.match(/^#{1,6}\s+(.+)$/m);
+  if (!headingMatch) {
+    return undefined;
+  }
+
+  return cleanMarkdownTitle(headingMatch[1]);
+}
+
+function trimKnownBoilerplate(markdown: string): string {
+  const normalized = normalizeMarkdown(markdown);
+  const lines = normalized.split("\n");
+
+  while (lines.length > 0) {
+    const lastLine = lines[lines.length - 1]?.trim();
+    if (!lastLine) {
+      lines.pop();
+      continue;
+    }
+
+    if (/^继续滑动看下一个$/.test(lastLine) || /^轻触阅读原文$/.test(lastLine)) {
+      lines.pop();
+      continue;
+    }
+
+    break;
+  }
+
+  return normalizeMarkdown(lines.join("\n"));
+}
+
+function buildDefuddleApiUrl(targetUrl: string): string {
+  return `${DEFUDDLE_API_ORIGIN}/${encodeURIComponent(targetUrl)}`;
+}
+
+async function fetchDefuddleApiMarkdown(
+  targetUrl: string,
+): Promise<{ markdown: string; title?: string }> {
+  const response = await fetch(buildDefuddleApiUrl(targetUrl), {
+    headers: {
+      accept: "text/markdown,text/plain;q=0.9,*/*;q=0.1",
+    },
+    redirect: "follow",
+  });
+
+  if (!response.ok) {
+    throw new Error(`defuddle.md returned ${response.status} ${response.statusText}`);
+  }
+
+  const rawMarkdown = (await response.text()).replace(/\r\n/g, "\n").trim();
+  if (!rawMarkdown) {
+    throw new Error("defuddle.md returned empty markdown");
+  }
+
+  const title = extractTitleFromMarkdownDocument(rawMarkdown);
+  const markdown = trimKnownBoilerplate(stripMarkdownFrontmatter(rawMarkdown));
+  if (!markdown) {
+    throw new Error("defuddle.md returned empty markdown");
+  }
+
+  return {
+    markdown,
+    title,
+  };
+}
+
+function sanitizeHtmlFragment(html: string): string {
+  const dom = new JSDOM(`<div id="__root">${html}</div>`);
+  const root = dom.window.document.querySelector("#__root");
+  if (!root) {
+    return html;
+  }
+
+  for (const selector of ["script", "style", "iframe", "noscript", "template", "svg", "path"]) {
+    root.querySelectorAll(selector).forEach((element) => element.remove());
+  }
+
+  return root.innerHTML;
+}
+
+function extractTextFromHtml(html: string): string {
+  const dom = new JSDOM(`<!doctype html><html><body>${html}</body></html>`);
+  const { document } = dom.window;
+  for (const selector of ["script", "style", "noscript", "template", "iframe", "svg", "path"]) {
+    document.querySelectorAll(selector).forEach((element) => element.remove());
+  }
+  return document.body?.textContent?.replace(/\s+/g, " ").trim() ?? "";
+}
+
+function getMetaContent(document: Document, names: string[]): string | undefined {
+  for (const name of names) {
+    const element =
+      document.querySelector(`meta[name="${name}"]`) ??
+      document.querySelector(`meta[property="${name}"]`);
+    const content = element?.getAttribute("content")?.trim();
+    if (content) {
+      return content;
+    }
+  }
+  return undefined;
+}
+
+function normalizeLanguageTag(value: string | null | undefined): string | undefined {
+  if (!value) {
+    return undefined;
+  }
+
+  const trimmed = value.trim();
+  if (!trimmed) {
+    return undefined;
+  }
+
+  const primary = trimmed.split(/[,\s;]/, 1)[0]?.trim();
+  if (!primary) {
+    return undefined;
+  }
+
+  return primary.replace(/_/g, "-");
+}
+
+function flattenJsonLdItems(data: unknown): JsonObject[] {
+  if (!data || typeof data !== "object") {
+    return [];
+  }
+
+  if (Array.isArray(data)) {
+    return data.flatMap(flattenJsonLdItems);
+  }
+
+  const item = data as JsonObject;
+  if (Array.isArray(item["@graph"])) {
+    return (item["@graph"] as unknown[]).flatMap(flattenJsonLdItems);
+  }
+
+  return [item];
+}
+
+function parseJsonLdScripts(document: Document): JsonObject[] {
+  const results: JsonObject[] = [];
+  document.querySelectorAll("script[type='application/ld+json']").forEach((script) => {
+    try {
+      const data = JSON.parse(script.textContent ?? "");
+      results.push(...flattenJsonLdItems(data));
+    } catch {
+      // Ignore malformed json-ld blocks.
+    }
+  });
+  return results;
+}
+
+function extractAuthorFromJsonLd(authorData: unknown): string | undefined {
+  if (typeof authorData === "string") {
+    return authorData.trim() || undefined;
+  }
+
+  if (!authorData || typeof authorData !== "object") {
+    return undefined;
+  }
+
+  if (Array.isArray(authorData)) {
+    return authorData
+      .map((author) => extractAuthorFromJsonLd(author))
+      .filter((value): value is string => Boolean(value))
+      .join(", ") || undefined;
+  }
+
+  const author = authorData as JsonObject;
+  return pickString(author.name);
+}
+
+function extractPrimaryJsonLdMeta(document: Document): Partial<HtmlConversionMetadata> {
+  for (const item of parseJsonLdScripts(document)) {
+    const type = Array.isArray(item["@type"]) ? item["@type"][0] : item["@type"];
+    if (typeof type !== "string" || !ARTICLE_TYPES.has(type)) {
+      continue;
+    }
+
+    return {
+      title: pickString(item.headline, item.name),
+      summary: pickString(item.description),
+      author: extractAuthorFromJsonLd(item.author),
+      publishedAt: pickString(item.datePublished, item.dateCreated),
+      coverImage: pickString(
+        item.image,
+        (item.image as JsonObject | undefined)?.url,
+        Array.isArray(item.image) ? item.image[0] : undefined,
+      ),
+    };
+  }
+
+  return {};
+}
+
+function extractPageMetadata(
+  html: string,
+  url: string,
+  capturedAt: string,
+): HtmlConversionMetadata {
+  const dom = new JSDOM(html, { url });
+  const { document } = dom.window;
+  const jsonLd = extractPrimaryJsonLdMeta(document);
+
+  return {
+    url,
+    canonicalUrl:
+      document.querySelector('link[rel="canonical"]')?.getAttribute("href")?.trim() ??
+      getMetaContent(document, ["og:url"]),
+    siteName: pickString(
+      getMetaContent(document, ["og:site_name"]),
+      document.querySelector('meta[name="application-name"]')?.getAttribute("content"),
+    ),
+    title: pickString(
+      getMetaContent(document, ["og:title", "twitter:title"]),
+      jsonLd.title,
+      document.querySelector("h1")?.textContent,
+      document.title,
+    ),
+    summary: pickString(
+      getMetaContent(document, ["description", "og:description", "twitter:description"]),
+      jsonLd.summary,
+    ),
+    author: pickString(
+      getMetaContent(document, ["author", "article:author", "twitter:creator"]),
+      jsonLd.author,
+    ),
+    publishedAt: pickString(
+      document.querySelector("time[datetime]")?.getAttribute("datetime"),
+      getMetaContent(document, ["article:published_time", "datePublished", "publishdate", "date"]),
+      jsonLd.publishedAt,
+    ),
+    coverImage: pickString(
+      getMetaContent(document, ["og:image", "twitter:image", "twitter:image:src"]),
+      jsonLd.coverImage,
+    ),
+    language: pickString(
+      normalizeLanguageTag(document.documentElement.getAttribute("lang")),
+      normalizeLanguageTag(
+        pickString(
+          getMetaContent(document, ["language", "content-language", "og:locale"]),
+          document.querySelector("meta[http-equiv='content-language']")?.getAttribute("content"),
+        ),
+      ),
+    ),
+    capturedAt,
+  };
+}
+
+function isMarkdownUsable(markdown: string, html: string): boolean {
+  const normalized = normalizeMarkdown(markdown);
+  if (!normalized) {
+    return false;
+  }
+
+  const htmlTextLength = extractTextFromHtml(html).length;
+  if (htmlTextLength < MIN_CONTENT_LENGTH) {
+    return true;
+  }
+
+  if (normalized.length >= 80) {
+    return true;
+  }
+
+  return normalized.length >= Math.min(200, Math.floor(htmlTextLength * 0.2));
+}
+
+function countMarkerHits(markdown: string, markers: RegExp[]): number {
+  let hits = 0;
+  for (const marker of markers) {
+    if (marker.test(markdown)) {
+      hits += 1;
+    }
+  }
+  return hits;
+}
+
+function countUsefulParagraphs(markdown: string): number {
+  const paragraphs = normalizeMarkdown(markdown).split(/\n{2,}/);
+  let count = 0;
+
+  for (const paragraph of paragraphs) {
+    const trimmed = paragraph.trim();
+    if (!trimmed) {
+      continue;
+    }
+    if (/^!?\[[^\]]*\]\([^)]+\)$/.test(trimmed)) {
+      continue;
+    }
+    if (/^#{1,6}\s+/.test(trimmed)) {
+      continue;
+    }
+    if ((trimmed.match(/\b[\p{L}\p{N}']+\b/gu) || []).length < 8) {
+      continue;
+    }
+    count += 1;
+  }
+
+  return count;
+}
+
+function scoreMarkdownQuality(markdown: string): number {
+  const normalized = normalizeMarkdown(markdown);
+  const wordCount = (normalized.match(/\b[\p{L}\p{N}']+\b/gu) || []).length;
+  const usefulParagraphs = countUsefulParagraphs(normalized);
+  const headingCount = (normalized.match(/^#{1,6}\s+/gm) || []).length;
+  const markerHits = countMarkerHits(normalized, LOW_QUALITY_MARKERS);
+  return Math.min(wordCount, 4000) + usefulParagraphs * 40 + headingCount * 10 - markerHits * 180;
+}
+
+function shouldCompareWithFallback(markdown: string): boolean {
+  const normalized = normalizeMarkdown(markdown);
+  return countMarkerHits(normalized, LOW_QUALITY_MARKERS) > 0 || countUsefulParagraphs(normalized) < 6;
+}
+
+function hasMeaningfulMarkdownStructure(markdown: string): boolean {
+  const normalized = normalizeMarkdown(markdown);
+  if (!normalized) {
+    return false;
+  }
+
+  return (
+    countUsefulParagraphs(normalized) > 0 ||
+    /^#{1,6}\s+/m.test(normalized) ||
+    /^[-*]\s+/m.test(normalized) ||
+    /^\d+\.\s+/m.test(normalized) ||
+    /!\[[^\]]*\]\([^)]+\)/.test(normalized)
+  );
+}
+
+function shouldTryRemoteMarkdownFallback(
+  markdown: string,
+  html: string,
+  options: ConvertHtmlToMarkdownOptions,
+): boolean {
+  if (!options.enableRemoteMarkdownFallback) {
+    return false;
+  }
+
+  return !isMarkdownUsable(markdown, html) || shouldCompareWithFallback(markdown);
+}
+
+function shouldPreferRemoteMarkdown(
+  current: HtmlToMarkdownResult,
+  remote: HtmlToMarkdownResult,
+  html: string,
+): boolean {
+  if (!isMarkdownUsable(current.markdown, html)) {
+    return true;
+  }
+
+  if (!hasMeaningfulMarkdownStructure(current.markdown) && hasMeaningfulMarkdownStructure(remote.markdown)) {
+    return true;
+  }
+
+  return scoreMarkdownQuality(remote.markdown) > scoreMarkdownQuality(current.markdown) + REMOTE_FALLBACK_SCORE_DELTA;
+}
+
+function buildRemoteFallbackReason(current: HtmlToMarkdownResult, html: string): string {
+  if (!isMarkdownUsable(current.markdown, html)) {
+    return current.fallbackReason
+      ? `Used defuddle.md markdown fallback after local extraction failed: ${current.fallbackReason}`
+      : "Used defuddle.md markdown fallback after local extraction returned empty or incomplete markdown";
+  }
+
+  return "defuddle.md produced higher-quality markdown than local extraction";
+}
+
+async function tryDefuddleConversion(
+  html: string,
+  url: string,
+  baseMetadata: HtmlConversionMetadata,
+): Promise<{ ok: true; result: HtmlToMarkdownResult } | { ok: false; reason: string }> {
+  try {
+    const virtualConsole = new VirtualConsole();
+    virtualConsole.on("jsdomError", (error: Error & { type?: string }) => {
+      if (error.type === "css parsing" || /Could not parse CSS stylesheet/i.test(error.message)) {
+        return;
+      }
+    });
+
+    const dom = new JSDOM(html, { url, virtualConsole });
+    const result = await Defuddle(dom, url, { markdown: true });
+    const markdown = trimKnownBoilerplate(result.content || "");
+
+    if (!isMarkdownUsable(markdown, html)) {
+      return { ok: false, reason: "Defuddle returned empty or incomplete markdown" };
+    }
+
+    const metadata: HtmlConversionMetadata = {
+      ...baseMetadata,
+      title: pickString(result.title, baseMetadata.title),
+      summary: pickString(result.description, baseMetadata.summary),
+      author: pickString(result.author, baseMetadata.author),
+      publishedAt: pickString(result.published, baseMetadata.publishedAt),
+      coverImage: pickString(result.image, baseMetadata.coverImage),
+      language: pickString(result.language, baseMetadata.language),
+    };
+
+    return {
+      ok: true,
+      result: {
+        metadata,
+        markdown,
+        rawHtml: html,
+        cleanedHtml: html,
+        media: collectMediaFromMarkdown(markdown).concat(
+          metadata.coverImage
+            ? [{ url: metadata.coverImage, kind: "image", role: "cover" as const }]
+            : [],
+        ),
+        conversionMethod: "defuddle",
+      },
+    };
+  } catch (error) {
+    return {
+      ok: false,
+      reason: error instanceof Error ? error.message : String(error),
+    };
+  }
+}
+
+async function tryDefuddleApiConversion(
+  html: string,
+  url: string,
+  baseMetadata: HtmlConversionMetadata,
+): Promise<{ ok: true; result: HtmlToMarkdownResult } | { ok: false; reason: string }> {
+  try {
+    const result = await fetchDefuddleApiMarkdown(url);
+    const markdown = result.markdown;
+
+    if (!isMarkdownUsable(markdown, html) && scoreMarkdownQuality(markdown) < 80) {
+      return { ok: false, reason: "defuddle.md returned empty or incomplete markdown" };
+    }
+
+    const metadata: HtmlConversionMetadata = {
+      ...baseMetadata,
+      title: pickString(result.title, baseMetadata.title),
+    };
+
+    return {
+      ok: true,
+      result: {
+        metadata,
+        markdown,
+        rawHtml: html,
+        cleanedHtml: html,
+        media: collectMediaFromMarkdown(markdown).concat(
+          metadata.coverImage
+            ? [{ url: metadata.coverImage, kind: "image", role: "cover" as const }]
+            : [],
+        ),
+        conversionMethod: "defuddle-api",
+      },
+    };
+  } catch (error) {
+    return {
+      ok: false,
+      reason: error instanceof Error ? error.message : String(error),
+    };
+  }
+}
+
+function convertHtmlFragmentToMarkdown(html: string): string {
+  if (!html.trim()) {
+    return "";
+  }
+
+  try {
+    return turndown.turndown(sanitizeHtmlFragment(html));
+  } catch {
+    return "";
+  }
+}
+
+function fallbackPlainText(html: string): string {
+  return trimKnownBoilerplate(extractTextFromHtml(html));
+}
+
+function convertWithReadability(
+  rawHtml: string,
+  cleanedHtml: string,
+  url: string,
+  baseMetadata: HtmlConversionMetadata,
+): HtmlToMarkdownResult {
+  const dom = new JSDOM(cleanedHtml, { url });
+  const document = dom.window.document;
+  const article = new Readability(document).parse();
+
+  const contentHtml =
+    article?.content?.trim() ??
+    document.querySelector("main")?.innerHTML?.trim() ??
+    document.body?.innerHTML?.trim() ??
+    "";
+
+  let markdown = contentHtml ? convertHtmlFragmentToMarkdown(contentHtml) : "";
+  if (!markdown) {
+    markdown = fallbackPlainText(cleanedHtml);
+  }
+
+  const metadata: HtmlConversionMetadata = {
+    ...baseMetadata,
+    title: pickString(article?.title, baseMetadata.title),
+    summary: pickString(article?.excerpt, baseMetadata.summary),
+    author: pickString(article?.byline, baseMetadata.author),
+  };
+
+  const media = collectMediaFromMarkdown(markdown);
+  if (metadata.coverImage) {
+    media.unshift({
+      url: metadata.coverImage,
+      kind: "image",
+      role: "cover",
+    });
+  }
+
+  return {
+    metadata,
+    markdown: trimKnownBoilerplate(markdown),
+    rawHtml,
+    cleanedHtml,
+    media,
+    conversionMethod: article?.content ? "legacy:readability" : "legacy:body",
+  };
+}
+
+export async function convertHtmlToMarkdown(
+  html: string,
+  url: string,
+  options: ConvertHtmlToMarkdownOptions = {},
+): Promise<HtmlToMarkdownResult> {
+  const capturedAt = new Date().toISOString();
+  const baseMetadata = extractPageMetadata(html, url, capturedAt);
+
+  let cleanedHtml = html;
+  try {
+    cleanedHtml = cleanHtml(html, url, {
+      removeBase64Images: !options.preserveBase64Images,
+    });
+  } catch {
+    cleanedHtml = html;
+  }
+
+  let selectedResult: HtmlToMarkdownResult;
+  const defuddleResult = await tryDefuddleConversion(cleanedHtml, url, baseMetadata);
+  if (defuddleResult.ok) {
+    if (shouldCompareWithFallback(defuddleResult.result.markdown)) {
+      const fallbackResult = convertWithReadability(html, cleanedHtml, url, baseMetadata);
+      if (
+        scoreMarkdownQuality(fallbackResult.markdown) >
+        scoreMarkdownQuality(defuddleResult.result.markdown) + LOCAL_FALLBACK_SCORE_DELTA
+      ) {
+        selectedResult = {
+          ...fallbackResult,
+          fallbackReason: "Readability/Turndown produced higher-quality markdown than Defuddle",
+        };
+      } else {
+        selectedResult = {
+          ...defuddleResult.result,
+          rawHtml: html,
+          cleanedHtml,
+        };
+      }
+    } else {
+      selectedResult = {
+        ...defuddleResult.result,
+        rawHtml: html,
+        cleanedHtml,
+      };
+    }
+  } else {
+    selectedResult = {
+      ...convertWithReadability(html, cleanedHtml, url, baseMetadata),
+      fallbackReason: defuddleResult.reason,
+    };
+  }
+
+  if (!shouldTryRemoteMarkdownFallback(selectedResult.markdown, cleanedHtml, options)) {
+    return selectedResult;
+  }
+
+  const remoteDefuddleResult = await tryDefuddleApiConversion(cleanedHtml, url, baseMetadata);
+  if (!remoteDefuddleResult.ok || !shouldPreferRemoteMarkdown(selectedResult, remoteDefuddleResult.result, cleanedHtml)) {
+    return selectedResult;
+  }
+
+  return {
+    ...remoteDefuddleResult.result,
+    rawHtml: html,
+    cleanedHtml,
+    fallbackReason: buildRemoteFallbackReason(selectedResult, cleanedHtml),
+  };
+}

packages/baoyu-fetch/src/extract/markdown-renderer.ts

@@ -0,0 +1,169 @@
+import TurndownService from "turndown";
+import { gfm } from "turndown-plugin-gfm";
+import { normalizeMarkdownMediaLinks } from "../media/markdown-media";
+import type { ContentBlock, ExtractedDocument } from "./document";
+
+const turndownService = new TurndownService({
+  codeBlockStyle: "fenced",
+  headingStyle: "atx",
+  bulletListMarker: "-",
+});
+
+turndownService.use(gfm);
+
+function renderBlock(block: ContentBlock): string {
+  switch (block.type) {
+    case "paragraph":
+      return block.text.trim();
+    case "heading":
+      return `${"#".repeat(Math.min(Math.max(block.depth, 1), 6))} ${block.text.trim()}`;
+    case "list":
+      return block.items
+        .map((item, index) => (block.ordered ? `${index + 1}. ${item.trim()}` : `- ${item.trim()}`))
+        .join("\n");
+    case "quote":
+      return block.text
+        .split("\n")
+        .map((line) => `> ${line}`)
+        .join("\n");
+    case "code":
+      return `\`\`\`${block.language ?? ""}\n${block.code.trimEnd()}\n\`\`\``;
+    case "image":
+      return `![${block.alt ?? ""}](${block.url})`;
+    case "html":
+      return turndownService.turndown(block.html).trim();
+    case "markdown":
+      return block.markdown.trim();
+  }
+}
+
+function isDefinedValue(value: unknown): boolean {
+  return value !== undefined && value !== null && value !== "";
+}
+
+function renderFrontmatterValue(value: unknown): string {
+  if (typeof value === "string") {
+    if (value.includes("\n")) {
+      return `|-\n${value
+        .replace(/\r\n/g, "\n")
+        .split("\n")
+        .map((line) => `  ${line}`)
+        .join("\n")}`;
+    }
+    return JSON.stringify(value);
+  }
+  if (typeof value === "number" || typeof value === "boolean") {
+    return String(value);
+  }
+  return JSON.stringify(value);
+}
+
+function renderFrontmatter(document: ExtractedDocument): string {
+  const fields = new Map<string, unknown>();
+  const preferredOrder = [
+    "title",
+    "url",
+    "requestedUrl",
+    "author",
+    "authorName",
+    "authorUsername",
+    "authorUrl",
+    "coverImage",
+    "siteName",
+    "publishedAt",
+    "summary",
+    "adapter",
+  ];
+
+  fields.set("title", document.title);
+  fields.set("url", document.canonicalUrl ?? document.url);
+  fields.set("requestedUrl", document.requestedUrl ?? document.url);
+  fields.set("author", document.author);
+  fields.set("siteName", document.siteName);
+  fields.set("publishedAt", document.publishedAt);
+  fields.set("summary", document.summary);
+  fields.set("adapter", document.adapter);
+
+  for (const [key, value] of Object.entries(document.metadata ?? {})) {
+    if (!fields.has(key)) {
+      fields.set(key, value);
+    }
+  }
+
+  const orderedKeys = [
+    ...preferredOrder.filter((key) => fields.has(key)),
+    ...Array.from(fields.keys()).filter((key) => !preferredOrder.includes(key)).sort(),
+  ];
+
+  const lines = orderedKeys
+    .map((key) => [key, fields.get(key)] as const)
+    .filter(([, value]) => isDefinedValue(value))
+    .map(([key, value]) => `${key}: ${renderFrontmatterValue(value)}`);
+
+  if (lines.length === 0) {
+    return "";
+  }
+
+  return `---\n${lines.join("\n")}\n---`;
+}
+
+function cleanMarkdown(markdown: string): string {
+  return normalizeMarkdownMediaLinks(markdown.replace(/\n{3,}/g, "\n\n").trim());
+}
+
+function normalizeComparableTitle(value: string): string {
+  return value
+    .trim()
+    .toLowerCase()
+    .replace(/^>\s*/, "")
+    .replace(/^#+\s+/, "")
+    .replace(/(?:\.{3}|…)\s*$/, "");
+}
+
+function bodyStartsWithTitle(body: string, title: string): boolean {
+  const firstMeaningfulLine = body
+    .replace(/\r\n/g, "\n")
+    .split("\n")
+    .map((line) => line.trim())
+    .find((line) => line && !/^!?\[[^\]]*\]\([^)]+\)$/.test(line));
+
+  if (!firstMeaningfulLine) {
+    return false;
+  }
+
+  const comparableTitle = normalizeComparableTitle(title);
+  const comparableFirstLine = normalizeComparableTitle(firstMeaningfulLine);
+  if (!comparableTitle || !comparableFirstLine) {
+    return false;
+  }
+
+  return (
+    comparableFirstLine === comparableTitle ||
+    comparableFirstLine.startsWith(comparableTitle) ||
+    comparableTitle.startsWith(comparableFirstLine)
+  );
+}
+
+export function renderMarkdown(document: ExtractedDocument): string {
+  const sections: string[] = [];
+  const frontmatter = renderFrontmatter(document);
+
+  if (frontmatter) {
+    sections.push(frontmatter);
+  }
+
+  const body = document.content
+    .map((block) => renderBlock(block))
+    .filter(Boolean)
+    .join("\n\n");
+
+  if (document.title && !bodyStartsWithTitle(body, document.title)) {
+    sections.push(`# ${document.title}`);
+  }
+
+  if (body) {
+    sections.push(body);
+  }
+
+  return cleanMarkdown(sections.join("\n\n"));
+}

packages/baoyu-fetch/src/media/default-downloader.ts

@@ -0,0 +1,161 @@
+import path from "node:path";
+import { mkdir, writeFile } from "node:fs/promises";
+import {
+  buildFileName,
+  isDataUri,
+  normalizeContentType,
+  normalizeMediaUrl,
+  resolveExtensionFromContentType,
+  resolveExtensionFromUrl,
+  resolveMediaKind,
+  resolveOutputExtension,
+  toPosixPath,
+} from "./media-utils";
+import type { MediaAsset, MediaDownloadRequest, MediaDownloadResult, MediaKind } from "./types";
+
+const DOWNLOAD_USER_AGENT =
+  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36";
+
+function parseBase64DataUri(rawUrl: string): { contentType: string; bytes: Buffer } | null {
+  const match = rawUrl.match(/^data:([^;,]+);base64,([A-Za-z0-9+/=\s]+)$/i);
+  if (!match?.[1] || !match[2]) {
+    return null;
+  }
+
+  const contentType = normalizeContentType(match[1]);
+  if (!contentType) {
+    return null;
+  }
+
+  try {
+    const bytes = Buffer.from(match[2].replace(/\s+/g, ""), "base64");
+    if (bytes.length === 0) {
+      return null;
+    }
+    return { contentType, bytes };
+  } catch {
+    return null;
+  }
+}
+
+function dedupeMedia(media: MediaAsset[]): MediaAsset[] {
+  const deduped: MediaAsset[] = [];
+  const seen = new Set<string>();
+  for (const item of media) {
+    const normalizedUrl = normalizeMediaUrl(item.url);
+    if (!normalizedUrl || seen.has(normalizedUrl)) {
+      continue;
+    }
+    seen.add(normalizedUrl);
+    deduped.push({
+      ...item,
+      url: normalizedUrl,
+    });
+  }
+  return deduped;
+}
+
+function toRelativePath(fromDir: string, absoluteTarget: string): string {
+  const relative = path.relative(fromDir, absoluteTarget) || path.basename(absoluteTarget);
+  return toPosixPath(relative);
+}
+
+export async function downloadMediaAssets(
+  request: MediaDownloadRequest,
+): Promise<MediaDownloadResult> {
+  const dedupedMedia = dedupeMedia(request.media);
+  const absoluteOutputPath = path.resolve(request.outputPath);
+  const markdownDir = path.dirname(absoluteOutputPath);
+  const baseDir = request.mediaDir ? path.resolve(request.mediaDir) : markdownDir;
+  const replacements: MediaDownloadResult["replacements"] = [];
+
+  let downloadedImages = 0;
+  let downloadedVideos = 0;
+
+  for (const asset of dedupedMedia) {
+    try {
+      let sourceUrl = normalizeMediaUrl(asset.url);
+      let contentType = "";
+      let extension: string | undefined;
+      let kind: MediaKind | undefined;
+      let bytes: Buffer | null = null;
+
+      if (isDataUri(asset.url)) {
+        const parsed = parseBase64DataUri(asset.url);
+        if (!parsed) {
+          request.log.warn(`Skipping unsupported embedded media: ${asset.url.slice(0, 32)}...`);
+          continue;
+        }
+
+        contentType = parsed.contentType;
+        extension =
+          resolveExtensionFromContentType(contentType) ??
+          resolveExtensionFromUrl(asset.fileNameHint ?? "");
+        kind = resolveMediaKind(sourceUrl, contentType, extension, asset.kind);
+        bytes = parsed.bytes;
+      } else {
+        const response = await fetch(sourceUrl, {
+          method: "GET",
+          redirect: "follow",
+          headers: {
+            "user-agent": DOWNLOAD_USER_AGENT,
+            ...(asset.headers ?? {}),
+          },
+        });
+
+        if (!response.ok) {
+          request.log.warn(`Skipping media (${response.status}): ${asset.url}`);
+          continue;
+        }
+
+        sourceUrl = normalizeMediaUrl(response.url || sourceUrl);
+        contentType = normalizeContentType(response.headers.get("content-type"));
+        extension =
+          resolveExtensionFromUrl(sourceUrl) ??
+          resolveExtensionFromUrl(asset.url) ??
+          resolveExtensionFromUrl(asset.fileNameHint ?? "");
+        kind = resolveMediaKind(sourceUrl, contentType, extension, asset.kind);
+        bytes = Buffer.from(await response.arrayBuffer());
+      }
+
+      if (!kind || !bytes) {
+        request.log.debug(`Skipping media with unresolved kind: ${asset.url}`);
+        continue;
+      }
+
+      const outputExtension = resolveOutputExtension(contentType, extension, kind);
+      const nextIndex = kind === "image" ? downloadedImages + 1 : downloadedVideos + 1;
+      const dirName = kind === "image" ? "imgs" : "videos";
+      const targetDir = path.join(baseDir, dirName);
+      await mkdir(targetDir, { recursive: true });
+
+      const fileName = buildFileName(kind, nextIndex, sourceUrl, outputExtension, asset.fileNameHint);
+      const absolutePath = path.join(targetDir, fileName);
+      await writeFile(absolutePath, bytes);
+
+      replacements.push({
+        url: asset.url,
+        localPath: toRelativePath(markdownDir, absolutePath),
+        absolutePath,
+        kind,
+      });
+
+      if (kind === "image") {
+        downloadedImages = nextIndex;
+      } else {
+        downloadedVideos = nextIndex;
+      }
+    } catch (error) {
+      const message = error instanceof Error ? error.message : String(error);
+      request.log.warn(`Failed to download media ${asset.url}: ${message}`);
+    }
+  }
+
+  return {
+    replacements,
+    downloadedImages,
+    downloadedVideos,
+    imageDir: downloadedImages > 0 ? path.join(baseDir, "imgs") : null,
+    videoDir: downloadedVideos > 0 ? path.join(baseDir, "videos") : null,
+  };
+}

packages/baoyu-fetch/src/media/markdown-media.ts

@@ -0,0 +1,458 @@
+import remarkGfm from "remark-gfm";
+import remarkParse from "remark-parse";
+import { unified } from "unified";
+import type { ContentBlock, ExtractedDocument } from "../extract/document";
+import {
+  isDataUri,
+  normalizeContentType,
+  normalizeMediaUrl,
+  resolveExtensionFromContentType,
+  resolveExtensionFromUrl,
+  resolveKindFromExtension,
+} from "./media-utils";
+import type { MediaAsset, MediaReplacement } from "./types";
+
+const MARKDOWN_LINK_RE =
+  /(!?\[[^\]\n]*\])\((<)?((?:https?:\/\/[^)\s>]+)|(?:data:[^)>\s]+))(>)?\)/g;
+const FRONTMATTER_COVER_RE = /^(coverImage:\s*")((?:https?:\/\/[^"]+)|(?:data:[^"]+))(")/m;
+const RAW_URL_RE = /(?:https?:\/\/[^\s<>"')\]]+|data:[^\s<>"')\]]+)/g;
+
+interface MarkdownAstNode {
+  type: string;
+  url?: string | null;
+  alt?: string | null;
+  title?: string | null;
+  value?: string | null;
+  children?: MarkdownAstNode[];
+  position?: {
+    start?: { offset?: number | null };
+    end?: { offset?: number | null };
+  };
+}
+
+interface MarkdownReplacementRange {
+  start: number;
+  end: number;
+  value: string;
+}
+
+function inferMediaKindFromLabel(label: string, rawUrl: string): "image" | "video" | undefined {
+  if (label.startsWith("![")) {
+    return "image";
+  }
+
+  const normalizedLabel = label.replace(/[!\[\]]/g, "").trim().toLowerCase();
+  if (/\b(video|animated[_ -]?gif|gif)\b/.test(normalizedLabel)) {
+    return "video";
+  }
+
+  if (isDataUri(rawUrl)) {
+    const contentType = normalizeContentType(rawUrl.slice(5, rawUrl.indexOf(";")));
+    return contentType.startsWith("image/") ? "image" : contentType.startsWith("video/") ? "video" : undefined;
+  }
+
+  return resolveKindFromExtension(resolveExtensionFromUrl(rawUrl));
+}
+
+function inferMediaKindFromRawUrl(rawUrl: string): "image" | "video" | undefined {
+  if (isDataUri(rawUrl)) {
+    const contentType = normalizeContentType(rawUrl.slice(5, rawUrl.indexOf(";")));
+    return contentType.startsWith("image/") ? "image" : contentType.startsWith("video/") ? "video" : undefined;
+  }
+
+  return resolveKindFromExtension(resolveExtensionFromUrl(rawUrl));
+}
+
+function pushMedia(assets: MediaAsset[], seen: Set<string>, media: MediaAsset): void {
+  const normalizedUrl = normalizeMediaUrl(media.url);
+  if (!normalizedUrl || seen.has(normalizedUrl)) {
+    return;
+  }
+  seen.add(normalizedUrl);
+  assets.push({
+    ...media,
+    url: normalizedUrl,
+  });
+}
+
+function getNodeOffsets(node: MarkdownAstNode): { start: number; end: number } | null {
+  const start = node.position?.start?.offset;
+  const end = node.position?.end?.offset;
+  if (typeof start !== "number" || typeof end !== "number" || start < 0 || end < start) {
+    return null;
+  }
+  return { start, end };
+}
+
+function escapeMarkdownLabel(value: string): string {
+  return value.replace(/\\/g, "\\\\").replace(/\[/g, "\\[").replace(/\]/g, "\\]");
+}
+
+function escapeMarkdownTitle(value: string): string {
+  return value.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
+}
+
+function formatMarkdownDestination(url: string): string {
+  return /[\s()<>]/.test(url) ? `<${url}>` : url;
+}
+
+function serializeImageNode(node: MarkdownAstNode): string {
+  const rawUrl = node.url ?? "";
+  const normalizedUrl = normalizeMediaUrl(rawUrl);
+  const alt = escapeMarkdownLabel(node.alt ?? "");
+  const title = node.title ? ` "${escapeMarkdownTitle(node.title)}"` : "";
+  return `![${alt}](${formatMarkdownDestination(normalizedUrl)}${title})`;
+}
+
+function serializeLinkedImageNode(linkNode: MarkdownAstNode, imageNode: MarkdownAstNode): string {
+  const imageMarkdown = serializeImageNode(imageNode);
+  const imageUrl = normalizeMediaUrl(imageNode.url ?? "");
+  const linkUrl = normalizeMediaUrl(linkNode.url ?? "");
+
+  if (!linkUrl || linkUrl === imageUrl) {
+    return imageMarkdown;
+  }
+
+  const title = linkNode.title ? ` "${escapeMarkdownTitle(linkNode.title)}"` : "";
+  return `[${imageMarkdown}](${formatMarkdownDestination(linkUrl)}${title})`;
+}
+
+function isParagraphWithSingleText(node: MarkdownAstNode | undefined, expectedValue: string): boolean {
+  if (node?.type !== "paragraph" || node.children?.length !== 1) {
+    return false;
+  }
+
+  const child = node.children[0];
+  return child?.type === "text" && child.value?.trim() === expectedValue;
+}
+
+function getSingleImageFromParagraph(node: MarkdownAstNode | undefined): MarkdownAstNode | null {
+  if (node?.type !== "paragraph" || node.children?.length !== 1) {
+    return null;
+  }
+
+  return node.children[0]?.type === "image" ? node.children[0] : null;
+}
+
+function extractBrokenLinkedImageDestination(node: MarkdownAstNode | undefined): string | null {
+  if (node?.type !== "paragraph") {
+    return null;
+  }
+
+  const children = node.children ?? [];
+  if (children.length !== 3) {
+    return null;
+  }
+
+  const [prefix, linkNode, suffix] = children;
+  if (prefix?.type !== "text" || prefix.value?.trim() !== "](") {
+    return null;
+  }
+  if (linkNode?.type !== "link" || !linkNode.url) {
+    return null;
+  }
+  if (suffix?.type !== "text" || suffix.value?.trim() !== ")") {
+    return null;
+  }
+
+  return linkNode.url;
+}
+
+function collectLinkedImageReplacements(
+  node: MarkdownAstNode,
+  replacements: MarkdownReplacementRange[],
+): void {
+  const children = node.children ?? [];
+
+  if (node.type === "link" && children.length === 1 && children[0]?.type === "image") {
+    const offsets = getNodeOffsets(node);
+    if (offsets) {
+      replacements.push({
+        start: offsets.start,
+        end: offsets.end,
+        value: serializeLinkedImageNode(node, children[0]),
+      });
+    }
+    return;
+  }
+
+  for (const child of children) {
+    collectLinkedImageReplacements(child, replacements);
+  }
+}
+
+function collectBrokenLinkedImageReplacements(
+  node: MarkdownAstNode,
+  replacements: MarkdownReplacementRange[],
+): void {
+  const children = node.children ?? [];
+  for (let index = 0; index <= children.length - 3; index += 1) {
+    const openParagraph = children[index];
+    const imageParagraph = children[index + 1];
+    const closeParagraph = children[index + 2];
+
+    if (!isParagraphWithSingleText(openParagraph, "[")) {
+      continue;
+    }
+
+    const imageNode = getSingleImageFromParagraph(imageParagraph);
+    if (!imageNode) {
+      continue;
+    }
+
+    const linkUrl = extractBrokenLinkedImageDestination(closeParagraph);
+    if (!linkUrl) {
+      continue;
+    }
+
+    const start = openParagraph.position?.start?.offset;
+    const end = closeParagraph.position?.end?.offset;
+    if (typeof start !== "number" || typeof end !== "number" || end < start) {
+      continue;
+    }
+
+    replacements.push({
+      start,
+      end,
+      value: serializeLinkedImageNode({ type: "link", url: linkUrl }, imageNode),
+    });
+
+    index += 2;
+  }
+
+  for (const child of children) {
+    collectBrokenLinkedImageReplacements(child, replacements);
+  }
+}
+
+function applyReplacements(source: string, replacements: MarkdownReplacementRange[]): string {
+  if (replacements.length === 0) {
+    return source;
+  }
+
+  let result = source;
+  const sorted = [...replacements].sort((left, right) => right.start - left.start);
+  for (const replacement of sorted) {
+    result = `${result.slice(0, replacement.start)}${replacement.value}${result.slice(replacement.end)}`;
+  }
+  return result;
+}
+
+function normalizeLinkedImageMarkdown(markdown: string): string {
+  let tree: MarkdownAstNode;
+  try {
+    tree = unified().use(remarkParse).use(remarkGfm).parse(markdown) as MarkdownAstNode;
+  } catch {
+    return markdown;
+  }
+
+  const replacements: MarkdownReplacementRange[] = [];
+  collectLinkedImageReplacements(tree, replacements);
+  collectBrokenLinkedImageReplacements(tree, replacements);
+  return applyReplacements(markdown, replacements);
+}
+
+export function normalizeMarkdownMediaLinks(markdown: string): string {
+  MARKDOWN_LINK_RE.lastIndex = 0;
+  let result = markdown.replace(MARKDOWN_LINK_RE, (full, label, openAngle, rawUrl, closeAngle) => {
+    const normalizedUrl = normalizeMediaUrl(rawUrl);
+    if (normalizedUrl === rawUrl) {
+      return full;
+    }
+    return `${label}(${openAngle ?? ""}${normalizedUrl}${closeAngle ?? ""})`;
+  });
+
+  result = result.replace(FRONTMATTER_COVER_RE, (full, prefix, rawUrl, suffix) => {
+    const normalizedUrl = normalizeMediaUrl(rawUrl);
+    if (normalizedUrl === rawUrl) {
+      return full;
+    }
+    return `${prefix}${normalizedUrl}${suffix}`;
+  });
+
+  RAW_URL_RE.lastIndex = 0;
+  result = result.replace(RAW_URL_RE, (rawUrl) => normalizeMediaUrl(rawUrl));
+  return normalizeLinkedImageMarkdown(result);
+}
+
+export function collectMediaFromText(
+  text: string,
+  options: {
+    role?: MediaAsset["role"];
+    defaultKind?: MediaAsset["kind"];
+    seen?: Set<string>;
+    into?: MediaAsset[];
+  } = {},
+): MediaAsset[] {
+  const assets = options.into ?? [];
+  const seen = options.seen ?? new Set<string>();
+
+  MARKDOWN_LINK_RE.lastIndex = 0;
+  let linkMatch: RegExpExecArray | null;
+  while ((linkMatch = MARKDOWN_LINK_RE.exec(text))) {
+    const label = linkMatch[1] ?? "";
+    const rawUrl = linkMatch[3] ?? "";
+    const kind = inferMediaKindFromLabel(label, rawUrl) ?? options.defaultKind;
+    if (!kind) {
+      continue;
+    }
+    pushMedia(assets, seen, {
+      url: rawUrl,
+      kind,
+      role: options.role ?? "inline",
+    });
+  }
+
+  RAW_URL_RE.lastIndex = 0;
+  let rawMatch: RegExpExecArray | null;
+  while ((rawMatch = RAW_URL_RE.exec(text))) {
+    const rawUrl = rawMatch[0] ?? "";
+    const kind = inferMediaKindFromRawUrl(rawUrl) ?? options.defaultKind;
+    if (!kind) {
+      continue;
+    }
+    pushMedia(assets, seen, {
+      url: rawUrl,
+      kind,
+      role: options.role ?? "inline",
+    });
+  }
+
+  return assets;
+}
+
+function collectMediaFromBlock(
+  block: ContentBlock,
+  assets: MediaAsset[],
+  seen: Set<string>,
+): void {
+  switch (block.type) {
+    case "image":
+      pushMedia(assets, seen, {
+        url: block.url,
+        kind: "image",
+        role: "inline",
+        alt: block.alt,
+      });
+      return;
+    case "html":
+    case "markdown":
+      collectMediaFromText(block.type === "html" ? block.html : block.markdown, {
+        role: "inline",
+        seen,
+        into: assets,
+      });
+      return;
+    case "paragraph":
+    case "quote":
+      collectMediaFromText(block.text, {
+        role: "inline",
+        seen,
+        into: assets,
+      });
+      return;
+    case "list":
+      for (const item of block.items) {
+        collectMediaFromText(item, {
+          role: "attachment",
+          seen,
+          into: assets,
+        });
+      }
+      return;
+    case "heading":
+    case "code":
+      return;
+  }
+}
+
+export function collectMediaFromDocument(document: ExtractedDocument): MediaAsset[] {
+  const assets: MediaAsset[] = [];
+  const seen = new Set<string>();
+  const coverImage =
+    typeof document.metadata?.coverImage === "string" ? document.metadata.coverImage : undefined;
+
+  if (coverImage) {
+    pushMedia(assets, seen, {
+      url: coverImage,
+      kind: "image",
+      role: "cover",
+    });
+  }
+
+  for (const block of document.content) {
+    collectMediaFromBlock(block, assets, seen);
+  }
+
+  return assets;
+}
+
+export function collectMediaFromMarkdown(markdown: string): MediaAsset[] {
+  const assets: MediaAsset[] = [];
+  const seen = new Set<string>();
+  const fmMatch = markdown.match(/^---\n([\s\S]*?)\n---/);
+  if (fmMatch) {
+    const coverMatch = fmMatch[1]?.match(FRONTMATTER_COVER_RE);
+    if (coverMatch?.[2]) {
+      pushMedia(assets, seen, {
+        url: coverMatch[2],
+        kind: "image",
+        role: "cover",
+      });
+    }
+  }
+
+  collectMediaFromText(markdown, { seen, into: assets });
+  return assets;
+}
+
+export function rewriteMarkdownMediaLinks(
+  markdown: string,
+  replacements: MediaReplacement[],
+): string {
+  if (replacements.length === 0) {
+    return markdown;
+  }
+
+  const replacementMap = new Map<string, string>();
+  for (const item of replacements) {
+    replacementMap.set(item.url, item.localPath);
+    replacementMap.set(normalizeMediaUrl(item.url), item.localPath);
+  }
+
+  MARKDOWN_LINK_RE.lastIndex = 0;
+  let result = markdown.replace(MARKDOWN_LINK_RE, (full, label, _openAngle, rawUrl) => {
+    const replacement = replacementMap.get(rawUrl) ?? replacementMap.get(normalizeMediaUrl(rawUrl));
+    if (!replacement) {
+      return full;
+    }
+    return `${label}(${replacement})`;
+  });
+
+  result = result.replace(FRONTMATTER_COVER_RE, (full, prefix, rawUrl, suffix) => {
+    const replacement = replacementMap.get(rawUrl) ?? replacementMap.get(normalizeMediaUrl(rawUrl));
+    if (!replacement) {
+      return full;
+    }
+    return `${prefix}${replacement}${suffix}`;
+  });
+
+  for (const { url, localPath } of replacements) {
+    result = result.split(url).join(localPath);
+    const normalizedUrl = normalizeMediaUrl(url);
+    if (normalizedUrl !== url) {
+      result = result.split(normalizedUrl).join(localPath);
+    }
+  }
+
+  return result;
+}
+
+export function resolveDataUriExtension(rawUrl: string): string | undefined {
+  if (!isDataUri(rawUrl)) {
+    return undefined;
+  }
+  const separatorIndex = rawUrl.indexOf(";");
+  const contentType = normalizeContentType(rawUrl.slice(5, separatorIndex === -1 ? undefined : separatorIndex));
+  return resolveExtensionFromContentType(contentType);
+}

packages/baoyu-fetch/src/media/media-utils.ts

@@ -0,0 +1,261 @@
+import path from "node:path";
+import type { MediaKind } from "./types";
+
+const IMAGE_EXTENSIONS = new Set([
+  "jpg",
+  "jpeg",
+  "png",
+  "webp",
+  "gif",
+  "bmp",
+  "avif",
+  "heic",
+  "heif",
+  "svg",
+]);
+
+const VIDEO_EXTENSIONS = new Set(["mp4", "m4v", "mov", "webm", "mkv"]);
+
+const MIME_EXTENSION_MAP: Record<string, string> = {
+  "image/jpeg": "jpg",
+  "image/jpg": "jpg",
+  "image/png": "png",
+  "image/webp": "webp",
+  "image/gif": "gif",
+  "image/bmp": "bmp",
+  "image/avif": "avif",
+  "image/heic": "heic",
+  "image/heif": "heif",
+  "image/svg+xml": "svg",
+  "video/mp4": "mp4",
+  "video/webm": "webm",
+  "video/quicktime": "mov",
+  "video/x-m4v": "m4v",
+};
+
+export function normalizeContentType(raw: string | null): string {
+  return raw?.split(";")[0]?.trim().toLowerCase() ?? "";
+}
+
+export function normalizeExtension(raw: string | undefined | null): string | undefined {
+  if (!raw) {
+    return undefined;
+  }
+  const trimmed = raw.replace(/^\./, "").trim().toLowerCase();
+  if (!trimmed) {
+    return undefined;
+  }
+  if (trimmed === "jpeg" || trimmed === "jpg") {
+    return "jpg";
+  }
+  return trimmed;
+}
+
+export function resolveExtensionFromUrl(rawUrl: string): string | undefined {
+  try {
+    const parsed = new URL(rawUrl);
+    const extFromPath = normalizeExtension(path.posix.extname(parsed.pathname));
+    if (extFromPath) {
+      return extFromPath;
+    }
+    const extFromFormat = normalizeExtension(parsed.searchParams.get("format"));
+    if (extFromFormat) {
+      return extFromFormat;
+    }
+  } catch {
+    return undefined;
+  }
+  return undefined;
+}
+
+export function resolveExtensionFromContentType(contentType: string): string | undefined {
+  return normalizeExtension(MIME_EXTENSION_MAP[contentType]);
+}
+
+export function resolveKindFromContentType(contentType: string): MediaKind | undefined {
+  if (!contentType) {
+    return undefined;
+  }
+  if (contentType.startsWith("image/")) {
+    return "image";
+  }
+  if (contentType.startsWith("video/")) {
+    return "video";
+  }
+  return undefined;
+}
+
+export function resolveKindFromExtension(extension: string | undefined): MediaKind | undefined {
+  if (!extension) {
+    return undefined;
+  }
+  if (IMAGE_EXTENSIONS.has(extension)) {
+    return "image";
+  }
+  if (VIDEO_EXTENSIONS.has(extension)) {
+    return "video";
+  }
+  return undefined;
+}
+
+export function resolveMediaKind(
+  rawUrl: string,
+  contentType: string,
+  extension: string | undefined,
+  hint?: MediaKind,
+): MediaKind | undefined {
+  const kindFromType = resolveKindFromContentType(contentType);
+  if (kindFromType) {
+    return kindFromType;
+  }
+
+  const kindFromExtension = resolveKindFromExtension(extension);
+  if (kindFromExtension) {
+    return kindFromExtension;
+  }
+
+  if (contentType && contentType !== "application/octet-stream") {
+    return undefined;
+  }
+
+  if (hint) {
+    return hint;
+  }
+
+  if (rawUrl.startsWith("data:image/")) {
+    return "image";
+  }
+
+  if (rawUrl.startsWith("data:video/")) {
+    return "video";
+  }
+
+  return undefined;
+}
+
+export function resolveOutputExtension(
+  contentType: string,
+  extension: string | undefined,
+  kind: MediaKind,
+): string {
+  const fromMime = resolveExtensionFromContentType(contentType);
+  if (fromMime) {
+    return fromMime;
+  }
+  const normalized = normalizeExtension(extension);
+  if (normalized) {
+    return normalized;
+  }
+  return kind === "video" ? "mp4" : "jpg";
+}
+
+export function isDataUri(value: string): boolean {
+  return value.startsWith("data:");
+}
+
+export function safeDecodeURIComponent(value: string): string {
+  try {
+    return decodeURIComponent(value);
+  } catch {
+    return value;
+  }
+}
+
+function extractEmbeddedUrl(value: string): string | undefined {
+  const encodedMatch = value.match(/https?%3A%2F%2F.+$/i)?.[0];
+  if (encodedMatch) {
+    const decoded = safeDecodeURIComponent(encodedMatch);
+    try {
+      return new URL(decoded).href;
+    } catch {
+      return undefined;
+    }
+  }
+
+  const literalMatch = value.match(/https?:\/\/.+$/i)?.[0];
+  if (!literalMatch) {
+    return undefined;
+  }
+
+  try {
+    return new URL(literalMatch).href;
+  } catch {
+    return undefined;
+  }
+}
+
+export function normalizeMediaUrl(rawUrl: string): string {
+  if (isDataUri(rawUrl)) {
+    return rawUrl;
+  }
+
+  try {
+    const parsed = new URL(rawUrl);
+    const hostname = parsed.hostname.toLowerCase();
+
+    if (hostname === "substackcdn.com" || hostname.endsWith(".substackcdn.com")) {
+      const embeddedUrl = extractEmbeddedUrl(`${parsed.pathname}${parsed.search}`);
+      if (embeddedUrl) {
+        return embeddedUrl;
+      }
+    }
+
+    return parsed.href;
+  } catch {
+    return rawUrl;
+  }
+}
+
+export function sanitizeFileSegment(input: string): string {
+  return input
+    .replace(/[^a-zA-Z0-9_-]+/g, "-")
+    .replace(/-+/g, "-")
+    .replace(/^[-_]+|[-_]+$/g, "")
+    .slice(0, 48);
+}
+
+export function resolveFileStem(rawUrl: string, extension: string, fileNameHint?: string): string {
+  const hintBase = fileNameHint?.trim();
+  if (hintBase) {
+    const parsed = path.posix.parse(hintBase);
+    const stem = parsed.name || parsed.base;
+    return sanitizeFileSegment(stem);
+  }
+
+  if (isDataUri(rawUrl)) {
+    return "";
+  }
+
+  try {
+    const parsed = new URL(rawUrl);
+    const base = path.posix.basename(parsed.pathname);
+    if (!base) {
+      return "";
+    }
+    const decodedBase = safeDecodeURIComponent(base);
+    const normalizedExtension = normalizeExtension(extension);
+    const stripExtension = normalizedExtension ? new RegExp(`\\.${normalizedExtension}$`, "i") : null;
+    const rawStem = stripExtension ? decodedBase.replace(stripExtension, "") : decodedBase;
+    return sanitizeFileSegment(rawStem);
+  } catch {
+    return "";
+  }
+}
+
+export function buildFileName(
+  kind: MediaKind,
+  index: number,
+  sourceUrl: string,
+  extension: string,
+  fileNameHint?: string,
+): string {
+  const stem = resolveFileStem(sourceUrl, extension, fileNameHint);
+  const prefix = kind === "image" ? "img" : "video";
+  const serial = String(index).padStart(3, "0");
+  const suffix = stem ? `-${stem}` : "";
+  return `${prefix}-${serial}${suffix}.${extension}`;
+}
+
+export function toPosixPath(value: string): string {
+  return value.split(path.sep).join(path.posix.sep);
+}

packages/baoyu-fetch/src/media/types.ts

@@ -0,0 +1,34 @@
+import type { Logger } from "../utils/logger";
+
+export type MediaKind = "image" | "video";
+
+export interface MediaAsset {
+  url: string;
+  kind?: MediaKind;
+  role?: "cover" | "inline" | "attachment";
+  alt?: string;
+  fileNameHint?: string;
+  headers?: Record<string, string>;
+}
+
+export interface MediaReplacement {
+  url: string;
+  localPath: string;
+  absolutePath: string;
+  kind: MediaKind;
+}
+
+export interface MediaDownloadRequest {
+  media: MediaAsset[];
+  outputPath: string;
+  mediaDir?: string;
+  log: Logger;
+}
+
+export interface MediaDownloadResult {
+  replacements: MediaReplacement[];
+  downloadedImages: number;
+  downloadedVideos: number;
+  imageDir: string | null;
+  videoDir: string | null;
+}

packages/baoyu-fetch/src/types/defuddle-node.d.ts

@@ -0,0 +1,31 @@
+declare module "defuddle/node" {
+  export interface DefuddleResponse {
+    content?: string;
+    title?: string;
+    description?: string;
+    author?: string;
+    published?: string;
+    image?: string;
+    language?: string;
+  }
+
+  export interface DefuddleOptions {
+    markdown?: boolean;
+  }
+
+  export function Defuddle(
+    input:
+      | Document
+      | string
+      | {
+          window: {
+            document: Document;
+            location: {
+              href: string;
+            };
+          };
+        },
+    url?: string,
+    options?: DefuddleOptions,
+  ): Promise<DefuddleResponse>;
+}

packages/baoyu-fetch/src/types/shims.d.ts

@@ -0,0 +1,17 @@
+declare module "turndown" {
+  export interface TurndownOptions {
+    codeBlockStyle?: "indented" | "fenced";
+    headingStyle?: "setext" | "atx";
+    bulletListMarker?: "-" | "*" | "+";
+  }
+
+  export default class TurndownService {
+    constructor(options?: TurndownOptions);
+    use(plugin: unknown): void;
+    turndown(input: string): string;
+  }
+}
+
+declare module "turndown-plugin-gfm" {
+  export const gfm: unknown;
+}

packages/baoyu-fetch/src/utils/logger.ts

@@ -0,0 +1,30 @@
+export interface Logger {
+  info(message: string): void;
+  warn(message: string): void;
+  error(message: string): void;
+  debug(message: string): void;
+}
+
+export function createLogger(debugEnabled = false): Logger {
+  const print = (level: string, message: string): void => {
+    console.error(`[${level}] ${message}`);
+  };
+
+  return {
+    info(message: string) {
+      print("info", message);
+    },
+    warn(message: string) {
+      print("warn", message);
+    },
+    error(message: string) {
+      print("error", message);
+    },
+    debug(message: string) {
+      if (debugEnabled) {
+        print("debug", message);
+      }
+    },
+  };
+}
+

packages/baoyu-fetch/src/utils/url.ts

@@ -0,0 +1,12 @@
+export function normalizeUrl(input: string): URL {
+  try {
+    return new URL(input);
+  } catch {
+    throw new Error(`Invalid URL: ${input}`);
+  }
+}
+
+export function sanitizeFilename(input: string): string {
+  return input.replace(/[^a-zA-Z0-9._-]+/g, "-").replace(/^-+|-+$/g, "") || "document";
+}
+

packages/baoyu-fetch/tsconfig.json

@@ -0,0 +1,17 @@
+{
+  "compilerOptions": {
+    "target": "ES2022",
+    "module": "ESNext",
+    "moduleResolution": "Bundler",
+    "lib": ["ES2022", "DOM"],
+    "types": ["bun"],
+    "strict": true,
+    "noEmit": true,
+    "skipLibCheck": true,
+    "allowSyntheticDefaultImports": true,
+    "forceConsistentCasingInFileNames": true,
+    "verbatimModuleSyntax": true,
+    "resolveJsonModule": true
+  },
+  "include": ["src/**/*.ts", "src/**/*.d.ts"]
+}

packages/baoyu-md/dist/code-themes/1c-light.min.css

@@ -0,0 +1,9 @@
+pre code.hljs{display:block;overflow-x:auto;padding:1em}code.hljs{padding:3px 5px}/*!
+  Theme: 1c-light
+  Description: Style IDE 1C:Enterprise 8
+  Author: (c) Barilko Vitaliy <barilkovetal@gmail.com>
+  Maintainer: @Diversus23
+  Website: https://softonit.ru/
+  License: see project LICENSE
+  Touched: 2023
+*/.hljs{color:#00f;background:#fff}.hljs-comment{color:green}.hljs-tag{color:#444a}.hljs-tag .hljs-attr,.hljs-tag .hljs-name{color:#444}.hljs-attribute,.hljs-doctag,.hljs-function,.hljs-keyword,.hljs-name,.hljs-punctuation,.hljs-selector-tag{color:red}.hljs-params,.hljs-type{color:#00f}.hljs-deletion,.hljs-number,.hljs-quote,.hljs-selector-class,.hljs-selector-id,.hljs-string,.hljs-symbol,.hljs-template-tag{color:#000}.hljs-section,.hljs-title{color:#00f}.hljs-link,.hljs-operator,.hljs-regexp,.hljs-selector-attr,.hljs-selector-pseudo,.hljs-template-variable,.hljs-variable{color:#ab5656}.hljs-literal{color:red}.hljs-addition,.hljs-built_in,.hljs-bullet,.hljs-code{color:#00f}.hljs-meta,.hljs-meta .hljs-keyword,.hljs-meta .hljs-string{color:#963200}.hljs-emphasis{font-style:italic}.hljs-strong{font-weight:700}

packages/baoyu-md/dist/code-themes/a11y-dark.min.css

@@ -0,0 +1,7 @@
+pre code.hljs{display:block;overflow-x:auto;padding:1em}code.hljs{padding:3px 5px}/*!
+  Theme: a11y-dark
+  Author: @ericwbailey
+  Maintainer: @ericwbailey
+
+  Based on the Tomorrow Night Eighties theme: https://github.com/isagalaev/highlight.js/blob/master/src/styles/tomorrow-night-eighties.css
+*/.hljs{background:#2b2b2b;color:#f8f8f2}.hljs-comment,.hljs-quote{color:#d4d0ab}.hljs-deletion,.hljs-name,.hljs-regexp,.hljs-selector-class,.hljs-selector-id,.hljs-tag,.hljs-template-variable,.hljs-variable{color:#ffa07a}.hljs-built_in,.hljs-link,.hljs-literal,.hljs-meta,.hljs-number,.hljs-params,.hljs-type{color:#f5ab35}.hljs-attribute{color:gold}.hljs-addition,.hljs-bullet,.hljs-string,.hljs-symbol{color:#abe338}.hljs-section,.hljs-title{color:#00e0e0}.hljs-keyword,.hljs-selector-tag{color:#dcc6e0}.hljs-emphasis{font-style:italic}.hljs-strong{font-weight:700}@media screen and (-ms-high-contrast:active){.hljs-addition,.hljs-attribute,.hljs-built_in,.hljs-bullet,.hljs-comment,.hljs-link,.hljs-literal,.hljs-meta,.hljs-number,.hljs-params,.hljs-quote,.hljs-string,.hljs-symbol,.hljs-type{color:highlight}.hljs-keyword,.hljs-selector-tag{font-weight:700}}

packages/baoyu-md/dist/code-themes/a11y-light.min.css

@@ -0,0 +1,7 @@
+pre code.hljs{display:block;overflow-x:auto;padding:1em}code.hljs{padding:3px 5px}/*!
+  Theme: a11y-light
+  Author: @ericwbailey
+  Maintainer: @ericwbailey
+
+  Based on the Tomorrow Night Eighties theme: https://github.com/isagalaev/highlight.js/blob/master/src/styles/tomorrow-night-eighties.css
+*/.hljs{background:#fefefe;color:#545454}.hljs-comment,.hljs-quote{color:#696969}.hljs-deletion,.hljs-name,.hljs-regexp,.hljs-selector-class,.hljs-selector-id,.hljs-tag,.hljs-template-variable,.hljs-variable{color:#d91e18}.hljs-attribute,.hljs-built_in,.hljs-link,.hljs-literal,.hljs-meta,.hljs-number,.hljs-params,.hljs-type{color:#aa5d00}.hljs-addition,.hljs-bullet,.hljs-string,.hljs-symbol{color:green}.hljs-section,.hljs-title{color:#007faa}.hljs-keyword,.hljs-selector-tag{color:#7928a1}.hljs-emphasis{font-style:italic}.hljs-strong{font-weight:700}@media screen and (-ms-high-contrast:active){.hljs-addition,.hljs-attribute,.hljs-built_in,.hljs-bullet,.hljs-comment,.hljs-link,.hljs-literal,.hljs-meta,.hljs-number,.hljs-params,.hljs-quote,.hljs-string,.hljs-symbol,.hljs-type{color:highlight}.hljs-keyword,.hljs-selector-tag{font-weight:700}}