Compare commits

..

624 Commits

Author SHA1 Message Date
Jim Liu 宝玉 b0ac5233cc docs(image-generation): document Cursor GenerateImage backend 2026-06-12 23:56:34 -05:00
Jim Liu 宝玉 c1f96f8421 chore: release v2.5.0 2026-06-12 20:29:18 -05:00
Jim Liu 宝玉 4d2322fd02 feat(baoyu-wechat-summary): add per-group fact memory (memory.md) 2026-06-12 20:28:46 -05:00
Jim Liu 宝玉 154d0d1f52 Merge pull request #179 from sandypoli-boop/fix/windows-bun-mkdir-eexist
fix(baoyu-image-gen): tolerate Bun-on-Windows EEXIST from mkdir(recursive)
2026-06-12 19:45:52 -05:00
sandy 7a956f6e7d fix(baoyu-image-gen): tolerate Bun-on-Windows EEXIST from mkdir(recursive)
On Windows, Bun throws EEXIST for mkdir(dir, { recursive: true }) when the
directory already exists, contradicting Node's documented contract (it should
resolve silently). Image generation then succeeds but the file save fails
whenever the output directory already exists (e.g. the Desktop):

    EEXIST: file already exists, mkdir 'C:\Users\...\Desktop'

Add an ensureDir() helper that tolerates EEXIST only when the path really is a
directory (rethrowing otherwise, so a genuine EEXIST against an existing file
is not swallowed), and route writeImage() and migrateLegacyExtendConfig()
through it. Covered by a cross-platform unit test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 15:13:22 +08:00
Jim Liu 宝玉 55223daf5c Merge pull request #176 from Osamaali313/fix/youtube-embed-url
fix(baoyu-fetch): parse YouTube /embed/ URLs in parseYouTubeVideoId
2026-06-10 17:56:23 -05:00
Syed Osama Ali Shah a1c4b732c5 fix(baoyu-fetch): parse YouTube /embed/ URLs in parseYouTubeVideoId
parseYouTubeVideoId handled watch, youtu.be, /shorts/ and /live/ URLs but
not the common /embed/<id> player form, so embed links returned null and the
YouTube adapter treated them as "no document". Add an /embed/ branch
mirroring the existing /shorts/ and /live/ handling, with a regression test.
2026-06-09 23:55:40 +03:00
Jim Liu 宝玉 894008c7f6 Promote baoyu-design in README skill listings 2026-06-09 01:47:53 -05:00
Jim Liu 宝玉 9daa6f5db3 Merge pull request #174 from Davidlaizz/custom/main
feat(baoyu-image-gen): add Agnes AI image generation provider
2026-06-08 23:32:34 -05:00
Davidlaizz 53aa30bbca fix(baoyu-image-gen): remove gcd from resolveSize to fix decimal aspect ratio distortion 2026-06-06 05:46:38 +08:00
Davidlaizz 3f1120e903 fix(baoyu-image-gen): add agnes to provider_limits YAML whitelist 2026-06-06 05:18:27 +08:00
Davidlaizz 7ea2692acd fix(baoyu-image-gen): add agnes to default_model YAML whitelist and remote ref allowlist 2026-06-06 05:15:26 +08:00
Davidlaizz 591614cfa5 docs(baoyu-image-gen): add Agnes to SKILL.md, README, and config docs 2026-06-06 04:56:19 +08:00
Davidlaizz ad7a7a646d feat(baoyu-image-gen): add Agnes AI image generation provider 2026-06-06 04:53:00 +08:00
Jim Liu 宝玉 ce84174bf7 Merge pull request #173 from yanghaod2278827/improve-skill-descriptions
docs: improve skill description for better trigger accuracy
2026-06-03 12:43:13 -05:00
yanghaod2278827 06e84b92c3 docs: improve skill description for better trigger accuracy 2026-06-03 11:30:52 +08:00
Jim Liu 宝玉 67fa5cd329 docs: add ebook link and fix WeChat config spacing 2026-06-02 17:01:56 -05:00
Jim Liu 宝玉 3907281f48 docs: add book info to Chinese README 2026-06-02 11:01:23 -05:00
Jim Liu 宝玉 011406036c fix: bump baoyu package dependencies 2026-06-01 23:21:05 -05:00
Jim Liu 宝玉 e6f4cd8a46 Merge pull request #171 from NTLx/feat/content-source-url
feat(baoyu-post-to-wechat): add content_source_url support for original article link
2026-05-29 23:01:42 -05:00
Jim Liu 宝玉 a80eec7d75 Merge pull request #170 from hypn4/feat/google-ga-image-models
feat(baoyu-image-gen): migrate Google image generation to GA Gemini endpoints
2026-05-29 23:01:12 -05:00
Jim Liu 宝玉 ec704c8afd chore: release v2.4.0 2026-05-29 18:59:31 -05:00
Jim Liu 宝玉 a85c81e8db feat(baoyu-wechat-summary): add @bot Q&A section to normal and roast digests 2026-05-29 18:59:26 -05:00
NTLx f06a9021a0 feat(baoyu-post-to-wechat): add content_source_url support for "阅读原文" link
Add the ability to specify the original article URL ("阅读原文" / "Read
Original" link) when publishing to WeChat Official Account draft via API.

Changes:
- ArticleOptions interface: add contentSourceUrl optional field
- publishToDraft: conditionally write content_source_url for news articles
- CLI: add --source-url parameter
- Frontmatter: extract sourceUrl / contentSourceUrl / content_source_url
- Priority chain: CLI --source-url → frontmatter → (none)
- SKILL.md: document the new option and draft/add payload rule

The field is only written for article_type=news per WeChat API spec.
content_source_url is optional (max 1KB) per the official draft/add API.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-29 13:20:58 +08:00
hypn4 d79ebe4838 feat(baoyu-image-gen): default to GA Gemini image endpoints and sync docs 2026-05-29 12:35:28 +09:00
hypn4 d3d1a9f7cd feat(baoyu-image-gen): allow reference images with GA Gemini image endpoints 2026-05-29 11:44:28 +09:00
Jim Liu 宝玉 77dd193b58 fix: sync npm lockfile 2026-05-27 23:26:46 -05:00
Jim Liu 宝玉 84d817ed52 chore: release v2.3.0 2026-05-27 23:24:29 -05:00
Jim Liu 宝玉 639e0b4193 feat: support Obsidian wikilink images
Support Obsidian image wikilinks alongside standard markdown images, preserve mixed image order, and resolve Obsidian default Attachments/ paths.

Co-authored-by: Chao Zheng <10296164+zcqqq@users.noreply.github.com>
2026-05-27 23:06:56 -05:00
Jim Liu 宝玉 84cefc2784 fix: harden URL-decoded image paths
Co-authored-by: zcqqq <10296164+zcqqq@users.noreply.github.com>
2026-05-27 21:44:22 -05:00
Chao Zheng 876f01ac19 fix(baoyu-md, baoyu-post-to-x): decode URL-encoded image paths
Obsidian creates markdown image links with URL-encoded filenames
(e.g. `Pasted%20image%2020260524.png`) but the actual file has spaces.
Add `decodeURIComponent()` before path resolution in both baoyu-md's
shared `resolveImagePath()` and baoyu-post-to-x's independent version.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-27 21:43:53 -05:00
Jim Liu 宝玉 e1c0ff7c02 chore: release v2.2.1 2026-05-26 00:53:56 -05:00
Jim Liu 宝玉 860dd36bb6 docs(baoyu-image-gen): surface build-batch.ts in Usage and clarify {baseDir} script paths
Add the outline.md + prompts/ → batch.json one-liner to SKILL.md's Usage section so the
build-batch helper is discoverable next to --batchfile, and update build-batch.ts --help
to print the bun / npx-y bun invocations with the {baseDir}/scripts/... layout used by
the rest of the skill.
2026-05-26 00:52:51 -05:00
Jim Liu 宝玉 aab0e28823 chore: release v2.2.0 2026-05-25 00:21:09 -05:00
Jim Liu 宝玉 95cdad2a2a docs(baoyu-article-illustrator,baoyu-comic,baoyu-infographic,baoyu-slide-deck,baoyu-xhs-images): point codex-imagegen flow at baoyu-image-gen --provider codex-cli
Add a `Codex via codex exec` branch to each skill's backend-selection
ladder and ship a per-skill references/codex-imagegen.md with the
invocation contract (preferred baoyu-image-gen --provider codex-cli
path, direct-wrapper fallback, parameter notes, stdout schema, batch
semantics).
2026-05-25 00:18:41 -05:00
Jim Liu 宝玉 8838161729 docs(baoyu-cover-image): redirect codex-imagegen invocation through baoyu-image-gen --provider codex-cli
Replace the long inline `scripts/codex-imagegen.sh` invocation block in
SKILL.md with a pointer to references/codex-imagegen.md. The reference
documents the preferred `baoyu-image-gen --provider codex-cli` path and
keeps the direct-wrapper fallback for runtimes without baoyu-image-gen.
2026-05-25 00:18:32 -05:00
Jim Liu 宝玉 49e9c46ca4 feat(baoyu-image-gen): add codex-cli provider wrapping codex-imagegen
Expose Codex CLI's built-in image_gen tool through baoyu-image-gen's
standard CLI + batch flow as a dedicated provider. The provider spawns
the bundled scripts/codex-imagegen/main.ts (synced from
packages/baoyu-codex-imagegen/src/) so the skill remains self-contained
and inherits retry, cache, file-lock, and JSONL logging. No
OPENAI_API_KEY required — uses the user's Codex subscription.

New env vars: BAOYU_CODEX_IMAGEGEN_{BIN,CACHE_DIR,TIMEOUT_MS,RETRIES,LOG_FILE}.
Hyphenated provider names now resolve under-scored env vars (e.g.,
BAOYU_IMAGE_GEN_CODEX_CLI_CONCURRENCY). codex-cli is never auto-selected
— pin via --provider or default_provider in EXTEND.md.
2026-05-25 00:18:27 -05:00
Jim Liu 宝玉 23fac03691 refactor(codex-imagegen): extract backend to packages/baoyu-codex-imagegen workspace
Move the codex-imagegen wrapper out of scripts/ into its own workspace
package alongside baoyu-md, baoyu-chrome-cdp, and baoyu-fetch. Drop the
bash shim — src/main.ts now carries a `#\!/usr/bin/env bun` shebang and
serves as the sole entrypoint. Add scripts/sync-codex-imagegen.sh so
skills/baoyu-image-gen/scripts/codex-imagegen/ can be regenerated from
packages/baoyu-codex-imagegen/src/ for skill self-containment. Update
CLAUDE.md, docs/codex-imagegen-backend.md, and the CI workflow paths
accordingly.
2026-05-25 00:18:16 -05:00
Jim Liu 宝玉 66160abb9f chore: release v2.1.0 2026-05-24 21:27:57 -05:00
Jim Liu 宝玉 b34e19daad feat(baoyu-post-to-wechat,baoyu-post-to-weibo,baoyu-post-to-x): cascade Mermaid PNG preprocessing into publishing pipelines 2026-05-24 21:27:53 -05:00
Jim Liu 宝玉 cda76e20fd feat(baoyu-markdown-to-html): render Mermaid code blocks to PNG via headless Chrome 2026-05-24 21:27:43 -05:00
Jim Liu 宝玉 8d2fd7aae0 feat(baoyu-md): add Mermaid preprocessing and utility modules 2026-05-24 21:27:38 -05:00
Jim Liu 宝玉 f60702cc8e feat(baoyu-chrome-cdp): add Mermaid-to-PNG rendering via CDP 2026-05-24 21:27:33 -05:00
Jim Liu 宝玉 613bee6bb8 feat(build): support multi-entry output in shared package build script 2026-05-24 21:27:28 -05:00
Jim Liu 宝玉 e63e21786d chore: release v2.0.1 2026-05-24 18:59:16 -05:00
Jim Liu 宝玉 b3a3b632c8 fix(baoyu-post-to-wechat): repair htmlToPlainText syntax error and entity decoding
Follow-up to the previous newspic conversion commit:

- `&ldquo;` / `&rdquo;` mapped to `"""` (three straight double-quotes),
  producing an unterminated string literal that prevented wechat-api.ts
  from parsing. Replaced with the intended curly Unicode characters.
- `&lsquo;` / `&rsquo;` similarly mapped to straight `'`; switched to the
  curly singles to match the entity meaning.
- Numeric entities used `String.fromCharCode`, which mangles code points
  above 0xFFFF (e.g. emoji like `&#128512;` → empty string). Switched to
  `String.fromCodePoint`.
- Added hexadecimal numeric entity support (`&#x1F600;` etc.); the
  previous regex only matched decimal forms.
- Anchored the entity sub-patterns with `^...$` so the callback cannot
  accidentally re-match a substring of the captured entity.
2026-05-24 18:57:01 -05:00
Go1dFinger 3aafe60463 fix(baoyu-post-to-wechat): improve newspic plain text conversion for WeChat API
The previous fix used a simple regex to strip HTML tags for newspic
content, but it had several issues:

1. HTML entities (&nbsp;, &lt;, &gt;, &amp;, etc.) were not decoded
2. Block-level tags (</p>, </div>, </h1> etc.) did not produce line breaks
3. Multiple consecutive whitespace characters were not collapsed

This commit introduces a dedicated htmlToPlainText() function that:
- Converts <br> and block-level closing tags to line breaks
- Strips all remaining HTML tags
- Decodes common HTML entities (including numeric entities)
- Collapses consecutive whitespace into single spaces
- Trims whitespace from each line

This ensures the content field for newspic articles is properly
formatted plain text that complies with WeChat API requirements,
preventing error 45166 (invalid content) especially when publishing
with multiple images.

Closes #163
Merges #164
2026-05-24 18:55:59 -05:00
Jim Liu 宝玉 246e90bf28 chore: align baoyu-{article-illustrator,comic,infographic,slide-deck} SKILL.md versions with ClawHub 2026-05-24 18:36:45 -05:00
Jim Liu 宝玉 c3cbce9ce3 feat\!: rename baoyu-imagine→baoyu-image-gen, baoyu-image-cards→baoyu-xhs-images (v2.0.0)
BREAKING CHANGE: removed `baoyu-imagine` and `baoyu-image-cards`. All
functionality now lives under `baoyu-image-gen` and `baoyu-xhs-images`
respectively. Cross-skill `## Image Generation Tools` examples updated
across baoyu-article-illustrator, baoyu-comic, baoyu-cover-image,
baoyu-infographic, and baoyu-slide-deck.

Migration: existing `~/.baoyu-skills/baoyu-imagine/EXTEND.md` configs are
auto-renamed to `…/baoyu-image-gen/EXTEND.md` on first run via the
legacy-path resolver in `scripts/main.ts`.
2026-05-24 18:35:05 -05:00
Jim Liu 宝玉 bc303345f4 chore(baoyu-imagine): align SKILL.md version with ClawHub 2026-05-24 17:14:11 -05:00
Jim Liu 宝玉 17d7d6105c chore(baoyu-cover-image): align SKILL.md version with ClawHub 2026-05-24 17:14:11 -05:00
Jim Liu 宝玉 50c6c72944 docs(readme): warn users to install only the skills they need
The marketplace bundles 20+ skills into one plugin, and bulk-installing
all of them adds context overhead to the AI agent on every run. Add a
short tip at the top of the Installation section so users browse the
available list and pick what they actually need.
2026-05-24 17:05:49 -05:00
Jim Liu 宝玉 ad1755fa23 chore: release v1.119.0 2026-05-24 17:00:39 -05:00
Jim Liu 宝玉 3cc1225b18 feat(baoyu-electron-extract): add new skill to extract Electron app sources
Extracts resources and JavaScript from any installed Electron app's
app.asar. When .js.map files embed sourcesContent, restores the
original source tree (TypeScript/JSX included); otherwise formats
minified JS/CSS with Prettier in place. Always skips node_modules
and webpack/runtime/* entries.

Source-map paths are resolved relative to each .js.map file first,
so bundler-relative entries like ../../src/main.ts restore to
readable paths under restored/ instead of hashed placeholders.

Auto-discovers apps on macOS (/Applications, ~/Applications) and
Windows (%LOCALAPPDATA%\Programs, %PROGRAMFILES%, %PROGRAMFILES(X86)%,
%APPDATA%). On other platforms pass --asar <path> explicitly.

Safety: refuses to write to /, the user home, or the current working
directory, and refuses non-empty existing output dirs without --force.
2026-05-24 17:00:31 -05:00
Jim Liu 宝玉 460bd087c6 Merge pull request #161 from xiaoyaner0201/docs/baoyu-imagine-codex-image2-notes
docs(baoyu-imagine): clarify Codex image2 fallback
2026-05-23 11:23:19 -05:00
千乘妍 (Xiaoyaner) daf0fb7bec docs(baoyu-imagine): clarify Codex image2 fallback 2026-05-22 16:45:30 +08:00
Jim Liu 宝玉 adbfa3036b chore: release v1.118.0 2026-05-21 16:56:17 -05:00
Jim Liu 宝玉 6026b619f0 Merge pull request #158 from yelban/feat/codex-imagegen-backend
feat: add codex-imagegen backend for non-Codex runtimes
2026-05-21 16:45:22 -05:00
Jim Liu 宝玉 1406a85331 chore: release v1.117.5 2026-05-21 08:55:07 -05:00
Jim Liu 宝玉 dac5867f8a chore: release v1.117.4 2026-05-21 08:52:09 -05:00
Jim Liu 宝玉 49829e7f98 Merge pull request #159 from JimLiu/feat/wechat-remote-api-bun-fix
feat(baoyu-post-to-wechat): remote-api publishing via SSH SOCKS5 tunnel
2026-05-21 08:46:11 -05:00
Yelban df16bd5d1a feat(codex-imagegen): cwd-drift immunity (path resolve + skip-git-repo-check)
Two improvements so the wrapper works the same regardless of caller cwd
or whether cwd is a git repo:

1. main.ts: resolve all filesystem path args (--prompt-file, --ref,
   --cache-dir, --log-file) to absolute paths up front. Previously only
   --image was resolved. Agents passing relative paths from arbitrary
   working directories (e.g. when SKILL.md is interpreted from a plugin
   install dir) now produce correct file lookups.

2. spawn.ts: pass --skip-git-repo-check to 'codex exec'. Without it,
   codex refuses to run outside a trusted directory:
   'Not inside a trusted directory and --skip-git-repo-check was not
   specified.' This made the wrapper fail when invoked from /tmp or
   from a non-git project tree.

3. baoyu-cover-image/SKILL.md: explicit path resolution note —
   scripts/codex-imagegen.sh lives at plugin/repo root (not shell
   cwd-relative). From the skill base directory it is at
   '../../scripts/codex-imagegen.sh'. Agents should resolve to an
   absolute path before invoking.

Verified:
- 16 unit tests pass
- e2e from /tmp (non-git, relative-path args) → 45s, 1.6MB PNG, status:ok

Composes with the existing cover-image wiring (commit e3932e4).
2026-05-21 16:26:03 +08:00
Yelban 9596d39e7b feat(baoyu-cover-image): wire SKILL.md to call codex-imagegen wrapper
Per review feedback on #158: a backend wrapper alone is not enough —
skills need explicit invocation guidance so the agent can find and
call it.

Updates skills/baoyu-cover-image/SKILL.md in two places:

1. ## Image Generation Tools — new bullet under the backend resolution
   list: 'Codex via codex exec (codex-imagegen)' with the exact command
   shape (--image / --prompt-file / --aspect / --ref / --cache-dir /
   --log-file), JSON output handling, and prerequisites.

2. ### Step 4: Generate Image, step 5 — added a concrete example
   command for the codex-imagegen branch alongside the existing generic
   instruction.

End-to-end verified: with EXTEND.md set to preferred_image_backend:
codex-imagegen, calling the command shape written in SKILL.md produces
a 1672x941 PNG in 83s (status: ok, attempts: 1, cached: false).
Output: /tmp/cover-e2e/cover.png. Structured log:
/tmp/cover-e2e/run.jsonl.

baoyu-cover-image is the validation skill; if reviewers want the same
guidance in baoyu-article-illustrator / baoyu-comic / baoyu-image-cards
/ baoyu-infographic / baoyu-slide-deck, happy to mirror the change.
2026-05-21 15:23:42 +08:00
Jim Liu 宝玉 156f8627c2 ci: install baoyu-post-to-wechat script deps before running tests
`wechat-socks-http.test.ts` (and `wechat-remote-publish.test.ts`
transitively) import `socks` from `wechat-socks-http.ts`. The CI
`npm ci` only installs root deps, so module resolution failed with
`ERR_MODULE_NOT_FOUND: Cannot find package 'socks'`.

Add a scoped `npm install` step for `skills/baoyu-post-to-wechat/scripts/`
with `--ignore-scripts` to skip postinstalls. Other skills with
private script package.json files don't yet have tests that import
their deps, so leaving them out keeps the install fast.

Co-authored-by: Dame5211 <1079825614@qq.com>
2026-05-21 02:16:31 -05:00
Jim Liu 宝玉 e0b861c148 fix(baoyu-post-to-wechat): make remote-api work under Bun & validate config strictly
The initial remote-api implementation (3b29f3c) relied on
`https.request({ agent: SocksProxyAgent })` to route token/upload/draft
calls through the SSH tunnel. Bun's `https.request` does not honor
Node's `http.Agent` contract, so the agent was silently bypassed and
requests still originated from the local IP — defeating the entire
IP-allowlist purpose. Two follow-on issues compounded it: tests read
the real `~/.baoyu-skills/.env` because Bun's `os.homedir()` ignores
test-time `process.env.HOME` mutations, and invalid config values were
silently coerced to defaults.

P1 — Bun-portable SOCKS routing:
- Drop `socks-proxy-agent` dependency. Add `socks` direct dep.
- New `wechat-socks-http.ts`: raw TCP via `SocksClient.createConnection`
  + `tls.connect({ socket, servername })` + hand-built HTTP/1.1 (status
  line parser, case-insensitive headers, chunked & content-length body
  framing). Works identically under Node and Bun because it avoids
  `http.Agent` entirely.
- Rewrite `wechat-http.ts` as a fetch-based local client and expose
  a `WechatClient = (url, init?) => Promise<WechatHttpResponse>`
  functional abstraction.
- `wechat-api.ts`: replace `agent?: http.Agent` with
  `client: WechatClient = wechatHttp` on the five HTTP-touching
  functions; `withSshTunnel` now yields a `WechatClient`.
- New `wechat-socks-http.test.ts` stands up a real SOCKS5 server
  stub + HTTP echo server and asserts `connectionCount === 1`,
  proving bytes actually traverse the proxy under both runtimes.

P2 — `HOME` honored under Bun:
- `homeDir()` reads `process.env.HOME` / `USERPROFILE` first, falling
  back to `os.homedir()`. `loadWechatExtendConfig` and `loadCredentials`
  use it, restoring test isolation.

P3 — Strict config validation:
- Replace lenient `toOptional*` helpers with `parsePort` /
  `parsePositiveInt` / `parseStrictHostKeyChecking` that throw with
  the key name. `loadWechatExtendConfig` only catches file-read
  errors so parse errors surface to the caller. Flip the corresponding
  test cases.

Verification:
- `npm test`: 261/261 pass.
- `bun test` in `scripts/`: 39/39 pass.

Co-authored-by: Dame5211 <1079825614@qq.com>
2026-05-21 02:10:23 -05:00
Jim Liu 宝玉 3b29f3c57c feat(baoyu-post-to-wechat): add remote-api publishing via SSH SOCKS5 tunnel
Re-implements PR #156 (remote WeChat API publishing for IP-allowlist
constraints) with a fully local code path: spawn a background `ssh -N -D`
SOCKS5 dynamic forward, wrap a `SocksProxyAgent`, and thread it through
the existing token/upload/draft flow. No Python helper, no SCP, no
remote files, no `AppSecret` ever leaving the local process.

Reusing `uploadImagesInHtml` (which replaces the matched `<img>` fullTag
rather than substring-replacing `src`) naturally avoids the original
PR's `html.replace(src, url)` HTML-replacement bug.

Changes:
- New `wechat-remote-publish.ts`: typed-whitelist SSH args, free-port
  allocation, SOCKS readiness polling, signal-handler-backed cleanup.
- New `wechat-http.ts`: minimal `https.request`-backed HTTP helper so a
  `SocksProxyAgent` can be passed through (undici `fetch` ignores agent).
- New `wechat-image-loader.ts`: extracts `loadUploadAsset` for reuse.
- `wechat-api.ts`: thread optional `agent?` through token/upload/draft;
  add `--remote*` CLI flags; dispatch via `withSshTunnel` when remote
  mode is selected by flag or `default_publish_method: remote-api`.
- `wechat-extend-config.ts`: add typed `remote_publish_*` keys with
  account-over-global fallback and value validation.
- Docs: SKILL.md, references/multi-account.md, references/config/
  first-time-setup.md, README.md, README.zh.md.
- Tests: 17 new node:test cases covering config parsing, SSH arg
  whitelisting, free-port allocation, multipart assembly, and HTTP
  agent threading.

Co-authored-by: Dame5211 <1079825614@qq.com>
2026-05-21 00:59:36 -05:00
Jim Liu 宝玉 f8fb457f36 chore: release v1.117.3 2026-05-20 07:58:44 -05:00
Jim Liu 宝玉 174e472a39 docs(baoyu-wechat-summary): restructure profile fields
Split aliases into group_nicknames (user's own prior names) and
aliases (nicknames from other members). Add tags field for
cross-cutting attributes. Sync SKILL.md version.
2026-05-20 07:58:20 -05:00
Jim Liu 宝玉 8b99fa7af0 fix(baoyu-post-to-wechat): sync SKILL.md version to 1.117.3 2026-05-20 07:58:18 -05:00
Jim Liu 宝玉 6699b802c0 fix(baoyu-diagram): add version field to SKILL.md 2026-05-20 07:58:17 -05:00
Jim Liu 宝玉 edcdc19ac5 feat(ci): add skill release commit validation
Add CI check to ensure commits touching skills/<name>/** use
Conventional Commit subjects. Also validates SKILL.md version
alignment during publish/sync.
2026-05-20 07:58:14 -05:00
Yelban e98fa33bc2 feat: add codex-imagegen backend for non-Codex runtimes
Implements the preferred_image_backend: codex-imagegen config key
already referenced in several SKILL.md files but with no corresponding
backend. Bridges non-Codex runtimes (Claude Code, etc.) to Codex CLI's
built-in image_gen tool via 'codex exec'. Uses the user's Codex
subscription — no OPENAI_API_KEY required.

Reliability:
- Retry with exponential backoff (default 2 retries, configurable)
- 9 classified error kinds (retryable vs non-retryable)
- Node child_process timeout with SIGTERM→SIGKILL ladder

Correctness:
- Verifies image_gen was actually invoked (not bypassed) by checking
  $CODEX_HOME/generated_images/{thread_id}/ for a PNG
- PNG magic byte validation
- Output file size sanity check

Observability:
- Structured JSONL logging with --log-file
- Verbose stderr mode with --verbose
- Returns thread_id, tool_calls, token usage in result JSON
- Raw codex event stream preserved per attempt for debugging

Efficiency:
- Idempotency cache via --cache-dir (sha256 of prompt+aspect+refs)
- Cache hit returns in <0.3s vs 50-90s cold

Features:
- --ref reference images (repeatable, passed through to codex exec --image)
- --retries, --retry-delay, --timeout all configurable
- File lock serializes concurrent invocations

Testing:
- 16 Bun unit tests across parser, cache, validator
- Path-filtered CI workflow at .github/workflows/codex-imagegen-tests.yml
- Composes with existing test.yml without conflicts

Architecture:
- scripts/codex-imagegen.sh: thin bash entrypoint (bun → npx -y bun fallback)
- scripts/codex-imagegen/: TypeScript module
  - main.ts (orchestrator), types.ts (CliOptions/GenerateResult/GenError),
    spawn.ts (child_process), parser.ts (JSONL events), validator.ts
    (image_gen check + PNG magic), cache.ts (sha256 cache + FileLock),
    logger.ts (JSONL structured logger)

Trade-offs documented in docs/codex-imagegen-backend.md:
- 5-10x slower than direct OpenAI API (or near-free on cache hit)
- ToS gray area: image_gen is documented for interactive use; non-
  interactive invocation via codex exec is not explicitly addressed.
  Suitable for personal low-volume use; users are responsible for
  ensuring their usage complies with applicable terms of service.
2026-05-20 00:50:24 +08:00
Jim Liu 宝玉 5d2a39c636 chore: release v1.117.2 2026-05-17 21:10:55 -05:00
Jim Liu 宝玉 db58bdee8c docs(skills): ban programmatic text repair on generated bitmaps
Add a text-correction policy to all raster-image skills: title/subtitle/labels/dialogue/etc. inside generated bitmaps must NOT be patched with ImageMagick, Pillow, Canvas, SVG, HTML/CSS, OCR overlays, or any other programmatic painter. On error, regenerate from a corrected prompt or switch to a lower-text variant.

Synced to: baoyu-cover-image, baoyu-article-illustrator, baoyu-comic, baoyu-image-cards, baoyu-xhs-images, baoyu-infographic, baoyu-slide-deck.
2026-05-17 21:10:50 -05:00
Jim Liu 宝玉 38cc497748 chore: release v1.117.1 2026-05-16 23:09:11 -05:00
Jim Liu 宝玉 0f95d12a09 Merge pull request #155 from JimLiu/codex/wechat-image-upload-fallback-webp
[codex] Fix WeChat image upload fallback and WebP clipboard
2026-05-16 23:07:09 -05:00
Jim Liu 宝玉 7fd9e51fc2 Fix WeChat image upload fallback and WebP clipboard 2026-05-16 22:33:07 -05:00
Jim Liu 宝玉 a5d227b4e8 Merge pull request #154 from zhangga/codex/fix-wechat-browser-article-publish
Fix WeChat browser article publishing reliability
2026-05-16 22:29:28 -05:00
zeqiang.zhang 81377416b4 Fix WeChat browser article publishing 2026-05-16 22:47:41 +08:00
Jim Liu 宝玉 a44bb08360 chore: release v1.117.0 2026-05-16 00:51:34 -05:00
Jim Liu 宝玉 a07669136c feat(baoyu-xhs-images): sync batch generation policy from baoyu-image-cards 2026-05-16 00:50:36 -05:00
Jim Liu 宝玉 b7298a60c6 feat(baoyu-slide-deck): add batch generation policy with configurable batch size 2026-05-16 00:50:33 -05:00
Jim Liu 宝玉 309f078efb feat(baoyu-image-cards): add batch generation policy with configurable batch size 2026-05-16 00:50:30 -05:00
Jim Liu 宝玉 8958ba5409 feat(baoyu-comic): add batch generation policy with configurable batch size 2026-05-16 00:50:28 -05:00
Jim Liu 宝玉 e6612628dc feat(baoyu-article-illustrator): add batch generation policy with configurable batch size 2026-05-16 00:50:25 -05:00
Jim Liu 宝玉 bd5745f837 chore: release v1.116.5 2026-05-14 01:49:35 -05:00
Jim Liu 宝玉 cb26732559 refactor(baoyu-post-to-wechat): harden Telegram QR notification
- Add 10s timeout to Telegram fetch so unreachable api.telegram.org
  doesn't block waitForLogin indefinitely.
- Move 2s QR-render wait inside sendQrToTelegram, after the env-var
  early return, so the no-op path doesn't pay the delay.
- Use viewport screenshot (captureBeyondViewport: false) as fallback
  to reduce payload size and keep the QR scannable.
2026-05-14 01:45:58 -05:00
孙斌锋 9baf570caa feat(baoyu-post-to-wechat): send WeChat login QR code to Telegram (#150)
When `TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHAT_ID` env vars are set,
the browser login flow automatically sends the WeChat QR code image
to the configured Telegram chat so headless / remote runs don't need
a screen to scan the code.

Strategy (in priority order):
1. Extract QR img.src from known DOM selectors
2. If URL-based src: re-fetch inside Chrome to carry session cookies
3. Fallback: full-page CDP screenshot

Feature is fully opt-in: skipped silently when env vars are absent.

Co-authored-by: Before SUN <beforesun@BeforedeMac-mini.local>
2026-05-14 01:21:45 -05:00
Jim Liu 宝玉 f99815cec9 chore: release v1.116.4 2026-05-14 01:10:54 -05:00
Jim Liu 宝玉 8f0663d515 refactor(baoyu-wechat-summary): streamline roast version prompts 2026-05-14 01:10:51 -05:00
Jim Liu 宝玉 20ebf6126c chore: release v1.116.3 2026-05-13 22:33:29 -05:00
Jim Liu 宝玉 64db328e61 docs: replace Claude Code references with Agent in READMEs 2026-05-13 22:33:16 -05:00
Jim Liu 宝玉 a3819b8e30 chore: release v1.116.2 2026-05-13 22:30:54 -05:00
Jim Liu 宝玉 234c2a832b docs(baoyu-wechat-summary): update example group name in SKILL.md 2026-05-13 22:30:38 -05:00
Jim Liu 宝玉 7943eb7b05 chore: release v1.116.1 2026-05-13 22:28:34 -05:00
Jim Liu 宝玉 4b4a4d3863 feat(baoyu-wechat-summary): add data_root to first-time setup flow 2026-05-13 22:28:13 -05:00
Jim Liu 宝玉 919849c863 chore: release v1.116.0 2026-05-13 22:18:01 -05:00
Jim Liu 宝玉 1faee80da4 feat(baoyu-wechat-summary): add WeChat group chat summary skill 2026-05-13 22:16:39 -05:00
Jim Liu 宝玉 dea9322d0d chore: release v1.115.4 2026-05-11 18:45:31 -05:00
Jim Liu 宝玉 86b1be83b8 docs(image-generation): emphasize Codex imagegen priority and forbid SVG/HTML substitution
Strengthen the shared image-generation backend selection rule:
- Codex `imagegen` MUST be used when listed in the available-skills inventory;
  invoke via the `Skill` tool with `skill: "imagegen"`.
- Never substitute SVG, HTML, canvas, or other code-based rendering when no
  raster backend can be resolved — fall through and ask the user instead.

Applied to docs/image-generation-tools.md and inlined into baoyu-article-illustrator
(plus its references/workflow.md), baoyu-comic, baoyu-cover-image, baoyu-image-cards,
baoyu-infographic, baoyu-slide-deck, baoyu-xhs-images per the self-containment rule.
2026-05-11 18:45:13 -05:00
Jim Liu 宝玉 a1b935b2d8 chore: release v1.115.3 2026-05-11 16:24:14 -05:00
Jim Liu 宝玉 9968d79a26 fix(baoyu-post-to-x): use toolbar media upload instead of image clipboard paste 2026-05-11 16:23:17 -05:00
Jim Liu 宝玉 6f75fb17e4 chore: add .codex-tmp and outputs to .gitignore 2026-05-11 16:23:09 -05:00
Jim Liu 宝玉 adba24281b Merge pull request #149 from fengxiaodong28/fix-browser-copy-paste
fix(browser): ensure tab activation before copy/paste in WeChat editor
2026-05-10 11:37:40 -05:00
Jim Liu 宝玉 61342ecfea chore: release v1.115.2 2026-05-10 02:49:50 -05:00
Jim Liu 宝玉 5f753dd584 fix(baoyu-post-to-x): respect Chrome plugin mode 2026-05-10 02:49:42 -05:00
Jim Liu 宝玉 076192d58e chore: release v1.115.1 2026-05-09 22:10:54 -05:00
Jim Liu 宝玉 d6d434e714 fix(image): update MiniMax default endpoint 2026-05-09 22:07:26 -05:00
Jim Liu 宝玉 6f3600d8e5 chore: release v1.115.0 2026-05-09 22:05:27 -05:00
Jim Liu 宝玉 045fe5e57e feat(baoyu-post-to-x): add Chrome Computer Use as preferred execution mode
In Codex, prefer the bundled Chrome Computer Use path for all X UI
actions (compose, article, quote, video). CDP scripts become a fallback
when Computer Use is unavailable or explicitly not requested.
2026-05-09 22:04:58 -05:00
FENG/XIAODONG 0b3b7d13b5 fix(browser): ensure tab activation before copy/paste in WeChat editor
When posting articles via browser automation, the HTML preview tab
needs to be active before copying content, and the editor tab needs
to be active before pasting. Without explicit Target.activateTarget
calls, AppleScript Cmd+C/Cmd+V would act on the wrong tab, causing
the editor body to remain empty after paste.

This fix adds Target.activateTarget before both copy and paste
operations, ensuring the correct tab is in focus for system-level
clipboard operations.
2026-05-09 22:13:06 +08:00
Jim Liu 宝玉 aa1a967a9f chore: release v1.114.1 2026-05-08 17:45:03 -05:00
Jim Liu 宝玉 d643bad53c test(baoyu-danger-gemini-web): cover generated image response fallback 2026-05-08 17:43:31 -05:00
Jim Liu 宝玉 0d977787b5 Merge pull request #146 from evilstar2016/fix/gemini-generated-image-detection
fix(gemini-webapi): add fallback scan for generated images
2026-05-08 17:40:45 -05:00
evilstar2016 516803feb4 fix(gemini-webapi): add fallback scan for generated images when wants_generated fails
The `wants_generated` detection checks `candidate[12][7][0]` and an old
`googleusercontent.com/image_generation_content/` URL pattern in the response
text. Both are no longer present in the current Gemini Web API response format,
causing the entire generated-image extraction block to be skipped even when
Gemini successfully generates images — resulting in "No image returned in
response" errors.

Generated image URLs now appear as `https://lh3.googleusercontent.com/gg-dl/`
somewhere in the response parts. This commit adds an unconditional fallback that
scans all response parts for those URLs when `generated_images` is still empty
after the existing `wants_generated` block, reusing the already-present
`collect_strings` helper and `GeneratedImage` constructor.

The existing code path is untouched — the fallback only runs when no images
were found through the original logic, so old response formats continue to work.
2026-05-06 18:48:08 +09:00
Jim Liu 宝玉 4af0506fa3 chore: release v1.114.0 2026-05-05 11:17:55 -05:00
Jim Liu 宝玉 80a1c2970a docs(release-skills): add GitHub release publishing workflow 2026-05-05 11:17:45 -05:00
Jim Liu 宝玉 cdfa0dbff9 feat(baoyu-infographic): add retro popup pop style 2026-05-05 11:17:42 -05:00
Jim Liu 宝玉 505a7e10ce chore: release v1.113.0 2026-04-25 15:03:22 -05:00
Jim Liu 宝玉 6d063734ae feat(baoyu-imagine): add DashScope Wan 2.7 image model support (#141)
* feat(baoyu-imagine): add DashScope Wan 2.7 image model support

Closes #139.

Adds the new `wan2.7-image-pro` and `wan2.7-image` model family to the
DashScope provider so users can call Wan 2.7 directly through the
official Aliyun (Bailian) API instead of going through Replicate.

- Register `wan2.7-image-pro` and `wan2.7-image` as a new `wan27` family
  in the DashScope provider with their own size resolution rules:
  pixel range `[768*768, 4096*4096]` for `wan2.7-image-pro` text-to-image,
  `[768*768, 2048*2048]` for `wan2.7-image-pro` with refs and for the
  base `wan2.7-image` model in any mode, with aspect ratios validated
  against the documented `[1:8, 8:1]` band.
- Allow up to 9 reference images per request (image editing /
  multi-image fusion). Local files are inlined as base64 data URLs;
  `http(s)://` paths are forwarded as-is. Other DashScope models still
  reject `--ref` with a hint to switch to a wan2.7 model or another
  provider.
- Drop `prompt_extend` from the request body for the Wan 2.7 family
  (not part of the Wan 2.7 API surface) and skip the Qwen-only negative
  prompt for this family.
- Allow `--provider dashscope --ref ...` in `detectProvider` so users
  can opt into Wan 2.7 reference workflows, while keeping Wan 2.7 out
  of the auto-detect ref priority list.
- Add provider, reference, and usage-example documentation, plus
  unit tests covering family routing, size derivation across the
  three pixel-budget modes, ratio rejection, explicit-size validation,
  and the new `--provider dashscope` ref opt-in path.

Made-with: Cursor

* fix(baoyu-imagine): force n=1 for DashScope wan2.7 to avoid silent multi-image billing

Cross-checked the implementation against the official Wan 2.7 image
generation & editing API reference and found that the API defaults
`parameters.n` to 4 in non-collage mode (1-4 range, billed per image).
baoyu-imagine has single-image save semantics — only the first image
in the response is kept — so without an explicit `n: 1` users would
silently pay for 3 discarded images per request.

- Always send `parameters.n: 1` in the wan2.7 request body
- Reject `--n > 1` for wan2.7 with a clear error pointing at the
  single-image save semantics
- Add tests asserting the request body shape (n=1, no prompt_extend,
  no negative_prompt) and the --n>1 rejection
- Document the defaults-vs-skill mismatch in the dashscope reference

Made-with: Cursor

* Fix DashScope Wan 2.7 review feedback
2026-04-25 14:54:08 -05:00
Jim Liu 宝玉 31d728b505 chore: release v1.112.0 2026-04-24 02:15:57 -05:00
Jim Liu 宝玉 f6d5df0594 fix(baoyu-post-to-x): add entry point guard to md-to-html.ts for module import compatibility
Wrap main() in an import.meta.url check so that importing parseMarkdown
from x-article.ts no longer triggers the CLI entry point. Mirrors the
same fix applied to baoyu-post-to-weibo.
2026-04-24 02:15:30 -05:00
Jim Liu 宝玉 4bd5fe573e feat(baoyu-article-illustrator): default to sketch-notes educational infographic style
Make `hand-drawn-edu` (infographic + sketch-notes + macaron) the universal
fallback preset when content analysis surfaces no strong signal. Rework
sketch-notes style spec around warm cream paper + black hand-drawn lines +
soft pastel section blocks.

- Change `hand-drawn-edu` preset type from flowchart to infographic; add
  `hand-drawn-edu-flow` (flowchart) and `hand-drawn-edu-compare` (comparison)
  as variants for users who need those layouts in the same warm style
- Elevate `sketch-notes` to primary style across infographic / flowchart /
  comparison / framework auto-selection; add sketch-notes column to
  Type x Style compatibility matrix
- Rewrite sketch-notes.md: macaron pastel palette, canonical single-page
  layout (title / sectioned boxes / takeaway), diagram-only rule
- Add infographic + sketch-notes + macaron prompt block to
  prompt-construction.md
- Update workflow, style-presets, and first-time-setup defaults to match
2026-04-24 02:15:25 -05:00
Jim Liu 宝玉 8c17d77209 chore: release v1.111.1 2026-04-21 16:59:35 -05:00
Jim Liu 宝玉 fb749aae36 docs(baoyu-article-illustrator): add Confirmation Policy
State that defaults / signals / EXTEND.md preferences are recommendation
inputs only — Step 3 is mandatory unless the current request opts out
explicitly.
2026-04-21 16:59:11 -05:00
Jim Liu 宝玉 a23f2e6a4b docs(baoyu-xhs-images): add Confirmation Policy
Sync Confirmation Policy with baoyu-image-cards (deprecated skill kept
functional per CLAUDE.md).
2026-04-21 16:59:11 -05:00
Jim Liu 宝玉 ae32c56a3c docs(baoyu-image-cards): add Confirmation Policy
State that defaults / signals / EXTEND.md preferences are recommendation
inputs only — Step 2 is mandatory unless the current request opts out via
`--yes` / equivalent wording.
2026-04-21 16:59:07 -05:00
Jim Liu 宝玉 4b2ad3ad18 docs(baoyu-slide-deck): add Confirmation Policy
State that defaults / signals / EXTEND.md preferences are recommendation
inputs only — Step 2 is mandatory unless the current request opts out
explicitly.
2026-04-21 16:59:07 -05:00
Jim Liu 宝玉 512b6d2e15 docs(baoyu-cover-image): add Confirmation Policy
State that defaults / keywords / EXTEND.md preferences are recommendation
inputs only — Step 2 is mandatory unless the current request opts out via
`--quick` / `quick_mode: true` / equivalent wording.
2026-04-21 16:59:03 -05:00
Jim Liu 宝玉 60d0b9c95b docs(baoyu-infographic): add Confirmation Policy as single source of truth
Consolidate the confirmation rule into one section so defaults, keywords,
and EXTEND.md preferences are treated as recommendation inputs only — never
as authorization to skip Step 4. Remove the repeated reminders that were
scattered across Step 5, Step 6, Default combination, Keyword Shortcuts,
and the preferences docs.
2026-04-21 16:58:58 -05:00
Jim Liu 宝玉 acf1d42a64 chore: release v1.111.0 2026-04-21 16:44:01 -05:00
Jim Liu 宝玉 adb783ae6d refactor: unify image-backend resolution with preferred_image_backend preference
Replaces the stateless ask-once backend rule with a 4-step resolution
(current-request override > saved preference > auto-select > ask) and
adds a single `preferred_image_backend` field (auto | ask | <backend-id>)
to every image-consuming skill's EXTEND.md schema. Runtime-native tools
(Codex `imagegen`, Hermes `image_generate`, ...) win by default; absent
field equals `auto` so existing user EXTEND.md files stay valid with no
schema version bump.

Each image-consuming skill also gains a top-level `## Changing
Preferences` section as a first-class surface for pinning a backend and
editing common one-line preferences.

Applies across:
  - baoyu-infographic (reverts in-progress two-field image_backend_mode design)
  - baoyu-comic
  - baoyu-cover-image
  - baoyu-image-cards
  - baoyu-article-illustrator
  - baoyu-slide-deck
  - baoyu-xhs-images
  - docs/image-generation-tools.md (canonical author-side doc)
2026-04-21 16:43:58 -05:00
Jim Liu 宝玉 a4bd37c390 chore: release v1.110.0 2026-04-21 15:15:19 -05:00
Jim Liu 宝玉 f8e434d2a0 chore: release v1.109.0 2026-04-21 13:39:42 -05:00
Jim Liu 宝玉 bb66585c00 fix(project): include vendored skill files in releases 2026-04-21 13:39:13 -05:00
Jim Liu 宝玉 d6442e1ab4 feat(baoyu-url-to-markdown): vendor baoyu-fetch runtime 2026-04-21 13:39:09 -05:00
Jim Liu 宝玉 2f09f48726 chore(deps): upgrade Defuddle extraction stack 2026-04-21 13:39:05 -05:00
Jim Liu 宝玉 fe26c2b639 fix(baoyu-fetch): extract X video variants 2026-04-21 13:39:01 -05:00
Jim Liu 宝玉 d7f623158c chore: release v1.108.0 2026-04-19 00:51:23 -05:00
Jim Liu 宝玉 2c800c670a [codex] Refactor skills into focused references (#135)
* docs: add runtime-neutral User Input Tools convention across skills

Introduce docs/user-input-tools.md as the author-side canonical source
and inline the tool-selection rule into every SKILL.md that prompts
the user. Also add Skill Self-Containment and User Input Tools sections
to CLAUDE.md and the copy-verbatim template to docs/creating-skills.md,
so skills stay portable across Claude Code, Codex, Hermes, and other
runtimes.

* feat: runtime-neutral image generation convention across skills

- Introduce inline `## Image Generation Tools` rule in every rendering SKILL.md so skills delegate backend choice instead of hard-coding one; author-side canonical copy lives in docs/image-generation-tools.md.
- Add `## Reference Images` support (`--ref`, frontmatter `references:` with direct/style/palette usage) to all seven image-rendering skills.
- Move build-batch.ts (with ref propagation into batch JSON) from baoyu-article-illustrator to baoyu-imagine so non-backend skills don't own backend-specific scripts; update baoyu-image-gen stub in sync and relax the CLAUDE.md deprecation note accordingly.

* refactor: slim heavy SKILL.md files and move detail to references/

Trim the four largest active skills and move presets, option tables, and
confirmation scripts into per-skill references/ so SKILL.md stays focused
on the decision flow.

- baoyu-slide-deck: 761→258, + styles-gallery.md, confirmation.md
- baoyu-image-cards: 657→280, + gallery.md, confirmation.md
- baoyu-post-to-wechat: 518→267, + multi-account.md, api-setup.md
- baoyu-imagine: 500→230, + providers/, usage-examples.md

Also un-deprecate baoyu-image-gen (drop stub warning) so it stays functional
alongside baoyu-imagine, and update CLAUDE.md to reflect that both
superseded skills are kept in sync rather than stubbed.

* refactor: slim four medium SKILL.md files into references/

Continue the P2 pattern on the next tier of skills — move option catalogs,
per-provider/adapter detail, and repeated EXTEND.md path boilerplate into
their own references so SKILL.md stays focused on the decision flow.

- baoyu-comic: 380→297 (art/tone/preset tables → auto-selection.md;
  Step 7 expanded detail → workflow.md)
- baoyu-infographic: 312→207 (layouts/styles/combinations/keywords →
  gallery.md; ASCII box tables → markdown tables)
- baoyu-format-markdown: 376→296 (title + summary generation →
  title-summary.md; ASCII box tables → markdown tables)
- baoyu-url-to-markdown: 334→169 (quality gate + recovery →
  quality-gate.md; adapters + media download → adapters.md)

* chore: sync deprecated skills with their replacements

Per project policy, baoyu-xhs-images and baoyu-image-gen are kept
functional alongside the active skills they were superseded by. Sync
their SKILL.md bodies and references/ to the slimmed baoyu-image-cards
and baoyu-imagine versions respectively, so cross-cutting fixes stay
consistent. Only the frontmatter (name, description, version, homepage)
differs — content is identical.

- baoyu-xhs-images: 657→281 (synced with baoyu-image-cards + new
  confirmation.md, gallery.md)
- baoyu-image-gen: 408→231 (synced with baoyu-imagine + new
  providers/, usage-examples.md)

* refactor: collapse EXTEND.md boilerplate into priority tables

Replace the dual bash/powershell existence-check blocks and ASCII box
art with a single markdown priority table across nine SKILL.md files.
The runtime-neutral phrasing removes shell-specific snippets without
losing the priority semantics.

* fix: address refactor-skills branch review findings

- image-gen: restore EXTEND.md paths to baoyu-image-gen (were pointing at
  baoyu-imagine) and mark descriptions of both deprecated skills as
  [Deprecated].
- xhs-images: sync neon/warm palettes with image-cards to add the
  "do not render color names/hex as visible text" safety sentence.
- infographic: restore Layout Gallery (21), Style Gallery (21),
  Recommended Combinations, and Keyword Shortcuts inline (previous
  refactor split them out but SKILL.md still depended on them), and add
  the missing references/config/first-time-setup.md + preferences-schema.md.
- image-cards / xhs-images / slide-deck / format-markdown: restore the
  sections that got over-slimmed into references/ (galleries, presets,
  dimensions, auto-selection, style x layout matrix, title/summary flow)
  and drop the now-empty shell files.
- docs/image-generation-tools.md: note that backend skills themselves
  (baoyu-imagine, baoyu-image-gen, baoyu-danger-gemini-web) are exempt
  from the ## Image Generation Tools section requirement.

* feat(image-gen): sync Z.AI GLM-Image provider from baoyu-imagine

Add Z.AI as a full provider in the deprecated baoyu-image-gen skill so
it stays in sync with baoyu-imagine's provider list.

- new scripts/providers/zai.ts + zai.test.ts (verbatim port; test
  factory trimmed to match image-gen's CliArgs shape).
- types.ts: "zai" added to Provider union and default_model.
- main.ts: rate-limit defaults, provider help text, env var help,
  --provider validation, loadProviderModule, detectProvider
  auto-detect chain, getModelForProvider, YAML parser allow-lists.
- references/config: Q2e Z.AI model question + zai slot in the
  preferences schema and batch.provider_limits.

Scope is intentionally limited to the Z.AI chain; unrelated drift
between image-gen and imagine (OpenAI image-API dialect,
aspectRatioSource, imageSizeSource) is left alone.

* docs: align inline-convention wording and note backend-skill exemption

- docs/user-input-tools.md: fix stale "links here" wording so it
  matches the inline convention already enforced everywhere else.
- CLAUDE.md §Image Generation Tools: inline the backend-skill
  exemption so readers don't need to cross-reference
  docs/image-generation-tools.md.
2026-04-19 00:48:44 -05:00
Jim Liu 宝玉 5b20f9a746 Use npm packages for shared skill code (#136) 2026-04-18 21:09:58 -05:00
Jim Liu 宝玉 9977ff520c chore: release v1.107.0 2026-04-15 21:34:22 -05:00
Jim Liu 宝玉 03bd68d3a4 docs: add Cocoon-AI credit for diagram skill design 2026-04-15 21:33:59 -05:00
Jim Liu 宝玉 39792f4360 feat(baoyu-diagram): add SVG-to-PNG @2x conversion script and consolidate references 2026-04-15 21:33:56 -05:00
Kayue Yeung 25b8a7f73d fix(claude-plugin): allow inline marketplace manifest (#130)
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-15 19:38:38 -05:00
Jim Liu 宝玉 e13092ba2a chore: release v1.106.0 2026-04-14 11:33:44 -05:00
Jim Liu 宝玉 acbcf19ba2 feat(baoyu-diagram): add architecture enrichment and structural layout patterns 2026-04-14 11:32:55 -05:00
Jim Liu 宝玉 dcd0f81433 chore: release v1.105.0 2026-04-13 19:51:57 -05:00
Jim Liu 宝玉 c938396efc feat(baoyu-diagram): unify to analyze→confirm→generate workflow
Remove single/multi mode split. Skill now analyzes any input material,
recommends diagram types and splitting strategy, confirms once, then generates.
2026-04-13 19:42:29 -05:00
Jim Liu 宝玉 1ba68c9a9c chore: release v1.104.0 2026-04-13 14:24:04 -05:00
Jim Liu 宝玉 b3f5c0a8aa fix(baoyu-post-to-wechat): verify editor focus before paste operations 2026-04-13 14:23:41 -05:00
Jim Liu 宝玉 9c45ede0c7 feat(baoyu-diagram): add Mermaid sketch step before SVG generation 2026-04-13 14:23:39 -05:00
Jim Liu 宝玉 30d2ac98ce chore: release v1.103.1 2026-04-13 11:13:06 -05:00
Jim Liu 宝玉 bfdd64bd4e chore: sync baoyu-md vendor to baoyu-markdown-to-html and baoyu-post-to-weibo 2026-04-12 20:18:05 -05:00
Jim Liu 宝玉 434d4857da chore: release v1.103.0 2026-04-12 20:17:25 -05:00
Jim Liu 宝玉 990fea4f7b fix(baoyu-post-to-wechat): decode HTML entities and strip tags from article summary
Add cleanSummaryText() to baoyu-md package: decodes HTML entities (&amp;, &lt;, &#x..., etc.)
and strips HTML tags before using frontmatter description/summary as WeChat article digest
2026-04-12 20:16:53 -05:00
Jim Liu 宝玉 517ff566a1 fix(baoyu-image-cards): prevent color names from appearing as visible text in images 2026-04-12 20:16:48 -05:00
Jim Liu 宝玉 4c9af7d92f fix(baoyu-cover-image): prevent color names and hex codes from appearing as visible text in images
Add semantic constraint to all palette references and prompt template:
color values are rendering guidance only and must not be displayed as text labels
2026-04-12 20:16:44 -05:00
Jim Liu 宝玉 46c4859d48 fix(baoyu-article-illustrator): prevent color names and hex codes from appearing as visible text in images
Add semantic constraint to palette references and prompt-construction rules:
color values are rendering guidance only and must not be displayed as text labels
2026-04-12 20:16:40 -05:00
Jim Liu 宝玉 f3f886217b feat(baoyu-diagram): add multi-diagram mode for article-wide diagram generation
- New multi-diagram mode: analyze article content and generate diagrams at identified positions
- New options: --density (minimal/balanced/per-section/rich), --mode (single/multi/auto)
- Auto mode detection: file path or multi-paragraph content → multi, short topic → single
- Output structure: diagram/{article-slug}/NN-{type}-{slug}/diagram.svg + outline.md
- Step 7: insert diagram image links into article at identified positions
- Modification guide: regenerate/add/remove individual diagrams
- Version bumped to 1.1.0 in SKILL.md
2026-04-12 20:16:36 -05:00
Jim Liu 宝玉 4d465d55d0 chore: release v1.102.0 2026-04-12 02:15:47 -05:00
Jim Liu 宝玉 11d80eeaa9 feat(baoyu-imagine): add OpenAI-compatible image API dialect support
Add --imageApiDialect flag, OPENAI_IMAGE_API_DIALECT env var, and
default_image_api_dialect config for gateways that expect aspect-ratio
size plus metadata.resolution instead of pixel size.
2026-04-12 02:14:18 -05:00
Jim Liu 宝玉 58ba4579ef chore: release v1.101.0 2026-04-12 01:20:43 -05:00
Jim Liu 宝玉 67a45a57a0 Improve baoyu-imagine Replicate compatibility (#125)
* Align Replicate image behavior with the models we actually support

Replicate image generation in baoyu-imagine no longer assumes that every model
accepts the nano-banana request schema. The Replicate provider now defaults to
google/nano-banana-2, routes supported model families through family-specific
builders and validators, blocks misleading multi-output requests before they
reach the API, and updates user-facing docs/config guidance to match the actual
contract.

Constraint: Replicate model families expose different input schemas
Constraint: Current Replicate path only saves one output image per request
Constraint: Must not change non-Replicate providers
Rejected: Keep one nano-banana-style payload for all Replicate models | triggers remote schema errors on Seedream and Wan
Rejected: Continue accepting multi-image Replicate requests and save only the first result | silently drops outputs
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Add a family-specific validator and input builder before exposing more Replicate model IDs or multi-output flags
Tested: npm test
Tested: node --test skills/baoyu-imagine/scripts/providers/replicate.test.ts skills/baoyu-imagine/scripts/main.test.ts
Not-tested: Live Replicate API calls against production models
Co-authored-by: justnode <justnode@users.noreply.github.com>

* Preserve Replicate compatibility when shared defaults leak across providers

Addressed the new PR review findings by teaching baoyu-imagine to track
where aspect-ratio defaults came from, mirroring the earlier imageSize fix,
so unsupported Replicate models can still run prompt-only requests when the
value was inherited from shared config. Also corrected Seedream 4.5 custom
size encoding to use the API's custom width/height schema instead of sending
literal WxH strings.

Constraint: Shared EXTEND defaults still need to apply globally for providers that support them
Constraint: Seedream 4.5 custom sizes must follow Replicate's documented custom size schema
Rejected: Ignore all aspect ratios for unknown Replicate models | would hide explicit unsupported CLI/task input
Rejected: Keep Seedream custom sizes as literal strings | validated locally but fails against the provider API
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Any future inherited-default validation for provider-specific flags should record the source explicitly before rejecting it
Tested: node --import tsx --test skills/baoyu-imagine/scripts/main.test.ts skills/baoyu-imagine/scripts/providers/replicate.test.ts
Tested: npm test
Not-tested: Live Replicate API calls for Seedream 4.5 custom-size requests

---------

Co-authored-by: justnode <justnode@users.noreply.github.com>
2026-04-12 01:16:32 -05:00
Jim Liu 宝玉 0b8ac256f4 chore: release v1.100.0 2026-04-12 00:30:55 -05:00
Jim Liu 宝玉 eaa0f1aa11 feat(baoyu-imagine): add Z.AI GLM-Image provider
Adds the Z.AI (智谱) provider supporting glm-image and cogview-4-250304
models via the Z.AI sync image API. Configure with ZAI_API_KEY (or
BIGMODEL_API_KEY for backward compat). Reference images are not supported yet.
2026-04-12 00:30:49 -05:00
Jim Liu 宝玉 ec5f4ffcc9 chore: release v1.99.1 2026-04-11 21:02:36 -05:00
Jim Liu 宝玉 d206e1674d fix(baoyu-article-illustrator): omit model field from batch tasks when unspecified
When --model is not provided, build-batch no longer writes a model field
into batch task entries; baoyu-imagine resolves the default from env or
EXTEND config instead of inheriting a hardcoded script default.

Adds tests covering both the omit-by-default and explicit-override paths.
2026-04-11 21:02:18 -05:00
Jim Liu 宝玉 b62ad26098 feat: add baoyu-diagram skill and baoyu-article-illustrator ink-notes style (v1.99.0)
- baoyu-diagram: new skill for generating publication-ready SVG diagrams (flowchart, structural, illustrative) with embedded styles and auto dark-mode
- baoyu-article-illustrator: add ink-notes style and mono-ink palette with presets for before/after, manifesto, and framework diagrams
2026-04-11 20:54:32 -05:00
Jim Liu 宝玉 31b2929d1c chore: release v1.98.0 2026-04-10 10:58:50 -05:00
Jim Liu 宝玉 7c2253dd3e feat(baoyu-xhs-images): add sketch-notes style, palette system, and new presets
- Restore baoyu-xhs-images as active skill (remove deprecated warning)
- Add sketch-notes style: hand-drawn educational infographic with macaron pastels
- Add palette system (macaron/warm/neon) as optional color override dimension
- Add 3 new presets: hand-drawn-edu, sketch-card, sketch-summary
- Add palette gallery documentation and palette override workflow
- Update style-presets.md with palette column
- Update prompt-assembly.md with palette override logic
- Update READMEs to reference baoyu-xhs-images
2026-04-10 10:58:29 -05:00
Jim Liu 宝玉 8be98c5afe chore: release v1.97.1 2026-04-09 13:20:21 -05:00
Jim Liu 宝玉 fe1a854bed fix(baoyu-image-cards): rename palette Zone labels to prevent text rendering in images 2026-04-09 13:20:14 -05:00
Jim Liu 宝玉 eeba585315 chore: release v1.97.0 2026-04-09 13:05:50 -05:00
Jim Liu 宝玉 1ca99ae5ae chore: add .omx to gitignore and update lockfile 2026-04-09 13:05:04 -05:00
Jim Liu 宝玉 c5df9d01bf fix(baoyu-xhs-images): improve deprecated skill description for better routing 2026-04-09 13:05:00 -05:00
Jim Liu 宝玉 ee1fc3c823 feat(baoyu-image-cards): add sketch-notes style, palette system, and 3 new presets 2026-04-09 13:04:57 -05:00
Jim Liu 宝玉 b7725a4ba8 chore: release v1.96.0 2026-04-09 10:47:51 -05:00
Jim Liu 宝玉 a596c653d3 feat(baoyu-image-cards): add image card series skill migrated from baoyu-xhs-images
Rename baoyu-xhs-images to baoyu-image-cards to decouple from Xiaohongshu platform.
Deprecate baoyu-xhs-images with migration notice pointing to baoyu-image-cards.
2026-04-09 10:47:28 -05:00
Jim Liu 宝玉 3017bfe423 chore: release v1.95.1 2026-04-09 10:00:43 -05:00
Jim Liu 宝玉 ae1d619ab2 fix(baoyu-slide-deck): add pptxgenjs dependency and detect image format by magic bytes 2026-04-09 10:00:29 -05:00
Jim Liu 宝玉 4821508c34 chore: release v1.95.0 2026-04-08 22:10:03 -05:00
Jim Liu 宝玉 f1042c8a6c feat(baoyu-slide-deck): add hand-drawn-edu preset and macaron mood 2026-04-08 22:06:54 -05:00
Jim Liu 宝玉 fa0fe441f5 feat(baoyu-infographic): add hand-drawn-edu style with macaron palette 2026-04-08 22:06:50 -05:00
Jim Liu 宝玉 90b2205914 chore: release v1.94.0 2026-04-08 17:58:33 -05:00
Jim Liu 宝玉 8e111c17b3 feat(baoyu-cover-image): add macaron palette and hand-drawn-edu style preset 2026-04-08 17:58:29 -05:00
Jim Liu 宝玉 80d7675355 chore: release v1.93.0 2026-04-08 15:49:00 -05:00
Jim Liu 宝玉 538ede2b32 feat(baoyu-article-illustrator): add hand-drawn-edu preset with sketch-notes + macaron 2026-04-08 15:48:20 -05:00
Jim Liu 宝玉 038e67fd9b refactor(baoyu-article-illustrator): extract palette as independent third dimension
- Create palettes/ directory with macaron, warm, neon palette files
- Move macaron from styles/ to palettes/ (color scheme, not rendering style)
- Extract warm palette variant from vector-illustration into shared palette
- Add neon palette extracted from retro style colors
- Update architecture from Type × Style to Type × Style × Palette
- Add Palette Gallery and override rules to styles.md
- Add Palette Override section to prompt-construction.md
- Update presets table with Palette column
- Add palette selection step to workflow
- Add preferred_palette to EXTEND.md schema
2026-04-08 14:35:51 -05:00
Jim Liu 宝玉 049462d6dd fix(baoyu-article-illustrator): add explicit style file loading step in workflow 2026-04-08 12:52:13 -05:00
Jim Liu 宝玉 c8042cef0d chore: release v1.92.0 2026-04-08 11:51:23 -05:00
Jim Liu 宝玉 15508eae43 feat(baoyu-article-illustrator): add macaron style with soft pastel color blocks 2026-04-08 11:49:49 -05:00
Jim Liu 宝玉 94eab2de63 chore: release v1.91.0 2026-04-07 01:32:21 -05:00
Jim Liu 宝玉 126b72ed36 feat: add pdf-lib dependency for PDF generation 2026-04-07 01:31:35 -05:00
Jim Liu 宝玉 38f4f253df feat(baoyu-comic): add four-panel preset, minimalist art style, and flexible image workflow
- Add minimalist art style (clean line art, limited spot color, stick-figure characters)
- Add four-panel layout with 起承转合 structure
- Add four-panel preset combining minimalist + neutral + four-panel layout
- Make character sheet conditional (skip for simple presets like four-panel)
- Add --ref failure recovery: compress → retry → fall back to prompt-only
2026-04-07 01:31:32 -05:00
Jim Liu 宝玉 8bc01debac Add concept-story preset to baoyu-comic 2026-04-07 01:09:05 -05:00
Jim Liu 宝玉 c5c54e26da chore: release v1.90.1 2026-04-05 13:55:58 -05:00
Jim Liu 宝玉 2a0bba6161 fix(baoyu-post-to-wechat): detect actual image format from buffer magic bytes
CDNs may serve WebP for URLs with .png extension. Detect real format
from magic bytes and correct content-type/extension before upload.
Also treat .webp as PNG-preferred for transparency handling.
2026-04-05 13:36:51 -05:00
Jim Liu 宝玉 c44a524fa6 chore: release v1.90.0 2026-04-03 23:45:44 -05:00
Jim Liu 宝玉 826535abe4 feat(baoyu-xhs-images): add --yes non-interactive mode for automation 2026-04-03 23:45:10 -05:00
Jim Liu 宝玉 fc50f31694 chore: release v1.89.2 2026-04-03 23:35:58 -05:00
Jim Liu 宝玉 204765a137 feat(baoyu-youtube-transcript): auto-retry with yt-dlp on empty InnerTube transcript 2026-04-03 23:35:42 -05:00
Jim Liu 宝玉 4874cd2dae feat(baoyu-post-to-wechat): use fallback chain for article summary field 2026-04-03 23:35:34 -05:00
Jim Liu 宝玉 b791ee5dc7 chore: release v1.89.1 2026-04-01 02:12:07 -05:00
Jim Liu 宝玉 450c76d955 chore(baoyu-url-to-markdown): sync vendor baoyu-fetch with login auto-detect 2026-04-01 02:12:04 -05:00
Jim Liu 宝玉 db33da26e7 feat(baoyu-fetch): auto-detect login state before extraction in interaction mode 2026-04-01 02:12:00 -05:00
Jim Liu 宝玉 c7c98ba034 chore: sync vendor baoyu-chrome-cdp across CDP skills 2026-04-01 02:11:56 -05:00
Jim Liu 宝玉 60ab574559 feat(baoyu-chrome-cdp): add gracefulKillChrome and fix killChrome process state check 2026-04-01 02:11:51 -05:00
Jim Liu 宝玉 8e2967d4a2 chore: release v1.89.0 2026-03-31 18:36:02 -05:00
Jim Liu 宝玉 3a8b0cc158 chore: update bun.lock files 2026-03-31 18:24:26 -05:00
Jim Liu 宝玉 9e3d72cf42 chore(baoyu-url-to-markdown): sync vendor baoyu-fetch with session and lifecycle changes 2026-03-31 18:24:25 -05:00
Jim Liu 宝玉 5eeb1e6d8d refactor(baoyu-danger-x-to-markdown): use graceful Chrome shutdown 2026-03-31 18:24:22 -05:00
Jim Liu 宝玉 0ee6dd4305 refactor(baoyu-danger-gemini-web): use graceful Chrome shutdown 2026-03-31 18:24:22 -05:00
Jim Liu 宝玉 7891f3c3c0 docs(baoyu-post-to-weibo): add post type auto-selection and safer CDP kill instructions 2026-03-31 18:24:17 -05:00
Jim Liu 宝玉 74f4a48ca7 feat(baoyu-post-to-x): add X session persistence and Chrome lock recovery 2026-03-31 18:24:17 -05:00
Jim Liu 宝玉 881c03262e feat(baoyu-article-illustrator): add warm palette variant for vector-illustration style 2026-03-31 18:24:15 -05:00
Jim Liu 宝玉 6afcfa80cc feat(baoyu-fetch): add X session cookie sidecar and graceful Chrome lifecycle
Add cookie sidecar to export/restore X session cookies across runs.
Implement graceful Browser.close via WebSocket before falling back to kill.
Auto-detect and clean stale Chrome profile lock artifacts on launch failure.
Wait for X session readiness (auth_token + ct0) before auto-continuing login flows.
2026-03-31 18:24:10 -05:00
Jim Liu 宝玉 9eb032a22f fix(ci): sync npm lockfile and root node tests 2026-03-27 14:23:59 -05:00
Jim Liu 宝玉 c51ae47eac chore: release v1.88.0 2026-03-27 14:12:23 -05:00
Jim Liu 宝玉 2ff139112f refactor(baoyu-url-to-markdown): replace custom pipeline with baoyu-fetch CLI 2026-03-27 14:11:05 -05:00
Jim Liu 宝玉 d0764c2739 feat(baoyu-fetch): add URL reader CLI with Chrome CDP and site adapters 2026-03-27 14:11:00 -05:00
Jim Liu 宝玉 2c14872e88 fix(baoyu-md): rename test image paths from images/ to imgs/ 2026-03-27 14:10:55 -05:00
Jim Liu 宝玉 e6d54f7492 refactor(shared-skill-packages): add files allowlist support and filter test/changelog files 2026-03-27 14:10:50 -05:00
Jim Liu 宝玉 02a4ca498a chore: release v1.87.2 2026-03-26 23:03:02 -05:00
Jim Liu 宝玉 31994be0e1 refactor(baoyu-translate): simplify translation prompts and workflow
Condense 15+ verbose translation principles to 7 concise ones.
Consolidate analysis from 8 sections to 4, review from 8 to 3.
Simplify subagent prompt template accordingly.
2026-03-26 23:02:28 -05:00
Jim Liu 宝玉 9137c5ab8c chore: release v1.87.1 2026-03-26 09:52:54 -05:00
Jim Liu 宝玉 e5d8ad91bc docs: document deprecated skills policy in CLAUDE.md 2026-03-26 09:51:51 -05:00
Jim Liu 宝玉 9c06b92a74 chore(baoyu-image-gen): add deprecation notice redirecting to baoyu-imagine 2026-03-26 09:51:48 -05:00
Jim Liu 宝玉 41a75584b3 chore: release v1.87.0 2026-03-26 09:44:06 -05:00
Jim Liu 宝玉 88843b0276 chore(baoyu-image-gen): remove deprecated skill from plugin manifest 2026-03-26 09:43:05 -05:00
Jim Liu 宝玉 6909c016b2 chore(baoyu-image-gen): remove deprecated redirect skill 2026-03-26 09:41:04 -05:00
Jim Liu 宝玉 bec1f1e2a1 chore: release v1.86.0 2026-03-25 20:09:44 -05:00
Jim Liu 宝玉 39a97678bb feat(baoyu-translate): enrich translation prompt with full analysis context
Restructure 02-prompt.md template to better leverage analysis results:
- Add source voice assessment alongside target style preset
- Extract figurative language mapping into dedicated structured table
- Include reasoning in comprehension challenges for annotation depth
- Add translation challenges section for structural/creative issues
- Provide chunk position context in subagent spawn prompts
2026-03-25 20:09:30 -05:00
Jim Liu 宝玉 6cd709b9e7 chore: release v1.85.0 2026-03-25 17:37:18 -05:00
Jim Liu 宝玉 aaf0f188dd feat(baoyu-image-gen): add deprecation redirect skill to guide migration to baoyu-imagine 2026-03-25 17:36:49 -05:00
Jim Liu 宝玉 b6bf8ecd06 feat(baoyu-imagine): auto-migrate legacy baoyu-image-gen EXTEND.md config path 2026-03-25 17:36:46 -05:00
Jim Liu 宝玉 7a0ffd9533 chore: release v1.84.0 2026-03-25 16:29:22 -05:00
Jim Liu 宝玉 69355b4ee1 feat(baoyu-imagine): rename baoyu-image-gen to baoyu-imagine 2026-03-25 16:28:06 -05:00
Jim Liu 宝玉 23b7487321 chore: release v1.83.0 2026-03-25 15:40:22 -05:00
Jim Liu 宝玉 ad8781c1c5 feat(baoyu-image-gen): add MiniMax provider with subject reference and custom sizes 2026-03-25 15:39:40 -05:00
Jim Liu 宝玉 86a3d6521b chore: release v1.82.0 2026-03-24 22:40:23 -05:00
Jim Liu 宝玉 e99ce744cd feat(baoyu-url-to-markdown): add browser fallback strategy, content cleaner, and data URI support
- Browser strategy: headless first with automatic retry in visible Chrome on failure
- New --browser auto|headless|headed flag with --headless/--headed shortcuts
- Content cleaner module for HTML preprocessing (remove ads, base64 images, scripts)
- Media localizer now handles base64 data URIs alongside remote URLs
- Capture finalUrl from browser to track redirects for output path
- Agent quality gate documentation for post-capture validation
- Upgrade defuddle ^0.12.0 → ^0.14.0
- Add unit tests for content-cleaner, html-to-markdown, legacy-converter, media-localizer
2026-03-24 22:39:17 -05:00
Jim Liu 宝玉 40f9f05c22 chore: release v1.81.0 2026-03-24 20:59:56 -05:00
Jim Liu 宝玉 09ce80357f feat(baoyu-youtube-transcript): add yt-dlp fallback and modularize codebase
Retry with alternate InnerTube client identities when YouTube returns
anti-bot responses, then fall back to yt-dlp when available. Split
monolithic main.ts into typed modules (youtube, transcript, storage,
shared, types) and add unit tests.
2026-03-24 20:59:04 -05:00
Jim Liu 宝玉 7c995fcc24 chore: release v1.80.1 2026-03-24 20:06:02 -05:00
Jim Liu 宝玉 151f1ec2a8 fix(baoyu-image-gen): use correct prompt field name for Jimeng API 2026-03-24 20:04:21 -05:00
Jim Liu 宝玉 12e207dc3f chore: release v1.80.0 2026-03-24 19:27:57 -05:00
Jim Liu 宝玉 00e74ab071 feat(baoyu-image-gen): improve Azure OpenAI provider with flexible endpoint parsing and deployment resolution 2026-03-24 19:19:49 -05:00
优弧 1653b8544b feat(baoyu-image-gen): add Azure OpenAI as independent image generation provider (#111)
Azure OpenAI differs from standard OpenAI in two ways:
1. Auth via api-key header instead of Authorization: Bearer
2. URL requires ?api-version query param with deployment path

Changes:
- New file: scripts/providers/azure.ts (generations + edits, reuses openai utilities)
- types.ts: add "azure" to Provider type and default_model
- main.ts: register azure across rate limits, CLI args, auto-detection,
  provider loading, model resolution, help text, ref validation,
  EXTEND.md parsing, and batch logging

Env vars: AZURE_OPENAI_API_KEY, AZURE_OPENAI_BASE_URL (required),
AZURE_API_VERSION, AZURE_OPENAI_IMAGE_MODEL (optional)

Co-authored-by: CatFly <zw.catfly@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 19:04:34 -05:00
Jim Liu 宝玉 dad8f3a800 chore: release v1.79.2 2026-03-23 22:39:14 -05:00
Jim Liu 宝玉 35298d7c9d fix(baoyu-post-to-weibo): add no-theme rule for article markdown-to-HTML conversion 2026-03-23 22:38:30 -05:00
Jim Liu 宝玉 f22374ab62 fix(baoyu-cover-image): simplify reference image handling based on model capability 2026-03-23 22:38:27 -05:00
Jim Liu 宝玉 d4e80b1bc3 Fix Node-compatible parser tests (#107)
* Fix Node-compatible parser tests

* Add parser test dependencies to root test env
2026-03-23 15:30:42 -05:00
Jim Liu 宝玉 a5761dc71a chore: release v1.79.1 2026-03-23 12:02:44 -05:00
Jim Liu 宝玉 a5189dff37 fix(baoyu-xhs-images): remove opacity from watermark prompt and fix CJK spacing 2026-03-23 12:01:03 -05:00
Jim Liu 宝玉 39fe872bf3 fix(baoyu-comic): fix Doraemon naming spacing and remove opacity from watermark prompt 2026-03-23 12:01:00 -05:00
Jim Liu 宝玉 52813504f8 fix(baoyu-article-illustrator): remove opacity parameter from watermark prompt 2026-03-23 12:00:53 -05:00
Jim Liu 宝玉 a4d4108cd1 docs(project): update documentation to reflect single-plugin architecture 2026-03-23 12:00:38 -05:00
Yizhou Qian 钱亦舟 d7e763f1f5 fix: consolidate to single plugin to prevent duplicate skill registration (#106)
Merge the three plugins (content-skills, ai-generation-skills,
utility-skills) into one plugin entry. Since all three shared the same
source ("./"), Claude Code cached every skill three times. A single
plugin with one source keeps the flat skills/ layout while ensuring
each skill is registered exactly once.
2026-03-23 09:46:50 -05:00
Jim Liu 宝玉 097c09c59b chore: release v1.79.0 2026-03-22 15:42:33 -05:00
Jim Liu 宝玉 e4cd8bfefc feat(baoyu-post-to-wechat): improve credential loading with multi-source resolution and diagnostics 2026-03-22 15:42:08 -05:00
Jim Liu 宝玉 3dc5f2e06f chore: release v1.78.0 2026-03-22 15:19:29 -05:00
Jim Liu 宝玉 e5d6c8ec68 feat(baoyu-url-to-markdown): add URL-specific parser layer for X/Twitter and archive.ph
- New parsers/ module with pluggable rule system for site-specific HTML extraction
- X status parser: extract tweet text, media, quotes, author from data-testid elements
- X article parser: extract long-form article content with inline media
- archive.ph parser: restore original URL and prefer #CONTENT container
- Improved slug generation with stop words and content-aware slugs
- Output path uses subdirectory structure (domain/slug/slug.md)
- Fix: preserve anchor elements containing media in legacy converter
- Fix: smarter title deduplication in markdown document builder
2026-03-22 15:18:46 -05:00
Jim Liu 宝玉 6a4b312146 chore: release v1.77.0 2026-03-22 15:17:12 -05:00
Jim Liu 宝玉 2d6fe533eb Merge pull request #105 from jzOcb/feat/chapter-end-times
feat(youtube-transcript): add end times to chapter data
2026-03-22 15:13:44 -05:00
jzocb f53af25e65 fix: address review feedback on chapter end times
- Guard last chapter end against duration=0: use Math.max(duration, ch.start)
- Remove unnecessary 'as any' cast in backfill
- Check all chapters for missing end (not just first) via .some()
- Skip backfill when needsFetch is true (about to refetch anyway)
- Wrap backfill writeFileSync in try/catch (best-effort persistence)
2026-03-22 16:07:05 -04:00
jzocb c7e32b4590 fix: backfill chapter end times for cached videos
Videos cached before the chapter end-time change would silently
lack the 'end' field when loaded from cache. This adds a migration
that detects missing 'end' fields on cache hit, computes them from
adjacent chapters, and persists the updated meta.json.

This ensures consistent output regardless of whether the data was
freshly fetched or loaded from cache.
2026-03-22 15:58:13 -04:00
jzocb 8d973f2bc5 feat(youtube-transcript): add end times to chapter data
Add 'end' field to Chapter interface and parseChapters output.
Each chapter's end is derived from the next chapter's start time,
with the last chapter ending at the video's total duration.

This makes chapter data complete and ready for downstream consumers
(e.g. video clipping with ffmpeg) without requiring them to compute
end times from adjacent chapters.

Before: { title: 'Overview', start: 0 }
After:  { title: 'Overview', start: 0, end: 21 }
2026-03-22 15:52:30 -04:00
Jim Liu 宝玉 ba20cf89f2 fix(sync-clawhub): skip failed skills instead of aborting 2026-03-21 23:25:45 -05:00
Jim Liu 宝玉 1827be9234 chore: release v1.76.1 2026-03-21 23:17:04 -05:00
Jim Liu 宝玉 93c98dfc3c docs(baoyu-youtube-transcript): fix zsh glob issue for YouTube URLs 2026-03-21 23:16:51 -05:00
Jim Liu 宝玉 fbd9f9b622 chore: release v1.76.0 2026-03-21 23:08:21 -05:00
Jim Liu 宝玉 b6e293d059 fix(baoyu-markdown-to-html): use process.execPath and tsx import in test runner 2026-03-21 23:07:46 -05:00
Jim Liu 宝玉 bb78aab095 feat(baoyu-youtube-transcript): add title heading, description summary, and cover image to markdown output 2026-03-21 23:07:44 -05:00
Jim Liu 宝玉 5071a1d0d0 chore: release v1.75.0 2026-03-21 22:44:00 -05:00
Jim Liu 宝玉 e413ade164 feat(baoyu-youtube-transcript): add sentence segmentation and improve caching 2026-03-21 22:42:43 -05:00
Jim Liu 宝玉 e52f92b193 new skill 2026-03-21 21:03:06 -05:00
Jim Liu 宝玉 603cabaef4 chore: release v1.74.1 2026-03-21 00:03:51 -05:00
Jim Liu 宝玉 7d12526e90 fix(baoyu-image-gen): broaden OpenRouter model detection and aspect ratio validation 2026-03-21 00:03:20 -05:00
Jim Liu 宝玉 e7f9764a49 Merge pull request #101 from cwandev/fix/openrouter
fix(baoyu-image-gen): align `OpenRouter` image generation with current API
2026-03-20 23:32:26 -05:00
Jim Liu 宝玉 e55f91b0ea chore: release v1.74.0 2026-03-20 23:17:14 -05:00
Jim Liu 宝玉 fe3b3d9125 feat(baoyu-markdown-to-html): pass through all rendering options from CLI
- CLI now supports --color, --font-family, --font-size, --code-theme, --mac-code-block, --line-number, --count, --legend
- convertMarkdown accepts full CliOptions instead of limited subset
- Dynamic help text showing available theme/color/font options
2026-03-20 23:17:10 -05:00
Jim Liu 宝玉 105339cf3f fix(baoyu-md): fix CSS custom property regex for quoted values and add theme layering
- Remove quotes from CSS custom property regex character class so values containing quotes are fully stripped
- grace/simple themes now layer default CSS before their own rules
- Add tests for quoted property stripping and theme layering
2026-03-20 23:17:04 -05:00
Jim Liu 宝玉 dcfd9033ae chore: release v1.73.3 2026-03-20 18:46:01 -05:00
Jim Liu 宝玉 eb416d174c fix(baoyu-post-to-wechat): prevent placeholder regex from matching longer numbered variants 2026-03-20 18:45:05 -05:00
Jim Liu 宝玉 83afacb00e chore: release v1.73.2 2026-03-20 14:46:06 -05:00
Jim Liu 宝玉 0e6bfbcabd refactor: 提取图片处理模块,本地转换不支持的格式而非回退到material接口
将WeChat正文图片的格式转换和压缩逻辑提取到独立的wechat-image-processor模块,
使用jimp和@jsquash/webp在本地将WebP/BMP/GIF等不支持的格式转换为JPEG/PNG并
压缩到1MB以内,避免依赖material接口的回退策略。同时简化了news类型封面兜底逻辑。
2026-03-20 14:43:49 -05:00
Jim Liu 宝玉 7b67ff44b8 Merge pull request #100 from AICreator-Wind/fix-body-image-upload
fix: 正文图片上传使用 media/uploadimg 接口
2026-03-20 14:00:14 -05:00
浪不能停 79c289ca92 fix: 处理WebP等不支持的格式并优化回退逻辑
- 添加WebP、BMP、TIFF等格式到不支持列表
- 对于需要回退到material接口的情况,同时调用两个接口获取URL和media_id
- 提取uploadToWechat函数避免代码重复
2026-03-20 18:18:01 +08:00
浪不能停 fc5ad4b762 fix: 处理media/uploadimg接口的文件格式和大小限制
- 添加文件大小和格式检查常量
- GIF图片或大于1MB的图片自动回退使用material接口
- 确保向后兼容现有工作流
2026-03-20 14:25:48 +08:00
浪不能停 747977416d fix: 修复newspic类型图片上传和news类型封面兜底逻辑
- uploadImagesInHtml 函数添加 articleType 参数支持
- 为 newspic 类型正文图片额外调用 material 接口收集 media_id
- 返回 firstImageSource 用于 news 类型封面兜底逻辑
- 恢复 news 类型没有显式封面时使用第一张正文图作为封面的逻辑
2026-03-20 13:57:59 +08:00
cwandev e43eec260a fix(baoyu-image-gen): narrow OpenRouter Gemini ratios 2026-03-19 22:09:23 +08:00
cwandev 96ef6e2251 fix(baoyu-image-gen): harden OpenRouter image support 2026-03-19 21:59:41 +08:00
cwandev efb7a1917a fix(baoyu-image-gen): require OpenRouter image parameters 2026-03-19 21:26:19 +08:00
cwandev 1af984a64f fix(baoyu-image-gen): align OpenRouter image generation with current API 2026-03-19 17:36:03 +08:00
Jim Liu 宝玉 60363fc2df chore: release v1.73.1 2026-03-18 22:00:27 -05:00
Jim Liu 宝玉 28ec1053f6 refactor(baoyu-danger-x-to-markdown): migrate tests from bun:test to node:test 2026-03-18 21:59:49 -05:00
Jim Liu 宝玉 5a6afd576f chore: release v1.73.0 2026-03-18 21:56:14 -05:00
Jim Liu 宝玉 ebc74a10ad feat(baoyu-danger-x-to-markdown): add video media support for X articles 2026-03-18 21:54:21 -05:00
浪不能停 e79a42fd94 fix: 正文图片上传使用 media/uploadimg 接口
- 区分正文图片和封面图片的上传接口
- 正文图片使用 media/uploadimg (返回 URL)
- 封面图片使用 material/add_material (返回 media_id)
- 添加 uploadType 参数支持两种上传方式
- 优化错误提示,告知用户 news 类型需要封面图
2026-03-18 17:23:48 +08:00
Jim Liu 宝玉 ea84f21439 chore: release v1.72.0 2026-03-18 02:25:15 -05:00
Jim Liu 宝玉 0b9e51d6cc feat(baoyu-danger-x-to-markdown): add MARKDOWN entity support for X articles 2026-03-18 02:24:15 -05:00
Jim Liu 宝玉 7cc9c92722 chore: release v1.71.0 2026-03-17 11:41:28 -05:00
Jim Liu 宝玉 fc2f0d042a feat(baoyu-image-gen): add Seedream reference image support and model-specific validation 2026-03-17 11:40:24 -05:00
Jim Liu 宝玉 cf76a0d4d5 chore: release v1.70.0 2026-03-17 01:59:13 -05:00
Jim Liu 宝玉 4a25bc2651 feat(baoyu-format-markdown): optimize title generation and auto-generate dual summaries 2026-03-17 01:59:10 -05:00
Jim Liu 宝玉 f407c950c3 docs(gemini-web): clarify CDP session reuse 2026-03-16 20:01:09 -05:00
Jim Liu 宝玉 93e54a7b86 chore: release v1.69.1 2026-03-16 12:59:35 -05:00
Jim Liu 宝玉 c1f8a9ad07 chore: sync vendored baoyu-chrome-cdp copies 2026-03-16 12:57:39 -05:00
Jim Liu 宝玉 8b8ecf61a6 fix(baoyu-chrome-cdp): tighten chrome auto-connect 2026-03-16 12:55:29 -05:00
Jim Liu 宝玉 4be6f3682a chore: sync vendored baoyu-chrome-cdp across all skills 2026-03-16 12:54:36 -05:00
Jim Liu 宝玉 b89ef02221 chore(baoyu-post-to-wechat): sync vendored baoyu-chrome-cdp 2026-03-16 12:53:51 -05:00
Jim Liu 宝玉 55356c8820 chore: release v1.69.0 2026-03-16 12:53:25 -05:00
Jim Liu 宝玉 f23d4eebc3 docs(baoyu-cover-image): clarify people reference image workflow and interactive confirmation 2026-03-16 12:52:16 -05:00
Jim Liu 宝玉 3b795b6ef3 Merge pull request #92 from bviews/main
feat(baoyu-chrome-cdp): support Chrome 146 auto-connect
2026-03-16 12:40:22 -05:00
Jim Liu 宝玉 c62d9d5a1b Merge pull request #94 from cfh-7598/fix/wechat-article-reliability
fix(baoyu-post-to-wechat): improve browser publishing reliability
2026-03-16 12:32:49 -05:00
cfh 84c56b0da3 fix(baoyu-post-to-wechat): improve browser publishing reliability
- Add retry logic (5 attempts with progressive backoff) to clickMenuByText
  to handle slow-loading home page menus
- Increase post-login wait from 2s to 5s and menu timeout from 20s to 40s
- Replace fixed 3s sleep after editor tab opens with waitForElement polling
  for #title (30s) and .ProseMirror (15s) to reliably wait for full load
- Improve title/author filling with focus() and change event dispatch
  for more reliable value setting

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 00:43:39 +08:00
Alec Jiang 462d080a0e fix(gemini-web): also respect BAOYU_CHROME_PROFILE_DIR in auto-discovery skip
Both GEMINI_WEB_CHROME_PROFILE_DIR and BAOYU_CHROME_PROFILE_DIR are
valid profile overrides (see resolveGeminiWebChromeProfileDir). Skip
existing Chrome auto-discovery when either is set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 19:02:58 +08:00
Alec Jiang bd01c370c9 fix(gemini-web): respect explicit profile config over auto-discovery
Skip existing Chrome auto-discovery when GEMINI_WEB_CHROME_PROFILE_DIR
is explicitly set, to avoid binding to the wrong browser profile/account.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 18:39:53 +08:00
Alec Jiang b7bcc8c094 fix(baoyu-chrome-cdp): keep HTTP validation in findExistingChromeDebugPort
findExistingChromeDebugPort callers (fetch_google_cookies_via_cdp) depend
on /json/version being available. TCP-only check could misclassify
non-CDP listeners as reusable, causing waitForChromeDebugPort to timeout
instead of falling back to launching a new Chrome. Restore isDebugPortReady
(HTTP) as the validator for this function; TCP-only check remains in
discoverRunningChromeDebugPort for Chrome 146 approval mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 18:11:37 +08:00
Alec Jiang d9f9da639d fix(baoyu-chrome-cdp): support Chrome 146 native remote debugging (approval mode)
Chrome 146+ blocks all HTTP endpoints (/json/version) in approval mode.
Read DevToolsActivePort ws path directly and use TCP port check instead
of HTTP discovery. Add WebSocket connect retry loop for approval dialog.
Unify findExistingChromeDebugPort to use the same mechanism.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 18:00:58 +08:00
bviews 126b3040e6 fix(gemini-web): reuse openPageSession and fix orphaned tab leak
- Replace manual CDP target orchestration with openPageSession, keeping
  behavior consistent with fetch_google_cookies_via_cdp
- Move created-tab cleanup into finally block so tabs are always closed
  even when Target.attachToTarget or Network.enable throws

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:44:53 +08:00
bviews 682888cc95 feat(baoyu-chrome-cdp): support connecting to existing Chrome session
Add ability to discover and connect to an already-running Chrome browser
(v144+) for cookie extraction, avoiding the need to launch a new window
and re-login. Uses Chrome's DevToolsActivePort from default user data
directories and process scanning as discovery mechanisms.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 15:51:57 +08:00
Jim Liu 宝玉 e31294415d chore: release v1.68.0 2026-03-14 23:08:00 -05:00
Jim Liu 宝玉 926f74e9c9 feat(baoyu-cover-image): add character preservation from reference images 2026-03-14 23:07:56 -05:00
Jim Liu 宝玉 339990e87e feat(baoyu-article-illustrator): add configurable output directory support 2026-03-14 23:07:56 -05:00
Jim Liu 宝玉 3c5c3e006d Merge pull request #89 from shixy96/fix/markdown-to-html-inline-code-in-emphasis
fix(markdown-to-html): preserve inline code inside bold/emphasis
2026-03-14 22:27:14 -05:00
shixy 2aa9790789 fix: preserve inline code in cjk emphasis 2026-03-14 17:12:03 +08:00
shixy 38fc733b99 fix(markdown-to-html): preserve inline code inside bold/emphasis
The `extractText()` helper in `preprocessCjkEmphasis()` only handled
`text` nodes and nodes with `children`. `inlineCode` AST nodes (which
have a `value` but no `children`) fell through to the default empty-
string return, silently dropping their content.

For example `**算出 \`logits\`**` rendered as `<strong>算出 </strong>`
with the code span completely lost.

Add an `inlineCode` branch that wraps the node value in backticks so
the downstream `marked` pass can turn it into a proper `<code>` element.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 16:13:02 +08:00
Jim Liu 宝玉 4d2b95d1d1 chore: release v1.67.0 2026-03-13 19:12:47 -05:00
JianJang2017 ac2ce0b8b6 Add qwen-image-2.0-pro support for baoyu-image-gen 2026-03-13 19:09:54 -05:00
Jim Liu 宝玉 de7dc85361 chore: sync shared skill package vendor test files 2026-03-13 17:56:53 -05:00
Jim Liu 宝玉 0c02b81885 chore: release v1.66.1 2026-03-13 17:56:30 -05:00
Jim Liu 宝玉 774ad784d8 test: migrate tests from centralized mjs to colocated TypeScript
Move test files from tests/ directory to colocate with source code,
convert from .mjs to .ts using tsx runner, add workspaces and npm
cache to CI workflow.
2026-03-13 16:36:06 -05:00
Jim Liu 宝玉 484b00109f chore: release v1.66.0 2026-03-13 16:20:38 -05:00
Jim Liu 宝玉 ac217d5402 test: add test infrastructure with CI workflow and image-gen unit tests 2026-03-13 16:17:01 -05:00
Jim Liu 宝玉 70d9f63727 docs(baoyu-image-gen): add Jimeng and Seedream provider documentation 2026-03-13 16:16:57 -05:00
Jim Liu 宝玉 3398509d9e refactor(baoyu-image-gen): export functions for testability and add module entry guard 2026-03-13 16:16:50 -05:00
Jim Liu 宝玉 a11613c11b Merge pull request #88 from JimLiu/lindaifeng/main
fix(image-gen): tighten Jimeng provider behavior
2026-03-13 15:58:07 -05:00
Jim Liu 宝玉 cb17cb9cca fix(image-gen): tighten Jimeng provider behavior 2026-03-13 15:39:46 -05:00
Jim Liu 宝玉 88d6e09472 Merge pull request #87 from lindaifeng/main
Integrate DreamGraph and DoubaoGraph
2026-03-13 15:35:15 -05:00
Jim Liu 宝玉 004236682d chore: release v1.65.1 2026-03-13 15:29:52 -05:00
Jim Liu 宝玉 7e07c1bb84 refactor(baoyu-translate): replace remark/unified with markdown-it for chunk parsing
Simplifies dependencies and adds main.ts CLI entry point with exported
functions for programmatic reuse.
2026-03-13 15:27:25 -05:00
lindaifeng c151f33775 Merge branch 'JimLiu:main' into main 2026-03-14 00:39:44 +08:00
ldf 32003da694 集成即梦生图和豆包生图 2026-03-14 00:37:46 +08:00
Jim Liu 宝玉 6f38724163 chore: release v1.65.0 2026-03-13 10:58:13 -05:00
Jim Liu 宝玉 374a6b28fd feat(baoyu-post-to-wechat): add placeholder image upload and fix frontmatter parsing 2026-03-13 10:58:13 -05:00
Jim Liu 宝玉 95970480c8 Merge pull request #86 from JimLiu/codex/baoyu-md-shared
refactor: share markdown renderer via baoyu-md
2026-03-13 10:45:34 -05:00
Jim Liu 宝玉 fc324319d8 fix: preserve weibo image metadata in shared md package 2026-03-13 10:44:23 -05:00
Jim Liu 宝玉 056fa06c3b refactor: share markdown renderer via baoyu-md 2026-03-13 10:20:28 -05:00
Jim Liu 宝玉 c62cda4424 chore: release v1.64.0 2026-03-13 09:20:41 -05:00
Jim Liu 宝玉 12b43e166d feat(baoyu-image-gen): add OpenRouter provider support 2026-03-13 09:18:34 -05:00
Jim Liu 宝玉 c1e1526c84 chore: release v1.63.0 2026-03-13 00:24:32 -05:00
Jim Liu 宝玉 0279fa403d feat(baoyu-url-to-markdown): add defuddle.md API fallback, YouTube transcripts, and modular converter architecture 2026-03-13 00:22:03 -05:00
Jim Liu 宝玉 994e47d1be docs: fix Claude Code marketplace repo casing 2026-03-12 12:46:25 -05:00
Jim Liu 宝玉 6a71d5080d chore: release v1.62.0 2026-03-12 01:51:18 -05:00
Jim Liu 宝玉 1e6e6637ac feat(baoyu-infographic): support flexible aspect ratios
Accept custom W:H ratios (e.g., 3:4, 4:3, 2.35:1) in addition to
named presets (landscape/portrait/square).
2026-03-12 01:50:36 -05:00
Jim Liu 宝玉 873b60aee5 docs(wechat): replace credential-like placeholders 2026-03-12 00:53:33 -05:00
Jim Liu 宝玉 b88ac59133 fix: set strict mode on plugins to prevent duplicated slash commands
Closes #79
2026-03-11 23:29:27 -05:00
Jim Liu 宝玉 d889c04084 chore: release v1.61.0 2026-03-11 23:24:43 -05:00
Jim Liu 宝玉 5276fae6bd feat(baoyu-post-to-wechat): add multi-account support
Support multiple WeChat Official Accounts via EXTEND.md accounts block.
Each account gets isolated Chrome profile, credential resolution chain
(inline → prefixed env → unprefixed env), and --account CLI arg.
2026-03-11 23:24:02 -05:00
Jim Liu 宝玉 c0941f8089 fix: exclude out/dist/build dirs and bun.lockb from skill release files 2026-03-11 22:55:47 -05:00
Jim Liu 宝玉 42b8b1fc99 fix: use proper MIME types in skill publish to fix ClawhHub rejection 2026-03-11 22:46:06 -05:00
Jim Liu 宝玉 53c788eb3b chore: release v1.60.0 2026-03-11 22:22:03 -05:00
Jim Liu 宝玉 c79066e96e fix(baoyu-post-to-x): add missing fs import in x-article 2026-03-11 22:18:11 -05:00
Jim Liu 宝玉 05dba5c320 refactor: publish skills directly from synced vendor 2026-03-11 21:36:19 -05:00
Jim Liu 宝玉 270a9af804 refactor: simplify CLAUDE.md, move detailed docs to docs/
CLAUDE.md reduced from 621 to 76 lines (↓88%). Detailed guidelines
moved to on-demand reference files under docs/.
2026-03-11 21:01:59 -05:00
Jim Liu 宝玉 3bba18c1fe build: commit vendored shared skill packages 2026-03-11 20:45:25 -05:00
Jim Liu 宝玉 069c5dc7d7 refactor: unify skill cdp and release artifacts 2026-03-11 19:38:59 -05:00
Jim Liu 宝玉 00bf946403 支持复用已有 Chrome CDP 实例,修复端口检测顺序问题 2026-03-11 17:24:18 -05:00
Jim Liu 宝玉 1cb54420e0 chore: release v1.59.1 2026-03-11 17:08:33 -05:00
Jim Liu 宝玉 6363bd83e2 refactor(scripts): replace clawhub CLI with local sync-clawhub.mjs 2026-03-11 16:45:57 -05:00
Jim Liu 宝玉 7b8247544d refactor(baoyu-post-to-weibo): add entry point guard to md-to-html.ts 2026-03-11 16:45:56 -05:00
Jim Liu 宝玉 a9576ebc67 fix(baoyu-post-to-x): remove disable-blink-features Chrome flag 2026-03-11 16:45:55 -05:00
Jim Liu 宝玉 79ca378229 fix(baoyu-translate): improve annotation density and style preset passing
- Add short text (<5 sentences) annotation reduction rule to translator's notes
- Explicitly pass resolved style preset to 02-prompt.md assembly in all modes
2026-03-11 16:45:51 -05:00
Jim Liu 宝玉 e1a1fe23cb Merge pull request #77 from jackL1020/claude/claude-md-mmlh0po6953zhhbf-D8rFo
docs: Update CLAUDE.md with v1.59.0 features and skill registry
2026-03-11 13:40:27 -05:00
Claude 3d85a7e663 docs: fix marketplace.json path (it's in .claude-plugin/, not root)
https://claude.ai/code/session_01DREj1A7KKhNVE1RXjmg9ZH
2026-03-11 03:28:56 +00:00
Claude 10aabb39f8 docs: update CLAUDE.md to reflect v1.59.0 codebase state
- Add 8 missing skills to architecture tree (baoyu-post-to-weibo,
  baoyu-infographic, baoyu-image-gen, baoyu-url-to-markdown,
  baoyu-format-markdown, baoyu-markdown-to-html, baoyu-translate)
- Fix marketplace.json path (root, not .claude-plugin/)
- Rename SKILL_DIR → baseDir in Script Directory template (v1.57.0 change)
- Add ClawHub/OpenClaw publishing section with sync-clawhub.sh usage
- Add openclaw frontmatter template and requirement for new skills
- Document baoyu-image-gen batch parallel generation capability
- Update Project Overview to mention all AI providers
- Expand Key Dependencies and Choosing a Category tables
- Add image-gen API key requirement note

https://claude.ai/code/session_01DREj1A7KKhNVE1RXjmg9ZH
2026-03-11 03:28:14 +00:00
Jim Liu 宝玉 661c74cce4 docs: add contributor credit to v1.59.0 changelog 2026-03-09 00:58:14 -05:00
Jim Liu 宝玉 82445f2a39 chore: release v1.59.0 2026-03-09 00:55:52 -05:00
Jim Liu 宝玉 df9ce95c46 fix: restore Google as default provider when multiple keys available
EXTEND.md default_provider still takes priority over auto-detect.
Priority: CLI --provider > EXTEND.md default_provider > auto-detect (Google first).
2026-03-09 00:52:53 -05:00
Jim Liu 宝玉 88b433d565 docs: improve skill documentation clarity
- Fix Provider Selection: default to Replicate when multiple keys available (matches code)
- Add Replicate column to Quality Presets table (normal→1K, 2k→2K)
- Add Replicate aspect ratio behavior (match_input_image when --ref without --ar)
- Remove stale Google Imagen reference from Aspect Ratios
- Add batch file format example with JSON schema to SKILL.md
- Note that batch paths resolve relative to batch file directory
- Move batch execution strategy in article-illustrator before numbered steps
- Fix translate image-language reminder to use standard markdown syntax
  with note to match article's own image syntax convention
2026-03-09 00:43:05 -05:00
Jim Liu 宝玉 5acef7151b feat: add batch parallel image generation and provider-level throttling
- Add --batchfile and --jobs flags for multi-image parallel generation
  with per-provider concurrency control and rate limiting
- Refactor main.ts into prepareSingleTask/prepareBatchTasks/runBatchTasks
  with worker pool pattern and up to 3 retries per image
- Fix Replicate provider: use image_input array (nano-banana-pro schema),
  add match_input_image aspect ratio, add quality-to-resolution mapping
- Improve OpenAI error message for missing API key (Codex auth hint)
- Expand non-retryable error detection (4xx codes, disabled models)
- Add batch config to EXTEND.md schema (max_workers, provider_limits)
- Add build-batch.ts for article-illustrator batch workflow integration
- Add image-language awareness pass to baoyu-translate

Co-authored-by: 敖氏 <aoshi@MacBook-Air.local>
2026-03-09 00:07:45 -05:00
Jim Liu 宝玉 e4d4ec8334 chore: release v1.58.0 2026-03-08 22:30:46 -05:00
Jim Liu 宝玉 4935dc8b1a fix: surface agent-browser startup errors 2026-03-08 22:25:34 -05:00
Jim Liu 宝玉 c43ed57ffc Merge pull request #69 from liby/feat/xdg-config-path
feat: add XDG config path support for EXTEND.md
2026-03-08 22:19:52 -05:00
Jim Liu 宝玉 d96368492e Merge pull request #66 from luojiyin1987/fix/harden-command-exec-and-js-escaping
fix: harden command execution and JS literal escaping
2026-03-08 22:10:45 -05:00
Jim Liu 宝玉 cfc647fd38 Merge pull request #68 from luojiyin1987/docs/sync-readme-and-claude-deps
docs: sync README utility skills and CLAUDE dependency notes
2026-03-08 22:04:40 -05:00
Jim Liu 宝玉 5144335916 chore: release v1.57.0 2026-03-08 19:24:45 -05:00
Jim Liu 宝玉 fa82c3d910 refactor(baoyu-xhs-images): use skill interface instead of direct script invocation for image generation 2026-03-08 19:24:03 -05:00
Jim Liu 宝玉 90730bc551 refactor(baoyu-comic): use skill interface instead of direct script invocation for image generation 2026-03-08 19:24:02 -05:00
Jim Liu 宝玉 154ccfe3ff refactor: add openclaw metadata and rename SKILL_DIR to baseDir across all skills 2026-03-08 19:22:24 -05:00
Jim Liu 宝玉 93efa5aeb1 refactor(baoyu-comic): add openclaw metadata, baseDir rename, and decouple image skill path 2026-03-08 19:22:19 -05:00
Jim Liu 宝玉 40ccbed74a refactor(baoyu-danger-x-to-markdown): add openclaw metadata, baseDir rename, and dynamic script path 2026-03-08 19:22:18 -05:00
Jim Liu 宝玉 fbf2bc02c5 refactor(baoyu-danger-gemini-web): add openclaw metadata, baseDir rename, and dynamic script path 2026-03-08 19:22:18 -05:00
Jim Liu 宝玉 a37c80e142 feat: add ClawHub/OpenClaw publishing support 2026-03-08 19:22:13 -05:00
Jim Liu 宝玉 2e68aa64a4 chore: release v1.56.1 2026-03-08 17:42:38 -05:00
Bryan Lee a702513487 feat: add XDG config path support for EXTEND.md
Add $XDG_CONFIG_HOME/baoyu-skills/<skill>/EXTEND.md as second priority
path between project-level and legacy $HOME/.baoyu-skills/ paths.

Priority order:
1. .baoyu-skills/<skill>/EXTEND.md (project)
2. $XDG_CONFIG_HOME/baoyu-skills/<skill>/EXTEND.md (XDG, new)
3. $HOME/.baoyu-skills/<skill>/EXTEND.md (legacy fallback)

Updated: CLAUDE.md template, 16 SKILL.md files, 2 workflow.md
references, and 2 extend-config.ts runtime loaders.
2026-03-09 04:54:26 +08:00
Jim Liu 宝玉 059ebc759c fix(baoyu-post-to-weibo): simplify article image insertion with Backspace-based placeholder deletion 2026-03-08 15:49:54 -05:00
Jim Liu 宝玉 d4a5a34865 chore: release v1.56.0 2026-03-08 14:08:40 -05:00
Jim Liu 宝玉 b3b2f8ea36 feat(baoyu-xhs-images): streamline workflow with smart confirm (6→4 steps) 2026-03-08 14:07:59 -05:00
Jim Liu 宝玉 f19ed00cb2 fix(baoyu-post-to-wechat): improve image upload with file chooser interception 2026-03-08 14:07:55 -05:00
Jim Liu 宝玉 8e58855047 feat(baoyu-article-illustrator): add preset-first selection and categorized style presets 2026-03-08 14:07:53 -05:00
Jim Liu 宝玉 fdfb0e2dfe chore: release v1.55.0 2026-03-08 13:07:49 -05:00
Jim Liu 宝玉 b4e319ba8f docs: add credits section to READMEs 2026-03-08 13:07:18 -05:00
Jim Liu 宝玉 691a9c2272 feat(baoyu-xhs-images): add screen-print style and style presets
Add screen-print as 11th visual style with poster aesthetic. Add --preset flag
for quick style + layout selection with 23 built-in presets (knowledge-card,
poster, cinematic, etc.). Update auto-selection and compatibility matrices.
2026-03-08 13:07:14 -05:00
Jim Liu 宝玉 16db99a3ef feat(baoyu-cover-image): add screen-print rendering and duotone palette
Add screen-print as 7th rendering style with halftone textures and limited
colors. Add duotone as 10th color palette for dramatic two-color compositions.
Update all compatibility matrices and style presets (poster-art, mondo,
art-deco, propaganda, cinematic).
2026-03-08 13:07:10 -05:00
Jim Liu 宝玉 adb587439d feat(baoyu-article-illustrator): add screen-print style and style presets
Add screen-print as new style option with poster aesthetic (bold silhouettes,
halftone textures, limited colors). Add --preset flag for quick type + style
selection (e.g., --preset tech-explainer, --preset opinion-piece).
2026-03-08 13:07:05 -05:00
luojiyin 6b2b28b421 docs: fix CLAUDE dependency statement 2026-03-08 21:21:54 +08:00
luojiyin 0df63c3a0b docs: add markdown-to-html to README.zh utility skills 2026-03-08 21:21:54 +08:00
luojiyin 7834d632e0 docs: add markdown-to-html to README utility skills 2026-03-08 21:21:47 +08:00
luojiyin ca9a0a1404 fix: harden wechat agent-browser command and eval handling 2026-03-08 20:34:43 +08:00
luojiyin 366e7b5403 fix: use execFileSync for google curl requests 2026-03-08 20:34:39 +08:00
luojiyin 709e026be1 fix: use spawnSync for autocorrect command 2026-03-08 20:34:35 +08:00
Jim Liu 宝玉 f6cef6bcbb chore: release v1.54.1 2026-03-07 16:11:24 -06:00
Jim Liu 宝玉 4300e6899c fix(baoyu-post-to-x): keep browser open for manual publishing 2026-03-07 16:10:19 -06:00
Jim Liu 宝玉 4435366898 docs: add star history to readmes 2026-03-07 16:07:29 -06:00
Jim Liu 宝玉 69cd89184e chore: release v1.54.0 2026-03-06 23:40:53 -06:00
Jim Liu 宝玉 5771e6d8c5 feat(baoyu-translate): add external glossary files and frontmatter transformation
Support glossary_files in EXTEND.md for loading glossaries from external
markdown tables or YAML files. Add frontmatter transformation rules to
rename source metadata fields with source prefix during translation.
2026-03-06 23:38:27 -06:00
Jim Liu 宝玉 970f5e741a feat(baoyu-post-to-wechat): default bottom citations for markdown input
Enable bottom citations by default for markdown articles. Add --no-cite
flag to all posting scripts (wechat-api, wechat-article, md-to-wechat)
to optionally disable. WeChat mp links stay inline.
2026-03-06 23:38:23 -06:00
Jim Liu 宝玉 8cea550627 feat(baoyu-markdown-to-html): add --cite option for bottom citations
Add optional --cite flag to convert ordinary external links to numbered
bottom citations. WeChat links stay inline. Also translate theme and
color descriptions from Chinese to English for consistency.
2026-03-06 23:38:19 -06:00
Jim Liu 宝玉 82029160d1 feat(baoyu-format-markdown): improve title and summary generation quality
Add style-differentiated title candidates (subversive, solution, suspense,
concrete number), prohibited patterns, hook-first principles, and stronger
summary guidelines with concrete detail requirements.
2026-03-06 23:38:15 -06:00
Jim Liu 宝玉 46af919634 chore: release v1.53.0 2026-03-06 21:20:22 -06:00
Jim Liu 宝玉 5560db595a feat(baoyu-url-to-markdown): add HTML snapshot saving and Defuddle fallback pipeline
- Save rendered HTML as sibling -captured.html file alongside markdown
- Defuddle-first conversion with automatic fallback to legacy Readability/selector extractor
- Add rawHtml, conversionMethod, fallbackReason to ConversionResult
- Log converter method and fallback reason in CLI output
2026-03-06 21:18:21 -06:00
Jim Liu 宝玉 3b031c7768 chore: release v1.52.0 2026-03-06 20:54:11 -06:00
Jim Liu 宝玉 39c7e86a8d feat(baoyu-post-to-weibo): add video support and improve upload reliability
- Add --video flag for video uploads (max 18 files total)
- Switch from clipboard paste to DOM.setFileInputFiles for uploads
- Add Chrome health check with auto-restart for unresponsive instances
- Add navigation check to ensure Weibo home page before posting
2026-03-06 20:27:18 -06:00
Jim Liu 宝玉 f00ca11d9a chore: release v1.51.2 2026-03-06 17:21:40 -06:00
Jim Liu 宝玉 87c2509268 fix(baoyu-infographic): add credential stripping to address Snyk W007 audit 2026-03-06 17:21:10 -06:00
Jim Liu 宝玉 20bf6d9ad8 fix(release-skills): use generic changelog language patterns to avoid URL scanner false positive 2026-03-06 17:21:06 -06:00
Jim Liu 宝玉 33bc0a6255 chore: release v1.51.1 2026-03-06 16:05:25 -06:00
Jim Liu 宝玉 c15a44b439 fix(security): remove curl|bash, enforce HTTPS-only downloads
- Replace all curl|bash install suggestions with brew/npm
- downloadFile: HTTPS-only, reject http:// URLs
- downloadFile: add redirect limit (max 5)
- Remove unused http import from md-to-html scripts
- Add Security Guidelines section to CLAUDE.md
- Update SKILL.md profile dir references
2026-03-06 16:03:10 -06:00
Jim Liu 宝玉 6e533f938f refactor: unify Chrome CDP profile path across all skills
All skills now share a single Chrome profile at:
- macOS: ~/Library/Application Support/baoyu-skills/chrome-profile
- Linux: $XDG_DATA_HOME/baoyu-skills/chrome-profile
- Env override: BAOYU_CHROME_PROFILE_DIR

Fixes baoyu-post-to-weibo incorrectly reusing x-browser-profile.
Legacy per-skill env vars retained as fallback.
2026-03-06 16:03:01 -06:00
Jim Liu 宝玉 c60eb85629 docs: add baoyu-post-to-weibo to README (en/zh) 2026-03-06 14:59:37 -06:00
Jim Liu 宝玉 20d2a78f87 chore: release v1.51.0 2026-03-06 14:56:49 -06:00
Jim Liu 宝玉 bb63ee2a2e feat(baoyu-format-markdown): add title/summary multi-candidate selection with auto_select support 2026-03-06 14:56:32 -06:00
Jim Liu 宝玉 5fc697166d feat(baoyu-post-to-weibo): add Weibo posting skill with text, images, and headline articles 2026-03-06 14:56:28 -06:00
Jim Liu 宝玉 1ed204bd5a chore: release v1.50.0 2026-03-06 02:33:47 -06:00
Jim Liu 宝玉 be2cbecfb0 feat(baoyu-translate): expand translation style presets from 4 to 9 with CLI flag support
Add 5 new style presets (academic, business, humorous, conversational,
elegant) to existing options. Wire --style CLI flag, update subagent
prompt template with style section, and document in both READMEs.
2026-03-06 02:33:25 -06:00
Jim Liu 宝玉 bce96e411d chore: release v1.49.0 2026-03-06 01:55:40 -06:00
Jim Liu 宝玉 fe404c493d feat(baoyu-translate): extract workflow mechanics, expand triggers, and save frontmatter in chunks
- Extract Step 2 materialization details to references/workflow-mechanics.md
- Expand description trigger keywords (改成中文, 快翻, 本地化, etc.)
- Add proactive warning for long content in quick mode
- Save frontmatter to chunks/frontmatter.md in chunk.ts
- Fix step number references in upgrade prompt
- Simplify refined-workflow draft step to reference SKILL.md principles
2026-03-06 01:55:18 -06:00
Jim Liu 宝玉 6dd754ccdd feat(baoyu-format-markdown): add reader-perspective analysis phase and restructure workflow
- Add Step 2 content analysis from reader's perspective (highlights, structure, formatting issues)
- Output analysis to {filename}-analysis.md as formatting blueprint
- Restructure from 8 steps to 7 steps with clearer phase separation
- Add explicit do/don't formatting principles
- Add completion report template with change summary
2026-03-06 01:55:13 -06:00
Jim Liu 宝玉 1f4415c8ff chore: release v1.48.2 2026-03-06 00:01:19 -06:00
Jim Liu 宝玉 b25eb87bb4 feat(baoyu-translate): add figurative language & emotional fidelity checks to refined workflow and quick mode 2026-03-06 00:01:07 -06:00
Jim Liu 宝玉 dbf35503a3 chore: add posts/ to .gitignore 2026-03-06 00:01:04 -06:00
Jim Liu 宝玉 36ccbd0300 chore: release v1.48.1 2026-03-05 23:35:52 -06:00
Jim Liu 宝玉 bc878e5157 feat(baoyu-translate): add figurative language analysis and meaning-first translation principles 2026-03-05 23:35:08 -06:00
Jim Liu 宝玉 727375afa3 chore: release v1.48.0 2026-03-05 20:54:11 -06:00
Jim Liu 宝玉 f2c914887a feat(baoyu-translate): add --output-dir to chunk.ts and improve refined workflow
- chunk.ts: add --output-dir option so chunks write to output directory instead of source directory
- Refined workflow: split Review into Critical Review + Revision (5→6 steps)
- Add Europeanized language diagnosis for CJK targets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 20:53:54 -06:00
Jim Liu 宝玉 9f76a96741 chore: release v1.47.0 2026-03-05 19:35:07 -06:00
Jim Liu 宝玉 67e3e11cce feat(skills): add cross-platform PowerShell support for EXTEND.md checks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:32:35 -06:00
Jim Liu 宝玉 5b4ba3ac3f feat(baoyu-translate): add three-mode translation skill with glossary support
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:32:27 -06:00
Jim Liu 宝玉 ce259e4547 chore: release v1.46.0 2026-03-05 14:34:35 -06:00
Jim Liu 宝玉 fff1a54b6b feat(baoyu-url-to-markdown): add --output-dir option for custom output directory
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:34:07 -06:00
Jim Liu 宝玉 9437581c48 chore: release v1.45.1 2026-03-05 10:31:54 -06:00
Jim Liu 宝玉 bd4db203f8 refactor(skills): replace hardcoded npx -y bun with ${BUN_X} runtime variable
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:31:29 -06:00
Jim Liu 宝玉 51387498a5 refactor(project): add BUN_X runtime detection with bun-first fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:31:24 -06:00
Jim Liu 宝玉 6d1336d8bd chore: release v1.45.0 2026-03-05 00:55:59 -06:00
Jim Liu 宝玉 42931d6294 feat(baoyu-post-to-x): add post-composition verification and improve stability
Add automatic check after article images are inserted: verifies remaining
XIMGPH placeholders and expected vs actual image count. Increase CDP timeout
to 60s and add 3s DOM stabilization delay between image insertions.
2026-03-05 00:55:56 -06:00
Jim Liu 宝玉 4c9e57777b chore: release v1.44.0 2026-03-05 00:37:52 -06:00
Jim Liu 宝玉 210905ef66 feat(baoyu-url-to-markdown): add media download and cover image extraction
- Add --download-media flag to download images/videos to local dirs and rewrite markdown links
- Extract coverImage from page meta (og:image) into YAML front matter
- Handle data-src lazy loading for WeChat and similar sites
- Add EXTEND.md preferences with first-time setup for download_media setting
- Add media-localizer module adapted from x-to-markdown
2026-03-05 00:37:09 -06:00
Jim Liu 宝玉 bd4de7b995 chore: release v1.43.2 2026-03-05 00:12:44 -06:00
Jim Liu 宝玉 832f06e86e refactor(baoyu-url-to-markdown): replace custom extraction with defuddle library 2026-03-05 00:11:59 -06:00
Jim Liu 宝玉 a7ba3d73db chore: release v1.43.1 2026-03-02 12:07:42 -06:00
Jim Liu 宝玉 e2a15aadcc feat(baoyu-danger-x-to-markdown): WSL auto-detection and debug port env var
Add X_DEBUG_PORT env var for fixed port in WSL.
Auto-detect WSL and resolve Chrome profile to Windows-native path.
2026-03-02 12:07:16 -06:00
Jim Liu 宝玉 e3f00c103e feat(baoyu-danger-gemini-web): WSL auto-detection and debug port env var
Add GEMINI_WEB_DEBUG_PORT env var for fixed port in WSL.
Auto-detect WSL and resolve Chrome profile to Windows-native path.
2026-03-02 12:07:12 -06:00
Jim Liu 宝玉 a501202ab6 feat(baoyu-post-to-wechat): auto-detect WSL for Chrome profile path
In WSL2, auto-resolve Windows USERPROFILE via cmd.exe/wslpath
so Chrome profile is stored on Windows-native filesystem.
2026-03-02 12:07:07 -06:00
Jim Liu 宝玉 166cb323dd feat(baoyu-post-to-x): auto-detect WSL for Chrome profile path
In WSL2, auto-resolve Windows USERPROFILE via cmd.exe/wslpath
so Chrome profile is stored on Windows-native filesystem.
Fixes login state persistence when using Windows Chrome from WSL.
2026-03-02 12:07:02 -06:00
Jim Liu 宝玉 b7585ebf71 chore: release v1.43.0 2026-03-02 11:53:22 -06:00
Jim Liu 宝玉 3205239067 feat(baoyu-post-to-x): support env var overrides for debug port and profile directory 2026-03-02 11:52:49 -06:00
Jim Liu 宝玉 c5b3066962 feat(baoyu-post-to-wechat): support env var overrides for debug port and profile directory 2026-03-02 11:52:45 -06:00
Jim Liu 宝玉 84d7777246 chore: release v1.42.3 2026-03-02 00:09:33 -06:00
Jim Liu 宝玉 8412392788 fix(baoyu-image-gen): use standard size presets for DashScope aspect ratio mapping
Replace free-form dimension calculation with closest-match from predefined
standard sizes to avoid invalid resolution errors from the API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 00:08:30 -06:00
Jim Liu 宝玉 7cc2e640b0 chore: release v1.42.2 2026-03-01 00:09:10 -06:00
Jim Liu 宝玉 e2fa3065f7 chore: add sync-md-to-wechat utility script
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 00:01:09 -06:00
Jim Liu 宝玉 8787cbe85b feat(baoyu-post-to-wechat): internalize markdown conversion with modular renderer and color support
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 00:01:05 -06:00
Jim Liu 宝玉 226d501e9e feat(baoyu-markdown-to-html): inline rendering pipeline and enhance modern theme
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 00:00:57 -06:00
Jim Liu 宝玉 6796ec67bd chore: release v1.42.1 2026-02-28 12:14:15 -06:00
Jim Liu 宝玉 856e8e5205 feat(baoyu-markdown-to-html): modularize render.ts and bundle code themes locally
Split monolithic render.ts into cli, constants, extend-config, html-builder, renderer, themes, and types modules. Bundle highlight.js code theme CSS files locally instead of CDN loading.
2026-02-28 12:13:39 -06:00
Jim Liu 宝玉 a7d6b5c69b chore: release v1.42.0 2026-02-28 11:51:46 -06:00
Jim Liu 宝玉 6e18725906 feat(baoyu-post-to-wechat): add default color preference and modern theme support 2026-02-28 11:48:59 -06:00
Jim Liu 宝玉 7133d54f04 feat(baoyu-markdown-to-html): consolidate heritage and warm into modern theme with per-theme color defaults 2026-02-28 11:48:55 -06:00
Jim Liu 宝玉 208bb20e98 chore: release v1.41.0 2026-02-28 11:25:22 -06:00
Jim Liu 宝玉 6edc1b7822 feat(baoyu-markdown-to-html): rename themes and add color presets with theme defaults
Rename red→heritage and orange→warm themes. Add 13 named color presets,
serif-cjk font family, and per-theme style defaults with CLI override support.
2026-02-28 11:18:16 -06:00
Jim Liu 宝玉 e754eae5ae chore: release v1.40.1 2026-02-28 01:38:40 -06:00
Jim Liu 宝玉 cc016f3b16 feat(baoyu-image-gen): clarify model resolution priority and add model display requirement
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 01:38:10 -06:00
Jim Liu 宝玉 2188160418 chore: release v1.40.0 2026-02-28 01:15:53 -06:00
Jim Liu 宝玉 77204d2e4a feat(baoyu-markdown-to-html): add CLI customization options and EXTEND.md config support
Add --color, --font-family, --font-size, --code-theme, --mac-code-block,
--line-number, --cite, --count, --legend options. Support EXTEND.md for
default preferences. Add code highlight theme fetching and mermaid fix.
2026-02-28 01:15:16 -06:00
Jim Liu 宝玉 35eeb62f10 Merge pull request #57 from zhao-newname/main
feat(baoyu-image-gen): support OpenAI chat completions endpoint for i…
2026-02-28 01:13:19 -06:00
jeff.zhao 7dfa2b6e6c feat(baoyu-image-gen): support OpenAI chat completions endpoint for image generation
Add OPENAI_IMAGE_USE_CHAT env var (true|false) to route image generation
through /chat/completions instead of /images/generations. Extracts base64
image data URLs from the chat response, enabling compatibility with
OpenAI-compatible providers that return images via chat API.
2026-02-28 14:46:05 +08:00
Jim Liu 宝玉 a2eecec26f chore: release v1.39.0 2026-02-28 00:33:53 -06:00
Jim Liu 宝玉 903e7ad79d feat(baoyu-markdown-to-html): add red and orange themes 2026-02-28 00:33:28 -06:00
Jim Liu 宝玉 13707fa2cf chore: release v1.38.0 2026-02-28 00:07:01 -06:00
Jim Liu 宝玉 b56e503b16 feat(baoyu-danger-x-to-markdown): add referenced tweet rendering, media reuse, and hi-res downloads
- Render embedded tweets in articles as blockquotes with author info and text summary
- Reuse existing markdown when --download-media targets already-converted URLs
- Upgrade Twitter image downloads to 4096x4096 resolution
- Improve entity resolution with logical key lookup
- Support trailing media for all block types (headings, lists, blockquotes)
- Change output path to {username}/{tweet-id}/{content-slug}.md for stable paths
2026-02-28 00:06:43 -06:00
Jim Liu 宝玉 7d03685ade chore: release v1.37.1 2026-02-27 18:35:25 -06:00
Jim Liu 宝玉 b305c386bc chore: remove unused Gemini model comparison images
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 18:32:05 -06:00
Jim Liu 宝玉 240dd7d314 Merge pull request #56 from xkcoding/fix/update-gemini-web-model-headers
fix(baoyu-danger-gemini-web): sync model headers with upstream and update model list
2026-02-27 18:30:48 -06:00
Jim Liu 宝玉 b02ceacfd9 chore: release v1.37.0 2026-02-27 18:29:04 -06:00
Jim Liu 宝玉 fdf9007e2c feat(baoyu-danger-x-to-markdown): improve article rendering with inline links and content-based output paths 2026-02-27 18:28:34 -06:00
Jim Liu 宝玉 08cee885d3 chore: release v1.36.0 2026-02-27 10:13:27 -06:00
Jim Liu 宝玉 3bd5fdeb1b feat(baoyu-image-gen): add gemini-3.1-flash-image-preview model and improve first-time setup
- Add gemini-3.1-flash-image-preview to supported Google multimodal models
- Improve preferences loading with blocking first-time setup flow
- Add first-time-setup.md reference for guided configuration
- Update model references in SKILL.md and preferences schema
2026-02-27 10:12:52 -06:00
Yangkai.Shen e737c4a611 docs: add model comparison images for PR #56
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 19:59:42 +08:00
Yangkai.Shen 2eec4f3639 fix(baoyu-danger-gemini-web): sync model headers with upstream and update model list
- Update x-goog-ext-525001261-jspb header format to match upstream
  HanaokaYuzu/Gemini-API (commit 42900f7, 2026-02-03), appending
  `,null,null,1` to fix image generation failures
- Replace deprecated gemini-2.5-pro and gemini-2.5-flash with
  gemini-3.0-flash and gemini-3.0-flash-thinking
- Add gemini-3.1-pro-preview (empty header, server auto-routed)
- Update SKILL.md and CLI help text to reflect new model options

Closes #50

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 19:40:50 +08:00
Jim Liu 宝玉 e5912018f3 Merge pull request #55 from liye71023326/fix/google-proxy-support
fix(baoyu-image-gen): use curl fallback when HTTP proxy is detected
2026-02-26 14:33:55 -06:00
李野 b1f568d03d fix(baoyu-image-gen): use curl fallback for Google API when HTTP proxy is detected
Bun's fetch implementation has a known issue where long-lived connections
through HTTP proxies (e.g., Clash, V2Ray) get their sockets closed
unexpectedly, causing Google image generation requests to fail with
"The socket connection was closed unexpectedly".

This change adds automatic proxy detection and falls back to curl as the
HTTP client when a proxy is configured (via https_proxy, http_proxy,
HTTPS_PROXY, HTTP_PROXY, or ALL_PROXY environment variables). When no
proxy is detected, the original fetch-based implementation is used.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 17:16:15 +08:00
Jim Liu 宝玉 e736707628 chore: release v1.35.0 2026-02-24 22:15:29 -06:00
Jim Liu 宝玉 d863f11f61 feat(baoyu-infographic): add dense-modules layout and 3 new styles for high-density infographics
Add dense-modules layout for data-rich guides and 3 new styles:
morandi-journal, pop-laboratory, retro-pop-grid. Add keyword shortcuts
for 高密度信息大图 auto-selection.

Prompt credit: AJ (https://waytoagi.feishu.cn/wiki/YG0zwalijihRREkgmPzcWRInnUg)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 22:13:47 -06:00
Jim Liu 宝玉 2ce873c65c Merge pull request #47 from justnode/feature/add-baoyu-image-gen-provider
Feature/add baoyu image gen provider
2026-02-24 21:53:19 -06:00
Jim Liu 宝玉 964cf1e045 chore: release v1.34.2 2026-02-24 18:51:36 -06:00
Jim Liu 宝玉 eded9a98bb docs(baoyu-post-to-wechat): enforce explicit theme passing for markdown conversion 2026-02-24 18:51:10 -06:00
Jim Liu 宝玉 a64fdbd23f docs(baoyu-markdown-to-html): clarify theme resolution with EXTEND.md fallbacks 2026-02-24 18:51:04 -06:00
justnodejs 36b9c5e197 docs(baoyu-image-gen): add replicate model configuration documentation 2026-02-24 23:25:38 +08:00
justnodejs 851497abbd refactor(baoyu-image-gen): update replicate default model to nano-banana-pro 2026-02-24 20:26:47 +08:00
justnodejs 65a561e654 feat(baoyu-image-gen): add replicate provider 2026-02-24 19:12:36 +08:00
Jim Liu 宝玉 7b2c02a007 chore: release v1.34.1 2026-02-20 03:00:26 -06:00
Jim Liu 宝玉 98f49eae96 Merge pull request #45 from LyInfi/fix/wechat-browser-upload-progress-reeval
fix(wechat-browser): fix upload progress check crashing on second iteration
2026-02-19 18:41:51 -06:00
LyInfi 1bdf44df9e fix(wechat-browser): fix upload progress check crashing on second iteration
Runtime.evaluate reuses the same JS execution context across calls in a
session. The previous expression used `const thumbs = ...` which throws
"Identifier 'thumbs' has already been declared" on the second loop
iteration, causing result.value to be undefined and JSON.parse to throw
"JSON Parse error: Unexpected identifier 'undefined'".

Fix by inlining the querySelector into a single expression with no
variable declaration, eliminating the re-declaration error.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 12:12:26 +08:00
Jim Liu 宝玉 d59ecf22c1 chore: release v1.34.0 2026-02-17 13:32:50 -06:00
Jim Liu 宝玉 22fa1a62fc refactor(baoyu-article-illustrator): enforce prompt file creation before image generation 2026-02-17 13:30:34 -06:00
Jim Liu 宝玉 3d48eed33a Merge pull request #44 from jeffrey94/feat/ref-image-chain
feat(baoyu-xhs-images): add reference image chain for visual consistency
2026-02-17 13:19:34 -06:00
Jim Liu 宝玉 8f1c4a65dd chore: release v1.33.1 2026-02-14 14:52:51 -06:00
Jim Liu 宝玉 9b97720f16 docs(baoyu-post-to-x): remove --submit flag and clarify manual publish workflow 2026-02-14 14:48:18 -06:00
Jim Liu 宝玉 145e1d2d04 refactor(baoyu-post-to-x): replace hand-rolled markdown parser with marked ecosystem
Switch md-to-html from manual line-by-line parsing to marked + front-matter +
highlight.js + remark-cjk-friendly for robust markdown conversion with syntax
highlighting, proper CJK handling, and standard frontmatter parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 14:37:14 -06:00
jeffrey94 49403b6fab feat(baoyu-xhs-images): add reference image chain for visual consistency
When generating multi-image series, use image 1 as --ref for all
subsequent images. This anchors character design, color rendering,
and illustration style across the entire series — critical for styles
with recurring characters or mascots.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 00:49:09 +08:00
Jim Liu 宝玉 6d56f1dae8 chore: release v1.33.0 2026-02-13 20:28:11 -06:00
Jim Liu 宝玉 dd08a2aa89 feat(baoyu-post-to-wechat): add pre-flight environment check script 2026-02-13 20:27:31 -06:00
Jim Liu 宝玉 d9b47debb3 feat(baoyu-post-to-x): add pre-flight check and image upload verification 2026-02-13 20:27:27 -06:00
Jim Liu 宝玉 6cc8627ca8 chore: release v1.32.0 2026-02-12 01:58:23 -06:00
Jim Liu 宝玉 54103dfd7d refactor(baoyu-post-to-wechat): prioritize coverImage over featureImage in frontmatter lookup 2026-02-12 01:56:32 -06:00
Jim Liu 宝玉 6012b9cc88 refactor(baoyu-format-markdown): rename featureImage to coverImage as primary frontmatter key 2026-02-12 01:56:28 -06:00
Jim Liu 宝玉 260b71cdd7 feat(baoyu-danger-x-to-markdown): add --download-media flag and media localization 2026-02-12 01:56:24 -06:00
Jim Liu 宝玉 9ff468a6a7 chore: release v1.31.2 2026-02-10 16:05:48 -06:00
Jim Liu 宝玉 c83d114a73 fix(baoyu-post-to-x): fix Windows clipboard and path handling
- Embed file path directly in PowerShell script with single-quote escaping
  instead of using param()/−Path which fails with -Command parameter.
- Use fileURLToPath() for getScriptDir() to handle Windows drive-letter paths.
2026-02-10 16:05:09 -06:00
Jim Liu 宝玉 e964100dd9 fix(baoyu-post-to-wechat): fix PowerShell clipboard copy on Windows
Embed file path directly in PowerShell script with single-quote escaping
instead of using param()/−Path which fails with -Command parameter.
2026-02-10 16:05:03 -06:00
Jim Liu 宝玉 6c0ae7a86a chore: release v1.31.1 2026-02-10 15:50:31 -06:00
Jim Liu 宝玉 52ade637af fix(baoyu-post-to-x): fix Chrome launch on macOS and cover image path resolution
Use open -na on macOS to avoid process blocking; resolve relative cover image
paths to absolute paths.
2026-02-10 15:46:14 -06:00
Jim Liu 宝玉 569beebdd6 feat(baoyu-post-to-wechat): adapt to new WeChat UI and fix digest/cover handling
Rename 图文 to 贴图 throughout; add ProseMirror editor support with old editor
fallback; add fallback file input selector; add upload progress monitoring;
improve save button detection with toast verification; truncate digest > 120
chars at punctuation boundary; fix cover image relative path resolution.
2026-02-10 15:46:09 -06:00
Jim Liu 宝玉 6cbf0f4e52 chore: release v1.31.0 2026-02-07 15:51:09 -06:00
Jim Liu 宝玉 5615e6150f feat(baoyu-post-to-wechat): add comment control, cover fallback chain, and first-time setup 2026-02-07 15:50:23 -06:00
Jim Liu 宝玉 7465f37dcf chore: release v1.30.3 2026-02-06 16:22:02 -06:00
Jim Liu 宝玉 38a1b90db3 refactor(baoyu-article-illustrator): optimize SKILL.md from 197 to 150 lines 2026-02-06 16:21:58 -06:00
Jim Liu 宝玉 b53a5db60a chore: release v1.30.2 2026-02-06 16:13:51 -06:00
Jim Liu 宝玉 372e03cef3 refactor(baoyu-cover-image): optimize SKILL.md from 532 to 233 lines 2026-02-06 16:13:47 -06:00
Jim Liu 宝玉 ea8ad82786 chore: release v1.30.1 2026-02-06 16:07:34 -06:00
Jim Liu 宝玉 6fffe252c0 fix(baoyu-cover-image): enhance reference image analysis and prompt generation
- Add deep analysis template for extracting specific visual elements
- Require MUST INCORPORATE section with concrete, reproducible details
- Add integration approach for spatial layout instructions
- Clarify ref-capable backend requirements (Google/OpenAI)
2026-02-06 16:07:09 -06:00
Jim Liu 宝玉 86a84739e8 feat(baoyu-image-gen): add OpenAI GPT Image edits support for reference images
- Support --ref with OpenAI GPT Image models (gpt-image-1.5)
- Auto-select Google or OpenAI when --ref provided
- Change ref-related warnings to explicit errors with fix hints
- Add reference image validation before generation
- Improve retry logic to skip non-retryable errors
2026-02-06 16:06:58 -06:00
Jim Liu 宝玉 7f80100b3e chore: release v1.30.0 2026-02-06 14:56:00 -06:00
Jim Liu 宝玉 61960961ba feat(baoyu-cover-image): add font dimension with 4 styles
Add 6th dimension for typography control:
- clean: modern geometric sans-serif
- handwritten: warm hand-lettered style
- serif: classic elegant typography
- display: bold decorative headlines

Includes auto-selection rules, compatibility matrix, and warm-flat preset.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 14:55:11 -06:00
Jim Liu 宝玉 cdd9ee042d chore: release v1.29.0 2026-02-06 13:36:08 -06:00
Jim Liu 宝玉 c742bfa1af fix(baoyu-url-to-markdown): improve html extraction and markdown conversion 2026-02-06 13:35:28 -06:00
Jim Liu 宝玉 0faea4ecaa Merge pull request #38 from kingdomad/feature/baoyu-image-gen-extend-config
add EXTEND.md configuration support
2026-02-05 23:10:11 -06:00
Jim Liu 宝玉 7c9ec259a1 Merge pull request #36 from NantesCheval/fix/wechat-title-and-list-duplication
fix(baoyu-post-to-wechat): fix title and list number duplication in WeChat articles
2026-02-05 23:09:17 -06:00
史提芬达 24a17709a8 add EXTEND.md configuration support 2026-02-05 19:46:22 +08:00
AlexCheval a3849af0cf fix(baoyu-post-to-wechat): fix title and list number duplication in WeChat articles
- Remove title from body content when extracting it for WeChat title field
  to prevent duplicate title display (one in header, one in content)
- Remove manual ordered list prefix since HTML <ol> already provides numbering,
  preventing "1.1.", "2.2.", "3.3." duplication

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 10:04:34 +08:00
Jim Liu 宝玉 cc95e6fe05 chore: release v1.28.4 2026-02-03 11:49:41 -06:00
Jim Liu 宝玉 e7d9ed7917 fix(baoyu-post-to-wechat): remove extra empty lines after image paste and fix summary timing
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 11:49:13 -06:00
Jim Liu 宝玉 876c6332f6 feat(baoyu-markdown-to-html): add HTML meta tags and quote stripping for frontmatter
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 11:49:07 -06:00
Jim Liu 宝玉 502d2448b2 chore: release v1.28.3 2026-02-03 11:06:59 -06:00
Jim Liu 宝玉 b9e48d9483 fix(baoyu-post-to-wechat): fix placeholder matching to avoid WECHATIMGPH_1 matching WECHATIMGPH_10 2026-02-03 11:06:41 -06:00
Jim Liu 宝玉 b0554f8d0c chore: release v1.28.2 2026-02-03 11:01:25 -06:00
Jim Liu 宝玉 12f3d95639 fix(baoyu-post-to-x): improve Chrome reuse and placeholder matching in x-article 2026-02-03 11:01:09 -06:00
Jim Liu 宝玉 809fb29ca9 chore: release v1.28.1 2026-02-02 09:11:49 -06:00
Jim Liu 宝玉 be20bdf0be refactor(baoyu-article-illustrator): simplify SKILL.md and add Core Styles tier
- Extract detailed workflow to references/workflow.md
- Add Core Styles for quick selection (vector, minimal-flat, sci-fi, hand-drawn, editorial, scene)
- Add vector-illustration as recommended default style
- Add Illustration Purpose (information/visualization/imagination)
- Add default composition, character rendering, text styling to prompts
2026-02-02 09:11:43 -06:00
Jim Liu 宝玉 7c36b2a6a5 chore: release v1.28.0 2026-02-01 19:49:49 -06:00
Jim Liu 宝玉 7979cb9dde chore: ignore baoyu-skill-evolution directory 2026-02-01 19:49:22 -06:00
Jim Liu 宝玉 e563bbae7b feat(baoyu-post-to-wechat): add newspic article type support
- Add ArticleOptions interface with news/newspic types
- Track all image media IDs for newspic format
- Restructure publishToDraft for dual article type handling
2026-02-01 19:49:14 -06:00
Jim Liu 宝玉 c63d1eec69 refactor(baoyu-xhs-images): enforce blocking first-time setup
- Mark preferences loading as BLOCKING operation
- Update workflow flow diagram and checklist
2026-02-01 19:49:05 -06:00
Jim Liu 宝玉 78d6d23359 refactor(baoyu-comic): enforce blocking first-time setup
- Mark preferences loading as BLOCKING operation
- Update workflow flow diagram
2026-02-01 19:48:56 -06:00
Jim Liu 宝玉 ec6781a231 feat(baoyu-article-illustrator): add reference image support
- Add reference image handling with direct/style/palette usage types
- Enhance first-time setup as blocking operation
- Update prompt construction with reference frontmatter format
2026-02-01 19:48:46 -06:00
Jim Liu 宝玉 edf9030675 feat(baoyu-cover-image): add reference image support and visual elements library
- Add --ref parameter for reference images (style/palette/direct usage)
- Add visual-elements.md with icon vocabulary by topic
- Enhance first-time setup as blocking operation
- Remove character limits from titles, use original source titles
- Update prompt template with reference handling
2026-02-01 19:48:37 -06:00
Jim Liu 宝玉 10e1c426a0 chore: release v1.27.0 2026-02-01 02:15:08 -06:00
Jim Liu 宝玉 dbab83ff82 feat(baoyu-markdown-to-html): add remark-cjk-friendly for CJK emphasis
Preprocess markdown with remark-cjk-friendly to properly handle
CJK emphasis markers before HTML conversion.
2026-02-01 02:14:12 -06:00
Jim Liu 宝玉 478e66a93a docs(baoyu-infographic): clarify AskUserQuestion usage
Emphasize combining multiple questions into single call.
2026-02-01 02:14:06 -06:00
Jim Liu 宝玉 3a5866eb4b refactor(baoyu-format-markdown): use remark-cjk-friendly for CJK emphasis
Replace custom CJK emphasis handling with remark-cjk-friendly library,
significantly simplifying the codebase.
2026-02-01 02:13:58 -06:00
Jim Liu 宝玉 c0162bb3af chore: release v1.26.1 2026-01-29 22:02:42 -06:00
Jim Liu 宝玉 e9342a16f8 fix(baoyu-xhs-images): remove notebook style, add backup rules for prompts and images 2026-01-29 22:01:47 -06:00
Jim Liu 宝玉 5cb86d8e51 feat(baoyu-slide-deck): add backup rules for source, prompt, and slide image files 2026-01-29 22:01:42 -06:00
Jim Liu 宝玉 66ed08f010 feat(baoyu-infographic): add backup rules for source, analysis, prompt, and image files 2026-01-29 22:01:22 -06:00
Jim Liu 宝玉 b179166d28 feat(baoyu-cover-image): add backup rules for source and prompt files 2026-01-29 22:01:17 -06:00
Jim Liu 宝玉 4805526649 feat(baoyu-comic): add backup rules for source, character sheet, prompts, and page images 2026-01-29 22:00:59 -06:00
Jim Liu 宝玉 89159bf86e feat(baoyu-article-illustrator): add backup rules for source, prompt, and image files 2026-01-29 22:00:37 -06:00
Jim Liu 宝玉 b45b5d2c6e chore: release v1.26.0 2026-01-29 20:59:45 -06:00
Jim Liu 宝玉 62866426c5 feat(baoyu-xhs-images): add notebook and study-notes styles with mindmap and quadrant layouts
- notebook: hand-drawn infographic style with watercolor rendering and Morandi palette
- study-notes: realistic handwritten photo style with blue pen, red annotations, yellow highlighter
- mindmap: center radial layout (4-8 branches)
- quadrant: four-quadrant / circular section layout
2026-01-29 20:59:24 -06:00
Jim Liu 宝玉 28a7db6129 chore: release v1.25.4 2026-01-29 17:17:58 -06:00
Jim Liu 宝玉 7c5e81fd15 chore: release v1.25.3 2026-01-28 22:31:47 -06:00
Jim Liu 宝玉 058d25d02c feat(baoyu-format-markdown): add content type detection and CJK punctuation handling 2026-01-28 22:31:25 -06:00
Jim Liu 宝玉 fe09ec546f chore: release v1.25.2 2026-01-28 22:10:32 -06:00
Jim Liu 宝玉 cf611a4514 docs(baoyu-post-to-wechat): add WeChat API credentials configuration guide 2026-01-28 22:10:27 -06:00
Jim Liu 宝玉 7cf30e969f chore: release v1.25.1 2026-01-28 21:49:51 -06:00
Jim Liu 宝玉 18b71157db feat(baoyu-markdown-to-html): add CJK content pre-check step
Suggest formatting with baoyu-format-markdown before conversion
to fix bold markers with punctuation issues.
2026-01-28 21:49:47 -06:00
Jim Liu 宝玉 9904c490e4 chore: release v1.25.0 2026-01-28 21:38:06 -06:00
Jim Liu 宝玉 32ed09128c docs: add baoyu-format-markdown and baoyu-markdown-to-html to plugin registry 2026-01-28 21:37:49 -06:00
Jim Liu 宝玉 81e96ee43c feat(baoyu-post-to-wechat): add API publishing and external theme support
- Add wechat-api.ts for direct API-based article publishing
- Support plain text, HTML, and markdown inputs with auto-conversion
- Add external theme discovery from MD_THEME_DIR environment variable
- Improve CSS inlining with juice library integration
- Support WeChat API credentials via .baoyu-skills/.env
2026-01-28 21:37:40 -06:00
Jim Liu 宝玉 5f994589b3 feat(baoyu-markdown-to-html): add markdown to HTML converter skill
Converts markdown to styled HTML with inline CSS, code highlighting,
math, PlantUML, footnotes, alerts, and WeChat-compatible themes.
2026-01-28 21:37:30 -06:00
Jim Liu 宝玉 40363194be feat(baoyu-format-markdown): add markdown formatter skill
Formats plain text or markdown with frontmatter, titles, summaries,
headings, bold, lists, code blocks, and CJK typography fixes.
2026-01-28 21:37:23 -06:00
Jim Liu 宝玉 21a5167b61 chore: release v1.24.4 2026-01-28 14:54:44 -06:00
Jim Liu 宝玉 4d3957ca06 fix(baoyu-post-to-x): ensure Apply button click closes modal before continuing 2026-01-28 14:54:39 -06:00
Jim Liu 宝玉 013c72fce7 chore: release v1.24.3 2026-01-28 13:56:48 -06:00
Jim Liu 宝玉 40fadcb1f6 docs: emphasize updating prompt files before regenerating 2026-01-28 13:56:35 -06:00
Jim Liu 宝玉 499a1ee478 chore: release v1.24.2 2026-01-28 13:14:44 -06:00
Jim Liu 宝玉 fa89eaf2f7 refactor(baoyu-image-gen): default to sequential generation 2026-01-28 13:14:39 -06:00
Jim Liu 宝玉 7964e20bc3 chore: release v1.24.1 2026-01-28 12:29:43 -06:00
Jim Liu 宝玉 45c109ce88 Merge pull request #27 from JianJang2017/my-baoyu-skills
Adapting baoyu-image-gen to the Tongyi Text-to-Image Model
2026-01-28 12:23:34 -06:00
jianzhang50 74c08def06 docs: add Aliyun text-to-image model config to README.md 2026-01-28 22:04:53 +08:00
jianzhang50 907c8ab852 Adapting baoyu-image-gen to the Tongyi Text-to-Image Model 2026-01-28 09:04:57 +08:00
Jim Liu 宝玉 64945f3341 chore: release v1.24.0 2026-01-27 10:10:44 -06:00
Jim Liu 宝玉 050d37c344 docs: backfill third-party contributor attributions in changelogs 2026-01-27 10:06:22 -06:00
Jim Liu 宝玉 f9f168fc1f docs(release-skills): add third-party contributor attribution to release workflow 2026-01-27 10:06:15 -06:00
Jim Liu 宝玉 b1af3a3e45 fix(baoyu-post-to-wechat): improve title extraction, summary auto-fill, and content verification 2026-01-27 10:06:11 -06:00
Jim Liu 宝玉 0727296592 Merge pull request #23 from AliceLJY/feat/reuse-existing-chrome
feat(baoyu-post-to-wechat): reuse existing Chrome instead of requiring all windows closed
2026-01-27 08:03:46 -06:00
Jim Liu 宝玉 0e571b72fb chore: release v1.23.1 2026-01-27 00:59:34 -06:00
Jim Liu 宝玉 366630f8c3 fix(baoyu-compress-image): rename original as backup instead of deleting 2026-01-27 00:59:30 -06:00
安闲静雅 6bfafe0ec5 feat(baoyu-post-to-wechat): reuse existing Chrome instead of requiring all windows closed
Previously, the script always launched a new Chrome instance, which failed
when Chrome was already running due to profile lock conflicts on macOS.
Users had to close all Chrome windows before publishing.

This change adds auto-detection of existing Chrome debug ports and reuses
an already-logged-in WeChat tab when available, falling back to launching
a new instance only when no existing Chrome is found.

Changes:
- cdp.ts: add tryConnectExisting() and findExistingChromeDebugPort()
- wechat-article.ts: try existing Chrome before launching new one,
  reuse logged-in WeChat tab (identified by token= in URL),
  add waitForElement() for reliable menu detection,
  add --cdp-port option for manual override,
  fix process not exiting after completion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 13:16:33 +08:00
Jim Liu 宝玉 5e597f0a8b chore: release v1.23.0 2026-01-26 11:36:06 -06:00
Jim Liu 宝玉 40f4d2f730 refactor(baoyu-cover-image): replace 20 styles with 5D palette×rendering system
Migrate from 4-dimension (Type×Style×Text×Mood) to 5-dimension
(Type×Palette×Rendering×Text×Mood) system. 9 palettes × 6 renderings
= 54 combinations replacing 20 fixed styles. Add style presets for
backward compatibility, v2→v3 schema migration, and new reference
structure (palettes/, renderings/, workflow/).
2026-01-26 11:32:30 -06:00
Jim Liu 宝玉 b12ae7374d chore: release v1.22.0 2026-01-25 21:05:37 -06:00
Jim Liu 宝玉 1afa6f5a58 refactor(baoyu-post-to-x): simplify image placeholders from bracket format to XIMGPH 2026-01-25 21:05:08 -06:00
Jim Liu 宝玉 977598d5ae feat(baoyu-post-to-wechat): add theme selection, HTML preview, and simplify image placeholders 2026-01-25 21:05:04 -06:00
Jim Liu 宝玉 eb738aa61a feat(baoyu-cover-image): add default_output_dir preference with output directory selection 2026-01-25 21:04:59 -06:00
Jim Liu 宝玉 e9a030f917 feat(baoyu-article-illustrator): add imgs-subdir output option and improve style selection 2026-01-25 21:04:56 -06:00
Jim Liu 宝玉 e6a9b217e6 chore: release v1.21.4 2026-01-25 16:10:08 -06:00
Jim Liu 宝玉 ec1743592c fix(baoyu-post-to-wechat): fix regressions from Windows compatibility PR
- Fix broken md-to-wechat-fixed.ts/wechat-article-fixed.ts filename
  references that cause runtime crash (files were never renamed)
- Restore frontmatter quote stripping for title/summary values
- Restore --title CLI parameter functionality (was silently ignored)
- Fix summary extraction to properly skip headings, quotes, lists
- Fix argument parsing to reject single-dash args as file paths
- Remove debug console.error logs left from development
2026-01-25 16:09:24 -06:00
Jim Liu 宝玉 fe647a11bb Merge pull request #20 from JadeLiang003/fix/windows-copy-paste-support
Fix: Windows compatibility for baoyu-post-to-wechat
2026-01-25 16:04:01 -06:00
JadeLiang 5a11b49b00 Fix: Windows compatibility for baoyu-post-to-wechat
## Problem
The baoyu-post-to-wechat skill failed on Windows due to:
1. `new URL(import.meta.url).pathname` returns incorrect path format on Windows
2. Copy/paste operations used `xdotool` (Linux tool) which doesn't exist on Windows

## Solution
1. **md-to-wechat.ts**: Use `fileURLToPath()` to correctly resolve file paths on Windows
2. **wechat-article.ts**: Use CDP `Input.dispatchKeyEvent` for copy/paste operations
   - Replaces system-dependent tools (xdotool/osascript)
   - Works consistently across Windows/macOS/Linux
   - More reliable as it operates within Chrome session

## Changes
- Import `fileURLToPath` from 'node:url'
- Replace `new URL(import.meta.url).pathname` with `fileURLToPath(import.meta.url)`
- Replace system tool calls with CDP keyboard events for Ctrl+C and Ctrl+V
- Add platform-specific modifier handling (Cmd for macOS, Ctrl for Windows/Linux)

## Testing
- Tested successfully on Windows 11
- Markdown conversion works correctly
- HTML copy/paste to WeChat editor works correctly

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-25 15:24:04 +08:00
Jim Liu 宝玉 04692515bb fix: remove opacity of watermark 2026-01-24 17:23:04 -06:00
663 changed files with 230719 additions and 6845 deletions
+21 -29
View File
@@ -6,44 +6,36 @@
},
"metadata": {
"description": "Skills shared by Baoyu for improving daily work efficiency",
"version": "1.21.3"
"version": "2.5.1"
},
"plugins": [
{
"name": "content-skills",
"description": "Content generation and publishing skills",
"name": "baoyu-skills",
"description": "Content generation, AI backends, and utility tools for daily work efficiency",
"source": "./",
"strict": false,
"skills": [
"./skills/baoyu-xhs-images",
"./skills/baoyu-post-to-x",
"./skills/baoyu-post-to-wechat",
"./skills/baoyu-article-illustrator",
"./skills/baoyu-cover-image",
"./skills/baoyu-slide-deck",
"./skills/baoyu-comic",
"./skills/baoyu-infographic"
]
},
{
"name": "ai-generation-skills",
"description": "AI-powered generation backends",
"source": "./",
"strict": false,
"skills": [
"./skills/baoyu-danger-gemini-web",
"./skills/baoyu-image-gen"
]
},
{
"name": "utility-skills",
"description": "Utility tools for content processing",
"source": "./",
"strict": false,
"skills": [
"./skills/baoyu-danger-x-to-markdown",
"./skills/baoyu-compress-image",
"./skills/baoyu-url-to-markdown"
"./skills/baoyu-cover-image",
"./skills/baoyu-danger-gemini-web",
"./skills/baoyu-danger-x-to-markdown",
"./skills/baoyu-diagram",
"./skills/baoyu-electron-extract",
"./skills/baoyu-format-markdown",
"./skills/baoyu-image-gen",
"./skills/baoyu-infographic",
"./skills/baoyu-markdown-to-html",
"./skills/baoyu-post-to-weibo",
"./skills/baoyu-post-to-wechat",
"./skills/baoyu-post-to-x",
"./skills/baoyu-slide-deck",
"./skills/baoyu-translate",
"./skills/baoyu-url-to-markdown",
"./skills/baoyu-wechat-summary",
"./skills/baoyu-xhs-images",
"./skills/baoyu-youtube-transcript"
]
}
]
+141 -22
View File
@@ -1,12 +1,22 @@
---
name: release-skills
description: Universal release workflow. Auto-detects version files and changelogs. Supports Node.js, Python, Rust, Claude Plugin, and generic projects. Use when user says "release", "发布", "new version", "bump version", "push", "推送".
description: Universal release workflow. Auto-detects version files and changelogs. Supports Node.js, Python, Rust, Claude Plugin, GitHub Releases, annotated tags, historical release backfill, and generic projects. Use when user says "release", "发布", "new version", "bump version", "push", "推送", "release notes", "GitHub Release", or "回填 Release".
---
# Release Skills
Universal release workflow supporting any project type with multi-language changelog.
## User Input Tools
When this skill prompts the user, follow this tool-selection rule (priority order):
1. **Prefer built-in user-input tools** exposed by the current agent runtime — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.
2. **Fallback**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
3. **Batching**: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.
Concrete `AskUserQuestion` references below are examples — substitute the local equivalent in other runtimes.
## Quick Start
Just run `/release-skills` - auto-detects your project configuration.
@@ -29,12 +39,14 @@ Just run `/release-skills` - auto-detects your project configuration.
| `--major` | Force major version bump |
| `--minor` | Force minor version bump |
| `--patch` | Force patch version bump |
| `--backfill-releases` | Create missing GitHub Releases for existing tags from changelog sections |
## Workflow
### Step 1: Detect Project Configuration
1. Check for `.releaserc.yml` (optional config override)
- If present, inspect whether it defines release hooks
2. Auto-detect version file by scanning (priority order):
- `package.json` (Node.js)
- `pyproject.toml` (Python)
@@ -46,20 +58,52 @@ Just run `/release-skills` - auto-detects your project configuration.
- `HISTORY*.md`
- `CHANGES*.md`
4. Identify language of each changelog by filename suffix
5. Display detected configuration
5. Detect GitHub release support:
- Check whether `origin` points to GitHub
- Check whether `gh` is installed and authenticated
- Check existing releases with `gh release list --limit 5` when available
6. Display detected configuration
**Project Hook Contract**:
If `.releaserc.yml` defines `release.hooks`, keep the release workflow generic and delegate project-specific packaging/publishing to those hooks.
Supported hooks:
| Hook | Purpose | Expected Responsibility |
|------|---------|-------------------------|
| `prepare_artifact` | Make one target releasable | Validate the target is self-contained, sync/embed local dependencies, optionally stage extra files |
| `publish_artifact` | Publish one releasable target | Upload the prepared target (or a staged directory if the project uses one), attach version/changelog/tags |
Supported placeholders:
| Placeholder | Meaning |
|-------------|---------|
| `{project_root}` | Absolute path to repository root |
| `{target}` | Absolute path to the module/skill being released |
| `{artifact_dir}` | Absolute path to a temporary staging directory for this target, when the project uses one |
| `{version}` | Version selected by the release workflow |
| `{dry_run}` | `true` or `false` |
| `{release_notes_file}` | Absolute path to a UTF-8 file containing release notes/changelog text |
Execution rules:
- Keep the skill generic: do not hardcode registry/package-manager/project layout details into this SKILL.
- If `prepare_artifact` exists, run it once per target before publish-related checks that need the final releasable target state.
- Write release notes to a temp file and pass that file path to `publish_artifact`; do not inline multiline changelog text into shell commands.
- If hooks are absent, fall back to the default project-agnostic release workflow.
**Language Detection Rules**:
| Filename Pattern | Language |
|------------------|----------|
| `CHANGELOG.md` (no suffix) | en (default) |
| `CHANGELOG.zh.md` / `CHANGELOG_CN.md` / `CHANGELOG.zh-CN.md` | zh |
| `CHANGELOG.ja.md` / `CHANGELOG_JP.md` | ja |
| `CHANGELOG.ko.md` / `CHANGELOG_KR.md` | ko |
| `CHANGELOG.de.md` / `CHANGELOG_DE.md` | de |
| `CHANGELOG.fr.md` / `CHANGELOG_FR.md` | fr |
| `CHANGELOG.es.md` / `CHANGELOG_ES.md` | es |
| `CHANGELOG.{lang}.md` | Corresponding language code |
Changelog files follow the pattern `CHANGELOG_{LANG}.md` or `CHANGELOG.{lang}.md`, where `{lang}` / `{LANG}` is a language or region code.
| Pattern | Example | Language |
|---------|---------|----------|
| No suffix | `CHANGELOG.md` | en (default) |
| `_{LANG}` (uppercase) | `CHANGELOG_CN.md`, `CHANGELOG_JP.md` | Corresponding language |
| `.{lang}` (lowercase) | `CHANGELOG.zh.md`, `CHANGELOG.ja.md` | Corresponding language |
| `.{lang-region}` | `CHANGELOG.zh-CN.md` | Corresponding region variant |
Common language codes: `zh` (Chinese), `ja` (Japanese), `ko` (Korean), `de` (German), `fr` (French), `es` (Spanish).
**Output Example**:
```
@@ -114,11 +158,17 @@ Display version change: `1.2.3 → 1.3.0`
For each detected changelog file:
1. **Identify language** from filename suffix
2. **Generate content in that language**:
2. **Detect third-party contributors**:
- Check merge commits: `git log ${LAST_TAG}..HEAD --merges --pretty=format:"%H %s"`
- For each merged PR, identify the PR author via `gh pr view <number> --json author --jq '.author.login'`
- Compare against repo owner (`gh repo view --json owner --jq '.owner.login'`)
- If PR author ≠ repo owner → third-party contributor
3. **Generate content in that language**:
- Section titles in target language
- Change descriptions written naturally in target language (not translated)
- Date format: YYYY-MM-DD (universal)
3. **Insert at file head** (preserve existing content)
- **Third-party contributions**: Append contributor attribution `(by @username)` to the changelog entry
4. **Insert at file head** (preserve existing content)
**Section Title Translations** (built-in):
@@ -138,6 +188,7 @@ For each detected changelog file:
### Features
- Description of new feature
- Description of third-party contribution (by @username)
### Fixes
- Description of fix
@@ -148,6 +199,12 @@ For each detected changelog file:
Only include sections that have changes. Omit empty sections.
**Third-Party Attribution Rules**:
- Only add `(by @username)` for contributors who are NOT the repo owner
- Use GitHub username with `@` prefix
- Place at the end of the changelog entry line
- Apply to all languages consistently (always use `(by @username)` format, not translated)
**Multi-language Example**:
English (CHANGELOG.md):
@@ -155,7 +212,7 @@ English (CHANGELOG.md):
## 1.3.0 - 2026-01-22
### Features
- Add user authentication module
- Add user authentication module (by @contributor1)
- Support OAuth2 login
### Fixes
@@ -167,7 +224,7 @@ Chinese (CHANGELOG.zh.md):
## 1.3.0 - 2026-01-22
### 新功能
- 新增用户认证模块
- 新增用户认证模块 (by @contributor1)
- 支持 OAuth2 登录
### 修复
@@ -179,7 +236,7 @@ Japanese (CHANGELOG.ja.md):
## 1.3.0 - 2026-01-22
### 新機能
- ユーザー認証モジュールを追加
- ユーザー認証モジュールを追加 (by @contributor1)
- OAuth2 ログインをサポート
### 修正
@@ -258,6 +315,14 @@ git commit -m "docs(project): update architecture documentation"
- Read version file (JSON/TOML/text)
- Update version number
- Write back (preserve formatting)
3. **Create release notes file**:
- Prefer the new version section from `CHANGELOG.md`
- If no English/default changelog exists, use the first detected changelog
- Extract only the exact `## {VERSION} - {YYYY-MM-DD}` section through the next `##`
- Match both plain version and tag-prefixed headings when needed, e.g. `1.2.3` and `v1.2.3`
- Keep breaking changes near the top; if needed, add a short highlight before other sections
- Write notes to a UTF-8 temp file and reuse it for annotated tag messages, GitHub Releases, and `publish_artifact`
- In normal mode, stop rather than creating an empty tag or GitHub Release when notes cannot be found
**Version Paths by File Type**:
@@ -273,7 +338,7 @@ git commit -m "docs(project): update architecture documentation"
Before creating the release commit, ask user to confirm:
**Use AskUserQuestion with two questions**:
**Use AskUserQuestion with three questions**:
1. **Version bump** (single select):
- Show recommended version based on Step 3 analysis
@@ -283,6 +348,11 @@ Before creating the release commit, ask user to confirm:
2. **Push to remote** (single select):
- Options: "Yes, push after commit", "No, keep local only"
3. **Publish GitHub Release** (single select):
- Offer this only when GitHub release support is available
- Default to "Yes, publish after tag push" when the user also chose push
- If the user keeps the release local, do not create or edit a GitHub Release
**Example Output Before Confirmation**:
```
Commits created:
@@ -297,10 +367,11 @@ Changelog preview (en):
### Fixes
- Improve panel layout for long dialogues in comic
Ready to create release commit and tag.
Release notes source: CHANGELOG.md#1.3.0
Ready to create release commit, annotated tag, and GitHub Release.
```
### Step 9: Create Release Commit and Tag
### Step 9: Create Release Commit and Annotated Tag
After user confirmation:
@@ -315,10 +386,11 @@ After user confirmation:
git commit -m "chore: release v{VERSION}"
```
3. **Create tag**:
3. **Create annotated tag**:
```bash
git tag v{VERSION}
git tag -a v{VERSION} -F <release-notes-file>
```
If `.releaserc.yml` sets `tag.sign: true`, use `git tag -s` with the same notes file.
4. **Push if user confirmed** (Step 8):
```bash
@@ -328,6 +400,28 @@ After user confirmation:
**Note**: Do NOT add Co-Authored-By line. This is a release commit, not a code contribution.
### Step 10: Publish Release Artifacts and GitHub Release
Project artifact publishing and GitHub Releases are separate outputs:
1. **Project artifacts**:
- If `release.hooks.publish_artifact` exists, run it once per prepared target
- Pass the same `{release_notes_file}` used for the tag and GitHub Release
- In dry-run mode, pass `{dry_run}=true` and report what would be published
2. **GitHub Release**:
- Run only if the user confirmed remote publishing and GitHub support is available
- Ensure the tag exists on the remote before creating the release
- Create or update using the extracted notes:
```bash
if gh release view v{VERSION} >/dev/null 2>&1; then
gh release edit v{VERSION} --title "v{VERSION}" --notes-file <release-notes-file>
else
gh release create v{VERSION} --title "v{VERSION}" --notes-file <release-notes-file> --verify-tag
fi
```
- Never inline multiline release notes into shell commands
**Post-Release Output**:
```
Release v1.3.0 created.
@@ -339,9 +433,32 @@ Commits:
4. chore: release v1.3.0
Tag: v1.3.0
Tag type: annotated
GitHub Release: published # or "skipped/local only"
Status: Pushed to origin # or "Local only - run git push when ready"
```
## Backfill Existing GitHub Releases
Use this mode when the user asks to backfill historical releases or passes `--backfill-releases`.
1. Do not bump versions, edit changelogs, or create release commits.
2. List existing tags in version order and detect missing releases:
```bash
git tag --sort=v:refname
gh release view <tag>
```
3. For each tag without a GitHub Release:
- Normalize the changelog lookup by stripping the configured tag prefix, e.g. `v1.2.3` -> `1.2.3`
- Extract the matching section from `CHANGELOG.md`; fall back to the first matching changelog file
- Skip or ask before publishing if no matching changelog section exists
- Create the release with:
```bash
gh release create <tag> --title "<tag>" --notes-file <release-notes-file> --verify-tag
```
4. Detect lightweight tags with `git cat-file -t <tag>` (`commit` means lightweight, `tag` means annotated).
5. Do not rewrite public lightweight tags by default. Converting an existing remote tag to an annotated tag requires explicit user confirmation because it rewrites a published reference.
## Configuration (.releaserc.yml)
Optional config file in project root to override defaults:
@@ -446,6 +563,7 @@ No changes made. Run without --dry-run to execute.
/release-skills --minor # Force minor bump
/release-skills --patch # Force patch bump
/release-skills --major # Force major bump (with confirmation)
/release-skills --backfill-releases # Create missing GitHub Releases for existing tags
```
## When to Use
@@ -454,6 +572,7 @@ Trigger this skill when user requests:
- "release", "发布", "create release", "new version", "新版本"
- "bump version", "update version", "更新版本"
- "prepare release"
- "release notes", "GitHub Release", "回填 Release"
- "push to remote" (with uncommitted changes)
**Important**: If user says "just push" or "直接 push" with uncommitted changes, STILL follow all steps above first.
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
set -eu
REPO_ROOT=$(git rev-parse --show-toplevel)
cd "$REPO_ROOT"
node scripts/verify-shared-package-deps.mjs --repo-root "$REPO_ROOT"
@@ -0,0 +1,38 @@
name: codex-imagegen tests
on:
push:
paths:
- 'packages/baoyu-codex-imagegen/**'
- '.github/workflows/codex-imagegen-tests.yml'
pull_request:
paths:
- 'packages/baoyu-codex-imagegen/**'
- '.github/workflows/codex-imagegen-tests.yml'
workflow_dispatch:
jobs:
unit-tests:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Show Bun version
run: bun --version
- name: Run unit tests
working-directory: packages/baoyu-codex-imagegen
run: bun test
- name: Bundle smoke test (catches import/syntax errors)
run: bun build --target=node packages/baoyu-codex-imagegen/src/main.ts --outfile /tmp/main-build.js
- name: Help output smoke test
run: bun packages/baoyu-codex-imagegen/src/main.ts --help
+35
View File
@@ -0,0 +1,35 @@
name: Test
on:
push:
pull_request:
workflow_dispatch:
jobs:
node-tests:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- name: Verify skill release commits
run: npm run verify:skill-release-commits
- name: Install dependencies
run: npm ci
- name: Install baoyu-post-to-wechat script dependencies
run: |
cd skills/baoyu-post-to-wechat/scripts
npm install --no-audit --no-fund --ignore-scripts
- name: Run tests
run: npm test
+24
View File
@@ -83,6 +83,10 @@ out
.nuxt
dist
# Shared package builds are published to npm.
!packages/*/dist/
!packages/*/dist/**
# Gatsby files
.cache/
# Comment in the public line in if your project uses Gatsby and not Next.js
@@ -152,3 +156,23 @@ slide-deck/
infographic/
illustrations/
comic/
translate/
posts/
diagram/
### IntelliJ IDEA ###
.idea
*.iws
*.iml
*.ipr
.claude/skills/baoyu-skill-evolution
# ClawHub local state (current and legacy directory names from the official CLI)
.clawhub/
.clawdhub/
.release-artifacts/
.worktrees/
youtube-transcript/
.omx/
.codex-tmp/
outputs/
wechat/
+6
View File
@@ -0,0 +1,6 @@
release:
target_globs:
- skills/*
hooks:
prepare_artifact: node scripts/verify-shared-package-deps.mjs --repo-root "{project_root}" --target "{target}"
publish_artifact: node scripts/publish-skill.mjs --skill-dir "{target}" --version "{version}" --changelog-file "{release_notes_file}" --dry-run "{dry_run}"
+1311 -3
View File
File diff suppressed because it is too large Load Diff
+1313 -5
View File
File diff suppressed because it is too large Load Diff
+74 -392
View File
@@ -1,430 +1,112 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Claude Code marketplace plugin providing AI-powered content generation skills. Skills use Gemini Web API (reverse-engineered) for text/image generation and Chrome CDP for browser automation.
Claude Code marketplace plugin providing AI-powered content generation skills. Version: **2.0.0**.
## Architecture
Skills are organized into three plugin categories in `marketplace.json`:
Skills are exposed through the single `baoyu-skills` plugin in `.claude-plugin/marketplace.json` (which defines plugin metadata, version, and skill paths). The repo docs still group them into three logical areas:
```
skills/
├── [content-skills] # Content generation and publishing
│ ├── baoyu-xhs-images/ # Xiaohongshu infographic series (1-10 images)
│ ├── baoyu-cover-image/ # Article cover images (2.35:1 aspect)
│ ├── baoyu-slide-deck/ # Presentation slides with outlines
│ ├── baoyu-article-illustrator/ # Smart illustration placement
│ ├── baoyu-comic/ # Knowledge comics (Logicomix/Ohmsha style)
│ ├── baoyu-post-to-x/ # X/Twitter posting automation
│ └── baoyu-post-to-wechat/ # WeChat Official Account posting
├── [ai-generation-skills] # AI-powered generation backends
│ └── baoyu-danger-gemini-web/ # Gemini API wrapper (text + image gen)
└── [utility-skills] # Utility tools for content processing
├── baoyu-danger-x-to-markdown/ # X/Twitter content to markdown
└── baoyu-compress-image/ # Image compression
```
| Group | Description |
|-------|-------------|
| Content Skills | Generate or publish content (images, slides, comics, posts) |
| AI Generation Skills | AI generation backends |
| Utility Skills | Content processing (conversion, compression, translation) |
**Plugin Categories**:
| Category | Description |
|----------|-------------|
| `content-skills` | Skills that generate or publish content (images, slides, comics, posts) |
| `ai-generation-skills` | Backend skills providing AI generation capabilities |
| `utility-skills` | Helper tools for content processing (conversion, compression) |
Each skill contains `SKILL.md` (YAML front matter + docs), optional `scripts/`, `references/`, `prompts/`.
Each skill contains:
- `SKILL.md` - YAML front matter (name, description) + documentation
- `scripts/` - TypeScript implementations
- `prompts/system.md` - AI generation guidelines (optional)
Top-level `scripts/` contains repo maintenance utilities (sync, hooks, publish).
## Running Skills
All scripts run via Bun (no build step):
TypeScript via Bun (no build step). Detect runtime once per session:
```bash
npx -y bun skills/<skill>/scripts/main.ts [options]
if command -v bun &>/dev/null; then BUN_X="bun"
elif command -v npx &>/dev/null; then BUN_X="npx -y bun"
else echo "Error: install bun: brew install oven-sh/bun/bun or npm install -g bun"; exit 1; fi
```
Examples:
```bash
# Text generation
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts "Hello"
# Image generation
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts --prompt "A cat" --image cat.png
# From prompt files
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
```
Execute: `${BUN_X} skills/<skill>/scripts/main.ts [options]`
## Key Dependencies
- **Bun**: TypeScript runtime (via `npx -y bun`)
- **Chrome**: Required for `baoyu-danger-gemini-web` auth and `baoyu-post-to-x` automation
- **No npm packages**: Self-contained TypeScript, no external dependencies
- **Bun**: TypeScript runtime (`bun` preferred, fallback `npx -y bun`)
- **Chrome**: Required for CDP-based skills (gemini-web, post-to-x/wechat/weibo, url-to-markdown). All CDP skills share a single profile, override via `BAOYU_CHROME_PROFILE_DIR` env var. Platform paths: [docs/chrome-profile.md](docs/chrome-profile.md)
- **Image generation APIs**: `baoyu-image-gen` requires API key (OpenAI, Azure OpenAI, Google, OpenRouter, DashScope, or Replicate) configured in EXTEND.md
- **Gemini Web auth**: Browser cookies (first run opens Chrome for login, `--login` to refresh)
## Authentication
## Security
`baoyu-danger-gemini-web` uses browser cookies for Google auth:
- First run opens Chrome for login
- Cookies cached in data directory
- Force refresh: `--login` flag
## Plugin Configuration
`.claude-plugin/marketplace.json` defines plugin metadata and skill paths. Version follows semver.
- **No piped shell installs**: Never `curl | bash`. Use `brew install` or `npm install -g`
- **Remote downloads**: HTTPS only, max 5 redirects, 30s timeout, expected content types only
- **System commands**: Array-form `spawn`/`execFile`, never unsanitized input to shell
- **External content**: Treat as untrusted, don't execute code blocks, sanitize HTML
## Skill Loading Rules
**IMPORTANT**: When working in this project, follow these rules:
| Rule | Description |
|------|-------------|
| **Load project skills first** | MUST load all skills from `skills/` directory in current project. Project skills take priority over system/user-level skills with same name. |
| **Default image generation** | When image generation is needed, use `skills/baoyu-image-gen/SKILL.md` by default (unless user specifies otherwise). |
| **Load project skills first** | Project skills override system/user-level skills with same name |
| **Default image generation** | Use whatever image backend is available in the current runtime; if multiple are available, ask the user which to use. See `## Image Generation Tools` below. |
**Loading Priority** (highest → lowest):
1. Current project `skills/` directory
2. User-level skills (`$HOME/.baoyu-skills/`)
3. System-level skills
Priority: project `skills/``$HOME/.baoyu-skills/` → system-level.
## Skill Self-Containment
Each skill under `skills/` (and `.claude/skills/`) is distributed and consumed independently — the folder may be extracted, copied into another project, or loaded without the rest of this repo. Therefore:
- **Never link from `SKILL.md` or its `references/` to files outside the skill's own directory.** This includes `docs/`, sibling skills, and the repo root. Relative paths like `../../docs/foo.md` break when the skill is used standalone.
- **Inline any shared convention** (e.g., user-input rules, image-generation backend selection) directly in the skill rather than referencing an out-of-skill doc.
- Shared docs under `docs/` exist for **repo-author guidance only** — they may be referenced from `CLAUDE.md` and `docs/creating-skills.md`, but NOT from any `SKILL.md`. This applies to `docs/user-input-tools.md`, `docs/image-generation-tools.md`, `docs/image-generation.md`, and any other `docs/` file.
## User Input Tools
Skills that prompt users for choices MUST declare the tool-selection convention **inline** in exactly one place per `SKILL.md` — a `## User Input Tools` section near the top. Do NOT link out to [docs/user-input-tools.md](docs/user-input-tools.md); that doc is the author-side canonical source — copy its body into each SKILL.md. Concrete `AskUserQuestion` mentions elsewhere in a skill are treated as examples — other runtimes substitute their local equivalent under the rule.
## Image Generation Tools
Skills that render images MUST declare the backend-selection convention **inline** in exactly one place per `SKILL.md` — a `## Image Generation Tools` section near the top (after `## User Input Tools`). Do NOT link out to [docs/image-generation-tools.md](docs/image-generation-tools.md); that doc is the author-side canonical source — copy its body into each SKILL.md. Concrete tool names (`imagegen`, `GenerateImage`, `image_generate`, `baoyu-image-gen`) elsewhere in a skill are treated as examples — other runtimes substitute their local equivalent under the rule. The rule is stateless: use whatever backend is available; if multiple, ask the user once; if none, ask how to proceed. Every rendered image's full prompt must be written to a standalone `prompts/NN-*.md` file before any backend is invoked. Backend skills (`baoyu-image-gen`, `baoyu-danger-gemini-web`) are exempt — they render directly rather than selecting a backend.
### `codex-imagegen` Backend
A backend for non-Codex runtimes (e.g., Claude Code) that generates images by spawning `codex exec --json --sandbox danger-full-access` and delegating to Codex CLI's built-in `image_gen` tool. Uses the user's Codex subscription — no `OPENAI_API_KEY` required. Lives under [packages/baoyu-codex-imagegen](packages/baoyu-codex-imagegen) so it follows the same workspace layout as other shared packages.
Invoke via (TS entrypoint with `#!/usr/bin/env bun` shebang):
```bash
bun packages/baoyu-codex-imagegen/src/main.ts \
--image <output.png> \
--prompt-file prompts/01-cover.md \
--aspect 16:9 \
--cache-dir ~/.cache/baoyu-codex-imagegen
```
Without bun installed: `npx -y bun packages/baoyu-codex-imagegen/src/main.ts …`.
Stdout emits a single JSON line: `{"status":"ok","path":...,"bytes":N,...}`. On failure, `{"status":"error","error_kind":...}`. Skills route here by setting `preferred_image_backend: codex-imagegen` in EXTEND.md, or by running `baoyu-image-gen --provider codex-cli` (which spawns the same wrapper internally). Full reference: [docs/codex-imagegen-backend.md](docs/codex-imagegen-backend.md).
## Release Process
**IMPORTANT**: When user requests release/发布/push, ALWAYS use `/release-skills` workflow.
**Never skip**:
1. `CHANGELOG.md` + `CHANGELOG.zh.md` - Both must be updated
Use `/release-skills` workflow. Never skip:
1. `CHANGELOG.md` + `CHANGELOG.zh.md`
2. `marketplace.json` version bump
3. `README.md` + `README.zh.md` if applicable
4. All files committed together before tag
## Adding New Skills
**IMPORTANT**: All skills MUST use `baoyu-` prefix to avoid conflicts when users import this plugin.
**REQUIRED READING**: Before creating a new skill, read the official [Skill authoring best practices](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices).
### Key Requirements from Official Best Practices
| Requirement | Details |
|-------------|---------|
| **Concise is key** | Claude is smart—only add context it doesn't have. Challenge each token. |
| **name field** | Max 64 chars, lowercase letters/numbers/hyphens only, no "anthropic"/"claude" |
| **description field** | Max 1024 chars, non-empty, MUST be third person, include what + when to use |
| **SKILL.md body** | Keep under 500 lines; use separate files for additional content |
| **Naming convention** | Gerund form preferred (e.g., `processing-pdfs`), but `baoyu-` prefix required here |
| **References** | Keep one level deep from SKILL.md; avoid nested references |
| **No time-sensitive info** | Avoid dates/versions that become outdated |
### Steps
1. Create `skills/baoyu-<name>/SKILL.md` with YAML front matter
- Directory name: `baoyu-<name>`
- SKILL.md `name` field: `baoyu-<name>`
2. Add TypeScript in `skills/baoyu-<name>/scripts/`
3. Add prompt templates in `skills/baoyu-<name>/prompts/` if needed
4. **Choose the appropriate category** and register in `marketplace.json`:
- `content-skills`: For content generation/publishing (images, slides, posts)
- `ai-generation-skills`: For AI backend capabilities
- `utility-skills`: For helper tools (conversion, compression)
- If none fit, create a new category with descriptive name
5. **Add Script Directory section** to SKILL.md (see template below)
### Choosing a Category
| If your skill... | Use category |
|------------------|--------------|
| Generates visual content (images, slides, comics) | `content-skills` |
| Publishes to platforms (X, WeChat, etc.) | `content-skills` |
| Provides AI generation backend | `ai-generation-skills` |
| Converts or processes content | `utility-skills` |
| Compresses or optimizes files | `utility-skills` |
**Creating a new category**: If the skill doesn't fit existing categories, add a new plugin object to `marketplace.json` with:
- `name`: Descriptive kebab-case name (e.g., `analytics-skills`)
- `description`: Brief description of the category
- `skills`: Array with the skill path
### Writing Effective Descriptions
**MUST write in third person** (not "I can help you" or "You can use this"):
```yaml
# Good
description: Generates Xiaohongshu infographic series from content. Use when user asks for "小红书图片", "XHS images", or "RedNote infographics".
# Bad
description: I can help you create Xiaohongshu images
description: You can use this to generate XHS content
```
Include both **what** the skill does and **when** to use it (triggers/keywords).
### Script Directory Template
Every SKILL.md with scripts MUST include this section after Usage:
```markdown
## Script Directory
**Important**: All scripts are located in the `scripts/` subdirectory of this skill.
**Agent Execution Instructions**:
1. Determine this SKILL.md file's directory path as `SKILL_DIR`
2. Script path = `${SKILL_DIR}/scripts/<script-name>.ts`
3. Replace all `${SKILL_DIR}` in this document with the actual path
**Script Reference**:
| Script | Purpose |
|--------|---------|
| `scripts/main.ts` | Main entry point |
| `scripts/other.ts` | Other functionality |
```
When referencing scripts in workflow sections, use `${SKILL_DIR}/scripts/<name>.ts` so agents can resolve the correct path.
### Progressive Disclosure
For skills with extensive content, use separate reference files:
```
skills/baoyu-example/
├── SKILL.md # Main instructions (<500 lines)
├── references/
│ ├── styles.md # Style definitions (loaded as needed)
│ └── examples.md # Usage examples (loaded as needed)
└── scripts/
└── main.ts # Executable script
```
In SKILL.md, link to reference files (one level deep only):
```markdown
**Available styles**: See [references/styles.md](references/styles.md)
**Examples**: See [references/examples.md](references/examples.md)
```
## Code Style
- TypeScript throughout, no comments
- Async/await patterns
- Short variable names
- Type-safe interfaces
TypeScript, no comments, async/await, short variable names, type-safe interfaces.
## Image Generation Guidelines
## Adding New Skills
Skills that require image generation MUST delegate to available image generation skills rather than implementing their own.
All skills MUST use `baoyu-` prefix. Details: [docs/creating-skills.md](docs/creating-skills.md)
### Image Generation Skill Selection
## Reference Docs
**Default**: Use `skills/baoyu-image-gen/SKILL.md` (unless user specifies otherwise).
1. Read `skills/baoyu-image-gen/SKILL.md` for parameters and capabilities
2. If user explicitly requests a different skill, check `skills/` directory for alternatives
3. Only ask user to choose when they haven't specified and multiple viable options exist
### Generation Flow Template
Use this template when implementing image generation in skills:
```markdown
### Step N: Generate Images
**Skill Selection**:
1. Check available image generation skills (e.g., `baoyu-danger-gemini-web`)
2. Read selected skill's SKILL.md for parameter reference
3. If multiple skills available, ask user to choose
**Generation Flow**:
1. Call selected image generation skill with:
- Prompt file path (or inline prompt)
- Output image path
- Any skill-specific parameters (refer to skill's SKILL.md)
2. Generate images sequentially (one at a time)
3. After each image, output progress: "Generated X/N"
4. On failure, auto-retry once before reporting error
```
### Output Path Convention
Each session creates an independent directory. Even the same source file generates a new directory per session.
**Output Directory**:
```
<skill-suffix>/<topic-slug>/
```
- `<skill-suffix>`: Skill name suffix (e.g., `xhs-images`, `cover-image`, `slide-deck`, `comic`)
- `<topic-slug>`: Generated from content topic (2-4 words, kebab-case)
- Example: `xhs-images/ai-future-trends/`
**Slug Generation**:
1. Extract main topic from content (2-4 words, kebab-case)
2. Example: "Introduction to Machine Learning" → `intro-machine-learning`
**Conflict Resolution**:
If `<skill-suffix>/<topic-slug>/` already exists:
- Append timestamp: `<topic-slug>-YYYYMMDD-HHMMSS`
- Example: `ai-future` exists → `ai-future-20260118-143052`
**Source Files**:
- Copy all sources to `<skill-suffix>/<topic-slug>/` with naming: `source-{slug}.{ext}`
- Multiple sources supported: text, images, files from conversation
- Examples:
- `source-article.md` (main text content)
- `source-reference.png` (image from conversation)
- `source-data.csv` (additional file)
- Original source files remain unchanged
### Image Naming Convention
Image filenames MUST include meaningful slugs for readability:
**Format**: `NN-{type}-[slug].png`
- `NN`: Two-digit sequence number (01, 02, ...)
- `{type}`: Image type (cover, content, page, slide, illustration, etc.)
- `[slug]`: Descriptive kebab-case slug derived from content
**Examples**:
```
01-cover-ai-future.png
02-content-key-benefits.png
03-page-enigma-machine.png
04-slide-architecture-overview.png
```
**Slug Rules**:
- Derived from image purpose or content (kebab-case)
- Must be unique within the output directory
- 2-5 words, concise but descriptive
- When content changes significantly, update slug accordingly
### Best Practices
- Always read the image generation skill's SKILL.md before calling
- Pass parameters exactly as documented in the skill
- Handle failures gracefully with retry logic
- Provide clear progress feedback to user
## Style Maintenance (baoyu-comic)
When adding, updating, or deleting styles for `baoyu-comic`, follow this workflow:
### Adding a New Style
1. **Create style definition**: `skills/baoyu-comic/references/styles/<style-name>.md`
2. **Update SKILL.md**:
- Add style to `--style` options table
- Add auto-selection entry if applicable
3. **Generate showcase image**:
```bash
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts \
--prompt "A single comic book page in <style-name> style showing [appropriate scene]. Features: [style characteristics from style definition]. 3:4 portrait aspect ratio comic page." \
--image screenshots/comic-styles/<style-name>.png
```
4. **Compress to WebP**:
```bash
npx -y bun skills/baoyu-compress-image/scripts/main.ts screenshots/comic-styles/<style-name>.png
```
5. **Update both READMEs** (`README.md` and `README.zh.md`):
- Add style to `--style` options
- Add row to style description table
- Add image to style preview grid
### Updating an Existing Style
1. **Update style definition**: `skills/baoyu-comic/references/styles/<style-name>.md`
2. **Regenerate showcase image** (if visual characteristics changed):
- Follow steps 3-4 from "Adding a New Style"
3. **Update READMEs** if description changed
### Deleting a Style
1. **Delete style definition**: `skills/baoyu-comic/references/styles/<style-name>.md`
2. **Delete showcase image**: `screenshots/comic-styles/<style-name>.webp`
3. **Update SKILL.md**:
- Remove from `--style` options
- Remove auto-selection entry
4. **Update both READMEs**:
- Remove from `--style` options
- Remove from style description table
- Remove from style preview grid
### Style Preview Grid Format
READMEs use 3-column tables for style previews:
```markdown
| | | |
|:---:|:---:|:---:|
| ![style1](./screenshots/comic-styles/style1.webp) | ![style2](./screenshots/comic-styles/style2.webp) | ![style3](./screenshots/comic-styles/style3.webp) |
| style1 | style2 | style3 |
```
## Extension Support
Every SKILL.md MUST include two parts for extension support:
### 1. Load Preferences Section (in Step 1 or as "Preferences" section)
For skills with workflows, add as Step 1.1. For utility skills, add as "Preferences (EXTEND.md)" section before Usage:
```markdown
**1.1 Load Preferences (EXTEND.md)**
Use Bash to check EXTEND.md existence (priority order):
\`\`\`bash
# Check project-level first
test -f .baoyu-skills/<skill-name>/EXTEND.md && echo "project"
# Then user-level (cross-platform: $HOME works on macOS/Linux/WSL)
test -f "$HOME/.baoyu-skills/<skill-name>/EXTEND.md" && echo "user"
\`\`\`
┌────────────────────────────────────────────┬───────────────────┐
│ Path │ Location │
├────────────────────────────────────────────┼───────────────────┤
│ .baoyu-skills/<skill-name>/EXTEND.md │ Project directory │
├────────────────────────────────────────────┼───────────────────┤
│ $HOME/.baoyu-skills/<skill-name>/EXTEND.md │ User home │
└────────────────────────────────────────────┴───────────────────┘
┌───────────┬───────────────────────────────────────────────────────────────────────────┐
│ Result │ Action │
├───────────┼───────────────────────────────────────────────────────────────────────────┤
│ Found │ Read, parse, display summary │
├───────────┼───────────────────────────────────────────────────────────────────────────┤
│ Not found │ Ask user with AskUserQuestion (see references/config/first-time-setup.md) │
└───────────┴───────────────────────────────────────────────────────────────────────────┘
**EXTEND.md Supports**: [List supported configuration options for this skill]
Schema: `references/config/preferences-schema.md`
```
### 2. Extension Support Section (at the end)
Simplified section that references the preferences section:
```markdown
## Extension Support
Custom configurations via EXTEND.md. See **Step 1.1** for paths and supported options.
```
Or for utility skills without workflow steps:
```markdown
## Extension Support
Custom configurations via EXTEND.md. See **Preferences** section for paths and supported options.
```
**Notes**:
- Replace `<skill-name>` with actual skill name (e.g., `baoyu-cover-image`)
- Use `$HOME` instead of `~` for cross-platform compatibility (macOS/Linux/WSL)
- Use `test -f` Bash command for explicit file existence check
- ASCII tables for clear visual formatting
| Topic | File |
|-------|------|
| Image generation output guidelines | [docs/image-generation.md](docs/image-generation.md) |
| Image generation backend selection | [docs/image-generation-tools.md](docs/image-generation-tools.md) |
| User input tool convention | [docs/user-input-tools.md](docs/user-input-tools.md) |
| Chrome profile platform paths | [docs/chrome-profile.md](docs/chrome-profile.md) |
| Comic style maintenance | [docs/comic-style-maintenance.md](docs/comic-style-maintenance.md) |
| ClawHub/OpenClaw publishing | [docs/publishing.md](docs/publishing.md) |
+666 -88
View File
@@ -2,7 +2,7 @@
English | [中文](./README.zh.md)
Skills shared by Baoyu for improving daily work efficiency with Claude Code.
Skills shared by Baoyu for improving daily work efficiency with AI Agents (Claude Code, Codex, etc.).
## Prerequisites
@@ -11,18 +11,41 @@ Skills shared by Baoyu for improving daily work efficiency with Claude Code.
## Installation
> **Tip**: This repository contains 20+ skills. Install only the ones you actually need — bulk-installing every skill adds unnecessary context overhead for your AI agent on every run.
### Quick Install (Recommended)
```bash
npx skills add jimliu/baoyu-skills
```
### Register as Plugin Marketplace
### Publish to ClawHub / OpenClaw
Run the following command in Claude Code:
This repository now supports publishing each `skills/baoyu-*` directory as an individual ClawHub skill.
```bash
/plugin marketplace add jimliu/baoyu-skills
# Preview what would be published
./scripts/sync-clawhub.sh --dry-run
# Publish all changed skills from ./skills
./scripts/sync-clawhub.sh --all
```
ClawHub installs skills individually, not as one marketplace bundle. After publishing, users can install specific skills such as:
```bash
clawhub install baoyu-image-gen
clawhub install baoyu-markdown-to-html
```
Publishing to ClawHub releases the published skill under `MIT-0`, per ClawHub's registry rules.
### Register as Plugin Marketplace
Run the following command in the Agent:
```bash
/plugin marketplace add JimLiu/baoyu-skills
```
### Install Skills
@@ -31,37 +54,35 @@ Run the following command in Claude Code:
1. Select **Browse and install plugins**
2. Select **baoyu-skills**
3. Select the plugin(s) you want to install
3. Select the **baoyu-skills** plugin
4. Select **Install now**
**Option 2: Direct Install**
```bash
# Install specific plugin
/plugin install content-skills@baoyu-skills
/plugin install ai-generation-skills@baoyu-skills
/plugin install utility-skills@baoyu-skills
# Install the marketplace's single plugin
/plugin install baoyu-skills@baoyu-skills
```
**Option 3: Ask the Agent**
Simply tell Claude Code:
Simply tell the Agent:
> Please install Skills from github.com/JimLiu/baoyu-skills
### Available Plugins
### Available Plugin
| Plugin | Description | Skills |
|--------|-------------|--------|
| **content-skills** | Content generation and publishing | [xhs-images](#baoyu-xhs-images), [infographic](#baoyu-infographic), [cover-image](#baoyu-cover-image), [slide-deck](#baoyu-slide-deck), [comic](#baoyu-comic), [article-illustrator](#baoyu-article-illustrator), [post-to-x](#baoyu-post-to-x), [post-to-wechat](#baoyu-post-to-wechat) |
| **ai-generation-skills** | AI-powered generation backends | [image-gen](#baoyu-image-gen), [danger-gemini-web](#baoyu-danger-gemini-web) |
| **utility-skills** | Utility tools for content processing | [url-to-markdown](#baoyu-url-to-markdown), [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image) |
The marketplace now exposes a single plugin so each skill is registered exactly once.
| Plugin | Description | Includes |
|--------|-------------|----------|
| **baoyu-skills** | Content generation, AI backends, and utility tools for daily work efficiency | All skills in this repository, organized below as Content Skills, AI Generation Skills, and Utility Skills |
## Update Skills
To update skills to the latest version:
1. Run `/plugin` in Claude Code
1. Run `/plugin` in the Agent
2. Switch to **Marketplaces** tab (use arrow keys or Tab)
3. Select **baoyu-skills**
4. Choose **Update marketplace**
@@ -74,13 +95,25 @@ You can also **Enable auto-update** to get the latest versions automatically.
Skills are organized into three categories:
### Featured Design Skill: baoyu-design
If you want a design-focused Agent Skill, check out [JimLiu/baoyu-design](https://github.com/JimLiu/baoyu-design). It is a separate project that runs Claude Design locally in Cursor, Claude Code, Codex, Claude Desktop, or any file-capable coding agent, producing polished UI mockups, interactive prototypes, wireframes, landing pages, dashboards, mobile apps, and slide decks as self-contained HTML.
<a href="https://github.com/JimLiu/baoyu-design">
<img src="https://raw.githubusercontent.com/JimLiu/baoyu-design/main/assets/screenshots/cursor-reader-mac-app.webp" alt="Cursor running baoyu-design" width="720">
</a>
```bash
npx skills add JimLiu/baoyu-design
```
### Content Skills
Content generation and publishing skills.
#### baoyu-xhs-images
Xiaohongshu (RedNote) infographic series generator. Breaks down content into 1-10 cartoon-style infographics with **Style × Layout** two-dimensional system.
Xiaohongshu image card series generator. Breaks down content into 1-10 cartoon-style image cards with **Style × Layout** system and optional palette override.
```bash
# Auto-select style and layout
@@ -93,13 +126,22 @@ Xiaohongshu (RedNote) infographic series generator. Breaks down content into 1-1
/baoyu-xhs-images posts/ai-future/article.md --layout dense
# Combine style and layout
/baoyu-xhs-images posts/ai-future/article.md --style tech --layout list
/baoyu-xhs-images posts/ai-future/article.md --style notion --layout list
# Override palette
/baoyu-xhs-images posts/ai-future/article.md --style notion --palette macaron
# Direct content input
/baoyu-xhs-images 今日星座运势
# Non-interactive (skip all confirmations, for scheduled tasks)
/baoyu-xhs-images posts/ai-future/article.md --yes
/baoyu-xhs-images posts/ai-future/article.md --yes --preset knowledge-card
```
**Styles** (visual aesthetics): `cute` (default), `fresh`, `warm`, `bold`, `minimal`, `retro`, `pop`, `notion`, `chalkboard`
**Styles** (visual aesthetics): `cute` (default), `fresh`, `warm`, `bold`, `minimal`, `retro`, `pop`, `notion`, `chalkboard`, `study-notes`, `screen-print`, `sketch-notes`
**Palettes** (optional color override): `macaron`, `warm`, `neon`
**Style Previews**:
@@ -133,7 +175,7 @@ Xiaohongshu (RedNote) infographic series generator. Breaks down content into 1-1
#### baoyu-infographic
Generate professional infographics with 20 layout types and 17 visual styles. Analyzes content, recommends layout×style combinations, and generates publication-ready infographics.
Generate professional infographics with 21 layout types and 21 visual styles. Analyzes content, recommends layout×style combinations, and generates publication-ready infographics.
```bash
# Auto-recommend combinations based on content
@@ -148,8 +190,9 @@ Generate professional infographics with 20 layout types and 17 visual styles. An
# Specify both
/baoyu-infographic path/to/content.md --layout funnel --style corporate-memphis
# With aspect ratio
# With aspect ratio (named preset or custom W:H)
/baoyu-infographic path/to/content.md --aspect portrait
/baoyu-infographic path/to/content.md --aspect 3:4
```
**Options**:
@@ -157,7 +200,7 @@ Generate professional infographics with 20 layout types and 17 visual styles. An
|--------|-------------|
| `--layout <name>` | Information layout (20 options) |
| `--style <name>` | Visual style (17 options, default: craft-handmade) |
| `--aspect <ratio>` | landscape (16:9), portrait (9:16), square (1:1) |
| `--aspect <ratio>` | Named: landscape (16:9), portrait (9:16), square (1:1). Custom: any W:H ratio (e.g., 3:4, 4:3, 2.35:1) |
| `--lang <code>` | Output language (en, zh, ja, etc.) |
**Layouts** (information structure):
@@ -243,9 +286,46 @@ Generate professional infographics with 20 layout types and 17 visual styles. An
| ![knolling](./screenshots/infographic-styles/knolling.webp) | ![lego-brick](./screenshots/infographic-styles/lego-brick.webp) | |
| knolling | lego-brick | |
#### baoyu-diagram
Generate publication-ready SVG diagrams from source material — flowcharts, sequence/protocol diagrams, structural/architecture diagrams, and illustrative intuition diagrams. Analyzes input material to recommend diagram type(s) and splitting strategy, confirms the plan once, then generates all diagrams. Claude writes real SVG code directly following a cohesive design system. Output is self-contained `.svg` files with embedded styles and auto dark-mode.
```bash
# Topic string — skill analyzes and proposes a plan
/baoyu-diagram "how JWT authentication works"
/baoyu-diagram "Kubernetes architecture" --type structural
/baoyu-diagram "OAuth 2.0 flow" --type sequence
# File path — skill reads, analyzes, and proposes a plan
/baoyu-diagram path/to/article.md
# Language and output path
/baoyu-diagram "微服务架构" --lang zh
/baoyu-diagram "build pipeline" --out docs/build-pipeline.svg
```
**Options**:
| Option | Description |
|--------|-------------|
| `--type <name>` | `flowchart`, `sequence`, `structural`, `illustrative`, `class`, `auto` (default). Skips type recommendation. |
| `--lang <code>` | Output language (en, zh, ja, ...) |
| `--out <path>` | Output file path. Generates exactly one diagram focused on the most important aspect. |
**Diagram types**:
| Type | Reader need | Verbs that trigger it |
|------|-------------|------------------------|
| `flowchart` | Walk me through the steps in order | walk through, steps, process, lifecycle, workflow, state machine |
| `sequence` | Who talks to whom, in what order | protocol, handshake, auth flow, OAuth, TCP, request/response |
| `structural` | Show me what's inside what, how it's organised | architecture, components, topology, layout, what's inside |
| `illustrative` | Give me the intuition — draw the mechanism | how does X work, explain X, intuition for, why does X do Y |
| `class` | What are the types and how are they related | class diagram, UML, inheritance, interface, schema |
Not an image-generation skill — no LLM image model is called. Claude writes the SVG by hand with hand-computed layout math, so every diagram honors the design system. Embedded `<style>` block with `@media (prefers-color-scheme: dark)` means the same file renders correctly in both light and dark mode anywhere it's embedded.
#### baoyu-cover-image
Generate cover images for articles with 4 dimensions: Type × Style × Text × Mood.
Generate cover images for articles with 5 dimensions: Type × Palette × Rendering × Text × Mood. Combines 11 color palettes with 7 rendering styles for 77 unique combinations.
```bash
# Auto-select all dimensions based on content
@@ -254,44 +334,27 @@ Generate cover images for articles with 4 dimensions: Type × Style × Text × M
# Quick mode: skip confirmation, use auto-selection
/baoyu-cover-image path/to/article.md --quick
# Specify dimensions
/baoyu-cover-image path/to/article.md --type conceptual --style blueprint
# Specify dimensions (5D system)
/baoyu-cover-image path/to/article.md --type conceptual --palette cool --rendering digital
/baoyu-cover-image path/to/article.md --text title-subtitle --mood bold
# Specify aspect ratio (default: 2.35:1)
/baoyu-cover-image path/to/article.md --aspect 16:9
# Style presets (backward-compatible shorthand)
/baoyu-cover-image path/to/article.md --style blueprint
# Specify aspect ratio (default: 16:9)
/baoyu-cover-image path/to/article.md --aspect 2.35:1
# Visual only (no title text)
/baoyu-cover-image path/to/article.md --no-title
```
**Four Dimensions**:
**Five Dimensions**:
- **Type**: `hero`, `conceptual`, `typography`, `metaphor`, `scene`, `minimal`
- **Style**: 20 built-in styles (see previews below)
- **Palette**: `warm`, `elegant`, `cool`, `dark`, `earth`, `vivid`, `pastel`, `mono`, `retro`, `duotone`, `macaron`
- **Rendering**: `flat-vector`, `hand-drawn`, `painterly`, `digital`, `pixel`, `chalk`, `screen-print`
- **Text**: `none`, `title-only` (default), `title-subtitle`, `text-rich`
- **Mood**: `subtle`, `balanced` (default), `bold`
Available styles: `elegant` (default), `blueprint`, `bold-editorial`, `chalkboard`, `dark-atmospheric`, `editorial-infographic`, `fantasy-animation`, `flat-doodle`, `intuition-machine`, `minimal`, `nature`, `notion`, `pixel-art`, `playful`, `retro`, `sketch-notes`, `vector-illustration`, `vintage`, `warm`, `watercolor`
**Style Previews**:
| | | |
|:---:|:---:|:---:|
| ![elegant](./screenshots/cover-image-styles/elegant.webp) | ![blueprint](./screenshots/cover-image-styles/blueprint.webp) | ![bold-editorial](./screenshots/cover-image-styles/bold-editorial.webp) |
| elegant | blueprint | bold-editorial |
| ![chalkboard](./screenshots/cover-image-styles/chalkboard.webp) | ![dark-atmospheric](./screenshots/cover-image-styles/dark-atmospheric.webp) | ![editorial-infographic](./screenshots/cover-image-styles/editorial-infographic.webp) |
| chalkboard | dark-atmospheric | editorial-infographic |
| ![fantasy-animation](./screenshots/cover-image-styles/fantasy-animation.webp) | ![intuition-machine](./screenshots/cover-image-styles/intuition-machine.webp) | ![minimal](./screenshots/cover-image-styles/minimal.webp) |
| fantasy-animation | intuition-machine | minimal |
| ![nature](./screenshots/cover-image-styles/nature.webp) | ![notion](./screenshots/cover-image-styles/notion.webp) | ![pixel-art](./screenshots/cover-image-styles/pixel-art.webp) |
| nature | notion | pixel-art |
| ![playful](./screenshots/cover-image-styles/playful.webp) | ![retro](./screenshots/cover-image-styles/retro.webp) | ![sketch-notes](./screenshots/cover-image-styles/sketch-notes.webp) |
| playful | retro | sketch-notes |
| ![vector-illustration](./screenshots/cover-image-styles/vector-illustration.webp) | ![vintage](./screenshots/cover-image-styles/vintage.webp) | ![warm](./screenshots/cover-image-styles/warm.webp) |
| vector-illustration | vintage | warm |
| ![watercolor](./screenshots/cover-image-styles/watercolor.webp) | ![flat-doodle](./screenshots/cover-image-styles/flat-doodle.webp) | |
| watercolor | flat-doodle | |
#### baoyu-slide-deck
Generate professional slide deck images from content. Creates comprehensive outlines with style instructions, then generates individual slide images.
@@ -466,20 +529,17 @@ Knowledge comic creator with flexible art style × tone combinations. Creates or
#### baoyu-article-illustrator
Smart article illustration skill with Type × Style two-dimension approach. Analyzes article structure, identifies positions requiring visual aids, and generates illustrations.
Smart article illustration skill with Type × Style × Palette three-dimension approach. Analyzes article structure, identifies positions requiring visual aids, and generates illustrations.
```bash
# Auto-select type and style based on content
/baoyu-article-illustrator path/to/article.md
# Specify type
/baoyu-article-illustrator path/to/article.md --type infographic
# Specify style
/baoyu-article-illustrator path/to/article.md --style blueprint
# Combine type and style
# Specify type and style
/baoyu-article-illustrator path/to/article.md --type flowchart --style notion
# With palette override
/baoyu-article-illustrator path/to/article.md --style vector-illustration --palette macaron
```
**Types** (information structure):
@@ -493,7 +553,7 @@ Smart article illustration skill with Type × Style two-dimension approach. Anal
| `framework` | Concept maps, relationship diagrams | Methodologies, architecture |
| `timeline` | Chronological progression | History, project progress |
**Styles** (visual aesthetics):
**Styles** (rendering approach):
| Style | Description | Best For |
|-------|-------------|----------|
@@ -506,6 +566,14 @@ Smart article illustration skill with Type × Style two-dimension approach. Anal
| `editorial` | Magazine-style infographic | Tech explainers, journalism |
| `scientific` | Academic precise diagrams | Biology, chemistry, technical |
**Palettes** (optional color override):
| Palette | Description | Best For |
|---------|-------------|----------|
| `macaron` | Soft pastel blocks (blue, mint, lavender, peach) on warm cream | Educational, knowledge, tutorials |
| `warm` | Warm earth tones on soft peach, no cool colors | Brand, product, lifestyle |
| `neon` | Vibrant neon on dark purple | Gaming, retro, pop culture |
**Style Previews**:
| | | |
@@ -521,9 +589,11 @@ Smart article illustration skill with Type × Style two-dimension approach. Anal
Post content and articles to X (Twitter). Supports regular posts with images and X Articles (long-form Markdown). Uses real Chrome with CDP to bypass anti-automation.
Plain text input is treated as a regular post. Markdown files are treated as X Articles. Scripts fill content into the browser, and the user reviews and publishes manually.
```bash
# Post with text
/baoyu-post-to-x "Hello from Claude Code!"
/baoyu-post-to-x "Hello from AI Agent!"
# Post with images
/baoyu-post-to-x "Check this out" --image photo.png
@@ -536,12 +606,12 @@ Post content and articles to X (Twitter). Supports regular posts with images and
Post content to WeChat Official Account (微信公众号). Two modes available:
**Image-Text (图)** - Multiple images with short title/content:
**Image-Text (图)** - Multiple images with short title/content:
```bash
/baoyu-post-to-wechat 图 --markdown article.md --images ./photos/
/baoyu-post-to-wechat 图 --markdown article.md --image img1.png --image img2.png --image img3.png
/baoyu-post-to-wechat 图 --title "标题" --content "内容" --image img1.png --submit
/baoyu-post-to-wechat 图 --markdown article.md --images ./photos/
/baoyu-post-to-wechat 图 --markdown article.md --image img1.png --image img2.png --image img3.png
/baoyu-post-to-wechat 图 --title "标题" --content "内容" --image img1.png --submit
```
**Article (文章)** - Full markdown/HTML with rich formatting:
@@ -552,7 +622,117 @@ Post content to WeChat Official Account (微信公众号). Two modes available:
/baoyu-post-to-wechat 文章 --html article.html
```
Prerequisites: Google Chrome installed. First run requires QR code login (session preserved).
**Publishing Methods**:
| Method | Speed | Requirements |
|--------|-------|--------------|
| API (Recommended) | Fast | API credentials (local IP allowlisted in WeChat) |
| Browser | Slow | Chrome, login session |
| Remote API | Fast | API credentials + SSH-reachable server whose IP is on WeChat's allowlist |
**API Configuration** (for faster publishing):
```bash
# Add to .baoyu-skills/.env (project-level) or ~/.baoyu-skills/.env (user-level)
WECHAT_APP_ID=your_app_id
WECHAT_APP_SECRET=your_app_secret
```
To obtain credentials:
1. Visit https://developers.weixin.qq.com/platform/
2. Go to: 我的业务 → 公众号 → 开发密钥
3. Create development key and copy AppID/AppSecret
4. Add your machine's IP to the whitelist
**Browser Method** (no API setup needed): Requires Google Chrome. First run opens browser for QR code login (session preserved).
**Remote API Method** (for when WeChat's IP allowlist excludes your local machine): tunnels WeChat API calls through an SSH SOCKS5 dynamic port forward to a server whose IP is on the allowlist. No files are written to the remote host and `AppSecret` never leaves the local process. Add to your EXTEND.md:
```yaml
# Optional: only set when WeChat's IP allowlist excludes your local machine
remote_publish_host: server.example.com
remote_publish_user: deploy
remote_publish_identity_file: ~/.ssh/id_ed25519
```
Then publish with `--remote` (or set `default_publish_method: remote-api`). Authentication is SSH key only; only the typed `remote_publish_*` keys are honored.
**Multi-Account Support**: Manage multiple WeChat Official Accounts via `EXTEND.md`:
```bash
mkdir -p .baoyu-skills/baoyu-post-to-wechat
```
Create `.baoyu-skills/baoyu-post-to-wechat/EXTEND.md`:
```yaml
# Global settings (shared across all accounts)
default_theme: default
default_color: blue
# Account list
accounts:
- name: My Tech Blog
alias: tech-blog
default: false
default_publish_method: api
default_author: Author Name
need_open_comment: 1
only_fans_can_comment: 0
app_id: your_wechat_app_id
app_secret: your_wechat_app_secret
- name: AI Newsletter
alias: ai-news
default_publish_method: browser
default_author: AI Newsletter
need_open_comment: 1
only_fans_can_comment: 0
```
| Accounts configured | Behavior |
|---------------------|----------|
| No `accounts` block | Single-account mode (backward compatible) |
| 1 account | Auto-select, no prompt |
| 2+ accounts | Prompt to select, or use `--account <alias>` |
| 1 account has `default: true` | Pre-selected as default |
Each account gets an isolated Chrome profile for independent login sessions (browser method). API credentials can be set inline in EXTEND.md or via `.env` with alias-prefixed keys (e.g., `WECHAT_TECH_BLOG_APP_ID`).
#### baoyu-post-to-weibo
Post content to Weibo (微博). Supports regular posts with text, images, and videos, and headline articles (头条文章) with Markdown input. Uses real Chrome with CDP to bypass anti-automation.
**Regular Posts** - Text + images/videos (max 18 files):
```bash
# Post with text
/baoyu-post-to-weibo "Hello Weibo!"
# Post with images
/baoyu-post-to-weibo "Check this out" --image photo.png
# Post with video
/baoyu-post-to-weibo "Watch this" --video clip.mp4
```
**Headline Articles (头条文章)** - Long-form Markdown:
```bash
# Publish article
/baoyu-post-to-weibo --article article.md
# With cover image
/baoyu-post-to-weibo --article article.md --cover cover.jpg
```
**Article Options**:
| Option | Description |
|--------|-------------|
| `--cover <path>` | Cover image |
| `--title <text>` | Override title (max 32 chars) |
| `--summary <text>` | Override summary (max 44 chars) |
**Note**: Scripts fill content into the browser. User reviews and publishes manually. First run requires manual Weibo login (session persists).
### AI Generation Skills
@@ -560,7 +740,7 @@ AI-powered generation backends.
#### baoyu-image-gen
AI SDK-based image generation using official OpenAI and Google APIs. Supports text-to-image, reference images, aspect ratios, and quality presets.
AI SDK-based image generation using OpenAI GPT Image 2, Azure OpenAI, Google, OpenRouter, DashScope (Aliyun Tongyi Wanxiang), MiniMax, Jimeng (即梦), Seedream (豆包), and Replicate APIs. Supports text-to-image, reference images, aspect ratios, custom sizes, batch generation, and quality presets.
```bash
# Basic generation (auto-detect provider)
@@ -573,10 +753,52 @@ AI SDK-based image generation using official OpenAI and Google APIs. Supports te
/baoyu-image-gen --prompt "A banner" --image banner.png --quality 2k
# Specific provider
/baoyu-image-gen --prompt "A cat" --image cat.png --provider openai
/baoyu-image-gen --prompt "A cat" --image cat.png --provider openai --model gpt-image-2
# With reference images (Google multimodal only)
# Azure OpenAI (model = deployment name)
/baoyu-image-gen --prompt "A cat" --image cat.png --provider azure --model gpt-image-2
# OpenRouter
/baoyu-image-gen --prompt "A cat" --image cat.png --provider openrouter
# OpenRouter with reference images
/baoyu-image-gen --prompt "Make it blue" --image out.png --provider openrouter --model google/gemini-3.1-flash-image --ref source.png
# DashScope (Aliyun Tongyi Wanxiang)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider dashscope
# DashScope with custom size
/baoyu-image-gen --prompt "为咖啡品牌设计一张 21:9 横幅海报,包含清晰中文标题" --image banner.png --provider dashscope --model qwen-image-2.0-pro --size 2048x872
# Z.AI GLM-Image
/baoyu-image-gen --prompt "一张带清晰中文标题的科技海报" --image out.png --provider zai
# MiniMax
/baoyu-image-gen --prompt "A fashion editorial portrait by a bright studio window" --image out.jpg --provider minimax
# MiniMax with subject reference
/baoyu-image-gen --prompt "A girl stands by the library window, cinematic lighting" --image out.jpg --provider minimax --model image-01 --ref portrait.png --ar 16:9
# Replicate (default: google/nano-banana-2)
/baoyu-image-gen --prompt "A cat" --image cat.png --provider replicate
# Replicate Seedream 4.5
/baoyu-image-gen --prompt "A studio portrait" --image portrait.png --provider replicate --model bytedance/seedream-4.5 --ar 3:2
# Replicate Wan 2.7 Image Pro
/baoyu-image-gen --prompt "A concept frame" --image frame.png --provider replicate --model wan-video/wan-2.7-image-pro --size 2048x1152
# Jimeng (即梦)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider jimeng
# Seedream (豆包)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider seedream
# With reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate, MiniMax, or Seedream 5.0/4.5/4.0)
/baoyu-image-gen --prompt "Make it blue" --image out.png --ref source.png
# Batch mode
/baoyu-image-gen --batchfile batch.json --jobs 4 --json
```
**Options**:
@@ -585,27 +807,85 @@ AI SDK-based image generation using official OpenAI and Google APIs. Supports te
| `--prompt`, `-p` | Prompt text |
| `--promptfiles` | Read prompt from files (concatenated) |
| `--image` | Output image path (required) |
| `--provider` | `google` or `openai` (default: google) |
| `--model`, `-m` | Model ID |
| `--batchfile` | JSON batch file for multi-image generation |
| `--jobs` | Worker count for batch mode |
| `--provider` | `google`, `openai`, `azure`, `openrouter`, `dashscope`, `zai`, `minimax`, `jimeng`, `seedream`, `replicate`, or `agnes` |
| `--model`, `-m` | Model ID or deployment name. Azure uses deployment name; OpenRouter uses full model IDs; Z.AI uses `glm-image`; MiniMax uses `image-01` / `image-01-live` |
| `--ar` | Aspect ratio (e.g., `16:9`, `1:1`, `4:3`) |
| `--size` | Size (e.g., `1024x1024`) |
| `--quality` | `normal` or `2k` (default: normal) |
| `--ref` | Reference images (Google multimodal only) |
| `--size` | Size (e.g., `1024x1024`; `gpt-image-2` accepts valid custom sizes up to 3840px max edge) |
| `--quality` | `normal` or `2k` (default: `2k`) |
| `--imageSize` | `1K`, `2K`, or `4K` for Google/OpenRouter |
| `--imageApiDialect` | `openai-native` or `ratio-metadata` for OpenAI-compatible gateways |
| `--ref` | Reference images (Google, OpenAI, Azure OpenAI, OpenRouter, Replicate supported families, MiniMax, or Seedream 5.0/4.5/4.0) |
| `--n` | Number of images per request (`replicate` currently requires `--n 1`) |
| `--json` | JSON output |
**Environment Variables** (see [Environment Configuration](#environment-configuration) for setup):
| Variable | Description | Default |
|----------|-------------|---------|
| `OPENAI_API_KEY` | OpenAI API key | - |
| `AZURE_OPENAI_API_KEY` | Azure OpenAI API key | - |
| `OPENROUTER_API_KEY` | OpenRouter API key | - |
| `GOOGLE_API_KEY` | Google API key | - |
| `OPENAI_IMAGE_MODEL` | OpenAI model | `gpt-image-1.5` |
| `GOOGLE_IMAGE_MODEL` | Google model | `gemini-3-pro-image-preview` |
| `GEMINI_API_KEY` | Alias for `GOOGLE_API_KEY` | - |
| `DASHSCOPE_API_KEY` | DashScope API key (Aliyun) | - |
| `ZAI_API_KEY` | Z.AI API key | - |
| `BIGMODEL_API_KEY` | Backward-compatible alias for Z.AI API key | - |
| `MINIMAX_API_KEY` | MiniMax API key | - |
| `REPLICATE_API_TOKEN` | Replicate API token | - |
| `JIMENG_ACCESS_KEY_ID` | Jimeng Volcengine access key | - |
| `JIMENG_SECRET_ACCESS_KEY` | Jimeng Volcengine secret key | - |
| `ARK_API_KEY` | Seedream Volcengine ARK API key | - |
| `OPENAI_IMAGE_MODEL` | OpenAI model | `gpt-image-2` |
| `AZURE_OPENAI_DEPLOYMENT` | Azure default deployment name | - |
| `AZURE_OPENAI_IMAGE_MODEL` | Backward-compatible Azure deployment/model alias | `gpt-image-2` |
| `OPENROUTER_IMAGE_MODEL` | OpenRouter model | `google/gemini-3.1-flash-image` |
| `GOOGLE_IMAGE_MODEL` | Google model | `gemini-3-pro-image` |
| `DASHSCOPE_IMAGE_MODEL` | DashScope model | `qwen-image-2.0-pro` |
| `ZAI_IMAGE_MODEL` | Z.AI model | `glm-image` |
| `BIGMODEL_IMAGE_MODEL` | Backward-compatible alias for Z.AI model | `glm-image` |
| `MINIMAX_IMAGE_MODEL` | MiniMax model | `image-01` |
| `REPLICATE_IMAGE_MODEL` | Replicate model | `google/nano-banana-2` |
| `JIMENG_IMAGE_MODEL` | Jimeng model | `jimeng_t2i_v40` |
| `SEEDREAM_IMAGE_MODEL` | Seedream model | `doubao-seedream-5-0-260128` |
| `OPENAI_BASE_URL` | Custom OpenAI endpoint | - |
| `OPENAI_IMAGE_API_DIALECT` | OpenAI-compatible image API dialect (`openai-native` or `ratio-metadata`) | `openai-native` |
| `OPENAI_IMAGE_USE_CHAT` | Use `/chat/completions` for OpenAI image generation | `false` |
| `AZURE_OPENAI_BASE_URL` | Azure resource or deployment endpoint | - |
| `AZURE_API_VERSION` | Azure image API version | `2025-04-01-preview` |
| `OPENROUTER_BASE_URL` | Custom OpenRouter endpoint | `https://openrouter.ai/api/v1` |
| `OPENROUTER_HTTP_REFERER` | Optional app/site URL for OpenRouter attribution | - |
| `OPENROUTER_TITLE` | Optional app name for OpenRouter attribution | - |
| `GOOGLE_BASE_URL` | Custom Google endpoint | - |
| `DASHSCOPE_BASE_URL` | Custom DashScope endpoint | - |
| `ZAI_BASE_URL` | Custom Z.AI endpoint | `https://api.z.ai/api/paas/v4` |
| `BIGMODEL_BASE_URL` | Backward-compatible alias for Z.AI endpoint | - |
| `MINIMAX_BASE_URL` | Custom MiniMax endpoint | `https://api.minimaxi.com` |
| `REPLICATE_BASE_URL` | Custom Replicate endpoint | - |
| `JIMENG_BASE_URL` | Custom Jimeng endpoint | `https://visual.volcengineapi.com` |
| `JIMENG_REGION` | Jimeng region | `cn-north-1` |
| `SEEDREAM_BASE_URL` | Custom Seedream endpoint | `https://ark.cn-beijing.volces.com/api/v3` |
| `BAOYU_IMAGE_GEN_MAX_WORKERS` | Override batch worker cap | `10` |
| `BAOYU_IMAGE_GEN_<PROVIDER>_CONCURRENCY` | Override provider concurrency | provider-specific |
| `BAOYU_IMAGE_GEN_<PROVIDER>_START_INTERVAL_MS` | Override provider request start gap | provider-specific |
**Provider Notes**:
- Azure OpenAI: `--model` means Azure deployment name, not the underlying model family.
- DashScope: `qwen-image-2.0-pro` is the recommended default for custom `--size`, `21:9`, and strong Chinese/English text rendering.
- Z.AI: `glm-image` is recommended for posters, diagrams, and text-heavy Chinese/English images. Reference images are not supported.
- MiniMax: `image-01` supports documented custom `width` / `height`; `image-01-live` is lower latency and works best with `--ar`.
- MiniMax reference images are sent as `subject_reference`; the current API is specialized toward character / portrait consistency.
- Jimeng does not support reference images.
- Seedream reference images are supported by Seedream 5.0 / 4.5 / 4.0, not Seedream 3.0.
- Replicate defaults to `google/nano-banana-2`. `baoyu-image-gen` only enables Replicate advanced options for `google/nano-banana*`, `bytedance/seedream-4.5`, `bytedance/seedream-5-lite`, `wan-video/wan-2.7-image`, and `wan-video/wan-2.7-image-pro`.
- Replicate currently saves exactly one output image per request. `--n > 1` is blocked locally instead of silently dropping extra results.
- Replicate model behavior is family-specific: nano-banana uses `--quality` / `--ar`, Seedream uses validated `--size` / `--ar`, and Wan uses validated `--size` (with `--ar` converted locally to a concrete size).
**Provider Auto-Selection**:
1. If `--provider` specified → use it
2. If only one API key available → use that provider
3. If both available → default to Google
1. If `--provider` is specified → use it
2. If `--ref` is provided and no provider is specified → try Google, then OpenAI, Azure, OpenRouter, Replicate, Seedream, MiniMax, and finally Agnes
3. If only one API key is available → use that provider
4. If multiple providers are available → default to Google, then OpenAI, Azure, OpenRouter, DashScope, Z.AI, MiniMax, Replicate, Jimeng, Seedream, Agnes
#### baoyu-danger-gemini-web
@@ -629,9 +909,43 @@ Interacts with Gemini Web to generate text and images.
Utility tools for content processing.
#### baoyu-youtube-transcript
Download YouTube video transcripts/subtitles and cover images. Supports multiple languages, translation, chapters, and speaker identification. Caches raw data for fast re-formatting.
```bash
# Default: markdown with timestamps
/baoyu-youtube-transcript https://www.youtube.com/watch?v=VIDEO_ID
# Specify languages (priority order)
/baoyu-youtube-transcript https://youtu.be/VIDEO_ID --languages zh,en,ja
# With chapters and speaker identification
/baoyu-youtube-transcript https://youtu.be/VIDEO_ID --chapters --speakers
# SRT subtitle format
/baoyu-youtube-transcript https://youtu.be/VIDEO_ID --format srt
# List available transcripts
/baoyu-youtube-transcript https://youtu.be/VIDEO_ID --list
```
**Options**:
| Option | Description | Default |
|--------|-------------|---------|
| `<url-or-id>` | YouTube URL or video ID | Required |
| `--languages <codes>` | Language codes, comma-separated | `en` |
| `--format <fmt>` | Output format: `text`, `srt` | `text` |
| `--translate <code>` | Translate to specified language | |
| `--chapters` | Chapter segmentation from video description | |
| `--speakers` | Speaker identification (requires AI post-processing) | |
| `--no-timestamps` | Disable timestamps | |
| `--list` | List available transcripts | |
| `--refresh` | Force re-fetch, ignore cache | |
#### baoyu-url-to-markdown
Fetch any URL via Chrome CDP and convert to clean markdown. Supports two capture modes for different scenarios.
Fetch any URL via Chrome CDP and convert to clean markdown. Saves rendered HTML snapshot alongside the markdown, and automatically falls back to a legacy extractor when Defuddle fails.
```bash
# Auto mode (default) - capture when page loads
@@ -671,6 +985,9 @@ Converts X (Twitter) content to markdown format. Supports tweet threads and X Ar
# JSON output
/baoyu-danger-x-to-markdown https://x.com/username/status/123456 --json
# Download media (images/videos) to local files
/baoyu-danger-x-to-markdown https://x.com/username/status/123456 --download-media
```
**Supported URLs:**
@@ -689,6 +1006,206 @@ Compress images to reduce file size while maintaining quality.
/baoyu-compress-image path/to/images/ --quality 80
```
#### baoyu-format-markdown
Format plain text or markdown files with proper frontmatter, titles, summaries, headings, bold, lists, and code blocks.
```bash
# Format a markdown file
/baoyu-format-markdown path/to/article.md
# Format with specific output
/baoyu-format-markdown path/to/draft.md
```
**Workflow**:
1. Read source file and analyze content structure
2. Check/create YAML frontmatter (title, slug, summary, coverImage)
3. Handle title: use existing, extract from H1, or generate candidates
4. Apply formatting: headings, bold, lists, code blocks, quotes
5. Save to `{filename}-formatted.md`
6. Run typography script: ASCII→fullwidth quotes, CJK spacing, autocorrect
**Frontmatter Fields**:
| Field | Processing |
|-------|------------|
| `title` | Use existing, extract H1, or generate candidates |
| `slug` | Infer from file path or generate from title |
| `summary` | Generate engaging summary (100-150 chars) |
| `coverImage` | Check for `imgs/cover.png` in same directory |
**Formatting Rules**:
| Element | Format |
|---------|--------|
| Titles | `#`, `##`, `###` hierarchy |
| Key points | `**bold**` |
| Parallel items | `-` unordered or `1.` ordered lists |
| Code/commands | `` `inline` `` or ` ```block``` ` |
| Quotes | `>` blockquote |
#### baoyu-markdown-to-html
Convert markdown files into styled HTML with WeChat-compatible themes, syntax highlighting, and optional bottom citations for external links.
```bash
# Basic conversion
/baoyu-markdown-to-html article.md
# Theme + color
/baoyu-markdown-to-html article.md --theme grace --color red
# Convert ordinary external links to bottom citations
/baoyu-markdown-to-html article.md --cite
```
#### baoyu-translate
Translate articles and documents between languages with three modes: quick (direct), normal (analysis-informed), and refined (full publication-quality workflow with review and polish).
```bash
# Normal mode (default) - analyze then translate
/translate article.md --to zh-CN
# Quick mode - direct translation
/translate article.md --mode quick --to ja
# Refined mode - full workflow with review and polish
/translate article.md --mode refined --to zh-CN
# Translate a URL
/translate https://example.com/article --to zh-CN
# Specify audience
/translate article.md --to zh-CN --audience technical
# Specify style
/translate article.md --to zh-CN --style humorous
# With additional glossary
/translate article.md --to zh-CN --glossary my-terms.md
```
**Options**:
| Option | Description |
|--------|-------------|
| `<source>` | File path, URL, or inline text |
| `--mode <mode>` | `quick`, `normal` (default), `refined` |
| `--from <lang>` | Source language (auto-detect if omitted) |
| `--to <lang>` | Target language (default: `zh-CN`) |
| `--audience <type>` | Target reader profile (default: `general`) |
| `--style <style>` | Translation style (default: `storytelling`) |
| `--glossary <file>` | Additional glossary file |
**Modes**:
| Mode | Steps | Use Case |
|------|-------|----------|
| Quick | Translate | Short texts, informal content |
| Normal | Analyze → Translate | Articles, blog posts |
| Refined | Analyze → Translate → Review → Polish | Publication-quality documents |
After normal mode completes, you can reply "继续润色" or "refine" to continue with review and polish steps.
**Audience Presets**:
| Value | Description |
|-------|-------------|
| `general` | General readers (default) — plain language, more translator's notes |
| `technical` | Developers / engineers — less annotation on common tech terms |
| `academic` | Researchers / scholars — formal register, precise terminology |
| `business` | Business professionals — business-friendly tone |
Custom audience descriptions are also accepted, e.g., `--audience "AI-interested general readers"`.
**Style Presets**:
| Value | Description |
|-------|-------------|
| `storytelling` | Engaging narrative flow (default) — smooth transitions, vivid phrasing |
| `formal` | Professional, structured — neutral tone, no colloquialisms |
| `technical` | Precise, documentation-style — concise, terminology-heavy |
| `literal` | Close to original structure — minimal restructuring |
| `academic` | Scholarly, rigorous — formal register, complex clauses OK |
| `business` | Concise, results-focused — action-oriented, executive-friendly |
| `humorous` | Preserves and adapts humor — witty, recreates comedic effect |
| `conversational` | Casual, spoken-like — friendly, as if explaining to a friend |
| `elegant` | Literary, polished prose — aesthetically refined, carefully crafted |
Custom style descriptions are also accepted, e.g., `--style "poetic and lyrical"`.
**Features**:
- Custom glossaries via EXTEND.md with built-in EN→ZH glossary
- Audience-aware translation with adjustable annotation depth
- Automatic chunking for long documents (4000+ words) with parallel subagent translation
- Figurative language interpreted by meaning, not word-for-word
- Translator's notes for cultural/domain-specific references
- Output directory with all intermediate files preserved
#### baoyu-wechat-summary
Summarize WeChat group chat highlights into a structured digest. Extracts topics, quotes, and stats from group messages using [wx-cli](https://github.com/jackwener/wx-cli). Maintains per-group history, per-user profiles, and per-group fact memory across runs. Supports normal and roast (毒舌) versions, and answers `@bot` questions raised in the chat.
```bash
# Summarize a group's recent messages
/baoyu-wechat-summary 相亲相爱一家人 最近 1
# Weekly summary
/baoyu-wechat-summary AI 技术群 最近 7
# Incremental (since last digest)
/baoyu-wechat-summary 相亲相爱一家人
# Roast version
/baoyu-wechat-summary 相亲相爱一家人 最近 3 天 毒舌版
```
**Requirements**:
- [wx-cli](https://github.com/jackwener/wx-cli) installed (`npm install -g @jackwener/wx-cli`)
- WeChat 4.x running and logged in on macOS
**Features**:
- Topic extraction with attribution and quotes
- Message leaderboard and per-user profiles
- Per-group fact memory: corrections confirmed in chat persist across digests (with injection guardrails)
- Incremental mode (picks up where last digest left off)
- Multi-day range splitting for large batches
- Normal and roast (毒舌) digest versions
- Profile backfill from historical digests
#### baoyu-electron-extract
Extract resources and JavaScript from any installed Electron app's `app.asar`. When `.js.map` files embed `sourcesContent`, restores the original source tree (TypeScript/JSX included); otherwise formats the minified JS/CSS with Prettier in place. Always skips `node_modules`. Works on macOS and Windows; pass `--asar <path>` on other platforms.
```bash
# Extract by app name (default output: ~/Downloads/Codex-electron-extract/)
/baoyu-electron-extract Codex
# Extract by absolute path (.app bundle, install dir, or .asar file)
/baoyu-electron-extract "/Applications/Visual Studio Code.app"
/baoyu-electron-extract --asar /Applications/Codex.app/Contents/Resources/app.asar Codex
# Custom output directory
/baoyu-electron-extract Codex --output ~/work/codex-source
# Preview discovery without writing anything
/baoyu-electron-extract Codex --dry-run
# Overwrite an existing output directory
/baoyu-electron-extract Codex --force
```
**Options**:
| Option | Description | Default |
|--------|-------------|---------|
| `<app>` | App name or absolute path (required unless `--asar`) | — |
| `--output`, `-o` | Output directory | `~/Downloads/<AppName>-electron-extract` |
| `--asar` | Override the resolved `.asar` path | auto-discovered |
| `--force`, `-f` | Allow writing into a non-empty existing output dir | false |
| `--skip-format` | Skip Prettier formatting | false |
| `--skip-restore` | Skip source-map restoration | false |
| `--no-unpacked` | Don't copy `app.asar.unpacked/` alongside | false |
| `--dry-run` | Print resolved paths and exit without writing | false |
| `--json` | Emit one JSON-line summary on stdout | false |
**Output layout**: `extract-report.json` (counts, warnings, paths), `extracted/` (raw asar, formatted in place when no map), `extracted.unpacked/` (native modules if present), and `restored/` (rebuilt source tree from `.js.map` files).
## Environment Configuration
Some skills require API keys or custom configuration. Environment variables can be set in `.env` files:
@@ -709,13 +1226,59 @@ mkdir -p ~/.baoyu-skills
cat > ~/.baoyu-skills/.env << 'EOF'
# OpenAI
OPENAI_API_KEY=sk-xxx
OPENAI_IMAGE_MODEL=gpt-image-1.5
OPENAI_IMAGE_MODEL=gpt-image-2
# OPENAI_BASE_URL=https://api.openai.com/v1
# OPENAI_IMAGE_USE_CHAT=false
# Azure OpenAI
AZURE_OPENAI_API_KEY=xxx
AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
AZURE_OPENAI_DEPLOYMENT=gpt-image-2
# AZURE_API_VERSION=2025-04-01-preview
# OpenRouter
OPENROUTER_API_KEY=sk-or-xxx
OPENROUTER_IMAGE_MODEL=google/gemini-3.1-flash-image
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# OPENROUTER_HTTP_REFERER=https://your-app.example.com
# OPENROUTER_TITLE=Your App Name
# Google
GOOGLE_API_KEY=xxx
GOOGLE_IMAGE_MODEL=gemini-3-pro-image-preview
GOOGLE_IMAGE_MODEL=gemini-3-pro-image
# GOOGLE_BASE_URL=https://generativelanguage.googleapis.com/v1beta
# DashScope (Aliyun Tongyi Wanxiang)
DASHSCOPE_API_KEY=sk-xxx
DASHSCOPE_IMAGE_MODEL=qwen-image-2.0-pro
# DASHSCOPE_BASE_URL=https://dashscope.aliyuncs.com/api/v1
# Z.AI
ZAI_API_KEY=xxx
ZAI_IMAGE_MODEL=glm-image
# ZAI_BASE_URL=https://api.z.ai/api/paas/v4
# MiniMax
MINIMAX_API_KEY=xxx
MINIMAX_IMAGE_MODEL=image-01
# MINIMAX_BASE_URL=https://api.minimaxi.com
# Replicate
REPLICATE_API_TOKEN=r8_xxx
REPLICATE_IMAGE_MODEL=google/nano-banana-2
# REPLICATE_BASE_URL=https://api.replicate.com
# Jimeng (即梦)
JIMENG_ACCESS_KEY_ID=xxx
JIMENG_SECRET_ACCESS_KEY=xxx
JIMENG_IMAGE_MODEL=jimeng_t2i_v40
# JIMENG_BASE_URL=https://visual.volcengineapi.com
# JIMENG_REGION=cn-north-1
# Seedream (豆包)
ARK_API_KEY=xxx
SEEDREAM_IMAGE_MODEL=doubao-seedream-5-0-260128
# SEEDREAM_BASE_URL=https://ark.cn-beijing.volces.com/api/v3
EOF
```
@@ -744,13 +1307,14 @@ mkdir -p .baoyu-skills/baoyu-cover-image
Then create `.baoyu-skills/baoyu-cover-image/EXTEND.md`:
```markdown
## Custom Styles
## Custom Palettes
### brand
- Primary color: #1a73e8
- Secondary color: #34a853
- Font style: Modern sans-serif
- Always include company logo watermark
### corporate-tech
- Primary colors: #1a73e8, #4A90D9
- Background: #F5F7FA
- Accent colors: #00B4D8, #48CAE4
- Decorative hints: Clean lines, subtle gradients
- Best for: SaaS, enterprise, technical
```
The extension content will be loaded before skill execution and override defaults.
@@ -786,6 +1350,20 @@ This skill uses a reverse-engineered X (Twitter) API.
- First use requires consent acknowledgment
- Authentication via environment variables or Chrome login
## Credits
This project was inspired by and builds upon the following open source projects:
- [x-article-publisher-skill](https://github.com/wshuyi/x-article-publisher-skill) by [@wshuyi](https://github.com/wshuyi) — Inspiration for the X article publishing skill
- [doocs/md](https://github.com/doocs/md) by [@doocs](https://github.com/doocs) — Core implementation logic for Markdown to HTML conversion
- [High-density Infographic Prompt](https://waytoagi.feishu.cn/wiki/YG0zwalijihRREkgmPzcWRInnUg) by AJ@WaytoAGI — Inspiration for the infographic skill
- [qiaomu-mondo-poster-design](https://github.com/joeseesun/qiaomu-mondo-poster-design) by [@joeseesun](https://github.com/joeseesun)(乔木) — Inspiration for the Mondo style
- [architecture-diagram-generator](https://github.com/Cocoon-AI/architecture-diagram-generator) by [@Cocoon-AI](https://github.com/Cocoon-AI) — Inspiration for the diagram skill's design system
## License
MIT
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=JimLiu/baoyu-skills&type=Date)](https://www.star-history.com/#JimLiu/baoyu-skills&Date)
+663 -84
View File
@@ -2,7 +2,17 @@
[English](./README.md) | 中文
宝玉分享的 Claude Code 技能集,提升日常工作效率。
宝玉分享的 AI Agent 技能集(适用于 Claude Code、Codex 等),提升日常工作效率。
## 作者的图书
<img width="500" height="500" alt="图解 Skill —— AI 提效实战指南" src="https://github.com/user-attachments/assets/6caef6a2-6f11-490e-a43b-e810df8e9354" />
作者的图书《图解 Skill —— AI 提效实战指南》系统讲解如何设计、编写、安装和迭代 Skill,并配有完整示例、提示词、插图生成工作流和章节配套资源。
- 官方配套仓库:[JimLiu/Illustrated-Agent-Skills](https://github.com/JimLiu/Illustrated-Agent-Skills)
- 购买链接:[京东购买](https://u.jd.com/RDY9YwC)
- 电子书购买链接:https://www.ituring.com.cn/book/3616
## 前置要求
@@ -11,18 +21,41 @@
## 安装
> **提示**:本仓库已收录 20+ 个 skill,请按需安装你真正会用到的那几个,不要一次性全装 —— 每个加载的 skill 都会在 Agent 每次运行时占用额外上下文。
### 快速安装(推荐)
```bash
npx skills add jimliu/baoyu-skills
```
### 注册插件市场
### 发布到 ClawHub / OpenClaw
在 Claude Code 中运行:
现在这个仓库支持把每个 `skills/baoyu-*` 目录作为独立 ClawHub skill 发布。
```bash
/plugin marketplace add jimliu/baoyu-skills
# 预览将要发布的变更
./scripts/sync-clawhub.sh --dry-run
# 发布 ./skills 下所有已变更的 skill
./scripts/sync-clawhub.sh --all
```
ClawHub 按“单个 skill”安装,不是把整个 marketplace 一次性装进去。发布后,用户可以按需安装:
```bash
clawhub install baoyu-image-gen
clawhub install baoyu-markdown-to-html
```
根据 ClawHub 的 registry 规则,发布到 ClawHub 的 skill 会以 `MIT-0` 许可分发。
### 注册插件市场
在 Agent 中运行:
```bash
/plugin marketplace add JimLiu/baoyu-skills
```
### 安装技能
@@ -31,37 +64,35 @@ npx skills add jimliu/baoyu-skills
1. 选择 **Browse and install plugins**
2. 选择 **baoyu-skills**
3. 选择要安装的插件
3. 选择 **baoyu-skills** 插件
4. 选择 **Install now**
**方式二:直接安装**
```bash
# 安装指定插件
/plugin install content-skills@baoyu-skills
/plugin install ai-generation-skills@baoyu-skills
/plugin install utility-skills@baoyu-skills
# 安装 marketplace 中唯一的插件
/plugin install baoyu-skills@baoyu-skills
```
**方式三:告诉 Agent**
直接告诉 Claude Code
直接告诉 Agent
> 请帮我安装 github.com/JimLiu/baoyu-skills 中的 Skills
### 可用插件
| 插件 | 说明 | 包含技能 |
现在 marketplace 只暴露一个插件,这样每个 skill 只会注册一次。
| 插件 | 说明 | 包含内容 |
|------|------|----------|
| **content-skills** | 内容生成和发布 | [xhs-images](#baoyu-xhs-images), [infographic](#baoyu-infographic), [cover-image](#baoyu-cover-image), [slide-deck](#baoyu-slide-deck), [comic](#baoyu-comic), [article-illustrator](#baoyu-article-illustrator), [post-to-x](#baoyu-post-to-x), [post-to-wechat](#baoyu-post-to-wechat) |
| **ai-generation-skills** | AI 生成后端 | [image-gen](#baoyu-image-gen), [danger-gemini-web](#baoyu-danger-gemini-web) |
| **utility-skills** | 内容处理工具 | [url-to-markdown](#baoyu-url-to-markdown), [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image) |
| **baoyu-skills** | 提供内容生成、AI 后端和日常效率工具技能 | 仓库中的全部 skills,仍按下方的内容技能、AI 生成技能、工具技能三个分类展示 |
## 更新技能
更新技能到最新版本:
1.Claude Code 中运行 `/plugin`
1.Agent 中运行 `/plugin`
2. 切换到 **Marketplaces** 标签页(使用方向键或 Tab
3. 选择 **baoyu-skills**
4. 选择 **Update marketplace**
@@ -74,13 +105,25 @@ npx skills add jimliu/baoyu-skills
技能分为三大类:
### 设计技能推荐:baoyu-design
如果你想让本地 Agent 直接做设计,可以试试 [JimLiu/baoyu-design](https://github.com/JimLiu/baoyu-design)。这是一个独立项目,它把 Claude Design 打包成可移植的 Agent Skill,可在 Cursor、Claude Code、Codex、Claude Desktop 或其他能读写文件的编码 Agent 中运行,用来生成精致 UI 稿、可交互原型、线框图、落地页、仪表盘、移动 App 和幻灯片,产物都是自包含 HTML,留在你自己的仓库里。
<a href="https://github.com/JimLiu/baoyu-design">
<img src="https://raw.githubusercontent.com/JimLiu/baoyu-design/main/assets/screenshots/cursor-reader-mac-app.webp" alt="Cursor 运行 baoyu-design" width="720">
</a>
```bash
npx skills add JimLiu/baoyu-design
```
### 内容技能 (Content Skills)
内容生成和发布技能。
#### baoyu-xhs-images
小红书信息图系列生成器。将内容拆解为 1-10 张卡通风格信息图,支持 **风格 × 布局** 二维系统。
小红书图片卡片系列生成器。将内容拆解为 1-10 张卡通风格图片卡片,支持 **风格 × 布局** 系统和可选配色覆盖
```bash
# 自动选择风格和布局
@@ -93,13 +136,22 @@ npx skills add jimliu/baoyu-skills
/baoyu-xhs-images posts/ai-future/article.md --layout dense
# 组合风格和布局
/baoyu-xhs-images posts/ai-future/article.md --style tech --layout list
/baoyu-xhs-images posts/ai-future/article.md --style notion --layout list
# 覆盖配色
/baoyu-xhs-images posts/ai-future/article.md --style notion --palette macaron
# 直接输入内容
/baoyu-xhs-images 今日星座运势
# 非交互模式(跳过所有确认,适用于定时任务)
/baoyu-xhs-images posts/ai-future/article.md --yes
/baoyu-xhs-images posts/ai-future/article.md --yes --preset knowledge-card
```
**风格**(视觉美学):`cute`(默认)、`fresh``warm``bold``minimal``retro``pop``notion``chalkboard`
**风格**(视觉美学):`cute`(默认)、`fresh``warm``bold``minimal``retro``pop``notion``chalkboard``study-notes``screen-print``sketch-notes`
**配色**(可选颜色覆盖):`macaron``warm``neon`
**风格预览**
@@ -133,7 +185,7 @@ npx skills add jimliu/baoyu-skills
#### baoyu-infographic
专业信息图生成器,支持 20 种布局和 17 种视觉风格。分析内容后推荐布局×风格组合,生成可发布的信息图。
专业信息图生成器,支持 21 种布局和 21 种视觉风格。分析内容后推荐布局×风格组合,生成可发布的信息图。
```bash
# 根据内容自动推荐组合
@@ -148,8 +200,9 @@ npx skills add jimliu/baoyu-skills
# 同时指定布局和风格
/baoyu-infographic path/to/content.md --layout funnel --style corporate-memphis
# 指定比例
# 指定比例(预设名称或自定义 W:H
/baoyu-infographic path/to/content.md --aspect portrait
/baoyu-infographic path/to/content.md --aspect 3:4
```
**选项**
@@ -157,7 +210,7 @@ npx skills add jimliu/baoyu-skills
|------|------|
| `--layout <name>` | 信息布局(20 种选项) |
| `--style <name>` | 视觉风格(17 种选项,默认:craft-handmade |
| `--aspect <ratio>` | landscape (16:9)、portrait (9:16)、square (1:1) |
| `--aspect <ratio>` | 预设:landscape (16:9)、portrait (9:16)、square (1:1)。自定义:任意 W:H 比例(如 3:4、4:3、2.35:1 |
| `--lang <code>` | 输出语言(en、zh、ja 等) |
**布局**(信息结构):
@@ -243,9 +296,46 @@ npx skills add jimliu/baoyu-skills
| ![knolling](./screenshots/infographic-styles/knolling.webp) | ![lego-brick](./screenshots/infographic-styles/lego-brick.webp) | |
| knolling | lego-brick | |
#### baoyu-diagram
从源素材生成可直接发布的 SVG 图表 —— 包括流程图、时序/协议图、架构/结构图、示意图(直觉图解)。分析输入素材,推荐图表类型和拆分策略,一次确认后批量生成。Claude 直接输出符合统一设计规范的真实 SVG 代码,产物是自包含的 `.svg` 文件,内嵌样式并自动支持深色模式。
```bash
# 主题描述 —— 技能分析并提出方案
/baoyu-diagram "JWT 认证流程是怎么工作的"
/baoyu-diagram "Kubernetes 架构" --type structural
/baoyu-diagram "OAuth 2.0 流程" --type sequence
# 文件路径 —— 技能读取、分析并提出方案
/baoyu-diagram path/to/article.md
# 语言和输出路径
/baoyu-diagram "微服务架构" --lang zh
/baoyu-diagram "build pipeline" --out docs/build-pipeline.svg
```
**参数**
| 参数 | 说明 |
|------|------|
| `--type <name>` | `flowchart``sequence``structural``illustrative``class``auto`(默认)。跳过类型推荐直接生成。 |
| `--lang <code>` | 输出语言(en、zh、ja 等) |
| `--out <path>` | 输出文件路径。生成聚焦于最重要内容的单张图表。 |
**五种图表类型**
| 类型 | 适用场景 | 触发动词 |
|------|----------|----------|
| `flowchart` | 按顺序走一遍流程 | 流程、步骤、工作流、生命周期、状态机 |
| `sequence` | 谁和谁通信、按什么顺序 | 协议、握手、认证流程、OAuth、TCP、请求/响应 |
| `structural` | 展示什么包含什么、如何组织 | 架构、组件、拓扑、布局、什么在什么里面 |
| `illustrative` | 建立直觉 —— 画出机制本身 | 怎么工作、原理、为什么、直观解释 |
| `class` | 类型是什么、它们如何关联 | 类图、UML、继承、接口、数据模型 |
本技能不调用任何图像生成模型 —— Claude 通过手算坐标直接写 SVG 代码,确保每个图表都遵守设计规范。内嵌的 `<style>` 块包含 `@media (prefers-color-scheme: dark)`,同一个文件在浅色和深色模式下均正确渲染,可嵌入到任意支持 SVG 的宿主环境中。
#### baoyu-cover-image
为文章生成封面图,支持维定制系统:类型 × 风格 × 文字 × 氛围。
为文章生成封面图,支持维定制系统:类型 × 配色 × 渲染 × 文字 × 氛围。11 种配色方案与 7 种渲染风格组合,提供 77 种独特效果。
```bash
# 根据内容自动选择所有维度
@@ -254,44 +344,27 @@ npx skills add jimliu/baoyu-skills
# 快速模式:跳过确认,使用自动选择
/baoyu-cover-image path/to/article.md --quick
# 指定维度
/baoyu-cover-image path/to/article.md --type conceptual --style blueprint
# 指定维度5D 系统)
/baoyu-cover-image path/to/article.md --type conceptual --palette cool --rendering digital
/baoyu-cover-image path/to/article.md --text title-subtitle --mood bold
# 指定宽高比(默认:2.35:1
/baoyu-cover-image path/to/article.md --aspect 16:9
# 风格预设(向后兼容的简写方式
/baoyu-cover-image path/to/article.md --style blueprint
# 指定宽高比(默认:16:9
/baoyu-cover-image path/to/article.md --aspect 2.35:1
# 纯视觉(不含标题文字)
/baoyu-cover-image path/to/article.md --no-title
```
**个维度**
**个维度**
- **类型 (Type)**`hero``conceptual``typography``metaphor``scene``minimal`
- **风格 (Style)**:20 种内置风格(见下方预览)
- **配色 (Palette)**`warm``elegant``cool``dark``earth``vivid``pastel``mono``retro``duotone``macaron`
- **渲染 (Rendering)**`flat-vector``hand-drawn``painterly``digital``pixel``chalk``screen-print`
- **文字 (Text)**`none``title-only`(默认)、`title-subtitle``text-rich`
- **氛围 (Mood)**`subtle``balanced`(默认)、`bold`
可用风格:`elegant`(默认)、`blueprint``bold-editorial``chalkboard``dark-atmospheric``editorial-infographic``fantasy-animation``flat-doodle``intuition-machine``minimal``nature``notion``pixel-art``playful``retro``sketch-notes``vector-illustration``vintage``warm``watercolor`
**风格预览**
| | | |
|:---:|:---:|:---:|
| ![elegant](./screenshots/cover-image-styles/elegant.webp) | ![blueprint](./screenshots/cover-image-styles/blueprint.webp) | ![bold-editorial](./screenshots/cover-image-styles/bold-editorial.webp) |
| elegant | blueprint | bold-editorial |
| ![chalkboard](./screenshots/cover-image-styles/chalkboard.webp) | ![dark-atmospheric](./screenshots/cover-image-styles/dark-atmospheric.webp) | ![editorial-infographic](./screenshots/cover-image-styles/editorial-infographic.webp) |
| chalkboard | dark-atmospheric | editorial-infographic |
| ![fantasy-animation](./screenshots/cover-image-styles/fantasy-animation.webp) | ![intuition-machine](./screenshots/cover-image-styles/intuition-machine.webp) | ![minimal](./screenshots/cover-image-styles/minimal.webp) |
| fantasy-animation | intuition-machine | minimal |
| ![nature](./screenshots/cover-image-styles/nature.webp) | ![notion](./screenshots/cover-image-styles/notion.webp) | ![pixel-art](./screenshots/cover-image-styles/pixel-art.webp) |
| nature | notion | pixel-art |
| ![playful](./screenshots/cover-image-styles/playful.webp) | ![retro](./screenshots/cover-image-styles/retro.webp) | ![sketch-notes](./screenshots/cover-image-styles/sketch-notes.webp) |
| playful | retro | sketch-notes |
| ![vector-illustration](./screenshots/cover-image-styles/vector-illustration.webp) | ![vintage](./screenshots/cover-image-styles/vintage.webp) | ![warm](./screenshots/cover-image-styles/warm.webp) |
| vector-illustration | vintage | warm |
| ![watercolor](./screenshots/cover-image-styles/watercolor.webp) | ![flat-doodle](./screenshots/cover-image-styles/flat-doodle.webp) | |
| watercolor | flat-doodle | |
#### baoyu-slide-deck
从内容生成专业的幻灯片图片。先创建包含样式说明的完整大纲,然后逐页生成幻灯片图片。
@@ -466,20 +539,17 @@ npx skills add jimliu/baoyu-skills
#### baoyu-article-illustrator
智能文章插图技能,采用类型 × 风格维系统。分析文章结构,识别需要视觉辅助的位置,生成插图。
智能文章插图技能,采用类型 × 风格 × 色板三维系统。分析文章结构,识别需要视觉辅助的位置,生成插图。
```bash
# 根据内容自动选择类型和风格
/baoyu-article-illustrator path/to/article.md
# 指定类型
/baoyu-article-illustrator path/to/article.md --type infographic
# 指定风格
/baoyu-article-illustrator path/to/article.md --style blueprint
# 组合类型和风格
/baoyu-article-illustrator path/to/article.md --type flowchart --style notion
# 使用色板覆盖
/baoyu-article-illustrator path/to/article.md --style vector-illustration --palette macaron
```
**类型**(信息结构):
@@ -493,7 +563,7 @@ npx skills add jimliu/baoyu-skills
| `framework` | 概念图、关系图 | 方法论、架构 |
| `timeline` | 时间线进展 | 历史、项目进度 |
**风格**视觉美学):
**风格**渲染手法):
| 风格 | 描述 | 适用场景 |
|------|------|----------|
@@ -506,6 +576,14 @@ npx skills add jimliu/baoyu-skills
| `editorial` | 杂志风格信息图 | 科技解说、新闻 |
| `scientific` | 学术精确图表 | 生物、化学、技术 |
**色板**(可选配色覆盖):
| 色板 | 描述 | 适用场景 |
|------|------|----------|
| `macaron` | 马卡龙柔和色块(浅蓝、浅绿、浅紫、浅橙)暖白底 | 教育、知识分享、教程 |
| `warm` | 暖色系(橙、赭石、金)无冷色 | 品牌、产品、生活方式 |
| `neon` | 霓虹色(粉、青、黄)深色底 | 游戏、复古、潮流 |
**风格预览**
| | | |
@@ -521,9 +599,11 @@ npx skills add jimliu/baoyu-skills
发布内容和文章到 X (Twitter)。支持带图片的普通帖子和 X 文章(长篇 Markdown)。使用真实 Chrome + CDP 绕过反自动化检测。
纯文本输入默认按普通帖子处理,Markdown 文件默认按 X 文章处理。脚本会将内容填入浏览器,用户需手动检查并发布。
```bash
# 发布文字
/baoyu-post-to-x "Hello from Claude Code!"
/baoyu-post-to-x "Hello from AI Agent!"
# 发布带图片
/baoyu-post-to-x "看看这个" --image photo.png
@@ -536,12 +616,12 @@ npx skills add jimliu/baoyu-skills
发布内容到微信公众号,支持两种模式:
**图模式** - 多图配短标题和正文:
**图模式** - 多图配短标题和正文:
```bash
/baoyu-post-to-wechat 图 --markdown article.md --images ./photos/
/baoyu-post-to-wechat 图 --markdown article.md --image img1.png --image img2.png --image img3.png
/baoyu-post-to-wechat 图 --title "标题" --content "内容" --image img1.png --submit
/baoyu-post-to-wechat 图 --markdown article.md --images ./photos/
/baoyu-post-to-wechat 图 --markdown article.md --image img1.png --image img2.png --image img3.png
/baoyu-post-to-wechat 图 --title "标题" --content "内容" --image img1.png --submit
```
**文章模式** - 完整 markdown/HTML 富文本格式:
@@ -552,7 +632,108 @@ npx skills add jimliu/baoyu-skills
/baoyu-post-to-wechat 文章 --html article.html
```
前置要求:已安装 Google Chrome,首次运行需扫码登录(登录状态会保存)
**发布方式**
| 方式 | 速度 | 要求 |
|------|------|------|
| API(推荐) | 快 | API 凭证(本机 IP 在公众号白名单内) |
| 浏览器 | 慢 | Chrome,登录会话 |
| 远程 API | 快 | API 凭证 + 一台 IP 在公众号白名单内、可 SSH 登录的服务器 |
**API 配置**(更快的发布方式):
```bash
# 添加到 .baoyu-skills/.env(项目级)或 ~/.baoyu-skills/.env(用户级)
WECHAT_APP_ID=你的AppID
WECHAT_APP_SECRET=你的AppSecret
```
获取凭证方法:
1. 访问 https://developers.weixin.qq.com/platform/
2. 进入:我的业务 → 公众号 → 开发密钥
3. 添加开发密钥,复制 AppID 和 AppSecret
4. 将你操作的机器 IP 加入白名单
**浏览器方式**(无需 API 配置):需已安装 Google Chrome,首次运行需扫码登录(登录状态会保存)
**远程 API 方式**(适用于本机 IP 不在公众号白名单内的情况):通过 SSH SOCKS5 动态端口转发,将对 `api.weixin.qq.com` 的 HTTPS 调用转发到 IP 在白名单内的服务器上。Markdown 渲染、图片处理、草稿组装仍在本地完成;远端不会落任何文件,`AppSecret` 不会离开本地进程。仅支持 SSH 密钥认证,且只接受类型化的 `remote_publish_*` 配置项,不透传任意 ssh 选项。在 EXTEND.md 中配置 `remote_publish_host` 等字段后,发布时加上 `--remote`(或将 `default_publish_method` 设为 `remote-api`)。
**多账号支持**:通过 `EXTEND.md` 管理多个微信公众号:
```bash
mkdir -p .baoyu-skills/baoyu-post-to-wechat
```
创建 `.baoyu-skills/baoyu-post-to-wechat/EXTEND.md`
```yaml
# 全局设置(所有账号共享)
default_theme: default
default_color: blue
# 账号列表
accounts:
- name: 宝玉的技术分享
alias: baoyu
default: false
default_publish_method: api
default_author: 宝玉
need_open_comment: 1
only_fans_can_comment: 0
app_id: 你的微信 AppID
app_secret: 你的微信 AppSecret
- name: AI 工具集
alias: ai-tools
default_publish_method: browser
default_author: AI 工具集
need_open_comment: 1
only_fans_can_comment: 0
```
| 账号配置情况 | 行为 |
|-------------|------|
| 无 `accounts` 块 | 单账号模式(向后兼容) |
| 1 个账号 | 自动选择,无需提示 |
| 2+ 个账号 | 提示选择,或使用 `--account <别名>` |
| 某账号设置 `default: true` | 预选为默认账号 |
每个账号拥有独立的 Chrome 配置目录,保证浏览器方式下的登录会话互不干扰。API 凭证可在 EXTEND.md 中直接配置,也可通过 `.env` 文件使用别名前缀的环境变量(如 `WECHAT_BAOYU_APP_ID`)。
#### baoyu-post-to-weibo
发布内容到微博。支持文字、图片、视频发布和头条文章(长篇 Markdown)。使用真实 Chrome + CDP 绕过反自动化检测。
**普通微博** - 文字 + 图片/视频(最多 18 个文件):
```bash
# 发布文字
/baoyu-post-to-weibo "Hello Weibo!"
# 发布带图片
/baoyu-post-to-weibo "看看这个" --image photo.png
# 发布带视频
/baoyu-post-to-weibo "看这个" --video clip.mp4
```
**头条文章** - 长篇 Markdown 文章:
```bash
# 发布文章
/baoyu-post-to-weibo --article article.md
# 带封面图
/baoyu-post-to-weibo --article article.md --cover cover.jpg
```
**文章选项**
| 选项 | 说明 |
|------|------|
| `--cover <path>` | 封面图 |
| `--title <text>` | 覆盖标题(最多 32 字) |
| `--summary <text>` | 覆盖摘要(最多 44 字) |
**说明**:脚本会将内容填入浏览器,用户需手动检查并发布。首次运行需手动登录微博(登录状态会保存)。
### AI 生成技能 (AI Generation Skills)
@@ -560,7 +741,7 @@ AI 驱动的生成后端。
#### baoyu-image-gen
基于 AI SDK 的图像生成,使用官方 OpenAI 和 Google API。支持文生图、参考图、宽高比和质量预设。
基于 AI SDK 的图像生成,支持 OpenAI GPT Image 2、Azure OpenAI、Google、OpenRouter、DashScope(阿里通义万相)、MiniMax、即梦(Jimeng)、豆包(Seedream)和 Replicate API。支持文生图、参考图、宽高比、自定义尺寸、批量生成和质量预设。
```bash
# 基础生成(自动检测服务商)
@@ -573,10 +754,52 @@ AI 驱动的生成后端。
/baoyu-image-gen --prompt "横幅图" --image banner.png --quality 2k
# 指定服务商
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider openai
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider openai --model gpt-image-2
# 带参考图(仅 Google 多模态支持
# Azure OpenAImodel 为部署名称
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider azure --model gpt-image-2
# OpenRouter
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider openrouter
# OpenRouter + 参考图
/baoyu-image-gen --prompt "把它变成蓝色" --image out.png --provider openrouter --model google/gemini-3.1-flash-image --ref source.png
# DashScope(阿里通义万相)
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider dashscope
# DashScope 自定义尺寸
/baoyu-image-gen --prompt "为咖啡品牌设计一张 21:9 横幅海报,包含清晰中文标题" --image banner.png --provider dashscope --model qwen-image-2.0-pro --size 2048x872
# Z.AI GLM-Image
/baoyu-image-gen --prompt "一张带清晰中文标题的科技海报" --image out.png --provider zai
# MiniMax
/baoyu-image-gen --prompt "A fashion editorial portrait by a bright studio window" --image out.jpg --provider minimax
# MiniMax + 角色参考图
/baoyu-image-gen --prompt "A girl stands by the library window, cinematic lighting" --image out.jpg --provider minimax --model image-01 --ref portrait.png --ar 16:9
# Replicate(默认:google/nano-banana-2
/baoyu-image-gen --prompt "一只猫" --image cat.png --provider replicate
# Replicate Seedream 4.5
/baoyu-image-gen --prompt "一张影棚人像" --image portrait.png --provider replicate --model bytedance/seedream-4.5 --ar 3:2
# Replicate Wan 2.7 Image Pro
/baoyu-image-gen --prompt "一张概念分镜" --image frame.png --provider replicate --model wan-video/wan-2.7-image-pro --size 2048x1152
# 即梦(Jimeng
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider jimeng
# 豆包(Seedream
/baoyu-image-gen --prompt "一只可爱的猫" --image cat.png --provider seedream
# 带参考图(Google、OpenAI、Azure OpenAI、OpenRouter、Replicate、MiniMax 或 Seedream 5.0/4.5/4.0
/baoyu-image-gen --prompt "把它变成蓝色" --image out.png --ref source.png
# 批量模式
/baoyu-image-gen --batchfile batch.json --jobs 4 --json
```
**选项**
@@ -585,27 +808,85 @@ AI 驱动的生成后端。
| `--prompt`, `-p` | 提示词文本 |
| `--promptfiles` | 从文件读取提示词(多文件拼接) |
| `--image` | 输出图片路径(必需) |
| `--provider` | `google``openai`(默认:google |
| `--model`, `-m` | 模型 ID |
| `--batchfile` | 多图批量生成的 JSON 文件 |
| `--jobs` | 批量模式的并发 worker 数 |
| `--provider` | `google``openai``azure``openrouter``dashscope``zai``minimax``jimeng``seedream``replicate``agnes` |
| `--model`, `-m` | 模型 ID 或部署名。Azure 使用部署名;OpenRouter 使用完整模型 IDZ.AI 使用 `glm-image`MiniMax 使用 `image-01` / `image-01-live` |
| `--ar` | 宽高比(如 `16:9``1:1``4:3` |
| `--size` | 尺寸(如 `1024x1024` |
| `--quality` | `normal``2k`(默认:normal |
| `--ref` | 参考图片(仅 Google 多模态支持) |
| `--size` | 尺寸(如 `1024x1024``gpt-image-2` 支持最长边不超过 3840px 的有效自定义尺寸 |
| `--quality` | `normal``2k`(默认:`2k` |
| `--imageSize` | Google/OpenRouter 使用的 `1K``2K``4K` |
| `--imageApiDialect` | OpenAI 兼容网关的图像 API 方言(`openai-native``ratio-metadata` |
| `--ref` | 参考图片(Google、OpenAI、Azure OpenAI、OpenRouter、Replicate 支持的模型家族、MiniMax 或 Seedream 5.0/4.5/4.0 |
| `--n` | 单次请求生成图片数量(`replicate` 当前只支持 `--n 1` |
| `--json` | 输出 JSON 结果 |
**环境变量**(配置方法见[环境配置](#环境配置)):
| 变量 | 说明 | 默认值 |
|------|------|--------|
| `OPENAI_API_KEY` | OpenAI API 密钥 | - |
| `AZURE_OPENAI_API_KEY` | Azure OpenAI API 密钥 | - |
| `OPENROUTER_API_KEY` | OpenRouter API 密钥 | - |
| `GOOGLE_API_KEY` | Google API 密钥 | - |
| `OPENAI_IMAGE_MODEL` | OpenAI 模型 | `gpt-image-1.5` |
| `GOOGLE_IMAGE_MODEL` | Google 模型 | `gemini-3-pro-image-preview` |
| `GEMINI_API_KEY` | `GOOGLE_API_KEY` 的别名 | - |
| `DASHSCOPE_API_KEY` | DashScope API 密钥(阿里云) | - |
| `ZAI_API_KEY` | Z.AI API 密钥 | - |
| `BIGMODEL_API_KEY` | Z.AI API 密钥向后兼容别名 | - |
| `MINIMAX_API_KEY` | MiniMax API 密钥 | - |
| `REPLICATE_API_TOKEN` | Replicate API Token | - |
| `JIMENG_ACCESS_KEY_ID` | 即梦火山引擎 Access Key | - |
| `JIMENG_SECRET_ACCESS_KEY` | 即梦火山引擎 Secret Key | - |
| `ARK_API_KEY` | 豆包火山引擎 ARK API 密钥 | - |
| `OPENAI_IMAGE_MODEL` | OpenAI 模型 | `gpt-image-2` |
| `AZURE_OPENAI_DEPLOYMENT` | Azure 默认部署名 | - |
| `AZURE_OPENAI_IMAGE_MODEL` | 兼容旧配置的 Azure 部署/模型别名 | `gpt-image-2` |
| `OPENROUTER_IMAGE_MODEL` | OpenRouter 模型 | `google/gemini-3.1-flash-image` |
| `GOOGLE_IMAGE_MODEL` | Google 模型 | `gemini-3-pro-image` |
| `DASHSCOPE_IMAGE_MODEL` | DashScope 模型 | `qwen-image-2.0-pro` |
| `ZAI_IMAGE_MODEL` | Z.AI 模型 | `glm-image` |
| `BIGMODEL_IMAGE_MODEL` | Z.AI 模型向后兼容别名 | `glm-image` |
| `MINIMAX_IMAGE_MODEL` | MiniMax 模型 | `image-01` |
| `REPLICATE_IMAGE_MODEL` | Replicate 模型 | `google/nano-banana-2` |
| `JIMENG_IMAGE_MODEL` | 即梦模型 | `jimeng_t2i_v40` |
| `SEEDREAM_IMAGE_MODEL` | 豆包模型 | `doubao-seedream-5-0-260128` |
| `OPENAI_BASE_URL` | 自定义 OpenAI 端点 | - |
| `OPENAI_IMAGE_API_DIALECT` | OpenAI 兼容图像 API 方言(`openai-native``ratio-metadata` | `openai-native` |
| `OPENAI_IMAGE_USE_CHAT` | OpenAI 改走 `/chat/completions` | `false` |
| `AZURE_OPENAI_BASE_URL` | Azure 资源或部署端点 | - |
| `AZURE_API_VERSION` | Azure 图像 API 版本 | `2025-04-01-preview` |
| `OPENROUTER_BASE_URL` | 自定义 OpenRouter 端点 | `https://openrouter.ai/api/v1` |
| `OPENROUTER_HTTP_REFERER` | OpenRouter 归因用站点 URL | - |
| `OPENROUTER_TITLE` | OpenRouter 归因用应用名 | - |
| `GOOGLE_BASE_URL` | 自定义 Google 端点 | - |
| `DASHSCOPE_BASE_URL` | 自定义 DashScope 端点 | - |
| `ZAI_BASE_URL` | 自定义 Z.AI 端点 | `https://api.z.ai/api/paas/v4` |
| `BIGMODEL_BASE_URL` | Z.AI 端点向后兼容别名 | - |
| `MINIMAX_BASE_URL` | 自定义 MiniMax 端点 | `https://api.minimaxi.com` |
| `REPLICATE_BASE_URL` | 自定义 Replicate 端点 | - |
| `JIMENG_BASE_URL` | 自定义即梦端点 | `https://visual.volcengineapi.com` |
| `JIMENG_REGION` | 即梦区域 | `cn-north-1` |
| `SEEDREAM_BASE_URL` | 自定义豆包端点 | `https://ark.cn-beijing.volces.com/api/v3` |
| `BAOYU_IMAGE_GEN_MAX_WORKERS` | 批量模式最大 worker 数 | `10` |
| `BAOYU_IMAGE_GEN_<PROVIDER>_CONCURRENCY` | 覆盖 provider 并发数 | provider 默认值 |
| `BAOYU_IMAGE_GEN_<PROVIDER>_START_INTERVAL_MS` | 覆盖 provider 请求启动间隔 | provider 默认值 |
**Provider 说明**
- Azure OpenAI`--model` 表示 Azure deployment name,不是底层模型家族名。
- DashScope`qwen-image-2.0-pro` 是自定义 `--size``21:9` 和中英文排版的推荐默认模型。
- Z.AI`glm-image` 适合海报、图表和中英文排版密集的图片生成,暂不支持参考图。
- MiniMax`image-01` 支持官方文档里的自定义 `width` / `height``image-01-live` 更偏低延迟,适合配合 `--ar` 使用。
- MiniMax 参考图会走 `subject_reference`,当前能力更偏角色 / 人像一致性。
- 即梦不支持参考图。
- 豆包参考图能力仅适用于 Seedream 5.0 / 4.5 / 4.0,不适用于 Seedream 3.0。
- Replicate 默认模型改为 `google/nano-banana-2``baoyu-image-gen` 目前只对 `google/nano-banana*``bytedance/seedream-4.5``bytedance/seedream-5-lite``wan-video/wan-2.7-image``wan-video/wan-2.7-image-pro` 开启本地能力识别与校验。
- Replicate 当前只保存单张输出图,`--n > 1` 会在本地直接报错,避免多图结果被静默丢弃。
- Replicate 的参数能力按模型家族区分:nano-banana 走 `--quality` / `--ar`Seedream 走校验后的 `--size` / `--ar`Wan 走校验后的 `--size``--ar` 会先在本地换算成具体尺寸)。
**服务商自动选择**
1. 如果指定了 `--provider` → 使用指定的
2. 如果只有一个 API 密钥 → 使用对应服务商
3. 如果两个都有 → 默认使用 Google
2. 如果传了 `--ref` 且未指定 provider → 依次尝试 Google、OpenAI、Azure、OpenRouter、Replicate、Seedream、MiniMax,最后是 Agnes
3. 如果只有一个 API 密钥 → 使用对应服务商
4. 如果多个可用 → 默认使用 Google,然后依次为 OpenAI、Azure、OpenRouter、DashScope、Z.AI、MiniMax、Replicate、即梦、豆包、Agnes
#### baoyu-danger-gemini-web
@@ -629,9 +910,43 @@ AI 驱动的生成后端。
内容处理工具。
#### baoyu-youtube-transcript
下载 YouTube 视频字幕/转录文本和封面图片。支持多语言、翻译、章节分段和说话人识别。缓存原始数据以便快速重新格式化。
```bash
# 默认:带时间戳的 Markdown
/baoyu-youtube-transcript https://www.youtube.com/watch?v=VIDEO_ID
# 指定语言(按优先级排列)
/baoyu-youtube-transcript https://youtu.be/VIDEO_ID --languages zh,en,ja
# 章节分段 + 说话人识别
/baoyu-youtube-transcript https://youtu.be/VIDEO_ID --chapters --speakers
# SRT 字幕格式
/baoyu-youtube-transcript https://youtu.be/VIDEO_ID --format srt
# 列出可用字幕
/baoyu-youtube-transcript https://youtu.be/VIDEO_ID --list
```
**选项**
| 选项 | 说明 | 默认值 |
|------|------|--------|
| `<url-or-id>` | YouTube URL 或视频 ID | 必填 |
| `--languages <codes>` | 语言代码,逗号分隔 | `en` |
| `--format <fmt>` | 输出格式:`text``srt` | `text` |
| `--translate <code>` | 翻译为指定语言 | |
| `--chapters` | 根据视频描述进行章节分段 | |
| `--speakers` | 说话人识别(需 AI 后处理) | |
| `--no-timestamps` | 禁用时间戳 | |
| `--list` | 列出可用字幕 | |
| `--refresh` | 强制重新获取,忽略缓存 | |
#### baoyu-url-to-markdown
通过 Chrome CDP 抓取任意 URL 并转换为干净的 Markdown。支持两种抓取模式,适应不同场景
通过 Chrome CDP 抓取任意 URL 并转换为 Markdown。同时保存渲染后的 HTML 快照,Defuddle 失败时自动回退到旧版提取器
```bash
# 自动模式(默认)- 页面加载后立即抓取
@@ -671,6 +986,9 @@ AI 驱动的生成后端。
# JSON 输出
/baoyu-danger-x-to-markdown https://x.com/username/status/123456 --json
# 下载媒体文件(图片/视频)到本地
/baoyu-danger-x-to-markdown https://x.com/username/status/123456 --download-media
```
**支持的 URL**
@@ -689,6 +1007,206 @@ AI 驱动的生成后端。
/baoyu-compress-image path/to/images/ --quality 80
```
#### baoyu-format-markdown
格式化纯文本或 Markdown 文件,添加 frontmatter、标题、摘要、层级标题、加粗、列表和代码块。
```bash
# 格式化 markdown 文件
/baoyu-format-markdown path/to/article.md
# 格式化指定文件
/baoyu-format-markdown path/to/draft.md
```
**工作流程**
1. 读取源文件并分析内容结构
2. 检查/创建 YAML frontmattertitle、slug、summary、coverImage
3. 处理标题:使用现有标题、提取 H1 或生成候选标题
4. 应用格式:层级标题、加粗、列表、代码块、引用
5. 保存为 `{文件名}-formatted.md`
6. 运行排版脚本:半角引号→全角引号、中英文空格、autocorrect
**Frontmatter 字段**
| 字段 | 处理方式 |
|------|----------|
| `title` | 使用现有、提取 H1 或生成候选 |
| `slug` | 从文件路径推断或根据标题生成 |
| `summary` | 生成吸引人的摘要(100-150 字) |
| `coverImage` | 检查同目录下 `imgs/cover.png` |
**格式化规则**
| 元素 | 格式 |
|------|------|
| 标题 | `#``##``###` 层级 |
| 重点内容 | `**加粗**` |
| 并列要点 | `-` 无序列表或 `1.` 有序列表 |
| 代码/命令 | `` `行内` `` 或 ` ```代码块``` ` |
| 引用 | `>` 引用块 |
#### baoyu-markdown-to-html
将 Markdown 文件转换为样式化 HTML,支持微信公众号兼容主题、代码高亮,以及可选的外链底部引用。
```bash
# 基础转换
/baoyu-markdown-to-html article.md
# 主题 + 颜色
/baoyu-markdown-to-html article.md --theme grace --color red
# 将普通外链转换为文末引用
/baoyu-markdown-to-html article.md --cite
```
#### baoyu-translate
三模式翻译技能:快速(直接翻译)、标准(分析后翻译)、精翻(完整出版级工作流,含审校与润色)。
```bash
# 标准模式(默认)- 先分析再翻译
/translate article.md --to zh-CN
# 快速模式 - 直接翻译
/translate article.md --mode quick --to ja
# 精翻模式 - 完整工作流,含审校与润色
/translate article.md --mode refined --to zh-CN
# 翻译 URL
/translate https://example.com/article --to zh-CN
# 指定受众
/translate article.md --to zh-CN --audience technical
# 指定风格
/translate article.md --to zh-CN --style humorous
# 附加术语表
/translate article.md --to zh-CN --glossary my-terms.md
```
**选项**
| 选项 | 说明 |
|------|------|
| `<source>` | 文件路径、URL 或行内文本 |
| `--mode <mode>` | `quick``normal`(默认)、`refined` |
| `--from <lang>` | 源语言(省略则自动检测) |
| `--to <lang>` | 目标语言(默认:`zh-CN` |
| `--audience <type>` | 目标读者(默认:`general` |
| `--style <style>` | 翻译风格(默认:`storytelling` |
| `--glossary <file>` | 附加术语表文件 |
**模式**
| 模式 | 步骤 | 适用场景 |
|------|------|----------|
| 快速 | 翻译 | 短文本、非正式内容 |
| 标准 | 分析 → 翻译 | 文章、博客 |
| 精翻 | 分析 → 翻译 → 审校 → 润色 | 出版级文档 |
标准模式完成后,可回复「继续润色」或「refine」继续审校润色步骤。
**受众预设**
| 值 | 说明 |
|----|------|
| `general` | 普通读者(默认)— 通俗语言,更多译注 |
| `technical` | 开发者/工程师 — 常见技术术语少加注释 |
| `academic` | 研究者/学者 — 正式语体,精确术语 |
| `business` | 商务人士 — 商务友好语气 |
也支持自定义受众描述,如 `--audience "对 AI 感兴趣的普通读者"`
**风格预设**
| 值 | 说明 |
|----|------|
| `storytelling` | 叙事流畅(默认)— 过渡自然,表达生动 |
| `formal` | 正式、结构化 — 中性语气,无口语化表达 |
| `technical` | 精确、文档风格 — 简洁,术语密集 |
| `literal` | 贴近原文结构 — 最小化重构 |
| `academic` | 学术、严谨 — 正式语体,复杂从句可接受 |
| `business` | 简洁、结果导向 — 行动导向,高管友好 |
| `humorous` | 保留幽默感 — 诙谐,在目标语言中重现喜剧效果 |
| `conversational` | 口语化、亲切 — 友好,如同朋友间解释 |
| `elegant` | 文学性、优雅 — 精心雕琢,注重韵律美感 |
也支持自定义风格描述,如 `--style "诗意而抒情"`
**特性**
- 通过 EXTEND.md 自定义术语表,内置英中术语表
- 面向受众的翻译,可调节注释深度
- 长文档(4000+ 词)自动分块并行翻译
- 比喻和修辞按意译而非逐字翻译
- 为文化/专业术语添加译注
- 输出目录保留所有中间文件
#### baoyu-wechat-summary
微信群聊精华提取。使用 [wx-cli](https://github.com/jackwener/wx-cli) 从群消息中提取话题、引言和统计数据,生成结构化简报。支持跨次运行的群聊历史、群友画像和群级事实记忆维护,可生成正常版和毒舌版,并在简报中回应群里向 `@bot` 提出的问题。
```bash
# 总结群最近消息
/baoyu-wechat-summary 相亲相爱一家人 最近 1
# 周报
/baoyu-wechat-summary AI 技术群 最近 7
# 增量模式(从上次摘要继续)
/baoyu-wechat-summary 相亲相爱一家人
# 毒舌版
/baoyu-wechat-summary 相亲相爱一家人 最近 3 天 毒舌版
```
**前置要求**
- 安装 [wx-cli](https://github.com/jackwener/wx-cli)`npm install -g @jackwener/wx-cli`
- macOS 上运行并登录 WeChat 4.x
**特性**
- 话题提取,带归属和引言
- 发言排行榜和群友画像
- 群级事实记忆:群友指正过的事实跨期生效(内置防注入规则)
- 增量模式(从上次摘要断点继续)
- 大批量消息自动按天分割
- 正常版和毒舌版两种风格
- 支持从历史摘要回溯初始化画像
#### baoyu-electron-extract
从任意已安装的 Electron 应用的 `app.asar` 中提取资源和 JavaScript。当 `.js.map` 内嵌 `sourcesContent` 时,还原原始源码树(含 TypeScript/JSX);否则用 Prettier 原地美化压缩后的 JS/CSS。始终跳过 `node_modules`。支持 macOS 和 Windows,其他平台请用 `--asar <path>` 指定 asar 文件。
```bash
# 按应用名提取(默认输出:~/Downloads/<AppName>-electron-extract/
/baoyu-electron-extract Codex
# 按绝对路径提取(.app 包、安装目录或 .asar 文件均可)
/baoyu-electron-extract "/Applications/Visual Studio Code.app"
/baoyu-electron-extract --asar /Applications/Codex.app/Contents/Resources/app.asar Codex
# 自定义输出目录
/baoyu-electron-extract Codex --output ~/work/codex-source
# 仅预览发现的路径,不写入任何文件
/baoyu-electron-extract Codex --dry-run
# 覆盖已存在的输出目录
/baoyu-electron-extract Codex --force
```
**选项**
| 选项 | 说明 | 默认值 |
|------|------|--------|
| `<app>` | 应用名或绝对路径(未给 `--asar` 时必填) | — |
| `--output`, `-o` | 输出目录 | `~/Downloads/<AppName>-electron-extract` |
| `--asar` | 覆盖解析得到的 `.asar` 路径 | 自动发现 |
| `--force`, `-f` | 允许写入非空的已有输出目录 | false |
| `--skip-format` | 跳过 Prettier 格式化 | false |
| `--skip-restore` | 跳过 source-map 还原 | false |
| `--no-unpacked` | 不复制同级的 `app.asar.unpacked/` | false |
| `--dry-run` | 打印解析路径后退出,不写文件 | false |
| `--json` | 在 stdout 输出一行 JSON 概要 | false |
**输出结构**`extract-report.json`(计数、警告、路径),`extracted/`(asar 原始内容,无 map 时原地美化),`extracted.unpacked/`(存在时复制的原生模块),以及 `restored/`(基于 `.js.map` 重建的源码树)。
## 环境配置
部分技能需要 API 密钥或自定义配置。环境变量可以在 `.env` 文件中设置:
@@ -709,13 +1227,59 @@ mkdir -p ~/.baoyu-skills
cat > ~/.baoyu-skills/.env << 'EOF'
# OpenAI
OPENAI_API_KEY=sk-xxx
OPENAI_IMAGE_MODEL=gpt-image-1.5
OPENAI_IMAGE_MODEL=gpt-image-2
# OPENAI_BASE_URL=https://api.openai.com/v1
# OPENAI_IMAGE_USE_CHAT=false
# Azure OpenAI
AZURE_OPENAI_API_KEY=xxx
AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
AZURE_OPENAI_DEPLOYMENT=gpt-image-2
# AZURE_API_VERSION=2025-04-01-preview
# OpenRouter
OPENROUTER_API_KEY=sk-or-xxx
OPENROUTER_IMAGE_MODEL=google/gemini-3.1-flash-image
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# OPENROUTER_HTTP_REFERER=https://your-app.example.com
# OPENROUTER_TITLE=你的应用名
# Google
GOOGLE_API_KEY=xxx
GOOGLE_IMAGE_MODEL=gemini-3-pro-image-preview
GOOGLE_IMAGE_MODEL=gemini-3-pro-image
# GOOGLE_BASE_URL=https://generativelanguage.googleapis.com/v1beta
# DashScope(阿里通义万相)
DASHSCOPE_API_KEY=sk-xxx
DASHSCOPE_IMAGE_MODEL=qwen-image-2.0-pro
# DASHSCOPE_BASE_URL=https://dashscope.aliyuncs.com/api/v1
# Z.AI
ZAI_API_KEY=xxx
ZAI_IMAGE_MODEL=glm-image
# ZAI_BASE_URL=https://api.z.ai/api/paas/v4
# MiniMax
MINIMAX_API_KEY=xxx
MINIMAX_IMAGE_MODEL=image-01
# MINIMAX_BASE_URL=https://api.minimaxi.com
# Replicate
REPLICATE_API_TOKEN=r8_xxx
REPLICATE_IMAGE_MODEL=google/nano-banana-2
# REPLICATE_BASE_URL=https://api.replicate.com
# 即梦(Jimeng
JIMENG_ACCESS_KEY_ID=xxx
JIMENG_SECRET_ACCESS_KEY=xxx
JIMENG_IMAGE_MODEL=jimeng_t2i_v40
# JIMENG_BASE_URL=https://visual.volcengineapi.com
# JIMENG_REGION=cn-north-1
# 豆包(Seedream
ARK_API_KEY=xxx
SEEDREAM_IMAGE_MODEL=doubao-seedream-5-0-260128
# SEEDREAM_BASE_URL=https://ark.cn-beijing.volces.com/api/v3
EOF
```
@@ -744,13 +1308,14 @@ mkdir -p .baoyu-skills/baoyu-cover-image
然后创建 `.baoyu-skills/baoyu-cover-image/EXTEND.md`
```markdown
## 自定义风格
## 自定义配色
### brand
- 主色:#1a73e8
- 色:#34a853
- 字体风格:现代无衬线
- 始终包含公司 logo 水印
### corporate-tech
- 主色:#1a73e8#4A90D9
- 背景色:#F5F7FA
- 强调色:#00B4D8#48CAE4
- 装饰提示:简洁线条、渐变效果
- 适用于:SaaS、企业、技术内容
```
扩展内容会在技能执行前加载,并覆盖默认设置。
@@ -786,6 +1351,20 @@ HTTP_PROXY=http://127.0.0.1:7890 HTTPS_PROXY=http://127.0.0.1:7890 /baoyu-danger
- 首次使用需确认免责声明
- 通过环境变量或 Chrome 登录进行身份验证
## 致谢
本项目受到以下开源项目的启发,感谢它们的作者:
- [x-article-publisher-skill](https://github.com/wshuyi/x-article-publisher-skill) by [@wshuyi](https://github.com/wshuyi) — 发布 X 文章技能的灵感来源
- [doocs/md](https://github.com/doocs/md) by [@doocs](https://github.com/doocs) — Markdown 转 HTML 的核心实现逻辑
- [高密度信息图 Prompt](https://waytoagi.feishu.cn/wiki/YG0zwalijihRREkgmPzcWRInnUg) by AJ@WaytoAGI — 信息图技能的灵感来源
- [qiaomu-mondo-poster-design](https://github.com/joeseesun/qiaomu-mondo-poster-design) by [@joeseesun](https://github.com/joeseesun)(乔木) — Mondo 风格的灵感来源
- [architecture-diagram-generator](https://github.com/Cocoon-AI/architecture-diagram-generator) by [@Cocoon-AI](https://github.com/Cocoon-AI) — 图表技能设计体系的灵感来源
## 许可证
MIT
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=JimLiu/baoyu-skills&type=Date)](https://www.star-history.com/#JimLiu/baoyu-skills&Date)
Executable
BIN
View File
Binary file not shown.
+27
View File
@@ -0,0 +1,27 @@
# Chrome Profile
All CDP skills share a single profile directory. Do NOT create per-skill profiles.
Override: `BAOYU_CHROME_PROFILE_DIR` env var (takes priority over all defaults).
| Platform | Default Path |
|----------|-------------|
| macOS | `~/Library/Application Support/baoyu-skills/chrome-profile` |
| Linux | `$XDG_DATA_HOME/baoyu-skills/chrome-profile` (fallback `~/.local/share/`) |
| Windows | `%APPDATA%/baoyu-skills/chrome-profile` |
| WSL | Windows home `/.local/share/baoyu-skills/chrome-profile` |
New skills: use `BAOYU_CHROME_PROFILE_DIR` only (not per-skill env vars like `X_BROWSER_PROFILE_DIR`).
## Implementation Pattern
```typescript
function getDefaultProfileDir(): string {
const override = process.env.BAOYU_CHROME_PROFILE_DIR?.trim();
if (override) return path.resolve(override);
const base = process.platform === 'darwin'
? path.join(os.homedir(), 'Library', 'Application Support')
: process.env.XDG_DATA_HOME || path.join(os.homedir(), '.local', 'share');
return path.join(base, 'baoyu-skills', 'chrome-profile');
}
```
+272
View File
@@ -0,0 +1,272 @@
# `codex-imagegen` Backend
Generate images via Codex CLI's built-in `image_gen` tool from non-Codex runtimes (e.g., Claude Code). The wrapper spawns `codex exec --json` and lets the user's existing Codex subscription drive image generation — **no `OPENAI_API_KEY` required**.
This backend implements the `preferred_image_backend: codex-imagegen` config key already referenced in several `SKILL.md` files across this repo.
## Features
| Feature | Status |
|---------|--------|
| **Reliability**: retry + exponential backoff | Default 2 retries |
| **Verification**: confirms `image_gen` was actually invoked (not bypassed) | Checks `$CODEX_HOME/generated_images/{thread_id}/` |
| **Verification**: PNG magic-byte sanity check | ✓ |
| **Idempotency cache**: reuses output for same prompt+aspect+refs | `--cache-dir` |
| **Concurrency control**: file lock prevents parallel `codex exec` collisions | Built-in |
| **Structured logging**: JSONL log file | `--log-file` |
| **Token usage returned** | Embedded in result JSON |
| **`--ref` reference images** | Repeatable |
| **Unit tests** | 16 tests (parser / cache / validator) |
| **Error classification**: retryable vs non-retryable | 9 `error_kind` values |
## Why this backend
| Scenario | Conventional backend | This backend |
|----------|---------------------|--------------|
| You have a Codex subscription | OpenAI Images API costs add up per image | Subscription already covers it — zero marginal API cost |
| No `OPENAI_API_KEY` available | `baoyu-image-gen` needs an API key | `codex login` is enough |
| Want to use GPT Image 2 | Only via OpenAI API | Codex's `image_gen` *is* GPT Image 2 |
## Prerequisites
```bash
npm install -g @openai/codex
codex login # signs in with your OpenAI account (subscription)
codex --version # confirm >= 0.130
```
`bun` is required for running the wrapper. On macOS:
```bash
brew install oven-sh/bun/bun
```
If `bun` is not on `PATH`, fall back to `npx -y bun packages/baoyu-codex-imagegen/src/main.ts …`.
## Usage
### Direct CLI
```bash
# Inline prompt
bun packages/baoyu-codex-imagegen/src/main.ts \
--image /tmp/cat.png \
--prompt "A friendly orange cat, watercolor"
# Prompt from file
bun packages/baoyu-codex-imagegen/src/main.ts \
--image cover.png \
--prompt-file prompts/01-cover.md \
--aspect 16:9
# Verbose mode for debugging
bun packages/baoyu-codex-imagegen/src/main.ts -v --image dog.png --prompt "A corgi" --aspect 1:1
```
### Through `baoyu-image-gen`
```bash
${BUN_X} skills/baoyu-image-gen/scripts/main.ts \
--provider codex-cli \
--prompt "A friendly orange cat, watercolor" \
--image /tmp/cat.png \
--ar 1:1
```
The `codex-cli` provider spawns the bundled `codex-imagegen` TS entrypoint internally and surfaces its retry/cache machinery through baoyu-image-gen's standard CLI + batch flow.
On success, stdout emits a single JSON line:
```json
{"status":"ok","path":"/tmp/cat.png","bytes":2567101,"elapsed_seconds":53}
```
On failure, exit code is non-zero and stderr contains the error message.
### Enabling within image skills
Image-generating skills (e.g., `baoyu-cover-image`, `baoyu-article-illustrator`) already support a `preferred_image_backend` preference. To route them through this backend, set the following in the corresponding `EXTEND.md`:
```yaml
# ~/.baoyu-skills/baoyu-cover-image/EXTEND.md
preferred_image_backend: codex-imagegen
```
When the LLM runs the skill, it reads the preference and — guided by the `### codex-imagegen Backend` section in `CLAUDE.md` — invokes `bun packages/baoyu-codex-imagegen/src/main.ts`.
> **Note**: The integration is mediated by the LLM reading `CLAUDE.md`. It is not a hard binding. If a skill does not route to the backend automatically, mentioning it explicitly in the prompt works.
## Parameters
| Flag | Required | Description |
|------|----------|-------------|
| `--image <path>` | ✓ | Output PNG path (absolute recommended; relative paths are resolved against cwd) |
| `--prompt <text>` | one of | Prompt string (mutually exclusive with `--prompt-file`) |
| `--prompt-file <path>` | one of | Read prompt from file (mutually exclusive with `--prompt`) |
| `--aspect <ratio>` | | Aspect ratio. Default `1:1`. Common: `16:9`, `9:16`, `4:3`, `2.35:1` |
| `--ref <file>` | | Reference image path (repeatable) |
| `--timeout <ms>` | | `codex exec` timeout in ms. Default `300000` |
| `--retries <n>` | | Retry count on retryable errors. Default `2` (total attempts = retries + 1) |
| `--retry-delay <ms>` | | Base delay between retries (exponential backoff). Default `1500` |
| `--cache-dir <path>` | | Enable idempotency cache (reuses output for same prompt+aspect+refs) |
| `--log-file <path>` | | Structured JSONL log path (appended) |
| `-v` / `--verbose` | | Mirror log entries to stderr |
| `-h` / `--help` | | Show usage |
## Structured Output
On success, stdout contains a single JSON line:
```json
{
"status": "ok",
"path": "/tmp/owl.png",
"bytes": 1693831,
"elapsed_seconds": 87,
"thread_id": "019e40e8-daef-7c60-943d-5e7bb3f6cb3d",
"attempts": 1,
"cached": false,
"usage": {
"input": 110899,
"cached_input": 83456,
"output": 457,
"reasoning": 47
},
"tool_calls": [
{"tool": "shell", "status": "completed"},
{"tool": "agent_message", "status": "completed"}
]
}
```
Cache hits return with `elapsed_seconds: 0`, `cached: true`, `attempts: 0`.
On failure, exit code is `1` and the JSON contains `error` and `error_kind`:
```json
{
"status": "error",
"error": "image_gen was not invoked: no PNG in ...",
"error_kind": "no_image_gen_tool_use"
}
```
## Error Kinds
| `error_kind` | Retryable | Meaning |
|--------------|-----------|---------|
| `codex_not_installed` | ✗ | `codex` CLI not found |
| `invalid_args` | ✗ | Argument parsing error |
| `prompt_file_missing` | ✗ | `--prompt-file` path does not exist |
| `spawn_failed` | ✓ | `codex exec` exited non-zero |
| `timeout` | ✓ | Exceeded `--timeout` |
| `no_image_gen_tool_use` | ✓ | Agent did not invoke `image_gen` (it took another path) |
| `output_missing` | ✓ | Output file not created |
| `invalid_png` | ✓ | Output is not a valid PNG |
| `agent_refused` | ✓ | No `thread_id` in event stream (Codex refused to respond) |
| `lock_busy` | ✗ | Concurrency lock acquisition timed out |
## Measured Performance
| Metric | Value |
|--------|-------|
| First-run latency | 5090 s |
| Cache-hit latency | < 0.3 s |
| Output dimensions | 1024×1024, 1672×941 (16:9), etc. — chosen by `image_gen` |
| Output format | PNG (RGB, 8-bit) |
| Token usage per call | ~110k input (~80k cached) + ~500 output |
| Quota source | Codex subscription (does not consume OpenAI API quota) |
| Default timeout | 300 s (5 min) |
## Limitations & Risks
1. **510× slower than direct API**. `codex exec` cold-starts the agent, loads the built-in `image_gen` SKILL.md, and runs reasoning before invoking the tool. Cache hits avoid this for repeated prompts.
2. **ToS gray area**. Codex's `image_gen` tool is designed for interactive use. Invoking it programmatically via `codex exec` from an external agent is not explicitly addressed by current OpenAI policies. Suggested guardrails:
- Personal, low-volume use is reasonable.
- Not recommended for production automation or high-volume batch jobs.
- Users are responsible for ensuring their usage complies with applicable terms of service.
3. **Sandbox permissions**. The wrapper passes `--sandbox danger-full-access` so the spawned agent can move the rendered PNG out of `$CODEX_HOME/generated_images/`. This is necessary because the agent must `cp`/`mv` the file to the user-specified output path.
4. **Concurrency = 1**. The file lock serializes concurrent invocations to avoid `codex exec` collisions. Parallel calls queue.
## Troubleshooting
| Symptom | `error_kind` | Resolution |
|---------|--------------|------------|
| `command not found: codex` | `codex_not_installed` | `npm install -g @openai/codex` |
| `codex exec` fails | `spawn_failed` | Check `codex login` status; inspect `raw_log` path |
| Timeout | `timeout` | Pass `--timeout 600000` (10 min) for slow networks |
| Agent skipped `image_gen` | `no_image_gen_tool_use` | Auto-retries; consider sharpening the prompt — abstract prompts let the agent wander |
| Output missing | `output_missing` | Agent did not `cp` to the target path; check `raw_log` for the actual save location under `generated_images/` |
| Lock held | `lock_busy` | Wait for the in-flight request to finish; or `rm ~/.cache/baoyu-codex-imagegen/codex-exec.lock` |
| Low image quality | — | Sharpen the prompt, try a different aspect, or supply `--ref` |
## Architecture
```
packages/baoyu-codex-imagegen/
├── src/
│ ├── main.ts # parseArgs → cache → lock → retry loop → emit JSON (`#!/usr/bin/env bun`)
│ ├── types.ts # CliOptions, GenerateResult, GenError, ErrorKind
│ ├── spawn.ts # spawn codex exec --json --sandbox danger-full-access
│ ├── parser.ts # parse JSONL event stream → toolCalls, usage, thread_id
│ ├── validator.ts # verify image_gen invocation + PNG magic + file size
│ ├── cache.ts # cacheKey(sha256), FileLock, lookup/store
│ ├── logger.ts # JsonLogger (verbose stderr + JSONL file)
│ ├── parser.test.ts
│ ├── cache.test.ts
│ └── validator.test.ts
├── package.json # workspace package: `bin` → `src/main.ts`, no build step
└── README.md
```
Run tests:
```bash
cd packages/baoyu-codex-imagegen && bun test
```
## Internal Flow
```mermaid
flowchart LR
CC[Claude Code / any caller]
WRAPPER[bun packages/baoyu-codex-imagegen/<br/>src/main.ts]
CODEX["codex exec --json<br/>--sandbox danger-full-access"]
AGENT[Codex agent]
TOOL[image_gen built-in tool]
DEFAULT["$CODEX_HOME/<br/>generated_images/{thread_id}/"]
OUT[/specified OUTPUT path/]
CC -->|exec wrapper| WRAPPER
WRAPPER -->|stdin: instruction| CODEX
CODEX --> AGENT
AGENT -->|tool call| TOOL
TOOL -->|writes file| DEFAULT
AGENT -->|agent cp/mv| OUT
WRAPPER -->|verify + parse| CC
classDef cc fill:#1e40af,color:#fff,stroke:#93c5fd
classDef cdx fill:#7c2d12,color:#fff,stroke:#fdba74
class CC,WRAPPER cc
class CODEX,AGENT,TOOL cdx
```
## Design Decisions
1. **Pure TypeScript entrypoint**`src/main.ts` carries a `#!/usr/bin/env bun` shebang and is the sole entry. There is no shell shim: callers either invoke `bun src/main.ts …` directly, run the file as an executable (when `bun` is on `PATH`), or fall back to `npx -y bun src/main.ts …`. This matches the project's `skills/<skill>/scripts/main.ts` convention.
2. **`--sandbox danger-full-access`** — necessary so the spawned agent can `cp`/`mv` the rendered PNG out of `$CODEX_HOME/generated_images/` to the user-specified path. Standard sandboxes block this.
3. **Parse the JSONL event stream** — the final `agent_message` and intermediate `command_execution` events let the wrapper verify what actually happened (was `image_gen` called? did `cp` reach the right destination?), which is far more reliable than scraping freeform stdout.
4. **Shared package, not a skill** — this backend is a CLI utility that skills route to via `preferred_image_backend` and that `baoyu-image-gen --provider codex-cli` spawns internally. It lives under `packages/` alongside the other shared workspaces (`baoyu-md`, `baoyu-chrome-cdp`, `baoyu-fetch`) because it has no `SKILL.md` and is never loaded directly by an agent.
5. **File lock instead of internal queue** — keeps the implementation small and works across multiple shell sessions or processes invoking the same wrapper concurrently.
## Related Files
| File | Role |
|------|------|
| `packages/baoyu-codex-imagegen/src/main.ts` | TypeScript CLI entrypoint (`#!/usr/bin/env bun`) |
| `packages/baoyu-codex-imagegen/src/` | TypeScript implementation |
| `packages/baoyu-codex-imagegen/package.json` | Workspace manifest |
| `skills/baoyu-image-gen/scripts/providers/codex-cli.ts` | Provider adapter that lets `baoyu-image-gen --provider codex-cli` spawn this wrapper |
| `docs/codex-imagegen-backend.md` | This document |
| `CLAUDE.md` | Tells LLMs how to invoke this backend |
| `.github/workflows/codex-imagegen-tests.yml` | CI unit tests |
+35
View File
@@ -0,0 +1,35 @@
# Style Maintenance (baoyu-comic)
## Adding a New Style
1. Create style definition: `skills/baoyu-comic/references/styles/<style-name>.md`
2. Update SKILL.md: add to `--style` options table + auto-selection entry
3. Generate showcase image:
```bash
${BUN_X} skills/baoyu-danger-gemini-web/scripts/main.ts \
--prompt "A single comic book page in <style-name> style showing [scene]. Features: [characteristics]. 3:4 portrait aspect ratio comic page." \
--image screenshots/comic-styles/<style-name>.png
```
4. Compress: `${BUN_X} skills/baoyu-compress-image/scripts/main.ts screenshots/comic-styles/<style-name>.png`
5. Update both READMEs (`README.md` + `README.zh.md`): add style to options, description table, preview grid
## Updating an Existing Style
1. Update style definition in `references/styles/`
2. Regenerate showcase image if visual characteristics changed (steps 3-4 above)
3. Update READMEs if description changed
## Deleting a Style
1. Delete style definition + showcase image (`.webp`)
2. Remove from SKILL.md `--style` options + auto-selection
3. Remove from both READMEs (options, description table, preview grid)
## Style Preview Grid Format
```markdown
| | | |
|:---:|:---:|:---:|
| ![style1](./screenshots/comic-styles/style1.webp) | ![style2](./screenshots/comic-styles/style2.webp) | ![style3](./screenshots/comic-styles/style3.webp) |
| style1 | style2 | style3 |
```
+175
View File
@@ -0,0 +1,175 @@
# Creating New Skills
**REQUIRED READING**: [Skill authoring best practices](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices)
## Key Requirements
| Requirement | Details |
|-------------|---------|
| **Prefix** | All skills MUST use `baoyu-` prefix |
| **name field** | Max 64 chars, lowercase letters/numbers/hyphens only, no "anthropic"/"claude" |
| **description** | Max 1024 chars, third person, include what + when to use |
| **SKILL.md body** | Keep under 500 lines; use `references/` for additional content |
| **References** | One level deep from SKILL.md; avoid nested references |
## SKILL.md Frontmatter Template
```yaml
---
name: baoyu-<name>
description: <Third-person description. What it does + when to use it.>
version: <semver matching marketplace.json>
metadata:
openclaw:
homepage: https://github.com/JimLiu/baoyu-skills#baoyu-<name>
requires: # include only if skill has scripts
anyBins:
- bun
- npx
---
```
## Steps
1. Create `skills/baoyu-<name>/SKILL.md` with YAML front matter
2. Add TypeScript in `skills/baoyu-<name>/scripts/` (if applicable)
3. Add prompt templates in `skills/baoyu-<name>/prompts/` if needed
4. Register the skill in `.claude-plugin/marketplace.json` under the `baoyu-skills` plugin entry
5. Add Script Directory section to SKILL.md if skill has scripts
6. Add openclaw metadata to frontmatter
## Skill Grouping
All skills are registered under the single `baoyu-skills` plugin. Use these logical groups when deciding where the skill should appear in the docs:
| If your skill... | Use group |
|------------------|-----------|
| Generates visual content (images, slides, comics) | Content Skills |
| Publishes to platforms (X, WeChat, Weibo) | Content Skills |
| Provides AI generation backend | AI Generation Skills |
| Converts or processes content | Utility Skills |
If you add a new logical group, update the docs that present grouped skills, but keep the skill registered under the single `baoyu-skills` plugin entry.
## Writing Descriptions
**MUST write in third person**:
```yaml
# Good
description: Generates Xiaohongshu infographic series from content. Use when user asks for "小红书图片", "XHS images".
# Bad
description: I can help you create Xiaohongshu images
```
## Script Directory Template
Every SKILL.md with scripts MUST include:
```markdown
## Script Directory
**Important**: All scripts are located in the `scripts/` subdirectory of this skill.
**Agent Execution Instructions**:
1. Determine this SKILL.md file's directory path as `{baseDir}`
2. Script path = `{baseDir}/scripts/<script-name>.ts`
3. Resolve `${BUN_X}` runtime: if `bun` installed → `bun`; if `npx` available → `npx -y bun`; else suggest installing bun
4. Replace all `{baseDir}` and `${BUN_X}` in this document with actual values
**Script Reference**:
| Script | Purpose |
|--------|---------|
| `scripts/main.ts` | Main entry point |
```
## Progressive Disclosure
For skills with extensive content:
```
skills/baoyu-example/
├── SKILL.md # Main instructions (<500 lines)
├── references/
│ ├── styles.md # Loaded as needed
│ └── examples.md # Loaded as needed
└── scripts/
└── main.ts
```
Link from SKILL.md (one level deep only):
```markdown
**Available styles**: See [references/styles.md](references/styles.md)
```
## Extension Support (EXTEND.md)
Every SKILL.md MUST include EXTEND.md loading. Add as Step 1.1 (workflow skills) or "Preferences" section (utility skills):
```markdown
**1.1 Load Preferences (EXTEND.md)**
Check EXTEND.md existence (priority order):
\`\`\`bash
test -f .baoyu-skills/<skill-name>/EXTEND.md && echo "project"
test -f "${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/<skill-name>/EXTEND.md" && echo "xdg"
test -f "$HOME/.baoyu-skills/<skill-name>/EXTEND.md" && echo "user"
\`\`\`
| Path | Location |
|------|----------|
| `.baoyu-skills/<skill-name>/EXTEND.md` | Project directory |
| `$XDG_CONFIG_HOME/baoyu-skills/<skill-name>/EXTEND.md` | XDG config (~/.config) |
| `$HOME/.baoyu-skills/<skill-name>/EXTEND.md` | User home (legacy) |
| Result | Action |
|--------|--------|
| Found | Read, parse, display summary |
| Not found | Ask user via the runtime's user-input tool (see [user-input-tools.md](user-input-tools.md)) |
```
End of SKILL.md should include:
```markdown
## Extension Support
Custom configurations via EXTEND.md. See **Step 1.1** for paths and supported options.
```
## User Input Tools Section (Required)
Every SKILL.md that prompts the user for choices MUST include exactly one `## User Input Tools` section near the top (right after the intro, before the main workflow). The rule must be **inlined** — do NOT link to `docs/user-input-tools.md` (skills are self-contained; see [CLAUDE.md → Skill Self-Containment](../CLAUDE.md)). The author-side canonical reference lives at [user-input-tools.md](user-input-tools.md); copy its body into each new SKILL.md.
Standard snippet (copy verbatim):
```markdown
## User Input Tools
When this skill prompts the user, follow this tool-selection rule (priority order):
1. **Prefer built-in user-input tools** exposed by the current agent runtime — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.
2. **Fallback**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
3. **Batching**: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.
Concrete `AskUserQuestion` references below are examples — substitute the local equivalent in other runtimes.
```
## Image Generation Tools Section (Required for image-gen skills)
Every SKILL.md that renders images — whether by calling an image-generation API directly or by delegating to another skill — MUST include exactly one `## Image Generation Tools` section near the top (after `## User Input Tools`, before the main workflow). The rule must be **inlined** — do NOT link to `docs/image-generation-tools.md` (skills are self-contained; see [CLAUDE.md → Skill Self-Containment](../CLAUDE.md)). The author-side canonical reference lives at [image-generation-tools.md](image-generation-tools.md); copy its body into each new SKILL.md.
Standard snippet (copy verbatim):
```markdown
## Image Generation Tools
When this skill needs to render an image:
- **Use whatever image-generation tool or skill is available** in the current runtime — e.g., Codex `imagegen`, Cursor `GenerateImage`, Hermes `image_generate`, `baoyu-image-gen`, or any equivalent the user has installed.
- **If multiple are available**, ask the user **once** at the start which to use (batch with any other initial questions).
- **If none are available**, tell the user and ask how to proceed.
**Prompt file requirement (hard)**: write each image's full, final prompt to a standalone file under `prompts/` (naming: `NN-{type}-[slug].md`) BEFORE invoking any backend. The backend receives the prompt file (or its content); the file is the reproducibility record and lets you switch backends without regenerating prompts.
Concrete tool names (`imagegen`, `GenerateImage`, `image_generate`, `baoyu-image-gen`) above are examples — substitute the local equivalents under the same rule.
```
+49
View File
@@ -0,0 +1,49 @@
# Image Generation Tools
Skills in this repo are loaded by multiple agent runtimes (Claude Code, Codex, Cursor, Hermes, other agents, bare CLI). Each runtime exposes a different image-generation capability — some have a runtime-native tool (Codex `imagegen`, Cursor `GenerateImage`, Hermes `image_generate`), others rely on an installed skill (`baoyu-image-gen`, or user-defined). This document defines the canonical **backend-selection rule** every skill that renders images follows so skills stay portable.
## The Rule
When a skill needs to render an image, resolve the backend in this order:
1. **Current-request override** — if the user names a specific backend in the current message, use it.
2. **Saved preference** — if the skill's `EXTEND.md` sets `preferred_image_backend` to a backend available right now, use it.
3. **Auto-select** (when the preference is `auto`, unset, or the pinned backend isn't available):
- **Codex (`imagegen`)** — first, inspect your available-skills / tool inventory. If a skill named `imagegen` is listed, you are running inside Codex and MUST use it: invoke via the `Skill` tool with `skill: "imagegen"`, passing the saved prompt file's content (plus output path and aspect ratio per Codex `imagegen`'s own args). Codex `imagegen` is the official raster backend in that runtime and outranks any non-native skill (e.g., `baoyu-image-gen`) unless the user has explicitly pinned a different `preferred_image_backend`.
- **Cursor (`GenerateImage`)** — if the runtime exposes a native `GenerateImage` tool, you are running inside Cursor and it outranks any non-native skill the same way Codex `imagegen` does. Two hard caveats: (a) it has no aspect-ratio parameter — state the target aspect ratio / dimensions explicitly in the prompt text passed as `description`; (b) it does not accept an output directory — it saves to a tool-managed location, so after generation copy/move the file to the skill's expected output path (e.g., `outputs/.../NN-xxx.png`). Reference images go in `reference_image_paths`.
- **Other runtime-native tools** — if the runtime exposes a different native image tool (e.g., Hermes `image_generate`), use it the same way.
- Otherwise, if exactly one non-native backend is installed (e.g., `baoyu-image-gen`), use it.
- Otherwise (multiple non-native backends with no runtime-native tool), ask the user once — batch with any other initial questions.
4. **If none are available**, tell the user and ask how to proceed.
**⛔ Never substitute SVG, HTML, canvas, or other code-based rendering for raster image generation.** Codex `imagegen`'s own description says it should be used "when the output should be a bitmap asset rather than repo-native code or vector." If you cannot resolve a raster backend via step 3, fall through to step 4 and ask the user — do **not** silently emit SVG, write inline `<svg>` markup, or produce HTML/CSS art as a substitute. This applies even if the article/section seems "diagram-like": the consumer skill calling this rule has already decided that a raster image is what it needs.
Setting `preferred_image_backend: ask` forces the step-3 prompt every run regardless of available backends.
## The Preference Field
Each image-consuming skill's `EXTEND.md` carries a single `preferred_image_backend` field:
| Value | Meaning |
|---|---|
| `auto` (default) | Apply the auto-select rule — runtime-native preferred, fall back to only installed backend, ask if multiple non-native. |
| `ask` | Always confirm the backend on every run, even when a runtime-native tool exists. |
| `<backend-id>` (e.g., `codex-imagegen`, `baoyu-image-gen`, `GenerateImage`, `image_generate`) | Pin this backend when available; fall back to `auto` if it isn't. |
The field is **absent-equals-auto**: older `EXTEND.md` files without this field behave exactly as if `preferred_image_backend: auto` were set. No schema version bump is needed to introduce it.
## Prompt File Requirement (hard)
Regardless of which backend is chosen, every skill that renders images MUST write each image's full, final prompt to a standalone file under `prompts/` (naming: `NN-{type}-[slug].md`) BEFORE invoking any backend. The backend receives the prompt file (or its content); the file is the reproducibility record and allows switching backends without regenerating prompts.
## How Skills Declare This
Each `SKILL.md` that renders images includes **exactly one** `## Image Generation Tools` section (near the top, after `## User Input Tools` and before the main workflow) that **inlines** this rule. Skills are self-contained and cannot link to `docs/` — each skill folder must ship the rule inside its own `SKILL.md`. See [CLAUDE.md → Skill Self-Containment](../CLAUDE.md).
Each skill's `references/config/preferences-schema.md` (and its `EXTEND.md` template in `first-time-setup.md`) lists `preferred_image_backend` alongside other preference fields. First-time setup does NOT ask the user about the backend — `auto` is set silently. Users who want to pin a specific backend edit `EXTEND.md` later, and each skill's `## Changing Preferences` section documents the common one-line edits.
Concrete tool names (`imagegen`, `GenerateImage`, `image_generate`, `baoyu-image-gen`) in this document and in SKILL.md are **examples** — agents in other runtimes apply the rule above and substitute the local equivalent. Skill-specific parameters for these backends are illustrative; runtimes without those knobs can omit them.
## Backend Skills Are Exempt
Skills that **are themselves** image-generation backends — currently `baoyu-image-gen`, `baoyu-image-gen` (deprecated), and `baoyu-danger-gemini-web` — do NOT include a `## Image Generation Tools` section. They render directly via their own provider integrations and have no need to "select a backend." The rule applies only to consumer skills that delegate rendering to whatever backend the runtime exposes.
+54
View File
@@ -0,0 +1,54 @@
# Image Generation Guidelines
Skills that require image generation MUST delegate to available image generation tools (runtime-native tools or installed skills).
**Backend selection convention**: see [image-generation-tools.md](image-generation-tools.md) for the runtime-neutral rule. Short version: use whatever backend is available; if multiple, ask the user once; if none, ask how to proceed. This document covers output conventions (naming, paths) that apply regardless of which backend is selected.
## Skill Selection
1. Follow the rule in [image-generation-tools.md](image-generation-tools.md): use whatever backend is available; ask only on ambiguity.
2. Read the chosen backend's documentation for parameters and capabilities.
3. If user requests a specific backend, honor it.
## Generation Flow Template
```markdown
### Step N: Generate Images
**Backend Selection**:
1. Detect available image-generation tools/skills (runtime-native + installed)
2. If one available → use it. If multiple → ask user once. If none → ask how to proceed.
3. Read the chosen backend's docs for parameters
**Generation Flow**:
1. Write the full prompt to `prompts/NN-{type}-[slug].md` BEFORE invoking the backend
2. Call backend with the prompt (or prompt file), output path, and parameters
3. Generate sequentially by default (batch parallel only when backend supports it and user has multiple prompts)
4. Output progress: "Generated X/N"
5. On failure, auto-retry once before reporting error
```
**Batch Parallel** (`baoyu-image-gen` only): concurrent workers with per-provider throttling via `batch.max_workers` in EXTEND.md.
## Output Path Convention
**Output Directory**: `<skill-suffix>/<topic-slug>/`
- `<skill-suffix>`: e.g., `xhs-images`, `cover-image`, `slide-deck`, `comic`
- `<topic-slug>`: 2-4 words, kebab-case from content topic
- Conflict: append timestamp `<topic-slug>-YYYYMMDD-HHMMSS`
**Source Files**: Copy to output dir as `source-{slug}.{ext}`
## Image Naming Convention
**Format**: `NN-{type}-[slug].png`
- `NN`: Two-digit sequence (01, 02, ...)
- `{type}`: cover, content, page, slide, illustration, etc.
- `[slug]`: 2-5 word kebab-case descriptor, unique within directory
Examples:
```
01-cover-ai-future.png
02-content-key-benefits.png
03-slide-architecture-overview.png
```
+48
View File
@@ -0,0 +1,48 @@
# ClawHub / OpenClaw Publishing
## OpenClaw Metadata
Skills include `metadata.openclaw` in YAML front matter:
```yaml
metadata:
openclaw:
homepage: https://github.com/JimLiu/baoyu-skills#<skill-name>
requires: # only for skills with scripts
anyBins:
- bun
- npx
```
## Publishing Commands
```bash
bash scripts/sync-clawhub.sh # sync all skills
bash scripts/sync-clawhub.sh <skill> # sync one skill
```
Release hooks are configured via `.releaserc.yml`. This repo does not stage a separate release directory: publish reads the skill directory directly and validates that local package references and CLI bin targets are self-contained.
Every skill release must keep the `version:` in that skill's `SKILL.md` aligned with the version being published. `publish-skill.mjs` and `sync-clawhub.mjs` both reject mismatches so a registry payload cannot ship with stale skill metadata.
Commits that touch `skills/<name>/**` must use Conventional Commit subjects, for example `fix(baoyu-post-to-wechat): handle WeChat editor focus`. CI runs `npm run verify:skill-release-commits` against the pushed or PR commit range so bare subjects like `Fix WeChat browser article publishing` cannot bypass per-skill release versioning silently.
## Shared Workspace Packages
`packages/` is the source of truth for shared runtime code. Most skills consume shared packages from npm with semver ranges. `baoyu-url-to-markdown` is the exception: it vendors the `baoyu-fetch` runtime into `skills/baoyu-url-to-markdown/scripts/lib/` so the published skill is self-contained and does not depend on the `baoyu-fetch` npm package.
Current packages:
- `baoyu-chrome-cdp` (Chrome CDP utilities), consumed by 5 skills (`baoyu-danger-gemini-web`, `baoyu-danger-x-to-markdown`, `baoyu-post-to-wechat`, `baoyu-post-to-weibo`, `baoyu-post-to-x`)
- `baoyu-md` (shared Markdown rendering and placeholder pipeline), consumed by 3 skills (`baoyu-markdown-to-html`, `baoyu-post-to-wechat`, `baoyu-post-to-weibo`)
- `baoyu-fetch` (URL-to-Markdown CLI), vendored into 1 skill (`baoyu-url-to-markdown`)
**How it works**: npm packages are built from `packages/` and published to the public npm registry. Skills normally depend on those packages with `^<version>` specs. Release prep runs `node scripts/verify-shared-package-deps.mjs` so accidental `file:` dependencies cannot slip back in. For vendored skill runtimes, keep the copied code under the skill directory and run `node scripts/publish-skill.mjs --skill-dir <skill> --version <version> --dry-run` before publishing.
**Update workflow**:
1. Edit package under `packages/`
2. Run the package build, e.g. `bun run --cwd packages/baoyu-md build`
3. Publish the changed npm package with `npm publish --access public`
4. Update consuming skill `package.json` semver ranges if the package version changed
5. Run `node scripts/verify-shared-package-deps.mjs`
**Git hook**: Run `node scripts/install-git-hooks.mjs` once to enable the `pre-push` hook. It blocks pushes when a skill uses a local `file:` dependency or a vendored workspace package.
+73
View File
@@ -0,0 +1,73 @@
# Testing Strategy
This repository has many scripts, but they do not share a single runtime or dependency graph. The lowest-risk testing strategy is to start from stable Node-based library code, then expand outward to CLI and skill-specific smoke tests.
## Current Baseline
- Root test runner: `node:test`
- Entry point: `npm test`
- Coverage command: `npm run test:coverage`
- CI trigger: GitHub Actions on `push`, `pull_request`, and manual dispatch
This avoids introducing Jest/Vitest across a repo that already mixes plain Node scripts, Bun-based skill packages, npm-published shared packages, and browser automation.
## Rollout Plan
### Phase 1: Stable library coverage
Focus on pure functions under `scripts/lib/` first.
- `scripts/lib/release-files.mjs`
- `scripts/verify-shared-package-deps.mjs`
Goals:
- Validate file filtering and release packaging rules
- Catch regressions that reintroduce local `file:` dependencies or vendored workspace packages
- Keep tests deterministic and free of network, Bun, or browser requirements
### Phase 2: Root CLI integration tests
Add temp-directory integration tests for root CLIs that already support dry-run or local-only flows.
- `scripts/verify-shared-package-deps.mjs`
- `scripts/publish-skill.mjs --dry-run`
- `scripts/sync-clawhub.mjs` argument handling and local skill discovery
Goals:
- Assert exit codes and stdout for common flows
- Cover CLI argument parsing without hitting external services
### Phase 3: Skill script smoke tests
Add opt-in smoke tests for selected `skills/*/scripts/` packages, starting with those that:
- accept local input files
- have deterministic output
- do not require authenticated browser sessions
Examples:
- markdown transforms
- file conversion helpers
- local content analyzers
Keep browser automation, login flows, and live API publishing scripts outside the default CI path unless they are explicitly mocked.
### Phase 4: Coverage gates
After the stable Node path has enough breadth, add coverage thresholds in CI for the tested root modules.
Recommended order:
1. Start with reporting only
2. Add line/function thresholds for `scripts/lib/**`
3. Expand include patterns once skill-level smoke tests are reliable
## Conventions For New Tests
- Prefer temp directories over committed fixtures unless the fixture is reused heavily
- Test exported functions before testing CLI wrappers
- Avoid network, browser, and credential dependencies in default CI
- Keep tests isolated so they can run with plain `node --test`
+17
View File
@@ -0,0 +1,17 @@
# User Input Tools
Skills in this repo are loaded by multiple agent runtimes (Claude Code, other agents, bare CLI). Each runtime exposes a different API for asking the user questions. This document defines the canonical **tool-selection rule** every skill follows so skills stay portable.
## Tool Selection (priority order)
1. **Prefer built-in user-input tools** if the current agent runtime exposes one — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.
2. **Fallback to plain text**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
3. **Batching rule**:
- If the tool supports **multiple questions per call** (e.g., `AskUserQuestion`): **Combine all applicable questions into a single call. Do NOT split into separate calls.**
- If the tool supports **only one question per call** (e.g., single-prompt `clarify`): ask **one question per call, in priority order**.
## How Skills Declare This
Each `SKILL.md` that uses interactive user input includes **exactly one** `## User Input Tools` section (typically near the top, right after the intro) that **inlines** this rule. Do NOT link here from a SKILL.md — skills are self-contained (see [CLAUDE.md → Skill Self-Containment](../CLAUDE.md)). This document is the author-side canonical source; copy its body into each SKILL.md. The rule then governs every user-input interaction in that skill and its `references/` files.
Specific mentions of a concrete tool (e.g., `AskUserQuestion`) elsewhere in a skill are **concrete examples** — agents in other runtimes apply the rule above and substitute the local equivalent. Tool-specific parameters (e.g., `header:`, `multiSelect:`) are illustrative; runtimes without those knobs can omit them.
+5081
View File
File diff suppressed because it is too large Load Diff
+28
View File
@@ -0,0 +1,28 @@
{
"name": "baoyu-skills",
"private": true,
"type": "module",
"workspaces": [
"packages/*"
],
"scripts": {
"test": "node ./scripts/run-node-tests.mjs",
"test:coverage": "node ./scripts/run-node-tests.mjs --experimental-test-coverage",
"verify:skill-release-commits": "node ./scripts/verify-skill-release-commits.mjs"
},
"devDependencies": {
"@mozilla/readability": "^0.6.0",
"linkedom": "^0.18.12",
"turndown": "^7.2.2",
"turndown-plugin-gfm": "^1.0.2",
"tsx": "^4.20.5"
},
"dependencies": {
"pdf-lib": "^1.17.1",
"pptxgenjs": "^4.0.1",
"sharp": "^0.34.5"
},
"overrides": {
"@xmldom/xmldom": "0.8.13"
}
}
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+479
View File
@@ -0,0 +1,479 @@
var __create = Object.create;
var __getProtoOf = Object.getPrototypeOf;
var __defProp = Object.defineProperty;
var __getOwnPropNames = Object.getOwnPropertyNames;
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
var __hasOwnProp = Object.prototype.hasOwnProperty;
function __accessProp(key) {
return this[key];
}
var __toESMCache_node;
var __toESMCache_esm;
var __toESM = (mod, isNodeMode, target) => {
var canCache = mod != null && typeof mod === "object";
if (canCache) {
var cache = isNodeMode ? __toESMCache_node ??= new WeakMap : __toESMCache_esm ??= new WeakMap;
var cached = cache.get(mod);
if (cached)
return cached;
}
target = mod != null ? __create(__getProtoOf(mod)) : {};
const to = isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target;
for (let key of __getOwnPropNames(mod))
if (!__hasOwnProp.call(to, key))
__defProp(to, key, {
get: __accessProp.bind(mod, key),
enumerable: true
});
if (canCache)
cache.set(mod, to);
return to;
};
var __toCommonJS = (from) => {
var entry = (__moduleCache ??= new WeakMap).get(from), desc;
if (entry)
return entry;
entry = __defProp({}, "__esModule", { value: true });
if (from && typeof from === "object" || typeof from === "function") {
for (var key of __getOwnPropNames(from))
if (!__hasOwnProp.call(entry, key))
__defProp(entry, key, {
get: __accessProp.bind(from, key),
enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable
});
}
__moduleCache.set(from, entry);
return entry;
};
var __moduleCache;
var __returnValue = (v) => v;
function __exportSetter(name, newValue) {
this[name] = __returnValue.bind(null, newValue);
}
var __export = (target, all) => {
for (var name in all)
__defProp(target, name, {
get: all[name],
enumerable: true,
configurable: true,
set: __exportSetter.bind(all, name)
});
};
// src/index.ts
var exports_src = {};
__export(exports_src, {
waitForChromeDebugPort: () => waitForChromeDebugPort,
sleep: () => sleep,
resolveSharedChromeProfileDir: () => resolveSharedChromeProfileDir,
openPageSession: () => openPageSession,
launchChrome: () => launchChrome,
killChrome: () => killChrome,
gracefulKillChrome: () => gracefulKillChrome,
getFreePort: () => getFreePort,
getDefaultChromeUserDataDirs: () => getDefaultChromeUserDataDirs,
findExistingChromeDebugPort: () => findExistingChromeDebugPort,
findChromeExecutable: () => findChromeExecutable,
discoverRunningChromeDebugPort: () => discoverRunningChromeDebugPort,
CdpConnection: () => CdpConnection
});
module.exports = __toCommonJS(exports_src);
var import_node_child_process = require("node:child_process");
var import_node_fs = __toESM(require("node:fs"));
var import_node_net = __toESM(require("node:net"));
var import_node_os = __toESM(require("node:os"));
var import_node_path = __toESM(require("node:path"));
var import_node_process = __toESM(require("node:process"));
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function getFreePort(fixedEnvName) {
const fixed = fixedEnvName ? Number.parseInt(import_node_process.default.env[fixedEnvName] ?? "", 10) : NaN;
if (Number.isInteger(fixed) && fixed > 0)
return fixed;
return await new Promise((resolve, reject) => {
const server = import_node_net.default.createServer();
server.unref();
server.on("error", reject);
server.listen(0, "127.0.0.1", () => {
const address = server.address();
if (!address || typeof address === "string") {
server.close(() => reject(new Error("Unable to allocate a free TCP port.")));
return;
}
const port = address.port;
server.close((err) => {
if (err)
reject(err);
else
resolve(port);
});
});
});
}
function findChromeExecutable(options) {
for (const envName of options.envNames ?? []) {
const override = import_node_process.default.env[envName]?.trim();
if (override && import_node_fs.default.existsSync(override))
return override;
}
const candidates = import_node_process.default.platform === "darwin" ? options.candidates.darwin ?? options.candidates.default : import_node_process.default.platform === "win32" ? options.candidates.win32 ?? options.candidates.default : options.candidates.default;
for (const candidate of candidates) {
if (import_node_fs.default.existsSync(candidate))
return candidate;
}
return;
}
function resolveSharedChromeProfileDir(options = {}) {
for (const envName of options.envNames ?? []) {
const override = import_node_process.default.env[envName]?.trim();
if (override)
return import_node_path.default.resolve(override);
}
const appDataDirName = options.appDataDirName ?? "baoyu-skills";
const profileDirName = options.profileDirName ?? "chrome-profile";
if (options.wslWindowsHome) {
return import_node_path.default.join(options.wslWindowsHome, ".local", "share", appDataDirName, profileDirName);
}
const base = import_node_process.default.platform === "darwin" ? import_node_path.default.join(import_node_os.default.homedir(), "Library", "Application Support") : import_node_process.default.platform === "win32" ? import_node_process.default.env.APPDATA ?? import_node_path.default.join(import_node_os.default.homedir(), "AppData", "Roaming") : import_node_process.default.env.XDG_DATA_HOME ?? import_node_path.default.join(import_node_os.default.homedir(), ".local", "share");
return import_node_path.default.join(base, appDataDirName, profileDirName);
}
async function fetchWithTimeout(url, timeoutMs) {
if (!timeoutMs || timeoutMs <= 0)
return await fetch(url, { redirect: "follow" });
const ctl = new AbortController;
const timer = setTimeout(() => ctl.abort(), timeoutMs);
try {
return await fetch(url, { redirect: "follow", signal: ctl.signal });
} finally {
clearTimeout(timer);
}
}
async function fetchJson(url, options = {}) {
const response = await fetchWithTimeout(url, options.timeoutMs);
if (!response.ok) {
throw new Error(`Request failed: ${response.status} ${response.statusText}`);
}
return await response.json();
}
async function isDebugPortReady(port, timeoutMs = 3000) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
return !!version.webSocketDebuggerUrl;
} catch {
return false;
}
}
function isPortListening(port, timeoutMs = 3000) {
return new Promise((resolve) => {
const socket = new import_node_net.default.Socket;
const timer = setTimeout(() => {
socket.destroy();
resolve(false);
}, timeoutMs);
socket.once("connect", () => {
clearTimeout(timer);
socket.destroy();
resolve(true);
});
socket.once("error", () => {
clearTimeout(timer);
resolve(false);
});
socket.connect(port, "127.0.0.1");
});
}
function parseDevToolsActivePort(filePath) {
try {
const content = import_node_fs.default.readFileSync(filePath, "utf-8");
const lines = content.split(/\r?\n/);
const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
const wsPath = lines[1]?.trim();
if (port > 0 && wsPath)
return { port, wsPath };
} catch {}
return null;
}
async function findExistingChromeDebugPort(options) {
const timeoutMs = options.timeoutMs ?? 3000;
const parsed = parseDevToolsActivePort(import_node_path.default.join(options.profileDir, "DevToolsActivePort"));
if (parsed && parsed.port > 0 && await isDebugPortReady(parsed.port, timeoutMs))
return parsed.port;
if (import_node_process.default.platform === "win32")
return null;
try {
const result = import_node_child_process.spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
if (result.status !== 0 || !result.stdout)
return null;
const lines = result.stdout.split(`
`).filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
for (const line of lines) {
const portMatch = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(portMatch?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs))
return port;
}
} catch {}
return null;
}
function getDefaultChromeUserDataDirs(channels = ["stable"]) {
const home = import_node_os.default.homedir();
const dirs = [];
const channelDirs = {
stable: {
darwin: import_node_path.default.join(home, "Library", "Application Support", "Google", "Chrome"),
linux: import_node_path.default.join(home, ".config", "google-chrome"),
win32: import_node_path.default.join(import_node_process.default.env.LOCALAPPDATA ?? import_node_path.default.join(home, "AppData", "Local"), "Google", "Chrome", "User Data")
},
beta: {
darwin: import_node_path.default.join(home, "Library", "Application Support", "Google", "Chrome Beta"),
linux: import_node_path.default.join(home, ".config", "google-chrome-beta"),
win32: import_node_path.default.join(import_node_process.default.env.LOCALAPPDATA ?? import_node_path.default.join(home, "AppData", "Local"), "Google", "Chrome Beta", "User Data")
},
canary: {
darwin: import_node_path.default.join(home, "Library", "Application Support", "Google", "Chrome Canary"),
linux: import_node_path.default.join(home, ".config", "google-chrome-canary"),
win32: import_node_path.default.join(import_node_process.default.env.LOCALAPPDATA ?? import_node_path.default.join(home, "AppData", "Local"), "Google", "Chrome SxS", "User Data")
},
dev: {
darwin: import_node_path.default.join(home, "Library", "Application Support", "Google", "Chrome Dev"),
linux: import_node_path.default.join(home, ".config", "google-chrome-dev"),
win32: import_node_path.default.join(import_node_process.default.env.LOCALAPPDATA ?? import_node_path.default.join(home, "AppData", "Local"), "Google", "Chrome Dev", "User Data")
}
};
const platform = import_node_process.default.platform === "darwin" ? "darwin" : import_node_process.default.platform === "win32" ? "win32" : "linux";
for (const ch of channels) {
const entry = channelDirs[ch];
if (entry)
dirs.push(entry[platform]);
}
return dirs;
}
async function discoverRunningChromeDebugPort(options = {}) {
const channels = options.channels ?? ["stable", "beta", "canary", "dev"];
const timeoutMs = options.timeoutMs ?? 3000;
const userDataDirs = (options.userDataDirs ?? getDefaultChromeUserDataDirs(channels)).map((dir) => import_node_path.default.resolve(dir));
for (const dir of userDataDirs) {
const parsed = parseDevToolsActivePort(import_node_path.default.join(dir, "DevToolsActivePort"));
if (!parsed)
continue;
if (await isPortListening(parsed.port, timeoutMs)) {
return { port: parsed.port, wsUrl: `ws://127.0.0.1:${parsed.port}${parsed.wsPath}` };
}
}
if (import_node_process.default.platform !== "win32") {
try {
const result = import_node_child_process.spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
if (result.status === 0 && result.stdout) {
const lines = result.stdout.split(`
`).filter((line) => line.includes("--remote-debugging-port=") && userDataDirs.some((dir) => line.includes(dir)));
for (const line of lines) {
const portMatch = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(portMatch?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs)) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
if (version.webSocketDebuggerUrl)
return { port, wsUrl: version.webSocketDebuggerUrl };
} catch {}
}
}
}
} catch {}
}
return null;
}
async function waitForChromeDebugPort(port, timeoutMs, options) {
const start = Date.now();
let lastError = null;
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs: 5000 });
if (version.webSocketDebuggerUrl)
return version.webSocketDebuggerUrl;
lastError = new Error("Missing webSocketDebuggerUrl");
} catch (error) {
lastError = error;
}
await sleep(200);
}
if (options?.includeLastError && lastError) {
throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
}
throw new Error("Chrome debug port not ready");
}
class CdpConnection {
ws;
nextId = 0;
pending = new Map;
eventHandlers = new Map;
defaultTimeoutMs;
constructor(ws, defaultTimeoutMs = 15000) {
this.ws = ws;
this.defaultTimeoutMs = defaultTimeoutMs;
this.ws.addEventListener("message", (event) => {
try {
const data = typeof event.data === "string" ? event.data : new TextDecoder().decode(event.data);
const msg = JSON.parse(data);
if (msg.method) {
const handlers = this.eventHandlers.get(msg.method);
if (handlers) {
handlers.forEach((handler) => handler(msg.params));
}
}
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer)
clearTimeout(pending.timer);
if (msg.error?.message)
pending.reject(new Error(msg.error.message));
else
pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener("close", () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer)
clearTimeout(pending.timer);
pending.reject(new Error("CDP connection closed."));
}
});
}
static async connect(url, timeoutMs, options) {
const ws = new WebSocket(url);
await new Promise((resolve, reject) => {
const timer = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
ws.addEventListener("open", () => {
clearTimeout(timer);
resolve();
});
ws.addEventListener("error", () => {
clearTimeout(timer);
reject(new Error("CDP connection failed."));
});
});
return new CdpConnection(ws, options?.defaultTimeoutMs ?? 15000);
}
on(method, handler) {
if (!this.eventHandlers.has(method)) {
this.eventHandlers.set(method, new Set);
}
this.eventHandlers.get(method)?.add(handler);
}
off(method, handler) {
this.eventHandlers.get(method)?.delete(handler);
}
async send(method, params, options) {
const id = ++this.nextId;
const message = { id, method };
if (params)
message.params = params;
if (options?.sessionId)
message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
const result = await new Promise((resolve, reject) => {
const timer = timeoutMs > 0 ? setTimeout(() => {
this.pending.delete(id);
reject(new Error(`CDP timeout: ${method}`));
}, timeoutMs) : null;
this.pending.set(id, { resolve, reject, timer });
this.ws.send(JSON.stringify(message));
});
return result;
}
close() {
try {
this.ws.close();
} catch {}
}
}
async function launchChrome(options) {
await import_node_fs.default.promises.mkdir(options.profileDir, { recursive: true });
const args = [
`--remote-debugging-port=${options.port}`,
`--user-data-dir=${options.profileDir}`,
"--no-first-run",
"--no-default-browser-check",
...options.extraArgs ?? []
];
if (options.headless)
args.push("--headless=new");
if (options.url)
args.push(options.url);
return import_node_child_process.spawn(options.chromePath, args, { stdio: "ignore" });
}
function killChrome(chrome) {
try {
chrome.kill("SIGTERM");
} catch {}
setTimeout(() => {
if (chrome.exitCode === null && chrome.signalCode === null) {
try {
chrome.kill("SIGKILL");
} catch {}
}
}, 2000).unref?.();
}
async function gracefulKillChrome(chrome, port, timeoutMs = 6000) {
if (chrome.exitCode !== null || chrome.signalCode !== null)
return;
const exitPromise = new Promise((resolve) => {
chrome.once("exit", () => resolve());
});
killChrome(chrome);
const start = Date.now();
while (Date.now() - start < timeoutMs) {
if (chrome.exitCode !== null || chrome.signalCode !== null)
return;
if (port !== undefined && !await isPortListening(port, 250))
return;
const exited = await Promise.race([
exitPromise.then(() => true),
sleep(100).then(() => false)
]);
if (exited)
return;
}
await Promise.race([
exitPromise,
sleep(250)
]);
}
async function openPageSession(options) {
let targetId;
let createdTarget = false;
if (options.reusing) {
const created = await options.cdp.send("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
} else {
const targets = await options.cdp.send("Target.getTargets");
const existing = targets.targetInfos.find(options.matchTarget);
if (existing) {
targetId = existing.targetId;
} else {
const created = await options.cdp.send("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
}
}
const { sessionId } = await options.cdp.send("Target.attachToTarget", { targetId, flatten: true });
if (options.activateTarget ?? true) {
await options.cdp.send("Target.activateTarget", { targetId });
}
if (options.enablePage)
await options.cdp.send("Page.enable", {}, { sessionId });
if (options.enableRuntime)
await options.cdp.send("Runtime.enable", {}, { sessionId });
if (options.enableDom)
await options.cdp.send("DOM.enable", {}, { sessionId });
if (options.enableNetwork)
await options.cdp.send("Network.enable", {}, { sessionId });
return { sessionId, targetId, createdTarget };
}
+415
View File
@@ -0,0 +1,415 @@
// src/index.ts
import { spawn, spawnSync } from "node:child_process";
import fs from "node:fs";
import net from "node:net";
import os from "node:os";
import path from "node:path";
import process from "node:process";
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function getFreePort(fixedEnvName) {
const fixed = fixedEnvName ? Number.parseInt(process.env[fixedEnvName] ?? "", 10) : NaN;
if (Number.isInteger(fixed) && fixed > 0)
return fixed;
return await new Promise((resolve, reject) => {
const server = net.createServer();
server.unref();
server.on("error", reject);
server.listen(0, "127.0.0.1", () => {
const address = server.address();
if (!address || typeof address === "string") {
server.close(() => reject(new Error("Unable to allocate a free TCP port.")));
return;
}
const port = address.port;
server.close((err) => {
if (err)
reject(err);
else
resolve(port);
});
});
});
}
function findChromeExecutable(options) {
for (const envName of options.envNames ?? []) {
const override = process.env[envName]?.trim();
if (override && fs.existsSync(override))
return override;
}
const candidates = process.platform === "darwin" ? options.candidates.darwin ?? options.candidates.default : process.platform === "win32" ? options.candidates.win32 ?? options.candidates.default : options.candidates.default;
for (const candidate of candidates) {
if (fs.existsSync(candidate))
return candidate;
}
return;
}
function resolveSharedChromeProfileDir(options = {}) {
for (const envName of options.envNames ?? []) {
const override = process.env[envName]?.trim();
if (override)
return path.resolve(override);
}
const appDataDirName = options.appDataDirName ?? "baoyu-skills";
const profileDirName = options.profileDirName ?? "chrome-profile";
if (options.wslWindowsHome) {
return path.join(options.wslWindowsHome, ".local", "share", appDataDirName, profileDirName);
}
const base = process.platform === "darwin" ? path.join(os.homedir(), "Library", "Application Support") : process.platform === "win32" ? process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming") : process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share");
return path.join(base, appDataDirName, profileDirName);
}
async function fetchWithTimeout(url, timeoutMs) {
if (!timeoutMs || timeoutMs <= 0)
return await fetch(url, { redirect: "follow" });
const ctl = new AbortController;
const timer = setTimeout(() => ctl.abort(), timeoutMs);
try {
return await fetch(url, { redirect: "follow", signal: ctl.signal });
} finally {
clearTimeout(timer);
}
}
async function fetchJson(url, options = {}) {
const response = await fetchWithTimeout(url, options.timeoutMs);
if (!response.ok) {
throw new Error(`Request failed: ${response.status} ${response.statusText}`);
}
return await response.json();
}
async function isDebugPortReady(port, timeoutMs = 3000) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
return !!version.webSocketDebuggerUrl;
} catch {
return false;
}
}
function isPortListening(port, timeoutMs = 3000) {
return new Promise((resolve) => {
const socket = new net.Socket;
const timer = setTimeout(() => {
socket.destroy();
resolve(false);
}, timeoutMs);
socket.once("connect", () => {
clearTimeout(timer);
socket.destroy();
resolve(true);
});
socket.once("error", () => {
clearTimeout(timer);
resolve(false);
});
socket.connect(port, "127.0.0.1");
});
}
function parseDevToolsActivePort(filePath) {
try {
const content = fs.readFileSync(filePath, "utf-8");
const lines = content.split(/\r?\n/);
const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
const wsPath = lines[1]?.trim();
if (port > 0 && wsPath)
return { port, wsPath };
} catch {}
return null;
}
async function findExistingChromeDebugPort(options) {
const timeoutMs = options.timeoutMs ?? 3000;
const parsed = parseDevToolsActivePort(path.join(options.profileDir, "DevToolsActivePort"));
if (parsed && parsed.port > 0 && await isDebugPortReady(parsed.port, timeoutMs))
return parsed.port;
if (process.platform === "win32")
return null;
try {
const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
if (result.status !== 0 || !result.stdout)
return null;
const lines = result.stdout.split(`
`).filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
for (const line of lines) {
const portMatch = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(portMatch?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs))
return port;
}
} catch {}
return null;
}
function getDefaultChromeUserDataDirs(channels = ["stable"]) {
const home = os.homedir();
const dirs = [];
const channelDirs = {
stable: {
darwin: path.join(home, "Library", "Application Support", "Google", "Chrome"),
linux: path.join(home, ".config", "google-chrome"),
win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome", "User Data")
},
beta: {
darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Beta"),
linux: path.join(home, ".config", "google-chrome-beta"),
win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome Beta", "User Data")
},
canary: {
darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Canary"),
linux: path.join(home, ".config", "google-chrome-canary"),
win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome SxS", "User Data")
},
dev: {
darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Dev"),
linux: path.join(home, ".config", "google-chrome-dev"),
win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome Dev", "User Data")
}
};
const platform = process.platform === "darwin" ? "darwin" : process.platform === "win32" ? "win32" : "linux";
for (const ch of channels) {
const entry = channelDirs[ch];
if (entry)
dirs.push(entry[platform]);
}
return dirs;
}
async function discoverRunningChromeDebugPort(options = {}) {
const channels = options.channels ?? ["stable", "beta", "canary", "dev"];
const timeoutMs = options.timeoutMs ?? 3000;
const userDataDirs = (options.userDataDirs ?? getDefaultChromeUserDataDirs(channels)).map((dir) => path.resolve(dir));
for (const dir of userDataDirs) {
const parsed = parseDevToolsActivePort(path.join(dir, "DevToolsActivePort"));
if (!parsed)
continue;
if (await isPortListening(parsed.port, timeoutMs)) {
return { port: parsed.port, wsUrl: `ws://127.0.0.1:${parsed.port}${parsed.wsPath}` };
}
}
if (process.platform !== "win32") {
try {
const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
if (result.status === 0 && result.stdout) {
const lines = result.stdout.split(`
`).filter((line) => line.includes("--remote-debugging-port=") && userDataDirs.some((dir) => line.includes(dir)));
for (const line of lines) {
const portMatch = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(portMatch?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs)) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
if (version.webSocketDebuggerUrl)
return { port, wsUrl: version.webSocketDebuggerUrl };
} catch {}
}
}
}
} catch {}
}
return null;
}
async function waitForChromeDebugPort(port, timeoutMs, options) {
const start = Date.now();
let lastError = null;
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs: 5000 });
if (version.webSocketDebuggerUrl)
return version.webSocketDebuggerUrl;
lastError = new Error("Missing webSocketDebuggerUrl");
} catch (error) {
lastError = error;
}
await sleep(200);
}
if (options?.includeLastError && lastError) {
throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
}
throw new Error("Chrome debug port not ready");
}
class CdpConnection {
ws;
nextId = 0;
pending = new Map;
eventHandlers = new Map;
defaultTimeoutMs;
constructor(ws, defaultTimeoutMs = 15000) {
this.ws = ws;
this.defaultTimeoutMs = defaultTimeoutMs;
this.ws.addEventListener("message", (event) => {
try {
const data = typeof event.data === "string" ? event.data : new TextDecoder().decode(event.data);
const msg = JSON.parse(data);
if (msg.method) {
const handlers = this.eventHandlers.get(msg.method);
if (handlers) {
handlers.forEach((handler) => handler(msg.params));
}
}
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer)
clearTimeout(pending.timer);
if (msg.error?.message)
pending.reject(new Error(msg.error.message));
else
pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener("close", () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer)
clearTimeout(pending.timer);
pending.reject(new Error("CDP connection closed."));
}
});
}
static async connect(url, timeoutMs, options) {
const ws = new WebSocket(url);
await new Promise((resolve, reject) => {
const timer = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
ws.addEventListener("open", () => {
clearTimeout(timer);
resolve();
});
ws.addEventListener("error", () => {
clearTimeout(timer);
reject(new Error("CDP connection failed."));
});
});
return new CdpConnection(ws, options?.defaultTimeoutMs ?? 15000);
}
on(method, handler) {
if (!this.eventHandlers.has(method)) {
this.eventHandlers.set(method, new Set);
}
this.eventHandlers.get(method)?.add(handler);
}
off(method, handler) {
this.eventHandlers.get(method)?.delete(handler);
}
async send(method, params, options) {
const id = ++this.nextId;
const message = { id, method };
if (params)
message.params = params;
if (options?.sessionId)
message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
const result = await new Promise((resolve, reject) => {
const timer = timeoutMs > 0 ? setTimeout(() => {
this.pending.delete(id);
reject(new Error(`CDP timeout: ${method}`));
}, timeoutMs) : null;
this.pending.set(id, { resolve, reject, timer });
this.ws.send(JSON.stringify(message));
});
return result;
}
close() {
try {
this.ws.close();
} catch {}
}
}
async function launchChrome(options) {
await fs.promises.mkdir(options.profileDir, { recursive: true });
const args = [
`--remote-debugging-port=${options.port}`,
`--user-data-dir=${options.profileDir}`,
"--no-first-run",
"--no-default-browser-check",
...options.extraArgs ?? []
];
if (options.headless)
args.push("--headless=new");
if (options.url)
args.push(options.url);
return spawn(options.chromePath, args, { stdio: "ignore" });
}
function killChrome(chrome) {
try {
chrome.kill("SIGTERM");
} catch {}
setTimeout(() => {
if (chrome.exitCode === null && chrome.signalCode === null) {
try {
chrome.kill("SIGKILL");
} catch {}
}
}, 2000).unref?.();
}
async function gracefulKillChrome(chrome, port, timeoutMs = 6000) {
if (chrome.exitCode !== null || chrome.signalCode !== null)
return;
const exitPromise = new Promise((resolve) => {
chrome.once("exit", () => resolve());
});
killChrome(chrome);
const start = Date.now();
while (Date.now() - start < timeoutMs) {
if (chrome.exitCode !== null || chrome.signalCode !== null)
return;
if (port !== undefined && !await isPortListening(port, 250))
return;
const exited = await Promise.race([
exitPromise.then(() => true),
sleep(100).then(() => false)
]);
if (exited)
return;
}
await Promise.race([
exitPromise,
sleep(250)
]);
}
async function openPageSession(options) {
let targetId;
let createdTarget = false;
if (options.reusing) {
const created = await options.cdp.send("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
} else {
const targets = await options.cdp.send("Target.getTargets");
const existing = targets.targetInfos.find(options.matchTarget);
if (existing) {
targetId = existing.targetId;
} else {
const created = await options.cdp.send("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
}
}
const { sessionId } = await options.cdp.send("Target.attachToTarget", { targetId, flatten: true });
if (options.activateTarget ?? true) {
await options.cdp.send("Target.activateTarget", { targetId });
}
if (options.enablePage)
await options.cdp.send("Page.enable", {}, { sessionId });
if (options.enableRuntime)
await options.cdp.send("Runtime.enable", {}, { sessionId });
if (options.enableDom)
await options.cdp.send("DOM.enable", {}, { sessionId });
if (options.enableNetwork)
await options.cdp.send("Network.enable", {}, { sessionId });
return { sessionId, targetId, createdTarget };
}
export {
waitForChromeDebugPort,
sleep,
resolveSharedChromeProfileDir,
openPageSession,
launchChrome,
killChrome,
gracefulKillChrome,
getFreePort,
getDefaultChromeUserDataDirs,
findExistingChromeDebugPort,
findChromeExecutable,
discoverRunningChromeDebugPort,
CdpConnection
};
+729
View File
@@ -0,0 +1,729 @@
var __create = Object.create;
var __getProtoOf = Object.getPrototypeOf;
var __defProp = Object.defineProperty;
var __getOwnPropNames = Object.getOwnPropertyNames;
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
var __hasOwnProp = Object.prototype.hasOwnProperty;
function __accessProp(key) {
return this[key];
}
var __toESMCache_node;
var __toESMCache_esm;
var __toESM = (mod, isNodeMode, target) => {
var canCache = mod != null && typeof mod === "object";
if (canCache) {
var cache = isNodeMode ? __toESMCache_node ??= new WeakMap : __toESMCache_esm ??= new WeakMap;
var cached = cache.get(mod);
if (cached)
return cached;
}
target = mod != null ? __create(__getProtoOf(mod)) : {};
const to = isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target;
for (let key of __getOwnPropNames(mod))
if (!__hasOwnProp.call(to, key))
__defProp(to, key, {
get: __accessProp.bind(mod, key),
enumerable: true
});
if (canCache)
cache.set(mod, to);
return to;
};
var __toCommonJS = (from) => {
var entry = (__moduleCache ??= new WeakMap).get(from), desc;
if (entry)
return entry;
entry = __defProp({}, "__esModule", { value: true });
if (from && typeof from === "object" || typeof from === "function") {
for (var key of __getOwnPropNames(from))
if (!__hasOwnProp.call(entry, key))
__defProp(entry, key, {
get: __accessProp.bind(from, key),
enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable
});
}
__moduleCache.set(from, entry);
return entry;
};
var __moduleCache;
var __returnValue = (v) => v;
function __exportSetter(name, newValue) {
this[name] = __returnValue.bind(null, newValue);
}
var __export = (target, all) => {
for (var name in all)
__defProp(target, name, {
get: all[name],
enumerable: true,
configurable: true,
set: __exportSetter.bind(all, name)
});
};
// src/mermaid.ts
var exports_mermaid = {};
__export(exports_mermaid, {
renderMermaidToPng: () => renderMermaidToPng,
closeRenderer: () => closeRenderer,
MermaidRenderError: () => MermaidRenderError
});
module.exports = __toCommonJS(exports_mermaid);
var import_node_child_process2 = require("node:child_process");
var import_node_fs2 = __toESM(require("node:fs"));
var import_node_path2 = __toESM(require("node:path"));
var import_node_process2 = __toESM(require("node:process"));
var import_node_url = require("node:url");
// src/index.ts
var import_node_child_process = require("node:child_process");
var import_node_fs = __toESM(require("node:fs"));
var import_node_net = __toESM(require("node:net"));
var import_node_os = __toESM(require("node:os"));
var import_node_path = __toESM(require("node:path"));
var import_node_process = __toESM(require("node:process"));
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function getFreePort(fixedEnvName) {
const fixed = fixedEnvName ? Number.parseInt(import_node_process.default.env[fixedEnvName] ?? "", 10) : NaN;
if (Number.isInteger(fixed) && fixed > 0)
return fixed;
return await new Promise((resolve, reject) => {
const server = import_node_net.default.createServer();
server.unref();
server.on("error", reject);
server.listen(0, "127.0.0.1", () => {
const address = server.address();
if (!address || typeof address === "string") {
server.close(() => reject(new Error("Unable to allocate a free TCP port.")));
return;
}
const port = address.port;
server.close((err) => {
if (err)
reject(err);
else
resolve(port);
});
});
});
}
function findChromeExecutable(options) {
for (const envName of options.envNames ?? []) {
const override = import_node_process.default.env[envName]?.trim();
if (override && import_node_fs.default.existsSync(override))
return override;
}
const candidates = import_node_process.default.platform === "darwin" ? options.candidates.darwin ?? options.candidates.default : import_node_process.default.platform === "win32" ? options.candidates.win32 ?? options.candidates.default : options.candidates.default;
for (const candidate of candidates) {
if (import_node_fs.default.existsSync(candidate))
return candidate;
}
return;
}
function resolveSharedChromeProfileDir(options = {}) {
for (const envName of options.envNames ?? []) {
const override = import_node_process.default.env[envName]?.trim();
if (override)
return import_node_path.default.resolve(override);
}
const appDataDirName = options.appDataDirName ?? "baoyu-skills";
const profileDirName = options.profileDirName ?? "chrome-profile";
if (options.wslWindowsHome) {
return import_node_path.default.join(options.wslWindowsHome, ".local", "share", appDataDirName, profileDirName);
}
const base = import_node_process.default.platform === "darwin" ? import_node_path.default.join(import_node_os.default.homedir(), "Library", "Application Support") : import_node_process.default.platform === "win32" ? import_node_process.default.env.APPDATA ?? import_node_path.default.join(import_node_os.default.homedir(), "AppData", "Roaming") : import_node_process.default.env.XDG_DATA_HOME ?? import_node_path.default.join(import_node_os.default.homedir(), ".local", "share");
return import_node_path.default.join(base, appDataDirName, profileDirName);
}
async function fetchWithTimeout(url, timeoutMs) {
if (!timeoutMs || timeoutMs <= 0)
return await fetch(url, { redirect: "follow" });
const ctl = new AbortController;
const timer = setTimeout(() => ctl.abort(), timeoutMs);
try {
return await fetch(url, { redirect: "follow", signal: ctl.signal });
} finally {
clearTimeout(timer);
}
}
async function fetchJson(url, options = {}) {
const response = await fetchWithTimeout(url, options.timeoutMs);
if (!response.ok) {
throw new Error(`Request failed: ${response.status} ${response.statusText}`);
}
return await response.json();
}
async function isDebugPortReady(port, timeoutMs = 3000) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
return !!version.webSocketDebuggerUrl;
} catch {
return false;
}
}
function parseDevToolsActivePort(filePath) {
try {
const content = import_node_fs.default.readFileSync(filePath, "utf-8");
const lines = content.split(/\r?\n/);
const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
const wsPath = lines[1]?.trim();
if (port > 0 && wsPath)
return { port, wsPath };
} catch {}
return null;
}
async function findExistingChromeDebugPort(options) {
const timeoutMs = options.timeoutMs ?? 3000;
const parsed = parseDevToolsActivePort(import_node_path.default.join(options.profileDir, "DevToolsActivePort"));
if (parsed && parsed.port > 0 && await isDebugPortReady(parsed.port, timeoutMs))
return parsed.port;
if (import_node_process.default.platform === "win32")
return null;
try {
const result = import_node_child_process.spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
if (result.status !== 0 || !result.stdout)
return null;
const lines = result.stdout.split(`
`).filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
for (const line of lines) {
const portMatch = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(portMatch?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs))
return port;
}
} catch {}
return null;
}
async function waitForChromeDebugPort(port, timeoutMs, options) {
const start = Date.now();
let lastError = null;
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs: 5000 });
if (version.webSocketDebuggerUrl)
return version.webSocketDebuggerUrl;
lastError = new Error("Missing webSocketDebuggerUrl");
} catch (error) {
lastError = error;
}
await sleep(200);
}
if (options?.includeLastError && lastError) {
throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
}
throw new Error("Chrome debug port not ready");
}
class CdpConnection {
ws;
nextId = 0;
pending = new Map;
eventHandlers = new Map;
defaultTimeoutMs;
constructor(ws, defaultTimeoutMs = 15000) {
this.ws = ws;
this.defaultTimeoutMs = defaultTimeoutMs;
this.ws.addEventListener("message", (event) => {
try {
const data = typeof event.data === "string" ? event.data : new TextDecoder().decode(event.data);
const msg = JSON.parse(data);
if (msg.method) {
const handlers = this.eventHandlers.get(msg.method);
if (handlers) {
handlers.forEach((handler) => handler(msg.params));
}
}
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer)
clearTimeout(pending.timer);
if (msg.error?.message)
pending.reject(new Error(msg.error.message));
else
pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener("close", () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer)
clearTimeout(pending.timer);
pending.reject(new Error("CDP connection closed."));
}
});
}
static async connect(url, timeoutMs, options) {
const ws = new WebSocket(url);
await new Promise((resolve, reject) => {
const timer = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
ws.addEventListener("open", () => {
clearTimeout(timer);
resolve();
});
ws.addEventListener("error", () => {
clearTimeout(timer);
reject(new Error("CDP connection failed."));
});
});
return new CdpConnection(ws, options?.defaultTimeoutMs ?? 15000);
}
on(method, handler) {
if (!this.eventHandlers.has(method)) {
this.eventHandlers.set(method, new Set);
}
this.eventHandlers.get(method)?.add(handler);
}
off(method, handler) {
this.eventHandlers.get(method)?.delete(handler);
}
async send(method, params, options) {
const id = ++this.nextId;
const message = { id, method };
if (params)
message.params = params;
if (options?.sessionId)
message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
const result = await new Promise((resolve, reject) => {
const timer = timeoutMs > 0 ? setTimeout(() => {
this.pending.delete(id);
reject(new Error(`CDP timeout: ${method}`));
}, timeoutMs) : null;
this.pending.set(id, { resolve, reject, timer });
this.ws.send(JSON.stringify(message));
});
return result;
}
close() {
try {
this.ws.close();
} catch {}
}
}
async function launchChrome(options) {
await import_node_fs.default.promises.mkdir(options.profileDir, { recursive: true });
const args = [
`--remote-debugging-port=${options.port}`,
`--user-data-dir=${options.profileDir}`,
"--no-first-run",
"--no-default-browser-check",
...options.extraArgs ?? []
];
if (options.headless)
args.push("--headless=new");
if (options.url)
args.push(options.url);
return import_node_child_process.spawn(options.chromePath, args, { stdio: "ignore" });
}
async function openPageSession(options) {
let targetId;
let createdTarget = false;
if (options.reusing) {
const created = await options.cdp.send("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
} else {
const targets = await options.cdp.send("Target.getTargets");
const existing = targets.targetInfos.find(options.matchTarget);
if (existing) {
targetId = existing.targetId;
} else {
const created = await options.cdp.send("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
}
}
const { sessionId } = await options.cdp.send("Target.attachToTarget", { targetId, flatten: true });
if (options.activateTarget ?? true) {
await options.cdp.send("Target.activateTarget", { targetId });
}
if (options.enablePage)
await options.cdp.send("Page.enable", {}, { sessionId });
if (options.enableRuntime)
await options.cdp.send("Runtime.enable", {}, { sessionId });
if (options.enableDom)
await options.cdp.send("DOM.enable", {}, { sessionId });
if (options.enableNetwork)
await options.cdp.send("Network.enable", {}, { sessionId });
return { sessionId, targetId, createdTarget };
}
// src/mermaid.ts
class MermaidRenderError extends Error {
constructor(message, options) {
super(message);
this.name = "MermaidRenderError";
if (options?.cause !== undefined) {
this.cause = options.cause;
}
}
}
function resolveRenderScale(scale) {
const resolved = scale ?? 2;
if (!Number.isFinite(resolved) || resolved <= 0) {
throw new MermaidRenderError(`Invalid Mermaid render scale: ${scale}`);
}
return resolved;
}
function resolveMinWidth(minWidth) {
if (minWidth === undefined)
return;
if (!Number.isFinite(minWidth) || minWidth <= 0) {
throw new MermaidRenderError(`Invalid Mermaid render minWidth: ${minWidth}`);
}
return minWidth;
}
var CHROME_CANDIDATES = {
darwin: [
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary",
"/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"
],
win32: [
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe",
"C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe"
],
default: [
"/usr/bin/google-chrome",
"/usr/bin/google-chrome-stable",
"/usr/bin/chromium",
"/usr/bin/chromium-browser",
"/snap/bin/chromium",
"/usr/bin/microsoft-edge"
]
};
var wslHome;
function getWslWindowsHome() {
if (wslHome !== undefined)
return wslHome;
if (!import_node_process2.default.env.WSL_DISTRO_NAME) {
wslHome = null;
return null;
}
try {
const raw = import_node_child_process2.execSync('cmd.exe /C "echo %USERPROFILE%"', {
encoding: "utf-8",
timeout: 5000
}).trim().replace(/\r/g, "");
wslHome = import_node_child_process2.execSync(`wslpath -u "${raw}"`, {
encoding: "utf-8",
timeout: 5000
}).trim() || null;
} catch {
wslHome = null;
}
return wslHome;
}
function getProfileDir() {
return resolveSharedChromeProfileDir({
envNames: ["BAOYU_CHROME_PROFILE_DIR", "MERMAID_RENDER_PROFILE_DIR"],
wslWindowsHome: getWslWindowsHome()
});
}
function resolveAssetsDir() {
const here = import_node_url.fileURLToPath(undefined);
const dir = import_node_path2.default.dirname(here);
const candidates = [
import_node_path2.default.resolve(dir, "..", "assets"),
import_node_path2.default.resolve(dir, "assets")
];
for (const candidate of candidates) {
if (import_node_fs2.default.existsSync(import_node_path2.default.join(candidate, "mermaid.min.js")))
return candidate;
}
throw new MermaidRenderError(`Cannot locate mermaid.min.js. Looked in: ${candidates.join(", ")}`);
}
var cachedMermaidScript = null;
function loadMermaidScript() {
if (cachedMermaidScript)
return cachedMermaidScript;
const assetsDir = resolveAssetsDir();
cachedMermaidScript = import_node_fs2.default.readFileSync(import_node_path2.default.join(assetsDir, "mermaid.min.js"), "utf-8");
return cachedMermaidScript;
}
var rendererState = null;
var connectingPromise = null;
var exitHookInstalled = false;
function installExitHook() {
if (exitHookInstalled)
return;
exitHookInstalled = true;
const cleanup = () => {
const state = rendererState;
rendererState = null;
if (!state)
return;
try {
state.cdp.close();
} catch {}
if (state.ownsChrome && state.chrome) {
try {
state.chrome.kill("SIGTERM");
} catch {}
}
};
import_node_process2.default.on("exit", cleanup);
import_node_process2.default.on("beforeExit", cleanup);
}
async function tryConnectExisting(port) {
try {
const wsUrl = await waitForChromeDebugPort(port, 5000, { includeLastError: true });
return await CdpConnection.connect(wsUrl, 5000);
} catch {
return null;
}
}
async function ensureRenderer() {
if (rendererState)
return rendererState;
if (connectingPromise)
return await connectingPromise;
connectingPromise = (async () => {
const profileDir = getProfileDir();
const existingPort = await findExistingChromeDebugPort({ profileDir });
if (existingPort) {
const cdp2 = await tryConnectExisting(existingPort);
if (cdp2) {
const state2 = {
cdp: cdp2,
chrome: null,
port: existingPort,
ownsChrome: false
};
rendererState = state2;
installExitHook();
return state2;
}
}
const chromePath = findChromeExecutable({
candidates: CHROME_CANDIDATES,
envNames: ["BAOYU_CHROME_PATH", "MERMAID_RENDER_CHROME_PATH"]
});
if (!chromePath) {
throw new MermaidRenderError("Chrome not found. Install Google Chrome / Chromium / Edge, or set BAOYU_CHROME_PATH.");
}
const port = await getFreePort("MERMAID_RENDER_DEBUG_PORT");
const chrome = await launchChrome({
chromePath,
profileDir,
port,
headless: true,
extraArgs: [
"--disable-blink-features=AutomationControlled",
"--disable-gpu",
"--hide-scrollbars"
]
});
const wsUrl = await waitForChromeDebugPort(port, 30000, { includeLastError: true });
const cdp = await CdpConnection.connect(wsUrl, 30000);
const state = { cdp, chrome, port, ownsChrome: true };
rendererState = state;
installExitHook();
return state;
})();
try {
return await connectingPromise;
} finally {
connectingPromise = null;
}
}
function buildHostHtml(code, theme, background) {
const script = loadMermaidScript();
const safeCode = code.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
const safeTheme = JSON.stringify(theme);
const cssBackground = background === "transparent" ? "transparent" : background;
return `<!doctype html>
<html>
<head>
<meta charset="utf-8">
<style>
html, body {
margin: 0;
padding: 0;
background: ${cssBackground};
font-family: -apple-system, BlinkMacSystemFont, "PingFang SC", "Microsoft YaHei", "Hiragino Sans GB", "Noto Sans CJK SC", sans-serif;
}
#host {
display: inline-block;
padding: 16px;
}
#host svg {
max-width: none !important;
}
</style>
</head>
<body>
<div id="host"><div class="mermaid">${safeCode}</div></div>
<script>${script}</script>
<script>
window.__mermaidReady = false;
window.__mermaidError = null;
try {
mermaid.initialize({ startOnLoad: false, theme: ${safeTheme}, securityLevel: "loose" });
mermaid.run({ querySelector: ".mermaid" })
.then(function () { window.__mermaidReady = true; })
.catch(function (err) { window.__mermaidError = String(err && err.message || err); });
} catch (err) {
window.__mermaidError = String(err && err.message || err);
}
</script>
</body>
</html>`;
}
async function evaluate(cdp, sessionId, expression) {
const result = await cdp.send("Runtime.evaluate", {
expression,
returnByValue: true,
awaitPromise: false
}, { sessionId });
const exception = result.exceptionDetails;
if (exception) {
return {
value: undefined,
exceptionText: exception.exception?.description ?? exception.text ?? "evaluation failed"
};
}
return { value: result.result.value };
}
async function waitForMermaidSvg(cdp, sessionId, timeoutMs) {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const status = await evaluate(cdp, sessionId, `(function () {
if (window.__mermaidError) {
return { ready: false, error: window.__mermaidError, rect: null };
}
var svg = document.querySelector(".mermaid svg");
if (!svg) return { ready: false, error: null, rect: null };
var bbox = svg.getBoundingClientRect();
return {
ready: window.__mermaidReady === true,
error: null,
rect: {
x: bbox.x,
y: bbox.y,
width: bbox.width,
height: bbox.height,
},
};
})()`);
if (status.exceptionText) {
throw new MermaidRenderError(`Mermaid evaluation failed: ${status.exceptionText}`);
}
const value = status.value;
if (value?.error) {
throw new MermaidRenderError(`Mermaid render failed: ${value.error}`);
}
if (value?.ready && value.rect && value.rect.width > 0 && value.rect.height > 0) {
const host = await evaluate(cdp, sessionId, `(function () {
var host = document.getElementById("host");
if (!host) return null;
var rect = host.getBoundingClientRect();
return { x: rect.x, y: rect.y, width: rect.width, height: rect.height };
})()`);
if (host.value)
return host.value;
return value.rect;
}
await sleep(80);
}
throw new MermaidRenderError(`Mermaid render timed out after ${timeoutMs}ms`);
}
async function withPageSession(state, fn) {
let session = null;
try {
session = await openPageSession({
cdp: state.cdp,
reusing: !state.ownsChrome,
url: "about:blank",
matchTarget: () => false,
enablePage: true,
enableRuntime: true
});
return await fn(session.sessionId, session.targetId);
} finally {
if (session?.createdTarget) {
try {
await state.cdp.send("Target.closeTarget", { targetId: session.targetId });
} catch {}
}
}
}
async function renderMermaidToPng(code, outputPath, options = {}) {
const theme = options.theme ?? "default";
const scale = resolveRenderScale(options.scale);
const minWidth = resolveMinWidth(options.minWidth);
const background = options.background ?? "white";
const timeoutMs = options.timeoutMs ?? 15000;
if (!code.trim()) {
throw new MermaidRenderError("Mermaid code is empty");
}
await import_node_fs2.default.promises.mkdir(import_node_path2.default.dirname(outputPath), { recursive: true });
const state = await ensureRenderer();
const html = buildHostHtml(code, theme, background);
return await withPageSession(state, async (sessionId) => {
await state.cdp.send("Emulation.setDeviceMetricsOverride", {
width: 1280,
height: 800,
deviceScaleFactor: 1,
mobile: false
}, { sessionId });
await state.cdp.send("Page.setDocumentContent", { frameId: await getFrameId(state.cdp, sessionId), html }, { sessionId });
const rect = await waitForMermaidSvg(state.cdp, sessionId, timeoutMs);
const cssWidth = Math.max(1, Math.ceil(rect.width));
const cssHeight = Math.max(1, Math.ceil(rect.height));
const targetCssWidth = Math.max(cssWidth, Math.ceil(minWidth ?? cssWidth));
const captureScale = scale * (targetCssWidth / cssWidth);
const bitmapWidth = Math.max(1, Math.ceil(cssWidth * captureScale));
const bitmapHeight = Math.max(1, Math.ceil(cssHeight * captureScale));
await state.cdp.send("Emulation.setDeviceMetricsOverride", {
width: cssWidth,
height: cssHeight,
deviceScaleFactor: 1,
mobile: false
}, { sessionId });
const shot = await state.cdp.send("Page.captureScreenshot", {
format: "png",
clip: {
x: rect.x,
y: rect.y,
width: cssWidth,
height: cssHeight,
scale: captureScale
},
captureBeyondViewport: true
}, { sessionId });
const buffer = Buffer.from(shot.data, "base64");
await import_node_fs2.default.promises.writeFile(outputPath, buffer);
return {
width: bitmapWidth,
height: bitmapHeight,
bytes: buffer.length
};
});
}
async function getFrameId(cdp, sessionId) {
const result = await cdp.send("Page.getFrameTree", {}, { sessionId });
return result.frameTree.frame.id;
}
async function closeRenderer() {
const state = rendererState;
rendererState = null;
if (!state)
return;
try {
state.cdp.close();
} catch {}
if (state.ownsChrome && state.chrome) {
try {
state.chrome.kill("SIGTERM");
} catch {}
}
}
+665
View File
@@ -0,0 +1,665 @@
// src/mermaid.ts
import { execSync } from "node:child_process";
import fs2 from "node:fs";
import path2 from "node:path";
import process2 from "node:process";
import { fileURLToPath } from "node:url";
// src/index.ts
import { spawn, spawnSync } from "node:child_process";
import fs from "node:fs";
import net from "node:net";
import os from "node:os";
import path from "node:path";
import process from "node:process";
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function getFreePort(fixedEnvName) {
const fixed = fixedEnvName ? Number.parseInt(process.env[fixedEnvName] ?? "", 10) : NaN;
if (Number.isInteger(fixed) && fixed > 0)
return fixed;
return await new Promise((resolve, reject) => {
const server = net.createServer();
server.unref();
server.on("error", reject);
server.listen(0, "127.0.0.1", () => {
const address = server.address();
if (!address || typeof address === "string") {
server.close(() => reject(new Error("Unable to allocate a free TCP port.")));
return;
}
const port = address.port;
server.close((err) => {
if (err)
reject(err);
else
resolve(port);
});
});
});
}
function findChromeExecutable(options) {
for (const envName of options.envNames ?? []) {
const override = process.env[envName]?.trim();
if (override && fs.existsSync(override))
return override;
}
const candidates = process.platform === "darwin" ? options.candidates.darwin ?? options.candidates.default : process.platform === "win32" ? options.candidates.win32 ?? options.candidates.default : options.candidates.default;
for (const candidate of candidates) {
if (fs.existsSync(candidate))
return candidate;
}
return;
}
function resolveSharedChromeProfileDir(options = {}) {
for (const envName of options.envNames ?? []) {
const override = process.env[envName]?.trim();
if (override)
return path.resolve(override);
}
const appDataDirName = options.appDataDirName ?? "baoyu-skills";
const profileDirName = options.profileDirName ?? "chrome-profile";
if (options.wslWindowsHome) {
return path.join(options.wslWindowsHome, ".local", "share", appDataDirName, profileDirName);
}
const base = process.platform === "darwin" ? path.join(os.homedir(), "Library", "Application Support") : process.platform === "win32" ? process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming") : process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share");
return path.join(base, appDataDirName, profileDirName);
}
async function fetchWithTimeout(url, timeoutMs) {
if (!timeoutMs || timeoutMs <= 0)
return await fetch(url, { redirect: "follow" });
const ctl = new AbortController;
const timer = setTimeout(() => ctl.abort(), timeoutMs);
try {
return await fetch(url, { redirect: "follow", signal: ctl.signal });
} finally {
clearTimeout(timer);
}
}
async function fetchJson(url, options = {}) {
const response = await fetchWithTimeout(url, options.timeoutMs);
if (!response.ok) {
throw new Error(`Request failed: ${response.status} ${response.statusText}`);
}
return await response.json();
}
async function isDebugPortReady(port, timeoutMs = 3000) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
return !!version.webSocketDebuggerUrl;
} catch {
return false;
}
}
function parseDevToolsActivePort(filePath) {
try {
const content = fs.readFileSync(filePath, "utf-8");
const lines = content.split(/\r?\n/);
const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
const wsPath = lines[1]?.trim();
if (port > 0 && wsPath)
return { port, wsPath };
} catch {}
return null;
}
async function findExistingChromeDebugPort(options) {
const timeoutMs = options.timeoutMs ?? 3000;
const parsed = parseDevToolsActivePort(path.join(options.profileDir, "DevToolsActivePort"));
if (parsed && parsed.port > 0 && await isDebugPortReady(parsed.port, timeoutMs))
return parsed.port;
if (process.platform === "win32")
return null;
try {
const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5000 });
if (result.status !== 0 || !result.stdout)
return null;
const lines = result.stdout.split(`
`).filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
for (const line of lines) {
const portMatch = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(portMatch?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs))
return port;
}
} catch {}
return null;
}
async function waitForChromeDebugPort(port, timeoutMs, options) {
const start = Date.now();
let lastError = null;
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson(`http://127.0.0.1:${port}/json/version`, { timeoutMs: 5000 });
if (version.webSocketDebuggerUrl)
return version.webSocketDebuggerUrl;
lastError = new Error("Missing webSocketDebuggerUrl");
} catch (error) {
lastError = error;
}
await sleep(200);
}
if (options?.includeLastError && lastError) {
throw new Error(`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
}
throw new Error("Chrome debug port not ready");
}
class CdpConnection {
ws;
nextId = 0;
pending = new Map;
eventHandlers = new Map;
defaultTimeoutMs;
constructor(ws, defaultTimeoutMs = 15000) {
this.ws = ws;
this.defaultTimeoutMs = defaultTimeoutMs;
this.ws.addEventListener("message", (event) => {
try {
const data = typeof event.data === "string" ? event.data : new TextDecoder().decode(event.data);
const msg = JSON.parse(data);
if (msg.method) {
const handlers = this.eventHandlers.get(msg.method);
if (handlers) {
handlers.forEach((handler) => handler(msg.params));
}
}
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer)
clearTimeout(pending.timer);
if (msg.error?.message)
pending.reject(new Error(msg.error.message));
else
pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener("close", () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer)
clearTimeout(pending.timer);
pending.reject(new Error("CDP connection closed."));
}
});
}
static async connect(url, timeoutMs, options) {
const ws = new WebSocket(url);
await new Promise((resolve, reject) => {
const timer = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
ws.addEventListener("open", () => {
clearTimeout(timer);
resolve();
});
ws.addEventListener("error", () => {
clearTimeout(timer);
reject(new Error("CDP connection failed."));
});
});
return new CdpConnection(ws, options?.defaultTimeoutMs ?? 15000);
}
on(method, handler) {
if (!this.eventHandlers.has(method)) {
this.eventHandlers.set(method, new Set);
}
this.eventHandlers.get(method)?.add(handler);
}
off(method, handler) {
this.eventHandlers.get(method)?.delete(handler);
}
async send(method, params, options) {
const id = ++this.nextId;
const message = { id, method };
if (params)
message.params = params;
if (options?.sessionId)
message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
const result = await new Promise((resolve, reject) => {
const timer = timeoutMs > 0 ? setTimeout(() => {
this.pending.delete(id);
reject(new Error(`CDP timeout: ${method}`));
}, timeoutMs) : null;
this.pending.set(id, { resolve, reject, timer });
this.ws.send(JSON.stringify(message));
});
return result;
}
close() {
try {
this.ws.close();
} catch {}
}
}
async function launchChrome(options) {
await fs.promises.mkdir(options.profileDir, { recursive: true });
const args = [
`--remote-debugging-port=${options.port}`,
`--user-data-dir=${options.profileDir}`,
"--no-first-run",
"--no-default-browser-check",
...options.extraArgs ?? []
];
if (options.headless)
args.push("--headless=new");
if (options.url)
args.push(options.url);
return spawn(options.chromePath, args, { stdio: "ignore" });
}
async function openPageSession(options) {
let targetId;
let createdTarget = false;
if (options.reusing) {
const created = await options.cdp.send("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
} else {
const targets = await options.cdp.send("Target.getTargets");
const existing = targets.targetInfos.find(options.matchTarget);
if (existing) {
targetId = existing.targetId;
} else {
const created = await options.cdp.send("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
}
}
const { sessionId } = await options.cdp.send("Target.attachToTarget", { targetId, flatten: true });
if (options.activateTarget ?? true) {
await options.cdp.send("Target.activateTarget", { targetId });
}
if (options.enablePage)
await options.cdp.send("Page.enable", {}, { sessionId });
if (options.enableRuntime)
await options.cdp.send("Runtime.enable", {}, { sessionId });
if (options.enableDom)
await options.cdp.send("DOM.enable", {}, { sessionId });
if (options.enableNetwork)
await options.cdp.send("Network.enable", {}, { sessionId });
return { sessionId, targetId, createdTarget };
}
// src/mermaid.ts
class MermaidRenderError extends Error {
constructor(message, options) {
super(message);
this.name = "MermaidRenderError";
if (options?.cause !== undefined) {
this.cause = options.cause;
}
}
}
function resolveRenderScale(scale) {
const resolved = scale ?? 2;
if (!Number.isFinite(resolved) || resolved <= 0) {
throw new MermaidRenderError(`Invalid Mermaid render scale: ${scale}`);
}
return resolved;
}
function resolveMinWidth(minWidth) {
if (minWidth === undefined)
return;
if (!Number.isFinite(minWidth) || minWidth <= 0) {
throw new MermaidRenderError(`Invalid Mermaid render minWidth: ${minWidth}`);
}
return minWidth;
}
var CHROME_CANDIDATES = {
darwin: [
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary",
"/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"
],
win32: [
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe",
"C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe"
],
default: [
"/usr/bin/google-chrome",
"/usr/bin/google-chrome-stable",
"/usr/bin/chromium",
"/usr/bin/chromium-browser",
"/snap/bin/chromium",
"/usr/bin/microsoft-edge"
]
};
var wslHome;
function getWslWindowsHome() {
if (wslHome !== undefined)
return wslHome;
if (!process2.env.WSL_DISTRO_NAME) {
wslHome = null;
return null;
}
try {
const raw = execSync('cmd.exe /C "echo %USERPROFILE%"', {
encoding: "utf-8",
timeout: 5000
}).trim().replace(/\r/g, "");
wslHome = execSync(`wslpath -u "${raw}"`, {
encoding: "utf-8",
timeout: 5000
}).trim() || null;
} catch {
wslHome = null;
}
return wslHome;
}
function getProfileDir() {
return resolveSharedChromeProfileDir({
envNames: ["BAOYU_CHROME_PROFILE_DIR", "MERMAID_RENDER_PROFILE_DIR"],
wslWindowsHome: getWslWindowsHome()
});
}
function resolveAssetsDir() {
const here = fileURLToPath(import.meta.url);
const dir = path2.dirname(here);
const candidates = [
path2.resolve(dir, "..", "assets"),
path2.resolve(dir, "assets")
];
for (const candidate of candidates) {
if (fs2.existsSync(path2.join(candidate, "mermaid.min.js")))
return candidate;
}
throw new MermaidRenderError(`Cannot locate mermaid.min.js. Looked in: ${candidates.join(", ")}`);
}
var cachedMermaidScript = null;
function loadMermaidScript() {
if (cachedMermaidScript)
return cachedMermaidScript;
const assetsDir = resolveAssetsDir();
cachedMermaidScript = fs2.readFileSync(path2.join(assetsDir, "mermaid.min.js"), "utf-8");
return cachedMermaidScript;
}
var rendererState = null;
var connectingPromise = null;
var exitHookInstalled = false;
function installExitHook() {
if (exitHookInstalled)
return;
exitHookInstalled = true;
const cleanup = () => {
const state = rendererState;
rendererState = null;
if (!state)
return;
try {
state.cdp.close();
} catch {}
if (state.ownsChrome && state.chrome) {
try {
state.chrome.kill("SIGTERM");
} catch {}
}
};
process2.on("exit", cleanup);
process2.on("beforeExit", cleanup);
}
async function tryConnectExisting(port) {
try {
const wsUrl = await waitForChromeDebugPort(port, 5000, { includeLastError: true });
return await CdpConnection.connect(wsUrl, 5000);
} catch {
return null;
}
}
async function ensureRenderer() {
if (rendererState)
return rendererState;
if (connectingPromise)
return await connectingPromise;
connectingPromise = (async () => {
const profileDir = getProfileDir();
const existingPort = await findExistingChromeDebugPort({ profileDir });
if (existingPort) {
const cdp2 = await tryConnectExisting(existingPort);
if (cdp2) {
const state2 = {
cdp: cdp2,
chrome: null,
port: existingPort,
ownsChrome: false
};
rendererState = state2;
installExitHook();
return state2;
}
}
const chromePath = findChromeExecutable({
candidates: CHROME_CANDIDATES,
envNames: ["BAOYU_CHROME_PATH", "MERMAID_RENDER_CHROME_PATH"]
});
if (!chromePath) {
throw new MermaidRenderError("Chrome not found. Install Google Chrome / Chromium / Edge, or set BAOYU_CHROME_PATH.");
}
const port = await getFreePort("MERMAID_RENDER_DEBUG_PORT");
const chrome = await launchChrome({
chromePath,
profileDir,
port,
headless: true,
extraArgs: [
"--disable-blink-features=AutomationControlled",
"--disable-gpu",
"--hide-scrollbars"
]
});
const wsUrl = await waitForChromeDebugPort(port, 30000, { includeLastError: true });
const cdp = await CdpConnection.connect(wsUrl, 30000);
const state = { cdp, chrome, port, ownsChrome: true };
rendererState = state;
installExitHook();
return state;
})();
try {
return await connectingPromise;
} finally {
connectingPromise = null;
}
}
function buildHostHtml(code, theme, background) {
const script = loadMermaidScript();
const safeCode = code.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
const safeTheme = JSON.stringify(theme);
const cssBackground = background === "transparent" ? "transparent" : background;
return `<!doctype html>
<html>
<head>
<meta charset="utf-8">
<style>
html, body {
margin: 0;
padding: 0;
background: ${cssBackground};
font-family: -apple-system, BlinkMacSystemFont, "PingFang SC", "Microsoft YaHei", "Hiragino Sans GB", "Noto Sans CJK SC", sans-serif;
}
#host {
display: inline-block;
padding: 16px;
}
#host svg {
max-width: none !important;
}
</style>
</head>
<body>
<div id="host"><div class="mermaid">${safeCode}</div></div>
<script>${script}</script>
<script>
window.__mermaidReady = false;
window.__mermaidError = null;
try {
mermaid.initialize({ startOnLoad: false, theme: ${safeTheme}, securityLevel: "loose" });
mermaid.run({ querySelector: ".mermaid" })
.then(function () { window.__mermaidReady = true; })
.catch(function (err) { window.__mermaidError = String(err && err.message || err); });
} catch (err) {
window.__mermaidError = String(err && err.message || err);
}
</script>
</body>
</html>`;
}
async function evaluate(cdp, sessionId, expression) {
const result = await cdp.send("Runtime.evaluate", {
expression,
returnByValue: true,
awaitPromise: false
}, { sessionId });
const exception = result.exceptionDetails;
if (exception) {
return {
value: undefined,
exceptionText: exception.exception?.description ?? exception.text ?? "evaluation failed"
};
}
return { value: result.result.value };
}
async function waitForMermaidSvg(cdp, sessionId, timeoutMs) {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const status = await evaluate(cdp, sessionId, `(function () {
if (window.__mermaidError) {
return { ready: false, error: window.__mermaidError, rect: null };
}
var svg = document.querySelector(".mermaid svg");
if (!svg) return { ready: false, error: null, rect: null };
var bbox = svg.getBoundingClientRect();
return {
ready: window.__mermaidReady === true,
error: null,
rect: {
x: bbox.x,
y: bbox.y,
width: bbox.width,
height: bbox.height,
},
};
})()`);
if (status.exceptionText) {
throw new MermaidRenderError(`Mermaid evaluation failed: ${status.exceptionText}`);
}
const value = status.value;
if (value?.error) {
throw new MermaidRenderError(`Mermaid render failed: ${value.error}`);
}
if (value?.ready && value.rect && value.rect.width > 0 && value.rect.height > 0) {
const host = await evaluate(cdp, sessionId, `(function () {
var host = document.getElementById("host");
if (!host) return null;
var rect = host.getBoundingClientRect();
return { x: rect.x, y: rect.y, width: rect.width, height: rect.height };
})()`);
if (host.value)
return host.value;
return value.rect;
}
await sleep(80);
}
throw new MermaidRenderError(`Mermaid render timed out after ${timeoutMs}ms`);
}
async function withPageSession(state, fn) {
let session = null;
try {
session = await openPageSession({
cdp: state.cdp,
reusing: !state.ownsChrome,
url: "about:blank",
matchTarget: () => false,
enablePage: true,
enableRuntime: true
});
return await fn(session.sessionId, session.targetId);
} finally {
if (session?.createdTarget) {
try {
await state.cdp.send("Target.closeTarget", { targetId: session.targetId });
} catch {}
}
}
}
async function renderMermaidToPng(code, outputPath, options = {}) {
const theme = options.theme ?? "default";
const scale = resolveRenderScale(options.scale);
const minWidth = resolveMinWidth(options.minWidth);
const background = options.background ?? "white";
const timeoutMs = options.timeoutMs ?? 15000;
if (!code.trim()) {
throw new MermaidRenderError("Mermaid code is empty");
}
await fs2.promises.mkdir(path2.dirname(outputPath), { recursive: true });
const state = await ensureRenderer();
const html = buildHostHtml(code, theme, background);
return await withPageSession(state, async (sessionId) => {
await state.cdp.send("Emulation.setDeviceMetricsOverride", {
width: 1280,
height: 800,
deviceScaleFactor: 1,
mobile: false
}, { sessionId });
await state.cdp.send("Page.setDocumentContent", { frameId: await getFrameId(state.cdp, sessionId), html }, { sessionId });
const rect = await waitForMermaidSvg(state.cdp, sessionId, timeoutMs);
const cssWidth = Math.max(1, Math.ceil(rect.width));
const cssHeight = Math.max(1, Math.ceil(rect.height));
const targetCssWidth = Math.max(cssWidth, Math.ceil(minWidth ?? cssWidth));
const captureScale = scale * (targetCssWidth / cssWidth);
const bitmapWidth = Math.max(1, Math.ceil(cssWidth * captureScale));
const bitmapHeight = Math.max(1, Math.ceil(cssHeight * captureScale));
await state.cdp.send("Emulation.setDeviceMetricsOverride", {
width: cssWidth,
height: cssHeight,
deviceScaleFactor: 1,
mobile: false
}, { sessionId });
const shot = await state.cdp.send("Page.captureScreenshot", {
format: "png",
clip: {
x: rect.x,
y: rect.y,
width: cssWidth,
height: cssHeight,
scale: captureScale
},
captureBeyondViewport: true
}, { sessionId });
const buffer = Buffer.from(shot.data, "base64");
await fs2.promises.writeFile(outputPath, buffer);
return {
width: bitmapWidth,
height: bitmapHeight,
bytes: buffer.length
};
});
}
async function getFrameId(cdp, sessionId) {
const result = await cdp.send("Page.getFrameTree", {}, { sessionId });
return result.frameTree.frame.id;
}
async function closeRenderer() {
const state = rendererState;
rendererState = null;
if (!state)
return;
try {
state.cdp.close();
} catch {}
if (state.ownsChrome && state.chrome) {
try {
state.chrome.kill("SIGTERM");
} catch {}
}
}
export {
renderMermaidToPng,
closeRenderer,
MermaidRenderError
};
+50
View File
@@ -0,0 +1,50 @@
{
"name": "baoyu-chrome-cdp",
"version": "0.1.1",
"type": "module",
"files": [
"dist",
"src/**/*.ts",
"assets",
"!src/**/*.test.ts"
],
"exports": {
".": {
"types": "./src/index.ts",
"import": "./dist/index.js",
"require": "./dist/index.cjs",
"default": "./dist/index.js"
},
"./mermaid": {
"types": "./src/mermaid.ts",
"import": "./dist/mermaid.js",
"require": "./dist/mermaid.cjs",
"default": "./dist/mermaid.js"
},
"./src/*": "./src/*",
"./assets/*": "./assets/*"
},
"description": "Shared Chrome DevTools Protocol utilities for baoyu skills.",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
"types": "./src/index.ts",
"repository": {
"type": "git",
"url": "git+https://github.com/JimLiu/baoyu-skills.git",
"directory": "packages/baoyu-chrome-cdp"
},
"bugs": {
"url": "https://github.com/JimLiu/baoyu-skills/issues"
},
"homepage": "https://github.com/JimLiu/baoyu-skills/tree/main/packages/baoyu-chrome-cdp#readme",
"publishConfig": {
"access": "public"
},
"scripts": {
"build": "node ../../scripts/build-shared-package.mjs --entry-out src/index.ts:index --entry-out src/mermaid.ts:mermaid --asset assets:assets",
"prepack": "bun run build"
},
"engines": {
"bun": ">=1.2.0"
}
}
+362
View File
@@ -0,0 +1,362 @@
import assert from "node:assert/strict";
import { spawn, type ChildProcess } from "node:child_process";
import fs from "node:fs/promises";
import http from "node:http";
import os from "node:os";
import path from "node:path";
import process from "node:process";
import test, { type TestContext } from "node:test";
import {
discoverRunningChromeDebugPort,
findChromeExecutable,
findExistingChromeDebugPort,
gracefulKillChrome,
getFreePort,
openPageSession,
resolveSharedChromeProfileDir,
waitForChromeDebugPort,
} from "./index.ts";
function useEnv(
t: TestContext,
values: Record<string, string | null>,
): void {
const previous = new Map<string, string | undefined>();
for (const [key, value] of Object.entries(values)) {
previous.set(key, process.env[key]);
if (value == null) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
t.after(() => {
for (const [key, value] of previous.entries()) {
if (value == null) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
});
}
async function makeTempDir(prefix: string): Promise<string> {
return fs.mkdtemp(path.join(os.tmpdir(), prefix));
}
async function startDebugServer(port: number): Promise<http.Server> {
const server = http.createServer((req, res) => {
if (req.url === "/json/version") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({
webSocketDebuggerUrl: `ws://127.0.0.1:${port}/devtools/browser/demo`,
}));
return;
}
res.writeHead(404);
res.end();
});
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(port, "127.0.0.1", () => resolve());
});
return server;
}
async function closeServer(server: http.Server): Promise<void> {
await new Promise<void>((resolve, reject) => {
server.close((error) => {
if (error) reject(error);
else resolve();
});
});
}
function shellPathForPlatform(): string | null {
if (process.platform === "win32") return null;
return "/bin/bash";
}
async function startFakeChromiumProcess(port: number): Promise<ChildProcess | null> {
const shell = shellPathForPlatform();
if (!shell) return null;
const child = spawn(
shell,
[
"-lc",
`exec -a chromium-mock ${JSON.stringify(process.execPath)} -e 'setInterval(() => {}, 1000)' -- --remote-debugging-port=${port}`,
],
{ stdio: "ignore" },
);
await new Promise((resolve) => setTimeout(resolve, 250));
return child;
}
async function stopProcess(child: ChildProcess | null): Promise<void> {
if (!child) return;
if (child.exitCode !== null || child.signalCode !== null) return;
child.kill("SIGTERM");
await new Promise((resolve) => setTimeout(resolve, 100));
if (child.exitCode === null && child.signalCode === null) child.kill("SIGKILL");
if (child.exitCode !== null || child.signalCode !== null) return;
await new Promise((resolve) => child.once("exit", resolve));
}
async function startPortHoldingProcess(port: number): Promise<ChildProcess> {
const child = spawn(
process.execPath,
[
"-e",
`
const http = require("node:http");
const port = Number(process.argv[1]);
const server = http.createServer((_req, res) => res.end("ok"));
server.listen(port, "127.0.0.1", () => process.stdout.write("ready\\n"));
setInterval(() => {}, 1000);
`,
String(port),
],
{
stdio: ["ignore", "pipe", "ignore"],
},
);
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => reject(new Error("Timed out waiting for child server to start.")), 3_000);
child.once("error", (error) => {
clearTimeout(timer);
reject(error);
});
child.stdout?.once("data", () => {
clearTimeout(timer);
resolve();
});
child.once("exit", () => {
clearTimeout(timer);
reject(new Error("Child server exited before becoming ready."));
});
});
return child;
}
test("getFreePort honors a fixed environment override and otherwise allocates a TCP port", async (t) => {
useEnv(t, { TEST_FIXED_PORT: "45678" });
assert.equal(await getFreePort("TEST_FIXED_PORT"), 45678);
const dynamicPort = await getFreePort();
assert.ok(Number.isInteger(dynamicPort));
assert.ok(dynamicPort > 0);
});
test("findChromeExecutable prefers env overrides and falls back to candidate paths", async (t) => {
const root = await makeTempDir("baoyu-chrome-bin-");
t.after(() => fs.rm(root, { recursive: true, force: true }));
const envChrome = path.join(root, "env-chrome");
const fallbackChrome = path.join(root, "fallback-chrome");
await fs.writeFile(envChrome, "");
await fs.writeFile(fallbackChrome, "");
useEnv(t, { BAOYU_CHROME_PATH: envChrome });
assert.equal(
findChromeExecutable({
envNames: ["BAOYU_CHROME_PATH"],
candidates: { default: [fallbackChrome] },
}),
envChrome,
);
useEnv(t, { BAOYU_CHROME_PATH: null });
assert.equal(
findChromeExecutable({
envNames: ["BAOYU_CHROME_PATH"],
candidates: { default: [fallbackChrome] },
}),
fallbackChrome,
);
});
test("resolveSharedChromeProfileDir supports env overrides, WSL paths, and default suffixes", (t) => {
useEnv(t, { BAOYU_SHARED_PROFILE: "/tmp/custom-profile" });
assert.equal(
resolveSharedChromeProfileDir({
envNames: ["BAOYU_SHARED_PROFILE"],
appDataDirName: "demo-app",
profileDirName: "demo-profile",
}),
path.resolve("/tmp/custom-profile"),
);
useEnv(t, { BAOYU_SHARED_PROFILE: null });
assert.equal(
resolveSharedChromeProfileDir({
wslWindowsHome: "/mnt/c/Users/demo",
appDataDirName: "demo-app",
profileDirName: "demo-profile",
}),
path.join("/mnt/c/Users/demo", ".local", "share", "demo-app", "demo-profile"),
);
const fallback = resolveSharedChromeProfileDir({
appDataDirName: "demo-app",
profileDirName: "demo-profile",
});
assert.match(fallback, /demo-app[\\/]demo-profile$/);
});
test("findExistingChromeDebugPort reads DevToolsActivePort and validates it against a live endpoint", async (t) => {
const root = await makeTempDir("baoyu-cdp-profile-");
t.after(() => fs.rm(root, { recursive: true, force: true }));
const port = await getFreePort();
const server = await startDebugServer(port);
t.after(() => closeServer(server));
await fs.writeFile(path.join(root, "DevToolsActivePort"), `${port}\n/devtools/browser/demo\n`);
const found = await findExistingChromeDebugPort({ profileDir: root, timeoutMs: 1000 });
assert.equal(found, port);
});
test("discoverRunningChromeDebugPort reads DevToolsActivePort from the provided user-data dir", async (t) => {
const root = await makeTempDir("baoyu-cdp-user-data-");
t.after(() => fs.rm(root, { recursive: true, force: true }));
const port = await getFreePort();
const server = await startDebugServer(port);
t.after(() => closeServer(server));
await fs.writeFile(path.join(root, "DevToolsActivePort"), `${port}\n/devtools/browser/demo\n`);
const found = await discoverRunningChromeDebugPort({
userDataDirs: [root],
timeoutMs: 1000,
});
assert.deepEqual(found, {
port,
wsUrl: `ws://127.0.0.1:${port}/devtools/browser/demo`,
});
});
test("discoverRunningChromeDebugPort ignores unrelated debugging processes", async (t) => {
if (process.platform === "win32") {
t.skip("Process discovery fallback is not used on Windows.");
return;
}
const root = await makeTempDir("baoyu-cdp-user-data-");
t.after(() => fs.rm(root, { recursive: true, force: true }));
const port = await getFreePort();
const server = await startDebugServer(port);
t.after(() => closeServer(server));
const fakeChromium = await startFakeChromiumProcess(port);
t.after(async () => { await stopProcess(fakeChromium); });
const found = await discoverRunningChromeDebugPort({
userDataDirs: [root],
timeoutMs: 1000,
});
assert.equal(found, null);
});
test("openPageSession reports whether it created a new target", async () => {
const calls: string[] = [];
const cdpExisting = {
send: async <T>(method: string): Promise<T> => {
calls.push(method);
if (method === "Target.getTargets") {
return {
targetInfos: [{ targetId: "existing-target", type: "page", url: "https://gemini.google.com/app" }],
} as T;
}
if (method === "Target.attachToTarget") return { sessionId: "session-existing" } as T;
throw new Error(`Unexpected method: ${method}`);
},
};
const existing = await openPageSession({
cdp: cdpExisting as never,
reusing: false,
url: "https://gemini.google.com/app",
matchTarget: (target) => target.url.includes("gemini.google.com"),
activateTarget: false,
});
assert.deepEqual(existing, {
sessionId: "session-existing",
targetId: "existing-target",
createdTarget: false,
});
assert.deepEqual(calls, ["Target.getTargets", "Target.attachToTarget"]);
const createCalls: string[] = [];
const cdpCreated = {
send: async <T>(method: string): Promise<T> => {
createCalls.push(method);
if (method === "Target.getTargets") return { targetInfos: [] } as T;
if (method === "Target.createTarget") return { targetId: "created-target" } as T;
if (method === "Target.attachToTarget") return { sessionId: "session-created" } as T;
throw new Error(`Unexpected method: ${method}`);
},
};
const created = await openPageSession({
cdp: cdpCreated as never,
reusing: false,
url: "https://gemini.google.com/app",
matchTarget: (target) => target.url.includes("gemini.google.com"),
activateTarget: false,
});
assert.deepEqual(created, {
sessionId: "session-created",
targetId: "created-target",
createdTarget: true,
});
assert.deepEqual(createCalls, ["Target.getTargets", "Target.createTarget", "Target.attachToTarget"]);
});
test("waitForChromeDebugPort retries until the debug endpoint becomes available", async (t) => {
const port = await getFreePort();
const serverPromise = (async () => {
await new Promise((resolve) => setTimeout(resolve, 200));
const server = await startDebugServer(port);
t.after(() => closeServer(server));
})();
const websocketUrl = await waitForChromeDebugPort(port, 4000, {
includeLastError: true,
});
await serverPromise;
assert.equal(websocketUrl, `ws://127.0.0.1:${port}/devtools/browser/demo`);
});
test("gracefulKillChrome waits for the Chrome process to exit and release its port", async (t) => {
const port = await getFreePort();
const child = await startPortHoldingProcess(port);
t.after(async () => { await stopProcess(child); });
assert.equal(await waitForChromeDebugPort(port, 1_000).catch(() => null), null);
await gracefulKillChrome(child, port, 4_000);
assert.ok(child.exitCode !== null || child.signalCode !== null);
assert.equal(
await fetch(`http://127.0.0.1:${port}`).then(() => true).catch(() => false),
false,
);
});
+554
View File
@@ -0,0 +1,554 @@
import { spawn, spawnSync, type ChildProcess } from "node:child_process";
import fs from "node:fs";
import net from "node:net";
import os from "node:os";
import path from "node:path";
import process from "node:process";
export type PlatformCandidates = {
darwin?: string[];
win32?: string[];
default: string[];
};
type PendingRequest = {
resolve: (value: unknown) => void;
reject: (error: Error) => void;
timer: ReturnType<typeof setTimeout> | null;
};
type CdpSendOptions = {
sessionId?: string;
timeoutMs?: number;
};
type FetchJsonOptions = {
timeoutMs?: number;
};
type FindChromeExecutableOptions = {
candidates: PlatformCandidates;
envNames?: string[];
};
type ResolveSharedChromeProfileDirOptions = {
envNames?: string[];
appDataDirName?: string;
profileDirName?: string;
wslWindowsHome?: string | null;
};
type FindExistingChromeDebugPortOptions = {
profileDir: string;
timeoutMs?: number;
};
export type ChromeChannel = "stable" | "beta" | "canary" | "dev";
export type DiscoveredChrome = {
port: number;
wsUrl: string;
};
type DiscoverRunningChromeOptions = {
channels?: ChromeChannel[];
userDataDirs?: string[];
timeoutMs?: number;
};
type LaunchChromeOptions = {
chromePath: string;
profileDir: string;
port: number;
url?: string;
headless?: boolean;
extraArgs?: string[];
};
type ChromeTargetInfo = {
targetId: string;
url: string;
type: string;
};
type OpenPageSessionOptions = {
cdp: CdpConnection;
reusing: boolean;
url: string;
matchTarget: (target: ChromeTargetInfo) => boolean;
enablePage?: boolean;
enableRuntime?: boolean;
enableDom?: boolean;
enableNetwork?: boolean;
activateTarget?: boolean;
};
export type PageSession = {
sessionId: string;
targetId: string;
createdTarget: boolean;
};
export function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
export async function getFreePort(fixedEnvName?: string): Promise<number> {
const fixed = fixedEnvName ? Number.parseInt(process.env[fixedEnvName] ?? "", 10) : NaN;
if (Number.isInteger(fixed) && fixed > 0) return fixed;
return await new Promise((resolve, reject) => {
const server = net.createServer();
server.unref();
server.on("error", reject);
server.listen(0, "127.0.0.1", () => {
const address = server.address();
if (!address || typeof address === "string") {
server.close(() => reject(new Error("Unable to allocate a free TCP port.")));
return;
}
const port = address.port;
server.close((err) => {
if (err) reject(err);
else resolve(port);
});
});
});
}
export function findChromeExecutable(options: FindChromeExecutableOptions): string | undefined {
for (const envName of options.envNames ?? []) {
const override = process.env[envName]?.trim();
if (override && fs.existsSync(override)) return override;
}
const candidates = process.platform === "darwin"
? options.candidates.darwin ?? options.candidates.default
: process.platform === "win32"
? options.candidates.win32 ?? options.candidates.default
: options.candidates.default;
for (const candidate of candidates) {
if (fs.existsSync(candidate)) return candidate;
}
return undefined;
}
export function resolveSharedChromeProfileDir(options: ResolveSharedChromeProfileDirOptions = {}): string {
for (const envName of options.envNames ?? []) {
const override = process.env[envName]?.trim();
if (override) return path.resolve(override);
}
const appDataDirName = options.appDataDirName ?? "baoyu-skills";
const profileDirName = options.profileDirName ?? "chrome-profile";
if (options.wslWindowsHome) {
return path.join(options.wslWindowsHome, ".local", "share", appDataDirName, profileDirName);
}
const base = process.platform === "darwin"
? path.join(os.homedir(), "Library", "Application Support")
: process.platform === "win32"
? (process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming"))
: (process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share"));
return path.join(base, appDataDirName, profileDirName);
}
async function fetchWithTimeout(url: string, timeoutMs?: number): Promise<Response> {
if (!timeoutMs || timeoutMs <= 0) return await fetch(url, { redirect: "follow" });
const ctl = new AbortController();
const timer = setTimeout(() => ctl.abort(), timeoutMs);
try {
return await fetch(url, { redirect: "follow", signal: ctl.signal });
} finally {
clearTimeout(timer);
}
}
async function fetchJson<T = unknown>(url: string, options: FetchJsonOptions = {}): Promise<T> {
const response = await fetchWithTimeout(url, options.timeoutMs);
if (!response.ok) {
throw new Error(`Request failed: ${response.status} ${response.statusText}`);
}
return await response.json() as T;
}
async function isDebugPortReady(port: number, timeoutMs = 3_000): Promise<boolean> {
try {
const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(
`http://127.0.0.1:${port}/json/version`,
{ timeoutMs }
);
return !!version.webSocketDebuggerUrl;
} catch {
return false;
}
}
function isPortListening(port: number, timeoutMs = 3_000): Promise<boolean> {
return new Promise((resolve) => {
const socket = new net.Socket();
const timer = setTimeout(() => { socket.destroy(); resolve(false); }, timeoutMs);
socket.once("connect", () => { clearTimeout(timer); socket.destroy(); resolve(true); });
socket.once("error", () => { clearTimeout(timer); resolve(false); });
socket.connect(port, "127.0.0.1");
});
}
function parseDevToolsActivePort(filePath: string): { port: number; wsPath: string } | null {
try {
const content = fs.readFileSync(filePath, "utf-8");
const lines = content.split(/\r?\n/);
const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
const wsPath = lines[1]?.trim();
if (port > 0 && wsPath) return { port, wsPath };
} catch {}
return null;
}
export async function findExistingChromeDebugPort(options: FindExistingChromeDebugPortOptions): Promise<number | null> {
const timeoutMs = options.timeoutMs ?? 3_000;
const parsed = parseDevToolsActivePort(path.join(options.profileDir, "DevToolsActivePort"));
if (parsed && parsed.port > 0 && await isDebugPortReady(parsed.port, timeoutMs)) return parsed.port;
if (process.platform === "win32") return null;
try {
const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5_000 });
if (result.status !== 0 || !result.stdout) return null;
const lines = result.stdout
.split("\n")
.filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
for (const line of lines) {
const portMatch = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(portMatch?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs)) return port;
}
} catch {}
return null;
}
export function getDefaultChromeUserDataDirs(channels: ChromeChannel[] = ["stable"]): string[] {
const home = os.homedir();
const dirs: string[] = [];
const channelDirs: Record<string, { darwin: string; linux: string; win32: string }> = {
stable: {
darwin: path.join(home, "Library", "Application Support", "Google", "Chrome"),
linux: path.join(home, ".config", "google-chrome"),
win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome", "User Data"),
},
beta: {
darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Beta"),
linux: path.join(home, ".config", "google-chrome-beta"),
win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome Beta", "User Data"),
},
canary: {
darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Canary"),
linux: path.join(home, ".config", "google-chrome-canary"),
win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome SxS", "User Data"),
},
dev: {
darwin: path.join(home, "Library", "Application Support", "Google", "Chrome Dev"),
linux: path.join(home, ".config", "google-chrome-dev"),
win32: path.join(process.env.LOCALAPPDATA ?? path.join(home, "AppData", "Local"), "Google", "Chrome Dev", "User Data"),
},
};
const platform = process.platform === "darwin" ? "darwin" : process.platform === "win32" ? "win32" : "linux";
for (const ch of channels) {
const entry = channelDirs[ch];
if (entry) dirs.push(entry[platform]);
}
return dirs;
}
// Best-effort reuse of an already-running local CDP session discovered from
// known Chrome user-data dirs. This is distinct from Chrome DevTools MCP's
// prompt-based --autoConnect flow.
export async function discoverRunningChromeDebugPort(options: DiscoverRunningChromeOptions = {}): Promise<DiscoveredChrome | null> {
const channels = options.channels ?? ["stable", "beta", "canary", "dev"];
const timeoutMs = options.timeoutMs ?? 3_000;
const userDataDirs = (options.userDataDirs ?? getDefaultChromeUserDataDirs(channels))
.map((dir) => path.resolve(dir));
for (const dir of userDataDirs) {
const parsed = parseDevToolsActivePort(path.join(dir, "DevToolsActivePort"));
if (!parsed) continue;
if (await isPortListening(parsed.port, timeoutMs)) {
return { port: parsed.port, wsUrl: `ws://127.0.0.1:${parsed.port}${parsed.wsPath}` };
}
}
if (process.platform !== "win32") {
try {
const result = spawnSync("ps", ["aux"], { encoding: "utf-8", timeout: 5_000 });
if (result.status === 0 && result.stdout) {
const lines = result.stdout
.split("\n")
.filter((line) =>
line.includes("--remote-debugging-port=") &&
userDataDirs.some((dir) => line.includes(dir))
);
for (const line of lines) {
const portMatch = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(portMatch?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs)) {
try {
const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(`http://127.0.0.1:${port}/json/version`, { timeoutMs });
if (version.webSocketDebuggerUrl) return { port, wsUrl: version.webSocketDebuggerUrl };
} catch {}
}
}
}
} catch {}
}
return null;
}
export async function waitForChromeDebugPort(
port: number,
timeoutMs: number,
options?: { includeLastError?: boolean }
): Promise<string> {
const start = Date.now();
let lastError: unknown = null;
while (Date.now() - start < timeoutMs) {
try {
const version = await fetchJson<{ webSocketDebuggerUrl?: string }>(
`http://127.0.0.1:${port}/json/version`,
{ timeoutMs: 5_000 }
);
if (version.webSocketDebuggerUrl) return version.webSocketDebuggerUrl;
lastError = new Error("Missing webSocketDebuggerUrl");
} catch (error) {
lastError = error;
}
await sleep(200);
}
if (options?.includeLastError && lastError) {
throw new Error(
`Chrome debug port not ready: ${lastError instanceof Error ? lastError.message : String(lastError)}`
);
}
throw new Error("Chrome debug port not ready");
}
export class CdpConnection {
private ws: WebSocket;
private nextId = 0;
private pending = new Map<number, PendingRequest>();
private eventHandlers = new Map<string, Set<(params: unknown) => void>>();
private defaultTimeoutMs: number;
private constructor(ws: WebSocket, defaultTimeoutMs = 15_000) {
this.ws = ws;
this.defaultTimeoutMs = defaultTimeoutMs;
this.ws.addEventListener("message", (event) => {
try {
const data = typeof event.data === "string"
? event.data
: new TextDecoder().decode(event.data as ArrayBuffer);
const msg = JSON.parse(data) as {
id?: number;
method?: string;
params?: unknown;
result?: unknown;
error?: { message?: string };
};
if (msg.method) {
const handlers = this.eventHandlers.get(msg.method);
if (handlers) {
handlers.forEach((handler) => handler(msg.params));
}
}
if (msg.id) {
const pending = this.pending.get(msg.id);
if (pending) {
this.pending.delete(msg.id);
if (pending.timer) clearTimeout(pending.timer);
if (msg.error?.message) pending.reject(new Error(msg.error.message));
else pending.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener("close", () => {
for (const [id, pending] of this.pending.entries()) {
this.pending.delete(id);
if (pending.timer) clearTimeout(pending.timer);
pending.reject(new Error("CDP connection closed."));
}
});
}
static async connect(
url: string,
timeoutMs: number,
options?: { defaultTimeoutMs?: number }
): Promise<CdpConnection> {
const ws = new WebSocket(url);
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
ws.addEventListener("open", () => {
clearTimeout(timer);
resolve();
});
ws.addEventListener("error", () => {
clearTimeout(timer);
reject(new Error("CDP connection failed."));
});
});
return new CdpConnection(ws, options?.defaultTimeoutMs ?? 15_000);
}
on(method: string, handler: (params: unknown) => void): void {
if (!this.eventHandlers.has(method)) {
this.eventHandlers.set(method, new Set());
}
this.eventHandlers.get(method)?.add(handler);
}
off(method: string, handler: (params: unknown) => void): void {
this.eventHandlers.get(method)?.delete(handler);
}
async send<T = unknown>(method: string, params?: Record<string, unknown>, options?: CdpSendOptions): Promise<T> {
const id = ++this.nextId;
const message: Record<string, unknown> = { id, method };
if (params) message.params = params;
if (options?.sessionId) message.sessionId = options.sessionId;
const timeoutMs = options?.timeoutMs ?? this.defaultTimeoutMs;
const result = await new Promise<unknown>((resolve, reject) => {
const timer = timeoutMs > 0
? setTimeout(() => {
this.pending.delete(id);
reject(new Error(`CDP timeout: ${method}`));
}, timeoutMs)
: null;
this.pending.set(id, { resolve, reject, timer });
this.ws.send(JSON.stringify(message));
});
return result as T;
}
close(): void {
try {
this.ws.close();
} catch {}
}
}
export async function launchChrome(options: LaunchChromeOptions): Promise<ChildProcess> {
await fs.promises.mkdir(options.profileDir, { recursive: true });
const args = [
`--remote-debugging-port=${options.port}`,
`--user-data-dir=${options.profileDir}`,
"--no-first-run",
"--no-default-browser-check",
...(options.extraArgs ?? []),
];
if (options.headless) args.push("--headless=new");
if (options.url) args.push(options.url);
return spawn(options.chromePath, args, { stdio: "ignore" });
}
export function killChrome(chrome: ChildProcess): void {
try {
chrome.kill("SIGTERM");
} catch {}
setTimeout(() => {
if (chrome.exitCode === null && chrome.signalCode === null) {
try {
chrome.kill("SIGKILL");
} catch {}
}
}, 2_000).unref?.();
}
export async function gracefulKillChrome(
chrome: ChildProcess,
port?: number,
timeoutMs = 6_000,
): Promise<void> {
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
const exitPromise = new Promise<void>((resolve) => {
chrome.once("exit", () => resolve());
});
killChrome(chrome);
const start = Date.now();
while (Date.now() - start < timeoutMs) {
if (chrome.exitCode !== null || chrome.signalCode !== null) return;
if (port !== undefined && !await isPortListening(port, 250)) return;
const exited = await Promise.race([
exitPromise.then(() => true),
sleep(100).then(() => false),
]);
if (exited) return;
}
await Promise.race([
exitPromise,
sleep(250),
]);
}
export async function openPageSession(options: OpenPageSessionOptions): Promise<PageSession> {
let targetId: string;
let createdTarget = false;
if (options.reusing) {
const created = await options.cdp.send<{ targetId: string }>("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
} else {
const targets = await options.cdp.send<{ targetInfos: ChromeTargetInfo[] }>("Target.getTargets");
const existing = targets.targetInfos.find(options.matchTarget);
if (existing) {
targetId = existing.targetId;
} else {
const created = await options.cdp.send<{ targetId: string }>("Target.createTarget", { url: options.url });
targetId = created.targetId;
createdTarget = true;
}
}
const { sessionId } = await options.cdp.send<{ sessionId: string }>(
"Target.attachToTarget",
{ targetId, flatten: true }
);
if (options.activateTarget ?? true) {
await options.cdp.send("Target.activateTarget", { targetId });
}
if (options.enablePage) await options.cdp.send("Page.enable", {}, { sessionId });
if (options.enableRuntime) await options.cdp.send("Runtime.enable", {}, { sessionId });
if (options.enableDom) await options.cdp.send("DOM.enable", {}, { sessionId });
if (options.enableNetwork) await options.cdp.send("Network.enable", {}, { sessionId });
return { sessionId, targetId, createdTarget };
}
+499
View File
@@ -0,0 +1,499 @@
import { execSync, type ChildProcess } from "node:child_process";
import fs from "node:fs";
import path from "node:path";
import process from "node:process";
import { fileURLToPath } from "node:url";
import {
CdpConnection,
findChromeExecutable,
findExistingChromeDebugPort,
getFreePort,
launchChrome,
openPageSession,
resolveSharedChromeProfileDir,
sleep,
waitForChromeDebugPort,
type PageSession,
type PlatformCandidates,
} from "./index.js";
export class MermaidRenderError extends Error {
constructor(message: string, options?: { cause?: unknown }) {
super(message);
this.name = "MermaidRenderError";
if (options?.cause !== undefined) {
(this as { cause?: unknown }).cause = options.cause;
}
}
}
export interface MermaidRenderOptions {
theme?: string;
scale?: number;
background?: string;
timeoutMs?: number;
minWidth?: number;
}
export interface MermaidRenderResult {
width: number;
height: number;
bytes: number;
}
function resolveRenderScale(scale: number | undefined): number {
const resolved = scale ?? 2;
if (!Number.isFinite(resolved) || resolved <= 0) {
throw new MermaidRenderError(`Invalid Mermaid render scale: ${scale}`);
}
return resolved;
}
function resolveMinWidth(minWidth: number | undefined): number | undefined {
if (minWidth === undefined) return undefined;
if (!Number.isFinite(minWidth) || minWidth <= 0) {
throw new MermaidRenderError(`Invalid Mermaid render minWidth: ${minWidth}`);
}
return minWidth;
}
const CHROME_CANDIDATES: PlatformCandidates = {
darwin: [
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary",
"/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge",
],
win32: [
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe",
"C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe",
],
default: [
"/usr/bin/google-chrome",
"/usr/bin/google-chrome-stable",
"/usr/bin/chromium",
"/usr/bin/chromium-browser",
"/snap/bin/chromium",
"/usr/bin/microsoft-edge",
],
};
let wslHome: string | null | undefined;
function getWslWindowsHome(): string | null {
if (wslHome !== undefined) return wslHome;
if (!process.env.WSL_DISTRO_NAME) {
wslHome = null;
return null;
}
try {
const raw = execSync('cmd.exe /C "echo %USERPROFILE%"', {
encoding: "utf-8",
timeout: 5_000,
}).trim().replace(/\r/g, "");
wslHome = execSync(`wslpath -u "${raw}"`, {
encoding: "utf-8",
timeout: 5_000,
}).trim() || null;
} catch {
wslHome = null;
}
return wslHome;
}
function getProfileDir(): string {
return resolveSharedChromeProfileDir({
envNames: ["BAOYU_CHROME_PROFILE_DIR", "MERMAID_RENDER_PROFILE_DIR"],
wslWindowsHome: getWslWindowsHome(),
});
}
function resolveAssetsDir(): string {
const here = fileURLToPath(import.meta.url);
const dir = path.dirname(here);
const candidates = [
path.resolve(dir, "..", "assets"),
path.resolve(dir, "assets"),
];
for (const candidate of candidates) {
if (fs.existsSync(path.join(candidate, "mermaid.min.js"))) return candidate;
}
throw new MermaidRenderError(
`Cannot locate mermaid.min.js. Looked in: ${candidates.join(", ")}`,
);
}
let cachedMermaidScript: string | null = null;
function loadMermaidScript(): string {
if (cachedMermaidScript) return cachedMermaidScript;
const assetsDir = resolveAssetsDir();
cachedMermaidScript = fs.readFileSync(path.join(assetsDir, "mermaid.min.js"), "utf-8");
return cachedMermaidScript;
}
interface RendererState {
cdp: CdpConnection;
chrome: ChildProcess | null;
port: number | null;
ownsChrome: boolean;
}
let rendererState: RendererState | null = null;
let connectingPromise: Promise<RendererState> | null = null;
let exitHookInstalled = false;
function installExitHook(): void {
if (exitHookInstalled) return;
exitHookInstalled = true;
const cleanup = () => {
const state = rendererState;
rendererState = null;
if (!state) return;
try { state.cdp.close(); } catch {}
if (state.ownsChrome && state.chrome) {
try { state.chrome.kill("SIGTERM"); } catch {}
}
};
process.on("exit", cleanup);
process.on("beforeExit", cleanup);
}
async function tryConnectExisting(port: number): Promise<CdpConnection | null> {
try {
const wsUrl = await waitForChromeDebugPort(port, 5_000, { includeLastError: true });
return await CdpConnection.connect(wsUrl, 5_000);
} catch {
return null;
}
}
async function ensureRenderer(): Promise<RendererState> {
if (rendererState) return rendererState;
if (connectingPromise) return await connectingPromise;
connectingPromise = (async (): Promise<RendererState> => {
const profileDir = getProfileDir();
const existingPort = await findExistingChromeDebugPort({ profileDir });
if (existingPort) {
const cdp = await tryConnectExisting(existingPort);
if (cdp) {
const state: RendererState = {
cdp,
chrome: null,
port: existingPort,
ownsChrome: false,
};
rendererState = state;
installExitHook();
return state;
}
}
const chromePath = findChromeExecutable({
candidates: CHROME_CANDIDATES,
envNames: ["BAOYU_CHROME_PATH", "MERMAID_RENDER_CHROME_PATH"],
});
if (!chromePath) {
throw new MermaidRenderError(
"Chrome not found. Install Google Chrome / Chromium / Edge, or set BAOYU_CHROME_PATH.",
);
}
const port = await getFreePort("MERMAID_RENDER_DEBUG_PORT");
const chrome = await launchChrome({
chromePath,
profileDir,
port,
headless: true,
extraArgs: [
"--disable-blink-features=AutomationControlled",
"--disable-gpu",
"--hide-scrollbars",
],
});
const wsUrl = await waitForChromeDebugPort(port, 30_000, { includeLastError: true });
const cdp = await CdpConnection.connect(wsUrl, 30_000);
const state: RendererState = { cdp, chrome, port, ownsChrome: true };
rendererState = state;
installExitHook();
return state;
})();
try {
return await connectingPromise;
} finally {
connectingPromise = null;
}
}
function buildHostHtml(code: string, theme: string, background: string): string {
const script = loadMermaidScript();
const safeCode = code
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;");
const safeTheme = JSON.stringify(theme);
const cssBackground = background === "transparent" ? "transparent" : background;
return `<!doctype html>
<html>
<head>
<meta charset="utf-8">
<style>
html, body {
margin: 0;
padding: 0;
background: ${cssBackground};
font-family: -apple-system, BlinkMacSystemFont, "PingFang SC", "Microsoft YaHei", "Hiragino Sans GB", "Noto Sans CJK SC", sans-serif;
}
#host {
display: inline-block;
padding: 16px;
}
#host svg {
max-width: none !important;
}
</style>
</head>
<body>
<div id="host"><div class="mermaid">${safeCode}</div></div>
<script>${script}</script>
<script>
window.__mermaidReady = false;
window.__mermaidError = null;
try {
mermaid.initialize({ startOnLoad: false, theme: ${safeTheme}, securityLevel: "loose" });
mermaid.run({ querySelector: ".mermaid" })
.then(function () { window.__mermaidReady = true; })
.catch(function (err) { window.__mermaidError = String(err && err.message || err); });
} catch (err) {
window.__mermaidError = String(err && err.message || err);
}
</script>
</body>
</html>`;
}
async function evaluate<T = unknown>(
cdp: CdpConnection,
sessionId: string,
expression: string,
): Promise<{ value: T | undefined; exceptionText?: string }> {
const result = await cdp.send<{
result: { value?: T };
exceptionDetails?: { text?: string; exception?: { description?: string } };
}>(
"Runtime.evaluate",
{
expression,
returnByValue: true,
awaitPromise: false,
},
{ sessionId },
);
const exception = result.exceptionDetails;
if (exception) {
return {
value: undefined,
exceptionText: exception.exception?.description ?? exception.text ?? "evaluation failed",
};
}
return { value: result.result.value };
}
interface BoundingRect {
x: number;
y: number;
width: number;
height: number;
}
async function waitForMermaidSvg(
cdp: CdpConnection,
sessionId: string,
timeoutMs: number,
): Promise<BoundingRect> {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const status = await evaluate<{
ready: boolean;
error: string | null;
rect: BoundingRect | null;
}>(
cdp,
sessionId,
`(function () {
if (window.__mermaidError) {
return { ready: false, error: window.__mermaidError, rect: null };
}
var svg = document.querySelector(".mermaid svg");
if (!svg) return { ready: false, error: null, rect: null };
var bbox = svg.getBoundingClientRect();
return {
ready: window.__mermaidReady === true,
error: null,
rect: {
x: bbox.x,
y: bbox.y,
width: bbox.width,
height: bbox.height,
},
};
})()`,
);
if (status.exceptionText) {
throw new MermaidRenderError(`Mermaid evaluation failed: ${status.exceptionText}`);
}
const value = status.value;
if (value?.error) {
throw new MermaidRenderError(`Mermaid render failed: ${value.error}`);
}
if (value?.ready && value.rect && value.rect.width > 0 && value.rect.height > 0) {
const host = await evaluate<BoundingRect | null>(
cdp,
sessionId,
`(function () {
var host = document.getElementById("host");
if (!host) return null;
var rect = host.getBoundingClientRect();
return { x: rect.x, y: rect.y, width: rect.width, height: rect.height };
})()`,
);
if (host.value) return host.value;
return value.rect;
}
await sleep(80);
}
throw new MermaidRenderError(`Mermaid render timed out after ${timeoutMs}ms`);
}
async function withPageSession<T>(
state: RendererState,
fn: (sessionId: string, targetId: string) => Promise<T>,
): Promise<T> {
let session: PageSession | null = null;
try {
session = await openPageSession({
cdp: state.cdp,
reusing: !state.ownsChrome,
url: "about:blank",
matchTarget: () => false,
enablePage: true,
enableRuntime: true,
});
return await fn(session.sessionId, session.targetId);
} finally {
if (session?.createdTarget) {
try {
await state.cdp.send("Target.closeTarget", { targetId: session.targetId });
} catch {}
}
}
}
export async function renderMermaidToPng(
code: string,
outputPath: string,
options: MermaidRenderOptions = {},
): Promise<MermaidRenderResult> {
const theme = options.theme ?? "default";
const scale = resolveRenderScale(options.scale);
const minWidth = resolveMinWidth(options.minWidth);
const background = options.background ?? "white";
const timeoutMs = options.timeoutMs ?? 15_000;
if (!code.trim()) {
throw new MermaidRenderError("Mermaid code is empty");
}
await fs.promises.mkdir(path.dirname(outputPath), { recursive: true });
const state = await ensureRenderer();
const html = buildHostHtml(code, theme, background);
return await withPageSession(state, async (sessionId) => {
await state.cdp.send(
"Emulation.setDeviceMetricsOverride",
{
width: 1280,
height: 800,
deviceScaleFactor: 1,
mobile: false,
},
{ sessionId },
);
await state.cdp.send(
"Page.setDocumentContent",
{ frameId: await getFrameId(state.cdp, sessionId), html },
{ sessionId },
);
const rect = await waitForMermaidSvg(state.cdp, sessionId, timeoutMs);
const cssWidth = Math.max(1, Math.ceil(rect.width));
const cssHeight = Math.max(1, Math.ceil(rect.height));
const targetCssWidth = Math.max(cssWidth, Math.ceil(minWidth ?? cssWidth));
const captureScale = scale * (targetCssWidth / cssWidth);
const bitmapWidth = Math.max(1, Math.ceil(cssWidth * captureScale));
const bitmapHeight = Math.max(1, Math.ceil(cssHeight * captureScale));
await state.cdp.send(
"Emulation.setDeviceMetricsOverride",
{
width: cssWidth,
height: cssHeight,
deviceScaleFactor: 1,
mobile: false,
},
{ sessionId },
);
const shot = await state.cdp.send<{ data: string }>(
"Page.captureScreenshot",
{
format: "png",
clip: {
x: rect.x,
y: rect.y,
width: cssWidth,
height: cssHeight,
scale: captureScale,
},
captureBeyondViewport: true,
},
{ sessionId },
);
const buffer = Buffer.from(shot.data, "base64");
await fs.promises.writeFile(outputPath, buffer);
return {
width: bitmapWidth,
height: bitmapHeight,
bytes: buffer.length,
};
});
}
async function getFrameId(cdp: CdpConnection, sessionId: string): Promise<string> {
const result = await cdp.send<{ frameTree: { frame: { id: string } } }>(
"Page.getFrameTree",
{},
{ sessionId },
);
return result.frameTree.frame.id;
}
export async function closeRenderer(): Promise<void> {
const state = rendererState;
rendererState = null;
if (!state) return;
try { state.cdp.close(); } catch {}
if (state.ownsChrome && state.chrome) {
try { state.chrome.kill("SIGTERM"); } catch {}
}
}
+102
View File
@@ -0,0 +1,102 @@
# baoyu-codex-imagegen
Generate images via Codex CLI's built-in `image_gen` tool from non-Codex runtimes (e.g., Claude Code). The wrapper spawns `codex exec --json` and lets the user's existing Codex subscription drive image generation — **no `OPENAI_API_KEY` required**.
This package implements the `preferred_image_backend: codex-imagegen` config key referenced across the `baoyu-skills` plugin and is the engine behind `baoyu-image-gen --provider codex-cli`.
## Layout
```
packages/baoyu-codex-imagegen/
├── src/
│ ├── main.ts # CLI orchestrator (executable via `#!/usr/bin/env bun`)
│ ├── spawn.ts # codex exec child-process wrapper
│ ├── parser.ts # JSONL event-stream parser
│ ├── validator.ts # Output PNG / image_gen-invocation verification
│ ├── cache.ts # SHA256 idempotency cache + file lock
│ ├── logger.ts # Structured JSONL logging
│ ├── types.ts # Shared types and `GenError`
│ └── *.test.ts # Bun unit tests
└── package.json # `bin` points to `src/main.ts`
```
## Prerequisites
```bash
npm install -g @openai/codex
codex login # signs in with your OpenAI account (subscription)
codex --version # confirm >= 0.130
```
`bun` is required for running the wrapper:
```bash
brew install oven-sh/bun/bun
```
If `bun` is not on `PATH`, `npx -y bun src/main.ts …` works as a fallback.
## Usage
```bash
# Inline prompt (executes via shebang once bun is on PATH)
./src/main.ts \
--image /tmp/cat.png \
--prompt "A friendly orange cat, watercolor"
# Or invoke bun explicitly
bun src/main.ts \
--image cover.png \
--prompt-file prompts/01-cover.md \
--aspect 16:9 \
--cache-dir ~/.cache/baoyu-codex-imagegen
# Without bun installed
npx -y bun src/main.ts --image cover.png --prompt "..."
```
Stdout emits a single JSON line:
```json
{"status":"ok","path":"…","bytes":1234567,"elapsed_seconds":62,"thread_id":"…","attempts":1,"cached":false,"usage":{}}
```
On failure:
```json
{"status":"error","path":"…","bytes":0,"error":"…","error_kind":"timeout"}
```
`error_kind` values: `codex_not_installed`, `invalid_args`, `prompt_file_missing`, `spawn_failed`, `timeout`, `no_image_gen_tool_use`, `output_missing`, `invalid_png`, `agent_refused`, `lock_busy`.
## Options
| Flag | Description |
|---|---|
| `--image <path>` | Output PNG path (required) |
| `--prompt <text>` | Prompt text |
| `--prompt-file <path>` | Read prompt from file (mutually exclusive with `--prompt`) |
| `--aspect <ratio>` | Aspect ratio (`1:1`, `16:9`, `9:16`, `4:3`, `2.35:1`). Default: `1:1` |
| `--ref <file>` | Reference image (repeatable) |
| `--timeout <ms>` | Codex exec timeout in ms. Default: `300000` |
| `--retries <n>` | Retry attempts on retryable errors. Default: `2` |
| `--retry-delay <ms>` | Base retry delay (exponential). Default: `1500` |
| `--cache-dir <path>` | Enable idempotency cache. Disabled by default. |
| `--log-file <path>` | Append structured JSONL log |
| `-v, --verbose` | Verbose stderr logging |
| `-h, --help` | Show help |
## Test
```bash
cd packages/baoyu-codex-imagegen
bun test
```
## Trade-offs
- 510× slower than direct OpenAI API calls (except on cache hits)
- Uses your Codex subscription — programmatic use of `codex exec` falls into the same terms as interactive use
- Requires `codex` CLI and active login session
See [`docs/codex-imagegen-backend.md`](../../docs/codex-imagegen-backend.md) for the full background.
@@ -0,0 +1,39 @@
{
"name": "baoyu-codex-imagegen",
"version": "0.1.0",
"type": "module",
"description": "Generate images via Codex CLI's built-in image_gen tool from non-Codex runtimes (Claude Code, Hermes, …).",
"bin": {
"codex-imagegen": "./src/main.ts"
},
"files": [
"src/**/*.ts",
"!src/**/*.test.ts"
],
"exports": {
".": {
"types": "./src/main.ts",
"default": "./src/main.ts"
},
"./src/*": "./src/*"
},
"scripts": {
"test": "bun test",
"smoke": "bun src/main.ts --help"
},
"repository": {
"type": "git",
"url": "git+https://github.com/JimLiu/baoyu-skills.git",
"directory": "packages/baoyu-codex-imagegen"
},
"bugs": {
"url": "https://github.com/JimLiu/baoyu-skills/issues"
},
"homepage": "https://github.com/JimLiu/baoyu-skills/tree/main/packages/baoyu-codex-imagegen#readme",
"publishConfig": {
"access": "public"
},
"engines": {
"bun": ">=1.2.0"
}
}
@@ -0,0 +1,63 @@
import { test, expect } from "bun:test";
import { mkdtemp, writeFile, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { cacheKey, lookupCache, storeCache, FileLock } from "./cache.ts";
test("cacheKey is deterministic and order-independent for refs", () => {
const k1 = cacheKey("hello", "16:9", ["a.png", "b.png"]);
const k2 = cacheKey("hello", "16:9", ["b.png", "a.png"]);
expect(k1).toBe(k2);
const k3 = cacheKey("hello", "16:9", []);
expect(k3).not.toBe(k1);
const k4 = cacheKey("hello", "1:1", []);
expect(k4).not.toBe(k3);
});
test("lookupCache returns null on miss, path on hit", async () => {
const dir = await mkdtemp(path.join(tmpdir(), "cig-test-"));
try {
expect(await lookupCache(dir, "abc")).toBeNull();
const fake = path.join(dir, "abc.png");
await writeFile(fake, Buffer.alloc(2000));
expect(await lookupCache(dir, "abc")).toBe(fake);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("storeCache copies source into cache", async () => {
const dir = await mkdtemp(path.join(tmpdir(), "cig-test-"));
const src = path.join(dir, "src.png");
try {
await writeFile(src, Buffer.from("xxxx".repeat(1000)));
await storeCache(dir, "key1", src);
const cached = await lookupCache(dir, "key1");
expect(cached).not.toBeNull();
const a = await readFile(src);
const b = await readFile(cached!);
expect(a.equals(b)).toBe(true);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("FileLock prevents concurrent acquisition", async () => {
const dir = await mkdtemp(path.join(tmpdir(), "cig-lock-"));
try {
const lockPath = path.join(dir, "x.lock");
const lock1 = new FileLock(lockPath);
const lock2 = new FileLock(lockPath);
await lock1.acquire(1000);
let lock2Acquired = false;
const p = lock2.acquire(500).then(() => (lock2Acquired = true)).catch(() => {});
await new Promise((r) => setTimeout(r, 300));
expect(lock2Acquired).toBe(false);
await lock1.release();
await p;
expect(lock2Acquired).toBe(true);
await lock2.release();
} finally {
await rm(dir, { recursive: true, force: true });
}
});
@@ -0,0 +1,80 @@
import { createHash } from "node:crypto";
import { mkdir, readFile, writeFile, copyFile, stat } from "node:fs/promises";
import { existsSync, openSync, closeSync } from "node:fs";
import path from "node:path";
import { setTimeout as delay } from "node:timers/promises";
export function cacheKey(prompt: string, aspect: string, refs: string[]): string {
const h = createHash("sha256");
h.update(prompt);
h.update("|");
h.update(aspect);
h.update("|");
for (const r of [...refs].sort()) h.update(r);
return h.digest("hex").slice(0, 16);
}
export async function lookupCache(cacheDir: string, key: string): Promise<string | null> {
const entry = path.join(cacheDir, `${key}.png`);
try {
const s = await stat(entry);
if (s.size > 1000) return entry;
} catch {}
return null;
}
export async function storeCache(cacheDir: string, key: string, sourcePath: string): Promise<void> {
await mkdir(cacheDir, { recursive: true });
const entry = path.join(cacheDir, `${key}.png`);
await copyFile(sourcePath, entry);
}
export class FileLock {
private fd: number | null = null;
constructor(private lockPath: string) {}
async acquire(timeoutMs = 30_000): Promise<void> {
const start = Date.now();
await mkdir(path.dirname(this.lockPath), { recursive: true });
while (Date.now() - start < timeoutMs) {
try {
this.fd = openSync(this.lockPath, "wx");
return;
} catch (e: any) {
if (e.code !== "EEXIST") throw e;
if (await this.isStale()) {
try {
await this.release(true);
} catch {}
continue;
}
await delay(200);
}
}
throw new Error(`Failed to acquire lock at ${this.lockPath} within ${timeoutMs}ms`);
}
private async isStale(): Promise<boolean> {
try {
const s = await stat(this.lockPath);
return Date.now() - s.mtimeMs > 10 * 60 * 1000;
} catch {
return true;
}
}
async release(force = false): Promise<void> {
if (this.fd != null) {
try {
closeSync(this.fd);
} catch {}
this.fd = null;
}
if (existsSync(this.lockPath) || force) {
const { unlink } = await import("node:fs/promises");
try {
await unlink(this.lockPath);
} catch {}
}
}
}
@@ -0,0 +1,39 @@
import { appendFile, mkdir } from "node:fs/promises";
import path from "node:path";
export interface LogEntry {
ts: string;
level: "info" | "warn" | "error";
event: string;
[k: string]: unknown;
}
export class JsonLogger {
constructor(private logFile: string | null, public verbose: boolean) {}
async log(level: LogEntry["level"], event: string, extra: Record<string, unknown> = {}): Promise<void> {
const entry: LogEntry = { ts: new Date().toISOString(), level, event, ...extra };
const line = JSON.stringify(entry);
if (this.verbose) process.stderr.write(`[${level}] ${event} ${jsonExtras(extra)}\n`);
if (this.logFile) {
await mkdir(path.dirname(this.logFile), { recursive: true });
await appendFile(this.logFile, line + "\n", "utf-8");
}
}
info(event: string, extra?: Record<string, unknown>) {
return this.log("info", event, extra);
}
warn(event: string, extra?: Record<string, unknown>) {
return this.log("warn", event, extra);
}
error(event: string, extra?: Record<string, unknown>) {
return this.log("error", event, extra);
}
}
function jsonExtras(extra: Record<string, unknown>): string {
const entries = Object.entries(extra);
if (entries.length === 0) return "";
return entries.map(([k, v]) => `${k}=${typeof v === "string" ? v : JSON.stringify(v)}`).join(" ");
}
+326
View File
@@ -0,0 +1,326 @@
#!/usr/bin/env bun
import { readFile, mkdir, copyFile, stat } from "node:fs/promises";
import { homedir } from "node:os";
import path from "node:path";
import process from "node:process";
import { setTimeout as delay } from "node:timers/promises";
import { GenError, type CliOptions, type GenerateResult } from "./types.ts";
import { runCodexExec } from "./spawn.ts";
import { findCpToTarget, verifyImageGenWasInvoked, verifyOutput } from "./validator.ts";
import { cacheKey, lookupCache, storeCache, FileLock } from "./cache.ts";
import { JsonLogger } from "./logger.ts";
const HELP = `codex-imagegen — generate images via Codex CLI's image_gen tool
Usage:
codex-imagegen --image <output.png> [--prompt <text> | --prompt-file <path>] [options]
Required:
--image <path> Output PNG path
--prompt <text> Prompt text (or use --prompt-file)
--prompt-file <path> Read prompt from file
Options:
--aspect <ratio> Aspect ratio (1:1, 16:9, 9:16, 4:3, 2.35:1). Default: 1:1
--ref <file> Reference image (repeatable)
--timeout <ms> Codex exec timeout in ms. Default: 300000
--retries <n> Retry attempts on retryable errors. Default: 2
--retry-delay <ms> Base retry delay (exponential). Default: 1500
--cache-dir <path> Enable idempotency cache. Disabled by default.
--log-file <path> Append JSONL log
-v, --verbose Verbose stderr logging
-h, --help Show this help
Stdout: single JSON line on success or failure.
`;
const SHELL_METACHAR = /[;|&`$<>\n\r()'"]/;
function assertSafePath(label: string, value: string): void {
if (SHELL_METACHAR.test(value)) {
throw new GenError(
"invalid_args",
`${label} contains shell metacharacters and would be unsafe to interpolate into the codex instruction: ${value}`,
false,
);
}
}
function parseArgs(argv: string[]): CliOptions {
const opts: CliOptions = {
prompt: "",
promptFile: null,
outputPath: "",
aspect: "1:1",
refImages: [],
timeoutMs: 300_000,
retries: 2,
retryDelayMs: 1500,
cacheDir: null,
logFile: null,
verbose: false,
};
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
const next = () => argv[++i];
switch (a) {
case "--prompt": opts.prompt = next(); break;
case "--prompt-file": opts.promptFile = next(); break;
case "--image": opts.outputPath = next(); break;
case "--aspect": opts.aspect = next(); break;
case "--ref": opts.refImages.push(next()); break;
case "--timeout": opts.timeoutMs = Number(next()); break;
case "--retries": opts.retries = Number(next()); break;
case "--retry-delay": opts.retryDelayMs = Number(next()); break;
case "--cache-dir": opts.cacheDir = next(); break;
case "--log-file": opts.logFile = next(); break;
case "-v":
case "--verbose": opts.verbose = true; break;
case "-h":
case "--help": process.stdout.write(HELP); process.exit(0);
default: throw new GenError("invalid_args", `Unknown argument: ${a}`, false);
}
}
if (!opts.outputPath) throw new GenError("invalid_args", "--image is required", false);
if (opts.prompt && opts.promptFile) {
throw new GenError("invalid_args", "--prompt and --prompt-file are mutually exclusive", false);
}
if (!opts.prompt && !opts.promptFile) {
throw new GenError("invalid_args", "--prompt or --prompt-file required", false);
}
// Resolve every filesystem path to absolute up front, so behavior is
// independent of the caller's cwd. This matters when the wrapper is
// invoked from a skill running in an arbitrary working directory.
const cwd = process.cwd();
const toAbs = (p: string) => (path.isAbsolute(p) ? p : path.resolve(cwd, p));
opts.outputPath = toAbs(opts.outputPath);
if (opts.promptFile) opts.promptFile = toAbs(opts.promptFile);
opts.refImages = opts.refImages.map(toAbs);
if (opts.cacheDir) opts.cacheDir = toAbs(opts.cacheDir);
if (opts.logFile) opts.logFile = toAbs(opts.logFile);
// The output and ref paths are interpolated raw into the agent instruction
// sent to `codex exec --sandbox danger-full-access`. A path containing shell
// metacharacters could be misread by the agent's shell when it cp's the
// result into place. Reject upfront rather than trusting the agent to quote.
assertSafePath("--image path", opts.outputPath);
for (const ref of opts.refImages) assertSafePath("--ref path", ref);
return opts;
}
async function loadPrompt(opts: CliOptions): Promise<string> {
if (opts.prompt) return opts.prompt;
const file = opts.promptFile!;
try {
return await readFile(file, "utf-8");
} catch {
throw new GenError("prompt_file_missing", `Prompt file not found: ${file}`, false);
}
}
function buildInstruction(prompt: string, opts: CliOptions): string {
const refHint = opts.refImages.length > 0
? `\nREFERENCE IMAGES (attached above): ${opts.refImages.length} image(s) provided for style/composition guidance.\n`
: "";
return `You have an internal tool called image_gen for image generation. Use it.
TASK: Generate an image with the spec below, then save to disk.
PROMPT:
${prompt}
ASPECT RATIO: ${opts.aspect}
OUTPUT PATH: ${opts.outputPath}
${refHint}
STEPS:
1. Call image_gen with the prompt and aspect ratio above${opts.refImages.length > 0 ? " (using the attached reference images for guidance)" : ""}.
2. Move or copy the resulting image from Codex default location ($CODEX_HOME/generated_images/...) to: ${opts.outputPath}
3. Verify with: ls -la ${opts.outputPath}
4. Reply with ONLY this JSON line (no markdown fences, no other text):
{"status":"ok","path":"${opts.outputPath}","bytes":<file_size_in_bytes>}
HARD CONSTRAINTS:
- Do NOT use curl, wget, Python, or any external API.
- Do NOT use bash to fabricate an image; only image_gen produces real pixels.
- Use ONLY the image_gen internal tool.`;
}
async function attemptGenerate(
opts: CliOptions,
instruction: string,
attempt: number,
log: JsonLogger,
): Promise<{ bytes: number; threadId: string | null; usage: any; toolCalls: any[] }> {
await log.info("attempt.start", { attempt, output: opts.outputPath, aspect: opts.aspect });
const run = await runCodexExec({
instruction,
timeoutMs: opts.timeoutMs,
refImages: opts.refImages,
});
await log.info("codex.completed", {
duration_ms: run.durationMs,
thread_id: run.threadId,
tool_calls: run.toolCalls.length,
usage: run.usage,
raw_log: run.rawLogPath,
});
// verify: thread id must be present
if (!run.threadId) {
throw new GenError("agent_refused", "No thread id in event stream");
}
// verify: image_gen was actually invoked (check $CODEX_HOME/generated_images/{threadId}/)
const ver = await verifyImageGenWasInvoked(run.threadId);
if (!ver.ok) {
// secondary verify: did tool_calls include cp/mv from generated_images to our target
if (!findCpToTarget(run.toolCalls, opts.outputPath)) {
throw new GenError("no_image_gen_tool_use", `image_gen was not invoked: ${ver.reason}`);
}
}
// verify output
const { bytes } = await verifyOutput(opts.outputPath);
return {
bytes,
threadId: run.threadId,
usage: run.usage,
toolCalls: run.toolCalls.map((tc) => ({ tool: tc.tool, status: tc.status })),
};
}
async function generate(opts: CliOptions, log: JsonLogger): Promise<GenerateResult> {
const startEpoch = Date.now();
const prompt = await loadPrompt(opts);
// Cache lookup
if (opts.cacheDir) {
const key = cacheKey(prompt, opts.aspect, opts.refImages);
const cached = await lookupCache(opts.cacheDir, key);
if (cached) {
await mkdir(path.dirname(opts.outputPath), { recursive: true });
await copyFile(cached, opts.outputPath);
const s = await stat(opts.outputPath);
await log.info("cache.hit", { key, source: cached });
return {
status: "ok",
path: opts.outputPath,
bytes: s.size,
elapsed_seconds: 0,
thread_id: null,
attempts: 0,
cached: true,
usage: null,
tool_calls: [],
};
}
await log.info("cache.miss", { key });
}
// lock to prevent concurrent codex exec
const lockDir = opts.cacheDir ?? path.join(homedir(), ".cache", "baoyu-codex-imagegen");
const lock = new FileLock(path.join(lockDir, "codex-exec.lock"));
try {
await lock.acquire(60_000);
} catch (e) {
throw new GenError("lock_busy", String(e), false);
}
await mkdir(path.dirname(opts.outputPath), { recursive: true });
const instruction = buildInstruction(prompt, opts);
let lastErr: GenError | null = null;
let lastAttempt = 0;
try {
for (let attempt = 1; attempt <= opts.retries + 1; attempt++) {
lastAttempt = attempt;
try {
const result = await attemptGenerate(opts, instruction, attempt, log);
// write to cache
if (opts.cacheDir) {
const key = cacheKey(prompt, opts.aspect, opts.refImages);
await storeCache(opts.cacheDir, key, opts.outputPath);
await log.info("cache.stored", { key });
}
return {
status: "ok",
path: opts.outputPath,
bytes: result.bytes,
elapsed_seconds: Math.round((Date.now() - startEpoch) / 1000),
thread_id: result.threadId,
attempts: attempt,
cached: false,
usage: result.usage,
tool_calls: result.toolCalls,
};
} catch (e) {
lastErr = e instanceof GenError ? e : new GenError("spawn_failed", String(e));
await log.warn("attempt.failed", {
attempt,
kind: lastErr.kind,
retryable: lastErr.retryable,
error: lastErr.message,
});
if (!lastErr.retryable || attempt > opts.retries) break;
const wait = opts.retryDelayMs * Math.pow(2, attempt - 1);
await log.info("retry.wait", { wait_ms: wait, next_attempt: attempt + 1 });
await delay(wait);
}
}
} finally {
await lock.release();
}
const err = lastErr ?? new GenError("spawn_failed", "Unknown failure");
err.attempts = lastAttempt;
throw err;
}
async function main() {
let opts: CliOptions;
try {
opts = parseArgs(process.argv.slice(2));
} catch (e) {
const err = e instanceof GenError ? e : new GenError("invalid_args", String(e), false);
process.stderr.write(`Error: ${err.message}\n`);
process.exit(2);
}
const log = new JsonLogger(opts.logFile, opts.verbose);
await log.info("start", { output: opts.outputPath, aspect: opts.aspect, refs: opts.refImages.length });
try {
const result = await generate(opts, log);
await log.info("done", { bytes: result.bytes, attempts: result.attempts, cached: result.cached });
process.stdout.write(JSON.stringify(result) + "\n");
process.exit(0);
} catch (e) {
const err = e instanceof GenError ? e : new GenError("spawn_failed", String(e));
await log.error("failed", { kind: err.kind, error: err.message, attempts: err.attempts ?? 0 });
const out: GenerateResult = {
status: "error",
path: opts.outputPath,
bytes: 0,
elapsed_seconds: 0,
thread_id: null,
attempts: err.attempts ?? 0,
cached: false,
usage: null,
tool_calls: [],
error: err.message,
error_kind: err.kind,
};
process.stdout.write(JSON.stringify(out) + "\n");
process.exit(1);
}
}
main();
@@ -0,0 +1,49 @@
import { test, expect } from "bun:test";
import { parseEventStream, hasImageGenInvocation } from "./parser.ts";
const REAL_PoC_STREAM = `{"type":"thread.started","thread_id":"019e40d3-30e3-7030-874d-773bc0d6d1eb"}
{"type":"turn.started"}
{"type":"item.started","item":{"id":"item_0","type":"command_execution","command":"sed -n '1,5p' /tmp/x.md","status":"in_progress"}}
{"type":"item.completed","item":{"id":"item_0","type":"command_execution","command":"sed -n '1,5p' /tmp/x.md","exit_code":0,"status":"completed"}}
{"type":"item.started","item":{"id":"item_1","type":"command_execution","command":"cp /Users/x/.codex/generated_images/019e40d3/ig_abc.png /tmp/out.png","status":"in_progress"}}
{"type":"item.completed","item":{"id":"item_1","type":"command_execution","command":"cp /Users/x/.codex/generated_images/019e40d3/ig_abc.png /tmp/out.png","exit_code":0,"status":"completed"}}
{"type":"item.completed","item":{"id":"item_2","type":"agent_message","text":"{\\"status\\":\\"ok\\",\\"path\\":\\"/tmp/out.png\\",\\"bytes\\":1234567}"}}
{"type":"turn.completed","usage":{"input_tokens":100000,"cached_input_tokens":80000,"output_tokens":500,"reasoning_output_tokens":50}}`;
test("parseEventStream extracts threadId, toolCalls, agentMessage, usage", () => {
const r = parseEventStream(REAL_PoC_STREAM);
expect(r.threadId).toBe("019e40d3-30e3-7030-874d-773bc0d6d1eb");
expect(r.toolCalls.length).toBe(3);
expect(r.usage).toEqual({
input: 100000,
cached_input: 80000,
output: 500,
reasoning: 50,
});
expect(r.agentMessage).toContain('"status":"ok"');
});
test("parseEventStream tolerates malformed lines", () => {
const stream = `not json at all
{"type":"thread.started","thread_id":"abc"}
{partial json
{"type":"turn.completed","usage":{"input_tokens":1,"cached_input_tokens":0,"output_tokens":1,"reasoning_output_tokens":0}}`;
const r = parseEventStream(stream);
expect(r.threadId).toBe("abc");
expect(r.usage?.input).toBe(1);
});
test("hasImageGenInvocation finds shell calls touching generated_images", () => {
const r = parseEventStream(REAL_PoC_STREAM);
// image_gen itself is not an event; inferred via generated_images cp path
// this test only verifies parser behavior; driver logic lives in validator
const hasCp = r.toolCalls.some((tc) => tc.command?.includes("generated_images"));
expect(hasCp).toBe(true);
});
test("hasImageGenInvocation (proper) returns false when no image_gen tool", () => {
expect(hasImageGenInvocation([{ id: "1", tool: "shell", status: "completed" }])).toBe(false);
expect(
hasImageGenInvocation([{ id: "1", tool: "image_gen", status: "completed" }]),
).toBe(true);
});
@@ -0,0 +1,64 @@
import type { CodexRunResult, ToolCall, TokenUsage } from "./types.ts";
export function parseEventStream(raw: string): Omit<CodexRunResult, "rawLogPath" | "durationMs"> {
const lines = raw.split("\n").filter((l) => l.trim().length > 0);
let threadId: string | null = null;
let agentMessage: string | null = null;
let usage: TokenUsage | null = null;
const toolCallsById = new Map<string, ToolCall>();
for (const line of lines) {
let event: any;
try {
event = JSON.parse(line);
} catch {
continue;
}
const type = event?.type;
if (type === "thread.started") {
threadId = event.thread_id ?? null;
} else if (type === "item.started" || type === "item.completed") {
const item = event.item;
if (!item?.id) continue;
const tc: ToolCall = {
id: item.id,
tool: deriveToolName(item),
status: item.status ?? (type === "item.completed" ? "completed" : "in_progress"),
command: item.command,
};
toolCallsById.set(item.id, tc);
if (item.type === "agent_message" && type === "item.completed") {
agentMessage = String(item.text ?? "");
}
} else if (type === "turn.completed") {
const u = event.usage;
if (u) {
usage = {
input: u.input_tokens ?? 0,
cached_input: u.cached_input_tokens ?? 0,
output: u.output_tokens ?? 0,
reasoning: u.reasoning_output_tokens ?? 0,
};
}
}
}
return {
threadId,
toolCalls: Array.from(toolCallsById.values()),
agentMessage,
usage,
};
}
function deriveToolName(item: any): string {
if (item.type === "command_execution") return "shell";
if (item.type === "agent_message") return "agent_message";
if (item.type === "image_gen" || item.type === "image_generation") return "image_gen";
if (typeof item.tool === "string") return item.tool;
return item.type ?? "unknown";
}
export function hasImageGenInvocation(toolCalls: ToolCall[]): boolean {
return toolCalls.some((tc) => tc.tool === "image_gen");
}
@@ -0,0 +1,81 @@
import { spawn } from "node:child_process";
import { writeFile, mkdtemp } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { GenError, type CodexRunResult } from "./types.ts";
import { parseEventStream } from "./parser.ts";
export interface SpawnInput {
instruction: string;
timeoutMs: number;
refImages?: string[];
}
export async function runCodexExec(input: SpawnInput): Promise<CodexRunResult> {
const start = Date.now();
const logDir = await mkdtemp(path.join(tmpdir(), "codex-imggen-"));
const rawLogPath = path.join(logDir, "stream.jsonl");
// --skip-git-repo-check: lets the wrapper run from non-git cwds
// (e.g. /tmp, or a skill installed under ~/.claude/plugins/...).
// Without it, codex refuses with "Not inside a trusted directory".
const args = [
"exec",
"--json",
"--sandbox",
"danger-full-access",
"--skip-git-repo-check",
];
for (const img of input.refImages ?? []) {
args.push("--image", img);
}
args.push("-");
let timedOut = false;
const child = spawn("codex", args, { stdio: ["pipe", "pipe", "pipe"] });
let stdout = "";
let stderr = "";
child.stdout.on("data", (chunk) => {
stdout += chunk.toString();
});
child.stderr.on("data", (chunk) => {
stderr += chunk.toString();
});
child.stdin.write(input.instruction);
child.stdin.end();
const timer = setTimeout(() => {
timedOut = true;
child.kill("SIGTERM");
setTimeout(() => child.kill("SIGKILL"), 2000);
}, input.timeoutMs);
const exit = await new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve) => {
child.on("close", (code, signal) => resolve({ code, signal }));
});
clearTimeout(timer);
await writeFile(rawLogPath, stdout + (stderr ? `\n--- stderr ---\n${stderr}` : ""));
if (timedOut) {
throw new GenError("timeout", `codex exec exceeded ${input.timeoutMs}ms (log: ${rawLogPath})`);
}
if (exit.code !== 0) {
if (stderr.includes("command not found") || stderr.includes("not found: codex")) {
throw new GenError("codex_not_installed", "codex CLI not installed", false);
}
throw new GenError(
"spawn_failed",
`codex exec exited ${exit.code} signal=${exit.signal} (log: ${rawLogPath})`,
);
}
const parsed = parseEventStream(stdout);
return {
...parsed,
rawLogPath,
durationMs: Date.now() - start,
};
}
@@ -0,0 +1,79 @@
export interface CliOptions {
prompt: string;
promptFile: string | null;
outputPath: string;
aspect: string;
refImages: string[];
timeoutMs: number;
retries: number;
retryDelayMs: number;
cacheDir: string | null;
logFile: string | null;
verbose: boolean;
}
export interface ToolCall {
id: string;
tool: string;
status: string;
command?: string;
}
export interface TokenUsage {
input: number;
cached_input: number;
output: number;
reasoning: number;
}
export interface CodexRunResult {
threadId: string | null;
toolCalls: ToolCall[];
agentMessage: string | null;
usage: TokenUsage | null;
rawLogPath: string;
durationMs: number;
}
export interface GenerateResult {
status: "ok" | "error";
path: string;
bytes: number;
elapsed_seconds: number;
thread_id: string | null;
attempts: number;
cached: boolean;
usage: TokenUsage | null;
tool_calls: { tool: string; status: string }[];
error?: string;
error_kind?: ErrorKind;
}
export type ErrorKind =
| "codex_not_installed"
| "invalid_args"
| "prompt_file_missing"
| "spawn_failed"
| "timeout"
| "no_image_gen_tool_use"
| "output_missing"
| "invalid_png"
| "agent_refused"
| "lock_busy";
export const RETRYABLE: ReadonlySet<ErrorKind> = new Set([
"spawn_failed",
"timeout",
"no_image_gen_tool_use",
"output_missing",
"invalid_png",
"agent_refused",
]);
export class GenError extends Error {
attempts?: number;
constructor(public kind: ErrorKind, message: string, public retryable?: boolean) {
super(message);
this.retryable = retryable ?? RETRYABLE.has(kind);
}
}
@@ -0,0 +1,98 @@
import { test, expect } from "bun:test";
import { mkdtemp, writeFile, rm, mkdir } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { verifyOutput, verifyImageGenWasInvoked, findCpToTarget } from "./validator.ts";
import { GenError } from "./types.ts";
const PNG_HEADER = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
test("verifyOutput passes for valid PNG", async () => {
const dir = await mkdtemp(path.join(tmpdir(), "cig-val-"));
try {
const p = path.join(dir, "good.png");
await writeFile(p, Buffer.concat([PNG_HEADER, Buffer.alloc(5000)]));
const r = await verifyOutput(p);
expect(r.bytes).toBeGreaterThan(1000);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("verifyOutput rejects missing file", async () => {
await expect(verifyOutput("/no/such/file.png")).rejects.toBeInstanceOf(GenError);
});
test("verifyOutput rejects tiny file", async () => {
const dir = await mkdtemp(path.join(tmpdir(), "cig-val-"));
try {
const p = path.join(dir, "tiny.png");
await writeFile(p, "tiny");
await expect(verifyOutput(p)).rejects.toThrow(/too small/);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("verifyOutput rejects non-PNG magic", async () => {
const dir = await mkdtemp(path.join(tmpdir(), "cig-val-"));
try {
const p = path.join(dir, "fake.png");
await writeFile(p, Buffer.concat([Buffer.from("GIF89a"), Buffer.alloc(5000)]));
await expect(verifyOutput(p)).rejects.toThrow(/not a valid PNG/);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("verifyImageGenWasInvoked false when no thread directory", async () => {
const orig = process.env.CODEX_HOME;
const tempHome = await mkdtemp(path.join(tmpdir(), "cig-home-"));
process.env.CODEX_HOME = tempHome;
try {
const r = await verifyImageGenWasInvoked("no-such-thread");
expect(r.ok).toBe(false);
} finally {
process.env.CODEX_HOME = orig;
await rm(tempHome, { recursive: true, force: true });
}
});
test("verifyImageGenWasInvoked true when PNG exists in thread dir", async () => {
const orig = process.env.CODEX_HOME;
const tempHome = await mkdtemp(path.join(tmpdir(), "cig-home-"));
process.env.CODEX_HOME = tempHome;
try {
const threadDir = path.join(tempHome, "generated_images", "thread-xyz");
await mkdir(threadDir, { recursive: true });
await writeFile(path.join(threadDir, "ig_abc.png"), Buffer.alloc(100));
const r = await verifyImageGenWasInvoked("thread-xyz");
expect(r.ok).toBe(true);
} finally {
process.env.CODEX_HOME = orig;
await rm(tempHome, { recursive: true, force: true });
}
});
test("findCpToTarget detects cp from generated_images", () => {
expect(
findCpToTarget(
[
{
id: "1",
tool: "shell",
status: "completed",
command: "cp ~/.codex/generated_images/thread/ig_x.png /tmp/out.png",
},
],
"/tmp/out.png",
),
).toBe(true);
expect(
findCpToTarget(
[{ id: "1", tool: "shell", status: "completed", command: "ls /tmp" }],
"/tmp/out.png",
),
).toBe(false);
});
@@ -0,0 +1,55 @@
import { stat, readdir } from "node:fs/promises";
import { homedir } from "node:os";
import path from "node:path";
import { GenError } from "./types.ts";
import type { ToolCall } from "./types.ts";
const PNG_MAGIC = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
export function codexHome(): string {
return process.env.CODEX_HOME ?? path.join(homedir(), ".codex");
}
export async function verifyImageGenWasInvoked(threadId: string | null): Promise<{ ok: boolean; reason?: string }> {
if (!threadId) return { ok: false, reason: "no thread id" };
const dir = path.join(codexHome(), "generated_images", threadId);
try {
const entries = await readdir(dir);
const pngs = entries.filter((e) => e.toLowerCase().endsWith(".png"));
if (pngs.length === 0) return { ok: false, reason: `no PNG in ${dir}` };
return { ok: true };
} catch (e: any) {
return { ok: false, reason: `cannot read ${dir}: ${e?.code ?? e?.message}` };
}
}
export function findCpToTarget(toolCalls: ToolCall[], target: string): boolean {
return toolCalls.some(
(tc) =>
tc.tool === "shell" &&
typeof tc.command === "string" &&
(tc.command.includes(target) || tc.command.includes(path.basename(target))) &&
/\b(cp|mv|cat)\b/.test(tc.command) &&
tc.command.includes("generated_images"),
);
}
export async function verifyOutput(outputPath: string): Promise<{ bytes: number }> {
let s;
try {
s = await stat(outputPath);
} catch {
throw new GenError("output_missing", `Output file not created: ${outputPath}`);
}
if (s.size < 1000) {
throw new GenError("invalid_png", `Output file too small (${s.size} bytes)`);
}
const file = Bun.file(outputPath);
const head = new Uint8Array(await file.slice(0, 8).arrayBuffer());
for (let i = 0; i < PNG_MAGIC.length; i++) {
if (head[i] !== PNG_MAGIC[i]) {
throw new GenError("invalid_png", `Output is not a valid PNG (magic mismatch)`);
}
}
return { bytes: s.size };
}
+11
View File
@@ -0,0 +1,11 @@
# Changesets
This folder stores release notes for version bumps managed by Changesets.
Create a new changeset before merging a user-facing change:
```bash
bunx changeset
```
After the changeset lands on `main`, GitHub Actions will open or update the release PR automatically. Merging that release PR publishes the next npm version.
@@ -0,0 +1,11 @@
{
"$schema": "https://unpkg.com/@changesets/config@3.1.3/schema.json",
"changelog": "@changesets/cli/changelog",
"commit": false,
"fixed": [],
"linked": [],
"access": "public",
"baseBranch": "main",
"updateInternalDependencies": "patch",
"ignore": []
}
@@ -0,0 +1,5 @@
---
"baoyu-reader": patch
---
Add a `defuddle.md` markdown fallback for generic extraction when the CLI is asked to produce markdown output.
+58
View File
@@ -0,0 +1,58 @@
name: Release
on:
push:
branches:
- main
workflow_dispatch:
concurrency: ${{ github.workflow }}-${{ github.ref }}
permissions:
contents: write
pull-requests: write
id-token: write
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: 1.2.23
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: 24
registry-url: https://registry.npmjs.org
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Type check
run: bun run check
- name: Test
run: bun test
- name: Verify npm package contents
run: npm pack --dry-run
- name: Create release PR or publish to npm
id: changesets
uses: changesets/action@v1
with:
version: bun run version-packages
publish: bun run release
commit: "chore: release"
title: "chore: release"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+6
View File
@@ -0,0 +1,6 @@
node_modules
dist
.DS_Store
.debug
coverage
+65
View File
@@ -0,0 +1,65 @@
# Changelog
English | [简体中文](./CHANGELOG.zh-CN.md)
The format is inspired by Keep a Changelog, and the project follows Semantic
Versioning.
## [0.1.2] - 2026-04-21
#### Changed
- Upgraded Defuddle to 0.17.0 and jsdom to 29.0.2 for generic extraction.
- Added an `@xmldom/xmldom` override to keep the optional Defuddle MathML
dependency chain on a non-vulnerable release.
#### Fixed
- Fixed X/Twitter video extraction for single posts and X Articles by selecting
the highest-bitrate MP4 variant instead of the preview image URL.
- Fixed X Article media rendering so video entities are emitted as
`[video](...)` links instead of image embeds.
## [0.1.1] - 2026-03-27
#### Added
- Added the `hn` adapter for Hacker News stories and comment threads.
- Added `--download-media` and `--media-dir` to download extracted media and
rewrite Markdown links.
- Added Defuddle as the first generic extraction pass, with Readability +
HTML-to-Markdown as fallback.
- Added interactive wait modes for login and verification flows, including
manual verification handoff and force-wait resume behavior.
- Added `--format markdown|json` while keeping `--json` as a compatibility
alias.
- Added Changesets-based release automation for npm publishing.
#### Changed
- Renamed the package and CLI from `baoyu-markdown` to `baoyu-fetch`.
- Changed the published package to run `src/cli.ts` directly with Bun instead of
shipping a prebuilt `dist`.
- Improved X extraction for threads, articles, note tweets, embeds, image URLs,
login state handling, and media metadata.
- Improved YouTube transcript extraction and normalized Markdown image output.
#### Fixed
- Fixed X note tweet URL expansion.
- Fixed media URL normalization before download, including Substack media links.
- Fixed foreground behavior for interactive flows so manual steps are easier to
complete.
## [0.1.0] - 2026-03-25
#### Added
- Initial public release as `baoyu-markdown`.
- Added Chrome CDP session management, controlled tabs, and network journaling.
- Added built-in adapters for `x`, `youtube`, and the generic fallback.
- Added X article parsing, X single/tweet extraction, and YouTube transcript
extraction.
- Added Markdown rendering, document metadata output, and CLI support for file
output, JSON output, debug exports, custom Chrome connection settings,
headless mode, and timeout control.
+55
View File
@@ -0,0 +1,55 @@
# 更新日志
[English](./CHANGELOG.md) | 简体中文
格式参考 Keep a Changelog,版本号遵循 Semantic Versioning。
## [0.1.2] - 2026-04-21
### 变更
- 将 Defuddle 升级到 0.17.0、jsdom 升级到 29.0.2,用于通用页面提取。
- 新增 `@xmldom/xmldom` override,使 Defuddle 的可选 MathML 依赖链保持在无漏洞版本。
### 修复
- 修复 X/Twitter 单条内容和 X Article 的视频提取逻辑,改为选择最高码率 MP4 变体,而不是预览图 URL。
- 修复 X Article 媒体渲染,视频实体现在输出为 `[video](...)` 链接,而不是图片嵌入。
## [0.1.1] - 2026-03-27
### 新增
- 新增 `hn` adapter,可提取 Hacker News 帖子与评论串。
- 新增 `--download-media``--media-dir`,可下载提取出的媒体文件并重写
Markdown 链接。
- 通用提取链路新增 Defuddle 首选路径,并保留 Readability + HTML to Markdown
作为回退方案。
- 新增登录/验证场景的交互等待模式,支持手动验证接管和 force wait 自动恢复。
- 新增 `--format markdown|json`,同时保留 `--json` 作为兼容别名。
- 新增基于 Changesets 的 npm 发版自动化流程。
### 变更
- 将包名和 CLI 名称从 `baoyu-markdown` 更名为 `baoyu-fetch`
- npm 发布物改为直接以 Bun 执行 `src/cli.ts`,不再附带预构建的 `dist`
- 强化 X 提取链路,覆盖 thread、article、note tweet、embed、图片 URL、
登录态判断与媒体元数据。
- 增强 YouTube transcript 提取,并规范化 Markdown 图片输出。
### 修复
- 修复 X note tweet 的 URL 展开问题。
- 修复媒体下载前的 URL 规范化问题,包括 Substack 媒体链接。
- 修复交互模式的前台行为,使手动登录/验证流程更稳定。
## [0.1.0] - 2026-03-25
### 新增
- `baoyu-markdown` 的首个公开版本。
- 新增 Chrome CDP 会话管理、受控 tab 与网络日志采集能力。
- 新增内置 `x``youtube` 与通用 fallback adapters。
- 新增 X article 解析、X 单条内容提取,以及 YouTube transcript 提取。
- 新增 Markdown 渲染与文档元数据输出,并提供文件输出、JSON 输出、调试导出、
自定义 Chrome 连接、headless 模式和超时控制等 CLI 能力。
+124
View File
@@ -0,0 +1,124 @@
# baoyu-fetch
English | [简体中文](./README.zh-CN.md) | [Changelog](./CHANGELOG.md) | [中文更新日志](./CHANGELOG.zh-CN.md)
`baoyu-fetch` is a Bun CLI built on Chrome CDP. Give it a URL and it returns
high-quality `markdown` or `json`. When a site adapter matches, it prefers API
responses or structured page data; otherwise it falls back to generic HTML
extraction.
## Features
- Capture rendered page content through Chrome CDP
- Observe network requests and responses, and fetch bodies when needed
- Adapter registry that auto-selects a handler from the URL
- Built-in adapters for `x`, `youtube`, and `hn`
- Generic fallback: Defuddle first, then Readability + HTML-to-Markdown; when `--format markdown` is requested, it can also fall back to `defuddle.md`
- Print `markdown` / `json` to stdout or save with `--output`
- Optionally download extracted images or videos and rewrite Markdown links
- Optional wait modes for login and verification flows
- Chrome profile defaults to `baoyu-skills/chrome-profile`
## Installation
```bash
bun install
```
For package usage, the quickest option is:
```bash
bunx baoyu-fetch https://example.com
```
You can also install it globally:
```bash
npm install -g baoyu-fetch
```
The npm package ships TypeScript source entrypoints instead of a prebuilt
`dist`, so Bun is required at runtime.
## Usage
```bash
bun run src/cli.ts https://example.com
bunx baoyu-fetch https://example.com
baoyu-fetch https://example.com
baoyu-fetch https://example.com --format markdown --output article.md
baoyu-fetch https://example.com --format markdown --output article.md --download-media
baoyu-fetch https://x.com/jack/status/20 --format json --output article.json
baoyu-fetch https://x.com/jack/status/20 --json
baoyu-fetch https://x.com/jack/status/20 --wait-for interaction
baoyu-fetch https://x.com/jack/status/20 --wait-for force
baoyu-fetch https://x.com/jack/status/20 --chrome-profile-dir ~/Library/Application\\ Support/baoyu-skills/chrome-profile
```
## Options
```bash
baoyu-fetch <url> [options]
Options:
--output <file> Save output to file
--format <type> Output format: markdown | json
--json Alias for --format json
--adapter <name> Force an adapter (for example x / hn / generic)
--download-media Download adapter-reported media into ./imgs and ./videos, then rewrite markdown links
--media-dir <dir> Base directory for downloaded media. Defaults to the output directory
--debug-dir <dir> Write debug artifacts (html, document.json, network.json)
--cdp-url <url> Reuse an existing Chrome DevTools endpoint
--browser-path <path> Explicit Chrome binary path
--chrome-profile-dir <path>
Chrome user data dir. Defaults to BAOYU_CHROME_PROFILE_DIR
or baoyu-skills/chrome-profile
--headless Launch a temporary headless Chrome if needed
--wait-for <mode> Wait mode: interaction | force
--wait-for-interaction
Alias for --wait-for interaction
--wait-for-login Alias for --wait-for interaction
--interaction-timeout <ms>
Manual interaction timeout. Default: 600000
--interaction-poll-interval <ms>
Poll interval while waiting. Default: 1500
--login-timeout <ms> Alias for --interaction-timeout
--login-poll-interval <ms>
Alias for --interaction-poll-interval
--timeout <ms> Page load timeout. Default: 30000
--help Show help
```
## How It Works
1. The CLI parses the target URL and options.
2. It opens or connects to a Chrome CDP session and creates a controlled tab.
3. `NetworkJournal` records requests and responses.
4. The adapter registry resolves a site-specific adapter when possible.
5. The adapter returns a structured `ExtractedDocument`.
6. If nothing matches, generic HTML extraction runs instead.
7. The result is rendered as Markdown, or returned as JSON with both
`document` and `markdown`.
## Development
```bash
bun run check
bun run test
bun run build
```
## Release
When you make a user-visible change, add a changeset first:
```bash
bunx changeset
```
After the generated `.changeset/*.md` file lands on `main`, GitHub Actions will
open or update the release PR. Merging that release PR publishes the package to
npm.
The publish flow does not build `dist`; it publishes `src/*.ts` for Bun
execution directly.
+122
View File
@@ -0,0 +1,122 @@
# baoyu-fetch
[English](./README.md) | 简体中文 | [更新日志](./CHANGELOG.zh-CN.md) | [English Changelog](./CHANGELOG.md)
`baoyu-fetch` 是一个基于 Chrome CDP 的 Bun CLI。输入 URL,它会输出高质量
`markdown``json`;命中站点 adapter 时优先消费 API 返回或页面内结构化
数据,未命中时回退到通用 HTML 提取。
## 当前能力
- 通过 Chrome CDP 抓取渲染后的页面内容
- 监听网络请求与响应,按需拉取响应体
- adapter registry,支持按 URL 自动命中站点处理器
- 内置 `x``youtube``hn` adapters
- 通用 fallbackDefuddle 优先,Readability + HTML to Markdown 回退;`--format markdown` 时会再尝试 `defuddle.md` 兜底
- `stdout``--output` 输出 `markdown` / `json`
- 可选下载提取出的图片/视频并重写 Markdown 链接
- 提供登录/验证场景下的交互等待模式
- Chrome profile 默认对齐 `baoyu-skills/chrome-profile`
## 安装
```bash
bun install
```
作为包使用时,推荐直接这样运行:
```bash
bunx baoyu-fetch https://example.com
```
也可以全局安装:
```bash
npm install -g baoyu-fetch
```
npm 包发布的是 TypeScript 源码入口,不包含预编译的 `dist`,所以运行时需要
Bun。
## 用法
```bash
bun run src/cli.ts https://example.com
bunx baoyu-fetch https://example.com
baoyu-fetch https://example.com
baoyu-fetch https://example.com --format markdown --output article.md
baoyu-fetch https://example.com --format markdown --output article.md --download-media
baoyu-fetch https://x.com/jack/status/20 --format json --output article.json
baoyu-fetch https://x.com/jack/status/20 --json
baoyu-fetch https://x.com/jack/status/20 --wait-for interaction
baoyu-fetch https://x.com/jack/status/20 --wait-for force
baoyu-fetch https://x.com/jack/status/20 --chrome-profile-dir ~/Library/Application\\ Support/baoyu-skills/chrome-profile
```
## 主要参数
```bash
baoyu-fetch <url> [options]
Options:
--output <file> 保存输出内容到文件
--format <type> 输出格式:markdown | json
--json `--format json` 的兼容别名
--adapter <name> 强制使用指定 adapter(如 x / hn / generic
--download-media 下载 adapter 返回的媒体到 ./imgs 和 ./videos,并重写 markdown 链接
--media-dir <dir> 指定媒体下载根目录;默认使用输出文件所在目录
--debug-dir <dir> 导出调试信息(html、document.json、network.json
--cdp-url <url> 连接现有 Chrome 调试地址
--browser-path <path> 指定 Chrome 可执行文件
--chrome-profile-dir <path>
指定 Chrome profile 目录。默认使用 BAOYU_CHROME_PROFILE_DIR
否则回退到 baoyu-skills/chrome-profile
--headless 启动临时 headless Chrome(未连现有实例时)
--wait-for <mode> 等待模式:interaction | force
--wait-for-interaction
`--wait-for interaction` 的别名
--wait-for-login `--wait-for interaction` 的别名
--interaction-timeout <ms>
手动交互等待超时,默认 600000
--interaction-poll-interval <ms>
等待期间的轮询间隔,默认 1500
--login-timeout <ms> `--interaction-timeout` 的别名
--login-poll-interval <ms>
`--interaction-poll-interval` 的别名
--timeout <ms> 页面加载超时,默认 30000
--help 显示帮助
```
## 设计
核心链路:
1. CLI 解析 URL 和选项
2. 建立 CDP 会话并创建受控 tab
3. 启动 `NetworkJournal` 收集所有请求/响应
4. 由 adapter registry 匹配站点 adapter
5. adapter 返回结构化 `ExtractedDocument`
6. 没命中则走通用 HTML 提取
7. 按请求输出 Markdown,或输出包含 `document``markdown` 的 JSON
## 开发
```bash
bun run check
bun run test
bun run build
```
## 发版
新增用户可见改动后,先添加一个 changeset:
```bash
bunx changeset
```
把生成的 `.changeset/*.md` 一起合并到 `main` 后,GitHub Actions 会自动创建或
更新 release PR;合并 release PR 之后,会自动发布到 npm。
发布流程不会编译 `dist`,而是直接把 `src/*.ts` 发布出去供 Bun 执行。
+530
View File
@@ -0,0 +1,530 @@
{
"lockfileVersion": 1,
"configVersion": 0,
"workspaces": {
"": {
"name": "baoyu-fetch",
"dependencies": {
"@mozilla/readability": "^0.6.0",
"chrome-launcher": "^1.2.1",
"defuddle": "^0.17.0",
"jsdom": "^29.0.2",
"remark-gfm": "^4.0.1",
"remark-parse": "^11.0.0",
"turndown": "^7.2.0",
"turndown-plugin-gfm": "^1.0.2",
"unified": "^11.0.5",
"ws": "^8.18.3",
},
"devDependencies": {
"@changesets/cli": "^2.30.0",
"@types/bun": "^1.2.23",
"@types/jsdom": "^21.1.7",
"@types/ws": "^8.18.1",
"typescript": "^5.9.2",
},
},
},
"overrides": {
"@xmldom/xmldom": "0.8.13",
},
"packages": {
"@asamuzakjp/css-color": ["@asamuzakjp/css-color@5.1.11", "", { "dependencies": { "@asamuzakjp/generational-cache": "^1.0.1", "@csstools/css-calc": "^3.2.0", "@csstools/css-color-parser": "^4.1.0", "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-KVw6qIiCTUQhByfTd78h2yD1/00waTmm9uy/R7Ck/ctUyAPj+AEDLkQIdJW0T8+qGgj3j5bpNKK7Q3G+LedJWg=="],
"@asamuzakjp/dom-selector": ["@asamuzakjp/dom-selector@7.1.1", "", { "dependencies": { "@asamuzakjp/generational-cache": "^1.0.1", "@asamuzakjp/nwsapi": "^2.3.9", "bidi-js": "^1.0.3", "css-tree": "^3.2.1", "is-potential-custom-element-name": "^1.0.1" } }, "sha512-67RZDnYRc8H/8MLDgQCDE//zoqVFwajkepHZgmXrbwybzXOEwOWGPYGmALYl9J2DOLfFPPs6kKCqmbzV895hTQ=="],
"@asamuzakjp/generational-cache": ["@asamuzakjp/generational-cache@1.0.1", "", {}, "sha512-wajfB8KqzMCN2KGNFdLkReeHncd0AslUSrvHVvvYWuU8ghncRJoA50kT3zP9MVL0+9g4/67H+cdvBskj9THPzg=="],
"@asamuzakjp/nwsapi": ["@asamuzakjp/nwsapi@2.3.9", "", {}, "sha512-n8GuYSrI9bF7FFZ/SjhwevlHc8xaVlb/7HmHelnc/PZXBD2ZR49NnN9sMMuDdEGPeeRQ5d0hqlSlEpgCX3Wl0Q=="],
"@babel/runtime": ["@babel/runtime@7.29.2", "", {}, "sha512-JiDShH45zKHWyGe4ZNVRrCjBz8Nh9TMmZG1kh4QTK8hCBTWBi8Da+i7s1fJw7/lYpM4ccepSNfqzZ/QvABBi5g=="],
"@bramus/specificity": ["@bramus/specificity@2.4.2", "", { "dependencies": { "css-tree": "^3.0.0" }, "bin": { "specificity": "bin/cli.js" } }, "sha512-ctxtJ/eA+t+6q2++vj5j7FYX3nRu311q1wfYH3xjlLOsczhlhxAg2FWNUXhpGvAw3BWo1xBcvOV6/YLc2r5FJw=="],
"@changesets/apply-release-plan": ["@changesets/apply-release-plan@7.1.0", "", { "dependencies": { "@changesets/config": "^3.1.3", "@changesets/get-version-range-type": "^0.4.0", "@changesets/git": "^3.0.4", "@changesets/should-skip-package": "^0.1.2", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "detect-indent": "^6.0.0", "fs-extra": "^7.0.1", "lodash.startcase": "^4.4.0", "outdent": "^0.5.0", "prettier": "^2.7.1", "resolve-from": "^5.0.0", "semver": "^7.5.3" } }, "sha512-yq8ML3YS7koKQ/9bk1PqO0HMzApIFNwjlwCnwFEXMzNe8NpzeeYYKCmnhWJGkN8g7E51MnWaSbqRcTcdIxUgnQ=="],
"@changesets/assemble-release-plan": ["@changesets/assemble-release-plan@6.0.9", "", { "dependencies": { "@changesets/errors": "^0.2.0", "@changesets/get-dependents-graph": "^2.1.3", "@changesets/should-skip-package": "^0.1.2", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "semver": "^7.5.3" } }, "sha512-tPgeeqCHIwNo8sypKlS3gOPmsS3wP0zHt67JDuL20P4QcXiw/O4Hl7oXiuLnP9yg+rXLQ2sScdV1Kkzde61iSQ=="],
"@changesets/changelog-git": ["@changesets/changelog-git@0.2.1", "", { "dependencies": { "@changesets/types": "^6.1.0" } }, "sha512-x/xEleCFLH28c3bQeQIyeZf8lFXyDFVn1SgcBiR2Tw/r4IAWlk1fzxCEZ6NxQAjF2Nwtczoen3OA2qR+UawQ8Q=="],
"@changesets/cli": ["@changesets/cli@2.30.0", "", { "dependencies": { "@changesets/apply-release-plan": "^7.1.0", "@changesets/assemble-release-plan": "^6.0.9", "@changesets/changelog-git": "^0.2.1", "@changesets/config": "^3.1.3", "@changesets/errors": "^0.2.0", "@changesets/get-dependents-graph": "^2.1.3", "@changesets/get-release-plan": "^4.0.15", "@changesets/git": "^3.0.4", "@changesets/logger": "^0.1.1", "@changesets/pre": "^2.0.2", "@changesets/read": "^0.6.7", "@changesets/should-skip-package": "^0.1.2", "@changesets/types": "^6.1.0", "@changesets/write": "^0.4.0", "@inquirer/external-editor": "^1.0.2", "@manypkg/get-packages": "^1.1.3", "ansi-colors": "^4.1.3", "enquirer": "^2.4.1", "fs-extra": "^7.0.1", "mri": "^1.2.0", "package-manager-detector": "^0.2.0", "picocolors": "^1.1.0", "resolve-from": "^5.0.0", "semver": "^7.5.3", "spawndamnit": "^3.0.1", "term-size": "^2.1.0" }, "bin": { "changeset": "bin.js" } }, "sha512-5D3Nk2JPqMI1wK25pEymeWRSlSMdo5QOGlyfrKg0AOufrUcjEE3RQgaCpHoBiM31CSNrtSgdJ0U6zL1rLDDfBA=="],
"@changesets/config": ["@changesets/config@3.1.3", "", { "dependencies": { "@changesets/errors": "^0.2.0", "@changesets/get-dependents-graph": "^2.1.3", "@changesets/logger": "^0.1.1", "@changesets/should-skip-package": "^0.1.2", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "fs-extra": "^7.0.1", "micromatch": "^4.0.8" } }, "sha512-vnXjcey8YgBn2L1OPWd3ORs0bGC4LoYcK/ubpgvzNVr53JXV5GiTVj7fWdMRsoKUH7hhhMAQnsJUqLr21EncNw=="],
"@changesets/errors": ["@changesets/errors@0.2.0", "", { "dependencies": { "extendable-error": "^0.1.5" } }, "sha512-6BLOQUscTpZeGljvyQXlWOItQyU71kCdGz7Pi8H8zdw6BI0g3m43iL4xKUVPWtG+qrrL9DTjpdn8eYuCQSRpow=="],
"@changesets/get-dependents-graph": ["@changesets/get-dependents-graph@2.1.3", "", { "dependencies": { "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "picocolors": "^1.1.0", "semver": "^7.5.3" } }, "sha512-gphr+v0mv2I3Oxt19VdWRRUxq3sseyUpX9DaHpTUmLj92Y10AGy+XOtV+kbM6L/fDcpx7/ISDFK6T8A/P3lOdQ=="],
"@changesets/get-release-plan": ["@changesets/get-release-plan@4.0.15", "", { "dependencies": { "@changesets/assemble-release-plan": "^6.0.9", "@changesets/config": "^3.1.3", "@changesets/pre": "^2.0.2", "@changesets/read": "^0.6.7", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3" } }, "sha512-Q04ZaRPuEVZtA+auOYgFaVQQSA98dXiVe/yFaZfY7hoSmQICHGvP0TF4u3EDNHWmmCS4ekA/XSpKlSM2PyTS2g=="],
"@changesets/get-version-range-type": ["@changesets/get-version-range-type@0.4.0", "", {}, "sha512-hwawtob9DryoGTpixy1D3ZXbGgJu1Rhr+ySH2PvTLHvkZuQ7sRT4oQwMh0hbqZH1weAooedEjRsbrWcGLCeyVQ=="],
"@changesets/git": ["@changesets/git@3.0.4", "", { "dependencies": { "@changesets/errors": "^0.2.0", "@manypkg/get-packages": "^1.1.3", "is-subdir": "^1.1.1", "micromatch": "^4.0.8", "spawndamnit": "^3.0.1" } }, "sha512-BXANzRFkX+XcC1q/d27NKvlJ1yf7PSAgi8JG6dt8EfbHFHi4neau7mufcSca5zRhwOL8j9s6EqsxmT+s+/E6Sw=="],
"@changesets/logger": ["@changesets/logger@0.1.1", "", { "dependencies": { "picocolors": "^1.1.0" } }, "sha512-OQtR36ZlnuTxKqoW4Sv6x5YIhOmClRd5pWsjZsddYxpWs517R0HkyiefQPIytCVh4ZcC5x9XaG8KTdd5iRQUfg=="],
"@changesets/parse": ["@changesets/parse@0.4.3", "", { "dependencies": { "@changesets/types": "^6.1.0", "js-yaml": "^4.1.1" } }, "sha512-ZDmNc53+dXdWEv7fqIUSgRQOLYoUom5Z40gmLgmATmYR9NbL6FJJHwakcCpzaeCy+1D0m0n7mT4jj2B/MQPl7A=="],
"@changesets/pre": ["@changesets/pre@2.0.2", "", { "dependencies": { "@changesets/errors": "^0.2.0", "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3", "fs-extra": "^7.0.1" } }, "sha512-HaL/gEyFVvkf9KFg6484wR9s0qjAXlZ8qWPDkTyKF6+zqjBe/I2mygg3MbpZ++hdi0ToqNUF8cjj7fBy0dg8Ug=="],
"@changesets/read": ["@changesets/read@0.6.7", "", { "dependencies": { "@changesets/git": "^3.0.4", "@changesets/logger": "^0.1.1", "@changesets/parse": "^0.4.3", "@changesets/types": "^6.1.0", "fs-extra": "^7.0.1", "p-filter": "^2.1.0", "picocolors": "^1.1.0" } }, "sha512-D1G4AUYGrBEk8vj8MGwf75k9GpN6XL3wg8i42P2jZZwFLXnlr2Pn7r9yuQNbaMCarP7ZQWNJbV6XLeysAIMhTA=="],
"@changesets/should-skip-package": ["@changesets/should-skip-package@0.1.2", "", { "dependencies": { "@changesets/types": "^6.1.0", "@manypkg/get-packages": "^1.1.3" } }, "sha512-qAK/WrqWLNCP22UDdBTMPH5f41elVDlsNyat180A33dWxuUDyNpg6fPi/FyTZwRriVjg0L8gnjJn2F9XAoF0qw=="],
"@changesets/types": ["@changesets/types@6.1.0", "", {}, "sha512-rKQcJ+o1nKNgeoYRHKOS07tAMNd3YSN0uHaJOZYjBAgxfV7TUE7JE+z4BzZdQwb5hKaYbayKN5KrYV7ODb2rAA=="],
"@changesets/write": ["@changesets/write@0.4.0", "", { "dependencies": { "@changesets/types": "^6.1.0", "fs-extra": "^7.0.1", "human-id": "^4.1.1", "prettier": "^2.7.1" } }, "sha512-CdTLvIOPiCNuH71pyDu3rA+Q0n65cmAbXnwWH84rKGiFumFzkmHNT8KHTMEchcxN+Kl8I54xGUhJ7l3E7X396Q=="],
"@csstools/color-helpers": ["@csstools/color-helpers@6.0.2", "", {}, "sha512-LMGQLS9EuADloEFkcTBR3BwV/CGHV7zyDxVRtVDTwdI2Ca4it0CCVTT9wCkxSgokjE5Ho41hEPgb8OEUwoXr6Q=="],
"@csstools/css-calc": ["@csstools/css-calc@3.2.0", "", { "peerDependencies": { "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-bR9e6o2BDB12jzN/gIbjHa5wLJ4UjD1CB9pM7ehlc0ddk6EBz+yYS1EV2MF55/HUxrHcB/hehAyt5vhsA3hx7w=="],
"@csstools/css-color-parser": ["@csstools/css-color-parser@4.1.0", "", { "dependencies": { "@csstools/color-helpers": "^6.0.2", "@csstools/css-calc": "^3.2.0" }, "peerDependencies": { "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-U0KhLYmy2GVj6q4T3WaAe6NPuFYCPQoE3b0dRGxejWDgcPp8TP7S5rVdM5ZrFaqu4N67X8YaPBw14dQSYx3IyQ=="],
"@csstools/css-parser-algorithms": ["@csstools/css-parser-algorithms@4.0.0", "", { "peerDependencies": { "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-+B87qS7fIG3L5h3qwJ/IFbjoVoOe/bpOdh9hAjXbvx0o8ImEmUsGXN0inFOnk2ChCFgqkkGFQ+TpM5rbhkKe4w=="],
"@csstools/css-syntax-patches-for-csstree": ["@csstools/css-syntax-patches-for-csstree@1.1.3", "", { "peerDependencies": { "css-tree": "^3.2.1" }, "optionalPeers": ["css-tree"] }, "sha512-SH60bMfrRCJF3morcdk57WklujF4Jr/EsQUzqkarfHXEFcAR1gg7fS/chAE922Sehgzc1/+Tz5H3Ypa1HiEKrg=="],
"@csstools/css-tokenizer": ["@csstools/css-tokenizer@4.0.0", "", {}, "sha512-QxULHAm7cNu72w97JUNCBFODFaXpbDg+dP8b/oWFAZ2MTRppA3U00Y2L1HqaS4J6yBqxwa/Y3nMBaxVKbB/NsA=="],
"@exodus/bytes": ["@exodus/bytes@1.15.0", "", { "peerDependencies": { "@noble/hashes": "^1.8.0 || ^2.0.0" }, "optionalPeers": ["@noble/hashes"] }, "sha512-UY0nlA+feH81UGSHv92sLEPLCeZFjXOuHhrIo0HQydScuQc8s0A7kL/UdgwgDq8g8ilksmuoF35YVTNphV2aBQ=="],
"@inquirer/external-editor": ["@inquirer/external-editor@1.0.3", "", { "dependencies": { "chardet": "^2.1.1", "iconv-lite": "^0.7.0" }, "peerDependencies": { "@types/node": ">=18" }, "optionalPeers": ["@types/node"] }, "sha512-RWbSrDiYmO4LbejWY7ttpxczuwQyZLBUyygsA9Nsv95hpzUWwnNTVQmAq3xuh7vNwCp07UTmE5i11XAEExx4RA=="],
"@manypkg/find-root": ["@manypkg/find-root@1.1.0", "", { "dependencies": { "@babel/runtime": "^7.5.5", "@types/node": "^12.7.1", "find-up": "^4.1.0", "fs-extra": "^8.1.0" } }, "sha512-mki5uBvhHzO8kYYix/WRy2WX8S3B5wdVSc9D6KcU5lQNglP2yt58/VfLuAK49glRXChosY8ap2oJ1qgma3GUVA=="],
"@manypkg/get-packages": ["@manypkg/get-packages@1.1.3", "", { "dependencies": { "@babel/runtime": "^7.5.5", "@changesets/types": "^4.0.1", "@manypkg/find-root": "^1.1.0", "fs-extra": "^8.1.0", "globby": "^11.0.0", "read-yaml-file": "^1.1.0" } }, "sha512-fo+QhuU3qE/2TQMQmbVMqaQ6EWbMhi4ABWP+O4AM1NqPBuy0OrApV5LO6BrrgnhtAHS2NH6RrVk9OL181tTi8A=="],
"@mixmark-io/domino": ["@mixmark-io/domino@2.2.0", "", {}, "sha512-Y28PR25bHXUg88kCV7nivXrP2Nj2RueZ3/l/jdx6J9f8J4nsEGcgX0Qe6lt7Pa+J79+kPiJU3LguR6O/6zrLOw=="],
"@mozilla/readability": ["@mozilla/readability@0.6.0", "", {}, "sha512-juG5VWh4qAivzTAeMzvY9xs9HY5rAcr2E4I7tiSSCokRFi7XIZCAu92ZkSTsIj1OPceCifL3cpfteP3pDT9/QQ=="],
"@nodelib/fs.scandir": ["@nodelib/fs.scandir@2.1.5", "", { "dependencies": { "@nodelib/fs.stat": "2.0.5", "run-parallel": "^1.1.9" } }, "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g=="],
"@nodelib/fs.stat": ["@nodelib/fs.stat@2.0.5", "", {}, "sha512-RkhPPp2zrqDAQA/2jNhnztcPAlv64XdhIp7a7454A5ovI7Bukxgt7MX7udwAu3zg1DcpPU0rz3VV1SeaqvY4+A=="],
"@nodelib/fs.walk": ["@nodelib/fs.walk@1.2.8", "", { "dependencies": { "@nodelib/fs.scandir": "2.1.5", "fastq": "^1.6.0" } }, "sha512-oGB+UxlgWcgQkgwo8GcEGwemoTFt3FIO9ababBmaGwXIoBKZ+GTy0pP185beGg7Llih/NSHSV2XAs1lnznocSg=="],
"@types/bun": ["@types/bun@1.3.11", "", { "dependencies": { "bun-types": "1.3.11" } }, "sha512-5vPne5QvtpjGpsGYXiFyycfpDF2ECyPcTSsFBMa0fraoxiQyMJ3SmuQIGhzPg2WJuWxVBoxWJ2kClYTcw/4fAg=="],
"@types/debug": ["@types/debug@4.1.13", "", { "dependencies": { "@types/ms": "*" } }, "sha512-KSVgmQmzMwPlmtljOomayoR89W4FynCAi3E8PPs7vmDVPe84hT+vGPKkJfThkmXs0x0jAaa9U8uW8bbfyS2fWw=="],
"@types/jsdom": ["@types/jsdom@21.1.7", "", { "dependencies": { "@types/node": "*", "@types/tough-cookie": "*", "parse5": "^7.0.0" } }, "sha512-yOriVnggzrnQ3a9OKOCxaVuSug3w3/SbOj5i7VwXWZEyUNl3bLF9V3MfxGbZKuwqJOQyRfqXyROBB1CoZLFWzA=="],
"@types/mdast": ["@types/mdast@4.0.4", "", { "dependencies": { "@types/unist": "*" } }, "sha512-kGaNbPh1k7AFzgpud/gMdvIm5xuECykRR+JnWKQno9TAXVa6WIVCGTPvYGekIDL4uwCZQSYbUxNBSb1aUo79oA=="],
"@types/ms": ["@types/ms@2.1.0", "", {}, "sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA=="],
"@types/node": ["@types/node@25.5.0", "", { "dependencies": { "undici-types": "~7.18.0" } }, "sha512-jp2P3tQMSxWugkCUKLRPVUpGaL5MVFwF8RDuSRztfwgN1wmqJeMSbKlnEtQqU8UrhTmzEmZdu2I6v2dpp7XIxw=="],
"@types/tough-cookie": ["@types/tough-cookie@4.0.5", "", {}, "sha512-/Ad8+nIOV7Rl++6f1BdKxFSMgmoqEoYbHRpPcx3JEfv8VRsQe9Z4mCXeJBzxs7mbHY/XOZZuXlRNfhpVPbs6ZA=="],
"@types/unist": ["@types/unist@3.0.3", "", {}, "sha512-ko/gIFJRv177XgZsZcBwnqJN5x/Gien8qNOn0D5bQU/zAzVf9Zt3BlcUiLqhV9y4ARk0GbT3tnUiPNgnTXzc/Q=="],
"@types/ws": ["@types/ws@8.18.1", "", { "dependencies": { "@types/node": "*" } }, "sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg=="],
"@xmldom/xmldom": ["@xmldom/xmldom@0.8.13", "", {}, "sha512-KRYzxepc14G/CEpEGc3Yn+JKaAeT63smlDr+vjB8jRfgTBBI9wRj/nkQEO+ucV8p8I9bfKLWp37uHgFrbntPvw=="],
"ansi-colors": ["ansi-colors@4.1.3", "", {}, "sha512-/6w/C21Pm1A7aZitlI5Ni/2J6FFQN8i1Cvz3kHABAAbw93v/NlvKdVOqz7CCWz/3iv/JplRSEEZ83XION15ovw=="],
"ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="],
"argparse": ["argparse@2.0.1", "", {}, "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q=="],
"array-union": ["array-union@2.1.0", "", {}, "sha512-HGyxoOTYUyCM6stUe6EJgnd4EoewAI7zMdfqO+kGjnlZmBDz/cR5pf8r/cR4Wq60sL/p0IkcjUEEPwS3GFrIyw=="],
"bail": ["bail@2.0.2", "", {}, "sha512-0xO6mYd7JB2YesxDKplafRpsiOzPt9V02ddPCLbY1xYGPOX24NTyN50qnUxgCPcSoYMhKpAuBTjQoRZCAkUDRw=="],
"better-path-resolve": ["better-path-resolve@1.0.0", "", { "dependencies": { "is-windows": "^1.0.0" } }, "sha512-pbnl5XzGBdrFU/wT4jqmJVPn2B6UHPBOhzMQkY/SPUPB6QtUXtmBHBIwCbXJol93mOpGMnQyP/+BB19q04xj7g=="],
"bidi-js": ["bidi-js@1.0.3", "", { "dependencies": { "require-from-string": "^2.0.2" } }, "sha512-RKshQI1R3YQ+n9YJz2QQ147P66ELpa1FQEg20Dk8oW9t2KgLbpDLLp9aGZ7y8WHSshDknG0bknqGw5/tyCs5tw=="],
"boolbase": ["boolbase@1.0.0", "", {}, "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww=="],
"braces": ["braces@3.0.3", "", { "dependencies": { "fill-range": "^7.1.1" } }, "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA=="],
"bun-types": ["bun-types@1.3.11", "", { "dependencies": { "@types/node": "*" } }, "sha512-1KGPpoxQWl9f6wcZh57LvrPIInQMn2TQ7jsgxqpRzg+l0QPOFvJVH7HmvHo/AiPgwXy+/Thf6Ov3EdVn1vOabg=="],
"ccount": ["ccount@2.0.1", "", {}, "sha512-eyrF0jiFpY+3drT6383f1qhkbGsLSifNAjA61IUjZjmLCWjItY6LB9ft9YhoDgwfmclB2zhu51Lc7+95b8NRAg=="],
"character-entities": ["character-entities@2.0.2", "", {}, "sha512-shx7oQ0Awen/BRIdkjkvz54PnEEI/EjwXDSIZp86/KKdbafHh1Df/RYGBhn4hbe2+uKC9FnT5UCEdyPz3ai9hQ=="],
"chardet": ["chardet@2.1.1", "", {}, "sha512-PsezH1rqdV9VvyNhxxOW32/d75r01NY7TQCmOqomRo15ZSOKbpTFVsfjghxo6JloQUCGnH4k1LGu0R4yCLlWQQ=="],
"chrome-launcher": ["chrome-launcher@1.2.1", "", { "dependencies": { "@types/node": "*", "escape-string-regexp": "^4.0.0", "is-wsl": "^2.2.0", "lighthouse-logger": "^2.0.1" }, "bin": { "print-chrome-path": "bin/print-chrome-path.cjs" } }, "sha512-qmFR5PLMzHyuNJHwOloHPAHhbaNglkfeV/xDtt5b7xiFFyU1I+AZZX0PYseMuhenJSSirgxELYIbswcoc+5H4A=="],
"commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="],
"cross-spawn": ["cross-spawn@7.0.6", "", { "dependencies": { "path-key": "^3.1.0", "shebang-command": "^2.0.0", "which": "^2.0.1" } }, "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA=="],
"css-select": ["css-select@5.2.2", "", { "dependencies": { "boolbase": "^1.0.0", "css-what": "^6.1.0", "domhandler": "^5.0.2", "domutils": "^3.0.1", "nth-check": "^2.0.1" } }, "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw=="],
"css-tree": ["css-tree@3.2.1", "", { "dependencies": { "mdn-data": "2.27.1", "source-map-js": "^1.2.1" } }, "sha512-X7sjQzceUhu1u7Y/ylrRZFU2FS6LRiFVp6rKLPg23y3x3c3DOKAwuXGDp+PAGjh6CSnCjYeAul8pcT8bAl+lSA=="],
"css-what": ["css-what@6.2.2", "", {}, "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA=="],
"cssom": ["cssom@0.5.0", "", {}, "sha512-iKuQcq+NdHqlAcwUY0o/HL69XQrUaQdMjmStJ8JFmUaiiQErlhrmuigkg/CU4E2J0IyUKUrMAgl36TvN67MqTw=="],
"data-urls": ["data-urls@7.0.0", "", { "dependencies": { "whatwg-mimetype": "^5.0.0", "whatwg-url": "^16.0.0" } }, "sha512-23XHcCF+coGYevirZceTVD7NdJOqVn+49IHyxgszm+JIiHLoB2TkmPtsYkNWT1pvRSGkc35L6NHs0yHkN2SumA=="],
"debug": ["debug@4.4.3", "", { "dependencies": { "ms": "^2.1.3" } }, "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA=="],
"decimal.js": ["decimal.js@10.6.0", "", {}, "sha512-YpgQiITW3JXGntzdUmyUR1V812Hn8T1YVXhCu+wO3OpS4eU9l4YdD3qjyiKdV6mvV29zapkMeD390UVEf2lkUg=="],
"decode-named-character-reference": ["decode-named-character-reference@1.3.0", "", { "dependencies": { "character-entities": "^2.0.0" } }, "sha512-GtpQYB283KrPp6nRw50q3U9/VfOutZOe103qlN7BPP6Ad27xYnOIWv4lPzo8HCAL+mMZofJ9KEy30fq6MfaK6Q=="],
"defuddle": ["defuddle@0.17.0", "", { "dependencies": { "commander": "^12.1.0" }, "optionalDependencies": { "linkedom": "^0.18.12", "mathml-to-latex": "^1.5.0", "temml": "^0.13.1", "turndown": "^7.2.0" }, "bin": { "defuddle": "dist/cli.js" } }, "sha512-rjRb9kxgrJLFhFJjXTSAitbmmSyvjEqGCi5JiyM6ImOf4leC/O+f7TPVPAymk/gnENFTrn9j7T/LTfW9IVwiLw=="],
"dequal": ["dequal@2.0.3", "", {}, "sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA=="],
"detect-indent": ["detect-indent@6.1.0", "", {}, "sha512-reYkTUJAZb9gUuZ2RvVCNhVHdg62RHnJ7WJl8ftMi4diZ6NWlciOzQN88pUhSELEwflJht4oQDv0F0BMlwaYtA=="],
"devlop": ["devlop@1.1.0", "", { "dependencies": { "dequal": "^2.0.0" } }, "sha512-RWmIqhcFf1lRYBvNmr7qTNuyCt/7/ns2jbpp1+PalgE/rDQcBT0fioSMUpJ93irlUhC5hrg4cYqe6U+0ImW0rA=="],
"dir-glob": ["dir-glob@3.0.1", "", { "dependencies": { "path-type": "^4.0.0" } }, "sha512-WkrWp9GR4KXfKGYzOLmTuGVi1UWFfws377n9cc55/tb6DuqyF6pcQ5AbiHEshaDpY9v6oaSr2XCDidGmMwdzIA=="],
"dom-serializer": ["dom-serializer@2.0.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.2", "entities": "^4.2.0" } }, "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg=="],
"domelementtype": ["domelementtype@2.3.0", "", {}, "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw=="],
"domhandler": ["domhandler@5.0.3", "", { "dependencies": { "domelementtype": "^2.3.0" } }, "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w=="],
"domutils": ["domutils@3.2.2", "", { "dependencies": { "dom-serializer": "^2.0.0", "domelementtype": "^2.3.0", "domhandler": "^5.0.3" } }, "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw=="],
"enquirer": ["enquirer@2.4.1", "", { "dependencies": { "ansi-colors": "^4.1.1", "strip-ansi": "^6.0.1" } }, "sha512-rRqJg/6gd538VHvR3PSrdRBb/1Vy2YfzHqzvbhGIQpDRKIa4FgV/54b5Q1xYSxOOwKvjXweS26E0Q+nAMwp2pQ=="],
"entities": ["entities@6.0.1", "", {}, "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g=="],
"escape-string-regexp": ["escape-string-regexp@4.0.0", "", {}, "sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA=="],
"esprima": ["esprima@4.0.1", "", { "bin": { "esparse": "./bin/esparse.js", "esvalidate": "./bin/esvalidate.js" } }, "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A=="],
"extend": ["extend@3.0.2", "", {}, "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g=="],
"extendable-error": ["extendable-error@0.1.7", "", {}, "sha512-UOiS2in6/Q0FK0R0q6UY9vYpQ21mr/Qn1KOnte7vsACuNJf514WvCCUHSRCPcgjPT2bAhNIJdlE6bVap1GKmeg=="],
"fast-glob": ["fast-glob@3.3.3", "", { "dependencies": { "@nodelib/fs.stat": "^2.0.2", "@nodelib/fs.walk": "^1.2.3", "glob-parent": "^5.1.2", "merge2": "^1.3.0", "micromatch": "^4.0.8" } }, "sha512-7MptL8U0cqcFdzIzwOTHoilX9x5BrNqye7Z/LuC7kCMRio1EMSyqRK3BEAUD7sXRq4iT4AzTVuZdhgQ2TCvYLg=="],
"fastq": ["fastq@1.20.1", "", { "dependencies": { "reusify": "^1.0.4" } }, "sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw=="],
"fill-range": ["fill-range@7.1.1", "", { "dependencies": { "to-regex-range": "^5.0.1" } }, "sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg=="],
"find-up": ["find-up@4.1.0", "", { "dependencies": { "locate-path": "^5.0.0", "path-exists": "^4.0.0" } }, "sha512-PpOwAdQ/YlXQ2vj8a3h8IipDuYRi3wceVQQGYWxNINccq40Anw7BlsEXCMbt1Zt+OLA6Fq9suIpIWD0OsnISlw=="],
"fs-extra": ["fs-extra@7.0.1", "", { "dependencies": { "graceful-fs": "^4.1.2", "jsonfile": "^4.0.0", "universalify": "^0.1.0" } }, "sha512-YJDaCJZEnBmcbw13fvdAM9AwNOJwOzrE4pqMqBq5nFiEqXUqHwlK4B+3pUw6JNvfSPtX05xFHtYy/1ni01eGCw=="],
"glob-parent": ["glob-parent@5.1.2", "", { "dependencies": { "is-glob": "^4.0.1" } }, "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow=="],
"globby": ["globby@11.1.0", "", { "dependencies": { "array-union": "^2.1.0", "dir-glob": "^3.0.1", "fast-glob": "^3.2.9", "ignore": "^5.2.0", "merge2": "^1.4.1", "slash": "^3.0.0" } }, "sha512-jhIXaOzy1sb8IyocaruWSn1TjmnBVs8Ayhcy83rmxNJ8q2uWKCAj3CnJY+KpGSXCueAPc0i05kVvVKtP1t9S3g=="],
"graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="],
"html-encoding-sniffer": ["html-encoding-sniffer@6.0.0", "", { "dependencies": { "@exodus/bytes": "^1.6.0" } }, "sha512-CV9TW3Y3f8/wT0BRFc1/KAVQ3TUHiXmaAb6VW9vtiMFf7SLoMd1PdAc4W3KFOFETBJUb90KatHqlsZMWV+R9Gg=="],
"html-escaper": ["html-escaper@3.0.3", "", {}, "sha512-RuMffC89BOWQoY0WKGpIhn5gX3iI54O6nRA0yC124NYVtzjmFWBIiFd8M0x+ZdX0P9R4lADg1mgP8C7PxGOWuQ=="],
"htmlparser2": ["htmlparser2@10.1.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.3", "domutils": "^3.2.2", "entities": "^7.0.1" } }, "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ=="],
"human-id": ["human-id@4.1.3", "", { "bin": { "human-id": "dist/cli.js" } }, "sha512-tsYlhAYpjCKa//8rXZ9DqKEawhPoSytweBC2eNvcaDK+57RZLHGqNs3PZTQO6yekLFSuvA6AlnAfrw1uBvtb+Q=="],
"iconv-lite": ["iconv-lite@0.7.2", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw=="],
"ignore": ["ignore@5.3.2", "", {}, "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g=="],
"is-docker": ["is-docker@2.2.1", "", { "bin": { "is-docker": "cli.js" } }, "sha512-F+i2BKsFrH66iaUFc0woD8sLy8getkwTwtOBjvs56Cx4CgJDeKQeqfz8wAYiSb8JOprWhHH5p77PbmYCvvUuXQ=="],
"is-extglob": ["is-extglob@2.1.1", "", {}, "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ=="],
"is-glob": ["is-glob@4.0.3", "", { "dependencies": { "is-extglob": "^2.1.1" } }, "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg=="],
"is-number": ["is-number@7.0.0", "", {}, "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng=="],
"is-plain-obj": ["is-plain-obj@4.1.0", "", {}, "sha512-+Pgi+vMuUNkJyExiMBt5IlFoMyKnr5zhJ4Uspz58WOhBF5QoIZkFyNHIbBAtHwzVAgk5RtndVNsDRN61/mmDqg=="],
"is-potential-custom-element-name": ["is-potential-custom-element-name@1.0.1", "", {}, "sha512-bCYeRA2rVibKZd+s2625gGnGF/t7DSqDs4dP7CrLA1m7jKWz6pps0LpYLJN8Q64HtmPKJ1hrN3nzPNKFEKOUiQ=="],
"is-subdir": ["is-subdir@1.2.0", "", { "dependencies": { "better-path-resolve": "1.0.0" } }, "sha512-2AT6j+gXe/1ueqbW6fLZJiIw3F8iXGJtt0yDrZaBhAZEG1raiTxKWU+IPqMCzQAXOUCKdA4UDMgacKH25XG2Cw=="],
"is-windows": ["is-windows@1.0.2", "", {}, "sha512-eXK1UInq2bPmjyX6e3VHIzMLobc4J94i4AWn+Hpq3OU5KkrRC96OAcR3PRJ/pGu6m8TRnBHP9dkXQVsT/COVIA=="],
"is-wsl": ["is-wsl@2.2.0", "", { "dependencies": { "is-docker": "^2.0.0" } }, "sha512-fKzAra0rGJUUBwGBgNkHZuToZcn+TtXHpeCgmkMJMMYx1sQDYaCSyjJBSCa2nH1DGm7s3n1oBnohoVTBaN7Lww=="],
"isexe": ["isexe@2.0.0", "", {}, "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw=="],
"js-yaml": ["js-yaml@4.1.1", "", { "dependencies": { "argparse": "^2.0.1" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA=="],
"jsdom": ["jsdom@29.0.2", "", { "dependencies": { "@asamuzakjp/css-color": "^5.1.5", "@asamuzakjp/dom-selector": "^7.0.6", "@bramus/specificity": "^2.4.2", "@csstools/css-syntax-patches-for-csstree": "^1.1.1", "@exodus/bytes": "^1.15.0", "css-tree": "^3.2.1", "data-urls": "^7.0.0", "decimal.js": "^10.6.0", "html-encoding-sniffer": "^6.0.0", "is-potential-custom-element-name": "^1.0.1", "lru-cache": "^11.2.7", "parse5": "^8.0.0", "saxes": "^6.0.0", "symbol-tree": "^3.2.4", "tough-cookie": "^6.0.1", "undici": "^7.24.5", "w3c-xmlserializer": "^5.0.0", "webidl-conversions": "^8.0.1", "whatwg-mimetype": "^5.0.0", "whatwg-url": "^16.0.1", "xml-name-validator": "^5.0.0" }, "peerDependencies": { "canvas": "^3.0.0" }, "optionalPeers": ["canvas"] }, "sha512-9VnGEBosc/ZpwyOsJBCQ/3I5p7Q5ngOY14a9bf5btenAORmZfDse1ZEheMiWcJ3h81+Fv7HmJFdS0szo/waF2w=="],
"jsonfile": ["jsonfile@4.0.0", "", { "optionalDependencies": { "graceful-fs": "^4.1.6" } }, "sha512-m6F1R3z8jjlf2imQHS2Qez5sjKWQzbuuhuJ/FKYFRZvPE3PuHcSMVZzfsLhGVOkfd20obL5SWEBew5ShlquNxg=="],
"lighthouse-logger": ["lighthouse-logger@2.0.2", "", { "dependencies": { "debug": "^4.4.1", "marky": "^1.2.2" } }, "sha512-vWl2+u5jgOQuZR55Z1WM0XDdrJT6mzMP8zHUct7xTlWhuQs+eV0g+QL0RQdFjT54zVmbhLCP8vIVpy1wGn/gCg=="],
"linkedom": ["linkedom@0.18.12", "", { "dependencies": { "css-select": "^5.1.0", "cssom": "^0.5.0", "html-escaper": "^3.0.3", "htmlparser2": "^10.0.0", "uhyphen": "^0.2.0" }, "peerDependencies": { "canvas": ">= 2" }, "optionalPeers": ["canvas"] }, "sha512-jalJsOwIKuQJSeTvsgzPe9iJzyfVaEJiEXl+25EkKevsULHvMJzpNqwvj1jOESWdmgKDiXObyjOYwlUqG7wo1Q=="],
"locate-path": ["locate-path@5.0.0", "", { "dependencies": { "p-locate": "^4.1.0" } }, "sha512-t7hw9pI+WvuwNJXwk5zVHpyhIqzg2qTlklJOf0mVxGSbe3Fp2VieZcduNYjaLDoy6p9uGpQEGWG87WpMKlNq8g=="],
"lodash.startcase": ["lodash.startcase@4.4.0", "", {}, "sha512-+WKqsK294HMSc2jEbNgpHpd0JfIBhp7rEV4aqXWqFr6AlXov+SlcgB1Fv01y2kGe3Gc8nMW7VA0SrGuSkRfIEg=="],
"longest-streak": ["longest-streak@3.1.0", "", {}, "sha512-9Ri+o0JYgehTaVBBDoMqIl8GXtbWg711O3srftcHhZ0dqnETqLaoIK0x17fUw9rFSlK/0NlsKe0Ahhyl5pXE2g=="],
"lru-cache": ["lru-cache@11.3.5", "", {}, "sha512-NxVFwLAnrd9i7KUBxC4DrUhmgjzOs+1Qm50D3oF1/oL+r1NpZ4gA7xvG0/zJ8evR7zIKn4vLf7qTNduWFtCrRw=="],
"markdown-table": ["markdown-table@3.0.4", "", {}, "sha512-wiYz4+JrLyb/DqW2hkFJxP7Vd7JuTDm77fvbM8VfEQdmSMqcImWeeRbHwZjBjIFki/VaMK2BhFi7oUUZeM5bqw=="],
"marky": ["marky@1.3.0", "", {}, "sha512-ocnPZQLNpvbedwTy9kNrQEsknEfgvcLMvOtz3sFeWApDq1MXH1TqkCIx58xlpESsfwQOnuBO9beyQuNGzVvuhQ=="],
"mathml-to-latex": ["mathml-to-latex@1.5.0", "", { "dependencies": { "@xmldom/xmldom": "^0.8.10" } }, "sha512-rrWn0eEvcEcdMM4xfHcSGIy+i01DX9byOdXTLWg+w1iJ6O6ohP5UXY1dVzNUZLhzfl3EGcRekWLhY7JT5Omaew=="],
"mdast-util-find-and-replace": ["mdast-util-find-and-replace@3.0.2", "", { "dependencies": { "@types/mdast": "^4.0.0", "escape-string-regexp": "^5.0.0", "unist-util-is": "^6.0.0", "unist-util-visit-parents": "^6.0.0" } }, "sha512-Tmd1Vg/m3Xz43afeNxDIhWRtFZgM2VLyaf4vSTYwudTyeuTneoL3qtWMA5jeLyz/O1vDJmmV4QuScFCA2tBPwg=="],
"mdast-util-from-markdown": ["mdast-util-from-markdown@2.0.3", "", { "dependencies": { "@types/mdast": "^4.0.0", "@types/unist": "^3.0.0", "decode-named-character-reference": "^1.0.0", "devlop": "^1.0.0", "mdast-util-to-string": "^4.0.0", "micromark": "^4.0.0", "micromark-util-decode-numeric-character-reference": "^2.0.0", "micromark-util-decode-string": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0", "unist-util-stringify-position": "^4.0.0" } }, "sha512-W4mAWTvSlKvf8L6J+VN9yLSqQ9AOAAvHuoDAmPkz4dHf553m5gVj2ejadHJhoJmcmxEnOv6Pa8XJhpxE93kb8Q=="],
"mdast-util-gfm": ["mdast-util-gfm@3.1.0", "", { "dependencies": { "mdast-util-from-markdown": "^2.0.0", "mdast-util-gfm-autolink-literal": "^2.0.0", "mdast-util-gfm-footnote": "^2.0.0", "mdast-util-gfm-strikethrough": "^2.0.0", "mdast-util-gfm-table": "^2.0.0", "mdast-util-gfm-task-list-item": "^2.0.0", "mdast-util-to-markdown": "^2.0.0" } }, "sha512-0ulfdQOM3ysHhCJ1p06l0b0VKlhU0wuQs3thxZQagjcjPrlFRqY215uZGHHJan9GEAXd9MbfPjFJz+qMkVR6zQ=="],
"mdast-util-gfm-autolink-literal": ["mdast-util-gfm-autolink-literal@2.0.1", "", { "dependencies": { "@types/mdast": "^4.0.0", "ccount": "^2.0.0", "devlop": "^1.0.0", "mdast-util-find-and-replace": "^3.0.0", "micromark-util-character": "^2.0.0" } }, "sha512-5HVP2MKaP6L+G6YaxPNjuL0BPrq9orG3TsrZ9YXbA3vDw/ACI4MEsnoDpn6ZNm7GnZgtAcONJyPhOP8tNJQavQ=="],
"mdast-util-gfm-footnote": ["mdast-util-gfm-footnote@2.1.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "devlop": "^1.1.0", "mdast-util-from-markdown": "^2.0.0", "mdast-util-to-markdown": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0" } }, "sha512-sqpDWlsHn7Ac9GNZQMeUzPQSMzR6Wv0WKRNvQRg0KqHh02fpTz69Qc1QSseNX29bhz1ROIyNyxExfawVKTm1GQ=="],
"mdast-util-gfm-strikethrough": ["mdast-util-gfm-strikethrough@2.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "mdast-util-from-markdown": "^2.0.0", "mdast-util-to-markdown": "^2.0.0" } }, "sha512-mKKb915TF+OC5ptj5bJ7WFRPdYtuHv0yTRxK2tJvi+BDqbkiG7h7u/9SI89nRAYcmap2xHQL9D+QG/6wSrTtXg=="],
"mdast-util-gfm-table": ["mdast-util-gfm-table@2.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "devlop": "^1.0.0", "markdown-table": "^3.0.0", "mdast-util-from-markdown": "^2.0.0", "mdast-util-to-markdown": "^2.0.0" } }, "sha512-78UEvebzz/rJIxLvE7ZtDd/vIQ0RHv+3Mh5DR96p7cS7HsBhYIICDBCu8csTNWNO6tBWfqXPWekRuj2FNOGOZg=="],
"mdast-util-gfm-task-list-item": ["mdast-util-gfm-task-list-item@2.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "devlop": "^1.0.0", "mdast-util-from-markdown": "^2.0.0", "mdast-util-to-markdown": "^2.0.0" } }, "sha512-IrtvNvjxC1o06taBAVJznEnkiHxLFTzgonUdy8hzFVeDun0uTjxxrRGVaNFqkU1wJR3RBPEfsxmU6jDWPofrTQ=="],
"mdast-util-phrasing": ["mdast-util-phrasing@4.1.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "unist-util-is": "^6.0.0" } }, "sha512-TqICwyvJJpBwvGAMZjj4J2n0X8QWp21b9l0o7eXyVJ25YNWYbJDVIyD1bZXE6WtV6RmKJVYmQAKWa0zWOABz2w=="],
"mdast-util-to-markdown": ["mdast-util-to-markdown@2.1.2", "", { "dependencies": { "@types/mdast": "^4.0.0", "@types/unist": "^3.0.0", "longest-streak": "^3.0.0", "mdast-util-phrasing": "^4.0.0", "mdast-util-to-string": "^4.0.0", "micromark-util-classify-character": "^2.0.0", "micromark-util-decode-string": "^2.0.0", "unist-util-visit": "^5.0.0", "zwitch": "^2.0.0" } }, "sha512-xj68wMTvGXVOKonmog6LwyJKrYXZPvlwabaryTjLh9LuvovB/KAH+kvi8Gjj+7rJjsFi23nkUxRQv1KqSroMqA=="],
"mdast-util-to-string": ["mdast-util-to-string@4.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0" } }, "sha512-0H44vDimn51F0YwvxSJSm0eCDOJTRlmN0R1yBh4HLj9wiV1Dn0QoXGbvFAWj2hSItVTlCmBF1hqKlIyUBVFLPg=="],
"mdn-data": ["mdn-data@2.27.1", "", {}, "sha512-9Yubnt3e8A0OKwxYSXyhLymGW4sCufcLG6VdiDdUGVkPhpqLxlvP5vl1983gQjJl3tqbrM731mjaZaP68AgosQ=="],
"merge2": ["merge2@1.4.1", "", {}, "sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg=="],
"micromark": ["micromark@4.0.2", "", { "dependencies": { "@types/debug": "^4.0.0", "debug": "^4.0.0", "decode-named-character-reference": "^1.0.0", "devlop": "^1.0.0", "micromark-core-commonmark": "^2.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-combine-extensions": "^2.0.0", "micromark-util-decode-numeric-character-reference": "^2.0.0", "micromark-util-encode": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-resolve-all": "^2.0.0", "micromark-util-sanitize-uri": "^2.0.0", "micromark-util-subtokenize": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-zpe98Q6kvavpCr1NPVSCMebCKfD7CA2NqZ+rykeNhONIJBpc1tFKt9hucLGwha3jNTNI8lHpctWJWoimVF4PfA=="],
"micromark-core-commonmark": ["micromark-core-commonmark@2.0.3", "", { "dependencies": { "decode-named-character-reference": "^1.0.0", "devlop": "^1.0.0", "micromark-factory-destination": "^2.0.0", "micromark-factory-label": "^2.0.0", "micromark-factory-space": "^2.0.0", "micromark-factory-title": "^2.0.0", "micromark-factory-whitespace": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-classify-character": "^2.0.0", "micromark-util-html-tag-name": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-resolve-all": "^2.0.0", "micromark-util-subtokenize": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-RDBrHEMSxVFLg6xvnXmb1Ayr2WzLAWjeSATAoxwKYJV94TeNavgoIdA0a9ytzDSVzBy2YKFK+emCPOEibLeCrg=="],
"micromark-extension-gfm": ["micromark-extension-gfm@3.0.0", "", { "dependencies": { "micromark-extension-gfm-autolink-literal": "^2.0.0", "micromark-extension-gfm-footnote": "^2.0.0", "micromark-extension-gfm-strikethrough": "^2.0.0", "micromark-extension-gfm-table": "^2.0.0", "micromark-extension-gfm-tagfilter": "^2.0.0", "micromark-extension-gfm-task-list-item": "^2.0.0", "micromark-util-combine-extensions": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-vsKArQsicm7t0z2GugkCKtZehqUm31oeGBV/KVSorWSy8ZlNAv7ytjFhvaryUiCUJYqs+NoE6AFhpQvBTM6Q4w=="],
"micromark-extension-gfm-autolink-literal": ["micromark-extension-gfm-autolink-literal@2.1.0", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-sanitize-uri": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-oOg7knzhicgQ3t4QCjCWgTmfNhvQbDDnJeVu9v81r7NltNCVmhPy1fJRX27pISafdjL+SVc4d3l48Gb6pbRypw=="],
"micromark-extension-gfm-footnote": ["micromark-extension-gfm-footnote@2.1.0", "", { "dependencies": { "devlop": "^1.0.0", "micromark-core-commonmark": "^2.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-normalize-identifier": "^2.0.0", "micromark-util-sanitize-uri": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-/yPhxI1ntnDNsiHtzLKYnE3vf9JZ6cAisqVDauhp4CEHxlb4uoOTxOCJ+9s51bIB8U1N1FJ1RXOKTIlD5B/gqw=="],
"micromark-extension-gfm-strikethrough": ["micromark-extension-gfm-strikethrough@2.1.0", "", { "dependencies": { "devlop": "^1.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-classify-character": "^2.0.0", "micromark-util-resolve-all": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-ADVjpOOkjz1hhkZLlBiYA9cR2Anf8F4HqZUO6e5eDcPQd0Txw5fxLzzxnEkSkfnD0wziSGiv7sYhk/ktvbf1uw=="],
"micromark-extension-gfm-table": ["micromark-extension-gfm-table@2.1.1", "", { "dependencies": { "devlop": "^1.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-t2OU/dXXioARrC6yWfJ4hqB7rct14e8f7m0cbI5hUmDyyIlwv5vEtooptH8INkbLzOatzKuVbQmAYcbWoyz6Dg=="],
"micromark-extension-gfm-tagfilter": ["micromark-extension-gfm-tagfilter@2.0.0", "", { "dependencies": { "micromark-util-types": "^2.0.0" } }, "sha512-xHlTOmuCSotIA8TW1mDIM6X2O1SiX5P9IuDtqGonFhEK0qgRI4yeC6vMxEV2dgyr2TiD+2PQ10o+cOhdVAcwfg=="],
"micromark-extension-gfm-task-list-item": ["micromark-extension-gfm-task-list-item@2.1.0", "", { "dependencies": { "devlop": "^1.0.0", "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-qIBZhqxqI6fjLDYFTBIa4eivDMnP+OZqsNwmQ3xNLE4Cxwc+zfQEfbs6tzAo2Hjq+bh6q5F+Z8/cksrLFYWQQw=="],
"micromark-factory-destination": ["micromark-factory-destination@2.0.1", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-Xe6rDdJlkmbFRExpTOmRj9N3MaWmbAgdpSrBQvCFqhezUn4AHqJHbaEnfbVYYiexVSs//tqOdY/DxhjdCiJnIA=="],
"micromark-factory-label": ["micromark-factory-label@2.0.1", "", { "dependencies": { "devlop": "^1.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-VFMekyQExqIW7xIChcXn4ok29YE3rnuyveW3wZQWWqF4Nv9Wk5rgJ99KzPvHjkmPXF93FXIbBp6YdW3t71/7Vg=="],
"micromark-factory-space": ["micromark-factory-space@2.0.1", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-zRkxjtBxxLd2Sc0d+fbnEunsTj46SWXgXciZmHq0kDYGnck/ZSGj9/wULTV95uoeYiK5hRXP2mJ98Uo4cq/LQg=="],
"micromark-factory-title": ["micromark-factory-title@2.0.1", "", { "dependencies": { "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-5bZ+3CjhAd9eChYTHsjy6TGxpOFSKgKKJPJxr293jTbfry2KDoWkhBb6TcPVB4NmzaPhMs1Frm9AZH7OD4Cjzw=="],
"micromark-factory-whitespace": ["micromark-factory-whitespace@2.0.1", "", { "dependencies": { "micromark-factory-space": "^2.0.0", "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-Ob0nuZ3PKt/n0hORHyvoD9uZhr+Za8sFoP+OnMcnWK5lngSzALgQYKMr9RJVOWLqQYuyn6ulqGWSXdwf6F80lQ=="],
"micromark-util-character": ["micromark-util-character@2.1.1", "", { "dependencies": { "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-wv8tdUTJ3thSFFFJKtpYKOYiGP2+v96Hvk4Tu8KpCAsTMs6yi+nVmGh1syvSCsaxz45J6Jbw+9DD6g97+NV67Q=="],
"micromark-util-chunked": ["micromark-util-chunked@2.0.1", "", { "dependencies": { "micromark-util-symbol": "^2.0.0" } }, "sha512-QUNFEOPELfmvv+4xiNg2sRYeS/P84pTW0TCgP5zc9FpXetHY0ab7SxKyAQCNCc1eK0459uoLI1y5oO5Vc1dbhA=="],
"micromark-util-classify-character": ["micromark-util-classify-character@2.0.1", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-K0kHzM6afW/MbeWYWLjoHQv1sgg2Q9EccHEDzSkxiP/EaagNzCm7T/WMKZ3rjMbvIpvBiZgwR3dKMygtA4mG1Q=="],
"micromark-util-combine-extensions": ["micromark-util-combine-extensions@2.0.1", "", { "dependencies": { "micromark-util-chunked": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-OnAnH8Ujmy59JcyZw8JSbK9cGpdVY44NKgSM7E9Eh7DiLS2E9RNQf0dONaGDzEG9yjEl5hcqeIsj4hfRkLH/Bg=="],
"micromark-util-decode-numeric-character-reference": ["micromark-util-decode-numeric-character-reference@2.0.2", "", { "dependencies": { "micromark-util-symbol": "^2.0.0" } }, "sha512-ccUbYk6CwVdkmCQMyr64dXz42EfHGkPQlBj5p7YVGzq8I7CtjXZJrubAYezf7Rp+bjPseiROqe7G6foFd+lEuw=="],
"micromark-util-decode-string": ["micromark-util-decode-string@2.0.1", "", { "dependencies": { "decode-named-character-reference": "^1.0.0", "micromark-util-character": "^2.0.0", "micromark-util-decode-numeric-character-reference": "^2.0.0", "micromark-util-symbol": "^2.0.0" } }, "sha512-nDV/77Fj6eH1ynwscYTOsbK7rR//Uj0bZXBwJZRfaLEJ1iGBR6kIfNmlNqaqJf649EP0F3NWNdeJi03elllNUQ=="],
"micromark-util-encode": ["micromark-util-encode@2.0.1", "", {}, "sha512-c3cVx2y4KqUnwopcO9b/SCdo2O67LwJJ/UyqGfbigahfegL9myoEFoDYZgkT7f36T0bLrM9hZTAaAyH+PCAXjw=="],
"micromark-util-html-tag-name": ["micromark-util-html-tag-name@2.0.1", "", {}, "sha512-2cNEiYDhCWKI+Gs9T0Tiysk136SnR13hhO8yW6BGNyhOC4qYFnwF1nKfD3HFAIXA5c45RrIG1ub11GiXeYd1xA=="],
"micromark-util-normalize-identifier": ["micromark-util-normalize-identifier@2.0.1", "", { "dependencies": { "micromark-util-symbol": "^2.0.0" } }, "sha512-sxPqmo70LyARJs0w2UclACPUUEqltCkJ6PhKdMIDuJ3gSf/Q+/GIe3WKl0Ijb/GyH9lOpUkRAO2wp0GVkLvS9Q=="],
"micromark-util-resolve-all": ["micromark-util-resolve-all@2.0.1", "", { "dependencies": { "micromark-util-types": "^2.0.0" } }, "sha512-VdQyxFWFT2/FGJgwQnJYbe1jjQoNTS4RjglmSjTUlpUMa95Htx9NHeYW4rGDJzbjvCsl9eLjMQwGeElsqmzcHg=="],
"micromark-util-sanitize-uri": ["micromark-util-sanitize-uri@2.0.1", "", { "dependencies": { "micromark-util-character": "^2.0.0", "micromark-util-encode": "^2.0.0", "micromark-util-symbol": "^2.0.0" } }, "sha512-9N9IomZ/YuGGZZmQec1MbgxtlgougxTodVwDzzEouPKo3qFWvymFHWcnDi2vzV1ff6kas9ucW+o3yzJK9YB1AQ=="],
"micromark-util-subtokenize": ["micromark-util-subtokenize@2.1.0", "", { "dependencies": { "devlop": "^1.0.0", "micromark-util-chunked": "^2.0.0", "micromark-util-symbol": "^2.0.0", "micromark-util-types": "^2.0.0" } }, "sha512-XQLu552iSctvnEcgXw6+Sx75GflAPNED1qx7eBJ+wydBb2KCbRZe+NwvIEEMM83uml1+2WSXpBAcp9IUCgCYWA=="],
"micromark-util-symbol": ["micromark-util-symbol@2.0.1", "", {}, "sha512-vs5t8Apaud9N28kgCrRUdEed4UJ+wWNvicHLPxCa9ENlYuAY31M0ETy5y1vA33YoNPDFTghEbnh6efaE8h4x0Q=="],
"micromark-util-types": ["micromark-util-types@2.0.2", "", {}, "sha512-Yw0ECSpJoViF1qTU4DC6NwtC4aWGt1EkzaQB8KPPyCRR8z9TWeV0HbEFGTO+ZY1wB22zmxnJqhPyTpOVCpeHTA=="],
"micromatch": ["micromatch@4.0.8", "", { "dependencies": { "braces": "^3.0.3", "picomatch": "^2.3.1" } }, "sha512-PXwfBhYu0hBCPw8Dn0E+WDYb7af3dSLVWKi3HGv84IdF4TyFoC0ysxFd0Goxw7nSv4T/PzEJQxsYsEiFCKo2BA=="],
"mri": ["mri@1.2.0", "", {}, "sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA=="],
"ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="],
"nth-check": ["nth-check@2.1.1", "", { "dependencies": { "boolbase": "^1.0.0" } }, "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w=="],
"outdent": ["outdent@0.5.0", "", {}, "sha512-/jHxFIzoMXdqPzTaCpFzAAWhpkSjZPF4Vsn6jAfNpmbH/ymsmd7Qc6VE9BGn0L6YMj6uwpQLxCECpus4ukKS9Q=="],
"p-filter": ["p-filter@2.1.0", "", { "dependencies": { "p-map": "^2.0.0" } }, "sha512-ZBxxZ5sL2HghephhpGAQdoskxplTwr7ICaehZwLIlfL6acuVgZPm8yBNuRAFBGEqtD/hmUeq9eqLg2ys9Xr/yw=="],
"p-limit": ["p-limit@2.3.0", "", { "dependencies": { "p-try": "^2.0.0" } }, "sha512-//88mFWSJx8lxCzwdAABTJL2MyWB12+eIY7MDL2SqLmAkeKU9qxRvWuSyTjm3FUmpBEMuFfckAIqEaVGUDxb6w=="],
"p-locate": ["p-locate@4.1.0", "", { "dependencies": { "p-limit": "^2.2.0" } }, "sha512-R79ZZ/0wAxKGu3oYMlz8jy/kbhsNrS7SKZ7PxEHBgJ5+F2mtFW2fK2cOtBh1cHYkQsbzFV7I+EoRKe6Yt0oK7A=="],
"p-map": ["p-map@2.1.0", "", {}, "sha512-y3b8Kpd8OAN444hxfBbFfj1FY/RjtTd8tzYwhUqNYXx0fXx2iX4maP4Qr6qhIKbQXI02wTLAda4fYUbDagTUFw=="],
"p-try": ["p-try@2.2.0", "", {}, "sha512-R4nPAVTAU0B9D35/Gk3uJf/7XYbQcyohSKdvAxIRSNghFl4e71hVoGnBNQz9cWaXxO2I10KTC+3jMdvvoKw6dQ=="],
"package-manager-detector": ["package-manager-detector@0.2.11", "", { "dependencies": { "quansync": "^0.2.7" } }, "sha512-BEnLolu+yuz22S56CU1SUKq3XC3PkwD5wv4ikR4MfGvnRVcmzXR9DwSlW2fEamyTPyXHomBJRzgapeuBvRNzJQ=="],
"parse5": ["parse5@7.3.0", "", { "dependencies": { "entities": "^6.0.0" } }, "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw=="],
"path-exists": ["path-exists@4.0.0", "", {}, "sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w=="],
"path-key": ["path-key@3.1.1", "", {}, "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q=="],
"path-type": ["path-type@4.0.0", "", {}, "sha512-gDKb8aZMDeD/tZWs9P6+q0J9Mwkdl6xMV8TjnGP3qJVJ06bdMgkbBlLU8IdfOsIsFz2BW1rNVT3XuNEl8zPAvw=="],
"picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="],
"picomatch": ["picomatch@2.3.2", "", {}, "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA=="],
"pify": ["pify@4.0.1", "", {}, "sha512-uB80kBFb/tfd68bVleG9T5GGsGPjJrLAUpR5PZIrhBnIaRTQRjqdJSsIKkOP6OAIFbj7GOrcudc5pNjZ+geV2g=="],
"prettier": ["prettier@2.8.8", "", { "bin": { "prettier": "bin-prettier.js" } }, "sha512-tdN8qQGvNjw4CHbY+XXk0JgCXn9QiF21a55rBe5LJAU+kDyC4WQn4+awm2Xfk2lQMk5fKup9XgzTZtGkjBdP9Q=="],
"punycode": ["punycode@2.3.1", "", {}, "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg=="],
"quansync": ["quansync@0.2.11", "", {}, "sha512-AifT7QEbW9Nri4tAwR5M/uzpBuqfZf+zwaEM/QkzEjj7NBuFD2rBuy0K3dE+8wltbezDV7JMA0WfnCPYRSYbXA=="],
"queue-microtask": ["queue-microtask@1.2.3", "", {}, "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A=="],
"read-yaml-file": ["read-yaml-file@1.1.0", "", { "dependencies": { "graceful-fs": "^4.1.5", "js-yaml": "^3.6.1", "pify": "^4.0.1", "strip-bom": "^3.0.0" } }, "sha512-VIMnQi/Z4HT2Fxuwg5KrY174U1VdUIASQVWXXyqtNRtxSr9IYkn1rsI6Tb6HsrHCmB7gVpNwX6JxPTHcH6IoTA=="],
"remark-gfm": ["remark-gfm@4.0.1", "", { "dependencies": { "@types/mdast": "^4.0.0", "mdast-util-gfm": "^3.0.0", "micromark-extension-gfm": "^3.0.0", "remark-parse": "^11.0.0", "remark-stringify": "^11.0.0", "unified": "^11.0.0" } }, "sha512-1quofZ2RQ9EWdeN34S79+KExV1764+wCUGop5CPL1WGdD0ocPpu91lzPGbwWMECpEpd42kJGQwzRfyov9j4yNg=="],
"remark-parse": ["remark-parse@11.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "mdast-util-from-markdown": "^2.0.0", "micromark-util-types": "^2.0.0", "unified": "^11.0.0" } }, "sha512-FCxlKLNGknS5ba/1lmpYijMUzX2esxW5xQqjWxw2eHFfS2MSdaHVINFmhjo+qN1WhZhNimq0dZATN9pH0IDrpA=="],
"remark-stringify": ["remark-stringify@11.0.0", "", { "dependencies": { "@types/mdast": "^4.0.0", "mdast-util-to-markdown": "^2.0.0", "unified": "^11.0.0" } }, "sha512-1OSmLd3awB/t8qdoEOMazZkNsfVTeY4fTsgzcQFdXNq8ToTN4ZGwrMnlda4K6smTFKD+GRV6O48i6Z4iKgPPpw=="],
"require-from-string": ["require-from-string@2.0.2", "", {}, "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw=="],
"resolve-from": ["resolve-from@5.0.0", "", {}, "sha512-qYg9KP24dD5qka9J47d0aVky0N+b4fTU89LN9iDnjB5waksiC49rvMB0PrUJQGoTmH50XPiqOvAjDfaijGxYZw=="],
"reusify": ["reusify@1.1.0", "", {}, "sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw=="],
"run-parallel": ["run-parallel@1.2.0", "", { "dependencies": { "queue-microtask": "^1.2.2" } }, "sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA=="],
"safer-buffer": ["safer-buffer@2.1.2", "", {}, "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg=="],
"saxes": ["saxes@6.0.0", "", { "dependencies": { "xmlchars": "^2.2.0" } }, "sha512-xAg7SOnEhrm5zI3puOOKyy1OMcMlIJZYNJY7xLBwSze0UjhPLnWfj2GF2EpT0jmzaJKIWKHLsaSSajf35bcYnA=="],
"semver": ["semver@7.7.4", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-vFKC2IEtQnVhpT78h1Yp8wzwrf8CM+MzKMHGJZfBtzhZNycRFnXsHk6E5TxIkkMsgNS7mdX3AGB7x2QM2di4lA=="],
"shebang-command": ["shebang-command@2.0.0", "", { "dependencies": { "shebang-regex": "^3.0.0" } }, "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA=="],
"shebang-regex": ["shebang-regex@3.0.0", "", {}, "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A=="],
"signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
"slash": ["slash@3.0.0", "", {}, "sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q=="],
"source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="],
"spawndamnit": ["spawndamnit@3.0.1", "", { "dependencies": { "cross-spawn": "^7.0.5", "signal-exit": "^4.0.1" } }, "sha512-MmnduQUuHCoFckZoWnXsTg7JaiLBJrKFj9UI2MbRPGaJeVpsLcVBu6P/IGZovziM/YBsellCmsprgNA+w0CzVg=="],
"sprintf-js": ["sprintf-js@1.0.3", "", {}, "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g=="],
"strip-ansi": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="],
"strip-bom": ["strip-bom@3.0.0", "", {}, "sha512-vavAMRXOgBVNF6nyEEmL3DBK19iRpDcoIwW+swQ+CbGiu7lju6t+JklA1MHweoWtadgt4ISVUsXLyDq34ddcwA=="],
"symbol-tree": ["symbol-tree@3.2.4", "", {}, "sha512-9QNk5KwDF+Bvz+PyObkmSYjI5ksVUYtjW7AU22r2NKcfLJcXp96hkDWU3+XndOsUb+AQ9QhfzfCT2O+CNWT5Tw=="],
"temml": ["temml@0.13.2", "", {}, "sha512-n8fDRSsLscq9nh9j6z+FgkCvFMT0IJm6GCgwfzh+7AHT3Sfb4jFTQlsA6hVcF2dYYr3b66oDBVES95RfoukyrA=="],
"term-size": ["term-size@2.2.1", "", {}, "sha512-wK0Ri4fOGjv/XPy8SBHZChl8CM7uMc5VML7SqiQ0zG7+J5Vr+RMQDoHa2CNT6KHUnTGIXH34UDMkPzAUyapBZg=="],
"tldts": ["tldts@7.0.28", "", { "dependencies": { "tldts-core": "^7.0.28" }, "bin": { "tldts": "bin/cli.js" } }, "sha512-+Zg3vWhRUv8B1maGSTFdev9mjoo8Etn2Ayfs4cnjlD3CsGkxXX4QyW3j2WJ0wdjYcYmy7Lx2RDsZMhgCWafKIw=="],
"tldts-core": ["tldts-core@7.0.28", "", {}, "sha512-7W5Efjhsc3chVdFhqtaU0KtK32J37Zcr9RKtID54nG+tIpcY79CQK/veYPODxtD/LJ4Lue66jvrQzIX2Z2/pUQ=="],
"to-regex-range": ["to-regex-range@5.0.1", "", { "dependencies": { "is-number": "^7.0.0" } }, "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ=="],
"tough-cookie": ["tough-cookie@6.0.1", "", { "dependencies": { "tldts": "^7.0.5" } }, "sha512-LktZQb3IeoUWB9lqR5EWTHgW/VTITCXg4D21M+lvybRVdylLrRMnqaIONLVb5mav8vM19m44HIcGq4qASeu2Qw=="],
"tr46": ["tr46@6.0.0", "", { "dependencies": { "punycode": "^2.3.1" } }, "sha512-bLVMLPtstlZ4iMQHpFHTR7GAGj2jxi8Dg0s2h2MafAE4uSWF98FC/3MomU51iQAMf8/qDUbKWf5GxuvvVcXEhw=="],
"trough": ["trough@2.2.0", "", {}, "sha512-tmMpK00BjZiUyVyvrBK7knerNgmgvcV/KLVyuma/SC+TQN167GrMRciANTz09+k3zW8L8t60jWO1GpfkZdjTaw=="],
"turndown": ["turndown@7.2.2", "", { "dependencies": { "@mixmark-io/domino": "^2.2.0" } }, "sha512-1F7db8BiExOKxjSMU2b7if62D/XOyQyZbPKq/nUwopfgnHlqXHqQ0lvfUTeUIr1lZJzOPFn43dODyMSIfvWRKQ=="],
"turndown-plugin-gfm": ["turndown-plugin-gfm@1.0.2", "", {}, "sha512-vwz9tfvF7XN/jE0dGoBei3FXWuvll78ohzCZQuOb+ZjWrs3a0XhQVomJEb2Qh4VHTPNRO4GPZh0V7VRbiWwkRg=="],
"typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="],
"uhyphen": ["uhyphen@0.2.0", "", {}, "sha512-qz3o9CHXmJJPGBdqzab7qAYuW8kQGKNEuoHFYrBwV6hWIMcpAmxDLXojcHfFr9US1Pe6zUswEIJIbLI610fuqA=="],
"undici": ["undici@7.25.0", "", {}, "sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ=="],
"undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
"unified": ["unified@11.0.5", "", { "dependencies": { "@types/unist": "^3.0.0", "bail": "^2.0.0", "devlop": "^1.0.0", "extend": "^3.0.0", "is-plain-obj": "^4.0.0", "trough": "^2.0.0", "vfile": "^6.0.0" } }, "sha512-xKvGhPWw3k84Qjh8bI3ZeJjqnyadK+GEFtazSfZv/rKeTkTjOJho6mFqh2SM96iIcZokxiOpg78GazTSg8+KHA=="],
"unist-util-is": ["unist-util-is@6.0.1", "", { "dependencies": { "@types/unist": "^3.0.0" } }, "sha512-LsiILbtBETkDz8I9p1dQ0uyRUWuaQzd/cuEeS1hoRSyW5E5XGmTzlwY1OrNzzakGowI9Dr/I8HVaw4hTtnxy8g=="],
"unist-util-stringify-position": ["unist-util-stringify-position@4.0.0", "", { "dependencies": { "@types/unist": "^3.0.0" } }, "sha512-0ASV06AAoKCDkS2+xw5RXJywruurpbC4JZSm7nr7MOt1ojAzvyyaO+UxZf18j8FCF6kmzCZKcAgN/yu2gm2XgQ=="],
"unist-util-visit": ["unist-util-visit@5.1.0", "", { "dependencies": { "@types/unist": "^3.0.0", "unist-util-is": "^6.0.0", "unist-util-visit-parents": "^6.0.0" } }, "sha512-m+vIdyeCOpdr/QeQCu2EzxX/ohgS8KbnPDgFni4dQsfSCtpz8UqDyY5GjRru8PDKuYn7Fq19j1CQ+nJSsGKOzg=="],
"unist-util-visit-parents": ["unist-util-visit-parents@6.0.2", "", { "dependencies": { "@types/unist": "^3.0.0", "unist-util-is": "^6.0.0" } }, "sha512-goh1s1TBrqSqukSc8wrjwWhL0hiJxgA8m4kFxGlQ+8FYQ3C/m11FcTs4YYem7V664AhHVvgoQLk890Ssdsr2IQ=="],
"universalify": ["universalify@0.1.2", "", {}, "sha512-rBJeI5CXAlmy1pV+617WB9J63U6XcazHHF2f2dbJix4XzpUF0RS3Zbj0FGIOCAva5P/d/GBOYaACQ1w+0azUkg=="],
"vfile": ["vfile@6.0.3", "", { "dependencies": { "@types/unist": "^3.0.0", "vfile-message": "^4.0.0" } }, "sha512-KzIbH/9tXat2u30jf+smMwFCsno4wHVdNmzFyL+T/L3UGqqk6JKfVqOFOZEpZSHADH1k40ab6NUIXZq422ov3Q=="],
"vfile-message": ["vfile-message@4.0.3", "", { "dependencies": { "@types/unist": "^3.0.0", "unist-util-stringify-position": "^4.0.0" } }, "sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw=="],
"w3c-xmlserializer": ["w3c-xmlserializer@5.0.0", "", { "dependencies": { "xml-name-validator": "^5.0.0" } }, "sha512-o8qghlI8NZHU1lLPrpi2+Uq7abh4GGPpYANlalzWxyWteJOCsr/P+oPBA49TOLu5FTZO4d3F9MnWJfiMo4BkmA=="],
"webidl-conversions": ["webidl-conversions@8.0.1", "", {}, "sha512-BMhLD/Sw+GbJC21C/UgyaZX41nPt8bUTg+jWyDeg7e7YN4xOM05YPSIXceACnXVtqyEw/LMClUQMtMZ+PGGpqQ=="],
"whatwg-mimetype": ["whatwg-mimetype@5.0.0", "", {}, "sha512-sXcNcHOC51uPGF0P/D4NVtrkjSU2fNsm9iog4ZvZJsL3rjoDAzXZhkm2MWt1y+PUdggKAYVoMAIYcs78wJ51Cw=="],
"whatwg-url": ["whatwg-url@16.0.1", "", { "dependencies": { "@exodus/bytes": "^1.11.0", "tr46": "^6.0.0", "webidl-conversions": "^8.0.1" } }, "sha512-1to4zXBxmXHV3IiSSEInrreIlu02vUOvrhxJJH5vcxYTBDAx51cqZiKdyTxlecdKNSjj8EcxGBxNf6Vg+945gw=="],
"which": ["which@2.0.2", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="],
"ws": ["ws@8.20.0", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-sAt8BhgNbzCtgGbt2OxmpuryO63ZoDk/sqaB/znQm94T4fCEsy/yV+7CdC1kJhOU9lboAEU7R3kquuycDoibVA=="],
"xml-name-validator": ["xml-name-validator@5.0.0", "", {}, "sha512-EvGK8EJ3DhaHfbRlETOWAS5pO9MZITeauHKJyb8wyajUfQUenkIg2MvLDTZ4T/TgIcm3HU0TFBgWWboAZ30UHg=="],
"xmlchars": ["xmlchars@2.2.0", "", {}, "sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw=="],
"zwitch": ["zwitch@2.0.4", "", {}, "sha512-bXE4cR/kVZhKZX/RjPEflHaKVhUVl85noU3v6b8apfQEc1x4A+zBxjZ4lN8LqGd6WZ3dl98pY4o717VFmoPp+A=="],
"@manypkg/find-root/@types/node": ["@types/node@12.20.55", "", {}, "sha512-J8xLz7q2OFulZ2cyGTLE1TbbZcjpno7FaN6zdJNrgAdrJ+DZzh/uFR6YrTb4C+nXakvud8Q4+rbhoIWlYQbUFQ=="],
"@manypkg/find-root/fs-extra": ["fs-extra@8.1.0", "", { "dependencies": { "graceful-fs": "^4.2.0", "jsonfile": "^4.0.0", "universalify": "^0.1.0" } }, "sha512-yhlQgA6mnOJUKOsRUFsgJdQCvkKhcz8tlZG5HBQfReYZy46OwLcY+Zia0mtdHsOo9y/hP+CxMN0TU9QxoOtG4g=="],
"@manypkg/get-packages/@changesets/types": ["@changesets/types@4.1.0", "", {}, "sha512-LDQvVDv5Kb50ny2s25Fhm3d9QSZimsoUGBsUioj6MC3qbMUCuC8GPIvk/M6IvXx3lYhAs0lwWUQLb+VIEUCECw=="],
"@manypkg/get-packages/fs-extra": ["fs-extra@8.1.0", "", { "dependencies": { "graceful-fs": "^4.2.0", "jsonfile": "^4.0.0", "universalify": "^0.1.0" } }, "sha512-yhlQgA6mnOJUKOsRUFsgJdQCvkKhcz8tlZG5HBQfReYZy46OwLcY+Zia0mtdHsOo9y/hP+CxMN0TU9QxoOtG4g=="],
"dom-serializer/entities": ["entities@4.5.0", "", {}, "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw=="],
"htmlparser2/entities": ["entities@7.0.1", "", {}, "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA=="],
"jsdom/parse5": ["parse5@8.0.1", "", { "dependencies": { "entities": "^8.0.0" } }, "sha512-z1e/HMG90obSGeidlli3hj7cbocou0/wa5HacvI3ASx34PecNjNQeaHNo5WIZpWofN9kgkqV1q5YvXe3F0FoPw=="],
"mdast-util-find-and-replace/escape-string-regexp": ["escape-string-regexp@5.0.0", "", {}, "sha512-/veY75JbMK4j1yjvuUxuVsiS/hr/4iHs9FTT6cgTexxdE0Ly/glccBAkloH/DofkjRbZU3bnoj38mOmhkZ0lHw=="],
"read-yaml-file/js-yaml": ["js-yaml@3.14.2", "", { "dependencies": { "argparse": "^1.0.7", "esprima": "^4.0.0" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-PMSmkqxr106Xa156c2M265Z+FTrPl+oxd/rgOQy2tijQeK5TxQ43psO1ZCwhVOSdnn+RzkzlRz/eY4BgJBYVpg=="],
"jsdom/parse5/entities": ["entities@8.0.0", "", {}, "sha512-zwfzJecQ/Uej6tusMqwAqU/6KL2XaB2VZ2Jg54Je6ahNBGNH6Ek6g3jjNCF0fG9EWQKGZNddNjU5F1ZQn/sBnA=="],
"read-yaml-file/js-yaml/argparse": ["argparse@1.0.10", "", { "dependencies": { "sprintf-js": "~1.0.2" } }, "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg=="],
}
}
+68
View File
@@ -0,0 +1,68 @@
{
"name": "baoyu-fetch",
"version": "0.1.2",
"description": "Read URLs into high-quality Markdown or JSON with Chrome CDP and site adapters.",
"type": "module",
"bin": {
"baoyu-fetch": "dist/cli.js"
},
"files": [
"dist",
"README.zh-CN.md",
"src/adapters",
"src/browser",
"src/cli.ts",
"src/commands",
"src/extract",
"src/media",
"src/types",
"src/utils",
"README.md"
],
"repository": {
"type": "git",
"url": "git+https://github.com/JimLiu/baoyu-skills.git",
"directory": "packages/baoyu-fetch"
},
"bugs": {
"url": "https://github.com/JimLiu/baoyu-skills/issues"
},
"homepage": "https://github.com/JimLiu/baoyu-skills/tree/main/packages/baoyu-fetch#readme",
"publishConfig": {
"access": "public"
},
"scripts": {
"build": "rm -rf dist && bun build ./src/cli.ts --target bun --outfile ./dist/cli.js && chmod +x ./dist/cli.js",
"check": "tsc --noEmit",
"dev": "bun run ./src/cli.ts",
"release": "changeset publish",
"test": "bun test",
"version-packages": "changeset version",
"prepack": "bun run build"
},
"engines": {
"bun": ">=1.2.0"
},
"dependencies": {
"@mozilla/readability": "^0.6.0",
"chrome-launcher": "^1.2.1",
"defuddle": "^0.17.0",
"jsdom": "^29.0.2",
"remark-gfm": "^4.0.1",
"remark-parse": "^11.0.0",
"turndown": "^7.2.0",
"turndown-plugin-gfm": "^1.0.2",
"unified": "^11.0.5",
"ws": "^8.18.3"
},
"devDependencies": {
"@changesets/cli": "^2.30.0",
"@types/bun": "^1.2.23",
"@types/jsdom": "^21.1.7",
"@types/ws": "^8.18.1",
"typescript": "^5.9.2"
},
"overrides": {
"@xmldom/xmldom": "0.8.13"
}
}
@@ -0,0 +1,25 @@
import { describe, expect, test } from "bun:test";
import { resolveAdapter } from "../adapters";
describe("adapter registry", () => {
test("matches x adapter for x.com status URLs", () => {
const adapter = resolveAdapter({
url: new URL("https://x.com/openai/status/1234567890"),
});
expect(adapter.name).toBe("x");
});
test("matches hn adapter for item URLs", () => {
const adapter = resolveAdapter({
url: new URL("https://news.ycombinator.com/item?id=47534848"),
});
expect(adapter.name).toBe("hn");
});
test("falls back to generic adapter", () => {
const adapter = resolveAdapter({
url: new URL("https://example.com/post"),
});
expect(adapter.name).toBe("generic");
});
});
@@ -0,0 +1,69 @@
import { describe, expect, test } from "bun:test";
import { HELP_TEXT, parseArgs } from "../cli";
describe("parseArgs", () => {
test("defaults to markdown output", () => {
const options = parseArgs(["bun", "src/cli.ts", "https://example.com"]);
expect(options.format).toBe("markdown");
});
test("parses explicit json output format", () => {
const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--format", "json"]);
expect(options.format).toBe("json");
});
test("maps --json to json output format", () => {
const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--json"]);
expect(options.format).toBe("json");
});
test("parses --wait-for interaction", () => {
const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--wait-for", "interaction"]);
expect(options.waitMode).toBe("interaction");
});
test("parses --wait-for force", () => {
const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--wait-for", "force"]);
expect(options.waitMode).toBe("force");
});
test("maps legacy wait flags to interaction mode", () => {
const options = parseArgs(["bun", "src/cli.ts", "https://example.com", "--wait-for-interaction"]);
expect(options.waitMode).toBe("interaction");
});
test("parses media download options", () => {
const options = parseArgs([
"bun",
"src/cli.ts",
"https://example.com",
"--download-media",
"--media-dir",
"./assets",
]);
expect(options.downloadMedia).toBe(true);
expect(options.mediaDir).toBe("./assets");
});
test("rejects invalid wait modes", () => {
expect(() =>
parseArgs(["bun", "src/cli.ts", "https://example.com", "--wait-for", "unknown"]),
).toThrow("Invalid wait mode");
});
test("rejects invalid output formats", () => {
expect(() =>
parseArgs(["bun", "src/cli.ts", "https://example.com", "--format", "xml"]),
).toThrow("Invalid output format");
});
test("documents wait modes in help text", () => {
expect(HELP_TEXT).toContain("baoyu-fetch");
expect(HELP_TEXT).toContain("--format <type>");
expect(HELP_TEXT).toContain("--wait-for <mode>");
expect(HELP_TEXT).toContain("--download-media");
expect(HELP_TEXT).toContain("force: start visible Chrome, then auto-continue");
expect(HELP_TEXT).toContain("or continue immediately when you press Enter");
});
});
@@ -0,0 +1,55 @@
import { describe, expect, test } from "bun:test";
import { formatOutputContent } from "../commands/convert";
describe("formatOutputContent", () => {
test("returns raw markdown for markdown output", () => {
expect(
formatOutputContent("markdown", {
adapter: "generic",
status: "ok",
media: [],
downloads: null,
document: {
url: "https://example.com",
content: [],
},
markdown: "# Example",
}),
).toBe("# Example");
});
test("returns structured json for json output", () => {
const parsed = JSON.parse(
formatOutputContent("json", {
adapter: "generic",
status: "ok",
media: [],
downloads: null,
document: {
url: "https://example.com",
content: [],
},
markdown: "# Example",
}),
);
expect(parsed.status).toBe("ok");
expect(parsed.markdown).toBe("# Example");
expect(parsed.document.url).toBe("https://example.com");
});
test("rejects markdown output for interaction-required payloads", () => {
expect(() =>
formatOutputContent("markdown", {
adapter: "x",
status: "needs_interaction",
interaction: {
type: "wait_for_interaction",
kind: "login",
provider: "x",
prompt: "Login required",
},
}),
).toThrow("Markdown output is only available");
});
});
@@ -0,0 +1,167 @@
import { describe, expect, test } from "bun:test";
import { renderMarkdown } from "../extract/markdown-renderer";
import {
buildHnDocument,
buildHnThreadMarkdown,
extractHnThreadFromHtml,
parseHnItemId,
type HnCommentNode,
type HnItem,
} from "../adapters/hn";
describe("hn adapter helpers", () => {
test("parses item id from hn item url", () => {
expect(parseHnItemId(new URL("https://news.ycombinator.com/item?id=47534848"))).toBe(47534848);
expect(parseHnItemId(new URL("https://news.ycombinator.com/newest"))).toBeNull();
});
test("renders threaded comments with author, time, and nested indentation", () => {
const story: HnItem = {
id: 47534848,
type: "story",
by: "mmcclure",
time: 1774554485,
title: "Example &amp; Title",
url: "https://example.com/post",
score: 257,
descendants: 2,
};
const comments: HnCommentNode[] = [
{
item: {
id: 47535377,
type: "comment",
by: "jackfruitpeel",
time: 1774557334,
text: "Root comment<p>With two paragraphs.",
},
children: [
{
item: {
id: 47535469,
type: "comment",
by: "__MatrixMan__",
time: 1774557848,
text: "Nested reply with a <a href=\"item?id=1\">relative link</a>.",
},
children: [],
},
],
},
];
const body = buildHnThreadMarkdown(story, comments, "https://news.ycombinator.com/item?id=47534848");
expect(body).toContain("Source: [https://example.com/post](https://example.com/post)");
expect(body).toContain("Submitted by mmcclure at 2026-03-26 19:48:05 UTC");
expect(body).toContain("- jackfruitpeel · [2026-03-26 20:35:34 UTC](https://news.ycombinator.com/item?id=47534848#47535377)");
expect(body).toContain(" Root comment");
expect(body).toContain(" With two paragraphs.");
expect(body).toContain(" - __MatrixMan__ · [2026-03-26 20:44:08 UTC](https://news.ycombinator.com/item?id=47534848#47535469)");
expect(body).toContain(" Nested reply with a [relative link](https://news.ycombinator.com/item?id=1).");
});
test("extracts story metadata and nested comments from hn html", () => {
const parsed = extractHnThreadFromHtml(
`
<html>
<body>
<table class="fatitem">
<tr class="athing submission" id="47534848">
<td class="title">
<span class="titleline">
<a href="https://example.com/post">Example story</a>
</span>
</td>
</tr>
<tr>
<td class="subtext">
<span class="subline">
<span class="score">257 points</span>
by <a href="user?id=mmcclure" class="hnuser">mmcclure</a>
<span class="age" title="2026-03-26T19:48:05 1774554485">
<a href="item?id=47534848">1 hour ago</a>
</span>
<a href="item?id=47534848">152 comments</a>
</span>
</td>
</tr>
<tr>
<td><div class="toptext">Story <p>body</p></div></td>
</tr>
</table>
<table class="comment-tree">
<tr class="athing comtr" id="47535377">
<td class="ind" indent="0"></td>
<td class="default">
<span class="comhead">
<a href="user?id=jackfruitpeel" class="hnuser">jackfruitpeel</a>
<span class="age" title="2026-03-26T20:35:34 1774557334">
<a href="item?id=47535377">36 minutes ago</a>
</span>
</span>
<div class="comment"><div class="commtext c00">Root</div></div>
</td>
</tr>
<tr class="athing comtr" id="47535469">
<td class="ind" indent="1"></td>
<td class="default">
<span class="comhead">
<a href="user?id=willio58" class="hnuser">willio58</a>
<span class="age" title="2026-03-26T20:44:08 1774557848">
<a href="item?id=47535469">27 minutes ago</a>
</span>
</span>
<div class="comment"><div class="commtext c00">Child</div></div>
</td>
</tr>
</table>
</body>
</html>
`,
"https://news.ycombinator.com/item?id=47534848",
);
expect(parsed).not.toBeNull();
expect(parsed?.story.title).toBe("Example story");
expect(parsed?.story.url).toBe("https://example.com/post");
expect(parsed?.story.by).toBe("mmcclure");
expect(parsed?.story.time).toBe(1774554485);
expect(parsed?.story.score).toBe(257);
expect(parsed?.story.descendants).toBe(152);
expect(parsed?.story.text).toContain("Story");
expect(parsed?.comments).toHaveLength(1);
expect(parsed?.comments[0]?.item.by).toBe("jackfruitpeel");
expect(parsed?.comments[0]?.children).toHaveLength(1);
expect(parsed?.comments[0]?.children[0]?.item.by).toBe("willio58");
});
test("builds hn document with metadata and markdown body", () => {
const document = buildHnDocument(
{
id: 123,
type: "story",
by: "pg",
time: 1175714200,
title: "Ask HN: Example",
text: "What are you working on?",
score: 111,
descendants: 0,
},
[],
"https://news.ycombinator.com/item?id=123",
);
const markdown = renderMarkdown(document);
expect(document.adapter).toBe("hn");
expect(document.siteName).toBe("Hacker News");
expect(document.publishedAt).toBe("2007-04-04T19:16:40.000Z");
expect(markdown).toContain('adapter: "hn"');
expect(markdown).toContain('siteName: "Hacker News"');
expect(markdown).toContain("# Ask HN: Example");
expect(markdown).toContain("## Post");
expect(markdown).toContain("What are you working on?");
expect(markdown).toContain("## Comments");
expect(markdown).toContain("No comments.");
});
});
@@ -0,0 +1,105 @@
import { afterEach, describe, expect, test } from "bun:test";
import {
convertHtmlToMarkdown,
extractTitleFromMarkdownDocument,
} from "../extract/html-to-markdown";
const originalFetch = globalThis.fetch;
afterEach(() => {
globalThis.fetch = originalFetch;
});
describe("extractTitleFromMarkdownDocument", () => {
test("prefers frontmatter title when present", () => {
const title = extractTitleFromMarkdownDocument(`---
title: "Frontmatter Title"
---
# Heading Title
`);
expect(title).toBe("Frontmatter Title");
});
test("falls back to the first markdown heading", () => {
const title = extractTitleFromMarkdownDocument(`
Intro text
# Heading Title
Body text.
`);
expect(title).toBe("Heading Title");
});
});
describe("convertHtmlToMarkdown remote fallback", () => {
test("does not call defuddle.md when the remote fallback option is disabled", async () => {
let fetchCalls = 0;
globalThis.fetch = Object.assign(
async () => {
fetchCalls += 1;
return new Response("# Remote Title\n\nRemote body.", {
headers: {
"content-type": "text/markdown",
},
});
},
{
preconnect: originalFetch.preconnect,
},
) as typeof fetch;
const result = await convertHtmlToMarkdown(
"<!doctype html><html><head><title>Local Title</title></head><body></body></html>",
"https://example.com/post",
);
expect(fetchCalls).toBe(0);
expect(result.conversionMethod).not.toBe("defuddle-api");
});
test("uses defuddle.md markdown when local extraction is empty", async () => {
const fetchCalls: Array<{ input: RequestInfo | URL; init?: RequestInit }> = [];
globalThis.fetch = Object.assign(
async (input: RequestInfo | URL, init?: RequestInit) => {
fetchCalls.push({ input, init });
return new Response(`---
title: "Remote Title"
---
# Remote Title
Remote body.
`, {
headers: {
"content-type": "text/markdown",
},
});
},
{
preconnect: originalFetch.preconnect,
},
) as typeof fetch;
const result = await convertHtmlToMarkdown(
"<!doctype html><html><head><title>Local Title</title></head><body></body></html>",
"https://example.com/post",
{ enableRemoteMarkdownFallback: true },
);
expect(fetchCalls).toHaveLength(1);
expect(String(fetchCalls[0]?.input)).toBe(
"https://defuddle.md/https%3A%2F%2Fexample.com%2Fpost",
);
expect(fetchCalls[0]?.init?.headers).toEqual({
accept: "text/markdown,text/plain;q=0.9,*/*;q=0.1",
});
expect(result.conversionMethod).toBe("defuddle-api");
expect(result.metadata.title).toBe("Remote Title");
expect(result.markdown).toBe("# Remote Title\n\nRemote body.");
expect(result.fallbackReason).toContain("defuddle.md");
});
});
@@ -0,0 +1,54 @@
import { describe, expect, test } from "bun:test";
import { detectInteractionGateFromSnapshot } from "../browser/interaction-gates";
describe("detectInteractionGateFromSnapshot", () => {
test("detects cloudflare challenge", () => {
const gate = detectInteractionGateFromSnapshot({
title: "Just a moment...",
currentUrl: "https://example.com/cdn-cgi/challenge-platform/h/b",
bodyText: "Checking your browser before accessing example.com",
hasCloudflareTurnstile: true,
hasCloudflareChallenge: true,
hasRecaptcha: false,
hasRecaptchaIframe: false,
hasHcaptcha: false,
hasHcaptchaIframe: false,
});
expect(gate?.kind).toBe("cloudflare");
expect(gate?.provider).toBe("cloudflare");
});
test("detects google recaptcha", () => {
const gate = detectInteractionGateFromSnapshot({
title: "Protected page",
currentUrl: "https://example.com/form",
bodyText: "Please verify that you're not a robot via reCAPTCHA",
hasCloudflareTurnstile: false,
hasCloudflareChallenge: false,
hasRecaptcha: true,
hasRecaptchaIframe: true,
hasHcaptcha: false,
hasHcaptchaIframe: false,
});
expect(gate?.kind).toBe("recaptcha");
expect(gate?.provider).toBe("google_recaptcha");
});
test("returns null when no challenge is present", () => {
const gate = detectInteractionGateFromSnapshot({
title: "Example",
currentUrl: "https://example.com/article",
bodyText: "Normal article body",
hasCloudflareTurnstile: false,
hasCloudflareChallenge: false,
hasRecaptcha: false,
hasRecaptchaIframe: false,
hasHcaptcha: false,
hasHcaptchaIframe: false,
});
expect(gate).toBeNull();
});
});
@@ -0,0 +1,138 @@
import { describe, expect, test } from "bun:test";
import {
collectMediaFromDocument,
collectMediaFromMarkdown,
normalizeMarkdownMediaLinks,
rewriteMarkdownMediaLinks,
} from "../media/markdown-media";
describe("markdown media helpers", () => {
test("collects cover, image markdown, and plain media urls from a document", () => {
const media = collectMediaFromDocument({
url: "https://example.com/post",
metadata: {
coverImage: "https://cdn.example.com/cover.jpg",
},
content: [
{ type: "paragraph", text: "Poster: https://cdn.example.com/poster.png" },
{ type: "markdown", markdown: "![inline](https://cdn.example.com/body.webp)\n\n[video](https://cdn.example.com/clip.mp4)" },
],
});
expect(media).toEqual([
{ url: "https://cdn.example.com/cover.jpg", kind: "image", role: "cover" },
{ url: "https://cdn.example.com/poster.png", kind: "image", role: "inline" },
{ url: "https://cdn.example.com/body.webp", kind: "image", role: "inline" },
{ url: "https://cdn.example.com/clip.mp4", kind: "video", role: "inline" },
]);
});
test("rewrites markdown links, frontmatter cover images, and plain url mentions", () => {
const markdown = `---
coverImage: "https://cdn.example.com/cover.jpg"
---
![inline](https://cdn.example.com/body.webp)
Poster: https://cdn.example.com/poster.png
`;
const rewritten = rewriteMarkdownMediaLinks(markdown, [
{
url: "https://cdn.example.com/cover.jpg",
localPath: "imgs/img-001-cover.jpg",
absolutePath: "/tmp/imgs/img-001-cover.jpg",
kind: "image",
},
{
url: "https://cdn.example.com/body.webp",
localPath: "imgs/img-002-body.webp",
absolutePath: "/tmp/imgs/img-002-body.webp",
kind: "image",
},
{
url: "https://cdn.example.com/poster.png",
localPath: "imgs/img-003-poster.png",
absolutePath: "/tmp/imgs/img-003-poster.png",
kind: "image",
},
]);
expect(rewritten).toContain('coverImage: "imgs/img-001-cover.jpg"');
expect(rewritten).toContain("![inline](imgs/img-002-body.webp)");
expect(rewritten).toContain("Poster: imgs/img-003-poster.png");
});
test("normalizes and dedupes linked Substack CDN image variants", () => {
const resizedUrl =
"https://substackcdn.com/image/fetch/$s_!wORh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const linkedUrl =
"https://substackcdn.com/image/fetch/$s_!wORh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const canonicalUrl =
"https://substack-post-media.s3.amazonaws.com/public/images/b83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const markdown = `[![](${resizedUrl})](${linkedUrl})`;
expect(normalizeMarkdownMediaLinks(markdown)).toBe(`![](${canonicalUrl})`);
expect(collectMediaFromMarkdown(markdown)).toEqual([
{
url: canonicalUrl,
kind: "image",
role: "inline",
},
]);
});
test("collapses linked images when href equals image url after normalization", () => {
const resizedUrl =
"https://substackcdn.com/image/fetch/$s_!wORh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const linkedUrl =
"https://substackcdn.com/image/fetch/$s_!wORh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const canonicalUrl =
"https://substack-post-media.s3.amazonaws.com/public/images/b83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const markdown = `[
![](${resizedUrl})
](${linkedUrl})`;
expect(normalizeMarkdownMediaLinks(markdown)).toBe(`![](${canonicalUrl})`);
});
test("compacts linked images when href differs from the image url", () => {
const markdown = `[
![diagram](https://cdn.example.com/body.webp)
](https://example.com/source)`;
expect(normalizeMarkdownMediaLinks(markdown)).toBe(
"[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)",
);
});
test("keeps single-line linked images on one line after parser-based normalization", () => {
const markdown = `[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)`;
expect(normalizeMarkdownMediaLinks(markdown)).toBe(
"[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)",
);
});
test("repairs broken linked image blocks without disturbing surrounding paragraphs", () => {
const markdown = `Before
[
![diagram](https://cdn.example.com/body.webp)
](https://example.com/source)
After`;
expect(normalizeMarkdownMediaLinks(markdown)).toBe(`Before
[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)
After`);
});
});
@@ -0,0 +1,99 @@
import { describe, expect, test } from "bun:test";
import { renderMarkdown } from "../extract/markdown-renderer";
describe("renderMarkdown", () => {
test("renders frontmatter and content blocks", () => {
const markdown = renderMarkdown({
url: "https://example.com/post",
requestedUrl: "https://example.com/post?ref=test",
title: "Example Title",
author: "Alice",
siteName: "Example",
publishedAt: "2026-03-25",
adapter: "generic",
metadata: {
authorName: "Alice Example",
authorUsername: "alice",
authorUrl: "https://example.com/@alice",
kind: "generic/article",
},
content: [
{ type: "paragraph", text: "First paragraph." },
{ type: "list", ordered: false, items: ["One", "Two"] },
],
});
expect(markdown).toContain("---");
expect(markdown).toContain('title: "Example Title"');
expect(markdown).toContain('url: "https://example.com/post"');
expect(markdown).toContain('requestedUrl: "https://example.com/post?ref=test"');
expect(markdown).toContain('author: "Alice"');
expect(markdown).toContain('authorName: "Alice Example"');
expect(markdown).toContain('authorUsername: "alice"');
expect(markdown).toContain('authorUrl: "https://example.com/@alice"');
expect(markdown).toContain("# Example Title");
expect(markdown).toContain("First paragraph.");
expect(markdown).toContain("- One");
});
test("avoids duplicating the title when body already starts with it", () => {
const markdown = renderMarkdown({
url: "https://example.com/post",
title: "Example Title",
content: [{ type: "markdown", markdown: "# Example Title\n\nBody text." }],
});
expect(markdown.match(/# Example Title/g)?.length).toBe(1);
expect(markdown).toContain("Body text.");
});
test("normalizes Substack CDN image links in rendered markdown", () => {
const resizedUrl =
"https://substackcdn.com/image/fetch/$s_!wORh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const linkedUrl =
"https://substackcdn.com/image/fetch/$s_!wORh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const canonicalUrl =
"https://substack-post-media.s3.amazonaws.com/public/images/b83f9d2f-711f-4edd-bc8a-303b8de422e5_1600x1300.png";
const markdown = renderMarkdown({
url: "https://example.com/post",
metadata: {
coverImage: resizedUrl,
},
content: [
{
type: "markdown",
markdown: `[
![](${resizedUrl})
](${linkedUrl})`,
},
],
});
expect(markdown).toContain(`coverImage: "${canonicalUrl}"`);
expect(markdown).toContain(`![](${canonicalUrl})`);
expect(markdown).not.toContain(`[![](${canonicalUrl})](${canonicalUrl})`);
expect(markdown).not.toContain("substackcdn.com/image/fetch");
});
test("renders linked images on a single line when href differs from the image url", () => {
const markdown = renderMarkdown({
url: "https://example.com/post",
content: [
{
type: "markdown",
markdown: `[
![diagram](https://cdn.example.com/body.webp)
](https://example.com/source)`,
},
],
});
expect(markdown).toContain("[![diagram](https://cdn.example.com/body.webp)](https://example.com/source)");
expect(markdown).not.toContain("](https://example.com/source)\n");
});
});
@@ -0,0 +1,68 @@
import { afterEach, describe, expect, test } from "bun:test";
import fs from "node:fs";
import path from "node:path";
import os from "node:os";
import {
ensureChromeProfileDir,
hasChromeLockArtifacts,
resolveChromeProfileDir,
shouldRetryChromeLaunchRecovery,
} from "../browser/profile";
const originalProfile = process.env.BAOYU_CHROME_PROFILE_DIR;
afterEach(() => {
if (originalProfile === undefined) {
delete process.env.BAOYU_CHROME_PROFILE_DIR;
} else {
process.env.BAOYU_CHROME_PROFILE_DIR = originalProfile;
}
});
describe("resolveChromeProfileDir", () => {
test("uses BAOYU_CHROME_PROFILE_DIR when set", () => {
process.env.BAOYU_CHROME_PROFILE_DIR = "/tmp/baoyu-profile";
expect(resolveChromeProfileDir()).toBe("/tmp/baoyu-profile");
});
test("falls back to shared baoyu-skills profile path", () => {
delete process.env.BAOYU_CHROME_PROFILE_DIR;
const resolved = resolveChromeProfileDir();
if (process.platform === "darwin") {
expect(resolved).toBe(path.join(os.homedir(), "Library", "Application Support", "baoyu-skills", "chrome-profile"));
} else if (process.platform === "win32") {
expect(resolved.endsWith(path.join("baoyu-skills", "chrome-profile"))).toBe(true);
} else {
expect(resolved.endsWith(path.join("baoyu-skills", "chrome-profile"))).toBe(true);
}
});
});
describe("ensureChromeProfileDir", () => {
test("creates the profile directory when missing", () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "baoyu-fetch-profile-"));
const profileDir = path.join(tempRoot, "nested", "chrome-profile");
try {
expect(fs.existsSync(profileDir)).toBe(false);
expect(ensureChromeProfileDir(profileDir)).toBe(profileDir);
expect(fs.statSync(profileDir).isDirectory()).toBe(true);
} finally {
fs.rmSync(tempRoot, { force: true, recursive: true });
}
});
});
describe("stale lock recovery helpers", () => {
test("detects Chrome singleton lock artifacts", () => {
expect(hasChromeLockArtifacts(["Cookies", "SingletonLock"])).toBe(true);
expect(hasChromeLockArtifacts(["chrome.pid"])).toBe(true);
expect(hasChromeLockArtifacts(["Preferences", "Cookies"])).toBe(false);
});
test("only retries stale-lock recovery when no live owner exists", () => {
expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: true, hasLiveOwner: false })).toBe(true);
expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: true, hasLiveOwner: true })).toBe(false);
expect(shouldRetryChromeLaunchRecovery({ hasLockArtifacts: false, hasLiveOwner: false })).toBe(false);
});
});
@@ -0,0 +1,105 @@
import { describe, expect, test } from "bun:test";
import { shouldAutoContinueForceWait, shouldKeepBrowserOpenAfterInteraction } from "../commands/convert";
describe("shouldAutoContinueForceWait", () => {
test("continues when a challenge disappears", () => {
expect(
shouldAutoContinueForceWait(
{
url: "https://example.com/challenge",
hasGate: true,
loginState: "unknown",
sessionReady: true,
},
{
url: "https://example.com/article",
hasGate: false,
loginState: "unknown",
sessionReady: true,
},
),
).toBe(true);
});
test("waits for X session cookies before continuing after login", () => {
expect(
shouldAutoContinueForceWait(
{
url: "https://x.com/i/flow/login",
hasGate: false,
loginState: "logged_out",
sessionReady: false,
},
{
url: "https://x.com/home",
hasGate: false,
loginState: "logged_in",
sessionReady: false,
},
),
).toBe(false);
expect(
shouldAutoContinueForceWait(
{
url: "https://x.com/i/flow/login",
hasGate: false,
loginState: "logged_out",
sessionReady: false,
},
{
url: "https://x.com/home",
hasGate: false,
loginState: "logged_in",
sessionReady: true,
},
),
).toBe(true);
});
test("does not continue when nothing changed yet", () => {
expect(
shouldAutoContinueForceWait(
{
url: "https://x.com/lennysan/status/2036483059407810640",
hasGate: false,
loginState: "unknown",
sessionReady: false,
},
{
url: "https://x.com/lennysan/status/2036483059407810640",
hasGate: false,
loginState: "unknown",
sessionReady: false,
},
),
).toBe(false);
});
});
describe("shouldKeepBrowserOpenAfterInteraction", () => {
test("keeps launched X login browsers open", () => {
expect(
shouldKeepBrowserOpenAfterInteraction({
launched: true,
interaction: { kind: "login", provider: "x" },
}),
).toBe(true);
});
test("does not keep reused or non-login browsers open", () => {
expect(
shouldKeepBrowserOpenAfterInteraction({
launched: false,
interaction: { kind: "login", provider: "x" },
}),
).toBe(false);
expect(
shouldKeepBrowserOpenAfterInteraction({
launched: true,
interaction: { kind: "cloudflare", provider: "cloudflare" },
}),
).toBe(false);
});
});
@@ -0,0 +1,443 @@
import { describe, expect, test } from "bun:test";
import { extractArticleDocumentFromPayload } from "../adapters/x/article";
describe("x article extraction", () => {
test("renders markdown entities referenced by atomic blocks", () => {
const payload = {
data: {
tweetResult: {
result: {
rest_id: "2036762680401223946",
legacy: {
full_text: "Fallback text",
favorite_count: 12,
retweet_count: 3,
reply_count: 1,
created_at: "Wed Mar 25 11:10:38 +0000 2026",
},
core: {
user_results: {
result: {
legacy: {
name: "Eric Zakariasson",
screen_name: "ericzakariasson",
},
},
},
},
article: {
article_results: {
result: {
title: "Building CLIs for agents",
content_state: {
blocks: [
{
type: "unstyled",
text: "Make it non-interactive.",
data: {},
entityRanges: [],
inlineStyleRanges: [],
},
{
type: "atomic",
text: " ",
data: {},
entityRanges: [{ key: 0, length: 1, offset: 0 }],
inlineStyleRanges: [],
},
{
type: "unstyled",
text: "Return data on success.",
data: {},
entityRanges: [],
inlineStyleRanges: [],
},
],
entityMap: [
{
key: "0",
value: {
type: "MARKDOWN",
mutability: "Mutable",
data: {
markdown: "```bash\n$ mycli deploy --env production --dry-run\n```",
},
},
},
],
},
},
},
},
},
},
},
};
const document = extractArticleDocumentFromPayload(
payload,
"2036762680401223946",
"https://x.com/ericzakariasson/status/2036762680401223946",
);
expect(document).not.toBeNull();
expect(document?.metadata?.kind).toBe("x/article");
const content = document?.content[0];
expect(content?.type).toBe("markdown");
if (!content || content.type !== "markdown") {
throw new Error("Expected markdown content");
}
expect(content.markdown).toContain("```bash");
expect(content.markdown).toContain("$ mycli deploy --env production --dry-run");
expect(content.markdown).toContain("Make it non-interactive.");
expect(content.markdown).toContain("Return data on success.");
});
test("renders media, embedded tweets, and cover image from article entities", () => {
const embeddedTweetPayload = {
data: {
tweetResult: {
result: {
rest_id: "999",
legacy: {
full_text: "Embedded tweet text",
favorite_count: 4,
retweet_count: 2,
reply_count: 1,
created_at: "Wed Mar 25 11:10:38 +0000 2026",
extended_entities: {
media: [
{
type: "photo",
media_url_https: "https://pbs.twimg.com/media/embedded.jpg",
},
],
},
},
core: {
user_results: {
result: {
core: {
name: "Embedded Author",
screen_name: "embedded_author",
},
legacy: {},
},
},
},
},
},
},
};
const articlePayload = {
data: {
tweetResult: {
result: {
rest_id: "2036670816344064290",
legacy: {
full_text: "Fallback text",
favorite_count: 12,
retweet_count: 3,
reply_count: 1,
created_at: "Wed Mar 25 11:10:38 +0000 2026",
},
core: {
user_results: {
result: {
legacy: {
name: "Eric Zakariasson",
screen_name: "ericzakariasson",
},
},
},
},
article: {
article_results: {
result: {
title: "Article with media",
cover_media: {
media_info: {
original_img_url: "https://pbs.twimg.com/media/cover?format=jpeg&name=small",
},
},
media_entities: [
{
media_id: "42",
media_info: {
original_img_url: "https://pbs.twimg.com/media/body.jpg",
},
},
],
content_state: {
blocks: [
{
type: "unstyled",
text: "Read more: https://t.co/example",
data: {},
entityRanges: [{ key: 2, length: 20, offset: 11 }],
inlineStyleRanges: [],
},
{
type: "atomic",
text: " ",
data: {},
entityRanges: [{ key: 0, length: 1, offset: 0 }],
inlineStyleRanges: [],
},
{
type: "atomic",
text: " ",
data: {},
entityRanges: [{ key: 1, length: 1, offset: 0 }],
inlineStyleRanges: [],
},
],
entityMap: [
{
key: "0",
value: {
type: "MEDIA",
mutability: "Immutable",
data: {
mediaItems: [{ mediaId: "42" }],
},
},
},
{
key: "1",
value: {
type: "TWEET",
mutability: "Immutable",
data: {
tweetId: "999",
},
},
},
{
key: "2",
value: {
type: "LINK",
mutability: "Mutable",
data: {
url: "https://example.com/report",
},
},
},
],
},
},
},
},
},
},
},
};
const document = extractArticleDocumentFromPayload(
articlePayload,
"2036670816344064290",
"https://x.com/ericzakariasson/status/2036670816344064290",
[articlePayload, embeddedTweetPayload],
);
expect(document).not.toBeNull();
expect(document?.metadata?.coverImage).toBe(
"https://pbs.twimg.com/media/cover?format=jpg&name=4096x4096",
);
const content = document?.content[0];
expect(content?.type).toBe("markdown");
if (!content || content.type !== "markdown") {
throw new Error("Expected markdown content");
}
expect(content.markdown).toContain("https://example.com/report");
expect(content.markdown).toContain("![](https://pbs.twimg.com/media/body?format=jpg&name=4096x4096)");
expect(content.markdown).toContain("> Embedded Author (@embedded_author)");
expect(content.markdown).toContain("> Embedded tweet text");
expect(content.markdown).toContain(
"> ![](https://pbs.twimg.com/media/embedded?format=jpg&name=4096x4096)",
);
});
test("prefers expanded link entity urls in article blocks", () => {
const payload = {
data: {
tweetResult: {
result: {
rest_id: "2036670816344064290",
legacy: {
full_text: "Fallback text",
favorite_count: 12,
retweet_count: 3,
reply_count: 1,
created_at: "Wed Mar 25 11:10:38 +0000 2026",
},
core: {
user_results: {
result: {
legacy: {
name: "Eric Zakariasson",
screen_name: "ericzakariasson",
},
},
},
},
article: {
article_results: {
result: {
title: "Article with expanded links",
content_state: {
blocks: [
{
type: "unstyled",
text: "Read more: https://t.co/example",
data: {},
entityRanges: [{ key: 0, length: 20, offset: 11 }],
inlineStyleRanges: [],
},
],
entityMap: [
{
key: "0",
value: {
type: "LINK",
mutability: "Mutable",
data: {
expanded_url: "https://example.com/report",
url: "https://t.co/example",
},
},
},
],
},
},
},
},
},
},
},
};
const document = extractArticleDocumentFromPayload(
payload,
"2036670816344064290",
"https://x.com/ericzakariasson/status/2036670816344064290",
);
expect(document).not.toBeNull();
const content = document?.content[0];
expect(content?.type).toBe("markdown");
if (!content || content.type !== "markdown") {
throw new Error("Expected markdown content");
}
expect(content.markdown).toContain("https://example.com/report");
expect(content.markdown).not.toContain("https://t.co/example");
});
test("renders article video media as the highest bitrate mp4 link", () => {
const payload = {
data: {
tweetResult: {
result: {
rest_id: "2046628728210350366",
legacy: {
full_text: "Fallback text",
favorite_count: 12,
retweet_count: 3,
reply_count: 1,
created_at: "Tue Apr 21 16:34:47 +0000 2026",
},
core: {
user_results: {
result: {
legacy: {
name: "Google AI Studio",
screen_name: "GoogleAIStudio",
},
},
},
},
article: {
article_results: {
result: {
title: "Article with video",
media_entities: [
{
media_id: "2046627051822530560",
media_info: {
__typename: "ApiVideo",
variants: [
{
bit_rate: 2176000,
content_type: "video/mp4",
url: "https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1280x720/medium.mp4",
},
{
content_type: "application/x-mpegURL",
url: "https://video.twimg.com/amplify_video/2046627051822530560/pl/playlist.m3u8",
},
{
bit_rate: 10368000,
content_type: "video/mp4",
url: "https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1920x1080/high.mp4",
},
],
},
},
],
content_state: {
blocks: [
{
type: "atomic",
text: " ",
data: {},
entityRanges: [{ key: 0, length: 1, offset: 0 }],
inlineStyleRanges: [],
},
],
entityMap: [
{
key: "0",
value: {
type: "MEDIA",
mutability: "Immutable",
data: {
mediaItems: [{ mediaId: "2046627051822530560" }],
},
},
},
],
},
},
},
},
},
},
},
};
const document = extractArticleDocumentFromPayload(
payload,
"2046628728210350366",
"https://x.com/GoogleAIStudio/status/2046628728210350366",
);
expect(document).not.toBeNull();
const content = document?.content[0];
expect(content?.type).toBe("markdown");
if (!content || content.type !== "markdown") {
throw new Error("Expected markdown content");
}
expect(content.markdown).toBe(
"[video](https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1920x1080/high.mp4)",
);
});
});
@@ -0,0 +1,28 @@
import { describe, expect, test } from "bun:test";
import { buildXSessionCookieMap, hasRequiredXSessionCookies } from "../adapters/x/session";
describe("X session helpers", () => {
test("keeps non-empty X session cookies", () => {
expect(
buildXSessionCookieMap([
{ name: "auth_token", value: "auth" },
{ name: "ct0", value: "csrf" },
{ name: "twid", value: "u=123" },
{ name: "ct0", value: "" },
{ name: "", value: "ignored" },
{ name: "gt", value: undefined },
]),
).toEqual({
auth_token: "auth",
ct0: "csrf",
twid: "u=123",
});
});
test("requires auth_token and ct0 for a ready X session", () => {
expect(hasRequiredXSessionCookies({ auth_token: "auth" })).toBe(false);
expect(hasRequiredXSessionCookies({ ct0: "csrf" })).toBe(false);
expect(hasRequiredXSessionCookies({ auth_token: "auth", ct0: "csrf" })).toBe(true);
});
});
@@ -0,0 +1,260 @@
import { describe, expect, test } from "bun:test";
import { extractSingleTweetDocumentFromPayload } from "../adapters/x/single";
describe("x single tweet extraction", () => {
test("replaces t.co links in note tweets with expanded urls", () => {
const payload = {
data: {
tweetResult: {
result: {
rest_id: "2036483061635039711",
legacy: {
full_text:
"First, some context:\n\n1. This analysis is based on data from @trueupio, one of my favorite collaborators and sources of data. They track job openings at tech companies and top startups around the world (over 9,000 companies) and make it easy to browse open gigs. Their data looks",
favorite_count: 43,
retweet_count: 1,
reply_count: 1,
created_at: "Tue Mar 24 16:39:32 +0000 2026",
entities: {
hashtags: [],
symbols: [],
timestamps: [],
urls: [],
user_mentions: [
{
id_str: "1407256023547613193",
indices: [61, 70],
name: "TrueUp",
screen_name: "trueupio",
},
],
},
},
note_tweet: {
note_tweet_results: {
result: {
text:
"First, some context:\n\n1. This analysis is based on data from @trueupio, one of my favorite collaborators and sources of data. They track job openings at tech companies and top startups around the world (over 9,000 companies) and make it easy to browse open gigs. Their data looks at roles at tech companies—the most sought-after and lucrative jobs. (It doesnt include roles at non-tech companies and consulting agencies.) Browse open roles here: https://t.co/x7ff2NjpP1\n\n2. Keep reading for highlights, or jump straight to the full report: https://t.co/AbqPp2TEde",
entity_set: {
hashtags: [],
symbols: [],
urls: [
{
display_url: "trueup.io/jobs",
expanded_url: "https://trueup.io/jobs",
indices: [447, 470],
url: "https://t.co/x7ff2NjpP1",
},
{
display_url: "lennysnewsletter.com/i/191595250/if…",
expanded_url:
"https://www.lennysnewsletter.com/i/191595250/if-youre-having-trouble-finding-a-job",
indices: [541, 564],
url: "https://t.co/AbqPp2TEde",
},
],
user_mentions: [
{
id_str: "1407256023547613193",
indices: [61, 70],
name: "TrueUp",
screen_name: "trueupio",
},
],
},
},
},
},
core: {
user_results: {
result: {
legacy: {
name: "Lenny Rachitsky",
screen_name: "lennysan",
},
},
},
},
},
},
},
};
const document = extractSingleTweetDocumentFromPayload(
payload,
"2036483061635039711",
"https://x.com/lennysan/status/2036483061635039711",
);
expect(document).not.toBeNull();
const paragraphBlock = document?.content.find((block) => block.type === "paragraph");
expect(paragraphBlock).toEqual({
type: "paragraph",
text:
"First, some context:\n\n1. This analysis is based on data from @trueupio, one of my favorite collaborators and sources of data. They track job openings at tech companies and top startups around the world (over 9,000 companies) and make it easy to browse open gigs. Their data looks at roles at tech companies—the most sought-after and lucrative jobs. (It doesnt include roles at non-tech companies and consulting agencies.) Browse open roles here: https://trueup.io/jobs\n\n2. Keep reading for highlights, or jump straight to the full report: https://www.lennysnewsletter.com/i/191595250/if-youre-having-trouble-finding-a-job",
});
});
test("upgrades image urls to high resolution for tweet and quoted tweet media", () => {
const payload = {
data: {
tweetResult: {
result: {
rest_id: "2036762680401223946",
legacy: {
full_text: "Main tweet text https://t.co/media",
favorite_count: 12,
retweet_count: 3,
reply_count: 1,
created_at: "Wed Mar 25 11:10:38 +0000 2026",
extended_entities: {
media: [
{
type: "photo",
media_url_https: "https://pbs.twimg.com/media/main-image.png",
url: "https://t.co/media",
},
],
},
},
core: {
user_results: {
result: {
legacy: {
name: "Eric Zakariasson",
screen_name: "ericzakariasson",
},
},
},
},
quoted_status_result: {
result: {
rest_id: "999",
legacy: {
full_text: "Quoted tweet text",
favorite_count: 4,
retweet_count: 2,
reply_count: 1,
created_at: "Wed Mar 25 10:10:38 +0000 2026",
extended_entities: {
media: [
{
type: "photo",
media_url_https: "https://pbs.twimg.com/media/quoted?format=jpeg&name=small",
},
],
},
},
core: {
user_results: {
result: {
legacy: {
name: "Quoted Author",
screen_name: "quoted_author",
},
},
},
},
},
},
},
},
},
};
const document = extractSingleTweetDocumentFromPayload(
payload,
"2036762680401223946",
"https://x.com/ericzakariasson/status/2036762680401223946",
);
expect(document).not.toBeNull();
const imageBlock = document?.content.find((block) => block.type === "image");
expect(imageBlock).toEqual({
type: "image",
url: "https://pbs.twimg.com/media/main-image?format=png&name=4096x4096",
});
const quoteBlock = document?.content.find((block) => block.type === "quote");
expect(quoteBlock).toEqual({
type: "quote",
text:
"Quoted Author (@quoted_author)\n\nQuoted tweet text\n\nphoto: https://pbs.twimg.com/media/quoted?format=jpg&name=4096x4096",
});
});
test("uses the highest bitrate mp4 variant for tweet video media", () => {
const payload = {
data: {
tweetResult: {
result: {
rest_id: "2046628728210350366",
legacy: {
full_text: "Video post https://t.co/video",
favorite_count: 12,
retweet_count: 3,
reply_count: 1,
created_at: "Tue Apr 21 16:34:47 +0000 2026",
extended_entities: {
media: [
{
type: "video",
media_url_https: "https://pbs.twimg.com/amplify_video_thumb/2046627051822530560/img/poster.jpg",
url: "https://t.co/video",
video_info: {
variants: [
{
content_type: "application/x-mpegURL",
url: "https://video.twimg.com/amplify_video/2046627051822530560/pl/playlist.m3u8",
},
{
bitrate: 256000,
content_type: "video/mp4",
url: "https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/480x270/low.mp4",
},
{
bitrate: 10368000,
content_type: "video/mp4",
url: "https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1920x1080/high.mp4",
},
],
},
},
],
},
},
core: {
user_results: {
result: {
legacy: {
name: "Google AI Studio",
screen_name: "GoogleAIStudio",
},
},
},
},
},
},
},
};
const document = extractSingleTweetDocumentFromPayload(
payload,
"2046628728210350366",
"https://x.com/GoogleAIStudio/status/2046628728210350366",
);
expect(document).not.toBeNull();
const listBlock = document?.content.find((block) => block.type === "list");
expect(listBlock).toEqual({
type: "list",
ordered: false,
items: [
"video: https://video.twimg.com/amplify_video/2046627051822530560/vid/avc1/1920x1080/high.mp4",
],
});
});
});
@@ -0,0 +1,305 @@
import { describe, expect, test } from "bun:test";
import { extractThreadDocumentFromPayloads, extractThreadTweetsFromPayloads } from "../adapters/x/thread";
function buildTweet(options: {
id: string;
text: string;
createdAt: string;
userId?: string;
screenName?: string;
name?: string;
conversationId?: string;
inReplyToStatusId?: string;
inReplyToUserId?: string;
quotedTweet?: unknown;
}) {
const userId = options.userId ?? "3178231";
const screenName = options.screenName ?? "dotey";
const name = options.name ?? "宝玉";
return {
__typename: "Tweet",
rest_id: options.id,
legacy: {
id_str: options.id,
full_text: options.text,
favorite_count: 0,
retweet_count: 0,
reply_count: 0,
created_at: options.createdAt,
user_id_str: userId,
conversation_id_str: options.conversationId ?? options.id,
in_reply_to_status_id_str: options.inReplyToStatusId,
in_reply_to_user_id_str: options.inReplyToUserId,
},
core: {
user_results: {
result: {
core: {
name,
screen_name: screenName,
},
legacy: {},
},
},
},
quoted_status_result: options.quotedTweet
? {
result: options.quotedTweet,
}
: undefined,
};
}
function tweetEntry(tweet: unknown) {
return {
content: {
itemContent: {
tweet_results: {
result: tweet,
},
},
},
};
}
function moduleTweetItem(tweet: unknown) {
return {
item: {
itemContent: {
tweet_results: {
result: tweet,
},
},
},
};
}
describe("x thread extraction", () => {
test("keeps only the continuous same-author reply chain", () => {
const rootId = "1996285439867556304";
const reply1Id = "1996285442275340783";
const reply2Id = "1996285444582146559";
const quotedId = "1993729800922341810";
const quotedInsideThreadId = "1993729800922341811";
const otherAuthorReplyId = "2000000000000000001";
const sameAuthorAfterOtherId = "2000000000000000002";
const root = buildTweet({
id: rootId,
text: "A thread for my nana banana pro prompts 🧵",
createdAt: "Wed Dec 03 18:28:32 +0000 2025",
conversationId: rootId,
});
const reply1 = buildTweet({
id: reply1Id,
text: "Prompt 1",
createdAt: "Wed Dec 03 18:28:33 +0000 2025",
conversationId: rootId,
inReplyToStatusId: rootId,
inReplyToUserId: "3178231",
quotedTweet: buildTweet({
id: quotedInsideThreadId,
text: "Quoted inside the thread body",
createdAt: "Tue Nov 25 18:28:35 +0000 2025",
screenName: "quoted_author",
name: "Quoted Author",
}),
});
const reply2 = buildTweet({
id: reply2Id,
text: "Prompt 2",
createdAt: "Wed Dec 03 18:28:34 +0000 2025",
conversationId: rootId,
inReplyToStatusId: reply1Id,
inReplyToUserId: "3178231",
});
const quotedSameAuthor = buildTweet({
id: quotedId,
text: "Quoted standalone tweet",
createdAt: "Tue Nov 25 18:28:34 +0000 2025",
});
const otherAuthorReply = buildTweet({
id: otherAuthorReplyId,
text: "Another author joined the conversation",
createdAt: "Wed Dec 03 18:28:35 +0000 2025",
userId: "42",
screenName: "someone_else",
name: "Someone Else",
conversationId: rootId,
inReplyToStatusId: reply2Id,
inReplyToUserId: "3178231",
});
const sameAuthorAfterOther = buildTweet({
id: sameAuthorAfterOtherId,
text: "This should not be part of the continuous author chain",
createdAt: "Wed Dec 03 18:28:36 +0000 2025",
conversationId: rootId,
inReplyToStatusId: otherAuthorReplyId,
inReplyToUserId: "42",
});
const payloads = [
{
data: {
threaded_conversation_with_injections_v2: {
instructions: [
{
type: "TimelineAddEntries",
entries: [
tweetEntry(root),
tweetEntry(reply1),
tweetEntry(quotedSameAuthor),
{
content: {
items: [moduleTweetItem(reply2)],
},
},
],
},
{
type: "TimelineAddToModule",
moduleItems: [moduleTweetItem(otherAuthorReply), moduleTweetItem(sameAuthorAfterOther)],
},
],
},
},
},
];
const tweets = extractThreadTweetsFromPayloads(
payloads,
rootId,
"https://x.com/dotey/status/1996285439867556304",
);
expect(tweets.map((tweet) => tweet.id)).toEqual([rootId, reply1Id, reply2Id]);
const document = extractThreadDocumentFromPayloads(
payloads,
rootId,
"https://x.com/dotey/status/1996285439867556304",
);
expect(document).not.toBeNull();
expect(document?.metadata?.tweetCount).toBe(3);
expect(document?.metadata?.lastTweetId).toBe(reply2Id);
const content = document?.content[0];
expect(content?.type).toBe("markdown");
if (!content || content.type !== "markdown") {
throw new Error("Expected markdown content");
}
expect(content.markdown).toContain("Prompt 1");
expect(content.markdown).toContain("Prompt 2");
expect(content.markdown).toContain("Quoted inside the thread body");
expect(content.markdown).not.toContain("Quoted standalone tweet");
expect(content.markdown).not.toContain("This should not be part of the continuous author chain");
});
test("returns null when there is no same-author reply chain", () => {
const rootId = "1996285439867556304";
const root = buildTweet({
id: rootId,
text: "Root tweet",
createdAt: "Wed Dec 03 18:28:32 +0000 2025",
conversationId: rootId,
});
const quotedSameAuthor = buildTweet({
id: "1993729800922341810",
text: "Quoted standalone tweet",
createdAt: "Tue Nov 25 18:28:34 +0000 2025",
});
const payloads = [
{
data: {
threaded_conversation_with_injections_v2: {
instructions: [
{
type: "TimelineAddEntries",
entries: [tweetEntry(root), tweetEntry(quotedSameAuthor)],
},
],
},
},
},
];
expect(
extractThreadDocumentFromPayloads(
payloads,
rootId,
"https://x.com/dotey/status/1996285439867556304",
),
).toBeNull();
});
test("restores ancestors when the requested tweet is in the middle of a thread", () => {
const rootId = "1996285439867556304";
const reply1Id = "1996285442275340783";
const reply2Id = "1996285444582146559";
const root = buildTweet({
id: rootId,
text: "Root tweet",
createdAt: "Wed Dec 03 18:28:32 +0000 2025",
conversationId: rootId,
});
const reply1 = buildTweet({
id: reply1Id,
text: "Middle tweet",
createdAt: "Wed Dec 03 18:28:33 +0000 2025",
conversationId: rootId,
inReplyToStatusId: rootId,
inReplyToUserId: "3178231",
});
const reply2 = buildTweet({
id: reply2Id,
text: "Last tweet",
createdAt: "Wed Dec 03 18:28:34 +0000 2025",
conversationId: rootId,
inReplyToStatusId: reply1Id,
inReplyToUserId: "3178231",
});
const payloads = [
{
data: {
threaded_conversation_with_injections_v2: {
instructions: [
{
type: "TimelineAddEntries",
entries: [
tweetEntry(root),
tweetEntry(reply1),
tweetEntry(reply2),
],
},
],
},
},
},
];
const tweets = extractThreadTweetsFromPayloads(
payloads,
reply1Id,
"https://x.com/dotey/status/1996285442275340783",
);
expect(tweets.map((tweet) => tweet.id)).toEqual([rootId, reply1Id, reply2Id]);
const document = extractThreadDocumentFromPayloads(
payloads,
reply1Id,
"https://x.com/dotey/status/1996285442275340783",
);
expect(document).not.toBeNull();
expect(document?.metadata?.tweetId).toBe(rootId);
expect(document?.metadata?.lastTweetId).toBe(reply2Id);
expect(document?.metadata?.tweetCount).toBe(3);
});
});
@@ -0,0 +1,101 @@
import { describe, expect, test } from "bun:test";
import {
buildYouTubeThumbnailCandidates,
formatTimestampRange,
parseYouTubeDescriptionChapters,
parseYouTubeVideoId,
renderYouTubeTranscriptMarkdown,
} from "../adapters/youtube/utils";
describe("parseYouTubeVideoId", () => {
test("parses watch URLs", () => {
expect(parseYouTubeVideoId(new URL("https://www.youtube.com/watch?v=abc123"))).toBe("abc123");
});
test("parses youtu.be URLs", () => {
expect(parseYouTubeVideoId(new URL("https://youtu.be/abc123"))).toBe("abc123");
});
test("parses shorts URLs", () => {
expect(parseYouTubeVideoId(new URL("https://www.youtube.com/shorts/abc123"))).toBe("abc123");
});
test("parses embed URLs", () => {
expect(parseYouTubeVideoId(new URL("https://www.youtube.com/embed/abc123"))).toBe("abc123");
});
});
describe("parseYouTubeDescriptionChapters", () => {
test("extracts chapter timestamps from description lines", () => {
expect(
parseYouTubeDescriptionChapters(`0:00 Intro
2:15 What is a product engineer?
10:05 Career paths`),
).toEqual([
{ title: "Intro", time: 0 },
{ title: "What is a product engineer?", time: 135 },
{ title: "Career paths", time: 605 },
]);
});
test("ignores isolated timestamps that do not look like chapters", () => {
expect(parseYouTubeDescriptionChapters("Published on 2026-03-26\nSee you at 1:23")).toEqual([]);
});
});
describe("renderYouTubeTranscriptMarkdown", () => {
test("renders description before chapters and keeps every segment on its own line", () => {
const markdown = renderYouTubeTranscriptMarkdown({
description: "Line one\nLine two",
chapters: [
{ title: "Intro", time: 0 },
{ title: "Deep Dive", time: 4 },
],
segments: [
{ start: 0, end: 2, text: "Hello everyone." },
{ start: 2, end: 4, text: "Welcome back." },
{ start: 4, end: 7, text: "Now the details." },
],
});
expect(markdown).toContain("## Description");
expect(markdown).toContain("Line one \nLine two");
expect(markdown).toContain("## Chapters");
expect(markdown).toContain("### Intro [0:00 -> 0:04]");
expect(markdown).toContain("[0:00 -> 0:02] Hello everyone.");
expect(markdown).toContain("[0:02 -> 0:04] Welcome back.");
expect(markdown).toContain("### Deep Dive [0:04 -> 0:07]");
expect(markdown).toContain("[0:04 -> 0:07] Now the details.");
});
test("falls back to a transcript section when chapters are unavailable", () => {
const markdown = renderYouTubeTranscriptMarkdown({
segments: [{ start: 65, end: 70, text: "Single line." }],
chapters: [],
});
expect(markdown).toContain("## Transcript");
expect(markdown).toContain("[1:05 -> 1:10] Single line.");
});
});
describe("thumbnail helpers", () => {
test("prefers max resolution thumbnail candidates before listed fallbacks", () => {
expect(
buildYouTubeThumbnailCandidates("abc123", [
"https://i.ytimg.com/vi/abc123/hqdefault.jpg",
"https://i.ytimg.com/vi/abc123/mqdefault.jpg?foo=bar",
]),
).toEqual([
"https://i.ytimg.com/vi/abc123/maxresdefault.jpg",
"https://i.ytimg.com/vi/abc123/sddefault.jpg",
"https://i.ytimg.com/vi/abc123/hqdefault.jpg",
"https://i.ytimg.com/vi/abc123/mqdefault.jpg",
"https://i.ytimg.com/vi/abc123/default.jpg",
]);
});
test("renders timestamp ranges with start and end values", () => {
expect(formatTimestampRange(3661, 3675)).toBe("[1:01:01 -> 1:01:15]");
});
});
@@ -0,0 +1,74 @@
import type { Adapter } from "../types";
import { detectInteractionGate } from "../../browser/interaction-gates";
import { captureNormalizedPageSnapshot } from "../../browser/page-snapshot";
import { convertHtmlToMarkdown } from "../../extract/html-to-markdown";
export const genericAdapter: Adapter = {
name: "generic",
match() {
return true;
},
async process(context) {
context.log.info(`Loading ${context.input.url.toString()} with generic adapter`);
await context.browser.goto(context.input.url.toString(), context.timeoutMs);
try {
await context.network.waitForIdle({
idleMs: 1_200,
timeoutMs: Math.min(context.timeoutMs, 15_000),
});
} catch {
context.log.debug("Network idle timed out on initial load; continuing.");
}
await context.browser.scrollToEnd({ maxSteps: 4, delayMs: 300 });
try {
await context.network.waitForIdle({
idleMs: 900,
timeoutMs: Math.min(context.timeoutMs, 10_000),
});
} catch {
context.log.debug("Network idle timed out after scrolling; continuing.");
}
const interaction = await detectInteractionGate(context.browser);
if (interaction) {
return {
status: "needs_interaction",
interaction,
};
}
const snapshot = await captureNormalizedPageSnapshot(context.browser);
const converted = await convertHtmlToMarkdown(snapshot.html, snapshot.finalUrl, {
enableRemoteMarkdownFallback: context.outputFormat === "markdown",
preserveBase64Images: context.downloadMedia,
});
const document = {
url: snapshot.finalUrl,
canonicalUrl: converted.metadata.canonicalUrl,
title: converted.metadata.title,
author: converted.metadata.author,
siteName: converted.metadata.siteName,
publishedAt: converted.metadata.publishedAt,
summary: converted.metadata.summary,
adapter: "generic",
metadata: {
coverImage: converted.metadata.coverImage,
language: converted.metadata.language,
capturedAt: converted.metadata.capturedAt,
conversionMethod: converted.conversionMethod,
fallbackReason: converted.fallbackReason,
kind: "generic/article",
},
content: converted.markdown ? [{ type: "markdown" as const, markdown: converted.markdown }] : [],
};
return {
status: "ok",
document,
media: converted.media,
};
},
};
@@ -0,0 +1,391 @@
import { JSDOM } from "jsdom";
import TurndownService from "turndown";
import { gfm } from "turndown-plugin-gfm";
import type { Adapter } from "../types";
import type { ExtractedDocument } from "../../extract/document";
import { collectMediaFromDocument } from "../../media/markdown-media";
const HN_BASE_URL = "https://news.ycombinator.com";
const turndown = new TurndownService({
headingStyle: "atx",
bulletListMarker: "-",
codeBlockStyle: "fenced",
});
turndown.use(gfm);
export interface HnItem {
id: number;
type: "story" | "comment" | "job" | "poll" | "pollopt" | string;
by?: string;
time?: number;
text?: string;
title?: string;
url?: string;
score?: number;
descendants?: number;
kids?: number[];
parent?: number;
deleted?: boolean;
dead?: boolean;
}
export interface HnCommentNode {
item: HnItem;
children: HnCommentNode[];
}
interface ParsedHnThread {
story: HnItem;
comments: HnCommentNode[];
}
function decodeHtmlText(value: string | undefined): string | undefined {
if (!value) {
return undefined;
}
const dom = new JSDOM(`<!doctype html><html><body>${value}</body></html>`);
return dom.window.document.body.textContent?.trim() || undefined;
}
function normalizeMarkdown(markdown: string): string {
return markdown
.replace(/\r\n/g, "\n")
.replace(/[ \t]+\n/g, "\n")
.replace(/\n{3,}/g, "\n\n")
.trim();
}
function convertHnHtmlToMarkdown(html: string | undefined, baseUrl: string): string {
if (!html?.trim()) {
return "";
}
const dom = new JSDOM(`<div id="__root">${html}</div>`, { url: baseUrl });
const root = dom.window.document.querySelector("#__root");
if (!root) {
return "";
}
root.querySelectorAll("a[href]").forEach((element) => {
const href = element.getAttribute("href");
if (!href) {
return;
}
try {
element.setAttribute("href", new URL(href, baseUrl).toString());
} catch {
// Ignore malformed URLs and keep the original href.
}
});
return normalizeMarkdown(turndown.turndown(root.innerHTML));
}
function formatIsoTimestamp(unixSeconds: number | undefined): string | undefined {
if (!unixSeconds || !Number.isFinite(unixSeconds)) {
return undefined;
}
return new Date(unixSeconds * 1_000).toISOString();
}
function formatDisplayTimestamp(unixSeconds: number | undefined): string {
const iso = formatIsoTimestamp(unixSeconds);
if (!iso) {
return "unknown time";
}
return iso.replace("T", " ").replace(".000Z", " UTC");
}
function indentMarkdown(markdown: string, spaces: number): string {
const prefix = " ".repeat(spaces);
return markdown
.split("\n")
.map((line) => (line ? `${prefix}${line}` : prefix))
.join("\n");
}
function renderCommentHeader(item: HnItem, pageUrl: string): string {
const author = item.by ?? "[deleted]";
const time = item.id
? `[${formatDisplayTimestamp(item.time)}](${pageUrl}#${item.id})`
: formatDisplayTimestamp(item.time);
return `${author} · ${time}`;
}
function renderCommentNode(node: HnCommentNode, pageUrl: string, depth = 0): string {
const baseIndent = " ".repeat(depth * 4);
const lines = [`${baseIndent}- ${renderCommentHeader(node.item, pageUrl)}`];
const body = convertHnHtmlToMarkdown(node.item.text, pageUrl);
if (body) {
lines.push("");
lines.push(indentMarkdown(body, depth * 4 + 4));
} else if (node.item.deleted || node.item.dead) {
lines.push("");
lines.push(`${baseIndent} [comment unavailable]`);
}
for (const child of node.children) {
lines.push("");
lines.push(renderCommentNode(child, pageUrl, depth + 1));
}
return lines.join("\n");
}
export function buildHnThreadMarkdown(
story: HnItem,
comments: HnCommentNode[],
pageUrl: string,
): string {
const lines: string[] = [];
const storyUrl = story.url ? new URL(story.url, pageUrl).toString() : undefined;
const storyText = convertHnHtmlToMarkdown(story.text, pageUrl);
if (storyUrl && storyUrl !== pageUrl) {
lines.push(`Source: [${storyUrl}](${storyUrl})`);
}
lines.push(`HN Item: [${story.id}](${pageUrl})`);
const submittedBy = story.by ? ` by ${story.by}` : "";
const submittedAt = formatDisplayTimestamp(story.time);
lines.push(`Submitted${submittedBy} at ${submittedAt}`);
const stats: string[] = [];
if (typeof story.score === "number") {
stats.push(`${story.score} points`);
}
if (typeof story.descendants === "number") {
stats.push(`${story.descendants} comments`);
}
if (stats.length > 0) {
lines.push(stats.join(" | "));
}
if (storyText) {
lines.push("");
lines.push("## Post");
lines.push("");
lines.push(storyText);
}
lines.push("");
lines.push("## Comments");
lines.push("");
if (comments.length === 0) {
lines.push("No comments.");
} else {
lines.push(comments.map((comment) => renderCommentNode(comment, pageUrl)).join("\n\n"));
}
return normalizeMarkdown(lines.join("\n"));
}
export function buildHnDocument(
story: HnItem,
comments: HnCommentNode[],
pageUrl: string,
): ExtractedDocument {
const decodedTitle = decodeHtmlText(story.title) ?? `HN Item ${story.id}`;
return {
url: pageUrl,
canonicalUrl: pageUrl,
title: decodedTitle,
author: story.by,
siteName: "Hacker News",
publishedAt: formatIsoTimestamp(story.time),
adapter: "hn",
metadata: {
kind: "hn/story",
storyId: story.id,
storyUrl: story.url ? new URL(story.url, pageUrl).toString() : undefined,
points: story.score,
commentCount: story.descendants,
},
content: [
{
type: "markdown",
markdown: buildHnThreadMarkdown(story, comments, pageUrl),
},
],
};
}
export function parseHnItemId(url: URL): number | null {
if (url.hostname !== "news.ycombinator.com") {
return null;
}
if (url.pathname !== "/item") {
return null;
}
const value = url.searchParams.get("id");
if (!value || !/^\d+$/.test(value)) {
return null;
}
return Number(value);
}
function extractUnixSecondsFromAge(element: Element | null): number | undefined {
const title = element?.getAttribute("title")?.trim();
if (!title) {
return undefined;
}
const match = title.match(/(\d{9,})$/);
return match ? Number(match[1]) : undefined;
}
function extractScore(text: string | null | undefined): number | undefined {
if (!text) {
return undefined;
}
const match = text.match(/(\d+)/);
return match ? Number(match[1]) : undefined;
}
function extractCommentCount(container: ParentNode): number | undefined {
const anchors = Array.from(container.querySelectorAll("a"));
for (const anchor of anchors) {
const match = anchor.textContent?.trim().match(/(\d+)\s+comments?/i);
if (match) {
return Number(match[1]);
}
}
return undefined;
}
function normalizeStoryUrl(storyId: number, href: string | null | undefined, pageUrl: string): string | undefined {
if (!href) {
return undefined;
}
try {
const resolved = new URL(href, pageUrl).toString();
if (resolved === pageUrl || resolved === `${HN_BASE_URL}/item?id=${storyId}`) {
return undefined;
}
return resolved;
} catch {
return undefined;
}
}
export function extractHnThreadFromHtml(html: string, pageUrl: string): ParsedHnThread | null {
const dom = new JSDOM(html, { url: pageUrl });
const { document } = dom.window;
const storyRow = document.querySelector("table.fatitem tr.athing.submission");
if (!storyRow) {
return null;
}
const storyId = Number(storyRow.getAttribute("id"));
if (!Number.isFinite(storyId)) {
return null;
}
const titleLink = storyRow.querySelector(".titleline > a");
const subline = document.querySelector("table.fatitem .subline");
const topText = document.querySelector("table.fatitem .toptext");
const story: HnItem = {
id: storyId,
type: "story",
by: subline?.querySelector(".hnuser")?.textContent?.trim() || undefined,
time: extractUnixSecondsFromAge(subline?.querySelector(".age") ?? null),
title: titleLink?.innerHTML?.trim() || undefined,
url: normalizeStoryUrl(storyId, titleLink?.getAttribute("href"), pageUrl),
text: topText?.innerHTML?.trim() || undefined,
score: extractScore(subline?.querySelector(".score")?.textContent),
descendants: extractCommentCount(subline ?? document),
};
const roots: HnCommentNode[] = [];
const stack: HnCommentNode[] = [];
document.querySelectorAll("tr.athing.comtr").forEach((row) => {
const commentId = Number(row.getAttribute("id"));
if (!Number.isFinite(commentId)) {
return;
}
const indentRaw = row.querySelector("td.ind")?.getAttribute("indent");
const depth = indentRaw && /^\d+$/.test(indentRaw) ? Number(indentRaw) : 0;
const comhead = row.querySelector(".comhead");
const item: HnItem = {
id: commentId,
type: "comment",
by: comhead?.querySelector(".hnuser")?.textContent?.trim() || undefined,
time: extractUnixSecondsFromAge(comhead?.querySelector(".age") ?? null),
text: row.querySelector(".comment > .commtext")?.innerHTML?.trim() || undefined,
deleted: row.querySelector(".comment > .commtext") === null,
};
const node: HnCommentNode = {
item,
children: [],
};
while (stack.length > depth) {
stack.pop();
}
const parent = stack[stack.length - 1];
if (parent) {
parent.children.push(node);
} else {
roots.push(node);
}
stack.push(node);
});
return {
story,
comments: roots,
};
}
export const hnAdapter: Adapter = {
name: "hn",
match(input) {
return parseHnItemId(input.url) !== null;
},
async process(context) {
const itemId = parseHnItemId(context.input.url);
if (!itemId) {
return {
status: "no_document",
};
}
const pageUrl = context.input.url.toString();
context.log.info(`Loading ${pageUrl} with hn adapter`);
await context.browser.goto(pageUrl, context.timeoutMs);
const html = await context.browser.getHTML();
const thread = extractHnThreadFromHtml(html, pageUrl);
if (!thread) {
return {
status: "no_document",
};
}
const document = buildHnDocument(thread.story, thread.comments, pageUrl);
return {
status: "ok",
document,
media: collectMediaFromDocument(document),
};
},
};
@@ -0,0 +1,29 @@
import type { Adapter, AdapterInput } from "./types";
import { genericAdapter } from "./generic";
import { hnAdapter } from "./hn";
import { xAdapter } from "./x";
import { youtubeAdapter } from "./youtube";
const adapters: Adapter[] = [xAdapter, youtubeAdapter, hnAdapter, genericAdapter];
export function listAdapters(): Adapter[] {
return adapters;
}
export function resolveAdapter(input: AdapterInput, forcedName?: string): Adapter {
if (forcedName) {
const forced = adapters.find((adapter) => adapter.name === forcedName);
if (!forced) {
throw new Error(`Unknown adapter: ${forcedName}`);
}
return forced;
}
const matched = adapters.find((adapter) => adapter.match(input));
if (!matched) {
throw new Error("No adapter matched the URL");
}
return matched;
}
export { genericAdapter };
@@ -0,0 +1,73 @@
import type { BrowserSession } from "../browser/session";
import type { CdpClient } from "../browser/cdp-client";
import type { NetworkJournal } from "../browser/network-journal";
import type { ExtractedDocument } from "../extract/document";
import type { MediaDownloadRequest, MediaDownloadResult, MediaAsset } from "../media/types";
import type { Logger } from "../utils/logger";
export interface AdapterInput {
url: URL;
}
export type LoginState = "logged_in" | "logged_out" | "unknown";
export type InteractionKind = "login" | "cloudflare" | "recaptcha" | "hcaptcha" | "captcha" | "challenge";
export interface AdapterLoginInfo {
provider: string;
state: LoginState;
required?: boolean;
username?: string;
reason?: string;
}
export interface WaitForInteractionRequest {
type: "wait_for_interaction";
kind: InteractionKind;
provider: string;
prompt: string;
reason?: string;
timeoutMs?: number;
pollIntervalMs?: number;
requiresVisibleBrowser?: boolean;
}
export type AdapterProcessResult =
| {
status: "ok";
document: ExtractedDocument;
media?: MediaAsset[];
login?: AdapterLoginInfo;
}
| {
status: "needs_interaction";
interaction: WaitForInteractionRequest;
login?: AdapterLoginInfo;
}
| {
status: "no_document";
login?: AdapterLoginInfo;
};
export interface AdapterContext {
input: AdapterInput;
browser: BrowserSession;
network: NetworkJournal;
cdp: CdpClient;
log: Logger;
outputFormat: "markdown" | "json";
timeoutMs: number;
interactive: boolean;
downloadMedia: boolean;
}
export interface Adapter {
name: string;
match(input: AdapterInput): boolean;
checkLogin?(context: AdapterContext): Promise<AdapterLoginInfo>;
exportCookies?(context: AdapterContext, profileDir?: string): Promise<boolean>;
restoreCookies?(context: AdapterContext, profileDir?: string): Promise<boolean>;
downloadMedia?(request: MediaDownloadRequest): Promise<MediaDownloadResult>;
process(context: AdapterContext): Promise<AdapterProcessResult>;
}
export type { MediaAsset };
@@ -0,0 +1,461 @@
import type { ExtractedDocument } from "../../extract/document";
import {
findTweetNode,
findTweetNodeById,
formatMediaList,
formatTweetAuthor,
getTweetAuthorMetadata,
getTweetText,
getUser,
isRecord,
normalizeTitle,
resolveBestXVideoVariantUrl,
toHighResXImageUrl,
toXTweet,
} from "./shared";
import type { JsonObject } from "./types";
interface ArticleMedia {
kind: "image" | "video";
url: string;
}
function resolveArticleMedia(mediaInfo: JsonObject): ArticleMedia | null {
const videoUrl = resolveBestXVideoVariantUrl(mediaInfo);
if (videoUrl) {
return {
kind: "video",
url: videoUrl,
};
}
const rawUrl =
(typeof mediaInfo.original_img_url === "string" && mediaInfo.original_img_url) ||
(typeof mediaInfo.url === "string" && mediaInfo.url) ||
"";
if (!rawUrl) {
return null;
}
return {
kind: "image",
url: toHighResXImageUrl(rawUrl),
};
}
function resolveArticleMediaUrl(mediaInfo: JsonObject): string {
return resolveArticleMedia(mediaInfo)?.url ?? "";
}
function normalizeEntityMap(entityMap: unknown): Map<string, JsonObject> {
const normalized = new Map<string, JsonObject>();
if (Array.isArray(entityMap)) {
for (const entry of entityMap) {
if (!isRecord(entry)) {
continue;
}
const key =
typeof entry.key === "string" || typeof entry.key === "number"
? String(entry.key)
: undefined;
const value = isRecord(entry.value) ? entry.value : undefined;
if (!key || !value) {
continue;
}
normalized.set(key, value);
}
return normalized;
}
if (!isRecord(entityMap)) {
return normalized;
}
for (const [key, value] of Object.entries(entityMap)) {
if (!isRecord(value)) {
continue;
}
normalized.set(key, value);
}
return normalized;
}
function getEntityMarkdown(entityMap: Map<string, JsonObject>, entityKey: unknown): string | null {
const key =
typeof entityKey === "string" || typeof entityKey === "number"
? String(entityKey)
: undefined;
if (!key) {
return null;
}
const entity = entityMap.get(key);
if (!entity || entity.type !== "MARKDOWN") {
return null;
}
const data = isRecord(entity.data) ? entity.data : {};
if (typeof data.markdown !== "string") {
return null;
}
const markdown = data.markdown.trim();
return markdown || null;
}
function getLinkUrl(entityMap: Map<string, JsonObject>, entityKey: unknown): string | null {
const key =
typeof entityKey === "string" || typeof entityKey === "number"
? String(entityKey)
: undefined;
if (!key) {
return null;
}
const entity = entityMap.get(key);
if (!entity || entity.type !== "LINK") {
return null;
}
const data = isRecord(entity.data) ? entity.data : {};
const candidates = [
data.expanded_url,
data.expandedUrl,
data.original_url,
data.originalUrl,
data.url,
data.display_url,
data.displayUrl,
];
for (const candidate of candidates) {
if (typeof candidate === "string" && candidate.trim()) {
return candidate.trim();
}
}
return null;
}
function getTweetId(entityMap: Map<string, JsonObject>, entityKey: unknown): string | null {
const key =
typeof entityKey === "string" || typeof entityKey === "number"
? String(entityKey)
: undefined;
if (!key) {
return null;
}
const entity = entityMap.get(key);
if (!entity || entity.type !== "TWEET") {
return null;
}
const data = isRecord(entity.data) ? entity.data : {};
if (typeof data.tweetId !== "string") {
return null;
}
return data.tweetId;
}
function buildMediaMap(articleResult: JsonObject): Map<string, ArticleMedia> {
const mediaMap = new Map<string, ArticleMedia>();
const mediaEntities = Array.isArray(articleResult.media_entities) ? articleResult.media_entities : [];
for (const entity of mediaEntities) {
if (!isRecord(entity) || typeof entity.media_id !== "string" || !isRecord(entity.media_info)) {
continue;
}
const media = resolveArticleMedia(entity.media_info);
if (media) {
mediaMap.set(entity.media_id, media);
}
}
const coverMedia = isRecord(articleResult.cover_media) ? articleResult.cover_media : null;
if (coverMedia && typeof coverMedia.media_id === "string" && isRecord(coverMedia.media_info)) {
const media = resolveArticleMedia(coverMedia.media_info);
if (media) {
mediaMap.set(coverMedia.media_id, media);
}
}
return mediaMap;
}
function getMediaMarkdown(
entityMap: Map<string, JsonObject>,
entityKey: unknown,
mediaMap: Map<string, ArticleMedia>,
): string[] {
const key =
typeof entityKey === "string" || typeof entityKey === "number"
? String(entityKey)
: undefined;
if (!key) {
return [];
}
const entity = entityMap.get(key);
if (!entity || entity.type !== "MEDIA") {
return [];
}
const data = isRecord(entity.data) ? entity.data : {};
const mediaItems = Array.isArray(data.mediaItems) ? data.mediaItems : [];
const media: ArticleMedia[] = [];
for (const item of mediaItems) {
if (!isRecord(item) || typeof item.mediaId !== "string") {
continue;
}
const mediaItem = mediaMap.get(item.mediaId);
if (mediaItem && !media.some((value) => value.url === mediaItem.url)) {
media.push(mediaItem);
}
}
return media.map((item) => item.kind === "image" ? `![](${item.url})` : `[video](${item.url})`);
}
function resolveTweetMarkdown(payloads: unknown[], tweetId: string, pageUrl: string): string | null {
for (const payload of payloads) {
const tweet = findTweetNodeById(payload, tweetId);
if (!tweet) {
continue;
}
const xTweet = toXTweet(tweet, pageUrl);
const author = formatTweetAuthor(xTweet) ?? xTweet.url;
const lines = [`> ${author}`, ...xTweet.text.split("\n").map((line) => `> ${line}`)];
const media = formatMediaList(xTweet.media).map((line) =>
line.startsWith("photo: ") ? `> ![](${line.slice("photo: ".length)})` : `> - ${line}`,
);
const parts = [lines.join("\n")];
if (media.length > 0) {
parts.push([">", ...media].join("\n"));
}
parts.push(`> ${xTweet.url}`);
return parts.join("\n").trim();
}
return `> Embedded tweet: https://x.com/i/status/${tweetId}`;
}
function replaceLinkEntities(text: string, block: JsonObject, entityMap: Map<string, JsonObject>): string {
const entityRanges = Array.isArray(block.entityRanges) ? block.entityRanges : [];
const replacements = entityRanges
.filter((range): range is JsonObject => isRecord(range))
.map((range) => {
const offset = typeof range.offset === "number" ? range.offset : -1;
const length = typeof range.length === "number" ? range.length : -1;
const url = getLinkUrl(entityMap, range.key);
return { offset, length, url };
})
.filter((range) => range.offset >= 0 && range.length > 0 && range.url)
.sort((left, right) => right.offset - left.offset);
let next = text;
for (const replacement of replacements) {
next =
next.slice(0, replacement.offset) +
replacement.url +
next.slice(replacement.offset + replacement.length);
}
return next;
}
function renderAtomicBlock(
block: JsonObject,
entityMap: Map<string, JsonObject>,
mediaMap: Map<string, ArticleMedia>,
payloads: unknown[],
pageUrl: string,
): string | null {
const entityRanges = Array.isArray(block.entityRanges) ? block.entityRanges : [];
const parts: string[] = [];
for (const range of entityRanges) {
if (!isRecord(range)) {
continue;
}
const markdown = getEntityMarkdown(entityMap, range.key);
if (markdown) {
parts.push(markdown);
continue;
}
const mediaMarkdown = getMediaMarkdown(entityMap, range.key, mediaMap);
if (mediaMarkdown.length > 0) {
parts.push(mediaMarkdown.join("\n\n"));
continue;
}
const tweetId = getTweetId(entityMap, range.key);
if (tweetId) {
const tweetMarkdown = resolveTweetMarkdown(payloads, tweetId, pageUrl);
if (tweetMarkdown) {
parts.push(tweetMarkdown);
}
}
}
if (parts.length === 0) {
return null;
}
return parts.join("\n\n");
}
function renderArticleBlocks(
blocks: unknown[],
entityMap: Map<string, JsonObject>,
mediaMap: Map<string, ArticleMedia>,
payloads: unknown[],
pageUrl: string,
): string {
const parts: string[] = [];
let orderedCounter = 0;
for (const block of blocks) {
if (!isRecord(block)) {
continue;
}
const blockType = typeof block.type === "string" ? block.type : "unstyled";
const rawText = typeof block.text === "string" ? block.text : "";
const text = replaceLinkEntities(rawText, block, entityMap).trim();
if (!text && blockType !== "atomic") {
continue;
}
if (blockType !== "ordered-list-item") {
orderedCounter = 0;
}
switch (blockType) {
case "header-one":
parts.push(`# ${text}`);
break;
case "header-two":
parts.push(`## ${text}`);
break;
case "header-three":
parts.push(`### ${text}`);
break;
case "blockquote":
parts.push(`> ${text}`);
break;
case "unordered-list-item":
parts.push(`- ${text}`);
break;
case "ordered-list-item":
orderedCounter += 1;
parts.push(`${orderedCounter}. ${text}`);
break;
case "code-block":
parts.push(`\`\`\`\n${text}\n\`\`\``);
break;
case "atomic": {
const markdown = renderAtomicBlock(block, entityMap, mediaMap, payloads, pageUrl);
if (markdown) {
parts.push(markdown);
}
break;
}
default:
parts.push(text);
break;
}
}
return parts.join("\n\n").trim();
}
function getArticleResult(tweet: JsonObject): JsonObject | null {
if (
isRecord(tweet.article) &&
isRecord(tweet.article.article_results) &&
isRecord(tweet.article.article_results.result)
) {
return tweet.article.article_results.result as JsonObject;
}
return null;
}
function extractSummary(markdown: string): string | undefined {
const segments = markdown
.split(/\n\n+/)
.map((segment) => segment.trim())
.filter(Boolean);
const preferred = segments.find((segment) => !/^(#|>|- |\d+\. |\`\`\`)/.test(segment));
return preferred?.slice(0, 220);
}
export function extractArticleDocumentFromPayload(
payload: unknown,
statusId: string,
pageUrl: string,
payloads: unknown[] = [payload],
): ExtractedDocument | null {
const tweet = findTweetNode(payload, statusId);
if (!tweet) {
return null;
}
const articleResult = getArticleResult(tweet);
if (!articleResult) {
return null;
}
const title = typeof articleResult.title === "string" ? articleResult.title.trim() : undefined;
const contentState = isRecord(articleResult.content_state) ? articleResult.content_state : {};
const blocks = Array.isArray(contentState.blocks) ? contentState.blocks : [];
const entityMap = normalizeEntityMap(contentState.entityMap);
const mediaMap = buildMediaMap(articleResult);
const richMarkdown = renderArticleBlocks(blocks, entityMap, mediaMap, payloads, pageUrl);
const plainText = typeof articleResult.plain_text === "string" ? articleResult.plain_text.trim() : "";
const markdown = richMarkdown || plainText || getTweetText(tweet);
if (!markdown) {
return null;
}
const xTweet = toXTweet(tweet, pageUrl);
const user = getUser(tweet);
const coverMedia = isRecord(articleResult.cover_media) ? articleResult.cover_media : null;
const coverMediaInfo = coverMedia && isRecord(coverMedia.media_info) ? coverMedia.media_info : null;
const coverImage = coverMediaInfo ? resolveArticleMediaUrl(coverMediaInfo) || undefined : undefined;
return {
url: pageUrl,
canonicalUrl: xTweet.url,
title: title || normalizeTitle(xTweet.text, "X Article"),
author: formatTweetAuthor(xTweet),
siteName: "X",
publishedAt: xTweet.createdAt,
summary: extractSummary(markdown) || xTweet.text.slice(0, 200) || undefined,
adapter: "x",
metadata: {
kind: "x/article",
tweetId: xTweet.id,
coverImage,
authorName: xTweet.authorName ?? user.name,
authorUsername: xTweet.author ?? user.screenName,
authorUrl: (xTweet.author ?? user.screenName) ? `https://x.com/${xTweet.author ?? user.screenName}` : undefined,
...getTweetAuthorMetadata(xTweet),
},
content: [{ type: "markdown", markdown }],
};
}
@@ -0,0 +1,134 @@
import type { Adapter, AdapterLoginInfo } from "../types";
import { exportCookies, restoreCookies, type CookieSidecarConfig } from "../../browser/cookie-sidecar";
import { detectInteractionGate } from "../../browser/interaction-gates";
import type { ExtractedDocument } from "../../extract/document";
import { collectMediaFromDocument } from "../../media/markdown-media";
import { extractArticleDocumentFromPayload } from "./article";
import { buildNeedsLoginResult, detectXLogin } from "./login";
import { extractStatusId, isXHost } from "./match";
import { collectXJsonPayloads, waitForInitialXPayload } from "./payloads";
import { extractSingleTweetDocumentFromPayload } from "./single";
import { extractThreadDocumentFromPayloads } from "./thread";
import { loadFullXThread } from "./thread-loader";
const cookieConfig: CookieSidecarConfig = {
urls: ["https://x.com/", "https://twitter.com/"],
filename: "x-session-cookies.json",
requiredCookieNames: ["auth_token", "ct0"],
filterCookie: (c) => {
const d = c.domain ?? "";
return d.endsWith("x.com") || d.endsWith("twitter.com");
},
};
function extractDocumentFromPayloads(
payloads: unknown[],
statusId: string,
pageUrl: string,
): ExtractedDocument | null {
for (const payload of payloads) {
const articleDocument = extractArticleDocumentFromPayload(payload, statusId, pageUrl, payloads);
if (articleDocument) {
return articleDocument;
}
}
const threadDocument = extractThreadDocumentFromPayloads(payloads, statusId, pageUrl);
if (threadDocument) {
return threadDocument;
}
for (const payload of payloads) {
const singleDocument = extractSingleTweetDocumentFromPayload(payload, statusId, pageUrl);
if (singleDocument) {
return singleDocument;
}
}
return null;
}
async function ensureXLoginState(context: Parameters<Adapter["process"]>[0]): Promise<AdapterLoginInfo> {
return detectXLogin(context);
}
export const xAdapter: Adapter = {
name: "x",
match(input) {
return isXHost(input.url.hostname);
},
async checkLogin(context) {
return detectXLogin(context);
},
async exportCookies(context, profileDir) {
return exportCookies(context.browser.targetSession, cookieConfig, profileDir);
},
async restoreCookies(context, profileDir) {
return restoreCookies(context.browser.targetSession, cookieConfig, profileDir);
},
async process(context) {
const statusId = extractStatusId(context.input.url);
if (!statusId) {
return {
status: "no_document",
};
}
context.log.info(`Loading ${context.input.url.toString()} with x adapter`);
await context.browser.goto(context.input.url.toString(), context.timeoutMs);
const interaction = await detectInteractionGate(context.browser);
if (interaction) {
return {
status: "needs_interaction",
interaction,
};
}
let login = await ensureXLoginState(context);
if (login.state === "logged_out") {
return buildNeedsLoginResult(login);
}
await waitForInitialXPayload(context);
await loadFullXThread(context, statusId);
const pageUrl = await context.browser.getURL();
const postLoadInteraction = await detectInteractionGate(context.browser);
if (postLoadInteraction) {
return {
status: "needs_interaction",
interaction: postLoadInteraction,
login,
};
}
login = await ensureXLoginState(context).catch(() => login);
if (login.state === "logged_out") {
return buildNeedsLoginResult(login);
}
const payloads = await collectXJsonPayloads(context);
if (payloads.length === 0) {
return {
status: "no_document",
login,
};
}
const document = extractDocumentFromPayloads(payloads, statusId, pageUrl);
if (document) {
return {
status: "ok",
document,
media: collectMediaFromDocument(document),
login,
};
}
return {
status: "no_document",
login,
};
},
};
@@ -0,0 +1,80 @@
import type { AdapterContext, AdapterLoginInfo, AdapterProcessResult } from "../types";
interface XLoginSnapshot {
currentUrl: string;
hasAccountMenu: boolean;
hasLoginInputs: boolean;
bodyText: string;
}
export async function detectXLogin(context: AdapterContext): Promise<AdapterLoginInfo> {
const snapshot = await context.browser.evaluate<XLoginSnapshot>(`
(() => {
const bodyText = (document.body?.innerText ?? "").slice(0, 2500);
return {
currentUrl: window.location.href,
hasAccountMenu: Boolean(
document.querySelector(
'[data-testid="SideNav_AccountSwitcher_Button"], [data-testid="AppTabBar_Profile_Link"], [aria-label="Account menu"]'
)
),
hasLoginInputs: Boolean(
document.querySelector(
'input[name="text"], input[name="password"], input[autocomplete="username"], input[autocomplete="current-password"]'
)
),
bodyText,
};
})()
`).catch(async () => ({
currentUrl: await context.browser.getURL().catch(() => context.input.url.toString()),
hasAccountMenu: false,
hasLoginInputs: false,
bodyText: "",
}));
if (
/\/i\/flow\/login|\/login/i.test(snapshot.currentUrl) ||
snapshot.hasLoginInputs ||
/sign in to x|join x today|登录 x|注册 x|登录到 x/i.test(snapshot.bodyText)
) {
return {
provider: "x",
state: "logged_out",
required: true,
reason: "X login page detected",
};
}
if (snapshot.hasAccountMenu) {
return {
provider: "x",
state: "logged_in",
};
}
return {
provider: "x",
state: "unknown",
};
}
export function buildNeedsLoginResult(login: AdapterLoginInfo): AdapterProcessResult {
return {
status: "needs_interaction",
login: {
...login,
provider: "x",
state: login.state === "logged_in" ? "unknown" : login.state,
required: true,
},
interaction: {
type: "wait_for_interaction",
kind: "login",
provider: "x",
reason: login.reason,
prompt: "Please sign in to X in the opened Chrome window. Extraction will continue automatically once login is detected.",
requiresVisibleBrowser: true,
},
};
}
@@ -0,0 +1,9 @@
export function isXHost(hostname: string): boolean {
return ["x.com", "www.x.com", "twitter.com", "www.twitter.com"].includes(hostname);
}
export function extractStatusId(url: URL): string | undefined {
const match = url.pathname.match(/\/(?:status|article)\/(\d+)/);
return match?.[1];
}
@@ -0,0 +1,50 @@
import type { AdapterContext } from "../types";
import { filterXGraphQlEntries } from "./shared";
export function getRelevantXThreadEntries(context: AdapterContext) {
return filterXGraphQlEntries(context.network.getEntries()).filter(
(entry) =>
entry.method === "GET" &&
entry.finished &&
(
entry.url.includes("TweetDetail") ||
entry.url.includes("TweetResultByRestId") ||
entry.url.includes("TweetResultsByRestIds")
),
);
}
export async function prefetchRelevantXThreadBodies(context: AdapterContext): Promise<void> {
const entries = getRelevantXThreadEntries(context).filter((entry) => entry.body === undefined && !entry.bodyError);
for (const entry of entries) {
await context.network.ensureBody(entry);
}
}
export async function collectXJsonPayloads(context: AdapterContext): Promise<unknown[]> {
await prefetchRelevantXThreadBodies(context);
const entries = getRelevantXThreadEntries(context);
const payloads: unknown[] = [];
for (const entry of entries) {
const payload = await context.network.getJsonBody(entry);
if (payload) {
payloads.push(payload);
}
}
return payloads;
}
export async function waitForInitialXPayload(context: AdapterContext): Promise<void> {
try {
await context.network.waitForResponse(
(entry) =>
entry.url.includes("/graphql/") &&
(entry.url.includes("TweetDetail") || entry.url.includes("TweetResultByRestId")),
{ timeoutMs: Math.min(context.timeoutMs, 15_000) },
);
await prefetchRelevantXThreadBodies(context);
} catch {
context.log.debug("No tweet GraphQL response observed before timeout.");
}
}
@@ -0,0 +1,47 @@
import type { AdapterContext } from "../types";
const X_SESSION_URLS = ["https://x.com/", "https://twitter.com/"] as const;
const REQUIRED_X_SESSION_COOKIES = ["auth_token", "ct0"] as const;
interface CookieLike {
name?: string;
value?: string | null;
}
interface NetworkGetCookiesResult {
cookies?: CookieLike[];
}
export function buildXSessionCookieMap(cookies: readonly CookieLike[]): Record<string, string> {
const cookieMap: Record<string, string> = {};
for (const cookie of cookies) {
const name = cookie.name?.trim();
const value = cookie.value?.trim();
if (!name || !value) {
continue;
}
cookieMap[name] = value;
}
return cookieMap;
}
export function hasRequiredXSessionCookies(cookieMap: Record<string, string>): boolean {
return REQUIRED_X_SESSION_COOKIES.every((name) => Boolean(cookieMap[name]));
}
export async function readXSessionCookieMap(
context: Pick<AdapterContext, "browser">,
): Promise<Record<string, string>> {
const { cookies } = await context.browser.targetSession.send<NetworkGetCookiesResult>(
"Network.getCookies",
{ urls: [...X_SESSION_URLS] },
);
return buildXSessionCookieMap(cookies ?? []);
}
export async function isXSessionReady(
context: Pick<AdapterContext, "browser">,
): Promise<boolean> {
const cookieMap = await readXSessionCookieMap(context);
return hasRequiredXSessionCookies(cookieMap);
}
@@ -0,0 +1,423 @@
import path from "node:path";
import type { NetworkEntry } from "../../browser/network-journal";
import type { XMedia, XQuotedTweet, XTweet, XUser, JsonObject } from "./types";
const X_IMAGE_EXTENSIONS = new Set(["jpg", "jpeg", "png", "webp", "gif", "bmp", "avif"]);
function emptyObject(): JsonObject {
return {};
}
export function isRecord(value: unknown): value is JsonObject {
return Boolean(value) && typeof value === "object" && !Array.isArray(value);
}
export function walk(value: unknown, visitor: (node: unknown) => boolean | void): boolean {
if (visitor(value)) {
return true;
}
if (Array.isArray(value)) {
for (const item of value) {
if (walk(item, visitor)) {
return true;
}
}
return false;
}
if (isRecord(value)) {
for (const child of Object.values(value)) {
if (walk(child, visitor)) {
return true;
}
}
}
return false;
}
function hasTweetText(node: JsonObject): boolean {
const legacy = isRecord(node.legacy) ? node.legacy : emptyObject();
return (
typeof legacy.full_text === "string" ||
typeof getNoteTweetText(node) === "string"
);
}
export function findTweetNodeById(payload: unknown, tweetId: string): JsonObject | null {
let match: JsonObject | null = null;
walk(payload, (node) => {
if (!isRecord(node) || typeof node.rest_id !== "string" || !isRecord(node.legacy)) {
return false;
}
if (!hasTweetText(node)) {
return false;
}
if (node.rest_id === tweetId) {
match = node;
return true;
}
return false;
});
return match;
}
export function findTweetNode(payload: unknown, statusId: string): JsonObject | null {
let firstMatch: JsonObject | null = null;
const exactMatch = findTweetNodeById(payload, statusId);
if (exactMatch) {
return exactMatch;
}
walk(payload, (node) => {
if (!isRecord(node) || typeof node.rest_id !== "string" || !isRecord(node.legacy)) {
return false;
}
if (!hasTweetText(node)) {
return false;
}
if (!firstMatch) {
firstMatch = node;
}
return false;
});
return firstMatch;
}
export function getLegacy(tweet: JsonObject): JsonObject {
return isRecord(tweet.legacy) ? tweet.legacy : emptyObject();
}
export function unwrapTweetResult(node: unknown): JsonObject | null {
if (!isRecord(node)) {
return null;
}
if (node.__typename === "TweetWithVisibilityResults" && isRecord(node.tweet)) {
return unwrapTweetResult(node.tweet);
}
const tweet = isRecord(node.tweet) ? (node.tweet as JsonObject) : node;
if (typeof tweet.rest_id !== "string" || !isRecord(tweet.legacy)) {
return null;
}
return tweet;
}
export function getUser(tweet: JsonObject): XUser {
const result =
isRecord(tweet.core) &&
isRecord(tweet.core.user_results) &&
isRecord(tweet.core.user_results.result)
? (tweet.core.user_results.result as JsonObject)
: emptyObject();
const legacy = isRecord(result.legacy) ? result.legacy : emptyObject();
const core = isRecord(result.core) ? result.core : emptyObject();
return {
name:
(typeof legacy.name === "string" ? legacy.name : undefined) ??
(typeof core.name === "string" ? core.name : undefined),
screenName:
(typeof legacy.screen_name === "string" ? legacy.screen_name : undefined) ??
(typeof core.screen_name === "string" ? core.screen_name : undefined),
};
}
function getNoteTweetResult(tweet: JsonObject): JsonObject | null {
if (
!isRecord(tweet.note_tweet) ||
!isRecord(tweet.note_tweet.note_tweet_results) ||
!isRecord(tweet.note_tweet.note_tweet_results.result)
) {
return null;
}
return tweet.note_tweet.note_tweet_results.result as JsonObject;
}
function getNoteTweetText(tweet: JsonObject): string | undefined {
const noteTweet = getNoteTweetResult(tweet);
return typeof noteTweet?.text === "string" ? noteTweet.text : undefined;
}
interface TweetUrlEntity {
url: string;
expandedUrl?: string;
displayUrl?: string;
}
function collectTweetUrlEntities(values: unknown[]): TweetUrlEntity[] {
return values.reduce<TweetUrlEntity[]>((entities, value) => {
if (!isRecord(value) || typeof value.url !== "string" || !value.url) {
return entities;
}
entities.push({
url: value.url,
expandedUrl: typeof value.expanded_url === "string" ? value.expanded_url : undefined,
displayUrl: typeof value.display_url === "string" ? value.display_url : undefined,
});
return entities;
}, []);
}
function getTweetUrlEntities(tweet: JsonObject): TweetUrlEntity[] {
const noteTweet = getNoteTweetResult(tweet);
const noteTweetEntitySet = noteTweet && isRecord(noteTweet.entity_set) ? noteTweet.entity_set : emptyObject();
const noteTweetUrls = collectTweetUrlEntities(Array.isArray(noteTweetEntitySet.urls) ? noteTweetEntitySet.urls : []);
const legacy = getLegacy(tweet);
const legacyEntities = isRecord(legacy.entities) ? legacy.entities : emptyObject();
const legacyUrls = collectTweetUrlEntities(Array.isArray(legacyEntities.urls) ? legacyEntities.urls : []);
const seen = new Set<string>();
return [...noteTweetUrls, ...legacyUrls].filter((value) => {
if (seen.has(value.url)) {
return false;
}
seen.add(value.url);
return true;
});
}
export function getTweetText(tweet: JsonObject): string {
const legacy = getLegacy(tweet);
let text =
getNoteTweetText(tweet) ?? (typeof legacy.full_text === "string" ? legacy.full_text : "");
for (const value of getTweetUrlEntities(tweet)) {
const replacement =
(typeof value.expandedUrl === "string" && value.expandedUrl) ||
(typeof value.displayUrl === "string" && value.displayUrl) ||
value.url;
text = text.replaceAll(value.url, replacement);
}
const extendedEntities = isRecord(legacy.extended_entities) ? legacy.extended_entities : emptyObject();
const media = Array.isArray(extendedEntities.media) ? extendedEntities.media : [];
for (const value of media) {
if (isRecord(value) && typeof value.url === "string") {
text = text.replaceAll(value.url, "").trim();
}
}
return text.replace(/\n{3,}/g, "\n\n").trim();
}
function normalizeXImageExtension(raw: string | undefined | null): string | undefined {
if (!raw) {
return undefined;
}
const normalized = raw.replace(/^\./, "").trim().toLowerCase();
if (!normalized) {
return undefined;
}
return normalized === "jpeg" ? "jpg" : normalized;
}
export function toHighResXImageUrl(rawUrl: string): string {
try {
const parsed = new URL(rawUrl);
if (parsed.hostname.toLowerCase() !== "pbs.twimg.com") {
return rawUrl;
}
const pathExtension = normalizeXImageExtension(path.posix.extname(parsed.pathname));
const format = normalizeXImageExtension(parsed.searchParams.get("format")) ?? pathExtension;
if (!format || !X_IMAGE_EXTENSIONS.has(format)) {
return rawUrl;
}
if (pathExtension) {
parsed.pathname = parsed.pathname.replace(new RegExp(`\\.${pathExtension}$`, "i"), "");
}
parsed.searchParams.set("format", format);
parsed.searchParams.set("name", "4096x4096");
return parsed.toString();
} catch {
return rawUrl;
}
}
function getVideoVariantBitrate(variant: JsonObject): number {
const value = variant.bitrate ?? variant.bit_rate;
return typeof value === "number" && Number.isFinite(value) ? value : 0;
}
function getVideoVariantContentType(variant: JsonObject): string {
const value = variant.content_type ?? variant.contentType;
return typeof value === "string" ? value.toLowerCase() : "";
}
export function resolveBestXVideoVariantUrl(mediaInfo: unknown): string | undefined {
if (!isRecord(mediaInfo)) {
return undefined;
}
const variantsSource =
Array.isArray(mediaInfo.variants)
? mediaInfo.variants
: isRecord(mediaInfo.video_info) && Array.isArray(mediaInfo.video_info.variants)
? mediaInfo.video_info.variants
: [];
const variants = variantsSource
.filter(
(variant): variant is JsonObject =>
isRecord(variant) && typeof variant.url === "string" && variant.url.length > 0,
)
.filter((variant) => getVideoVariantContentType(variant) === "video/mp4")
.sort((left, right) => getVideoVariantBitrate(right) - getVideoVariantBitrate(left));
return typeof variants[0]?.url === "string" ? variants[0].url : undefined;
}
export function getTweetMedia(tweet: JsonObject): XMedia[] {
const legacy = getLegacy(tweet);
const extendedEntities = isRecord(legacy.extended_entities) ? legacy.extended_entities : emptyObject();
const media = Array.isArray(extendedEntities.media) ? extendedEntities.media : [];
return media
.map((value) => {
if (!isRecord(value) || typeof value.type !== "string") {
return null;
}
if (value.type === "photo" && typeof value.media_url_https === "string") {
return {
type: value.type,
url: toHighResXImageUrl(value.media_url_https),
alt: typeof value.ext_alt_text === "string" ? value.ext_alt_text : undefined,
};
}
if (value.type === "video" || value.type === "animated_gif") {
const videoUrl = resolveBestXVideoVariantUrl(value);
if (!videoUrl) {
return null;
}
return {
type: value.type,
url: videoUrl,
};
}
return null;
})
.filter((value): value is XMedia => value !== null);
}
export function getTweetUrl(tweet: JsonObject, fallbackUrl: string): string {
const user = getUser(tweet);
const fallbackScreenName = extractScreenNameFromUrl(fallbackUrl);
const id = typeof tweet.rest_id === "string" ? tweet.rest_id : "";
const screenName = user.screenName ?? fallbackScreenName;
if (screenName && id) {
return `https://x.com/${screenName}/status/${id}`;
}
return fallbackUrl;
}
export function getQuotedTweet(tweet: JsonObject, fallbackUrl: string): XQuotedTweet | undefined {
const quoted = unwrapTweetResult(
isRecord(tweet.quoted_status_result) ? tweet.quoted_status_result.result : null,
);
if (!quoted) {
return undefined;
}
const user = getUser(quoted);
return {
id: typeof quoted.rest_id === "string" ? quoted.rest_id : "",
author: user.screenName,
authorName: user.name,
text: getTweetText(quoted),
url: getTweetUrl(quoted, fallbackUrl),
media: getTweetMedia(quoted),
};
}
export function extractScreenNameFromUrl(url: string): string | undefined {
try {
const parsed = new URL(url);
const match = parsed.pathname.match(/^\/([^/]+)\/(?:status|article)\//);
if (!match) {
return undefined;
}
if (match[1] === "i") {
return undefined;
}
return match[1];
} catch {
return undefined;
}
}
export function toXTweet(tweet: JsonObject, fallbackUrl: string): XTweet {
const legacy = getLegacy(tweet);
const user = getUser(tweet);
const fallbackScreenName = extractScreenNameFromUrl(fallbackUrl);
const screenName = user.screenName ?? fallbackScreenName;
return {
id: typeof tweet.rest_id === "string" ? tweet.rest_id : "",
author: screenName,
authorName: user.name,
text: getTweetText(tweet),
likes: typeof legacy.favorite_count === "number" ? legacy.favorite_count : 0,
retweets: typeof legacy.retweet_count === "number" ? legacy.retweet_count : 0,
replies: typeof legacy.reply_count === "number" ? legacy.reply_count : 0,
createdAt: typeof legacy.created_at === "string" ? legacy.created_at : undefined,
inReplyTo: typeof legacy.in_reply_to_status_id_str === "string" ? legacy.in_reply_to_status_id_str : undefined,
url: getTweetUrl(tweet, fallbackUrl),
media: getTweetMedia(tweet),
quotedTweet: getQuotedTweet(tweet, fallbackUrl),
};
}
export function normalizeTitle(text: string, fallback: string): string {
const firstLine = text.split("\n")[0]?.trim();
if (!firstLine) {
return fallback;
}
return firstLine.slice(0, 120);
}
export function formatTweetAuthor(tweet: XTweet): string | undefined {
if (tweet.author && tweet.authorName) {
return `${tweet.authorName} (@${tweet.author})`;
}
if (tweet.author) {
return `@${tweet.author}`;
}
return tweet.authorName;
}
export function getTweetAuthorMetadata(tweet: XTweet): Record<string, unknown> {
return {
authorName: tweet.authorName,
authorUsername: tweet.author,
authorUrl: tweet.author ? `https://x.com/${tweet.author}` : undefined,
};
}
export function formatMediaList(media: XMedia[]): string[] {
return media.map((item) => {
if (item.type === "photo") {
return `photo: ${item.url}`;
}
return `${item.type}: ${item.url}`;
});
}
export function filterXGraphQlEntries(entries: NetworkEntry[]): NetworkEntry[] {
return entries.filter((entry) => entry.url.includes("/graphql/"));
}
@@ -0,0 +1,87 @@
import type { ExtractedDocument, ContentBlock } from "../../extract/document";
import { findTweetNode, formatMediaList, formatTweetAuthor, getTweetAuthorMetadata, normalizeTitle, toXTweet } from "./shared";
export function extractSingleTweetDocumentFromPayload(
payload: unknown,
statusId: string,
pageUrl: string,
): ExtractedDocument | null {
const tweet = findTweetNode(payload, statusId);
if (!tweet) {
return null;
}
const xTweet = toXTweet(tweet, pageUrl);
const content: ContentBlock[] = [];
if (xTweet.text) {
content.push({ type: "paragraph", text: xTweet.text });
}
for (const mediaLine of formatMediaList(xTweet.media)) {
if (mediaLine.startsWith("photo: ")) {
content.push({
type: "image",
url: mediaLine.slice("photo: ".length),
});
} else {
content.push({
type: "list",
ordered: false,
items: [mediaLine],
});
}
}
if (xTweet.quotedTweet) {
const quotedLines: string[] = [];
const quotedAuthor =
xTweet.quotedTweet.author && xTweet.quotedTweet.authorName
? `${xTweet.quotedTweet.authorName} (@${xTweet.quotedTweet.author})`
: xTweet.quotedTweet.author
? `@${xTweet.quotedTweet.author}`
: xTweet.quotedTweet.authorName;
if (quotedAuthor) {
quotedLines.push(quotedAuthor);
}
if (xTweet.quotedTweet.text) {
quotedLines.push(xTweet.quotedTweet.text);
}
quotedLines.push(...formatMediaList(xTweet.quotedTweet.media));
if (quotedLines.length > 0) {
content.push({ type: "heading", depth: 2, text: "Quoted Tweet" });
content.push({ type: "quote", text: quotedLines.join("\n\n") });
}
}
return {
url: pageUrl,
canonicalUrl: xTweet.url,
title: normalizeTitle(
xTweet.author ? `@${xTweet.author}: ${xTweet.text}` : xTweet.text,
"Tweet",
),
author: formatTweetAuthor(xTweet),
siteName: "X",
publishedAt: xTweet.createdAt,
summary: xTweet.text.slice(0, 200) || undefined,
adapter: "x",
metadata: {
kind: "x/post",
tweetId: xTweet.id,
...getTweetAuthorMetadata(xTweet),
conversationId:
typeof tweet.legacy === "object" &&
tweet.legacy !== null &&
typeof (tweet.legacy as Record<string, unknown>).conversation_id_str === "string"
? (tweet.legacy as Record<string, unknown>).conversation_id_str
: undefined,
favoriteCount: xTweet.likes,
replyCount: xTweet.replies,
retweetCount: xTweet.retweets,
},
content,
};
}
@@ -0,0 +1,286 @@
import type { AdapterContext } from "../types";
import { extractThreadTweetsFromPayloads } from "./thread";
import { collectXJsonPayloads, getRelevantXThreadEntries, prefetchRelevantXThreadBodies } from "./payloads";
interface ClickTextResult {
clicked: boolean;
text?: string;
}
interface ScrollStepResult {
moved: boolean;
atTop: boolean;
atBottom: boolean;
}
interface ThreadProgress {
tweetCount: number;
firstTweetId?: string;
lastTweetId?: string;
requestCount: number;
tweetDetailCount: number;
}
interface TopProbeState {
requestCount: number;
tweetDetailCount: number;
scrollHeight: number;
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function waitForXNetworkSettle(context: AdapterContext, reason: string): Promise<void> {
try {
await context.network.waitForIdle({
idleMs: 650,
timeoutMs: Math.min(context.timeoutMs, 5_000),
});
} catch {
context.log.debug(`Network idle timed out after ${reason}.`);
}
}
async function captureTopProbeState(context: AdapterContext): Promise<TopProbeState> {
const entries = getRelevantXThreadEntries(context);
const scrollHeight = await context.browser.evaluate<number>(`
(() => {
const scrollRoot = document.scrollingElement ?? document.documentElement ?? document.body;
return scrollRoot.scrollHeight;
})()
`);
return {
requestCount: entries.length,
tweetDetailCount: entries.filter((entry) => entry.url.includes("TweetDetail")).length,
scrollHeight,
};
}
async function waitForTopProbe(context: AdapterContext): Promise<boolean> {
const initial = await captureTopProbeState(context);
const deadline = Date.now() + 1_200;
while (Date.now() < deadline) {
try {
await context.network.waitForIdle({
idleMs: 250,
timeoutMs: 350,
});
} catch {
// Keep polling until the shorter top-probe budget expires.
}
await prefetchRelevantXThreadBodies(context);
const next = await captureTopProbeState(context);
if (
next.requestCount > initial.requestCount ||
next.tweetDetailCount > initial.tweetDetailCount ||
next.scrollHeight > initial.scrollHeight + 4
) {
context.log.debug("Observed additional X thread activity while probing the page top.");
return true;
}
await sleep(120);
}
return false;
}
async function scrollThreadToTop(context: AdapterContext): Promise<void> {
let settledTopChecks = 0;
while (settledTopChecks < 2) {
const scroll = await context.browser.evaluate<ScrollStepResult>(`
(() => {
const scrollRoot = document.scrollingElement ?? document.documentElement ?? document.body;
const before = window.scrollY;
window.scrollTo({ top: 0, left: 0, behavior: "instant" });
const after = window.scrollY;
return {
moved: after !== before,
atTop: after <= 4,
atBottom: window.innerHeight + after >= scrollRoot.scrollHeight - 4,
};
})()
`);
await sleep(140);
await waitForXNetworkSettle(context, "scrolling X thread to top");
await prefetchRelevantXThreadBodies(context);
if (scroll.moved) {
settledTopChecks = 0;
continue;
}
const observedTopActivity = await waitForTopProbe(context);
if (observedTopActivity) {
settledTopChecks = 0;
continue;
}
settledTopChecks += 1;
}
}
async function clickVisibleShowReplies(context: AdapterContext): Promise<ClickTextResult> {
return context.browser.evaluate<ClickTextResult>(`
(() => {
const normalize = (value) => value.replace(/\\s+/g, " ").trim();
const matches = [
/^Show replies$/i,
/^Show more replies$/i,
/^Show additional replies$/i,
/^显示回复$/,
/^展开回复$/,
];
const isVisible = (element) => {
if (!(element instanceof HTMLElement)) {
return false;
}
const rect = element.getBoundingClientRect();
const style = window.getComputedStyle(element);
return (
rect.width > 0 &&
rect.height > 0 &&
style.visibility !== "hidden" &&
style.display !== "none"
);
};
const selectors = [
"a",
"button",
'[role="button"]',
'[role="link"]',
];
for (const element of document.querySelectorAll(selectors.join(","))) {
if (!isVisible(element)) {
continue;
}
const text = normalize(element.textContent ?? "");
if (!text || !matches.some((pattern) => pattern.test(text))) {
continue;
}
element.scrollIntoView({ block: "center", inline: "nearest" });
if (element instanceof HTMLElement) {
element.click();
return { clicked: true, text };
}
}
return { clicked: false };
})()
`);
}
async function expandVisibleShowReplies(context: AdapterContext): Promise<number> {
let clickCount = 0;
while (clickCount < 8) {
const result = await clickVisibleShowReplies(context).catch<ClickTextResult>(() => ({ clicked: false }));
if (!result.clicked) {
break;
}
clickCount += 1;
context.log.debug(`Expanded X thread replies via "${result.text ?? "Show replies"}".`);
await sleep(250);
await waitForXNetworkSettle(context, "expanding Show replies");
await prefetchRelevantXThreadBodies(context);
}
return clickCount;
}
async function scrollThreadBy(context: AdapterContext, stepPx: number): Promise<ScrollStepResult> {
const result = await context.browser.evaluate<ScrollStepResult>(`
(() => {
const scrollRoot = document.scrollingElement ?? document.documentElement ?? document.body;
const before = window.scrollY;
window.scrollBy({ top: ${stepPx}, left: 0, behavior: "instant" });
const after = window.scrollY;
return {
moved: after !== before,
atTop: after <= 4,
atBottom: window.innerHeight + after >= scrollRoot.scrollHeight - 4,
};
})()
`);
await sleep(140);
await waitForXNetworkSettle(context, "scrolling X thread");
await prefetchRelevantXThreadBodies(context);
return result;
}
async function captureThreadProgress(context: AdapterContext, statusId: string): Promise<ThreadProgress> {
const entries = getRelevantXThreadEntries(context);
const payloads = await collectXJsonPayloads(context);
const tweets = extractThreadTweetsFromPayloads(payloads, statusId, context.input.url.toString());
return {
tweetCount: tweets.length,
firstTweetId: tweets[0]?.id,
lastTweetId: tweets[tweets.length - 1]?.id,
requestCount: entries.length,
tweetDetailCount: entries.filter((entry) => entry.url.includes("TweetDetail")).length,
};
}
export async function loadFullXThread(context: AdapterContext, statusId: string): Promise<void> {
await scrollThreadToTop(context);
let progress = await captureThreadProgress(context, statusId);
let stagnantRounds = 0;
let roundsWithoutMovement = 0;
let distanceWithoutThreadActivityPx = 0;
for (let round = 0; ; round += 1) {
const stepPx = round < 12 ? 1_200 : 1_600;
let expandedCount = await expandVisibleShowReplies(context);
const scroll = await scrollThreadBy(context, stepPx);
expandedCount += await expandVisibleShowReplies(context);
const nextProgress = await captureThreadProgress(context, statusId);
const grew =
nextProgress.tweetCount > progress.tweetCount ||
nextProgress.firstTweetId !== progress.firstTweetId ||
nextProgress.lastTweetId !== progress.lastTweetId ||
nextProgress.requestCount > progress.requestCount ||
nextProgress.tweetDetailCount > progress.tweetDetailCount;
if (grew) {
context.log.debug(
`X thread progress: ${nextProgress.tweetCount} tweets (${nextProgress.firstTweetId ?? "unknown"} -> ${nextProgress.lastTweetId ?? "unknown"}), ${nextProgress.requestCount} requests, ${nextProgress.tweetDetailCount} TweetDetail.`,
);
stagnantRounds = 0;
distanceWithoutThreadActivityPx = 0;
} else if (expandedCount > 0) {
stagnantRounds = 0;
distanceWithoutThreadActivityPx = 0;
} else {
stagnantRounds += 1;
distanceWithoutThreadActivityPx += stepPx;
}
roundsWithoutMovement = scroll.moved ? 0 : roundsWithoutMovement + 1;
progress = nextProgress;
if (scroll.atBottom && stagnantRounds >= 6) {
context.log.debug("Stopping X thread scroll after reaching page bottom with no further thread progress.");
break;
}
if (roundsWithoutMovement >= 2 && stagnantRounds >= 4) {
context.log.debug("Stopping X thread scroll after repeated downward scrolls no longer move the page.");
break;
}
if (distanceWithoutThreadActivityPx >= 24_000 && stagnantRounds >= 12) {
context.log.debug("Stopping X thread scroll after a long stretch with no thread-related progress.");
break;
}
}
}
@@ -0,0 +1,316 @@
import type { ExtractedDocument } from "../../extract/document";
import {
formatMediaList,
formatTweetAuthor,
getLegacy,
getTweetAuthorMetadata,
isRecord,
normalizeTitle,
toXTweet,
unwrapTweetResult,
} from "./shared";
import type { JsonObject, XQuotedTweet, XTweet } from "./types";
interface ParsedThreadTweet extends XTweet {
userId?: string;
conversationId?: string;
inReplyToUserId?: string;
sortTimestamp: number;
}
function compareTweetIds(left: string, right: string): number {
try {
const leftId = BigInt(left);
const rightId = BigInt(right);
if (leftId === rightId) {
return 0;
}
return leftId < rightId ? -1 : 1;
} catch {
return left.localeCompare(right);
}
}
function toTimestamp(value: string | undefined): number {
if (!value) {
return 0;
}
const parsed = Date.parse(value);
return Number.isNaN(parsed) ? 0 : parsed;
}
function scoreParsedTweet(tweet: ParsedThreadTweet): number {
return (
(tweet.text ? 4 : 0) +
(tweet.author ? 2 : 0) +
(tweet.authorName ? 2 : 0) +
(tweet.media.length > 0 ? 1 : 0)
);
}
function toParsedThreadTweet(tweet: JsonObject, pageUrl: string): ParsedThreadTweet {
const legacy = getLegacy(tweet);
const xTweet = toXTweet(tweet, pageUrl);
return {
...xTweet,
userId: typeof legacy.user_id_str === "string" ? legacy.user_id_str : undefined,
conversationId: typeof legacy.conversation_id_str === "string" ? legacy.conversation_id_str : undefined,
inReplyToUserId: typeof legacy.in_reply_to_user_id_str === "string" ? legacy.in_reply_to_user_id_str : undefined,
sortTimestamp: toTimestamp(xTweet.createdAt),
};
}
function collectTweetFromItemContent(
itemContent: unknown,
pageUrl: string,
tweets: Map<string, ParsedThreadTweet>,
): void {
if (!isRecord(itemContent)) {
return;
}
const tweet = unwrapTweetResult(
isRecord(itemContent.tweet_results) ? itemContent.tweet_results.result : null,
);
if (!tweet || typeof tweet.rest_id !== "string") {
return;
}
const parsed = toParsedThreadTweet(tweet, pageUrl);
const existing = tweets.get(parsed.id);
if (!existing || scoreParsedTweet(parsed) >= scoreParsedTweet(existing)) {
tweets.set(parsed.id, parsed);
}
}
function collectTweetsFromItems(
items: unknown,
pageUrl: string,
tweets: Map<string, ParsedThreadTweet>,
): void {
if (!Array.isArray(items)) {
return;
}
for (const item of items) {
if (!isRecord(item)) {
continue;
}
if (isRecord(item.item) && isRecord(item.item.itemContent)) {
collectTweetFromItemContent(item.item.itemContent, pageUrl, tweets);
continue;
}
if (isRecord(item.itemContent)) {
collectTweetFromItemContent(item.itemContent, pageUrl, tweets);
}
}
}
function getInstructions(payload: unknown): unknown[] {
if (!isRecord(payload) || !isRecord(payload.data)) {
return [];
}
const { data } = payload;
return (
(isRecord(data.threaded_conversation_with_injections_v2) &&
Array.isArray(data.threaded_conversation_with_injections_v2.instructions)
? data.threaded_conversation_with_injections_v2.instructions
: undefined) ??
(isRecord(data.threaded_conversation_with_injections) &&
Array.isArray(data.threaded_conversation_with_injections.instructions)
? data.threaded_conversation_with_injections.instructions
: undefined) ??
(isRecord(data.tweetResult) &&
isRecord(data.tweetResult.result) &&
isRecord(data.tweetResult.result.timeline) &&
Array.isArray(data.tweetResult.result.timeline.instructions)
? data.tweetResult.result.timeline.instructions
: [])
);
}
function parseTweetDetailPayload(payload: unknown, pageUrl: string): ParsedThreadTweet[] {
const tweets = new Map<string, ParsedThreadTweet>();
const instructions = getInstructions(payload);
for (const instruction of instructions) {
if (!isRecord(instruction)) {
continue;
}
collectTweetsFromItems(instruction.moduleItems, pageUrl, tweets);
if (!Array.isArray(instruction.entries)) {
continue;
}
for (const entry of instruction.entries) {
if (!isRecord(entry)) {
continue;
}
const content = isRecord(entry.content) ? entry.content : {};
collectTweetFromItemContent(content.itemContent, pageUrl, tweets);
collectTweetsFromItems(content.items, pageUrl, tweets);
}
}
return Array.from(tweets.values());
}
function buildContinuousThread(tweets: ParsedThreadTweet[], statusId: string): ParsedThreadTweet[] {
const byId = new Map<string, ParsedThreadTweet>();
for (const tweet of tweets) {
const existing = byId.get(tweet.id);
if (!existing || scoreParsedTweet(tweet) >= scoreParsedTweet(existing)) {
byId.set(tweet.id, tweet);
}
}
const rootTweet = byId.get(statusId);
if (!rootTweet?.userId || !rootTweet.conversationId) {
return [];
}
const candidates = Array.from(byId.values()).filter(
(tweet) =>
tweet.id === statusId ||
(tweet.userId === rootTweet.userId && tweet.conversationId === rootTweet.conversationId),
);
const repliesByParent = new Map<string, ParsedThreadTweet[]>();
for (const tweet of candidates) {
if (!tweet.inReplyTo || tweet.id === statusId) {
continue;
}
const bucket = repliesByParent.get(tweet.inReplyTo) ?? [];
bucket.push(tweet);
bucket.sort((left, right) => {
if (left.sortTimestamp !== right.sortTimestamp) {
return left.sortTimestamp - right.sortTimestamp;
}
return compareTweetIds(left.id, right.id);
});
repliesByParent.set(tweet.inReplyTo, bucket);
}
const ancestorPath: ParsedThreadTweet[] = [rootTweet];
const ancestorSeen = new Set<string>([rootTweet.id]);
let currentAncestor = rootTweet;
while (currentAncestor.inReplyTo) {
const parent = byId.get(currentAncestor.inReplyTo);
if (!parent || ancestorSeen.has(parent.id)) {
break;
}
ancestorPath.unshift(parent);
ancestorSeen.add(parent.id);
currentAncestor = parent;
}
const chain = ancestorPath.slice();
const seen = new Set<string>(chain.map((tweet) => tweet.id));
let currentId = rootTweet.id;
while (true) {
const next = (repliesByParent.get(currentId) ?? []).find((tweet) => !seen.has(tweet.id));
if (!next) {
break;
}
chain.push(next);
seen.add(next.id);
currentId = next.id;
}
return chain;
}
export function extractThreadTweetsFromPayloads(
payloads: unknown[],
statusId: string,
pageUrl: string,
): XTweet[] {
const parsedTweets: ParsedThreadTweet[] = [];
for (const payload of payloads) {
parsedTweets.push(...parseTweetDetailPayload(payload, pageUrl));
}
return buildContinuousThread(parsedTweets, statusId).map(({ sortTimestamp: _sortTimestamp, ...tweet }) => tweet);
}
function buildQuotedTweetMarkdown(quotedTweet: XQuotedTweet): string {
const author = quotedTweet.author ? `@${quotedTweet.author}` : "Unknown";
const name = quotedTweet.authorName ? `${quotedTweet.authorName} ` : "";
const lines: string[] = [`Quoted Tweet${quotedTweet.author || quotedTweet.authorName ? `: ${name}${author}`.trim() : ""}`];
if (quotedTweet.text) {
lines.push(...quotedTweet.text.split("\n"));
}
for (const mediaLine of formatMediaList(quotedTweet.media)) {
lines.push(mediaLine);
}
return lines.map((line) => (line ? `> ${line}` : ">")).join("\n");
}
function buildThreadMarkdown(tweets: XTweet[]): string {
return tweets
.map((tweet, index) => {
const lines: string[] = [];
const author = tweet.author ? `@${tweet.author}` : "Unknown";
const name = tweet.authorName ? `${tweet.authorName} ` : "";
lines.push(`## ${index + 1}. ${name}${author}`.trim());
if (tweet.createdAt) {
lines.push(`_Published: ${tweet.createdAt}_`);
}
lines.push(tweet.text || "(No text)");
const mediaLines = formatMediaList(tweet.media);
if (mediaLines.length > 0) {
lines.push(mediaLines.map((line) => `- ${line}`).join("\n"));
}
if (tweet.quotedTweet) {
lines.push(buildQuotedTweetMarkdown(tweet.quotedTweet));
}
return lines.join("\n\n");
})
.join("\n\n");
}
export function extractThreadDocumentFromPayloads(
payloads: unknown[],
statusId: string,
pageUrl: string,
): ExtractedDocument | null {
const tweets = extractThreadTweetsFromPayloads(payloads, statusId, pageUrl);
if (tweets.length <= 1) {
return null;
}
const rootTweet = tweets[0];
const rootAuthor = formatTweetAuthor(rootTweet);
return {
url: pageUrl,
canonicalUrl: rootTweet.url,
title: normalizeTitle(rootTweet.text, "X Thread"),
author: rootAuthor,
siteName: "X",
publishedAt: rootTweet.createdAt,
summary: rootTweet.text.slice(0, 200) || undefined,
adapter: "x",
metadata: {
kind: "x/thread",
tweetId: rootTweet.id,
tweetCount: tweets.length,
lastTweetId: tweets[tweets.length - 1]?.id,
...getTweetAuthorMetadata(rootTweet),
},
content: [{ type: "markdown", markdown: buildThreadMarkdown(tweets) }],
};
}
@@ -0,0 +1,36 @@
export type JsonObject = Record<string, unknown>;
export interface XUser {
name?: string;
screenName?: string;
}
export interface XMedia {
type: string;
url: string;
alt?: string;
}
export interface XQuotedTweet {
id: string;
author?: string;
authorName?: string;
text: string;
url: string;
media: XMedia[];
}
export interface XTweet {
id: string;
author?: string;
authorName?: string;
text: string;
likes: number;
retweets: number;
replies: number;
createdAt?: string;
inReplyTo?: string;
url: string;
media: XMedia[];
quotedTweet?: XQuotedTweet;
}
@@ -0,0 +1,33 @@
import type { Adapter } from "../types";
import { collectMediaFromDocument } from "../../media/markdown-media";
import { extractYouTubeTranscriptDocument } from "./transcript";
import { isYouTubeHost, parseYouTubeVideoId } from "./utils";
export const youtubeAdapter: Adapter = {
name: "youtube",
match(input) {
return isYouTubeHost(input.url.hostname);
},
async process(context) {
const videoId = parseYouTubeVideoId(context.input.url);
if (!videoId) {
return {
status: "no_document",
};
}
context.log.info(`Loading ${context.input.url.toString()} with youtube adapter`);
const document = await extractYouTubeTranscriptDocument(context, videoId);
if (!document) {
return {
status: "no_document",
};
}
return {
status: "ok",
document,
media: collectMediaFromDocument(document),
};
},
};
@@ -0,0 +1,392 @@
import type { ExtractedDocument } from "../../extract/document";
import { detectInteractionGate } from "../../browser/interaction-gates";
import {
buildYouTubeThumbnailCandidates,
parseYouTubeDescriptionChapters,
renderYouTubeTranscriptMarkdown,
type YouTubeChapter,
type YouTubeTranscriptSegment,
} from "./utils";
interface CaptionInfo {
captionUrl: string;
language: string;
kind: string;
available: string[];
title?: string;
author?: string;
authorUrl?: string;
channelId?: string;
description?: string;
publishedAt?: string;
viewCount?: number;
durationSeconds?: number;
keywords: string[];
category?: string;
isLiveContent?: boolean;
coverImages: string[];
}
function normalizeUrl(url: string | undefined): string | undefined {
if (!url) {
return undefined;
}
try {
const parsed = new URL(url);
if (parsed.protocol === "http:") {
parsed.protocol = "https:";
}
return parsed.toString();
} catch {
return url;
}
}
function buildSummary(description: string | undefined, segments: YouTubeTranscriptSegment[]): string | undefined {
const descriptionSummary = description
?.replace(/\r\n/g, "\n")
.split("\n")
.map((line) => line.trim())
.find((line) => line && !/^https?:\/\//i.test(line));
if (descriptionSummary) {
return descriptionSummary.slice(0, 240);
}
const transcriptSummary = segments
.slice(0, 8)
.map((segment) => segment.text)
.join(" ")
.slice(0, 240)
.trim();
return transcriptSummary || undefined;
}
async function canFetchThumbnail(url: string): Promise<boolean> {
try {
const response = await fetch(url, { method: "HEAD", redirect: "follow" });
if (response.ok) {
return true;
}
if (response.status === 405) {
const fallbackResponse = await fetch(url, {
method: "GET",
headers: { Range: "bytes=0-0" },
redirect: "follow",
});
return fallbackResponse.ok;
}
} catch {
return false;
}
return false;
}
async function resolveBestCoverImage(videoId: string, coverImages: string[]): Promise<string | undefined> {
const candidates = buildYouTubeThumbnailCandidates(videoId, coverImages);
for (const candidate of candidates) {
if (await canFetchThumbnail(candidate)) {
return candidate;
}
}
return candidates[0];
}
export async function extractYouTubeTranscriptDocument(
context: Parameters<import("../types").Adapter["process"]>[0],
videoId: string,
): Promise<ExtractedDocument | null> {
const videoUrl = `https://www.youtube.com/watch?v=${videoId}`;
await context.browser.goto(videoUrl, context.timeoutMs);
const interaction = await detectInteractionGate(context.browser);
if (interaction) {
context.log.debug(`Interaction gate detected on YouTube: ${interaction.provider}`);
return null;
}
try {
await context.network.waitForIdle({
idleMs: 1_000,
timeoutMs: Math.min(context.timeoutMs, 8_000),
});
} catch {
context.log.debug("Network idle timed out on YouTube load.");
}
const captionInfo = await context.browser.evaluate<CaptionInfo | { error: string }>(`
(async () => {
function readText(value) {
if (!value) return undefined;
if (typeof value === 'string') {
const text = value.trim();
return text || undefined;
}
if (typeof value.simpleText === 'string') {
const text = value.simpleText.trim();
return text || undefined;
}
if (Array.isArray(value.runs)) {
const text = value.runs
.map((run) => typeof run?.text === 'string' ? run.text : '')
.join('')
.trim();
return text || undefined;
}
return undefined;
}
function parsePositiveInteger(value) {
if (typeof value === 'number' && Number.isFinite(value) && value >= 0) {
return Math.floor(value);
}
if (typeof value !== 'string') {
return undefined;
}
const normalized = value.replace(/[^\\d]/g, '');
if (!normalized) {
return undefined;
}
const parsed = Number.parseInt(normalized, 10);
return Number.isFinite(parsed) ? parsed : undefined;
}
const apiKey = window.ytcfg?.data_?.INNERTUBE_API_KEY;
const playerResponse = window.ytInitialPlayerResponse;
const videoDetails = playerResponse?.videoDetails || {};
const microformat = playerResponse?.microformat?.playerMicroformatRenderer || {};
const title =
videoDetails.title ||
readText(microformat.title) ||
document.title.replace(/ - YouTube$/, '').trim();
const author =
videoDetails.author ||
microformat.ownerChannelName ||
document.querySelector('link[itemprop="name"]')?.getAttribute('content') ||
undefined;
const authorUrl =
microformat.ownerProfileUrl ||
(typeof videoDetails.channelId === 'string' && videoDetails.channelId
? 'https://www.youtube.com/channel/' + videoDetails.channelId
: undefined);
const description =
readText(microformat.description) ||
(typeof videoDetails.shortDescription === 'string' ? videoDetails.shortDescription.trim() : undefined);
const keywords = Array.isArray(videoDetails.keywords)
? videoDetails.keywords.filter((keyword) => typeof keyword === 'string' && keyword.trim())
: [];
const thumbnails = [
...(Array.isArray(videoDetails.thumbnail?.thumbnails) ? videoDetails.thumbnail.thumbnails : []),
...(Array.isArray(microformat.thumbnail?.thumbnails) ? microformat.thumbnail.thumbnails : []),
]
.filter((thumbnail) => typeof thumbnail?.url === 'string' && thumbnail.url)
.sort((left, right) => ((right?.width || 0) * (right?.height || 0)) - ((left?.width || 0) * (left?.height || 0)))
.map((thumbnail) => thumbnail.url);
if (!apiKey) {
return { error: 'INNERTUBE_API_KEY not found on page' };
}
const response = await fetch('/youtubei/v1/player?key=' + apiKey + '&prettyPrint=false', {
method: 'POST',
credentials: 'include',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
context: { client: { clientName: 'ANDROID', clientVersion: '20.10.38' } },
videoId: ${JSON.stringify(videoId)}
})
});
if (!response.ok) {
return { error: 'InnerTube player API returned HTTP ' + response.status };
}
const data = await response.json();
const renderer = data.captions?.playerCaptionsTracklistRenderer;
if (!renderer?.captionTracks?.length) {
return { error: 'No captions available for this video' };
}
const tracks = renderer.captionTracks;
const track = tracks.find((item) => item.kind !== 'asr') || tracks[0];
return {
captionUrl: track.baseUrl,
language: track.languageCode,
kind: track.kind || 'manual',
available: tracks.map((item) => {
const languageLabel = readText(item.name) || item.languageCode;
return item.kind === 'asr'
? languageLabel + ' [' + item.languageCode + ', auto]'
: languageLabel + ' [' + item.languageCode + ']';
}),
title,
author,
authorUrl,
channelId: typeof videoDetails.channelId === 'string' ? videoDetails.channelId : undefined,
description,
publishedAt:
(typeof microformat.publishDate === 'string' && microformat.publishDate) ||
(typeof microformat.uploadDate === 'string' && microformat.uploadDate) ||
document.querySelector('meta[itemprop="datePublished"]')?.getAttribute('content') ||
undefined,
viewCount: parsePositiveInteger(videoDetails.viewCount) ?? parsePositiveInteger(microformat.viewCount),
durationSeconds: parsePositiveInteger(videoDetails.lengthSeconds),
keywords,
category: typeof microformat.category === 'string' ? microformat.category : undefined,
isLiveContent: Boolean(videoDetails.isLiveContent || microformat.isLiveContent),
coverImages: thumbnails,
};
})()
`);
if ("error" in captionInfo) {
context.log.debug(`YouTube transcript unavailable: ${captionInfo.error}`);
return null;
}
const segments = await context.browser.evaluate<YouTubeTranscriptSegment[] | { error: string }>(`
(async () => {
const response = await fetch(${JSON.stringify(captionInfo.captionUrl)});
const xml = await response.text();
if (!xml) {
return { error: 'Caption XML is empty' };
}
function getAttr(tag, name) {
const needle = name + '="';
const index = tag.indexOf(needle);
if (index === -1) return '';
const valueStart = index + needle.length;
const valueEnd = tag.indexOf('"', valueStart);
if (valueEnd === -1) return '';
return tag.substring(valueStart, valueEnd);
}
function decodeEntities(value) {
return value
.replaceAll('&amp;', '&')
.replaceAll('&lt;', '<')
.replaceAll('&gt;', '>')
.replaceAll('&quot;', '"')
.replaceAll('&#39;', "'");
}
const marker = xml.includes('<p t="') ? '<p ' : '<text ';
const endMarker = marker === '<p ' ? '</p>' : '</text>';
const results = [];
let position = 0;
while (true) {
const tagStart = xml.indexOf(marker, position);
if (tagStart === -1) break;
let contentStart = xml.indexOf('>', tagStart);
if (contentStart === -1) break;
contentStart += 1;
const tagEnd = xml.indexOf(endMarker, contentStart);
if (tagEnd === -1) break;
const attrString = xml.substring(tagStart + marker.length, contentStart - 1);
const content = xml.substring(contentStart, tagEnd);
const start = marker === '<p '
? (parseFloat(getAttr(attrString, 't')) || 0) / 1000
: (parseFloat(getAttr(attrString, 'start')) || 0);
const duration = marker === '<p '
? (parseFloat(getAttr(attrString, 'd')) || 0) / 1000
: (parseFloat(getAttr(attrString, 'dur')) || 0);
const text = decodeEntities(content.replace(/<[^>]+>/g, '')).split('\\n').join(' ').trim();
if (text) {
results.push({ start, end: start + duration, text });
}
position = tagEnd + endMarker.length;
}
if (results.length === 0) {
return { error: 'Parsed 0 transcript segments' };
}
return results;
})()
`);
if (!Array.isArray(segments) || segments.length === 0) {
context.log.debug("Parsed no YouTube transcript segments.");
return null;
}
const extractedChapters = await context.browser.evaluate<YouTubeChapter[]>(`
(() => {
const data = window.ytInitialData;
const markers = data?.playerOverlays?.playerOverlayRenderer
?.decoratedPlayerBarRenderer?.decoratedPlayerBarRenderer
?.playerBar?.multiMarkersPlayerBarRenderer?.markersMap || [];
const results = [];
for (const marker of markers) {
const chapters = marker?.value?.chapters;
if (!Array.isArray(chapters)) continue;
for (const chapter of chapters) {
const renderer = chapter?.chapterRenderer;
const title = renderer?.title?.simpleText;
const timeRangeStartMillis = renderer?.timeRangeStartMillis;
if (title && typeof timeRangeStartMillis === 'number') {
results.push({ title, time: Math.floor(timeRangeStartMillis / 1000) });
}
}
}
return results;
})()
`).catch(() => []);
const descriptionChapters = parseYouTubeDescriptionChapters(captionInfo.description);
const chapters = extractedChapters.length > 0 ? extractedChapters : descriptionChapters;
const markdown = renderYouTubeTranscriptMarkdown({
description: captionInfo.description,
segments,
chapters,
});
if (!markdown) {
return null;
}
const pageUrl = await context.browser.getURL();
const coverImage = await resolveBestCoverImage(videoId, captionInfo.coverImages);
const summary = buildSummary(captionInfo.description, segments);
return {
url: pageUrl,
canonicalUrl: pageUrl,
title: captionInfo.title || "YouTube Transcript",
author: captionInfo.author,
publishedAt: captionInfo.publishedAt,
siteName: "YouTube",
summary,
adapter: "youtube",
metadata: {
kind: "youtube/transcript",
videoId,
authorUrl: normalizeUrl(captionInfo.authorUrl),
channelId: captionInfo.channelId,
coverImage,
description: captionInfo.description,
durationSeconds: captionInfo.durationSeconds,
language: captionInfo.language,
captionKind: captionInfo.kind,
availableLanguages: captionInfo.available,
viewCount: captionInfo.viewCount,
keywords: captionInfo.keywords,
category: captionInfo.category,
isLiveContent: captionInfo.isLiveContent,
chapterCount: chapters.length,
},
content: [{ type: "markdown", markdown }],
};
}
@@ -0,0 +1,258 @@
export interface YouTubeTranscriptSegment {
start: number;
end: number;
text: string;
}
export interface YouTubeChapter {
title: string;
time: number;
}
interface RenderYouTubeTranscriptMarkdownInput {
description?: string;
segments: YouTubeTranscriptSegment[];
chapters: YouTubeChapter[];
}
const DESCRIPTION_CHAPTER_RE = /^((?:\d{1,2}:)?\d{1,2}:\d{2})(?:\s+[-|:]\s+|\s+)(.+)$/;
const YOUTUBE_THUMBNAIL_VARIANTS = [
"maxresdefault.jpg",
"sddefault.jpg",
"hqdefault.jpg",
"mqdefault.jpg",
"default.jpg",
];
export function isYouTubeHost(hostname: string): boolean {
return [
"youtube.com",
"www.youtube.com",
"m.youtube.com",
"youtu.be",
].includes(hostname);
}
export function parseYouTubeVideoId(url: URL): string | null {
if (url.hostname === "youtu.be") {
return url.pathname.split("/").filter(Boolean)[0] ?? null;
}
if (url.pathname === "/watch") {
return url.searchParams.get("v");
}
const shortsMatch = url.pathname.match(/^\/shorts\/([^/?#]+)/);
if (shortsMatch) {
return shortsMatch[1];
}
const liveMatch = url.pathname.match(/^\/live\/([^/?#]+)/);
if (liveMatch) {
return liveMatch[1];
}
const embedMatch = url.pathname.match(/^\/embed\/([^/?#]+)/);
if (embedMatch) {
return embedMatch[1];
}
return null;
}
function parseTimestampValue(raw: string): number | null {
const parts = raw
.split(":")
.map((part) => Number.parseInt(part, 10))
.filter((part) => Number.isFinite(part));
if (parts.length < 2 || parts.length > 3) {
return null;
}
if (parts.some((part) => part < 0)) {
return null;
}
if (parts.length === 2) {
const [minutes, seconds] = parts;
return minutes * 60 + seconds;
}
const [hours, minutes, seconds] = parts;
return hours * 3600 + minutes * 60 + seconds;
}
export function formatTimestamp(totalSeconds: number): string {
const rounded = Math.max(0, Math.floor(totalSeconds));
const hours = Math.floor(rounded / 3600);
const minutes = Math.floor((rounded % 3600) / 60);
const seconds = rounded % 60;
if (hours > 0) {
return `${hours}:${String(minutes).padStart(2, "0")}:${String(seconds).padStart(2, "0")}`;
}
return `${minutes}:${String(seconds).padStart(2, "0")}`;
}
export function formatTimestampRange(start: number, end: number): string {
const safeStart = Math.max(0, start);
const safeEnd = Math.max(safeStart, end);
return `[${formatTimestamp(safeStart)} -> ${formatTimestamp(safeEnd)}]`;
}
export function normalizeYouTubeChapters(chapters: YouTubeChapter[]): YouTubeChapter[] {
const seenTimes = new Set<number>();
return chapters
.map((chapter) => ({
title: chapter.title.trim(),
time: Math.max(0, Math.floor(chapter.time)),
}))
.filter((chapter) => chapter.title)
.sort((left, right) => left.time - right.time)
.filter((chapter) => {
if (seenTimes.has(chapter.time)) {
return false;
}
seenTimes.add(chapter.time);
return true;
});
}
export function parseYouTubeDescriptionChapters(description?: string | null): YouTubeChapter[] {
if (!description) {
return [];
}
const chapters: YouTubeChapter[] = [];
const seen = new Set<string>();
for (const rawLine of description.replace(/\r\n/g, "\n").split("\n")) {
const line = rawLine.trim();
if (!line) {
continue;
}
const match = line.match(DESCRIPTION_CHAPTER_RE);
if (!match) {
continue;
}
const time = parseTimestampValue(match[1]);
const title = match[2]?.trim();
if (time === null || !title) {
continue;
}
const key = `${time}:${title.toLowerCase()}`;
if (seen.has(key)) {
continue;
}
seen.add(key);
chapters.push({ title, time });
}
const normalized = normalizeYouTubeChapters(chapters);
if (normalized.length >= 2) {
return normalized;
}
if (normalized.length === 1 && normalized[0]?.time === 0) {
return normalized;
}
return [];
}
function renderDescriptionMarkdown(description: string): string {
return description
.replace(/\r\n/g, "\n")
.trim()
.split(/\n{2,}/)
.map((block) => block.split("\n").map((line) => line.trimEnd()).join(" \n"))
.join("\n\n")
.trim();
}
function renderSegmentLine(segment: YouTubeTranscriptSegment): string {
return `${formatTimestampRange(segment.start, segment.end)} ${segment.text}`;
}
export function renderYouTubeTranscriptMarkdown({
description,
segments,
chapters,
}: RenderYouTubeTranscriptMarkdownInput): string {
if (segments.length === 0) {
return "";
}
const parts: string[] = [];
const normalizedDescription = description?.trim();
const transcriptEnd = segments.reduce((maxEnd, segment) => Math.max(maxEnd, segment.end, segment.start), 0);
const normalizedChapters = normalizeYouTubeChapters(chapters).filter(
(chapter) => transcriptEnd <= 0 || chapter.time < transcriptEnd,
);
if (normalizedDescription) {
parts.push("## Description");
parts.push(renderDescriptionMarkdown(normalizedDescription));
}
if (normalizedChapters.length > 0) {
parts.push("## Chapters");
for (let index = 0; index < normalizedChapters.length; index += 1) {
const chapter = normalizedChapters[index];
const nextChapter = normalizedChapters[index + 1];
const chapterEnd = nextChapter ? nextChapter.time : transcriptEnd;
const chapterSegments = segments.filter(
(segment) => segment.start >= chapter.time && segment.start < chapterEnd,
);
parts.push(`### ${chapter.title} ${formatTimestampRange(chapter.time, chapterEnd)}`);
if (chapterSegments.length > 0) {
parts.push(chapterSegments.map(renderSegmentLine).join("\n"));
}
}
} else {
parts.push("## Transcript");
parts.push(segments.map(renderSegmentLine).join("\n"));
}
return parts.filter(Boolean).join("\n\n").trim();
}
function normalizeThumbnailKey(url: string): string {
try {
const parsed = new URL(url);
return `${parsed.origin}${parsed.pathname}`;
} catch {
return url;
}
}
export function buildYouTubeThumbnailCandidates(videoId: string, listedUrls: string[]): string[] {
const candidates = [
...YOUTUBE_THUMBNAIL_VARIANTS.map((variant) => `https://i.ytimg.com/vi/${videoId}/${variant}`),
...listedUrls,
];
const seen = new Set<string>();
return candidates.filter((candidate) => {
if (!candidate) {
return false;
}
const key = normalizeThumbnailKey(candidate);
if (seen.has(key)) {
return false;
}
seen.add(key);
return true;
});
}
@@ -0,0 +1,258 @@
import { EventEmitter } from "node:events";
import WebSocket from "ws";
type JsonObject = Record<string, unknown>;
interface CdpPendingCommand {
resolve(value: unknown): void;
reject(error: unknown): void;
method: string;
}
interface CdpErrorShape {
message?: string;
}
interface CdpCommandResult<T> {
result?: T;
error?: CdpErrorShape;
}
interface CreatePageSessionOptions {
initialUrl?: string;
visible?: boolean;
}
export class TargetSession extends EventEmitter {
constructor(
private readonly client: CdpClient,
public readonly targetId: string,
public readonly sessionId: string,
) {
super();
}
async send<T>(method: string, params: JsonObject = {}): Promise<T> {
return this.client.sendSessionCommand<T>(this.sessionId, method, params);
}
handleEvent(method: string, params: JsonObject): void {
this.emit(method, params);
this.emit("event", { method, params });
}
async waitForEvent<T extends JsonObject>(
method: string,
predicate?: (params: T) => boolean,
timeoutMs = 30_000,
): Promise<T> {
return new Promise<T>((resolve, reject) => {
const timeout = setTimeout(() => {
this.off(method, listener);
reject(new Error(`Timed out waiting for ${method}`));
}, timeoutMs);
const listener = (params: T): void => {
if (predicate && !predicate(params)) {
return;
}
clearTimeout(timeout);
this.off(method, listener);
resolve(params);
};
this.on(method, listener);
});
}
}
export class CdpClient {
private readonly ws: WebSocket;
private readonly pending = new Map<number, CdpPendingCommand>();
private readonly sessions = new Map<string, TargetSession>();
private nextId = 1;
private constructor(ws: WebSocket) {
this.ws = ws;
this.ws.on("message", (raw) => {
this.handleMessage(raw.toString());
});
}
static async connect(browserWsUrl: string): Promise<CdpClient> {
const ws = await new Promise<WebSocket>((resolve, reject) => {
const socket = new WebSocket(browserWsUrl);
socket.once("open", () => resolve(socket));
socket.once("error", (error) => reject(error));
});
return new CdpClient(ws);
}
private handleMessage(rawMessage: string): void {
const message = JSON.parse(rawMessage) as {
id?: number;
sessionId?: string;
method?: string;
params?: JsonObject;
result?: unknown;
error?: CdpErrorShape;
};
if (typeof message.id === "number") {
const pending = this.pending.get(message.id);
if (!pending) {
return;
}
this.pending.delete(message.id);
if (message.error) {
pending.reject(new Error(`${pending.method}: ${message.error.message ?? "Unknown CDP error"}`));
return;
}
pending.resolve(message.result);
return;
}
if (typeof message.sessionId === "string" && typeof message.method === "string") {
const session = this.sessions.get(message.sessionId);
if (session) {
session.handleEvent(message.method, (message.params ?? {}) as JsonObject);
}
}
}
private async sendCommand<T>(
method: string,
params: JsonObject = {},
sessionId?: string,
): Promise<T> {
const id = this.nextId;
this.nextId += 1;
const payload = sessionId ? { id, method, params, sessionId } : { id, method, params };
const result = new Promise<T>((resolve, reject) => {
this.pending.set(id, {
resolve: (value) => resolve(value as T),
reject,
method,
});
});
this.ws.send(JSON.stringify(payload));
return result;
}
async sendBrowserCommand<T>(method: string, params: JsonObject = {}): Promise<T> {
return this.sendCommand<T>(method, params);
}
async sendSessionCommand<T>(sessionId: string, method: string, params: JsonObject = {}): Promise<T> {
return this.sendCommand<T>(method, params, sessionId);
}
private async createPageTarget(initialUrl: string, visible = false): Promise<{ targetId: string }> {
const attempts: JsonObject[] = visible
? [
{
url: initialUrl,
newWindow: true,
focus: true,
},
{
url: initialUrl,
focus: true,
},
{
url: initialUrl,
},
]
: [
{
url: initialUrl,
hidden: true,
},
{
url: initialUrl,
background: true,
focus: false,
},
{
url: initialUrl,
},
];
let lastError: unknown;
for (const params of attempts) {
try {
return await this.sendBrowserCommand<{ targetId: string }>("Target.createTarget", params);
} catch (error) {
lastError = error;
}
}
throw lastError instanceof Error ? lastError : new Error("Target.createTarget failed");
}
async createPageSession(options: CreatePageSessionOptions = {}): Promise<TargetSession> {
const initialUrl = options.initialUrl ?? "about:blank";
const created = await this.createPageTarget(initialUrl, Boolean(options.visible));
const attached = await this.sendBrowserCommand<{ sessionId: string }>("Target.attachToTarget", {
targetId: created.targetId,
flatten: true,
});
const session = new TargetSession(this, created.targetId, attached.sessionId);
this.sessions.set(attached.sessionId, session);
if (options.visible) {
await this.sendBrowserCommand("Target.activateTarget", {
targetId: created.targetId,
}).catch(() => {});
}
await session.send("Page.enable");
await session.send("Runtime.enable");
await session.send("DOM.enable");
if (options.visible) {
await session.send("Page.bringToFront").catch(() => {});
}
return session;
}
async closeTarget(targetId: string): Promise<void> {
try {
await this.sendBrowserCommand("Target.closeTarget", { targetId });
} catch {
// Target may already be gone.
}
}
async close(): Promise<void> {
await new Promise<void>((resolve) => {
if (this.ws.readyState === WebSocket.CLOSED) {
resolve();
return;
}
this.ws.once("close", () => resolve());
this.ws.close();
});
}
}
export async function evaluateRuntime<T>(session: TargetSession, expression: string): Promise<T> {
const response = await session.send<CdpCommandResult<{ value?: T; description?: string }>>("Runtime.evaluate", {
expression,
awaitPromise: true,
returnByValue: true,
});
if (response.error) {
throw new Error(response.error.message ?? "Runtime.evaluate failed");
}
return (response.result?.value as T | undefined) ?? (undefined as T);
}
@@ -0,0 +1,187 @@
import { launch, type LaunchedChrome } from "chrome-launcher";
import WebSocket from "ws";
import type { Logger } from "../utils/logger";
import {
cleanChromeLockArtifacts,
ensureChromeProfileDir,
findChromeProcessUsingProfile,
findExistingChromeDebugPort,
hasChromeLockArtifacts,
listChromeProfileEntries,
resolveChromeProfileDir,
shouldRetryChromeLaunchRecovery,
} from "./profile";
interface ChromeVersionResponse {
webSocketDebuggerUrl: string;
}
export interface ChromeConnectOptions {
cdpUrl?: string;
browserPath?: string;
headless?: boolean;
logger?: Logger;
profileDir?: string;
}
export interface ChromeConnection {
browserWsUrl: string;
origin?: string;
port?: number;
profileDir?: string;
launched: boolean;
close(): Promise<void>;
}
async function fetchJson<T>(url: string): Promise<T> {
const response = await fetch(url);
if (!response.ok) {
throw new Error(`Failed to fetch ${url}: HTTP ${response.status}`);
}
return (await response.json()) as T;
}
async function connectToHttpEndpoint(origin: string): Promise<ChromeConnection> {
const normalizedOrigin = origin.replace(/\/$/, "");
const version = await fetchJson<ChromeVersionResponse>(`${normalizedOrigin}/json/version`);
return {
browserWsUrl: version.webSocketDebuggerUrl,
origin: normalizedOrigin,
port: Number(new URL(normalizedOrigin).port || 80),
launched: false,
async close() {
// Reused external Chrome, nothing to close here.
},
};
}
async function tryReuseChrome(profileDir: string, logger?: Logger): Promise<ChromeConnection | null> {
const port = await findExistingChromeDebugPort({ profileDir });
if (!port) {
return null;
}
const origin = `http://127.0.0.1:${port}`;
try {
const connection = await connectToHttpEndpoint(origin);
logger?.info(`Reusing Chrome debugger at ${origin} for profile ${profileDir}`);
return {
...connection,
profileDir,
};
} catch {
// Debugger disappeared between detection and connect.
}
return null;
}
async function launchFreshChrome(
profileDir: string,
options: Pick<ChromeConnectOptions, "browserPath" | "headless">,
): Promise<ChromeConnection> {
let launchedChrome: LaunchedChrome | null = null;
try {
launchedChrome = await launch({
chromePath: options.browserPath,
userDataDir: profileDir,
chromeFlags: [
"--disable-background-networking",
"--disable-default-apps",
"--disable-popup-blocking",
"--disable-sync",
"--no-first-run",
"--no-default-browser-check",
"--remote-allow-origins=*",
...(!options.headless ? ["--no-startup-window"] : []),
...(options.headless ? ["--headless=new"] : []),
],
});
const origin = `http://127.0.0.1:${launchedChrome.port}`;
const version = await fetchJson<ChromeVersionResponse>(`${origin}/json/version`);
const chrome = launchedChrome;
return {
browserWsUrl: version.webSocketDebuggerUrl,
origin,
port: launchedChrome.port,
profileDir,
launched: true,
async close() {
if (!chrome) return;
await gracefulCloseChrome(chrome, origin);
},
};
} catch (error) {
launchedChrome?.kill();
throw error;
}
}
async function gracefulCloseChrome(chrome: LaunchedChrome, origin: string): Promise<void> {
try {
const resp = await fetch(`${origin}/json/version`);
const { webSocketDebuggerUrl } = (await resp.json()) as ChromeVersionResponse;
if (webSocketDebuggerUrl) {
const ws = await new Promise<WebSocket>((resolve, reject) => {
const socket = new WebSocket(webSocketDebuggerUrl);
socket.once("open", () => resolve(socket));
socket.once("error", reject);
});
const id = 1;
ws.send(JSON.stringify({ id, method: "Browser.close" }));
await new Promise<void>((resolve) => {
const timer = setTimeout(() => { ws.close(); resolve(); }, 5_000);
ws.once("close", () => { clearTimeout(timer); resolve(); });
});
const exited = await new Promise<boolean>((resolve) => {
if (chrome.pid && !isProcessAlive(chrome.pid)) { resolve(true); return; }
const timer = setTimeout(() => resolve(false), 3_000);
chrome.process.once("exit", () => { clearTimeout(timer); resolve(true); });
});
if (exited) return;
}
} catch {}
chrome.kill();
}
function isProcessAlive(pid: number): boolean {
try { process.kill(pid, 0); return true; } catch { return false; }
}
export async function connectChrome(options: ChromeConnectOptions): Promise<ChromeConnection> {
if (options.cdpUrl) {
if (options.cdpUrl.startsWith("ws://") || options.cdpUrl.startsWith("wss://")) {
return {
browserWsUrl: options.cdpUrl,
launched: false,
async close() {},
};
}
return connectToHttpEndpoint(options.cdpUrl);
}
const profileDir = ensureChromeProfileDir(resolveChromeProfileDir(options.profileDir));
const reused = await tryReuseChrome(profileDir, options.logger);
if (reused) {
return reused;
}
options.logger?.warn(`No running Chrome debugger found for profile ${profileDir}. Launching Chrome with that profile.`);
try {
return await launchFreshChrome(profileDir, options);
} catch (error) {
const entries = await listChromeProfileEntries(profileDir);
const shouldRetry = shouldRetryChromeLaunchRecovery({
hasLockArtifacts: hasChromeLockArtifacts(entries),
hasLiveOwner: findChromeProcessUsingProfile(profileDir),
});
if (!shouldRetry) {
throw error;
}
options.logger?.warn(`Chrome launch failed with stale profile locks. Cleaning ${profileDir} and retrying once.`);
cleanChromeLockArtifacts(profileDir);
return await launchFreshChrome(profileDir, options);
}
}
@@ -0,0 +1,100 @@
import { readFile, writeFile, mkdir } from "node:fs/promises";
import { dirname, join } from "node:path";
import { resolveChromeProfileDir } from "./profile";
import type { TargetSession } from "./cdp-client";
export interface CdpCookie {
name: string;
value: string;
domain: string;
path: string;
expires: number;
size: number;
httpOnly: boolean;
secure: boolean;
session: boolean;
sameSite?: string;
priority?: string;
sameParty?: boolean;
sourceScheme?: string;
sourcePort?: number;
partitionKey?: string;
}
interface SidecarData {
savedAt: string;
cookies: CdpCookie[];
}
export interface CookieSidecarConfig {
urls: readonly string[];
filename: string;
requiredCookieNames: readonly string[];
filterCookie?: (cookie: CdpCookie) => boolean;
}
function sidecarPath(filename: string, profileDir?: string): string {
return join(resolveChromeProfileDir(profileDir), filename);
}
function hasRequired(cookies: CdpCookie[], names: readonly string[]): boolean {
return names.every((name) =>
cookies.some((c) => c.name === name && Boolean(c.value)),
);
}
async function getCookies(session: TargetSession, urls: readonly string[]): Promise<CdpCookie[]> {
const { cookies } = await session.send<{ cookies: CdpCookie[] }>(
"Network.getCookies",
{ urls: [...urls] },
);
return cookies ?? [];
}
export async function exportCookies(
session: TargetSession,
config: CookieSidecarConfig,
profileDir?: string,
): Promise<boolean> {
const all = await getCookies(session, config.urls);
const filtered = config.filterCookie ? all.filter(config.filterCookie) : all;
if (!hasRequired(filtered, config.requiredCookieNames)) return false;
const filePath = sidecarPath(config.filename, profileDir);
await mkdir(dirname(filePath), { recursive: true });
const data: SidecarData = { savedAt: new Date().toISOString(), cookies: filtered };
await writeFile(filePath, JSON.stringify(data, null, 2));
return true;
}
export async function restoreCookies(
session: TargetSession,
config: CookieSidecarConfig,
profileDir?: string,
): Promise<boolean> {
const live = await getCookies(session, config.urls);
if (hasRequired(live, config.requiredCookieNames)) return false;
const filePath = sidecarPath(config.filename, profileDir);
const raw = await readFile(filePath, "utf8");
const data = JSON.parse(raw) as SidecarData;
if (!data.cookies?.length) return false;
const now = Date.now() / 1000;
const valid = data.cookies.filter((c) => c.session || !c.expires || c.expires > now);
if (!hasRequired(valid, config.requiredCookieNames)) return false;
await session.send("Network.setCookies", {
cookies: valid.map((c) => ({
name: c.name,
value: c.value,
domain: c.domain,
path: c.path,
httpOnly: c.httpOnly,
secure: c.secure,
sameSite: c.sameSite,
expires: c.expires,
})),
});
return true;
}
@@ -0,0 +1,123 @@
import type { WaitForInteractionRequest } from "../adapters/types";
import type { BrowserSession } from "./session";
interface GateSnapshot {
title: string;
currentUrl: string;
bodyText: string;
hasCloudflareTurnstile: boolean;
hasCloudflareChallenge: boolean;
hasRecaptcha: boolean;
hasRecaptchaIframe: boolean;
hasHcaptcha: boolean;
hasHcaptchaIframe: boolean;
}
export function detectInteractionGateFromSnapshot(snapshot: GateSnapshot): WaitForInteractionRequest | null {
const text = snapshot.bodyText.toLowerCase();
const title = snapshot.title.toLowerCase();
const url = snapshot.currentUrl.toLowerCase();
if (
snapshot.hasCloudflareTurnstile ||
snapshot.hasCloudflareChallenge ||
title.includes("just a moment") ||
text.includes("verify you are human") ||
text.includes("checking your browser before accessing") ||
text.includes("enable javascript and cookies to continue") ||
url.includes("/cdn-cgi/challenge-platform/")
) {
return {
type: "wait_for_interaction",
kind: "cloudflare",
provider: "cloudflare",
reason: "Cloudflare human verification detected",
prompt: "Please complete the Cloudflare verification in the opened Chrome window. Extraction will continue automatically once the challenge disappears.",
requiresVisibleBrowser: true,
};
}
if (
snapshot.hasRecaptcha ||
snapshot.hasRecaptchaIframe ||
text.includes("i'm not a robot") ||
text.includes("recaptcha")
) {
return {
type: "wait_for_interaction",
kind: "recaptcha",
provider: "google_recaptcha",
reason: "Google reCAPTCHA detected",
prompt: "Please complete the reCAPTCHA verification in the opened Chrome window. Extraction will continue automatically once the challenge disappears.",
requiresVisibleBrowser: true,
};
}
if (
snapshot.hasHcaptcha ||
snapshot.hasHcaptchaIframe ||
text.includes("hcaptcha")
) {
return {
type: "wait_for_interaction",
kind: "hcaptcha",
provider: "hcaptcha",
reason: "hCaptcha verification detected",
prompt: "Please complete the hCaptcha verification in the opened Chrome window. Extraction will continue automatically once the challenge disappears.",
requiresVisibleBrowser: true,
};
}
return null;
}
export async function detectInteractionGate(browser: BrowserSession): Promise<WaitForInteractionRequest | null> {
const snapshot = await browser.evaluate<GateSnapshot>(`
(() => {
const bodyText = (document.body?.innerText ?? "").slice(0, 4000);
return {
title: document.title ?? "",
currentUrl: window.location.href,
bodyText,
hasCloudflareTurnstile: Boolean(
document.querySelector(
'.cf-turnstile, [name="cf-turnstile-response"], iframe[src*="challenges.cloudflare.com"]'
)
),
hasCloudflareChallenge: Boolean(
document.querySelector(
'#challenge-running, #cf-challenge-running, .challenge-platform, [data-ray], [data-translate="checking_browser"]'
)
),
hasRecaptcha: Boolean(
document.querySelector(
'.g-recaptcha, textarea[name="g-recaptcha-response"], iframe[title*="reCAPTCHA"]'
)
),
hasRecaptchaIframe: Boolean(
document.querySelector('iframe[src*="google.com/recaptcha"], iframe[src*="recaptcha/api2"]')
),
hasHcaptcha: Boolean(
document.querySelector(
'.h-captcha, textarea[name="h-captcha-response"], iframe[title*="hCaptcha"]'
)
),
hasHcaptchaIframe: Boolean(
document.querySelector('iframe[src*="hcaptcha.com"]')
),
};
})()
`).catch(() => ({
title: "",
currentUrl: "",
bodyText: "",
hasCloudflareTurnstile: false,
hasCloudflareChallenge: false,
hasRecaptcha: false,
hasRecaptchaIframe: false,
hasHcaptcha: false,
hasHcaptchaIframe: false,
}));
return detectInteractionGateFromSnapshot(snapshot);
}
@@ -0,0 +1,235 @@
import type { TargetSession } from "./cdp-client";
import type { Logger } from "../utils/logger";
type JsonObject = Record<string, unknown>;
export interface NetworkEntry {
requestId: string;
url: string;
method: string;
resourceType: string;
timestamp: number;
requestHeaders?: Record<string, string>;
requestBody?: string;
status?: number;
statusText?: string;
responseHeaders?: Record<string, string>;
mimeType?: string;
body?: string;
bodyBase64?: boolean;
bodyError?: string;
failed?: boolean;
failureReason?: string;
finished: boolean;
}
function normalizeHeaders(headers: unknown): Record<string, string> | undefined {
if (!headers || typeof headers !== "object") {
return undefined;
}
return Object.fromEntries(
Object.entries(headers as Record<string, unknown>).map(([key, value]) => [key, String(value)]),
);
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
export class NetworkJournal {
private readonly entries = new Map<string, NetworkEntry>();
private lastActivityAt = Date.now();
private started = false;
constructor(
private readonly session: TargetSession,
private readonly log: Logger,
) {}
async start(): Promise<void> {
if (this.started) {
return;
}
this.started = true;
this.session.on("Network.requestWillBeSent", this.handleRequestWillBeSent);
this.session.on("Network.responseReceived", this.handleResponseReceived);
this.session.on("Network.loadingFinished", this.handleLoadingFinished);
this.session.on("Network.loadingFailed", this.handleLoadingFailed);
await this.session.send("Network.enable");
}
stop(): void {
if (!this.started) {
return;
}
this.session.off("Network.requestWillBeSent", this.handleRequestWillBeSent);
this.session.off("Network.responseReceived", this.handleResponseReceived);
this.session.off("Network.loadingFinished", this.handleLoadingFinished);
this.session.off("Network.loadingFailed", this.handleLoadingFailed);
this.started = false;
}
private touch(): void {
this.lastActivityAt = Date.now();
}
private readonly handleRequestWillBeSent = (params: JsonObject): void => {
const requestId = typeof params.requestId === "string" ? params.requestId : undefined;
const request = params.request as JsonObject | undefined;
if (!requestId || !request) {
return;
}
this.touch();
this.entries.set(requestId, {
requestId,
url: String(request.url ?? ""),
method: String(request.method ?? "GET"),
resourceType: String(params.type ?? "Other"),
timestamp: Date.now(),
requestHeaders: normalizeHeaders(request.headers),
requestBody: typeof request.postData === "string" ? request.postData : undefined,
finished: false,
});
};
private readonly handleResponseReceived = (params: JsonObject): void => {
const requestId = typeof params.requestId === "string" ? params.requestId : undefined;
const response = params.response as JsonObject | undefined;
if (!requestId || !response) {
return;
}
this.touch();
const existing = this.entries.get(requestId);
if (!existing) {
return;
}
existing.status = typeof response.status === "number" ? response.status : undefined;
existing.statusText = typeof response.statusText === "string" ? response.statusText : undefined;
existing.responseHeaders = normalizeHeaders(response.headers);
existing.mimeType = typeof response.mimeType === "string" ? response.mimeType : undefined;
this.entries.set(requestId, existing);
};
private readonly handleLoadingFinished = (params: JsonObject): void => {
const requestId = typeof params.requestId === "string" ? params.requestId : undefined;
if (!requestId) {
return;
}
this.touch();
const existing = this.entries.get(requestId);
if (!existing) {
return;
}
existing.finished = true;
this.entries.set(requestId, existing);
};
private readonly handleLoadingFailed = (params: JsonObject): void => {
const requestId = typeof params.requestId === "string" ? params.requestId : undefined;
if (!requestId) {
return;
}
this.touch();
const existing = this.entries.get(requestId);
if (!existing) {
return;
}
existing.finished = true;
existing.failed = true;
existing.failureReason = typeof params.errorText === "string" ? params.errorText : "Unknown error";
this.entries.set(requestId, existing);
};
getEntries(): NetworkEntry[] {
return Array.from(this.entries.values());
}
findEntries(predicate: (entry: NetworkEntry) => boolean): NetworkEntry[] {
return this.getEntries().filter(predicate);
}
async waitForIdle(options: { idleMs?: number; timeoutMs?: number } = {}): Promise<void> {
const idleMs = options.idleMs ?? 1_200;
const timeoutMs = options.timeoutMs ?? 15_000;
const startedAt = Date.now();
while (Date.now() - startedAt < timeoutMs) {
if (Date.now() - this.lastActivityAt >= idleMs) {
return;
}
await sleep(Math.min(150, idleMs));
}
throw new Error("Timed out waiting for network idle");
}
async waitForResponse(
predicate: (entry: NetworkEntry) => boolean,
options: { timeoutMs?: number } = {},
): Promise<NetworkEntry> {
const timeoutMs = options.timeoutMs ?? 10_000;
const startedAt = Date.now();
while (Date.now() - startedAt < timeoutMs) {
const matched = this.getEntries().find((entry) => entry.finished && predicate(entry));
if (matched) {
return matched;
}
await sleep(150);
}
throw new Error("Timed out waiting for matching network response");
}
async ensureBody(entry: NetworkEntry): Promise<string | undefined> {
if (entry.body !== undefined) {
return entry.body;
}
if (entry.bodyError || entry.failed || !entry.finished) {
return undefined;
}
try {
const result = await this.session.send<{ body: string; base64Encoded: boolean }>("Network.getResponseBody", {
requestId: entry.requestId,
});
entry.bodyBase64 = result.base64Encoded;
entry.body = result.base64Encoded ? Buffer.from(result.body, "base64").toString("utf8") : result.body;
return entry.body;
} catch (error) {
entry.bodyError = error instanceof Error ? error.message : String(error);
this.log.debug(`Failed to fetch response body for ${entry.url}: ${entry.bodyError}`);
return undefined;
}
}
async getJsonBody(entry: NetworkEntry): Promise<unknown | null> {
const body = await this.ensureBody(entry);
if (!body) {
return null;
}
try {
return JSON.parse(body);
} catch {
return null;
}
}
async toJSON(options: { includeBodies?: boolean } = {}): Promise<NetworkEntry[]> {
const entries = this.getEntries();
if (!options.includeBodies) {
return entries;
}
await Promise.all(entries.map((entry) => this.ensureBody(entry)));
return entries;
}
}
@@ -0,0 +1,105 @@
import type { BrowserSession } from "./session";
export interface CapturedPageSnapshot {
html: string;
finalUrl: string;
}
export const CAPTURE_NORMALIZED_PAGE_SCRIPT = String.raw`
(() => {
const baseUrl = document.baseURI || location.href;
const htmlClone = document.documentElement.cloneNode(true);
function materializeShadowDom(sourceRoot, cloneRoot) {
const sourceElements = Array.from(sourceRoot.querySelectorAll("*"));
const cloneElements = Array.from(cloneRoot.querySelectorAll("*"));
for (let index = sourceElements.length - 1; index >= 0; index -= 1) {
const sourceElement = sourceElements[index];
const cloneElement = cloneElements[index];
const shadowRoot = sourceElement && sourceElement.shadowRoot;
if (!shadowRoot || !cloneElement || !shadowRoot.innerHTML) {
continue;
}
if (cloneElement.tagName && cloneElement.tagName.includes("-")) {
const wrapper = document.createElement("div");
wrapper.setAttribute("data-shadow-host", cloneElement.tagName.toLowerCase());
wrapper.innerHTML = shadowRoot.innerHTML;
cloneElement.replaceWith(wrapper);
} else {
cloneElement.innerHTML = shadowRoot.innerHTML;
}
}
}
function toAbsolute(url) {
if (!url) return url;
try {
return new URL(url, baseUrl).href;
} catch {
return url;
}
}
function absolutizeAttribute(root, selector, attribute) {
root.querySelectorAll(selector).forEach((element) => {
const value = element.getAttribute(attribute);
if (!value) return;
const absolute = toAbsolute(value);
if (absolute) {
element.setAttribute(attribute, absolute);
}
});
}
function absolutizeSrcset(root, selector) {
root.querySelectorAll(selector).forEach((element) => {
const srcset = element.getAttribute("srcset");
if (!srcset) return;
element.setAttribute(
"srcset",
srcset
.split(",")
.map((part) => {
const trimmed = part.trim();
if (!trimmed) return "";
const [url, ...descriptor] = trimmed.split(/\s+/);
const absolute = toAbsolute(url);
return descriptor.length > 0 ? absolute + " " + descriptor.join(" ") : absolute;
})
.filter(Boolean)
.join(", "),
);
});
}
materializeShadowDom(document.documentElement, htmlClone);
htmlClone
.querySelectorAll("img[data-src], video[data-src], audio[data-src], source[data-src]")
.forEach((element) => {
const dataSource = element.getAttribute("data-src");
const current = element.getAttribute("src");
if (dataSource && (!current || current === "" || current.startsWith("data:"))) {
element.setAttribute("src", dataSource);
}
});
absolutizeAttribute(htmlClone, "a[href]", "href");
absolutizeAttribute(htmlClone, "img[src], video[src], audio[src], source[src], iframe[src]", "src");
absolutizeAttribute(htmlClone, "video[poster]", "poster");
absolutizeSrcset(htmlClone, "img[srcset], source[srcset]");
return {
html: "<!doctype html>\n" + htmlClone.outerHTML,
finalUrl: location.href,
};
})()
`;
export async function captureNormalizedPageSnapshot(
browser: BrowserSession,
): Promise<CapturedPageSnapshot> {
return browser.evaluate<CapturedPageSnapshot>(CAPTURE_NORMALIZED_PAGE_SCRIPT);
}
+201
View File
@@ -0,0 +1,201 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import process from "node:process";
import { spawnSync } from "node:child_process";
export interface ResolveSharedChromeProfileDirOptions {
envNames?: string[];
appDataDirName?: string;
profileDirName?: string;
}
export interface FindExistingChromeDebugPortOptions {
profileDir: string;
timeoutMs?: number;
}
interface ChromeVersionResponse {
webSocketDebuggerUrl?: string;
}
const CHROME_LOCK_FILE_NAMES = ["SingletonLock", "SingletonSocket", "SingletonCookie", "chrome.pid"] as const;
function resolveDataBaseDir(): string {
if (process.platform === "darwin") {
return path.join(os.homedir(), "Library", "Application Support");
}
if (process.platform === "win32") {
return process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming");
}
return process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share");
}
export function resolveSharedChromeProfileDir(
options: ResolveSharedChromeProfileDirOptions = {},
): string {
for (const envName of options.envNames ?? []) {
const override = process.env[envName]?.trim();
if (override) {
return path.resolve(override);
}
}
const appDataDirName = options.appDataDirName ?? "baoyu-skills";
const profileDirName = options.profileDirName ?? "chrome-profile";
return path.join(resolveDataBaseDir(), appDataDirName, profileDirName);
}
export function resolveChromeProfileDir(profileDir?: string): string {
if (profileDir?.trim()) {
return path.resolve(profileDir.trim());
}
return resolveSharedChromeProfileDir({
envNames: ["BAOYU_CHROME_PROFILE_DIR"],
appDataDirName: "baoyu-skills",
profileDirName: "chrome-profile",
});
}
export function ensureChromeProfileDir(profileDir: string): string {
fs.mkdirSync(profileDir, { recursive: true });
return profileDir;
}
export function hasChromeLockArtifacts(entries: readonly string[]): boolean {
return CHROME_LOCK_FILE_NAMES.some((name) => entries.includes(name));
}
export function shouldRetryChromeLaunchRecovery(options: {
hasLockArtifacts: boolean;
hasLiveOwner: boolean;
}): boolean {
return options.hasLockArtifacts && !options.hasLiveOwner;
}
export function findChromeProcessUsingProfile(profileDir: string): boolean {
if (process.platform === "win32") {
return false;
}
try {
const result = spawnSync("ps", ["aux"], {
encoding: "utf8",
timeout: 5_000,
});
if (result.status !== 0 || !result.stdout) {
return false;
}
return result.stdout
.split("\n")
.some((line) => line.includes(`--user-data-dir=${profileDir}`));
} catch {
return false;
}
}
export function cleanChromeLockArtifacts(profileDir: string): void {
for (const name of CHROME_LOCK_FILE_NAMES) {
try {
fs.unlinkSync(path.join(profileDir, name));
} catch {
// Ignore missing files and continue cleaning the remaining artifacts.
}
}
}
export async function listChromeProfileEntries(profileDir: string): Promise<string[]> {
try {
return await fs.promises.readdir(profileDir);
} catch {
return [];
}
}
async function fetchWithTimeout(url: string, timeoutMs = 3_000): Promise<Response> {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeoutMs);
try {
return await fetch(url, {
redirect: "follow",
signal: controller.signal,
});
} finally {
clearTimeout(timer);
}
}
async function fetchJson<T>(url: string, timeoutMs = 3_000): Promise<T> {
const response = await fetchWithTimeout(url, timeoutMs);
if (!response.ok) {
throw new Error(`Request failed: ${response.status} ${response.statusText}`);
}
return (await response.json()) as T;
}
async function isDebugPortReady(port: number, timeoutMs = 3_000): Promise<boolean> {
try {
const version = await fetchJson<ChromeVersionResponse>(`http://127.0.0.1:${port}/json/version`, timeoutMs);
return Boolean(version.webSocketDebuggerUrl);
} catch {
return false;
}
}
function parseDevToolsActivePort(filePath: string): { port: number; wsPath: string } | null {
try {
const content = fs.readFileSync(filePath, "utf8");
const lines = content.split(/\r?\n/);
const port = Number.parseInt(lines[0]?.trim() ?? "", 10);
const wsPath = lines[1]?.trim() ?? "";
if (port > 0 && wsPath) {
return { port, wsPath };
}
} catch {
// Ignore and fall back to process inspection.
}
return null;
}
export async function findExistingChromeDebugPort(
options: FindExistingChromeDebugPortOptions,
): Promise<number | null> {
const timeoutMs = options.timeoutMs ?? 3_000;
const activePort = parseDevToolsActivePort(path.join(options.profileDir, "DevToolsActivePort"));
if (activePort && await isDebugPortReady(activePort.port, timeoutMs)) {
return activePort.port;
}
if (process.platform === "win32") {
return null;
}
try {
const result = spawnSync("ps", ["aux"], {
encoding: "utf8",
timeout: 5_000,
});
if (result.status !== 0 || !result.stdout) {
return null;
}
const lines = result.stdout
.split("\n")
.filter((line) => line.includes(options.profileDir) && line.includes("--remote-debugging-port="));
for (const line of lines) {
const match = line.match(/--remote-debugging-port=(\d+)/);
const port = Number.parseInt(match?.[1] ?? "", 10);
if (port > 0 && await isDebugPortReady(port, timeoutMs)) {
return port;
}
}
} catch {
// Ignore and report no reusable debugger.
}
return null;
}
+155
View File
@@ -0,0 +1,155 @@
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { CdpClient, TargetSession, evaluateRuntime } from "./cdp-client";
interface NavigationResult {
errorText?: string;
}
const execFileAsync = promisify(execFile);
const MACOS_BROWSER_APP_IDS = [
"com.google.Chrome",
"org.chromium.Chromium",
"com.brave.Browser",
"com.microsoft.edgemac",
];
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function activateBrowserApp(): Promise<void> {
if (process.platform !== "darwin") {
return;
}
for (const appId of MACOS_BROWSER_APP_IDS) {
try {
await execFileAsync("osascript", ["-e", `tell application id "${appId}" to activate`]);
return;
} catch {
// Try the next installed browser bundle id.
}
}
}
export class BrowserSession {
private constructor(
private readonly cdp: CdpClient,
public readonly targetSession: TargetSession,
public readonly interactive: boolean,
) {}
static async open(
cdp: CdpClient,
options: {
initialUrl?: string;
interactive?: boolean;
} = {},
): Promise<BrowserSession> {
const targetSession = await cdp.createPageSession({
initialUrl: options.initialUrl,
visible: options.interactive,
});
const browser = new BrowserSession(cdp, targetSession, Boolean(options.interactive));
if (browser.interactive) {
await browser.bringToFront().catch(() => {});
}
return browser;
}
async goto(url: string, timeoutMs = 30_000): Promise<void> {
const loadPromise = this.targetSession.waitForEvent("Page.loadEventFired", undefined, timeoutMs).catch(() => null);
const result = await this.targetSession.send<NavigationResult>("Page.navigate", { url });
if (result.errorText) {
throw new Error(`Navigation failed: ${result.errorText}`);
}
await loadPromise;
await this.waitForReadyState(timeoutMs);
}
async waitForReadyState(timeoutMs = 30_000): Promise<void> {
const startedAt = Date.now();
while (Date.now() - startedAt < timeoutMs) {
const state = await this.evaluate<string>("document.readyState");
if (state === "interactive" || state === "complete") {
return;
}
await sleep(150);
}
throw new Error("Timed out waiting for document.readyState");
}
async evaluate<T>(expression: string): Promise<T> {
return evaluateRuntime<T>(this.targetSession, expression);
}
async getHTML(): Promise<string> {
return this.evaluate<string>("document.documentElement.outerHTML");
}
async getTitle(): Promise<string> {
return this.evaluate<string>("document.title");
}
async getURL(): Promise<string> {
return this.evaluate<string>("window.location.href");
}
async bringToFront(): Promise<void> {
await this.targetSession.send("Page.bringToFront").catch(async () => {
await this.cdp.sendBrowserCommand("Target.activateTarget", {
targetId: this.targetSession.targetId,
});
});
if (this.interactive) {
await activateBrowserApp().catch(() => {});
}
}
async click(selector: string): Promise<void> {
const result = await this.evaluate<{ ok: boolean; error?: string }>(`
(() => {
const element = document.querySelector(${JSON.stringify(selector)});
if (!element) {
return { ok: false, error: "Element not found" };
}
element.scrollIntoView({ block: "center", inline: "center" });
if (element instanceof HTMLElement) {
element.click();
return { ok: true };
}
return { ok: false, error: "Element is not clickable" };
})()
`);
if (!result.ok) {
throw new Error(result.error ?? `Failed to click ${selector}`);
}
}
async scrollToEnd(options: { stepPx?: number; delayMs?: number; maxSteps?: number } = {}): Promise<void> {
const stepPx = options.stepPx ?? 1_400;
const delayMs = options.delayMs ?? 250;
const maxSteps = options.maxSteps ?? 6;
for (let step = 0; step < maxSteps; step += 1) {
const done = await this.evaluate<boolean>(`
(() => {
const before = window.scrollY;
window.scrollBy(0, ${stepPx});
const atBottom = window.innerHeight + window.scrollY >= document.body.scrollHeight - 4;
return atBottom || window.scrollY === before;
})()
`);
if (done) {
break;
}
await sleep(delayMs);
}
}
async close(): Promise<void> {
await this.cdp.closeTarget(this.targetSession.targetId);
}
}
+227
View File
@@ -0,0 +1,227 @@
#!/usr/bin/env bun
import {
runConvertCommand,
type ConvertCommandOptions,
type OutputFormat,
type WaitMode,
} from "./commands/convert";
export const HELP_TEXT = `
baoyu-fetch - Read a URL into Markdown or JSON with Chrome CDP
Usage:
baoyu-fetch <url> [options]
Options:
--output <file> Save output to file
--format <type> Output format: markdown | json
--json Alias for --format json
--adapter <name> Force an adapter (e.g. x, generic)
--download-media Download adapter-reported media into ./imgs and ./videos, then rewrite markdown links
--media-dir <dir> Base directory for downloaded media. Defaults to the output directory
--debug-dir <dir> Write debug artifacts
--cdp-url <url> Reuse an existing Chrome DevTools endpoint
--browser-path <path> Explicit Chrome binary path
--chrome-profile-dir <path>
Chrome user data dir. Defaults to BAOYU_CHROME_PROFILE_DIR
or baoyu-skills/chrome-profile.
--headless Launch a temporary headless Chrome if needed
--wait-for <mode> Wait mode: interaction | force
interaction: start visible Chrome and auto-wait only when login or verification is required
force: start visible Chrome, then auto-continue after it detects login/challenge progress
or continue immediately when you press Enter
--wait-for-interaction
Alias for --wait-for interaction
--wait-for-login Alias for --wait-for interaction
--interaction-timeout <ms>
How long to wait for manual interaction before failing (default: 600000)
--interaction-poll-interval <ms>
How often to poll interaction state while waiting (default: 1500)
--login-timeout <ms> Alias for --interaction-timeout
--login-poll-interval <ms>
Alias for --interaction-poll-interval
--timeout <ms> Page timeout in milliseconds (default: 30000)
--help Show help
Examples:
baoyu-fetch https://example.com
baoyu-fetch https://example.com --format markdown --output article.md --download-media
baoyu-fetch https://example.com --format json --output article.json
baoyu-fetch https://x.com/lennysan/status/2036483059407810640 --wait-for interaction
baoyu-fetch https://x.com/lennysan/status/2036483059407810640 --wait-for force
`.trim();
interface CliOptions extends ConvertCommandOptions {
url?: string;
help: boolean;
}
function normalizeWaitMode(raw: string): WaitMode {
const value = raw.toLowerCase();
if (value === "interaction" || value === "auto") {
return "interaction";
}
if (value === "force" || value === "manual" || value === "always") {
return "force";
}
throw new Error(`Invalid wait mode: ${raw}. Expected interaction or force.`);
}
function normalizeOutputFormat(raw: string): OutputFormat {
const value = raw.toLowerCase();
if (value === "markdown" || value === "json") {
return value;
}
throw new Error(`Invalid output format: ${raw}. Expected markdown or json.`);
}
export function parseArgs(argv: string[]): CliOptions {
const options: CliOptions = {
format: "markdown",
headless: false,
downloadMedia: false,
waitMode: "none",
interactionTimeoutMs: 600_000,
interactionPollIntervalMs: 1_500,
timeoutMs: 30_000,
help: false,
};
const args = argv.slice(2);
for (let index = 0; index < args.length; index += 1) {
const value = args[index];
if (value === "--help" || value === "-h") {
options.help = true;
continue;
}
if (value === "--format") {
const format = args[index + 1];
if (!format) {
throw new Error("--format requires a value");
}
options.format = normalizeOutputFormat(format);
index += 1;
continue;
}
if (value === "--json") {
options.format = "json";
continue;
}
if (value === "--download-media") {
options.downloadMedia = true;
continue;
}
if (value === "--headless") {
options.headless = true;
continue;
}
if (value === "--wait-for") {
const mode = args[index + 1];
if (!mode) {
throw new Error("--wait-for requires a mode");
}
options.waitMode = normalizeWaitMode(mode);
index += 1;
continue;
}
if (value === "--wait-for-interaction" || value === "--wait-for-login") {
options.waitMode = "interaction";
continue;
}
if (value === "--output") {
options.output = args[index + 1];
index += 1;
continue;
}
if (value === "--adapter") {
options.adapter = args[index + 1];
index += 1;
continue;
}
if (value === "--debug-dir") {
options.debugDir = args[index + 1];
index += 1;
continue;
}
if (value === "--media-dir") {
options.mediaDir = args[index + 1];
index += 1;
continue;
}
if (value === "--cdp-url") {
options.cdpUrl = args[index + 1];
index += 1;
continue;
}
if (value === "--browser-path") {
options.browserPath = args[index + 1];
index += 1;
continue;
}
if (value === "--chrome-profile-dir") {
options.chromeProfileDir = args[index + 1];
index += 1;
continue;
}
if (value === "--timeout") {
const parsed = Number(args[index + 1]);
if (!Number.isFinite(parsed) || parsed <= 0) {
throw new Error(`Invalid timeout: ${args[index + 1]}`);
}
options.timeoutMs = parsed;
index += 1;
continue;
}
if (value === "--interaction-timeout" || value === "--login-timeout") {
const parsed = Number(args[index + 1]);
if (!Number.isFinite(parsed) || parsed <= 0) {
throw new Error(`Invalid interaction timeout: ${args[index + 1]}`);
}
options.interactionTimeoutMs = parsed;
index += 1;
continue;
}
if (value === "--interaction-poll-interval" || value === "--login-poll-interval") {
const parsed = Number(args[index + 1]);
if (!Number.isFinite(parsed) || parsed <= 0) {
throw new Error(`Invalid interaction poll interval: ${args[index + 1]}`);
}
options.interactionPollIntervalMs = parsed;
index += 1;
continue;
}
if (value.startsWith("-")) {
throw new Error(`Unknown option: ${value}`);
}
if (!options.url) {
options.url = value;
continue;
}
throw new Error(`Unexpected argument: ${value}`);
}
return options;
}
async function main(): Promise<void> {
try {
const options = parseArgs(process.argv);
if (options.help || !options.url) {
console.log(HELP_TEXT);
return;
}
await runConvertCommand(options);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.error(message);
process.exitCode = 1;
}
}
if (import.meta.main) {
void main();
}
@@ -0,0 +1,580 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { createInterface } from "node:readline";
import { connectChrome, type ChromeConnection } from "../browser/chrome-launcher";
import { CdpClient } from "../browser/cdp-client";
import { detectInteractionGate } from "../browser/interaction-gates";
import { NetworkJournal } from "../browser/network-journal";
import { BrowserSession } from "../browser/session";
import { genericAdapter, resolveAdapter } from "../adapters";
import { isXSessionReady } from "../adapters/x/session";
import type { ExtractedDocument } from "../extract/document";
import { renderMarkdown } from "../extract/markdown-renderer";
import { downloadMediaAssets } from "../media/default-downloader";
import { rewriteMarkdownMediaLinks } from "../media/markdown-media";
import { createLogger } from "../utils/logger";
import { normalizeUrl } from "../utils/url";
import type {
Adapter,
AdapterContext,
AdapterLoginInfo,
LoginState,
MediaAsset,
WaitForInteractionRequest,
} from "../adapters/types";
export type WaitMode = "none" | "interaction" | "force";
export type OutputFormat = "markdown" | "json";
export interface ConvertCommandOptions {
url?: string;
output?: string;
format: OutputFormat;
adapter?: string;
debugDir?: string;
cdpUrl?: string;
browserPath?: string;
chromeProfileDir?: string;
headless: boolean;
downloadMedia: boolean;
mediaDir?: string;
waitMode: WaitMode;
interactionTimeoutMs: number;
interactionPollIntervalMs: number;
timeoutMs: number;
}
interface RuntimeResources {
chrome: ChromeConnection;
cdp: CdpClient;
browser: BrowserSession;
network: NetworkJournal;
interactive: boolean;
}
interface ForceWaitSnapshot {
url: string;
hasGate: boolean;
loginState: LoginState | "unavailable";
sessionReady: boolean;
}
interface SuccessfulConvertOutput {
adapter: string;
status: "ok";
login?: AdapterLoginInfo;
media: MediaAsset[];
downloads: Awaited<ReturnType<typeof downloadMediaAssets>> | null;
document: ExtractedDocument;
markdown: string;
}
interface InteractionRequiredOutput {
adapter: string;
status: "needs_interaction";
login?: AdapterLoginInfo;
interaction: WaitForInteractionRequest;
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function isForceWaitSessionReady(snapshot: ForceWaitSnapshot): boolean {
return snapshot.sessionReady;
}
export function shouldKeepBrowserOpenAfterInteraction(options: {
launched: boolean;
interaction: Pick<WaitForInteractionRequest, "kind" | "provider">;
}): boolean {
return options.launched && options.interaction.kind === "login" && options.interaction.provider === "x";
}
export function shouldAutoContinueForceWait(
initial: ForceWaitSnapshot,
current: ForceWaitSnapshot,
): boolean {
if (initial.hasGate && !current.hasGate) {
return true;
}
if (initial.loginState === "logged_out" && current.loginState !== "logged_out" && isForceWaitSessionReady(current)) {
return true;
}
if (initial.loginState !== "logged_in" && current.loginState === "logged_in" && isForceWaitSessionReady(current)) {
return true;
}
if (
current.url !== initial.url &&
!current.hasGate &&
current.loginState !== "logged_out" &&
isForceWaitSessionReady(current)
) {
return true;
}
return false;
}
async function writeOutput(path: string, content: string): Promise<void> {
const directory = path.includes("/") ? path.slice(0, path.lastIndexOf("/")) : "";
if (directory) {
await mkdir(directory, { recursive: true });
}
await writeFile(path, content, "utf8");
}
async function writeDebugArtifacts(
debugDir: string,
document: ExtractedDocument,
markdown: string,
browser: BrowserSession,
network: NetworkJournal,
): Promise<void> {
await mkdir(debugDir, { recursive: true });
const html = await browser.getHTML().catch(() => "");
const networkDump = await network.toJSON({ includeBodies: true });
await Promise.all([
writeFile(join(debugDir, "document.json"), JSON.stringify(document, null, 2), "utf8"),
writeFile(join(debugDir, "markdown.md"), markdown, "utf8"),
writeFile(join(debugDir, "page.html"), html, "utf8"),
writeFile(join(debugDir, "network.json"), JSON.stringify(networkDump, null, 2), "utf8"),
]);
}
async function openRuntime(
options: ConvertCommandOptions,
interactive: boolean,
debugEnabled: boolean,
): Promise<RuntimeResources> {
const logger = createLogger(debugEnabled);
if (interactive) {
logger.info("Opening Chrome in interactive mode.");
}
const chrome = await connectChrome({
cdpUrl: options.cdpUrl,
browserPath: options.browserPath,
profileDir: options.chromeProfileDir,
headless: interactive ? false : options.headless,
logger,
});
const cdp = await CdpClient.connect(chrome.browserWsUrl);
const browser = await BrowserSession.open(cdp, { interactive });
if (interactive) {
await browser.bringToFront().catch(() => {});
}
const network = new NetworkJournal(browser.targetSession, logger);
await network.start();
return {
chrome,
cdp,
browser,
network,
interactive,
};
}
async function closeRuntime(runtime: RuntimeResources | null | undefined): Promise<void> {
if (!runtime) {
return;
}
runtime.network.stop();
await runtime.browser.close().catch(() => {});
await runtime.cdp.close().catch(() => {});
await runtime.chrome.close().catch(() => {});
}
async function isInteractionSessionReady(
context: AdapterContext,
interaction: WaitForInteractionRequest,
): Promise<boolean> {
if (interaction.provider !== "x") {
return true;
}
return await isXSessionReady(context).catch(() => false);
}
async function reopenInteractiveRuntime(
runtime: RuntimeResources,
options: ConvertCommandOptions,
debugEnabled: boolean,
): Promise<RuntimeResources> {
if (runtime.interactive) {
return runtime;
}
await closeRuntime(runtime);
return openRuntime(options, true, debugEnabled);
}
async function captureForceWaitSnapshot(
adapter: Adapter,
context: AdapterContext,
): Promise<ForceWaitSnapshot> {
const [gate, url, login] = await Promise.all([
detectInteractionGate(context.browser).catch(() => null),
context.browser.getURL().catch(() => context.input.url.toString()),
adapter.checkLogin?.(context).catch(() => ({
provider: adapter.name,
state: "unknown" as const,
})),
]);
return {
url,
hasGate: Boolean(gate),
loginState: login?.state ?? "unavailable",
sessionReady: adapter.name === "x" ? await isXSessionReady(context).catch(() => false) : true,
};
}
async function waitForForceResume(
adapter: Adapter,
context: AdapterContext,
options: ConvertCommandOptions,
): Promise<void> {
if (context.interactive) {
await context.browser.bringToFront().catch(() => {});
}
const prompt =
"Chrome is ready. Complete any manual login or verification. Extraction will continue automatically after it detects progress, or press Enter to continue immediately.";
context.log.info(prompt);
const rl = createInterface({
input: process.stdin,
output: process.stderr,
});
let manualContinue = false;
let closed = false;
const closeReadline = (): void => {
if (!closed) {
closed = true;
rl.close();
}
};
rl.once("line", () => {
manualContinue = true;
closeReadline();
});
const initial = await captureForceWaitSnapshot(adapter, context);
const startedAt = Date.now();
try {
while (Date.now() - startedAt < options.interactionTimeoutMs) {
if (manualContinue) {
return;
}
const current = await captureForceWaitSnapshot(adapter, context);
if (shouldAutoContinueForceWait(initial, current)) {
return;
}
await sleep(options.interactionPollIntervalMs);
}
} finally {
closeReadline();
}
throw new Error("Timed out waiting for force-mode interaction to complete");
}
async function waitForInteraction(
adapter: Adapter,
context: AdapterContext,
interaction: WaitForInteractionRequest,
options: ConvertCommandOptions,
): Promise<AdapterLoginInfo> {
const timeoutMs = interaction.timeoutMs ?? options.interactionTimeoutMs;
const pollIntervalMs = interaction.pollIntervalMs ?? options.interactionPollIntervalMs;
if (context.interactive) {
await context.browser.bringToFront().catch(() => {});
}
context.log.info(interaction.prompt);
const startedAt = Date.now();
let lastLogin: AdapterLoginInfo | null = null;
while (Date.now() - startedAt < timeoutMs) {
if (interaction.kind === "login" && adapter.checkLogin) {
lastLogin = await adapter.checkLogin(context);
if (lastLogin.state === "logged_in" && await isInteractionSessionReady(context, interaction)) {
return lastLogin;
}
}
const gate = await detectInteractionGate(context.browser);
if (!gate) {
if (interaction.kind !== "login") {
return lastLogin ?? {
provider: interaction.provider,
state: "unknown",
reason: `${interaction.provider} challenge cleared`,
};
}
if (!adapter.checkLogin) {
return {
provider: interaction.provider,
state: "unknown",
};
}
lastLogin = await adapter.checkLogin(context);
if (lastLogin.state !== "logged_out" && await isInteractionSessionReady(context, interaction)) {
return lastLogin;
}
}
await sleep(pollIntervalMs);
}
const reason = lastLogin?.reason ? ` (${lastLogin.reason})` : "";
throw new Error(`Timed out waiting for ${interaction.provider} interaction${reason}`);
}
export function formatOutputContent(
format: OutputFormat,
payload: SuccessfulConvertOutput | InteractionRequiredOutput,
): string {
if (format === "json") {
return JSON.stringify(payload, null, 2);
}
if (payload.status !== "ok") {
throw new Error("Markdown output is only available for successful extraction results");
}
return payload.markdown;
}
function printOutput(content: string): void {
process.stdout.write(content);
if (!content.endsWith("\n")) {
process.stdout.write("\n");
}
}
export async function runConvertCommand(options: ConvertCommandOptions): Promise<void> {
if (!options.url) {
throw new Error("URL is required");
}
if (options.downloadMedia && !options.output) {
throw new Error("--download-media requires --output so media paths can be rewritten relative to the saved output file");
}
const url = normalizeUrl(options.url);
let runtime = await openRuntime(options, options.waitMode !== "none", Boolean(options.debugDir));
const logger = createLogger(Boolean(options.debugDir));
let didLogin = false;
let adapter: Adapter | null = null;
let context: AdapterContext | null = null;
try {
adapter = resolveAdapter({ url }, options.adapter);
context = {
input: { url },
browser: runtime.browser,
network: runtime.network,
cdp: runtime.cdp,
log: logger,
outputFormat: options.format,
timeoutMs: options.timeoutMs,
interactive: runtime.interactive,
downloadMedia: options.downloadMedia,
};
if (adapter.restoreCookies) {
const restored = await adapter.restoreCookies(context, runtime.chrome.profileDir).catch(() => false);
if (restored) logger.info(`Restored ${adapter.name} session cookies from sidecar.`);
}
if (options.waitMode === "interaction" && adapter.checkLogin) {
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
const preLogin = await adapter.checkLogin(context);
if (preLogin.state !== "logged_in") {
didLogin = true;
await waitForInteraction(adapter, context, {
type: "wait_for_interaction",
kind: "login",
provider: preLogin.provider ?? adapter.name,
prompt: `Please sign in to ${adapter.name === "x" ? "X" : adapter.name} in the opened Chrome window. Extraction will continue automatically once login is detected.`,
reason: preLogin.reason ?? `Not logged in to ${adapter.name}`,
requiresVisibleBrowser: true,
}, options);
}
}
if (options.waitMode === "force") {
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
await waitForForceResume(adapter, context, options);
}
let result = await adapter.process(context);
if (result.status === "no_document") {
const interaction = await detectInteractionGate(context.browser);
if (interaction) {
result = {
status: "needs_interaction",
interaction,
login: result.login,
};
}
}
while (result.status === "needs_interaction") {
if (options.waitMode === "none") {
if (options.format === "json") {
printOutput(
formatOutputContent(options.format, {
adapter: adapter.name,
status: result.status,
login: result.login,
interaction: result.interaction,
}),
);
return;
}
throw new Error(`${adapter.name} requires manual interaction. Re-run with --wait-for interaction to continue after completing it.`);
}
if (result.interaction.requiresVisibleBrowser !== false) {
runtime = await reopenInteractiveRuntime(runtime, options, Boolean(options.debugDir));
}
context = {
input: { url },
browser: runtime.browser,
network: runtime.network,
cdp: runtime.cdp,
log: logger,
outputFormat: options.format,
timeoutMs: options.timeoutMs,
interactive: runtime.interactive,
downloadMedia: options.downloadMedia,
};
await context.browser.goto(url.toString(), options.timeoutMs).catch(() => {});
if (result.interaction.kind === "login") {
didLogin = true;
}
await waitForInteraction(adapter, context, result.interaction, options);
result = await adapter.process(context);
if (result.status === "no_document") {
const interaction = await detectInteractionGate(context.browser);
if (interaction) {
result = {
status: "needs_interaction",
interaction,
login: result.login,
};
}
}
}
let document: ExtractedDocument | null = result.status === "ok" ? result.document : null;
let media: MediaAsset[] = result.status === "ok" ? (result.media ?? []) : [];
let login = result.login;
let mediaAdapter = adapter;
if (!document && adapter.name !== genericAdapter.name && result.status === "no_document") {
logger.info(`Adapter ${adapter.name} returned no structured document; falling back to generic extraction`);
const fallback = await genericAdapter.process(context);
if (fallback.status === "ok") {
document = fallback.document;
media = fallback.media ?? [];
mediaAdapter = genericAdapter;
}
}
if (!document) {
throw new Error("Failed to extract a document from the target URL");
}
document.requestedUrl ??= url.toString();
let markdown = renderMarkdown(document);
let downloadResult:
| Awaited<ReturnType<typeof downloadMediaAssets>>
| null = null;
if (options.downloadMedia && options.output) {
downloadResult = mediaAdapter.downloadMedia
? await mediaAdapter.downloadMedia({
media,
outputPath: options.output,
mediaDir: options.mediaDir,
log: logger,
})
: await downloadMediaAssets({
media,
outputPath: options.output,
mediaDir: options.mediaDir,
log: logger,
});
markdown = rewriteMarkdownMediaLinks(markdown, downloadResult.replacements);
if (downloadResult.downloadedImages > 0 || downloadResult.downloadedVideos > 0) {
logger.info(
`Downloaded ${downloadResult.downloadedImages} images and ${downloadResult.downloadedVideos} videos`,
);
}
}
if (options.output) {
await writeOutput(
options.output,
formatOutputContent(options.format, {
adapter: document.adapter ?? adapter.name,
status: "ok",
login,
media,
downloads: downloadResult,
document,
markdown,
}),
);
logger.info(`Saved ${options.format} to ${options.output}`);
}
if (options.debugDir) {
await writeDebugArtifacts(options.debugDir, document, markdown, runtime.browser, runtime.network);
logger.info(`Wrote debug artifacts to ${options.debugDir}`);
}
if (options.format === "json") {
printOutput(
formatOutputContent(options.format, {
adapter: document.adapter ?? adapter.name,
status: "ok",
login,
media,
downloads: downloadResult,
document,
markdown,
}),
);
return;
}
printOutput(markdown);
} finally {
if (adapter?.exportCookies && context) {
await adapter.exportCookies(context, runtime.chrome.profileDir).catch(() => {});
}
await closeRuntime(runtime);
}
}

Some files were not shown because too many files have changed in this diff Show More