mirror of
https://github.com/JimLiu/baoyu-skills.git
synced 2026-08-08 17:53:03 +08:00
Jim Liu 宝玉
2c800c670a
* docs: add runtime-neutral User Input Tools convention across skills Introduce docs/user-input-tools.md as the author-side canonical source and inline the tool-selection rule into every SKILL.md that prompts the user. Also add Skill Self-Containment and User Input Tools sections to CLAUDE.md and the copy-verbatim template to docs/creating-skills.md, so skills stay portable across Claude Code, Codex, Hermes, and other runtimes. * feat: runtime-neutral image generation convention across skills - Introduce inline `## Image Generation Tools` rule in every rendering SKILL.md so skills delegate backend choice instead of hard-coding one; author-side canonical copy lives in docs/image-generation-tools.md. - Add `## Reference Images` support (`--ref`, frontmatter `references:` with direct/style/palette usage) to all seven image-rendering skills. - Move build-batch.ts (with ref propagation into batch JSON) from baoyu-article-illustrator to baoyu-imagine so non-backend skills don't own backend-specific scripts; update baoyu-image-gen stub in sync and relax the CLAUDE.md deprecation note accordingly. * refactor: slim heavy SKILL.md files and move detail to references/ Trim the four largest active skills and move presets, option tables, and confirmation scripts into per-skill references/ so SKILL.md stays focused on the decision flow. - baoyu-slide-deck: 761→258, + styles-gallery.md, confirmation.md - baoyu-image-cards: 657→280, + gallery.md, confirmation.md - baoyu-post-to-wechat: 518→267, + multi-account.md, api-setup.md - baoyu-imagine: 500→230, + providers/, usage-examples.md Also un-deprecate baoyu-image-gen (drop stub warning) so it stays functional alongside baoyu-imagine, and update CLAUDE.md to reflect that both superseded skills are kept in sync rather than stubbed. * refactor: slim four medium SKILL.md files into references/ Continue the P2 pattern on the next tier of skills — move option catalogs, per-provider/adapter detail, and repeated EXTEND.md path boilerplate into their own references so SKILL.md stays focused on the decision flow. - baoyu-comic: 380→297 (art/tone/preset tables → auto-selection.md; Step 7 expanded detail → workflow.md) - baoyu-infographic: 312→207 (layouts/styles/combinations/keywords → gallery.md; ASCII box tables → markdown tables) - baoyu-format-markdown: 376→296 (title + summary generation → title-summary.md; ASCII box tables → markdown tables) - baoyu-url-to-markdown: 334→169 (quality gate + recovery → quality-gate.md; adapters + media download → adapters.md) * chore: sync deprecated skills with their replacements Per project policy, baoyu-xhs-images and baoyu-image-gen are kept functional alongside the active skills they were superseded by. Sync their SKILL.md bodies and references/ to the slimmed baoyu-image-cards and baoyu-imagine versions respectively, so cross-cutting fixes stay consistent. Only the frontmatter (name, description, version, homepage) differs — content is identical. - baoyu-xhs-images: 657→281 (synced with baoyu-image-cards + new confirmation.md, gallery.md) - baoyu-image-gen: 408→231 (synced with baoyu-imagine + new providers/, usage-examples.md) * refactor: collapse EXTEND.md boilerplate into priority tables Replace the dual bash/powershell existence-check blocks and ASCII box art with a single markdown priority table across nine SKILL.md files. The runtime-neutral phrasing removes shell-specific snippets without losing the priority semantics. * fix: address refactor-skills branch review findings - image-gen: restore EXTEND.md paths to baoyu-image-gen (were pointing at baoyu-imagine) and mark descriptions of both deprecated skills as [Deprecated]. - xhs-images: sync neon/warm palettes with image-cards to add the "do not render color names/hex as visible text" safety sentence. - infographic: restore Layout Gallery (21), Style Gallery (21), Recommended Combinations, and Keyword Shortcuts inline (previous refactor split them out but SKILL.md still depended on them), and add the missing references/config/first-time-setup.md + preferences-schema.md. - image-cards / xhs-images / slide-deck / format-markdown: restore the sections that got over-slimmed into references/ (galleries, presets, dimensions, auto-selection, style x layout matrix, title/summary flow) and drop the now-empty shell files. - docs/image-generation-tools.md: note that backend skills themselves (baoyu-imagine, baoyu-image-gen, baoyu-danger-gemini-web) are exempt from the ## Image Generation Tools section requirement. * feat(image-gen): sync Z.AI GLM-Image provider from baoyu-imagine Add Z.AI as a full provider in the deprecated baoyu-image-gen skill so it stays in sync with baoyu-imagine's provider list. - new scripts/providers/zai.ts + zai.test.ts (verbatim port; test factory trimmed to match image-gen's CliArgs shape). - types.ts: "zai" added to Provider union and default_model. - main.ts: rate-limit defaults, provider help text, env var help, --provider validation, loadProviderModule, detectProvider auto-detect chain, getModelForProvider, YAML parser allow-lists. - references/config: Q2e Z.AI model question + zai slot in the preferences schema and batch.provider_limits. Scope is intentionally limited to the Z.AI chain; unrelated drift between image-gen and imagine (OpenAI image-API dialect, aspectRatioSource, imageSizeSource) is left alone. * docs: align inline-convention wording and note backend-skill exemption - docs/user-input-tools.md: fix stale "links here" wording so it matches the inline convention already enforced everywhere else. - CLAUDE.md §Image Generation Tools: inline the backend-skill exemption so readers don't need to cross-reference docs/image-generation-tools.md.
173 lines
8.3 KiB
Markdown
173 lines
8.3 KiB
Markdown
---
|
|
name: baoyu-url-to-markdown
|
|
description: Fetch any URL and convert to markdown using baoyu-fetch CLI (Chrome CDP with site-specific adapters). Built-in adapters for X/Twitter, YouTube transcripts, Hacker News threads, and generic pages via Defuddle. Handles login/CAPTCHA via interaction wait modes. Use when user wants to save a webpage as markdown.
|
|
version: 1.60.0
|
|
metadata:
|
|
openclaw:
|
|
homepage: https://github.com/JimLiu/baoyu-skills#baoyu-url-to-markdown
|
|
requires:
|
|
anyBins:
|
|
- bun
|
|
- npx
|
|
---
|
|
|
|
# URL to Markdown
|
|
|
|
Fetches any URL via `baoyu-fetch` CLI (Chrome CDP + site-specific adapters) and converts it to clean markdown.
|
|
|
|
## User Input Tools
|
|
|
|
When this skill prompts the user, follow this tool-selection rule (priority order):
|
|
|
|
1. **Prefer built-in user-input tools** exposed by the current agent runtime — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.
|
|
2. **Fallback**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
|
|
3. **Batching**: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.
|
|
|
|
Concrete `AskUserQuestion` references below are examples — substitute the local equivalent in other runtimes.
|
|
|
|
## CLI Setup
|
|
|
|
**Important**: The CLI is provided by the npm package dependency `baoyu-fetch`.
|
|
|
|
**Agent Execution Instructions**:
|
|
1. Determine this SKILL.md file's directory path as `{baseDir}`
|
|
2. Resolve `${BUN}` runtime: if `bun` installed → `bun`; else suggest installing Bun
|
|
3. If `{baseDir}/scripts/node_modules/.bin/baoyu-fetch` does not exist, run `${BUN} install --cwd {baseDir}/scripts`
|
|
4. `${READER}` = `{baseDir}/scripts/node_modules/.bin/baoyu-fetch`
|
|
5. Replace all `${READER}` in this document with the resolved value
|
|
|
|
## Preferences (EXTEND.md)
|
|
|
|
Check EXTEND.md in priority order — the first one found wins:
|
|
|
|
| Priority | Path | Scope |
|
|
|----------|------|-------|
|
|
| 1 | `.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | Project |
|
|
| 2 | `${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | XDG |
|
|
| 3 | `$HOME/.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | User home |
|
|
|
|
| Result | Action |
|
|
|--------|--------|
|
|
| Found | Read, parse, apply settings |
|
|
| Not found | **MUST** run first-time setup (see below) — do NOT silently create defaults |
|
|
|
|
**EXTEND.md supports**: download media by default, default output directory.
|
|
|
|
### First-Time Setup ⛔ BLOCKING
|
|
|
|
When EXTEND.md is not found, you **MUST** use `AskUserQuestion` to gather preferences before creating EXTEND.md. **NEVER** create EXTEND.md with silent defaults. Generation is BLOCKED until setup completes. Batch all three questions into a single call:
|
|
|
|
- **Q1 — Media** (header "Media"): "How to handle images and videos in pages?"
|
|
- "Ask each time (Recommended)" — Prompt after each save
|
|
- "Always download" — Download to local `imgs/` and `videos/`
|
|
- "Never download" — Keep remote URLs
|
|
- **Q2 — Output** (header "Output"): "Default output directory?"
|
|
- "url-to-markdown (Recommended)" — Save to `./url-to-markdown/{domain}/{slug}.md`
|
|
- User may pick "Other" and type a custom path
|
|
- **Q3 — Save** (header "Save"): "Where to save preferences?"
|
|
- "User (Recommended)" — `~/.baoyu-skills/` (all projects)
|
|
- "Project" — `.baoyu-skills/` (this project only)
|
|
|
|
After answers, write EXTEND.md, confirm "Preferences saved to [path]", then continue.
|
|
|
|
Full template: [references/config/first-time-setup.md](references/config/first-time-setup.md).
|
|
|
|
### Supported Keys
|
|
|
|
| Key | Default | Values | Description |
|
|
|-----|---------|--------|-------------|
|
|
| `download_media` | `ask` | `ask` / `1` / `0` | `ask` = prompt each time, `1` = always, `0` = never |
|
|
| `default_output_dir` | empty | path or empty | Default output directory (empty = `./url-to-markdown/`) |
|
|
|
|
**EXTEND.md → CLI mapping**:
|
|
|
|
| EXTEND.md key | CLI argument | Notes |
|
|
|---------------|-------------|-------|
|
|
| `download_media: 1` | `--download-media` | Requires `--output` to be set |
|
|
| `default_output_dir: ./posts/` | Agent constructs `--output ./posts/{domain}/{slug}.md` | Agent generates path, not a direct flag |
|
|
|
|
**Value priority**: CLI arguments → EXTEND.md → skill defaults.
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
# Default: headless capture, markdown to stdout
|
|
${READER} <url>
|
|
|
|
# Save to file
|
|
${READER} <url> --output article.md
|
|
|
|
# Save with media download
|
|
${READER} <url> --output article.md --download-media
|
|
|
|
# Wait for interaction (login/CAPTCHA) — auto-detect and continue
|
|
${READER} <url> --wait-for interaction --output article.md
|
|
|
|
# Wait for interaction — manual control (Enter to continue)
|
|
${READER} <url> --wait-for force --output article.md
|
|
|
|
# JSON output
|
|
${READER} <url> --format json --output article.json
|
|
|
|
# Force specific adapter
|
|
${READER} <url> --adapter youtube --output transcript.md
|
|
```
|
|
|
|
## Options
|
|
|
|
| Option | Description |
|
|
|--------|-------------|
|
|
| `<url>` | URL to fetch |
|
|
| `--output <path>` | Output file path (default: stdout) |
|
|
| `--format <type>` | Output format: `markdown` (default) or `json` |
|
|
| `--json` | Shorthand for `--format json` |
|
|
| `--adapter <name>` | Force adapter: `x`, `youtube`, `hn`, or `generic` (default: auto-detect) |
|
|
| `--headless` | Force headless Chrome (no visible window) |
|
|
| `--wait-for <mode>` | Interaction wait mode: `none` (default), `interaction`, or `force` |
|
|
| `--wait-for-interaction` | Alias for `--wait-for interaction` |
|
|
| `--wait-for-login` | Alias for `--wait-for interaction` |
|
|
| `--timeout <ms>` | Page load timeout (default: 30000) |
|
|
| `--interaction-timeout <ms>` | Login/CAPTCHA wait timeout (default: 600000 = 10 min) |
|
|
| `--interaction-poll-interval <ms>` | Poll interval for interaction checks (default: 1500) |
|
|
| `--download-media` | Download images/videos to local `imgs/` and `videos/`, rewrite markdown links. Requires `--output` |
|
|
| `--media-dir <dir>` | Base directory for downloaded media (default: same as `--output` directory) |
|
|
| `--cdp-url <url>` | Reuse existing Chrome DevTools Protocol endpoint |
|
|
| `--browser-path <path>` | Custom Chrome/Chromium binary path |
|
|
| `--chrome-profile-dir <path>` | Chrome user data directory (default: `BAOYU_CHROME_PROFILE_DIR` env or `./baoyu-skills/chrome-profile`) |
|
|
| `--debug-dir <dir>` | Write debug artifacts (document.json, markdown.md, page.html, network.json) |
|
|
|
|
## Agent Quality Gate
|
|
|
|
**CRITICAL**: treat default headless capture as provisional. Some sites render differently in headless mode and can silently return low-quality content without failing the CLI.
|
|
|
|
After every headless run, inspect the saved markdown. See [references/quality-gate.md](references/quality-gate.md) for the full checklist, recovery workflow, and capture-mode table. Read it whenever a run looks suspicious or the user asks about login/CAPTCHA handling.
|
|
|
|
## Output Path Generation
|
|
|
|
The agent must construct the output file path — `baoyu-fetch` does not auto-generate paths.
|
|
|
|
**Algorithm**:
|
|
1. Determine base directory from EXTEND.md `default_output_dir` or default `./url-to-markdown/`
|
|
2. Extract domain from URL (e.g., `example.com`)
|
|
3. Generate slug from URL path or page title (kebab-case, 2-6 words)
|
|
4. Construct: `{base_dir}/{domain}/{slug}/{slug}.md` — each URL gets its own directory so media files stay isolated
|
|
5. Conflict resolution: append timestamp `{slug}-YYYYMMDD-HHMMSS/{slug}-YYYYMMDD-HHMMSS.md`
|
|
|
|
Pass the constructed path to `--output`. Media files (`--download-media`) are saved into subdirectories next to the markdown file, keeping each URL's assets self-contained.
|
|
|
|
## Adapters & Media
|
|
|
|
See [references/adapters.md](references/adapters.md) for the adapter catalog (X, YouTube, Hacker News, generic), per-adapter notes, the media download flow (`ask` / always / never), and the JSON output schema. Read it before answering adapter-specific questions or handling media prompts.
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `BAOYU_CHROME_PROFILE_DIR` | Chrome user data directory (can also use `--chrome-profile-dir`) |
|
|
|
|
**Troubleshooting**: Chrome not found → use `--browser-path`. Timeout → increase `--timeout`. Login/CAPTCHA → `--wait-for interaction`. Debug → `--debug-dir` to inspect captured HTML and network logs.
|
|
|
|
## Extension Support
|
|
|
|
Custom configurations via EXTEND.md. See **Preferences** section above for paths and supported keys.
|