Compare commits

...

9 Commits

Author SHA1 Message Date
Jim Liu 宝玉 76ddcf134e chore: release v1.2.0 2026-01-18 20:42:40 -06:00
Jim Liu 宝玉 5a0feba96e docs: update installation instructions and add available plugins section 2026-01-18 20:28:36 -06:00
Jim Liu 宝玉 97bc68efd8 chore: release v1.1.0 2026-01-18 20:11:49 -06:00
Jim Liu 宝玉 464edf0656 chore: release v1.0.1 2026-01-18 19:15:34 -06:00
Jim Liu 宝玉 43c2589a38 Refactor code structure for improved readability and maintainability 2026-01-18 18:18:54 -06:00
Jim Liu 宝玉 4c82884722 chore: release v1.0.0
- Add baoyu-danger-x-to-markdown skill to marketplace
- Update README files with new skill documentation
- Bump version to 1.0.0

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 18:00:58 -06:00
Jim Liu 宝玉 56d0485412 chore: rename baoyu-gemini-web to baoyu-danger-gemini-web 2026-01-18 17:58:12 -06:00
Jim Liu 宝玉 4998eaf8c2 feat: add baoyu-danger-x-to-markdown skill
Ignore generated x-to-markdown output directory.
2026-01-18 17:42:25 -06:00
Jim Liu 宝玉 5993b6969d chore: release v0.11.0 2026-01-18 09:53:48 -06:00
78 changed files with 4212 additions and 203 deletions
+21 -3
View File
@@ -6,16 +6,15 @@
},
"metadata": {
"description": "Skills shared by Baoyu for improving daily work efficiency",
"version": "0.10.0"
"version": "1.2.0"
},
"plugins": [
{
"name": "content-skills",
"description": "Skills shared by Baoyu for improving daily work efficiency",
"description": "Content generation and publishing skills",
"source": "./",
"strict": false,
"skills": [
"./skills/baoyu-gemini-web",
"./skills/baoyu-xhs-images",
"./skills/baoyu-post-to-x",
"./skills/baoyu-post-to-wechat",
@@ -24,6 +23,25 @@
"./skills/baoyu-slide-deck",
"./skills/baoyu-comic"
]
},
{
"name": "ai-generation-skills",
"description": "AI-powered generation backends",
"source": "./",
"strict": false,
"skills": [
"./skills/baoyu-danger-gemini-web"
]
},
{
"name": "utility-skills",
"description": "Utility tools for content processing",
"source": "./",
"strict": false,
"skills": [
"./skills/baoyu-danger-x-to-markdown",
"./skills/baoyu-compress-image"
]
}
]
}
+1
View File
@@ -143,3 +143,4 @@ tests-data/
# Skill extensions (user customization)
.baoyu-skills/
x-to-markdown/
+50 -6
View File
@@ -2,6 +2,50 @@
English | [中文](./CHANGELOG.zh.md)
## 1.2.0 - 2026-01-18
### Features
- Session-independent output directories: each generation session creates a new directory (`<skill-suffix>/<topic-slug>/`), even for the same source file. Conflicts resolved by appending timestamp.
- Multi-source file support: source files now saved as `source-{slug}.{ext}`, supporting multiple inputs (text, images, files from conversation).
### Documentation
- `CLAUDE.md`: updates Output Path Convention with new session-independent directory structure and multi-source file naming.
- Multiple skills: updates file management sections to reflect new directory and source file conventions.
- `baoyu-slide-deck`, `baoyu-article-illustrator`, `baoyu-cover-image`, `baoyu-xhs-images`, `baoyu-comic`
## 1.1.0 - 2026-01-18
### Features
- `baoyu-compress-image`: new utility skill for cross-platform image compression. Converts to WebP by default with PNG-to-PNG support. Uses system tools (sips, cwebp, ImageMagick) with Sharp fallback.
### Refactor
- Marketplace structure: reorganizes plugins into three categories—`content-skills`, `ai-generation-skills`, and `utility-skills`—for better organization.
### Documentation
- `CLAUDE.md`, `README.md`, `README.zh.md`: updates skill architecture documentation to reflect the new three-category structure.
## 1.0.1 - 2026-01-18
### Refactor
- Code structure improvements for better readability and maintainability.
- `baoyu-slide-deck`: unified style reference file formats.
### Other
- Screenshots: converted from PNG to WebP format for smaller file sizes; added screenshots for new styles.
## 1.0.0 - 2026-01-18
### Features
- `baoyu-danger-x-to-markdown`: new skill to convert X/Twitter posts and threads to Markdown format.
### Breaking Changes
- `baoyu-gemini-web` renamed to `baoyu-danger-gemini-web` to indicate potential risks of using reverse-engineered APIs.
## 0.11.0 - 2026-01-18
### Features
- `baoyu-danger-gemini-web`: adds disclaimer consent check flow—requires user acceptance before first use, with persistent consent storage per platform.
## 0.10.0 - 2026-01-18
### Features
@@ -21,12 +65,12 @@ English | [中文](./CHANGELOG.zh.md)
## 0.8.2 - 2026-01-17
### Refactor
- `baoyu-gemini-web`: reorganizes script architecture—moves modular files into `gemini-webapi/` subdirectory and updates SKILL.md with `${SKILL_DIR}` path references.
- `baoyu-danger-gemini-web`: reorganizes script architecture—moves modular files into `gemini-webapi/` subdirectory and updates SKILL.md with `${SKILL_DIR}` path references.
## 0.8.1 - 2026-01-17
### Refactor
- `baoyu-gemini-web`: refactors script architecture—consolidates 10 separate files into a structured `gemini-webapi/` module (TypeScript port of gemini_webapi Python library).
- `baoyu-danger-gemini-web`: refactors script architecture—consolidates 10 separate files into a structured `gemini-webapi/` module (TypeScript port of gemini_webapi Python library).
## 0.8.0 - 2026-01-17
@@ -56,7 +100,7 @@ English | [中文](./CHANGELOG.zh.md)
- `baoyu-slide-deck`: adds `scripts/merge-to-pdf.ts` to export generated slides into a single PDF; docs updated with pptx/pdf outputs.
- `baoyu-comic`: adds `scripts/merge-to-pdf.ts` to merge cover/pages into a PDF; docs clarify character reference handling (image vs text).
- Docs conventions: adds a “Script Directory” template to `CLAUDE.md`; aligns `baoyu-gemini-web` / `baoyu-slide-deck` / `baoyu-comic` docs to use `${SKILL_DIR}` in commands so agents can run scripts from any install location.
- Docs conventions: adds a “Script Directory” template to `CLAUDE.md`; aligns `baoyu-danger-gemini-web` / `baoyu-slide-deck` / `baoyu-comic` docs to use `${SKILL_DIR}` in commands so agents can run scripts from any install location.
## 0.6.0 - 2026-01-17
@@ -71,8 +115,8 @@ English | [中文](./CHANGELOG.zh.md)
## 0.5.2 - 2026-01-16
- `baoyu-gemini-web`: adds `--sessionId` (local persisted sessions, plus `--list-sessions`) for multi-turn conversations and consistent multi-image generation.
- `baoyu-gemini-web`: adds `--reference/--ref` for reference images (vision input), plus stronger timeout handling and cookie refresh recovery.
- `baoyu-danger-gemini-web`: adds `--sessionId` (local persisted sessions, plus `--list-sessions`) for multi-turn conversations and consistent multi-image generation.
- `baoyu-danger-gemini-web`: adds `--reference/--ref` for reference images (vision input), plus stronger timeout handling and cookie refresh recovery.
- Docs: `baoyu-xhs-images` / `baoyu-slide-deck` / `baoyu-comic` document session usage (reuse one `sessionId` per set) to improve visual consistency.
## 0.5.1 - 2026-01-16
@@ -88,7 +132,7 @@ English | [中文](./CHANGELOG.zh.md)
## 0.4.2 - 2026-01-15
- `baoyu-gemini-web`: updates description to clarify it as the image-generation backend for other skills (e.g. `cover-image`, `xhs-images`, `article-illustrator`).
- `baoyu-danger-gemini-web`: updates description to clarify it as the image-generation backend for other skills (e.g. `cover-image`, `xhs-images`, `article-illustrator`).
## 0.4.1 - 2026-01-15
+50 -6
View File
@@ -2,6 +2,50 @@
[English](./CHANGELOG.md) | 中文
## 1.2.0 - 2026-01-18
### 新功能
- Session 独立输出目录:每次生成创建独立目录(`<skill-suffix>/<topic-slug>/`),即使是同一源文件也会新建目录。目录冲突时追加时间戳。
- 多源文件支持:源文件现以 `source-{slug}.{ext}` 命名,支持多个输入(文本、图片、会话中的文件)。
### 文档
- `CLAUDE.md`:更新 Output Path Convention,采用新的 session 独立目录结构和多源文件命名规范。
- 多个技能:更新文件管理部分,反映新的目录和源文件规范。
- `baoyu-slide-deck``baoyu-article-illustrator``baoyu-cover-image``baoyu-xhs-images``baoyu-comic`
## 1.1.0 - 2026-01-18
### 新功能
- `baoyu-compress-image`:新增跨平台图片压缩技能。默认转换为 WebP 格式,支持 PNG 转 PNG。自动选择系统工具(sips、cwebp、ImageMagick),Sharp 作为兜底方案。
### 重构
- Marketplace 结构重组:将插件分为三大类——`content-skills`(内容技能)、`ai-generation-skills`AI 生成技能)和 `utility-skills`(工具技能),便于管理和发现。
### 文档
- `CLAUDE.md``README.md``README.zh.md`:更新技能架构文档,反映新的三类分组结构。
## 1.0.1 - 2026-01-18
### 重构
- 代码结构优化,提升可读性和可维护性。
- `baoyu-slide-deck`:统一风格参考文件格式。
### 其他
- 截图:从 PNG 转换为 WebP 格式,减小文件体积;新增新风格的截图。
## 1.0.0 - 2026-01-18
### 新功能
- `baoyu-danger-x-to-markdown`:新增技能,将 X/Twitter 帖子和线程转换为 Markdown 格式。
### 破坏性变更
- `baoyu-gemini-web` 重命名为 `baoyu-danger-gemini-web`,以提示使用逆向工程 API 的潜在风险。
## 0.11.0 - 2026-01-18
### 新功能
- `baoyu-danger-gemini-web`:新增 Disclaimer 同意检查流程——首次使用前需用户确认接受,同意状态按平台持久化存储。
## 0.10.0 - 2026-01-18
### 新功能
@@ -21,12 +65,12 @@
## 0.8.2 - 2026-01-17
### 重构
- `baoyu-gemini-web`:重组脚本架构——将模块文件移至 `gemini-webapi/` 子目录,并更新 SKILL.md 使用 `${SKILL_DIR}` 路径引用。
- `baoyu-danger-gemini-web`:重组脚本架构——将模块文件移至 `gemini-webapi/` 子目录,并更新 SKILL.md 使用 `${SKILL_DIR}` 路径引用。
## 0.8.1 - 2026-01-17
### 重构
- `baoyu-gemini-web`:重构脚本架构——将 10 个分散的脚本文件整合为结构化的 `gemini-webapi/` 模块(gemini_webapi Python 库的 TypeScript 移植版)。
- `baoyu-danger-gemini-web`:重构脚本架构——将 10 个分散的脚本文件整合为结构化的 `gemini-webapi/` 模块(gemini_webapi Python 库的 TypeScript 移植版)。
## 0.8.0 - 2026-01-17
@@ -56,7 +100,7 @@
- `baoyu-slide-deck`:新增 `scripts/merge-to-pdf.ts`,可将生成的 slide 图片一键合并为 PDF;文档补充导出步骤与产物命名(pptx/pdf)。
- `baoyu-comic`:新增 `scripts/merge-to-pdf.ts`,将封面/分页图片合并为 PDF;补充角色参考(图片/文本)处理说明。
- 文档规范:在 `CLAUDE.md` 中补充“Script Directory”模板;`baoyu-gemini-web` / `baoyu-slide-deck` / `baoyu-comic` 文档统一用 `${SKILL_DIR}` 引用脚本路径,方便 agent 在任意安装目录运行。
- 文档规范:在 `CLAUDE.md` 中补充“Script Directory”模板;`baoyu-danger-gemini-web` / `baoyu-slide-deck` / `baoyu-comic` 文档统一用 `${SKILL_DIR}` 引用脚本路径,方便 agent 在任意安装目录运行。
## 0.6.0 - 2026-01-17
@@ -71,8 +115,8 @@
## 0.5.2 - 2026-01-16
- `baoyu-gemini-web`:新增 `--sessionId`(本地持久化会话,支持 `--list-sessions`),用于多轮对话/多图生成保持上下文一致。
- `baoyu-gemini-web`:新增 `--reference/--ref` 传入参考图片(vision 输入),并增强超时与 cookie 失效自动恢复逻辑。
- `baoyu-danger-gemini-web`:新增 `--sessionId`(本地持久化会话,支持 `--list-sessions`),用于多轮对话/多图生成保持上下文一致。
- `baoyu-danger-gemini-web`:新增 `--reference/--ref` 传入参考图片(vision 输入),并增强超时与 cookie 失效自动恢复逻辑。
- `baoyu-xhs-images` / `baoyu-slide-deck` / `baoyu-comic`:文档补充 session 约定(整套图使用同一 `sessionId`,增强风格一致性)。
## 0.5.1 - 2026-01-16
@@ -88,7 +132,7 @@
## 0.4.2 - 2026-01-15
- `baoyu-gemini-web`:描述信息更新,明确其作为 `cover-image` / `xhs-images` / `article-illustrator` 等技能的图片生成后端。
- `baoyu-danger-gemini-web`:描述信息更新,明确其作为 `cover-image` / `xhs-images` / `article-illustrator` 等技能的图片生成后端。
## 0.4.1 - 2026-01-15
+72 -30
View File
@@ -8,17 +8,34 @@ Claude Code marketplace plugin providing AI-powered content generation skills. S
## Architecture
Skills are organized into three plugin categories in `marketplace.json`:
```
skills/
├── baoyu-gemini-web/ # Core: Gemini API wrapper (text + image gen)
├── baoyu-xhs-images/ # Xiaohongshu infographic series (1-10 images)
├── baoyu-cover-image/ # Article cover images (2.35:1 aspect)
├── baoyu-slide-deck/ # Presentation slides with outlines
├── baoyu-article-illustrator/ # Smart illustration placement
├── baoyu-post-to-x/ # X/Twitter posting automation
── baoyu-post-to-wechat/ # WeChat Official Account posting
├── [content-skills] # Content generation and publishing
├── baoyu-xhs-images/ # Xiaohongshu infographic series (1-10 images)
├── baoyu-cover-image/ # Article cover images (2.35:1 aspect)
├── baoyu-slide-deck/ # Presentation slides with outlines
├── baoyu-article-illustrator/ # Smart illustration placement
├── baoyu-comic/ # Knowledge comics (Logicomix/Ohmsha style)
│ ├── baoyu-post-to-x/ # X/Twitter posting automation
│ └── baoyu-post-to-wechat/ # WeChat Official Account posting
├── [ai-generation-skills] # AI-powered generation backends
│ └── baoyu-danger-gemini-web/ # Gemini API wrapper (text + image gen)
└── [utility-skills] # Utility tools for content processing
├── baoyu-danger-x-to-markdown/ # X/Twitter content to markdown
└── baoyu-compress-image/ # Image compression
```
**Plugin Categories**:
| Category | Description |
|----------|-------------|
| `content-skills` | Skills that generate or publish content (images, slides, comics, posts) |
| `ai-generation-skills` | Backend skills providing AI generation capabilities |
| `utility-skills` | Helper tools for content processing (conversion, compression) |
Each skill contains:
- `SKILL.md` - YAML front matter (name, description) + documentation
- `scripts/` - TypeScript implementations
@@ -35,24 +52,24 @@ npx -y bun skills/<skill>/scripts/main.ts [options]
Examples:
```bash
# Text generation
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "Hello"
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts "Hello"
# Image generation
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --prompt "A cat" --image cat.png
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts --prompt "A cat" --image cat.png
# From prompt files
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
```
## Key Dependencies
- **Bun**: TypeScript runtime (via `npx -y bun`)
- **Chrome**: Required for `baoyu-gemini-web` auth and `baoyu-post-to-x` automation
- **Chrome**: Required for `baoyu-danger-gemini-web` auth and `baoyu-post-to-x` automation
- **No npm packages**: Self-contained TypeScript, no external dependencies
## Authentication
`baoyu-gemini-web` uses browser cookies for Google auth:
`baoyu-danger-gemini-web` uses browser cookies for Google auth:
- First run opens Chrome for login
- Cookies cached in data directory
- Force refresh: `--login` flag
@@ -70,9 +87,28 @@ npx -y bun skills/baoyu-gemini-web/scripts/main.ts --promptfiles system.md conte
- SKILL.md `name` field: `baoyu-<name>`
2. Add TypeScript in `skills/baoyu-<name>/scripts/`
3. Add prompt templates in `skills/baoyu-<name>/prompts/` if needed
4. Register in `marketplace.json` plugins[0].skills array as `./skills/baoyu-<name>`
4. **Choose the appropriate category** and register in `marketplace.json`:
- `content-skills`: For content generation/publishing (images, slides, posts)
- `ai-generation-skills`: For AI backend capabilities
- `utility-skills`: For helper tools (conversion, compression)
- If none fit, create a new category with descriptive name
5. **Add Script Directory section** to SKILL.md (see template below)
### Choosing a Category
| If your skill... | Use category |
|------------------|--------------|
| Generates visual content (images, slides, comics) | `content-skills` |
| Publishes to platforms (X, WeChat, etc.) | `content-skills` |
| Provides AI generation backend | `ai-generation-skills` |
| Converts or processes content | `utility-skills` |
| Compresses or optimizes files | `utility-skills` |
**Creating a new category**: If the skill doesn't fit existing categories, add a new plugin object to `marketplace.json` with:
- `name`: Descriptive kebab-case name (e.g., `analytics-skills`)
- `description`: Brief description of the category
- `skills`: Array with the skill path
### Script Directory Template
Every SKILL.md with scripts MUST include this section after Usage:
@@ -121,7 +157,7 @@ Use this template when implementing image generation in skills:
### Step N: Generate Images
**Skill Selection**:
1. Check available image generation skills (e.g., `baoyu-gemini-web`)
1. Check available image generation skills (e.g., `baoyu-danger-gemini-web`)
2. Read selected skill's SKILL.md for parameter reference
3. If multiple skills available, ask user to choose
@@ -137,27 +173,33 @@ Use this template when implementing image generation in skills:
### Output Path Convention
Generated images from the same skill and source file MUST be grouped together:
Each session creates an independent directory. Even the same source file generates a new directory per session.
**With source file** (e.g., `/path/to/project/content/my-article.md`):
**Output Directory**:
```
/path/to/project/content/my-article/<skill-suffix>/
<skill-suffix>/<topic-slug>/
```
- Remove file extension from source filename
- Use skill name suffix (e.g., `xhs-images`, `cover-image`, `slide-deck`)
- Example: source `/tests-data/anthropic-economic-index.md` + skill `baoyu-xhs-images``/tests-data/anthropic-economic-index/xhs-images/`
- `<skill-suffix>`: Skill name suffix (e.g., `xhs-images`, `cover-image`, `slide-deck`, `comic`)
- `<topic-slug>`: Generated from content topic (2-4 words, kebab-case)
- Example: `xhs-images/ai-future-trends/`
**Without source file**:
```
./<skill-suffix>/<source-slug>/
```
- Place under current project directory
- Use descriptive slug for the content
**Slug Generation**:
1. Extract main topic from content (2-4 words, kebab-case)
2. Example: "Introduction to Machine Learning" → `intro-machine-learning`
**Directory Backup**:
- If output directory already exists, rename existing directory with timestamp
- Format: `<dirname>-backup-YYYYMMDD-HHMMSS`
- Example: `xhs-images``xhs-images-backup-20260117-143052`
**Conflict Resolution**:
If `<skill-suffix>/<topic-slug>/` already exists:
- Append timestamp: `<topic-slug>-YYYYMMDD-HHMMSS`
- Example: `ai-future` exists → `ai-future-20260118-143052`
**Source Files**:
- Copy all sources to `<skill-suffix>/<topic-slug>/` with naming: `source-{slug}.{ext}`
- Multiple sources supported: text, images, files from conversation
- Examples:
- `source-article.md` (main text content)
- `source-reference.png` (image from conversation)
- `source-data.csv` (additional file)
- Original source files remain unchanged
### Image Naming Convention
+127 -22
View File
@@ -31,13 +31,16 @@ Run the following command in Claude Code:
1. Select **Browse and install plugins**
2. Select **baoyu-skills**
3. Select **content-skills**
3. Select the plugin(s) you want to install
4. Select **Install now**
**Option 2: Direct Install**
```bash
# Install specific plugin
/plugin install content-skills@baoyu-skills
/plugin install ai-generation-skills@baoyu-skills
/plugin install utility-skills@baoyu-skills
```
**Option 3: Ask the Agent**
@@ -46,6 +49,14 @@ Simply tell Claude Code:
> Please install Skills from github.com/JimLiu/baoyu-skills
### Available Plugins
| Plugin | Description | Skills |
|--------|-------------|--------|
| **content-skills** | Content generation and publishing | [xhs-images](#baoyu-xhs-images), [cover-image](#baoyu-cover-image), [slide-deck](#baoyu-slide-deck), [comic](#baoyu-comic), [article-illustrator](#baoyu-article-illustrator), [post-to-x](#baoyu-post-to-x), [post-to-wechat](#baoyu-post-to-wechat) |
| **ai-generation-skills** | AI-powered generation backends | [danger-gemini-web](#baoyu-danger-gemini-web) |
| **utility-skills** | Utility tools for content processing | [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image) |
## Update Skills
To update skills to the latest version:
@@ -61,25 +72,13 @@ You can also **Enable auto-update** to get the latest versions automatically.
## Available Skills
### baoyu-gemini-web
Skills are organized into three categories:
Interacts with Gemini Web to generate text and images.
### Content Skills
**Text Generation:**
Content generation and publishing skills.
```bash
/baoyu-gemini-web "Hello, Gemini"
/baoyu-gemini-web --prompt "Explain quantum computing"
```
**Image Generation:**
```bash
/baoyu-gemini-web --prompt "A cute cat" --image cat.png
/baoyu-gemini-web --promptfiles system.md content.md --image out.png
```
### baoyu-xhs-images
#### baoyu-xhs-images
Xiaohongshu (RedNote) infographic series generator. Breaks down content into 1-10 cartoon-style infographics with **Style × Layout** two-dimensional system.
@@ -112,7 +111,7 @@ Xiaohongshu (RedNote) infographic series generator. Breaks down content into 1-1
| `comparison` | 2 sides | Before/after, pros/cons |
| `flow` | 3-6 steps | Processes, timelines |
### baoyu-cover-image
#### baoyu-cover-image
Generate hand-drawn style cover images for articles with multiple style options.
@@ -130,7 +129,7 @@ Generate hand-drawn style cover images for articles with multiple style options.
Available styles: `elegant` (default), `tech`, `warm`, `bold`, `minimal`, `playful`, `nature`, `retro`
### baoyu-slide-deck
#### baoyu-slide-deck
Generate professional slide deck images from content. Creates comprehensive outlines with style instructions, then generates individual slide images.
@@ -169,9 +168,24 @@ Generate professional slide deck images from content. Creates comprehensive outl
| `vintage` | Aged-paper aesthetic, historical document styling | Historical, heritage, biography |
| `watercolor` | Soft hand-painted textures, natural warmth | Lifestyle, wellness, travel |
**Style Previews**:
| | | |
|:---:|:---:|:---:|
| ![blueprint](./screenshots/slide-deck-styles/blueprint.webp) | ![bold-editorial](./screenshots/slide-deck-styles/bold-editorial.webp) | ![corporate](./screenshots/slide-deck-styles/corporate.webp) |
| blueprint | bold-editorial | corporate |
| ![dark-atmospheric](./screenshots/slide-deck-styles/dark-atmospheric.webp) | ![editorial-infographic](./screenshots/slide-deck-styles/editorial-infographic.webp) | ![fantasy-animation](./screenshots/slide-deck-styles/fantasy-animation.webp) |
| dark-atmospheric | editorial-infographic | fantasy-animation |
| ![intuition-machine](./screenshots/slide-deck-styles/intuition-machine.webp) | ![minimal](./screenshots/slide-deck-styles/minimal.webp) | ![notion](./screenshots/slide-deck-styles/notion.webp) |
| intuition-machine | minimal | notion |
| ![pixel-art](./screenshots/slide-deck-styles/pixel-art.webp) | ![scientific](./screenshots/slide-deck-styles/scientific.webp) | ![sketch-notes](./screenshots/slide-deck-styles/sketch-notes.webp) |
| pixel-art | scientific | sketch-notes |
| ![vector-illustration](./screenshots/slide-deck-styles/vector-illustration.webp) | ![vintage](./screenshots/slide-deck-styles/vintage.webp) | ![watercolor](./screenshots/slide-deck-styles/watercolor.webp) |
| vector-illustration | vintage | watercolor |
After generation, slides are automatically merged into a `.pptx` file for easy sharing.
### baoyu-comic
#### baoyu-comic
Knowledge comic creator supporting multiple styles (Logicomix/Ligne Claire, Ohmsha manga guide). Creates original educational comics with detailed panel layouts and sequential image generation.
@@ -228,7 +242,30 @@ Knowledge comic creator supporting multiple styles (Logicomix/Ligne Claire, Ohms
| `mixed` | 3-7 varies | Complex narratives, emotional arcs |
| `webtoon` | 3-5 vertical | Ohmsha tutorials, mobile reading |
### baoyu-post-to-wechat
#### baoyu-article-illustrator
Smart article illustration skill. Analyzes article content and generates illustrations at positions requiring visual aids.
```bash
/baoyu-article-illustrator path/to/article.md
```
#### baoyu-post-to-x
Post content and articles to X (Twitter). Supports regular posts with images and X Articles (long-form Markdown). Uses real Chrome with CDP to bypass anti-automation.
```bash
# Post with text
/baoyu-post-to-x "Hello from Claude Code!"
# Post with images
/baoyu-post-to-x "Check this out" --image photo.png
# Post X Article
/baoyu-post-to-x --article path/to/article.md
```
#### baoyu-post-to-wechat
Post content to WeChat Official Account (微信公众号). Two modes available:
@@ -250,6 +287,63 @@ Post content to WeChat Official Account (微信公众号). Two modes available:
Prerequisites: Google Chrome installed. First run requires QR code login (session preserved).
### AI Generation Skills
AI-powered generation backends.
#### baoyu-danger-gemini-web
Interacts with Gemini Web to generate text and images.
**Text Generation:**
```bash
/baoyu-danger-gemini-web "Hello, Gemini"
/baoyu-danger-gemini-web --prompt "Explain quantum computing"
```
**Image Generation:**
```bash
/baoyu-danger-gemini-web --prompt "A cute cat" --image cat.png
/baoyu-danger-gemini-web --promptfiles system.md content.md --image out.png
```
### Utility Skills
Utility tools for content processing.
#### baoyu-danger-x-to-markdown
Converts X (Twitter) content to markdown format. Supports tweet threads and X Articles.
```bash
# Convert tweet to markdown
/baoyu-danger-x-to-markdown https://x.com/username/status/123456
# Save to specific file
/baoyu-danger-x-to-markdown https://x.com/username/status/123456 -o output.md
# JSON output
/baoyu-danger-x-to-markdown https://x.com/username/status/123456 --json
```
**Supported URLs:**
- `https://x.com/<user>/status/<id>`
- `https://twitter.com/<user>/status/<id>`
- `https://x.com/i/article/<id>`
**Authentication:** Uses environment variables (`X_AUTH_TOKEN`, `X_CT0`) or Chrome login for cookie-based auth.
#### baoyu-compress-image
Compress images to reduce file size while maintaining quality.
```bash
/baoyu-compress-image path/to/image.png
/baoyu-compress-image path/to/images/ --quality 80
```
## Customization
All skills support customization via `EXTEND.md` files. Create an extension file to override default styles, add custom configurations, or define your own presets.
@@ -280,7 +374,7 @@ The extension content will be loaded before skill execution and override default
## Disclaimer
### baoyu-gemini-web
### baoyu-danger-gemini-web
This skill uses the Gemini Web API (reverse-engineered).
@@ -290,6 +384,17 @@ This skill uses the Gemini Web API (reverse-engineered).
- Cookies are cached for subsequent runs
- No guarantees on API stability or availability
### baoyu-danger-x-to-markdown
This skill uses a reverse-engineered X (Twitter) API.
**Warning:** This is NOT an official API. Use at your own risk.
- May break without notice if X changes their API
- Account restrictions possible if API usage detected
- First use requires consent acknowledgment
- Authentication via environment variables or Chrome login
## License
MIT
+127 -22
View File
@@ -31,13 +31,16 @@ npx add-skill jimliu/baoyu-skills
1. 选择 **Browse and install plugins**
2. 选择 **baoyu-skills**
3. 选择 **content-skills**
3. 选择要安装的插件
4. 选择 **Install now**
**方式二:直接安装**
```bash
# 安装指定插件
/plugin install content-skills@baoyu-skills
/plugin install ai-generation-skills@baoyu-skills
/plugin install utility-skills@baoyu-skills
```
**方式三:告诉 Agent**
@@ -46,6 +49,14 @@ npx add-skill jimliu/baoyu-skills
> 请帮我安装 github.com/JimLiu/baoyu-skills 中的 Skills
### 可用插件
| 插件 | 说明 | 包含技能 |
|------|------|----------|
| **content-skills** | 内容生成和发布 | [xhs-images](#baoyu-xhs-images), [cover-image](#baoyu-cover-image), [slide-deck](#baoyu-slide-deck), [comic](#baoyu-comic), [article-illustrator](#baoyu-article-illustrator), [post-to-x](#baoyu-post-to-x), [post-to-wechat](#baoyu-post-to-wechat) |
| **ai-generation-skills** | AI 生成后端 | [danger-gemini-web](#baoyu-danger-gemini-web) |
| **utility-skills** | 内容处理工具 | [danger-x-to-markdown](#baoyu-danger-x-to-markdown), [compress-image](#baoyu-compress-image) |
## 更新技能
更新技能到最新版本:
@@ -61,25 +72,13 @@ npx add-skill jimliu/baoyu-skills
## 可用技能
### baoyu-gemini-web
技能分为三大类:
与 Gemini Web 交互,生成文本和图片。
### 内容技能 (Content Skills)
**文本生成:**
内容生成和发布技能。
```bash
/baoyu-gemini-web "你好,Gemini"
/baoyu-gemini-web --prompt "解释量子计算"
```
**图片生成:**
```bash
/baoyu-gemini-web --prompt "一只可爱的猫" --image cat.png
/baoyu-gemini-web --promptfiles system.md content.md --image out.png
```
### baoyu-xhs-images
#### baoyu-xhs-images
小红书信息图系列生成器。将内容拆解为 1-10 张卡通风格信息图,支持 **风格 × 布局** 二维系统。
@@ -112,7 +111,7 @@ npx add-skill jimliu/baoyu-skills
| `comparison` | 双栏 | 对比、优劣 |
| `flow` | 3-6 步 | 流程、时间线 |
### baoyu-cover-image
#### baoyu-cover-image
为文章生成手绘风格封面图,支持多种风格选项。
@@ -130,7 +129,7 @@ npx add-skill jimliu/baoyu-skills
可用风格:`elegant`(默认)、`tech``warm``bold``minimal``playful``nature``retro`
### baoyu-slide-deck
#### baoyu-slide-deck
从内容生成专业的幻灯片图片。先创建包含样式说明的完整大纲,然后逐页生成幻灯片图片。
@@ -169,9 +168,24 @@ npx add-skill jimliu/baoyu-skills
| `vintage` | 做旧纸张美学,历史文档风格 | 历史、传记、人文 |
| `watercolor` | 柔和手绘水彩纹理,自然温暖 | 生活方式、健康、旅行 |
**风格预览**
| | | |
|:---:|:---:|:---:|
| ![blueprint](./screenshots/slide-deck-styles/blueprint.webp) | ![bold-editorial](./screenshots/slide-deck-styles/bold-editorial.webp) | ![corporate](./screenshots/slide-deck-styles/corporate.webp) |
| blueprint | bold-editorial | corporate |
| ![dark-atmospheric](./screenshots/slide-deck-styles/dark-atmospheric.webp) | ![editorial-infographic](./screenshots/slide-deck-styles/editorial-infographic.webp) | ![fantasy-animation](./screenshots/slide-deck-styles/fantasy-animation.webp) |
| dark-atmospheric | editorial-infographic | fantasy-animation |
| ![intuition-machine](./screenshots/slide-deck-styles/intuition-machine.webp) | ![minimal](./screenshots/slide-deck-styles/minimal.webp) | ![notion](./screenshots/slide-deck-styles/notion.webp) |
| intuition-machine | minimal | notion |
| ![pixel-art](./screenshots/slide-deck-styles/pixel-art.webp) | ![scientific](./screenshots/slide-deck-styles/scientific.webp) | ![sketch-notes](./screenshots/slide-deck-styles/sketch-notes.webp) |
| pixel-art | scientific | sketch-notes |
| ![vector-illustration](./screenshots/slide-deck-styles/vector-illustration.webp) | ![vintage](./screenshots/slide-deck-styles/vintage.webp) | ![watercolor](./screenshots/slide-deck-styles/watercolor.webp) |
| vector-illustration | vintage | watercolor |
生成完成后,所有幻灯片会自动合并为 `.pptx` 文件,方便分享。
### baoyu-comic
#### baoyu-comic
知识漫画创作器,支持多种风格(Logicomix/清线风格、欧姆社漫画教程风格)。创作带有详细分镜布局的原创教育漫画,逐页生成图片。
@@ -228,7 +242,30 @@ npx add-skill jimliu/baoyu-skills
| `mixed` | 3-7 不等 | 复杂叙事、情感弧线 |
| `webtoon` | 3-5 竖向 | 欧姆社教程、手机阅读 |
### baoyu-post-to-wechat
#### baoyu-article-illustrator
智能文章插图技能。分析文章内容,在需要视觉辅助的位置生成插图。
```bash
/baoyu-article-illustrator path/to/article.md
```
#### baoyu-post-to-x
发布内容和文章到 X (Twitter)。支持带图片的普通帖子和 X 文章(长篇 Markdown)。使用真实 Chrome + CDP 绕过反自动化检测。
```bash
# 发布文字
/baoyu-post-to-x "Hello from Claude Code!"
# 发布带图片
/baoyu-post-to-x "看看这个" --image photo.png
# 发布 X 文章
/baoyu-post-to-x --article path/to/article.md
```
#### baoyu-post-to-wechat
发布内容到微信公众号,支持两种模式:
@@ -250,6 +287,63 @@ npx add-skill jimliu/baoyu-skills
前置要求:已安装 Google Chrome,首次运行需扫码登录(登录状态会保存)
### AI 生成技能 (AI Generation Skills)
AI 驱动的生成后端。
#### baoyu-danger-gemini-web
与 Gemini Web 交互,生成文本和图片。
**文本生成:**
```bash
/baoyu-danger-gemini-web "你好,Gemini"
/baoyu-danger-gemini-web --prompt "解释量子计算"
```
**图片生成:**
```bash
/baoyu-danger-gemini-web --prompt "一只可爱的猫" --image cat.png
/baoyu-danger-gemini-web --promptfiles system.md content.md --image out.png
```
### 工具技能 (Utility Skills)
内容处理工具。
#### baoyu-danger-x-to-markdown
将 X (Twitter) 内容转换为 markdown 格式。支持推文串和 X 文章。
```bash
# 将推文转换为 markdown
/baoyu-danger-x-to-markdown https://x.com/username/status/123456
# 保存到指定文件
/baoyu-danger-x-to-markdown https://x.com/username/status/123456 -o output.md
# JSON 输出
/baoyu-danger-x-to-markdown https://x.com/username/status/123456 --json
```
**支持的 URL**
- `https://x.com/<user>/status/<id>`
- `https://twitter.com/<user>/status/<id>`
- `https://x.com/i/article/<id>`
**身份验证:** 使用环境变量(`X_AUTH_TOKEN``X_CT0`)或 Chrome 登录进行 cookie 认证。
#### baoyu-compress-image
压缩图片以减小文件大小,同时保持质量。
```bash
/baoyu-compress-image path/to/image.png
/baoyu-compress-image path/to/images/ --quality 80
```
## 自定义扩展
所有技能支持通过 `EXTEND.md` 文件自定义。创建扩展文件可覆盖默认样式、添加自定义配置或定义个人预设。
@@ -280,7 +374,7 @@ mkdir -p .baoyu-skills/baoyu-cover-image
## 免责声明
### baoyu-gemini-web
### baoyu-danger-gemini-web
此技能使用 Gemini Web API(逆向工程)。
@@ -290,6 +384,17 @@ mkdir -p .baoyu-skills/baoyu-cover-image
- Cookies 会被缓存供后续使用
- 不保证 API 的稳定性或可用性
### baoyu-danger-x-to-markdown
此技能使用逆向工程的 X (Twitter) API。
**警告:** 这不是官方 API。使用风险自负。
- 如果 X 更改其 API,可能会无预警失效
- 如检测到 API 使用,账号可能受限
- 首次使用需确认免责声明
- 通过环境变量或 Chrome 登录进行身份验证
## 许可证
MIT
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 125 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 782 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 936 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 129 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 402 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 926 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 107 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 106 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 248 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

+26 -27
View File
@@ -102,43 +102,42 @@ When no `--style` is specified, analyze content to select the best style:
## File Management
### With Article Path
### Output Directory
Save illustrations to `[source-name-no-ext]/illustrations/` subdirectory in the same folder as the article:
Each session creates an independent directory named by content slug:
```
path/to/
├── article.md
── article/
└── illustrations/
├── outline.md
├── prompts/
│ ├── illustration-concept-a.md
── illustration-concept-b.md
│ └── ...
├── illustration-concept-a.png
├── illustration-concept-b.png
└── ...
illustrations/{topic-slug}/
├── source-{slug}.{ext} # Source files (text, images, etc.)
── outline.md
├── outline-{style}.md # Style variant outlines
├── prompts/
├── illustration-concept-a.md
│ ├── illustration-concept-b.md
── ...
├── illustration-concept-a.png
├── illustration-concept-b.png
└── ...
```
Example: `/posts/ai-future.md``/posts/ai-future/illustrations/`
**Slug Generation**:
1. Extract main topic from content (2-4 words, kebab-case)
2. Example: "The Future of AI" → `future-of-ai`
### Without Article Path (Pasted Content)
### Conflict Resolution
Save to `./illustrations/[topic-slug]/`:
If `illustrations/{topic-slug}/` already exists:
- Append timestamp: `{topic-slug}-YYYYMMDD-HHMMSS`
- Example: `ai-future` exists → `ai-future-20260118-143052`
```
illustrations/
└── ai-future/
├── source.md
├── outline.md
├── prompts/
└── *.png
```
### Source Files
### Directory Backup
Copy all sources with naming `source-{slug}.{ext}`:
- `source-article.md` (main text content)
- `source-photo.jpg` (image from conversation)
- `source-reference.pdf` (additional file)
If target directory exists, rename existing to `<dirname>-backup-YYYYMMDD-HHMMSS`
Multiple sources supported: text, images, files from conversation.
## Workflow
+16 -8
View File
@@ -55,9 +55,11 @@ Style × Layout × Aspect can be freely combined. Custom styles can be described
## File Structure
Each session creates an independent directory named by content slug:
```
[target]/
├── source.md # Source content (if pasted, not file)
comic/{topic-slug}/
├── source-{slug}.{ext} # Source files (text, images, etc.)
├── analysis.md # Deep analysis results (YAML+MD)
├── storyboard-chronological.md # Variant A (preserved)
├── storyboard-thematic.md # Variant B (preserved)
@@ -83,13 +85,19 @@ Style × Layout × Aspect can be freely combined. Custom styles can be described
└── {topic-slug}.pdf
```
**Target directory**:
- With source path: `[source-dir]/[source-name-no-ext]/comic/`
- Example: `/posts/turing-story.md``/posts/turing-story/comic/`
- Without source: `./comic/[topic-slug]/`
**Slug Generation**:
1. Extract main topic from content (2-4 words, kebab-case)
2. Example: "Alan Turing Biography" → `alan-turing-bio`
**Directory backup**:
- If target directory exists, rename existing to `<dirname>-backup-YYYYMMDD-HHMMSS`
**Conflict Resolution**:
If `comic/{topic-slug}/` already exists:
- Append timestamp: `{topic-slug}-YYYYMMDD-HHMMSS`
- Example: `turing-story` exists → `turing-story-20260118-143052`
**Source Files**:
Copy all sources with naming `source-{slug}.{ext}`:
- `source-biography.md`, `source-portrait.jpg`, `source-timeline.png`, etc.
- Multiple sources supported: text, images, files from conversation
## Workflow
+188
View File
@@ -0,0 +1,188 @@
---
name: baoyu-compress-image
description: Cross-platform image compression skill. Converts images to WebP by default with PNG-to-PNG support. Uses system tools (sips, cwebp, ImageMagick) with Sharp fallback.
---
# Image Compressor
Cross-platform image compression with WebP default output, PNG-to-PNG support, preferring system tools with Sharp fallback.
## Script Directory
**Important**: All scripts are located in the `scripts/` subdirectory of this skill.
**Agent Execution Instructions**:
1. Determine this SKILL.md file's directory path as `SKILL_DIR`
2. Script path = `${SKILL_DIR}/scripts/<script-name>.ts`
3. Replace all `${SKILL_DIR}` in this document with the actual path
**Script Reference**:
| Script | Purpose |
|--------|---------|
| `scripts/main.ts` | CLI entry point for image compression |
## Quick Start
```bash
# Compress to WebP (default)
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png
# Keep original format (PNG → PNG)
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png --format png
# Custom quality
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png -q 75
# Process directory
npx -y bun ${SKILL_DIR}/scripts/main.ts ./images/ -r
```
## Commands
### Single File Compression
```bash
# Basic (converts to WebP, replaces original)
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png
# Custom output path
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png -o compressed.webp
# Keep original file
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png --keep
# Custom quality (0-100, default: 80)
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png -q 75
# Keep original format
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png -f png
```
### Directory Processing
```bash
# Process all images in directory
npx -y bun ${SKILL_DIR}/scripts/main.ts ./images/
# Recursive processing
npx -y bun ${SKILL_DIR}/scripts/main.ts ./images/ -r
# With custom quality
npx -y bun ${SKILL_DIR}/scripts/main.ts ./images/ -r -q 75
```
### Output Formats
```bash
# Plain text (default)
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png
# JSON output
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png --json
```
## Options
| Option | Short | Description | Default |
|--------|-------|-------------|---------|
| `<input>` | | Input file or directory | Required |
| `--output <path>` | `-o` | Output path | Same path, new extension |
| `--format <fmt>` | `-f` | webp, png, jpeg | webp |
| `--quality <n>` | `-q` | Quality 0-100 | 80 |
| `--keep` | `-k` | Keep original file | false |
| `--recursive` | `-r` | Process directories recursively | false |
| `--json` | | JSON output | false |
| `--help` | `-h` | Show help | |
## Compressor Selection
Priority order (auto-detected):
1. **sips** (macOS built-in, WebP support since macOS 11)
2. **cwebp** (Google's official WebP tool)
3. **ImageMagick** (`convert` command)
4. **Sharp** (npm package, auto-installed by Bun)
The skill automatically selects the best available compressor.
## Output Format
### Text Mode (default)
```
image.png → image.webp (245KB → 89KB, 64% reduction)
```
### JSON Mode
```json
{
"input": "image.png",
"output": "image.webp",
"inputSize": 250880,
"outputSize": 91136,
"ratio": 0.36,
"compressor": "sips"
}
```
### Directory JSON Mode
```json
{
"files": [...],
"summary": {
"totalFiles": 10,
"totalInputSize": 2508800,
"totalOutputSize": 911360,
"ratio": 0.36,
"compressor": "sips"
}
}
```
## Examples
### Compress single image
```bash
npx -y bun ${SKILL_DIR}/scripts/main.ts photo.png
# photo.png → photo.webp (1.2MB → 340KB, 72% reduction)
```
### Compress with custom quality
```bash
npx -y bun ${SKILL_DIR}/scripts/main.ts photo.png -q 60
# photo.png → photo.webp (1.2MB → 280KB, 77% reduction)
```
### Keep original format
```bash
npx -y bun ${SKILL_DIR}/scripts/main.ts screenshot.png -f png --keep
# screenshot.png → screenshot-compressed.png (500KB → 380KB, 24% reduction)
```
### Process entire directory
```bash
npx -y bun ${SKILL_DIR}/scripts/main.ts ./screenshots/ -r
# Processed 15 files: 12.5MB → 4.2MB (66% reduction)
```
### Get JSON for scripting
```bash
npx -y bun ${SKILL_DIR}/scripts/main.ts image.png --json | jq '.ratio'
```
## Extension Support
Custom configurations via EXTEND.md.
**Check paths** (priority order):
1. `.baoyu-skills/baoyu-compress-image/EXTEND.md` (project)
2. `~/.baoyu-skills/baoyu-compress-image/EXTEND.md` (user)
If found, load before workflow. Extension content overrides defaults.
+315
View File
@@ -0,0 +1,315 @@
#!/usr/bin/env bun
import { existsSync, statSync, readdirSync, unlinkSync, renameSync } from "fs";
import { basename, dirname, extname, join, resolve } from "path";
import { spawn } from "child_process";
type Compressor = "sips" | "cwebp" | "imagemagick" | "sharp";
type Format = "webp" | "png" | "jpeg";
interface Options {
input: string;
output?: string;
format: Format;
quality: number;
keep: boolean;
recursive: boolean;
json: boolean;
}
interface Result {
input: string;
output: string;
inputSize: number;
outputSize: number;
ratio: number;
compressor: Compressor;
}
const SUPPORTED_EXTS = [".png", ".jpg", ".jpeg", ".webp", ".gif", ".tiff"];
async function commandExists(cmd: string): Promise<boolean> {
try {
const proc = spawn("which", [cmd], { stdio: "pipe" });
return new Promise((res) => {
proc.on("close", (code) => res(code === 0));
proc.on("error", () => res(false));
});
} catch {
return false;
}
}
async function detectCompressor(format: Format): Promise<Compressor> {
if (format === "webp") {
if (await commandExists("cwebp")) return "cwebp";
if (await commandExists("convert")) return "imagemagick";
return "sharp";
}
if (process.platform === "darwin") return "sips";
if (await commandExists("convert")) return "imagemagick";
return "sharp";
}
function runCmd(cmd: string, args: string[]): Promise<{ code: number; stderr: string }> {
return new Promise((res) => {
const proc = spawn(cmd, args, { stdio: ["ignore", "ignore", "pipe"] });
let stderr = "";
proc.stderr?.on("data", (d) => (stderr += d.toString()));
proc.on("close", (code) => res({ code: code ?? 1, stderr }));
proc.on("error", (e) => res({ code: 1, stderr: e.message }));
});
}
async function compressWithSips(input: string, output: string, format: Format, quality: number): Promise<void> {
const fmt = format === "jpeg" ? "jpeg" : format;
const args = ["-s", "format", fmt, "-s", "formatOptions", String(quality), input, "--out", output];
const { code, stderr } = await runCmd("sips", args);
if (code !== 0) throw new Error(`sips failed: ${stderr}`);
}
async function compressWithCwebp(input: string, output: string, quality: number): Promise<void> {
const args = ["-q", String(quality), input, "-o", output];
const { code, stderr } = await runCmd("cwebp", args);
if (code !== 0) throw new Error(`cwebp failed: ${stderr}`);
}
async function compressWithImagemagick(input: string, output: string, quality: number): Promise<void> {
const args = [input, "-quality", String(quality), output];
const { code, stderr } = await runCmd("convert", args);
if (code !== 0) throw new Error(`convert failed: ${stderr}`);
}
async function compressWithSharp(input: string, output: string, format: Format, quality: number): Promise<void> {
const sharp = (await import("sharp")).default;
let pipeline = sharp(input);
if (format === "webp") pipeline = pipeline.webp({ quality });
else if (format === "png") pipeline = pipeline.png({ quality });
else if (format === "jpeg") pipeline = pipeline.jpeg({ quality });
await pipeline.toFile(output);
}
async function compress(
compressor: Compressor,
input: string,
output: string,
format: Format,
quality: number
): Promise<void> {
switch (compressor) {
case "sips":
await compressWithSips(input, output, format, quality);
break;
case "cwebp":
if (format !== "webp") {
await compressWithSharp(input, output, format, quality);
} else {
await compressWithCwebp(input, output, quality);
}
break;
case "imagemagick":
await compressWithImagemagick(input, output, quality);
break;
case "sharp":
await compressWithSharp(input, output, format, quality);
break;
}
}
function getOutputPath(input: string, format: Format, keep: boolean, customOutput?: string): string {
if (customOutput) return resolve(customOutput);
const dir = dirname(input);
const base = basename(input, extname(input));
const ext = format === "jpeg" ? ".jpg" : `.${format}`;
if (keep && extname(input).toLowerCase() === ext) {
return join(dir, `${base}-compressed${ext}`);
}
return join(dir, `${base}${ext}`);
}
function formatSize(bytes: number): string {
if (bytes < 1024) return `${bytes}B`;
if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)}KB`;
return `${(bytes / (1024 * 1024)).toFixed(1)}MB`;
}
async function processFile(
compressor: Compressor,
input: string,
opts: Options
): Promise<Result> {
const absInput = resolve(input);
const inputSize = statSync(absInput).size;
const output = getOutputPath(absInput, opts.format, opts.keep, opts.output);
const tempOutput = output + ".tmp";
await compress(compressor, absInput, tempOutput, opts.format, opts.quality);
const outputSize = statSync(tempOutput).size;
if (!opts.keep && absInput !== output) {
unlinkSync(absInput);
}
renameSync(tempOutput, output);
return {
input: absInput,
output,
inputSize,
outputSize,
ratio: outputSize / inputSize,
compressor,
};
}
function collectFiles(dir: string, recursive: boolean): string[] {
const files: string[] = [];
const entries = readdirSync(dir, { withFileTypes: true });
for (const entry of entries) {
const full = join(dir, entry.name);
if (entry.isDirectory() && recursive) {
files.push(...collectFiles(full, recursive));
} else if (entry.isFile() && SUPPORTED_EXTS.includes(extname(entry.name).toLowerCase())) {
files.push(full);
}
}
return files;
}
function printHelp() {
console.log(`Usage: bun main.ts <input> [options]
Options:
-o, --output <path> Output path
-f, --format <fmt> Output format: webp, png, jpeg (default: webp)
-q, --quality <n> Quality 0-100 (default: 80)
-k, --keep Keep original file
-r, --recursive Process directories recursively
--json JSON output
-h, --help Show help`);
}
function parseArgs(args: string[]): Options | null {
const opts: Options = {
input: "",
format: "webp",
quality: 80,
keep: false,
recursive: false,
json: false,
};
for (let i = 0; i < args.length; i++) {
const arg = args[i];
if (arg === "-h" || arg === "--help") {
printHelp();
process.exit(0);
} else if (arg === "-o" || arg === "--output") {
opts.output = args[++i];
} else if (arg === "-f" || arg === "--format") {
const fmt = args[++i]?.toLowerCase();
if (fmt === "webp" || fmt === "png" || fmt === "jpeg" || fmt === "jpg") {
opts.format = fmt === "jpg" ? "jpeg" : (fmt as Format);
} else {
console.error(`Invalid format: ${fmt}`);
return null;
}
} else if (arg === "-q" || arg === "--quality") {
const q = parseInt(args[++i], 10);
if (isNaN(q) || q < 0 || q > 100) {
console.error(`Invalid quality: ${args[i]}`);
return null;
}
opts.quality = q;
} else if (arg === "-k" || arg === "--keep") {
opts.keep = true;
} else if (arg === "-r" || arg === "--recursive") {
opts.recursive = true;
} else if (arg === "--json") {
opts.json = true;
} else if (!arg.startsWith("-") && !opts.input) {
opts.input = arg;
}
}
if (!opts.input) {
console.error("Error: Input file or directory required");
printHelp();
return null;
}
return opts;
}
async function main() {
const args = process.argv.slice(2);
const opts = parseArgs(args);
if (!opts) process.exit(1);
const input = resolve(opts.input);
if (!existsSync(input)) {
console.error(`Error: ${input} not found`);
process.exit(1);
}
const compressor = await detectCompressor(opts.format);
const isDir = statSync(input).isDirectory();
if (isDir) {
const files = collectFiles(input, opts.recursive);
if (files.length === 0) {
console.error("No supported images found");
process.exit(1);
}
const results: Result[] = [];
for (const file of files) {
try {
const r = await processFile(compressor, file, { ...opts, output: undefined });
results.push(r);
if (!opts.json) {
const reduction = Math.round((1 - r.ratio) * 100);
console.log(`${r.input}${r.output} (${formatSize(r.inputSize)}${formatSize(r.outputSize)}, ${reduction}% reduction)`);
}
} catch (e) {
if (!opts.json) console.error(`Error processing ${file}: ${(e as Error).message}`);
}
}
if (opts.json) {
const totalInput = results.reduce((s, r) => s + r.inputSize, 0);
const totalOutput = results.reduce((s, r) => s + r.outputSize, 0);
console.log(
JSON.stringify({
files: results,
summary: {
totalFiles: results.length,
totalInputSize: totalInput,
totalOutputSize: totalOutput,
ratio: totalInput > 0 ? totalOutput / totalInput : 0,
compressor,
},
}, null, 2)
);
} else {
const totalInput = results.reduce((s, r) => s + r.inputSize, 0);
const totalOutput = results.reduce((s, r) => s + r.outputSize, 0);
const reduction = Math.round((1 - totalOutput / totalInput) * 100);
console.log(`\nProcessed ${results.length} files: ${formatSize(totalInput)}${formatSize(totalOutput)} (${reduction}% reduction)`);
}
} else {
try {
const r = await processFile(compressor, input, opts);
if (opts.json) {
console.log(JSON.stringify(r, null, 2));
} else {
const reduction = Math.round((1 - r.ratio) * 100);
console.log(`${r.input}${r.output} (${formatSize(r.inputSize)}${formatSize(r.outputSize)}, ${reduction}% reduction)`);
}
} catch (e) {
console.error(`Error: ${(e as Error).message}`);
process.exit(1);
}
}
}
main();
+19 -22
View File
@@ -74,38 +74,35 @@ When no `--style` is specified, the system analyzes content to select the best s
## File Management
### With Article Path
### Output Directory
Save to `[source-name-no-ext]/cover-image/` subdirectory in the same folder as the article:
Each session creates an independent directory named by content slug:
```
path/to/
├── article.md
── article/
└── cover-image/
├── prompts/
│ └── cover.md
└── cover.png
cover-image/{topic-slug}/
├── source-{slug}.{ext} # Source files (text, images, etc.)
── prompts/
└── cover.md
└── cover.png
```
Example: `/posts/ai-future.md``/posts/ai-future/cover-image/`
**Slug Generation**:
1. Extract main topic from content (2-4 words, kebab-case)
2. Example: "The Future of AI" → `future-of-ai`
### Without Article Path (Pasted Content)
### Conflict Resolution
Save to `./cover-image/[topic-slug]/`:
If `cover-image/{topic-slug}/` already exists:
- Append timestamp: `{topic-slug}-YYYYMMDD-HHMMSS`
- Example: `ai-future` exists → `ai-future-20260118-143052`
```
cover-image/
└── ai-future/
├── source.md # Saved pasted content
├── prompts/
│ └── cover.md
└── cover.png
```
### Source Files
### Directory Backup
Copy all sources with naming `source-{slug}.{ext}`:
- `source-article.md` (main text content)
- `source-logo.png` (image from conversation)
If target directory exists, rename existing to `<dirname>-backup-YYYYMMDD-HHMMSS`
Multiple sources supported: text, images, files from conversation.
## Workflow
@@ -1,5 +1,5 @@
---
name: baoyu-gemini-web
name: baoyu-danger-gemini-web
description: Image generation skill using Gemini Web. Generates images from text prompts via Google Gemini. Also supports text generation. Use as the image generation backend for other skills like cover-image, xhs-images, article-illustrator.
---
@@ -26,6 +26,86 @@ Supports:
| `scripts/main.ts` | CLI entry point for text/image generation |
| `scripts/gemini-webapi/*` | TypeScript port of `gemini_webapi` (GeminiClient, types, utils) |
## ⚠️ Disclaimer (REQUIRED)
**Before using this skill**, the consent check MUST be performed.
### Consent Check Flow
**Step 1**: Check consent file
```bash
# macOS
cat ~/Library/Application\ Support/baoyu-skills/gemini-web/consent.json 2>/dev/null
# Linux
cat ~/.local/share/baoyu-skills/gemini-web/consent.json 2>/dev/null
# Windows (PowerShell)
Get-Content "$env:APPDATA\baoyu-skills\gemini-web\consent.json" 2>$null
```
**Step 2**: If consent exists and `accepted: true` with matching `disclaimerVersion: "1.0"`:
Print warning and proceed:
```
⚠️ Warning: Using reverse-engineered Gemini Web API (not official). Accepted on: <acceptedAt date>
```
**Step 3**: If consent file doesn't exist or `disclaimerVersion` mismatch:
Display disclaimer and ask user:
```
⚠️ DISCLAIMER
This tool uses a reverse-engineered Gemini Web API, NOT an official Google API.
Risks:
- May break without notice if Google changes their API
- No official support or guarantees
- Use at your own risk
Do you accept these terms and wish to continue?
```
Use `AskUserQuestion` tool with options:
- **Yes, I accept** - Continue and save consent
- **No, I decline** - Exit immediately
**Step 4**: On acceptance, create consent file:
```bash
# macOS
mkdir -p ~/Library/Application\ Support/baoyu-skills/gemini-web
cat > ~/Library/Application\ Support/baoyu-skills/gemini-web/consent.json << 'EOF'
{
"version": 1,
"accepted": true,
"acceptedAt": "<ISO timestamp>",
"disclaimerVersion": "1.0"
}
EOF
# Linux
mkdir -p ~/.local/share/baoyu-skills/gemini-web
cat > ~/.local/share/baoyu-skills/gemini-web/consent.json << 'EOF'
{
"version": 1,
"accepted": true,
"acceptedAt": "<ISO timestamp>",
"disclaimerVersion": "1.0"
}
EOF
```
**Step 5**: On decline, output message and stop:
```
User declined the disclaimer. Exiting.
```
---
## Quick start
```bash
@@ -181,7 +261,7 @@ Session files are stored in `~/Library/Application Support/baoyu-skills/gemini-w
Custom configurations via EXTEND.md.
**Check paths** (priority order):
1. `.baoyu-skills/baoyu-gemini-web/EXTEND.md` (project)
2. `~/.baoyu-skills/baoyu-gemini-web/EXTEND.md` (user)
1. `.baoyu-skills/baoyu-danger-gemini-web/EXTEND.md` (project)
2. `~/.baoyu-skills/baoyu-danger-gemini-web/EXTEND.md` (user)
If found, load before workflow. Extension content overrides defaults.
@@ -61,14 +61,14 @@ function normalizeSessionMetadata(input: unknown): Array<string | null> {
function printUsage(cookiePath: string, profileDir: string): void {
console.log(`Usage:
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --prompt "Hello"
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "Hello"
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --prompt "A cute cat" --image generated.png
npx -y bun skills/baoyu-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts --prompt "Hello"
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts "Hello"
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts --prompt "A cute cat" --image generated.png
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts --promptfiles system.md content.md --image out.png
Multi-turn conversation (agent generates unique sessionId):
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "Remember 42" --sessionId abc123
npx -y bun skills/baoyu-gemini-web/scripts/main.ts "What number?" --sessionId abc123
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts "Remember 42" --sessionId abc123
npx -y bun skills/baoyu-danger-gemini-web/scripts/main.ts "What number?" --sessionId abc123
Options:
-p, --prompt <text> Prompt text
+177
View File
@@ -0,0 +1,177 @@
---
name: baoyu-danger-x-to-markdown
description: Convert X (Twitter) tweet or article URL to markdown. Uses reverse-engineered X API (private). Requires user consent before use.
---
# X to Markdown
Converts X (Twitter) content to markdown format:
- Tweet threads → Markdown with YAML front matter
- X Articles → Full article content extraction
## Script Directory
**Important**: All scripts are located in the `scripts/` subdirectory of this skill.
**Agent Execution Instructions**:
1. Determine this SKILL.md file's directory path as `SKILL_DIR`
2. Script path = `${SKILL_DIR}/scripts/<script-name>.ts`
3. Replace all `${SKILL_DIR}` in this document with the actual path
**Script Reference**:
| Script | Purpose |
|--------|---------|
| `scripts/main.ts` | CLI entry point for URL conversion |
## ⚠️ Disclaimer (REQUIRED)
**Before using this skill**, the consent check MUST be performed.
### Consent Check Flow
**Step 1**: Check consent file
```bash
# macOS
cat ~/Library/Application\ Support/baoyu-skills/x-to-markdown/consent.json 2>/dev/null
# Linux
cat ~/.local/share/baoyu-skills/x-to-markdown/consent.json 2>/dev/null
# Windows (PowerShell)
Get-Content "$env:APPDATA\baoyu-skills\x-to-markdown\consent.json" 2>$null
```
**Step 2**: If consent exists and `accepted: true` with matching `disclaimerVersion: "1.0"`:
Print warning and proceed:
```
⚠️ Warning: Using reverse-engineered X API (not official). Accepted on: <acceptedAt date>
```
**Step 3**: If consent file doesn't exist or `disclaimerVersion` mismatch:
Display disclaimer and ask user:
```
⚠️ DISCLAIMER
This tool uses a reverse-engineered X (Twitter) API, NOT an official API.
Risks:
- May break without notice if X changes their API
- No official support or guarantees
- Account restrictions possible if API usage detected
- Use at your own risk
Do you accept these terms and wish to continue?
```
Use `AskUserQuestion` tool with options:
- **Yes, I accept** - Continue and save consent
- **No, I decline** - Exit immediately
**Step 4**: On acceptance, create consent file:
```bash
# macOS
mkdir -p ~/Library/Application\ Support/baoyu-skills/x-to-markdown
cat > ~/Library/Application\ Support/baoyu-skills/x-to-markdown/consent.json << 'EOF'
{
"version": 1,
"accepted": true,
"acceptedAt": "<ISO timestamp>",
"disclaimerVersion": "1.0"
}
EOF
# Linux
mkdir -p ~/.local/share/baoyu-skills/x-to-markdown
cat > ~/.local/share/baoyu-skills/x-to-markdown/consent.json << 'EOF'
{
"version": 1,
"accepted": true,
"acceptedAt": "<ISO timestamp>",
"disclaimerVersion": "1.0"
}
EOF
```
**Step 5**: On decline, output message and stop:
```
User declined the disclaimer. Exiting.
```
---
## Usage
```bash
# Convert tweet (outputs markdown path)
npx -y bun ${SKILL_DIR}/scripts/main.ts <url>
# Save to specific file
npx -y bun ${SKILL_DIR}/scripts/main.ts <url> -o output.md
# JSON output
npx -y bun ${SKILL_DIR}/scripts/main.ts <url> --json
```
## Options
| Option | Description |
|--------|-------------|
| `<url>` | Tweet or article URL |
| `-o <path>` | Output path (file or dir) |
| `--json` | Output as JSON |
| `--login` | Refresh cookies only |
## File Structure
```
x-to-markdown/
└── {username}/
└── {tweet-id}.md
```
## Supported URLs
- `https://x.com/<user>/status/<id>`
- `https://twitter.com/<user>/status/<id>`
- `https://x.com/i/article/<id>`
## Output Format
```markdown
---
url: https://x.com/username/status/123
author: "Display Name (@username)"
tweet_count: 3
---
Tweet content...
---
Thread continuation...
```
## Authentication
**Option 1**: Environment variables (recommended)
- `X_AUTH_TOKEN` - auth_token cookie
- `X_CT0` - ct0 cookie
**Option 2**: Chrome login (auto if env vars not set)
- First run opens Chrome for login
- Cookies cached locally
## Extension Support
Custom configurations via EXTEND.md.
**Check paths** (priority order):
1. `.baoyu-skills/baoyu-danger-x-to-markdown/EXTEND.md` (project)
2. `~/.baoyu-skills/baoyu-danger-x-to-markdown/EXTEND.md` (user)
If found, load before workflow. Extension content overrides defaults.
@@ -0,0 +1,143 @@
import { resolveXToMarkdownChromeProfileDir } from "./paths.js";
export const DEFAULT_BEARER_TOKEN =
"Bearer AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA";
export const DEFAULT_USER_AGENT =
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36";
export const X_LOGIN_URL = "https://x.com/home";
export const X_USER_DATA_DIR = resolveXToMarkdownChromeProfileDir();
export const X_COOKIE_NAMES = ["auth_token", "ct0", "gt", "twid"] as const;
export const X_REQUIRED_COOKIES = ["auth_token", "ct0"] as const;
export const FALLBACK_QUERY_ID = "id8pHQbQi7eZ6P9mA1th1Q";
export const FALLBACK_FEATURE_SWITCHES = [
"profile_label_improvements_pcf_label_in_post_enabled",
"responsive_web_profile_redirect_enabled",
"rweb_tipjar_consumption_enabled",
"verified_phone_label_enabled",
"responsive_web_graphql_skip_user_profile_image_extensions_enabled",
"responsive_web_graphql_timeline_navigation_enabled",
];
export const FALLBACK_FIELD_TOGGLES = ["withPayments", "withAuxiliaryUserLabels"];
export const FALLBACK_TWEET_QUERY_ID = "HJ9lpOL-ZlOk5CkCw0JW6Q";
export const FALLBACK_TWEET_FEATURE_SWITCHES = [
"creator_subscriptions_tweet_preview_api_enabled",
"premium_content_api_read_enabled",
"communities_web_enable_tweet_community_results_fetch",
"c9s_tweet_anatomy_moderator_badge_enabled",
"responsive_web_grok_analyze_button_fetch_trends_enabled",
"responsive_web_grok_analyze_post_followups_enabled",
"responsive_web_jetfuel_frame",
"responsive_web_grok_share_attachment_enabled",
"responsive_web_grok_annotations_enabled",
"articles_preview_enabled",
"responsive_web_edit_tweet_api_enabled",
"graphql_is_translatable_rweb_tweet_is_translatable_enabled",
"view_counts_everywhere_api_enabled",
"longform_notetweets_consumption_enabled",
"responsive_web_twitter_article_tweet_consumption_enabled",
"tweet_awards_web_tipping_enabled",
"responsive_web_grok_show_grok_translated_post",
"responsive_web_grok_analysis_button_from_backend",
"post_ctas_fetch_enabled",
"creator_subscriptions_quote_tweet_preview_enabled",
"freedom_of_speech_not_reach_fetch_enabled",
"standardized_nudges_misinfo",
"tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled",
"longform_notetweets_rich_text_read_enabled",
"longform_notetweets_inline_media_enabled",
"profile_label_improvements_pcf_label_in_post_enabled",
"responsive_web_profile_redirect_enabled",
"rweb_tipjar_consumption_enabled",
"verified_phone_label_enabled",
"responsive_web_grok_image_annotation_enabled",
"responsive_web_grok_imagine_annotation_enabled",
"responsive_web_grok_community_note_auto_translation_is_enabled",
"responsive_web_graphql_skip_user_profile_image_extensions_enabled",
"responsive_web_graphql_timeline_navigation_enabled",
"responsive_web_enhance_cards_enabled",
];
export const FALLBACK_TWEET_FIELD_TOGGLES = [
"withArticleRichContentState",
"withArticlePlainText",
"withGrokAnalyze",
"withDisallowedReplyControls",
"withPayments",
"withAuxiliaryUserLabels",
];
export const FALLBACK_TWEET_DETAIL_QUERY_ID = "_8aYOgEDz35BrBcBal1-_w";
export const FALLBACK_TWEET_DETAIL_FEATURE_SWITCHES = [
"rweb_video_screen_enabled",
"profile_label_improvements_pcf_label_in_post_enabled",
"rweb_tipjar_consumption_enabled",
"verified_phone_label_enabled",
"creator_subscriptions_tweet_preview_api_enabled",
"responsive_web_graphql_timeline_navigation_enabled",
"responsive_web_graphql_skip_user_profile_image_extensions_enabled",
"premium_content_api_read_enabled",
"communities_web_enable_tweet_community_results_fetch",
"c9s_tweet_anatomy_moderator_badge_enabled",
"responsive_web_grok_analyze_button_fetch_trends_enabled",
"responsive_web_grok_analyze_post_followups_enabled",
"responsive_web_jetfuel_frame",
"responsive_web_grok_share_attachment_enabled",
"articles_preview_enabled",
"responsive_web_edit_tweet_api_enabled",
"graphql_is_translatable_rweb_tweet_is_translatable_enabled",
"view_counts_everywhere_api_enabled",
"longform_notetweets_consumption_enabled",
"responsive_web_twitter_article_tweet_consumption_enabled",
"tweet_awards_web_tipping_enabled",
"responsive_web_grok_show_grok_translated_post",
"responsive_web_grok_analysis_button_from_backend",
"creator_subscriptions_quote_tweet_preview_enabled",
"freedom_of_speech_not_reach_fetch_enabled",
"standardized_nudges_misinfo",
"tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled",
"longform_notetweets_rich_text_read_enabled",
"longform_notetweets_inline_media_enabled",
"responsive_web_grok_image_annotation_enabled",
"responsive_web_enhance_cards_enabled",
];
export const FALLBACK_TWEET_DETAIL_FEATURE_DEFAULTS: Record<string, boolean> = {
rweb_video_screen_enabled: false,
profile_label_improvements_pcf_label_in_post_enabled: true,
rweb_tipjar_consumption_enabled: true,
verified_phone_label_enabled: false,
creator_subscriptions_tweet_preview_api_enabled: true,
responsive_web_graphql_timeline_navigation_enabled: true,
responsive_web_graphql_skip_user_profile_image_extensions_enabled: false,
premium_content_api_read_enabled: false,
communities_web_enable_tweet_community_results_fetch: true,
c9s_tweet_anatomy_moderator_badge_enabled: true,
responsive_web_grok_analyze_button_fetch_trends_enabled: false,
responsive_web_grok_analyze_post_followups_enabled: true,
responsive_web_jetfuel_frame: false,
responsive_web_grok_share_attachment_enabled: true,
articles_preview_enabled: true,
responsive_web_edit_tweet_api_enabled: true,
graphql_is_translatable_rweb_tweet_is_translatable_enabled: true,
view_counts_everywhere_api_enabled: true,
longform_notetweets_consumption_enabled: true,
responsive_web_twitter_article_tweet_consumption_enabled: true,
tweet_awards_web_tipping_enabled: false,
responsive_web_grok_show_grok_translated_post: false,
responsive_web_grok_analysis_button_from_backend: true,
creator_subscriptions_quote_tweet_preview_enabled: false,
freedom_of_speech_not_reach_fetch_enabled: true,
standardized_nudges_misinfo: true,
tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled: true,
longform_notetweets_rich_text_read_enabled: true,
longform_notetweets_inline_media_enabled: true,
responsive_web_grok_image_annotation_enabled: true,
responsive_web_enhance_cards_enabled: false,
};
export const FALLBACK_TWEET_DETAIL_FIELD_TOGGLES = [
"withArticleRichContentState",
"withArticlePlainText",
"withGrokAnalyze",
"withDisallowedReplyControls",
];
@@ -0,0 +1,85 @@
import fs from "node:fs";
import path from "node:path";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { resolveXToMarkdownCookiePath } from "./paths.js";
export type CookieMap = Record<string, string>;
export type CookieFileData =
| {
cookies: CookieMap;
updated_at: number;
source?: string;
}
| {
version: number;
updatedAt: string;
cookieMap: CookieMap;
source?: string;
};
export async function read_cookie_file(
p: string = resolveXToMarkdownCookiePath()
): Promise<CookieMap | null> {
try {
if (!fs.existsSync(p) || !fs.statSync(p).isFile()) return null;
const raw = await readFile(p, "utf8");
const data = JSON.parse(raw) as unknown;
if (data && typeof data === "object" && "cookies" in (data as any)) {
const cookies = (data as any).cookies as unknown;
if (cookies && typeof cookies === "object") {
const out: CookieMap = {};
for (const [k, v] of Object.entries(cookies as Record<string, unknown>)) {
if (typeof v === "string") out[k] = v;
}
return Object.keys(out).length > 0 ? out : null;
}
}
if (data && typeof data === "object" && "cookieMap" in (data as any)) {
const cookies = (data as any).cookieMap as unknown;
if (cookies && typeof cookies === "object") {
const out: CookieMap = {};
for (const [k, v] of Object.entries(cookies as Record<string, unknown>)) {
if (typeof v === "string") out[k] = v;
}
return Object.keys(out).length > 0 ? out : null;
}
}
if (data && typeof data === "object") {
const out: CookieMap = {};
for (const [k, v] of Object.entries(data as Record<string, unknown>)) {
if (typeof v === "string") out[k] = v;
}
return Object.keys(out).length > 0 ? out : null;
}
return null;
} catch {
return null;
}
}
export async function write_cookie_file(
cookies: CookieMap,
p: string = resolveXToMarkdownCookiePath(),
source?: string
): Promise<void> {
const dir = path.dirname(p);
await mkdir(dir, { recursive: true });
const payload: CookieFileData = {
version: 1,
updatedAt: new Date().toISOString(),
cookieMap: cookies,
source,
};
await writeFile(p, JSON.stringify(payload, null, 2), "utf8");
}
export const readCookieFile = read_cookie_file;
export const writeCookieFile = write_cookie_file;
@@ -0,0 +1,423 @@
import { spawn, type ChildProcess } from "node:child_process";
import fs from "node:fs";
import { mkdir } from "node:fs/promises";
import net from "node:net";
import process from "node:process";
import { read_cookie_file, write_cookie_file } from "./cookie-file.js";
import { resolveXToMarkdownCookiePath } from "./paths.js";
import { X_COOKIE_NAMES, X_REQUIRED_COOKIES, X_LOGIN_URL, X_USER_DATA_DIR } from "./constants.js";
import type { CookieLike } from "./types.js";
type CdpSendOptions = { sessionId?: string; timeoutMs?: number };
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function fetchWithTimeout(
url: string,
init: RequestInit & { timeoutMs?: number } = {}
): Promise<Response> {
const { timeoutMs, ...rest } = init;
if (!timeoutMs || timeoutMs <= 0) return fetch(url, rest);
const ctl = new AbortController();
const t = setTimeout(() => ctl.abort(), timeoutMs);
try {
return await fetch(url, { ...rest, signal: ctl.signal });
} finally {
clearTimeout(t);
}
}
class CdpConnection {
private ws: WebSocket;
private nextId = 0;
private pending = new Map<
number,
{ resolve: (v: unknown) => void; reject: (e: Error) => void; timer: ReturnType<typeof setTimeout> | null }
>();
private constructor(ws: WebSocket) {
this.ws = ws;
this.ws.addEventListener("message", (event) => {
try {
const data =
typeof event.data === "string"
? event.data
: new TextDecoder().decode(event.data as ArrayBuffer);
const msg = JSON.parse(data) as { id?: number; result?: unknown; error?: { message?: string } };
if (msg.id) {
const p = this.pending.get(msg.id);
if (p) {
this.pending.delete(msg.id);
if (p.timer) clearTimeout(p.timer);
if (msg.error?.message) p.reject(new Error(msg.error.message));
else p.resolve(msg.result);
}
}
} catch {}
});
this.ws.addEventListener("close", () => {
for (const [id, p] of this.pending.entries()) {
this.pending.delete(id);
if (p.timer) clearTimeout(p.timer);
p.reject(new Error("CDP connection closed."));
}
});
}
static async connect(url: string, timeoutMs: number): Promise<CdpConnection> {
const ws = new WebSocket(url);
await new Promise<void>((resolve, reject) => {
const t = setTimeout(() => reject(new Error("CDP connection timeout.")), timeoutMs);
ws.addEventListener("open", () => {
clearTimeout(t);
resolve();
});
ws.addEventListener("error", () => {
clearTimeout(t);
reject(new Error("CDP connection failed."));
});
});
return new CdpConnection(ws);
}
async send<T = unknown>(
method: string,
params?: Record<string, unknown>,
opts?: CdpSendOptions
): Promise<T> {
const id = ++this.nextId;
const msg: Record<string, unknown> = { id, method };
if (params) msg.params = params;
if (opts?.sessionId) msg.sessionId = opts.sessionId;
const timeoutMs = opts?.timeoutMs ?? 15_000;
const out = await new Promise<unknown>((resolve, reject) => {
const t =
timeoutMs > 0
? setTimeout(() => {
this.pending.delete(id);
reject(new Error(`CDP timeout: ${method}`));
}, timeoutMs)
: null;
this.pending.set(id, { resolve, reject, timer: t });
this.ws.send(JSON.stringify(msg));
});
return out as T;
}
close(): void {
try {
this.ws.close();
} catch {}
}
}
async function getFreePort(): Promise<number> {
return await new Promise((resolve, reject) => {
const srv = net.createServer();
srv.unref();
srv.on("error", reject);
srv.listen(0, "127.0.0.1", () => {
const addr = srv.address();
if (!addr || typeof addr === "string") {
srv.close(() => reject(new Error("Unable to allocate a free TCP port.")));
return;
}
const port = addr.port;
srv.close((err) => (err ? reject(err) : resolve(port)));
});
});
}
function findChromeExecutable(): string | null {
const override = process.env.X_CHROME_PATH?.trim();
if (override && fs.existsSync(override)) return override;
const candidates: string[] = [];
switch (process.platform) {
case "darwin":
candidates.push(
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary",
"/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"
);
break;
case "win32":
candidates.push(
"C:\\\\Program Files\\\\Google\\\\Chrome\\\\Application\\\\chrome.exe",
"C:\\\\Program Files (x86)\\\\Google\\\\Chrome\\\\Application\\\\chrome.exe",
"C:\\\\Program Files\\\\Microsoft\\\\Edge\\\\Application\\\\msedge.exe",
"C:\\\\Program Files (x86)\\\\Microsoft\\\\Edge\\\\Application\\\\msedge.exe"
);
break;
default:
candidates.push(
"/usr/bin/google-chrome",
"/usr/bin/google-chrome-stable",
"/usr/bin/chromium",
"/usr/bin/chromium-browser",
"/snap/bin/chromium",
"/usr/bin/microsoft-edge"
);
break;
}
for (const p of candidates) {
if (fs.existsSync(p)) return p;
}
return null;
}
async function waitForChromeDebugPort(port: number, timeoutMs: number): Promise<string> {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
try {
const res = await fetchWithTimeout(`http://127.0.0.1:${port}/json/version`, { timeoutMs: 5_000 });
if (!res.ok) throw new Error(`status=${res.status}`);
const j = (await res.json()) as { webSocketDebuggerUrl?: string };
if (j.webSocketDebuggerUrl) return j.webSocketDebuggerUrl;
} catch {}
await sleep(200);
}
throw new Error("Chrome debug port not ready");
}
async function launchChrome(profileDir: string, port: number): Promise<ChildProcess> {
const chrome = findChromeExecutable();
if (!chrome) throw new Error("Chrome executable not found.");
const args = [
`--remote-debugging-port=${port}`,
`--user-data-dir=${profileDir}`,
"--no-first-run",
"--no-default-browser-check",
"--disable-popup-blocking",
X_LOGIN_URL,
];
return spawn(chrome, args, { stdio: "ignore" });
}
async function fetchXCookiesViaCdp(
profileDir: string,
timeoutMs: number,
verbose: boolean,
log?: (message: string) => void
): Promise<Record<string, string>> {
await mkdir(profileDir, { recursive: true });
const port = await getFreePort();
const chrome = await launchChrome(profileDir, port);
let cdp: CdpConnection | null = null;
try {
const wsUrl = await waitForChromeDebugPort(port, 30_000);
cdp = await CdpConnection.connect(wsUrl, 15_000);
const { targetId } = await cdp.send<{ targetId: string }>("Target.createTarget", {
url: X_LOGIN_URL,
newWindow: true,
});
const { sessionId } = await cdp.send<{ sessionId: string }>("Target.attachToTarget", { targetId, flatten: true });
await cdp.send("Network.enable", {}, { sessionId });
if (verbose) {
log?.("[x-cookies] Chrome opened. If needed, complete X login in the window. Waiting for cookies...");
}
const start = Date.now();
let last: Record<string, string> = {};
while (Date.now() - start < timeoutMs) {
const { cookies } = await cdp.send<{ cookies: CookieLike[] }>(
"Network.getCookies",
{ urls: ["https://x.com/", "https://twitter.com/"] },
{ sessionId, timeoutMs: 10_000 }
);
const m = buildXCookieMap((cookies ?? []).filter(Boolean));
last = m;
if (hasRequiredXCookies(m)) {
return m;
}
await sleep(1000);
}
throw new Error(`Timed out waiting for X cookies. Last keys: ${Object.keys(last).join(", ")}`);
} finally {
if (cdp) {
try {
await cdp.send("Browser.close", {}, { timeoutMs: 5_000 });
} catch {}
cdp.close();
}
try {
chrome.kill("SIGTERM");
} catch {}
setTimeout(() => {
if (!chrome.killed) {
try {
chrome.kill("SIGKILL");
} catch {}
}
}, 2_000).unref?.();
}
}
function resolveCookieDomain(cookie: CookieLike): string | null {
const rawDomain = cookie.domain?.trim();
if (rawDomain) {
return rawDomain.startsWith(".") ? rawDomain.slice(1) : rawDomain;
}
const rawUrl = cookie.url?.trim();
if (rawUrl) {
try {
return new URL(rawUrl).hostname;
} catch {
return null;
}
}
return null;
}
function pickCookieValue<T extends CookieLike>(cookies: T[], name: string): string | undefined {
const matches = cookies.filter((cookie) => cookie.name === name && typeof cookie.value === "string");
if (matches.length === 0) return undefined;
const preferred = matches.find((cookie) => {
const domain = resolveCookieDomain(cookie);
return domain === "x.com" && (cookie.path ?? "/") === "/";
});
const xDomain = matches.find((cookie) => (resolveCookieDomain(cookie) ?? "").endsWith("x.com"));
const twitterDomain = matches.find((cookie) => (resolveCookieDomain(cookie) ?? "").endsWith("twitter.com"));
return (preferred ?? xDomain ?? twitterDomain ?? matches[0])?.value;
}
function buildXCookieMap<T extends CookieLike>(cookies: T[]): Record<string, string> {
const cookieMap: Record<string, string> = {};
for (const name of X_COOKIE_NAMES) {
const value = pickCookieValue(cookies, name);
if (value) cookieMap[name] = value;
}
return cookieMap;
}
export function hasRequiredXCookies(cookieMap: Record<string, string>): boolean {
return X_REQUIRED_COOKIES.every((name) => Boolean(cookieMap[name]));
}
function filterXCookieMap(cookieMap: Record<string, string>): Record<string, string> {
const filtered: Record<string, string> = {};
for (const name of X_COOKIE_NAMES) {
const value = cookieMap[name];
if (value) filtered[name] = value;
}
return filtered;
}
function buildInlineCookiesFromEnv(): CookieLike[] {
const cookies: CookieLike[] = [];
const authToken = process.env.X_AUTH_TOKEN?.trim();
const ct0 = process.env.X_CT0?.trim();
const gt = process.env.X_GUEST_TOKEN?.trim();
const twid = process.env.X_TWID?.trim();
if (authToken) {
cookies.push({ name: "auth_token", value: authToken, domain: "x.com", path: "/" });
}
if (ct0) {
cookies.push({ name: "ct0", value: ct0, domain: "x.com", path: "/" });
}
if (gt) {
cookies.push({ name: "gt", value: gt, domain: "x.com", path: "/" });
}
if (twid) {
cookies.push({ name: "twid", value: twid, domain: "x.com", path: "/" });
}
return cookies;
}
async function loadXCookiesFromInline(log?: (message: string) => void): Promise<Record<string, string>> {
const inline = buildInlineCookiesFromEnv();
if (inline.length === 0) return {};
const cookieMap = buildXCookieMap(
inline.filter((cookie): cookie is CookieLike => Boolean(cookie?.name && typeof cookie.value === "string"))
);
if (Object.keys(cookieMap).length > 0) {
log?.(`[x-cookies] Loaded X cookies from env: ${Object.keys(cookieMap).length} cookie(s).`);
} else {
log?.("[x-cookies] Env cookies provided but no X cookies matched.");
}
return cookieMap;
}
async function loadXCookiesFromFile(log?: (message: string) => void): Promise<Record<string, string>> {
const cookiePath = resolveXToMarkdownCookiePath();
const fileMap = filterXCookieMap((await read_cookie_file(cookiePath)) ?? {});
if (Object.keys(fileMap).length > 0) {
log?.(`[x-cookies] Loaded X cookies from file: ${cookiePath} (${Object.keys(fileMap).length} cookie(s))`);
}
return fileMap;
}
async function loadXCookiesFromCdp(log?: (message: string) => void): Promise<Record<string, string>> {
try {
const cookieMap = await fetchXCookiesViaCdp(X_USER_DATA_DIR, 5 * 60 * 1000, true, log);
if (!hasRequiredXCookies(cookieMap)) return cookieMap;
const cookiePath = resolveXToMarkdownCookiePath();
try {
await write_cookie_file(cookieMap, cookiePath, "cdp");
log?.(`[x-cookies] Cookies saved to ${cookiePath}`);
} catch (error) {
log?.(
`[x-cookies] Failed to write cookie file (${cookiePath}): ${
error instanceof Error ? error.message : String(error ?? "")
}`
);
}
if (cookieMap.auth_token) log?.(`[x-cookies] auth_token: ${cookieMap.auth_token.slice(0, 20)}...`);
if (cookieMap.ct0) log?.(`[x-cookies] ct0: ${cookieMap.ct0.slice(0, 20)}...`);
return cookieMap;
} catch (error) {
log?.(
`[x-cookies] Failed to load cookies via Chrome DevTools Protocol: ${
error instanceof Error ? error.message : String(error ?? "")
}`
);
return {};
}
}
export async function loadXCookies(log?: (message: string) => void): Promise<Record<string, string>> {
const inlineMap = await loadXCookiesFromInline(log);
const fileMap = await loadXCookiesFromFile(log);
const combined = { ...fileMap, ...inlineMap };
if (hasRequiredXCookies(combined)) return combined;
const cdpMap = await loadXCookiesFromCdp(log);
return { ...fileMap, ...cdpMap, ...inlineMap };
}
export async function refreshXCookies(log?: (message: string) => void): Promise<Record<string, string>> {
return loadXCookiesFromCdp(log);
}
export function buildCookieHeader(cookieMap: Record<string, string>): string | undefined {
const entries = Object.entries(cookieMap).filter(([, value]) => value);
if (entries.length === 0) return undefined;
return entries.map(([key, value]) => `${key}=${value}`).join("; ");
}
@@ -0,0 +1,389 @@
import {
DEFAULT_BEARER_TOKEN,
DEFAULT_USER_AGENT,
FALLBACK_FEATURE_SWITCHES,
FALLBACK_FIELD_TOGGLES,
FALLBACK_QUERY_ID,
FALLBACK_TWEET_DETAIL_FEATURE_DEFAULTS,
FALLBACK_TWEET_DETAIL_FEATURE_SWITCHES,
FALLBACK_TWEET_DETAIL_FIELD_TOGGLES,
FALLBACK_TWEET_DETAIL_QUERY_ID,
FALLBACK_TWEET_FEATURE_SWITCHES,
FALLBACK_TWEET_FIELD_TOGGLES,
FALLBACK_TWEET_QUERY_ID,
} from "./constants.js";
import {
buildFeatureMap,
buildFieldToggleMap,
buildRequestHeaders,
buildTweetFieldToggleMap,
fetchHomeHtml,
fetchText,
parseStringList,
} from "./http.js";
import type { ArticleQueryInfo } from "./types.js";
function isNonEmptyObject(value: unknown): value is Record<string, unknown> {
return Boolean(value && typeof value === "object" && Object.keys(value as Record<string, unknown>).length > 0);
}
function unwrapTweetResult(result: any): any {
if (!result) return null;
if (result.__typename === "TweetWithVisibilityResults" && result.tweet) {
return result.tweet;
}
return result;
}
function extractArticleFromTweet(payload: unknown): unknown {
const root = (payload as { data?: any }).data ?? payload;
const result = root?.tweetResult?.result ?? root?.tweet_result?.result ?? root?.tweet_result;
const tweet = unwrapTweetResult(result);
const legacy = tweet?.legacy ?? {};
const article = legacy?.article ?? tweet?.article;
return (
article?.article_results?.result ??
legacy?.article_results?.result ??
tweet?.article_results?.result ??
null
);
}
function extractTweetFromPayload(payload: unknown): unknown {
const root = (payload as { data?: any }).data ?? payload;
const result = root?.tweetResult?.result ?? root?.tweet_result?.result ?? root?.tweet_result;
return unwrapTweetResult(result);
}
function extractArticleFromEntity(payload: unknown): unknown {
const root = (payload as { data?: any }).data ?? payload;
return (
root?.article_result_by_rest_id?.result ??
root?.article_result_by_rest_id ??
root?.article_entity_result?.result ??
null
);
}
async function resolveArticleQueryInfo(userAgent: string): Promise<ArticleQueryInfo> {
const html = await fetchHomeHtml(userAgent);
const bundleMatch = html.match(/"bundle\\.TwitterArticles":"([a-z0-9]+)"/);
if (!bundleMatch) {
return {
queryId: FALLBACK_QUERY_ID,
featureSwitches: FALLBACK_FEATURE_SWITCHES,
fieldToggles: FALLBACK_FIELD_TOGGLES,
html,
};
}
const bundleHash = bundleMatch[1];
const chunkUrl = `https://abs.twimg.com/responsive-web/client-web/bundle.TwitterArticles.${bundleHash}a.js`;
const chunk = await fetchText(chunkUrl, {
headers: {
"user-agent": userAgent,
},
});
const queryIdMatch = chunk.match(/queryId:\"([^\"]+)\",operationName:\"ArticleEntityResultByRestId\"/);
const featureMatch = chunk.match(
/operationName:\"ArticleEntityResultByRestId\"[\s\S]*?featureSwitches:\[(.*?)\]/
);
const fieldToggleMatch = chunk.match(
/operationName:\"ArticleEntityResultByRestId\"[\s\S]*?fieldToggles:\[(.*?)\]/
);
const featureSwitches = parseStringList(featureMatch?.[1]);
const fieldToggles = parseStringList(fieldToggleMatch?.[1]);
return {
queryId: queryIdMatch?.[1] ?? FALLBACK_QUERY_ID,
featureSwitches: featureSwitches.length > 0 ? featureSwitches : FALLBACK_FEATURE_SWITCHES,
fieldToggles: fieldToggles.length > 0 ? fieldToggles : FALLBACK_FIELD_TOGGLES,
html,
};
}
function resolveMainChunkHash(html: string): string | null {
const match = html.match(/main\\.([a-z0-9]+)\\.js/);
return match?.[1] ?? null;
}
function resolveApiChunkHash(html: string): string | null {
const match = html.match(/api:\"([a-zA-Z0-9_-]+)\"/);
return match?.[1] ?? null;
}
async function resolveTweetDetailQueryInfo(userAgent: string): Promise<ArticleQueryInfo> {
const html = await fetchHomeHtml(userAgent);
const apiHash = resolveApiChunkHash(html);
if (!apiHash) {
return {
queryId: FALLBACK_TWEET_DETAIL_QUERY_ID,
featureSwitches: FALLBACK_TWEET_DETAIL_FEATURE_SWITCHES,
fieldToggles: FALLBACK_TWEET_DETAIL_FIELD_TOGGLES,
html,
};
}
const chunkUrl = `https://abs.twimg.com/responsive-web/client-web/api.${apiHash}a.js`;
const chunk = await fetchText(chunkUrl, {
headers: {
"user-agent": userAgent,
},
});
const queryIdMatch = chunk.match(/queryId:\"([^\"]+)\",operationName:\"TweetDetail\"/);
const featureMatch = chunk.match(
/operationName:\"TweetDetail\"[\s\S]*?featureSwitches:\[(.*?)\]/
);
const fieldToggleMatch = chunk.match(
/operationName:\"TweetDetail\"[\s\S]*?fieldToggles:\[(.*?)\]/
);
const featureSwitches = parseStringList(featureMatch?.[1]);
const fieldToggles = parseStringList(fieldToggleMatch?.[1]);
return {
queryId: queryIdMatch?.[1] ?? FALLBACK_TWEET_DETAIL_QUERY_ID,
featureSwitches: featureSwitches.length > 0 ? featureSwitches : FALLBACK_TWEET_DETAIL_FEATURE_SWITCHES,
fieldToggles: fieldToggles.length > 0 ? fieldToggles : FALLBACK_TWEET_DETAIL_FIELD_TOGGLES,
html,
};
}
function buildTweetDetailFieldToggleMap(keys: string[]): Record<string, boolean> {
const toggles = buildFieldToggleMap(keys);
if (Object.prototype.hasOwnProperty.call(toggles, "withArticlePlainText")) {
toggles.withArticlePlainText = false;
}
if (Object.prototype.hasOwnProperty.call(toggles, "withGrokAnalyze")) {
toggles.withGrokAnalyze = false;
}
if (Object.prototype.hasOwnProperty.call(toggles, "withDisallowedReplyControls")) {
toggles.withDisallowedReplyControls = false;
}
return toggles;
}
async function resolveTweetQueryInfo(userAgent: string): Promise<ArticleQueryInfo> {
const html = await fetchHomeHtml(userAgent);
const mainHash = resolveMainChunkHash(html);
if (!mainHash) {
return {
queryId: FALLBACK_TWEET_QUERY_ID,
featureSwitches: FALLBACK_TWEET_FEATURE_SWITCHES,
fieldToggles: FALLBACK_TWEET_FIELD_TOGGLES,
html,
};
}
const chunkUrl = `https://abs.twimg.com/responsive-web/client-web/main.${mainHash}.js`;
const chunk = await fetchText(chunkUrl, {
headers: {
"user-agent": userAgent,
},
});
const queryIdMatch = chunk.match(/queryId:\"([^\"]+)\",operationName:\"TweetResultByRestId\"/);
const featureMatch = chunk.match(
/operationName:\"TweetResultByRestId\"[\s\S]*?featureSwitches:\[(.*?)\]/
);
const fieldToggleMatch = chunk.match(
/operationName:\"TweetResultByRestId\"[\s\S]*?fieldToggles:\[(.*?)\]/
);
const featureSwitches = parseStringList(featureMatch?.[1]);
const fieldToggles = parseStringList(fieldToggleMatch?.[1]);
return {
queryId: queryIdMatch?.[1] ?? FALLBACK_TWEET_QUERY_ID,
featureSwitches: featureSwitches.length > 0 ? featureSwitches : FALLBACK_TWEET_FEATURE_SWITCHES,
fieldToggles: fieldToggles.length > 0 ? fieldToggles : FALLBACK_TWEET_FIELD_TOGGLES,
html,
};
}
async function fetchTweetResult(
tweetId: string,
cookieMap: Record<string, string>,
userAgent: string,
bearerToken: string
): Promise<unknown> {
const queryInfo = await resolveTweetQueryInfo(userAgent);
const features = buildFeatureMap(queryInfo.html, queryInfo.featureSwitches);
const fieldToggles = buildTweetFieldToggleMap(queryInfo.fieldToggles);
const url = new URL(`https://x.com/i/api/graphql/${queryInfo.queryId}/TweetResultByRestId`);
url.searchParams.set(
"variables",
JSON.stringify({
tweetId,
withCommunity: false,
includePromotedContent: false,
withVoice: true,
})
);
if (Object.keys(features).length > 0) {
url.searchParams.set("features", JSON.stringify(features));
}
if (Object.keys(fieldToggles).length > 0) {
url.searchParams.set("fieldToggles", JSON.stringify(fieldToggles));
}
const response = await fetch(url.toString(), {
headers: buildRequestHeaders(cookieMap, userAgent, bearerToken),
});
const text = await response.text();
if (!response.ok) {
throw new Error(`X API error (${response.status}): ${text.slice(0, 400)}`);
}
try {
return JSON.parse(text);
} catch (error) {
throw new Error(`Failed to parse response JSON: ${error instanceof Error ? error.message : String(error)}`);
}
}
export async function fetchTweetDetail(
tweetId: string,
cookieMap: Record<string, string>,
cursor?: string
): Promise<unknown> {
const userAgent = process.env.X_USER_AGENT?.trim() || DEFAULT_USER_AGENT;
const bearerToken = process.env.X_BEARER_TOKEN?.trim() || DEFAULT_BEARER_TOKEN;
const queryInfo = await resolveTweetDetailQueryInfo(userAgent);
const features = buildFeatureMap(
queryInfo.html,
queryInfo.featureSwitches,
FALLBACK_TWEET_DETAIL_FEATURE_DEFAULTS
);
const fieldToggles = buildTweetDetailFieldToggleMap(queryInfo.fieldToggles);
const url = new URL(`https://x.com/i/api/graphql/${queryInfo.queryId}/TweetDetail`);
url.searchParams.set(
"variables",
JSON.stringify({
focalTweetId: tweetId,
cursor,
referrer: cursor ? "tweet" : undefined,
with_rux_injections: false,
includePromotedContent: true,
withCommunity: true,
withQuickPromoteEligibilityTweetFields: true,
withBirdwatchNotes: true,
withVoice: true,
withV2Timeline: true,
withDownvotePerspective: false,
withReactionsMetadata: false,
withReactionsPerspective: false,
withSuperFollowsTweetFields: false,
withSuperFollowsUserFields: false,
})
);
if (Object.keys(features).length > 0) {
url.searchParams.set("features", JSON.stringify(features));
}
if (Object.keys(fieldToggles).length > 0) {
url.searchParams.set("fieldToggles", JSON.stringify(fieldToggles));
}
const response = await fetch(url.toString(), {
headers: buildRequestHeaders(cookieMap, userAgent, bearerToken),
});
const text = await response.text();
if (!response.ok) {
throw new Error(`X API error (${response.status}): ${text.slice(0, 400)}`);
}
try {
return JSON.parse(text);
} catch (error) {
throw new Error(`Failed to parse response JSON: ${error instanceof Error ? error.message : String(error)}`);
}
}
async function fetchArticleEntityById(
articleEntityId: string,
cookieMap: Record<string, string>,
userAgent: string,
bearerToken: string
): Promise<unknown> {
const queryInfo = await resolveArticleQueryInfo(userAgent);
const features = buildFeatureMap(queryInfo.html, queryInfo.featureSwitches);
const fieldToggles = buildFieldToggleMap(queryInfo.fieldToggles);
const url = new URL(`https://x.com/i/api/graphql/${queryInfo.queryId}/ArticleEntityResultByRestId`);
url.searchParams.set("variables", JSON.stringify({ articleEntityId }));
if (Object.keys(features).length > 0) {
url.searchParams.set("features", JSON.stringify(features));
}
if (Object.keys(fieldToggles).length > 0) {
url.searchParams.set("fieldToggles", JSON.stringify(fieldToggles));
}
const response = await fetch(url.toString(), {
headers: buildRequestHeaders(cookieMap, userAgent, bearerToken),
});
const text = await response.text();
if (!response.ok) {
throw new Error(`X API error (${response.status}): ${text.slice(0, 400)}`);
}
try {
return JSON.parse(text);
} catch (error) {
throw new Error(`Failed to parse response JSON: ${error instanceof Error ? error.message : String(error)}`);
}
}
export async function fetchXArticle(
articleId: string,
cookieMap: Record<string, string>,
raw: boolean
): Promise<unknown> {
const userAgent = process.env.X_USER_AGENT?.trim() || DEFAULT_USER_AGENT;
const bearerToken = process.env.X_BEARER_TOKEN?.trim() || DEFAULT_BEARER_TOKEN;
const tweetPayload = await fetchTweetResult(articleId, cookieMap, userAgent, bearerToken);
if (raw) {
return tweetPayload;
}
const articleFromTweet = extractArticleFromTweet(tweetPayload);
if (isNonEmptyObject(articleFromTweet)) {
return articleFromTweet;
}
const articlePayload = await fetchArticleEntityById(articleId, cookieMap, userAgent, bearerToken);
const articleFromEntity = extractArticleFromEntity(articlePayload);
if (isNonEmptyObject(articleFromEntity)) {
return articleFromEntity;
}
return articleFromEntity ?? articlePayload;
}
export async function fetchXTweet(
tweetId: string,
cookieMap: Record<string, string>,
raw: boolean
): Promise<unknown> {
const userAgent = process.env.X_USER_AGENT?.trim() || DEFAULT_USER_AGENT;
const bearerToken = process.env.X_BEARER_TOKEN?.trim() || DEFAULT_BEARER_TOKEN;
const tweetPayload = await fetchTweetResult(tweetId, cookieMap, userAgent, bearerToken);
if (raw) {
return tweetPayload;
}
const tweet = extractTweetFromPayload(tweetPayload);
if (isNonEmptyObject(tweet)) {
return tweet;
}
return tweet ?? tweetPayload;
}
@@ -0,0 +1,117 @@
import { buildCookieHeader } from "./cookies.js";
let cachedHomeHtml: { userAgent: string; html: string } | null = null;
export async function fetchText(url: string, init?: RequestInit): Promise<string> {
const response = await fetch(url, init);
const text = await response.text();
if (!response.ok) {
throw new Error(`Request failed (${response.status}) for ${url}: ${text.slice(0, 200)}`);
}
return text;
}
export async function fetchHomeHtml(userAgent: string): Promise<string> {
if (cachedHomeHtml?.userAgent === userAgent) {
return cachedHomeHtml.html;
}
const html = await fetchText("https://x.com", {
headers: {
"user-agent": userAgent,
},
});
cachedHomeHtml = { userAgent, html };
return html;
}
export function parseStringList(raw: string | undefined): string[] {
if (!raw) return [];
return raw
.split(",")
.map((item) => item.trim())
.filter(Boolean)
.map((item) => item.replace(/^\"|\"$/g, ""));
}
export function resolveFeatureValue(html: string, key: string): boolean | undefined {
const keyPattern = key.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const unescaped = new RegExp(`"${keyPattern}"\\s*:\\s*\\{"value"\\s*:\\s*(true|false)`);
const escaped = new RegExp(`\\\\"${keyPattern}\\\\"\\s*:\\s*\\\\{\\\\"value\\\\"\\s*:\\s*(true|false)`);
const match = html.match(unescaped) ?? html.match(escaped);
if (!match) return undefined;
return match[1] === "true";
}
export function buildFeatureMap(
html: string,
keys: string[],
defaults?: Record<string, boolean>
): Record<string, boolean> {
const features: Record<string, boolean> = {};
for (const key of keys) {
const value = resolveFeatureValue(html, key);
if (value !== undefined) {
features[key] = value;
} else if (defaults && Object.prototype.hasOwnProperty.call(defaults, key)) {
features[key] = defaults[key] ?? true;
} else {
features[key] = true;
}
}
if (!Object.prototype.hasOwnProperty.call(features, "responsive_web_graphql_exclude_directive_enabled")) {
features.responsive_web_graphql_exclude_directive_enabled = true;
}
return features;
}
export function buildFieldToggleMap(keys: string[]): Record<string, boolean> {
const toggles: Record<string, boolean> = {};
for (const key of keys) {
toggles[key] = true;
}
return toggles;
}
export function buildTweetFieldToggleMap(keys: string[]): Record<string, boolean> {
const toggles: Record<string, boolean> = {};
for (const key of keys) {
if (key === "withGrokAnalyze" || key === "withDisallowedReplyControls") {
toggles[key] = false;
} else {
toggles[key] = true;
}
}
return toggles;
}
export function buildRequestHeaders(
cookieMap: Record<string, string>,
userAgent: string,
bearerToken: string
): Record<string, string> {
const headers: Record<string, string> = {
authorization: bearerToken,
"user-agent": userAgent,
accept: "application/json",
"x-twitter-active-user": "yes",
"x-twitter-client-language": "en",
"accept-language": "en",
};
if (cookieMap.auth_token) {
headers["x-twitter-auth-type"] = "OAuth2Session";
}
const cookieHeader = buildCookieHeader(cookieMap);
if (cookieHeader) {
headers.cookie = cookieHeader;
}
if (cookieMap.ct0) {
headers["x-csrf-token"] = cookieMap.ct0;
}
if (process.env.X_CLIENT_TRANSACTION_ID?.trim()) {
headers["x-client-transaction-id"] = process.env.X_CLIENT_TRANSACTION_ID.trim();
}
return headers;
}
@@ -0,0 +1,419 @@
import fs from "node:fs";
import path from "node:path";
import readline from "node:readline";
import process from "node:process";
import { mkdir, readFile, rename, writeFile } from "node:fs/promises";
import { fetchXArticle } from "./graphql.js";
import { formatArticleMarkdown } from "./markdown.js";
import { hasRequiredXCookies, loadXCookies, refreshXCookies } from "./cookies.js";
import { resolveXToMarkdownConsentPath } from "./paths.js";
import { tweetToMarkdown } from "./tweet-to-markdown.js";
type CliArgs = {
url: string | null;
output: string | null;
json: boolean;
login: boolean;
help: boolean;
};
type ConsentRecord = {
version: number;
accepted: boolean;
acceptedAt: string;
disclaimerVersion: string;
};
const DISCLAIMER_VERSION = "1.0";
function printUsage(exitCode: number): never {
const cmd = "npx -y bun skills/baoyu-danger-x-to-markdown/scripts/main.ts";
console.log(`X (Twitter) to Markdown
Usage:
${cmd} <url>
${cmd} --url <url>
Options:
--output <path>, -o Output path (file or dir). Default: ./x-to-markdown/<slug>/
--json Output as JSON
--login Refresh cookies only, then exit
--help, -h Show help
Examples:
${cmd} https://x.com/username/status/1234567890
${cmd} https://x.com/i/article/1234567890 -o ./article.md
${cmd} https://x.com/username/status/1234567890 -o ./out/
${cmd} https://x.com/username/status/1234567890 --json | jq -r '.markdownPath'
${cmd} --login
`);
process.exit(exitCode);
}
function parseArgs(argv: string[]): CliArgs {
const out: CliArgs = {
url: null,
output: null,
json: false,
login: false,
help: false,
};
const positional: string[] = [];
for (let i = 0; i < argv.length; i++) {
const a = argv[i]!;
if (a === "--help" || a === "-h") {
out.help = true;
continue;
}
if (a === "--json") {
out.json = true;
continue;
}
if (a === "--login") {
out.login = true;
continue;
}
if (a === "--url") {
const v = argv[++i];
if (!v) throw new Error("Missing value for --url");
out.url = v;
continue;
}
if (a === "--output" || a === "-o") {
const v = argv[++i];
if (!v) throw new Error(`Missing value for ${a}`);
out.output = v;
continue;
}
if (a.startsWith("-")) {
throw new Error(`Unknown option: ${a}`);
}
positional.push(a);
}
if (!out.url && positional.length > 0) {
out.url = positional[0]!;
}
return out;
}
function normalizeInputUrl(input: string): string {
const trimmed = input.trim();
if (!trimmed) return "";
try {
return new URL(trimmed).toString();
} catch {
return trimmed;
}
}
function parseArticleId(input: string): string | null {
const trimmed = input.trim();
if (!trimmed) return null;
try {
const parsed = new URL(trimmed);
const match = parsed.pathname.match(/\/(?:i\/)?article\/(\d+)/);
if (match?.[1]) return match[1];
} catch {
return null;
}
return null;
}
function parseTweetId(input: string): string | null {
const trimmed = input.trim();
if (!trimmed) return null;
if (/^\d+$/.test(trimmed)) return trimmed;
try {
const parsed = new URL(trimmed);
const match = parsed.pathname.match(/\/status(?:es)?\/(\d+)/);
if (match?.[1]) return match[1];
} catch {
return null;
}
return null;
}
function parseTweetUsername(input: string): string | null {
const trimmed = input.trim();
if (!trimmed) return null;
try {
const parsed = new URL(trimmed);
const match = parsed.pathname.match(/^\/([^/]+)\/status(?:es)?\/\d+/);
if (match?.[1]) return match[1];
} catch {
return null;
}
return null;
}
function sanitizeSlug(input: string): string {
return input
.trim()
.replace(/^@/, "")
.replace(/[^a-zA-Z0-9_-]+/g, "-")
.replace(/-+/g, "-")
.replace(/^[-_]+|[-_]+$/g, "")
.slice(0, 120);
}
function formatBackupTimestamp(date: Date = new Date()): string {
const pad2 = (n: number) => String(n).padStart(2, "0");
return `${date.getFullYear()}${pad2(date.getMonth() + 1)}${pad2(date.getDate())}-${pad2(date.getHours())}${pad2(
date.getMinutes()
)}${pad2(date.getSeconds())}`;
}
async function backupDirIfExists(dir: string, log: (message: string) => void): Promise<void> {
try {
if (!fs.existsSync(dir)) return;
const stat = fs.statSync(dir);
if (!stat.isDirectory()) return;
const backup = `${dir}-backup-${formatBackupTimestamp()}`;
await rename(dir, backup);
log(`[x-to-markdown] Existing directory moved to: ${backup}`);
} catch (error) {
throw new Error(
`Failed to backup existing directory (${dir}): ${error instanceof Error ? error.message : String(error ?? "")}`
);
}
}
function resolveDefaultOutputDir(slug: string): string {
return path.resolve(process.cwd(), "x-to-markdown", slug);
}
async function resolveOutputPath(
normalizedUrl: string,
kind: "tweet" | "article",
argsOutput: string | null,
log: (message: string) => void
): Promise<{ outputDir: string; markdownPath: string; slug: string }> {
const articleId = kind === "article" ? parseArticleId(normalizedUrl) : null;
const tweetId = kind === "tweet" ? parseTweetId(normalizedUrl) : null;
const username = kind === "tweet" ? parseTweetUsername(normalizedUrl) : null;
const userSlug = username ? sanitizeSlug(username) : null;
const idPart = articleId ?? tweetId ?? String(Date.now());
const slug = userSlug ?? idPart;
const defaultFileName = kind === "article" ? `${idPart}.md` : `${idPart}.md`;
if (argsOutput) {
const wantsDir = argsOutput.endsWith("/") || argsOutput.endsWith("\\");
const resolved = path.resolve(argsOutput);
try {
if (wantsDir || (fs.existsSync(resolved) && fs.statSync(resolved).isDirectory())) {
const outputDir = path.join(resolved, slug);
await backupDirIfExists(outputDir, log);
await mkdir(outputDir, { recursive: true });
return { outputDir, markdownPath: path.join(outputDir, defaultFileName), slug };
}
} catch {
// treat as file path
}
const outputDir = path.dirname(resolved);
await mkdir(outputDir, { recursive: true });
return { outputDir, markdownPath: resolved, slug };
}
const outputDir = resolveDefaultOutputDir(slug);
await backupDirIfExists(outputDir, log);
await mkdir(outputDir, { recursive: true });
return { outputDir, markdownPath: path.join(outputDir, defaultFileName), slug };
}
function formatMetaMarkdown(meta: Record<string, string | number | null | undefined>): string {
const lines = ["---"];
for (const [key, value] of Object.entries(meta)) {
if (value === undefined || value === null || value === "") continue;
if (typeof value === "number") {
lines.push(`${key}: ${value}`);
} else {
lines.push(`${key}: ${JSON.stringify(value)}`);
}
}
lines.push("---");
return lines.join("\n");
}
async function promptYesNo(question: string): Promise<boolean> {
if (!process.stdin.isTTY) return false;
const rl = readline.createInterface({
input: process.stdin,
output: process.stderr,
});
try {
const answer = await new Promise<string>((resolve) => rl.question(question, resolve));
const normalized = answer.trim().toLowerCase();
return normalized === "y" || normalized === "yes";
} finally {
rl.close();
}
}
function isValidConsent(value: unknown): value is ConsentRecord {
if (!value || typeof value !== "object") return false;
const record = value as Partial<ConsentRecord>;
return (
record.accepted === true &&
record.disclaimerVersion === DISCLAIMER_VERSION &&
typeof record.acceptedAt === "string" &&
record.acceptedAt.length > 0
);
}
async function ensureConsent(log: (message: string) => void): Promise<void> {
const consentPath = resolveXToMarkdownConsentPath();
try {
if (fs.existsSync(consentPath) && fs.statSync(consentPath).isFile()) {
const raw = await readFile(consentPath, "utf8");
const parsed = JSON.parse(raw) as unknown;
if (isValidConsent(parsed)) {
log(
`⚠️ Warning: Using reverse-engineered X API (not official). Accepted on: ${(parsed as ConsentRecord).acceptedAt}`
);
return;
}
}
} catch {
// fall through to prompt
}
log(`⚠️ DISCLAIMER
This tool uses a reverse-engineered X (Twitter) API, NOT an official API.
Risks:
- May break without notice if X changes their API
- No official support or guarantees
- Account restrictions possible if API usage detected
- Use at your own risk
`);
if (!process.stdin.isTTY) {
throw new Error(
`Consent required. Run in a TTY or create ${consentPath} with accepted: true and disclaimerVersion: ${DISCLAIMER_VERSION}`
);
}
const accepted = await promptYesNo("Do you accept these terms and wish to continue? (y/N): ");
if (!accepted) {
throw new Error("User declined the disclaimer. Exiting.");
}
await mkdir(path.dirname(consentPath), { recursive: true });
const payload: ConsentRecord = {
version: 1,
accepted: true,
acceptedAt: new Date().toISOString(),
disclaimerVersion: DISCLAIMER_VERSION,
};
await writeFile(consentPath, JSON.stringify(payload, null, 2), "utf8");
log(`[x-to-markdown] Consent saved to: ${consentPath}`);
}
async function convertArticleToMarkdown(
inputUrl: string,
articleId: string,
log: (message: string) => void
): Promise<string> {
log("[x-to-markdown] Loading cookies...");
const cookieMap = await loadXCookies(log);
if (!hasRequiredXCookies(cookieMap)) {
throw new Error("Missing auth cookies. Provide X_AUTH_TOKEN and X_CT0 or log in via Chrome.");
}
log(`[x-to-markdown] Fetching article ${articleId}...`);
const article = await fetchXArticle(articleId, cookieMap, false);
const body = formatArticleMarkdown(article).trimEnd();
const title = typeof (article as any)?.title === "string" ? String((article as any).title).trim() : "";
const meta = formatMetaMarkdown({
url: `https://x.com/i/article/${articleId}`,
requested_url: inputUrl,
title: title || null,
});
return [meta, body].filter(Boolean).join("\n\n").trimEnd();
}
async function main(): Promise<void> {
const args = parseArgs(process.argv.slice(2));
if (args.help) printUsage(0);
if (!args.login && !args.url) printUsage(1);
const log = (message: string) => console.error(message);
await ensureConsent(log);
if (args.login) {
log("[x-to-markdown] Refreshing cookies via browser login...");
const cookieMap = await refreshXCookies(log);
if (!hasRequiredXCookies(cookieMap)) {
throw new Error("Missing auth cookies after login. Please ensure you are logged in to X.");
}
log("[x-to-markdown] Cookies refreshed.");
return;
}
const normalizedUrl = normalizeInputUrl(args.url ?? "");
const articleId = parseArticleId(normalizedUrl);
const tweetId = parseTweetId(normalizedUrl);
if (!articleId && !tweetId) {
throw new Error("Invalid X url. Examples: https://x.com/<user>/status/<id> or https://x.com/i/article/<id>");
}
const kind = articleId ? ("article" as const) : ("tweet" as const);
const { outputDir, markdownPath, slug } = await resolveOutputPath(normalizedUrl, kind, args.output, log);
const markdown =
kind === "article" && articleId
? await convertArticleToMarkdown(normalizedUrl, articleId, log)
: await tweetToMarkdown(normalizedUrl, { log });
await writeFile(markdownPath, markdown, "utf8");
log(`[x-to-markdown] Saved: ${markdownPath}`);
if (args.json) {
console.log(
JSON.stringify(
{
url: articleId ? `https://x.com/i/article/${articleId}` : normalizedUrl,
requested_url: normalizedUrl,
type: kind,
slug,
outputDir,
markdownPath,
},
null,
2
)
);
} else {
console.log(markdownPath);
}
}
await main().catch((error) => {
console.error(error instanceof Error ? error.message : String(error ?? ""));
process.exit(1);
});
@@ -0,0 +1,306 @@
import type {
ArticleBlock,
ArticleContentState,
ArticleEntity,
ArticleMediaInfo,
} from "./types.js";
function coerceArticleEntity(value: unknown): ArticleEntity | null {
if (!value || typeof value !== "object") return null;
const candidate = value as ArticleEntity;
if (
typeof candidate.title === "string" ||
typeof candidate.plain_text === "string" ||
typeof candidate.preview_text === "string" ||
candidate.content_state
) {
return candidate;
}
return null;
}
function escapeMarkdownAlt(text: string): string {
return text.replace(/[\[\]]/g, "\\$&");
}
function normalizeCaption(caption?: string): string {
const trimmed = caption?.trim();
if (!trimmed) return "";
return trimmed.replace(/\s+/g, " ");
}
function resolveMediaUrl(info?: ArticleMediaInfo): string | undefined {
if (!info) return undefined;
if (info.original_img_url) return info.original_img_url;
if (info.preview_image?.original_img_url) return info.preview_image.original_img_url;
const variants = info.variants ?? [];
const mp4 = variants
.filter((variant) => variant?.content_type?.includes("video"))
.sort((a, b) => (b.bit_rate ?? 0) - (a.bit_rate ?? 0))[0];
return mp4?.url ?? variants[0]?.url;
}
function buildMediaById(article: ArticleEntity): Map<string, string> {
const map = new Map<string, string>();
for (const entity of article.media_entities ?? []) {
if (!entity?.media_id) continue;
const url = resolveMediaUrl(entity.media_info);
if (url) {
map.set(entity.media_id, url);
}
}
return map;
}
function collectMediaUrls(
article: ArticleEntity,
usedUrls: Set<string>,
excludeUrl?: string
): string[] {
const urls: string[] = [];
const addUrl = (url?: string) => {
if (!url) return;
if (excludeUrl && url === excludeUrl) {
usedUrls.add(url);
return;
}
if (usedUrls.has(url)) return;
usedUrls.add(url);
urls.push(url);
};
for (const entity of article.media_entities ?? []) {
addUrl(resolveMediaUrl(entity?.media_info));
}
return urls;
}
function resolveEntityMediaLines(
entityKey: number | undefined,
entityMap: ArticleContentState["entityMap"] | undefined,
mediaById: Map<string, string>,
usedUrls: Set<string>
): string[] {
if (entityKey === undefined || !entityMap) return [];
const entry = entityMap[String(entityKey)];
const value = entry?.value;
if (!value) return [];
const type = value.type;
if (type !== "MEDIA" && type !== "IMAGE") return [];
const caption = normalizeCaption(value.data?.caption);
const altText = caption ? escapeMarkdownAlt(caption) : "";
const lines: string[] = [];
const mediaItems = value.data?.mediaItems ?? [];
for (const item of mediaItems) {
const mediaId =
typeof item?.mediaId === "string"
? item.mediaId
: typeof item?.media_id === "string"
? item.media_id
: undefined;
const url = mediaId ? mediaById.get(mediaId) : undefined;
if (url && !usedUrls.has(url)) {
usedUrls.add(url);
lines.push(`![${altText}](${url})`);
}
}
const fallbackUrl = typeof value.data?.url === "string" ? value.data.url : undefined;
if (fallbackUrl && !usedUrls.has(fallbackUrl)) {
usedUrls.add(fallbackUrl);
lines.push(`![${altText}](${fallbackUrl})`);
}
return lines;
}
function renderContentBlocks(
blocks: ArticleBlock[],
entityMap: ArticleContentState["entityMap"] | undefined,
mediaById: Map<string, string>,
usedUrls: Set<string>
): string[] {
const lines: string[] = [];
let previousKind: "list" | "quote" | "heading" | "text" | "code" | "media" | null = null;
let listKind: "ordered" | "unordered" | null = null;
let orderedIndex = 0;
let inCodeBlock = false;
const pushBlock = (
blockLines: string[],
kind: "list" | "quote" | "heading" | "text" | "media"
) => {
if (blockLines.length === 0) return;
if (
lines.length > 0 &&
previousKind &&
!(previousKind === kind && (kind === "list" || kind === "quote" || kind === "media"))
) {
lines.push("");
}
lines.push(...blockLines);
previousKind = kind;
};
const collectMediaLines = (block: ArticleBlock): string[] => {
const ranges = Array.isArray(block.entityRanges) ? block.entityRanges : [];
const mediaLines: string[] = [];
for (const range of ranges) {
if (typeof range?.key !== "number") continue;
mediaLines.push(...resolveEntityMediaLines(range.key, entityMap, mediaById, usedUrls));
}
return mediaLines;
};
for (const block of blocks) {
const type = typeof block?.type === "string" ? block.type : "unstyled";
const text = typeof block?.text === "string" ? block.text : "";
if (type === "code-block") {
if (!inCodeBlock) {
if (lines.length > 0) {
lines.push("");
}
lines.push("```");
inCodeBlock = true;
}
lines.push(text);
previousKind = "code";
listKind = null;
orderedIndex = 0;
continue;
}
if (type === "atomic") {
if (inCodeBlock) {
lines.push("```");
inCodeBlock = false;
previousKind = "code";
}
listKind = null;
orderedIndex = 0;
const mediaLines = collectMediaLines(block);
if (mediaLines.length > 0) {
pushBlock(mediaLines, "media");
}
continue;
}
if (inCodeBlock) {
lines.push("```");
inCodeBlock = false;
previousKind = "code";
}
if (type === "unordered-list-item") {
listKind = "unordered";
orderedIndex = 0;
pushBlock([`- ${text}`], "list");
continue;
}
if (type === "ordered-list-item") {
if (listKind !== "ordered") {
orderedIndex = 0;
}
listKind = "ordered";
orderedIndex += 1;
pushBlock([`${orderedIndex}. ${text}`], "list");
continue;
}
listKind = null;
orderedIndex = 0;
switch (type) {
case "header-one":
pushBlock([`# ${text}`], "heading");
break;
case "header-two":
pushBlock([`## ${text}`], "heading");
break;
case "header-three":
pushBlock([`### ${text}`], "heading");
break;
case "header-four":
pushBlock([`#### ${text}`], "heading");
break;
case "header-five":
pushBlock([`##### ${text}`], "heading");
break;
case "header-six":
pushBlock([`###### ${text}`], "heading");
break;
case "blockquote": {
const quoteLines = text.length > 0 ? text.split("\n") : [""];
pushBlock(quoteLines.map((line) => `> ${line}`), "quote");
break;
}
default:
pushBlock([text], "text");
break;
}
const trailingMediaLines = collectMediaLines(block);
if (trailingMediaLines.length > 0) {
pushBlock(trailingMediaLines, "media");
}
}
if (inCodeBlock) {
lines.push("```");
}
return lines;
}
export function formatArticleMarkdown(article: unknown): string {
const candidate = coerceArticleEntity(article);
if (!candidate) {
return `\`\`\`json\n${JSON.stringify(article, null, 2)}\n\`\`\``;
}
const lines: string[] = [];
const usedUrls = new Set<string>();
const mediaById = buildMediaById(candidate);
const title = typeof candidate.title === "string" ? candidate.title.trim() : "";
if (title) {
lines.push(`# ${title}`);
}
const coverUrl = resolveMediaUrl(candidate.cover_media?.media_info);
if (coverUrl) {
if (lines.length > 0) lines.push("");
lines.push(`![](${coverUrl})`);
usedUrls.add(coverUrl);
}
const blocks = candidate.content_state?.blocks;
const entityMap = candidate.content_state?.entityMap;
if (Array.isArray(blocks) && blocks.length > 0) {
const rendered = renderContentBlocks(blocks, entityMap, mediaById, usedUrls);
if (rendered.length > 0) {
if (lines.length > 0) lines.push("");
lines.push(...rendered);
}
} else if (typeof candidate.plain_text === "string") {
if (lines.length > 0) lines.push("");
lines.push(candidate.plain_text.trim());
} else if (typeof candidate.preview_text === "string") {
if (lines.length > 0) lines.push("");
lines.push(candidate.preview_text.trim());
}
const mediaUrls = collectMediaUrls(candidate, usedUrls, coverUrl);
if (mediaUrls.length > 0) {
lines.push("", "## Media", "");
for (const url of mediaUrls) {
lines.push(`![](${url})`);
}
}
return lines.join("\n").trimEnd();
}
@@ -0,0 +1,41 @@
import os from "node:os";
import path from "node:path";
import process from "node:process";
const APP_DATA_DIR = "baoyu-skills";
const X_TO_MARKDOWN_DATA_DIR = "x-to-markdown";
const COOKIE_FILE_NAME = "cookies.json";
const PROFILE_DIR_NAME = "chrome-profile";
const CONSENT_FILE_NAME = "consent.json";
export function resolveUserDataRoot(): string {
if (process.platform === "win32") {
return process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming");
}
if (process.platform === "darwin") {
return path.join(os.homedir(), "Library", "Application Support");
}
return process.env.XDG_DATA_HOME ?? path.join(os.homedir(), ".local", "share");
}
export function resolveXToMarkdownDataDir(): string {
const override = process.env.X_DATA_DIR?.trim();
if (override) return path.resolve(override);
return path.join(resolveUserDataRoot(), APP_DATA_DIR, X_TO_MARKDOWN_DATA_DIR);
}
export function resolveXToMarkdownCookiePath(): string {
const override = process.env.X_COOKIE_PATH?.trim();
if (override) return path.resolve(override);
return path.join(resolveXToMarkdownDataDir(), COOKIE_FILE_NAME);
}
export function resolveXToMarkdownChromeProfileDir(): string {
const override = process.env.X_CHROME_PROFILE_DIR?.trim();
if (override) return path.resolve(override);
return path.join(resolveXToMarkdownDataDir(), PROFILE_DIR_NAME);
}
export function resolveXToMarkdownConsentPath(): string {
return path.join(resolveXToMarkdownDataDir(), CONSENT_FILE_NAME);
}
@@ -0,0 +1,295 @@
type ThreadLike = {
requestedId?: string;
rootId?: string;
tweets?: unknown[];
totalTweets?: number;
user?: any;
};
type TweetPhoto = {
src: string;
alt?: string;
};
type TweetVideo = {
url: string;
poster?: string;
alt?: string;
type?: string;
};
export type ThreadTweetsMarkdownOptions = {
username?: string;
headingLevel?: number;
startIndex?: number;
includeTweetUrls?: boolean;
};
export type ThreadMarkdownOptions = ThreadTweetsMarkdownOptions & {
includeHeader?: boolean;
title?: string;
sourceUrl?: string;
};
function coerceThread(value: unknown): ThreadLike | null {
if (!value || typeof value !== "object") return null;
const candidate = value as ThreadLike;
if (!Array.isArray(candidate.tweets)) return null;
return candidate;
}
function escapeMarkdownAlt(text: string): string {
return text.replace(/[\[\]]/g, "\\$&");
}
function normalizeAlt(text?: string | null): string {
const trimmed = text?.trim();
if (!trimmed) return "";
return trimmed.replace(/\s+/g, " ");
}
function parseTweetText(tweet: any): string {
const noteText = tweet?.note_tweet?.note_tweet_results?.result?.text;
const legacyText = tweet?.legacy?.full_text ?? tweet?.legacy?.text ?? "";
return (noteText ?? legacyText ?? "").trim();
}
function parsePhotos(tweet: any): TweetPhoto[] {
const media = tweet?.legacy?.extended_entities?.media ?? [];
return media
.reduce((acc: TweetPhoto[], item: any) => {
if (item?.type !== "photo") {
return acc;
}
const src = item.media_url_https ?? item.media_url;
if (!src) {
return acc;
}
const alt = normalizeAlt(item.ext_alt_text);
acc.push({ src, alt });
return acc;
}, [])
.filter((photo) => Boolean(photo.src));
}
function parseVideos(tweet: any): TweetVideo[] {
const media = tweet?.legacy?.extended_entities?.media ?? [];
return media
.reduce((acc: TweetVideo[], item: any) => {
if (!item?.type || !["animated_gif", "video"].includes(item.type)) {
return acc;
}
const variants = item?.video_info?.variants ?? [];
const sources = variants
.map((variant: any) => ({
contentType: variant?.content_type,
url: variant?.url,
bitrate: variant?.bitrate ?? 0,
}))
.filter((variant: any) => Boolean(variant.url));
const videoSources = sources.filter((variant: any) =>
String(variant.contentType ?? "").includes("video")
);
const sorted = (videoSources.length > 0 ? videoSources : sources).sort(
(a: any, b: any) => (b.bitrate ?? 0) - (a.bitrate ?? 0)
);
const best = sorted[0];
if (!best?.url) {
return acc;
}
const alt = normalizeAlt(item.ext_alt_text);
acc.push({
url: best.url,
poster: item.media_url_https ?? item.media_url ?? undefined,
alt,
type: item.type,
});
return acc;
}, [])
.filter((video) => Boolean(video.url));
}
function unwrapTweetResult(result: any): any {
if (!result) return null;
if (result.__typename === "TweetWithVisibilityResults" && result.tweet) {
return result.tweet;
}
return result;
}
function resolveTweetId(tweet: any): string | undefined {
return tweet?.legacy?.id_str ?? tweet?.rest_id;
}
function buildTweetUrl(username: string | undefined, tweetId: string | undefined): string | null {
if (!tweetId) return null;
if (username) {
return `https://x.com/${username}/status/${tweetId}`;
}
return `https://x.com/i/web/status/${tweetId}`;
}
function formatTweetMarkdown(
tweet: any,
index: number,
options: ThreadTweetsMarkdownOptions
): string[] {
const headingLevel = options.headingLevel ?? 2;
const includeTweetUrls = options.includeTweetUrls ?? true;
const headingPrefix = "#".repeat(Math.min(Math.max(headingLevel, 1), 6));
const tweetId = resolveTweetId(tweet);
const tweetUrl = includeTweetUrls ? buildTweetUrl(options.username, tweetId) : null;
const lines: string[] = [];
lines.push(`${headingPrefix} ${index}`);
if (tweetUrl) {
lines.push(tweetUrl);
}
lines.push("");
const text = parseTweetText(tweet);
const photos = parsePhotos(tweet);
const videos = parseVideos(tweet);
const quoted = unwrapTweetResult(tweet?.quoted_status_result?.result);
const bodyLines: string[] = [];
if (text) {
bodyLines.push(...text.split(/\r?\n/));
}
const quotedLines = formatQuotedTweetMarkdown(quoted);
if (quotedLines.length > 0) {
if (bodyLines.length > 0) bodyLines.push("");
bodyLines.push(...quotedLines);
}
const photoLines = photos.map((photo) => {
const alt = photo.alt ? escapeMarkdownAlt(photo.alt) : "";
return `![${alt}](${photo.src})`;
});
if (photoLines.length > 0) {
if (bodyLines.length > 0) bodyLines.push("");
bodyLines.push(...photoLines);
}
const videoLines: string[] = [];
for (const video of videos) {
if (video.poster) {
const alt = video.alt ? escapeMarkdownAlt(video.alt) : "video";
videoLines.push(`![${alt}](${video.poster})`);
}
videoLines.push(`[${video.type ?? "video"}](${video.url})`);
}
if (videoLines.length > 0) {
if (bodyLines.length > 0) bodyLines.push("");
bodyLines.push(...videoLines);
}
if (bodyLines.length === 0) {
bodyLines.push("_No text or media._");
}
lines.push(...bodyLines);
return lines;
}
function formatQuotedTweetMarkdown(quoted: any): string[] {
if (!quoted) return [];
const quotedUser = quoted?.core?.user_results?.result?.legacy;
const quotedUsername = quotedUser?.screen_name;
const quotedName = quotedUser?.name;
const quotedAuthor =
quotedUsername && quotedName
? `${quotedName} (@${quotedUsername})`
: quotedUsername
? `@${quotedUsername}`
: quotedName ?? "Unknown";
const quotedId = resolveTweetId(quoted);
const quotedUrl =
buildTweetUrl(quotedUsername, quotedId) ??
(quotedId ? `https://x.com/i/web/status/${quotedId}` : "unavailable");
const quotedText = parseTweetText(quoted);
const lines: string[] = [];
lines.push(`Author: ${quotedAuthor}`);
lines.push(`URL: ${quotedUrl}`);
if (quotedText) {
lines.push("", ...quotedText.split(/\r?\n/));
} else {
lines.push("", "(no content)");
}
return lines.map((line) => `> ${line}`.trimEnd());
}
export function formatThreadTweetsMarkdown(
tweets: unknown[],
options: ThreadTweetsMarkdownOptions = {}
): string {
const lines: string[] = [];
const startIndex = options.startIndex ?? 1;
if (!Array.isArray(tweets) || tweets.length === 0) {
return "";
}
tweets.forEach((tweet, index) => {
if (lines.length > 0) {
lines.push("");
}
lines.push(...formatTweetMarkdown(tweet, startIndex + index, options));
});
return lines.join("\n").trimEnd();
}
export function formatThreadMarkdown(
thread: unknown,
options: ThreadMarkdownOptions = {}
): string {
const candidate = coerceThread(thread);
if (!candidate) {
return `\`\`\`json\n${JSON.stringify(thread, null, 2)}\n\`\`\``;
}
const tweets = candidate.tweets ?? [];
const firstTweet = tweets[0] as any;
const user = candidate.user ?? firstTweet?.core?.user_results?.result?.legacy;
const username = user?.screen_name;
const name = user?.name;
const includeHeader = options.includeHeader ?? true;
const lines: string[] = [];
if (includeHeader) {
if (options.title) {
lines.push(`# ${options.title}`);
} else if (username) {
lines.push(`# Thread by @${username}${name ? ` (${name})` : ""}`);
} else {
lines.push("# Thread");
}
const sourceUrl = options.sourceUrl ?? buildTweetUrl(username, candidate.rootId ?? candidate.requestedId);
if (sourceUrl) {
lines.push(`Source: ${sourceUrl}`);
}
if (typeof candidate.totalTweets === "number") {
lines.push(`Tweets: ${candidate.totalTweets}`);
}
}
const tweetMarkdown = formatThreadTweetsMarkdown(tweets, {
...options,
username,
});
if (tweetMarkdown) {
if (lines.length > 0) {
lines.push("");
}
lines.push(tweetMarkdown);
}
return lines.join("\n").trimEnd();
}
@@ -0,0 +1,311 @@
import { fetchTweetDetail } from "./graphql.js";
type TweetEntry = {
tweet: any;
user?: any;
};
type ParsedEntries = {
entries: TweetEntry[];
moreCursor?: string;
topCursor?: string;
bottomCursor?: string;
};
type ThreadResult = {
requestedId: string;
rootId: string;
tweets: any[];
totalTweets: number;
user?: any;
responses?: unknown[];
};
function unwrapTweetResult(result: any): any {
if (!result) return null;
if (result.__typename === "TweetWithVisibilityResults" && result.tweet) {
return result.tweet;
}
return result;
}
function extractTweetEntry(itemContent: any): TweetEntry | null {
const result = itemContent?.tweet_results?.result;
if (!result) return null;
const resolved = unwrapTweetResult(result?.tweet ?? result);
if (!resolved) return null;
const user = resolved?.core?.user_results?.result?.legacy;
return { tweet: resolved, user };
}
function parseInstruction(instruction?: any): ParsedEntries {
const { entries: entities, moduleItems } = instruction || {};
const entries: TweetEntry[] = [];
let moreCursor: string | undefined;
let topCursor: string | undefined;
let bottomCursor: string | undefined;
const parseItems = (items: any[]) => {
items?.forEach((item) => {
const itemContent = item?.item?.itemContent ?? item?.itemContent;
if (!itemContent) {
return;
}
if (
itemContent.cursorType &&
["ShowMore", "ShowMoreThreads"].includes(itemContent.cursorType) &&
itemContent.itemType === "TimelineTimelineCursor"
) {
moreCursor = itemContent.value;
return;
}
const entry = extractTweetEntry(itemContent);
if (entry) {
entries.push(entry);
}
});
};
if (moduleItems) {
parseItems(moduleItems);
}
for (const entity of entities ?? []) {
if (entity?.content?.clientEventInfo?.component === "you_might_also_like") {
continue;
}
const { itemContent, items, cursorType, entryType, value } = entity?.content ?? {};
if (cursorType === "Bottom" && entryType === "TimelineTimelineCursor") {
bottomCursor = value;
}
if (
itemContent?.cursorType === "Bottom" &&
itemContent?.itemType === "TimelineTimelineCursor"
) {
bottomCursor = bottomCursor ?? itemContent?.value;
}
if (cursorType === "Top" && entryType === "TimelineTimelineCursor") {
topCursor = topCursor ?? value;
}
if (itemContent) {
const entry = extractTweetEntry(itemContent);
if (entry) {
entries.push(entry);
}
if (
itemContent.cursorType &&
["ShowMore", "ShowMoreThreads"].includes(itemContent.cursorType) &&
itemContent.itemType === "TimelineTimelineCursor"
) {
moreCursor = moreCursor ?? itemContent.value;
}
if (itemContent.cursorType === "Top" && itemContent.itemType === "TimelineTimelineCursor") {
topCursor = topCursor ?? itemContent.value;
}
}
if (items) {
parseItems(items);
}
}
return { entries, moreCursor, topCursor, bottomCursor };
}
function parseTweetsAndToken(response: any): ParsedEntries {
const instruction =
response?.data?.threaded_conversation_with_injections_v2?.instructions?.find(
(ins: any) => ins?.type === "TimelineAddEntries" || ins?.type === "TimelineAddToModule"
) ??
response?.data?.threaded_conversation_with_injections?.instructions?.find(
(ins: any) => ins?.type === "TimelineAddEntries" || ins?.type === "TimelineAddToModule"
);
return parseInstruction(instruction);
}
function toTimestamp(value: string | undefined): number {
if (!value) return 0;
const parsed = Date.parse(value);
return Number.isNaN(parsed) ? 0 : parsed;
}
export async function fetchTweetThread(
tweetId: string,
cookieMap: Record<string, string>,
includeResponses = false
): Promise<ThreadResult | null> {
const responses: unknown[] = [];
const res = await fetchTweetDetail(tweetId, cookieMap);
if (includeResponses) {
responses.push(res);
}
let { entries, moreCursor, topCursor, bottomCursor } = parseTweetsAndToken(res);
if (!entries.length) {
const errorMessage = res?.errors?.[0]?.message;
if (errorMessage) {
throw new Error(errorMessage);
}
return null;
}
let allEntries = entries.slice();
const root = allEntries.find((entry) => entry.tweet?.legacy?.id_str === tweetId);
if (!root) {
throw new Error("Can not fetch the root tweet");
}
let rootEntry = root.tweet.legacy;
const isSameThread = (entry: TweetEntry) => {
const tweet = entry.tweet?.legacy;
if (!tweet) return false;
return (
tweet.user_id_str === rootEntry.user_id_str &&
tweet.conversation_id_str === rootEntry.conversation_id_str &&
(tweet.id_str === rootEntry.id_str ||
tweet.in_reply_to_user_id_str === rootEntry.user_id_str ||
tweet.in_reply_to_status_id_str === rootEntry.conversation_id_str ||
!tweet.in_reply_to_user_id_str)
);
};
const inThread = (items: TweetEntry[]) => items.some(isSameThread);
let hasThread = inThread(entries);
let maxRequestCount = 1000;
let topHasThread = true;
while (topCursor && topHasThread && maxRequestCount > 0) {
const newRes = await fetchTweetDetail(tweetId, cookieMap, topCursor);
if (includeResponses) {
responses.push(newRes);
}
const parsed = parseTweetsAndToken(newRes);
topHasThread = inThread(parsed.entries);
topCursor = parsed.topCursor;
allEntries = parsed.entries.concat(allEntries);
maxRequestCount--;
}
async function checkMoreTweets(focalId: string) {
while (moreCursor && hasThread && maxRequestCount > 0) {
const newRes = await fetchTweetDetail(focalId, cookieMap, moreCursor);
if (includeResponses) {
responses.push(newRes);
}
const parsed = parseTweetsAndToken(newRes);
moreCursor = parsed.moreCursor;
bottomCursor = bottomCursor ?? parsed.bottomCursor;
hasThread = inThread(parsed.entries);
allEntries = allEntries.concat(parsed.entries);
maxRequestCount--;
}
if (bottomCursor) {
const newRes = await fetchTweetDetail(focalId, cookieMap, bottomCursor);
if (includeResponses) {
responses.push(newRes);
}
const parsed = parseTweetsAndToken(newRes);
allEntries = allEntries.concat(parsed.entries);
bottomCursor = undefined;
}
}
await checkMoreTweets(tweetId);
const allThreadEntries = allEntries.filter(
(entry) => entry.tweet?.legacy?.id_str === tweetId || isSameThread(entry)
);
const lastEntity = allThreadEntries[allThreadEntries.length - 1];
if (lastEntity?.tweet?.legacy?.id_str) {
const lastRes = await fetchTweetDetail(lastEntity.tweet.legacy.id_str, cookieMap);
if (includeResponses) {
responses.push(lastRes);
}
const parsed = parseTweetsAndToken(lastRes);
hasThread = inThread(parsed.entries);
allEntries = allEntries.concat(parsed.entries);
moreCursor = parsed.moreCursor;
bottomCursor = parsed.bottomCursor;
maxRequestCount--;
await checkMoreTweets(lastEntity.tweet.legacy.id_str);
}
const distinctEntries: TweetEntry[] = [];
const entriesMap = allEntries.reduce((acc, entry) => {
const id = entry.tweet?.legacy?.id_str ?? entry.tweet?.rest_id;
if (id && !acc.has(id)) {
distinctEntries.push(entry);
acc.set(id, entry);
}
return acc;
}, new Map<string, TweetEntry>());
allEntries = distinctEntries;
while (rootEntry.in_reply_to_status_id_str) {
const parent = entriesMap.get(rootEntry.in_reply_to_status_id_str)?.tweet?.legacy;
if (
parent &&
parent.user_id_str === rootEntry.user_id_str &&
parent.conversation_id_str === rootEntry.conversation_id_str &&
parent.id_str !== rootEntry.id_str
) {
rootEntry = parent;
} else {
break;
}
}
allEntries = allEntries.sort((a, b) => {
const aTime = toTimestamp(a.tweet?.legacy?.created_at);
const bTime = toTimestamp(b.tweet?.legacy?.created_at);
return aTime - bTime;
});
const rootIndex = allEntries.findIndex(
(entry) => entry.tweet?.legacy?.id_str === rootEntry.id_str
);
if (rootIndex > 0) {
allEntries = allEntries.slice(rootIndex);
}
const threadEntries = allEntries.filter(
(entry) => entry.tweet?.legacy?.id_str === tweetId || isSameThread(entry)
);
if (!threadEntries.length) {
return null;
}
const tweets = threadEntries.map((entry) => entry.tweet);
const user = threadEntries[0].user ?? threadEntries[0].tweet?.core?.user_results?.result?.legacy;
const result: ThreadResult = {
requestedId: tweetId,
rootId: rootEntry.id_str ?? tweetId,
tweets,
totalTweets: tweets.length,
user,
};
if (includeResponses) {
result.responses = responses;
}
return result;
}
@@ -0,0 +1,96 @@
import { fetchXArticle } from "./graphql.js";
import type { ArticleEntity } from "./types.js";
function coerceArticleEntity(value: unknown): ArticleEntity | null {
if (!value || typeof value !== "object") return null;
const candidate = value as ArticleEntity;
if (
typeof candidate.title === "string" ||
typeof candidate.plain_text === "string" ||
typeof candidate.preview_text === "string" ||
candidate.content_state
) {
return candidate;
}
return null;
}
function hasArticleContent(article: ArticleEntity): boolean {
const blocks = article.content_state?.blocks;
if (Array.isArray(blocks) && blocks.length > 0) {
return true;
}
if (typeof article.plain_text === "string" && article.plain_text.trim()) {
return true;
}
if (typeof article.preview_text === "string" && article.preview_text.trim()) {
return true;
}
return false;
}
function parseArticleIdFromUrl(raw: string | undefined): string | null {
if (!raw) return null;
try {
const parsed = new URL(raw);
const match = parsed.pathname.match(/\/(?:i\/)?article\/(\d+)/);
if (match?.[1]) return match[1];
} catch {
return null;
}
return null;
}
function extractArticleIdFromUrls(urls: any[] | undefined): string | null {
if (!Array.isArray(urls)) return null;
for (const url of urls) {
const candidate =
url?.expanded_url ?? url?.url ?? (url?.display_url ? `https://${url.display_url}` : undefined);
const id = parseArticleIdFromUrl(candidate);
if (id) return id;
}
return null;
}
export function extractArticleEntityFromTweet(tweet: any): unknown | null {
return (
tweet?.article?.article_results?.result ??
tweet?.article?.result ??
tweet?.legacy?.article?.article_results?.result ??
tweet?.legacy?.article?.result ??
tweet?.article_results?.result ??
null
);
}
export function extractArticleIdFromTweet(tweet: any): string | null {
const embedded = extractArticleEntityFromTweet(tweet);
const embeddedArticle = embedded as { rest_id?: string } | null;
if (embeddedArticle?.rest_id) {
return embeddedArticle.rest_id;
}
const noteUrls = tweet?.note_tweet?.note_tweet_results?.result?.entity_set?.urls;
const legacyUrls = tweet?.legacy?.entities?.urls;
return extractArticleIdFromUrls(noteUrls) ?? extractArticleIdFromUrls(legacyUrls);
}
export async function resolveArticleEntityFromTweet(
tweet: any,
cookieMap: Record<string, string>
): Promise<unknown | null> {
if (!tweet) return null;
const embedded = extractArticleEntityFromTweet(tweet);
const embeddedArticle = coerceArticleEntity(embedded);
if (embeddedArticle && hasArticleContent(embeddedArticle)) {
return embedded;
}
const articleId = extractArticleIdFromTweet(tweet);
if (!articleId) {
return embedded ?? null;
}
const fetched = await fetchXArticle(articleId, cookieMap, false);
return fetched ?? embedded ?? null;
}
@@ -0,0 +1,190 @@
#!/usr/bin/env npx tsx
import * as path from "node:path";
import { fileURLToPath } from "node:url";
import { hasRequiredXCookies, loadXCookies } from "./cookies.js";
import { fetchTweetThread } from "./thread.js";
import { formatArticleMarkdown } from "./markdown.js";
import { formatThreadTweetsMarkdown } from "./thread-markdown.js";
import { resolveArticleEntityFromTweet } from "./tweet-article.js";
type TweetToMarkdownOptions = {
log?: (message: string) => void;
};
function parseArgs(): { url?: string } {
const args = process.argv.slice(2);
let url: string | undefined;
for (const arg of args) {
if (!arg.startsWith("-") && !url) {
url = arg;
}
}
return { url };
}
function normalizeInputUrl(input: string): string {
const trimmed = input.trim();
if (!trimmed) return "";
try {
return new URL(trimmed).toString();
} catch {
return trimmed;
}
}
function parseTweetId(input: string): string | null {
const trimmed = input.trim();
if (!trimmed) return null;
if (/^\d+$/.test(trimmed)) return trimmed;
try {
const parsed = new URL(trimmed);
const match = parsed.pathname.match(/\/status(?:es)?\/(\d+)/);
if (match?.[1]) return match[1];
} catch {
return null;
}
return null;
}
function buildTweetUrl(username: string | undefined, tweetId: string | undefined): string | null {
if (!tweetId) return null;
if (username) {
return `https://x.com/${username}/status/${tweetId}`;
}
return `https://x.com/i/web/status/${tweetId}`;
}
function formatMetaMarkdown(meta: Record<string, string | number | null | undefined>): string {
const lines = ["---"];
for (const [key, value] of Object.entries(meta)) {
if (value === undefined || value === null || value === "") continue;
if (typeof value === "number") {
lines.push(`${key}: ${value}`);
} else {
lines.push(`${key}: ${JSON.stringify(value)}`);
}
}
lines.push("---");
return lines.join("\n");
}
function extractTweetText(tweet: any): string {
const noteText = tweet?.note_tweet?.note_tweet_results?.result?.text;
const legacyText = tweet?.legacy?.full_text ?? tweet?.legacy?.text ?? "";
return (noteText ?? legacyText ?? "").trim();
}
function isOnlyUrl(text: string): boolean {
const trimmed = text.trim();
if (!trimmed) return true;
return /^https?:\/\/\S+$/.test(trimmed);
}
export async function tweetToMarkdown(
inputUrl: string,
options: TweetToMarkdownOptions = {}
): Promise<string> {
const normalizedUrl = normalizeInputUrl(inputUrl);
const tweetId = parseTweetId(normalizedUrl);
if (!tweetId) {
throw new Error("Invalid tweet url. Example: https://x.com/<user>/status/<tweet_id>");
}
const log = options.log ?? (() => {});
log("[tweet-to-markdown] Loading cookies...");
const cookieMap = await loadXCookies(log);
if (!hasRequiredXCookies(cookieMap)) {
throw new Error("Missing auth cookies. Provide X_AUTH_TOKEN and X_CT0 or log in via Chrome.");
}
log(`[tweet-to-markdown] Fetching thread for ${tweetId}...`);
const thread = await fetchTweetThread(tweetId, cookieMap);
if (!thread) {
throw new Error("Failed to fetch thread.");
}
const tweets = thread.tweets ?? [];
if (tweets.length === 0) {
throw new Error("No tweets found in thread.");
}
const firstTweet = tweets[0] as any;
const user = thread.user ?? firstTweet?.core?.user_results?.result?.legacy;
const username = user?.screen_name;
const name = user?.name;
const author =
username && name ? `${name} (@${username})` : username ? `@${username}` : name ?? null;
const authorUrl = username ? `https://x.com/${username}` : undefined;
const requestedUrl = normalizedUrl || buildTweetUrl(username, tweetId) || inputUrl.trim();
const rootUrl = buildTweetUrl(username, thread.rootId ?? tweetId) ?? requestedUrl;
const meta = formatMetaMarkdown({
url: rootUrl,
requested_url: requestedUrl,
author,
author_name: name ?? null,
author_username: username ?? null,
author_url: authorUrl ?? null,
tweet_count: thread.totalTweets ?? tweets.length,
});
const parts: string[] = [meta];
const articleEntity = await resolveArticleEntityFromTweet(firstTweet, cookieMap);
let remainingTweets = tweets;
if (articleEntity) {
const articleMarkdown = formatArticleMarkdown(articleEntity).trimEnd();
if (articleMarkdown) {
parts.push(articleMarkdown);
const firstTweetText = extractTweetText(firstTweet);
if (isOnlyUrl(firstTweetText)) {
remainingTweets = tweets.slice(1);
}
}
}
if (remainingTweets.length > 0) {
const hasArticle = parts.length > 1;
if (hasArticle) {
parts.push("## Thread");
}
const tweetMarkdown = formatThreadTweetsMarkdown(remainingTweets, {
username,
headingLevel: hasArticle ? 3 : 2,
startIndex: 1,
includeTweetUrls: true,
});
if (tweetMarkdown) {
parts.push(tweetMarkdown);
}
}
return parts.join("\n\n").trimEnd();
}
async function main() {
const { url } = parseArgs();
if (!url) {
console.error("Usage:");
console.error(" npx -y bun skills/baoyu-danger-x-to-markdown/scripts/tweet-to-markdown.ts <tweet url>");
process.exit(1);
}
const markdown = await tweetToMarkdown(url, { log: console.log });
console.log(markdown);
}
const isCliExecution =
process.argv[1] && fileURLToPath(import.meta.url) === path.resolve(process.argv[1]);
if (isCliExecution) {
main().catch((error) => {
console.error(error instanceof Error ? error.message : error);
process.exit(1);
});
}
@@ -0,0 +1,79 @@
export type CookieLike = {
name?: string;
value?: string;
domain?: string;
path?: string;
url?: string;
};
export type ArticleQueryInfo = {
queryId: string;
featureSwitches: string[];
fieldToggles: string[];
html: string;
};
export type ArticleEntityRange = {
key?: number;
offset?: number;
length?: number;
};
export type ArticleBlock = {
type?: string;
text?: string;
entityRanges?: ArticleEntityRange[];
};
export type ArticleEntityMapMediaItem = {
mediaId?: string;
media_id?: string;
localMediaId?: string;
};
export type ArticleEntityMapEntry = {
key?: string;
value?: {
type?: string;
mutability?: string;
data?: {
caption?: string;
mediaItems?: ArticleEntityMapMediaItem[];
url?: string;
};
};
};
export type ArticleContentState = {
blocks?: ArticleBlock[];
entityMap?: Record<string, ArticleEntityMapEntry>;
};
export type ArticleMediaInfo = {
__typename?: string;
original_img_url?: string;
preview_image?: {
original_img_url?: string;
};
variants?: Array<{
content_type?: string;
url?: string;
bit_rate?: number;
}>;
};
export type ArticleMediaEntity = {
media_id?: string;
media_info?: ArticleMediaInfo;
};
export type ArticleEntity = {
title?: string;
plain_text?: string;
preview_text?: string;
content_state?: ArticleContentState;
cover_media?: {
media_info?: ArticleMediaInfo;
};
media_entities?: ArticleMediaEntity[];
};
+23 -19
View File
@@ -95,36 +95,40 @@ This deck is designed for **reading and sharing**, not live presentation:
## File Management
### With Content Path
```
content-dir/
├── source-content.md
└── source-content/
└── slide-deck/
├── outline.md
├── prompts/
│ └── 01-slide-cover.md, 02-slide-{slug}.md, ...
├── 01-slide-cover.png, 02-slide-{slug}.png, ...
├── {topic-slug}.pptx
└── {topic-slug}.pdf
```
### Output Directory
Example: `/posts/ai-intro.md``/posts/ai-intro/slide-deck/`
Each session creates an independent directory named by content slug:
### Without Content Path (Pasted Content)
```
slide-deck/{topic-slug}/
├── source.md
├── source-{slug}.{ext} # Source files (text, images, etc.)
├── outline.md
├── outline-{style}.md # Style variant outlines
├── prompts/
├── *.png
│ └── 01-slide-cover.md, 02-slide-{slug}.md, ...
├── 01-slide-cover.png, 02-slide-{slug}.png, ...
├── {topic-slug}.pptx
└── {topic-slug}.pdf
```
### Directory Backup
**Slug Generation**:
1. Extract main topic from content (2-4 words, kebab-case)
2. Example: "Introduction to Machine Learning" → `intro-machine-learning`
If target directory exists, rename existing to `<dirname>-backup-YYYYMMDD-HHMMSS`
### Conflict Resolution
If `slide-deck/{topic-slug}/` already exists:
- Append timestamp: `{topic-slug}-YYYYMMDD-HHMMSS`
- Example: `intro-ml` exists → `intro-ml-20260118-143052`
### Source Files
Copy all sources with naming `source-{slug}.{ext}`:
- `source-article.md` (main text content)
- `source-diagram.png` (image from conversation)
- `source-data.xlsx` (additional file)
Multiple sources supported: text, images, files from conversation.
## Workflow
@@ -67,3 +67,4 @@ Clean sans-serif such as Inter, SF Pro, or Helvetica Neue. Medium weight for bod
## Best For
Product launches, marketing presentations, keynote speeches, brand showcases, investor pitches, high-stakes presentations
@@ -49,16 +49,6 @@ Clean geometric sans-serif for text box content. Readable at smaller sizes. Cons
- "KEY QUOTE:" box at bottom with core insight
- No title blocks, stamps, or watermarks in corners
## Text Box Types
| Label | Purpose |
|-------|---------|
| DEFINITION 定义: | Core concept definition from source |
| KEY INSIGHT 核心洞察: | Main analytical insight |
| EXAMPLE 案例: | Concrete example from content |
| KEY QUOTE 关键引文: | Important quote from source |
| CONTEXT 背景: | Contextual information |
## Style Rules
### Do
@@ -77,17 +67,6 @@ Clean geometric sans-serif for text box content. Readable at smaller sizes. Cons
- Use gradients or glossy effects
- Add slide numbers, footers, or logos
## Prompt Structure
1. Start with "Technical blueprint infographic."
2. Title format: "Title: '[中文标题]' in dark maroon ALL CAPS in brackets, with English subtitle below"
3. Describe main visual (isometric 3D or 2D technical illustration)
4. Describe background pattern (faded, thematic)
5. List 3-5 text boxes with specific content
6. Include bilingual callout labels
7. Specify paper texture and colors
8. End with: "Clean corners with no title blocks or stamps. 4K ultra-high resolution. Technical blueprint aesthetic."
## Best For
Technical explanations, concept breakdowns, academic presentations, knowledge documentation, research summaries, educational content with depth, bilingual audiences
+16 -8
View File
@@ -92,9 +92,11 @@ Detailed layout definitions: `references/layouts/<layout>.md`
## File Structure
Each session creates an independent directory named by content slug:
```
[target]/
├── source.md # Source content (if pasted)
xhs-images/{topic-slug}/
├── source-{slug}.{ext} # Source files (text, images, etc.)
├── analysis.md # Deep analysis results
├── outline-style-[slug].md # Variant A (e.g., outline-style-tech.md)
├── outline-style-[slug].md # Variant B (e.g., outline-style-notion.md)
@@ -109,13 +111,19 @@ Detailed layout definitions: `references/layouts/<layout>.md`
└── NN-ending-[slug].png
```
**Target directory**:
- With source path: `[source-dir]/[source-name-no-ext]/xhs-images/`
- Example: `/tests-data/article.md``/tests-data/article/xhs-images/`
- Without source: `./xhs-images/[topic-slug]/`
**Slug Generation**:
1. Extract main topic from content (2-4 words, kebab-case)
2. Example: "AI工具推荐" → `ai-tools-recommend`
**Directory backup**:
- If target directory exists, rename existing to `<dirname>-backup-YYYYMMDD-HHMMSS`
**Conflict Resolution**:
If `xhs-images/{topic-slug}/` already exists:
- Append timestamp: `{topic-slug}-YYYYMMDD-HHMMSS`
- Example: `ai-tools` exists → `ai-tools-20260118-143052`
**Source Files**:
Copy all sources with naming `source-{slug}.{ext}`:
- `source-article.md`, `source-photo.jpg`, etc.
- Multiple sources supported: text, images, files from conversation
## Workflow