seahi/baoyu-skills
1
0
Fork 0
mirror of https://github.com/JimLiu/baoyu-skills.git synced 2026-08-08 01:43:03 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(baoyu-post-to-x): add Chrome Computer Use as preferred execution mode

Browse Source 
In Codex, prefer the bundled Chrome Computer Use path for all X UI
actions (compose, article, quote, video). CDP scripts become a fallback
when Computer Use is unavailable or explicitly not requested.
This commit is contained in:
Jim Liu 宝玉
2026-05-09 22:04:58 -05:00
parent aa1a967a9f
commit 045fe5e57e
3 changed files with 207 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/baoyu-post-to-x/references/articles.md
+74 -9
View File
@@ -2,6 +2,17 @@
Publish Markdown articles to X Articles editor with rich text formatting and images.
## Mode Selection
In Codex, prefer **Chrome Computer Use** when available:
1. If Computer Use tools are already visible, call `get_app_state` for `Google Chrome`.
2. If not, use `tool_search` for `computer-use get_app_state click press_key drag scroll Google Chrome`, then call `get_app_state`.
3. If that succeeds, use the Computer Use workflow below.
4. Use the CDP script workflow only when Computer Use is unavailable or explicitly requested.
If the user explicitly asks for Chrome Computer Use, do not fall back to CDP, Playwright, or the in-app Browser without approval.
## Prerequisites
- X Premium subscription (required for Articles)
@@ -10,6 +21,24 @@ Publish Markdown articles to X Articles editor with rich text formatting and ima
## Usage
### Chrome Computer Use (Codex Preferred)
Prepare the article HTML and image map:
```bash
${BUN_X} {baseDir}/scripts/md-to-html.ts article.md --save-html /tmp/x-article-body.html > /tmp/x-article.json
```
Copy the generated HTML as rich text:
```bash
${BUN_X} {baseDir}/scripts/copy-to-clipboard.ts html --file /tmp/x-article-body.html
```
Then use Codex Computer Use against `Google Chrome` for all X UI operations.
### CDP Script Fallback
```bash
# Publish markdown article (preview mode)
${BUN_X} {baseDir}/scripts/x-article.ts article.md
@@ -21,6 +50,8 @@ ${BUN_X} {baseDir}/scripts/x-article.ts article.md --cover ./cover.jpg
${BUN_X} {baseDir}/scripts/x-article.ts article.md --submit
```
Do not use `--submit` unless the user has explicitly confirmed the final public publish action.
## Markdown Format
```markdown
@@ -118,7 +149,33 @@ JSON output:
| `1. item` | `<ol><li>` |
| `![](img)` | Image placeholder |
## Workflow
## Computer Use Workflow (Preferred in Codex)
1. **Detect Computer Use**: call `get_app_state` for `Google Chrome`; use `tool_search` first if the tools are not visible.
2. **Parse Markdown**: run `md-to-html.ts --save-html /tmp/x-article-body.html > /tmp/x-article.json`.
3. **Read the map**: use `/tmp/x-article.json` for `title`, `coverImage`, and `contentImages`.
4. **Open X Articles**: use Chrome Computer Use to navigate to `https://x.com/compose/articles`.
5. **Create Draft**: click the create/write button if needed, or open the target draft.
6. **Upload Cover**: if `coverImage` exists, use Chrome's visible upload/file picker UI. If the file picker cannot be operated reliably, stop and ask for help rather than switching to CDP silently.
7. **Fill Title**: type the title into the title field.
8. **Paste Content**:
- Run `copy-to-clipboard.ts html --file /tmp/x-article-body.html`.
- Click the article body.
- Press `super+v` on macOS or `control+v` on Windows/Linux with Computer Use.
9. **Insert Images**: for each `contentImages` item in placeholder order:
- Run `copy-to-clipboard.ts image <localPath>`.
- Select the exact placeholder text (`XIMGPH_N`) in the editor.
- Press `super+v`/`control+v` with Computer Use.
- Wait for X to finish uploading media.
- If `XIMGPH_N` remains above the inserted image, reselect that exact text and press `BackSpace`.
- Do not press `BackSpace` unless the Computer Use state confirms the selected text is exactly the placeholder.
10. **Verify**:
- Inspect the Computer Use state for `XIMGPH_` residue.
- Confirm the expected number of image blocks is visible.
- Open Preview and verify title, cover, body, links, and images.
11. **Publish Safety**: ask the user for explicit final confirmation before clicking `Publish`.
## CDP Script Workflow (Fallback)
1. **Parse Markdown**: Extract title, cover, content images, generate HTML
2. **Launch Chrome**: Real browser with CDP, persistent login
@@ -137,7 +194,7 @@ JSON output:
- Compare expected vs actual image count
- Warn if issues found
10. **Review**: Browser stays open for 60s preview
11. **Publish**: Only with `--submit` flag
11. **Publish**: Only with `--submit` flag and explicit user confirmation
## Example Session
@@ -145,15 +202,15 @@ JSON output:
User: /post-to-x article ./blog/my-post.md --cover ./thumbnail.png
Claude:
1. Parses markdown: title="My Post", 3 content images
2. Launches Chrome with CDP
3. Navigates to x.com/compose/articles
4. Clicks create button
1. Detects Chrome Computer Use
2. Parses markdown: title="My Post", 3 content images
3. Saves `/tmp/x-article-body.html` and `/tmp/x-article.json`
4. Uses Chrome Computer Use to open X Articles and create a draft
5. Uploads thumbnail.png as cover
6. Fills title "My Post"
7. Pastes HTML content
7. Pastes HTML content with a real Chrome paste
8. Inserts 3 images at placeholder positions
9. Reports: "Article composed. Review and use --submit to publish."
9. Opens Preview and asks before publishing
```
## Troubleshooting
@@ -162,6 +219,8 @@ Claude:
- **Cover upload fails**: Check file path and format (PNG, JPEG)
- **Images not inserting**: Verify placeholders exist in pasted content
- **Content not pasting**: Check HTML clipboard: `${BUN_X} {baseDir}/scripts/copy-to-clipboard.ts html --file /tmp/test.html`
- **Computer Use unavailable**: Use the CDP fallback script, unless the user explicitly required Chrome Computer Use.
- **Placeholder remains after paste**: Select only the placeholder text and press BackSpace after upload completes.
## How It Works
@@ -172,7 +231,13 @@ Claude:
- Downloads remote images locally
- Returns structured JSON
2. `x-article.ts` publishes via CDP:
2. Chrome Computer Use publishes through the user's visible Chrome UI:
- Uses the user's active Chrome profile and logged-in X session
- Uses `copy-to-clipboard.ts` for rich HTML and image clipboard payloads
- Uses real keystrokes (`super+v`/`control+v`) through Codex Computer Use
- Keeps the final publish click under user confirmation
3. `x-article.ts` publishes via CDP as a fallback:
- Launches real Chrome (bypasses detection)
- Uses persistent profile (saved login)
- Navigates and fills editor via DOM manipulation
skills/baoyu-post-to-x/references/regular-posts.md
+32 -27
View File
@@ -6,6 +6,17 @@ Detailed documentation for posting text and images to X.
If you prefer step-by-step control:
### Step 0: Prefer Chrome Computer Use in Codex
When running inside Codex, first detect whether Chrome Computer Use is enabled:
1. If Computer Use tools are already visible, call `get_app_state` for `Google Chrome`.
2. If not, use `tool_search` for `computer-use get_app_state click press_key drag scroll Google Chrome`, then call `get_app_state`.
3. If `get_app_state` succeeds, use the user's real Chrome with Computer Use for all X UI actions.
4. Use CDP scripts only when Computer Use is unavailable or explicitly requested.
If the user explicitly asks for Chrome Computer Use, do not use Playwright, the in-app Browser, or CDP without approval.
### Step 1: Copy Image to Clipboard
```bash
@@ -25,27 +36,15 @@ ${BUN_X} {baseDir}/scripts/paste-from-clipboard.ts --app "Google Chrome" --retri
${BUN_X} {baseDir}/scripts/paste-from-clipboard.ts --delay 200
```
### Step 3: Use Playwright MCP (if Chrome session available)
### Step 3: Use Chrome Computer Use (Preferred)
```bash
# Navigate
mcp__playwright__browser_navigate url="https://x.com/compose/post"
# Get element refs
mcp__playwright__browser_snapshot
# Type text
mcp__playwright__browser_click element="editor" ref="<ref>"
mcp__playwright__browser_type element="editor" ref="<ref>" text="Your content"
# Paste image (after copying to clipboard)
mcp__playwright__browser_press_key key="Meta+v" # macOS
# or
mcp__playwright__browser_press_key key="Control+v" # Windows/Linux
# Screenshot to verify
mcp__playwright__browser_take_screenshot filename="preview.png"
```
1. Use `get_app_state` for `Google Chrome`.
2. Navigate Chrome to `https://x.com/compose/post` if needed.
3. Click the composer and type the post text.
4. Copy each image to the clipboard with `copy-to-clipboard.ts image <path>`.
5. Press `super+v` on macOS or `control+v` on Windows/Linux with Computer Use.
6. Wait until X finishes media upload.
7. Ask for explicit confirmation before clicking `Post`.
## Image Support
@@ -59,12 +58,12 @@ mcp__playwright__browser_take_screenshot filename="preview.png"
User: /post-to-x "Hello from Claude!" --image ./screenshot.png
Claude:
1. Runs: ${BUN_X} {baseDir}/scripts/x-browser.ts "Hello from Claude!" --image ./screenshot.png
2. Chrome opens with X compose page
3. Text is typed into editor
4. Image is copied to clipboard and pasted
5. Browser stays open 30s for preview
6. Reports: "Post composed. Use --submit to post."
1. Detects Chrome Computer Use
2. Opens X compose in the user's real Chrome
3. Types text into editor
4. Copies image to clipboard and pastes with Computer Use
5. Waits for upload and verifies the preview
6. Asks before clicking Post
```
## Troubleshooting
@@ -80,7 +79,13 @@ Claude:
## How It Works
The `x-browser.ts` script uses Chrome DevTools Protocol (CDP) to:
In Chrome Computer Use mode:
1. Codex controls the user's visible Google Chrome window
2. Text is typed through the real UI
3. Images are copied to the system clipboard and pasted with real keystrokes
4. The user confirms before the final public post
The `x-browser.ts` script is the CDP fallback. It uses Chrome DevTools Protocol (CDP) to:
1. Launch real Chrome (not Playwright) with `--disable-blink-features=AutomationControlled`
2. Use persistent profile directory for saved login sessions
3. Interact with X via CDP commands (Runtime.evaluate, Input.dispatchKeyEvent)