Compare commits

..

4 Commits

Author SHA1 Message Date
wangruofeng c9a50cc908 fix(baoyu-post-to-x): 修复紧贴中文的加粗/斜体在 X Articles 渲染为字面 ** 的问题 (#189)
* fix(baoyu-post-to-x): render CJK-adjacent bold/italics in articles

md-to-html.ts preprocessed markdown with remark + remark-cjk-friendly, then re-stringified and re-parsed with marked. remark-cjk-friendly parses emphasis adjacent to CJK correctly, but remark-stringify re-emits the **bold*/*italic* markers, which marked then FAILS to parse: its flanking test does not treat a CJK character after the closing delimiter as valid punctuation/whitespace. Result: literal **...** asterisks leak into the pasted X Article body whenever the closing delimiter is directly followed by a CJK character.

Fix: convert strong/emphasis mdast nodes (already identified correctly by remark-cjk-friendly) into raw inline <strong>/<em> HTML before stringify, so marked passes them through untouched. A small recursive serializer preserves nested links, inline code, images, etc.

- add remarkStrongEmToHtml remark plugin + inline mdast serializer
- add unist-util-visit as an explicit dependency
- add regression test covering CJK-adjacent bold and italics

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(baoyu-post-to-x): 修复紧贴中文的加粗/斜体在 X Articles 渲染为字面 ** 的问题

按维护者建议改用 remark-cjk-friendly 标准用法,替代自定义插件/序列化器:

- 删除 remarkStrongEmToHtml 插件与 serializeMdastNode 序列化器
- preprocessCjkMarkdown 简化为 remarkParse + remarkCjkFriendly + remarkStringify 往返
- 移除原 markdown 阶段的 &#x...; 实体解码(该解码会还原 cjk-friendly 为
  骗过 marked flanking 判定而生成的实体,正是 CJK 加粗失败的根因)
- 改在 marked 渲染出最终 HTML 后再 decodeHtmlEntities,此时 markdown 语法
  已不存在,解码安全且保持输出干净
- 移除不再使用的 unist-util-visit 显式依赖
- 测试补充引用式链接 **[docs][d]** 在强调内的回归断言(顺带修复 Codex
  指出的 linkReference 渲染为纯文本的 P2 问题)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(baoyu-post-to-x): 移除渲染后无差别实体解码,避免篡改作者字面实体

回应 Codex review (P2):decodeHtmlEntities 对整个渲染 HTML 做无差别
&#x...; 解码,会把作者为显示字面 HTML 而写的实体(如 &#x3C;b&#x3E;)
解码成真实标签,存在内容篡改与 HTML 注入风险。

该解码本就只是为了让输出 HTML「好看」,而 remark-cjk-friendly 引入的
实体是合法 HTML 字符引用,粘进 X 编辑器时能正确渲染;且这些实体只会
出现在强调 span 的外侧(边界字符),不会出现在 span 内部。因此直接
移除解码,既消除过度解码风险,又无功能损失。

- 删除 decodeHtmlEntities 及其在 convertMarkdownToHtml 的调用
- 新增测试:作者字面实体 &#x3C;b&#x3E; 不被解码为真实 <b>,CJK 加粗
  仍正常渲染
- bun test 8 个用例全部通过;真实文章端到端验证 17 处加粗、0 字面 **、
  0 真实标签注入

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: ruofeng <ruofeng.wang@rd.group>
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-28 01:42:07 -05:00
Jim Liu 宝玉 348dc82612 Add explicit MIT license 2026-06-19 23:01:55 -05:00
Jim Liu 宝玉 dba3d732c3 Merge pull request #187 from jiji262/fix/gemini3-image-extraction
fix(gemini-web): recover generated images on Gemini 3
2026-06-19 22:45:31 -05:00
jiji262 661bdd08ff fix(gemini-web): recover generated images on Gemini 3
Gemini 3 stopped emitting the legacy `image_generation_content` marker in
the candidate text and moved the generated image into a later response
part. The `wants_generated` gate only checked the candidate text / a single
structured field, so on Gemini 3 it was never true and the image was
silently dropped: generate_content returned an empty result even though the
gg-dl image URL was present in the raw response.

- Broaden `wants_generated` to also fire when the raw response contains a
  `gg-dl/` generated-image URL or an `image_generation_content` marker.
- When a candidate has no image part, skip extraction instead of throwing
  ImageGenerationError, so multi-candidate responses don't fail wholesale.

Verified against a live Gemini 3 account: image generation returns the
generated image again with the default gemini-3-pro model.
2026-06-20 11:37:12 +08:00
12 changed files with 126 additions and 12 deletions
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Jim Liu
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+3 -1
View File
@@ -1387,7 +1387,9 @@ This project was inspired by and builds upon the following open source projects:
## License
MIT
Unless otherwise noted, this repository is licensed under the [MIT License](./LICENSE).
Published ClawHub skills follow ClawHub registry rules and are distributed under `MIT-0`. Third-party code and assets retain their original licenses where noted.
## Star History
+3 -1
View File
@@ -1388,7 +1388,9 @@ HTTP_PROXY=http://127.0.0.1:7890 HTTPS_PROXY=http://127.0.0.1:7890 /baoyu-danger
## 许可证
MIT
除另有说明外,本仓库采用 [MIT License](./LICENSE) 授权。
发布到 ClawHub 的 skill 根据 ClawHub registry 规则以 `MIT-0` 分发。第三方代码和素材按其注明的原始许可授权。
## Star History
+5
View File
@@ -5,6 +5,7 @@
"packages": {
"": {
"name": "baoyu-skills",
"license": "MIT",
"workspaces": [
"packages/*"
],
@@ -5018,12 +5019,14 @@
},
"packages/baoyu-chrome-cdp": {
"version": "0.1.1",
"license": "MIT",
"engines": {
"bun": ">=1.2.0"
}
},
"packages/baoyu-codex-imagegen": {
"version": "0.1.0",
"license": "MIT",
"bin": {
"codex-imagegen": "src/main.ts"
},
@@ -5033,6 +5036,7 @@
},
"packages/baoyu-fetch": {
"version": "0.1.2",
"license": "MIT",
"dependencies": {
"@mozilla/readability": "^0.6.0",
"chrome-launcher": "^1.2.1",
@@ -5061,6 +5065,7 @@
},
"packages/baoyu-md": {
"version": "0.1.1",
"license": "MIT",
"dependencies": {
"fflate": "^0.8.2",
"front-matter": "^4.0.2",
+1
View File
@@ -1,6 +1,7 @@
{
"name": "baoyu-skills",
"private": true,
"license": "MIT",
"type": "module",
"workspaces": [
"packages/*"
+1
View File
@@ -1,6 +1,7 @@
{
"name": "baoyu-chrome-cdp",
"version": "0.1.1",
"license": "MIT",
"type": "module",
"files": [
"dist",
@@ -1,6 +1,7 @@
{
"name": "baoyu-codex-imagegen",
"version": "0.1.0",
"license": "MIT",
"type": "module",
"description": "Generate images via Codex CLI's built-in image_gen tool from non-Codex runtimes (Claude Code, Hermes, …).",
"bin": {
+1
View File
@@ -1,6 +1,7 @@
{
"name": "baoyu-fetch",
"version": "0.1.2",
"license": "MIT",
"description": "Read URLs into high-quality Markdown or JSON with Chrome CDP and site adapters.",
"type": "module",
"bin": {
+1
View File
@@ -1,6 +1,7 @@
{
"name": "baoyu-md",
"version": "0.1.1",
"license": "MIT",
"type": "module",
"main": "./dist/index.cjs",
"module": "./dist/index.js",
@@ -456,19 +456,19 @@ export class GeminiClient extends GemMixin {
const generated_images: GeneratedImage[] = [];
const wants_generated =
get_nested_value(candidate, [12, 7, 0], null) != null ||
/http:\/\/googleusercontent\.com\/image_generation_content\/\d+/.test(text);
/http:\/\/googleusercontent\.com\/image_generation_content\/\d+/.test(text) ||
// Gemini 3 no longer emits the legacy marker in the candidate text; the generated
// image arrives in a later response part. Detect it from the raw response instead.
/\/gg-dl\//.test(txt) ||
/image_generation_content\/\d+/.test(txt);
if (wants_generated) {
const image_part = find_generated_image_part(response_json as unknown[], body_index, candidate_index, 1);
const img_body = image_part?.body ?? null;
if (!img_body) {
throw new ImageGenerationError(
'Failed to parse generated images. Please update gemini_webapi to the latest version. If the error persists and is caused by the package, please report it on GitHub.',
);
}
const img_candidate = get_nested_value<unknown[]>(img_body, [4, candidate_index], []);
// Not every candidate carries a generated image (e.g. multi-candidate responses).
// If this candidate has no image part, skip extraction instead of failing the whole call.
const img_candidate = img_body ? get_nested_value<unknown[]>(img_body, [4, candidate_index], []) : [];
const finished = get_nested_value<string | null>(img_candidate, [1, 0], null);
if (finished) {
text = finished.replace(/http:\/\/googleusercontent\.com\/image_generation_content\/\d+/g, '').trimEnd();
@@ -98,3 +98,68 @@ test('parseMarkdown resolves encoded spaces and literal percent image paths', as
assert.equal(result.contentImages[0]?.localPath, path.join(root, 'Pasted image.png'));
assert.equal(result.contentImages[1]?.localPath, path.join(root, '100%.png'));
});
test('parseMarkdown renders CJK-adjacent bold and italics (no literal asterisks)', async (t) => {
const root = await makeTempDir('x-md-to-html-cjk-bold-');
t.after(() => fs.rm(root, { recursive: true, force: true }));
const markdownPath = path.join(root, 'post.md');
const tempDir = path.join(root, 'tmp');
await fs.mkdir(tempDir, { recursive: true });
await fs.writeFile(
markdownPath,
[
'# 标题',
'',
'分工在变细。**国际大厂卷基础设施,中文项目卷场景落地。**这其实是生态成熟的表现。',
'',
'半角场景 **Top 10 里平均有 8 个** 项目。',
'',
'斜体 *数据来源 GitHub* 收尾。',
'',
'参考 **[docs][d]** 了解更多。',
'',
'[d]: https://example.com',
].join('\n'),
);
const result = await parseMarkdown(markdownPath, { tempDir });
// Bold directly adjacent to CJK (closing ** followed by CJK) must render.
assert.match(result.html, /<strong>国际大厂卷基础设施,中文项目卷场景落地。<\/strong>/);
assert.match(result.html, /<strong>Top 10 里平均有 8 个<\/strong>/);
// Italics.
assert.match(result.html, /<em>数据来源 GitHub<\/em>/);
// Reference-style links inside emphasis must render as links, not plain text.
assert.match(result.html, /<strong><a href="https:\/\/example\.com" rel="noopener noreferrer nofollow">docs<\/a><\/strong>/);
// No literal emphasis delimiters should leak into the output.
assert.doesNotMatch(result.html, /\*\*/);
assert.doesNotMatch(result.html, /(?<!\*)\*(?!\*)[^*\n]+\*(?!\*)/);
});
test('parseMarkdown does not decode author-written literal HTML entities into tags', async (t) => {
const root = await makeTempDir('x-md-to-html-entities-');
t.after(() => fs.rm(root, { recursive: true, force: true }));
const markdownPath = path.join(root, 'post.md');
const tempDir = path.join(root, 'tmp');
await fs.mkdir(tempDir, { recursive: true });
await fs.writeFile(
markdownPath,
[
'# 标题',
'',
'正文中写 &#x3C;b&#x3E;literal&#x3C;/b&#x3E; 想显示字面标签。**加粗**收尾。',
'',
'代码里写 `&#x3C;b&#x3E;` 同样保留。',
].join('\n'),
);
const result = await parseMarkdown(markdownPath, { tempDir });
// CJK-adjacent bold still renders.
assert.match(result.html, /<strong>加粗<\/strong>/);
// Author-written literal entities must NOT be decoded into real tags.
assert.doesNotMatch(result.html, /<b>literal<\/b>/);
assert.match(result.html, /&lt;b&gt;literal&lt;\/b&gt;/);
});
+16 -2
View File
@@ -136,6 +136,21 @@ function highlightCode(code: string, lang: string): string {
}
}
// Normalize CJK-adjacent emphasis so `marked` renders it correctly.
//
// `marked`'s emphasis tokenizer treats a closing `**`/`*` directly followed by a
// CJK character as not right-flanking, so it leaves the delimiters literal
// (e.g. `**加粗**这` renders as plain text with the asterisks intact). We round-trip
// the markdown through `remark-cjk-friendly`, whose stringify serializes the
// boundary character as an HTML entity (`&#x8FD9;`); the entity is treated as
// punctuation by `marked`'s flanking rules, so emphasis parses as expected.
//
// We deliberately do NOT decode the entities afterward. They are valid HTML
// character references that render correctly when the article HTML is pasted into
// the X editor, and `marked` only emits them for characters outside the emphasis
// span (the boundary char), never inside it. A blanket decode of the rendered
// HTML would risk turning author-written literal entities (e.g. `&#x3C;b&#x3E;`
// meant to display `<b>` as text) into real tags, so we leave them intact.
function preprocessCjkMarkdown(markdown: string): string {
try {
const processor = unified()
@@ -143,8 +158,7 @@ function preprocessCjkMarkdown(markdown: string): string {
.use(remarkCjkFriendly)
.use(remarkStringify);
const result = String(processor.processSync(markdown));
return result.replace(/&#x([0-9A-Fa-f]+);/g, (_, hex: string) => String.fromCodePoint(parseInt(hex, 16)));
return String(processor.processSync(markdown));
} catch {
return markdown;
}