{
  "markdown": "# web2md-core\n\nTurn messy HTML into clean, LLM-ready Markdown.\n\nThis is the extraction and conversion engine behind [Web2MD](https://web2md.org).\nIt is a pure library — give it an HTML string, get Markdown back. No network\ncalls, no API key, no account. Nothing in this package talks to a server.\n\n```bash\nnpm install web2md-core\n```\n\n## Why convert at all\n\nFeeding raw HTML to a language model wastes most of your context window on\nmarkup, navigation, and ads. Converting first cuts that down and gives the model\na document it can actually follow.\n\nThe library reports both numbers so you can see the difference:\n\n```ts\nimport { convertToMarkdown } from 'web2md-core'\n\nconst result = convertToMarkdown(html, { url: 'https://example.com/post' })\n\nconsole.log(result.markdown)\nconsole.log(result.metadata.originalTokenCount, '→', result.metadata.tokenCount)\n// e.g. 417 → 281\n```\n\n`convertToMarkdown` returns `null` when it cannot find a main content block —\ncheck for that rather than assuming a result.\n\n## What it does\n\n- **Finds the actual article.** Strips navigation, sidebars, ads, cookie banners,\n  and footers, keeping the content a reader came for.\n- **Preserves structure.** Headings, lists, tables, and fenced code blocks survive\n  the round trip — that structure is what lets a model answer questions about one\n  specific section.\n- **Reports tokens.** Estimated counts for both the original HTML and the cleaned\n  Markdown, plus helpers to split or trim for a target context window.\n- **Runs anywhere.** Uses [linkedom](https://github.com/WebReflection/linkedom)\n  for parsing, so it works in Node without a browser.\n\n## API\n\n### `convertToMarkdown(html, options?)`\n\nThe main entry point. Note the signature takes **two** arguments — the URL goes\ninside `options`, not as a positional parameter:\n\n```ts\nconvertToMarkdown(html, { url: 'https://example.com/post' })\n```\n\n| Option | Default | Meaning |\n| --- | --- | --- |\n| `url` | — | Source URL. Used to resolve relative links and fill `metadata.url`. |\n| `includeLinks` | `false` | Keep `<a>` as Markdown links. Off by default because link URLs are often the bulk of the tokens on navigation-heavy pages. |\n| `includeImages` | `false` | Keep images. Off by default for the same reason. |\n| `includeMeta` | `false` | Prepend a metadata block (title, source, timestamp). |\n| `customRule` | — | A `CustomRule` for site-specific extraction. |\n| `detectCodeLanguage` | `false` | Try to infer the language of fenced code blocks. |\n\n`includeLinks` and `includeImages` default to **off**. That is deliberate — the\nprimary use case is feeding an LLM, where both are usually noise. Turn them on\nwhen you are archiving rather than summarising.\n\n### Other exports\n\n```ts\nquickConvert(html, url?)        // same result, but with links, images and\n                                // metadata turned ON — the \"archive it\" preset\nextractContent(html, url?)      // main content element, before conversion\nhtmlToMarkdown(html, options?)  // low-level conversion, no extraction\ncountTokens(text)               // token estimate\nsplitByTokens(md, limit)        // chunk for RAG ingestion\noptimizeForContextWindow(md, model)\nhtmlLooksLikeLoginWall(html)    // detect login walls so you can fail loudly\nMODEL_CONTEXT_LIMITS            // context sizes for common models\n```\n\nMarkdown → sanitized HTML (via DOMPurify), for previewing output:\n\n```ts\nrenderMarkdownSync(md)\nrenderMarkdownFull(md)          // async; includes syntax highlighting\nrenderMarkdownWithFormulas(md)  // KaTeX math\n```\n\n`getPageHTML()` and `getSelectionHTML()` read `document` directly and therefore\nonly work in a browser. They throw in Node — that boundary is intentional.\n\n## Site-specific extraction\n\nGeneric extraction handles most pages. When a site needs special treatment,\npass a rule:\n\n```ts\nconvertToMarkdown(html, {\n  url: 'https://example.com/thread',\n  customRule: {\n    name: 'Example forum',\n    domain: 'example.com',\n    contentSelector: '.thread-body',\n    removeSelectors: ['.signature', '.ad-slot'],\n  },\n})\n```\n\n## Scope\n\nThis package covers extraction and conversion. It does not include Web2MD's\nbrowser extension, hosted API, or account system — those stay in the product.\n\nContributions to extraction quality are especially welcome: if a site converts\nbadly, an issue with the URL and what went wrong is genuinely useful.\n\n## License\n\nMIT\n",
  "bytes": 4380,
  "sha": "ee710b4efba2ac631e350c1b82e415ad4d52b65ffe969484a7e5d8199c1f181d",
  "repo_slug": "io-oi-ai/web2md",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_org_web2md_web2md_4ae99e9e/readme"
}