How to Download HTML Pages for Conversion

Before you can convert HTML to Markdown, you need the HTML files on your machine. This guide covers every common method — from one-click browser saves to full recursive site mirrors. Pick the approach that matches your scenario.

Method 1: wget — Recursive Site Mirroring

Best for: documentation sites, product manuals, developer references — any site with a clear URL hierarchy.

Basic Command

wget -r -np -k -E \
  -w 0.5 --random-wait \
  https://docs.example.com/en/latest/

What the Flags Do

Flag	Purpose
`-r`	Follow links recursively
`-np`	Never ascend above the starting path (stay inside the docs section)
`-k`	Rewrite links to point at local files
`-E`	Append `.html` to extensionless URLs so the converter can detect them
`-w 0.5`	Wait 0.5 seconds between requests
`--random-wait`	Randomize the delay so it looks less automated

Skip Images and Assets (Recommended)

You only need the HTML. Reject binary files to save bandwidth and disk space:

wget -r -np -k -E \
  --reject-regex '\.(png|jpe?g|gif|svg|webp|woff2?|ttf|css|js|zip|pdf)$' \
  -w 0.5 --random-wait \
  https://community.denodo.com/docs/html/browse/latest/en/vdp/developer/index

Limit Crawl Depth

For large sites, limit how deep wget follows links:

# Only 2 levels deep from the start URL
wget -r -np -k -E -l 2 \
  -w 0.5 --random-wait \
  https://docs.example.com/en/latest/

Installing wget

System	Command
macOS	`brew install wget`
Ubuntu/Debian	`sudo apt install wget`
Windows	`winget install JernejSimoncic.wget`

Method 2: curl — Download a List of URLs

Best for: when you have a specific list of pages (from a sitemap, a table of contents, or a URL file).

Download from a URL List

# urls.txt — one URL per line
https://docs.example.com/page-1.html
https://docs.example.com/page-2.html
https://docs.example.com/api/reference.html

# Download all pages, preserve directory structure
while IFS= read -r url; do
  path=$(echo "$url" | sed 's|https\?://[^/]*/||')
  mkdir -p "$(dirname "downloaded/$path")"
  curl -s -o "downloaded/$path" "$url"
  sleep 0.5
done < urls.txt

Extract URLs from a Sitemap

# Pull URLs from sitemap.xml
curl -s https://docs.example.com/sitemap.xml \
  | grep -oP '<loc>\K[^<]+' \
  | grep '\.html' \
  > urls.txt

Method 3: Browser — Save Individual Pages

Best for: a handful of pages, or sites that require login/authentication.

Open the page in Chrome/Firefox/Edge
Press Ctrl+S (Windows/Linux) or ⌘+S (macOS)
Choose "Web Page, HTML Only" (not "Complete" — you don't need images/CSS)
Save to a folder
Repeat for each page, or use a browser extension for batch saving

Browser Extensions for Batch Saving

SingleFile (Chrome/Firefox) — saves pages as self-contained HTML files
Save All Resources (Chrome) — bulk-saves open tabs as HTML
DownThemAll (Firefox) — download manager that can grab all links on a page

Method 4: HTTrack — GUI Website Copier

Best for: users who prefer a graphical interface over the command line.

Download HTTrack from httrack.com (Windows/Linux/macOS)
Enter the starting URL and configure filters (HTML only)
Let it mirror the site
Find the HTML files in the output folder

Method 5: Puppeteer/Playwright — JavaScript-Rendered Pages

Best for: single-page apps (SPAs) and sites that render content with JavaScript. wget and curl only get the initial HTML — if the content loads dynamically, you need a headless browser.

// save-pages.mjs — Node.js + Playwright
import { chromium } from 'playwright';
import { writeFileSync, mkdirSync } from 'fs';

const urls = [
  'https://spa-docs.example.com/getting-started',
  'https://spa-docs.example.com/api-reference',
];

const browser = await chromium.launch();
const page = await browser.newPage();

for (const url of urls) {
  await page.goto(url, { waitUntil: 'networkidle' });
  const html = await page.content();
  const filename = url.split('/').pop() + '.html';
  mkdirSync('pages', { recursive: true });
  writeFileSync(`pages/${filename}`, html);
  console.log(`Saved: ${filename}`);
}

await browser.close();

Creating the ZIP Package

Once you have your HTML files downloaded, bundle them into a ZIP for the converter:

macOS / Linux

# From inside the downloaded folder:
cd community.denodo.com

# ZIP only HTML files, preserving directory structure
find . -name '*.html' -print | zip ../docs.zip -@

Or zip everything (non-HTML files are ignored by the converter):

zip -r ../docs.zip .

Windows (PowerShell)

Compress-Archive -Path .\downloaded\* -DestinationPath docs.zip

Windows (File Explorer)

Select the folder with your HTML files
Right-click → Compress to ZIP file

Checking Your ZIP Before Upload

The converter accepts ZIPs up to 100 MB with up to 200 HTML files:

# Count HTML files
unzip -l docs.zip | grep -c '\.html$'

# Check total size
du -h docs.zip

If your archive exceeds the limits, split it by subdirectory and convert in batches.

Tips and Best Practices

Respect robots.txt — Check the site's robots.txt before crawling. Add -e robots=on to wget (it's on by default)
Throttle requests — Always add a delay between requests (-w 0.5 minimum) to avoid overloading the server
Check terms of service — Some sites prohibit automated downloading in their ToS
Use -E with wget — This ensures extensionless URLs get saved as .html files that the converter can detect
Test with a small section first — Add -l 1 to limit depth and verify the output before doing a full crawl
Login-protected sites — Use wget --load-cookies with exported browser cookies, or save pages manually via the browser

Next Step

Once you have your ZIP file ready, head to the converter and drop it in. The conversion happens entirely in your browser — your files are never uploaded to any server.

Try the Converter →