موسوعة البوكيمون — نسخة عربية (Pokémon Book in Arabic)

A script that scrapes Bulbapedia for every National Pokédex base species (no Mega/regional forms), translates each entry's intro + Biology from English to Arabic using the mando.family translate API, and renders a single, self-contained, RTL, A4 print-oriented HTML book.

License: personal noncommercial use only. Bulbapedia content is CC BY-NC-SA 2.5; the rendered book includes an attribution page. Pokémon and its trademarks belong to Nintendo / Creatures Inc. / GAME FREAK.

Features

• Base species only, keyed by National Pokédex number.
• Each entry: official artwork, translated intro, translated Biology (when present), and meta fields — type, category, height (cm), weight (kg), abilities.
• Fully resumable: raw pages, parsed species, images, and translations are all cached on disk.
• Self-contained HTML: images embedded as base64 data URIs (use --no-embed to keep paths).
• A4 / RTL print CSS ready for headless-Chrome PDF export.

Requirements

python3 -m pip install -r requirements.txt   # requests, beautifulsoup4, lxml

Usage

python3 build_book.py --start 1 --end 6        # first six Pokémon (smoke test)
python3 build_book.py --start 1 --end 1025     # full book
python3 build_book.py --only-render            # rebuild HTML from cache (no network)
python3 build_book.py --skip-translate         # scrape only, no translation
python3 build_book.py --force                  # re-scrape cached pages
python3 build_book.py --engine deepl           # switch translation engine
python3 build_book.py --no-embed               # reference images by path, not data URIs
python3 build_book.py --ethical                # random 1-3s waits between requests

Output is written to out/book.html.

PDF preview

google-chrome-stable --headless --no-sandbox --print-to-pdf=out/preview.pdf out/book.html

How it works

1. Scrape (scrape.py) — builds the species index from the National Pokédex list, fetches each X_(Pokémon) page (respecting the site's 5s crawl delay), and extracts the intro, Biology, infobox fields, and artwork.
2. Translate (translate.py) — sends text to the mando.family google engine (GET-only, ~1000-char chunks), retrying on failure; results cached by SHA-256.
3. Render (render.py) — writes the RTL HTML book with embedded images.

Project layout

build_book.py        # CLI entrypoint
scrape.py            # Bulbapedia fetch + parsing
translate.py         # mando.family translation client
render.py            # HTML writer
cache/               # gitignored — pages, species, images, dex index, translations
out/                 # gitignored — book.html, preview.pdf

Notes & gotchas

• The full 1025-species run takes several hours due to Bulbapedia's crawl-delay.
• --ethical spreads requests out with random 1-3s waits (jitter on top of the site crawl-delay for scraping, and between translation calls).
• Long Biology sections are chunked before translation to stay within URL length limits.
• Deleting cache/translations.json forces a full re-translate; deleting cache/pages/ or cache/dex.json re-fetches from Bulbapedia.
• No tests, linter, or CI are configured.

See AGENTS.md for the detailed parsing rules and caching behavior.