mirror of
https://github.com/tldr-pages/tldr.git
synced 2025-04-23 17:42:07 +02:00
8ebd171d6f
Co-authored-by: marchersimon <50295997+marchersimon@users.noreply.github.com> Co-authored-by: Seth Falco <seth@falco.fun> Co-authored-by: Patrice Denis <patricedenis@users.noreply.github.com>
32 lines
732 B
Markdown
32 lines
732 B
Markdown
# scrapy
|
|
|
|
> Web-crawling framework.
|
|
> More information: <https://scrapy.org>.
|
|
|
|
- Create a project:
|
|
|
|
`scrapy startproject {{project_name}}`
|
|
|
|
- Create a spider (in project directory):
|
|
|
|
`scrapy genspider {{spider_name}} {{website_domain}}`
|
|
|
|
- Edit spider (in project directory):
|
|
|
|
`scrapy edit {{spider_name}}`
|
|
|
|
- Run spider (in project directory):
|
|
|
|
`scrapy crawl {{spider_name}}`
|
|
|
|
- Fetch a webpage as scrapy sees it and print source in stdout:
|
|
|
|
`scrapy fetch {{url}}`
|
|
|
|
- Open a webpage in the default browser as scrapy sees it (disable JavaScript for extra fidelity):
|
|
|
|
`scrapy view {{url}}`
|
|
|
|
- Open scrapy shell for URL, which allows interaction with the page source in python shell (or ipython if available):
|
|
|
|
`scrapy shell {{url}}`
|