mirror of
https://github.com/tldr-pages/tldr.git
synced 2025-07-01 08:35:23 +02:00
add page for scrapy from scrapy.org (#939)
This commit is contained in:
parent
111d8aae0f
commit
9f5d5b571f
1 changed files with 31 additions and 0 deletions
31
pages/common/scrapy.md
Normal file
31
pages/common/scrapy.md
Normal file
|
@ -0,0 +1,31 @@
|
|||
# scrapy
|
||||
|
||||
> Web-crawling Framework.
|
||||
|
||||
- Create a project:
|
||||
|
||||
`scrapy startproject {{project_name}}`
|
||||
|
||||
- Create a spider (in project directory):
|
||||
|
||||
`scrapy genspider {{spider_name}} {{website_domain}}`
|
||||
|
||||
- Edit spider (in project directory):
|
||||
|
||||
`scrapy edit {{spider_name}}`
|
||||
|
||||
- Run spider (in project directory):
|
||||
|
||||
`scrapy crawl {{spider_name}}`
|
||||
|
||||
- Fetch a webpage as scrapy sees it and print source in stdout:
|
||||
|
||||
`scrapy fetch {{url}}`
|
||||
|
||||
- Open a webpage in the default browser as scrapy sees it (disable javascript for extra fidelity):
|
||||
|
||||
`scrapy view {{url}}`
|
||||
|
||||
- Open scrapy shell for url, which allows interaction with the page source in python shell (or ipython if available):
|
||||
|
||||
`scrapy shell {{url}}`
|
Loading…
Add table
Reference in a new issue