Recent Changes - Search:

Making

Photography?

Coding

Cooking

Projects

PmWiki

edit SideBar

Scrapy

Testing out web scraping.

Installing

  • (also works with Anaconda) @@pip install scrapy
  • if error Library not loaded: libssl.1.0.0.dylib appears, make sure the proper symlinks were created for libssl, as suggested here. In my case, the symlinks for ssl.1.0 did not exist, so I just had to run
    • $ sudo ln -s /Users/YOURUSERNAME/anaconda/lib/libssl.1.0.0.dylib /usr/lib
    • $ sudo ln -s /Users/YOURUSERNAME/anaconda/lib/libcrypto.1.0.0.dylib /usr/lib

Reference

  • Scrapy documentation
  • Scrapy tutorial
    • For tutorial to work, items.py has to be set to the example in the tutorial.
    • Also, must add from tutorial.items import DmozItem at the top of the dmoz_spider.py

Related

Edit - History - Print - Recent Changes - Search
Page last modified on February 22, 2015, at 15:27 EST