Metadata-Version: 1.1
Name: webby
Version: 1.2.1
Summary: Web Crawling and HTML Parsing Library
Home-page: https://github.com/matt-git/webby
Author: Matthew McCullough
Author-email: gitwebby@gmail.com
License: MIT
Description: Webby v 1.0.7

        

        

        Versions are currently out if sync. Give me some time to fix version control!

        

        Webby is built for Python v2.7.

        

        

        constants:

        GOOGLE = 'http://www.google.com'

        AMAZON = 'http://www.amazon.com'

        FACEBOOK = 'http://www.facebook.com'

        LINKEDIN = 'http://www.linkedin.com'

        

        1) The Crawler Object:

        

          - Crawler(root) -> The constructor takes in a root url.

          

          - Crawler.info() 

            Displays urllib2 current information.

            

            

          - Crawler.get_url()

            Returns current URL

            

            

          - Crawler.get_hyperlinks()

            quick function that returns all hyperlinks that exist on the webpage

            

            

          - Crawler.get_full_links()

            Returns a list of the full URL's occuring on the webpage

            

            

          - Crawler.search(keywords, url=None)

            The search function appends given words to the URL and navigates to the search page.

          

          

          - Crawler.open(url)

            Opens up that specific URL in the crawler.

            

          - Crawler.crawl(length=10, directory=None)

          

          

        

          from webby import *

          

          root = "http://example.com"

          

          spider = Crawler(root)   #The spider automatically connects to the URL and waits

          

          parse = Parser(spider.source)

          

          

          

          

          

          

          

        
Keywords: Web crawling data mining
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Build Tools :: Data Mining
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
