Metadata-Version: 2.1
Name: mathgenproject-jgolden17
Version: 0.1.0
Summary: Mathematics Genealogy Project Spider
Home-page: https://github.com/jgolden17/mathgenproject
Author: Jonathan Golden
License: UNKNOWN
Description: # mathgenproject
        > Mathematics Genealogy Project Spider
        
        A webspider for the [Mathematics Genealogy Project](https://genealogy.math.ndsu.nodak.edu/index.php).
        
        ## Installation
        
        Use the package manager pip to install the spider.
        
        ```bash
        pip install mathgenproject
        ```
        
        ## Usage
        
        Define a pipeline through which to process each mathematician returned from the spider.
        
        ```python
        class MyPipeline(object):
            def open_spider(self, spider):
                ...
        
            def process_item(self, item, spider):
                print(item['name'])
                return item
        
            def close_spider(self, spider):
                ...
        
        ```
        
        Run the spider using scrapy's `CrawlerProcess`, passing in the mathematician's MGP ID.
        
        ```python
        from scrapy.crawler import CrawlerProcess
        from mathgenproject.spiders import MathGenProjectSpider
        
        process = CrawlerProcess(settings={
            'FEED_FORMAT': 'json',
            'FEED_URI': 'items.json',
            'ITEM_PIPELINES': {
                'MyPipeline': 300,
            },
        })
        
        process.crawl(MathGenProjectSpider, mgp_id='216087')
        process.start()
        ```
        
        ## Contributing
        
        Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
        
        ## License
        
        [MIT](https://choosealicense.com/licenses/mit/)
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown
