Metadata-Version: 2.1
Name: ckanext-dcat_usmetadata
Version: 0.4.0
Summary: DCAT USMetadata Form App for CKAN
Home-page: https://github.com/GSA/ckanext-dcat_usmetadata
Author: Data.gov
Author-email: datagovhelp@gsa.gov
License: AGPL
Description: [![Github Actions](https://github.com/GSA/ckanext-dcat_usmetadata/actions/workflows/test.yml/badge.svg)](https://github.com/GSA/ckanext-dcat_usmetadata/actions)
        [![PyPI version](https://badge.fury.io/py/ckanext-dcat_usmetadata.svg)](https://badge.fury.io/py/ckanext-dcat_usmetadata)
        
        # ckanext-dcat_usmetadata
        
        This extension provides a new dataset form for [inventory.data.gov](https://inventory.data.gov/). The form is tailored to managing metadata meeting the [DCAT-US Schema](https://resources.data.gov/resources/dcat-us/).
        
        ## Usage
        
        ### Dependencies
        
        This module currently depends on the [USMetadata app](https://github.com/GSA/USMetadata) for server-side validation and rendering.
        Make sure it is enabled in CKAN's plugins.
        
        This extension is compatible with these versions of CKAN.
        
        | CKAN version | Compatibility |
        | ------------ | ------------- |
        | <=2.8        | no            |
        | 2.9          | yes           |
        
        ### Installation
        
        To install this package, activate CKAN virtualenv (e.g. "source /path/to/virtenv/bin/activate"), then run
        
        ```
        (virtualenv) pip install ckanext-dcat-usmetadata
        ```
        
        In your CKAN .ini file add `dcat_usmetadata` to your enabled plugins:
        
        `ckan.plugins = [YOUR PLUGINS HERE...] dcat_usmetadata`
        
        ### Commands
        
        ### publishers-import
        
        This extension adds a new CLI command for importing publishers linked to CKAN
        organizations. The [list of
        publishers](https://docs.google.com/spreadsheets/d/1BWpXWswnnMRaNazMCYrzl4W5JMWpGuyW0Wo2YaNH9jY/edit?usp=sharing)
        should be exported in a CSV data and it should have the following structure
        (note the headers):
        
        ```
        organization,publisher,publisher_1,publisher_2,publisher_3,publisher_4,publisher_5
        agricultural-marketing-service-department-of-agriculture,Department of Agriculture,Agricultural Marketing Service,,,,
        ars-usda-gov,Department of Agriculture,Agricultural Research Service,,,,
        aphis-usda-gov,Department of Agriculture,Animal and Plant Health Inspection Service,,,,
        risk-management-agency-department-of-agriculture,Department of Agriculture,Departmental Management,,,,
        usda-gov,Department of Agriculture,Office of Chief Information Officer,,,,
        usda-gov,Department of Agriculture,Economic Research Service,,,,
        usda-gov,Department of Agriculture,Farm Service Agency,,,,
        usda-gov,Department of Agriculture,Food and Nutrition Service,,,,
        usda-gov,Department of Agriculture,Food Safety and Inspection Service,,,,
        usda-gov,Department of Agriculture,Foreign Agricultural Service,,,,
        usda-gov,Department of Agriculture,National Agricultural Statistics Service,,,,
        usda-gov,Department of Agriculture,National Institute of Food and Agriculture,,,,
        usda-gov,Department of Agriculture,Natural Resources Conservation Service,Colorado State University,,,
        usda-gov,Department of Agriculture,Rural Development,,,,
        usda-gov,Department of Agriculture,GIPSA,Federal Grain Inspection Service,,,
        usda-gov,Department of Agriculture,Natural Resources Conservation Service,,,,
        usda-gov,Department of Agriculture,US Forest Service,,,,
        ```
        
        Each CKAN organization must have its own list of publishers.
        
        Example of running the command:
        
            $ ckan dcat-usmetadata import-publishers /path/to/publishers.csv
        
        ## Development
        
        ### Prerequisites
        
        These tools are required for development.
        
        - [Node.js](https://nodejs.org/) 12.x
        - [Yarn](https://yarnpkg.com/) 1.22.x
        - [Cypress](https://www.cypress.io/) 6.0.0+
        
        ### Setup
        
        Install Node.js dependencies.
        
        ```bash
        yarn install
        ```
        
        Build the JS application. The new build files can be found in `ckanext/dcat_usmetadata/public` folder.
        
        ```bash
        yarn build
        ```
        
        Build and start the docker containers.
        
        ```bash
        yarn build:docker
        yarn up
        ```
        
        ## Testing
        
        There are several levels of testing:
        
        | Suite                     | Description                         | Command                  |
        | ------------------------- | ----------------------------------- | ------------------------ |
        | Unit tests for the JS app | Tests for the React app.            | `yarn test:metadata-app` |
        | CKAN extension tests      | Python tests using Nosetests        | `yarn test`              |
        | End to end tests          | Cypress tests against inventory app | `yarn e2e`               |
        | End to end tests (local)  | Cypress tests against inventory app | `yarn front-end`         |
        
        ## Linting
        
        Lint the python code.
        
        ```bash
        yarn lint:python
        ```
        
        Lint the JavaScript code.
        
        ```bash
        yarn lint:js
        ```
        
        ## Metadata app
        
        The Metadata app is a [Create React App](https://create-react-app.dev/)-bootstrapped project.
        
        To run the app use `yarn && yarn start:metadata-app` command.
        
        _TODO briefly describe how the metadata application relates to the CKAN
        extension._
        
        ### Development
        
        This project uses [cosmos](https://reactcosmos.org/) for development.
        
        Run CKAN locally (`yarn up`), get the Admin user's API Key and add it in `/metadata-app/public/index.html` as `data-apiKey` attribute of the `div` element. Add a test org for development purposes.
        
        Run `yarn && yarn cosmos` to start the cosmos server, which will watch the `metadata-app/src` directory for changes.
        
        Run the unit tests:
        
        ```bash
        yarn test:metadata-app
        # To run it in watch mode:
        yarn test:metadata-app:watch
        ```
        
        ### Update Jest snapshots
        
        Some tests render a fixture component with [Jest](https://jestjs.io/) and then
        match against a known good snapshot (HTML rendering) of the component. When you
        edit a component, you'll usually have to update the snapshot and inspect the
        diff to make sure all changes are as intended.
        
        ```bash
        yarn test:metadata-app --updateSnapshot
        ```
        
        ## Local development and end-to-end testing
        
        We use the [inventory app](https://github.com/GSA/inventory-app) locally for development and end-to-end (e2e) testing.
        
        To build the latest JS code and update assets in the CKAN extension, you can run the following command from the root directory of this project:
        
        ```
        yarn build
        ```
        
        For convenience, we have prepared a single script that you can run to perform end-to-end tests locally. Don't forget to `yarn build` prior to running e2e tests, otherwise, the tests could run against older builds:
        
        ```bash
        yarn e2e
        ```
        
        Note, it may be necessary to remove cached images when rebuilding the inventory app docker container, in order to ensure that the new usmetadata-app template is included in the build. If you want to make sure that you aren't using cached builds, you can try:
        
        ```bash
        docker-compose build --no-cache --pull ckanext-dcat_usmetadata_app
        ```
        
        To run e2e tests interactively use:
        
        ```bash
        yarn e2e:interactive
        ```
        
        ## Publishing a new version of the extension
        
        We publish this extension to PyPI - https://pypi.org/project/ckanext-dcat-usmetadata/. This is done by CI job that is triggered on tagged commit on master branch. When you need to release a new version of the extension, you need to:
        
        0. Create a new branch for releasing a new version of the extension. You can name your branch with the following convention: `release/x.y.z`;
        1. Update version in `setup.py`;
        2. Get your PR merged to master branch;
        3. Tag the merged commit with the new version (`git tag $version`).
        
        In the CI job, the following is done for tagged commits:
        
        - It builds the JS bundles and puts them into the relevant directory so the extension can use them;
        - It runs integration tests to make sure everything is working as expected;
        - It packages the extension and publishes it to PyPI.
        
        Below is a sequence diagram demonstrating the flow (you need to have `github + mermaid` chrome extension to view it):
        
        ```mermaid
        sequenceDiagram
            Developer->>Git: Push tagged commit to master branch
            Git-->>CI/CD: Trigger deployment
            CI/CD-->>CI/CD: Build assets (JS bundles)
            CI/CD-->>CI/CD: Build python package
            CI/CD-->>CI/CD: Run tests
            CI/CD-->>PyPI: Publish the package
            Inventory-->>PyPI: Install
        ```
        
        ## Ways to Contribute
        
        The Data.gov team manages all Data.gov updates, bugs, and feature additions via GitHub's public [issue tracker](https://github.com/GSA/ckanext-dcat_usmetadata/issues) in this repository.
        
        If you do not already have a GitHub account, you can [sign up for GitHub here](https://github.com/). In the spirit of open source software, everyone is encouraged to help improve this project. Here are some ways you can contribute:
        
        - by reporting bugs
        - by suggesting new features
        - by translating content to a new language
        - by writing or editing documentation
        - by writing specifications
        - by writing code and documentation (no pull request is too small: fix typos, add code comments, clean up inconsistent whitespace)
        - by reviewing pull requests.
        - by closing issues
        
        ### Submit Great Issues
        
        - Before submitting a new issue, check to make sure a similar issue isn't already open. If one is, contribute to that issue thread with your feedback.
        - When submitting a bug report, please try to provide as much detail as possible, i.e. a screenshot or gist that demonstrates the problem, the technology you are using, and any relevant links.
        
        ### Ready for your Help
        
        Issues labeled [help wanted](https://github.com/GSA/data.gov/labels/help%20wanted) make it easy for you to find ways you can contribute today.
        
        ## Public Domain
        
        This project constitutes a work of the United States Government and is not subject to domestic copyright protection under 17 USC § 105. Additionally, we waive copyright and related rights in the work worldwide through the CC0 1.0 [Universal public domain dedication](https://creativecommons.org/publicdomain/zero/1.0/).
        
        All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.
        
Keywords: CKAN
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
Classifier: Programming Language :: Python :: 3
Description-Content-Type: text/markdown
