Metadata-Version: 2.5
Name: swh.vulns
Version: 0.3.0
Summary: Labeling of objects of the Software Heritage archives with known vulnerabilities affecting them.
Project-URL: Homepage, https://gitlab.softwareheritage.org/swh/devel/swh-vulns
Project-URL: Bug Reports, https://gitlab.softwareheritage.org/swh/devel/swh-vulns/-/issues
Project-URL: Funding, https://www.softwareheritage.org/donate
Project-URL: Documentation, https://docs.softwareheritage.org/devel/swh-vulns/
Project-URL: Source, https://gitlab.softwareheritage.org/swh/devel/swh-vulns.git
Author-email: Software Heritage developers <swh-devel@inria.fr>
License-File: AUTHORS
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11
Requires-Dist: swh-core[http]
Requires-Dist: swh-scheduler
Requires-Dist: swh-storage
Provides-Extra: luigi
Requires-Dist: swh-graph[luigi]; extra == 'luigi'
Provides-Extra: testing
Requires-Dist: grpcio; extra == 'testing'
Requires-Dist: pytest-click; extra == 'testing'
Requires-Dist: pytest>=8.1; extra == 'testing'
Requires-Dist: requests-mock; extra == 'testing'
Requires-Dist: swh-core[http]; extra == 'testing'
Requires-Dist: swh-core[testing]; extra == 'testing'
Requires-Dist: swh-graph[testing]; extra == 'testing'
Requires-Dist: swh-loader-git; extra == 'testing'
Requires-Dist: swh-osv; extra == 'testing'
Requires-Dist: swh-scheduler; extra == 'testing'
Requires-Dist: swh-storage; extra == 'testing'
Requires-Dist: swh-storage[pytest]; extra == 'testing'
Requires-Dist: types-grpcio; extra == 'testing'
Requires-Dist: types-protobuf; extra == 'testing'
Description-Content-Type: text/x-rst

Software Heritage - Labeling of objects in the archive with known vulnerabilities affecting them.
=================================================================================================

Software vulnerabilities reported in the `Open Source Vulnerabilities database
<https://osv.dev/>`__ were processed so they can be associated to objects in the
Software Heritage archive.

Software Heritage graph labeling with vulnerabilities
-----------------------------------------------------

Labeling of objects of the Software Heritage archive with known vulnerabilities affecting them.

Deployment
^^^^^^^^^^

Install dependencies:

1. ``cargo install --locked swh-graph swh-vulns-grpc-server``
2. ``pip3 install swh.graph``

Get data files:

1. Get swh-graph: ``swh graph download --name 2026-03-02`` (~15TB)
2. From ``s3://softwareheritage/derived_datasets/2026-03-02/vulnerabilities/``,
   download :`all.sqlite`, :`commit2vuln_without_cherrypicks.*`, and :`connected_components.wccs`,
   about 10GB. (Ignore other files in the directory, they weigh about 1.5TB.)

Run this:

.. code-block:: bash

    swh-vulns-grpc-serve \
        --db ./2026-03-02/all.sqlite \
        --commit2vuln ./2026-03-02/commit2vuln_without_cherrypicks \
        --subgraphwccs ./2026-03-02/connected_components.wccs \
        --graph ./2026-03-02/graph \
        --bind 127.0.0.1:<port>

note that ``--commit2vuln`` and ``--graph`` should omit the file extension


Database of OSV vulnerability events mapped to SWHIDs
-----------------------------------------------------

Each OSV vulnerability report can be processed so events (vulnerability introduction,
vulnerability fix, ...) can be associated to releases and revisions in the Software
Heritage archive. These enriched vulnerability events are then stored into a
PostgreSQL database.

Schema
^^^^^^

Vulnerability events are stored in an ``osv_vulnerability_event`` table with
the following schema:

.. code-block:: sql

    create table osv_vulnerability_event
    (
        id                      bigserial  not null,
        vulnerability_id        text       not null,
        vulnerability_severity  text,
        event_type              text       not null,
        origin_url              text       not null,
        swhid                   text,
        version                 text
    );

The description of the columns is given below:

- ``vulnerability_id``: vulnerability identifier from OSV,
  for instance `CVE-2026-9358 <https://osv.dev/vulnerability/CVE-2026-9358>`__.

- ``vulnerability_severity``: severity of the vulnerability,
  see `possible values <https://ossf.github.io/osv-schema/#severitytype-field>`__

- ``event_type``: type of vulnerability event, possible values are ``introduced``,
  ``fixed``, ``last_affected`` or ``limit``;
  see `more details <https://ossf.github.io/osv-schema/#severitytype-field>`__

- ``origin_url``: URL of software origin affected by the vulnerability

- ``swhid``: SWHID of a release or a revision related to the vulnerability_event,
  for ``introduced`` events it can be null which means vulnerability affects all
  versions of a software origin prior its resolution.

- ``version``: software origin version associated to the SWHID

Deployment
^^^^^^^^^^

The PostgreSQL database can be created with the following command.

.. code-block:: bash

    $ createdb swh-osv-vulnerabilities

The created database should then be referenced in the ``~/.pg_service.conf`` by adding
the following section:

.. code-block:: ini

    [swh-osv-vulnerabilities]
    dbname=swh-osv-vulnerabilities
    host=<db_host>
    port=<db_port>
    user=<db_user>

Database schema can then be created or upgraded using the following commands:

.. code-block:: bash

    $ POSTGRES_DB=swh-osv-vulnerabilities
    $ swh db init-admin -d service=$POSTGRES_DB vulns
    $ swh db init -d service=$POSTGRES_DB vulns
    $ swh db upgrade --non-interactive -d service=$POSTGRES_DB vulns

Populating the database
^^^^^^^^^^^^^^^^^^^^^^^

Database can be populated on a regular basis through the execution of dedicated
celery tasks than can be created using a CLI command.

The celery worker and the CLI command must use the following configuration file
and set its path in the ``SWH_CONFIG_FILENAME`` environment variable:

.. code-block:: yaml

    storage:
      cls: pipeline
      steps:
        - cls: retry
        - cls: remote
          url: <storage_url>

    vulnerabilities_storage:
      cls: remote
      url: <storage_url>

    celery:
      task_broker: <broker_url>
      task_modules:
        - swh.vulns.osv.tasks
      task_queues:
        - swh.vulns.osv.tasks.ProcessOSVReport

    scheduler:
      cls: remote
      url: <scheduler_url>

Once a celery worker was configured to execute the OSV report processing tasks,
those can be created by executing the following CLI command:

.. code-block:: bash

    $ swh vulns osv create-report-processing-tasks

It creates one task per OSV report to process, currently only the git related ones are considered.

Subsequent calls of that command will only create tasks for new reports or reports modified
since last processing.

Querying the database from Python
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A dedicated Python interface is available to easily query the database content,
see some example below:


.. code-block:: python

    >>> from swh.vulns.storage import get_vulnerabilities_storage
    >>> from swh.model.swhids import CoreSWHID

    # instantiate remote storage client
    >>> vulns_storage = get_vulnerabilities_storage("remote", url="<storage_url>")

    # get vulnerability events by vulnerability identifier
    >>> vulns_storage.osv_vulnerability_event_get_by_id(["CVE-2026-9227"])
    [OSVVulnerabilityEvent(vulnerability_id='CVE-2026-9227', event_type='fixed', origin_url='https://github.com/cssigniter/gutenbee', swhid=CoreSWHID.from_string('swh:1:rev:bde934cdecf67a4de1d6548cc1fc6c59bc6690e5'), version=None, vulnerability_severity='CVSS_V3')]

    # get vulnerability events by SWHID
    >>> vulns_storage.osv_vulnerability_event_get_by_swhid([CoreSWHID.from_string("swh:1:rev:5c4568a05a0a62b5947c55f68f9f2ecfb90a4f12")])
    [OSVVulnerabilityEvent(vulnerability_id='CVE-2016-0718', event_type='introduced', origin_url='https://github.com/python/cpython', swhid=CoreSWHID.from_string('swh:1:rev:5c4568a05a0a62b5947c55f68f9f2ecfb90a4f12'), version='3.6.0', vulnerability_severity='CVSS_V3'), OSVVulnerabilityEvent(vulnerability_id='CVE-2021-4189', event_type='introduced', origin_url='https://github.com/python/cpython', swhid=CoreSWHID.from_string('swh:1:rev:5c4568a05a0a62b5947c55f68f9f2ecfb90a4f12'), version='3.6.0', vulnerability_severity='CVSS_V3'), OSVVulnerabilityEvent(vulnerability_id='CVE-2019-15903', event_type='introduced', origin_url='https://github.com/python/cpython', swhid=CoreSWHID.from_string('swh:1:rev:5c4568a05a0a62b5947c55f68f9f2ecfb90a4f12'), version='3.6.0', vulnerability_severity='CVSS_V3')]

    # get vulnerability events by origin URL
    >>> vulns_storage.osv_vulnerability_event_get_by_origin_url(["git://git.gnupg.org/libgcrypt.git"])
    [OSVVulnerabilityEvent(vulnerability_id='CVE-2026-41989', event_type='fixed', origin_url='git://git.gnupg.org/libgcrypt.git', swhid=CoreSWHID.from_string('swh:1:rev:089ff0edf61ba829714a568778087eeac5b0df82'), version=None, vulnerability_severity='CVSS_V3'), OSVVulnerabilityEvent(vulnerability_id='CVE-2026-41989', event_type='introduced', origin_url='git://git.gnupg.org/libgcrypt.git', swhid=CoreSWHID.from_string('swh:1:rel:f65dd9e5c43f1ec66b14fec9a4b0fee0d32ca7df'), version='1.8.8', vulnerability_severity='CVSS_V3'), OSVVulnerabilityEvent(vulnerability_id='CVE-2026-41989', event_type='introduced', origin_url='git://git.gnupg.org/libgcrypt.git', swhid=CoreSWHID.from_string('swh:1:rev:efd5e1e7b4e7861b53eafdbf197fd6d4ff6f45e1'), version=None, vulnerability_severity='CVSS_V3'), OSVVulnerabilityEvent(vulnerability_id='CVE-2026-41989', event_type='introduced', origin_url='git://git.gnupg.org/libgcrypt.git', swhid=CoreSWHID.from_string('swh:1:rev:d3d4803ca1b1d50fdb0c8fa2891c75e0229ff25f'), version='1.8.8', vulnerability_severity='CVSS_V3'), OSVVulnerabilityEvent(vulnerability_id='CVE-2026-41989', event_type='fixed', origin_url='git://git.gnupg.org/libgcrypt.git', swhid=CoreSWHID.from_string('swh:1:rev:d365a41094571f2cce18f27b53ffdceb540f77bb'), version=None, vulnerability_severity='CVSS_V3'), OSVVulnerabilityEvent(vulnerability_id='CVE-2026-41989', event_type='introduced', origin_url='git://git.gnupg.org/libgcrypt.git', swhid=CoreSWHID.from_string('swh:1:rev:9d94d7846cde272b8b1519ba96e53967bf0b90d2'), version=None, vulnerability_severity='CVSS_V3'), OSVVulnerabilityEvent(vulnerability_id='CVE-2026-41989', event_type='fixed', origin_url='git://git.gnupg.org/libgcrypt.git', swhid=CoreSWHID.from_string('swh:1:rev:efc346430901b84f1f580a147191624d7ded0db6'), version=None, vulnerability_severity='CVSS_V3')]

