Metadata-Version: 2.1
Name: ts-ids-es-json-generator
Version: 1.0.1
Summary: Python utility for generating elasticsearch.json from schema.json for an IDS
Home-page: https://github.com/tetrascience/ts-ids-es-json-generator
Author: tetrascience
Author-email: developers@tetrascience.com
License: Apache License 2.0
Project-URL: Tetra Developer Site, https://developers.tetrascience.com
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jsonref (>=0.2)
Requires-Dist: pydash (>=5.0.0)

# ts-ids-es-json-generator

## Introduction

This simple utility generates `elasticsearch.json` using `schema.json`.

## How to Use

To run the generator, simply provide the program with the location of the IDS root directory:

```
$ pipenv run python -m ids_es_json_generator ~/path/to/ids/folder
```

`elasticsearch.json` may be manually edited after automatic generation (e.g. to update non-searchable fields).

## Output

The script saves `elasticsearch.json` to the same directory you provided.

## Assumptions

The generator operates under the following assumptions:

- It's the IDS designer's responsibility to make sure the schema and `elasticsearch.json` are correct.
  The generator will not validate the IDS for you.

## Restrictions

- JSON pointers are dereferenced to create the mapping, so schemas with circular references or missing references will fail.

## Tests

Run the tests using the following command:

`$ pipenv run python -m pytest`

## How it works

Anywhere an array of objects appears in the schema, a `nested` field type will be created in the output `elasticsearch.json`, including arrays of objects contained anywhere within other arrays of objects.
The steps are:

1. The schema is loaded into a dict with all of its JSON pointers dereferenced using `jsonref`, so that nested fields involving `definitions` are included in the elasticsearch mapping.
2. `find_array_object` will find all arrays of objects and create a dict representing their locations in the schema (see test case `test_array_simple_nested_case` in `__tests__/unit/test_find_array_object.py`).
3. `generate_es_mapping` uses this dict to create the elasticsearch mapping (see test case `test_nested_keys_case` in `__tests__/unit/test_generate_es_mapping.py`).

See the example files and test cases for more example usage.


