.. _dl1_datamodel:

.. currentmodule:: ctapipe.io.containers

==============
DL1 Data Model
==============

The DL1 files are HDF5 format files, with the following data set hierarchy.  The tables
should be written with `pytables` (not `h5py`), ideally with the `ctapipe.io.HDF5TableWriter`,
which ensures the unit and other descriptive metadata are attached to the output. Containers marked
with a `+` should be written without their prefix (all others should use column prefixes).

The following describes the contents of data level 1 (DL1) output files
generated by ctapipe (e.g. the `ctapipe-stage1-process` tool which uses the `ctapipe.io.DL1Writer` component to generate output data).


--------------
DL1 Data Model
--------------

This describes data that change per-event.
The following datasets will be written to the group `/dl1/event/` in the  output file:

.. list-table::
    :widths: 25 50 25
    :header-rows: 1

    * - Group/Dataset
      - Description
      - Contents
    * - event/subarray
      - event-wise data pertaining to a subarray
      - (group)
    * - event/subarray/trigger
      - subarray trigger information
      - `EventIndexContainer` +, `CentralTriggerContainer`
    * - event/telescope
      - Per-telescope Per-event information
      - (group)
    * - event/telescope/parameters/tel_{TEL_ID:03d}
      - tables of image parameters (one per telescope)
      - `TelEventIndexContainer` +, `ImageParametersContainer`
    * - event/telescope/images/tel_{TEL_ID:03d}
      - tables of telescope images (one per telescope)
      - `TelEventIndexContainer` +, `DL1CameraContainer`

---------------------
Simulation Data Model
---------------------
    * - /simulation/event/subarray/shower
      - true shower parameters from Monte-Carlo simulation
      - `EventIndexContainer` +, `SimulatedShowerContainer`
    * - /simulation/event/telescope/images/tel_{TEL_ID:03d}
      - simulated camera images 
      - `EventIndexContainer` +, `SimulatedCameraContainer`
    * - /simulation/event/telescope/parameters/tel_{TEL_ID:03d}
      - Parameters derived form the simulated camera images 
      - `EventIndexContainer` +, `ImageParametersContainer`
    * - /simulation/service/shower_distribution
      - simulated shower distribution histograms
      - `SimulatedShowerDistribution`

------------------------
Configuration Data Model
------------------------

The output file should also contain serializations of the instrument, observation (if
applicable), simulation (if applicable) configuration information, written to the
`/configuration` group:

.. list-table::
    :widths: 25 50 25
    :header-rows: 1

    * - Group/Dataset
      - Description
      - Contents
    * - /instrument
      - Serialized `SubarrayDescription`
      - (group)
    * - /instrument/subarray/layout
      - Subarray layout info
      - result of `SubarrayDescription.to_table()` output as HDF5 using `astropy.table`
        functionality
    * - /instrument/telescope/optics
      - telescope optics information
      - result of `SubarrayDescription.to_table(kind='optics')` output as HDF5 using
        `astropy.table` functionality
    * - /instrument/telescope/camera/{CAMERA_ID}
      - camera geometry information
      - result of `CameraGeometry.to_table()` output as HDF5 using `astropy.table`
        functionality
    * - /simulation
      - Monte-Carlo simulation configuration information
      - (group)
    * - /simulation/run
      - Monte-Carlo simulation run information
      - `SimulationConfigContainer`

---------------
Core Provenance
---------------

The root group of the file shall contain all of the "CTA Core Provenance Metadata"
headers as user attributes, with the hierarchy flattened and separated by spaces (e.g.
`"CTA ACTIVITY NAME" = "ctapipe-stage1-process"`)





