Metadata-Version: 2.5
Name: biocframe
Version: 0.8.1
Summary: Flexible dataframe representation to support nested structures.
Project-URL: Homepage, https://github.com/BiocPy/biocframe
Project-URL: Documentation, https://biocpy.github.io/BiocFrame/
Project-URL: Source, https://github.com/BiocPy/biocframe
Project-URL: Bug Tracker, https://github.com/BiocPy/biocframe/issues
Author-email: Jayaram Kancherla <jayaram.kancherla@gmail.com>
License: The MIT License (MIT)
        
        Copyright (c) 2022 Genentech, Inc.
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: AUTHORS.md
License-File: LICENSE.txt
Keywords: bioconductor,biocpy,bioinformatics,dataframe
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: biocutils>=0.3.3
Requires-Dist: numpy>=1.26.0
Provides-Extra: optional
Requires-Dist: pandas>=2.0.0; extra == 'optional'
Requires-Dist: polars>=0.20.0; extra == 'optional'
Provides-Extra: testing
Requires-Dist: pandas>=2.0.0; extra == 'testing'
Requires-Dist: polars>=0.20.0; extra == 'testing'
Requires-Dist: pytest-cov>=4.0.0; extra == 'testing'
Requires-Dist: pytest>=7.0.0; extra == 'testing'
Requires-Dist: setuptools>=68.0.0; extra == 'testing'
Description-Content-Type: text/markdown

<!-- These are examples of badges you might want to add to your README:
     please update the URLs accordingly

[![Built Status](https://api.cirrus-ci.com/github/<USER>/BiocFrame.svg?branch=main)](https://cirrus-ci.com/github/<USER>/BiocFrame)
[![ReadTheDocs](https://readthedocs.org/projects/BiocFrame/badge/?version=latest)](https://BiocFrame.readthedocs.io/en/stable/)
[![Coveralls](https://img.shields.io/coveralls/github/<USER>/BiocFrame/main.svg)](https://coveralls.io/r/<USER>/BiocFrame)
[![PyPI-Server](https://img.shields.io/pypi/v/BiocFrame.svg)](https://pypi.org/project/BiocFrame/)
[![Conda-Forge](https://img.shields.io/conda/vn/conda-forge/BiocFrame.svg)](https://anaconda.org/conda-forge/BiocFrame)
[![Monthly Downloads](https://pepy.tech/badge/BiocFrame/month)](https://pepy.tech/project/BiocFrame)
[![Twitter](https://img.shields.io/twitter/url/http/shields.io.svg?style=social&label=Twitter)](https://twitter.com/BiocFrame)
-->

[![Project generated with PyScaffold](https://img.shields.io/badge/-PyScaffold-005CA0?logo=pyscaffold)](https://pyscaffold.org/)
[![PyPI-Server](https://img.shields.io/pypi/v/BiocFrame.svg)](https://pypi.org/project/BiocFrame/)
![Unit tests](https://github.com/BiocPy/BiocFrame/actions/workflows/run-tests.yml/badge.svg)

# Bioconductor-like data frames

## Overview

This package implements the `BiocFrame` class, a Bioconductor-friendly alternative to Pandas `DataFrame`.
The main advantage is that the `BiocFrame` makes no assumption on the types of the columns -
as long as an object has a length (`__len__`) and slicing methods (`__getitem__`), it can be used inside a `BiocFrame`.
This allows us to accept arbitrarily complex objects as columns, which is often the case in Bioconductor objects.

To get started, install the package from [PyPI](https://pypi.org/project/biocframe/):

```shell
pip install biocframe

# To install optional dependencies
pip install biocframe[optional]
```

## Quick Examples

### Genomic Annotation Data

Genomic data often requires storing coordinates, annotations, and metadata together:

```python
# Gene annotation with nested structures
gene_annotations = BiocFrame({
    "gene_id": ["GENE1", "GENE2", "GENE3"],
    "symbol": ["BRCA1", "TP53", "EGFR"],
    "location": BiocFrame({
        "chromosome": ["chr17", "chr17", "chr7"],
        "start": [43044295, 7668422, 55019017],
        "end": [43125483, 7687550, 55211628],
        "strand": ["-", "-", "+"],
    }),
    "transcripts": [
        ["NM_007294", "NM_007297", "NM_007300"],
        ["NM_000546"],
        ["NM_005228", "NM_201282"],
    ],
    "pathways": [
        ["DNA repair", "Cell cycle"],
        ["Apoptosis", "Cell cycle", "DNA repair"],
        ["Cell growth", "Signal transduction"],
    ],
}, row_names=["ENSG00000012048", "ENSG00000141510", "ENSG00000146648"])

print(gene_annotations)
```

### Multi-Omics Data Integration

When combining different types of omics data with varying structures:

```python
# Multi-omics data with different measurement types
multi_omics = BiocFrame({
    "sample_id": ["S1", "S2", "S3"],
    "rna_seq": np.array([
        [100, 200, 150],
        [300, 250, 180],
        [120, 220, 160],
    ], dtype=np.float32),
    "methylation": BiocFrame({
        "cg0001": [0.85, 0.92, 0.78],
        "cg0002": [0.45, 0.38, 0.52],
        "cg0003": [0.12, 0.15, 0.10],
    }),
    "clinical": BiocFrame({
        "age": [45, 52, 38],
        "gender": ["M", "F", "F"],
        "diagnosis": ["Type A", "Type B", "Type A"],
    }),
}, column_data=BiocFrame({
    "data_type": ["identifier", "expression", "epigenetic", "clinical"],
    "source": ["lab", "sequencer", "array", "EHR"],
}))

print(multi_omics)
print("\nColumn metadata:")
print(multi_omics.get_column_data())
```

### Hierarchical Data Structures

For data with natural hierarchies (e.g., samples → patients → cohorts):

```python
# Hierarchical clinical trial data
clinical_trial = BiocFrame({
    "patient_id": ["P001", "P002", "P003"],
    "cohort": ["A", "A", "B"],
    "samples": [
        BiocFrame({
            "sample_id": ["S001", "S002"],
            "collection_date": ["2024-01-01", "2024-01-15"],
            "vital_status": ["alive", "alive"],
        }),
        BiocFrame({
            "sample_id": ["S003", "S004", "S005"],
            "collection_date": ["2024-01-02", "2024-01-16", "2024-01-30"],
            "vital_status": ["alive", "alive", "deceased"],
        }),
        BiocFrame({
            "sample_id": ["S006"],
            "collection_date": ["2024-01-03"],
            "vital_status": ["alive"],
        }),
    ],
}, metadata={
    "trial_name": "PHASE_III_STUDY",
    "start_date": "2024-01-01",
    "status": "ongoing",
})

print(clinical_trial)
```

## Construction

To construct a `BiocFrame` object, simply provide the data as a dictionary.

```python
from biocframe import BiocFrame

obj = {
    "ensembl": ["ENS00001", "ENS00002", "ENS00003"],
    "symbol": ["MAP1A", "BIN1", "ESR1"],
}
bframe = BiocFrame(obj)
print(bframe)
## BiocFrame with 3 rows and 2 columns
##      ensembl symbol
##       <list> <list>
## [0] ENS00001  MAP1A
## [1] ENS00002   BIN1
## [2] ENS00003   ESR1
```

You can specify complex objects as columns, as long as they have some "length" equal to the number of rows.
For example, we can nest a `BiocFrame` inside another `BiocFrame`:

```python
obj = {
    "ensembl": ["ENS00001", "ENS00002", "ENS00002"],
    "symbol": ["MAP1A", "BIN1", "ESR1"],
    "ranges": BiocFrame({
        "chr": ["chr1", "chr2", "chr3"],
        "start": [1000, 1100, 5000],
        "end": [1100, 4000, 5500]
    }),
}

bframe2 = BiocFrame(obj, row_names=["row1", "row2", "row3"])
print(bframe2)
## BiocFrame with 3 rows and 3 columns
##       ensembl symbol         ranges
##        <list> <list>    <BiocFrame>
## row1 ENS00001  MAP1A chr1:1000:1100
## row2 ENS00002   BIN1 chr2:1100:4000
## row3 ENS00002   ESR1 chr3:5000:5500
```

## Extracting data

Properties can be accessed directly from the object:

```python
print(bframe.shape)
## (3, 2)

print(bframe.get_column_names())
## ['ensembl', 'symbol']

print(bframe.column_names) # same as above
## ['ensembl', 'symbol']
```

We can fetch individual columns:

```python
bframe.get_column("ensembl")
## ['ENS00001', 'ENS00002', 'ENS00003']

bframe["ensembl"]
## ['ENS00001', 'ENS00002', 'ENS00003']
```

And we can get individual rows as a dictionary:

```python
bframe.get_row(2)
## {'ensembl': 'ENS00003', 'symbol': 'ESR1'}
```

To extract a subset of the data in the `BiocFrame`, we use the subset (`[]`) operator.
This accepts different subsetting arguments like a boolean vector, a `slice` object, a sequence of indices, or row/column names.

```python
sliced = bframe[1:2, [True, False, False]]
print(sliced)
## BiocFrame with 1 row and 1 column
##      column1
##       <list>
## [0] ENS00002

sliced = bframe[[0,2], ["symbol", "ensembl"]]
print(sliced)
## BiocFrame with 2 rows and 2 columns
##     symbol  ensembl
##     <list>   <list>
## [0]  MAP1A ENS00001
## [1]   ESR1 ENS00003

# Short-hand to get a single column:
bframe["ensembl"]
## ['ENS00001', 'ENS00002', 'ENS00003']
```

## Setting data

### Preferred approach

To set `BiocFrame` properties, we encourage a functional style of programming that avoids mutating the object.
This avoids inadvertent modification of `BiocFrame`s that are part of larger data structures.

```python
modified = bframe.set_column_names(["column1", "column2"])
print(modified)
## BiocFrame with 3 rows and 2 columns
##      column1 column2
##       <list>  <list>
## [0] ENS00001   MAP1A
## [1] ENS00002    BIN1
## [2] ENS00003    ESR1

# Original is unchanged:
print(bframe.get_column_names())
## ['ensembl', 'symbol']
```

To add new columns, or replace existing columns:

```python
modified = bframe.set_column("symbol", ["A", "B", "C"])
print(modified)
## BiocFrame with 3 rows and 2 columns
##      ensembl symbol
##       <list> <list>
## [0] ENS00001      A
## [1] ENS00002      B
## [2] ENS00003      C

modified = bframe.set_column("new_col_name", range(2, 5))
print(modified)
## BiocFrame with 3 rows and 3 columns
##      ensembl symbol new_col_name
##       <list> <list>      <range>
## [0] ENS00001  MAP1A            2
## [1] ENS00002   BIN1            3
## [2] ENS00003   ESR1            4
```

Change the row or column names:

```python
modified = bframe.\
    set_column_names(["FOO", "BAR"]).\
    set_row_names(['alpha', 'bravo', 'charlie'])
print(modified)
## BiocFrame with 3 rows and 2 columns
##              FOO    BAR
##           <list> <list>
##   alpha ENS00001  MAP1A
##   bravo ENS00002   BIN1
## charlie ENS00003   ESR1
```

We also support Bioconductor's metadata concepts, either along the columns or for the entire object:

```python
modified = bframe.\
    set_metadata({ "author": "Jayaram Kancherla" }).\
    set_column_data(BiocFrame({"column_source": ["Ensembl", "HGNC" ]}))
print(modified)
## BiocFrame with 3 rows and 2 columns
##      ensembl symbol
##       <list> <list>
## [0] ENS00001  MAP1A
## [1] ENS00002   BIN1
## [2] ENS00003   ESR1
## ------
## column_data(1): column_source
## metadata(1): author
```

### The other way

Properties can also be set by direct assignment for in-place modification.
We prefer not to do it this way as it can silently mutate ``BiocFrame`` instances inside other data structures.
Nonetheless:

```python
testframe = BiocFrame({ "A": [1,2,3], "B": [4,5,6] })
testframe.column_names = ["column1", "column2" ]
print(testframe)
## BiocFrame with 3 rows and 2 columns
##     column1 column2
##      <list>  <list>
## [0]       1       4
## [1]       2       5
## [2]       3       6
```

Similarly, we could set or replace columns directly:

```python
testframe["column2"] = ["A", "B", "C"]
testframe[1:3, ["column1","column2"]] = BiocFrame({"x":[4, 5], "y":["E", "F"]})
## BiocFrame with 3 rows and 2 columns
##     column1 column2
##      <list>  <list>
## [0]       1       A
## [1]       4       E
## [2]       5       F
```

These assignments are the same as calling the corresponding `set_*()` methods with `in_place = True`.
It is best to do this only if the `BiocFrame` object is not being used anywhere else;
otherwise, it is safer to just create a (shallow) copy via the default `in_place = False`.

## Combining objects

**BiocFrame** implements methods for the various `combine` generics from [**BiocUtils**](https://github.com/BiocPy/biocutils).
So, for example, to combine by row:

```python
import biocutils

bframe1 = BiocFrame(
    {
        "odd": [1, 3, 5, 7, 9],
        "even": [0, 2, 4, 6, 8],
    }
)

bframe2 = BiocFrame(
    {
        "odd": [11, 33, 55, 77, 99],
        "even": [0, 22, 44, 66, 88],
    }
)

combined = biocutils.combine_rows(bframe1, bframe2)
print(combined)
## BiocFrame with 10 rows and 2 columns
##        odd   even
##     <list> <list>
## [0]      1      0
## [1]      3      2
## [2]      5      4
## [3]      7      6
## [4]      9      8
## [5]     11      0
## [6]     33     22
## [7]     55     44
## [8]     77     66
## [9]     99     88
```

Similarly, to combine by column:

```python
bframe3 = BiocFrame(
    {
        "foo": ["A", "B", "C", "D", "E"],
        "bar": [True, False, True, False, True]
    }
)

combined = biocutils.combine_columns(bframe1, bframe3)
print(combined)
BiocFrame with 5 rows and 4 columns
       odd   even    foo    bar
    <list> <list> <list> <list>
[0]      1      0      A   True
[1]      3      2      B  False
[2]      5      4      C   True
[3]      7      6      D  False
[4]      9      8      E   True
```

By default, both methods above assume that the number and identity of columns (for `combine_rows()`) or rows (for `combine_columns()`) are the same across objects.
If this is not the case, e.g., with different columns across objects, we can use **BiocFrame**'s `relaxed_combine_rows()` instead:

```python
from biocframe import relaxed_combine_rows
modified2 = bframe2.set_column("foo", ["A", "B", "C", "D", "E"])
combined = relaxed_combine_rows(bframe1, modified2)
print(combined)
## BiocFrame with 10 rows and 3 columns
##        odd   even    foo
##     <list> <list> <list>
## [0]      1      0   None
## [1]      3      2   None
## [2]      5      4   None
## [3]      7      6   None
## [4]      9      8   None
## [5]     11      0      A
## [6]     33     22      B
## [7]     55     44      C
## [8]     77     66      D
## [9]     99     88      E
```

Similarly, if the rows are different, we can use **BiocFrame**'s `merge` function:

```python
from biocframe import merge
modified1 = bframe1.set_row_names(["A", "B", "C", "D", "E"])
modified3 = bframe3.set_row_names(["C", "D", "E", "F", "G"])
combined = merge([modified1, modified3], by=None, join="outer")
## BiocFrame with 7 rows and 4 columns
##      odd   even    foo    bar
##   <list> <list> <list> <list>
## A      1      0   None   None
## B      3      2   None   None
## C      5      4      A   True
## D      7      6      B  False
## E      9      8      C   True
## F   None   None      D  False
## G   None   None      E   True
```

## Playing nice with pandas

`BiocFrame` is intended for accurate representation of Bioconductor objects for interoperability with R.
Most users will probably prefer to work with **pandas** `DataFrame` objects for their actual analyses.
This conversion is easily achieved:

```python
from biocframe import BiocFrame
bframe = BiocFrame(
    {
        "foo": ["A", "B", "C", "D", "E"],
        "bar": [True, False, True, False, True]
    }
)

pd = bframe.to_pandas()
print(pd)
##   foo    bar
## 0   A   True
## 1   B  False
## 2   C   True
## 3   D  False
## 4   E   True
```

Conversion back to a ``BiocFrame`` is similarly easy:

```python
out = BiocFrame.from_pandas(pd)
print(out)
## BiocFrame with 5 rows and 2 columns
##      foo    bar
##   <list> <list>
## 0      A   True
## 1      B  False
## 2      C   True
## 3      D  False
## 4      E   True
```

## Further reading

Check out [the reference documentation](https://biocpy.github.io/BiocFrame/) for more details.

Also see check out Bioconductor's [**S4Vectors**](https://bioconductor.org/packages/S4Vectors) package,
which implements the `DFrame` class on which `BiocFrame` was based.
