Metadata-Version: 2.4
Name: jimg_ncd
Version: 0.2.1
Summary: 
License-File: license.txt
Author: Jakub Kubis - JBS
Author-email: jbiosystem@gmail.com
Requires-Python: >=3.12,<3.13
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: IPython
Requires-Dist: csbdeep
Requires-Dist: gdown (>=5.2.0,<6.0.0)
Requires-Dist: harmonypy (>=0.0.10,<0.0.11)
Requires-Dist: joblib
Requires-Dist: kaleido (==0.2.1)
Requires-Dist: matplotlib (<3.10.0)
Requires-Dist: matplotlib-venn (>=1.1.2,<2.0.0)
Requires-Dist: mpld3
Requires-Dist: nbformat (>=5.10.4,<6.0.0)
Requires-Dist: numpy (==1.26.0)
Requires-Dist: opencv-python
Requires-Dist: openpyxl (>=3.1.5,<4.0.0)
Requires-Dist: pandas
Requires-Dist: pingouin
Requires-Dist: plotly
Requires-Dist: scikit-image
Requires-Dist: scikit-learn (<1.6.0)
Requires-Dist: scipy
Requires-Dist: setuptools (>=80.9.0,<81.0.0)
Requires-Dist: stardist
Requires-Dist: tensorflow (>=2.20.0,<3.0.0)
Requires-Dist: tqdm
Requires-Dist: umap-learn (>=0.5.7,<0.6.0)
Description-Content-Type: text/markdown

### JIMG_ncd – Python library for automated nucleus detection and analysis

![Python version](https://img.shields.io/badge/python-%E2%89%A53.12%20%7C%20%3C3.13-blue?logo=python&logoColor=white.png)
![License](https://img.shields.io/badge/license-GPLv3-blue)
![Docs](https://img.shields.io/badge/docs-available-blueviolet)
</br>

<p align="right">
    <img src="https://github.com/jkubis96/Logos/blob/main/logos/jbs_current.png?raw=true" alt="drawing" width="250" />
    <img src="https://github.com/jkubis96/Logos/blob/main/logos/jbi_current.png?raw=true" alt="drawing" width="250" />
</p>

</br>

### Author: Jakub Kubiś 

<div align="left">
 Institute of Bioorganic Chemistry<br />
 Polish Academy of Sciences<br />
</div>


## Description


<div align="justify">


**JIMG_ncd** is a Python library designed for DL based automated nucleus detection and analysis in high-resolution images from confocal microscopy and flow cytometry, eg. [Amnis-ImageStream](https://cytekbio.com/pages/imagestream). It facilitates in-depth examination of nuclei and their chromatin organization, enabling precise analysis of nuclear morphology and chromatin structure.

Additionally, JIMG_ncd provides advanced tools for analyzing cell populations from cytometric data. Users can select distinctive cellular features, cluster cells based on these characteristics, and apply statistical analyses to uncover unique features within each cluster.
 
 

</div>

</br>



<br />

## 📚 Table of Contents
- 1.[Installation](#installation)
- 2.[Documentation](#doc)
- 3.[Example pipelines](#epip)
  - 3.1. [Nuclei analysis - confocal microscopy](#nacm)
    - 3.1.1 [Testing analysis parameters](#nacm1)
    - 3.1.2 [Performing nuclei analysis with adjusted parameters](#nacm2)
    - 3.1.3 [Selecting nuclei based on nucleus parameters](#nacm3)
    - 3.1.4 [Extracting nuclei chromanitization features](#nacm4)
    - 3.1.5 [Adjusting nuclei chromanitization parameters](#nacm5)
    - 3.1.6 [Analyzing nuclei series](#nacm6)
    - 3.1.7 [Obtaining nuclei series analysis results](#nacm7)
    - 3.1.8 [Analyzing nuclei chromatinization series](#nacm8)
    - 3.1.9 [Obtaining nuclei chromatinization series analysis results](#nacm9)
  - 3.2. [Nuclei analysis - flow cytometry](#nafc)
    - 3.2.1 [Testing analysis parameters](#nafc1)
    - 3.2.2 [Performing nuclei analysis with adjusted parameters](#nafc2)
    - 3.2.3 [Selecting nuclei based on nucleus parameters](#nafc3)
    - 3.2.4 [Extracting nuclei chromanitization features](#nafc4)
    - 3.2.5 [Adjusting nuclei chromanitization parameters](#nafc5)
    - 3.2.6 [Analyzing nuclei chromatinization series](#nafc6)
    - 3.2.7 [Obtaining nuclei chromatinization series analysis results](#nafc7)
    - 3.2.8 [Concatenating nuclei chromatinization analysis with ImageStream (IS) data](#nafc8)
    - 3.2.9 [Combining projects from nuclei chromatinization analysi](#nafc9)

  - 3.3. [Clustering and DFA (Differential Feature Analysis) – nuclei data](#cdnd)
    - 3.3.1 [Selecting feature analysis (DFA) for separate experiments data](#cdnd1)
    - 3.3.2 [Loading images IDs](#cdnd2)
    - 3.3.3 [Performing data scaling and dimensionality reduction](#cdnd3)
    - 3.3.4 [Performing UMAP & clustering](#cdnd4)
    - 3.3.5 [Obtaining complete data and metadata (clusters)](#cdnd5)
    - 3.3.6 [Preforming DFA analysis on clusters](#cdnd6)
    - 3.3.7 [Preforming proportion analysis](#cdnd7)
  - 3.4. [ImagesManagement – nuclei data](#im)
    - 3.4.1 [Loading & saving raw data from project](#im1)
    - 3.4.2 [Filtering project data to selected features](#im2)
    - 3.4.3 [Loading saved project with images](#im3)
    - 3.4.4 [Adjusting a series of images](#im4)
    - 3.4.5 [Stitching images](#im5)
    - 3.4.6 [Merging images](#im6)
    - 3.4.7 [Saving a series of images](#im7)
<br />

<br />

# 1. Installation <a id="installation"></a>

#### In command line write:

```
pip install jimg-ncd
```





# 2. Documenation <a id="doc"></a>

Documentation for classes and functions is available here 👉 [Documentation 📄](https://jkubis96.github.io/JIMG_ncd/jimg_ncd.html)


# 3. Example pipelines <a id="epip"></a>

If you want to run the examples, you must download the test data. To do this, use:

```
from jimg_ncd.nuclei import test_data

test_data()
```


<br />

#### 3.1 Nuclei analysis - confocal microscopy <a id="nacm"></a>

##### 3.1.1 Testing analysis parameters <a id="nacm1"></a>


```
from jimg_ncd.nuclei import NucleiFinder


# initiate class
nf = NucleiFinder()


image = nf.load_image('test_data/microscope_nuclei/r01c02f90p20-ch1sk1fk1fl1.tiff')


nf.input_image(image)


# Check the basic parameters
nf.current_parameters_nuclei


# Test nms & prob parmeters for nuclei segmentation
nf.nuclei_finder_test()

nf.browser_test()
```

<br/>

[Browse Report](https://jkubis96.github.io/webpages/ncdJIMG/report_microscope_image.html)

<br/>

```
# If required, change parameters
nf.set_nms(nms = 0.9)

nf.set_prob(prob = 0.5)


# Analysis

# 1. First step on nuclei analysis
nf.find_nuclei()


# Parameters for micrsocope image adjustment 
nf.current_parameters_img_adj
```

<br/>

***Image with 'Default' parameters:***
<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/find_nuclei_before.bmp" alt="drawing" width="600" />
</p>

<br/>

```
# If image required changes, change parameters and run again (nf.find_nuclei())
nf.set_adj_image_brightness(brightness = 1000)

nf.set_adj_image_gamma(gamma = 1.2)

nf.set_adj_image_contrast(contrast = 2)


# Check if parameters has changed
nf.current_parameters_nuclei


# Second execution with new parameters for image adjustment
nf.find_nuclei()
```
<br/>

***Image with adjusted parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/find_nuclei_after.bmp" alt="drawing" width="600" />
</p>

<br/>

##### 3.1.2 Performing nuclei analysis with adjusted parameters <a id="nacm2"></a>

```
# Return results
nf.find_nuclei()
nuclei_results, analysed_img = nf.get_results_nuclei()
```
<br/>

***Dictionary with nuclei results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/dict_nuclei.bmp" alt="drawing" width="600" />
</p>

<br/>


##### 3.1.3 Selecting nuclei based on nucleus parameters <a id="nacm3"></a>


```
# 2. Second step of analysis (selection)
nf.select_nuclei()
```
<br/>

***Image with 'Default' selection parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/select_nuclei_before.bmp" alt="drawing" width="600" />
</p>

<br/>

```
# Parameters for selecting nuclei; adjust if analysis results do not meet 
# requirements, and re-run the analysis as needed.
nf.current_parameters_nuclei

nf.set_nuclei_circularity(circ = 0.5)

nf.set_nuclei_size(size = (100,800))

nf.set_nuclei_min_mean_intensity(intensity = 2000)


# Check if parameters has changed
nf.current_parameters_nuclei


# Second execution with adjusted parameters of second step of analysis (selection)
nf.select_nuclei()
```
<br/>

***Image with adjusted selection parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/select_nuclei_after.bmp" alt="drawing" width="600" />
</p>

<br/>

```
# Return results
nuclei_selected_results, analysed_selected_img = nf.get_results_nuclei_selected()
```
<br/>

***Dictionary with nuclei results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/dict_nuclei.bmp" alt="drawing" width="600" />
</p>

<br/>

##### 3.1.4 Extracting nuclei chromanitization features <a id="nacm4"></a>

```
# 3. third step (chromatinization alaysis)
nf.nuclei_chromatinization()
```
<br/>

***Image with 'Default' chromatinization parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/nuclei_chromatinization_before.bmp" alt="drawing" width="600" />
</p>

<br/>

##### 3.1.5 Adjusting nuclei chromanitization parameters <a id="nacm5"></a>

```
# Parameters for nuclei chromatinization; adjust if analysis results do not meet 
# requirements, and re-run the analysis as needed.


# Chromatinization parameters

nf.current_parameters_chromatinization

nf.set_chromatinization_size(size = (2,400))

nf.set_chromatinization_ratio(ratio = .05)

nf.set_chromatinization_cut_point(cut_point = .95)

nf.current_parameters_chromatinization

# Chromatinization image parameters

nf.current_parameters_img_adj_chro

nf.set_adj_chrom_gamma(gamma = 0.25)

nf.set_adj_chrom_contrast(contrast = 3)

nf.set_adj_chrom_brightness(brightness = 950)

nf.current_parameters_img_adj_chro


# Second execution of the third step (chromatinization analysis)
nf.nuclei_chromatinization()
```
<br/>

***Image with adjusted chromatinization parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/nuclei_chromatinization_after.bmp" alt="drawing" width="600" />
</p>

<br/>

```
chromatinization_results, analysed_chromatinization_img = nf.get_results_nuclei_chromatinization()
```
<br/>

***Dictionary with nuclei chromatinization results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/dict_chrom.bmp" alt="drawing" width="600" />
</p>

<br/>

##### 3.1.6 Analyzing nuclei series <a id="nacm6"></a>

```
# If your parameters are correct for your data, you can run series analysis on more images

# Nuclei 

series_results_nuclei = nf.series_analysis_nuclei(path_to_images = 'test_data/microscope_nuclei', 
                                                  file_extension = 'tiff', 
                                                  selected_id = [], 
                                                  fille_name_part = 'ch1',
                                                  selection_opt = True, 
                                                  include_img = False, 
                                                  test_series = 0)
```

<br/>

***Dictionary with series nuclei results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/series.bmp" alt="drawing" width="600" />
</p>

<br/>

<br/>



##### 3.1.7 Obtaining nuclei series analysis results <a id="nacm7"></a>


```
# get & save results

from jimg_ncd.nuclei import NucleiDataManagement

# initiate class with NucleiFinder data
ndm = NucleiDataManagement(series_results_nuclei, 'example')

# get data as data frame
df = ndm.get_data()
print(df)

# save results as data frame
ndm.save_results_df(path='')

# save results as project *.nuc
ndm.save_nuc_project(path='')

# saved project by ndm.save_nuc_project(path='') can be then loaded for further analysis
ndm2 = NucleiDataManagement.load_nuc_dict('example.nuc')
ndm2.get_data()
```




<br/>

##### 3.1.8 Analyzing nuclei chromatinization series <a id="nacm8"></a>


```
# Chromatinization 

series_results_chromatinization = nf.series_analysis_chromatinization(path_to_images = 'test_data/microscope_nuclei', 
                                                  file_extension = 'tiff', 
                                                  selected_id = [], 
                                                  fille_name_part = 'ch1',
                                                  selection_opt = True, 
                                                  include_img = True, 
                                                  test_series = 0)
```
<br/>

***Dictionary with series nuclei chromatinization results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/series_chrom.bmp" alt="drawing" width="600" />
</p>

<br/>




##### 3.1.9 Obtaining nuclei chromatinization series analysis results <a id="nacm9"></a>



```
# get & save results

from jimg_ncd.nuclei import NucleiDataManagement

# initiate class with NucleiFinder data
ndm = NucleiDataManagement(series_results_chromatinization, 'example_chromatinization')

# get data as data frame
df = ndm.get_data()
print(df)

# save results as data frame
ndm.save_results_df(path='')

# save results as project *.nuc
ndm.save_nuc_project(path='')

# saved project by ndm.save_nuc_project(path='') can be then loaded for further analysis
ndm2 = NucleiDataManagement.load_nuc_dict('example_chromatinization.nuc')
ndm2.get_data()
```

<br/>

***Data table with series nuclei chromatinization results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/Microscope_nuclei/get_data_nuclei.bmp" alt="drawing" width="700" />
</p>




<br />

<br />

#### 3.2 Nuclei analysis - flow cytometry <a id="nafc"></a>

##### 3.2.1 Testing analysis parameters <a id="nafc1"></a>


```
from jimg_ncd.nuclei import NucleiFinder

# initiate class
nf = NucleiFinder()

image = nf.load_image("test_data/flow_cytometry/ctrl/3087_Ch7.ome.tif")

nf.input_image(image)

# Check the basic parameters
nf.current_parameters_nuclei


# Test nms & prob parmeters for nuclei segmentation
nf.nuclei_finder_test()

nf.browser_test()
```

<br/>

[Browse Report](https://jkubis96.github.io/webpages/ncdJIMG/report_flow_cytometry_image.html)

<br/>

```
# If required, change parameters
nf.set_nms(nms = 0.6)

nf.set_prob(prob = 0.3)


# Analysis

# 1. First step on nuclei analysis
nf.find_nuclei()
```

<br/>

***Image with 'Default' parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/find_nuclei_before.bmp" alt="drawing" width="500" />
</p>

<br/>

```
# Parameters for micrsocope image adjustment 
nf.current_parameters_img_adj


# If image required changes, change parameters and run again (nf.find_nuclei())
nf.set_adj_image_brightness(brightness = 1000)

nf.set_adj_image_gamma(gamma = 1.2)

nf.set_adj_image_contrast(contrast = 2)


# Check if parameters has changed
nf.current_parameters_nuclei


# Second execution with new parameters for image adjustment
nf.find_nuclei()
```
<br/>

***Image with adjusted parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/find_nuclei_after.bmp" alt="drawing" width="500" />
</p>

<br/>

##### 3.2.2 Performing nuclei analysis with adjusted parameters <a id="nafc2"></a>

```
# Return results
nf.find_nuclei()
nuclei_results, analysed_img = nf.get_results_nuclei()
```
<br/>

***Dictionary with nuclei results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/dict_nuclei.bmp" alt="drawing" width="400" />
</p>

<br/>

##### 3.2.3 Selecting nuclei based on nucleus parameters <a id="nafc3"></a>

```
# 2. Second step of analysis (selection)
nf.select_nuclei()
```
<br/>

***Image with 'Default' selection parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/select_nuclei_before.bmp" alt="drawing" width="500" />
</p>

<br/>

```
# Parameters for selecting nuclei; adjust if analysis results do not meet 
# requirements, and re-run the analysis as needed.
nf.current_parameters_nuclei

nf.set_nuclei_circularity(circ = 0.5)


nf.set_nuclei_size(size = (100,800))

nf.set_nuclei_min_mean_intensity(intensity = 2000)


# Check if parameters has changed
nf.current_parameters_nuclei


# Second execution with adjusted parameters of second step of analysis (selection)
nf.select_nuclei()
```
<br/>

***Image with adjusted selection parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/select_nuclei_after.bmp" alt="drawing" width="500" />
</p>

<br/>


```
# Return results
nuclei_selected_results, analysed_selected_img = nf.get_results_nuclei_selected()
```
<br/>

***Dictionary with nuclei results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/dict_nuclei.bmp" alt="drawing" width="400" />
</p>

<br/>

##### 3.2.4 Extracting nuclei chromanitization features <a id="nafc4"></a>

```
# 3. third step (chromatinization alaysis)
nf.nuclei_chromatinization()
```
<br/>

***Image with 'Default' chromatinization parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/nuclei_chromatinization_before.bmp" alt="drawing" width="500" />
</p>

<br/>

##### 3.2.5 Adjusting nuclei chromanitization parameters <a id="nafc5"></a>

<br/>

```
# Parameters for nuclei chromatinization; adjust if analysis results do not meet 
# requirements, and re-run the analysis as needed.


# Chromatinization parameters
nf.current_parameters_chromatinization

nf.set_chromatinization_size(size = (2,1000))

nf.set_chromatinization_ratio(ratio = 0.005)

nf.set_chromatinization_cut_point(cut_point = 1.05)

nf.current_parameters_chromatinization


# Chromatinization image parameters
nf.current_parameters_img_adj_chro

nf.set_adj_chrom_gamma(gamma = 0.25)

nf.set_adj_chrom_contrast(contrast = 4)

nf.set_adj_chrom_brightness(brightness = 950)

nf.current_parameters_img_adj_chro


# Second execution of the third step (chromatinization analysis)
nf.nuclei_chromatinization()
```
<br/>

***Image with adjusted chromatinization parameters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/nuclei_chromatinization_after.bmp" alt="drawing" width="500" />
</p>

<br/>


```
# Return results
chromatinization_results, analysed_chromatinization_img = nf.get_results_nuclei_chromatinization()
```
<br/>

***Dictionary with nuclei chromatinization results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/dict_chrom.bmp" alt="drawing" width="400" />
</p>

<br/>

##### 3.2.6 Analyzing nuclei chromatinization series <a id="nafc6"></a>

```
# If your parameters are correct for your data, you can run series analysis on more images

# Chromatinization CTRL CELLS
series_results_chromatinization = nf.series_analysis_chromatinization(path_to_images = 'test_data/flow_cytometry/ctrl', 
                                                  file_extension = 'tif', 
                                                  selected_id = [], 
                                                  selection_opt = True, 
                                                  include_img = False, 
                                                  test_series = 0)
```
<br/>

***Dictionary with series nuclei chromatinization results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/series_chrom.bmp" alt="drawing" width="600" />
</p>

<br/>

```
# Chromatinization DISEASE CELLS
series_results_chromatinization2 = nf.series_analysis_chromatinization(path_to_images = 'test_data/flow_cytometry/dis', 
                                                  file_extension = 'tif', 
                                                  selected_id = [], 
                                                  selection_opt = True, 
                                                  include_img = False, 
                                                  test_series = 0)
```
<br/>

***Dictionary with series nuclei chromatinization results:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/series_chrom.bmp" alt="drawing" width="600" />
</p>

<br/>

##### 3.2.7 Obtaining nuclei chromatinization series analysis results <a id="nafc7"></a>


```
from jimg_ncd.nuclei import NucleiDataManagement

# initiate class with NucleiFinder data
ndm = NucleiDataManagement(series_results_chromatinization, 'healthy_chromatinization')
ndm2 = NucleiDataManagement(series_results_chromatinization, 'disease_chromatinization')

# get data as data frame
df = ndm.get_data()
print(df)

ndm.save_results_df(path='')


df2 = ndm2.get_data()
print(df)

ndm2.save_results_df(path='')

```
<br/>

***Healthy results in data frame:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/ctr_df.bmp" alt="drawing" width="600" />
</p>


***Disease results in data frame:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/dis_df.bmp" alt="drawing" width="600" />
</p>

<br/>

##### 3.2.8 Concatenating nuclei chromatinization analysis with ImageStream (IS) data <a id="nafc8"></a>

```
# load IS data
import pandas as pd

healthy_data = pd.read_csv('test_data/flow_cytometry/ctrl.txt', sep='\t', header=1)

disease_data = pd.read_csv('test_data/flow_cytometry/dis.txt', sep='\t', header=1)

# select data with cell size info
selectes_columns = [ 
    'Area_M01',
    'Major Axis_M01',
    'Minor Axis_M01',
    'Aspect Ratio_M01',
    'Diameter_M01',
    'Area_M09',
    'Major Axis_M09',
    'Minor Axis_M09',
    'Aspect Ratio_M09',
    'Diameter_M09',
 ]

ndm.add_IS_data(healthy_data, IS_features=selectes_columns)

ndm2.add_IS_data(disease_data, IS_features=selectes_columns)

df_is = ndm.get_data_with_IS()
print(df_is)

ndm.save_results_df_with_IS(path = '')

df_is2 = ndm2.get_data_with_IS()
print(df_is2)

ndm2.save_results_df_with_IS(path = '')

# save projects
# NOTE:
# Image Stream (IS) data is NOT saved within the .nuc project file.
# After loading a project using `load_nuc_dict()`, IS data must be added again
# using the `add_IS_data()` method.

ndm.save_nuc_project(path='')

ndm2.save_nuc_project(path='')
```
<br/>

***Healthy results in data frame with IS data:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/ctr_df_is.bmp" alt="drawing" width="600" />
</p>


***Disease results in data frame with IS data:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/dis_df_is.bmp" alt="drawing" width="600" />
</p>

<br/>


##### 3.2.9 Combining projects from nuclei chromatinization analysi <a id="nafc9"></a>


```
from jimg_ncd.nuclei import NucleiDataManagement

# merge projects

ndm = NucleiDataManagement.load_nuc_dict('healthy_chromatinization.nuc')
ndm2 = NucleiDataManagement.load_nuc_dict('disease_chromatinization.nuc')


# load IS data

# NOTE:
# Image Stream (IS) data is NOT saved within the .nuc project file.
# After loading a project using `load_nuc_dict()`, IS data must be added again
# using the `add_IS_data()` method.

import pandas as pd

healthy_data = pd.read_csv('test_data/flow_cytometry/ctrl.txt', sep='\t', header=1)

disease_data = pd.read_csv('test_data/flow_cytometry/dis.txt', sep='\t', header=1)

# select data with cell size info
selectes_columns = [ 
    'Area_M01',
    'Major Axis_M01',
    'Minor Axis_M01',
    'Aspect Ratio_M01',
    'Diameter_M01',
    'Area_M09',
    'Major Axis_M09',
    'Minor Axis_M09',
    'Aspect Ratio_M09',
    'Diameter_M09',
 ]

ndm.add_IS_data(healthy_data, IS_features=selectes_columns)


ndm2.add_IS_data(disease_data, IS_features=selectes_columns)


ndm.add_experiment([ndm2])

df = ndm.get_mutual_experiments_data(inc_is=True)
print(df)

ndm.save_mutual_experiments(path='', inc_is=True)
```

<br/>

***Combined healthy and disease results in data frame with IS data:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/FlowCytometry_nuclei/concatenated_data.bmp" alt="drawing" width="600" />
</p>



<br />
<br />


#### 3.3 Clustering and DFA (Differential Feature Analysis) – Nuclei data <a id="cdnd"></a>

##### 3.3.1 Selecting feature analysis (DFA) for separate experiments data <a id="cdnd1"></a>


```
from jimg_ncd.nuclei import GroupAnalysis
import pandas as pd

data = pd.read_csv('healthy_chromatinization_disease_chromatinization_IS.csv', sep=',', header=0)

# initiate class
ga = GroupAnalysis.load_data(data, ids_col = 'id_name', set_col = 'set')

# check available groups for selection of differential features
ga.groups

# run DFA analysis on example sets

ga.DFA(meta_group_by = 'sets',
   sets = {'disease':['disease_chromatinization'],
           'ctrl':['healthy_chromatinization']}, 
   n_proc = 5)
 
group_diff_features = ga.get_DFA()

ga.heatmap_DFA()

fig = ga.get_DFA_plot()

fig.savefig('DFA_groups.png', dpi=300, bbox_inches='tight')
```
<br/>

***Data table presenting statistical analysis of differential features:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/primary_stat.bmp" alt="drawing" width="600" />
</p>

***DFA heatmap for groups:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/DFA_groups.png" alt="drawing" width="600" />
</p>


<br/>

##### 3.3.2 Filtering project data to selected features <a id="cdnd2"></a>

```
# select differential features

diff_features = list(group_diff_features['feature'][group_diff_features['p_val'] <= 0.05])

ga.select_data(features_list = diff_features)
```

<br/>


##### 3.3.3 Performing data scaling and dimensionality reduction <a id="cdnd3"></a>


```
# scale data
ga.data_scale()


# run PCA dimensionality reduction
ga.PCA()


# get PCA data, if required
pca_data = ga.get_PCA()


# run PC variance analysis
ga.var_plot()


# get var_data, if required
var_data = ga.get_var_data()


# get knee_plot, if required
knee_plot = ga.get_knee_plot(show = True)

knee_plot.savefig('knee_plot.png', dpi=300)
```
<br/>

***Knee plot - cumulative explanation of variance:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/knee_plot.png" alt="drawing" width="500" />
</p>

<br/>

##### 3.3.4 Performing UMAP & clustering <a id="cdnd4"></a>

```
# run UMAP dimensionality reduction
ga.UMAP(PC_num = 8,
     factorize_with_metadata = False, 
     harmonize_sets = True,
     n_neighbors = 10,
     min_dist = 0.01,
     n_components = 2)


# get UMAP_data, if required
UMAP_data = ga.get_UMAP_data()


# get UMAP_plots, if required
UMAP_plots = ga.get_UMAP_plots(show = True)

UMAP_plots.keys()

UMAP_plots['PrimaryUMAP'].savefig('UMAP.png', dpi=300)
```
<br/>

***UMAP plot - sets:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/UMAP.png" alt="drawing" width="500" />
</p>

<br/>

```
# run db_scan on UMAP components
ga.db_scan(eps = 0.5,
        min_samples = 10)


# run UMAP_on_clusters
ga.UMAP_on_clusters(min_entities = 5)


# get UMAP_plots, if required
UMAP_plots = ga.get_UMAP_plots(show = True)

UMAP_plots.keys()

UMAP_plots['ClusterUMAP'].savefig('UMAP_clusters.png', dpi=300)
UMAP_plots['ClusterXSetsUMAP'].savefig('ClusterXSetsUMAP.png', dpi=300)
```
<br/>

***UMAP plot - db_scan clusters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/UMAP_clusters.png" alt="drawing" width="500" />
</p>

<br/>


***UMAP plot -  set / cluster combination:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/ClusterXSetsUMAP.png" alt="drawing" width="500" />
</p>

<br/>

##### 3.3.5 Obtaining complete data and metadata (clusters) <a id="cdnd5"></a>

```
# get full_data [data + metadata], if required
full_data = ga.full_info()
```

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/full_data.bmp" alt="drawing" width="600" />
</p>


##### 3.3.6 Preforming DFA analysis on clusters <a id="cdnd6"></a>

```
# check available groups for selection of differential features
ga.groups


# run DFA analysis on finl clusters
ga.DFA(meta_group_by = 'full_name',
    sets = {}, 
    n_proc = 5)

dfa_clusters = ga.get_DFA()


ga.heatmap_DFA(top_n = 3)

fig = ga.get_DFA_plot()


fig.savefig('DFA_clusters.png', dpi=300, bbox_inches='tight')
```

<br/>

***Data table presenting statistical analysis of differential features for final clusters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/primary_stat.bmp" alt="drawing" width="600" />
</p>

***DFA heatmap for clusters:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/DFA_clusters.png" alt="drawing" width="600" />
</p>

<br />


##### 3.3.7 Preforming proportion analysis <a id="cdnd7"></a>


```
ga.print_avaiable_features()
          
ga.proportion_analysis(grouping_col = 'sets', 
                       val_col = 'nuclei_per_img', 
                       grouping_dict = None, 
                       omit = None)
     
pl = ga.get_proportion_plot(show = True)
      
pl.savefig('proportion.png', dpi=300, bbox_inches='tight')
  
ga.get_proportion_stats()
```

<br />


***Nuclei per cell proportion plot:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/documentation/fig/DFA_analysis_nuclei/proportion.png" alt="drawing" width="600" />
</p>




<br/>

<br/>


#### 3.4 ImagesManagement – nuclei data <a id="im"></a>

##### 3.4.1 Loading & saving raw data from project <a id="im1"></a>


```                                      
from jimg_ncd.nuclei import ImagesManagement

# load dictionary data obtained from series_analysis_chromatinization() 
# or series_analysis_nuclei() with include_img = True

indm = ImagesManagement.load_experimental_images(series_results_chromatinization, 'disease')

indm.save_raw(path_to_save = '')
```
<br/>

##### 3.4.2 Loading images IDs <a id="im2"></a>


```                                      
from jimg_ncd.nuclei import ImagesManagement

# instead of loading dictionary data obtained from 
# series_analysis_chromatinization() or series_analysis_nuclei() 
# only image IDs to adjust can be loaded

images_ids = [1523, 1528, 1589, 1335]

indm = ImagesManagement.load_images_ids(images_ids = images_ids, experiment_name = 'disease')
```
<br/>

##### 3.4.3 Loading saved project with images <a id="im3"></a>


```                                      
from jimg_ncd.nuclei import ImagesManagement

# loading whole project with data obtained from  
# series_analysis_chromatinization() or series_analysis_nuclei() 
# saved using self.save_raw(path_to_save = '')

indm = ImagesManagement.load_from_dict('disease.inuc.npz', 'disease')
```
<br/>

##### 3.4.4 Adjusting a series of images <a id="im4"></a>


```                                      
indm.adjust_images(acronyme='bright', 
                      color='gray', 
                      path_to_images='test_data/flow_cytometry/dis', 
                      fille_name_part='Ch9.ome', 
                      eq = False, 
                      clahe = False, 
                      gamma = 0.7, 
                      contrast = 1.2)

indm.adjust_images(acronyme='DAPI', 
                      color='blue', 
                      path_to_images='test_data/flow_cytometry/dis', 
                      fille_name_part='Ch7.ome', 
                      eq = True, 
                      clahe = True, 
                      gamma = 2, 
                      contrast = 1.2)
```
<br/>



##### 3.4.5 Stitching images <a id="im5"></a>


```         
# check available image acronyms defined in adjust_images
indm.get_included_acronyms()

# Stitch result images loaded from the project
# if they were loaded using:
#  - 3.4.3 Loading a saved project with images
#  - 3.4.1 Loading & saving raw data from the project

indm.image_stitching(acronyms=['bright'], to_results_image=True)

# you can also stitch different adjusted images with different acronyms,
# even if they were loaded using 3.4.2 Loading image IDs

indm.image_stitching(acronyms=['bright', 'DAPI'], to_results_image=False) 
```

<br/>

***Stitched image:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/images_management/fig/FlowCytometry_nuclei/18171_bright_res.png" alt="drawing" width="500" />
</p>

<br/>

##### 3.4.6 Merging images <a id="im6"></a>

```         
# check available image acronyms defined in adjust_images
indm.get_included_acronyms()

indm.image_merging(acronyms = ['bright', 'DAPI'], 
                   ratio_list = [0.9,0.6])  
```
<br/>


***Merged image:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/images_management/fig/FlowCytometry_nuclei/18414_bright_DAPI.png" alt="drawing" width="250" />
</p>

<br/>


##### 3.4.7 Saving a series of images <a id="im7"></a>

```         
# check available image acronyms defined in adjust_images, image_stitching and image_merging

indm.get_included_acronyms()


indm.save_prepared_images(acronyme = 'stitched_bright',  path_to_save = '')

indm.save_prepared_images(acronyme = 'stitched_bright_DAPI', path_to_save = '')

indm.save_prepared_images(acronyme = 'merged_bright_DAPI', path_to_save = '')
```

<br/>


***Saved images:***

<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/images_management/fig/FlowCytometry_nuclei/saved.bmp" alt="drawing" width="650" />
</p>


<p align="center">
<img  src="https://raw.githubusercontent.com/jkubis96/JIMG_ncd/refs/heads/images_management/fig/FlowCytometry_nuclei/saved2.bmp" alt="drawing" width="650" />
</p>


<br />
<br />



### Have fun JBS
