Contact: Luigi Silva (luigi.silva@linea.org.br)
Last verified: Apr. 08, 2025
Acknowledgments¶
This notebook used computational resources from the Associação Laboratório Interinstitucional de e-Astronomia (LIneA) with financial support from the INCT of e-Universo (Process no. 465376/2014-2).
This notebook used data obtained with the Dark Energy Spectroscopic Instrument (DESI). DESI construction and operations is managed by the Lawrence Berkeley National Laboratory. This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of High-Energy Physics, under Contract No. DE–AC02–05CH11231, and by the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility under the same contract. Additional support for DESI was provided by the U.S. National Science Foundation (NSF), Division of Astronomical Sciences under Contract No. AST-0950945 to the NSF’s National Optical-Infrared Astronomy Research Laboratory; the Science and Technology Facilities Council of the United Kingdom; the Gordon and Betty Moore Foundation; the Heising-Simons Foundation; the French Alternative Energies and Atomic Energy Commission (CEA); the National Council of Humanities, Science and Technology of Mexico (CONAHCYT); the Ministry of Science and Innovation of Spain (MICINN), and by the DESI Member Institutions: www.desi.lbl.gov/collaborating-institutions. The DESI collaboration is honored to be permitted to conduct scientific research on I’oligam Du’ag (Kitt Peak), a mountain with particular significance to the Tohono O’odham Nation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the U.S. National Science Foundation, the U.S. Department of Energy, or any of the listed funding agencies.
See complete DESI Data License and Acknowledgements information here.
Introduction¶
This notebook contains a brief characterization of the spectroscopic redshifts (spec-z) measurements in the DESI DR1 catalog, which has been publicly distributed and described in detail in scientific literature by DESI project.
The data presented here were obtained from the DESI data release web page.
The schema of the catalog can be found here.
Reference paper¶
Imports and configurations of libraries¶
Importing the required libraries¶
Importing the libraries.
Setting up a local cluster with dask¶
Defining a local Dask Client.
Printing info about the Client.
Client
Client-cfad6e26-14b8-11f0-91bf-c2d3b49f35e5
| Connection method: Cluster object | Cluster type: distributed.LocalCluster |
| Dashboard: http://127.0.0.1:8787/status |
Cluster Info
LocalCluster
f7524b6d
| Dashboard: http://127.0.0.1:8787/status | Workers: 2 |
| Total threads: 4 | Total memory: 13.04 GiB |
| Status: running | Using processes: True |
Scheduler Info
Scheduler
Scheduler-7ea5a17e-1031-474a-a674-6a1d712ead0a
| Comm: tcp://127.0.0.1:45620 | Workers: 2 |
| Dashboard: http://127.0.0.1:8787/status | Total threads: 4 |
| Started: Just now | Total memory: 13.04 GiB |
Workers
Worker: 0
| Comm: tcp://127.0.0.1:33151 | Total threads: 2 |
| Dashboard: http://127.0.0.1:39430/status | Memory: 6.52 GiB |
| Nanny: tcp://127.0.0.1:44213 | |
| Local directory: /tmp/dask-scratch-space/worker-okveid10 | |
Worker: 1
| Comm: tcp://127.0.0.1:40428 | Total threads: 2 |
| Dashboard: http://127.0.0.1:32772/status | Memory: 6.52 GiB |
| Nanny: tcp://127.0.0.1:44762 | |
| Local directory: /tmp/dask-scratch-space/worker-j3aywwft | |
Printing info about the Dask Dashboard.
Reading the entire catalog with Dask¶
Defining the path to the input files¶
Defining the path to the input files.
Reading the catalog¶
Reading the catalog with Dask.
Data characterization¶
Full catalog¶
Total number of lines (full catalog): 28425963
Total number of columns (full catalog): 136
Columns names (full catalog): ['TARGETID', 'SURVEY', 'PROGRAM', 'HEALPIX', 'SPGRPVAL', 'Z', 'ZERR', 'ZWARN', 'CHI2', 'COEFF', 'NPIXELS', 'SPECTYPE', 'SUBTYPE', 'NCOEFF', 'DELTACHI2', 'COADD_FIBERSTATUS', 'TARGET_RA', 'TARGET_DEC', 'PMRA', 'PMDEC', 'REF_EPOCH', 'FA_TARGET', 'FA_TYPE', 'OBJTYPE', 'SUBPRIORITY', 'OBSCONDITIONS', 'RELEASE', 'BRICKNAME', 'BRICKID', 'BRICK_OBJID', 'MORPHTYPE', 'EBV', 'FLUX_G', 'FLUX_R', 'FLUX_Z', 'FLUX_W1', 'FLUX_W2', 'FLUX_IVAR_G', 'FLUX_IVAR_R', 'FLUX_IVAR_Z', 'FLUX_IVAR_W1', 'FLUX_IVAR_W2', 'FIBERFLUX_G', 'FIBERFLUX_R', 'FIBERFLUX_Z', 'FIBERTOTFLUX_G', 'FIBERTOTFLUX_R', 'FIBERTOTFLUX_Z', 'MASKBITS', 'SERSIC', 'SHAPE_R', 'SHAPE_E1', 'SHAPE_E2', 'REF_ID', 'REF_CAT', 'GAIA_PHOT_G_MEAN_MAG', 'GAIA_PHOT_BP_MEAN_MAG', 'GAIA_PHOT_RP_MEAN_MAG', 'PARALLAX', 'PHOTSYS', 'PRIORITY_INIT', 'NUMOBS_INIT', 'CMX_TARGET', 'DESI_TARGET', 'BGS_TARGET', 'MWS_TARGET', 'SCND_TARGET', 'SV1_DESI_TARGET', 'SV1_BGS_TARGET', 'SV1_MWS_TARGET', 'SV1_SCND_TARGET', 'SV2_DESI_TARGET', 'SV2_BGS_TARGET', 'SV2_MWS_TARGET', 'SV2_SCND_TARGET', 'SV3_DESI_TARGET', 'SV3_BGS_TARGET', 'SV3_MWS_TARGET', 'SV3_SCND_TARGET', 'PLATE_RA', 'PLATE_DEC', 'COADD_NUMEXP', 'COADD_EXPTIME', 'COADD_NUMNIGHT', 'COADD_NUMTILE', 'MEAN_DELTA_X', 'RMS_DELTA_X', 'MEAN_DELTA_Y', 'RMS_DELTA_Y', 'MEAN_PSF_TO_FIBER_SPECFLUX', 'MEAN_FIBER_RA', 'STD_FIBER_RA', 'MEAN_FIBER_DEC', 'STD_FIBER_DEC', 'MIN_MJD', 'MAX_MJD', 'MEAN_MJD', 'TSNR2_GPBDARK_B', 'TSNR2_ELG_B', 'TSNR2_GPBBRIGHT_B', 'TSNR2_LYA_B', 'TSNR2_BGS_B', 'TSNR2_GPBBACKUP_B', 'TSNR2_QSO_B', 'TSNR2_LRG_B', 'TSNR2_GPBDARK_R', 'TSNR2_ELG_R', 'TSNR2_GPBBRIGHT_R', 'TSNR2_LYA_R', 'TSNR2_BGS_R', 'TSNR2_GPBBACKUP_R', 'TSNR2_QSO_R', 'TSNR2_LRG_R', 'TSNR2_GPBDARK_Z', 'TSNR2_ELG_Z', 'TSNR2_GPBBRIGHT_Z', 'TSNR2_LYA_Z', 'TSNR2_BGS_Z', 'TSNR2_GPBBACKUP_Z', 'TSNR2_QSO_Z', 'TSNR2_LRG_Z', 'TSNR2_GPBDARK', 'TSNR2_ELG', 'TSNR2_GPBBRIGHT', 'TSNR2_LYA', 'TSNR2_BGS', 'TSNR2_GPBBACKUP', 'TSNR2_QSO', 'TSNR2_LRG', 'MAIN_NSPEC', 'MAIN_PRIMARY', 'SV_NSPEC', 'SV_PRIMARY', 'ZCAT_NSPEC', 'ZCAT_PRIMARY', 'DESINAME']
Filtered catalog¶
Applying the SPECTYPE==GALAXY and ZCAT_PRIMARY==True filters¶
Applying the SPECTYPE==GALAXY and ZCAT_PRIMARY==True filters in the catalog.
Selecting the columns of interest¶
Selecting the columns of interest in the catalog.
['TARGETID', 'HEALPIX', 'Z', 'ZERR', 'ZWARN', 'TARGET_RA', 'TARGET_DEC', 'EBV', 'FLUX_G', 'FLUX_R', 'FLUX_Z', 'FLUX_W1', 'FLUX_W2', 'FLUX_IVAR_G', 'FLUX_IVAR_R', 'FLUX_IVAR_Z', 'FLUX_IVAR_W1', 'FLUX_IVAR_W2']
Number of lines and columns¶
Total number of lines (filtered catalog): 21044698
Total number of columns (filtered catalog): 18
Columns names (filtered catalog): ['TARGETID', 'HEALPIX', 'Z', 'ZERR', 'ZWARN', 'TARGET_RA', 'TARGET_DEC', 'EBV', 'FLUX_G', 'FLUX_R', 'FLUX_Z', 'FLUX_W1', 'FLUX_W2', 'FLUX_IVAR_G', 'FLUX_IVAR_R', 'FLUX_IVAR_Z', 'FLUX_IVAR_W1', 'FLUX_IVAR_W2']
First lines of the catalog¶
| TARGETID | HEALPIX | Z | ZERR | ZWARN | TARGET_RA | TARGET_DEC | EBV | FLUX_G | FLUX_R | FLUX_Z | FLUX_W1 | FLUX_W2 | FLUX_IVAR_G | FLUX_IVAR_R | FLUX_IVAR_Z | FLUX_IVAR_W1 | FLUX_IVAR_W2 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 39628473198710603 | 2152 | 0.804206 | 0.000010 | 0 | 23.764862 | 29.832379 | 0.054335 | 1.354010 | 2.976295 | 10.395741 | 27.525110 | 48.779099 | 531.460571 | 123.641281 | 19.293917 | 2.664506 | 0.641028 |
| 2 | 39628473198711342 | 2152 | 0.932752 | 2.232e-46 | 518 | 23.802207 | 29.832150 | 0.053668 | 2.653994 | 11.634748 | 23.412300 | 50.114071 | 35.767471 | 595.788574 | 118.594986 | 25.252254 | 2.198009 | 0.608285 |
| 3 | 39628473198711925 | 2152 | 0.390723 | 0.000020 | 0 | 23.832176 | 29.858176 | 0.053723 | 3.122899 | 13.893675 | 35.096653 | 78.427452 | 54.463345 | 436.552338 | 94.306648 | 18.671127 | 1.769941 | 0.524213 |
| 4 | 39628473202901028 | 2152 | 0.665803 | 0.000076 | 0 | 23.866668 | 29.816397 | 0.053278 | 0.331562 | 1.404510 | 4.291292 | 24.670589 | 14.502599 | 1265.926758 | 370.455292 | 56.139481 | 2.710926 | 0.662417 |
| 5 | 39628473202901626 | 2152 | 0.328418 | 0.000050 | 0 | 23.898508 | 29.849485 | 0.053137 | 8.766316 | 30.099854 | 60.953991 | 88.483856 | 50.180351 | 173.101532 | 25.908314 | 5.784760 | 1.403655 | 0.399923 |
Basic statistics¶
| TARGETID | HEALPIX | Z | ZERR | ZWARN | TARGET_RA | TARGET_DEC | EBV | FLUX_G | FLUX_R | FLUX_Z | FLUX_W1 | FLUX_W2 | FLUX_IVAR_G | FLUX_IVAR_R | FLUX_IVAR_Z | FLUX_IVAR_W1 | FLUX_IVAR_W2 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 | 21044698.000000 |
| mean | 5.282e+10 | 18979.565458 | 0.686863 | 0.000068 | 256.741457 | 184.477115 | 16.480597 | 0.041694 | 5.785048 | 13.613173 | 25.804079 | 31.345707 | 21.482312 | 543.947215 | 207.887829 | 43.991272 | 2.003729 | 0.522845 |
| std | 1.623e+17 | 10127.589749 | 0.464044 | 0.000058 | 674.851214 | 86.142084 | 21.210698 | 0.038225 | 49.013611 | 101.211184 | 167.558305 | 1976.882889 | 453.540389 | 653.421766 | 319.336411 | 54.513966 | 1.566050 | 0.466398 |
| min | -8.265e+08 | 0.000e+00 | -0.004312 | 2.232e-46 | 0.000e+00 | 0.000068 | -31.601866 | 0.000e+00 | -1770.626953 | -248.088272 | -444.662659 | -34785.710938 | -260170.796875 | -99.000000 | -99.000000 | -99.000000 | 0.000e+00 | 0.000e+00 |
| 25% | 3.963e+16 | 10512.000000 | 0.407712 | 0.000040 | 0.000e+00 | 156.217883 | 3.152669 | 0.025922 | 0.339624 | 0.453288 | 0.828235 | 0.856831 | 0.104785 | 175.050026 | 50.422132 | 11.490018 | 1.296442 | 0.370481 |
| 50% | 3.963e+16 | 20071.000000 | 0.816868 | 0.000074 | 0.000e+00 | 212.811514 | 17.130446 | 0.036964 | 0.822830 | 1.952401 | 7.305542 | 20.402670 | 13.988712 | 436.097748 | 153.232582 | 40.473501 | 2.593952 | 0.625707 |
| 75% | 3.963e+16 | 27247.000000 | 1.184093 | 0.000109 | 5.000000 | 314.948742 | 40.435969 | 0.062465 | 11.264684 | 27.309690 | 51.519180 | 63.298538 | 45.356457 | 1441.455536 | 530.845047 | 113.992617 | 3.476955 | 0.849641 |
| max | 2.882e+18 | 49151.000000 | 1.697883 | 0.007043 | 2695.000000 | 359.999980 | 84.609140 | 3.964358 | 70843.726562 | 149822.156250 | 193771.828125 | 9016309.000000 | 1548954.875000 | 12979.132812 | 5741.590332 | 808.293274 | 54.963043 | 16.473513 |
Spatial distribution plot¶
Getting RA and DEC columns in the Dask Dataframe.
Changing RA to the $[-180^{\circ}, 180^{\circ})$ interval.
Plot using the Plate Carrée projection¶
Defining the points element.
Defining the title, the axis labels and the plot sizes.
Making the plot with geoviews and datashader.
Plot using the Mollweide projection¶
Defining the points element.
Defining the title, the axis labels and the plot sizes.
Defining the RA and DEC ticks for the Mollweide projection.
Making the plot with datashader.