.. doctest-skip-all

.. _whatsnew-2.0:

**************************
What's New in Astropy 2.0?
**************************

Overview
========

Astropy 2.0 is a major release that adds significant new functionality since
the 1.3.x series of releases.

On this page, you can read about some of the big changes in this release:

* :ref:`whatsnew-2.0-models-units`
* :ref:`whatsnew-2.0-ccddata-class`
* :ref:`whatsnew-2.0-coordinates-velocities`
* :ref:`whatsnew-2.0-stats`
* :ref:`whatsnew-2.0-table-uni-sandwich`
* :ref:`whatsnew-2.0-bintablehdu-from_table`
* :ref:`whatsnew-2.0-fits-printdiff`
* :ref:`whatsnew-2.0-molar-mass`
* :ref:`whatsnew-2.0-ascii-ecsv-mixins`
* :ref:`whatsnew-2.0-convolution`
* :ref:`whatsnew-2.0-cosmo-def`
* :ref:`whatsnew-2.0-renamed-removed`

In addition to these major changes, Astropy 2.0 includes a large number of
smaller improvements and bug fixes, which are described in the :ref:`changelog`.
By the numbers:


* 701 issues have been closed since v1.3
* 319 pull requests have been merged since v1.3
* 232 distinct people have contributed code

.. _whatsnew-2.0-models-units:

New unit support for most models
================================

Most Astropy models now can handle inputs with units, and produce the
appropriate outputs with units as well. Some models cannot support this due
to their definitions (e.g., Legendre, Hermite, etc), while some will have
this capability added in a future release. Example usage::

    >>> from astropy import units as u
    >>> from astropy.modeling.models import Gaussian1D
    >>> g = Gaussian1D(amplitude=1*u.J, mean=1*u.m, stddev=0.1*u.m)
    >>> g([3, 4, 5.5] * u.cm)
    <Quantity [  3.70353198e-21,  9.72098502e-21,  4.05703276e-20] J>

For more information, see :ref:`modeling-units`.

.. _whatsnew-2.0-ccddata-class:

New image class CCDData added
=============================

A new class, :class:`~astropy.nddata.CCDData`, has been added to the
`astropy.nddata` package. It can read from/write to FITS files, provides methods
for arithmetic operations with propagation of uncertainty, and support for
binary masks. For examples of how to use this class, see the
:ref:`astropy_nddata_getting_started` section of the :ref:`astropy.nddata
<astropy_nddata>` documentation.

.. _whatsnew-2.0-coordinates-velocities:

Experimental velocity support in ``astropy.coordinates``
========================================================

Astropy coordinate frame objects now contains experimental support for storing
and transforming velocities. This includes, among other things, support for
transforming proper motion components between coordinate frames and transforming
full-space velocities to/from a local standard of rest (``LSR``) frame and a
:class:`~astropy.coordinates.Galactocentric` frame. For example, to transform a
set of proper motions from the :class:`~astropy.coordinates.Galactic` frame to
the :class:`~astropy.coordinates.ICRS` frame::

    >>> from astropy.coordinates import Galactic, ICRS
    >>> gal = Galactic(l=8.67*u.degree, b=53.09*u.degree,
    ...                pm_l_cosb=-117*u.mas/u.yr, pm_b=13*u.mas/u.yr)
    >>> gal.transform_to(ICRS)
    <ICRS Coordinate: (ra, dec) in deg
        ( 226.45743375,  8.3354549)
     (pm_ra_cosdec, pm_dec) in mas / yr
        (-77.61973364, -88.50523685)>

Or, for example, to transform a 3D velocity from the
:class:`~astropy.coordinates.ICRS` frame to a Galactocentric frame with custom
values for the sun-galactic center distance and solar velocity vector::

    >>> icrs = ICRS(ra=11.23*u.degree, dec=58.13*u.degree,
    ...             distance=213.4*u.pc,
    ...             pm_ra_cosdec=9*u.mas/u.yr, pm_dec=3*u.mas/u.yr,
    ...             radial_velocity=-61*u.km/u.s)
    >>> v_sun = coord.CartesianDifferential([10, 244, 7.])*u.km/u.s
    >>> gc = icrs.transform_to(coord.Galactocentric(galcen_distance=8*u.kpc,
                                                    galcen_v_sun=v_sun))
    >>> gc.x, gc.y, gc.z
    (<Quantity -8112.928728515727 pc>,
     <Quantity 180.22175948399217 pc>,
     <Quantity 9.781203623025618 pc>)
    >>> gc.v_x, gc.v_y, gc.v_z
    (<Quantity 34.40211035247248 km / s>,
     <Quantity 187.80653073084486 km / s>,
     <Quantity 14.74171285614737 km / s>)

The velocity support works by adding support for "differential" objects which
contain differences of representations. For more details, see
:ref:`astropy-coordinates-velocities`. This functionality will likely be added
to the :class:`~astropy.coordinates.SkyCoord` class in future.

In addition, the :class:`~astropy.coordinates.SkyCoord` class now has a
`~astropy.coordinates.SkyCoord.radial_velocity_correction` method which can be
used to compute heliocentric and barycentric corrections for radial velocity
measurements.  While in the future this may use the mechanisms described above,
currently it uses a simpler algorithm for numerical stability. A simple example
of using this functionality might be::

    >>> from astropy.coordinates import SkyCoord, EarthLocation
    >>> from astropy.time import Time
    >>> obstime = Time('2017-2-14')
    >>> target = SkyCoord.from_name('M31')
    >>> keck = EarthLocation.of_site('Keck')
    >>> target.radial_velocity_correction(obstime=obstime, location=keck).to('km/s')
    <Quantity -22.363056056262263 km / s>




.. _whatsnew-2.0-stats:

New functionality in astropy.stats
==================================

New sigma-clipping class
------------------------

A new :class:`~astropy.stats.SigmaClip` class has been added as an
object-oriented interface for sigma clipping::

    >>> from astropy.stats import SigmaClip
    >>> data = [1, 5, 6, 8, 100, 5, 3, 2]
    >>> sigclip = SigmaClip(sigma=2, iters=5)
    >>> print(sigclip)  # doctest: +SKIP
    <SigmaClip>
        sigma: 3
        sigma_lower: None
        sigma_upper: None
        iters: 10
        cenfunc: <function median at 0x108dbde18>
        stdfunc: <function std at 0x103ab52f0>
    >>> sigclip(data)
    masked_array(data = [1 5 6 8 -- 5 3 2],
                 mask = [False False False False  True False False False],
           fill_value = 999999)

Note that once the ``sigclip`` instance is defined above, it can be
applied to other data, using the same, already-defined, sigma-clipping
parameters.

New robust statistical functions
--------------------------------

New :func:`~astropy.stats.biweight_midcovariance` and
:func:`~astropy.stats.biweight_midcorrelation` functions were added to
`astropy.stats`. The :func:`~astropy.stats.biweight_midcovariance` function
computes the robust covariance between two or more variables, and
:func:`~astropy.stats.biweight_midcorrelation` computes a robust
measure of similarity between two variables.

For example::

    >>> import numpy as np
    >>> from astropy.stats import biweight_midcovariance
    >>> from astropy.stats import biweight_midcorrelation
    >>> # Generate two random variables x and y
    >>> rng = np.random.RandomState(1)
    >>> x = rng.normal(0, 1, 200)
    >>> y = rng.normal(0, 3, 200)
    >>> # Introduce an obvious outlier
    >>> x[0] = 30.0
    >>> # Calculate the biweight midcovariances between x and y
    >>> bicov = biweight_midcovariance([x, y])
    >>> print(bicov)  # doctest: +FLOAT_CMP
    [[ 0.82483155 -0.18961219]
     [-0.18961219 9.80265764]]
    >>> # Print standard deviation estimates
    >>> print(np.sqrt(bicov.diagonal()))  # doctest: +FLOAT_CMP
    [ 0.90820237  3.13091961]
    >>> # Compute the biweight midcorrelation between x and y
    >>> print(biweight_midcorrelation(x, y))  # doctest: +FLOAT_CMP
    -0.066682472486875297

New statistical estimators for Ripley's K Function
--------------------------------------------------

New statistical estimators for Ripley's K Function,
:class:`~astropy.stats.RipleysKEstimator`, in `astropy.stats`. For example:

.. plot::
   :include-source:

    import numpy as np
    from matplotlib import pyplot as plt
    from astropy.stats import RipleysKEstimator
    z = np.random.uniform(low=5, high=10, size=(100, 2))
    Kest = RipleysKEstimator(area=25, x_max=10, y_max=10, x_min=5, y_min=5)
    r = np.linspace(0, 2.5, 100)
    plt.plot(r, Kest.poisson(r), label='poisson')
    plt.plot(r, Kest(data=z, radii=r, mode='none'), label='none')
    plt.plot(r, Kest(data=z, radii=r, mode='translation'), label='translation')
    plt.plot(r, Kest(data=z, radii=r, mode='ohser'), label='ohser')
    plt.plot(r, Kest(data=z, radii=r, mode='var-width'), label='var-width')
    plt.plot(r, Kest(data=z, radii=r, mode='ripley'), label='ripley')
    plt.legend(loc='upper left')

.. _whatsnew-2.0-table-uni-sandwich:

Easier use of efficient bytestring Table columns in Python 3
============================================================

Working with bytestring Table columns (numpy ``'S'`` dtype) in Python
3 has been made more convenient because it is now possible to compare
and set array elements with the natural Python string (`str`) type.
Previously one had to use the Python `bytes` type and bytestring literals
like ``b'hello'``.  This change allows working with ASCII data columns
in Python 3 using only 1-byte per character instead of the default
4-bytes per character for the numpy ``'U'`` unicode dtype.  For large
datasets this improves memory performance.

Please see :ref:`bytestring-columns-python-3` for details.  Note that no
change has been made to behavior for Python 2.

.. note::

     This introduces an API change that affects comparison of
     bytestring column elements in Python 3.

     If comparison with `str` instead of `bytes` is a problem
     (and `bytes` is really more logical), please open an issue
     on GitHub.

.. _whatsnew-2.0-bintablehdu-from_table:

New way to instantiate a BinTableHDU directly from a Table
==========================================================

A new way to instantiate a FITS :class:`~astropy.io.fits.BinTableHDU` directly
from a :class:`~astropy.table.Table` object. For example::

    >>> from astropy.io import fits
    >>> from astropy.table import Table
    >>> tab = Table([[1, 2, 3], ['a', 'b', 'c'], [2.3, 4.5, 6.7]],
    ...             names=['a', 'b', 'c'], dtype=['i', 'U1', 'f'])
    >>> hdu = fits.BinTableHDU(tab)

.. _whatsnew-2.0-fits-printdiff:

New ``printdiff`` convenience function for FITS
===============================================

A new :func:`~astropy.io.fits.printdiff` convenience function was added for
comparison between FITS files or HDUs. For example::

    >>> from astropy.io import fits
    >>> hdu1 = fits.ImageHDU([1, 2, 3])
    >>> hdu2 = fits.ImageHDU([1, 2.1, 3])
    >>> fits.printdiff(hdu1, hdu2)

    Headers contain differences:
      Keyword BITPIX   has different values:
         a> 64
         b> -64
          ? +

    Data contains differences:
      Data differs at [2]:
           (int64) a> 2
         (float64) b> 2.1000000000000001
      1 different pixels found (33.33% different).

.. _whatsnew-2.0-molar-mass:

New ``molar_mass_amu`` unit equivalency
=======================================

A new equivalency named :class:`~astropy.units.molar_mass_amu` has been added to
convert between ``g/mol`` unit to atomic mass unit (amu). For example::

    >>> from astropy import constants as const
    >>> from astropy import units as u
    >>> x = 1 * (u.g / u.mol)
    >>> y = 1 * u.u
    >>> x.to(u.u, equivalencies=u.molar_mass_amu())
    <Quantity 1.0 u>
    >>> y.to(u.g/u.mol, equivalencies=u.molar_mass_amu())
    <Quantity 1.0 g / mol>

.. _whatsnew-2.0-ascii-ecsv-mixins:

Store astropy core object types in ASCII ECSV table file
========================================================

It is now possible to store the following :ref:`mixin column
<mixin_columns>` types in an ASCII :ref:`ECSV
<ecsv_format>` table file:
:class:`~astropy.time.Time`,
:class:`~astropy.time.TimeDelta`,
:class:`~astropy.units.Quantity`,
:class:`~astropy.coordinates.Latitude`,
:class:`~astropy.coordinates.Longitude`,
:class:`~astropy.coordinates.Angle`,
:class:`~astropy.coordinates.Distance`,
:class:`~astropy.coordinates.EarthLocation`,
:class:`~astropy.coordinates.SkyCoord`. The table file can then be read back
into astropy with no loss of object data or attributes.

.. _whatsnew-2.0-convolution:

Improvements to astropy.convolution
===================================

Convolution has undergone a significant overhaul to make fft and direct
convolution consistent.  They keyword arguments have changed and the behavior
of `~astropy.convolution.convolve` is no longer the same as in versions prior to
2.0 (although `~astropy.convolution.convolve_fft`'s behavior remains unchanged).
The details are given on the :ref:`astropy convolution <astropy_convolve>`.

.. _whatsnew-2.0-cosmo-def:

No relativistic species by default in cosmological models
=========================================================

For all of the built in cosmological model types (e.g.,
:class:`~astropy.cosmology.FlatLambdaCDM`) the default CMB temperature at z=0 is
now 0K, which corresponds to no contributions from photons or neutrinos (massive
or otherwise).  This does not affect built in literature models (such as the
WMAP or Planck models).  The justification is to avoid including mass-energy
components that the user has not explicitly requested.  This is a non-backwards
compatible change, although the effects are small for most use cases.

.. _whatsnew-2.0-renamed-removed:

Renamed/removed functionality
=============================

Several sub-packages have been moved or removed, and these are described in the
following sections.

astropy.tests.helper.pytest
---------------------------

The bundled version of ``pytest`` has now been removed, but the
``astropy.tests.helper.pytest`` import will continue to work properly.
Affiliated packages should nevertheless transition to importing ``pytest``
directly rather than from ``astropy.tests.helper``. This also means that
``pytest`` is now a formal requirement for testing for both Astropy and for
affiliated packages.

astropy.vo.conesearch
---------------------

The cone search module has been moved to `Astroquery
<http://astroquery.readthedocs.io>`_ (0.3.5 and later) and will be removed from
Astropy in a future version. The API here will be preserved as the "classic" API
in Astroquery, however some configuration behavior might change; See the
Astroquery `documentation
<http://astroquery.readthedocs.io/en/latest/vo_conesearch/vo_conesearch.html>`_
for new usage details.

astropy.vo.samp
---------------

The SAMP (Simple Application Messaging Protocol) module, formerly available in
``astropy.vo.samp``, has now been moved to `astropy.samp`, so you should
update any imports to this module.

Full change log
===============

To see a detailed list of all changes in version v2.0, including changes in
API, please see the :ref:`changelog`.
