HGXFile¶

class Stoner.HDF5.HGXFile(*args, **kargs)[source]¶

Bases: Data

A subclass of DataFile for reading GenX HDF Files.

These files typically have an extension .hgx. This class has been based on a limited sample of hgx files and so may not be sufficiently general to handle all cases.

Attributes Summary

`T`	Get the current data transposed.
`ax`	Return the current axis number.
`axes`	Return the current axes object.
`basename`	Return the basename of the current filename.
`clone`	Get a deep copy of the current DataFile.
`cmap`	Get the current cmap.
`column_headers`	Pass through to the setas attribute.
`data`	Property Accessors for the main numerical data.
`dict_records`	Return the data as a dictionary of single columns with column headers for the keys.
`dims`	Alias for self.data.axes.
`dtype`	Return the np dtype attribute of the data.
`fig`	Get the current figure.
`fignum`	Return the current figure number.
`filename`	Return DataFile filename, or make one up.
`filepath`	Return DataFile filename, or make one up, returning as a pathlib.Path.
`header`	Make a pretty header string that looks like the tabular representation.
`labels`	Return the labels for the plot columns.
`mask`	Return the mask of the data array.
`metadata`	Read the metadata dictionary.
`mime_type`
`multiple`
`no_fmt`
`pattern`
`positional_fmt`
`priority`
`records`	Return the data as a np structured data array.
`setas`	Get the list of column assignments.
`shape`	Pass through the numpy shape attribute of the data.
`showfig`	Return either the current figure or self or None.
`subplots`	Return the subplot instances.
`template`	Return the current plot template.

Methods Summary

`SG_Filter`([col, xcol, points, poly, order, ...])	Implement a Savitsky-Golay filtering of data for smoothing and differentiating data.
`__call__`(args, *kargs)	Clone the Data, but allowing additional arguments to modify the new clone.
`add`(col_a, col_b[, replace, header, index])	Add one column, number or array (col_b) to another column (col_a).
`add_column`(column_data[, header, index, ...])	Append a column of data or inserts a column to a datafile instance.
`annotate_fit`(model[, x, y, z, text_only])	Annotate a plot with some information about a fit.
`append`(value)	S.append(value) -- append value to the end of the sequence
`apply`(func[, col, replace, header])	Apply the given function to each row in the data set and adds to the data set.
`asarray`()	Provide a consistent way to get at the underlying array data.
`bin`([xcol, ycol, bins, mode, clone])	Bin x-y data into new values of x with an error bar.
`clear`()
`clip`(clipper[, column])	Clips the data based on the column and the clipper value.
`closest`(value[, xcol])	Return the row in a data file which has an x-column value closest to the given value.
`colormap_xyz`([xcol, ycol, zcol])	Make a xyz plot that forces the use of plt.colormap.
`column`(col)	Extract one or more columns of data from the datafile.
`columns`([not_masked, reset])	Iterate over the columns of data int he datafile.
`contour_xyz`([xcol, ycol, zcol, shape, xlim, ...])	Make a xyz plot that forces the use of plt.contour.
`count`([value, axis, col])	Count the number of un-masked elements in the `DataFile`.
`curve_fit`(func[, xcol, ycol, sigma])	General curve fitting function passed through from scipy.
`decompose`([xcol, ycol, sym, asym, replace])	Given (x,y) data, decomposes the y part into symmetric and antisymmetric contributions in x.
`del_column`([col, duplicates])	Delete a column from the current `DataFile` object.
`del_nan`([col, clone])	Remove rows that have nan in them.
`del_rows`([col, val, invert])	Search in the numerica data for the lines that match and deletes the corresponding rows.
`differential_evolution`(model[, xcol, ycol, ...])	Fit model to the data using a differential evolution algorithm.
`diffsum`(col_a, col_b[, replace, header, index])	Calculate \(\frac{a-b}{a+b}\) for the two columns a and b.
`dir`([pattern])	Return a list of keys in the metadata, filtering with a regular expression if necessary.
`divide`(col_a, col_b[, replace, header, index])	Divide one column (col_a) by another column, number or array (col_b).
`extend`(values)	S.extend(iterable) -- extend sequence by appending elements from the iterable
`extrapolate`(new_x[, xcol, ycol, yerr, ...])	Extrapolate data based on local fit to x,y data.
`figure`([figure, projection])	Set the figure used by `Stoner.plot.PlotMixin`.
`filter`([func, cols, reset])	Set the mask on rows of data by evaluating a function for each row.
`find_col`(col[, force_list])	Indexes the column headers in order to locate a column of data.shape.
`find_duplicates`([xcol, delta])	Find rows with duplicated values of the search column(s).
`find_peaks`(**kargs)	Interface to `scipy.signal.find_peaks()` for locating peaks in data.
`format`(key, **kargs)	Return the contents of key pretty formatted using `format_error()`.
`get`(k[,d])
`get_filename`(mode)	Force the user to choose a new filename using a system dialog box.
`griddata`([xcol, ycol, zcol, ucol, shape, ...])	Convert xyz data onto a regular grid.
`image_plot`([xcol, ycol, zcol, shape, xlim, ylim])	Grid up the three columns of data and plot.
`index`(value, [start, [stop]])	Raises ValueError if the value is not present.
`insert`(index, obj)	Implement the insert method.
`insert_rows`(row, new_data)	Insert new_data into the data array at position row.
`inset`([parent, loc, width, height])	Add a new set of axes as an inset to the current plot.
`integrate`([xcol, ycol, result, header, ...])	Integrate a column of data, optionally returning the cumulative integral.
`interpolate`(newX[, kind, xcol, replace])	Interpolate a dataset to get a new set of values for a given set of x data.
`items`()	Make sure we implement an items that doesn't just iterate over self.
`keys`()	Return the keys of the metadata dictionary.
`legend`(args, *kargs)	Pass Through to stop attribute access over-riding a handy method.
`lmfit`(model[, xcol, ycol, p0, sigma])	Wrap the lmfit module fitting.
`load`(args, *kargs)	Create a new `Data` from a file on disc guessing a better subclass if necessary.
`main_data`(data_grp)	Work through the main data group and build something that looks like a numpy 2D array.
`make_bins`(xcol, bins[, mode])	Generate bin boundaries and centres along an axis.
`max`([column, bounds])	Find maximum value and index in col_a column of data.
`mean`([column, sigma, bounds])	Find mean value of col_a data column.
`min`([column, bounds])	Find minimum value and index in col_a column of data.
`multiply`(col_a, col_b[, replace, header, index])	Multiply one column (col_a) by another column, number or array (col_b).
`normalise`([target, base, replace, header, ...])	Normalise data columns by dividing through by a base column value.
`odr`(model[, xcol, ycol])	Wrap the scipy.odr orthogonal distance regression fitting.
`outlier_detection`([column, window, shape, ...])	Detect outliers in a column of data.
`peaks`(**kargs)	Locates peaks and/or troughs in a column of data by using SG-differentiation.
`plot`(args, *kargs)	Try to make an appropriate plot based on the defined column assignments.
`plot_matrix`([xvals, yvals, rectang, cmap, ...])	Plot a surface plot by assuming that the current dataset represents a regular matrix of points.
`plot_voxels`([xcol, ycol, zcol, ucol, cmap])	Make a volumetric plot of data arranged as x,y,z,u.
`plot_xy`([xcol, ycol, fmt, xerr, yerr])	Makesa simple X-Y plot of the specified data.
`plot_xyuv`([xcol, ycol, ucol, vcol, wcol])	Make an overlaid image and quiver plot.
`plot_xyuvw`([xcol, ycol, ucol, vcol, wcol])	Make an overlaid image and quiver plot.
`plot_xyz`([xcol, ycol, zcol, shape, xlim, ...])	Plot a surface plot based on rows of X,Y,Z data using matplotlib.pcolor().
`plot_xyzuvw`([xcol, ycol, zcol, ucol, vcol, wcol])	Plot a vector field plot based on rows of X,Y,Z (U,V,W) data using ,ayavi.
`polyfit`([xcol, ycol, polynomial_order, ...])	Pass through to numpy.polyfit.
`pop`(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised.
`popitem`()	as a 2-tuple; but raise KeyError if D is empty.
`quiver_plot`([xcol, ycol, ucol, vcol])	Make a 2D Quiver plot from the data.
`remove`(value)	S.remove(value) -- remove first occurrence of value.
`remove_duplicates`([xcol, delta, strategy, ...])	Find and remove rows with duplicated values of the search column(s).
`rename`(old_col, new_col)	Rename columns without changing the underlying data.
`reorder_columns`(cols[, headers_too, setas_too])	Construct a new data array from the original data by assembling the columns in the order given.
`reverse`()	S.reverse() -- reverse IN PLACE
`rolling_window`([window, wrap, exclude_centre])	Iterate with a rolling window section of the data.
`rows`([not_masked, reset])	Iterate over rows of data.
`save`([filename])	Save a string representation of the current DataFile object into the file 'filename'.
`scale`(other[, xcol, ycol])	Scale the x and y data in this DataFile to match the x and y data in another DataFile.
`scan_group`(grp, pth)	Recursively list HDF5 Groups.
`search`([xcol, value, columns, accuracy])	Search the numerica data part of the file for lines that match and returns the corresponding rows.
`search_index`([xcol, value, accuracy, invert])	Return an array of booleans for indexing matching rows for use with search method.
`section`(**kargs)	Assuming data has x,y or x,y,z coordinates, return data from a section of the parameter space.
`select`(args, *kargs)	Produce a copy of the DataFile with only data rows that match a criteria.
`setdefault`(k[,d])
`smooth`([window, xcol, ycol, size])	Smooth data by convoluting with a window.
`sort`(order, *kargs)	Sort the data by column name.
`span`([column, bounds])	Return a tuple of the maximum and minimum values within the given column and bounds.
`spline`([xcol, ycol, sigma])	Construct a spline through x and y data and replace, add new data or return spline function.
`split`(*args[, final])	Recursively splits the current DataFile into a `Stoner.Folders.DataFolder`.
`std`([column, sigma, bounds])	Find standard deviation value of col_a data column.
`stitch`(other[, xcol, ycol, overlap, ...])	Apply a scaling to this data set to make it stich to another dataset.
`subplot`(args, *kargs)	Pass throuygh for `matplotlib.pyplot.subplot()`.
`subplot2grid`(args, *kargs)	Provide a pass through to `matplotlib.pyplot.subplot2grid()`.
`subtract`(col_a, col_b[, replace, header, index])	Subtract one column, number or array (col_b) from another column (col_a).
`swap_column`(swp, *kargs)	Swap pairs of columns in the data.
`threshold`(threshold, **kargs)	Find partial indices where the data in column passes the threshold, rising or falling.
`to_pandas`()	Create a pandas DataFrame from a `Stoner.Data` object.
`unique`(col[, return_index, return_inverse])	Return the unique values from the specified column - pass through for numpy.unique.
`update`([E, ]**F)	If E present and has a .keys() method, does: for k in E.keys(): D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
`values`()	Return the values of the metadata dictionary.
`x2`()	Generate a new set of axes with a second x-scale.
`y2`()	Generate a new set of axes with a second y-scale.

Attributes Documentation

T¶: Get the current data transposed.

ax¶: Return the current axis number.

axes¶: Return the current axes object.

basename¶: Return the basename of the current filename.

clone¶: Get a deep copy of the current DataFile.

cmap¶: Get the current cmap.

column_headers¶: Pass through to the setas attribute.

data¶: Property Accessors for the main numerical data.

dict_records¶: Return the data as a dictionary of single columns with column headers for the keys.

dims¶: Alias for self.data.axes.

dtype¶: Return the np dtype attribute of the data.

fig¶: Get the current figure.

fignum¶: Return the current figure number.

filename¶: Return DataFile filename, or make one up.

filepath¶: Return DataFile filename, or make one up, returning as a pathlib.Path.

header¶: Make a pretty header string that looks like the tabular representation.

labels¶: Return the labels for the plot columns.

mask¶: Return the mask of the data array.

metadata¶: Read the metadata dictionary.

mime_type = ['application/x-hdf', 'application/x-hdf5']¶

multiple = 'common'¶

no_fmt = [<function errorfill>]¶

pattern = ['*.hgx']¶

positional_fmt = [<function plot>, <function semilogx>, <function semilogy>, <function loglog>]¶

priority = 16¶

records¶

Return the data as a np structured data array.

If columns names are duplicated then they are made unique.

setas¶: Get the list of column assignments.

shape¶: Pass through the numpy shape attribute of the data.

showfig¶

Return either the current figure or self or None.

The return value depends on whether the attribute is True or False or None.

subplots¶: Return the subplot instances.

template¶: Return the current plot template.

Methods Documentation

SG_Filter(col=None, xcol=None, points=15, poly=1, order=0, pad=True, result=None, replace=False, header=None)¶

Implement a Savitsky-Golay filtering of data for smoothing and differentiating data.

Parameters:

col (column index) – Column of Data to be filtered. if None, first y-column in setas is filtered.
points (int) – Number of data points to use in the filtering window. Should be an odd number > poly+1 (default 15)

Keyword Arguments:

xcol (coilumn index) – If order>1 then can be used to specify an x-column to differentiate with respect to.
poly (int) – Order of polynomial to fit to the data. Must be equal or greater than order (default 1)
order (int) – Order of differentiation to carry out. Default=0 meaning smooth the data only.
pad (bool or float) – Pad the start and end of the array with the mean value (True, default) or specired value (float) or leave as is.
result (None,True, or column_index) – If not None, column index to insert new data, or True to append as last column
header (string or None) – Header for new column if result is not None. If header is Nne, a suitable column header is generated.

Returns:

(numpy array or datafile) – If result is None, a numpy array representing the smoothed or differentiated data is returned. Otherwise, a copy of the modified AnalysisMixin object is returned.

Notes

If col is not specified or is None then the DataFile.setas column assignments are used to set an x and y column. If col is a tuple, then it is assumed to specify and x-column and y-column for differentiating data. This is now a pass through to scipy.signal.savgol_filter()

Padding can help stop wildly wrong artefacts in the data at the start and enf of the data, particularly when the differential order is >1.

See also

User guide section Smoothing and Filtering Data

__call__(*args, **kargs)¶

Clone the Data, but allowing additional arguments to modify the new clone.

Parameters:

*args (tuple) – Positional arguments to pass through to the new clone.
**kargs (dict) – Keyword arguments to pass through to the new clone.

Raises:

TypeError – If a keyword argument doesn’t match an attribute.

Returns:

new_d (Data) – Modified clone of the current object.

add(col_a, col_b, replace=False, header=None, index=None)¶

Add one column, number or array (col_b) to another column (col_a).

Parameters:

col_a (index) – First column to work with
col_b (index, float or 1D array) – Second column to work with.

Keyword Arguments:

header (string or None) – new column header (defaults to a-b
replace (bool) – Replace the col_a column with the new data
index (column index or None) – Column to insert new data at.

Returns:

(Stoner.Data) – The newly modified Data object.

If col_a and col_b are tuples of length two, then the firstelement is assumed to be the value and the second element an uncertainty in the value. The uncertainties will then be propagated and an additional column with the uncertainites will be added to the data.

add_column(column_data, header=None, index=None, func_args=None, replace=False, setas=None)¶

Append a column of data or inserts a column to a datafile instance.

Parameters:

column_data (numpy.array or list or callable) – Data to append or insert or a callable function that will generate new data

Keyword Arguments:

header (string) – The text to set the column header to, if not supplied then defaults to ‘col#’
index (index type) – The index (numeric or string) to insert (or replace) the data
func_args (dict) – If column_data is a callable object, then this argument can be used to supply a dictionary of function arguments to the callable object.
replace (bool) – Replace the data or insert the data (default)
setas (str) – Set the type of column (x,y,z data etc - see Stoner.Core.DataFile.setas)

Returns:

datafile – The DataFile instance with the additional column inserted.

Note

Like most DataFile methods, this method operates in-place in that it also modifies the original DataFile Instance as well as returning it.

annotate_fit(model, x=None, y=None, z=None, text_only=False, **kargs)¶

Annotate a plot with some information about a fit.

Parameters:: mode (callable or lmfit_mod.Model) – The function/model used to describe the fit to be annotated.

Keyword Parameters:

x (float):: x coordinate of the label
y (float):: y coordinate of the label
z (float):: z co-ordinbate of the label if the current axes are 3D
prefix (str):: The prefix placed ahead of the model parameters in the metadata.
text_only (bool):: If False (default), add the text to the plot and return the current object, otherwise, return just the text and don’t add to a plot.
prefix(str):: If given overridges the prefix from the model to determine a prefix to the parameter names in the metadata

Returns:: (Datam, str) – A copy of the current Data instance if text_only is False, otherwise returns the text.

If prefix is not given, then the first prefix in the metadata lmfit.prefix is used if present, otherwise a prefix is generated from the model.prefix attribute. If x and y are not specified then they are set to be 0.75 * maximum x and y limit of the plot.

append(value)¶: S.append(value) – append value to the end of the sequence

apply(func, col=None, replace=True, header=None, **kargs)¶

Apply the given function to each row in the data set and adds to the data set.

Parameters:

func (callable) – The function to apply to each row of the data.
col (index) – The column in which to place the result of the function

Keyword Arguments:

replace (bool) – Either replace the existing column/complete data or create a new column or data file.
header (string or None) – The new column header(s) (defaults to the name of the function func

Note

If any extra keyword arguments are supplied then these are passed to the function directly. If you need to pass any arguments that overlap with the keyword arguments to :py:math:`AnalysisMixin.apply` then these can be supplied in a dictionary argument _extra.

The callable func should have a signature:

def func(row,**kargs):

and should return either a single float, in which case it will be used to repalce the specified column, or an array, in which case it is used to completely replace the row of data.

If the function returns a complete row of data, then the replace parameter will cause the return value to be a new datafile, leaving the original unchanged. The headers parameter can give the complete column headers for the new data file.

Returns:: (Stoner.Data) – The newly modified Data object.

asarray()¶: Provide a consistent way to get at the underlying array data.

bin(xcol=None, ycol=None, bins=0.03, mode='log', clone=True, **kargs)¶

Bin x-y data into new values of x with an error bar.

Parameters:

xcol (index) – Index of column of data with X values
ycol (index) – Index of column of data with Y values
bins (int, float or 1d array) – Number of bins (if integer) or size of bins (if float), or bin edges (if array)
mode (string) – “log” or “lin” for logarithmic or linear binning

Keyword Arguments:

yerr (index) – Column with y-error data if present.
bin_start (float) – Manually override the minimum bin value
bin_stop (float) – Manually override the maximum bin value
clone (bool) – Return a clone of the current AnalysisMixin with binned data (True) or just the numbers (False).

Returns:

(Stoner.Data or tuple of 4 array-like) – Either a clone of the current data set with the new binned data or tuple of (bin centres, bin values, bin errors, number points/bin), depending on the clone parameter.

Note

See also

User Guide section (Re)Binning Data

clear() → None. Remove all items from D.¶

clip(clipper, column=None)¶

Clips the data based on the column and the clipper value.

Parameters:

column (index) – Column to look for the maximum in
clipper (tuple or array) – Either a tuple of (min,max) or a numpy.ndarray - in which case the max and min values in that array will be used as the clip limits

Returns:

(Stoner.Data) – The newly modified Data object.

Note

If column is not defined (or is None) the DataFile.setas column assignments are used.

closest(value, xcol=None)¶

Return the row in a data file which has an x-column value closest to the given value.

Parameters:: value (float) – Value to search for.
Keyword Arguments:: xcol (index or None) – Column in which to look for value, or None to use setas.
Returns:: ndarray – A single row of data as a Stoner.Core.DataArray.

Notes

To find which row it is that has been returned, use the Stoner.Core.DataArray.i index attribute.

colormap_xyz(xcol=None, ycol=None, zcol=None, **kargs)¶

Make a xyz plot that forces the use of plt.colormap.

Parameters:

xcol (index) – Xcolumn index or label
ycol (index) – Y column index or label
zcol (index) – Z column index or label

Keyword Arguments:

shape (two-tuple) – Number of points along x and y in the grid - defaults to a square of sidelength = square root of the length of the data.
xlim (tuple) – The xlimits, defaults to automatically determined from data
ylim (tuple) – The ylimits, defaults to automatically determined from data
plotter (function) – Function to use to plot data. Defaults to plt.contour
colorbar (bool) – Draw the z-scale color bar beside the plot (True by default)
show_plot (bool) – Turn on interfactive plotting and show plot when drawn
save_filename (string or None) – If set to a string, save the plot with this filename
figure (integer or matplotlib.figure or boolean) – Controls which figure is used for the plot, or if a new figure is opened.
**kargs (dict) – Other arguments are passed on to the plotter.

Returns:

A matplotlib figure

column(col)¶

Extract one or more columns of data from the datafile.

Parameters:: col (int, string, list or re) – is the column index as defined for DataFile.find_col()
Returns:: (ndarray) – One or more columns of data as a numpy.ndarray.

columns(not_masked=False, reset=False)¶

Iterate over the columns of data int he datafile.

Keyword Arguments:

no_masked (bool) – Only iterate over columns that don’t have masked elements
reset (bool) – If true then reset the iterator (immediately stops the current iteration without returning any data)./

Yields:

1D array – Returns the next column of data.

contour_xyz(xcol=None, ycol=None, zcol=None, shape=None, xlim=None, ylim=None, plotter=None, **kargs)¶

Make a xyz plot that forces the use of plt.contour.

Args:

xcol (index):
Xcolumn index or label

ycol (index):
Y column index or label

zcol (index):
Z column index or label

Keyword Arguments:

shape (two-tuple) – Number of points along x and y in the grid - defaults to a square of sidelength = square root of the length of the data.
xlim (tuple) – The xlimits, defaults to automatically determined from data
ylim (tuple) – The ylimits, defaults to automatically determined from data
plotter (function) – Function to use to plot data. Defaults to plt.contour
show_plot (bool) – Turn on interfactive plotting and show plot when drawn
save_filename (string or None) – If set to a string, save the plot with this filename
figure (integer or matplotlib.figure or boolean) – Controls which figure is used for the plot, or if a new figure is opened.
**kargs (dict) – Other arguments are passed on to the plotter.

Returns:

A matplotlib figure

count(value=None, axis=0, col=None)¶

Count the number of un-masked elements in the DataFile.

Keywords:

valiue (float):: Value to count for
axis (int):: Which axis to count the unmasked elements along.
col (index, None):: Restrict to counting in a specific column. If left None, then the current ‘y’ column is used.

Returns:: (int) – Number of unmasked elements.

curve_fit(func, xcol=None, ycol=None, sigma=None, **kargs)¶

General curve fitting function passed through from scipy.

Parameters:

func (callable, lmfit_mod.Model, odr.Model) – The fitting function with the form def f(x,*p) where p is a list of fitting parameters
xcol (index, Iterable) – The index of the x-column data to fit. If list or other iterable sends a tuple of x columns to func for N-d fitting.
ycol (index, list of indices or array) – The index of the y-column data to fit. If an array, then should be 1D and the same length as the data. If ycol is a list of indices then the columns are iterated over in turn, fitting occurring for each one. In this case the return value is a list of what would be returned for a single column fit.

Keyword Arguments:

p0 (list, tuple, array or callable) – A vector of initial parameter values to try. See notes below.
sigma (index) – The index of the column with the y-error bars
bounds (callable) – A callable object that evaluates true if a row is to be included. Should be of the form f(x,y)
result (bool) – Determines whether the fitted data should be added into the DataFile object. If result is True then the last column will be used. If result is a string or an integer then it is used as a column index. Default to None for not adding fitted data
replace (bool) – Inidcatesa whether the fitted data replaces existing data or is inserted as a new column (default False)
header (string or None) – If this is a string then it is used as the name of the fitted data. (default None)
absolute_sigma (bool) – If False, sigma denotes relative weights of the data points. The default True means that the sigma parameter is the reciprocal of the absolute standard deviation.
output (str, default "fit") – Specify what to return.

Returns:

(various) –

The return value is determined by the output parameter. Options are:

”fit” (tuple of popt,pcov) Optimal values of the fitting parameters p, and the
variance-co-variance matrix for the fitting parameters.
”row” just a one dimensional numpy array of the fit parameters interleaved with their
uncertainties
”full” a tuple of (popt,pcov,dictionary of optional outputs, message, return code, row).
”data” a copy of the Stoner.Core.DataFile object with fit recorded in the
metadata and optionally as a new column.

Note

If the columns are not specified (or set to None) then the X and Y data are taken using the Stoner.Core.DataFile.setas attribute.

The fitting function should have prototype y=f(x,p[0],p[1],p[2]…) The x-column and y-column can be anything that Stoner.Core.DataFile.find_col() can use as an index but typucally either strings to be matched against column headings or integers. The initial parameter values and weightings default to None which corresponds to all parameters starting at 1 and all points equally weighted. The bounds function has format b(x, y-vec) and rewturns true if the point is to be used in the fit and false if not.

The absolute_sigma keyword determines whether the returned covariance matrix pcov is based on estimated errors in the data, and is not affected by the overall magnitude of the values in sigma. Only the relative magnitudes of the sigma values matter. If True, sigma describes one standard deviation errors of the input data points. The estimated covariance in pcov is based on these values.

The starting vector p0 can be either a list, tuple or array, or a callable that will produce a list, tuple or array. IF callable, it should take the form:

def p0_func(ydata,x=xdata):

and return a list of parameter values that is in the same order as the model function. If p0 is not given and a lmfit_mod.Model or scipy.odr.Model is supplied as the model function, then the model’s estimates of the starting values will be used instead.

See also

Stoner.Data.lmfit()
Stoner.Data.odr()
Stoner.Data.differential_evolution()
User guide section Curve Fitting in the Stoner Package

decompose(xcol=None, ycol=None, sym=None, asym=None, replace=True, **kwords)¶

Given (x,y) data, decomposes the y part into symmetric and antisymmetric contributions in x.

Keyword Arguments:

xcol (index) – Index of column with x data - defaults to first x column in self.setas
ycol (index or list of indices) – indices of y column(s) data
sym (index) – Index of column to place symmetric data in default, append to end of data
asym (index) – Index of column for asymmetric part of ata. Defaults to appending to end of data
replace (bool) – Overwrite data with output (true)

Returns:

self – The newly modified AnalysisMixin.

Example

"""Decompose Into symmetric and antisymmetric parts example."""

from numpy import linspace, reshape, array

from Stoner import Data
from Stoner.tools import format_val

x = linspace(-10, 10, 201)
y = 0.3 * x**3 - 6 * x**2 + 11 * x - 20
d = Data(x, y, setas="xy", column_headers=["X", "Y"])
d.decompose()
d.setas = "xyyy"
coeffs = d.polyfit(polynomial_order=3)
str_coeffs = [format_val(c, mode="eng", places=1) for c in coeffs.ravel()]
str_coeffs = reshape(array(str_coeffs), coeffs.shape)
d.plot()
d.text(
    -6,
    -800,
    "Coefficients\n{}".format(str_coeffs),
    fontdict={"size": "x-small"},
)
d.ylabel = "Data"
d.title = "Decompose Example"

(png, hires.png, pdf)

del_column(col=None, duplicates=False)¶

Delete a column from the current DataFile object.

Parameters:: col (int, string, iterable of booleans, list or re) – is the column index as defined for DataFile.find_col() to the column to be deleted
Keyword Arguments:: duplicates (bool) – (default False) look for duplicated columns
Returns:: datafile – The DataFile object with the column deleted.

Note

If duplicates is True and col is None then all duplicate columns are removed,
if col is not None and duplicates is True then all duplicates of the specified column are removed.
If duplicates is False and col is either None or False then all masked coplumns are deleeted. If
col is True, then all columns that are not set i the setas attrobute are deleted.
If col is a list (duplicates should not be None) then the all the matching columns are found.
If col is an iterable of booleans, then all columns whose elements are False are deleted.
If col is None and duplicates is None, then all columns with at least one elelemtn masked
will be deleted

del_nan(col=None, clone=False)¶

Remove rows that have nan in them.

eyword Arguments:

col (index types or None):: column(s) to look for nan’s in. If None or not given, use setas columns.
clone (boolean):: if True clone the current object before running and then return the clone not datafile.

Returns:: datafile (DataFile) – Returns a copy of the current object (or clone if *clone*=True)

del_rows(col=None, val=None, invert=False)¶

Search in the numerica data for the lines that match and deletes the corresponding rows.

Parameters:

col (list,slice,int,string, re, callable or None) – Column containing values to search for.
val (float or callable) –
Specifies rows to delete. Maybe:
- None - in which case the col argument is used to identify rows to be deleted,
- a float in which case rows whose columncol = val are deleted
- or a function - in which case rows where the function evaluates to be true are deleted.
- a tuple, in which case rows where column col takes value between the minimum and maximum of the tuple are deleted.

Keyword Arguments:

invert (bool) – Specifies whether to invert the logic of the test to delete a row. If True, keep the rows that would have been deleted otherwise.

Returns:

datafile – The current DataFile object

Note

If col is None, then all rows with masked data are deleted

if col is callable then it is passed each row as a DataArray and if it returns True, then the row will be deleted or kept depending on the value of invert.

If val is a callable it should take two arguments - a float and a list. The float is the value of the current row that corresponds to column col abd the second argument is the current row.

Todo

Implement val is a tuple for deletinging in a range of values.

differential_evolution(model, xcol=None, ycol=None, p0=None, sigma=None, **kargs)¶

Fit model to the data using a differential evolution algorithm.

Parameters:

model (lmfit_mod.Model) – An instance of an lmfit_mod.Model that represents the model to be fitted to the data
xcol (index or None) – Columns to be used for the x data for the fitting. If not givem defaults to the Stoner.Core.DataFile.setas x column
ycol (index or None) – Columns to be used for the y data for the fitting. If not givem defaults to the Stoner.Core.DataFile.setas y column

Keyword Arguments:

p0 (list, tuple, array or callable) – A vector of initial parameter values to try. See the notes in Stoner.Data.curve_fit() for more details.
sigma (index) – The index of the column with the y-error bars
bounds (callable) – A callable object that evaluates true if a row is to be included. Should be of the form f(x,y)
result (bool) – Determines whether the fitted data should be added into the DataFile object. If result is True then the last column will be used. If result is a string or an integer then it is used as a column index. Default to None for not adding fitted data
replace (bool) – Inidcatesa whether the fitted data replaces existing data or is inserted as a new column (default False)
header (string or None) – If this is a string then it is used as the name of the fitted data. (default None)
scale_covar (bool) – whether to automatically scale covariance matrix (leastsq only)
output (str, default "fit") – Specify what to return.

Returns:

( various ) –

The return value is determined by the output parameter. Options are

”fit” just the lmfit_mod.Model.ModelFit instance that contains all relevant
information about the fit.
”row” just a one dimensional numpy array of the fit parameters interleaved with their
uncertainties
”full” a tuple of the fit instance and the row.
”data” a copy of the Stoner.Core.DataFile object with the fit recorded in the
emtadata and optionally as a column of data.

This function is essentially a wrapper around the scipy.optimize.differential_evolution() function that presents the same interface as the other Stoner package curve fitting functions. The parent function, however, does not provide the variance-covariance matrix to estimate the fitting errors. To work around this, this function does the initial fit with the differential evolution, but then uses that to give a starting vector to a call to scipy.optimize.curve_fit() to calculate the covariance matrix.

See also

Stoner.Data.curve_fit()
Stoner.Data.lmfit()
Stoner.Data.odr()
User guide section Curve Fitting in the Stoner Package

Example

"""Simple use of lmfit to fit data."""

from numpy import linspace, exp, random

from Stoner import Data

random.seed(12345)  # Ensure consistent random numbers!
# Make some data
x = linspace(0, 10.0, 101)
y = 2 + 4 * exp(-x / 1.7) + random.normal(scale=0.2, size=101)

d = Data(x, y, column_headers=["Time", "Signal"], setas="xy")

func = lambda x, A, B, C: A + B * exp(-x / C)


# Do the fitting and plot the result
fit = d.differential_evolution(
    func,
    result=True,
    header="Fit",
    A=1,
    B=1,
    C=1,
    prefix="Model",
    residuals=True,
)

# Reset labels
d.labels = []

# Make nice two panel plot layout
ax = d.subplot2grid((3, 1), (2, 0))
d.setas = "x..y"
d.plot(fmt="g+")
d.title = ""

ax = d.subplot2grid((3, 1), (0, 0), rowspan=2)
d.setas = "xyy"
d.plot(fmt=["r.", "b-"])
d.xticklabels = [[]]
d.xlabel = ""

# Annotate plot with fitting parameters
d.annotate_fit(func, prefix="Model", x=0.7, y=0.3, fontdict={"size": "x-small"})
text = r"$y=A+Be^{-x/C}$" + "\n\n"
d.text(7.2, 3.9, text, fontdict={"size": "x-small"})
d.title = "Differential Evolution  Fit"

(png, hires.png, pdf)

diffsum(col_a, col_b, replace=False, header=None, index=None)¶

Calculate \(\frac{a-b}{a+b}\) for the two columns a and b.

Parameters:

col_a (index) – First column to work with
col_b (index, float or 1D array) – Second column to work with.

Keyword Arguments:

header (string or None) – new column header (defaults to a-b
replace (bool) – Replace the col_a column with the new data
index (column index or None) – Column to insert new data at.

Returns:

(Stoner.Data) – The newly modified Data object.

If col_a and col_b are tuples of length two, then the firstelement is assumed to be the value and the second element an uncertainty in the value. The uncertainties will then be propagated and an additional column with the uncertainites will be added to the data.

dir(pattern=None)¶

Return a list of keys in the metadata, filtering with a regular expression if necessary.

Keyword Arguments:: pattern (string or re) – is a regular expression or None to list all keys
Returns:: list – A list of metadata keys.

divide(col_a, col_b, replace=False, header=None, index=None)¶

Divide one column (col_a) by another column, number or array (col_b).

Parameters:

col_a (index) – First column to work with
col_b (index, float or 1D array) – Second column to work with.

Keyword Arguments:

header (string or None) – new column header (defaults to a-b
replace (bool) – Replace the col_a column with the new data
index (column index or None) – Column to insert new data at.

Returns:

(Stoner.Data) – The newly modified Data object.

If col_a and col_b are tuples of length two, then the firstelement is assumed to be the value and the second element an uncertainty in the value. The uncertainties will then be propagated and an additional column with the uncertainites will be added to the data.

extend(values)¶: S.extend(iterable) – extend sequence by appending elements from the iterable

extrapolate(new_x, xcol=None, ycol=None, yerr=None, overlap=20, kind='linear', errors=None)¶

Extrapolate data based on local fit to x,y data.

Parameters:

new_x (float or array) – New values of x data.

Keyword Arguments:

xcol (column index, None) – column containing x-data or None to use setas attribute
ycol (column index(es) or None) – column(s) containing the y-data or None to use setas attribute.
yerr (column index(es) or None) – y error data column or None to use setas attribute
overlap (float or int) – range of x-data used for the local fit for extrapolating. If int then overlap number of points is used, if float then that range x-axis space is used.
kind (str or callable) – Determines local fitting function. If string should be “linear”, “quadratic” or “cubic” if callable, then represents a function to be fitted to the data.
errors (callable or None) – If kind is a callable function, then errs must be defined and must also be a callable function.

Returns:

(array) – Extrapolated values.

Note

If the new_x values lie outside the span of the x-data, then the nearest overlap portion of the data is used to estimate the values. If the new_x values are within the span of the x-data then the portion of the data centred about the point and overlap points long will be used to interpolate a value.

If kind is callable, it should take x values in the first parameter and free fitting parameters as the other parameters (i.e. as with AnalysisMixin.curve_fit()).

figure(figure=None, projection='rectilinear', **kargs)¶

Set the figure used by Stoner.plot.PlotMixin.

Parameters:: figure (matplotlib.Figure or int) – Figure to switch to
Returns:: The current Stoner.plot.PlotMixin instance

filter(func=None, cols=None, reset=True)¶

Set the mask on rows of data by evaluating a function for each row.

Parameters:

func (callable) – is a callable object that should take a single list as a p[parameter representing one row.
cols (list) – a list of column indices that are used to form the list of values passed to func.
reset (bool) – determines whether the mask is reset before doing the filter (otherwise rows already masked out will be ignored in the filter (so the filter is logically or’d)) The default value of None results in a complete row being passed into func.

Returns:

datafile – The current DataFile object with the mask set

find_col(col, force_list=False)¶

Indexes the column headers in order to locate a column of data.shape.

Indexing can be by supplying an integer, a string, a regular expression, a slice or a list of any of the above.

Integer indices are simply checked to ensure that they are in range
String indices are first checked for an exact match against a column header if that fails they are then compiled to a regular expression and the first match to a column header is taken.
A regular expression index is simply matched against the column headers and the first match found is taken. This allows additional regular expression options such as case insensitivity.
A slice index is converted to a list of integers and processed as below
A list index returns the results of feading each item in the list at find_col() in turn.

Parameters:: col (int, a string, a re, a slice or a list) – Which column(s) to retuirn indices for.
Keyword Arguments:: force_list (bool) – Force the output always to be a list. Mainly for internal use only
Returns:: int, list of ints – The matching column index as an integer or a KeyError

find_duplicates(xcol=None, delta=1e-08)¶

Find rows with duplicated values of the search column(s).

Keyword Arguments:

xcol (index types) – The column)s) to search for duplicates in.
delta (float or array) – The absolute difference(s) to consider equal when comparing floats.

Returns:

(dictionary of value –

[list of row indices]):: The unique value and the associated rows that go with it.

Notes

If xcol is not specified, then the Data.setas attribute is used. If this is also not set, then all columns are considered.

find_peaks(**kargs)¶

Interface to scipy.signal.find_peaks() for locating peaks in data.

Parameters:

ycol (index) – the column name or index of the data in which to search for peaks

Keyword Arguments:

xcol (index) – the column name or index of the x data that the peaks correspond to.
height – number or ndarray or sequence, optional Required height of peaks. Either a number, None, an array matching ycol or a 2-element sequence of the former. The first element is always interpreted as the minimal and the second, if supplied, as the maximal required height.
threshold – number or ndarray or sequence, optional Required threshold of peaks, the vertical distance to its neighbouring samples. Either a number, None, an array matching ycol or a 2-element sequence of the former. The first element is always interpreted as the minimal and the second, if supplied, as the maximal required threshold.
distance – number, optional Required minimal horizontal distance (>= 1) in samples between neighbouring peaks. Smaller peaks are removed first until the condition is fulfilled for all remaining peaks. If this is a float and xcol is set, then the units are in terms of the x-data, otherwise in rwo indices.
prominence – number or ndarray or sequence, optional Required prominence of peaks. Either a number, None, an array matching ycol or a 2-element sequence of the former. The first element is always interpreted as the minimal and the second, if supplied, as the maximal required prominence.
width – number or ndarray or sequence, optional Required width of peaks in samples. Either a number, None, an array matching ycol or a 2-element sequence of the former. The first element is always interpreted as the minimal and the second, if supplied, as the maximal required width. If this is a float and xcol is set, then the units are in terms of the x-data, otherwise in rwo indices.
wlen – int, optional Used for calculation of the peaks prominences, thus it is only used if one of the arguments prominence or width is given. See argument wlen in peak_prominences for a full description of its effects.
rel_height – float, optional Used for calculation of the peaks width, thus it is only used if width is given. See argument rel_height in peak_widths for a full description of its effects.
plateau_size – number or ndarray or sequence, optional Required size of the flat top of peaks in samples. Either a number, None, an array matching ycol or a 2-element sequence of the former. The first element is always interpreted as the minimal and the second, if supplied as the maximal required plateau size. If this is a float and xcol is set, then the units are in terms of the x-data, otherwise in rwo indices.
prefix (str) – If et, then the metadata keys that return information about the peaks is returned with the given prefix. Default is None - no prefix.
sort (bool) – Sor the results by prominence of peak
modify (book) – If true, then the returned object is a copy of datafile with only the peaks left in the data.
full_data (bool) – If True (default) then all columns of the data at which peaks in the ycol column are found. modify true implies full_data is also true. If full_data is False, then only the x-column values of the peaks are returned.

Returns:

(various) – If modify is true, then returns a the AnalysisMixin with the data set to just the peaks/troughs. If modify is false (default), then the return value depends on ycol and xcol. If ycol is not None and xcol is None, then returns complete rows of data corresponding to the found peaks/troughs. If xcol is not None, or ycol is None and xcol is None, then returns a 1D array of the x positions of the peaks/troughs.

See also

User guide section Peak Finding

format(key, **kargs)¶

Return the contents of key pretty formatted using format_error().

Parameters:

fmt (str) – Specify the output format, options are:
- “text” - plain text output
- “latex” - latex output
- “html” - html entities
escape (bool) – Specifies whether to escape the prefix and units for unprintable characters in non text formats )default False)
mode (string) – If “float” (default) the number is formatted as is, if “eng” the value and error is converted to the next samllest power of 1000 and the appropriate SI index appended. If mode is “sci” then a scientific, i.e. mantissa and exponent format is used.
units (string) – A suffix providing the units of the value. If si mode is used, then appropriate si prefixes are prepended to the units string. In LaTeX mode, the units string is embedded in mathrm
prefix (string) – A prefix string that should be included before the value and error string. in LaTeX mode this is inside the math-mode markers, but not embedded in mathrm.

Returns:

A pretty string representation.

The if key=”key”, then the value is datafile[“key”], the error is datafile[“key err”], the default prefix is datafile[“key label”]+”=” or “key=”, the units are datafile[“key units”] or “”.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.¶

get_filename(mode)¶

Force the user to choose a new filename using a system dialog box.

Parameters:: mode (string) – The mode of file operation to be used when calling the dialog box
Returns:: str – The new filename

Note

The filename attribute of the current instance is updated by this method as well.

griddata(xcol=None, ycol=None, zcol=None, ucol=None, shape=None, xlim=None, ylim=None, zlim=None, method='linear', **kargs)¶

Convert xyz data onto a regular grid.

Parameters:

xcol (index) – Xcolumn index or label
ycol (index) – Y column index or label
zcol (index) – Z column index or label

Keyword Arguments:

ucol (index) – U (magnitude) column index or label
shape (two-tuple, three-ruple) – Number of points along x and y in the grid - defaults to a square of sidelength = square root of the length of the data.
xlim (tuple) – The xlimits
ylim (tuple) – The ylimits
zlim (tuple) – The ylimits
method (string) – Type of interpolation to use, default is linear
ReturnsL –

(X,Y,Z) or (X,Y,Z,M):
three two dimensional arrays of the coordinates of the interpolated data or 4 three diemensional arrays of the interpolated data

Notes

Depending on whether 3 or 4 columns of data can be identified, this method will produce data for a \(Z(X,Y)\) plot or a \(M(X,Y,Z)\) volumetric plot.

image_plot(xcol=None, ycol=None, zcol=None, shape=None, xlim=None, ylim=None, **kargs)¶

Grid up the three columns of data and plot.

Parameters:

xcol (index) – Xcolumn index or label
ycol (index) – Y column index or label
zcol (index) – Z column index or label

Keyword Arguments:

shape (two-tuple) – Number of points along x and y in the grid - defaults to a square of sidelength = square root of the length of the data.
xlim (tuple) – The xlimits, defaults to automatically determined from data
ylim (tuple) – The ylimits, defaults to automatically determined from data
xlabel (string) – X axes label. Default is None - guess from xvals or metadata
ylabel (string) – Y axes label, Default is None - guess from metadata
zlabel (string) – Z axis label, Default is None - guess from metadata
plotter (function) – Function to use to plot data. Defaults to plt.contour
show_plot (bool) – Turn on interfactive plotting and show plot when drawn
save_filename (string or None) – If set to a string, save the plot with this filename
figure (integer or matplotlib.figure or boolean) – Controls which figure is used for the plot, or if a new figure is opened.
**kargs (dict) – Other arguments are passed on to the plotter.

Returns:

A matplotlib figure

index(value[, start[, stop]]) → integer -- return first index of value.¶

Raises ValueError if the value is not present.

Supporting start and stop arguments is optional, but recommended.

insert(index, obj)¶: Implement the insert method.

insert_rows(row, new_data)¶

Insert new_data into the data array at position row. This is a wrapper for numpy.insert.

Parameters:

row (int) – Data row to insert into
new_data (numpy array) – An array with an equal number of columns as the main data array containing the new row(s) of data to insert

Returns:

datafile – A copy of the modified DataFile object

inset(parent=None, loc=None, width=0.35, height=0.3, **kargs)¶

Add a new set of axes as an inset to the current plot.

Keyword Arguments:

parent (matplotlib axes) – Which set of axes to add inset to, defaults to the current set
loc (int or string) – Inset location - can be a string like top right or upper right or a number.
width,height (int,float or string) – the dimensions of the inset specified as a integer %, or floating point fraction of the parent axes, or as a string measurement.
kargs (dictionary) – all other keywords are passed through to inset_locator.inset_axes

Returns:

A new set of axes

integrate(xcol=None, ycol=None, result=None, header=None, result_name=None, output='data', bounds=<function AnalysisMixin.<lambda>>, **kargs)¶

Integrate a column of data, optionally returning the cumulative integral.

Parameters:

xcol (index) – The X data column index (or header)
ycol (index)
The Y data column index (or header)

Keyword Arguments:

result (index or None) – Either a column index (or header) to overwrite with the cumulative data, or True to add a new column or None to not store the cumulative result.
result_name (str) – The metadata name for the final result
header (str) – The name of the header for the results column.
output (Str) – What to return - ‘data’ (default) - this object, ‘result’: final result
bounds (callable) – A function that evaluates for each row to determine if the data should be integrated over.
**kargs – Other keyword arguments are fed direct to the scipy.integrate.cumtrapz method

Returns:

(Stoner.Data) – The newly modified Data object.

Note

This is a pass through to the scipy.integrate.cumtrapz routine which just uses trapezoidal integration. A better alternative would be to offer a variety of methods including simpson’s rule and interpolation of data. If xcol or ycol are not specified then the current values from the Stoner.Core.DataFile.setas attribute are used.

interpolate(newX, kind='linear', xcol=None, replace=False)¶

Interpolate a dataset to get a new set of values for a given set of x data.

Parameters:

ewX (1D array or None) – Row indices or X column values to interpolate with. If None, then the AnalysisMixin.interpolate() returns an interpolation function. Unlike the raw interpolation function from scipy, this interpolation function will work with MaskedArrays by compressing them first.

Keyword Arguments:

kind (string) – Type of interpolation function to use - does a pass through from numpy. Default is linear.
xcol (index or None) – Column index or label that contains the data to use with newX to determine which rows to return. Defaults to None.
replace (bool) – If true, then the current AnalysisMixin’s data is replaced with the newly interpolated data and the current AnalysisMixin is returned.

Returns:

(2D numpy array) – Section of the current object’s data if replace is False(default) or the modofied AnalysisMixin if replace is true.

Note

Returns complete rows of data corresponding to the indices given in newX. if xcol is None, then newX is interpreted as (fractional) row indices. Otherwise, the column specified in xcol is thresholded with the values given in newX and the resultant row indices used to return the data.

If the positional argument, newX is None, then the return value is an interpolation function. This interpolation function takes one argument - if xcol was None, this argument is interpreted as array indices, but if xcol was specified, then this argument is interpreted as an array of xvalues.

items() → Tuple[str, Any]¶: Make sure we implement an items that doesn’t just iterate over self.

keys() → str¶: Return the keys of the metadata dictionary.

legend(*args, **kargs)¶: Pass Through to stop attribute access over-riding a handy method.

lmfit(model, xcol=None, ycol=None, p0=None, sigma=None, **kargs)¶

Wrap the lmfit module fitting.

Parameters:

model (lmfit_mod.Model) – An instance of an lmfit_mod.Model that represents the model to be fitted to the data
xcol (index or None) – Columns to be used for the x data for the fitting. If not givem defaults to the Stoner.Core.DataFile.setas x column
ycol (index or None) – Columns to be used for the y data for the fitting. If not givem defaults to the Stoner.Core.DataFile.setas y column

Keyword Arguments:

p0 (list, tuple, array or callable) – A vector of initial parameter values to try. See the notes in Stoner.Data.curve_fit() for more details.
sigma (index) – The index of the column with the y-error bars
bounds (callable) – A callable object that evaluates true if a row is to be included. Should be of the form f(x,y)
result (bool) – Determines whether the fitted data should be added into the DataFile object. If result is True then the last column will be used. If result is a string or an integer then it is used as a column index. Default to None for not adding fitted data
replace (bool) – Inidcatesa whether the fitted data replaces existing data or is inserted as a new column (default False)
header (string or None) – If this is a string then it is used as the name of the fitted data. (default None)
scale_covar (bool) – whether to automatically scale covariance matrix (leastsq only)
output (str, default "fit") – Specify what to return.

Returns:

( various ) –

The return value is determined by the output parameter. Options are

”fit” just the lmfit_mod.Model.ModelFit instance that contains all relevant
information about the fit.
”row” just a one dimensional numpy array of the fit parameters interleaved with their
uncertainties
”full” a tuple of the fit instance and the row.
”data” a copy of the Stoner.Core.DataFile object with the fit recorded in the
emtadata and optionally as a column of data.

See also

Note

If p0 is fed a 2D array, then it assumed that you want to calculate \(\chi^2\) for different starting parameters with some variables fixed. In this mode, fitting is carried out repeatedly with each row representing one attempt with different values of the parameters. In this mode the return value is a 2D array whose rows correspond to the inputs to the rows of p0, the columns are the fitted values of the parameters with an additional column for \(\chi^2\).

Example

"""Simple use of lmfit to fit data."""

from numpy import linspace, exp, random

from Stoner import Data

random.seed(12345)  # Ensure Consistent Random numbers
# Make some data
x = linspace(0, 10.0, 101)
y = 2 + 4 * exp(-x / 1.7) + random.normal(scale=0.2, size=101)

d = Data(x, y, column_headers=["Time", "Signal"], setas="xy")

# Do the fitting and plot the result
func = lambda x, A, B, C: A + B * exp(-x / C)
fit = d.lmfit(
    func,
    result=True,
    header="Fit",
    A=1,
    B=1,
    C=1,
    residuals=True,
    output="report",
)

# Reset labels
d.labels = []

# Make nice two panel plot layout
ax = d.subplot2grid((3, 1), (2, 0))
d.setas = "x..y"
d.plot(fmt="g+")
d.title = ""

ax = d.subplot2grid((3, 1), (0, 0), rowspan=2)
d.setas = "xyy"
d.plot(fmt=["ro", "b-"])
d.xticklabels = [[]]
d.xlabel = ""

# Annotate plot with fitting parameters
d.annotate_fit(func, prefix="Model", x=7.2, y=3, fontdict={"size": "x-small"})
text = r"$y=A+Be^{-x/C}$" + "\n\n"
d.text(7.2, 3.9, text, fontdict={"size": "x-small"})
d.title = "Levenberg-Marquardt Fit"

(png, hires.png, pdf)

classmethod load(*args, **kargs)¶

Create a new Data from a file on disc guessing a better subclass if necessary.

Parameters:

filename (string or None) – path to file to load

Keyword Arguments:

auto_load (bool) – If True (default) then the load routine tries all the subclasses of Data in turn to load the file
filetype (Data, str) – If not none then tries using filetype as the loader.
loaded_class (bool) – If True, the return object is kept as the class that managed to load it, otherwise it is copied into a Stoner.Data object. (Default False)

Returns:

(Data) – A new instance of Stoner.Data or a s subclass of Stoner.Data if loaded_class is True.

Note

If filetype is a string, then it is first tried as an exact match to a subclass name, otherwise it is used as a partial match and the first class in priority order is that matches is used.

Some subclasses can be found in the Stoner.formats package.

Each subclass is scanned in turn for a class attribute priority which governs the order in which they are tried. Subclasses which can make an early positive determination that a file has the correct format can have higher priority levels. Classes should return a suitable exception if they fail to load the file.

If no class can load a file successfully then a StonerUnrecognisedFormat exception is raised.

main_data(data_grp)[source]¶: Work through the main data group and build something that looks like a numpy 2D array.

make_bins(xcol, bins, mode='lin', **kargs)¶

Generate bin boundaries and centres along an axis.

Parameters:

xcol (index) – Column of data with X values
bins (1d_)array or int or float) – Number of bins (int) or width of bins (if float)
mode (string) – “lin” for linear binning, “log” for logarithmic binning.

Keyword Arguments:

bin_start (float) – Override minimum bin value
bin_stop (float) – Override the maximum bin value

Returns:

(tuple of 4 arrays) – bin_start,bin_stop,bin_centres (1D arrays): The locations of the bin boundaries and centres for each bin.

max(column=None, bounds=None)¶

Find maximum value and index in col_a column of data.

Parameters:: column (index) – Column to look for the maximum in
Keyword Arguments:: bounds (callable) – col_a callable function that takes col_a single argument list of numbers representing one row, and returns True for all rows to search in.
Returns:: (float,int) – (maximum value,row index of max value)

Note

If column is not defined (or is None) the DataFile.setas column assignments are used.

mean(column=None, sigma=None, bounds=None)¶

Find mean value of col_a data column.

Parameters:

column (index) – Column to look for the maximum in

Keyword Arguments:

sigma (column index or array) – The uncertainty noted for each value in the mean
bounds (callable) – col_a callable function that takes col_a single argument list of numbers representing one row, and returns True for all rows to search in.

Returns:

(float) – The mean of the data.

Note

If column is not defined (or is None) the DataFile.setas column assignments are used.

Todo

Fix the row index when the bounds function is used - see note of AnalysisMixin.max()

min(column=None, bounds=None)¶

Find minimum value and index in col_a column of data.

Parameters:: column (index) – Column to look for the maximum in
Keyword Arguments:: bounds (callable) – col_a callable function that takes col_a single argument list of numbers representing one row, and returns True for all rows to search in.
Returns:: (float,int) – (minimum value,row index of min value)

Note

If column is not defined (or is None) the DataFile.setas column assignments are used.

multiply(col_a, col_b, replace=False, header=None, index=None)¶

Multiply one column (col_a) by another column, number or array (col_b).

Parameters:

col_a (index) – First column to work with
col_b (index, float or 1D array) – Second column to work with.

Keyword Arguments:

header (string or None) – new column header (defaults to a-b
replace (bool) – Replace the col_a column with the new data
index (column index or None) – Column to insert new data at.

Returns:

(Stoner.Data) – The newly modified Data object.

If col_a and col_b are tuples of length two, then the firstelement is assumed to be the value and the second element an uncertainty in the value. The uncertainties will then be propagated and an additional column with the uncertainites will be added to the data.

normalise(target=None, base=None, replace=True, header=None, scale=None, limits=(0.0, 1.0))¶

Normalise data columns by dividing through by a base column value.

Parameters:

target (index) – One or more target columns to normalise can be a string, integer or list of strings or integers. If None then the default ‘y’ column is used.

Keyword Arguments:

base (index) – The column to normalise to, can be an integer or string. Deprecated can also be a tuple (low, high) being the output range
replace (bool) – Set True(default) to overwrite the target data columns
header (string or None) – The new column header - default is target name(norm)
scale (None or tuple of float,float) – Output range after normalising - low,high or None to map to -1,1
limits (float,float) – (low,high) - Take the input range from the high and low fraction of the input when sorted.

Returns:

(Stoner.Data) – The newly modified Data object.

Notes

The limits parameter is used to set the input scale being normalised from - if the data has a few outliers then this setting can be used to clip the input range before normalising. The parameters in the limit are the values at the low and high fractions of the cumulative distribution function of the data.

odr(model, xcol=None, ycol=None, **kargs)¶

Wrap the scipy.odr orthogonal distance regression fitting.

Parameters:

model (scipy.odr.Model, lmfit_mod.Models.Model or callable) – The model that describes the data. See below for more details.
xcol (index or None) – Columns to be used for the x data for the fitting. If not givem defaults to the Stoner.Core.DataFile.setas x column
ycol (index or None) – Columns to be used for the y data for the fitting. If not givem defaults to the Stoner.Core.DataFile.setas y column

Keyword Arguments:

p0 (list, tuple, array or callable) – A vector of initial parameter values to try. See the notes to Stoner.Data.curve_fit() for more details.
sigma_x (index) – The index of the column with the x-error bars
sigma_y (index) – The index of the column with the x-error bars
bounds (callable) – A callable object that evaluates true if a row is to be included. Should be of the form f(x,y)
result (bool) – Determines whether the fitted data should be added into the DataFile object. If result is True then the last column will be used. If result is a string or an integer then it is used as a column index. Default to None for not adding fitted data
replace (bool) – Inidcatesa whether the fitted data replaces existing data or is inserted as a new column (default False)
header (string or None) – If this is a string then it is used as the name of the fitted data. (default None)
output (str, default "fit") – Specify what to return.

Returns:

( various ) –

The return value is determined by the output parameter. Options are

”fit” just the scipy.odr.Output instance (default)
”row” just a one dimensional numpy array of the fit parameters interleaved with their
uncertainties
”full” a tuple of the fit instance and the row.
”data” a copy of the Stoner.Core.DataFile object with the fit recorded in the

emtadata and optionally

as a column of data.

Notes

The function tries to make use of whatever model you give it. Specifically, it accepts:

A subclass or an instance of scipy.odr.Model : this is the native model type for the underlying scipy odr package.

A subclass or instance of an lmfit_mod.Models.Model: the Stoner.analysis.fitting.models package has a number of useful prebuilt lmfit models that can be used directly by this function.

A callable function which should have a signature f(x,parameter1,parameter2…) and not the scip.odr standard f(beta,x)

This function is designed to be as compatible as possible with AnalysisMixin.curve_fit() and: AnalysisMixin.lmfit() to facilitate easy of switching between them.

See also

AnalysisMixin.curve_fit()
AnalysisMixin.lmfit()
Stoner.Data.differential_evolution()
User guide section Fitting with limits

Example

"""Simple use of lmfit to fit data."""

from numpy import linspace, exp, random

from Stoner import Data
from Stoner.plot.utils import errorfill

random.seed(12345)  # Ensure consistent random numbers!

# Make some data
x = linspace(0, 10.0, 101)
y = 2 + 4 * exp(-x / 1.7) + random.normal(scale=0.2, size=101)
x += +random.normal(scale=0.1, size=101)

d = Data(x, y, column_headers=["Time", "Signal"], setas="xy")

func = lambda x, A, B, C: A + B * exp(-x / C)

# Do the fitting and plot the result
fit = d.odr(
    func,
    result=True,
    header="Fit",
    A=1,
    B=1,
    C=1,
    prefix="Model",
    residuals=True,
)

# Reset labels
d.labels = []

# Make nice two panel plot layout
ax = d.subplot2grid((3, 1), (2, 0))
d.setas = "x..y"
d.plot(fmt="g+")
d.title = ""

# Plot up the data
ax = d.subplot2grid((3, 1), (0, 0), rowspan=2)
d.setas = "xy"
d.plot(fmt="ro")

d.setas = "x.y"
d.plot(plotter=errorfill, yerr=0.2, color="orange")
d.plot(plotter=errorfill, xerr=0.1, color="orange", label=None)
d.xticklabels = [[]]
d.xlabel = ""

# Annotate plot with fitting parameters
d.annotate_fit(func, prefix="Model", x=0.7, y=0.3, fontdict={"size": "x-small"})
text = r"$y=A+Be^{-x/C}$" + "\n\n"
d.text(7.2, 3.9, text, fontdict={"size": "x-small"})
d.title = "Orthogonal Distance Regression  Fit"

(png, hires.png, pdf)

outlier_detection(column=None, window=7, shape='boxcar', certainty=3.0, action='mask', width=1, func=None, **kargs)¶

Detect outliers in a column of data.

Parameters:

column (column index) – specifying column for outlier detection. If not set, defaults to the current y set column.

Keyword Arguments:

window (int) – data window for anomaly detection
shape (str) – The name of a scipy.signal windowing function to use when averaging the data. Defaults to ‘boxcar’ for a flat average.
certainty (float) – eg 3 detects data 3 standard deviations from average
action (str or callable) – what to do with outlying points, options are * ‘mask’ outlier points are masked (default) * ‘mask row’ outlier rows are masked * ‘delete’ outlier rows are deleted * callable the value of the action keyword is called with the outlier row * anything else defaults to do nothing.
width (odd integer) – Number of rows that an outliing spike could occupy. Defaults to 1.
func (callable) – A function that determines if the current row is an outlier.
action_args (tuple) – if action is callable, then action_args can be used to pass extra arguments to the action callable
action_kargs (dict) – If action is callable, then action_kargs can be used to pass extra keyword arguments to the action callable.

Returns:

(Stoner.Data) – The newly modified Data object.

outlier_detection will add row numbers of detected outliers to the metadata of d, also will perform action depending on request eg ‘mask’, ‘delete’ (any other action defaults to doing nothing).

The detection looks at a window of the data, takes the average and looks to see if the current data point falls certainty * std deviations away from data average.

The outlier detection function has the signatrure:

def outlier(row,column,window,certainty,**kargs)
    #code
    return True # or False

All extra keyword arguments are passed to the outlier detector.

IF action is a callable function then it should take the form of:

def action(i,column, data, *action_args, **action_kargs):
    pass

where i is the number of the outlier row, column the same value as above and data is the complete set of data.

In all cases the indices of the outlier rows are added to the ;outlier’ metadata.

Example

"""Detect outlying points from a lione."""

import numpy as np

from Stoner import Data
from Stoner.analysis.utils import poly_outlier

np.random.seed(12345)
x = np.linspace(0, 100, 201)
y = 0.01 * x**2 + 5 * np.sin(x / 10.0)

i = np.random.randint(len(x) - 20, size=20) + 10
y[i] += np.random.normal(size=len(i), scale=20)

d = Data(np.column_stack((x, y)), column_headers=["x", "y"], setas="xy")
d.plot(fmt="b.", label="raw data")
e = d.clone
e.outlier_detection(window=5, action="delete")
e.plot(fmt="r-", label="Default Outliers removed")
h = d.clone
h.outlier_detection(window=5, action="delete", shape="hamming")
h.plot(color="orange", label="Default Outliers removed with Hamming window")
f = d.clone
f.outlier_detection(
    window=21, order=3, certainty=2, width=3, action="delete", func=poly_outlier
)
f.plot(fmt="g-", label="Poly Outliers removed")
g = d.clone
g = g.outlier_detection(
    window=21, order=3, certainty=3, width=3, action="delete", func=poly_outlier
)
g.plot(color="purple", label="Masked outliers")
g = d.clone
e.title = "Outlier detection test"

(png, hires.png, pdf)

peaks(**kargs)¶

Locates peaks and/or troughs in a column of data by using SG-differentiation.

Parameters:

ycol (index) – the column name or index of the data in which to search for peaks
width (int or float) – the expected minimum halalf-width of a peak in terms of the number of data points (int) or distance in x (float). This is used in the differnetiation code to find local maxima. Bigger equals less sensitive to experimental noise, smaller means better eable to see sharp peaks
poly (int) – the order of polynomial to use when differentiating the data to locate a peak. Must >=2, higher numbers will find sharper peaks more accurately but at the risk of finding more false positives.

Keyword Arguments:

significance (float) – used to decide whether a local maxmima is a significant peak. Essentially just the curvature of the data. Bigger means less sensitive, smaller means more likely to detect noise. Default is the maximum curvature/(2*width)
xcol (index or None) – name or index of data column that p[provides the x-coordinate (default None)
peaks (bool) – select whether to measure peaks in data (default True)
troughs (bool) – select whether to measure troughs in data (default False)
sort (bool) – Sor the results by significance of peak
modify (book) – If true, then the returned object is a copy of datafile with only the peaks/troughs left in the data.
full_data (bool) – If True (default) then all columns of the data at which peaks in the ycol column are found. modify true implies full_data is also true. If full_data is False, then only the x-column values of the peaks are returned.

Returns:

(various) – If modify is true, then returns a the AnalysisMixin with the data set to just the peaks/troughs. If modify is false (default), then the return value depends on ycol and xcol. If ycol is not None and xcol is None, then returns complete rows of data corresponding to the found peaks/troughs. If xcol is not None, or ycol is None and xcol is None, then returns a 1D array of the x positions of the peaks/troughs.

See also

User guide section Peak Finding

plot(*args, **kargs)¶

Try to make an appropriate plot based on the defined column assignments.

The column assignments are examined to determine whether to plot and x,y plot or an x,y,z plot and whether to plot error bars (for an x,y plot). All keyword argume nts are passed through to the selected plotting routine.

plot_matrix(xvals=None, yvals=None, rectang=None, cmap=<matplotlib.colors.ListedColormap object>, show_plot=True, title='', xlabel=None, ylabel=None, zlabel=None, figure=None, plotter=None, **kwords)¶

Plot a surface plot by assuming that the current dataset represents a regular matrix of points.

Parameters:

xvals (index, list or numpy.array) – Either a column index or name or a list or numpytarray of column values. The default (None) uses the first column of data
yvals (int or list) – Either a row index or a list or numpy array of row values. The default (None) uses the column_ headings interpreted as floats
rectang (tuple) – a tuple of either 2 or 4 elements representing either the origin (row,column) or size (origin, number of rows, number of columns) of data to be used for the z0data matrix

Keyword Arguments:

cmap (matplotlib colour map) – Surface colour map - defaults to the jet colour map
show_plot (bool) – True Turns on interactive plot control
title (string) – Optional parameter that specifies the plot title - otherwise the current DataFile filename is used
xlabel (string) – X axes label. Default is None - guess from xvals or metadata
ylabel (string) – Y axes label, Default is None - guess from metadata
zlabel (string) – Z axis label, Default is None - guess from metadata
figure (matplotlib figure) – Controls what matplotlib figure to use. Can be an integer, or a matplotlib.figure or False. If False then a new figure is always used, otherwise it will default to using the last figure used by this DataFile object.
plotter (callable) – Optional argument that passes a plotting function into the routine. Sensible choices might be plt.plot (default), py.semilogy, plt.semilogx
kwords (dict) – A dictionary of other keyword arguments to pass into the plot function.
Returns – The matplotib figure with the data plotted

plot_voxels(xcol=None, ycol=None, zcol=None, ucol=None, cmap=None, **kargs)¶

Make a volumetric plot of data arranged as x,y,z,u.

Parameters:

xcol (index) – Xcolumn index or label
ycol (index) – Y column index or label
zcol (index) – Z column index or label
ucol (index) – U column index or label

Keyword Arguments:

visible (callable) – A function f(x,y,z) that returns True if a voxcel is to be visible
cmap (colourmap) – A Matplotlib colour map to apply for the magnitude (u column) data.

Returns:

(matplotlib.Figure) – The figure window contacting the plot

Example

"""3D surface plot example."""

import numpy as np
import matplotlib.cm

from Stoner import Data

x, y, z = np.meshgrid(
    np.linspace(-2, 2, 21), np.linspace(-2, 2, 21), np.linspace(-2, 2, 21)
)
x = x.ravel()
y = y.ravel()
z = z.ravel()
u = np.sin(x * y * z)

p = Data(x, y, z, u, setas="xyzu", column_headers=["X", "Y", "Z"])

p.plot_voxels(cmap=matplotlib.cm.jet, visible=lambda x, y, z: x - y + z < 2.0)
p.set_box_aspect((1, 1, 1.0))  # Passing through to the current axes
p.title = "Voxel plot"

(png, hires.png, pdf)

plot_xy(xcol=None, ycol=None, fmt=None, xerr=None, yerr=None, **kargs)¶

Makesa simple X-Y plot of the specified data.

Parameters:

xcol (index) – Xcolumn index or label
ycol (index) – Y column index or label

Keyword Arguments:

fmt (strong or sequence of strings) – Specifies the format for the plot - see matplotlib documentation for details
xerr,yerr (index) – C
plt.errorbar (olumns of data to get x and y errorbars from. Setting these turns the default plotter to)
xlabel (string) – X axes label. Default is None - guess from xvals or metadata
ylabel (string) – Y axes label, Default is None - guess from metadata
title (string) – Optional parameter that specifies the plot title - otherwise the current DataFile filename is used
plotter (function) – Function to use to plot data. Defaults to plt.plot unless error bars are set
show_plot (bool) – Turn on interfactive plotting and show plot when drawn
save_filename (string or None) – If set to a string, save the plot with this filename
figure (integer or matplotlib.figure or boolean) – Controls which figure is used for the plot, or if a new figure is opened.
multiple (string) –
how to handle multiple y-axes with a common x axis. Options are:
- common single y-axis (default)
- panels panels sharing common x axis
- sub plots sub plots
- y2 single axes with 2 y scales
**kargs (dict) – Other arguments are passed on to the plotter.

Returns:

A matplotlib.figure instance

plot_xyuv(xcol=None, ycol=None, ucol=None, vcol=None, wcol=None, **kargs)¶

Make an overlaid image and quiver plot.

Args:

!c xcol (index):

Xcolumn index or label

ycol (index):
Y column index or label

zcol (index):
Z column index or label

ucol (index):
U column index or label

vcol (index):
V column index or label

wcol (index):
W column index or label

Keyword Arguments:

show_plot (bool):: True Turns on interactive plot control
title (string):: Optional parameter that specifies the plot title - otherwise the current DataFile filename is used
save_filename (string):: Filename used to save the plot
figure (matplotlib figure):: Controls what matplotlib figure to use. Can be an integer, or a matplotlib.figure or False. If False then a new figure is always used, otherwise it will default to using the last figure used by this DataFile object.
no_quiver (bool):: Do not overlay quiver plot (in cases of dense meshes of points)
plotter (callable):: Optional argument that passes a plotting function into the routine. Default is a 3d surface plotter, but contour plot and pcolormesh also work.
**kargs (dict):: A dictionary of other keyword arguments to pass into the plot function.

plot_xyuvw(xcol=None, ycol=None, ucol=None, vcol=None, wcol=None, **kargs)¶

Make an overlaid image and quiver plot.

Args:

!c xcol (index):

Xcolumn index or label

ycol (index):
Y column index or label

zcol (index):
Z column index or label

ucol (index):
U column index or label

vcol (index):
V column index or label

wcol (index):
W column index or label

Keyword Arguments:

show_plot (bool):: True Turns on interactive plot control
title (string):: Optional parameter that specifies the plot title - otherwise the current DataFile filename is used
save_filename (string):: Filename used to save the plot
figure (matplotlib figure):: Controls what matplotlib figure to use. Can be an integer, or a matplotlib.figure or False. If False then a new figure is always used, otherwise it will default to using the last figure used by this DataFile object.
no_quiver (bool):: Do not overlay quiver plot (in cases of dense meshes of points)
plotter (callable):: Optional argument that passes a plotting function into the routine. Default is a 3d surface plotter, but contour plot and pcolormesh also work.
**kargs (dict):: A dictionary of other keyword arguments to pass into the plot function.

plot_xyz(xcol=None, ycol=None, zcol=None, shape=None, xlim=None, ylim=None, projection='3d', **kargs)¶

Plot a surface plot based on rows of X,Y,Z data using matplotlib.pcolor().

Parameters:

xcol (index) – Xcolumn index or label
ycol (index) – Y column index or label
zcol (index) – Z column index or label

Keyword Arguments:

shape (tuple) – Defines the shape of the surface (i.e. the number of X and Y value. If not provided or None, then the routine will attempt to calculate these from the data provided
xlim (tuple) – Defines the x-axis limits and grid of the data to be plotted
ylim (tuple) – Defines the Y-axis limits and grid of the data data to be plotted
cmap (matplotlib colour map) – Surface colour map - defaults to the jet colour map
show_plot (bool) – True Turns on interactive plot control
title (string) – Optional parameter that specifies the plot title - otherwise the current DataFile filename is used
save_filename (string) – Filename used to save the plot
figure (matplotlib figure) – Controls what matplotlib figure to use. Can be an integer, or a matplotlib.figure or False. If False then a new figure is always used, otherwise it will default to using the last figure used by this DataFile object.
plotter (callable) – Optional argument that passes a plotting function into the routine. Default is a 3d surface plotter, but contour plot and pcolormesh also work.
projection (string or None) – Whether to use a 3D projection or regular 2D axes (default is 3D)
**kargs (dict) – A dictionary of other keyword arguments to pass into the plot function.

Returns:

A matplotlib.figure instance

plot_xyzuvw(xcol=None, ycol=None, zcol=None, ucol=None, vcol=None, wcol=None, **kargs)¶

Plot a vector field plot based on rows of X,Y,Z (U,V,W) data using ,ayavi.

Parameters:

xcol (index) – Xcolumn index or label
ycol (index) – Y column index or label
zcol (index) – Z column index or label
ucol (index) – U column index or label
vcol (index) – V column i ndex or label
wcol (index) – W column index or label

Keyword Arguments:

colormap (string) – Vector field colour map - defaults to the jet colour map
colors (column index or numpy array) – Values used to map the colors of the resultant file.
mode (string) – glyph type, default is “cone”
scale_factor (float) – Scale-size of glyphs.
figure (mlab figure) – Controls what mlab figure to use. Can be an integer, or a mlab.figure or False. If False then a new figure is always used, otherwise it will default to using the last figure used by this DataFile object.
plotter (callable) – Optional argument that passes a plotting function into the routine. Sensible choices might be plt.plot (default), py.semilogy, plt.semilogx
kargs (dict) – A dictionary of other keyword arguments to pass into the plot function.

Returns:

A mayavi scene instance

polyfit(xcol=None, ycol=None, polynomial_order=2, bounds=<function <lambda>>, result=None, replace=False, header=None)¶

Pass through to numpy.polyfit.

Parameters:

xcol (index) – Index to the column in the data with the X data in it
ycol (index) – Index to the column int he data with the Y data in it
polynomial_order (int) – Order of polynomial to fit (default 2)
bounds (callable) – A function that evaluates True if the current row should be included in the fit
result (index or None) – Add the fitted data to the current data object in a new column (default don’t add)
replace (bool) – Overwrite or insert new data if result is not None (default False)
header (string or None) – Name of column_header of replacement data. Default is construct a string from the y column headser and polynomial order.

Returns:

(numpy.poly) – The best fit polynomial as a numpy.poly object.

Note

If the x or y columns are not specified (or are None) the the setas attribute is used instead.

This method is deprecated and may be removed in a future version in favour of the more general: curve_fit

pop(k[, d]) → v, remove specified key and return the corresponding value.¶: If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair¶: as a 2-tuple; but raise KeyError if D is empty.

quiver_plot(xcol=None, ycol=None, ucol=None, vcol=None, **kargs)¶

Make a 2D Quiver plot from the data.

Parameters:

xcol (index) – Xcolumn index or label
ycol (index) – Y column index or label
zcol (index) – Z column index or label
ucol (index) – U column index or label
vcol (index) – V column i ndex or label
wcol (index) – W column index or label

Keyword Arguments:

xlabel (string) – X axes label. Default is None - guess from xvals or metadata
ylabel (string) – Y axes label, Default is None - guess from metadata
zlabel (string) – Z axis label, Default is None - guess from metadata
plotter (function) – Function to use to plot data. Defaults to plt.contour
headlength,headwidth,headaxislength (float) – Controls the size of the quiver heads
show_plot (bool) – Turn on interfactive plotting and show plot when drawn
save_filename (string or None) – If set to a string, save the plot with this filename
figure (integer or matplotlib.figure or boolean) – Controls which figure is used for the plot, or if a new figure is opened.
**kargs (dict) – Other arguments are passed on to the plotter.

Returns:

A matplotlib figure instance.

Keyword arguments are all passed through to matplotlib.plt.quiver().

remove(value)¶: S.remove(value) – remove first occurrence of value. Raise ValueError if the value is not present.

remove_duplicates(xcol=None, delta=1e-08, strategy='keep first', ycol=None, yerr=None)¶

Find and remove rows with duplicated values of the search column(s).

Keyword Arguments:

xcol (index types) – The column)s) to search for duplicates in.
delta (float or array) – The absolute difference(s) to consider equal when comparing floats.
strategy (str, default keep first) –
What to do with duplicated rows. Options are:
- keep first - the first row is kept, others are discarded
- average - the duplicate rows are average together.
yerr (ycol,) – When using an average strategey identifies columns that represent values and uncertainties where the proper weighted standard error should be done.

Returns:

(dictionary of value –

[list of row indices]):: The unique value and the associated rows that go with it.

Notes

If ycol is not specified, then the Data.setas attribute is used. If this is also not set, then all columns are considered.

rename(old_col, new_col)¶

Rename columns without changing the underlying data.

Parameters:

old_col (string, int, re) – Old column index or name (using standard rules)
new_col (string) – New name of column

Returns:

datafile – A copy of the modified DataFile instance

reorder_columns(cols, headers_too=True, setas_too=True)¶

Construct a new data array from the original data by assembling the columns in the order given.

Parameters:

cols (list of column indices) – (referred to the oriignal data set) from which to assemble the new data set
headers_too (bool) – Reorder the column headers in the same way as the data (defaults to True)
setas_too (bool) – Reorder the column assignments in the same way as the data (defaults to True)

Returns:

datafile – A copy of the modified DataFile object

reverse()¶: S.reverse() – reverse IN PLACE

rolling_window(window=7, wrap=True, exclude_centre=False)¶

Iterate with a rolling window section of the data.

Keyword Arguments:

window (int) – Size of the rolling window (must be odd and >= 3)
wrap (bool) – Whether to use data from the other end of the array when at one end or the other.
exclude_centre (odd int or bool) – Exclude the ciurrent row from the rolling window (defaults to False)

Yields:

ndarray – Yields with a section of data that is window rows long, each iteration moves the marker one row further on.

rows(not_masked=False, reset=False)¶

Iterate over rows of data.

Keyword Arguments:

not_masked (bool) – If a row is masked and this is true, then don’t return this row.
reset (bool) – If true then reset the iterator (immediately stops the current iteration without returning any data)./

Yields:

1D array – Returns the next row of data

save(filename=None, **kargs)¶

Save a string representation of the current DataFile object into the file ‘filename’.

Parameters:

filename (string, bool or None) – Filename to save data as, if this is None then the current filename for the object is used. If this is not set, then then a file dialog is used. If filename is False then a file dialog is forced.
as_loaded (bool,str) – If True, then the Loaded as key is inspected to see what the original class of the DataFile was and then this class’ save method is used to save the data. If a str then the keyword value is interpreted as the name of a subclass of the the current DataFile.

Returns:

datafile – The current DataFile object

scale(other, xcol=None, ycol=None, **kargs)¶

Scale the x and y data in this DataFile to match the x and y data in another DataFile.

Parameters:

other (DataFile) – The other instance of a datafile to match to

Keyword Arguments:

xcol (column index) – Column with x points in it, default to None to use setas attribute value
ycol (column index) – Column with ypoints in it, default to None to use setas attribute value
xmode ('affine', 'linear','scale','offset') – How to manipulate the x-data to match up
ymode ('linear','scale','offset') – How to manipulate the y-data to match up.
bounds (callable) – Used to identiyf the set of (x,y) points to be used for scaling. Defaults to the whole data set if not speicifed.
otherbounds (callable) – Used to detemrine the set of (x,y) points in the other data file. Defaults to bounds if not given.
use_estimate (bool or 3x2 array) – Specifies whether to estimate an initial transformation value or to use the provided one, or start with an identity transformation.
replace (bool) – Whether to map the x,y data to the new coordinates and return a copy of this AnalysisMixin (true) or to just return the results of the scaling.
headers (2-element list or tuple of strings) – new column headers to use if replace is True.

Returns:

(various) – Either a copy of the :py:class:Stoner.Data` modified so that the x and y columns match other if replace is True, or opt_trans,*trans_err*,*new_xy_data*. Where opt_trans is the optimum affine transformation, trans_err is a matrix giving the standard error in the transformation matrix components and new_xy_data is an (n x 2) array of the transformed data.

Example

"""Example of using scale to overlap data."""

from numpy import linspace, sin, exp, pi, column_stack
from numpy.random import normal, seed
import matplotlib as mpl
from tabulate import tabulate

from Stoner import Data

seed(3)  # Just fix the random numbers to stop optimizer warnings
mpl.rc("text", usetex=True)


x = linspace(0, 10 * pi, 201)
x2 = x * 1.5 + 0.23
y = 10 * exp(-x / (2 * pi)) * sin(x) + normal(size=len(x), scale=0.1)
y2 = 3 * exp(-x / (2 * pi)) * sin(x) - 1 + normal(size=len(x), scale=0.1)

d = Data(x, y, column_headers=["Time", "Signal 1"], setas="xy")
d2 = Data(x2, y2, column_headers=["Time", "Signal 2"], setas="xy")

d.plot(label="1$^\\mathrm{st}$ signal")
d2.plot(figure=d.fig, label="2$^\\mathrm{nd}$ signal")
d3 = d2.scale(d, header="Signal 2 scaled", xmode="affine")
d3.plot(figure=d.fig, label="1$^\\mathrm{st}$ scaled signals")
d3["test"] = linspace(1, 10, 10)
txt = tabulate(d3["Transform"], floatfmt=".2f", tablefmt="grid")
d3.text(10, 4, "Transform\n{}".format(txt), fontdict={"size": "x-small"})

np_data = column_stack((x2, y2))
d4 = d.scale(
    np_data, header="Signal 2 scaled", xmode="affine", use_estimate=True
)
d4.plot(figure=d.fig, label="2$^\\mathrm{nd}$ scaled signal")
d4.ylim = (-7, 9)
txt = tabulate(d4["Transform"], floatfmt=".2f", tablefmt="grid")
d4.text(10, -7, "Transform\n{}".format(txt), fontdict={"size": "x-small"})

d4.title = "Scaling Example"

(png, hires.png, pdf)

scan_group(grp, pth)[source]¶: Recursively list HDF5 Groups.

search(xcol=None, value=None, columns=None, accuracy=0.0)¶

Search the numerica data part of the file for lines that match and returns the corresponding rows.

Keyword Arguments:

xcol (index types, None) – a Search Column Index. If None (default), use the current setas.x
value (float, tuple, list or callable, None) – Value to look for
columns (index or array of indices or None (default)) – columns of data to return - none represents all columns.
accuracy (float) – Uncertainty to accept when testing equalities

Returns:

ndarray – numpy array of matching rows or column values depending on the arguments.

Note

The value is interpreted as follows:

a float looks for an exact match
a list is a list of exact matches
an array or list of booleans (index like Numpy does)
a tuple should contain a (min,max) value.
A callable object should have accept a float and an array representing the value of the search col for the the current row and the entire row.
None opens an interactive span selector in a plot window.

search_index(xcol=None, value=None, accuracy=0.0, invert=False)¶: Return an array of booleans for indexing matching rows for use with search method.

section(**kargs)¶

Assuming data has x,y or x,y,z coordinates, return data from a section of the parameter space.

Keyword Arguments:

x (float, tuple, list or callable) – x values ,atch this condition are included inth e section
y (float, tuple, list or callable) – y values ,atch this condition are included inth e section
z (float, tuple,list or callable) – z values ,atch this condition are included inth e section
r (callable) – a
tuple (function that takes a)

Returns:

(DataFile) – A DataFile like object that includes only those lines from the original that match the section specification

Internally this function is calling DataFile.search() to pull out matching sections of the data array. To extract a 2D section of the parameter space orthogonal to one axis you just specify a condition on that axis. Specifying conditions on two axes will return a line of points along the third axis. The final keyword parameter allows you to select data points that lie in an arbitrary plane or line. eg:

d.section(r=lambda x,y,z:abs(2+3*x-2*y)<0.1 and z==2)

would extract points along the line 2y=3x+2 (note the use of an < operator to avoid floating point rounding errors) where the z-co-ordinate is 2.

select(*args, **kargs)¶

Produce a copy of the DataFile with only data rows that match a criteria.

Parameters:

args (various) – A single positional argument if present is interpreted as follows:

If a callable function is given, the entire row is presented to it. If it evaluates True then that row is selected. This allows arbitrary select operations
If a dict is given, then it and the kargs dictionary are merged and used to select the rows

Keyword Arguments:

kargs (various) –

Arbitrary keyword arguments are interpreted as requestion matches against the corresponding columns. The keyword argument may have an additional __operator* appended to it which is interpreted as follows:

eq value equals argument value (this is the default test for scalar argument)
ne value doe not equal argument value
gt value doe greater than argument value
lt value doe less than argument value
ge value doe greater than or equal to argument value
le value doe less than or equal to argument value
between value lies between the minimum and maximum values of the argument (the default test for 2-length tuple arguments)
ibetween,*ilbetween*,*iubetween* as above but include both,lower or upper values

Returns:

(DatFile) – a copy the DataFile instance that contains just the matching rows.

Note

if the operator is preceeded by __not__ then the sense of the test is negated.

If any of the tests is True, then the row will be selected, so the effect is a logical OR. To achieve a logical AND, you can chain two selects together:

d.select(temp__le=4.2,vti_temp__lt=4.2).select(field_gt=3.0)

will select rows that have either temp or vti_temp metadata values below 4.2 AND field metadata values greater than 3.

If you need to select on a row value that ends in an operator word, then append __eq in the keyword name to force the equality test. If the metadata keys to select on are not valid python identifiers, then pass them via the first positional dictionary value.

There is a “magic” column name “_i” which is interpreted as the row numbers of the data.

Example

"""Example using select method to pick out data."""

from Stoner import Data

d = Data("sample.txt", setas="xy")
d.plot(fmt="b-")
d.select(Temp__gt=75).select(Res__between=(5.3, 6.3)).plot(
    fmt="ro", label="portion 1"
)
d.select(Temp__lt=30).plot(fmt="g<", label="portion 2")

(png, hires.png, pdf)

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶

smooth(window='boxcar', xcol=None, ycol=None, size=None, **kargs)¶

Smooth data by convoluting with a window.

Parameters:

window (string or tuple) – Defines the window type to use by passing to scipy.signal.get_window().

Keyword Arguments:

xcol (column index or None) – Data to use as x data if needed to define a window. If None, use Stoner.Core.DataFile.setas
ycvol (column index or None) – Data to be smoothed
size (int or float) – If int, then the number of points to use in the smoothing window. If float, then the size in x-data to be used.
result (bool or column index) – Whether to add the smoothed data to the dataset and if so where.
replace (bool) – Replace the exiting data or insert as a new column.
header (string) – New column header for the new data.

Returns:

(datafile or array) – If result is False, then the return value will be a copy of the smoothed data, otherwise the return value is a copy of the AnalysisMixin object with the smoothed data added,

Notes

If size is float, then it is necessary to map the X-data to a number of rows and to ensure that the data is evenly spaced in x. To do this, the number of rows in the window is found by dividing the span in x by the size and multiplying by the total lenfth. Then the data is interpolated to a new set of evenly space X over the same range, smoothed and then interpoalted back to the original x values.

sort(*order, **kargs)¶

Sort the data by column name.

Parameters:: order (column index or list of indices or callable function) – One or more sort order keys.
Keyword Arguments:: reverse (boolean) – If true, the sorted array isreversed.
Returns:: (datafile) – A copy of the DataFile sorted object

Notes

Sorts in place and returns a copy of the sorted data object fo chaining methods.

If the argument is a callable function then it should take a two tuple arguments and return +1,0,-1 depending on whether the first argument is bigger, equal or smaller. Otherwise if the argument is interpreted as a column index. If a single argument is supplied, then it may be a list of column indices. If no sort orders are supplied then the data is sorted by the DataFile.setas attribute or if that is not set, then order of the columns in the data.

span(column=None, bounds=None)¶

Return a tuple of the maximum and minimum values within the given column and bounds.

Parameters:: column (index) – Column to look for the maximum in
Keyword Arguments:: bounds (callable) – col_a callable function that takes col_a single argument list of numbers representing one row, and returns True for all rows to search in.
Returns:: (float,float) – col_a tuple of (min value, max value)

Note

This works by calling into Data.max() and Data.min().

If column is not defined (or is None) the DataFile.setas column assignments are used.

spline(xcol=None, ycol=None, sigma=None, **kargs)¶

Construct a spline through x and y data and replace, add new data or return spline function.

Keyword Arguments:

xcol (column index) – Column with x data or if None, use setas attribute.
ycol (column index) – Column with y data or if None, use the setas attribute
sigma (column index, or array of data) – Column with weights, or if None use the 1/yerr column.
replace (Boolean or column index or None) – If True then the y-column data is repalced, if a column index then the new data is added after the specified index, if False then the new y-data is returned and if None, then spline object is returned.
header (string) – If replace is True or a column index then use this string as the new column header.
order (int) – The order of spline to use (1-5)
smoothing (float or None) – The smoothing factor to use when fitting the spline. A value of zero will create an interpolating spline.
bbox (tuple of length 2) – Bounding box for the spline - defaults to range of x values
ext (int or str) – How to extrapolate, default is “extrapolate”, but can also be “raise”,”zeros” or “const”.

Returns:

(various) – Depending on the value of replace, returns a copy of the AnalysisMixin, a 1D numpy array of data or an :[y:class:scipy.interpolate.UniverateSpline object.

This is really just a pass through to the scipy.interpolate.UnivariateSpline function. Also used in the extrapolate function.

split(*args, final='files')¶

Recursively splits the current DataFile into a Stoner.Folders.DataFolder.

Parameters:: *args (column index or function) – Each argument is used in turn to find key values for the files in the DataFolder
Keyword Arguments:: final (str) – Controls whether the final argument plaes the files in the DataFolder (default: “files”) or in groups (“groups”)
Returns:: Stoner.Folders.DataFolder – A Stoner.Folders.DataFolder object containing the individual AnalysisMixin objects

Note

Creates a DataFolder of DataFiles where each one contains the rows from the original object which had the same value of a given column(s) or function.

On each iteration the first argument is called. If it is a column type then rows which amtch each unique value are collated together and made into a separate file. If the argument is a callable, then it is called for each row, passing the row as a single 1D array and the return result is used to group lines together. The return value should be hashable.

Once this is done and the Stoner.Folders.DataFolder exists, if there are remaining argument, then the method is called recusivelyt for each file and the resulting DataFolder added into the root DataFolder and the file is removed.

Thus, when all of the arguments are evaluated, the resulting DataFolder is a multi-level tree.

Warning

There has been a change in the arguments for the split function from version 0.8 of the Stoner Package.

std(column=None, sigma=None, bounds=None)¶

Find standard deviation value of col_a data column.

Parameters:

column (index) – Column to look for the maximum in

Keyword Arguments:

sigma (column index or array) – The uncertainty noted for each value in the mean
bounds (callable) – col_a callable function that takes col_a single argument list of numbers representing one row, and returns True for all rows to search in.

Returns:

(float) – The standard deviation of the data.

Note

If column is not defined (or is None) the DataFile.setas column assignments are used.

Todo

Fix the row index when the bounds function is used - see note of AnalysisMixin.max()

stitch(other, xcol=None, ycol=None, overlap=None, min_overlap=0.0, mode='All', func=None, p0=None)¶

Apply a scaling to this data set to make it stich to another dataset.

Parameters:

other (DataFile) – Another data set that is used as the base to stitch this one on to
xcol,ycol (index or None) – The x and y data columns. If left as None then the current setas attribute is used.

Keyword Arguments:

overlap (tuple of (lower,higher) or None) – The band of x values that are used in both data sets to match, if left as None, thenthe common overlap of the x data is used.
min_overlap (float) – If you know that overlap must be bigger than a certain amount, the bounds between the two data sets needs to be adjusted. In this case min_overlap shifts the boundary of the overlap on this DataFile.
mode (str) – Unless func is specified, controls which parameters are actually variable, defaults to all of them.
func (callable) – a stitching function that transforms \((x,y)\rightarrow(x',y')\). Default is to use functions defined by mode
p0 (iterable) – if func is not None then p0 should be the starting values for the stitching function parameters

Returns:

(Stoner.Data) – A copy of the current AnalysisMixin with the x and y data columns adjusted to stitch

To stitch the data together, the x and y data in the current data file is transforms so that \(x'=x+A\) and \(y'=By+C\) where \(A,B,C\) are constants and \((x',y')\) are close matches to the \((x,y)\) data in other. The algorithm assumes that the overlap region contains equal numbers of \((x,y)\) points mode controls whether A,B, and C are fixed or adjustable

“All” - all three parameters adjustable

“Scale y, shift x” - C is fixed at 0.0

“Scale and shift y” A is fixed at 0.0

“Scale y” - only B is adjustable

“Shift y” - Only c is adjsutable

“Shift x” - Only A is adjustable

“Shift both” - B is fixed at 1.0

See also

User Guide section Stitching Datasets together

Example

subplot(*args, **kargs)¶

Pass throuygh for matplotlib.pyplot.subplot().

Parameters:

rows (int) – If this is the only argument, then a three digit number representing the rows,columns,index arguments. If separate rows, column and index are provided, then this is the number of rows of sub-plots in one figure.
columns (int) – The number of columns of sub-plots in one figure.
index (int) – Index (1 based) of the current sub-plot.

Returns:

A matplotlib.Axes instance representing the current sub-plot

As well as passing through to the plyplot routine of the same name, this function maintains a list of the current sub-plot axes via the subplots attribute.

subplot2grid(*args, **kargs)¶: Provide a pass through to matplotlib.pyplot.subplot2grid().

subtract(col_a, col_b, replace=False, header=None, index=None)¶

Subtract one column, number or array (col_b) from another column (col_a).

Parameters:

col_a (index) – First column to work with
col_b (index, float or 1D array) – Second column to work with.

Keyword Arguments:

header (string or None) – new column header (defaults to a-b
replace (bool) – Replace the col_a column with the new data
index (column index or None) – Column to insert new data at.

Returns:

(Stoner.Data) – The newly modified Data object.

If col_a and col_b are tuples of length two, then the firstelement is assumed to be the value and the second element an uncertainty in the value. The uncertainties will then be propagated and an additional column with the uncertainites will be added to the data.

swap_column(*swp, **kargs)¶

Swap pairs of columns in the data.

Useful for reordering data for idiot programs that expect columns in a fixed order.

Parameters:

swp (tuple of list of tuples of two elements) – Each element will be iused as a column index (using the normal rules for matching columns). The two elements represent the two columns that are to be swapped.
headers_too (bool) – Indicates the column headers are swapped as well

Returns:

datafile – A copy of the modified DataFile objects

Note

If swp is a list, then the function is called recursively on each element of the list. Thus in principle the @swp could contain lists of lists of tuples

threshold(threshold, **kargs)¶

Find partial indices where the data in column passes the threshold, rising or falling.

Parameters:

threshold (float) – Value to look for in column col

Keyword Arguments:

col (index) – Column index to look for data in
rising (bool) – look for case where the data is increasing in value (default True)
falling (bool) – look for case where data is fallinh in value (default False)
xcol (index, bool or None) – rather than returning a fractional row index, return the interpolated value in column xcol. If xcol is False, then return a complete row all_vals (bool): return all crossing points of the threshold or just the first. (default False)
transpose (bbool) – Swap the x and y columns around - this is most useful when the column assignments have been done via the setas attribute
all_vals (bool) – Return all values that match the criteria, or just the first in the file.

Returns:

(float) – Either a sing;le fractional row index, or an in terpolated x value

Note

If you don’t specify a col value or set it to None, then the assigned columns via the DataFile.setas attribute will be used.

Warning

There has been an API change. Versions prior to 0.1.9 placed the column before the threshold in the positional argument list. In order to support the use of assigned columns, this has been swapped to the present order.

to_pandas()¶

Create a pandas DataFrame from a Stoner.Data object.

Notes

In addition to transferring the numerical data, the DataFrame’s columns are set to a multi-level index of the Stoner.Data.column_headers and Stoner.Data.setas values. A pandas DataFrame extension attribute, metadata is registered and is used to store the metada from the :py:class:1Stoner.Data` object. This pandas extension attribute is in fact a trivial subclass of the Stoner.core.TypeHintedDict.

The inverse operation can be carried out simply by passing a DataFrame into the copnstructor of the Stoner.Data object.

Raises:: **NotImplementedError** if pandas didn't import correctly. –

unique(col, return_index=False, return_inverse=False)¶

Return the unique values from the specified column - pass through for numpy.unique.

Parameters:

col (index) – Column to look for unique values in

Keyword Arguments:

return_index (bool) – Pass through to np.unique()
reverse (bool) – Pass through to np.unique()

Returns:

(1D array) – Array of unique values from the column.

update([E, ]**F) → None. Update D from mapping/iterable E and F.¶: If E present and has a .keys() method, does: for k in E.keys(): D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → Any¶: Return the values of the metadata dictionary.

x2()¶

Generate a new set of axes with a second x-scale.

Returns:: The new matplotlib.axes instance.

y2()¶

Generate a new set of axes with a second y-scale.

Returns:: The new matplotlib.axes instance

Stoner Pacakge API Documentation

HGXFile¶

Table of Contents

Previous topic

Next topic

This Page