dvas.data.strategy package

Distributed under the terms of the GNU General Public License v3.0 or later.

SPDX-License-Identifier: GPL-3.0-or-later

Submodules

dvas.data.strategy.data module

Distributed under the terms of the GNU General Public License v3.0 or later.

SPDX-License-Identifier: GPL-3.0-or-later

Module contents: Data manager classes used in dvas.data.data.ProfileManager

class dvas.data.strategy.data.MPStrategyAC

Bases: object

Abstract class (AC) for a multiprofile (MP) strategy

abstract execute(*args, **kwargs): Execute strategy method

class dvas.data.strategy.data.ProfileAC

Bases: object

Abstract Profile class

REQUIRED_ATTRIBUTES = {'DF_COLS_ATTR': <class 'dict'>}

DF_COLS_ATTR = None

property data

Data.

Type:: pd.DataFrame

property columns

DataFrame columns name

Type:: pd.Index

classmethod reset_data_index(val)

Return the data with reset index

Parameters:: val (pandas.DataFrame) – DataFrame with index to be reset
Returns:: pandas.DataFrame

classmethod set_data_index(val)

Return the data with reset index

Parameters:: val (pandas.DataFrame) – DataFrame with index to be reset
Returns:: pandas.DataFrame

classmethod get_index_attr()

Get index attributes

Returns:: list

classmethod get_col_attr()

Get columns attributes

Returns:: list

abstract copy(): Copy method

class dvas.data.strategy.data.Profile(info, data=None)

Bases: ProfileAC

Base Profile class for atmospheric measurements. Requires only some measured values, together with their corresponding altitudes and flags.

The data is stored in a pandas DataFrame with column labels:

‘alt’ (float)
‘val’ (float)
‘flg’ (int)

The same format is expected as input.

FLG_BIT_POS_NM = 'bit_pos'

FLG_NAME_NM = 'flg_name'

FLG_DESC_NM = 'flg_desc'

DF_COLS_ATTR = {'alt': {'index': True, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'flg': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.int64'>}, 'val': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}}

__init__(info, data=None)

Profile Constructor.

Parameters:

info (InfoManager) – Data information
data (pd.DataFrame, optional) – The profile values in a pandas DataFrame. Default to None.

property info

Corresponding data info

Type:: InfoManager

property flg_names

Flag name, description and bit position.

Type:: dict

property alt

Corresponding data altitude

Type:: pd.Series

property val

Corresponding data ‘val’

Type:: pd.Series

property flg

Corresponding data ‘flag’

Type:: pd.Series

copy(): Copy method

set_flg(val, set_val, index=None)

Set flag values to True/False.

Parameters:

val (str) – Flag name
set_val (bool) – Turn on/off the flag. Defaults to True.
index (pd.Index, optional) – Specific Profile elements to set. Default to None (=all).

has_flg(val)

Check if a specific flag name is set.

Parameters:: val (str) – Flag name
Returns:: pd.Series – Series of int, with 1’s where the requested flag name is True.

has_tag(val)

Check if a specific tag name is set for the Profile.

Parameters:: val (str) – Tag name
Returns:: bool – True or False

class dvas.data.strategy.data.RSProfile(info, data=None)

Bases: Profile

Child Profile class for basic radiosonde atmospheric measurements. Requires some measured values, together with their corresponding measurement times since launch, altitudes, and flags.

The data is stored in a pandas DataFrame with column labels: - ‘alt’ (float) - ‘tdt’ (timedelta64[ns]) - ‘val’ (float) - ‘flg’ (int)

The same format is expected as input.

DF_COLS_ATTR = {'alt': {'index': True, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'flg': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.int64'>}, 'tdt': {'index': True, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>, <class 'pandas._libs.tslibs.timedeltas.Timedelta'>, <class 'pandas._libs.tslibs.nattype.NaTType'>), 'type': 'timedelta64[ns]'}, 'val': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}}

property tdt

Corresponding data time delta since launch

Type:: pd.Series

class dvas.data.strategy.data.GDPProfile(info, data=None)

Bases: RSProfile

Child RSProfile class for GDP-like radiosonde atmospheric measurements. Requires some measured values, together with their corresponding measurement times since launch, altitudes, flags, as well as 4 distinct types uncertainties:

‘ucs’ : Spatial-correlated uncertainties.

‘uct’ : Temporal correlated uncertainties.

‘ucu’ : Un-correlated uncertainties.

The property “uc_tot” returns the total uncertainty, and is prodvided for convenience.

The data is stored in a pandas DataFrame with column labels: - ‘alt’ (float) - ‘tdt’ (timedelta64[ns]) - ‘val’ (float) - ‘ucs’ (float) - ‘uct’ (float) - ‘ucu’ (float) - ‘flg’ (int)

The same format is expected as input.

DF_COLS_ATTR = {'alt': {'index': True, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'flg': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.int64'>}, 'tdt': {'index': True, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>, <class 'pandas._libs.tslibs.timedeltas.Timedelta'>, <class 'pandas._libs.tslibs.nattype.NaTType'>), 'type': 'timedelta64[ns]'}, 'ucs': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'uct': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'ucu': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'val': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}}

property ucs

Corresponding data time delta since launch

Type:: pd.Series

property uct

Corresponding data time delta since launch

Type:: pd.Series

property ucu

Corresponding data time delta since launch

Type:: pd.Series

property uc_tot

Computes the total uncertainty from the individual components.

Returns:: pd.Series – uc_tot = np.sqrt(uc_s**2 + uc_t**2 + uc_u**2)

class dvas.data.strategy.data.CWSProfile(info, data=None)

Bases: GDPProfile

Child GDPProfile class intended for CWS profiles.

class dvas.data.strategy.data.DeltaProfile(info, data=None)

Bases: GDPProfile

Child GDPProfile class intended for profile deltas between candidate and CWS profiles.

Unlike GDPs and CWS, this class no longer contains time delta information.

DF_COLS_ATTR = {'alt': {'index': True, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'flg': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.int64'>}, 'ucs': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'uct': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'ucu': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}, 'val': {'index': False, 'test': (<class 'numpy.float64'>, <class 'float'>, <class 'numpy.int64'>, <class 'int'>), 'type': <class 'numpy.float64'>}}

property tdt: DeltaProfile do not store any time delta information.

dvas.data.strategy.load module

Distributed under the terms of the GNU General Public License v3.0 or later.

SPDX-License-Identifier: GPL-3.0-or-later

Module contents: Loader strategies

class dvas.data.strategy.load.LoadStrategyAC

Bases: MPStrategyAC

Abstract load strategy class

static add_flg_prm(db_vs_df_keys)

Add flg parameter to DB vs Dataframe keys

db_vs_df_key (dict): DB vs Dataframe key

static fetch(search, **kwargs)

A base function that fetches data from the database.

Parameters:

search (str) – selection criteria
**kwargs (dict) – Key word parameter to extract.

Returns:

list of InfoManager – Data related info list of pd.DataFrame: Data

Example:

import dvas.data.strategy.load as ld
t = ld.LoadProfileStrategy()
t.fetch("dt('20160715T120000Z', '==')", {'alt':'altpros1', 'val':'trepros1'})

class dvas.data.strategy.load.LoadProfileStrategy

Bases: LoadStrategyAC

Class to manage the data loading strategy of Profile instances.

execute(search, val_abbr, alt_abbr)

Execute strategy method to fetch data from the databse.

Parameters:

search (str) – selection criteria
val_abbr (str) – name of the parameter values to extract
alt_abbr (str, optional) – name of the altitude parameter to extract.
flg_abbr (str, optional) – name of the flag parameter to extract. Defaults to None.

class dvas.data.strategy.load.LoadRSProfileStrategy

Bases: LoadProfileStrategy

Child class to manage the data loading strategy of RSProfile instances.

execute(search, val_abbr, tdt_abbr, alt_abbr=None)

Execute strategy method to fetch data from the database.

Parameters:

search (str) – selection criteria
val_abbr (str) – name of the parameter values to extract.
tdt_abbr (str) – name of the time delta parameter to extract.
alt_abbr (str, optional) – name of the altitude parameter to extract. Defaults to None.

class dvas.data.strategy.load.LoadGDPProfileStrategy

Bases: LoadProfileStrategy

Child class to manage the data loading strategy of GDPProfile instances.

execute(search, val_abbr, tdt_abbr, alt_abbr=None, ucs_abbr=None, uct_abbr=None, ucu_abbr=None)

Execute strategy method to fetch data from the database.

Parameters:

search (str) – selection criteria
val_abbr (str) – name of the parameter values to extract
tdt_abbr (str) – name of the time delta parameter to extract.
alt_abbr (str, optional) – name of the altitude parameter to extract. Dafaults to None.
ucs_abbr (str, optional) – name of the spatial-correlated uncertainty parameter to extract. Defaults to None.
uct_abbr (str, optional) – name of the time-correlated uncertainty parameter to extract. Defaults to None.
ucu_abbr (str, optional) – name of the un-correlated uncertainty parameter to extract. Defaults to None.

class dvas.data.strategy.load.LoadCWSProfileStrategy

Bases: LoadProfileStrategy

Child class to manage the data loading strategy of CWSProfile instances.

execute(search, val_abbr, tdt_abbr, alt_abbr=None, ucs_abbr=None, uct_abbr=None, ucu_abbr=None)

Execute strategy method to fetch data from the database.

Parameters:

search (str) – selection criteria
val_abbr (str) – name of the parameter values to extract
tdt_abbr (str) – name of the time delta parameter to extract.
alt_abbr (str, optional) – name of the altitude parameter to extract. Dafaults to None.
ucs_abbr (str, optional) – name of the spatial-correlated uncertainty parameter to extract. Defaults to None.
uct_abbr (str, optional) – name of the time-correlated uncertainty parameter to extract. Defaults to None.
ucu_abbr (str, optional) – name of the un-correlated uncertainty parameter to extract. Defaults to None.

class dvas.data.strategy.load.LoadDeltaProfileStrategy

Bases: LoadProfileStrategy

Child class to manage the data loading strategy of DeltaProfile instances.

execute(search, val_abbr, alt_abbr=None, ucs_abbr=None, uct_abbr=None, ucu_abbr=None)

Execute strategy method to fetch data from the database.

Parameters:

search (str) – selection criteria
val_abbr (str) – name of the parameter values to extract
alt_abbr (str, optional) – name of the altitude parameter to extract. Dafaults to None.
ucs_abbr (str, optional) – name of the spatial-correlated uncertainty parameter to extract. Defaults to None.
uct_abbr (str, optional) – name of the time-correlated uncertainty parameter to extract. Defaults to None.
ucu_abbr (str, optional) – name of the un-correlated uncertainty parameter to extract. Defaults to None.

dvas.data.strategy.plot module

Distributed under the terms of the GNU General Public License v3.0 or later.

SPDX-License-Identifier: GPL-3.0-or-later

Module contents: Plotting strategies

class dvas.data.strategy.plot.PlotStrategy

Bases: MPStrategyAC

Base class to manage the data plotting strategy for the MultiProfile class.

execute(prfs, **kwargs)

Call the proper plotting method for this strategy.

Parameters:

prfs (dict of Profile or RSProfile or GDPprofile) – data to plot
**kwargs – Keyword arguments to be passed down to the plotting function.

class dvas.data.strategy.plot.RSPlotStrategy

Bases: MPStrategyAC

Child class to manage the plotting strategy for the MultiRSProfile class.

execute(prfs, **kwargs)

Call the proper plotting method for this strategy.

Parameters:

prfs (dict of Profile or RSProfile or GDPprofile) – data to plot
**kwargs – Keyword arguments to be passed down to the plotting function.

class dvas.data.strategy.plot.GDPPlotStrategy

Bases: MPStrategyAC

Child class to manage the plotting strategy for the MultiGDPProfile class.

execute(prfs, **kwargs)

Call the proper plotting method for this strategy.

Parameters:

prfs (dict of Profile or RSProfile or GDPprofile) – data to plot
**kwargs – Keyword arguments to be passed down to the plotting function.

dvas.data.strategy.rebase module

Distributed under the terms of the GNU General Public License v3.0 or later.

SPDX-License-Identifier: GPL-3.0-or-later

Module contents: Rebase strategy

class dvas.data.strategy.rebase.RebaseStrategy

Bases: MPStrategyAC

Class to manage time data synchronization

execute(prfs, new_lengths, shifts=None)

Rebases Profiles on a DataFrame with a different length, possibly shifting values around.

Parameters:

prfs (list of dvas.data.strategy.data.Profile|RSProfile|GDPProfile) – Profiles to rebase.
new_lengths (int|list of int) – The new lengths for the Profiles. Will crop/add empty elements at the end of the profiles as needed.
shifts (int|list of int, optional) – row n of the existing data will become row n+shift. If specifiying an int, the same shift will be applied to all Profiles. Else, the list should specify a shift for each Profile. Defaults to None.

Returns:

dvas.data.MultiProfile|MultiRSProfile|MultiGDPProfile – the rebased MultiProfile.

Any missing data gets filled with NaN/NaT. Any superfulous data is be cropped. All non-integer indices get rebased as well (i.e. they are NOT interpolated). Any missing data gets flagged with FLG_NOPRF to indicate to it is not associated with any real profile measurement. A shift of -n crops the first n points, whereas a shift of +n crops the last n points (unless new_lengths is adjusted).

dvas.data.strategy.resample module

Distributed under the terms of the GNU General Public License v3.0 or later.

SPDX-License-Identifier: GPL-3.0-or-later

Module contents: Resample strategies

class dvas.data.strategy.resample.ResampleStrategy

Bases: MPStrategyAC

Class to handle the resample strategy for RS and GDPs Profiles.

execute(prfs, freq: str = '1s', chunk_size: int = 150, n_cpus: int = 1, interp_dist: float | int = 1, circular: bool = False)

Implementation of time resampling method for RS and GDP Profiles.

Parameters:

prfs (list of RSProfiles|GDPProfiles) – GDP Profiles to resample.
freq (str, optional) – see pandas.timedelta_range(). Defaults to ‘1s’.
interp_dist (float|int, optional) – define the distance between resampled points and their closest measurement, in s, from which the resampled element is forced to NaN. Defaults to 1, i.e. gaps that are >= 1s are not interpolated.
chunk_size (int, optional) – to speed up computation, Profiles get broken up in chunks of that length. The larger the chunks, the larger the memory requirements. The smaller the chunks the more items to process. Defaults to 150.
n_cpus (int|str, optional) – number of cpus to use. Can be a number, or ‘max’. Set to 1 to disable multiprocessing. Defaults to 1.
circular (bool, optional) – if True, will assume angular values and use np.unwrap() before interpolating. Defaults to False.

Returns:

dvas.data.MultiRSProfile|MultiGDPProfile – the resampled MultiProfile.

Note

This strategy does NOT treat NaN’s in a special way. This implies that if a NaN is one of the two closest original data points from a new location to be interpolated, that location will result in a NaN as well.

dvas.data.strategy.save module

Distributed under the terms of the GNU General Public License v3.0 or later.

SPDX-License-Identifier: GPL-3.0-or-later

Module contents: Strategy used to save data

class dvas.data.strategy.save.SaveDataStrategy

Bases: MPStrategyAC

Class to manage saving strategy of time data

execute(data, prms)

Implementation of save method.

Parameters:

data (MultiProfile) – the data to save into the database.
prms (list of str) – list of column names to save to the database.

dvas.data.strategy.sort module

Distributed under the terms of the GNU General Public License v3.0 or later.

SPDX-License-Identifier: GPL-3.0-or-later

Module contents: Sort strategy

class dvas.data.strategy.sort.SortProfileStrategy

Bases: MPStrategyAC

Class to manage sort of time data parameter

execute(data)

Implementation of sort method

Parameters:: data (list) – list of Profile
Returns:: list of Profile