Typical Processing Library Functionalities

The following is a general list of the type of processing operations that should be available in the PRISM Data Processing library. This list has been compiled from those operations already in the UKMO PV-WAVE library and the MPI PINGO (Procedural INterface for Grib formatted Objects) package.

Please indicate for each "General Functionality" whether you deem it Essential , Important, or Optional.

Priority
E, I or O General Functionality Examples of operations

Essential Information: Print size, num. of records etc, grid description, show level, status, version

Essential Averaging: basin / regional / area / vertical / zonal / meridional average, daily, monthly, seasonal, yearly mean/average, reduction of spatial dimensions (Collapse - E.g. cross-section field, Hovmuller field)
Note: vertical / zonal / meridional is meant in geographical coordinates, collapsing is in grid directions

Essential Arithmetic Operations: difference, add, subtract, multiply, divide, minus, inverse, modulus, power, restrict, sum

Essential Gather: Removing missing data (E.g. Removing land points from the ocean)

Essential Scatter: Opposite of Gather; i.e Adding in missing data

Essential Extraction of regions and slices: Restricting one or more dimensions to a range or a single value e.g. a region of a lat-long field, a vertical section through a lat-long-height field.

Essential Masking: Produce a mask field (e.g. missing/non-missing or binary 0/1) by applying a logical condition to some other field.

Essential Conversion to new grid: ... by interpolation and averaging.
For example changing the lat-long grid to a different resolution, transforming to a new kind of horizontal grid; e.g. lat-long to polar stereographic, changing to a new kind of vertical coordinate; e.g. atmos sigma levs to pressure levs or ocean depth to density.

Important Vector Operations: curl, dot-product, cross-product, magnitude, calculate laplacian, vector gradient, vector divergence, rotating vector field

Essential Sampling: constructing time-series or vertical profiles for scattered locations.

Essential Manipulation: reverse, transpose, split, concatenate, merge, replace

Optional Classes:

Classes - for counting and for averaging data which lie in user given quantity classes.

Durations/frequencyof events - for analysing events which are defined by a value lying in a user given range.

Optional Binning Data: E.g. Histogram

Essential Statistical Operations: variance, covariance, correlation, standard deviation, maximum, minimum, daily max/min, monthly max/min, yearly max/min

Essential Filtering and Smoothing should be considered as arithmetic operations

Important Compute Empirical Orthogonal Functions: with and without using weight file, in spatial space, in time space

Essential Integration and Differentiation: time-derivative and time-integral, area-integral, volume integral, vector gradient, vector-divergence

Essential Mathematical Functions: integer, sign, exponential, logarithm, square, trigonometric functions, polar, imaginary part, complex part

Important Regression: regression, detrend, anomaly coupling

Optional Tests and Confidence Intervals: normal, student-t, fisher, mean, variance, pooled variance, regression, correlation

Additional functionalities needed:

We would prefer an access to those functionnalities
through a common interface rather than having multiple packages.
Handling of metadata associated with processing is more than essential
and should be paid attention to.

Last Modified: 09 July 2002
Patrick Brockmann for IPSL

Priority E, I or O	General Functionality	Examples of operations
Essential	Information:	Print size, num. of records etc, grid description, show level, status, version
Essential	Averaging:	basin / regional / area / vertical / zonal / meridional average, daily, monthly, seasonal, yearly mean/average, reduction of spatial dimensions (Collapse - E.g. cross-section field, Hovmuller field) Note: vertical / zonal / meridional is meant in geographical coordinates, collapsing is in grid directions
Essential	Arithmetic Operations:	difference, add, subtract, multiply, divide, minus, inverse, modulus, power, restrict, sum
Essential	Gather:	Removing missing data (E.g. Removing land points from the ocean)
Essential	Scatter:	Opposite of Gather; i.e Adding in missing data
Essential	Extraction of regions and slices:	Restricting one or more dimensions to a range or a single value e.g. a region of a lat-long field, a vertical section through a lat-long-height field.
Essential	Masking:	Produce a mask field (e.g. missing/non-missing or binary 0/1) by applying a logical condition to some other field.
Essential	Conversion to new grid:	... by interpolation and averaging. For example changing the lat-long grid to a different resolution, transforming to a new kind of horizontal grid; e.g. lat-long to polar stereographic, changing to a new kind of vertical coordinate; e.g. atmos sigma levs to pressure levs or ocean depth to density.
Important	Vector Operations:	curl, dot-product, cross-product, magnitude, calculate laplacian, vector gradient, vector divergence, rotating vector field
Essential	Sampling:	constructing time-series or vertical profiles for scattered locations.
Essential	Manipulation:	reverse, transpose, split, concatenate, merge, replace
Optional	Classes:	Classes - for counting and for averaging data which lie in user given quantity classes. Durations/frequencyof events - for analysing events which are defined by a value lying in a user given range.
Optional	Binning Data:	E.g. Histogram
Essential	Statistical Operations:	variance, covariance, correlation, standard deviation, maximum, minimum, daily max/min, monthly max/min, yearly max/min
Essential	Filtering and Smoothing	should be considered as arithmetic operations
Important	Compute Empirical Orthogonal Functions:	with and without using weight file, in spatial space, in time space
Essential	Integration and Differentiation:	time-derivative and time-integral, area-integral, volume integral, vector gradient, vector-divergence
Essential	Mathematical Functions:	integer, sign, exponential, logarithm, square, trigonometric functions, polar, imaginary part, complex part
Important	Regression:	regression, detrend, anomaly coupling
Optional	Tests and Confidence Intervals:	normal, student-t, fisher, mean, variance, pooled variance, regression, correlation