Deedle
Series Module
The `Series` module provides an F#-friendly API for working with data and time series.
The API follows the usual design for collection-processing in F#, so the functions work
well with the pipelining (|>) operator. For example, given a series with ages,
we can use `Series.filterValues` to filter outliers and then `Stats.mean` to calculate
the mean:
ages
|> Series.filterValues (fun v -> v > 0.0 && v < 120.0)
|> Stats.mean
The module provides comprehensive set of functions for working with series. The same
API is also exposed using C#-friendly extension methods. In C#, the above snippet could
be written as:
[lang=csharp]
ages
.Where(kvp => kvp.Value > 0.0 && kvp.Value < 120.0)
.Mean()
For more information about similar frame-manipulation functions, see the `Frame` module.
For more information about C#-friendly extensions, see `SeriesExtensions`. The functions
in the `Series` module are grouped in a number of categories and documented below.
Accessing series data and lookup
--------------------------------
Functions in this category provide access to the values in the series.
- The term _observation_ is used for a key value pair in the series.
- When working with a sorted series, it is possible to perform lookup using
keys that are not present in the series - you can specify to search for the
previous or next available value using _lookup behavior_.
- Functions such as `get` and `getAll` have their counterparts `lookup` and
`lookupAll` that let you specify lookup behavior.
- For most of the functions that may fail, there is a `try[Foo]` variant that
returns `None` instead of failing.
- Functions with a name ending with `At` perform lookup based on the absolute
integer offset (and ignore the keys of the series)
Series transformations
----------------------
Functions in this category perform standard transformations on series including
projections, filtering, taking some sub-series of the series, aggregating values
using scanning and so on.
Projection and filtering functions generally skip over missing values, but there
are variants `filterAll` and `mapAll` that let you handle missing values explicitly.
Keys can be transformed using `mapKeys`. When you do not need to consider the keys,
and only care about values, use `filterValues` and `mapValues` (which is also aliased
as the `$` operator).
Series supports standard set of folding functions including `reduce` and `fold` (to
reduce series values into a single value) as well as the `scan[All]` function, which
can be used to fold values of a series into a series of intermeidate folding results.
The functions `take[Last]` and `skip[Last]` can be used to take a sub-series of the
original source series by skipping a specified number of elements. Note that this
does not require an ordered series and it ignores the index - for index-based lookup
use slicing, such as `series.[lo .. hi]`, instead.
Finally the `shift` function can be used to obtain a series with values shifted by
the specified offset. This can be used e.g. to get previous value for each key using
`Series.shift 1 ts`. The `diff` function calculates difference from previous value using
`ts - (Series.shift offs ts)`.
Processing series with exceptions
---------------------------------
The functions in this group can be used to write computations over series that may fail.
They use the type tryval<'T> which is defined as a discriminated union
with two cases: Success containing a value, or Error containing an exception.
The function `tryMap` lets you create Series<'K, tryval<'T>> by mapping over values
of an original series. You can then extract values using `tryValues`, which throws
`AggregateException` if there were any errors. Functions `tryErrors` and `trySuccesses`
give series containing only errors and successes. You can fill failed values with
a constant using `fillErrorsWith`.
Hierarchical index operations
-----------------------------
When the key of a series is tuple, the elements of the tuple can be treated
as multiple levels of a index. For example Series<'K1 * 'K2, 'V> has two
levels with keys of types 'K1 and 'K2 respectively.
The functions in this cateogry provide a way for aggregating values in the
series at one of the levels. For example, given a series `input` indexed by
two-element tuple, you can calculate mean for different first-level values as
follows:
input |> applyLevel fst Stats.mean
Note that the `Stats` module provides helpers for typical statistical operations,
so the above could be written just as `input |> Stats.levelMean fst`.
Grouping, windowing and chunking
--------------------------------
This category includes functions that group data from a series in some way. Two key
concepts here are _window_ and _chunk_. Window refers to (overlapping) sliding windows
over the input series while chunk refers to non-overlapping blocks of the series.
The boundary behavior can be specified using the `Boundary` flags. The value
`Skip` means that boundaries (incomplete windows or chunks) should be skipped. The value
`AtBeginning` and `AtEnding` can be used to define at which side should the boundary be
returned (or skipped). For chunking, `AtBeginning ||| Skip` makes sense and it means that
the incomplete chunk at the beginning should be skipped (aligning the last chunk with the end).
The behavior may be specified in a number of ways (which is reflected in the name):
- `dist` - using an absolute distance between the keys
- `while` - using a condition on the first and last key
- `size` - by specifying the absolute size of the window/chunk
The functions ending with `Into` take a function to be applied to the window/chunk.
The functions `window`, `windowInto` and `chunk`, `chunkInto` are simplified versions
that take a size. There is also `pairwise` function for sliding window of size two.
Missing values
--------------
This group of functions provides a way of working with missing values in a series.
The `dropMissing` function drops all keys for which there are no values in the series.
The `withMissingFrom` function lets you copy missing values from another series.
The remaining functions provide different mechanism for filling the missing values.
* `fillMissingWith` fills missing values with a specified constant
* `fillMissingUsing` calls a specified function for every missing value
* `fillMissing` and variants propagates values from previous/later keys
Sorting and index manipulation
------------------------------
A series that is sorted by keys allows a number of additional operations (such as lookup
using the `Lookp.ExactOrSmaller` lookup behavior). However, it is also possible to sort
series based on the values - although the functions for manipulation with series do not
guarantee that the order will be preserved.
To sort series by keys, use `sortByKey`. Other sorting functions let you sort the series
using a specified comparer function (`sortWith`), using a projection function (`sortBy`)
and using the default comparison (`sort`).
In addition, you can also replace the keys of a series with other keys using `indexWith`
or with integers using `indexOrdinally`. To pick and reorder series values using to match
a list of keys use `realign`.
Sampling, resampling and advanced lookup
----------------------------------------
Given a (typically) time series sampling or resampling makes it possible to
get time series with representative values at lower or uniform frequency.
We use the following terminology:
- `lookup` and `sample` functions find values at specified key; if a key is not
available, they can look for value associated with the nearest smaller or
the nearest greater key.
- `resample` function aggregate values values into chunks based
on a specified collection of keys (e.g. explicitly provided times), or
based on some relation between keys (e.g. date times having the same date).
- `resampleUniform` is similar to resampling, but we specify keys by
providing functions that generate a uniform sequence of keys (e.g. days),
the operation also fills value for days that have no corresponding
observations in the input sequence.
Joining, merging and zipping
----------------------------
Given two series, there are two ways to combine the values. If the keys in the series
are not overlapping (or you want to throw away values from one or the other series),
then you can use `merge` or `mergeUsing`. To merge more than 2 series efficiently, use
the `mergeAll` function, which has been optimized for large number of series.
If you want to align two series, you can use the _zipping_ operation. This aligns
two series based on their keys and gives you tuples of values. The default behavior
(`zip`) uses outer join and exact matching. For ordered series, you can specify
other forms of key lookups (e.g. find the greatest smaller key) using `zipAlign`.
functions ending with `Into` are generally easier to use as they call a specified
function to turn the tuple (of possibly missing values) into a new value.
For more complicated behaviors, it is often convenient to use joins on frames instead
of working with series. Create two frames with single columns and then use the join
operation. The result will be a frame with two columns (which is easier to use than
series of tuples).
Table of contents
- Other module members
- Accessing series data and lookup
- Creating series
- Grouping, windowing and chunking
- Hierarchical index operations
- Joining, merging and zipping
- Missing values
- Processing series with exceptions
- Sampling, resampling and advanced lookup
- Series transformations
- Sorting and index manipulation
Other module members
Functions and values
| Function or value |
Description
|
|
Retruns a new series whose values are converted using the specified conversion function. This operation is like `mapValues`, but it requires a pair of function that converts the values in _both ways_. This operation is only interesting when working with virtualized data sources. Using the `convert` function makes it possible to perfom additional operations on the resulting series - for example lookup - by converting the new value back and using the lookup of the underlying virtualized source.
|
Accessing series data and lookup
Functions and values
| Function or value |
Description
|
|
|
|
|
|
Returns a new series containing all values with keys in the range
[
|
|
Returns the total number of keys in the specified series. This returns the total length of the series, including keys for which there is no value available.
|
|
Returns the total number of values in the specified series. This excludes missing values or not available values (such as values created from `null`, `Double.NaN`, or those that are missing due to outer join etc.).
|
|
|
|
Returns the first key of the series, or throws exception if one doesn't exist
|
|
Returns the first value of the series. This fails if the first value is missing.
|
|
Get the value for the specified key. Uses exact lookup semantics for key lookup - use `lookup` for more options
|
|
|
|
Returns the value at the specified (integer) offset.
|
|
Returns true when the series contains value for the specified key (This is useful for checking prior to performing a computation)
|
|
Returns true when the series contains value for all of the specified keys (This is useful for checking prior to performing a computation)
|
|
Returns true when the series does not contains value for any of the specified keys (This is useful for checking prior to performing a computation)
|
|
Returns true when the series does not contains value for the specified key (This is useful for checking prior to performing a computation)
|
|
Returns true when the series contains value for some of the specified keys (This is useful for checking prior to performing a computation)
|
|
|
|
Returns the last key of the series, or throws exception if one doesn't exist
|
|
Returns the last value of the series. This fails if the last value is missing.
|
|
|
|
Create a new series that contains values for all provided keys. Use the specified lookup semantics - for exact matching, use `getAll`
|
|
Return observations with available values. The operation skips over all keys with missing values (such as values created from `null`, `Double.NaN`, or those that are missing due to outer join etc.).
|
Full Usage:
observationsAll series
Parameters:
Series<'K, 'T>
Returns: ('K * 'T option) seq
|
Returns all keys from the sequence, together with the associated (optional) values.
|
|
|
|
|
|
|
|
Get the value for the specified key. Returns `None` when the key does not exist or the value is missing. Uses exact lookup semantics for key lookup - use `tryLookup` for more options
|
Full Usage:
tryGetAt index series
Parameters:
int
series : Series<'K, 'T>
Returns: 'T option
|
Returns the value at the specified (integer) offset, or `None` if the value is missing.
|
|
|
|
|
|
Attempts to get an observation (key value pair) based on the specified key. The search uses the specified lookup semantics and so the returned key may differ from the key searched for. If the value is not available, `None` is returned.
|
|
Returns the (non-missing) values of the series as a sequence
|
|
Returns the series values (both missing and present) as a sequence
|
Creating series
Functions and values
| Function or value |
Description
|
Returns an empty series with no keys or values.
Example
let s : Series |
Grouping, windowing and chunking
Functions and values
| Function or value |
Description
|
Full Usage:
aggregate aggregation keySelector series
Parameters:
Aggregation<'K>
-
Specifies the aggregation method using Aggregation<K>. This is a discriminated union listing various chunking and windowing conditions.
keySelector : DataSegment<Series<'K, 'T>> -> 'TNewKey
-
A function that is called on each chunk to obtain a key.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'TNewKey, DataSegment<Series<'K, 'T>>>
|
Aggregates an ordered series using the method specified by
|
Full Usage:
aggregateInto aggregation keySelector f series
Parameters:
Aggregation<'K>
-
Specifies the aggregation method using Aggregation<K>. This is a discriminated union listing various chunking and windowing conditions.
keySelector : DataSegment<Series<'K, 'T>> -> 'TNewKey
-
A function that is called on each chunk to obtain a key.
f : DataSegment<Series<'K, 'T>> -> OptionalValue<'R>
-
A value selector function that is called to aggregate each chunk or window.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'TNewKey, 'R>
|
Aggregates an ordered series using the method specified by
|
|
Aggregates the input into a series of adacent chunks and returns the produced chunks as a nested series. The key in the new series is the last key of the chunk. This function skips incomplete chunks - you can use `Series.chunkSize` for more options.
|
Full Usage:
chunkDist distance series
Parameters:
^D
-
The maximal allowed distance between keys of a chunk. Note that this is an inline function - there must be `-` operator defined between `distance` and the keys of the series.
series : Series<^K, 'T>
-
The input series to be aggregated.
Returns: Series<^K, Series<^K, 'T>>
Modifiers: inline Type parameters: ^D, ^K, 'T |
Aggregates the input into a series of adacent chunks. A chunk is started once the distance between the first and the last key of a previous chunk is greater than the specified `distance`. The chunks are then returned as a nested series. The key of each chunk is the key of the first element in the chunk.
|
Full Usage:
chunkDistInto distance f series
Parameters:
^D
-
The maximal allowed distance between keys of a chunk. Note that this is an inline function - there must be `-` operator defined between `distance` and the keys of the series.
f : Series<^K, 'T> -> 'R
-
A value selector that is called to aggregate each chunk.
series : Series<^K, 'T>
-
The input series to be aggregated.
Returns: Series<^K, 'R>
Modifiers: inline Type parameters: ^D, ^K, 'T, 'R |
Aggregates the input into a series of adacent chunks. A chunk is started once the distance between the first and the last key of a previous chunk is greater than the specified `distance`. Each chunk is then aggregated into a value using the specified function `f`. The key of each chunk is the key of the first element in the chunk.
|
Full Usage:
chunkInto size f series
Parameters:
int
-
The size of the chunk.
f : Series<'K, 'T> -> 'R
-
A function that is called to aggregate each chunk into a single value.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'K, 'R>
Modifiers: inline Type parameters: 'K, 'T, 'R |
Aggregates the input into a series of adacent chunks and then applies the provided value selector `f` on each chunk to produce the result which is returned as a new series. The key in the new series is the last key of the chunk. This function skips incomplete chunks - you can use `Series.chunkSizeInto` for more options.
|
|
Aggregates the input into a series of adacent chunks using the specified size and boundary behavior and returns the produced chunks as a nested series. The key is the first key of the chunk, unless boundary behavior has `Boundary.AtBeginning` flag (in which case it is the last key). |
Full Usage:
chunkSizeInto (arg1, arg1) f series
Parameters:
int
arg1 : Boundary
f : DataSegment<Series<'K, 'T>> -> 'R
-
A value selector that is called to aggregate each chunk.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'K, 'R>
Modifiers: inline Type parameters: 'K, 'T, 'R |
Aggregates the input into a series of adacent chunks using the specified size and boundary behavior and then applies the provided value selector `f` on each chunk to produce the result which is returned as a new series. The key is the first key of the chunk, unless boundary behavior has `Boundary.AtBeginning` flag (in which case it is the last key).
|
Full Usage:
chunkWhile cond series
Parameters:
'K -> 'K -> bool
-
A function that is called on the first and the last key of a chunk to determine when a window should end.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'K, Series<'K, 'T>>
Modifiers: inline Type parameters: 'K, 'T |
Aggregates the input into a series of adacent chunks based on a condition on keys. A chunk is started once the specified `cond` function returns `false` when called on the first and the last key of the previous chunk. The chunks are then returned as a nested series. The key of each chunk is the key of the first element in the chunk.
|
Full Usage:
chunkWhileInto cond f series
Parameters:
'K -> 'K -> bool
-
A function that is called on the first and the last key of a chunk to determine when a window should end.
f : Series<'K, 'T> -> 'a
-
A value selector that is called to aggregate each chunk.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'K, 'a>
Modifiers: inline Type parameters: 'K, 'T, 'a |
Aggregates the input into a series of adacent chunks based on a condition on keys. A chunk is started once the specified `cond` function returns `false` when called on the first and the last key of the previous chunk. Each chunk is then aggregated into a value using the specified function `f`. The key of each chunk is the key of the first element in the chunk.
|
Full Usage:
groupBy keySelector series
Parameters:
'K -> 'T -> 'TNewKey
-
Generates a new key that is used for aggregation, based on the original key and value. The new key must support equality testing.
series : Series<'K, 'T>
-
An input series to be grouped.
Returns: Series<'TNewKey, Series<'K, 'T>>
|
Groups a series (ordered or unordered) using the specified key selector (`keySelector`) and then returns a series of (nested) series as the result. The outer series is indexed by the newly produced keys, the nested series are indexed with the original keys.
|
Full Usage:
groupInto keySelector f series
Parameters:
'K -> 'T -> 'TNewKey
-
Generates a new key that is used for aggregation, based on the original key and value. The new key must support equality testing.
f : 'TNewKey -> Series<'K, 'T> -> 'TNewValue
-
A function to aggregate each group of collected elements.
series : Series<'K, 'T>
-
An input series to be grouped.
Returns: Series<'TNewKey, 'TNewValue>
|
Groups a series (ordered or unordered) using the specified key selector (`keySelector`) and then aggregates each group into a single value, returned in the resulting series, using the provided `f` function.
|
|
Returns a series containing the predecessor and an element for each input, except for the first one. The returned series is one key shorter (it does not contain a value for the first key).
Example
val input: obj
val res: obj
|
|
Aggregates the input into pairs containing the predecessor and an element for each input, except for the first one. Then calls the specified aggregation function `f` with a tuple and a key. The returned series is one key shorter (it does not contain a value for the first key).
|
|
Creates a sliding window using the specified size and returns the produced windows as a nested series. The key in the new series is the last key of the window. This function skips incomplete chunks - you can use `Series.windowSize` for more options.
|
Full Usage:
windowDist distance series
Parameters:
'D
-
The maximal allowed distance between keys of a window. Note that this is an inline function - there must be `-` operator defined between `distance` and the keys of the series.
series : Series<^K, 'T>
-
The input series to be aggregated.
Returns: Series<^K, Series<^K, 'T>>
Modifiers: inline Type parameters: 'D, ^K, 'T |
Creates a sliding window based on distance between keys. A window is started at each input element and ends once the distance between the first and the last key is greater than the specified `distance`. The windows are then returned as a nested series. The key of each window is the key of the first element in the window.
|
Full Usage:
windowDistInto distance f series
Parameters:
'a
-
The maximal allowed distance between keys of a window. Note that this is an inline function - there must be `-` operator defined between `distance` and the keys of the series.
f : Series<^K, 'T> -> 'b
-
A function that is used to aggregate each window into a single value.
series : Series<^K, 'T>
-
The input series to be aggregated.
Returns: Series<^K, 'b>
Modifiers: inline Type parameters: 'a, ^K, 'T, 'b |
Creates a sliding window based on distance between keys. A window is started at each input element and ends once the distance between the first and the last key is greater than the specified `distance`. Each window is then aggregated into a value using the specified function `f`. The key of each window is the key of the first element in the window.
|
Full Usage:
windowInto size f series
Parameters:
int
-
The size of the sliding window.
f : Series<'K, 'T> -> 'R
-
A function that is called on each created window.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'K, 'R>
Modifiers: inline Type parameters: 'K, 'T, 'R |
Creates a sliding window using the specified size and then applies the provided value selector `f` on each window to produce the result which is returned as a new series. This function skips incomplete chunks - you can use `Series.windowSizeInto` for more options.
|
|
Creates a sliding window using the specified size and boundary behavior and returns the produced windows as a nested series. The key is the last key of the window, unless boundary behavior is `Boundary.AtEnding` (in which case it is the first key). |
Full Usage:
windowSizeInto (arg1, arg1) f series
Parameters:
int
arg1 : Boundary
f : DataSegment<Series<'K, 'T>> -> 'R
-
A value selector that is called to aggregate each window.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'K, 'R>
|
Creates a sliding window using the specified size and boundary behavior and then applies the provided value selector `f` on each window to produce the result which is returned as a new series. The key is the last key of the window, unless boundary behavior is `Boundary.AtEnding` (in which case it is the first key).
|
Full Usage:
windowWhile cond series
Parameters:
'K -> 'K -> bool
-
A function that is called on the first and the last key of a window to determine when a window should end.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'K, Series<'K, 'T>>
Modifiers: inline Type parameters: 'K, 'T |
Creates a sliding window based on a condition on keys. A window is started at each input element and ends once the specified `cond` function returns `false` when called on the first and the last key of the window. The windows are then returned as a nested series. The key of each window is the key of the first element in the window.
|
Full Usage:
windowWhileInto cond f series
Parameters:
'K -> 'K -> bool
-
A function that is called on the first and the last key of a window to determine when a window should end.
f : Series<'K, 'T> -> 'a
-
A function that is used to aggregate each window into a single value.
series : Series<'K, 'T>
-
The input series to be aggregated.
Returns: Series<'K, 'a>
Modifiers: inline Type parameters: 'K, 'T, 'a |
Creates a sliding window based on a condition on keys. A window is started at each input element and ends once the specified `cond` function returns `false` when called on the first and the last key of the window. Each window is then aggregated into a value using the specified function `f`. The key of each window is the key of the first element in the window.
|
Hierarchical index operations
Functions and values
| Function or value |
Description
|
Full Usage:
applyLevel level op series
Parameters:
'K1 -> 'K2
-
A delegate that returns a new group key, based on the key in the input series
op : Series<'K1, 'V> -> 'R
-
A function that takes a series and produces an aggregated result
series : Series<'K1, 'V>
-
An input series to be aggregated
Returns: Series<'K2, 'R>
Modifiers: inline Type parameters: 'K1, 'K2, 'V, 'R |
Groups the elements of the input series in groups based on the keys produced by `level` and then aggregates series representing each group using the specified function `op`. The result is a new series containing the aggregates of each group. This operation is designed to be used with [hierarchical indexing](../frame.html#indexing).
|
Full Usage:
applyLevelOptional level op series
Parameters:
'K1 -> 'K2
-
A delegate that returns a new group key, based on the key in the input series
op : Series<'K1, 'V> -> 'R option
-
A function that takes a series and produces an optional aggregated result
series : Series<'K1, 'V>
-
An input series to be aggregated
Returns: Series<'K2, 'R>
Modifiers: inline Type parameters: 'K1, 'K2, 'V, 'R |
Groups the elements of the input series in groups based on the keys produced by `level` and then aggregates series representing each group using the specified function `op`. The result is a new series containing the aggregates of each group. The result of a group may be None, in which case the group will have no representation in the resulting series. This operation is designed to be used with [hierarchical indexing](../frame.html#indexing).
|
Full Usage:
reduceLevel level op series
Parameters:
'K1 -> 'K2
-
A delegate that returns a new group key, based on the key in the input series
op : 'T -> 'T -> 'T
-
A function that is used to aggregate elements of each group
series : Series<'K1, 'T>
-
An input series to be aggregated
Returns: Series<'K2, 'T>
|
Groups the elements of the input series in groups based on the keys produced by `level` and then aggregates elements in each group using the specified function `op`. The result is a new series containing the aggregates of each group. This operation is designed to be used with [hierarchical indexing](../frame.html#indexing).
|
Joining, merging and zipping
Functions and values
| Function or value |
Description
|
|
|
|
|
|
|
|
|
Full Usage:
mergeUsing behavior series1 series2
Parameters:
UnionBehavior
-
specifies how to handle values available in both series. You can use `UnionBehavior.Exclusive` to throw an exception, or `UnionBehavior.PreferLeft` and `UnionBehavior.PreferRight` to prefer values from the first or the second series, respectively.
series1 : Series<'K, 'V>
-
the first (left) series to be merged
series2 : Series<'K, 'V>
-
the second (right) series to be merged
Returns: Series<'K, 'V>
|
Merge two series with possibly overlapping keys. The `behavior` parameter specifies how to handle situation when a value is definedin both series.
|
|
|
|
|
Full Usage:
zipAlign kind lookup series1 series2
Parameters:
JoinKind
-
specifies the kind of join you want to use (left, right, inner or outer). For inner join, it is better to use `zipInner` instead.
lookup : Lookup
-
specifies how matching keys are found when left or right join is used on a sorted series. Use this to find the nearest smaller or nearest greater key in the other series. Supported values are `Lookup.Exact`, `Lookup.ExactOrSmaller` and `Lookup.ExactOrGreater`.
series1 : Series<'K, 'V1>
-
The first (left) series to be aligned
series2 : Series<'K, 'V2>
-
The second (right) series to be aligned
Returns: Series<'K, ('V1 opt * 'V2 opt)>
|
Align and zip two series using the specified joining mechanism and key matching. The function returns a series of tuples where both elements may be missing. As a result, it is often easier to use join on frames instead.
|
Full Usage:
zipAlignInto kind lookup op series1 series2
Parameters:
JoinKind
-
specifies the kind of join you want to use (left, right, inner or outer). For inner join, it is better to use `zipInner` instead.
lookup : Lookup
-
specifies how matching keys are found when left or right join is used on a sorted series. Use this to find the nearest smaller or nearest greater key in the other series. Supported values are `Lookup.Exact`, `Lookup.ExactOrSmaller` and `Lookup.ExactOrGreater`.
op : 'V1 option -> 'V2 option -> 'R option
-
A function that combines values from the two series. In case of left, right or outer join, some of the values may be missing. The function can also return `None` to indicate a missing result.
series1 : Series<'K, 'V1>
-
The first (left) series to be aligned
series2 : Series<'K, 'V2>
-
The second (right) series to be aligned
Returns: Series<'K, 'R>
|
Align and zip two series using the specified joining mechanism and key matching. The function calls the specified function `op` to combine values from the two series
|
|
|
|
Align and zip two series using inner join and exact key matching (use `zipAlignInto` for more options). The function calls the specified function `op` to combine values from the two series
|
Missing values
Functions and values
| Function or value |
Description
|
|
Drop missing values from the specified series. The returned series contains only those keys for which there is a value available in the original one.
Example
val s: obj
|
Full Usage:
fillMissing direction series
Parameters:
Direction
-
Specifies the direction used when searching for the nearest available value. `Backward` means that we want to look for the first value with a smaller key while `Forward` searches for the nearest greater key.
series : Series<'K, 'T>
-
An input series that is to be filled
Returns: Series<'K, 'T>
|
Fill missing values in the series with the nearest available value (using the specified direction). Note that the series may still contain missing values after call to this function. This operation can only be used on ordered series.
Example
val sample: obj
|
Full Usage:
fillMissingBetween (startKey, endKey) direction series
Parameters:
'K
-
the lower bound at which values should be filled
endKey : 'K
-
the upper bound at which values should be filled
direction : Direction
-
Specifies the direction used when searching for the nearest available value. `Backward` means that we want to look for the first value with a smaller key while `Forward` searches for the nearest greater key.
series : Series<'K, 'T>
-
An input series that is to be filled
Returns: Series<'K, 'T>
|
Fill missing values only between `startKey` and `endKey`, inclusive.
|
|
|
|
Fill missing values in the series using the specified function. The specified function is called with all keys for which the series does not contain value and the result of the call is used in place of the missing value. This function can be used to implement more complex interpolation. For example see [handling missing values in the tutorial](../frame.html#missing)
|
|
|
|
Processing series with exceptions
Functions and values
| Function or value |
Description
|
|
|
|
|
|
|
|
|
|
Sampling, resampling and advanced lookup
Functions and values
| Function or value |
Description
|
Full Usage:
lookupTime interval dir lookup series
Parameters:
TimeSpan
-
The interval between the individual samples
dir : Direction
-
Specifies how the keys should be generated. `Direction.Forward` means that the key is the smallest value of each chunk (and so first key of the series is returned and the last is not, unless it matches exactly _start + k*interval_); `Direction.Backward` means that the first key is skipped and sample is generated at, or just before the end of interval and at the end of the series.
lookup : Lookup
-
Specifies how the lookup based on keys is performed. `Exact` means that the values at exact keys will be returned; `NearestGreater` returns the nearest greater key value (starting at the first key) and `NearestSmaller` returns the nearest smaller key value (starting at most `interval` after the end of the series)
series : Series<^K, ^V>
-
An input series to be resampled
Returns: Series<^K, ^V>
Modifiers: inline Type parameters: ^K, ^V |
Finds values at, or near, the specified times in a given series. The operation generates keys starting from the smallest key of the original series, using the specified `interval` and then finds values close to such keys using the specified `lookup` and `dir`. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
lookupTimeAt start interval dir lookup series
Parameters:
^K
-
The initial time to be used for sampling
interval : TimeSpan
-
The interval between the individual samples
dir : Direction
-
Specifies how the keys should be generated. `Direction.Forward` means that the key is the smallest value of each chunk (and so first key of the series is returned and the last is not, unless it matches exactly _start + k*interval_); `Direction.Backward` means that the first key is skipped and sample is generated at, or just before the end of interval and at the end of the series.
lookup : Lookup
-
Specifies how the lookup based on keys is performed. `Exact` means that the values at exact keys will be returned; `NearestGreater` returns the nearest greater key value (starting at the first key) and `NearestSmaller` returns the nearest smaller key value (starting at most `interval` after the end of the series)
series : Series<^K, ^V>
-
An input series to be resampled
Returns: Series<^K, ^V>
Modifiers: inline Type parameters: ^K, ^V |
Finds values at, or near, the specified times in a given series. The operation generates keys starting at the specified `start` time, using the specified `interval` and then finds values close to such keys using the specified `lookup` and `dir`. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
resample keys dir series
Parameters:
'K seq
-
A collection of keys to be used for resampling of the series
dir : Direction
-
If this parameter is `Direction.Forward`, then each key is used as the smallest key in a chunk; for `Direction.Backward`, the keys are used as the greatest keys in a chunk.
series : Series<'K, 'V>
-
An input series to be resampled
Returns: Series<'K, Series<'K, 'V>>
|
Resample the series based on a provided collection of keys. The values of the series are aggregated into chunks based on the specified keys. Depending on `direction`, the specified key is either used as the smallest or as the greatest key of the chunk (with the exception of boundaries that are added to the first/last chunk). Such chunks are then returned as nested series. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
resampleEquiv keyProj series
Parameters:
'K1 -> 'K2
-
A function that transforms keys from original space to a new space (which is then used for grouping based on equivalence)
series : Series<'K1, 'V1>
-
An input series to be resampled
Returns: Series<'K2, Series<'K1, 'V1>>
Modifiers: inline Type parameters: 'K1, 'K2, 'V1 |
Resample the series based on equivalence class on the keys. A specified function `keyProj` is used to project keys to another space and the observations for which the projected keys are equivalent are grouped into chunks. The chunks are then returned as nested series. This function is similar to `Series.chunkBy`, with the exception that it transforms keys to a new space. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered. For unordered series, similar functionality can be implemented using `Series.groupBy`.
|
Full Usage:
resampleEquivInto keyProj f series
Parameters:
'K1 -> 'K2
-
A function that transforms keys from original space to a new space (which is then used for grouping based on equivalence)
f : Series<'K1, 'V1> -> 'V2
-
A function that is used to collapse a generated chunk into a single value.
series : Series<'K1, 'V1>
-
An input series to be resampled
Returns: Series<'K2, 'V2>
Modifiers: inline Type parameters: 'K1, 'K2, 'V1, 'V2 |
Resample the series based on equivalence class on the keys. A specified function `keyProj` is used to project keys to another space and the observations for which the projected keys are equivalent are grouped into chunks. The chunks are then transformed to values using the provided function `f`. This function is similar to `Series.chunkBy`, with the exception that it transforms keys to a new space. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered. For unordered series, similar functionality can be implemented using `Series.groupBy`.
|
Full Usage:
resampleInto keys dir f series
Parameters:
'K seq
-
A collection of keys to be used for resampling of the series
dir : Direction
-
If this parameter is `Direction.Forward`, then each key is used as the smallest key in a chunk; for `Direction.Backward`, the keys are used as the greatest keys in a chunk.
f : 'K -> Series<'K, 'V> -> 'a
-
A function that is used to collapse a generated chunk into a single value. Note that this function may be called with empty series.
series : Series<'K, 'V>
-
An input series to be resampled
Returns: Series<'K, 'a>
|
Resample the series based on a provided collection of keys. The values of the series are aggregated into chunks based on the specified keys. Depending on `direction`, the specified key is either used as the smallest or as the greatest key of the chunk (with the exception of boundaries that are added to the first/last chunk). Such chunks are then aggregated using the provided function `f`. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
resampleUniform fillMode keyProj nextKey series
Parameters:
Lookup
-
When set to `Lookup.NearestSmaller` or `Lookup.NearestGreater`, the function searches for a nearest available observation in an neighboring chunk. Otherwise, the function `f` is called with an empty series as an argument.
keyProj : 'K1 -> 'K2
-
A function that transforms keys from original space to a new space (which is then used for grouping based on equivalence)
nextKey : 'K2 -> 'K2
-
A function that gets the next key in the transformed space
series : Series<'K1, 'V>
-
An input series to be resampled
Returns: Series<'K2, Series<'K1, 'V>>
|
Resample the series based on equivalence class on the keys and also generate values for all keys of the target space that are between the minimal and maximal key of the specified series (e.g. generate value for all days in the range covered by the series). A specified function `keyProj` is used to project keys to another space and `nextKey` is used to generate all keys in the range. Then return the chunks as nested series. When there are no values for a (generated) key, then the function behaves according to `fillMode`. It can look at the greatest value of previous chunk or smallest value of the next chunk, or it produces an empty series. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
resampleUniformInto fillMode keyProj nextKey f series
Parameters:
Lookup
-
When set to `Lookup.ExactOrSmaller` or `Lookup.ExactOrGreater`, the function searches for a nearest available observation in an neighboring chunk. Otherwise, the function `f` is called with an empty series as an argument. Values `Lookup.Smaller` and `Lookup.Greater` are not supported.
keyProj : 'K1 -> 'K2
-
A function that transforms keys from original space to a new space (which is then used for grouping based on equivalence)
nextKey : 'K2 -> 'K2
-
A function that gets the next key in the transformed space
f : Series<'K1, 'V> -> 'a
-
A function that is used to collapse a generated chunk into a single value. The function may be called on empty series when `fillMode` is `Lookup.Exact`.
series : Series<'K1, 'V>
-
An input series to be resampled
Returns: Series<'K2, 'a>
|
Resample the series based on equivalence class on the keys and also generate values for all keys of the target space that are between the minimal and maximal key of the specified series (e.g. generate value for all days in the range covered by the series). A specified function `keyProj` is used to project keys to another space and `nextKey` is used to generate all keys in the range. The chunk is then aggregated using `f`. When there are no values for a (generated) key, then the function behaves according to `fillMode`. It can look at the greatest value of previous chunk or smallest value of the next chunk, or it produces an empty series. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
sampleTime interval dir series
Parameters:
TimeSpan
-
The interval between the individual samples
dir : Direction
-
If this parameter is `Direction.Forward`, then each key is used as the smallest key in a chunk; for `Direction.Backward`, the keys are used as the greatest keys in a chunk.
series : Series<^a, 'b>
-
An input series to be resampled
Returns: Series<^a, Series<^a, 'b>>
Modifiers: inline Type parameters: ^a, 'b |
Performs sampling by time and returns chunks obtained by time-sampling as a nested series. The operation generates keys starting at the first key in the source series, using the specified `interval` and then obtains chunks based on these keys in a fashion similar to the `Series.resample` function. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
sampleTimeAt start interval dir series
Parameters:
^a
-
The initial time to be used for sampling
interval : TimeSpan
-
The interval between the individual samples
dir : Direction
-
If this parameter is `Direction.Forward`, then each key is used as the smallest key in a chunk; for `Direction.Backward`, the keys are used as the greatest keys in a chunk.
series : Series<^a, 'b>
-
An input series to be resampled
Returns: Series<^a, Series<^a, 'b>>
Modifiers: inline Type parameters: ^a, 'b |
Performs sampling by time and returns chunks obtained by time-sampling as a nested series. The operation generates keys starting at the given `start` time, using the specified `interval` and then obtains chunks based on these keys in a fashion similar to the `Series.resample` function. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
sampleTimeAtInto start interval dir f series
Parameters:
^K
-
The initial time to be used for sampling
interval : TimeSpan
-
The interval between the individual samples
dir : Direction
-
If this parameter is `Direction.Forward`, then each key is used as the smallest key in a chunk; for `Direction.Backward`, the keys are used as the greatest keys in a chunk.
f : Series<^K, ^V> -> 'a
-
A function that is called to aggregate each chunk into a single value.
series : Series<^K, ^V>
-
An input series to be resampled
Returns: Series<^K, 'a>
Modifiers: inline Type parameters: ^K, ^V, 'a |
Performs sampling by time and aggregates chunks obtained by time-sampling into a single value using a specified function. The operation generates keys starting at the given `start` time, using the specified `interval` and then obtains chunks based on these keys in a fashion similar to the `Series.resample` function. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Full Usage:
sampleTimeInto interval dir f series
Parameters:
TimeSpan
-
The interval between the individual samples
dir : Direction
-
If this parameter is `Direction.Forward`, then each key is used as the smallest key in a chunk; for `Direction.Backward`, the keys are used as the greatest keys in a chunk.
f : Series<^K, ^V> -> 'a
-
A function that is called to aggregate each chunk into a single value.
series : Series<^K, ^V>
-
An input series to be resampled
Returns: Series<^K, 'a>
Modifiers: inline Type parameters: ^K, ^V, 'a |
Performs sampling by time and aggregates chunks obtained by time-sampling into a single value using a specified function. The operation generates keys starting at the first key in the source series, using the specified `interval` and then obtains chunks based on these keys in a fashion similar to the `Series.resample` function. This operation is only supported on ordered series. The method throws `InvalidOperationException` when the series is not ordered.
|
Series transformations
Functions and values
| Function or value |
Description
|
Full Usage:
diff offset series
Parameters:
int
-
When positive, subtracts the past values from the current values; when negative, subtracts the future values from the current values.
series : Series<'K, ^T>
-
The input series, containing values that support the `-` operator.
Returns: Series<'K, ^T>
Modifiers: inline Type parameters: 'K, ^T |
Returns a series containing difference between a value in the original series and a value at the specified offset. For example, calling `Series.diff 1 s` returns a series where previous value is subtracted from the current one. In pseudo-code, the function behaves as follows: result[k] = series[k] - series[k - offset]
|
|
Returns a series containing the difference (as |
Full Usage:
diffDateOffset offset series
Parameters:
int
series : Series<'K, DateTimeOffset>
Returns: Series<'K, TimeSpan>
|
Returns a series containing the difference (as
|
|
|
|
|
|
Returns a new series containing only the elements of the input series whose key
maps to
|
|
|
|
|
Full Usage:
foldValues op init series
Parameters:
'a -> 'T -> 'a
-
A function that is used to aggregate elements of the series with the current state
init : 'a
-
An initial value for the aggregation
series : Series<'K, 'T>
-
An input series to be aggregated
Returns: 'a
|
Aggregates the values of the specified series using a function that can combine individual values. The folding starts with the specified initial value.
|
|
|
|
|
|
Returns a new series whose values are the results of applying the given function to
values of the original series. This specified function is called even when the value
is missing. It returns
|
|
|
|
Returns a new series whose values are the results of applying the given function to values of the original series. This function skips over missing values and call the function with just values. It is also aliased using the `$` operator so you can write `series $ func` for `series |> Series.mapValues func`.
|
|
Returns a new series with values replaced by missing where the predicate returns `true`. The predicate is called with both the key and an optional value (to handle missing values). This is the complement of `filterAll`: instead of dropping matching entries the values are replaced with missing so that the key alignment of the series is preserved.
|
|
|
Full Usage:
pctChange offset series
Parameters:
int
-
When positive, computes change from past values; when negative, computes change relative to future values.
series : Series<'K, ^T>
-
The input series, containing values that support the - and / operators.
Returns: Series<'K, ^T>
Modifiers: inline Type parameters: 'K, ^T, ^a |
Returns a series containing the percentage change between a value in the series and
a value at the specified offset. For example, calling
|
Full Usage:
reduceValues op series
Parameters:
'T -> 'T -> 'T
-
A function that is used to aggregate elements of the series
series : Series<'K, 'T>
-
An input series to be aggregated
Returns: 'T
|
Aggregates the values of the specified series using a function that can combine individual values. Fails if the series contains no values.
|
|
Applies a folding function starting with some initial optional value and the first optional value of the series, and continues to "scan" along the series, saving all values produced from the first function application, and yielding a new series having the original index and newly produced values.
|
|
Applies a folding function starting with some initial value and the first value of the series, and continues to "scan" along the series, saving all values produced from the first function application, and yielding a new series having the original index and newly produced values. Any application involving a missing value yields a missing value.
|
|
Returns a series with values shifted by the specified offset. When the offset is positive, the values are shifted forward and first `offset` keys are dropped. When the offset is negative, the values are shifted backwards and the last `offset` keys are dropped. Expressed in pseudo-code: result[k] = series[k - offset] If you want to calculate the difference, e.g. `s - (Series.shift 1 s)`, you can use `Series.diff` which will be a little bit faster.
|
|
|
|
|
|
|
|
Returns a series that contains the specified number of keys from the original series. The keys are taken from the end of the series.
|
Sorting and index manipulation
Functions and values
| Function or value |
Description
|
|
|
|
Returns a new series containing the specified keys mapped to the original values of the series. When the sequence contains _fewer_ keys, the values from the series are dropped. When it contains _more_ keys, the values for additional keys are missing.
|
|
Given an original series and a sequence of keys, returns a new series that contains the matching value for each of the specified keys. The `KeyCount` of the returned sequence is the length of `keys`. If there is no value for the specified keys in the input sequence, the returned series will contain a missing value.
|
|
|
|
|
|
Returns a new series, containing the observations of the original series sorted by values returned by the specified projection function.
|
|
|
Full Usage:
sortWith comparer series
Parameters:
'V -> 'V -> int
-
A comparer function on the series values. The function should return negative integer when the first value is smaller, positive when it is greater and 0 when the values are equal.
series : Series<'K, 'V>
-
An input series whose values are sorter
Returns: Series<'K, 'V>
|
Returns a new series, containing the observations of the original series sorted using the specified comparison function.
|