Deedle
Frame Module
The `Frame` module provides an F#-friendly API for working with data frames.
The module follows the usual desing for collection-processing in F#, so the
functions work well with the pipelining operator (`|>`). For example, given
a frame with two columns representing prices, we can use `Frame.pctChange` to
calculate daily returns like this:
let df = frame [ "MSFT" => prices1; "AAPL" => prices2 ]
let rets = df |> Frame.pctChange 1
rets |> Stats.mean
Note that the `Stats.mean` operation is overloaded and works both on series
(returning a number) and on frames (returning a series). You can also use
`Frame.diff` if you need absolute differences rather than relative changes.
The functions in this module are designed to be used from F#. For a C#-friendly
API, see the `FrameExtensions` type. For working with individual series, see the
`Series` module. The functions in the `Frame` module are grouped in a number of
categories and documented below.
Accessing frame data and lookup
-------------------------------
Functions in this category provide access to the values in the fame. You can
also add and remove columns from a frame (which both return a new value).
- `addCol`, `replaceCol` and `dropCol` can be used to create a new data frame
with a new column, by replacing an existing column with a new one, or by dropping
an existing column
- `cols` and `rows` return the columns or rows of a frame as a series containing
objects; `getCols` and `getRows` return a generic series and cast the values to
the type inferred from the context (columns or rows of incompatible types are skipped);
`getNumericCols` returns columns of a type convertible to `float` for convenience.
- You can get a specific row or column using `get[Col|Row]` or `lookup[Col|Row]` functions.
The `lookup` variant lets you specify lookup behavior for key matching (e.g. find the
nearest smaller key than the specified value). There are also `[try]get` and `[try]Lookup`
functions that return optional values and functions returning entire observations
(key together with the series).
- `sliceCols` and `sliceRows` return a sub-frame containing only the specified columns
or rows. Finally, `toArray2D` returns the frame data as a 2D array.
Grouping, windowing and chunking
--------------------------------
The basic grouping functions in this category can be used to group the rows of a
data frame by a specified projection or column to create a frame with hierarchical
index such as Frame<'K1 * 'K2, 'C>. The functions always aggregate rows, so if you
want to group columns, you need to use `Frame.transpose` first.
The function `groupRowsBy` groups rows by the value of a specified column. Use
`groupRowsBy[Int|Float|String...]` if you want to specify the type of the column in
an easier way than using type inference; `groupRowsUsing` groups rows using the
specified _projection function_ and `groupRowsByIndex` projects the grouping key just
from the row index.
More advanced functions include: `aggregateRowsBy` which groups the rows by a
specified sequence of columns and aggregates each group into a single value;
`pivotTable` implements the pivoting operation [as documented in the
tutorials](../frame.html#pivot).
The `melt` and `unmelt` functions turn the data frame into a single data frame
containing columns `Row`, `Column` and `Value` containing the data of the original
frame; `unmelt` can be used to turn this representation back into an original frame.
The `stack` and `unstack` functions implement pandas-style reshape operations.
`stack` converts `Frame<'R,'C>` to a long-format `Frame<'R*'C, string>` where each
cell becomes a row keyed by `(rowKey, colKey)` with a single `"Value"` column.
`unstack` promotes the inner row-key level to column keys, producing
`Frame<'R1, 'C*'R2>` from `Frame<'R1*'R2,'C>`.
A simple windowing functions that are exposed for an entire frame operations are
`window` and `windowInto`. For more complex windowing operations, you currently have
to use `mapRows` or `mapCols` and apply windowing on individual series.
Sorting and index manipulation
------------------------------
A frame is indexed by row keys and column keys. Both of these indices can be sorted
(by the keys). A frame that is sorted allows a number of additional operations (such
as lookup using the `Lookp.ExactOrSmaller` lookup behavior). The functions in this
category provide ways for manipulating the indices. It is expected that most operations
are done on rows and so more functions are available in a row-wise way. A frame can
alwyas be transposed using `Frame.transpose`.
Index operations:
The existing row/column keys can be replaced by a sequence of new keys using the
`indexColsWith` and `indexRowsWith` functions. Row keys can also be replaced by
ordinal numbers using `indexRowsOrdinally`.
The function `indexRows` uses the specified column of the original frame as the
index. It removes the column from the resulting frame (to avoid this, use overloaded
`IndexRows` method). This function infers the type of row keys from the context, so it
is usually more convenient to use `indexRows[Date|String|Int|...]` functions. Finally,
if you want to calculate the index value based on multiple columns of the row, you
can use `indexRowsUsing`.
Sorting frame rows:
Frame rows can be sorted according to the value of a specified column using the
`sortRows` function; `sortRowsBy` takes a projection function which lets you
transform the value of a column (e.g. to project a part of the value).
The functions `sortRowsByKey` and `sortColsByKey` sort the rows or columns
using the default ordering on the key values. The result is a frame with ordered
index.
Expanding columns:
When the frame contains a series with complex .NET objects such as F# records or
C# classes, it can be useful to "expand" the column. This operation looks at the
type of the objects, gets all properties of the objects (recursively) and
generates multiple series representing the properties as columns.
The function `expandCols` expands the specified columns while `expandAllCols`
applies the expansion to all columns of the data frame.
Frame transformations
---------------------
Functions in this category perform standard transformations on data frames including
projections, filtering, taking some sub-frame of the frame, aggregating values
using scanning and so on.
Projection and filtering functions such as `[map|filter][Cols|Rows]` call the
specified function with the column or row key and an ObjectSeries<'K> representing
the column or row. You can use functions ending with `Values` (such as `mapRowValues`)
when you do not require the row key, but only the row series; `mapRowKeys` and
`mapColKeys` can be used to transform the keys.
You can use `reduceValues` to apply a custom reduction to values of columns. Other
aggregations are available in the `Stats` module. You can also get a row with the
greaterst or smallest value of a given column using `[min|max]RowBy`.
The functions `take[Last]` and `skip[Last]` can be used to take a sub-frame of the
original source frame by skipping a specified number of rows. Note that this
does not require an ordered frame and it ignores the index - for index-based lookup
use slicing, such as `df.Rows.[lo .. hi]`, instead.
Finally the `shift` function can be used to obtain a frame with values shifted by
the specified offset. This can be used e.g. to get previous value for each key using
`Frame.shift 1 df`. The `diff` function calculates difference from previous value using
`df - (Frame.shift offs df)`.
Processing frames with exceptions
---------------------------------
The functions in this group can be used to write computations over frames that may fail.
They use the type tryval<'T> which is defined as a discriminated union
with two cases: Success containing a value, or Error containing an exception.
Using tryval<'T> as a value in a data frame is not generally recommended, because
the type of values cannot be tracked in the type. For this reason, it is better to use
tryval<'T> with individual series. However, `tryValues` and `fillErrorsWith` functions
can be used to get values, or fill failed values inside an entire data frame.
The `tryMapRows` function is more useful. It can be used to write a transformation
that applies a computation (which may fail) to each row of a data frame. The resulting
series is of type Series<'R, tryval<'T>> and can be processed using the Series module
functions.
Missing values
--------------
This group of functions provides a way of working with missing values in a data frame.
The category provides the following functions that can be used to fill missing values:
* `fillMissingWith` fills missing values with a specified constant
* `fillMissingUsing` calls a specified function for every missing value
* `fillMissing` and variants propagates values from previous/later keys
We use the terms _sparse_ and _dense_ to denote series that contain some missing values
or do not contain any missing values, respectively. The functions `denseCols` and
`denseRows` return a series that contains only dense columns or rows and all sparse
rows or columns are replaced with a missing value. The `dropSparseCols` and `dropSparseRows`
functions drop these missing values and return a frame with no missing values.
Joining, merging and zipping
----------------------------
The simplest way to join two frames is to use the `join` operation which can be used to
perform left, right, outer or inner join of two frames. When the row keys of the frames do
not match exactly, you can use `joinAlign` which takes an additional parameter that specifies
how to find matching key in left/right join (e.g. by taking the nearest smaller available key).
Frames that do not contian overlapping values can be combined using `merge` (when combining
just two frames) or using `mergeAll` (for larger number of frames). Tha latter is optimized
to work well for a large number of data frames.
Finally, frames with overlapping values can be combined using `zip`. It takes a function
that is used to combine the overlapping values. A `zipAlign` function provides a variant
with more flexible row key matching (as in `joinAlign`)
Hierarchical index operations
-----------------------------
A data frame has a hierarchical row index if the row index is formed by a tuple, such as
Frame<'R1 * 'R2, 'C>. Frames of this kind are returned, for example, by the grouping
functions such as Frame.groupRowsBy. The functions in this category provide ways for
working with data frames that have hierarchical row keys.
The functions applyLevel and reduceLevel can be used to reduce values according to
one of the levels. The applyLevel function takes a reduction of type Series<'K, 'T> -> 'T
while reduceLevel reduces individual values using a function of type 'T -> 'T -> 'T.
The functions nest and unnest can be used to convert between frames with
hierarchical indices (Frame<'K1 * 'K2, 'C>) and series of frames that represent
individual groups (Series<'K1, Frame<'K2, 'C>>). The nestBy function can be
used to perform group by operation and return the result as a series of frems.
Table of contents
- Other module members
- Accessing frame data and lookup
- Creating frames
- Frame transformations
- Grouping, windowing and chunking
- Hierarchical index operations
- Input and output
- Joining, merging and zipping
- Missing values
- Sorting and index manipulation
Other module members
Functions and values
| Function or value |
Description
|
|
Fills all error cases of a `tryval<'T>` value in a data frame with the specified
`value`. The function takes all columns of type `tryval<'T>` and uses `Series.fillErrorsWith`
to fill the error values with the specified default value.
|
|
|
|
|
|
|
|
Group rows of a data frame using the specified `keySelector`. The selector is called with
a key of each row and should return a new key. The result is a frame with multi-level index,
here the first level is formed by the newly created keys.
|
|
|
|
|
|
|
|
Replace the row index of the frame with ordinarilly generated integers starting from zero.
The rows of the frame are assigned index according to the current order, or in a
non-deterministic way, if the current row index is not ordered.
|
|
Returns a specified series (column) from a data frame. If the data frame has
ordered column index, the lookup semantics can be used to get series
with nearest greater/smaller key. For exact semantics, you can use `getCol`.
|
|
Returns a specified row from a data frame. If the data frame has
ordered row index, the lookup semantics can be used to get row with
nearest greater/smaller key. For exact semantics, you can use `getRow`.
|
|
|
|
Similar to `melt`, but keeps the specified identity columns as-is and melts the remaining columns into `Column` and `Value` pairs. This is analogous to pandas `DataFrame.melt(id_vars=...)` or R's `data.table::melt`. ## Parameters - `idCols` - The columns to use as identity variables (kept in the output) - `frame` - The input data frame ## Returns A data frame with the identity columns followed by `Column` and `Value` columns. The row index is a sequential integer starting from 0.
|
|
Append a sequence of data frames with non-overlapping values. The operation takes the union of
columns and rows of the source data frames and then unions the values. An exception is thrown when
both data frames define value for a column/row location, but the operation succeeds if one
all frames but one has a missing value at the location.
Note that the rows are *not* automatically reindexed to avoid overlaps. This means that when
a frame has rows indexed with ordinal numbers, you may need to explicitly reindex the row
keys before calling append.
|
|
Given a data frame, use the specified `keySelector` to generate a new, first-level
of indices based on the current indices. Returns a series (indexed by the first-level)
of frames (indexed by the second-level).
|
|
Reduce the values in each series according to the specified level of a hierarchical row key.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Returns data of the data frame as a 2D array containing data as `float` values.
Missing data are represented as `Double.NaN` in the returned array.
|
|
Returns data of the data frame as a jagged array containing data as `float` values.
Missing data are represented as `Double.NaN` in the returned array.
|
|
|
|
|
|
|
|
|
Full Usage:
tryMapRows f frame
Parameters:
'R -> ObjectSeries<'C> -> 'V
frame : Frame<'R, 'C>
Returns: Series<'R, 'V tryval>
|
Returns a series, obtained by applying the specified projection function `f` to all rows
of the input frame. The resulting series wraps the results in `tryval<'V>`. When the projection
function fails, the exception is wrapped using the `Error` case.
|
|
Given a data frame containing columns of type `tryval<'T>`, returns a new data frame
that contains the underlying values of type `'T`. When the frame contains one or more
failures, the operation throws `AggregateException`. Otherwise, it returns a frame containing values.
|
|
Accessing frame data and lookup
Functions and values
| Function or value |
Description
|
|
Creates a new data frame that contains all data from the original data frame, together with an additional series. The operation uses left join and aligns new series to the existing frame keys.
|
|
Returns the columns of the data frame as a series (indexed by the column keys of the source frame) containing untyped series representing individual columns of the frame.
|
|
Returns the total number of column keys in the specified frame. This returns the total length of columns, including keys for which there is no data available.
|
|
Returns the total number of row keys in the specified frame. This returns the total length of the row series, including keys for which there is no value available.
|
|
|
|
Creates a new data frame that contains all data from the original data frame without the specified series (column). The operation throws if the column key is not found.
|
|
|
|
Returns a series of columns of the data frame indexed by the column keys,
which contains those series whose values are convertible to
|
|
|
|
Returns a sub-frame containing the rows and columns at the specified integer (zero-based)
positions from the original data frame. Equivalent to pandas
Example
|
|
Returns a frame consisting of the columns at the specified integer (zero-based) positions
from the original data frame. Equivalent to pandas
|
|
|
Full Usage:
replaceCol column series frame
Parameters:
'C
-
A key (or name) for the column to be replaced or added
series : ISeries<'R>
-
A data series to be used (the row key type has to match)
frame : Frame<'R, 'C>
-
Source data frame (which is not mutated by the operation)
Returns: Frame<'R, 'C>
|
Creates a new data frame where the specified column is replaced with a new series. (If the series does not exist, only the new series is added.)
|
|
|
|
|
|
|
|
|
|
Creating frames
Functions and values
| Function or value |
Description
|
Returns an empty data frame with no rows or columns.
Example
let df : Frame |
Frame transformations
Functions and values
| Function or value |
Description
|
|
Returns a frame with columns containing difference between an original value and a value at the specified offset. For example, calling `Frame.diff 1 s` returns a frame where previous column values is subtracted from the current ones. In pseudo-code, the function behaves as follows: result[k] = series[k] - series[k - offset] Columns that cannot be converted to `float` are left without a change.
|
|
Returns a new data frame containing only the rows of the input frame that have
distinct values in the specified columns. When multiple rows have the same values
in those columns, only the first row (in index order) is preserved.
## Parameters
- `columns` - A sequence of column keys used to determine row uniqueness
- `frame` - Input data frame to be transformed
## Example
// Keep only the first row for each unique combination of "Category" and "Region"
df |> Frame.distinctRowsBy ["Category"; "Region"]
[category:Frame transformations]
|
Full Usage:
filterColValues f frame
Parameters:
ObjectSeries<'R> -> bool
-
Function of one argument that defines the predicate
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Frame<'R, 'C>
|
Returns a new data frame containing only the columns of the input frame for which the specified predicate returns `true`. The predicate is called with an object series that represents the column data (use `filterCols` if you need to access the column key).
|
Full Usage:
filterCols f frame
Parameters:
'C -> ObjectSeries<'R> -> bool
-
Function of two arguments that defines the predicate
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Frame<'R, 'C>
|
Returns a new data frame containing only the columns of the input frame for which the specified predicate returns `true`. The predicate is called with the column key and object series that represents the column data.
|
|
Returns a new data frame containing only the columns of the input frame whose
corresponding value in the boolean mask series is
|
Full Usage:
filterRowValues f frame
Parameters:
ObjectSeries<'C> -> bool
-
Function of one argument that defines the predicate
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Frame<'R, 'C>
|
Returns a new data frame containing only the rows of the input frame for which the specified predicate returns `true`. The predicate is called with an object series that represents the row data (use `filterRows` if you need to access the row key).
|
Full Usage:
filterRows f frame
Parameters:
'R -> ObjectSeries<'C> -> bool
-
Function of two arguments that defines the predicate
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Frame<'R, 'C>
|
Returns a new data frame containing only the rows of the input frame for which the specified predicate returns `true`. The predicate is called with the row key and object series that represents the row data.
|
Full Usage:
filterRowsBy column value frame
Parameters:
'C
-
The name of the column to be matched
value : 'V
-
Required value of the column. Note that the function is generic and no conversions are performed, so the value has to match including the actual type.
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Frame<'R, 'C>
|
Returns a new data frame containing only the rows of the input frame for which the specified `column` has the specified `value`. The operation may be implemented via an index for virtualized Deedle frames.
|
|
Returns a new data frame containing only the rows of the input frame whose
corresponding value in the boolean mask series is
|
|
Builds a new data frame whose values are the results of applying the specified function on these values, but only for those columns which can be converted to the appropriate type for input to the mapping function.
|
|
|
Full Usage:
mapColValues f frame
Parameters:
ObjectSeries<'R> -> 'a
-
Function of one argument that defines the column mapping
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Frame<'b, 'C>
|
Builds a new data frame whose columns are the results of applying the specified function on the columns of the input data frame. The function is called with an object series that represents the column data (use `mapCols` if you need to access the column key).
|
|
Builds a new data frame whose columns are the results of applying the specified
function on the typed columns of the input data frame. Unlike
|
Full Usage:
mapCols f frame
Parameters:
'C -> ObjectSeries<'R> -> 'a
-
Function of two arguments that defines the column mapping
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Frame<'b, 'C>
|
Builds a new data frame whose columns are the results of applying the specified function on the columns of the input data frame. The function is called with the column key and object series that represents the column data.
|
|
|
Full Usage:
mapRowValues f frame
Parameters:
ObjectSeries<'C> -> 'V
-
Function of one argument that defines the row mapping
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Series<'R, 'V>
Modifiers: inline Type parameters: 'C, 'V, 'R |
Builds a new data frame whose rows are the results of applying the specified function on the rows of the input data frame. The function is called with an object series that represents the row data (use `mapRows` if you need to access the row key).
|
Full Usage:
mapRows f frame
Parameters:
'R -> ObjectSeries<'C> -> 'V
-
Function of two arguments that defines the row mapping
frame : Frame<'R, 'C>
-
Input data frame to be transformed
Returns: Series<'R, 'V>
Modifiers: inline Type parameters: 'R, 'C, 'V |
Builds a new data frame whose rows are the results of applying the specified function on the rows of the input data frame. The function is called with the row key and object series that represents the row data.
|
|
Builds a new data frame whose values are the results of applying the specified function on these values, but only for those columns which can be converted to the appropriate type for input to the mapping function (use `map` if you need to access the row and column keys).
|
Full Usage:
maxRowBy column frame
Parameters:
'C
frame : Frame<'R, 'C>
Returns: ('R * ObjectSeries<'C>) option
Modifiers: inline Type parameters: 'C, 'R |
Returns a row of the data frame which has the greatest value of the specified `column`. The row is returned as an optional value (which is `None` for empty frame) and contains a key together with an object series representing the row.
|
Full Usage:
minRowBy column frame
Parameters:
'C
frame : Frame<'R, 'C>
Returns: ('R * ObjectSeries<'C>) option
Modifiers: inline Type parameters: 'C, 'R |
Returns a row of the data frame which has the smallest value of the specified `column`. The row is returned as an optional value (which is `None` for empty frame) and contains a key together with an object series representing the row.
|
|
Returns a frame where each value is the percentage change relative to the value at the
specified offset. For example, calling
|
|
Returns a series that contains the results of aggregating each column to a single value. The function takes columns that can be converted to the type expected by the specified `op` function and reduces the values in each column using `Series.reduceValues`.
ExampleThe following sums the values in each column that can be converted to `float` and returns the result as a new series: Multiple items
val float: value: 'T -> float (requires member op_Explicit) -------------------- type float = System.Double -------------------- type float<'Measure> = float |
|
Returns a frame with columns shifted by the specified offset. When the offset is positive, the values are shifted forward and first `offset` keys are dropped. When the offset is negative, the values are shifted backwards and the last `offset` keys are dropped. Expressed in pseudo-code: result[k] = series[k - offset] If you want to calculate the difference, e.g. `df - (Frame.shift 1 df)`, you can use `Frame.diff` which will be a little bit faster.
|
Grouping, windowing and chunking
Functions and values
| Function or value |
Description
|
Full Usage:
aggregateRowsBy groupBy aggBy aggFunc frame
Parameters:
'C seq
-
sequence of columns to group by
aggBy : 'C seq
-
sequence of columns to apply aggFunc to
aggFunc : Series<'R, 'V1> -> 'V2
-
invoked in order to aggregate values
frame : Frame<'R, 'C>
-
The input data frame to be aggregated
Returns: Frame<int, 'C>
|
Returns a data frame whose rows are grouped by `groupBy` and whose columns specified in `aggBy` are aggregated according to `aggFunc`.
|
|
Group rows of a data frame using the specified `column`. The type of the column is inferred from the usage of the resulting frame. The result is a frame with multi-level index, where the first level is formed by the newly created keys. Use `groupRowsBy[Int|String|...]` to explicitly specify the type of the column.
|
Full Usage:
groupRowsUsing selector frame
Parameters:
'R -> ObjectSeries<'C> -> 'K
frame : Frame<'R, 'C>
Returns: Frame<('K * 'R), 'C>
|
Group rows of a data frame using the specified `selector`. The selector is called with a row key and object series representing the row and should return a new key. The result is a frame with multi-level index, where the first level is formed by the newly created keys.
|
Full Usage:
pivotTable rowGrp colGrp op frame
Parameters:
'R -> ObjectSeries<'C> -> 'RNew
-
A function from rowkey & row to group value for the resulting row index
colGrp : 'R -> ObjectSeries<'C> -> 'CNew
-
A function from rowkey & row to group value for the resulting col index
op : Frame<'R, 'C> -> 'T
-
A function computing a value from the corresponding bucket frame
frame : Frame<'R, 'C>
-
The input data frame to pivot
Returns: Frame<'RNew, 'CNew>
|
Creates a new data frame resulting from a 'pivot' operation. Consider a denormalized data frame representing a table: column labels are field names & table values are observations of those fields. pivotTable buckets the rows along two axes, according to the results of the functions `rowGrp` and `colGrp`; and then computes a value for the frame of rows that land in each bucket.
|
|
Converts a data frame to a long-format frame by combining the row index and the
column index into a 2-tuple row index. Each cell of the original frame becomes a row
in the result keyed by
This is the pandas-style
|
|
|
|
Converts a data frame with a 2-tuple row index into a wide-format frame by promoting
the inner (second) element of each row key to the column index. The resulting frame has
row keys equal to the unique first tuple elements and column keys of the form
This is the pandas-style
|
|
Creates a sliding window using the specified size. The result is a series containing data frames that represent individual windows. This function skips incomplete chunks.
|
|
Creates a sliding window using the specified size and then applies the provided value selector `f` on each window to produce the result which is returned as a new series. This function skips incomplete chunks.
|
Hierarchical index operations
Functions and values
| Function or value |
Description
|
|
Apply a specified function to a group of values in each series according to the specified level of a hierarchical row key. For each group of rows as specified by `levelSel`, the function applies the specified function `op` to all columns. Columns that cannot be converted to a type required by `op` are skipped.
This function reduces a series of values using a function
Example
To get the standard deviation of values in all numerical columns according to the first
component of a two level row key val fst: tuple: ('T1 * 'T2) -> 'T1
|
|
Given a frame with two-level row index, returns a series indexed by the first part of the key, containing frames representing individual groups. This function can be used if you want to perform a transformation individually on each group (e.g. using `Series.mapValues` after calling `Frame.nest`).
|
|
Given a data frame, uses
This is a generalisation of
|
Input and output
Functions and values
| Function or value |
Description
|
Full Usage:
saveJson path frame
Parameters:
string
-
The output file path.
frame : Frame<'R, 'C>
-
The data frame to serialize.
|
Save the data frame as a JSON file at the specified path.
|
Full Usage:
toJson orient frame
Parameters:
string
-
Controls the JSON layout:
"columns" (default) produces a column-major object {"col":{"row":v,...},...};
"index" produces a row-major object {"row":{"col":v,...},...};
"records" produces an array of row objects [{"col":v,...},...].
frame : Frame<'R, 'C>
-
The data frame to serialize.
Returns: string
|
Serialize the data frame to a JSON string.
|
Joining, merging and zipping
Functions and values
| Function or value |
Description
|
|
Compares two frames column-by-column and returns a new frame of
|
Full Usage:
join kind frame1 frame2
Parameters:
JoinKind
-
Specifies the joining behavior on row indices. Use `JoinKind.Outer` and `JoinKind.Inner` to get the union and intersection of the row keys, respectively. Use `JoinKind.Left` and `JoinKind.Right` to use the current key of the left/right data frame.
frame1 : Frame<'R, 'C>
-
First data frame (left) to be used in the joining
frame2 : Frame<'R, 'C>
-
Other frame (right) to be joined with `frame1`
Returns: Frame<'R, 'C>
|
Join two data frames. The columns of the joined frames must not overlap and their rows are aligned and transformed according to the specified join kind. For more alignment options on ordered frames, see `joinAlign`.
|
Full Usage:
joinAlign kind lookup frame1 frame2
Parameters:
JoinKind
-
Specifies the joining behavior on row indices. Use `JoinKind.Outer` and `JoinKind.Inner` to get the union and intersection of the row keys, respectively. Use `JoinKind.Left` and `JoinKind.Right` to use the current key of the left/right data frame.
lookup : Lookup
-
When `kind` is `Left` or `Right` and the two frames have ordered row index, this parameter can be used to specify how to find value for a key when there is no exactly matching key or when there are missing values.
frame1 : Frame<'R, 'C>
-
First data frame (left) to be used in the joining
frame2 : Frame<'R, 'C>
-
Other frame (right) to be joined with `frame1`
Returns: Frame<'R, 'C>
|
Join two data frames. The columns of the joined frames must not overlap and their rows are aligned and transformed according to the specified join kind. When the index of both frames is ordered, it is possible to specify `lookup` in order to align indices from other frame to the indices of the main frame (typically, to find the nearest key with available value for a key).
|
Full Usage:
joinOn kind colKey frame1 frame2
Parameters:
JoinKind
-
Specifies the joining behavior on row indices. Use JoinKind.Outer and JoinKind.Inner to get the union and intersection of the row keys, respectively. Use JoinKind.Left and JoinKind.Right to use the current key of the left/right data frame.
colKey : 'C
-
The name of the column to join on. This column must exist in both frames and must have unique values.
frame1 : Frame<'R1, 'C>
-
First data frame (left) to be used in the joining.
frame2 : Frame<'R2, 'C>
-
Other frame (right) to be joined with frame1.
Returns: Frame<'K, 'C>
|
Join two data frames on a shared column, using that column's values as the row index for
alignment. Both frames must have unique values in the specified column (the unique-key case).
The join column is removed from the data columns of the result and becomes the row index.
This is a convenience wrapper around
|
Full Usage:
joinOnInt kind colKey frame1 frame2
Parameters:
JoinKind
-
Specifies the joining behavior on row indices. Use JoinKind.Outer and JoinKind.Inner to get the union and intersection of the row keys, respectively. Use JoinKind.Left and JoinKind.Right to use the current key of the left/right data frame.
colKey : 'C
-
The name of the column to join on.
frame1 : Frame<'R1, 'C>
-
First data frame (left).
frame2 : Frame<'R2, 'C>
-
Other frame (right).
Returns: Frame<int, 'C>
|
Join two data frames on a shared int-valued column. Both frames must have unique values
in the specified column (the unique-key case). The join column becomes the row index of the
result frame. This is the int-specialised variant of
|
Full Usage:
joinOnString kind colKey frame1 frame2
Parameters:
JoinKind
-
Specifies the joining behavior on row indices. Use JoinKind.Outer and JoinKind.Inner to get the union and intersection of the row keys, respectively. Use JoinKind.Left and JoinKind.Right to use the current key of the left/right data frame.
colKey : 'C
-
The name of the column to join on.
frame1 : Frame<'R1, 'C>
-
First data frame (left).
frame2 : Frame<'R2, 'C>
-
Other frame (right).
Returns: Frame<string, 'C>
|
Join two data frames on a shared string-valued column. Both frames must have unique values
in the specified column (the unique-key case). The join column becomes the row index of the
result frame. This is the string-specialised variant of
|
|
Append two data frames with non-overlapping values. The operation takes the union of columns and rows of the source data frames and then unions the values. An exception is thrown when both data frames define value for a column/row location, but the operation succeeds if one frame has a missing value at the location. Note that the rows are *not* automatically reindexed to avoid overlaps. This means that when a frame has rows indexed with ordinal numbers, you may need to explicitly reindex the row keys before calling append.
|
Full Usage:
zip op frame1 frame2
Parameters:
'V1 -> 'V2 -> 'V
-
A function that is applied to aligned values. The `Zip` operation is generic in the type of this function and the type of function is used to determine which values in the frames are zipped and which are left unchanged.
frame1 : Frame<'R, 'C>
-
First frame to be aligned and zipped with the other instance
frame2 : Frame<'R, 'C>
-
Other frame to be aligned and zipped with the first instance
Returns: Frame<'R, 'C>
|
Aligns two data frames using both column index and row index and apply the specified operation
on values of a specified type that are available in both data frames. This overload uses
`JoinKind.Outer` for both columns and rows.
Once aligned, the call
|
Full Usage:
zipAlign columnKind rowKind lookup op frame1 frame2
Parameters:
JoinKind
-
Specifies how to align columns (inner, outer, left or right join)
rowKind : JoinKind
-
Specifies how to align rows (inner, outer, left or right join)
lookup : Lookup
-
Specifies how to find matching value for a row (when using left or right join on rows)
op : 'V1 -> 'V2 -> 'V
-
A function that is applied to aligned values. The `Zip` operation is generic in the type of this function and the type of function is used to determine which values in the frames are zipped and which are left unchanged.
frame1 : Frame<'R, 'C>
-
First frame to be aligned and zipped with the other instance
frame2 : Frame<'R, 'C>
-
Other frame to be aligned and zipped with the first instance
Returns: Frame<'R, 'C>
|
Aligns two data frames using both column index and row index and apply the specified operation
on values of a specified type that are available in both data frames. The parameters `columnKind`,
and `rowKind` can be specified to determine how the alginment works (similarly to `Join`).
Column keys are always matched using `Lookup.Exact`, but `lookup` determines lookup for rows.
Once aligned, the call
|
Missing values
Functions and values
| Function or value |
Description
|
|
Returns the columns of the data frame that do not have any missing values. The operation returns a series (indexed by the column keys of the source frame) containing _series_ representing individual columns of the frame. This is similar to `Columns`, but it skips columns that contain missing value in _any_ row.
|
|
Returns the rows of the data frame that do not have any missing values. The operation returns a series (indexed by the row keys of the source frame) containing _series_ representing individual row of the frame. This is similar to `Rows`, but it skips rows that contain missing value in _any_ column.
|
|
|
|
|
|
|
|
|
|
Creates a new data frame that contains only those rows of the original data frame for which the given column has a value (i.e., is not missing). The resulting data frame has the same number of columns, but may have fewer rows (or no rows at all).
|
Full Usage:
fillMissing direction frame
Parameters:
Direction
-
Specifies the direction used when searching for the nearest available value. `Backward` means that we want to look for the first value with a smaller key while `Forward` searches for the nearest greater key.
frame : Frame<'R, 'C>
-
An input data frame that is to be filled
Returns: Frame<'R, 'C>
|
Fill missing values in the data frame with the nearest available value (using the specified direction). Note that the frame may still contain missing values after call to this function (e.g. if the first value is not available and we attempt to fill series with previous values). This operation can only be used on ordered frames.
|
Full Usage:
fillMissingUsing f frame
Parameters:
Series<'R, 'T> -> 'R -> 'T
-
A function that takes a series Series<R, T> together with a key `K` in the series and generates a value to be used in a place where the original series contains a missing value.
frame : Frame<'R, 'C>
-
An input data frame that is to be filled
Returns: Frame<'R, 'C>
|
Fill missing values in the frame using the specified function. The specified function is called with all series and keys for which the frame does not contain value and the result of the call is used in place of the missing value. The operation is only applied to columns (series) that contain values of the same type as the return type of the provided filling function. The operation does not attempt to convert between numeric values (so a series containing `float` will not be converted to a series of `int`).
|
|
Fill missing values of a given type in the frame with a constant value. The fill value is applied to columns whose element type matches the fill value type (using safe numeric widening). This includes both cases where the column type can be widened to the fill value type (e.g. decimal columns filled with a float value) and cases where the fill value can be widened to the column type (e.g. an integer fill value filling float columns). Columns whose element type is incompatible with the fill value type are left unchanged.
|
Sorting and index manipulation
Functions and values
| Function or value |
Description
|
|
Creates a new data frame where all columns are expanded based on runtime
structure of the objects they store. The expansion is performed recrusively
to the specified depth. A column can be expanded if it is
|
|
Creates a new data frame where the specified columns are expanded based on runtime
structure of the objects they store. A column can be expanded if it is
|
|
Replace the column index of the frame with the provided sequence of column keys. The columns of the frame are assigned keys according to the provided order. The specified column is removed from the resulting frame.
|
Full Usage:
indexRows column frame
Parameters:
'C
-
The name of a column in the original data frame that will be used for the new index. Note that the values in the column need to be unique.
frame : Frame<'R1, 'C>
-
Source data frame whose row index is to be replaced.
Returns: Frame<'R2, 'C>
|
Returns a data frame whose rows are indexed based on the specified column of the original data frame. The generic type parameter is specifies the type of the values in the required index column (and usually needs to be specified using a type annotation). The specified column is removed from the resulting frame.
|
Full Usage:
indexRowsApply column f frame
Parameters:
'C
-
The name of a column in the original data frame to use for the new index.
f : 'V -> 'R2
-
A function that converts a column value to the desired row key type.
frame : Frame<'R1, 'C>
-
Source data frame whose row index is to be replaced.
Returns: Frame<'R2, 'C>
|
Returns a data frame whose rows are indexed based on the specified column of the original data frame, with each index value transformed by the given function. This is useful when the column contains values that need to be converted or parsed into the desired key type (for example, parsing date strings from a CSV file). The specified column is removed from the resulting frame.
Example// Index by a "Date" string column, parsing each value as a DateTime frame |> Frame.indexRowsApply "Date" System.DateTime.Parse // Index by an "Id" column, converting strings to integers frame |> Frame.indexRowsApply "Id" int |
Full Usage:
indexRowsDateTime column frame
Parameters:
'C
-
The name of a column in the original data frame that will be used for the new index. Note that the values in the column need to be unique.
frame : Frame<'R1, 'C>
-
Source data frame whose row index is to be replaced.
Returns: Frame<DateTime, 'C>
|
Returns a data frame whose rows are indexed based on the specified column of the original data frame. This function casts (or converts) the column key to values of type `DateTime` (a generic variant that may require some type annotation is `Frame.indexRows`) The specified column is removed from the resulting frame.
|
Full Usage:
indexRowsDateTimeOffset column frame
Parameters:
'C
-
The name of a column in the original data frame that will be used for the new index. Note that the values in the column need to be unique.
frame : Frame<'R1, 'C>
-
Source data frame whose row index is to be replaced.
Returns: Frame<DateTimeOffset, 'C>
|
Returns a data frame whose rows are indexed based on the specified column of the original data frame. This function casts (or converts) the column key to values of type `DateTimeOffset` (a generic variant that may require some type annotation is `Frame.indexRows`) The specified column is removed from the resulting frame.
|
Full Usage:
indexRowsInt column frame
Parameters:
'C
-
The name of a column in the original data frame that will be used for the new index. Note that the values in the column need to be unique.
frame : Frame<'R1, 'C>
-
Source data frame whose row index is to be replaced.
Returns: Frame<int, 'C>
|
Returns a data frame whose rows are indexed based on the specified column of the original data frame. This function casts (or converts) the column key to values of type `int` (a generic variant that may require some type annotation is `Frame.indexRows`) The specified column is removed from the resulting frame.
|
Full Usage:
indexRowsObj column frame
Parameters:
'C
-
The name of a column in the original data frame that will be used for the new index. Note that the values in the column need to be unique.
frame : Frame<'R1, 'C>
-
Source data frame whose row index is to be replaced.
Returns: Frame<obj, 'C>
|
Returns a data frame whose rows are indexed based on the specified column of the original data frame. This function casts (or converts) the column key to values of type `obj` (a generic variant that may require some type annotation is `Frame.indexRows`) The specified column is removed from the resulting frame.
|
Full Usage:
indexRowsString column frame
Parameters:
'C
-
The name of a column in the original data frame that will be used for the new index. Note that the values in the column need to be unique.
frame : Frame<'R1, 'C>
-
Source data frame whose row index is to be replaced.
Returns: Frame<string, 'C>
|
Returns a data frame whose rows are indexed based on the specified column of the original data frame. This function casts (or converts) the column key to values of type `string` (a generic variant that may require some type annotation is `Frame.indexRows`) The specified column is removed from the resulting frame.
|
Full Usage:
indexRowsUsing f frame
Parameters:
ObjectSeries<'C> -> 'R2
-
A function from row (as object series) to new row key value
frame : Frame<'R1, 'C>
-
Source data frame whose row index are to be replaced.
Returns: Frame<'R2, 'C>
|
Replace the row index of the frame with a sequence of row keys generated using a function invoked on each row.
|
|
|
|
Returns a series containing the dense rank of each row in the data frame, ordered by the values in the specified column. The rank is 1-based: the row with the smallest column value receives rank 1. Rows with equal values receive the same rank. Rows where the specified column has a missing value receive a missing rank.
|
|
Align the existing data to a specified collection of row keys. Values in the data frame that do not match any new key are dropped, new keys (that were not in the original data frame) are assigned missing values.
|
|
Rename a single column of the data frame. Returns a new frame; does not mutate the
original. If
|
|
Rename all columns of the data frame by applying the specified mapping function. Returns a new frame; does not mutate the original.
|
|
Returns a transposed data frame. The rows of the original data frame are used as the columns of the new one (and vice versa). Use this operation if you have a data frame and you mostly need to access its rows as a series (because accessing columns as a series is more efficient).
|