Deedle Namespace
Contents
- Other namespace members
- Core frame and series types
- Frame and series operations
- Parameters and results of various operations
- Primitive types and values
- Specialized frame and series types
- Vectors and indices
Other namespace members
| Type/Module | Description |
|
|
|
|
|
|
|
Defines non-generic `Vector` type that provides functions for building vectors
(hard-bound to `ArrayVectorBuilder` type). In F#, the module is automatically opened
using `AutoOpen`. The methods are not designed for the use from C#.
|
|
|
Module with extensions for generic vector type. Given `vec` of type `IVector |
|
|
Set concrete IVectorBuilder implementation
|
|
|
A sequence of indicies together with the total number. Use `RangeRestriction.ofSeq` to create one from a sequence. This can be implemented by concrete vector/index builders to allow further optimizations (e.g. when the underlying source directly supports range operations). For example, if your source has an optimised way for getting every 10th address, you can create your own `IRangeRestriction` and then check for it in `LookupRange` and use optimised implementation rather than actually iterating over the sequence of indices. |
|
|
Represents an (untyped) vector that stores some values and provides access
to the values via a generic address. This type should be only used directly when
extending the DataFrame library and adding a new way of storing or loading data.
To allow invocation via Reflection, the vector exposes type of elements as `System.Type`.
|
|
|
A generic, typed vector. Represents mapping from addresses to values of type `T`.
The vector provides a minimal interface that is required by series and can be
implemented in a number of ways to provide vector backed by database or an
alternative representation of data.
|
|
|
Represents a location in a vector. In general, we always know the address, but
sometimes (BigDeedle) it is hard to get the offset (requires some data lookups),
so we use this interface to delay the calculation of the Offset (which is mainly
needed in one of the `series.Select` overloads)
|
|
|
Provides additional operations for working with the `RangeRestriction<'TAddress>` type |
|
|
Specifies a sub-range within index that can be accessed via slicing (see the `GetAddressRange` method). For in-memory data structures, accessing range via known addresses is typically sufficient, but for virtual Big Deedle sources, `Start` and `End` let us avoid fully evaluating addresses. `Custom` range can be used for optimizations. |
|
|
The type implements C# and F# extension methods that add numerical operations
to Deedle series.
|
|
|
|
|
|
Represents a generic function `\forall.'T.(IVector<'T> -> 'R)`. The function can be
generically invoked on an argument of type `IVector` using `IVector.Invoke`
|
Core frame and series types
| Type/Module | Description |
|
Contains extensions for creating values of type |
|
|
A frame is the key Deedle data structure (together with series). It represents a data table (think spreadsheet or CSV file) with multiple rows and columns. The frame consists of row index, column index and data. The indices are used for efficient lookup when accessing data by the row key `'TRowKey` or by the column key `'TColumnKey`. Deedle frames are optimized for the scenario when all values in a given column are of the same type (but types of different columns can differ). |
|
|
Represents the underlying (raw) data of the frame in a format that can be used for exporting data frame to other formats etc. (DataTable, CSV, Excel) |
|
|
Represents an untyped series with keys of type `K` and values of some unknown type (This type should not generally be used directly, but it can be used when you need to write code that works on a sequence of series of heterogeneous types). |
|
|
The type |
Frame and series operations
| Type/Module | Description |
|
Contains C#-friendly extension methods for various instances of `IEnumerable`
that can be used for creating |
|
|
This module contains F# functions and extensions for working with frames. This includes operations for creating frames such as the `frame` function, `=>` operator and `Frame.ofRows`, `Frame.ofColumns` and `Frame.ofRowKeys` functions. The module also provides additional F# extension methods including `ReadCsv`, `SaveCsv` and `PivotTable`. |
|
|
The `Frame` module provides an F#-friendly API for working with data frames.
The module follows the usual desing for collection-processing in F#, so the
functions work well with the pipelining operator (`|>`). For example, given
a frame with two columns representing prices, we can use `Frame.pctChange` to
calculate daily returns like this:
let df = frame [ "MSFT" => prices1; "AAPL" => prices2 ]
let rets = df |> Frame.pctChange 1
rets |> Stats.mean
Note that the `Stats.mean` operation is overloaded and works both on series
(returning a number) and on frames (returning a series). You can also use
`Frame.diff` if you need absolute differences rather than relative changes.
The functions in this module are designed to be used from F#. For a C#-friendly
API, see the `FrameExtensions` type. For working with individual series, see the
`Series` module. The functions in the `Frame` module are grouped in a number of
categories and documented below.
Accessing frame data and lookup
-------------------------------
Functions in this category provide access to the values in the fame. You can
also add and remove columns from a frame (which both return a new value).
- `addCol`, `replaceCol` and `dropCol` can be used to create a new data frame
with a new column, by replacing an existing column with a new one, or by dropping
an existing column
- `cols` and `rows` return the columns or rows of a frame as a series containing
objects; `getCols` and `getRows` return a generic series and cast the values to
the type inferred from the context (columns or rows of incompatible types are skipped);
`getNumericCols` returns columns of a type convertible to `float` for convenience.
- You can get a specific row or column using `get[Col|Row]` or `lookup[Col|Row]` functions.
The `lookup` variant lets you specify lookup behavior for key matching (e.g. find the
nearest smaller key than the specified value). There are also `[try]get` and `[try]Lookup`
functions that return optional values and functions returning entire observations
(key together with the series).
- `sliceCols` and `sliceRows` return a sub-frame containing only the specified columns
or rows. Finally, `toArray2D` returns the frame data as a 2D array.
Grouping, windowing and chunking
--------------------------------
The basic grouping functions in this category can be used to group the rows of a
data frame by a specified projection or column to create a frame with hierarchical
index such as |
|
|
Provides static methods for creating frames, reading frame data from CSV files and database (via IDataReader). The type also provides global configuration for reflection-based expansion. |
|
|
Type that can be used for creating frames using the C# collection initializer syntax.
You can use |
|
|
Contains C# and F# extension methods for the `Frame<'R, 'C>` type. The members are automatically available when you import the `Deedle` namespace. The type contains object-oriented counterparts to most of the functionality from the `Frame` module. |
|
|
The type implements C# and F# extension methods that add numerical operations
to Deedle series. With a few exceptions, the methods are only available for
series containing floating-point values, that is |
|
|
The `Series` module provides an F#-friendly API for working with data and time series.
The API follows the usual design for collection-processing in F#, so the functions work
well with the pipelining ( |
|
|
The type implements C# and F# extension methods for the |
|
|
The `Stats` type contains functions for fast calculation of statistics over series and frames as well as over a moving and an expanding window in a series. The resulting series has the same keys as the input series. When there are no values, or missing values, different functions behave in different ways. Statistics (e.g. mean) return missing value when any value is missing, while min/max functions return the minimal/maximal element (skipping over missing values). |
Parameters and results of various operations
| Type/Module | Description |
|
A non-generic type that simplifies the construction of |
|
|
Represents a strategy for aggregating data in an ordered series into data segments. To create a value of this type from C#, use the non-generic `Aggregation` type. Data can be aggregate using floating windows or chunks of a specified size or by specifying a condition on two keys (i.e. end a window/chunk when the condition no longer holds). |
|
|
Represents boundary behaviour for operations such as floating window. The type specifies whether incomplete windows (of smaller than required length) should be produced at the beginning (`AtBeginning`) or at the end (`AtEnding`) or skipped (`Skip`). For chunking, combinations are allowed too - to skip incomplete chunk at the beginning, use `Boundary.Skip ||| Boundary.AtBeginning`. |
|
|
Represents different kinds of type conversions that can be used by Deedle internally.
This is used, for example, when converting |
|
|
Provides helper functions and active patterns for working with `DataSegment` values |
|
|
Represents a segment of a series or sequence. The value is returned from various functions that aggregate data into chunks or floating windows. The `Complete` case represents complete segment (e.g. of the specified size) and `Boundary` represents segment at the boundary (e.g. smaller than the required size). |
|
|
Represents a kind of |
|
|
Specifies in which direction should we look when performing operations such as `Series.Pairwise`. |
|
|
Represents a special lookup. This can be used to support hierarchical or duplicate keys
in an index. A key type `K` can come with associated |
|
|
This enumeration specifies joining behavior for `Join` method provided by `Series` and `Frame`. Outer join unions the keys (and may introduce missing values), inner join takes the intersection of keys; left and right joins take the keys of the first or the second series/frame. |
|
|
Represents different behaviors of key lookup in series. For unordered series, the only available option is `Lookup.Exact` which finds the exact key - methods fail or return missing value if the key is not available in the index. For ordered series `Lookup.Greater` finds the first greater key (e.g. later date) with a value. `Lookup.Smaller` searches for the first smaller key. The options `Lookup.ExactOrGreater` and `Lookup.ExactOrSmaller` finds the exact key (if it is present) and otherwise search for the nearest larger or smaller key, respectively. |
|
|
F#-friendly functions for creating multi-level keys and lookups |
|
|
This enumeration specifies the behavior of `Union` operation on series when there are overlapping keys in two series that are being unioned. The options include preferring values from the left/right series or throwing an exception when both values are available. |
Primitive types and values
| Type/Module | Description |
|
A type with extension method for |
|
|
Thrown when a value at the specified index does not exist in the data frame or series. This exception is thrown only when the key is defined, but the value is not available, in other situations `KeyNotFoundException` is thrown |
|
|
A type alias for the |
|
|
Provides various helper functions for using the |
|
|
Non-generic type that makes it easier to create |
|
|
Value type that represents a potentially missing value. This is similar to
|
|
|
Extension methods for working with optional values from C#. These make it easier to provide default values and convert optional values to `Nullable` (when the contained value is value type) |
|
|
Module with helper functions for extracting values from hierarchical tuples |
|
|
A type alias for the |
|
|
Represents a value or an exception. This type is used by functions such as `Series.tryMap` and `Frame.tryMap` to capture the result of a lambda function, which may be either a value or an exception. The type is a discriminated union, so it can be processed using F# pattern matching, or using `Value`, `HasValue` and `Exception` properties |
Specialized frame and series types
| Type | Description |
|
Represents a series of columns from a frame. The type inherits from a series of
series representing individual columns ( |
|
|
This type exposes a single static method `DelayedSeries.Create` that can be used for
constructing data series (of type |
|
|
An empty interface that is implemented by |
|
|
Represents an operation that can be invoked on |
|
|
Represents a series containing boxed values. This type is inherited from |
|
|
Represents a series of rows from a frame. The type inherits from a series of
series representing individual rows ( |
|
|
A simple class that inherits from |
|
|
The type can be used for creating series using mutation. You can add
items using `Add` and get the resulting series using the |
Vectors and indices
| Type/Module | Description |
|
An `Address` value is used as an interface between vectors and indices. The index maps keys of various types to address, which is then used to get a value from the vector. Here is a brief summary of what we assume (and don't assume) about addresses: - Address is `int64` (although we might need to generalize this in the future) - Different data sources can use different addressing schemes (as long as both index and vector use the same scheme) - Addresses don't have to be continuous (e.g. if the source is partitioned, it can use 32bit partition index + 32bit offset in the partition) - In the in-memory representation, address is just index into an array - In the BigDeedle representation, address is abstracted and comes with `AddressOperations` that specifies how to use it (tests use linear offset and partitioned representation) |
|
|
Defines non-generic `Index` type that provides functions for building indices (hard-bound to `LinearIndexBuilder` type). In F#, the module is automatically opened using `AutoOpen`. The methods are not designed for the use from C#. |
|
|
Set concrete IIndexBuilder implementation |
|
|
Type that provides access to creating indices (represented as `LinearIndex` values) |
Deedle