renumics.spotlight.dataset

dataset

This module provides Spotlight dataset.

Dataset

Spotlight dataset.

filepath property

filepath: str

Dataset file name.

mode property

mode: str

Dataset file open mode.

open

open(mode: Optional[str] = None) -> None

Open previously closed file or reopen file with another mode.

Parameters:

Name	Type	Description	Default
mode	Optional[str]	Optional open mode. If not given, use self.mode.	None

close

close() -> None

Close file.

keys

keys() -> List[str]

Get dataset column names.

iterrows

iterrows() -> Iterable[Dict[str, Optional[OutputType]]]

iterrows(
    column_names: Union[str, Iterable[str]],
) -> Union[
    Iterable[Dict[str, Optional[OutputType]]],
    Iterable[Optional[OutputType]],
]

iterrows(
    column_names: Optional[
        Union[str, Iterable[str]]
    ] = None,
) -> Union[
    Iterable[Dict[str, Optional[OutputType]]],
    Iterable[Optional[OutputType]],
]

Iterate through dataset rows.

from_pandas

from_pandas(
    df: DataFrame,
    index: bool = False,
    dtypes: Optional[Dict[str, Any]] = None,
    workdir: Optional[PathType] = None,
) -> None

Import a pandas dataframe to the dataset.

Only scalar types supported by the Spotlight dataset are imported, the other are printed in a warning message.

Parameters:

Name	Type	Description	Default
df	DataFrame	pandas.DataFrame to import.	required
index	bool	Whether to import index of the dataframe as regular dataset column.	False
dtypes	Optional[Dict[str, Any]]	Optional dict with mapping column name -> column type with column types allowed by Spotlight.	None
workdir	Optional[PathType]	Optional folder where audio/images/meshes are stored. If None, current folder is used.	None

Example

from datetime import datetime import pandas as pd from renumics.spotlight import Dataset df = pd.DataFrame( ... { ... "bools": [True, False, False], ... "ints": [-1, 0, 1], ... "floats": [-1.0, 0.0, 1.0], ... "strings": ["a", "b", "c"], ... "datetimes": datetime.now().astimezone(), ... } ... ) with Dataset("docs/example.h5", "w") as dataset: ... dataset.from_pandas(df, index=False) with Dataset("docs/example.h5", "r") as dataset: ... print(len(dataset)) ... print(sorted(dataset.keys())) 3 ['bools', 'datetimes', 'floats', 'ints', 'strings']

from_csv

from_csv(
    filepath: PathType,
    dtypes: Optional[Dict[str, Any]] = None,
    columns: Optional[Iterable[str]] = None,
    workdir: Optional[PathType] = None,
) -> None

Parameters:

Name	Type	Description	Default
filepath	PathType	Path of csv file to read.	required
dtypes	Optional[Dict[str, Any]]	Optional dict with mapping column name -> column type with column types allowed by Spotlight.	None
columns	Optional[Iterable[str]]	Optional columns to read from csv. If not set, read all columns.	None
workdir	Optional[PathType]	Optional folder where audio/images/meshes are stored. If None, csv folder is used.	None

to_pandas

to_pandas() -> DataFrame

Export the dataset to pandas dataframe.

Only scalar types of the Spotlight dataset are exported, the others are printed in a warning message.

Returns:

Type	Description
DataFrame	pandas.DataFrame filled with the data of the Spotlight dataset.

Example

import pandas as pd from renumics.spotlight import Dataset with Dataset("docs/example.h5", "w") as dataset: ... dataset.append_bool_column("bools", [True, False, False]) ... dataset.append_int_column("ints", [-1, 0, 1]) ... dataset.append_float_column("floats", [-1.0, 0.0, 1.0]) ... dataset.append_string_column("strings", ["a", "b", "c"]) ... dataset.append_datetime_column("datetimes", optional=True) with Dataset("docs/example.h5", "r") as dataset: ... df = dataset.to_pandas() print(len(df)) 3 print(df.columns.sort_values()) Index(['bools', 'datetimes', 'floats', 'ints', 'strings'], dtype='object')

append_bool_column

append_bool_column(
    name: str,
    values: Union[
        BoolColumnInputType, Iterable[BoolColumnInputType]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: BoolColumnInputType = False,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    editable: bool = True,
) -> None

Create and optionally fill a boolean column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[BoolColumnInputType, Iterable[BoolColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional.	False
default	BoolColumnInputType	Value to use by default if column is optional and no value or None is given.	False
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
editable	bool	Whether column is editable in Spotlight.	True

Example

from renumics.spotlight import Dataset value = False with Dataset("docs/example.h5", "w") as dataset: ... dataset.append_bool_column("bool_values", 5*[value]) with Dataset("docs/example.h5", "r") as dataset: ... print(dataset["bool_values", 2]) False

append_int_column

append_int_column(
    name: str,
    values: Union[
        IntColumnInputType, Iterable[IntColumnInputType]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: IntColumnInputType = -1,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    editable: bool = True,
) -> None

Create and optionally fill an integer column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[IntColumnInputType, Iterable[IntColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional.	False
default	IntColumnInputType	Value to use by default if column is optional and no value or None is given.	-1
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
editable	bool	Whether column is editable in Spotlight.	True

Example

Find a similar example usage in renumics.spotlight.dataset.Dataset.append_bool_column.

append_float_column

append_float_column(
    name: str,
    values: Union[
        FloatColumnInputType, Iterable[FloatColumnInputType]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: FloatColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    editable: bool = True,
) -> None

Create and optionally fill a float column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[FloatColumnInputType, Iterable[FloatColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than NaN is specified, optional is automatically set to True.	False
default	FloatColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
editable	bool	Whether column is editable in Spotlight.	True

Example

Find a similar example usage in renumics.spotlight.dataset.Dataset.append_bool_column.

append_string_column

append_string_column(
    name: str,
    values: Union[
        StringColumnInputType,
        Iterable[StringColumnInputType],
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: Optional[str] = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    editable: bool = True,
) -> None

Create and optionally fill a float column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[StringColumnInputType, Iterable[StringColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than empty string is specified, optional is automatically set to True.	False
default	Optional[str]	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
editable	bool	Whether column is editable in Spotlight.	True

Example

Find a similar example usage in renumics.spotlight.dataset.Dataset.append_bool_column.

append_datetime_column

append_datetime_column(
    name: str,
    values: Union[
        DatetimeColumnInputType,
        Iterable[DatetimeColumnInputType],
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: DatetimeColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
) -> None

Create and optionally fill a datetime column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[DatetimeColumnInputType, Iterable[DatetimeColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	DatetimeColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None

Example

import numpy as np import datetime from renumics.spotlight import Dataset date = datetime.datetime.now() with Dataset("docs/example.h5", "w") as dataset: ... dataset.append_datetime_column("dates", 5*[date]) with Dataset("docs/example.h5", "r") as dataset: ... print(dataset["dates", 2] < datetime.datetime.now()) True

append_array_column

append_array_column(
    name: str,
    values: Union[
        ArrayColumnInputType, Iterable[ArrayColumnInputType]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: ArrayColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
) -> None

Create and optionally fill a numpy array column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[ArrayColumnInputType, Iterable[ArrayColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	ArrayColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None

Example

import numpy as np from renumics.spotlight import Dataset array_data = np.random.rand(5,3) with Dataset("docs/example.h5", "w") as dataset: ... dataset.append_array_column("arrays", 5*[array_data]) with Dataset("docs/example.h5", "r") as dataset: ... print(dataset["arrays", 2].shape) (5, 3)

append_categorical_column

append_categorical_column(
    name: str,
    values: Union[
        CategoricalColumnInputType,
        Iterable[CategoricalColumnInputType],
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: Optional[str] = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    editable: bool = True,
    categories: Optional[
        Union[Iterable[str], Dict[str, int]]
    ] = None,
) -> None

Create and optionally fill a categorical column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
categories	Optional[Union[Iterable[str], Dict[str, int]]]	The allowed categories for this column ("" is not allowed)	None
values	Union[CategoricalColumnInputType, Iterable[CategoricalColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than empty string is specified, optional is automatically set to True.	False
default	Optional[str]	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
editable	bool	Whether column is editable in Spotlight.	True

Example

Find an example usage in renumics.spotlight.dtypes.Category.

append_embedding_column

append_embedding_column(
    name: str,
    values: Union[
        EmbeddingColumnInputType,
        Iterable[EmbeddingColumnInputType],
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: EmbeddingColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    length: Optional[int] = None,
    dtype: Union[str, dtype] = "float32",
) -> None

Create and optionally fill a mesh column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[EmbeddingColumnInputType, Iterable[EmbeddingColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	EmbeddingColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
dtype	Union[str, dtype]	A valid float numpy dtype. Default is "float32".	'float32'

Example

Find an example usage in renumics.spotlight.dtypes.Embedding.

append_sequence_1d_column

append_sequence_1d_column(
    name: str,
    values: Union[
        Sequence1DColumnInputType,
        Iterable[Sequence1DColumnInputType],
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: Sequence1DColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    x_label: Optional[str] = None,
    y_label: Optional[str] = None,
) -> None

Create and optionally fill a 1d-sequence column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[Sequence1DColumnInputType, Iterable[Sequence1DColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	Sequence1DColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
x_label	Optional[str]	Optional x-axis label.	None
y_label	Optional[str]	Optional y-axis label. If None, column name is taken.	None

Example

Find an example usage in renumics.spotlight.dtypes.Sequence1D.

append_mesh_column

append_mesh_column(
    name: str,
    values: Optional[
        Union[
            MeshColumnInputType,
            Iterable[Optional[MeshColumnInputType]],
        ]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: Optional[MeshColumnInputType] = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    lookup: Optional[
        Union[
            BoolType,
            Iterable[MeshColumnInputType],
            Dict[str, MeshColumnInputType],
        ]
    ] = None,
    external: bool = False,
) -> None

Create and optionally fill a mesh column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Optional[Union[MeshColumnInputType, Iterable[Optional[MeshColumnInputType]]]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	Optional[MeshColumnInputType]	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
lookup	Optional[Union[BoolType, Iterable[MeshColumnInputType], Dict[str, MeshColumnInputType]]]	Optional data lookup/flag for automatic lookup creation. If False (default if external is True), never add data to lookup. If True (default if external is False), add all given files to the lookup, do nothing for explicitly given data. If lookup is given, store it explicit, further behaviour is as for True. If lookup is not a dict, keys are created automatically.	None
external	bool	Whether column should only contain paths/URLs to data and load it on demand.	False

Example

Find an example usage in renumics.spotlight.dtypes.Mesh.

append_image_column

append_image_column(
    name: str,
    values: Union[
        ImageColumnInputType, Iterable[ImageColumnInputType]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: ImageColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    lookup: Optional[
        Union[
            BoolType,
            Iterable[MeshColumnInputType],
            Dict[str, MeshColumnInputType],
        ]
    ] = None,
    external: bool = False,
) -> None

Create and optionally fill an image column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Union[ImageColumnInputType, Iterable[ImageColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	ImageColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
lookup	Optional[Union[BoolType, Iterable[MeshColumnInputType], Dict[str, MeshColumnInputType]]]	Optional data lookup/flag for automatic lookup creation. If False (default if external is True), never add data to lookup. If True (default if external is False), add all given files to the lookup, do nothing for explicitly given data. If lookup is given, store it explicit, further behaviour is as for True. If lookup is not a dict, keys are created automatically.	None
external	bool	Whether column should only contain paths/URLs to data and load it on demand.	False

Example

Find an example usage in renumics.spotlight.dtypes.Image.

append_audio_column

append_audio_column(
    name: str,
    values: Optional[
        Union[
            AudioColumnInputType,
            Iterable[AudioColumnInputType],
        ]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: ImageColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    lookup: Optional[
        Union[
            BoolType,
            Iterable[MeshColumnInputType],
            Dict[str, MeshColumnInputType],
        ]
    ] = None,
    external: bool = False,
    lossy: Optional[bool] = None,
) -> None

Create and optionally fill an audio column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Optional[Union[AudioColumnInputType, Iterable[AudioColumnInputType]]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	ImageColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
lookup	Optional[Union[BoolType, Iterable[MeshColumnInputType], Dict[str, MeshColumnInputType]]]	Optional data lookup/flag for automatic lookup creation. If False (default if external is True), never add data to lookup. If True (default if external is False), add all given files to the lookup, do nothing for explicitly given data. If lookup is given, store it explicit, further behaviour is as for True. If lookup is not a dict, keys are created automatically.	None
external	bool	Whether column should only contain paths/URLs to data and load it on demand.	False
lossy	Optional[bool]	Whether to store data lossy or lossless (default if external is False). Not recomended to use with external=True since it requires on demand transcoding which slows down the execution.	None

Example

Find an example usage in renumics.spotlight.media.Audio.

append_video_column

append_video_column(
    name: str,
    values: Optional[
        Union[
            VideoColumnInputType,
            Iterable[VideoColumnInputType],
        ]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: VideoColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    lookup: Optional[
        Union[
            BoolType,
            Iterable[MeshColumnInputType],
            Dict[str, MeshColumnInputType],
        ]
    ] = None,
    external: bool = False,
) -> None

Create and optionally fill an video column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Optional[Union[VideoColumnInputType, Iterable[VideoColumnInputType]]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	VideoColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
lookup	Optional[Union[BoolType, Iterable[MeshColumnInputType], Dict[str, MeshColumnInputType]]]	Optional data lookup/flag for automatic lookup creation. If False (default if external is True), never add data to lookup. If True (default if external is False), add all given files to the lookup, do nothing for explicitly given data. If lookup is given, store it explicit, further behaviour is as for True. If lookup is not a dict, keys are created automatically.	None
external	bool	Whether column should only contain paths/URLs to data and load it on demand.	False

append_window_column

append_window_column(
    name: str,
    values: Optional[
        Union[
            WindowColumnInputType,
            Iterable[WindowColumnInputType],
        ]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: WindowColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    editable: bool = True,
) -> None

Create and optionally fill window column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Optional[Union[WindowColumnInputType, Iterable[WindowColumnInputType]]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	WindowColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
editable	bool	Whether column is editable in Spotlight.	True

Example

Find an example usage in renumics.spotlight.dtypes.Window.

append_bounding_box_column

append_bounding_box_column(
    name: str,
    values: Optional[
        Union[
            BoundingBoxColumnInputType,
            Iterable[BoundingBoxColumnInputType],
        ]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: BoundingBoxColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    editable: bool = True,
) -> None

Create and optionally fill axis-aligned bounding box column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
values	Optional[Union[BoundingBoxColumnInputType, Iterable[BoundingBoxColumnInputType]]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	BoundingBoxColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
editable	bool	Whether column is editable in Spotlight.	True

append_column

append_column(
    name: str,
    dtype: Any,
    values: Union[
        ColumnInputType, Iterable[ColumnInputType]
    ] = None,
    order: Optional[int] = None,
    hidden: bool = False,
    optional: bool = False,
    default: ColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    **attrs: Any,
) -> None

Create and optionally fill a dataset column of the given type.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
dtype	Any	Column type.	required
values	Union[ColumnInputType, Iterable[ColumnInputType]]	Optional column values. If a single value, the whole column filled with this value.	None
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	bool	Whether column is hidden in Spotlight.	False
optional	bool	Whether column is optional. If default other than None is specified, optional is automatically set to True.	False
default	ColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
attrs	Any	Optional arguments for the respective append column method.	{}

Example

>>> from renumics.spotlight import Dataset
>>> with Dataset("docs/example.h5", "w") as dataset:
...     dataset.append_column("int", int, range(5))
...     dataset.append_column("float", float, 1.0)
...     dataset.append_column("bool", bool, True)
>>> with Dataset("docs/example.h5", "r") as dataset:
...     print(len(dataset))
...     print(sorted(dataset.keys()))
5
['bool', 'float', 'int']
>>> with Dataset("docs/example.h5", "r") as dataset:
...     print(dataset["int"])
...     print(dataset["bool"])
...     print(dataset["float"])
[0 1 2 3 4]
[ True  True  True  True  True]
[1. 1. 1. 1. 1.]

append_row

append_row(**values: ColumnInputType) -> None

Append a row to the dataset.

Parameters:

Name	Type	Description	Default
values	ColumnInputType	A mapping column name -> value. Keys of values should match dataset column names exactly except for optional columns.	{}

Example

from renumics.spotlight import Dataset with Dataset("docs/example.h5", "w") as dataset: ... dataset.append_bool_column("bool_values") ... dataset.append_float_column("float_values") data = {"bool_values":True, "float_values":0.2} with Dataset("docs/example.h5", "a") as dataset: ... dataset.append_row(data) ... dataset.append_row(data) ... print(dataset["float_values", 1]) 0.2

append_dataset

append_dataset(dataset: Dataset) -> None

Append a dataset to the current dataset row-wise.

insert_row

insert_row(
    index: IndexType, values: Dict[str, ColumnInputType]
) -> None

Insert a row into the dataset at the given index.

Example

>>> from renumics.spotlight import Dataset
>>> with Dataset("example.h5", "w") as dataset:
...     dataset.append_float_column("floats", [-1.0, 0.0, 1.0])
...     dataset.append_int_column("ints", [-1, 0, 2])
...     print(len(dataset))
...     print(dataset["floats"])
...     print(dataset["ints"])
3
[-1.  0.  1.]
[-1  0  2]
>>> with Dataset("example.h5", "a") as dataset:
...     dataset.insert_row(2, {"floats": float("nan"), "ints": 1000})
...     dataset.insert_row(-3, {"floats": 3.14, "ints": -1000})
...     print(len(dataset))
...     print(dataset["floats"])
...     print(dataset["ints"])
5
[-1.    3.14  0.     nan  1.  ]
[   -1 -1000     0  1000     2]

pop

pop(item: str) -> ndarray

pop(item: IndexType) -> Dict[str, Optional[OutputType]]

pop(
    item: Union[str, IndexType],
) -> Union[ndarray, Dict[str, Optional[OutputType]]]

Delete a dataset column or row and return it.

isnull

isnull(column_name: str) -> ndarray

Get missing values mask for the given column.

None, NaN and category "" values are mapped to True. So null-mask for columns of type bool, int and string always has only False values. A Window is mapped on True only if both start and end are NaN.

notnull

notnull(column_name: str) -> ndarray

Get non-missing values mask for the given column.

None, NaN and category "" values are mapped to False. So non-null-mask for columns of type bool, int and string always has only True values. A Window is mapped on True if at least one of its values is not NaN.

rename_column

rename_column(old_name: str, new_name: str) -> None

Rename a dataset column.

rebuild

rebuild() -> None

Update old-style columns in the dataset. Be aware, that it can take some time and memory. It is useful to do prune after rebuild.

prune

prune() -> None

Rebuild the whole dataset with the same content.

This method can be useful after column deletions, in order to decrease the dataset file size.

get_dtype

get_dtype(column_name: str) -> DType

Get type of dataset column.

Parameters:

Name	Type	Description	Default
column_name	str	Column name.	required

Example

from renumics.spotlight import Dataset with Dataset("docs/example.h5", "w") as dataset: ... dataset.append_bool_column("bool") ... dataset.append_datetime_column("datetime") ... dataset.append_array_column("array") ... dataset.append_mesh_column("mesh") with Dataset("docs/example.h5", "r") as dataset: ... for column_name in sorted(dataset.keys()): ... print(column_name, dataset.get_dtype(column_name)) array array bool bool datetime datetime mesh Mesh

get_column_attributes

get_column_attributes(name: str) -> Dict[str, Any]

Get attributes of a column. Available but unset attributes contain None.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required

Example

from renumics.spotlight import Dataset with Dataset("docs/example.h5", "w") as dataset: ... dataset.append_int_column("int", range(5)) ... dataset.append_int_column( ... "int1", ... hidden=True, ... default=10, ... description="integer column", ... tags=["important"], ... editable=False, ... ) with Dataset("docs/example.h5", "r") as dataset: ... attributes = dataset.get_column_attributes("int") ... for key in sorted(attributes.keys()): ... print(key, attributes[key]) default -1 description None editable True hidden False optional True order None tags None with Dataset("docs/example.h5", "r") as dataset: ... attributes = dataset.get_column_attributes("int1") ... for key in sorted(attributes.keys()): ... print(key, attributes[key]) default 10 description integer column editable False hidden True optional True order None tags ['important']

set_column_attributes

set_column_attributes(
    name: str,
    order: Optional[int] = None,
    hidden: Optional[bool] = None,
    optional: Optional[bool] = None,
    default: ColumnInputType = None,
    description: Optional[str] = None,
    tags: Optional[List[str]] = None,
    **attrs: Any,
) -> None

Set attributes of a column.

Parameters:

Name	Type	Description	Default
name	str	Column name.	required
order	Optional[int]	Optional Spotlight priority order value. None means the lowest priority.	None
hidden	Optional[bool]	Whether column is hidden in Spotlight.	None
optional	Optional[bool]	Whether column is optional. If default other than None is specified, optional is automatically set to True.	None
default	ColumnInputType	Value to use by default if column is optional and no value or None is given.	None
description	Optional[str]	Optional column description.	None
tags	Optional[List[str]]	Optional tags for the column.	None
attrs	Any	Optional more DType specific attributes .	{}

check_column_name staticmethod

check_column_name(name: str) -> None

Check a column name.

get_current_datetime

get_current_datetime() -> datetime

Get current datetime with timezone.

escape_dataset_name

escape_dataset_name(name: str) -> str

Replace "\" with "\" and "/" with "\s".

unescape_dataset_name

unescape_dataset_name(escaped_name: str) -> str

Replace "\" with "\" and "\s" with "/".