Skip to content

roverd.types

Data Type Library.

Types are generic dataclasses

Following the recommendations of the abstract dataloader, each type is a generic dataclass of arrays.

This means that each type can be specified with a particular array implementation, e.g., 

CameraData[np.ndarray], CameraData[jnp.ndarray], CameraData[torch.Tensor]
or left without any specific annotation to implicitly allow any backend to be used, i.e., 
CameraData = CameraData[np.ndarray | jnp.ndarray | torch.Tensor | ...]
Finally, callables accepting these types can be annotated using a type variable to designate that the callable preserves the array type: 
from abstract_dataloader.ext.types import TArray

def example_generic_fn(data: CameraData[TArray]) -> CameraData[TArray]:
    # if data is CameraData[np.ndarray], we return CameraData[np.ndarray]
    ...

Each type in this library is a dataclass of arrays:

  • Each entry has a leading batch axis, which is usually 1 when loading single samples. Some entries are annotated with #batch to indicate that broadcasting is allowed on this axis, e.g., all samples in the same batch having the same range_resolution.
  • Each entry has a second t axis, which indicates the time dimension for sequences of frames. If only single frames are loaded, this is just 1.

roverd.types.CameraData

Bases: Generic[TArray]

Video data.

Named Axes

  • height: image height.
  • width: image width.

Attributes:

Name Type Description
image UInt8[TArray, 'batch t height width 3']

raw image data in HWC format.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
Source code in format/src/roverd/types.py 
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
@dataclass
class CameraData(Generic[TArray]):
    """Video data.

    !!! info "Named Axes"
        - `height`: image height.
        - `width`: image width.

    Attributes:
        image: raw image data in HWC format.
        timestamps: timestamp for each frame; nominally in seconds.
    """

    image: UInt8[TArray, "batch t height width 3"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.CameraSemseg

Bases: Generic[TArray]

Camera-based semantic segmentation.

Named Axes

  • height: image height.
  • width: image width.

Attributes:

Name Type Description
semseg UInt8[TArray, 'batch t height width']

segmentation classes.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
Source code in format/src/roverd/types.py 
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
@dataclass
class CameraSemseg(Generic[TArray]):
    """Camera-based semantic segmentation.

    !!! info "Named Axes"
        - `height`: image height.
        - `width`: image width.

    Attributes:
        semseg: segmentation classes.
        timestamps: timestamp for each frame; nominally in seconds.
    """

    semseg: UInt8[TArray, "batch t height width"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.Depth

Bases: Generic[TArray]

Generic raw depth data.

Named Axes

  • beam: lidar beam.
  • time: index within each frame where the range measurements are taken.

Attributes:

Name Type Description
rng UInt16[TArray, 'batch t beam time']

raw range measurements in beam-time space; nominally in mm.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
Source code in format/src/roverd/types.py 
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
@dataclass
class Depth(Generic[TArray]):
    """Generic raw depth data.

    !!! info "Named Axes"
        - `beam`: lidar beam.
        - `time`: index within each frame where the range measurements are
            taken.

    Attributes:
        rng: raw range measurements in beam-time space; nominally in mm.
        timestamps: timestamp for each frame; nominally in seconds.
    """

    rng: UInt16[TArray, "batch t beam time"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.IMUData

Bases: Generic[TArray]

IMU data.

Attributes:

Name Type Description
acc Float32[TArray, 'batch t 3']

linear acceleration, in m/s^2.
rot Float32[TArray, 'batch t 3']

orientation (as euler angles).
avel Float32[TArray, 'batch t 3']

angular velocity.
timestamps Float64[TArray, 'batch t']

timestamp for each measurement; nominally in seconds.
Source code in format/src/roverd/types.py 
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
@dataclass
class IMUData(Generic[TArray]):
    """IMU data.

    Attributes:
        acc: linear acceleration, in m/s^2.
        rot: orientation (as euler angles).
        avel: angular velocity.
        timestamps: timestamp for each measurement; nominally in seconds.
    """

    acc: Float32[TArray, "batch t 3"]
    rot: Float32[TArray, "batch t 3"]
    avel: Float32[TArray, "batch t 3"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.OSData

Bases: Generic[TArray]

Depth and IR intensity data from an Ouster OSX sensor.

Named Axes

  • beam: lidar beam.
  • time: index within each frame where the range measurements are taken.

Warning

The measurements are recorded in time space, so are "staggered," and must be destaggered. See the ouster-sdk for details.

Attributes:

Name Type Description
rng UInt16[TArray, 'batch t beam time']

raw range measurements in beam-time space; nominally in mm.
rfl UInt8[TArray, 'batch t beam time']

reflectivity measurements (in NIR) for each beam (0-255).
nir UInt16[TArray, 'batch t beam time']

near-infrared ambient photos.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
intrinsics str

path to intrinsics file used by the Ouster SDK.
Source code in format/src/roverd/types.py 
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
@dataclass
class OSData(Generic[TArray]):
    """Depth and IR intensity data from an Ouster OSX sensor.

    !!! info "Named Axes"
        - `beam`: lidar beam.
        - `time`: index within each frame where the range measurements are
            taken.

    !!! warning

        The measurements are recorded in *time* space, so are "staggered,"
        and must be destaggered. See the [ouster-sdk](
        https://static.ouster.dev/sdk-docs/reference/lidar-scan.html#staggering-and-destaggering)
        for details.

    Attributes:
        rng: raw range measurements in beam-time space; nominally in mm.
        rfl: reflectivity measurements (in NIR) for each beam (0-255).
        nir: near-infrared ambient photos.
        timestamps: timestamp for each frame; nominally in seconds.
        intrinsics: path to intrinsics file used by the Ouster SDK.
    """

    rng: UInt16[TArray, "batch t beam time"]
    rfl: UInt8[TArray, "batch t beam time"]
    nir: UInt16[TArray, "batch t beam time"]
    timestamps: Float64[TArray, "batch t"]
    intrinsics: str

roverd.types.OSDepth

Bases: Generic[TArray]

Lidar raw depth data from an Ouster OSX sensor.

Warning

The measurements are recorded in time space, so are "staggered," and must be destaggered. See the ouster-sdk for details.

Attributes:

Name Type Description
rng UInt16[TArray, 'batch t beam time']

raw range measurements in beam-time space; nominally in mm.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
intrinsics str

path to intrinsics file used by the Ouster SDK.
Source code in format/src/roverd/types.py 
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
@dataclass
class OSDepth(Generic[TArray]):
    """Lidar raw depth data from an Ouster OSX sensor.

    !!! warning

        The measurements are recorded in *time* space, so are "staggered,"
        and must be destaggered. See the [ouster-sdk](
        https://static.ouster.dev/sdk-docs/reference/lidar-scan.html#staggering-and-destaggering)
        for details.

    Attributes:
        rng: raw range measurements in beam-time space; nominally in mm.
        timestamps: timestamp for each frame; nominally in seconds.
        intrinsics: path to intrinsics file used by the Ouster SDK.
    """

    rng: UInt16[TArray, "batch t beam time"]
    timestamps: Float64[TArray, "batch t"]
    intrinsics: str

roverd.types.PointCloud

Bases: Generic[TArray]

Generic point cloud data.

Named Axes

  • n: points within the point cloud.

Attributes:

Name Type Description
xyz Float32[TArray, 'batch t n 3']

point cloud coordinates, nominally in meters; zero-padded to the same number of points per frame.
length Integer[TArray, '#batch #t']

number of points in each point cloud.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
Source code in format/src/roverd/types.py 
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
@dataclass
class PointCloud(Generic[TArray]):
    """Generic point cloud data.

    !!! info "Named Axes"
        - `n`: points within the point cloud.

    Attributes:
        xyz: point cloud coordinates, nominally in meters; zero-padded to the
            same number of points per frame.
        length: number of points in each point cloud.
        timestamps: timestamp for each frame; nominally in seconds.
    """

    xyz: Float32[TArray, "batch t n 3"]
    length: Integer[TArray, "#batch #t"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.Pose

Bases: Generic[TArray]

Pose data.

Attributes:

Name Type Description
pos Float32[TArray, 'batch t 3']

position, in meters (front-left-up coordinate convention).
vel Float32[TArray, 'batch t 3']

velocity, in meters/second.
acc Float32[TArray, 'batch t 3']

acceleration, in meters/second^2.
rot Float32[TArray, 'batch t 3 3']

rotation (as matrix).
timestamps Float64[TArray, 'batch t']

timestamp for each measurement; nominally in seconds.
Source code in format/src/roverd/types.py 
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
@dataclass
class Pose(Generic[TArray]):
    """Pose data.

    Attributes:
        pos: position, in meters (front-left-up coordinate convention).
        vel: velocity, in meters/second.
        acc: acceleration, in meters/second^2.
        rot: rotation (as matrix).
        timestamps: timestamp for each measurement; nominally in seconds.
    """

    pos: Float32[TArray, "batch t 3"]
    vel: Float32[TArray, "batch t 3"]
    acc: Float32[TArray, "batch t 3"]
    rot: Float32[TArray, "batch t 3 3"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.XWR4DSpectrum

Bases: Generic[TArray]

4D radar spectrum data.

Named Axes

  • doppler: doppler bins.
  • elevation: elevation angle bins.
  • azimuth: azimuth angle bins.
  • range: range bins.

Attributes:

Name Type Description
spectrum Complex64[TArray, 'batch t doppler elevation azimuth range']

4D complex spectrum in doppler-elevation-azimuth-range order.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
range_resolution Float[TArray, '#batch']

range resolution for the modulation used; nominally in meters.
doppler_resolution Float[TArray, '#batch']

doppler resolution; nominally in m/s.
Source code in format/src/roverd/types.py 
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
@dataclass
class XWR4DSpectrum(Generic[TArray]):
    """4D radar spectrum data.

    !!! info "Named Axes"
        - `doppler`: doppler bins.
        - `elevation`: elevation angle bins.
        - `azimuth`: azimuth angle bins.
        - `range`: range bins.

    Attributes:
        spectrum: 4D complex spectrum in doppler-elevation-azimuth-range order.
        timestamps: timestamp for each frame; nominally in seconds.
        range_resolution: range resolution for the modulation used; nominally
            in meters.
        doppler_resolution: doppler resolution; nominally in m/s.
    """

    spectrum: Complex64[TArray, "batch t doppler elevation azimuth range"]
    timestamps: Float64[TArray, "batch t"]
    range_resolution: Float[TArray, "#batch"]
    doppler_resolution: Float[TArray, "#batch"]

roverd.types.XWRRadarIQ

Bases: Generic[TArray]

Raw I/Q data.

Named Axes

  • slow: slow time, i.e., number of chirps per frame.
  • tx: transmit antenna.
  • rx: receive antenna.
  • fast: fast time with interleaved IIQQ data, i.e., samples_per_chirp * 2.

Attributes:

Name Type Description
iq Int16[TArray, 'batch t slow tx rx fast']

raw I/Q data in slow-tx-rx-fast order; see the xwr documentation for details.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
range_resolution Float[TArray, '#batch']

range resolution for the modulation used; nominally in meters.
doppler_resolution Float[TArray, '#batch']

doppler resolution; nominally in m/s.
valid UInt8[TArray, 'batch t']

whether the entire frame is valid, or if the frame contains some zero-filled data due to dropped packets.
Source code in format/src/roverd/types.py 
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
@dataclass
class XWRRadarIQ(Generic[TArray]):
    """Raw I/Q data.

    !!! info "Named Axes"

        - `slow`: slow time, i.e., number of chirps per frame.
        - `tx`: transmit antenna.
        - `rx`: receive antenna.
        - `fast`: fast time with interleaved IIQQ data, i.e.,
            `samples_per_chirp * 2`.

    Attributes:
        iq: raw I/Q data in slow-tx-rx-fast order; see the
            [`xwr` documentation][xwr.capture.types.RadarFrame] for details.
        timestamps: timestamp for each frame; nominally in seconds.
        range_resolution: range resolution for the modulation used; nominally
            in meters.
        doppler_resolution: doppler resolution; nominally in m/s.
        valid: whether the entire frame is valid, or if the frame contains some
            zero-filled data due to dropped packets.
    """

    iq: Int16[TArray, "batch t slow tx rx fast"]
    timestamps: Float64[TArray, "batch t"]
    range_resolution: Float[TArray, "#batch"]
    doppler_resolution: Float[TArray, "#batch"]
    valid: UInt8[TArray, "batch t"]