Skip to content

roverd.types

Data Type Library.

Info

Following the recommendations of the abstract dataloader, each type is a generic dataclass of arrays.

Each type in this library is a dataclass of arrays:

  • Each entry has a leading batch axis, which is usually 1 when loading single samples. Some entries are annotated with #batch to indicate that broadcasting is allowed on this axis, e.g., all samples in the same batch having the same range_resolution.
  • Each entry has a second t axis, which indicates the time dimension for sequences of frames. If only single frames are loaded, this is just 1.

roverd.types.CameraData

Bases: Generic[TArray]

Video data.

Attributes:

Name Type Description
image UInt8[TArray, 'batch t height width 3']

raw image data in HWC format.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
Source code in format/src/roverd/types.py 
151
152
153
154
155
156
157
158
159
160
161
@dataclass
class CameraData(Generic[TArray]):
    """Video data.

    Attributes:
        image: raw image data in HWC format.
        timestamps: timestamp for each frame; nominally in seconds.
    """

    image: UInt8[TArray, "batch t height width 3"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.CameraSemseg

Bases: Generic[TArray]

Camera-based semantic segmentation.

Attributes:

Name Type Description
semseg UInt8[TArray, 'batch t height width']

segmentation classes.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
Source code in format/src/roverd/types.py 
164
165
166
167
168
169
170
171
172
173
174
@dataclass
class CameraSemseg(Generic[TArray]):
    """Camera-based semantic segmentation.

    Attributes:
        semseg: segmentation classes.
        timestamps: timestamp for each frame; nominally in seconds.
    """

    semseg: UInt8[TArray, "batch t height width"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.Depth

Bases: Generic[TArray]

Generic raw depth data.

Attributes:

Name Type Description
rng UInt16[TArray, 'batch t beam time']

raw range measurements in beam-time space; nominally in mm.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
Source code in format/src/roverd/types.py 
75
76
77
78
79
80
81
82
83
84
85
@dataclass
class Depth(Generic[TArray]):
    """Generic raw depth data.

    Attributes:
        rng: raw range measurements in beam-time space; nominally in mm.
        timestamps: timestamp for each frame; nominally in seconds.
    """

    rng: UInt16[TArray, "batch t beam time"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.IMUData

Bases: Generic[TArray]

IMU data.

Attributes:

Name Type Description
acc Float32[TArray, 'batch t 3']

linear acceleration, in m/s^2.
rot Float32[TArray, 'batch t 3']

orientation (as euler angles).
avel Float32[TArray, 'batch t 3']

angular velocity.
timestamps Float64[TArray, 'batch t']

timestamp for each measurement; nominally in seconds.
Source code in format/src/roverd/types.py 
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
@dataclass
class IMUData(Generic[TArray]):
    """IMU data.

    Attributes:
        acc: linear acceleration, in m/s^2.
        rot: orientation (as euler angles).
        avel: angular velocity.
        timestamps: timestamp for each measurement; nominally in seconds.
    """

    acc: Float32[TArray, "batch t 3"]
    rot: Float32[TArray, "batch t 3"]
    avel: Float32[TArray, "batch t 3"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.OSData

Bases: Generic[TArray]

Depth and IR intensity data from an Ouster OSX sensor.

Warning

The measurements are recorded in time space, so are "staggered," and must be destaggered. See the ouster-sdk for details.

Attributes:

Name Type Description
rng UInt16[TArray, 'batch t beam time']

raw range measurements in beam-time space; nominally in mm.
rfl UInt8[TArray, 'batch t beam time']

reflectivity measurements (in NIR) for each beam (0-255).
nir UInt16[TArray, 'batch t beam time']

near-infrared ambient photos.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
intrinsics str

path to intrinsics file used by the Ouster SDK.
Source code in format/src/roverd/types.py 
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
@dataclass
class OSData(Generic[TArray]):
    """Depth and IR intensity data from an Ouster OSX sensor.

    !!! warning

        The measurements are recorded in *time* space, so are "staggered,"
        and must be destaggered. See the [ouster-sdk](
        https://static.ouster.dev/sdk-docs/reference/lidar-scan.html#staggering-and-destaggering)
        for details.

    Attributes:
        rng: raw range measurements in beam-time space; nominally in mm.
        rfl: reflectivity measurements (in NIR) for each beam (0-255).
        nir: near-infrared ambient photos.
        timestamps: timestamp for each frame; nominally in seconds.
        intrinsics: path to intrinsics file used by the Ouster SDK.
    """

    rng: UInt16[TArray, "batch t beam time"]
    rfl: UInt8[TArray, "batch t beam time"]
    nir: UInt16[TArray, "batch t beam time"]
    timestamps: Float64[TArray, "batch t"]
    intrinsics: str

roverd.types.OSDepth

Bases: Generic[TArray]

Lidar raw depth data from an Ouster OSX sensor.

Warning

The measurements are recorded in time space, so are "staggered," and must be destaggered. See the ouster-sdk for details.

Attributes:

Name Type Description
rng UInt16[TArray, 'batch t beam time']

raw range measurements in beam-time space; nominally in mm.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
intrinsics str

path to intrinsics file used by the Ouster SDK.
Source code in format/src/roverd/types.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
@dataclass
class OSDepth(Generic[TArray]):
    """Lidar raw depth data from an Ouster OSX sensor.

    !!! warning

        The measurements are recorded in *time* space, so are "staggered,"
        and must be destaggered. See the [ouster-sdk](
        https://static.ouster.dev/sdk-docs/reference/lidar-scan.html#staggering-and-destaggering)
        for details.

    Attributes:
        rng: raw range measurements in beam-time space; nominally in mm.
        timestamps: timestamp for each frame; nominally in seconds.
        intrinsics: path to intrinsics file used by the Ouster SDK.
    """

    rng: UInt16[TArray, "batch t beam time"]
    timestamps: Float64[TArray, "batch t"]
    intrinsics: str

roverd.types.PointCloud

Bases: Generic[TArray]

Generic point cloud data.

Attributes:

Name Type Description
xyz Float32[TArray, 'batch t 3']

point cloud coordinates in meters.
length Integer[TArray, '#batch #t']

number of points in each point cloud.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
Source code in format/src/roverd/types.py 
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
@dataclass
class PointCloud(Generic[TArray]):
    """Generic point cloud data.

    Attributes:
        xyz: point cloud coordinates in meters.
        length: number of points in each point cloud.
        timestamps: timestamp for each frame; nominally in seconds.
    """

    xyz: Float32[TArray, "batch t 3"]
    length: Integer[TArray, "#batch #t"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.Pose

Bases: Generic[TArray]

Pose data.

Attributes:

Name Type Description
pos Float32[TArray, 'batch t 3']

position, in meters (front-left-up coordinate convention).
vel Float32[TArray, 'batch t 3']

velocity, in meters/second.
acc Float32[TArray, 'batch t 3']

acceleration, in meters/second^2.
rot Float32[TArray, 'batch t 3 3']

rotation (as matrix).
timestamps Float64[TArray, 'batch t']

timestamp for each measurement; nominally in seconds.
Source code in format/src/roverd/types.py 
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
@dataclass
class Pose(Generic[TArray]):
    """Pose data.

    Attributes:
        pos: position, in meters (front-left-up coordinate convention).
        vel: velocity, in meters/second.
        acc: acceleration, in meters/second^2.
        rot: rotation (as matrix).
        timestamps: timestamp for each measurement; nominally in seconds.
    """

    pos: Float32[TArray, "batch t 3"]
    vel: Float32[TArray, "batch t 3"]
    acc: Float32[TArray, "batch t 3"]
    rot: Float32[TArray, "batch t 3 3"]
    timestamps: Float64[TArray, "batch t"]

roverd.types.XWR4DSpectrum

Bases: Generic[TArray]

4D radar spectrum data.

Attributes:

Name Type Description
spectrum Complex64[TArray, 'batch t doppler elevation azimuth range']

4D complex spectrum in doppler-elevation-azimuth-range order.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
range_resolution Float[TArray, '#batch']

range resolution for the modulation used; nominally in meters.
doppler_resolution Float[TArray, '#batch']

doppler resolution; nominally in m/s.
Source code in format/src/roverd/types.py 
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
@dataclass
class XWR4DSpectrum(Generic[TArray]):
    """4D radar spectrum data.

    Attributes:
        spectrum: 4D complex spectrum in doppler-elevation-azimuth-range order.
        timestamps: timestamp for each frame; nominally in seconds.
        range_resolution: range resolution for the modulation used; nominally
            in meters.
        doppler_resolution: doppler resolution; nominally in m/s.
    """

    spectrum: Complex64[TArray, "batch t doppler elevation azimuth range"]
    timestamps: Float64[TArray, "batch t"]
    range_resolution: Float[TArray, "#batch"]
    doppler_resolution: Float[TArray, "#batch"]

roverd.types.XWRRadarIQ

Bases: Generic[TArray]

Raw I/Q data.

Attributes:

Name Type Description
iq Int16[TArray, 'batch t slow tx rx fast']

raw I/Q data in slow-tx-rx-fast order; see the xwr documentation for details.
timestamps Float64[TArray, 'batch t']

timestamp for each frame; nominally in seconds.
range_resolution Float[TArray, '#batch']

range resolution for the modulation used; nominally in meters.
doppler_resolution Float[TArray, '#batch']

doppler resolution; nominally in m/s.
valid UInt8[TArray, 'batch t']

whether the entire frame is valid, or if the frame contains some zero-filled data due to dropped packets.
Source code in format/src/roverd/types.py 
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
@dataclass
class XWRRadarIQ(Generic[TArray]):
    """Raw I/Q data.

    Attributes:
        iq: raw I/Q data in slow-tx-rx-fast order; see the [`xwr` documentation](
            https://wiselabcmu.github.io/xwr/dca/types/#xwr.capture.types.RadarFrame)
            for details.
        timestamps: timestamp for each frame; nominally in seconds.
        range_resolution: range resolution for the modulation used; nominally
            in meters.
        doppler_resolution: doppler resolution; nominally in m/s.
        valid: whether the entire frame is valid, or if the frame contains some
            zero-filled data due to dropped packets.
    """

    iq: Int16[TArray, "batch t slow tx rx fast"]
    timestamps: Float64[TArray, "batch t"]
    range_resolution: Float[TArray, "#batch"]
    doppler_resolution: Float[TArray, "#batch"]
    valid: UInt8[TArray, "batch t"]