Reading camera data

The dv-processing library provides a convenient way of reading camera data from live connected cameras and persistent files.

Camera name

Camera name is used to identify a unique camera produced by iniVation. Camera name consists of a camera model and a serial number, concatenated by an underscore (“_”) character. The library refers to camera name in multiple methods, this value can be consistently used across the library.

Some examples of camera names:

• DVXplorer: DVXplorer_DXA00093, DVXplorer_DXM00123

• DAVIS: DAVIS346_00000499

Note

This definition is valid for USB cameras, the camera name is also reported in network streaming sources. In that case, camera name can be manually set by the developer, so naming convention for the models might not be entirely followed.

From a camera

The easiest approach to access data from a live camera is to use the dv::io::camera::open() functions. This section provides in-depth explanations on their usage and code samples.

Discover connected cameras

The command-line utility dv-list-devices is available in the packages of dv-processing and can show information on the connected cameras. Sample output for the utility:

$ dv-list-devices
Device discovery: found 2 devices.
Detected device [DAVIS 00000499]
Detected device [DVXPLORER DXA00093]

Note

This utility reports type and serial number, not the full camera name described above. The getCameraName() function provides the full name once a device has been opened.

Device discovery is also possible with the use of library methods. Following is a sample on how to detect connected devices using discovery method:

C++Python

#include <dv-processing/io/camera/discovery.hpp>

#include <iostream>

int main() {
    // Call the discovery method
    const auto cameras = dv::io::camera::discover();

    std::cout << "Device discovery: found " << cameras.size() << " devices." << std::endl;

    // Loop through detected camera names and print them
    for (const auto &camera : cameras) {
        std::cout << "Detected device [" << camera.cameraModel << " " << camera.serialNumber << "]" << std::endl;
    }

    return 0;
}

Opening a camera

The dv::io::camera::DVS128, dv::io::camera::DAVIS, dv::io::camera::DVXplorer and dv::io::camera::DVXplorerM classes follow the RAII pattern for resource management. Creating an instance of the class will open the camera connected on USB and starts reading data immediately; the camera is shut-down when the object instance is destroyed. These classes allow full interaction with the cameras, exposing all available functionality.

It is also possible to open cameras in a more generic way, using the dv::io::camera::open() functions. For cameras with multi-camera synchronization support, you can also use the dv::io::camera::openSync() functions. This allows access to a set of features common to all cameras. For access to all features, you will have to up-cast the generic camera to the class representing its full type.

C++Python

// Open the first camera found
auto capture = dv::io::camera::open();

It is also possible to open a specific camera on the system by providing the serial number:

C++Python

// Open the specified camera
auto capture = dv::io::camera::open("DXA00093");

To access all features, it’s easiest to open the camera using its full class:

C++Python

// Open any DAVIS camera (camera name not specified)
auto capture = dv::io::camera::DAVIS();

Checking camera capabilities

Since some cameras provide different data types and capabilities, the base class dv::io::camera::CameraInputBase provides methods to test what data the camera can provide:

C++Python

#include <dv-processing/io/camera/discovery.hpp>

int main() {
    // Open any camera
    auto capture = dv::io::camera::open();

    // Print the camera name
    std::cout << "Opened [" << capture->getCameraName() << "] camera, it provides:" << std::endl;

    // Check whether event stream is available
    if (capture->isEventStreamAvailable()) {
        // Get the event stream resolution, the output is a std::optional, so the value() method is
        // used to get the actual resolution value
        const cv::Size resolution = capture->getEventResolution().value();

        // Print the event stream capability with resolution value
        std::cout << "* Events at " << resolution << " resolution" << std::endl;
    }

    // Check whether frame stream is available
    if (capture->isFrameStreamAvailable()) {
        // Get the frame stream resolution
        const cv::Size resolution = capture->getFrameResolution().value();

        // Print the frame stream capability with resolution value
        std::cout << "* Frames at " << resolution << " resolution" << std::endl;
    }

    // Check whether the IMU stream is available
    if (capture->isImuStreamAvailable()) {
        // Print the imu data stream capability
        std::cout << "* IMU measurements" << std::endl;
    }

    // Check whether the trigger stream is available
    if (capture->isTriggerStreamAvailable()) {
        // Print the trigger stream capability
        std::cout << "* Triggers" << std::endl;
    }

    return 0;
}

Configuring camera options

Some advanced properties of our cameras can be configured by a number of functions. Some are listed here for reference, please check the detailed API documentation for more details.

DVXplorer camera advanced control options

Sensitivity and eFPS

C++Python

#include <dv-processing/io/camera/dvxplorer.hpp>

int main() {
    // Open a DVXplorer camera
    dv::io::camera::DVXplorer capture{};

    // Configure event sensitivity. Range 0-17, default 9.
    capture.setContrastThresholdOn(9);
    capture.setContrastThresholdOff(9);

    // Configure event-frame readouts per second (here variable 5000 FPS, the default value)
    // See detailed API documentation for other available values
    capture.setReadoutFPS(dv::io::camera::DVXplorer::ReadoutFPS::VARIABLE_5000);

    // Enable global hold setting (already the default)
    capture.setGlobalHold(true);
    // Disable global reset setting (already the default)
    capture.setGlobalReset(false);

    return 0;
}

Read more about DVXplorer biases and details about the ReadoutFPS implementation in our documentation page.

Note

On DVXplorer, setting global hold to false can help for certain applications containing repeating patterns observation, such as flickering LEDs.

Crop events to a region of interest

C++Python

#include <dv-processing/io/camera/dvxplorer.hpp>

int main() {
    // Open a DVXplorer camera
    dv::io::camera::DVXplorer capture{};

    // Set ROI area, automatically takes effect
    capture.setCropArea({0, 0, 200, 200});

    return 0;
}

DAVIS camera advanced control options

General options

C++Python

#include <dv-processing/io/camera/davis.hpp>

int main() {
    // Open a Davis camera
    dv::io::camera::DAVIS capture{};

    // Setting camera readout to events and frames (default).
    capture.setEventsRunning(true);
    capture.setFramesRunning(true);

    // Configure frame output mode to color (default), only on COLOR cameras. Other mode available: GRAYSCALE
    capture.setColorMode(dv::io::camera::parser::DAVIS::ColorMode::DEFAULT);

    return 0;
}

Frame options

C++Python

#include <dv-processing/io/camera/davis.hpp>

#include <chrono>

int main() {
    using namespace std::chrono_literals;

    // Open a Davis camera
    dv::io::camera::DAVIS capture{};

    // Enable frame auto-exposure (default behavior)
    capture.setAutoExposure(true);

    // Disable auto-exposure, set frame exposure (here 10ms)
    capture.setAutoExposure(false);
    capture.setExposureDuration(10ms);

    // Read current frame exposure duration value
    std::chrono::microseconds duration = capture.getExposureDuration();

    // Set frame interval duration (here 33ms for ~30FPS)
    capture.setFrameInterval(33ms);

    // Read current frame interval duration value
    std::chrono::microseconds interval = capture.getFrameInterval();

    return 0;
}

Event options (biases)

Warning

Before using biases, make sure that you absolutely need to change them and that you understand them by reading about biases on our documentation page.

C++Python

#include <dv-processing/io/camera/davis.hpp>

int main() {
    // Open a Davis346 camera
    dv::io::camera::DAVIS capture{};

    /// Access biases raw value
    // Photoreceptor bias
    auto biasPhotoreceptor = capture.getDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::Photoreceptor);
    // Source follower bias
    auto biasPhotoreceptorSourceFollower
        = capture.getDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::PhotoreceptorSourceFollower);
    // Differential bias
    auto biasDiff = capture.getDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::Diff);
    // On threshold bias
    auto biasOn = capture.getDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::On);
    // Off threshold bias
    auto biasOff = capture.getDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::Off);
    // Refractory period bias
    auto biasRefractory = capture.getDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::Refractory);

    // Add 1 on the log-scale coarse value, i.e. multiply bias value by 8 (approximately)
    biasPhotoreceptor.first               += 1;
    biasPhotoreceptorSourceFollower.first += 1;
    biasDiff.first                        += 1;
    biasOn.first                          += 1;
    biasOff.first                         += 1;
    biasRefractory.first                  += 1;

    /// Set biases raw value
    // Setting photoreceptor bias
    capture.setDavis346BiasCoarseFine(
        dv::io::camera::DAVIS::Davis346BiasCF::Photoreceptor, biasPhotoreceptor.first, biasPhotoreceptor.second);
    // Setting source follower bias
    capture.setDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::PhotoreceptorSourceFollower,
        biasPhotoreceptorSourceFollower.first, biasPhotoreceptorSourceFollower.second);
    // Setting differential bias
    capture.setDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::Diff, biasDiff.first, biasDiff.second);
    // Setting on threshold bias
    capture.setDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::On, biasOn.first, biasOn.second);
    // Setting off threshold bias
    capture.setDavis346BiasCoarseFine(dv::io::camera::DAVIS::Davis346BiasCF::Off, biasOff.first, biasOff.second);
    // Setting refractory period bias
    capture.setDavis346BiasCoarseFine(
        dv::io::camera::DAVIS::Davis346BiasCF::Refractory, biasRefractory.first, biasRefractory.second);

    return 0;
}

Crop events or frames to a region of interest

C++Python

#include <dv-processing/io/camera/davis.hpp>

int main() {
    // Open a Davis camera
    dv::io::camera::DAVIS capture{};

    // Set ROI for events
    capture.setCropAreaEvents({0, 0, 200, 200});

    // Set ROI for frames
    capture.setCropAreaFrames({0, 0, 300, 150});

    return 0;
}

Read events from a live camera

Incoming data from a camera can be read sequentially using the dv::io::CameraInputBase::getNextEventBatch(). Following is a minimal sample on how to read events sequentially from a camera:

C++Python

#include <dv-processing/io/camera/discovery.hpp>

#include <chrono>

int main() {
    using namespace std::chrono_literals;

    // Open any camera
    auto capture = dv::io::camera::open();

    // Run the loop while camera is still connected
    while (capture->isRunning()) {
        // Read batch of events, check whether received data is correct.
        // The method does not wait for data arrive, it returns immediately with
        // the latest available data or if no data is available, returns a `std::nullopt`.
        if (const auto events = capture->getNextEventBatch(); events.has_value()) {
            // Print received packet information
            std::cout << *events << std::endl;
        }
        else {
            // No data has arrived yet, short sleep to reduce CPU load.
            std::this_thread::sleep_for(1ms);
        }
    }

    return 0;
}

Read frames from a live camera

Incoming frames from a camera can be read sequentially frame-by-frame using the dv::io::CameraInputBase::getNextFrame(). Following is a minimal sample on how to read frames sequentially from a camera:

C++Python

#include <dv-processing/io/camera/discovery.hpp>

#include <opencv2/highgui.hpp>

int main() {
    // Open any camera
    auto capture = dv::io::camera::open();

    // Initiate a preview window
    cv::namedWindow("Preview", cv::WINDOW_NORMAL);

    // Run the loop while camera is still connected
    while (capture->isRunning()) {
        // Read a frame, check whether it is correct.
        // The method does not wait for frame arrive, it returns immediately with
        // the latest available frame or if no data is available, returns a `std::nullopt`.
        if (const auto frame = capture->getNextFrame(); frame.has_value()) {
            std::cout << *frame << std::endl;

            // Show a preview of the image
            cv::imshow("Preview", frame->image);
        }
        cv::waitKey(2);
    }

    return 0;
}

Read IMU data from a live camera

Incoming imu data from a camera can be read sequentially using the {cpp:func} dv::io::CameraInputBase::getNextImuBatch(). Following is a minimal sample on how to read imu data sequentially from a camera:

C++Python

#include <dv-processing/io/camera/discovery.hpp>

#include <chrono>

int main() {
    using namespace std::chrono_literals;

    // Open any camera
    auto capture = dv::io::camera::open();

    // Run the loop while camera is still connected
    while (capture->isRunning()) {
        // Read IMU measurement batch, check whether it is correct.
        // The method does not wait for data to arrive, it returns immediately with
        // the latest available imu data or if no data is available, returns a `std::nullopt`.
        if (const auto imuBatch = capture->getNextImuBatch(); imuBatch.has_value() && !imuBatch->empty()) {
            std::cout << "Received " << imuBatch->size() << " IMU measurements" << std::endl;
        }
        else {
            // No data has arrived yet, short sleep to reduce CPU load.
            std::this_thread::sleep_for(1ms);
        }
    }

    return 0;
}

Read triggers from a live camera

Note

To understand what triggers are and where they come from, read more about them on our documentation page.

Incoming trigger data from a camera can be read sequentially using the dv::io::CameraInputBase::getNextTriggerBatch(). Following is a minimal sample on how to read trigger data sequentially from a camera:

C++Python

#include <dv-processing/io/camera/davis.hpp>

#include <chrono>

int main() {
    using namespace std::chrono_literals;

    // Open a DAVIS camera (DVXplorer also has triggers)
    dv::io::camera::DAVIS capture{};

    // Depending on the incoming signal, enable the detection of the desired type of pattern, here we enable everything.
    // Enable rising edge detection
    capture.setDetectorRisingEdges(true);
    // Enable falling edge detection
    capture.setDetectorFallingEdges(true);
    // Enable detector
    capture.setDetectorRunning(true);

    // Run the loop while camera is still connected
    while (capture.isRunning()) {
        // Read trigger batch, check whether it is correct.
        // The method does not wait for data to arrive, it returns immediately with
        // the latest available data or if no data is available, returns a `std::nullopt`.
        if (const auto triggers = capture.getNextTriggerBatch(); triggers.has_value() && !triggers->empty()) {
            std::cout << "Received " << triggers->size() << " Triggers" << std::endl;
        }
        else {
            // No data has arrived yet, short sleep to reduce CPU load.
            std::this_thread::sleep_for(1ms);
        }
    }

    return 0;
}

Sample application - reading data from a live camera

An application reading multiple types of data from a live camera can be found among the code samples in the source code repository of the dv-processing library:

• Camera capture in C++

• Camera capture in Python

From a file

Data from iniVation cameras are usually recorded using the AEDAT4 file format. The dv-processing library provide tools for reading such files. This section contains explanations and samples on how data can be read from AEDAT4 files. More detailed information on the AEDAT4 file format can be found here.

Inspecting AEDAT4 files

AEDAT4 file format supports recording of different data streams into single file, multiple cameras are also supported. The library provides a command-line utility for inspection of AEDAT4 files dv-filestat, it provides information on available streams recorded in it.

The utility provides information on the size, timestamp information, duration. More information about the utility can be found here.

Opening a file

AEDAT4 files can be opened and read using the dv::io::MonoCameraRecording class. This class assumes that the recording was performed using a single camera. Following is a minimal sample code on opening a recording and printing information about it.

A file can be opened by providing its path in the filesystem:

C++Python

#include <dv-processing/io/mono_camera_recording.hpp>

int main() {
    // Open a file
    dv::io::MonoCameraRecording reader("path/to/file.aedat4");

    // Get and print the camera name that data from recorded from
    std::cout << "Opened an AEDAT4 file which contains data from [" << reader.getCameraName() << "] camera"
              << std::endl;

    return 0;
}

Checking available streams

Data recordings might contain various data streams. The dv::io::MonoCameraRecording provides easy-to-use methods to inspect what data streams are available. Following sample code shows how to check for existence of various data streams:

C++Python

#include <dv-processing/io/mono_camera_recording.hpp>

int main() {
    // Store the file path
    const std::string pathToFile = "path/to/file.aedat4";

    // Open a file
    dv::io::MonoCameraRecording reader(pathToFile);

    // Print file path and camera name
    std::cout << "Available streams in [" << pathToFile << "]:" << std::endl;

    // Check if event stream is available
    if (reader.isEventStreamAvailable()) {
        // Check the resolution of event stream. Since the getEventResolution() method returns
        // a std::optional, we use *operator to get the value. The method returns std::nullopt
        // only in case the stream is unavailable, which is already checked.
        const cv::Size resolution = *reader.getEventResolution();

        // Print that the stream is present and its resolution
        std::cout << "  * Event stream with resolution " << resolution << std::endl;
    }

    // Check if frame stream is available
    if (reader.isFrameStreamAvailable()) {
        // Check the resolution of frame stream. Since the getFrameResolution() method returns
        // a std::optional, we use *operator to get the value. The method returns std::nullopt
        // only in case the stream is unavailable, which is already checked.
        const cv::Size resolution = *reader.getFrameResolution();

        // Print that the stream is available and its resolution
        std::cout << "  * Frame stream with resolution " << resolution << std::endl;
    }

    // Check if IMU stream is available
    if (reader.isImuStreamAvailable()) {
        // Print that the IMU stream is available
        std::cout << "  * IMU stream" << std::endl;
    }

    // Check if trigger stream is available
    if (reader.isTriggerStreamAvailable()) {
        // Print that the trigger stream is available
        std::cout << "  * Trigger stream " << std::endl;
    }

    return 0;
}

The dv-processing library also supports recording of other types, type agnostic methods are available as templated methods in C++, while Python only contains a limited set of named methods (since templating is unavailable in Python). The following sample shows the use of generic methods for checking the availability of certain streams with a name and a type:

#include <dv-processing/io/mono_camera_recording.hpp>

int main() {
    // Open a file
    dv::io::MonoCameraRecording reader("path/to/file.aedat4");

    // Store some stream name
    const std::string streamName = "poses";

    // Check if such stream name is available and validate the data type of this stream
    if (reader.isStreamAvailable("streamName") && reader.isStreamOfDataType<dv::Pose>("streamName")) {
        std::cout << "The file contains a stream named [" << streamName << "] and of data type [dv::Pose]" << std::endl;
    }

    return 0;
}

Read events from a file

Following sample reads events in batches while the stream has available data to read. While reading from a file, the dv::io::MonoCameraRecording::getNextEventBatch() will return data until the end of stream is reached, the dv::io::MonoCameraRecording::isRunning() method will return a false boolean when the end-of-file is reached for any stream. You can also use the dv::io::MonoCameraRecording::isRunning(streamName)() method to check for specific streams.

C++Python

#include <dv-processing/io/mono_camera_recording.hpp>

int main() {
    // Open a file
    dv::io::MonoCameraRecording reader("path/to/file.aedat4");

    // Run the loop while data is available
    while (reader.isRunning()) {
        // Read batch of events, check whether received data is correct.
        if (const auto events = reader.getNextEventBatch(); events.has_value()) {
            // Print received event packet information
            std::cout << *events << std::endl;
        }
    }

    return 0;
}

Read frames from a file

Following sample reads frames in batches while the stream has available data to read. While reading from a file, the dv::io::MonoCameraRecording::getNextFrame() will return a frame until the end of stream is reached, the dv::io::MonoCameraRecording::isRunning() method will return a false boolean when the end-of-file is reached for any stream. You can also use the dv::io::MonoCameraRecording::isRunning(streamName)() method to check for specific streams.

C++Python

#include <dv-processing/io/mono_camera_recording.hpp>

#include <opencv2/highgui.hpp>

int main() {
    // Open a file
    dv::io::MonoCameraRecording reader("path/to/file.aedat4");

    // Initiate a preview window
    cv::namedWindow("Preview", cv::WINDOW_NORMAL);

    // Variable to store the previous frame timestamp for correct playback
    std::optional<int64_t> lastTimestamp = std::nullopt;

    // Run the loop while data is available
    while (reader.isRunning()) {
        // Read a frame, check whether it is correct.
        // The method does not wait for frame arrive, it returns immediately with
        // latest available frame or if no data is available, returns a `std::nullopt`.
        if (const auto frame = reader.getNextFrame(); frame.has_value()) {
            // Print information about received frame
            std::cout << *frame << std::endl;

            // Show a preview of the image
            cv::imshow("Preview", frame->image);

            // Calculate the delay between last and current frame, divide by 1000 to convert microseconds
            // to milliseconds
            const int delay = lastTimestamp.has_value() ? (frame->timestamp - *lastTimestamp) / 1000 : 2;

            // Perform the sleep
            cv::waitKey(delay);

            // Store timestamp for the next frame
            lastTimestamp = frame->timestamp;
        }
    }

    return 0;
}

Read IMU data from a file

Following sample reads imu data in batches while the stream has available data to read. While reading from a file, the dv::io::MonoCameraRecording::getNextImuBatch() will return an IMU measurement batch until the end of stream is reached, the dv::io::MonoCameraRecording::isRunning() method will return a false boolean when the end-of-file is reached for any stream. You can also use the dv::io::MonoCameraRecording::isRunning(streamName)() method to check for specific streams.

C++Python

#include <dv-processing/io/mono_camera_recording.hpp>

int main() {
    // Open a file
    dv::io::MonoCameraRecording reader("path/to/file.aedat4");

    // Run the loop while data is available
    while (reader.isRunning()) {
        // Read IMU measurement batch, check whether it is correct.
        if (const auto imuBatch = reader.getNextImuBatch(); imuBatch.has_value() && !imuBatch->empty()) {
            // Print IMU batch information
            std::cout << "Received " << imuBatch->size() << " IMU measurements" << std::endl;
        }
    }

    return 0;
}

Read triggers from a file

Following sample reads triggers in batches while the stream has available data to read. While reading from a file, the dv::io::MonoCameraRecording::getNextTriggerBatch() will return a batch of triggers until the end of stream is reached, the dv::io::MonoCameraRecording::isRunning() method will return a false boolean when the end-of-file is reached for any stream. You can also use the dv::io::MonoCameraRecording::isRunning(streamName)() method to check for specific streams.

C++Python

#include <dv-processing/io/mono_camera_recording.hpp>

int main() {
    // Open a file
    dv::io::MonoCameraRecording reader("path/to/file.aedat4");

    // Run the loop while data is available
    while (reader.isRunning()) {
        // Read trigger batch, check whether it is correct.
        if (const auto triggers = reader.getNextTriggerBatch(); triggers.has_value() && !triggers->empty()) {
            // Print the trigger batch information
            std::cout << "Received " << triggers->size() << " triggers" << std::endl;
        }
    }

    return 0;
}

[Advanced] Reading custom data types

The previous samples show how to use named functions to read different data types. The C++ API provides templated methods to read any type of data. Below is a sample that shows how to read data using the generic templated API:

Note

Since templated methods are only available in C++, the generic writing methods are only available in the C++ API.

#include <dv-processing/data/timed_keypoint_base.hpp>
#include <dv-processing/io/mono_camera_recording.hpp>

int main() {
    // Open a file
    dv::io::MonoCameraRecording reader("path/to/file.aedat4");

    // Define and contain a stream name in a variable
    const std::string stream = "keypoints";

    // Check whether a timed-keypoint stream is available
    if (!reader.isStreamAvailable(stream) || !reader.isStreamOfDataType<dv::TimedKeyPointPacket>(stream)) {
        throw dv::exceptions::RuntimeError("Stream named 'keypoints' not found");
    }

    // Run the loop while data is available
    while (reader.isRunning()) {
        // Read timed keypoint batch, check whether it is correct.
        if (const auto keypoints = reader.getNextStreamPacket<dv::TimedKeyPointPacket>(stream); keypoints.has_value()) {
            // Print the number of keypoints read
            std::cout << "Read " << keypoints->elements.size() << " timed keypoints" << std::endl;
        }
    }

    return 0;
}

Sample application - reading data from a recorded AEDAT4 file

An application reading multiple types of data from an AEDAT4 file can be found in the source code repository of the dv-processing library:

• AEDAT4 player in C++

• AEDAT4 to CSV converter in Python