Aqueti developed ultra-high-resolution (>10X digital zoom out from single-pixel display size) cameras for security and other markets. ReliaSolve worked with them to develop the GPU-based decompression, tiling, rendering, and re-encoding portions of the pixel pipeline to enable viewers to pan, tilt, zoom, and move in time in a seamless view of the world. The resulting video can be displayed on with multiple views on different screens or on a tiled display wall. This includes reprojection, distortion correction, blending, and synchronization. The system includes GPU-based PBO caching of uncompressed texture using multiple CUDA streams for predictive decompression coupled with CPU-side predictive compressed-image caching.
The picture above shows the final product, the Aqueti Mantis Camputer. This model has a single wide-field-of-view camera in the center and 18 high-resolution inset cameras.
The picture above shows an early prototype of the camera input system whose set of synchronized, high-resolution microcameras produce the compressed video streams that are selected, reassembled, and rendered.
The image above shows a joystick-controlled interface that allows panning, tilting, zooming, and scrubbing through time on the video produced from the camera seen above. Differences in exposure time from the cameras during this early trial run make it possible to see the seams between the subsampled and tiled microcamera images. The final product includes correction for this, producing a seamless stitched video.
ReliaSolve also designed the Application Programmer’s Interface to Aqueti cameras, implementing the C++/C hourglass design and a SWIG Python layer to enable upgrades without re-linking applications in C, C++ and Python. The API was documented in DOxygen and includes test and example programs in C, C++, and Python for all functions.
ReliaSolve worked with Aqueti to help the cameras pass ONVIF conformance authentication in profiles S and G.
ReliaSolve also designed and implemented a version of the rendering system that performs acquisition, warping, stitching and rendering for a 4-microcamera solution that runs completely on an embedded nVidia Tegra processor to produce ultra-high-resolution 180-degree video for remote station monitoring.
We also provided a 360-view mode and the ability to see avatars of other zoomed-in viewers shown in the full view, so that a team leader can get an overview and see who is looking where.
In 2021, ReliaSolve developed a VR interface to 360-degree Aqueti cameras that lets the viewer zoom look in all directions and zoom in. It makes use of the Aqueti API to render images and uses their spatial and temporal metadata to provide smooth and jitter-free playback in VR that hides all of the “swimming” of the images due to end-to-end and rendering latencies.