Subproject B01

Coding of Multimodal Body Hull Data

Point Clouds are becoming one of the most common data structures to represent 3D scenes as it enable a six-degree of freedom (6DoF) viewing experience. However, a typical point cloud contains millions of 3D points and requires a huge amount of storage. Hence, efficient Point Cloud Compression (PCC) methods are inevitable to bring point clouds into practical applications. Unlike 2D image/video, point clouds are sparse and irregular (see the image), which makes the compression task even more difficult. Sub-project B01 aims to develop a source coding method for multimodal body envelope data that enables efficient transmission of the human body envelope including its motion.
Ricardo point cloud from MVUB dataset.
In recent years, the research society has been paying attention to this type of data, but the compression rate is still below the compression rates of 2D-image coding algorithms (JPEG, HEVC, VVC,…). In this project, we aim to tackle challenges in PCC including:
– Sparsity – most of the 3D space is empty, typically less than 2% of space is occupied, however, exploiting the redundancy and encoding the non-empty space are not easy tasks.
– Irregularity – unlike 2D images, where pixels are sampled uniformly over 2D planes, irregular sampling of point clouds makes it difficult to use traditional signal processing methods.
– Huge spatial volume – the information contained in a single 10-bit point cloud frame already equivalent to 1024 2D images of size 1024 × 1024. Such a point cloud would require enormous computational operations when applying any kind of signal-processing technique.
Point Clouds can be encoded and then used for different purposes such as VR, world heritage, medical analysis, etc. Thus, in this project, we investigate geometry and attribute coding in both lossless and lossy scenarios to provide solutions for various applications and purposes. Depending on the reconstruction accuracy of the biomechanical and psychomotor applications in the CRC, a reduction of the data volume by at least two orders of magnitude is expected.