Subproject B01

Subproject B01

Coding of Multimodal Body Hull Data

Point Clouds are becoming one of the most common data structures to represent 3D scenes as it enable a six-degree of freedom (6DoF) viewing experience. However, a typical point cloud contains millions of 3D points and requires a huge amount of storage. Hence, efficient Point Cloud Compression (PCC) methods are inevitable to bring point clouds into practical applications. Unlike 2D image/video, point clouds are sparse and irregular (see the image), which makes the compression task even more difficult. Sub-project B01 aims to develop a source coding method for multimodal body envelope data that enables efficient transmission of the human body envelope including its motion.
Ricardo point cloud from MVUB dataset.
In recent years, the research society has been paying attention to this type of data, but the compression rate is still below the compression rates of 2D-image coding algorithms (JPEG, HEVC, VVC,…). In this project, we aim to tackle challenges in PCC including:
– Sparsity – most of the 3D space is empty, typically less than 2% of space is occupied, however, exploiting the redundancy and encoding the non-empty space are not easy tasks.
– Irregularity – unlike 2D images, where pixels are sampled uniformly over 2D planes, irregular sampling of point clouds makes it difficult to use traditional signal processing methods.
– Huge spatial volume – the information contained in a single 10-bit point cloud frame already equivalent to 1024 2D images of size 1024 × 1024. Such a point cloud would require enormous computational operations when applying any kind of signal-processing technique.
Point Clouds can be encoded and then used for different purposes such as VR, world heritage, medical analysis, etc. Thus, in this project, we investigate geometry and attribute coding in both lossless and lossy scenarios to provide solutions for various applications and purposes. Depending on the reconstruction accuracy of the biomechanical and psychomotor applications in the CRC, a reduction of the data volume by at least two orders of magnitude is expected.

 

                                                                           

Contacts

Prof. Dr.-Ing. André Kaup

Principal Investigator

Xiumei Li, M. Sc.

Doctoral Candidate

Dat Thanh Nguyen, M. Sc.

Doctoral Candidate

Marina Ritthaler, M. Sc.

Associated Doctoral Candidate

PD Dr.-Ing. habil. Christian Herglotz

Postdoc

 

 

Additional Information

  • Nguyen DT., Zieger D., Gambietz M., Koelewijn A., Stamminger M., Kaup A.:
    Multiresolution point cloud compression for real-time visualization and streaming of large 3D datasets.
    Proceedings of the Asilomar Conferense on Signals, Systems, and Computers (2024).
  • Rückert R., Ullmann I., Vossiek M., Kaup A., Herglotz C.:
    Data Compression for Close-Range Radar Imaging.
    IEEE Transactions on Radar Systems 1 (2024), S. 421-433. ISSN: 2832-7357. DOI: 10.1109/TRS.2024.3387288
  • Nguyen DT., Kaup A.:
    Lossless Point Cloud Geometry and Attribute Compression Using a Learned Conditional Probability Model.
    IEEE Transactions on Circuits and Systems For Video Technology (2023), S. 1-1. ISSN: 1051-8215. DOI: 10.1109/TCSVT.2023.3239321
  • Nguyen DT., Nambiar KG., Kaup A.:
    Deep probabilistic model for lossless scalable point cloud attribute compression.
    International Conference on Acoustics, Speech, and Signal Processing (Rhodes Island, Greece, 2023). DOI: 10.1109/icassp49357.2023.10095385
  • Heimann V., Spruck A., Kaup A.:
    Frequency-Selective Geometry Upsampling of Point Clouds.
    IEEE International Conference on Image Processing (ICIP) 2022 (Bordeaux, 2022). DOI: 10.1109/ICIP46576.2022.9897920
  • Heimann V., Spruck A., Kaup A.:
    Jointly Resampling and Reconstructing Corrupted Images for Image Classification using Frequency-Selective Mesh-to-Grid Resampling.
    IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) (NEW YORK: 2022). DOI: 10.1109/MMSP55362.2022.9949143
  • Herglotz C., Genser N., Kaup A.:
    Rate-Distortion Optimal Transform Coefficient Selection for Unoccupied Regions in Video-Based Point Cloud Compression.
    IEEE Transactions on Circuits and Systems For Video Technology (2022), S. 1-1. ISSN: 1051-8215. DOI: 10.1109/TCSVT.2022.3185026
  • Nguyen DT., Kaup A.:
    Learning-based Lossless Point Cloud Geometry Coding using Sparse Tensors.
    IEEE International Conference on Image Processing (ICIP) (Bordeaux, 2022). DOI: 10.1109/icip46576.2022.9897827
  • Heimann V., Spruck A., Kaup A.:
    Frequency-Selective Mesh-to-Mesh Resampling for Color Upsampling of Point Clouds.
    IEEE 23rd International Workshop on Multimedia Signal Processing (Tampere, 2021) DOI: 10.1109/MMSP53017.2021.9733445
  • Körner G., Hoffmann M., Stief P., Zhang M., Rückert R., Herglotz C., Kaup A., Vossiek M.:
    Applicability and Performance of Standard Compression Methods for Efficient Data Transmission and Storage in Radar Networks.
    IEEE Journal of Microwaves 2 (2021), S. 78-96. ISSN: 2692-8388
    DOI: 10.1109/JMW.2021.3119781