Feature compression with 3d sparse convolution

Abstract

Feature compression is an important branch of video coding for machines (VCM). While existing methods draw inspiration from image compression, they have not fully utilized the unique characteristics of features. In this paper, we investigate feature characteristics in two key aspects: dimensionality and sparsity. Our analysis reveals that the low spatial dimensionality and high channel dimensionality of features make traditional 2D convolution-based methods, which usually downsample along spatial dimensions while increasing channels, unsuitable for feature compression. To address this, we propose compressing features using 3D convolution. Additionally, considering the sparsity characteristic, we propose applying sparse convolution to reduce model complexity. To thoroughly investigate the proposed 3D sparse convolution-based method, we verify it with various network structures and input features …

Publication
In VCIP

本文面向视频编码与机器(VCM)中的特征压缩任务,指出现有方法未充分利用特征的维度与稀疏性。研究发现特征具有低空间维度、高通道维度的特点,传统基于2D卷积的下采样方式并不适用,因此提出采用3D卷积进行特征压缩;同时利用特征的稀疏性引入稀疏卷积,以降低模型复杂度。作者在多种网络结构和输入特征上验证了所提方法的有效性,为特征压缩提供了新思路。