Event-adapted video super-resolution

Sun, 29 Sep 2024 00:00:00 +0000

Introducing event cameras into video super-resolution (VSR) shows great promise. In practice, however, integrating event data as a new modality necessitates a laborious model architecture design. This not only consumes substantial time and effort but also disregards valuable insights from successful existing VSR models. Furthermore, the resource-intensive process of retraining these newly designed models exacerbates the challenge. In this paper, inspired by the recent success of parameter-efficient tuning in reducing the number of trainable parameters of a pre-trained model for downstream tasks, we introduce the Event AdapTER (EATER) for VSR. EATER efficiently utilizes knowledge of VSR models at the feature level through two lightweight and trainable components: the event-adapted alignment (EAA) unit and the event-adapted fusion (EAF) unit. The EAA unit aligns multiple frames based on the event …

Deep spiking-unet for image processing

Sun, 01 Jan 2023 00:00:00 +0000

本文提出一种深度脉冲U-Net架构，将脉冲神经网络的生物合理性与U-Net的多尺度特征提取能力相结合，用于图像处理任务。通过脉冲神经元替代传统激活函数，在保持图像处理性能的同时显著降低计算能耗，为低功耗边缘端图像处理提供了新思路。

Zheng-Jun Zha | ViLab

Event-adapted video super-resolution

Deep spiking-unet for image processing