Dachun Kai | ViLab

Holo-World: Unified Camera, Object and Weather Control for Video World Model

Thu, 18 Jun 2026 00:00:00 +0000

本文提出Holo-World，一种统一的视频世界模型，可从单张图像出发，联合控制相机运动、物体动态和天气状态。作者构建了HoloStateData数据集，将多样视频转换为统一控制样本；并提出统一场景适配器，将世界保持与天气迁移分解到不同参数子空间，利用渲染背景、几何缓冲和物体控制维持场景结构，同时建模天气相关外观与粒子效果。场景-天气分解CFG进一步分别引导场景和天气残差，增强目标天气效果。实验表明，Holo-World在保持精确控制的同时，实现了优于视频到视频基线的天气状态生成。

Seeing the unseen: Zooming in the dark with event cameras

Sat, 14 Mar 2026 00:00:00 +0000

本文提出RetinexEVSR，首个事件驱动的低光视频超分辨率框架。该框架利用高对比度事件信号与Retinex先验，通过双向跨模态融合策略，有效整合噪声事件数据与退化RGB帧中的有用信息。其中，照明引导事件增强模块利用Retinex模型导出的光照图逐步细化事件特征，抑制低光伪影并保留高对比度细节；事件引导反射率增强模块则通过多尺度融合机制动态恢复反射率细节。实验表明，该方法在三个数据集上达到最优性能，在SDSD基准上相比先前事件方法提升2.95 dB，并减少65%运行时间。

EvTexture++: Event-Driven Texture Enhancement for Video Super-Resolution

Mon, 02 Feb 2026 00:00:00 +0000

本文提出EvTexture++，一种事件驱动的视频超分辨率纹理增强框架。与以往将事件用于运动估计不同，该方法利用事件的高频时空细节显式恢复纹理，通过定制纹理增强分支和迭代纹理增强模块，逐步挖掘高时间分辨率事件信息，实现纹理区域的渐进细化，从而生成更精确、细节更丰富的高分辨率视频。该框架还可作为即插即用模块提升现有VSR模型性能。

Event-enhanced blurry video super-resolution

Fri, 11 Apr 2025 00:00:00 +0000

In this paper, we tackle the task of blurry video super-resolution (BVSR), aiming to generate high-resolution (HR) videos from low-resolution (LR) and blurry inputs. Current BVSR methods often fail to restore sharp details at high resolutions, resulting in noticeable artifacts and jitter due to insufficient motion information for deconvolution and the lack of high-frequency details in LR frames. To address these challenges, we introduce event signals into BVSR and propose a novel event-enhanced network, Ev-DeblurVSR. To effectively fuse information from frames and events for feature deblurring, we introduce a reciprocal feature deblurring module that leverages motion information from intra-frame events to deblur frame features while reciprocally using global scene context from the frames to enhance event features. Furthermore, to enhance temporal consistency, we propose a hybrid deformable alignment module that fully exploits the complementary motion information from inter-frame events and optical flow to improve motion estimation in the deformable alignment process. Extensive evaluations demonstrate that Ev-DeblurVSR establishes a new state-of-the-art performance on both synthetic and real-world datasets. Notably, on real data, our method is 2.59 dB more accurate and 7.28× faster than the recent best BVSR baseline FMA-Net.

Enhancing Visual Tracking by Leveraging High-frequency Information within Event Signals

Wed, 01 Jan 2025 00:00:00 +0000

Traditional object trackers struggle in degraded scenarios, lacking sufficient appearance details of moving targets for precise tracking. Recent trackers have integrated highfrequency event signals to assist tracking. However, they neglect the high-temporalresolution motion information inherent in events, limiting their performance especially in occlusion and background clutter. To address these challenges, we propose HFTrack, a novel tracker designed to fully leverage the spatio-temporal high-frequency information within event signals, thereby enhancing the tracking performance. Specifically, we introduce a frequency-based feature enhancement module, which enriches the frame feature with high-frequency components from events in frequency space, capturing detailed appearance information of moving targets. Additionally, we propose a spatio-temporal information decoder with an auto-regressive temporal query, integrating both historical motion cues from events and enhanced spatial features for robust target localization. Experimental results demonstrate that our HFTrack significantly outperforms existing trackers, showcasing its strong ability to track the target under challenging conditions.

Asymmetric event-guided video super-resolution

Mon, 28 Oct 2024 00:00:00 +0000

本文首次提出非对称事件引导的视频超分辨率任务，针对事件相机与RGB相机难以严格标定的实际场景，构建了非对称事件引导视频超分辨率网络（AsEVSRN）。该网络通过专门设计的事件特征利用与跨模态融合机制，充分发挥事件相机高时间分辨率优势，有效提升视频超分辨率性能，拓展了事件相机在双摄手机、无人机等新兴高分辨率设备上的应用潜力。

Event-adapted video super-resolution

Sun, 29 Sep 2024 00:00:00 +0000

Introducing event cameras into video super-resolution (VSR) shows great promise. In practice, however, integrating event data as a new modality necessitates a laborious model architecture design. This not only consumes substantial time and effort but also disregards valuable insights from successful existing VSR models. Furthermore, the resource-intensive process of retraining these newly designed models exacerbates the challenge. In this paper, inspired by the recent success of parameter-efficient tuning in reducing the number of trainable parameters of a pre-trained model for downstream tasks, we introduce the Event AdapTER (EATER) for VSR. EATER efficiently utilizes knowledge of VSR models at the feature level through two lightweight and trainable components: the event-adapted alignment (EAA) unit and the event-adapted fusion (EAF) unit. The EAA unit aligns multiple frames based on the event …

A micro-expression recognition system with event cameras

Mon, 15 Jul 2024 00:00:00 +0000

本文提出了一种基于事件相机的微表情识别系统。针对微表情持续时间短、幅度微弱、难以用传统相机捕捉的问题，系统利用事件相机的高时间分辨率特性，设计了事件增强运动提取器（EEME）以放大细微运动，并引入事件引导注意力（EGA）聚焦关键面部区域，从而提升微表情识别的准确性与鲁棒性。该系统为情感计算领域提供了有效工具。

Estme: Event-driven spatio-temporal motion enhancement for micro-expression recognition

Mon, 15 Jul 2024 00:00:00 +0000

本文针对微表情识别中动作幅度小、持续时间短、难以捕捉的问题，提出了一种事件驱动的时空运动增强网络。该方法引入事件相机捕获的高时间分辨率事件信号，设计事件增强运动提取模块以增强细微运动细节，并利用事件引导注意力模块聚焦特定区域的微小变化，从而获取更精确的空间特征。在合成和真实数据集上的实验结果表明，该方法在微表情识别任务上具有优越性能。

Video super-resolution via event-driven temporal alignment

Sun, 08 Oct 2023 00:00:00 +0000

本文提出一种事件驱动的双向视频超分辨率框架（EBVSR），利用事件相机的高时间分辨率特性捕捉非线性运动，并设计事件辅助的时间对齐模块，以补充光流法在快速光照变化下的不足。同时构建基于事件的帧合成模块，通过双向跨模态融合增强网络对光照变化的鲁棒性。在合成和真实数据上的实验验证了该方法在视频超分辨率任务中的有效性。