Hebei Li | ViLab

Facm: Flow-anchored consistency models

Mon, 20 Apr 2026 00:00:00 +0000

本文针对连续时间一致性模型（CM）训练不稳定的问题，指出其根源在于捷径目标导致瞬时速度场被灾难性遗忘。为此提出流锚定一致性模型（FACM），以流匹配任务作为动态锚点，并设计扩展时间间隔策略统一优化、解耦两个任务，实现稳定且架构无关的训练。在ImageNet 256×256上，蒸馏LightningDiT模型取得NFE=2时FID 1.32、NFE=1时FID 1.70的SOTA结果；同时提出内存高效的Chain-JVP，将FACM扩展到140亿参数的Wan 2.2模型，加速文本到图像推理至2-8步。

RiO-DETR: DETR for Real-time Oriented Object Detection

Tue, 10 Mar 2026 00:00:00 +0000

本文提出RiO-DETR，一种面向实时旋转目标检测的DETR框架。针对方向语义依赖、角度周期性和搜索空间扩大等挑战，设计了内容驱动的角度估计、旋转校正正交注意力、解耦周期细化以及定向密集O2O机制，在保持实时效率的同时提升角度收敛速度与检测精度。在DOTA-1.0、DIOR-R和FAIR-1M-2.0上的实验表明，该方法实现了新的速度-精度平衡。

Dome-DETR: DETR with density-oriented feature-query manipulation for efficient tiny object detection

Mon, 27 Oct 2025 00:00:00 +0000

本文提出 Dome-DETR，一种面向微小物体检测的高效框架。针对现有方法特征利用不充分和计算成本高的问题，引入轻量级密度聚焦提取器（DeFE）生成紧凑前景掩码，并基于掩码的窗口注意力稀疏化（MWAS）将计算资源集中于关键区域。同时提出渐进自适应查询初始化（PAQI），自适应调节查询分布，提升检测效率与精度。

Dash: 4d hash encoding with self-supervised decomposition for real-time dynamic scene rendering

Sun, 19 Oct 2025 00:00:00 +0000

本文提出DASH，一种实时动态场景渲染框架，采用4D哈希编码结合自监督分解。针对现有平面基动态高斯溅射方法因低秩假设导致特征重叠和渲染质量差的问题，DASH通过自监督分解机制分离动态与静态组件，无需人工标注或预计算掩码，并引入多分辨率4D哈希编码器对动态元素进行显式表示，避免低秩约束，从而减少哈希冲突和冗余，实现高质量实时渲染。

Efficient spiking point mamba for point cloud analysis

Sun, 19 Oct 2025 00:00:00 +0000

本文提出 Spiking Point Mamba (SPM)，这是首个将 Mamba 引入三维点云分析的脉冲神经网络。针对直接适配 Mamba 时存在的时序动态不匹配和脉冲引起的信息损失问题，作者设计了层次动态编码 (HDE) 以增强动态时序建模，并提出 Spiking Mamba Block (SMB) 来学习跨时间步特征并减少脉冲信息丢失。此外，采用非对称 SNN 训练策略进一步提升性能。SPM 可作为高效骨干网络，适用于点云分类、部件分割与重建等任务。

Enhancing Visual Question Answering Via Clustered In-Context Sequence Configuration

Sun, 14 Sep 2025 00:00:00 +0000

本文针对多模态大语言模型在多模态上下文学习中的演示序列配置问题，提出一种基于聚类的上下文配置方法。该方法自适应地对候选数据进行分组，并从每个簇中选取演示样本，以增强序列内多样性并保持语义一致性，从而减少高相似演示带来的归纳偏置，使模型更关注演示的主要意图。在OK-VQA、VQAv2、VizWiz和TextVQA四个视觉问答基准上的实验验证了其有效性。

Create anything anywhere: Layout-controllable personalized diffusion model for multiple subjects

Mon, 30 Jun 2025 00:00:00 +0000

Diffusion models have significantly advanced text-to-image generation, laying the foundation for the development of personalized generative frameworks. However, existing methods lack precise layout controllability and overlook the potential of dynamic features of reference subjects in improving fidelity. In this work, we propose Layout-Controllable Personalized Diffusion (LCP-Diffusion) model, a novel framework that integrates subject identity preservation with flexible layout guidance in a tuning-free approach. Our model employs a Dynamic-Static Complementary Visual Refining module to comprehensively capture the intricate details of reference subjects, and introduces a Dual Layout Control mechanism to enforce robust spatial control across both training and inference stages. Extensive experiments validate that LCP-Diffusion excels in both identity preservation and layout controllability. To the best of our …

D-FINE: Redefine regression task of DETRs as fine-grained distribution refinement

Thu, 01 May 2025 00:00:00 +0000

D-FINE是一种实时目标检测器，通过重新定义DETR中的边界框回归任务实现高精度定位。其核心包含细粒度分布细化（FDR）和全局最优定位自蒸馏（GO-LSD）。FDR将回归从预测固定坐标改为迭代细化概率分布，提供细粒度中间表示；GO-LSD通过自蒸馏将定位知识从最终层传递到浅层，并简化深层残差预测。在COCO上达到54.0%/55.8% AP，124/78 FPS，预训练后达57.1%/59.3% AP，超越现有实时检测器，并显著提升多种DETR模型性能。

Efficient event-based semantic segmentation via exploiting frame-event fusion: A hybrid neural network approach

Fri, 11 Apr 2025 00:00:00 +0000

本文提出一种高效的混合神经网络框架，用于事件相机语义分割。该框架包含处理事件流的脉冲神经网络（SNN）分支和处理帧图像的人工神经网络（ANN）分支，并设计了自适应时间加权（ATW）注入器、事件驱动稀疏（EDS）注入器和通道选择融合（CSF）模块，以充分融合帧与事件的互补时空信息。在DDD17-Seg、DSEC-Semantic和M3ED-Semantic数据集上取得了最先进精度，并在DSEC-Semantic上降低63%能耗。

Spiking point transformer for point cloud classification

Fri, 11 Apr 2025 00:00:00 +0000

本文提出Spiking Point Transformer（SPT），首个基于Transformer的脉冲神经网络框架，用于三维点云分类。SPT设计队列驱动采样直接编码，在降低计算成本的同时保留关键支撑点；并引入混合动力学积分发放神经元（HD-IF），模拟选择性神经元激活，减少对特定人工神经元的过度依赖。在多个真实与合成点云基准上取得领先结果，理论能耗较ANN对应模型降低至少6.4倍。

Enhancing Visual Tracking by Leveraging High-frequency Information within Event Signals

Wed, 01 Jan 2025 00:00:00 +0000

Traditional object trackers struggle in degraded scenarios, lacking sufficient appearance details of moving targets for precise tracking. Recent trackers have integrated highfrequency event signals to assist tracking. However, they neglect the high-temporalresolution motion information inherent in events, limiting their performance especially in occlusion and background clutter. To address these challenges, we propose HFTrack, a novel tracker designed to fully leverage the spatio-temporal high-frequency information within event signals, thereby enhancing the tracking performance. Specifically, we introduce a frequency-based feature enhancement module, which enriches the frame feature with high-frequency components from events in frequency space, capturing detailed appearance information of moving targets. Additionally, we propose a spatio-temporal information decoder with an auto-regressive temporal query, integrating both historical motion cues from events and enhanced spatial features for robust target localization. Experimental results demonstrate that our HFTrack significantly outperforms existing trackers, showcasing its strong ability to track the target under challenging conditions.

Event-based head pose estimation: Benchmark and method

Sun, 29 Sep 2024 00:00:00 +0000

本文针对传统RGB方法在剧烈运动和极端光照下头部姿态估计困难的问题，引入事件相机的高时间分辨率与高动态范围优势。作者构建了两个大规模事件头部姿态数据集，包含282个序列，覆盖不同分辨率与场景；并提出事件头部姿态估计网络EV-HPE，设计了事件时空融合模块和事件运动感知注意力模块，有效结合事件流时空信息，提升姿态估计精度与鲁棒性。

Ee-mllm: A data-efficient and compute-efficient multimodal large language model

Wed, 21 Aug 2024 00:00:00 +0000

Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated satisfactory performance across various vision-language tasks. Current approaches for vision and language interaction fall into two categories: self-attention-based and cross-attention-based methods. However, both approaches present inherent limitations, forcing a trade-off between data and computational efficiency. To address this issue, we introduce the Data-$\textbf{E}$fficient and Compute-$\textbf{E}$fficient $\textbf{MLLM}$ ($\textbf{EE-MLLM}$). Specifically, we modify the original self-attention mechanism in MLLM to a composite attention mechanism. This mechanism has two key characteristics: 1) eliminating the computational overhead of self-attention among visual tokens to achieve $\textbf{compute efficiency}$, and 2) reusing the weights from each layer of LLM to facilitate effective vision-language modality alignment for $\textbf{data efficiency}$. As a result, EE-MLLM significantly outperforms Flamingo with limited training data, and reduces the prefilling time to 79 ms on an H800 GPU, compared to LLaVA’s 277 ms. To further investigate the efficiency of EE-MLLM, we present a training-free variant named EE-MLLM-F, which reduces the computation cost of self-attention-based method without additional training. Experimental results demonstrate the effectiveness of EE-MLLM across a range of benchmarks, including general-purpose datasets like MMBench and SeedBench, as well as fine-grained tasks such as TextVQA and DocVQA.

Event-assisted low-light video object segmentation

Sun, 16 Jun 2024 00:00:00 +0000

本文针对低光照条件下视频目标分割（VOS）性能严重下降的问题，提出一种利用事件相机数据辅助分割的新框架。该方法包含两个关键模块：自适应跨模态融合（ACMF）模块，用于提取并融合图像与事件模态特征以抑制噪声干扰；事件引导记忆匹配（EGMM）模块，用于修正低光下查询帧与记忆帧之间的相似度计算误差。实验表明，该方法在合成和真实低光数据集上均能显著提升分割精度，生成更准确的目标掩码。

Scene adaptive sparse transformer for event-based object detection

Sun, 16 Jun 2024 00:00:00 +0000

本文针对事件相机目标检测中Transformer计算成本过高的问题，提出场景自适应稀疏Transformer（SAST）。该方法通过窗口-令牌协同稀疏化与场景特定稀疏优化，在保持低计算量的同时实现高检测性能，并能根据场景复杂度自适应调整稀疏程度，有效平衡了检测精度与效率。

Deep multi-threshold spiking-UNet for image processing

Fri, 14 Jun 2024 00:00:00 +0000

本文提出Spiking-UNet，将脉冲神经网络与U-Net架构相结合用于图像处理任务。针对脉冲传播导致的信息损失问题，设计多阈值脉冲神经元以增强信息传递能力；同时采用基于预训练U-Net的转换与微调训练策略，有效解决了训练难题。在图像分割和去噪等任务上验证了所提方法的有效性。

Eoformer: Edge-oriented transformer for brain tumor segmentation

Sun, 01 Oct 2023 00:00:00 +0000

本文提出边缘导向Transformer（EoFormer），用于脑肿瘤MRI图像分割。该方法采用CNN-Transformer混合编码器，CNN提取局部低级特征，Transformer建模长距离依赖以生成全局高级特征；解码器集成边缘导向Sobel与Laplacian锐化模块，增强边缘信息。同时引入高效注意力与重参数化技术，提升特征表示能力与分割精度。

Deep spiking-unet for image processing

Sun, 01 Jan 2023 00:00:00 +0000

本文提出一种深度脉冲U-Net架构，将脉冲神经网络的生物合理性与U-Net的多尺度特征提取能力相结合，用于图像处理任务。通过脉冲神经元替代传统激活函数，在保持图像处理性能的同时显著降低计算能耗，为低功耗边缘端图像处理提供了新思路。

Dual progressive prototype network for generalized zero-shot learning

Fri, 01 Jan 2021 00:00:00 +0000

本文提出一种双渐进原型网络用于广义零样本学习。该方法通过渐进式地学习可见类和不可见类的原型表示，并利用双分支结构建模视觉特征与语义特征之间的映射，有效缓解了零样本学习中的领域偏移和投影偏差问题。在多个标准基准数据集上的实验验证了所提方法的有效性。

Task-independent knowledge makes for transferable representations for generalized zero-shot learning

Fri, 01 Jan 2021 00:00:00 +0000

本文针对广义零样本学习（GZSL）中可见类与未见类之间的表示偏差问题，提出利用任务无关知识来学习可迁移的视觉表示。通过在大规模辅助数据上预训练或引入外部知识，使模型捕获与类别标签无关的通用特征，从而提升对未见类别的识别能力。在多个基准数据集上验证了方法的有效性。