复旦大学跨媒体智能计算实验室（CMIT）主要研究方向包括人工智能、医学影像智能分析、跨媒体智能、视频大数据等，累计发表高水平学术论文100余篇。实验室研究成果与实际应用紧密结合，已完成各类已投入实际应用的产品与系统40余项，在国家核心区、新疆、上海等地成功示范应用。研发的产品设备通过了公安部产品鉴定和公安部产品列装测试，被公安部正式列装。研发的海量视频信息处理平台具备Tb/s级高清视频数据的在线分析处理能力，并在新疆反恐防暴、国家核心区被成功应用示范。近年来，与复旦附属华山医院、复旦附属儿科医院、复旦附属肿瘤医院、长海医院、上海市第一人民医院、上海市公共卫生中心、上海市疾病控制中心、上海市眼病防治中心等十余家三甲医院开展了长期稳定合作，在AI赋能医疗领域的人工智能治理解决方案、数据与算法治理体系、人工智能社会实验、大规模标准化医学数据集、人工智能医疗装备、新冠肺炎抗疫、人工智能2030、JF重大基础研究等重大项目上取得了一系列具有影响力的研究成果，并通过产学研合作开展规模化落地应用。

学术论文 Publications

2022

Jilan Xu, Junlin Hou, Yuejie Zhang, Rui Feng, Rui-Wei Zhao, Tao Zhang, Xuequan Lu, Shang Gao, CREAM: Weakly-Supervised Object Localization via Class RE-Activation Map, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, Louisiana, United States, June 2022.

Jiashuo Yu, Ying Cheng, Rui-Wei Zhao, Rui Feng, Yuejie Zhang, MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsing, ACM International Conference on Multimedia (ACM MM), Lisbon, Portugal, October 2022.

Jiashuo Yu#, Jinyu Liu#, Rui-Wei Zhao, Rui Feng, Yuejie Zhang, Modality-Aware Contrastive Instance Learning with Self-Distillation for Audio-Visual Violence Detection, ACM International Conference on Multimedia (ACM MM), Lisbon, Portugal, October 2022.

Xinyu Huang, Youcai Zhang, Ying Cheng, Weiwei Tian, Ruiwei Zhao, Rui Feng, Yuejie Zhang, Yaqian Li, Yandong Guo, Xiaobo Zhang, IDEA: Increasing Text Diversity via Online Multi-Label Recognition for Vision-Language Pre-training, ACM International Conference on Multimedia (ACM MM), Lisboa, Portugal, October 2022.

Jinyu Liu, Ying Cheng, Yuejie Zhang, Ruiwei Zhao, Rui Feng, Self-Supervised Video Representation Learning with Motion-Contrastive Perception, IEEE International Conference on Multimedia and Expo (ICME), Taipei, Taiwan, China, July 2022.

Shuyu Miao, Shanshan Du, Rui Feng, Yuejie Zhang, Huayu Li, Tianbi Liu, Lin Zheng, Weiguo Fan, Balanced single-shot object detection using cross-context attention-guided network, Pattern Recognition (PR), Vol. 122, 108258, 2022.

Chen Yangdong, Zhang Zhaolong, Wang Yanfei, Yuejie Zhang, Rui Feng, Tao Zhang, Weiguo Fan, AE-Net: Fine-grained Sketch-based Image Retrieval via Attention-Enhanced Network. Pattern Recognition (PR), Vol. 122, 108291, 2022.

Zheng Yi, Zhang Yuejie, Feng Rui, Zhang Tao, Fan Weiguo, Stacked multimodal attention network for context-aware video captioning. IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), Vol. 32, pp. 31-42, January 2022.

Junlin Hou, Jilan Xu, Nan Zhang, Yi Wang, Yuejie Zhang, Xiaobo Zhang, Rui Feng, CMC_v2: Towards More Accurate COVID-19 Detection with Discriminative Video Priors, European Conference on Computer Vision Workshop (ECCVW), Tel Aviv, Israel, October 2022.

Junlin Hou, Jilan Xu, Nan Zhang, Yuejie Zhang, Xiaobo Zhang, Rui Feng, Boosting COVID-19 Severity Detection with Infection-Aware Contrastive Mixup Classification, European Conference on Computer Vision Workshop (ECCVW), Tel Aviv, Israel, October 2022.

2021

Ying Cheng, Ruize Wang, Jiashuo Yu, Rui-Wei Zhao, Yuejie Zhang, Rui Feng, Exploring Logical Reasoning for Referring Expression Comprehension, ACM International Conference on Multimedia (ACM MM), Chengdu, China, October 2021.

Junlin Hou, Jilan Xu, Rui Feng, Yuejie Zhang, Fei Shan, Weiya Shi, CMC-COV19D: Contrastive Mixup Classification for COVID-19 Diagnosis, IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Montreal, British Columbia, Canada, October 2021. (1st prize in the ICCV 2021 Covid-19 Diagnosis Competition of AI-enabled Medical Image Analysis Workshop)

Junlin Hou, Jilan Xu, Longquan Jiang, Shanshan Du, Rui Feng, Yuejie Zhang, Fei Shan, Xiangyang Xue, Periphery-aware COVID-19 Diagnosis with Contrastive Representation Enhancement, Pattern Recognition (PR), Vol. 118, 108005, 2021.

Shuyu Miao, Rui Feng, Object-Oriented Relational Distillation for Object detection, IEEE International Conference on Acoustics, Speech, & Signal Processing (ICASSP), Toronto, Ontario, Canada, June 2021.

Ying Cheng, Mengyu He, Jiashuo Yu, Rui Feng, Improving Multimodal Speech Enhancement By Incorporating Self-supervised And Curriculum Learning, IEEE International Conference on Acoustics, Speech, & Signal Processing (ICASSP), Toronto, Ontario, Canada, June 2021.

Shuaicheng Li, Shuyu Miao, Rui-Wei Zhao, Rui Feng, Temporally Coarse to Fine Snippets Relationship Learning with Graph Convolution for Temporal Action Proposal Generation, IEEE International Conference on Multimedia and Expo (ICME), Virtual, July 2021.

Jiashuo Yu, Ying Cheng, Rui Feng, MPN: Multimodal Parallel Network for Audio-Visual Event Localization, IEEE International Conference on Multimedia and Expo (ICME), Virtual, July 2021.

Jiawang Cao, Lulu Jiang, Junlin Hou, Longquan Jiang, Ruiwei Zhao, Weiya Shi, Fei Shan, Rui Feng, Exploiting Deep Cross-Slice Features From CT Images For Multi-Class Pneumonia Classification, IEEE International Conference on Image Processing (ICIP), Anchorage, Alaska, United States, September 2021.

Lulu Jiang, Jiawang Cao, YuXin Li, Xuelin Liu, Ruiwei Zhao, Rui Feng, Atrous High-Order Attention Network for Medical Image Segmentation, International Symposium on Bioinformatics Research and Applications (ISBRA), China, November 2021.

Nan Zhang, Junlin Hou, Rui-Wei Zhao, Rui Feng, Yuejie Zhang, Semi-supervised Medical Image Segmentation with Distribution Calibration and Non-local Semantic Constraint, International Conference on Bioinformatics and Biomedicine (BIBM), Virtual, December 2021.

Wei Ji#, Weiwei Tian#, Wenbin Pan, Siqi Sun, Hui Jiang, Rui Feng, Yuejie Zhang, Rui-Wei Zhao, Gang Jin, CellDet: Dual-Task Cell Detection Network for Quantitative IHC-Stained Image Analysis, International Conference on Bioinformatics and Biomedicine (BIBM), Virtual, December 2021.

Xu Jilan, Hou Junlin, Zhang Yuejie, Feng Rui, Ruan Chunyang, Zhang Tao, Fan Weiguo, Histopathology Image Analysis with deformation Representation Learning, International Conference on Bioinformatics and Biomedicine (BIBM), Virtual, December 2021.

Wenxi Li, Zhuoqun Cao, Qian Wang, Songjian Chen, Rui Feng, Learning Error-Driven Curriculum for Crowd Counting, IEEE International Conference on Pattern Recognition (ICPR), Milan, Italy, January 2021.

2020

Ying Cheng, Ruize Wang, Zhihao Pan, Rui Feng, Yuejie Zhang, Look, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learning, ACM International Conference on Multimedia (ACM MM), Seattle, Washington, United States, October 2020.

Zhaolong Zhang, Yuejie Zhang, Rui Feng, Tao Zhang, Weiguo Fan, Zero-Shot Sketch-based Image Retrieval via Graph Convolution Network, AAAI Conference on Artificial Intelligence (AAAI), New York, New York, United States, February 2020.

Xinlong Xiao, Yuejie Zhang, Rui Feng, Tao Zhang, Shang Gao, Weiguo Fan, Video Captioning With Temporal And Region Graph Convolution Network, IEEE International Conference on Multimedia and Expo (ICME), London, United Kingdom, July 2020.

Yang Juexu, Zhang Yuejie, Feng Rui, Tao Zhang, Fan Weiguo, Deep Reinforcement Hashing with Redundancy Elimination for Effective Image Retrieval, Pattern Recognition (PR), Vol. 100, 107116, 2020.

Wang Yanfei, Huang Fei, Zhang Yuejie, Feng Rui, Zhang Tao, Fan Weiguo, Deep Cascaded Cross-modal Correlation Learning for Fine-grained Sketch-based Image Retrieval, Pattern Recognition (PR), Vol. 100, 107148, 2020.

Li Wenjie, Zheng Yi, Zhang Yuejie, Feng Rui, Tao Zhang, Fan Weiguo, Cross-modal Retrieval with Dual Multi-angle Self-attention, Journal of the Association for Information Science and Technology (JASIST), Vol. 72, pp. 46-65, 2020.

Shuyu Miao, Rui Feng, Yuejie Zhang, Representation Reconstruction Head for Object Detection, IEEE International Conference on Image Processing (ICIP), United Arab Emirates, October 2020. EI: 20210109724322.

Shuyu Miao, Rui Feng, Yuejie Zhang, Weiguo Fan, Learning Class-Based Graph Representation for Object Detection, European Conference on Artificial Intelligence (ECAI), Santiago, Compostela, August 2020.

Yang Shuyi, Jiang Longquan, Cao Zhuoqun, Wang Liya, Cao Jiawang, Feng Rui, Zhang Zhiyong, Xue Xiangyang, Shi Yuxin, Shan Fei, Deep Learning for Detecting Corona Virus Disease 2019 (COVID-19) on High-resolution Computed Tomography: a Pilot Study, Annals of Translational Medicine (ATM), Vol. 8, No. 7, 2020.

Qian Wang, Wenxi Li, Songjian Chen, Rui Feng, A Dynamic-Attention on Crowd Region with Physical Optical Flow Features for Crowd Counting, IEEE International Joint Conference on Neural Networks (IJCNN), Glasgow, United Kingdom, July 2020.

Tong Wei, Junlin Hou, Rui Feng, Fuzzy Graph Neural Network for Few-Shot Learning, IEEE International Joint Conference on Neural Networks (IJCNN), Glasgow, United Kingdom, July 2020.

Huayu Li, Shuyu Miao, Rui Feng, DG-FPN: Learning Dynamic Feature Fusion Based on Graph Convolution Network for Object Detection, IEEE International Conference on Multimedia and Expo (ICME), London, United Kingdom, July 2020.

Bo Zhang, Rui Feng, On the Possibility of Artificial Consciousness, Basic & Clinical Pharmacology & Toxicology, Vol. 127, pp. 6-9, 2020.

Jianxu Luo, Jialiang Wang, Bin Liu, Rui Feng, Lina Lu, Haidong Zhou, Automated Diabetic Retinopathy Grading and Lesion Detection based on the Modified R-FCN Object Detection Algorithm, IET Computer Vision, Vol. 14, No. 1, pp.1-8, 2020.

Before 2020

Zhanxiong Wang, Keke He, Yanwei Fu, Rui Feng, Yu-Gang Jiang, and Xiangyang Xue, Multi-task Deep Neural Network for Joint Face Recognition and Facial Attribute Prediction, ACM International Conference on Multimedia Retrieval(ACM ICMR), Bucharest, Romania, June 2017.

Keke He, Zhanxiong Wang, Yanwei Fu, Rui Feng, Yu-Gang Jiang, Xiangyang Xue, Adaptively Weighted Multi-task Deep Network for Person Attribute Classification, ACM International Conference on Multimedia (ACM MM), Mountain View, California, United States, October 2017.

Shi Zhong, Jian Pu, Yu-Gang Jiang, Rui Feng, Xiangyang Xue, Flexible Multi-task Learning with Latent Task Grouping, Neurocomputing, Vol. 189, pp. 179-188, 2016.

Yong Cheng, Zhengxiang Cai, Rui Feng, Cheng Jin,Yuejie Zhang, Tao Zhang, Cross-Modal Image-Tag Relevance Learning for Social Images, ACM International Conference on Multimedia (ACM MM), Brisbane, Australia, October 2015.

Yu-Gang Jiang, Yanran Wang, Rui Feng, Xiangyang Xue, Yingbin Zheng, Hanfang Yang, Understanding and Predicting Interestingness of Videos, AAAI Conference on Artificial Intelligence (AAAI), Bellevue, Washington, United States, July 2013.

Yanran Wang, Qi Dai, Rui Feng, Yu-Gang Jiang, Beauty is Here: Evaluating Aesthetics in Videos Using Multimodal Features and Free Training Data, ACM International Conference on Multimedia (ACM MM), Barcelona, Spain, October 2013.

Mingmin Chi, Peiwu Zhang, Yingbin Zhao, Rui Feng, Xiangyang Xue, Web Image Retrieval ReRanking with Multi-view Clustering, International World Wide Web Conference (WWW), April 2009.

已授权专利 Authorized Patents

202110367156.3 一种高速行人身份识别方法及装置

202110208326.3 一种实时行人检测与重识别方法及装置

202110208096.0 基于视听网络的多模态语音分离方法及装置

202011610911.8 一种有限元分析参数贝叶斯优化方法及装置

202010696441.5 面向校园的音视频智能分析系统

202010485289.6 一种基于web的分离式前端图像渲染方法

202010248309.8 糖网病眼底图像的样本数据标注系统

202010074352.7 基于嵌入式平台的轻量级视频分析系统

201911118136.1 基于长短时记忆神经网络的说话人分段标注方法及装置

201911100964.2 基于注意力机制深度学习网络的目标检测方法及装置

201911095764.2 基于光流融合型深度神经网络的人群密度估计方法及装置

201910509308.1 面向超高分辨率图像分析的高精度量化加速方法

201910276412.0 一种基于FPGA的超高分辨率视频图像实时校准方法

201810435661.5 基于残量化卷积神经网络的人脸识别方法及装置

201710226448.9 基于Kinect摄像头的高精度室内人物测距方法

201710226293.9 一种分布式大规模视频流处理系统

201710226292.4 基于深度卷积神经网络模型的大规模人脸识别方法

201610619786.4 基于FPGA+ARM多层卷积神经网络的手写体字符识别方法

201610615714.2 基于FPGA的深度卷积神经网络实现方法

201510428029.4 一种基于目标跟踪的行人徘徊检测算法

201410002727.3 基于机器学习的农药喷洒检测方法

201310695785.4 低碳节能实时温湿度采集系统

201310238940.X 一种基于分布式视频检测的座位统计方法

201310238694.8 基于多摄像头融合的上座率统计方法

201210240036.8 基于有限状态机的道路车辆拥堵判别方法

201210239902.1 基于视频的机动车绿灯跟进违章行为的检测方法

201110325073.4 基于广义霍夫变换的交通信号灯位置检测方法

201010167001.7 一种基于视频的过境车辆快速检测方法

201010166981.9 对道路图像进行车牌识别的方法和系统

软件著作权 Software Copyrights

2020SR0715744 视频行人目标重识别系统

2020SR0556142 车灯检测软件系统

2020SR0478157 复杂场景下车辆身份识别系统

2020SR0272777 面向说话人识别的样本数据标注系统V1.1.0

2020SR0272771 多语种智能识别系统V1.1.0

2020SR0132125 面向糖网病的眼底图像数据标注系统V1.1.0

2019SR0581709 人脸识别召回率测试系统V1.0

2019SR0018377 超高清视频图像实时处理系统V1.0

2017SR427388 基于人脸识别的实验教学系统软件V1.0

2017SR424416 基于视频分析的活体检测系统软件V1.0

2017SR424356 爱羽球运动社交平台软件V1.0

2017SR416566 通用车标识别软件V1.0

2017SR386706 大数据服务平台软件V1.0

2017SR257387 高速公路收费站综合业务管理平台软件V1.0

2017SR257331 基于深度学习敏感图像侦查系统V1.0