Yang Liu (刘阳)

IEEE/ACM Member, CCF/CAAI/CSIG Member
Email: liuy856@mail.sysu.edu.cn
Office: School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China

I am currently an associate professor at HCP Lab, School of Computer Science and Engineering, Sun Yat-sen University. From 2019-2024, I worked as a posdoctoral fellow (research fellow) at School of Computer Science and Engineering, Sun Yat-sen University, working with Prof. Liang Lin. I obtained my Ph.D degree of Telecommunications and Information Systems in June 2019 at Phoenix-eye (XD145) Lab, School of Telecommunications Engineering, Xidian University, advised by Prof. Zhaoyang Lu, Prof. Jing Li and Prof. Tao Yang. I serve as the reviewer of IEEE T-PAMI, T-IP, T-NNLS, T-MM, T-CSVT, CVPR, ICCV, AAAI, ACM MM, ECCV, etc. [中文版]

Looking for self-motivated Masters, RAs, visiting students, and interns. Please drop me an email if interested.

  • Computer Vision: Cross-modal Reasoning, Spatial-temporal Representation Learning
  • Machine Learning: Causality Inference, Self-supervised Learning, Transfer Learning
  • Embodied AI: Embodied Interaction, Embodied Manipulation, Robotic Control
    • CCF ChinaSoft 2023 Robotic Big Model and Embodied Intelligence Challenge, Third Prize, 2023.
    • The Third Guangdong Province Young Computer Science Academic Show, First Prize, 2023.
    • National Scholarship for PhD Students, 2018.
    • National English Competition, Third Prize, 2013.
    • National Mathematical Modeling Competition, Second Prize, 2012.


    News

    • 2024-07: We release the paper list for Embodied AI !
    • 2024-07: One paper is accepted by ACM MM 2024!
    • 2024-06: The book of multimodal large model 《多模态大模型:新一代人工智能技术范式》 is selected for the Sun Yat-sen University Undergraduate Textbook Series!
    • 2024-05: One first-author T-PAMI paper is selected as the ESI Hot Cited Paper!
    • 2024-05: One first-author T-PAMI paper is selected as the ESI Highly Cited Paper!
    • 2024-04: The book of multimodal large model 《多模态大模型:新一代人工智能技术范式》 is published!
    • 2023-12: I won the third prize of CCF ChinaSoft 2023 Robotic Big Model and Embodied Intelligence Challenge!
    • 2023-11: One first-author T-IP paper is selected as the ESI Hot Cited Paper!
    • 2023-11: One first-author T-IP paper is selected as the ESI Highly Cited Paper!
    • 2023-10: One accepted ACM MM 2023 paper is recommended as an oral!
    • 2023-10: An invention patent has been granted.
    • 2023-10: I have been selected as a member of the CSIG Visual Big Data Committee.
    • 2023-07: One paper is accepted by ACM MM 2023!
    • 2023-07: Two papers are accepted by ICCV 2023!
    • 2023-06: One paper is accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence!
    • 2023-06: I won the first prize in the Third Guangdong Province Young Computer Science Academic Show.
    • 2023-06: One paper is accepted by ICIP 2023.
    • 2023-04: One paper is accepted by IJCAI 2023!
    • 2023-03: The open-source framework for causal discovery and visual-linguistic reasoning Causal-VLReasoning is online!
    • 2023-03: I become a member of CAAI (Chinese Association for Artificial Intelligence).
    • 2023-03: I become a member of CCF (China Computer Federation).
    • 2023-03: I am invited as the Program Committee (PC) Member for the Computer Graphics International (CGI) 2023.
    • 2023-03: One paper is accepted by Information Sciences.
    • 2022-12: One review paper about causal reasoning is reported in social media by Machine Intelligence Research.
    • 2022-11: I received the funding from Guangdong Basic and Applied Basic Research Foundation 2023.
    • 2022-11: One review paper for causal reasoning is published online by Machine Intelligence Research.
    • 2022-11: I am invited as the Program Committee (PC) Member for the Computational Visual Media Conference 2023.
    • 2022-11: I am invited as the reviewer of IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023.
    • 2022-09: I become a member of IEEE Industrial Electronics Society.
    • 2022-08: My TCGL paper published in IEEE T-IP is invited to be exhibited as the poster in Valse 2022.
    • 2022-08: One review paper about causal reasoning is accepted by Machine Intelligence Research.
    • 2022-08: I am invited as the Program Committee (PC) Member for the AAAI 2023.
    • 2022-06: I become a member of CSIG.
    • 2022-05: One paper is accepted by IEEE Transactions on Industrial Informatics.
    • 2022-04: I received the funding from Science and Technology Projects in Guangzhou.
    • 2022-03: One paper is accepted by CVPR 2022 as an oral presentation.
    • 2022-01: One paper is accepted by IEEE Transactions on Image Processing. Code is available.
    • 2022-01: One paper is accepted by ICASSP 2022.
    • 2022-01: The code for TCGL is available.
    • 2021-12: I am invited as the reviewer of Chinese Conference on Pattern Recognition and Computer Vision (PRCV) 2022.
    • 2021-11: I am invited as the reviewer of IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022.
    • 2021-10: I start working as a research associate professor at Sun-Yat-Sen University.
    • 2021-05: One paper is accepted by IEEE Transactions on Image Processing. Code is available.
    • 2021-04: The code for SAKDN is available.
    • 2021-02: I become a Member of the IEEE.
    • 2020-12: I am invitied as the reviewing expert for NSFC.
    • 2020-11: I received the funding from National Natural Science Foundation of Guangdong Province of China 2021.
    • 2020-09: I received the funding from National Natural Science Foundation of China 2021.
    • 2020-06: I received the funding from China Postdoctoral Science Foundation 2020.
    • 2020-03: I received the funding from Fundamental Research of the Central Universities 2020.
    • 2020-01: My Ph.D Thesis (psw:ithd) is available now.
    • 2019-10: Codes and datasets for IEEE TIP is available.
    • 2019-10: One paper is accepted by IEEE Transactions on Image Processing.
    • 2019-08: I start working as a postdoctoral fellow at Sun-Yat-Sen University.
    • 2019-06: I obtain my Ph.D degree from Xidian University.

    Open-source Framework

    1. CausalVLR: A Toolbox and Benchmark for Visual-Linguistic Causal Reasoning
      Yang Liu, Weixing Chen, Guanbin Li, Liang Lin
      GitHub stars GitHub forks GitHub issues
    2. CausalVLR is a python open-source framework for causal relation discovery, causal inference that implements state-of-the-art causality learning algorithms for various visual-linguistic reasoning tasks, such as VQA, embodied interaction, model robustness, medical report generation, etc.

    Demos of Embodied Agents

    1. Robotic Inspection in Extreme Environments
    2. The robotic dog actively checks the surrounding states of extreme environments with the dark light condiction, including checking the machine states, detecting anomaly intrusion and human-robot dialog.
    3. Multi-robot Collaboration
    4. Multiple robotic dogs work collaboratively to finish complex tasks.
    5. Outdoor Navigation and Control
    6. The robotic dog actively explores the unknown environment and finds the optimal route for navigation.
    7. Embodied Interactive Robot
    8. We implemtent and integrate active visual sensing, visual navigation, embodied dialog, robotic manipulation and robotic arm controling algorithms into the robotic dog. The robotic dog can provide vairous services for humans in complex and dynamic environments.
    9. Embodied Interactive Agent for the Coffee Scene
    10. The multimodal embodied interactive agent (MEIA) can translate high-level tasks expressed in natural language into a sequence of executable actions. The MEIA can generate executable action plans based on diverse requirements and the robot's capabilities. Furthermore, we construct an embodied question answering dataset based on a dynamic virtual cafe environment with the large language model. The MEIA obtained the third prize of CCF ChinaSoft 2023 Robotic Big Model and Embodied Intelligence Challenge.

    Book

    1. Multimodal Large Models: The New Paradigm of Artificial General Intelligence
      《多模态大模型:新一代人工智能技术范式》
      Publishing House of Electronics Industry (PHE), ISBN 978-7-121-47547-4, 2024.
      电子工业出版社,ISBN 978-7-121-47547-4, 2024.
      Yang Liu, Liang Lin
      刘阳, 林倞
      Sun Yat-sen University Undergraduate Textbook Series
      GitHub stars GitHub forks GitHub issues

    Selected Papers

    1. Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI
      Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, Liang Lin
      arXiv:2407.06886.
      GitHub stars GitHub forks GitHub issues
    2. Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering
      Yang Liu, Guanbin Li, Liang Lin
      IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2023. (ESI Highly Cited & Hot Paper)
      GitHub stars GitHub forks GitHub issues
    3. Diversity Matters: User-Centric Multi-Interest Learning for Conversational Movie Recommendation
      Yongsen Zheng, Guohua Wang, Yang Liu, Liang Lin
      ACM International Conference on Multimedia (ACM MM), 2024.
    4. MEIA: Multimodal Embodied Perception and Interaction in Unknown Environments
      Yang Liu, Xinshuai Song, Kaixuan Jiang, Weixing Chen, Jingzhou Luo, Guanbin Li, Liang Lin
      arXiv:2402.00290
    5. VidMaestro: Towards Photo-realistic and High-dynamic Video Generations
      Binbin Yang, Kangyang Xie, Xinyu Xiao, Meng Wang, Yang Liu, Jingdong Chen, Ming Yang, Liang Lin
      Submitted to TMM, arXiv, 2024
    6. CausalGPT: Illuminating Faithfulness and Causality for Knowledge-based Reasoning with LLMs
      Ziyi Tang, Ruilin Wang, Weixing Chen, Yongsen Zheng, Yang Liu, Keze Wang, Tianshui Chen, Liang Lin
      arXiv:2308.11914
      GitHub stars GitHub forks GitHub issues
    7. ODMixer: Fine-grained Spatial-temporal MLP for Metro Origin-Destination Prediction
      Yang Liu, Binglin Chen, Yongsen Zheng, Lechao Cheng, Guanbin Li, Liang Lin
      Submitted to TKDE, arXiv:2404.15734
      GitHub stars GitHub forks GitHub issues
    8. Cross-Modal Causal Intervention for Medical Report Generation
      Weixing Chen, Yang Liu, Ce Wang, Jiarui Zhu, Shen Zhao, Guanbin Li, Cheng-Lin Liu, Liang Lin
      Arxiv, 2024.
      [Paper] [Code & Dataset] [BibTex]

      GitHub stars GitHub forks GitHub issues
    9. Visual Causal Scene Refinement for Video Question Answering
      Yushen Wei*, Yang Liu*, Hong Yan, Guanbin Li, Liang Lin
      ACM International Conference on Multimedia (ACM MM), 2023. (Oral)
      GitHub stars GitHub forks GitHub issues
    10. SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training
      Hong Yan, Yang Liu, Yushen Wei, Zhen Li, Guanbin Li, Liang Lin
      IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
      GitHub stars GitHub forks GitHub issues
    11. Enhanced Soft Label for Semi-Supervised Semantic Segmentation
      Jie Ma, Chuan Wang, Yang Liu, Liang Lin, Guanbin Li
      IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
      GitHub stars GitHub forks GitHub issues
    12. DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback
      Junfan Lin, Yuying Zhu, Lingbo Liu, Yang Liu, Guanbin Li, Liang Lin
      International Joint Conference on Artificial Intelligence (IJCAI), 2023.
      GitHub stars GitHub forks GitHub issues
    13. Hybrid-Order Representation Learning for Electricity Theft Detection
      Yuying Zhu, Yang Zhang, Lingbo Liu, Yang Liu, Guanbin Li, Mingzhi Mao, Liang Lin
      IEEE Transactions on Industrial Informatics (T-II), 2023.
      GitHub stars GitHub forks GitHub issues
    14. Urban Regional Function Guided Traffic Flow Prediction
      Kuo Wang, Lingbo Liu, Yang Liu, Guanbin Li, Liang Lin
      Information Sciences (INS), 2023.
    15. Dual adversarial adaptation for cross-device real-world image super-resolution
      Xiaoqian Xu, Pengxu Wei, Weikai Chen, Yang Liu, Mingzhi Mao, Liang Lin, Guanbin Li
      IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. (Oral)
      GitHub stars GitHub forks GitHub issues
    16. Causal Reasoning Meets Visual Representation Learning: A Prospective Study
      Yang Liu, Yushen Wei, Hong Yan, Guanbin Li, Liang Lin
      Machine Intelligence Research (MIR), 2022. (Top-10 Downloads)
      GitHub stars GitHub forks GitHub issues
    17. Cross-modal knowledge distillation for Vision-to-Sensor action recognition
      Jianyuan Ni, Raunak Sarbajna, Yang Liu, Anne HH Ngu, Yan Yan
      IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022.
    18. TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning
      Yang Liu, Keze Wang, Lingbo Liu, Haoyuan Lan, Liang Lin
      IEEE Transactions on Image Processing (T-IP), 2022. (ESI Highly Cited & Hot Paper)
      GitHub stars GitHub forks GitHub issues
    19. Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition
      Yang Liu, Keze Wang, Guanbin Li, Liang Lin
      IEEE Transactions on Image Processing (T-IP), 2021.
      GitHub stars GitHub forks GitHub issues
    20. Deep Image-to-Video Adaptation and Fusion Networks for Action Recognition
      Yang Liu, Zhaoyang Lu, Jing Li, Tao Yang, Chao Yao
      IEEE Transactions on Image Processing (T-IP), 2020.
      GitHub stars GitHub forks GitHub issues
    21. Hierarchically Learned View-Invariant Representations for Cross View Action Recognition
      Yang Liu, Zhaoyang Lu, Jing Li, Tao Yang
      IEEE Transactions on Circuits and Systems for Video Technology (T-CSVT), 2019.
      GitHub stars GitHub forks GitHub issues
    22. Global Temporal Representation based CNNs for Infrared Action Recognition
      Yang Liu, Zhaoyang Lu, Jing Li, Tao Yang, Chao Yao
      IEEE Signal Processing Letters (SPL), 2018.
      GitHub stars GitHub forks GitHub issues

    PhD Dissertation

    1. Cross-domain Human Action Recognition via Transfer Learning (基于迁移学习的跨域人体行为识别研究)
      PhD Dissertation (博士学位论文), Xidian University (西安电子科技大学), June 30, 2019.
      Yang Liu
      Supervisor:Prof. Zhaoyang Lu

    Academic services

      Reviewer for Journals
    • Advanced Science
    • IEEE Transactions on Pattern Analysis and Machine Intelligence
    • IEEE Transactions on Image Processing
    • IEEE Transactions on Neural Networks and Learning Systems
    • IEEE Transactions on Cybernetics
    • IEEE Transactions on Multimedia
    • IEEE Transactions on Circuits and Systems for Video Technology
    • IEEE Transactions on Human-Machine Systems
    • IEEE Transactions on Mobile Computing
    • IEEE Signal Processing Letters
    • IEEE Robotics and Automation Letters
    • ACM Transactions on Multimedia Computing Communications and Applications
    • ACM Transactions on Information Systems
    • IET Computer Vision
    • Pattern Recognition
    • Information Fusion
    • Neural Networks
    • Pattern Recognition Letters
    • Computer Vision and Image Understanding
    • Signal Processing: Image Communication
    • Machine Vision and Applications
    • Journal of Visual Communication and Image Representation
    • Visual Computer
      Program Committee (PC)/Reviewer for Conferences
    • IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
    • IEEE/CVF International Conference on Computer Vision (ICCV)
    • European Conference on Computer Vision (ECCV)
    • ACM Multimedia (ACM MM)
    • AAAI Conference on Artificial Intelligence (AAAI)
    • IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • Computational Visual Media Conference (CVM)
    • Computer Graphics International (CGI)
    • Chinese Conference on Pattern Recognition and Computer Vision (PRCV)
    • ACM international joint conference on pervasive and ubiquitous computing (UbiComp)
    • IEEE International Semantic Web Conference (ISWC)
    • IEEE International Symposium on Circuits and Systems (ISCAS)
    • International AAAI Conference on Web and Social Media (ICWSM)
    • CCF BigData Conference (CCF BigData)