Sijin Chen(思锦 陈)

Sijin Chen(思锦 陈)

Fudan University

I am currently a final-year Master’s student in Artificial Intelligence at Fudan University (Sep. 2021 - Jun. 2024, expected), where Prof. Tao Chen is my advisor. I am also fortunate to work closely with Dr. Hongyuan Zhu from A*STAR, Singapore, and Dr. Gang Yu, Dr. Xin Chen, and Dr. Chi Zhang from Tencent. Before this, I obtained my Bachelor’s degree in Data Science and Big Data Technology also from Fudan University (Sep. 2017 - Jun. 2021).

My long-term research goal is to develop vision-language systems that can comprehend, reason, and envision the physical world.

📣 I am actively looking for researcher / Ph.D. opportunities. Please check out my resume here.

🔥 News

  • Apr. 2024.  🎉🎉 Our state-of-the-art 3D dense captioning method Vote2Cap-DETR++ , is accepted to T-PAMI 2024.
  • Feb. 2024.  🎉🎉 Our Large Language 3D Assistant, LL3DA , is accepted to CVPR 2024.
  • Jan. 2024.  🐧🐧 Join Tencent as a research intern, working on 3D generation.
  • Oct. 2023.  🥇🥇 Win the Scan2Cap Challenge at ICCV 2023.
  • Feb. 2023.  🎉🎉 Our Vote2Cap-DETR paper is accepted to CVPR 2023.

📝 Selected Publications

CVPR 2024
sym

LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
CVPR 2024 |
Sijin Chen, Xin Chen, Chi Zhang, Mingsheng Li, Gang Yu, Hao Fei, Hongyuan Zhu, Jiayuan Fan, Tao Chen$^{\dagger}$

project | arXiv | github | youtube

  • Propose a Large Language 3D Assistant that responds to both visual interactions and textual instructions in complex 3D environments.
T-PAMI 2024
sym

Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning
T-PAMI 2024 |
Sijin Chen, Hongyuan Zhu, Mingsheng Li, Xin Chen, Peng Guo, Yinjie Lei, Gang Yu, Taihao Li, Tao Chen$^{\dagger}$

paper | arXiv | github

  • Decoupled feature extraction and task decoding for 3D Dense Captioning.
CVPR 2023
sym

End-to-End 3D Dense Captioning with Vote2Cap-DETR
CVPR 2023 |
Sijin Chen, Hongyuan Zhu, Xin Chen, Yinjie Lei, Gang Yu, Tao Chen$^{\dagger}$

paper | arXiv | github | youtube

  • We address 3D Dense Captioning as a set prediction problem with parallel decoding.
  • The first non-“detect-then-describe” framework for 3D Dense Captioning.
  • 🥇 Winner of the Scan2Cap Challenge in the 3rd Language for 3D Scene Workshop at ICCV 2023. [talk]

🥇 Awards and Scholarships

  • Oct. 2023. 1st place of the Scan2Cap Challenge in the 3rd Language for 3D Scene Workshop at ICCV 2023.
  • Sep. 2023. National Scholarship (rank 1/46).
  • Sep. 2022. 2nd prize of the Scholarship for Outstanding Students of Master’s Degrees at Fudan University.
  • Sep. 2021. Award for the Scholarship for Outstanding Students of Master’s Degrees at Fudan University.
  • Jun. 2021. 2nd prize of the Scholarship for Outstanding Students at Fudan University.

📖 Educations

  • Sep. 2021 - Jun. 2024 (expected). Master student at Fudan University.
  • Sep. 2017 - Jun. 2021. Bachelor student at Fudan University.

💬 Oral Presentations

  • Oct. 2023. Winner presentation of the Scan2Cap Challenge in the 3rd Language for 3D Scene Workshop at ICCV 2023. [talk | slides]
  • Jun. 2023. Paper presentation for “End-to-End 3D Dense Captioning with Vote2Cap-DETR” at the Workshop for Advances in 3D Vision, VALSE 2023, Wuxi, China.