Ruisheng Cao 💻
Ruisheng Cao Rhythm Tsao

4th-year CS PhD candidate

X-LANCE Lab, SJTU

About Me

Ruisheng Cao is a 4th-year PhD candidate at X-LANCE Lab supervised by professor Kai Yu, in Shanghai Jiao Tong University (SJTU). His research interests mainly lie in structured natural language understanding (including semantic parsing, text-to-SQL, code intelligence and task-oriented dialogue systems), model-based data generation or augmentation with cycle learning and iterative training, and large language model (LLM) or vision language model (VLM) based multi-modal agents in fields like coding, data science and engineering, website navigation and computer control.

Download CV
Interests
  • Semantic Parsing
  • Text-to-SQL
  • Task-oriented Dialogue System
  • Structured NLU and NLG
  • LLM/VLM-based Multimodal Agents
Education

  • PhD in Computer Technology

    Shanghai Jiao Tong University

  • M.Eng. in Computer Technology

    Shanghai Jiao Tong University

  • B.Eng. in Computer Science

    Shanghai Jiao Tong University

Featured Publications
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
A Heterogeneous Graph to Abstract Syntax Tree Framework for Text-to-SQL
LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations
Dual Learning for Semi-Supervised Natural Language Understanding
Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing
Semantic Parsing with Dual Learning
Recent Publications

Here are publications sorted by year.

Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao, Xinzhuang Xiong, Hanchong Zhang, Yuchen Mao, Wenjing Hu, Hongshen Xu, Tianbao Xie, Danyang Zhang, Sida Wang, Ruoxi Sun, Pengcheng Yin, Caiming Xiong, Ansong Ni, Qian Liu, Victor Zhong, Lu Chen, Kai Yu, Tao Yu (2024). Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?. Advances in Neural Information Processing Systems (NeurIPS), 2024.
PDF Cite Code Dataset Project DOI URL
Hanchong Zhang, Ruisheng Cao, Hongshen Xu, Lu Chen, Kai Yu (2024). CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers).
PDF Cite Code DOI URL
Ruisheng Cao, Hongshen Xu, Su Zhu, Sheng Jiang, Hanchong Zhang, Lu Chen, Kai Yu (2024). A Birgat Model for Multi-Intent Spoken Language Understanding with Hierarchical Semantic Frames. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).
PDF Cite Code DOI URL
Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, Tao Yu (2024). OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. Advances in Neural Information Processing Systems (NeurIPS), 2024.
PDF Cite Code Dataset Project Slides Video DOI URL
Hongshen Xu, Lu Chen, Zihan Zhao, Da Ma, Ruisheng Cao, Zichen Zhu, Kai Yu (2024). Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding. Proceedings of the 17th ACM International Conference on Web Search and Data Mining, WSDM 2024, Merida, Mexico, March 4-8, 2024.
PDF Cite Code DOI URL
See all publications