Starting from 2024, I have decided to discontinue the use of the list of publications format, as I believe that it fails to provide an intuitive view of our research. Instead, I am creating this active, self-curated selection of papers. I have intentionally excluded the publication venue information from these papers, which means they may be arXiv preprints or peer-reviewed conference / journal papers. For that information, kindly refer to my resume. The selection only represents my personal subjective preferences.
In this group of papers, we explore text diffusion models.
[2024] Jiacheng Ye*, Shansan Gong*, Liheng Chen*, Lin Zheng, Jiahui Gao, Han Shi, Chuan Wu, Zhenguo Li, Wei Bi, and Lingpeng Kong, Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models.
[2023] Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong, A Reparameterized Discrete Diffusion Model for Text Generation.
[2023] Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong, Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models.
[2023] Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong, DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models.
In this group of papers, we explore new transformer architectures with the goal of enhancing efficiency and performance in long sequence modeling.
[2023] Lin Zheng, Jianbo Yuan, Chong Wang, and Lingpeng Kong, Efficient Attention via Control Variates.
[2022] Lin Zheng, Chong Wang, and Lingpeng Kong, Linear Complexity Randomized Self-attention Mechanism.
[2022] Lin Zheng, Huijie Pan, and Lingpeng Kong, Ripple Attention for Visual Perception with Sub-quadratic Complexity.
[2021] Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, and Lingpeng Kong, Random Feature Attention.
The evaluation of these architectures.
[2023] Jun Zhang*, Shuyang Jiang*, Jiangtao Feng, Lin Zheng, and Lingpeng Kong, CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling.
In this group of papers, we explore key issues related to large language models (LLMs).
In-context Learning.
[2023] Zhiyong Wu*, Yaoxiang Wang*, Jiacheng Ye*, and Lingpeng Kong, Self-adaptive In-context Learning.
[2023] Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong, Compositional Exemplars for In-context Learning
Principles of Data Synthesis.
[2023] Jiahui Gao, Renjie Pi, Lin Yong, Hang Xu, Jiacheng Ye, Zhiyong Wu, Weizhong Zhang, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong, Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning.
[2022] Jiacheng Ye, Jiahui Gao, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong, ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback.
[2023] Jiacheng Ye, Chengzu Li, Lingpeng Kong, Tao Yu, Generating Data for Symbolic Language with Large Language Models.
[2022] Jiacheng Ye*, Jiahui Gao*, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong, ZeroGen: Efficient Zero-shot Learning via Dataset Generation.
Ultra-long Sequences.
[2023] Chenxin An, Fei Huang, Jun Zhang, Shansan Gong, Xipeng Qiu, Chang Zhou, and Lingpeng Kong, Training-Free Long-Context Scaling of Large Language Models.
[2023] Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, and Xipeng Qiu, L-Eval: Instituting Standardized Evaluation for Long Context Language Models.
In this group of papers, we explore computational science enabled by AI.
Mathematics.
[2024] Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi, GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers.
[2023] Xueliang Zhao, Wenda Li, and Lingpeng Kong, Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Proving.
[2023] Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, and Lingpeng Kong, G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model.
[2023] Xueliang Zhao, Xinting Huang, Wei Bi, and Lingpeng Kong, SEGO: Sequential Subgoal Optimization for Mathematical Problem-Solving.
Biology and Chemistry.
[2023] Chang Ma, Haiteng Zhao, Lin Zheng, Jiayi Xin, Qintong Li, Lijun Wu, Zhihong Deng, Yang Lu, Qi Liu, and Lingpeng Kong, Retrieved Sequence Augmentation for Protein Representation Learning.
[2023] Haiteng Zhao, Shengchao Liu, Chang Ma, Hannan Xu, Jie Fu, Zhi-Hong Deng, Lingpeng Kong, Qi Liu, GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning.