Publications

Published on
December 16, 2024
Specifications: The missing link to making the development of LLM systems an engineering discipline
PREPRINT
Ion Stoica, Matei Zaharia, Joseph Gonzalez, Ken Goldberg, Koushik Sen, Hao Zhang, Anastasios N. Angelopoulos, Shishir G. Patil, Lingjiao Chen, Wei-Lin Chiang, Jared Q. Davis
Published on
December 15, 2024
GameArena: Evaluating LLM Reasoning through Live Computer Games
PREPRINT
Lanxiang Hu*, Qiyu Li*, Anze Xie*, Nan Jiang, Ion Stoica, Haojian Jin, Hao Zhang
Published on
December 10, 2024
Efficient LLM Scheduling by Learning to Rank
NEURIPS2024
Yichao Fu, Siqi Zhu, Runlong Su, Aurick Qiao, Ion Stoica, Hao Zhang
Published on
November 15, 2024
DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training
COLM2024
Dacheng Li*, Rulin Shao*, Anze Xie, Eric P Xing, Joseph E Gonzalez, Ion Stoica, Xuezhe Ma, Hao Zhang
Published on
October 4, 2024
Learning to Maximize Mutual Information for Chain-of-Thought Distillation
ACL2024
Xin Chen, Hanxian Huang, Yanjun Gao, Yi Wang, Jishen Zhao, Ke Ding
Published on
September 16, 2024
Multi-modal Learning for WebAssembly Reverse Engineering
ISSTA2024
Hanxian Huang, Jishen Zhao
Published on
August 30, 2024
WikiDT: Visual-based Table Recognition and Question Answering Dataset
ICDAR2024
Hui Shi, Yusheng Xie, Luis Goncalves, Sicun Gao, Jishen Zhao
Published on
July 25, 2024
Optimizing Speculative Decoding for Serving Large Language Models Using Goodput
PREPRINT
Xiaoxuan Liu, Cade Daniel, Lanxiang Hu, Woosuk Kwon, Zhuohan Li, Xiangxi Mo, Alvin Cheung, Zhijie Deng, Ion Stoica, Hao Zhang
Published on
July 21, 2024
InferCept: Efficient Intercept Support for Augmented Large-Language Model Inferencing
ICML2024
Reyna Abhyankar, Zijian He, Vikranth Srivatsa, Hao Zhang, Yiying Zhang
Published on
July 15, 2024
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
ICML2024
Wei-Lin Chiang*, Lianmin Zheng*, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E. Gonzalez, Ion Stoica
Published on
July 15, 2024
CLLMs: Consistency Large Language Models
ICML2024
Siqi Kou*, Lanxiang Hu*, Zhezhi He, Zhijie Deng, Hao Zhang
Published on
July 15, 2024
Break the Sequential Dependency of LLM Inference using Lookahead Decoding
ICML2024
Yichao Fu, Peter Bailis, Ion Stoica, Hao Zhang
Published on
July 15, 2024
Online Speculative Decoding
ICML2024
Xiaoxuan Liu, Lanxiang Hu, Peter Bailis, Ion Stoica, Zhijie Deng, Alvin Cheung, Hao Zhang
Published on
June 19, 2024
AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models
PREPRINT
Zihao Zeng, Yibo Miao, Hongcheng Gao, Hao Zhang, Zhijie Deng
Published on
June 9, 2024
Sibyl: Forecasting Time-Evolving Query Workloads
SIGMOD2024
Hanxian Huang, Tarique Siddiqui, Rana Alotaibi, Carlo Curino, Jyoti Leeka, Alekh Jindal, Jishen Zhao, Jesús Camacho-Rodríguez, Yuanyuan Tian
Published on
June 4, 2024
Fasor: A Fast Tensor Program Optimization Framework for Efficient DNN Deployment
ICS2024
Hanxian Huang, Xin Chen, Jishen Zhao
Published on
May 31, 2024
Towards LLM-Powered Verilog RTL Assistant: Self-Verification and Self-Correction
PREPRINT
Hanxian Huang, Zhenghan Lin, Zixuan Wang, Xin Chen, Ke Ding, Jishen Zhao
Published on
May 17, 2024
Preble: Efficient Distributed Prompt Scheduling for LLM Serving
PREPRINT
Vikranth Srivatsa, Zijian He, Reyna Abhyankar, Dongming Li, Yiying Zhang
Published on
May 13, 2024
Safety-Critical Scenario Generation Via Reinforcement Learning Based Editing
ICRA2024
Haolan Liu, Liangjun Zhang, Siva Hari, Jishen Zhao
Published on
May 10, 2024
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
ICLR2024
Lianmin Zheng*, Wei-Lin Chiang*, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric Xing, Joseph E Gonzalez, Ion Stoica, Hao Zhang
Published on
April 15, 2024
In-Storage Domain-Specific Acceleration for Serverless Computing
ASPLOS2024
Rohan Mahapatra, Soroush Ghodrati, Byung Hoon Ahn, Sean Kinzer, Shu-ting Wang, Hanyang Xu, Lavanya Karthikeyan, Hardik Sharma, Amir Yazdanbakhsh, Mohammad Alian, Hadi Esmaeilzadeh
Published on
April 15, 2024
Restoring the Broken Covenant Between Compilers and Deep Learning Accelerators
preprint
Sean Kinzer, Soroush Ghodrati, Rohan Mahapatra, Byung Hoon Ahn, Edwin Mascarenhas, Xiaolong Li, Janarbek Matai, Liang Zhang, Hadi Esmaeilzadeh
Published on
April 15, 2024
Tandem processor: Grappling with Emerging Operators in Neural Networks
ASPLOS2024
Soroush Ghodrati, Sean Kinzer, Hanyang Xu, Rohan Mahapatra, Yoonsung, Byung Hoon Ahn, Dong Kai Wang, Lavanya Karthikeyan, Amir Yazdanbakhsh, Jongse Park, Nam Sung Kim, Hadi Esmaeilzadeh
Published on
April 3, 2024
Toward Inference-optimal Mixture-of-Expert Large Language Models
PREPRINT
Longfei Yun*, Yonghao Zhuang*, Yao Fu, Eric P Xing, Hao Zhang
Published on
March 1, 2024
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
OSDI2024
Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, Hao Zhang
Published on
February 15, 2024
Data Motion Acceleration for Heterogeneous Cross-Domain Accelerator Chaining
HPCA2024
Shu-Ting Wang, Hanyang Xu, Amin Mamandipoor, Rohan Mahapatra, Byung Hoon Ahn, Soroush Ghodrati, Krishnan Kailas, Mohammad Alian, Hadi Esmaeilzadeh
Published on
December 20, 2023
Judging LLM-as-a-judge with MT-Bench and Chatbot Arena
NeurIPS2023
Lianmin Zheng*, Wei-Lin Chiang*, Ying Sheng*, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E Gonzalez, Ion Stoica
Published on
December 1, 2023
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
OSDI2023
Zhuohan Li*, Lianmin Zheng*, Yinmin Zhong*, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E Gonzalez, Ion Stoica
Published on
November 1, 2023
How Long Can Context Length of Open-Source LLMs truly Promise?
Instruction-Tuning-and-Instruction-Following-Workshop-@-NeurIPS2023
Dacheng Li*, Rulin Shao*, Anze Xie, Ying Sheng, Lianmin Zheng, Joseph Gonzalez, Ion Stoica, Xuezhe Ma, Hao Zhang
Published on
October 23, 2023
Efficient Memory Management for Large Language Model Serving with PagedAttention
SOSP2023
Woosuk Kwon*, Zhuohan Li*, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Yu, Joey Gonzalez, Hao Zhang, Ion Stoica
Published on
October 2, 2023
TripLe: Revisiting Pretrained Model Reuse and Progressive Learning for Efficient Vision Transformer Scaling and Searching
ICCV2023
Cheng Fu, Hanxian Huang, Zixuan Jiang, Yun Ni, Lifeng Nai, Gang Wu, Liqun Cheng, Yanqi Zhou, Sheng Li, Andrew Li, Jishen Zhao
Published on
July 23, 2023
Everyone’s Preference Changes Differently: A Weighted Multi-Interest Model for Retrieval
ICML2023
Hui Shi, Yupeng Gu, Yitong Zhou, Bo Zhao, Sicun Gao, Jishen Zhao
Published on
May 15, 2023
On Optimizing the Communication of Model Parallelism
MLSYS2023
Yonghao Zhuang*, Hexu Zhao*, Lianmin Zheng, Zhuohan Li, Eric P. Xing, Qirong Ho, Joseph E. Gonzalez, Ion Stoica, Hao Zhang
Published on
May 1, 2023
MPCFormer: Fast, Performant and Private Transformer Inference with MPC
ICLR2023
Dacheng Li*, Rulin Shao*, Hongyi Wang*, Han Guo, Eric P. Xing, Hao Zhang
Published on
December 1, 2022
AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness
NeurIPS2022
Dacheng Li, Hongyi Wang, Eric Xing, Hao Zhang
Published on
October 10, 2022
Q-gym: An Equality Saturation Framework for DNN Inference Exploiting Weight Repetition
PACT2022
Cheng Fu, Hanxian Huang, Bram Wasti, Chris Cummins, Riyadh Baghdadi, Kim Hazelwood, Yuandong Tian, Jishen Zhao, Hugh Leather
Published on
July 1, 2022
Alpa: Automating Inter-and Intra-Operator Parallelism for Distributed Deep Learning
OSDI2022
Lianmin Zheng*, Zhuohan Li*, Hao Zhang*, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica
Published on
February 24, 2022
Learning Bounded Context-Free-Grammar via LSTM and the Transformer: the Difference and the Explanations
AAAI2022
Hui Shi, Sicun Gao, Yuandong Tian, Xinyun Chen, Jishen Zhao
Published on
July 1, 2021
Terapipe: Token-level Pipeline Parallelism for Training Large-scale Language Models
ICML2021
Zhuohan Li, Siyuan Zhuang, Shiyuan Guo, Danyang Zhuo, Hao Zhang, Dawn Song, Ion Stoica
Published on
February 1, 2021
Ada-segment: Automated Multi-loss Adaptation for Panoptic Segmentation
AAAI2021
Gengwei Zhang, Yiming Gao, Hang Xu, Hao Zhang, Zhenguo Li, Xiaodan Liang