- Published on
DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training
COLM2024
Dacheng Li*, Rulin Shao*, Anze Xie, Eric P Xing, Joseph E Gonzalez, Ion Stoica, Xuezhe Ma, Hao Zhang
Publications
- Published on
Learning to Maximize Mutual Information for Chain-of-Thought Distillation
ACL2024Xin Chen, Hanxian Huang, Yanjun Gao, Yi Wang, Jishen Zhao, Ke Ding- Published on
- Published on
WikiDT: Visual-based Table Recognition and Question Answering Dataset
ICDAR2024Hui Shi, Yusheng Xie, Luis Goncalves, Sicun Gao, Jishen Zhao- Published on
Optimizing Speculative Decoding for Serving Large Language Models Using Goodput
PREPRINTXiaoxuan Liu, Cade Daniel, Lanxiang Hu, Woosuk Kwon, Zhuohan Li, Xiangxi Mo, Alvin Cheung, Zhijie Deng, Ion Stoica, Hao Zhang- Published on
InferCept: Efficient Intercept Support for Augmented Large-Language Model Inferencing
ICML2024Reyna Abhyankar, Zijian He, Vikranth Srivatsa, Hao Zhang, Yiying Zhang- Published on
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
ICML2024Wei-Lin Chiang*, Lianmin Zheng*, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E. Gonzalez, Ion Stoica- Published on
CLLMs: Consistency Large Language Models
ICML2024Siqi Kou*, Lanxiang Hu*, Zhezhi He, Zhijie Deng, Hao Zhang- Published on
Break the Sequential Dependency of LLM Inference using Lookahead Decoding
ICML2024Yichao Fu, Peter Bailis, Ion Stoica, Hao Zhang- Published on
Online Speculative Decoding
ICML2024Xiaoxuan Liu, Lanxiang Hu, Peter Bailis, Ion Stoica, Zhijie Deng, Alvin Cheung, Hao Zhang- Published on
AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models
PREPRINTZihao Zeng, Yibo Miao, Hongcheng Gao, Hao Zhang, Zhijie Deng- Published on
Sibyl: Forecasting Time-Evolving Query Workloads
SIGMOD2024Hanxian Huang, Tarique Siddiqui, Rana Alotaibi, Carlo Curino, Jyoti Leeka, Alekh Jindal, Jishen Zhao, Jesús Camacho-Rodríguez, Yuanyuan Tian- Published on
Fasor: A Fast Tensor Program Optimization Framework for Efficient DNN Deployment
ICS2024Hanxian Huang, Xin Chen, Jishen Zhao- Published on
Towards LLM-Powered Verilog RTL Assistant: Self-Verification and Self-Correction
PREPRINTHanxian Huang, Zhenghan Lin, Zixuan Wang, Xin Chen, Ke Ding, Jishen Zhao- Published on
Preble: Efficient Distributed Prompt Scheduling for LLM Serving
PREPRINTVikranth Srivatsa, Zijian He, Reyna Abhyankar, Dongming Li, Yiying Zhang- Published on
Safety-Critical Scenario Generation Via Reinforcement Learning Based Editing
ICRA2024Haolan Liu, Liangjun Zhang, Siva Hari, Jishen Zhao- Published on
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
ICLR2024Lianmin Zheng*, Wei-Lin Chiang*, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric Xing, Joseph E Gonzalez, Ion Stoica, Hao Zhang- Published on
In-Storage Domain-Specific Acceleration for Serverless Computing
ASPLOS2024Rohan Mahapatra, Soroush Ghodrati, Byung Hoon Ahn, Sean Kinzer, Shu-ting Wang, Hanyang Xu, Lavanya Karthikeyan, Hardik Sharma, Amir Yazdanbakhsh, Mohammad Alian, Hadi Esmaeilzadeh- Published on
Restoring the Broken Covenant Between Compilers and Deep Learning Accelerators
preprintSean Kinzer, Soroush Ghodrati, Rohan Mahapatra, Byung Hoon Ahn, Edwin Mascarenhas, Xiaolong Li, Janarbek Matai, Liang Zhang, Hadi Esmaeilzadeh- Published on
Tandem processor: Grappling with Emerging Operators in Neural Networks
ASPLOS2024Soroush Ghodrati, Sean Kinzer, Hanyang Xu, Rohan Mahapatra, Yoonsung, Byung Hoon Ahn, Dong Kai Wang, Lavanya Karthikeyan, Amir Yazdanbakhsh, Jongse Park, Nam Sung Kim, Hadi Esmaeilzadeh- Published on
Toward Inference-optimal Mixture-of-Expert Large Language Models
PREPRINTLongfei Yun*, Yonghao Zhuang*, Yao Fu, Eric P Xing, Hao Zhang- Published on
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
OSDI2024Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, Hao Zhang- Published on
Data Motion Acceleration for Heterogeneous Cross-Domain Accelerator Chaining
HPCA2024Shu-Ting Wang, Hanyang Xu, Amin Mamandipoor, Rohan Mahapatra, Byung Hoon Ahn, Soroush Ghodrati, Krishnan Kailas, Mohammad Alian, Hadi Esmaeilzadeh- Published on
Judging LLM-as-a-judge with MT-Bench and Chatbot Arena
NeurIPS2023Lianmin Zheng*, Wei-Lin Chiang*, Ying Sheng*, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E Gonzalez, Ion Stoica- Published on
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
OSDI2023Zhuohan Li*, Lianmin Zheng*, Yinmin Zhong*, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E Gonzalez, Ion Stoica- Published on
How Long Can Context Length of Open-Source LLMs truly Promise?
Instruction-Tuning-and-Instruction-Following-Workshop-@-NeurIPS2023Dacheng Li*, Rulin Shao*, Anze Xie, Ying Sheng, Lianmin Zheng, Joseph Gonzalez, Ion Stoica, Xuezhe Ma, Hao Zhang- Published on
Efficient Memory Management for Large Language Model Serving with PagedAttention
SOSP2023Woosuk Kwon*, Zhuohan Li*, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Yu, Joey Gonzalez, Hao Zhang, Ion Stoica- Published on
TripLe: Revisiting Pretrained Model Reuse and Progressive Learning for Efficient Vision Transformer Scaling and Searching
ICCV2023Cheng Fu, Hanxian Huang, Zixuan Jiang, Yun Ni, Lifeng Nai, Gang Wu, Liqun Cheng, Yanqi Zhou, Sheng Li, Andrew Li, Jishen Zhao- Published on
Everyone’s Preference Changes Differently: A Weighted Multi-Interest Model for Retrieval
ICML2023Hui Shi, Yupeng Gu, Yitong Zhou, Bo Zhao, Sicun Gao, Jishen Zhao- Published on
On Optimizing the Communication of Model Parallelism
MLSYS2023Yonghao Zhuang*, Hexu Zhao*, Lianmin Zheng, Zhuohan Li, Eric P. Xing, Qirong Ho, Joseph E. Gonzalez, Ion Stoica, Hao Zhang- Published on
MPCFormer: Fast, Performant and Private Transformer Inference with MPC
ICLR2023Dacheng Li*, Rulin Shao*, Hongyi Wang*, Han Guo, Eric P. Xing, Hao Zhang- Published on
AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness
NeurIPS2022Dacheng Li, Hongyi Wang, Eric Xing, Hao Zhang- Published on
Q-gym: An Equality Saturation Framework for DNN Inference Exploiting Weight Repetition
PACT2022Cheng Fu, Hanxian Huang, Bram Wasti, Chris Cummins, Riyadh Baghdadi, Kim Hazelwood, Yuandong Tian, Jishen Zhao, Hugh Leather- Published on
Alpa: Automating Inter-and Intra-Operator Parallelism for Distributed Deep Learning
OSDI2022Lianmin Zheng*, Zhuohan Li*, Hao Zhang*, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica- Published on
Learning Bounded Context-Free-Grammar via LSTM and the Transformer: the Difference and the Explanations
AAAI2022Hui Shi, Sicun Gao, Yuandong Tian, Xinyun Chen, Jishen Zhao- Published on
Terapipe: Token-level Pipeline Parallelism for Training Large-scale Language Models
ICML2021Zhuohan Li, Siyuan Zhuang, Shiyuan Guo, Danyang Zhuo, Hao Zhang, Dawn Song, Ion Stoica- Published on
Ada-segment: Automated Multi-loss Adaptation for Panoptic Segmentation
AAAI2021Gengwei Zhang, Yiming Gao, Hang Xu, Hao Zhang, Zhenguo Li, Xiaodan Liang