Solar energy paper index
OmniPilot: An Uncertainty-Aware LLM Inference Advisor for Heterogeneous GPU Clusters
One-line summary
A solar energy research paper on OmniPilot: An Uncertainty-Aware LLM Inference Advisor for Heterogeneous GPU Clusters.
Engineering notes
Engineering notes will be added by the Power for Solar editorial team.
Chinese explanation / 中文解读
中文解读待补充：本站会优先为光伏效率、钙钛矿太阳能电池、储能技术、太阳能热利用、BIPV、并网技术等高价值论文补充中文说明。
Original abstract
Serving large language models (LLMs) on a shared, heterogeneous GPU cluster requires users and operators to select the GPU type, tensor-parallel degree, and precision before committing valuable node-hours. Making these choices is challenging because effective throughput, launch-success rates, and cluster demand and utilization continuously fluctuate. Furthermore, static configuration recipes miss critical interactions: quantization effects depend heavily on the model family, key-value cache pressure creates size-by-precision trade-offs, and failure rates vary by more than twofold across different tensor-parallel degrees. Additionally, cluster resources are frequently constrained by unpredictable hardware failures. To address these challenges, we present \textbf{OmniPilot}, a launch advisor that predicts serving costs for feasible configurations and abstains when requests fall outside its measured support envelope. OmniPilot pairs a conformally calibrated quantile cost model (spanning eight serving targets) with an out-of-distribution (OOD) abstention layer. It ranks configurations using an economic utility metric calibrated to an operator's revealed preferences. In evaluations across 460 benchmark runs on A100, H100, and H200 hardware across four precisions, OmniPilot predicts aggregate throughput with a 6.2\% mean absolute percentage error (MAPE) and a log-space $R^2=0.92$. The advisor achieves 95\% top-1 accuracy with a mean utility regret of just 0.003. When tested on an OOD holdout of unsupported cells, prediction error climbs to 24-46\% and conformal intervals cover 0 of 5 points; however, the abstention layer successfully flags all five as low-confidence. Over time, these OOD scenarios will be integrated into the training dataset to continuously expand the advisor's support envelope.
Links and sources
Need this topic turned into a technical roadmap?
Power for Solar can prepare a custom solar energy literature review, simulation code map, dataset map, and B2B photovoltaic technology assessment.
Request B2B research
Comments