Thanh Le

• Le, Thanh; La, Hoang Loc; Taherkordi, Amirhosein; Eliassen, Frank; Ha, Hoai Phuong & Guan, Peiyuan (2026). PM2Lat: Highly Accurate and Generalized Prediction of DNN Execution Latency on GPUs. IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing. ISSN 2376-4414. s. 604–614. doi: 10.1109/ccgrid68966.2026.00070. Fulltekst i vitenarkiv Vis sammendrag

  We present PM2Lat, a fast and generalized framework for accurately predicting the latency of deep neural network models on GPUs, with special focus on NVIDIA. Unlike prior methods that rely on deep learning models or handcrafted heuristics, PM2Lat leverages the Single-Instruction-Multiple-Thread architecture of GPUs to model execution time of DNN models. First, we conduct fine-grained GPU operation modeling by analyzing computational behavior and memory access patterns. Through this analysis, we observe that different GPU kernels can exhibit significant performance disparities, even when serving the same purpose. Hence, the core idea of PM2Lat is to differentiate kernels based on their configurations and analyze them accordingly. This kernel-aware modeling enables PM2Lat to achieve consistently low prediction error across diverse data types and hardware platforms. In addition, PM2Lat generalizes beyond standard matrix multiplication to support complex GPU kernels such as Triton, Flash Attention, and Cutlass Attention. Experimental results show that PM2Lat consistently achieves error rates below 10% across different data types and hardware platforms on Transformer models, outperforming the state-ofthe-art NeuSight by 10-20% for FP32 and by at least 50% for BF16. When applying to diverse kernels, the error rate is maintained at 3-8%.

• Sharma, Jivitesh; Vallejo, Islen; Ødegård, Rune Åvar; Le, Truong Thanh; Taherkordi, Amirhosein & Eliassen, Frank (2025). Physics-Informed Deep Learning for Wind Downscaling over Oslo. Proceedings - International Conference on Tools with Artificial Intelligence (ICTAI). ISSN 1082-3409. 37, s. 270–276. doi: 10.1109/ICTAI66417.2025.00042. Fulltekst i vitenarkiv Vis sammendrag

  Running a numerical weather model such as WRF at kilometre or sub-kilometre grid spacing over a regional domain is computationally expensive. We present physics-informed deeplearning models that ingest a single 9km WRF wind field and simultaneously predict two finer-scale wind fields at 3 km and 1 km resolution via dual decoder heads. Four representative architectures are benchmarked-Deep Residual U-Net (DeepRU), DEVINE, a bespoke 3-D Transformer, and a Fourier Neural Operator (FNO)-each trained with divergence-free, vorticity, and Navier-Stokes residual constraints plus Charbonnier and gradient perceptual losses. We train and validate our models on the city of Oslo for the year 2018. DeepRU achieves R2=0.94 (RMSE =0.050) at 3km and R2=0.89(RMSE=0.065) at 1 km. DEVINE, Transformer 3-D, and FNO yield 3 km scores of 0.91−0.93, with 1km scores lower by 0.02−0.08, illustrating the increased difficulty of finer-scale reconstruction. Physicsinformed losses improve all models compared to MSE-only baselines, and the residual architecture (DeepRU) remains most effective for this dual-scale task.

• Le, Truong Thanh; Taherkordi, Amirhosein; Eliassen, Frank & Guan, Peiyuan (2024). Optimal Distribution of ML Models Over Edge for Applications with High Input Frequency. IEEE International Conference on Cloud Computing Technology and Science (CloudCom). ISSN 2330-2194. s. 143–150. doi: 10.1109/cloudcom62794.2024.00033. Fulltekst i vitenarkiv Vis sammendrag

  The rise of complex and sizeable Machine Learning (ML) models challenges traditional cloud computing models with respect to the high volume of incoming data which results in increased bandwidth usage and network congestion, as well as delays in inference. Such ML models are being rapidly developed thanks to advances in computing platforms and the real-time computing demands of ML-driven applications such as autonomous vehicles and video processing. To mitigate these challenges, ML model distribution and inference offloading to computing devices close to data sources have been explored, especially through partitioning the models across the IoT-Edge-Cloud continuum. Existing efforts in this area have not successfully mastered the fully automatic and efficient determination of optimal partition points. Additionally, they have not effectively integrated Early-Exit layers that allow for early termination of model inference at earlier stages when feasible. In this paper, we introduce a novel partitioning algorithm designed to distribute ML models across edge devices with the goal of reducing response time when facing high-rate input data streams. Our proposed approach leverages the principles of Dynamic Programming to determine optimal partition points and establish appropriate exit thresholds for Early-Exit layers. Our evaluation results reveal that, in the context of continuous, high-rate input data, our method consistently lowers the maximum round-trip time for processing inference requests compared to state-of-the-art methods such as NeuroSurgeon and Genetics.

Se alle arbeider i NVA

• Le, Thanh; La, Hoang Loc; Taherkordi, Amirhosein; Eliassen, Frank; Ha, Hoai Phuong & Guan, Peiyuan (2026). Empirical Analysis of GPU Frequency Behavior Under ML Workloads. doi: 10.1109/ccgridw69005.2026.00018. Fulltekst i vitenarkiv Vis sammendrag

  This work presents ongoing research on the frequency-scaling behavior of NVIDIA GPUs when executing ML/AI workloads. Our preliminary findings show that, on lower-performance GPUs, the operating frequency is strongly affected by the recent workload history-typically within an 80ms window. This behavior challenges a common assumption under-lying several state-of-the-art ML latency-prediction techniques, which treat individual GPU kernel latencies as independent and therefore estimate total execution time by summing isolated per-kernel measurements. Our results indicate that such an assumption does not always hold, as the GPU's dynamic frequency scaling introduces inter-kernel dependencies. We also outline several promising directions for leveraging this observation in future work, including improved latency-prediction models, GPU kernel-reordering strategies, and NAS-driven guidelines for frequency/latency/energy-aware model design.

Se alle arbeider i NVA