Xing Cai

• Krishnasamy, Ezhilmathi; Trotter, James D.; Cai, Xing & Bouvry, Pascal (2026). Performance evaluation of the OpenACC API targeting multi-GPU data transfers: A case study of porting parallel 3D sweeping algorithms for geological folding. Parallel Computing. ISSN 0167-8191. 130. doi: 10.1016/j.parco.2026.103217. Fulltekst i vitenarkiv Vis sammendrag

  The increasing prevalence of GPUs in modern computing presents both performance opportunities and programming challenges. The complexity of heterogeneous CPU+GPU programming and the time-consuming process of adapting existing code to specific architectures often hinder performance optimization. We present a performance evaluation of the OpenACC API for various data transfer methodologies in the context of using multiple GPUs within a single compute node. Through a set of microbenchmarks, we first quantitatively study various scenarios of OpenACC-enabled data communication, encompassing high-level and low-level APIs, with synchronous, asynchronous and P2P variants for the latter. Then, we demonstrate how OpenACC can be used to offload the 3D Parallel Marching Method (3DPMM), a geological folding algorithm employed for mapping the Earth’s crust. When parallelizing the involved 3D stencil computations among multiple GPUs, we need to enable both the standard halo communication across the inter-GPU boundary surface and the non-standard volumetric data rearrangement between the GPUs. A proper use of the OpenACC API is therefore extremely important. To investigate the parallel capabilities of the 3DPMM algorithm, we employ a 3D computational domain and assess grid sizes in each direction ranging from 128 to 1280. Our findings reveal that for grid sizes up to 7683, a single GPU configuration outperforms setups using 2 and 4 GPUs, which exhibit performance slowdowns of up to 50% and 35%, respectively. In contrast, for problem sizes greater than 7683, the use of 4 GPUs shows the performance upper hand over 1 and 2 GPUs, achieving an average speedup of approximately 25%

• Huang, Cheng; Langguth, Johannes; Cai, Xing; Mottin, Davide & Assent, Ira (2026). Geld: Load-balanced D-Core Decomposition for Consumer GPUs. Proceedings of the ACM on Management of Data (PACMMOD). 4(3), s. 1–26. doi: 10.1145/3802064. Fulltekst i vitenarkiv Vis sammendrag

  D-core decomposition (DCD) identifies groups of nodes with strong cohesion at different levels of granularity, supporting e.g. clustering and community detection in directed graphs. Given the inefficiency of computing DCD, the computational power of GPUs is an attractive target. However, existing algorithms are ill-suited due to their restrictive programming model, as well as memory constraints in the widely available consumer-level GPUs. In particular, we show that any trivial adaptation of peeling algorithms, state-of-the-art for DCD, suffers from workload imbalance, redundant operations, and repeated atomic operations. To address these challenges, we propose a novel dynamic strategy to balance the computation on nodes with varying vertex degree and an enumeration-based lightweight memory footprint with greatly reduced thread contention. We conduct extensive experiments on 12 datasets showing that Geld achieves an average speedup of 31× over GPU peeling DCD, 6× over GPU H-Index DCD, 22× over the best existing parallel DCD algorithm with 33% memory efficiency, and 3 orders of magnitude over the best serial DCD algorithm.

• Sæternes, Erik Hide & Cai, Xing (2026). A control-theoretic approach to adaptive time stepping in reservoir simulations and its impacts on efficiency and robustness. Communications in nonlinear science & numerical simulation. ISSN 1007-5704. 158, s. 109828–109828. doi: 10.1016/j.cnsns.2026.109828. Fulltekst i vitenarkiv Vis sammendrag

  Adaptive time stepping is a central feature of most simulators solving time-dependent differential equations. Ideally, the time-stepping procedure should contribute to efficient and robust simulations. In this work, we first present a general overview of the adaptive-time-stepping design problem which is expected to help guide researchers and practitioners in constructing time-stepping procedures for their specific applications. Next, we provide a synthesis of the theory related to a control-theoretic approach to adaptive time stepping, and explain how this may form some of the parts of an adaptive-time-stepping procedure by helping to propose time-step sizes and decide whether executed time steps should be accepted or rejected. We combine our design framework with the control-theoretic approach, and formulate adaptive-time-stepping algorithms for reservoir simulations; a little-investigated application area for control theory. When doing so, we construct controllers which use the relative change in pressure and saturation as a proxy for the temporal discretisation error to propose new time steps, with the goal of keeping the relative change close to a target value. We then apply this complete time-stepping algorithm to real-field black-oil reservoir models using the industry-standard reservoir simulator OPM Flow. The simulator and all the tested reservoir models are open-source. By testing various versions of the novel time-stepping algorithm in the context of reservoir simulations, we arrive at a preferred third-order controller which seems to work well for several models. Our experiments also reveal that the choice of time-stepping algorithm has a notable impact on the efficiency of the simulator, and we observe impactful differences in robustness.

• Krishnasamy, Ezhilmathi; Trotter, James David; Cai, Xing; Pleiter, Dirk; Kos, Leon & Saavedra, Laura [Vis alle 7 forfattere av denne artikkelen] (2026). Performance and Programmability of MPI+X Integration with CUDA, HIP, SYCL, OpenACC, and OpenMP Offloading for Supercomputing: A Case Study on Dense Matrix–Vector Multiplication. I Mohr, Bernd; Fukaya, Takeshi & Miki, Yohei (Red.), Proceedings of Supercomputing Asia and International Conference on High Performance Computing in Asia Pacific Region Workshops (SCA/HPCAsia 2026 Workshops). Association for Computing Machinery (ACM). ISSN 9798400723285. s. 457–468. doi: 10.1145/3784828.3786264. Fulltekst i vitenarkiv Vis sammendrag

  Dense matrix-vector multiplication is a fundamental operation widely used in various application areas, including deep learning, computer graphics, and numerical analysis. With the increasing prevalence of large, heterogeneous high-performance computing (HPC) systems equipped with multiple Graphical Processing Units (GPUs), it is crucial to analyze effective strategies for utilizing these accelerators to enhance the efficiency of numerical computation. This study thus centers on the intricacies of accelerating dense matrix-vector multiplication on multi-GPU systems. We explore the combination of the Message Passing Interface (MPI) with various programming models—collectively referred to as MPI + X—highlighting its potential for advancing capabilities in large HPC environments, particularly those aimed at exascale computing and beyond. Here, "X" encompasses a range of programming models compatible with MPI. Specifically, this study investigates Compute Unified Device Architecture (CUDA), HIP, SYCL, Open Accelerators (OpenACC), and Open Multi-Processing (OpenMP) Offloading programming models within the context of large heterogeneous systems, as all can be integrated with MPI. We conduct a thorough examination of parallel dense matrix-vector multiplication and provide a detailed performance analysis of these programming models on NVIDIA, AMD, and Intel GPUs, addressing scenarios involving both GPU-aware and non-GPU-aware MPI. Our findings are substantiated by openly available code, which is essential for validation and the progression of scientific research.

• Ha, Hoai Phuong; Cai, Xing & Nguyen, Tan (2025). CAC: An asynchronous non-blocking consistency model with bounded staleness for distributed machine learning. Future Generation Computer Systems. ISSN 0167-739X. 178, s. 108317–108317. doi: 10.1016/j.future.2025.108317. Fulltekst i vitenarkiv Vis sammendrag

  Relaxed consistency models have been reported to significantly improve the performance of training machine learning (ML) models compared to strong consistency models. However, the existing relaxed consistency models for distributed ML either force fast workers to wait for stragglers (e.g., stale-synchronous models) or have no upper bound on data staleness (e.g., asynchronous models), thus negatively affecting the quality and performance of training ML models. We propose a new asynchronous non-blocking consistency model with bounded staleness for distributed ML that overcomes the drawbacks of blocking and unbounded staleness in previous relaxed consistency models. The new model, named Chained Asynchronous Consistency (CAC), guarantees an upper bound on data staleness in asynchronous computation without forcing fast workers to wait for slow workers. We theoretically prove that the Stochastic Gradient Descent (SGD) algorithm under CAC converges and the upper bound on the convergence expectation is independent of the number of workers. Based on the CAC model, we develop a new staleness-aware cacheable distributed object (CAC-object) for distributed ML where shared parameters are distributed among workers in a peer-to-peer manner. This approach avoids intermediate centralized storage, such as parameter servers, while maintaining simple one-sided communication (e.g., get and put). The CAC-object allows remote parameters to be cached and reused locally following a consistency model (e.g., CAC, stale-synchronous or asynchronous model). To demonstrate the applicability of the CAC-object in asynchronous distributed ML, we introduce a new asynchronous distributed matrix completion algorithm (CAC-MF) using the CAC-object. We develop the CAC-object and CAC-MF using UPC++, a Partitioned Global Address Space (PGAS) library for high-performance computing (HPC), and evaluate them in different execution scenarios (e.g., with and without stragglers and crash failures, different minibatch sizes) on HPC clusters. Our experimental results show that the CAC model scales with increasing workers, tolerates stragglers and crash failures, and achieves better convergence than stale-synchronous and asynchronous models. Particularly, in the case of halted stragglers, CAC’s Root Mean Square Error (RMSE) is up to 25 times less than the baseline models’ RMSE for the Netflix dataset, and 170 times less for the MovieLens dataset.

• Trotter, James David; Ekmekçibaşı, Sinan; Sağbili, Doğan; Langguth, Johannes; Cai, Xing & Unat, Didem (2025). CPU- and GPU-initiated Communication Strategies for Conjugate Gradient Methods on Large GPU Clusters. I Diachin, Lori; Schulz, Martin & Randles, Amanda (Red.), SC '25: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. Association for Computing Machinery (ACM). ISSN 9798400714665. s. 298–315. doi: 10.1145/3712285.3759774. Fulltekst i vitenarkiv Vis sammendrag

  The Conjugate Gradient (CG) method is a key building block in numerous applications, yet its low computational intensity and sensitivity to communication overhead make it difficult to scale efficiently on multi-GPU systems. In light of recent advances in multi-GPU communication technologies, we revisit CG parallelization for large-scale GPU clusters. This work presents scalable CG and pipelined CG solvers targeting NVIDIA and AMD GPUs, using GPU-aware MPI, NCCL/RCCL and NVSHMEM to implement both CPU- and GPU-initiated communication schemes. We also introduce a monolithic variant that offloads the entire CG loop to the GPU, enabling fully device-initiated execution via NVSHMEM. Optimizations across all variants reduce unnecessary data transfers and synchronization overheads; the GPU-initiated variant eliminates CPU involvement altogether. We benchmark our implementations on NVIDIA- and AMD-based supercomputers using SuiteSparse matrices and a real-world finite element application. By avoiding data transfers and synchronization bottlenecks, our single-GPU implementations achieve 8–14 % performance gains over state-of-the-art solvers. In strong scaling tests on over 1,000 GPUs, we outperform existing approaches by 5–15 %. While CPU-initiated variants remain favorable due to a lack of vendor supported device-side computational kernels and suboptimal NVSHMEM configurations at the clusters, the strong scaling properties of the GPU-initiated CG variant indicates that it will be highly competitive at even larger GPU counts and with further tuning.

• Langguth, Johannes; Trotter, James David & Cai, Xing (2025). Modelling Load Imbalance In Shared Memory Multicore Systems. I Pericàs, Miquel & Ellsworth, Danielle (Red.), Proceedings of the SC '25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis. Association for Computing Machinery (ACM). ISSN 9798400718717. s. 1433–1441. doi: 10.1145/3731599.3767707. Fulltekst i vitenarkiv Vis sammendrag

  Memory bandwidth has become the primary limiting factor of performance in many modern HPC applications, and it poses a limit to scalability because the achievable memory bandwidth only grows linearly with a small number of CPU cores. When the number of cores concurrently using the memory system exceeds a threshold, the aggregate memory bandwidth quickly saturates. To estimate the time usage of a computation dominated by memory traffic, the mainstream strategy is to divide the expected total memory traffic volume by the maximum memory bandwidth. However, this implicitly assumes homogeneous memory traffic which is often not the case, leading to inaccurate time estimates by the mainstream strategy. In this paper, we present a new performance model that specifically targets inhomogeneity in per-core memory traffic. The new requires only three hardware parameters. Using several cases of uneven per-core memory traffic, we demonstrate its advantage over the mainstream strategy.

• Zhao, Max; Burchard, Luk Bjarne; Schroeder, Daniel Thilo; Langguth, Johannes & Cai, Xing (2024). iPuma: High-Performance Sequence Alignment on the Graphcore IPU. I Taufer, Michela; Hoefler, Torsten; Hammond, Jeff & Randles, Amanda (Red.), ISC High Performance 2024 Research Paper Proceedings (39th International Conference). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9783982633602. doi: 10.23919/ISC.2024.10528941. Fulltekst i vitenarkiv Vis sammendrag

  String alignment algorithms are an essential tool for understanding DNA and protein sequences. They demand substantial computation in real-world applications, and are thus a prime target for hardware acceleration. However, GPUs struggle to provide sufficient acceleration. Meanwhile, the recent MIMD-capable AI accelerators such as the Graphcore Intelligence Processing Unit (IPU) have become technologically viable. In this paper we present iPuma, a new implementation of Smith-Waterman sequence alignment for the IPU, which offers generalized short and medium length, one-to-one, and many-to-many high-throughput alignments for both DNA and protein sequences. iPuma is integrated into two bioinformatics pipelines, MetaHipMer2 and PASTIS. On protein datasets, iPuma shows speedups of 2.7x and 1.6x over state-of-the-art GPU and CPU implementations, respectively. We test the scalability on up to 64 IPUs, attaining a peak scoring performance of 1763 GCUPS for protein and 1168 GCUPS for DNA sequences.

• Sæternes, Erik Hide; Thune, Andreas; Rustad, Alf Birger; Skeie, Tor & Cai, Xing (2024). Automated parameter tuning with accuracy control for efficient reservoir simulations. Journal of Computational Science. ISSN 1877-7503. 75. doi: 10.1016/j.jocs.2023.102205. Fulltekst i vitenarkiv Vis sammendrag

  Computer simulations of complex physical processes typically require sophisticated numerical schemes that internally involve many parameters. Different choices of such internal numerical parameters may lead to considerably different levels of computational efficiency, some may even result in wrong simulation results. The task of finding an optimal set of the numerical parameters (e.g. for the purpose of minimising the simulation time), while ensuring an accepted level of numerical accuracy, is therefore extremely important but challenging. In this paper, we propose a new automated search algorithm that is based on constrained stochastic searches within the parameter space. This iterative search scheme is also equipped with an accuracy check, which adopts several complementary measures for quantifying the similarities between time series from different simulations, such that parameter choices that lead to insufficiently accurate results will be automatically rejected. As a concrete scenario of usage, we have applied the automated parameter search scheme to the open-source reservoir simulation framework OPM. An empirical study shows that a suitable design of the optimisation objective function, together with an appropriate choice of the number of trials per search iteration and the perturbation scale per trial, can produce fast and convergent improvements with respect to the optimisation objective. For example, for a set of 12 numerical parameters, 30 trials from five search iterations are sufficient for reducing the objective function by 30% for the open Norne black-oil reservoir model. The robustness of the automated search scheme is also demonstrated for two other open reservoir models. Moreover, it is found that the parameter values automatically identified for the Norne model can also greatly improve the simulation efficiency of another proprietary reservoir model that has drastically different scale, resolution and geological properties.

• Jæger, Karoline Horgmo; Trotter, James David; Cai, Xing; Arevalo, Hermenegild & Tveito, Aslak (2024). Evaluating computational efforts and physiological resolution of mathematical models of cardiac tissue. Scientific Reports. 14. doi: 10.1038/s41598-024-67431-w. Fulltekst i vitenarkiv Vis sammendrag

  Computational techniques have significantly advanced our understanding of cardiac electrophysiology, yet they have predominantly concentrated on averaged models that do not represent the intricate dynamics near individual cardiomyocytes. Recently, accurate models representing individual cells have gained popularity, enabling analysis of the electrophysiology at the micrometer level. Here, we evaluate five mathematical models to determine their computational efficiency and physiological fidelity. Our findings reveal that cell-based models introduced in recent literature offer both efficiency and precision for simulating small tissue samples (comprising thousands of cardiomyocytes). Conversely, the traditional bidomain model and its simplified counterpart, the monodomain model, are more appropriate for larger tissue masses (encompassing millions to billions of cardiomyocytes). For simulations requiring detailed parameter variations along individual cell membranes, the EMI model emerges as the only viable choice. This model distinctively accounts for the extracellular (E), membrane (M), and intracellular (I) spaces, providing a comprehensive framework for detailed studies. Nonetheless, the EMI model’s applicability to large-scale tissues is limited by its substantial computational demands for subcellular resolution.

• Thune, Andreas; Reinemo, Sven- Arne; Skeie, Tor & Cai, Xing (2023). Detailed modeling of heterogeneous and contention-constrained point-to-point MPI communication. IEEE Transactions on Parallel and Distributed Systems. ISSN 1045-9219. 34(5), s. 1580–1593. doi: 10.1109/TPDS.2023.3253881. Fulltekst i vitenarkiv
• Trotter, James David; Langguth, Johannes & Cai, Xing (2023). Targeting performance and user-friendliness: GPU-accelerated finite element computation with automated code generation in FEniCS. Parallel Computing. ISSN 0167-8191. 118. doi: 10.1016/j.parco.2023.103051. Fulltekst i vitenarkiv Vis sammendrag

  This paper studies the use of automated code generation to provide user-friendly GPU acceleration for solving partial differential equations (PDEs) with finite element methods. By extending the FEniCS framework and its automated compiler, we have achieved that a high-level description of finite element computations written in the Unified Form Language is auto-translated to parallelised CUDA C++ code. The auto-generated code provides GPU offloading for the finite element assembly of linear equation systems which are then solved by a GPU-supported linear algebra backend. Specifically, we explore several auto-generated optimisations of the resulting CUDA C++ code. Numerical experiments show that GPU-based linear system assembly for a typical PDE with first-order elements can benefit from using a lookup table to avoid repeatedly carrying out numerous binary searches, and that further performance gains can be obtained by assembling a sparse matrix row by row. More importantly, the extended FEniCS compiler is able to seamlessly couple the assembly and solution phases for GPU acceleration, so that all unnecessary CPU–GPU data transfers are eliminated. Detailed experiments are used to quantify the negative impact of these data transfers, which can entirely destroy the potential of GPU acceleration if the assembly and solution phases are offloaded to GPU separately. Finally, a complete, auto-generated GPU-based PDE solver for a nonlinear solid mechanics application is used to demonstrate a substantial speedup over running on dual-socket multi-core CPUs, including GPU acceleration of algebraic multigrid as the preconditioner.

• Telle, Åshild; Trotter, James David; Cai, Xing; Finsberg, Henrik Nicolay Topnes; Kuchta, Miroslav & Sundnes, Joakim [Vis alle 7 forfattere av denne artikkelen] (2023). A cell-based framework for modeling cardiac mechanics. Biomechanics and Modeling in Mechanobiology. ISSN 1617-7959. 22, s. 515–539. doi: 10.1007/s10237-022-01660-8. Fulltekst i vitenarkiv Vis sammendrag

  Abstract Cardiomyocytes are the functional building blocks of the heart—yet most models developed to simulate cardiac mechanics do not represent the individual cells and their surrounding matrix. Instead, they work on a homogenized tissue level, assuming that cellular and subcellular structures and processes scale uniformly. Here we present a mathematical and numerical framework for exploring tissue-level cardiac mechanics on a microscale given an explicit three-dimensional geometrical representation of cells embedded in a matrix. We defined a mathematical model over such a geometry and parametrized our model using publicly available data from tissue stretching and shearing experiments. We then used the model to explore mechanical differences between the extracellular and the intracellular space. Through sensitivity analysis, we found the stiffness in the extracellular matrix to be most important for the intracellular stress values under contraction. Strain and stress values were observed to follow a normal-tangential pattern concentrated along the membrane, with substantial spatial variations both under contraction and stretching. We also examined how it scales to larger size simulations, considering multicellular domains. Our work extends existing continuum models, providing a new geometrical-based framework for exploring complex cell–cell and cell–matrix interactions.

• Burchard, Luk Bjarne; Hustad, Kristian Gregorius; Langguth, Johannes & Cai, Xing (2023). Enabling unstructured-mesh computation on massively tiled AI processors: An example of accelerating in silico cardiac simulation. Frontiers in Physics. 11. doi: 10.3389/fphy.2023.979699. Fulltekst i vitenarkiv Vis sammendrag

  A new trend in processor architecture design is the packaging of thousands of small processor cores into a single device, where there is no device-level shared memory but each core has its own local memory. Thus, both the work and data of an application code need to be carefully distributed among the small cores, also termed as tiles. In this paper, we investigate how numerical computations that involve unstructured meshes can be efficiently parallelized and executed on a massively tiled architecture. Graphcore IPUs are chosen as the target hardware platform, to which we port an existing monodomain solver that simulates cardiac electrophysiology over realistic 3D irregular heart geometries. There are two computational kernels in this simulator, where a 3D diffusion equation is discretized over an unstructured mesh and numerically approximated by repeatedly executing sparse matrix-vector multiplications (SpMVs), whereas an individual system of ordinary differential equations (ODEs) is explicitly integrated per mesh cell. We demonstrate how a new style of programming that uses Poplar/C++ can be used to port these commonly encountered computational tasks to Graphcore IPUs. In particular, we describe a per-tile data structure that is adapted to facilitate the inter-tile data exchange needed for parallelizing the SpMVs. We also study the achievable performance of the ODE solver that heavily depends on special mathematical functions, as well as their accuracy on Graphcore IPUs. Moreover, topics related to using multiple IPUs and performance analysis are addressed. In addition to demonstrating an impressive level of performance that can be achieved by IPUs for monodomain simulation, we also provide a discussion on the generic theme of parallelizing and executing unstructured-mesh multiphysics computations on massively tiled hardware.

• Trotter, James David; Cai, Xing & Funke, Simon Wolfgang (2022). On Memory Traffic and Optimisations for Low-order Finite Element Assembly Algorithms on Multi-core CPUs. ACM Transactions on Mathematical Software. ISSN 0098-3500. 48(2), s. 1–31. doi: 10.1145/3503925. Fulltekst i vitenarkiv Vis sammendrag

  Motivated by the wish to understand the achievable performance of finite element assembly on unstructured computational meshes, we dissect the standard cellwise assembly algorithm into four kernels, two of which are dominated by irregular memory traffic. Several optimisation schemes are studied together with associated lower and upper bounds on the estimated memory traffic volume. Apart from properly reordering the mesh entities, the two most significant optimisations include adopting a lookup table in adding element matrices or vectors to their global counterparts, and using a row-wise assembly algorithm for multi-threaded parallelisation. Rigorous benchmarking shows that, due to the various optimisations, the actual volumes of memory traffic are in many cases very close to the estimated lower bounds. These results confirm the effectiveness of the optimisations, while also providing a recipe for developing efficient software for finite element assembly.

• Hustad, Kristian Gregorius & Cai, Xing (2022). Resource-Efficient Use of Modern Processor Architectures For Numerically Solving Cardiac Ionic Cell Models. Frontiers in Physiology. 13. doi: 10.3389/fphys.2022.904648. Fulltekst i vitenarkiv Vis sammendrag

  A central component in simulating cardiac electrophysiology is the numerical solution of nonlinear ordinary differential equations, also called cardiac ionic cell models, that describe cross-cell-membrane ion transport. Biophysically detailed cell models often require a considerable amount of computation, including calls to special mathematical functions. This paper systematically studies how to efficiently use modern multicore CPUs for this costly computational task. We start by investigating the code restructurings needed to effectively enable compiler-supported SIMD vectorisation, which is the most important performance booster in this context. It is found that suitable OpenMP directives are sufficient for achieving both vectorisation and parallelisation. We then continue with an evaluation of the performance optimisation technique of using lookup tables. Due to increased challenges for automated vectorisation, the obtainable benefits of lookup tables are dependent on the hardware platforms chosen. Throughout the study, we report detailed time measurements obtained on Intel Xeon, Xeon Phi, AMD Epyc and two ARM processors including Fujitsu A64FX, while attention is also paid to the impact of SIMD vectorisation and lookup tables on the computational accuracy. As a realistic example, the benefits of performance enhancement are demonstrated by a 10 9 -run ensemble on the Oakforest-PACS system, where code restructurings and SIMD vectorisation yield an 84% reduction in computing time, corresponding to 63,270 node hours.

• Thune, Andreas; Cai, Xing & Rustad, Alf Birger (2021). On the impact of heterogeneity-aware mesh partitioning and non-contributing computation removal on parallel reservoir simulations. Journal of Mathematics in Industry. 11. doi: 10.1186/s13362-021-00108-5. Fulltekst i vitenarkiv Vis sammendrag

  Parallel computations have become standard practice for simulating the complicated multi-phase flow in a petroleum reservoir. Increasingly sophisticated numerical techniques have been developed in this context. During the chase of algorithmic superiority, however, there is a risk of forgetting the ultimate goal, namely, to efficiently simulate real-world reservoirs on realistic parallel hardware platforms. In this paper, we quantitatively analyse the negative performance impact caused by non-contributing computations that are associated with the “ghost computational cells” per subdomain, which is an insufficiently studied subject in parallel reservoir simulation. We also show how these non-contributing computations can be avoided by reordering the computational cells of each subdomain, such that the ghost cells are grouped together. Moreover, we propose a new graph-edge weighting scheme that can improve the mesh partitioning quality, aiming at a balance between handling the heterogeneity of geological properties and restricting the communication overhead. To put the study in a realistic setting, we enhance the open-source Flow simulator from the OPM framework, and provide comparisons with industrial-standard simulators for real-world reservoir models.

• Burchard, Luk; Cai, Xing & Langguth, Johannes (2021). iPUG for Multiple Graphcore IPUs: Optimizing Performance and Scalability of Parallel Breadth-First Search. I Sur, Chiranjib & Kothapalli, Kishore (Red.), 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665410168. s. 162–171. doi: 10.1109/HiPC53243.2021.00030. Fulltekst i vitenarkiv Vis sammendrag

  Parallel graph algorithms have become one of the principal applications of high-performance computing besides numerical simulations and machine learning workloads. However, due to their highly unstructured nature, graph algorithms remain extremely challenging for most parallel systems, with large gaps between observed performance and theoretical limits. Further-more, most mainstream architectures rely heavily on single instruction multiple data (SIMD) processing for high floating-point rates, which is not beneficial for graph processing which instead requires high memory bandwidth, low memory latency, and efficient processing of unstructured data. On the other hand, we are currently observing an explosion of new hardware architectures, many of which are adapted to specific purposes and diverge from traditional designs. A notable example is the Graphcore Intelligence Processing Unit (IPU), which is developed to meet the needs of upcoming machine intelligence applications. Its design eschews the traditional cache hierarchy, relying on SRAM as its main memory instead. The result is an extremely high-bandwidth, low-latency memory at the cost of capacity. In addition, the IPU consists of a large number of independent cores, allowing for true multiple instruction multiple data (MIMD) processing. Together, these features suggest that such a processor is well suited for graph processing. We test the limits of graph processing on multiple IPUs by implementing a low-level, high-performance code for breadth-first search (BFS), following the specifications of Graph500, the most widely used benchmark for parallel graph processing. Despite the simplicity of the BFS algorithm, implementing efficient parallel codes for it has proven to be a challenging task in the past. We show that our implementation scales well on a system with 8 IPUs and attains roughly twice the performance of an equal number of NVIDIA V100 GPUs using state-of-the-art CUDA code.

• Jæger, Karoline Horgmo; Hustad, Kristian Gregorius; Cai, Xing & Tveito, Aslak (2021). Efficient numerical solution of the EMI model representing the extracellular space (E), cell membrane (M) and intracellular space (I) of a collection of cardiac cells. Frontiers in Physics. 8(579461). doi: 10.3389/fphy.2020.579461. Fulltekst i vitenarkiv Vis sammendrag

  The EMI model represents excitable cells in a more accurate manner than traditional homogenized models at the price of increased computational complexity. The increased complexity of solving the EMI model stems from a significant increase in the number of computational nodes and from the form of the linear systems that need to be solved. Here, we will show that the latter problem can be solved by careful use of operator splitting of the spatially coupled equations. By using this method, the linear systems can be broken into sub-problems that are of the classical type of linear, elliptic boundary value problems. Therefore, the vast collection of methods for solving linear, elliptic partial differential equations can be used. We demonstrate that this enables us to solve the systems using shared-memory parallel computers. The computing time scales perfectly with the number of physical cells. For a collection of 512 × 256 cells, we solved linear systems with about 2.5×108 unknows. Since the computational effort scales linearly with the number of physical cells, we believe that larger computers can be used to simulate millions of excitable cells and thus allow careful analysis of physiological systems of great importance.

• Jæger, Karoline Horgmo; Hustad, Kristian Gregorius; Cai, Xing & Tveito, Aslak (2020). Operator Splitting and Finite Difference Schemes for Solving the EMI Model. I Tveito, Aslak; Mardal, Kent-Andre & Rognes, Marie (Red.), Modeling Excitable Tissue -- The EMI Framework. Springer. ISSN 9783030611569. s. 44–55. doi: 10.1007/978-3-030-61157-6_4. Fulltekst i vitenarkiv
• Trotter, James David; Langguth, Johannes & Cai, Xing (2020). Cache simulation for irregular memory traffic on multi-core CPUs: Case study on performance models for sparse matrix–vector multiplication. Journal of Parallel and Distributed Computing. ISSN 0743-7315. 144, s. 189–205. doi: 10.1016/j.jpdc.2020.05.020. Fulltekst i vitenarkiv
• Lagraviere, Jeremie Alexandre Emilien; Langguth, Johannes; Prugger, Martina; Einkemmer, Lukas; Ha, Hoai Phuong & Cai, Xing (2019). Performance optimization and modeling of fine-grained irregular communication in UPC. Scientific Programming. ISSN 1058-9244. 2019:6825728, s. 1–20. doi: 10.1155/2019/6825728. Fulltekst i vitenarkiv Vis sammendrag

  The Unified Parallel C (UPC) programming language offers parallelism via logically partitioned shared memory, which typically spans physically disjoint memory subsystems. One convenient feature of UPC is its ability to automatically execute between-thread data movement, such that the entire content of a shared data array appears to be freely accessible by all the threads. The programmer friendliness, however, can come at the cost of substantial performance penalties. This is especially true when indirectly indexing the elements of a shared array, for which the induced between-thread data communication can be irregular and have a fine-grained pattern. In this paper, we study performance enhancement strategies specifically targeting such fine-grained irregular communication in UPC. Starting from explicit thread privatization, continuing with block-wise communication, and arriving at message condensing and consolidation, we obtained considerable performance improvement of UPC programs that originally require fine-grained irregular communication. Besides the performance enhancement strategies, the main contribution of the present paper is to propose performance models for the different scenarios, in the form of quantifiable formulas that hinge on the actual volumes of various data movements plus a small number of easily obtainable hardware characteristic parameters. These performance models help to verify the enhancements obtained, while also providing insightful predictions of similar parallel implementations, not limited to UPC, that also involve between-thread or between-process irregular communication. As a further validation, we also apply our performance modeling methodology and hardware characteristic parameters to an existing UPC code for solving a 2D heat equation on a uniform mesh.

• Jarvis, Chad; Lines, Glenn Terje; Langguth, Johannes; Nakajima, Kengo & Cai, Xing (2019). Combining algorithmic rethinking and AVX-512 intrinsics for efficient simulation of subcellular calcium signaling. I Rodrigues, João M. F.; Cardoso, Pedro; Monteiro, Jânio; Lam, Roberto; Krzhizhanovskaya, Valeria V.; Lees, Michael H.; Dongarra, Jack & Sloot, Peter M.A. (Red.), Proceedings of 19th International Conference on Computational Science, ICCS2019, Part V. Springer. ISSN 9783030227494. s. 681–687. doi: 10.1007/978-3-030-22750-0_66. Fulltekst i vitenarkiv
• Langguth, Johannes; Cai, Xing & Sourouri, Mohammed (2018). Memory bandwidth contention: Communication vs computation Tradeoffs in supercomputers with multicore architectures. I Li, Mo & Tan, Kian-Lee (Red.), Proceedings 2018 IEEE 24th International Conference on Parallel and Distributed Systems ICPADS 2018. IEEE Sarnoff Symposium. ISSN 9781538673089. s. 497–506. doi: 10.1109/PADSW.2018.8644601. Fulltekst i vitenarkiv
• Langguth, Johannes; Jarvis, Chad & Cai, Xing (2017). Porting tissue-scale cardiac simulations to the knights landing platform. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10524 LNCS, s. 376–388. doi: 10.1007/978-3-319-67630-2_28. Fulltekst i vitenarkiv
• Paulsen, Geir; Clark, Stuart R; Nordmoen, Bjørn; Nenakhov, Sergey; Andersson, Aron & Cai, Xing [Vis alle 7 forfattere av denne artikkelen] (2017). Automated Translation of MATLAB Code to C++ with Performance and Traceability. I Rückemann, Claus-Peter & Vucinic, D (Red.), Proceedings of the Eleventh International Conference on Advanced Engineering Computing and Applications in Sciences (ADVCOMP 2017). International Academy, Research and Industry Association (IARIA). ISSN 9781612085999. s. 50–55. Fulltekst i vitenarkiv
• Lagraviere, Jeremie Alexandre Emilien; Langguth, Johannes; Sourouri, Mohammed; Ha, Hoai Phuong & Cai, Xing (2016). On the performance and energy efficiency of the PGAS programming model on multicore architectures. I Fox, Geoffrey C. & Nygård, Mads (Red.), Proceedings of the 14th IEEE International Conference on High Performance Computing & Simulation (HPCS 2016). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781509020881. s. 800–807. doi: 10.1109/HPCSim.2016.7568416. Fulltekst i vitenarkiv Vis sammendrag

  Using large-scale multicore systems to get the maximum performance and energy efficiency with manageable programmability is a major challenge. The partitioned global address space (PGAS) programming model enhances programmability by providing a global address space over large-scale computing systems. However, so far the performance and energy efficiency of the PGAS model on multicore-based parallel architectures have not been investigated thoroughly. In this paper we use a set of selected kernels from the well-known NAS Parallel Benchmarks to evaluate the performance and energy efficiency of the UPC programming language, which is a widely used implementation of the PGAS model. In addition, the MPI and OpenMP versions of the same parallel kernels are used for comparison with their UPC counterparts. The investigated hardware platforms are based on multicore CPUs, both within a single 16-core node and across multiple nodes involving up to 1024 physical cores. On the multi-node platform we used the hardware measurement solution called High definition Energy Efficiency Monitoring tool in order to measure energy. On the single-node system we used the hybrid measurement solution to make an effort into understanding the observed performance differences, we use the Intel Performance Counter Monitor to quantify in detail the communication time, cache hit/miss ratio and memory usage. Our experiments show that UPC is competitive with OpenMP and MPI on single and multiple nodes, with respect to both the performance and energy efficiency.

• Sourouri, Mohammed; Scott, Baden & Cai, Xing (2016). Panda: A Compiler Framework for Concurrent CPU+GPU Execution of 3D Stencil Computations on GPU-accelerated Supercomputers. International journal of parallel programming. ISSN 0885-7458. 45(3), s. 711–729. doi: 10.1007/s10766-016-0454-1. Fulltekst i vitenarkiv Vis sammendrag

  We present a new compiler framework for truly heterogeneous 3D stencil computation on GPU clusters. Our framework consists of a simple directive-based programming model and a tightly integrated source-to-source compiler. Annotated with a small number of directives, sequential stencil C codes can be automatically parallelized for large-scale GPU clusters. The most distinctive feature of the compiler is its capability to generate hybrid MPI + CUDA + OpenMP code that uses concurrent CPU + GPU computing to unleash the full potential of powerful GPU clusters. The auto-generated hybrid codes hide the overhead of various data motion by overlapping them with computation. Test results on the Titan supercomputer and the Wilkes cluster show that auto-translated codes can achieve about 90 % of the performance of highly optimized handwritten codes, for both a simple stencil benchmark and a real-world application in cardiac modeling. The user-friendliness and performance of our domain-specific compiler framework allow harnessing the full power of GPU-accelerated supercomputing without painstaking coding effort.

• Paulsen, Geir; Feinberg, Jonathan; Cai, Xing; Nordmoen, Bjørn & Dahle, Hans Petter (2016). Matlab2cpp: A Matlab-to-C++ code translator, 2016 11th System of Systems Engineering Conference (SoSE). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781467387279. doi: 10.1109/SYSOSE.2016.7542966. Fulltekst i vitenarkiv
• Zhang, Wei & Cai, Xing (2016). Solving 3D Time-Fractional Diffusion Equations by High-Performance Parallel Computing. Fractional Calculus and Applied Analysis. ISSN 1311-0454. 19(1), s. 140–160. doi: 10.1515/fca-2016-0008. Fulltekst i vitenarkiv
• Langguth, Johannes; Lan, Qiang; Gaur, Namit; Cai, Xing; Wen, Mei & Zhang, Chun-Yuan (2016). Enabling tissue-scale cardiac simulations using heterogeneous computing on tianhe-2. I Liao, Xiaofei; Lovas, Robert; Shen, Xipeng & Zheng, Ran (Red.), IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781509044573. s. 843–852. doi: 10.1109/ICPADS.2016.0114. Fulltekst i vitenarkiv
• Langguth, Johannes; Lan, Qiang; Gaur, Namit & Cai, Xing (2016). Accelerating Detailed Tissue-Scale 3D Cardiac Simulations Using Heterogeneous CPU-Xeon Phi Computing. International journal of parallel programming. ISSN 0885-7458. 45(5), s. 1236–1258. doi: 10.1007/s10766-016-0461-2. Fulltekst i vitenarkiv Vis sammendrag

  We investigate heterogeneous computing, which involves both multicore CPUs and manycore Xeon Phi coprocessors, as a new strategy for computational cardiology. In particular, 3D tissues of the human cardiac ventricle are studied with a physiologically realistic model that has 10,000 calcium release units per cell and 100 ryanodine receptors per release unit, together with tissue-scale simulations of the electrical activity and calcium handling. In order to attain resource-efficient use of heterogeneous computing systems that consist of both CPUs and Xeon Phis, we first direct the coding effort at ensuring good performance on the two types of compute devices individually. Although SIMD code vectorization is the main theme of performance programming, the actual implementation details differ considerably between CPU and Xeon Phi. Moreover, in addition to combined OpenMP+MPI programming, a suitable division of the cells between the CPUs and Xeon Phis is important for resource-efficient usage of an entire heterogeneous system. Numerical experiments show that good resource utilization is indeed achieved and that such a heterogeneous simulator paves the way for ultimately understanding the mechanisms of arrhythmia. The uncovered good programming practices can be used by computational scientists who want to adopt similar heterogeneous hardware platforms for a wide variety of applications.

• Langguth, Johannes; Wu, Nan; Chai, Jun & Cai, Xing (2015). Parallel performance modeling of irregular applications in cell-centered finite volume methods over unstructured tetrahedral meshes. Journal of Parallel and Distributed Computing. ISSN 0743-7315. 76, s. 120–131. doi: 10.1016/j.jpdc.2014.10.005. Fulltekst i vitenarkiv
• Lan, Qiang; Gaur, Namit; Langguth, Johannes & Cai, Xing (2015). Towards Detailed Tissue-Scale 3D Simulations of Electrical Activity and Calcium Handling in the Human Cardiac Ventricle. I Wang, Guojun; Zomaya, Albert; Perez, Gregorio Martinez & Li, Kenli (Red.), Algorithms and Architectures for Parallel Processing. Springer. ISSN 9783319271378. s. 79–92. doi: 10.1007/978-3-319-27137-8_7. Fulltekst i vitenarkiv
• Sourouri, Mohammed; Langguth, Johannes; Spiga, Filippo; Baden, Scott & Cai, Xing (2015). CPU+GPU Programming of Stencil Computations for Resource-Efficient Use of GPU Clusters. I Plessl, Christian (Red.), Proceedings of the 2015 IEEE 18th International Conference on Computational Science and Engineering. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781467382977. s. 17–26. doi: 10.1109/CSE.2015.33. Fulltekst i vitenarkiv
• Krishnasamy, Ezhilmathi; Sourouri, Mohammed & Cai, Xing (2015). Multi-GPU implementations of parallel 3D sweeping algorithms with application to geological folding. Procedia Computer Science. 51, s. 1494–1503. doi: 10.1016/j.procs.2015.05.339. Fulltekst i vitenarkiv
• Huang, Dafei; Xun, Changqing; Wu, Nan; Wen, Mei; Zhang, Chunyuan & Cai, Xing [Vis alle 7 forfattere av denne artikkelen] (2015). Enabling a uniform OpenCL device view for heterogeneous platforms. IEICE transactions on information and systems. ISSN 0916-8532. E98-D(4), s. 812–823. doi: 10.1587/transinf.2014EDP7244. Fulltekst i vitenarkiv
• Sourouri, Mohammed; Gillberg, Tor; Baden, Scott & Cai, Xing (2015). Effective multi-GPU communication using multiple CUDA streams and threads. Proceedings of the International Conference on Parallel and Distributed Systems. ISSN 1521-9097. 2015-April, s. 981–986. doi: 10.1109/PADSW.2014.7097919. Fulltekst i vitenarkiv
• Langguth, Johannes & Cai, Xing (2015). Heterogeneous CPU-GPU computing for the finite volume method on 3D unstructured meshes. Proceedings of the International Conference on Parallel and Distributed Systems. ISSN 1521-9097. 2015-April, s. 191–199. doi: 10.1109/PADSW.2014.7097808. Fulltekst i vitenarkiv
• Langguth, Johannes; Sourouri, Mohammed; Lines, Glenn Terje; Baden, Scott & Cai, Xing (2015). Scalable heterogeneous CPU-GPU computations for unstructured tetrahedral meshes. IEEE Micro. ISSN 0272-1732. 35(4), s. 6–15. doi: 10.1109/MM.2015.70. Fulltekst i vitenarkiv
• Dong, Xinnan; Wen, Mei; Chai, Jun; Cai, Xing; Zhao, Mandan & Zhang, Chunyuan (2015). Communication-hiding programming for clusters with multi-coprocessor nodes. Concurrency and Computation: Practice and Experience. ISSN 1532-0626. 27(16), s. 4172–4185. doi: 10.1002/cpe.3507. Fulltekst i vitenarkiv
• Chai, Jun; Hake, Johan Elon; Wu, Nan; Wen, Mei; Cai, Xing & Lines, Glenn Terje [Vis alle 10 forfattere av denne artikkelen] (2015). Towards simulation of subcellular calcium dynamics at nanometre resolution. The international journal of high performance computing applications. ISSN 1094-3420. 29(1), s. 51–63. doi: 10.1177/1094342013514465. Fulltekst i vitenarkiv
• Su, Huayou; Cai, Xing; Wen, Mei & Zhang, Chunyuan (2015). An analytical GPU performance model for 3D stencil computations from the angle of data traffic. The Journal of Supercomputing. ISSN 0920-8542. 71(7), s. 2433–2453. doi: 10.1007/s11227-015-1392-1. Fulltekst i vitenarkiv Vis sammendrag

  The achievable GPU performance of many scientific computations is not determined by a GPU’s peak floating-point rate, but rather how fast data are moved through different stages of the entire memory hierarchy. We take low-order 3D stencil computations as a representative class to study the reachable GPU performance from the angle of data traffic. Specifically, we propose a simple analytical model to estimate the execution time based on quantifying the data traffic volume at three stages: (1) between registers and on-streaming multiprocessor (SMX) storage, (2) between on-SMX storage and L2 cache, (3) between L2 cache and GPU’s device memory. Three associated granularities are used: a CUDA thread, a thread block, and a set of simultaneously active thread blocks. For four chosen 3D stencil computations, NVIDIA’s profiling tools are used to verify the accuracy of the quantified data traffic volumes, by examining a large number of executions with different problem sizes and thread block configurations. Moreover, by introducing an imbalance coefficient, together with the known realistic memory bandwidths, we can predict the execution time usage based on the quantified data traffic volumes. For the four 3D stencils, the average error of the time predictions is 6.9 % for a baseline implementation approach, whereas for a blocking implementation approach the average prediction error is 9.5 %.

• Cai, Xing (2015). Parallel Computing. I Engquist, Björn (Red.), Encyclopedia of Applied and Computational Mathematics. Springer. ISSN 9783540705284. s. 1129–1132. doi: 10.1007/978-3-540-70529-1_424. Fulltekst i vitenarkiv
• Huang, Dafei; Wen, Mei; Xun, Changqing; Chen, Dong; Cai, Xing & Qiao, Yuran [Vis alle 8 forfattere av denne artikkelen] (2014). Automated Transformation of GPU-Specific OpenCL Kernels Targeting Performance Portability on Multi-Core/Many-Core CPUs. I Silva, Fernando (Red.), Euro-Par 2014 Parallel Processing. Springer. ISSN 9783319098722. s. 210–221. doi: 10.1007/978-3-319-09873-9_18. Fulltekst i vitenarkiv
• Dong, Xinnan; Chai, Jun; Yang, Jing (NUDT); Wen, Mei; Wu, Nan & Cai, Xing [Vis alle 8 forfattere av denne artikkelen] (2014). Utilizing Multiple Xeon Phi Coprocessors on One Compute Node. I Sun, Xian-he (Red.), Algorithms and Architectures for Parallel Processing. Springer. ISSN 9783319111933. s. 68–81. doi: 10.1007/978-3-319-11194-0_6. Fulltekst i vitenarkiv
• Zhang, Wei; Wei, Wenjie & Cai, Xing (2014). Performance modeling of serial and parallel implementations of the fractional Adams-Bashforth-Moulton method. Fractional Calculus and Applied Analysis. ISSN 1311-0454. 17(3), s. 617–637. doi: 10.2478/s13540-014-0189-x. Fulltekst i vitenarkiv
• Zhang, Wei; Cai, Xing & Holm, Sverre (2014). Time-fractional heat equations and negative absolute temperatures. Computers and Mathematics with Applications. ISSN 0898-1221. 67(1), s. 164–171. doi: 10.1016/j.camwa.2013.11.007. Fulltekst i vitenarkiv
• Wen, Mei; Su, Huayou; Wei, Wenjie; Wu, Nan; Cai, Xing & Zhang, Chunyuan (2014). High efficient sedimentary basin simulations on hybrid CPU-GPU clusters. Cluster Computing. ISSN 1386-7857. 17(2), s. 359–369. doi: 10.1007/s10586-013-0300-9. Fulltekst i vitenarkiv
• Su, Huayou; Wu, Nan; Wen, Mei; Zhang, Chunyuan & Cai, Xing (2013). On the GPU-CPU Performance Portability of OpenCL for 3D Stencil Computations, 2013 International Conference on Parallel and Distributed Systems (ICPADS). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781479920815. s. 78–85. doi: 10.1109/ICPADS.2013.23. Fulltekst i vitenarkiv
• Wei, Wenjie; Clark, Stuart R; Su, Huayou; Wen, Mei & Cai, Xing (2013). Balancing efficiency and accuracy for sediment transport simulations. Computational Science & Discovery. ISSN 1749-4680. 6(1), s. 015011–015011. doi: 10.1088/1749-4699/6/1/015011. Fulltekst i vitenarkiv
• Langguth, Johannes; Wu, Nan; Chai, Jun & Cai, Xing (2013). On the GPU performance of cell-centered finite volume method over unstructured tetrahedral meshes. I Tumeo, Antonino; Feo, John; Villa, Oreste & Secchi, Simone (Red.), Proceedings of the 3rd Workshop on Irregular Applications: Architectures and Algorithms. ACM Publications. ISSN 9781450325035. doi: 10.1145/2535753.2535765. Fulltekst i vitenarkiv
• Chai, Jun; Su, Huayou; Wen, Mei; Cai, Xing; Wu, Nan & Zhang, Chunyuan (2013). Resource-efficient utilization of CPU/GPU-based heterogeneous supercomputers for Bayesian phylogenetic inference. The Journal of Supercomputing. ISSN 0920-8542. 66(1), s. 364–380. doi: 10.1007/s11227-013-0911-1. Fulltekst i vitenarkiv
• Chai, Jun; Wen, Mei; Wu, Nan; Huang, Dafei; Yang, Jing (NUDT) & Cai, Xing [Vis alle 8 forfattere av denne artikkelen] (2013). Simulating cardiac electrophysiology in the era of GPU-cluster computing. IEICE transactions on information and systems. ISSN 0916-8532. E96-D(12), s. 2587–2595. doi: 10.1587/transinf.E96.D.2587. Fulltekst i vitenarkiv
• Su, Huayou; Wu, Nan; Wen, Mei; Zhang, Chunyuan & Cai, Xing (2013). Performance of sediment transport simulations on NVIDIA’s Kepler architecture. Procedia Computer Science. 18, s. 1275–1281. doi: 10.1016/j.procs.2013.05.294. Fulltekst i vitenarkiv Vis sammendrag

  Aiming to understand how high-performance CUDA programming can be done for NVIDIA's new Kepler architecture, we have investigated a specific case of simulating sediment transport. The arisen stencil computations have distinct features connected to the two nonlinear partial differential equations that constitute the mathematical model. Consequently, the required CUDA programming effort differs for the two corresponding CUDA kernel functions. While Kepler's new read-only data cache brings enough benefits for one kernel function, performance of the other kernel function is further enhanceable through using the shared memory and so-called halo threads. The highest achieved performance of the stencil computation amounts to 190.45 GFLOPs on a Tesla K20 GPU.

Se alle arbeider i NVA

• Cai, Xing & Trotter, James David (2024). Enabling Cell-Based Simulations of Cardiac Electrophysiology with High-Performance Computing. Fulltekst i vitenarkiv
• Trotter, James David & Cai, Xing (2024). Distributed memory parallel simulator of the EMI model. Fulltekst i vitenarkiv
• Gatti, Alessandro; Trotter, James David; Skeie, Tor; Arevalo, Hermenegild & Cai, Xing (2024). A resource-efficient open-source solver for monodomain equations in cardiac electrophysiology. Fulltekst i vitenarkiv
• Jæger, Karoline Horgmo; Trotter, James David; Cai, Xing; Arevalo, Hermenegild & Tveito, Aslak (2024). Modeling ion channel reorganisation due to ischemia in cell-based models of the myocyte. Fulltekst i vitenarkiv
• Cai, Xing (2023). Towards high-performance unstructured-mesh computations. Fulltekst i vitenarkiv
• Pogorelov, Konstantin; Trotter, James David; Cai, Xing & Langguth, Johannes (2023). Machine learning approaches for sparse matrix vector optimization. Fulltekst i vitenarkiv
• Cai, Xing (2023). Quality control and innovation in computing and networking using the eX3 testbed. Fulltekst i vitenarkiv
• Bruaset, Are Magnus; Cai, Xing; Desprez, Frédéric & Parsons, Mark (2023). Editorial: Heterogeneous computing in physics-based models. Frontiers in Physics. 11. doi: 10.3389/fphy.2023.1320450. Fulltekst i vitenarkiv
• Telle, Åshild; Trotter, James; Cai, Xing; Kuchta, Miroslav; Finsberg, Henrik & Sundnes, Joakim [Vis alle 7 forfattere av denne artikkelen] (2022). Modeling cardiac mechanics using a cell-based framework. Fulltekst i vitenarkiv Vis sammendrag

  Cardiac tissue primarily consists of interconnected cardiac cells which contract in a synchronized manner as the heart beats. Most computational models of cardiac tissue, however, homogenize out the individual cells and their surroundings. This approach has been immensely useful for describing cardiac mechanics on an overall level, but gives very limited understanding of the interaction between individual cells and their intermediate surroundings. Several models have been developed for single cells, see e.g. [1, 2]. In this work, we extend the mechanical part of these frameworks to a domain representing multiple cells, allowing us to investigate cell-matrix and cell-cell interactions. We present a mechanical model in which each cell and the extracellular matrix have an explicit geometrical representation, similar to the electrophysiological model presented in [3]. The strain energy functions are defined separately for each of the intracellular and extracellular subdomains, while we assume continuity of displacement and stresses along the membrane. Active tension is only assigned to the intracellular subdomain. For each state, we find an equilibrium solution using the finite element method. We explore passive and active mechanics for a single cell surrounded by an extracellular matrix and for small collections of cells combined into tissue blocks. The explicit geometric representation gives rise to highly varying strain and stress patterns. We show that the extracellular matrix stiffness highly influences the cardiomyocyte stresses during contraction. Through large-scale simulations enabled by high-performance computing, we also demonstrate that our model can be scaled to small collections of cells, resembling small cardiac tissue samples. [1] Tracqui, T. and Ohayon, J. An integrated formulation of anisotropic force–calcium relations driving spatio-temporal contractions of cardiac myocytes. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences (2009). [2] Ruiz-Baier, R. Gizzi, A., Rossi, S. Cherubini, C. Laadhari, A. Filippi, S. and Quarteroni, A. Mathematical modelling of active contraction in isolated cardiomyocytes. Mathematical Medicine and Biology (2014). [3] Tveito, A., Jæger, KH. Kuchta, M. Mardal, K-A. and Rognes, ME. A cell-based framework for numerical modeling of electrical conduction in cardiac tissue. Frontiers in Physics (2017). \end{thebibliography}

• Trotter, James; Langguth, Johannes & Cai, Xing (2021). Automated Code Generation for GPU-Based Finite Element Computations in FEniCS. Fulltekst i vitenarkiv Vis sammendrag

  Developing high-performance finite element codes normally requires hand-crafting and fine tuning of computational kernels, which is not an easy task to carry out for each and every problem. Automated code generation has proved to be a highly productive alternative for frameworks like FEniCS, where a compiler is used to automatically generate suitable kernels from high-level mathematical descriptions of finite element problems. This strategy has so far enabled users to develop and run a variety of high-performance finite element solvers on clusters of multicore CPUs. We have recently enhanced FEniCS with GPU acceleration by enabling its internal compiler to generate CUDA kernels that are needed to offload finite element calculations to GPUs, particularly the assembly of linear systems. This poster presents the results of GPU-accelerating FEniCS and explores performance characteristics of auto-generated CUDA kernels and GPU-based assembly of linear systems for finite element methods.

• Cai, Xing (2020). On parallel simulation of porous media flow. Fulltekst i vitenarkiv Vis sammendrag

  Parallel computing has become an indispensable tool for efficiently simulating the complicated multi-phase flows in porous media. To achieve good performance, we need to pay special attention to the foundation of parallelization, namely, how the computation is decomposed among the hardware processing units. The prerequisite mesh-partitioning problem boils down to an intricate interplay among load balance, communication overhead, and effectiveness of the resulting numerical calculation. Specifically, we quantitatively analyse the negative performance impact caused by non-contributing computations that are associated with the ``ghost computational cells" per subdomain. We also show how these non-contributing computations can be avoided by reordering the computational cells of each subdomain. Moreover, we propose a new graph-edge weighting scheme that can improve the mesh partitioning quality, aiming at a balance between handling the heterogeneity of geological properties and restricting the communication overhead. The findings are applied to the open-source Flow simulator from the OPM framework (https://opm-project.org), leading to substantial improvements of the parallel performance.

• Langguth, Johannes; Gaur, Namit; Arevalo, Hermenegild; Jarvis, Chad; Altanaite, Neringa & Lan, Qiang [Vis alle 7 forfattere av denne artikkelen] (2020). Towards detailed Organ-Scale Simulations in Cardiac Electrophysiology. Fulltekst i vitenarkiv
• Langguth, Johannes; Gaur, Namit; Arevalo, Hermenegild; Jarvis, Chad; Altanaite, Neringa & Lan, Qiang [Vis alle 7 forfattere av denne artikkelen] (2020). Towards detailed Organ-Scale Simulations in Cardiac Electrophysiology. Fulltekst i vitenarkiv
• Hustad, Kristian Gregorius; Cai, Xing; Langguth, Johannes & Arevalo, Hermenegild (2020). Efficient simulations of patient-specific electrical heart activity on the DGX-2. Fulltekst i vitenarkiv
• Cai, Xing (2019). Use of modern processor architectures for computing the electrical activity in the heart. Fulltekst i vitenarkiv Vis sammendrag

  Recent advances in the mathematical models of electro-cardiology have prompted new requirements for computation. The computational challenges span from a detailed simulation of subcellular calcium handling, through a proper cell-tissue coupling, to a whole heart modeled on an unstructured computational mesh. The sheer size of the computation often can only be satisfied by cutting-edge computer clusters, where the main computing power is provided by non-conventional processors (also called accelerators), such as GPUs or many core processors. In this talk, we discuss the related issues of programming and performance engineering. The particular topics include CPU-accelerator computing, memory traffic minimization, and code vectorization.

• Langguth, Johannes; Arevalo, Hermenegild; Hustad, Kristian Gregorius & Cai, Xing (2019). Towards detailed real-time simulations of cardiac arrhythmia. Fulltekst i vitenarkiv
• Trotter, James David; Cai, Xing; Funke, Simon Wolfgang & Langguth, Johannes (2019). Compiling finite element variational forms for GPU-based assembly. Fulltekst i vitenarkiv
• Cai, Xing (2019). Unstructured computational meshes and data locality. Fulltekst i vitenarkiv
• Cai, Xing (2019). Heterogeneous computing for cardiac electrophysiology. Fulltekst i vitenarkiv
• Thune, Andreas & Cai, Xing (2019). Balancing the numerical and parallel performance for reservoir simulations. Fulltekst i vitenarkiv
• Thune, Andreas & Cai, Xing (2018). Unstructured mesh partitioning in the presence of strong coefficient heterogeneity. Fulltekst i vitenarkiv
• Cai, Xing (2018). Education in HPC and Data Science at Simula Research Lab and UiO. Fulltekst i vitenarkiv
• Trotter, James David; Langguth, Johannes & Cai, Xing (2018). Quantifying data traffic of sparse matrix-vector multiplication in a multi-level memory hierarchy. Fulltekst i vitenarkiv
• Cai, Xing (2018). Heterogeneous Computing: Programming, Performance and Applications. Fulltekst i vitenarkiv
• Langguth, Johannes; Hermenegild, Arevalo; Jarvis, Chad & Cai, Xing (2018). Towards Detailed Organ-Scale Simulations in Cardiac Electrophysiology. Fulltekst i vitenarkiv
• Langguth, Johannes; Cai, Xing & Chai, Jun (2017). Heterogeneous Manycore Simulations in Cardiac Electrophysiology. Fulltekst i vitenarkiv Vis sammendrag

  The demand for computing power in computational cardiology is continuously increasing due to the use of physiologically realistic cell models and the need to simulate the heart and use higher resolutions in time and space. And while parallel computing itself is widely used in the field by now, the arrival of heterogeneous multicore architectures presents an important opportunity for speeding up the cardiac simulation process. However, this speedup is not obtained effortlessly, and achieving it presents numerous computational challenges of its own.We present a summary of our experiences from the implementation of multiple cardiac research codes on multicore, manycore, and GPU processors. Our main goal is to highlight the platform-specific challenges we face in our applications, ranging from efficient data movement to optimizations for large-scale irregular computations. Based on that, we discuss the suitability of the different plattforms for cardiac computations.

• Cai, Xing & Langguth, Johannes (2017). Accelerated high-performance computing for computational cardiac electrophysiology. Fulltekst i vitenarkiv
• Cai, Xing (2017). OpenMP Programming. Fulltekst i vitenarkiv
• Lagraviere, Jeremie Alexandre Emilien; Langguth, Johannes; Sourouri, Mohammed; Ha, Hoai Phuong & Cai, Xing (2016). On the performance and energy efficiency of the PGAS programming model on multicore architectures. Fulltekst i vitenarkiv
• Lagravière, Jérémie; Ha, Phuong & Cai, Xing (2015). Is PGAS ready for the challenge of energy efficiency? A study with the NAS benchmark. Fulltekst i vitenarkiv Vis sammendrag

  In this study we compare the performance and power efficiency of Unified Parallel C (UPC), MPI and OpenMP by running a set of kernels from the NAS Benchmark.One of the goals of this study is to focus on the Partitioned Global Address Space (PGAS) model, in order to describe it and compare it to MPI and OpenMP.In particular we consider the power efficiency expressed in millions operations per second per watt as a criterion to evaluate the suitability of PGAS compared to MPI and OpenMP.Based on these measurements, we provide an analysis to explain the difference of performance between UPC, MPI, and OpenMP.

• Gaur, Namit; Cai, Xing & Rudy, Yoram (2015). Dysfunctional Sarcoplasmic Reticulum Ca2+ Release Underlies Arrhythmogenic Triggers in Catecholaminergic Polymorphic Ventricular Tachycardia: A Simulation Study in a Human Ventricular Myocyte Model. Fulltekst i vitenarkiv
• Gaur, Namit; Cai, Xing & Rudy, Yoram (2014). Mathematical Modeling of Ca Handling and Computational Studies of Ca-related Arrhythmogenesis in Heart. Fulltekst i vitenarkiv
• Gaur, Namit; Cai, Xing & Rudy, Yoram (2014). Arrhythmogenic Mechanisms and Therapeutic Targets for Catecholaminergic Polymorphic Ventricular Tachycardia: A Simulation Study in a Human Ventricular Myocyte. Fulltekst i vitenarkiv
• Gaur, Namit; Cai, Xing & Rudy, Yoram (2014). Cellular Arrhythmogenesis in CPVT in a computational model of cardiac ventricular myocyte. Fulltekst i vitenarkiv
• Gaur, Namit; Cai, Xing & Rudy, Yoram (2014). Spontaneous Ca2+ Release and Ca2+ Waves Underlie Early and Delayed Afterdepolarizations, and Triggered Activity in Ryanodine Receptor Mutation associated with Catecholaminergic Polymorphic Ventricular Tachycardia. Fulltekst i vitenarkiv
• Gaur, Namit; Cai, Xing & Rudy, Yoram (2014). Cellular Arrhythmogenesis in CPVT in a Computational Model of Cardiac Ventricular Myocyte. Fulltekst i vitenarkiv
• Gaur, Namit; Cai, Xing & Rudy, Yoram (2014). Spontaneous Ca2+ Release and Ca2+ Waves Underlie Early and Delayed Afterdepolarizations, and Triggered Activity, in Ryanodine Receptor Mutations Associated With Catecholaminergic Polymorphic Ventricular Tachycardia. Fulltekst i vitenarkiv
• Cai, Xing; Hake, Johan & Lines, Glenn (2014). Supercomputing-Enabled Study of Subcellular Calcium Dynamics. Fulltekst i vitenarkiv
• Cai, Xing (2013). Introduction to Scientific Writing. Fulltekst i vitenarkiv
• Cai, Xing (2013). Adopting Heterogeneous Hardware Platforms for Scientific Computing. Fulltekst i vitenarkiv
• Cai, Xing (2013). Scientific Computing on Accelerator-Based Supercomputers. Fulltekst i vitenarkiv
• Cai, Xing & Unat, Didem (2013). Mint: a User-Friendly C-to-CUDA Code Translator. Fulltekst i vitenarkiv Vis sammendrag

  Aiming at automated source-to-source code translation from C to CUDA, we have developed the Mint framework. Users only need to annotate serial C code with a few compiler directives, specifying host-device data transfers plus the parallelization depth and granularity of loop nests. Mint then generates CUDA code as output, while carrying out on-chip memory optimizations that will greatly benefit 3D stencil computations. Several real-world applications have been ported to GPU using Mint.

• Su, Huayou; Wu, Nan; Wen, Mei & Cai, Xing (2013). Performance of Sediment Transport Simulations on NVIDIA's Kepler Architecture. doi: 10.1016/j.procs.2013.05.294. Fulltekst i vitenarkiv
• Lagraviere, Jeremie Alexandre Emilien; Prugger, Martina; Einkemmer, Lukas; Langguth, Johannes; Ha, Hoai Phuong & Cai, Xing (2016). Implementing and optimizing a Sparse Matrix-Vector Multiplication with UPC. Institutt for informatikk Tromsø Tromsø. Fulltekst i vitenarkiv Vis sammendrag

  Programmability and performance-per-watt are the major challenges of the race to Exascale. In this study we focus on Partitioned Global Address Space (PGAS) languages, using UPC as a particular example. This category of parallel languages provides ease of programming as a strong advantage over the classic Message Passing Interface(MPI). PGAS has also advantages compared to classic shared memory programming (OpenMP), as by nature a PGAS program is meant to work on a single-node and multinode machine without changing the code. Our goal in this technical report, is to use UPC in order to implement a memory bound problem, which involves irregular inter-thread communication. To represent this problem we perform a SParse Matrix-Vector multiplication (SpMV) over unstructured data. We implemented different versions of the UPC-SpMV for different levels in the code complexity. In this technical report, we give a description of this various versions of the UPC-SpMV and a set of results using single-node and multi-node machine hardware scenarios.

• Lagraviere, Jeremie; Ha, Hoai Phuong & Cai, Xing (2015). Evaluation of the power efficiency of UPC, OpenMP and MPI. Institutt for informatikk Tromsø. Fulltekst i vitenarkiv Vis sammendrag

  In this study we compare the performance and power efficiency of Unified Parallel C (UPC), MPI and OpenMP by running a set of kernels from the NAS Benchmark. One of the goals of this study is to focus on the Partitioned Global Address Space (PGAS) model, in order to describe it and compare it to MPI and OpenMP. In particular we consider the power effi- ciency expressed in millions operations per second per watt as a criterion to evaluate the suitability of PGAS compared to MPI and OpenMP. Based on these measurements, we provide an analysis to explain the difference of performance between UPC, MPI, and OpenMP.

Se alle arbeider i NVA