Compiler-Driven Kernel Fusion for Efficient Graph Convolution Processing

Authors

  • Kornel Maziarczyk Department of Computer Science, Colorado State University, Fort Collins, CO, USA. Author
  • Ann-Louise Lindblad Department of Computer Science and Engineering, University at Buffalo, Buffalo, NY, USA. Author

Keywords:

graph convolution; kernel fusion; deep learning compilers; GPU acceleration; graph neural networks; systems architecture; sustainable computing; robustness; fairness; governance

Abstract

Graph convolution processing has become a foundational workload for relational learning, recommendation, fraud detection, scientific simulation, and socio-technical decision support. Yet its execution remains difficult because neighborhood aggregation, feature transformation, message passing, and normalization exhibit irregular memory access, variable work per vertex, and high sensitivity to graph topology. This paper examines compiler-driven kernel fusion as a system-level strategy for efficient graph convolution processing. Rather than treating fusion as a local operator optimization, the paper frames it as an architectural decision that reshapes execution graphs, memory traffic, scheduling, deployment planning, and governance. The analysis considers how compilers can combine aggregation, transformation, activation, and normalization stages to reduce intermediate materialization and launch overhead, while also exposing trade-offs in occupancy, register pressure, cache locality, and load balance. It compares dense and sparse graph settings, homogeneous and heterogeneous accelerators, and single-device and distributed infrastructures. The paper further discusses robustness, fairness, sustainability, auditability, and policy implications, arguing that performance gains from fusion are inseparable from questions of correctness, reproducibility, and equitable access to computational resources. Case illustrations from graph neural network compilers, GPU runtimes, and accelerator designs show that fusion is most effective when compiler transformations, runtime scheduling, and hardware abstractions are co-designed. The conclusion is that efficient graph convolution processing requires not only faster kernels, but also transparent compilation pipelines, robust validation, and governance frameworks that make optimization choices accountable across deployment contexts.

References

1. Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations. OpenReview.

2. Hamilton, W. L., Ying, R., & Leskovec, J. (2017). Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems (Vol. 30, pp. 1024-1034). Curran Associates.

3. Defferrard, M., Bresson, X., & Vandergheynst, P. (2016). Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems (Vol. 29, pp. 3844-3852). Curran Associates.

4. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2018). Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations. OpenReview.

5. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2020). A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1), 4-24. https://doi.org/10.1109/TNNLS.2020.2978386

6. Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., Gulcehre, C., Song, F., Ballard, A., Gilmer, J., Dahl, G., Vaswani, A., Allen, K., Nash, C., Langston, V., ... Pascanu, R. (2018). Relational inductive biases, deep learning, and graph networks. arXiv. https://arxiv.org/abs/1806.01261

7. Wang, M., Yu, L., Zheng, D., Gan, Q., Gai, Y., Ye, Z., Li, M., Zhou, J., Huang, Q., Ma, C., Huang, Z., Guo, Q., Zhang, H., Lin, H., Zhao, J., Li, J., Smola, A. J., & Zhang, Z. (2019). Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv. https://arxiv.org/abs/1909.01315

8. Fey, M., & Lenssen, J. E. (2019). Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds. https://arxiv.org/abs/1903.02428

9. Narayanan, D., Harlap, A., Phanishayee, A., Seshadri, V., Devanur, N. R., Ganger, G. R., Gibbons, P. B., & Zaharia, M. (2019). PipeDream: Generalized pipeline parallelism for DNN training. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (pp. 1-15). ACM.

10. Ragan-Kelley, J., Barnes, C., Adams, A., Paris, S., Durand, F., & Amarasinghe, S. (2013). Halide: A language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (pp. 519-530). ACM.

11. Chen, T., Moreau, T., Jiang, Z., Zheng, L., Yan, E., Shen, H., Cowan, M., Wang, L., Hu, Y., Ceze, L., Guestrin, C., & Krishnamurthy, A. (2018). TVM: An automated end-to-end optimizing compiler for deep learning. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (pp. 578-594). USENIX Association.

12. Li, M., Liu, Y., Liu, X., Sun, Q., You, X., Yang, H., Luan, Z., & Qian, D. (2020). The deep learning compiler: A comprehensive survey. IEEE Transactions on Parallel and Distributed Systems, 32(3), 708-727. https://doi.org/10.1109/TPDS.2020.3030548

13. Cyphers, S., Bansal, A. K., Bhiwandiwalla, A., Bobba, J., Brookhart, M., Chakraborty, A., Constable, W., Convey, C., Cook, L., Kanawi, O., Kim, R., Knight, J., Korovaiko, N., Palmer, J., Rakhmanov, A., & Sokolov, A. (2018). Intel nGraph: An intermediate representation, compiler, and executor for deep learning. arXiv. https://arxiv.org/abs/1801.08058

14. Jia, Z., Padon, O., Thomas, J., Warszawski, T., Zaharia, M., & Aiken, A. (2019). TASO: Optimizing deep learning computation with automatic generation of graph substitutions. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (pp. 47-62). ACM.

15. Ma, L., Xie, Z., Yang, Z., Xue, J., Miao, Y., Cui, W., Hu, W., Yang, F., Zhang, L., & Zhou, L. (2020). Rammer: Enabling holistic deep learning compiler optimizations with rTasks. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (pp. 881-897). USENIX Association.

16. Xie, X., Peng, H., Hasan, A., Huang, S., Zhao, J., Fang, H., ... & Ding, C. (2023, October). Accel-gcn: High-performance gpu accelerator design for graph convolution networks. In 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD) (pp. 01-09). IEEE.

17. Wang, Y., Feng, B., Li, G., Xie, X., Li, Y., Ding, Y., & Xie, Y. (2021). GNNAdvisor: An adaptive and efficient runtime system for GNN acceleration on GPUs. In Proceedings of the 15th USENIX Symposium on Operating Systems Design and Implementation (pp. 515-531). USENIX Association.

18. Hu, Y., Ye, Z., Wang, M., Yu, J., Zheng, D., Xu, M., Zhang, C., Zhang, Z., & Wang, Y. (2020). FeatGraph: A flexible and efficient backend for graph neural network systems. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 1-13). IEEE.

19. Zhang, M., Wu, Y., & Li, G. (2021). Dory: Overlapping and parallelizing GNN training on GPU. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (pp. 286-300). ACM.

20. Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020). Graph neural networks: A review of methods and applications. AI Open, 1, 57-81. https://doi.org/10.1016/j.aiopen.2021.01.001

Downloads

Published

2026-11-10