The Advanced Space-based Solar Observatory (ASO-S), as one of the second series scientific satellites in the space science pilot project, was proposed by the Purple Mountain Observatory of Chinese Academy of Sciences. ASO-S aims to observe and investigate the two solar eruptive phenomena, i.e., solar flares and coronal mass ejections, and their relationships with magnetic field. Hard X-ray Imager (HXI) is one of the payloads aboard ASO-S. As an important part of HXI, the calorimeter is responsible for measuring solar hard X-rays in the energy range of 30--200 keV. Before the launch of ASO-S, a ground test system was necessary to verify the function and performance of the HXI calorimeter. There are 99 lanthanum bromide detectors in the HXI calorimeter, each controlled by 8 identical front-end electronic boards. In addition to testing the performance of each channel, the ground test system also needs to simulate the operation of the calorimeter under different solar activities in orbit, conducting a comprehensive test. Moreover, the ground test system needs to be stable enough to meet the long-term testing needs of the calorimeter in different tests, including standalone testing, environmental experiments, thermal vacuum and vibration tests. According to the test requirements, a ground test board and a software in the host computer were designed, combined with radioactive sources, a DC (Direct Current) power supply, and a high-voltage module to form a ground test system for the HXI calorimeter. It can simultaneously configure and manage 8 front-end electronic boards, efficiently complete command sending and data receiving, and meet the maximum data output bandwidth of 400 Mbps of the calorimeter. The ground test system can fully meet the requirement of automatic control, status monitoring, data acquisition and data analysis for the HXI calorimeter. Using this ground test system, all function and performance tests for the HXI calorimeter were completed with the acquisition of the noise, the dead time, the energy resolution and other performance data. The ground test system provides strong guarantee for the high performance operation of the HXI calorimeter in orbit.