[en] Highlights: • A novel method is presented to optimize CCHP–GSHP system. • Multi-population genetic algorithm is used to solve the optimal method. • The electric efficiency and thermal efficiency of gas engine change with the part-load coefficient. • Comprehensive performance study is taken into consideration. - Abstract: A hybrid system, which couples Combined Cooling, Heating and Power system (CCHP) and Ground Source Heat Pump system (GSHP), simply or rather to say the CCHP–GSHP coupling system, or hybrid system, may efficiently solve the problems when a CCHP or a GSHP operates independently and obtain better system performance. This paper proposes a novel method to optimize the capacity and operation strategy for CCHP–GSHP coupling system. In this method, primary energy saving ratio, CO_2 emission reduction ratio and annual total cost saving ratio are optimization goals; the rated thermal capacity of gas engine in CCHP, the heating/cooling provided by GSHP system to total heating/cooling load ratio and the critical value to determine whether run the gas engine are variables. Multi-Population Genetic Algorithm (MPGA) is selected to solve the optimal model. Furthermore, a case study based on a hotel building is presented and studied to verify the effectiveness of this optimizing method and the corresponding algorithm to solve the model. The results in the case study show that the primary energy saving ratio, carbon dioxide emission reduction ratio, annual total cost saving ratio, comprehensive performance of the hybrid system comparing with the separated generation system are 26.10%, 35.02% 15.13% and 25.42%, respectively.