HEVC early termination methods for optimal CU decision utilizing encoding residual information

High-efficiency video coding (HEVC) utilizes a quadtree-structured coding unit (CU) for more possible sizes of basic blocks for later prediction. The exhaustive search for the optimal CU depth requires high complexity, which causes difficulties for real-time streaming. In the proposed method, we make use of the residual statistics of prediction units (PUs) in the current encoded CU to make a decision on CU early termination. The residual histogram is discussed, and the optimal early termination (ET) zone is designed; if the PU residual statistics fall in the ET zone, the early termination is triggered. The optimal parameters for the ET zone are solved and analyzed in the training stage, and can be estimated with table lookup and simple computations in the testing stage. Another early termination method using the neighboring SKIP information is also proposed. We use SKIP information from more meaningful CU neighbors defined and used in HEVC. We combine the proposed residual-based and SKIP-based algorithms with a state-of-the-art method. The experimental results show that compared to two state-of-the-art methods in both low delay (LP) and random access (RA) configurations, our preferred proposed combination can achieve the best BDBR results (0.737 % in LP and 0.471 % in RA) with a good corresponding TS performances (49.87 % in LP and 54.96 % in RA), making HEVC more suitable for real-time applications.