title: Analysis of dyadic approximation error for hybrid video codecs with integer transforms.
abstract: In this paper, we present an analysis of the dyadic approximation error introduced by the integerization of transform coding in H.264/AVC-like codecs. We derive the analytical formulations for dyadic approximation error and nonorthogonality error. We further classify the dyadic approximation error into a "system error" and a "nonflat error," and proposed two models for them. We found that the "nonflat error" has a substantial impact on video quality if the number of shifting bits at decoder side (DQ_BITS) is small. We also give a theoretical justification on why scaling factors at encoder side are better to be adapted to the rescaling factors at decoder side in H.264/AVC-like codecs.
authors.affiliation: Centre for Signal Processing, Department of Electronic and Information Engineering, Hong Kong Polytechnic University, Kowloon, Hong Kong. chauwai.wong@alumni.polyu.edu.hk
authors.forename: Chau-Wai
authors.initials: CW
authors.lastname: Wong
authors.forename: Wan-Chi
authors.initials: WC
authors.lastname: Siu