- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents AAAE, an application accuracy-aware evaluation framework for exploring the design space of approximate multipliers for energy-efficient Transformer and CNN inference. The framework is mainly composed of four parts: (1) approximate multiplier generation and error characterization; (2) application-level accuracy prediction; (3) hardware cost prediction; and (4) design optimization. For approximate multiplier generation, the work focuses on a compressor-based approximate 8-bit multiplier design in which the least significant nine 4:2 compressors are approximated using nine different approximate compressors proposed in the literature. For accuracy prediction, the work proposed a predictor that is based on three different types of features: multi-scale error features computed from error maps of approximate multipliers, model-conditioned error features computed from the error map and profiled input distribution at different locations in the network, and inference response features computed by running approximate inference using a small set of input images. The hardware cost predictor uses a GNN that leverages compressor hardware characteristics to predict the hardware characteristics of an approximate multiplier design. The optimization stage then searches for Pareto-optimal configurations.
Critical strengths
• A novel accuracy predictor design based on error properties of the multiplier, characteristics of input data, and inference response with approximations in place using a small set of inputs. • Targets the emerging area of using approximate hardware components for improving the energy-efficiency of Transformer and CNN inference
Critical weaknesses
• The method uses the same approximate multiplier across the entire network, so layer-wise approximation selection is not possible. • The work targets compressor-based 8-bit approximate multipliers despite mentioning that "the design space of approximate multipliers is extremely large, involving diverse architectural choices such as bit-width selection, truncation schemes, compressor structures, partial-product reduction, sign processing, and error compensation." • The approximation is limited to the least significant nine 4:2 compressors. Additionally, there is no detailed and convincing justification provided for this constraint specifically when the core purpose of the search is to find Preto optimal configurations from the complete design space. • The distribution profiles generated using accurate simulations are used for model-conditioned error estimation. The distribution drifts that occur due to approximations in the previous stages are not considered at all. Specifically, this becomes an issue when the design space includes aggressive approximations as well, which this work seems to exclude by limiting approximations to the least significant nine 4:2 compressors. • The literature has already widely explored GNN-based hardware characteristics estimation. • Considering that the level of approximation has been significantly constrained, the error metrics presented are insufficient to quantify the performance of the predictors because of the lack of diversity in the overall design space.
Questions for authors
Besides addressing the above mentioned weaknesses, address the following: • Can the proposed technique be used for aggressive approximations? The current approximation space considered is quite limited and does not cover aggressive approximations. • Is the high accuracy of the proposed predictors related to the lack of diversity and aggressiveness in the approximation space? • How generalizable is the proposed technique to a broader design space of approximate multipliers, beyond a small set of compressor-based approximate multipliers? • For aggressive approximations, where intermediate distributions are significantly affected by approximations in earlier stages, how does the overall performance of the proposed framework change? Additionally, how would the contribution of each feature set change for generating an accurate accuracy prediction?
1 comment
- High-Order Approximation Restrictions and Design Space Selection AAAE currently employs 8-bit approximate multipliers based on 4:2 compressors, generating designs with different arithmetic error characteristics and hardware costs by configuring different compressor types. To evaluate the impact of high-order approximation, we further include the eight previously exact high-order 4:2 compressors in the approximate design space. Combined with the nine low-order compressor positions, we generate 1,000 distinct configurations and evaluate their application-level accuracy on five INT8 vision models. Experimental results show that over 98% of the configurations cause severe accuracy degradation, with the approximate models' Top-1 accuracy dropping to 0.17%–1.11%. Since high-order partial-product columns carry larger numerical weights, introducing approximation at these positions generally has a more significant impact on the final computation results. Therefore, we keep the high-order compressors exact and restrict structural exploration to the nine low-order compressor positions, focusing the design space on configurations that offer meaningful trade-offs between application accuracy and hardware cost.
- Uniform Approximate Multiplier Across Different Network Layers AAAE currently employs the same approximate multiplier at the specified computation sites of a target neural network, while its application-level accuracy prediction accounts for the error sensitivities of different computation sites. Specifically, we construct model-conditioned error features using multiplication input distributions, local decision sensitivities, network depth, and computation roles. For Transformers, we separately consider the Query, Key, and Value projections, the attention output projection, and the linear layers in the feed-forward network to characterize how approximation errors at different computation sites affect the final classification results. Thus, AAAE does not assume uniform error tolerance across network layers; instead, it incorporates site-specific error sensitivity information into accuracy prediction. The current framework does not yet assign different approximate multipliers to individual network layers. Supporting layer-wise heterogeneous designs would require extending the design variables from a single multiplier configuration to joint configurations across multiple layers, together with a prediction model capable of capturing accumulated errors and interactions among different approximation configurations. This remains a limitation of the current framework and a direction for future work.
- Applicability to Aggressive Approximation and Different Multiplier Architectures AAAE's application-level accuracy analysis is not inherently restricted to a specific 4:2 compressor topology. Instead, it constructs an error surface from the complete input-output mapping of each approximate multiplier. Based on this representation, multi-scale error features characterize error magnitudes, signed biases, and input-dependent error distributions, while model-conditioned features incorporate the computational characteristics of the target neural network to predict their impact on application-level accuracy. Therefore, for other approximate multipliers with the same INT8 input-output definitions, including designs based on different compressor structures, truncation, or other approximation strategies, the same error-characterization pipeline can be applied as long as their complete input-output mappings are available. This allows the error-characterization approach to extend beyond the current nine low-order compressor configurations.