EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Problem

Framing

CNNs had been scaled one dimension at a time, which wastes compute and yields early saturation. EfficientNet closes this gap with compound scaling: jointly scale depth, width, and input resolution from one searched baseline. On ImageNet, EfficientNet-B7 reaches 84.3% top-1 with 66M parameters.

Currently Used Methods

Foundational

• @heDeepResidualLearning2016 — residual connections make very deep CNNs trainable.
  • Limitation in context: depth-only scaling gives diminishing accuracy gains.
• @huangDenseNet2017 — dense connectivity improves reuse and gradient flow.
  • Limitation in context: no principled rule for joint multi-axis scaling.
• @szegedyGoogLeNet2015 — Inception improves efficiency with hand-crafted factorization.
  • Limitation in context: scaling remains manual and architecture-specific.
• MnasNet: Platform-Aware Neural Architecture Search for Mobile — searched mobile backbone with strong efficiency.
  • Limitation in context: baseline search alone does not solve large-scale scaling.
• AmoebaNet: Regularized Evolution for Image Classifier Architecture Search — strong NAS classifier family.
  • Limitation in context: higher accuracy requires far larger parameter budgets.

Proposed Method

Architecture

EfficientNet starts from EfficientNet-B0, a NAS-designed MBConv network with squeeze-and-excitation and Swish. It keeps stage structure fixed and scales all stages with one global coefficient $\phi$.

Loss / Objective

The paper optimizes standard supervised classification loss while changing only the scaling rule.

Scaling Rule / Algorithm

Compound scaling assigns extra compute uniformly across depth, width, and resolution.

Training Procedure

• Optimizer: RMSProp
• Momentum: 0.9
• RMSProp decay: 0.9
• Weight decay: $1 \times 10^{-5}$
• Initial learning rate: 0.256
• LR decay: $\times 0.97$ every 2.4 epochs
• Activation: SiLU / Swish
• Model family: B0 through B7

Evaluation

Datasets

• ImageNet
• CIFAR-10
• CIFAR-100
• Flowers
• Stanford Cars
• FGVC Aircraft
• Oxford-IIIT Pets
• Food-101

Metrics

• Top-1 accuracy
• Top-5 accuracy
• Parameter count
• FLOPs
• Single-core CPU latency
• Transfer-learning accuracy

Headline results

• ImageNet, B0: top-1 77.1%, top-5 93.3%, 5.3M params, 0.39B FLOPs.
• ImageNet, B4: top-1 82.9%, top-5 96.4%, 19M params, 4.2B FLOPs.
• ImageNet, B7: top-1 84.3%, top-5 97.0%, 66M params, 37B FLOPs.
• ImageNet, B7 vs GPipe: same 84.3% top-1 with $8.4\times$ fewer parameters.
• Transfer learning: best on 5 of 8 datasets with $9.6\times$ fewer parameters on average.

Table 1: ImageNet comparisons at matched accuracy levels show EfficientNet using far fewer parameters and FLOPs.

Ablations

• Width scaling: accuracy rises, then saturates near 80%.
• Depth scaling: deeper models help first, then returns taper.
• Resolution scaling: larger inputs help, but gains diminish.
• Compound scaling: best accuracy-efficiency tradeoff at fixed compute.

Method Strengths and Weaknesses

Strengths

• One rule replaces ad hoc depth-only or width-only scaling.
• B7 reaches 84.3% top-1 with 66M parameters.
• Matches GPipe with $8.4\times$ fewer parameters.
• Wins 5 of 8 transfer datasets with much smaller models.

Weaknesses

• Depends on a strong searched baseline, B0.
• Global ratios limit stage-specific scaling choices.
• Largest model still costs 37B FLOPs.
• Coefficients require search, not full theory.

Suggestions from the authors

• Apply compound scaling to other vision architectures.
• Study scaling under stricter latency and hardware constraints.
• Improve baseline architecture search before scaling.
• Extend the rule to more transfer tasks.

Links

Prior Papers

• @heDeepResidualLearning2016 — establishes deep residual scaling baselines that EfficientNet outperforms at similar accuracy.
• @huangDenseNet2017 — provides compact CNN baselines used in the ImageNet efficiency comparisons.
• @szegedyGoogLeNet2015 — anchors the Inception efficiency line that EfficientNet improves on.
• @heKaimingInit2015 — supports stable optimization in the pre-EfficientNet deep CNN regime.

Further Papers

No vault papers identified as further work yet.