Going Deeper with Convolutions

Problem

Framing

ImageNet CNNs improved by getting deeper and wider, but naive scaling wasted parameters and compute. The paper closes this with Inception: parallel multi-scale branches plus $1 \times 1$ reductions that keep inference near $1.5$ billion multiply-adds. GoogLeNet reaches $6.67\%$ top-5 error on ILSVRC14.

Currently Used Methods

Foundational

• @krizhevskyAlexNet2012 — large-scale ImageNet CNN baseline with strong classification gains.
  • Limitation in context: much larger parameter count for worse top-5 accuracy.
• @simonyanVGGVeryDeep2014 — deeper CNNs from uniform stacks of small convolutions.
  • Limitation in context: depth scaling lacks Inception-style multi-branch efficiency.
• @lecunGradientbasedLearningApplied1998 — canonical conv-pool hierarchy for visual recognition.
  • Limitation in context: no explicit parallel multi-scale processing inside each stage.
• "Network In Network" — introduces $1 \times 1$ convolutions and local micro-networks.
  • Limitation in context: stops short of sparse multi-branch module design.
• @renFasterRCNN2015 — CNN-based region proposal and detection pipeline.
  • Limitation in context: backbone efficiency still constrains detection throughput and accuracy.

Proposed Method

Architecture

GoogLeNet is a 22-layer CNN built by stacking Inception modules. Each module runs parallel $1 \times 1$, $3 \times 3$, $5 \times 5$, and pooling branches, then concatenates channels; $1 \times 1$ projections reduce cost before the expensive branches. The classifier head replaces large fully connected blocks with global average pooling, and training adds two auxiliary classifiers.

Loss / Objective

Training sums the main softmax loss and two auxiliary softmax losses.

Algorithm

An Inception block applies four branch transforms to $\mathbf{x}$ and concatenates their outputs.

Training Procedure

• Inference budget: about $1.5$ billion multiply-adds.
• Input size: $224 \times 224$.
• Auxiliary heads: $2$.
• Auxiliary-loss weight: $0.3$ each.
• Auxiliary-head hidden layer: $1024$ units.
• Auxiliary-head dropout: $70\%$ dropped outputs.
• Final submission ensemble: $7$ models.
• Dense evaluation: $144$ crops per image.

Evaluation

Datasets

• ILSVRC 2014 classification: about $1.2$M train, $50$k validation, $100$k test, $1000$ classes.
• ILSVRC 2014 detection: $200$ classes.

Metrics

• Classification: top-5 error.
• Detection: mean average precision (mAP).

Headline results

• ILSVRC14 classification (single model, single crop): $10.07\%$ top-5 error.
• ILSVRC14 classification (single model, $144$ crops): $7.89\%$ top-5 error.
• ILSVRC14 classification ($7$-model ensemble): $6.67\%$ top-5 error.
• ILSVRC14 detection: $43.9\%$ mAP.
• Parameter efficiency: about $12\times$ fewer parameters than @krizhevskyAlexNet2012.

Table 1: ILSVRC classification challenge leaderboard by top-5 error

Ablations

• Crop count: denser evaluation cuts top-5 error from $10.07\%$ to $7.89\%$.
• Ensembling: $7$ models cut top-5 error to $6.67\%$.
• Auxiliary classifiers: improve optimization in deeper middle layers.
• Wider variant: gives small extra ensemble gains.

Method Strengths and Weaknesses

Strengths

• Wins ILSVRC14 classification with $6.67\%$ top-5 error.
• Uses about $12\times$ fewer parameters than @krizhevskyAlexNet2012.
• Captures multiple receptive-field scales within one module.
• $1 \times 1$ reductions cut cost before $3 \times 3$ and $5 \times 5$ branches.

Weaknesses

• Best headline number requires a $7$-model ensemble.
• Dense $144$-crop testing is expensive.
• Architecture remains hand-designed and branch-heavy.
• Detection result still depends on proposal-based R-CNN components.

Suggestions from the authors

• Automate Inception topology design instead of hand-crafting modules.
• Test whether the sparse-design principle transfers beyond vision.
• Analyze which architectural choices drive the accuracy gains.
• Reduce memory cost when training deep multi-branch networks.

Links

Prior Papers

• @krizhevskyAlexNet2012 — establishes the ImageNet CNN baseline that GoogLeNet surpasses with far fewer parameters.
• @simonyanVGGVeryDeep2014 — represents the deep uniform-stack design that Inception replaces with multi-branch modules.

Further Papers

• @heKaimingInit2015 — improves optimization for deep rectifier networks, directly relevant to post-GoogLeNet CNN scaling.
• @heDeepResidualLearning2016 — extends the depth-efficiency agenda with residual connections.
• @huangDenseNet2017 — continues efficient deep CNN design through feature reuse and shorter gradient paths.
• @tanEfficientNet2019 — revisits the same accuracy-efficiency tradeoff with principled CNN scaling.