Seminar paper · Deep learning · Computer vision

Instagram image analysis with CNNs

I fine-tuned an EfficientNet-B3 network on 4,070 Instagram images to score four content attributes: advertisement, body visibility, human focus, and nudity. On the test set, it more than halved the error of the best classical baseline, and Grad-CAM shows which image regions drive each score.

Solo project PyTorch Multi-output regression

GitHub ↗

01 / Overview

Motivation

Instagram users post billions of images, far too many to label by hand. For marketing research, the content of these images matters: is a post an ad, and how prominently is a person shown?

This project compares a convolutional neural network with seven classical machine learning methods for scoring four visual attributes automatically. The images come from the 74 highest-earning Instagram accounts of 2018, the profiles most relevant for influencer marketing.

Advertisement Body Visible Human Focus Nudity

02 / Dataset

Data and preprocessing

The dataset contains 4,070 images: 55 posts from each of the 74 accounts on the 2018 Instagram Rich List. Each image has a score from 1 to 5 for four target variables:

Advertisement: how strongly the image promotes a product or service
Body visible: the extent to which a human body is shown
Human focus: how much the composition centers on a person
Nudity: the degree of nudity depicted

The data was split 90/10 into 3,663 training and 407 test images, resized to 256 × 256 pixels. To prevent overfitting, I applied a set of random augmentations with Albumentations (see examples).

4,070 images 74 accounts 256×256 px

Data augmentation examples

Original

1 / 10

03 / Architecture

EfficientNet-B3

The model follows a standard transfer-learning design for multi-output regression:

Backbone: EfficientNet-B3, pretrained on ImageNet. It already recognizes general visual concepts, so training only has to adapt these features to the Instagram scoring task.
Regression head: a small fully connected head on top of the backbone outputs the four target scores.

04 / Training

Pipeline

Training follows a two-phase freeze-then-unfreeze schedule:

Epochs 1–5 (warm-up): the backbone is frozen and only the head is trained (learning rate 3e-4). This keeps the randomly initialized head from disrupting the pretrained weights.
Epochs 6–30 (fine-tuning): the whole model is trained end to end at a 10× lower learning rate (3e-5), decayed toward 1e-6 with cosine annealing.

Mixed-precision training (PyTorch AMP) speeds up training and reduces GPU memory use.

05 / Results

CNN outperforms all baselines

All methods were evaluated on the same 407 held-out test images. The CNN reaches an MAE of 0.387, so on the 1–5 scale its predictions are off by less than half a point on average, and an MSE of 0.390. That is 57% lower MAE than the best classical method (SVM) and 75% lower MSE than the best classical method (XGBoost).

All methods: error on 407 test images

Method	MAE	MSE
Linear	1.863	6.488
Decision Tree	1.195	3.235
AdaBoost	1.267	1.972
Random Forest	0.965	1.715
KNN	1.028	1.635
XGBoost	0.970	1.567
SVM	0.890	1.683
CNN (EfficientNet-B3)	0.387	0.390

Lower is better.

06 / Explainability

What the model looks at

Grad-CAM highlights the image regions that drive each score. Use the arrows to switch between the four attributes. The images are public Instagram posts from the seminar dataset, shown for academic illustration.

Source: Kendall Jenner

Source: Lele Pons

Source: Hannes Becker

Original

1 / 5

07 / Discussion

Applications and outlook

Automated image content scores can support influencer marketing analysis (for example, how often accounts post sponsored content), content recommendation, and content moderation such as nudity filters.

The model was trained on only 3,663 images. A larger, more diverse training set and newer architectures such as vision transformers are the obvious next steps toward a more precise model.