code/models/cnn/efficientnet-architecture.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "📝 **Author:** Amirhossein Heydari - 📧 **Email:** <amirhosseinheydari78@gmail.com> - 📍 **Origin:** [mr-pylin/pytorch-workshop](https://github.com/mr-pylin/pytorch-workshop)\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Table of contents**<a id='toc0_'></a>    \n",
    "- [Dependencies](#toc1_)    \n",
    "- [EfficientNet](#toc2_)    \n",
    "  - [Custom EfficientNet](#toc2_1_)    \n",
    "    - [EfficientNet-B0](#toc2_1_1_)    \n",
    "      - [Initialize the Model](#toc2_1_1_1_)    \n",
    "      - [Model Summary](#toc2_1_1_2_)    \n",
    "    - [EfficientNet-B1](#toc2_1_2_)    \n",
    "      - [Initialize the Model](#toc2_1_2_1_)    \n",
    "      - [Model Summary](#toc2_1_2_2_)    \n",
    "    - [EfficientNet-B2](#toc2_1_3_)    \n",
    "      - [Initialize the Model](#toc2_1_3_1_)    \n",
    "      - [Model Summary](#toc2_1_3_2_)    \n",
    "    - [EfficientNet-B3](#toc2_1_4_)    \n",
    "      - [Initialize the Model](#toc2_1_4_1_)    \n",
    "      - [Model Summary](#toc2_1_4_2_)    \n",
    "    - [EfficientNet-B4](#toc2_1_5_)    \n",
    "      - [Initialize the Model](#toc2_1_5_1_)    \n",
    "      - [Model Summary](#toc2_1_5_2_)    \n",
    "    - [EfficientNet-B5](#toc2_1_6_)    \n",
    "      - [Initialize the Model](#toc2_1_6_1_)    \n",
    "      - [Model Summary](#toc2_1_6_2_)    \n",
    "    - [EfficientNet-B6](#toc2_1_7_)    \n",
    "      - [Initialize the Model](#toc2_1_7_1_)    \n",
    "      - [Model Summary](#toc2_1_7_2_)    \n",
    "    - [EfficientNet-B7](#toc2_1_8_)    \n",
    "      - [Initialize the Model](#toc2_1_8_1_)    \n",
    "      - [Model Summary](#toc2_1_8_2_)    \n",
    "  - [PyTorch EfficientNet](#toc2_2_)    \n",
    "    - [EfficientNet-B0](#toc2_2_1_)    \n",
    "      - [Initialize the Model](#toc2_2_1_1_)    \n",
    "      - [Model Summary](#toc2_2_1_2_)    \n",
    "    - [EfficientNet-B1](#toc2_2_2_)    \n",
    "      - [Initialize the Model](#toc2_2_2_1_)    \n",
    "      - [Model Summary](#toc2_2_2_2_)    \n",
    "    - [EfficientNet-B2](#toc2_2_3_)    \n",
    "      - [Initialize the Model](#toc2_2_3_1_)    \n",
    "      - [Model Summary](#toc2_2_3_2_)    \n",
    "    - [EfficientNet-B3](#toc2_2_4_)    \n",
    "      - [Initialize the Model](#toc2_2_4_1_)    \n",
    "      - [Model Summary](#toc2_2_4_2_)    \n",
    "    - [EfficientNet-B4](#toc2_2_5_)    \n",
    "      - [Initialize the Model](#toc2_2_5_1_)    \n",
    "      - [Model Summary](#toc2_2_5_2_)    \n",
    "    - [EfficientNet-B5](#toc2_2_6_)    \n",
    "      - [Initialize the Model](#toc2_2_6_1_)    \n",
    "      - [Model Summary](#toc2_2_6_2_)    \n",
    "    - [EfficientNet-B6](#toc2_2_7_)    \n",
    "      - [Initialize the Model](#toc2_2_7_1_)    \n",
    "      - [Model Summary](#toc2_2_7_2_)    \n",
    "    - [EfficientNet-B7](#toc2_2_8_)    \n",
    "      - [Initialize the Model](#toc2_2_8_1_)    \n",
    "      - [Model Summary](#toc2_2_8_2_)    \n",
    "\n",
    "<!-- vscode-jupyter-toc-config\n",
    "\tnumbering=false\n",
    "\tanchor=true\n",
    "\tflat=false\n",
    "\tminLevel=1\n",
    "\tmaxLevel=6\n",
    "\t/vscode-jupyter-toc-config -->\n",
    "<!-- THIS CELL WILL BE REPLACED ON TOC UPDATE. DO NOT WRITE YOUR TEXT IN THIS CELL -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id='toc1_'></a>[Dependencies](#toc0_)\n",
    "\n",
    "- torchvision models:\n",
    "  - class\n",
    "    - brings in the model class directly\n",
    "    - Allows more control and customization since you are dealing directly with the class. You can override methods, customize initialization, etc.\n",
    "  - function\n",
    "    - This import brings in a function that returns an instance of the model\n",
    "    - Easier and quicker to use, especially for standard models\n",
    "- [pytorch.org/vision/stable/models.html](https://pytorch.org/vision/stable/models.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "import torch\n",
    "from torch import nn\n",
    "from torch.functional import F\n",
    "from torchinfo import summary\n",
    "from torchvision.models import (\n",
    "    EfficientNet,\n",
    "    efficientnet_b0,\n",
    "    efficientnet_b1,\n",
    "    efficientnet_b2,\n",
    "    efficientnet_b3,\n",
    "    efficientnet_b4,\n",
    "    efficientnet_b5,\n",
    "    efficientnet_b6,\n",
    "    efficientnet_b7,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id='toc2_'></a>[EfficientNet](#toc0_)\n",
    "\n",
    "- EfficientNet, developed in 2019 by researchers at [Google AI](https://research.google/)\n",
    "- It is based on the [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/pdf/1905.11946) paper\n",
    "- It was trained on the [ImageNet](https://www.image-net.org/) dataset (first resized to 256x256 then center cropped to 224x224) [[ImageNet viewer](https://navigu.net/#imagenet)]\n",
    "- Known for its balance of accuracy and efficiency, achieving state-of-the-art performance while being significantly more computationally efficient than previous models\n",
    "- The EfficientNet family includes several variants, such as `EfficientNet-B0` through `EfficientNet-B7`, where the number indicates the scaling factor, with B0 being the base model\n",
    "- Achieved high performance in various benchmarks and demonstrated significant efficiency improvements due to the compound scaling method\n",
    "\n",
    "<figure style=\"text-align: center;\">\n",
    "  <img src=\"../../../assets/images/third_party/cnn/architectures/efficientnet.svg\" alt=\"efficientnet-architecture.svg\" style=\"width: 100%;\">\n",
    "  <figcaption>Figure 2. Model Scaling. (a) is a baseline network example; (b)-(d) are conventional scaling that only increases one dimension of network width, depth, or resolution. (e) is our proposed compound scaling method that uniformly scales all three dimensions with a fixed ratio.<br>©️ Image: <a href= \"https://arxiv.org/pdf/1905.11946\">EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks</a></figcaption>\n",
    "</figure>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id='toc2_1_'></a>[Custom EfficientNet](#toc0_)\n",
    "\n",
    "- `Softmax` is missing due to internal implementation of `LogSoftmax` in the `CrossEntropyLoss` function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# swish activation function\n",
    "class Swish(nn.Module):\n",
    "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n",
    "        return x * torch.sigmoid(x)\n",
    "\n",
    "\n",
    "# squeeze-and-excitation (SE) block\n",
    "class SEBlock(nn.Module):\n",
    "    def __init__(self, in_channels: int, reduction: int = 4):\n",
    "        super().__init__()\n",
    "        reduced_channels = in_channels // reduction\n",
    "        self.fc1 = nn.Conv2d(in_channels, reduced_channels, 1)\n",
    "        self.fc2 = nn.Conv2d(reduced_channels, in_channels, 1)\n",
    "\n",
    "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n",
    "        out = F.adaptive_avg_pool2d(x, 1)\n",
    "        out = F.relu(self.fc1(out))\n",
    "        out = torch.sigmoid(self.fc2(out))\n",
    "        return x * out\n",
    "\n",
    "\n",
    "# mobile inverted bottleneck convolution (MBConv) block\n",
    "# the core building block of `EfficientNet`, originally introduced in `MobileNetV2`.\n",
    "class MBConvBlock(nn.Module):\n",
    "    def __init__(\n",
    "        self,\n",
    "        in_channels: int,\n",
    "        out_channels: int,\n",
    "        expand_ratio: int,\n",
    "        kernel_size: int,\n",
    "        stride: int,\n",
    "        reduction: int = 4,\n",
    "        drop_connect_rate: float = 0.2,\n",
    "    ):\n",
    "        super().__init__()\n",
    "        self.drop_connect_rate = drop_connect_rate\n",
    "        self.has_se = reduction > 0\n",
    "        self.stride = stride\n",
    "        mid_channels = in_channels * expand_ratio\n",
    "\n",
    "        # expand phase\n",
    "        if expand_ratio != 1:\n",
    "            self.expand = nn.Conv2d(in_channels, mid_channels, 1, bias=False)\n",
    "            self.bn0 = nn.BatchNorm2d(mid_channels)\n",
    "            self.act0 = Swish()\n",
    "\n",
    "        # depth-wise convolution phase\n",
    "        self.depthwise = nn.Conv2d(mid_channels, mid_channels, kernel_size, stride, (kernel_size - 1) // 2, groups=mid_channels, bias=False)\n",
    "        self.bn1 = nn.BatchNorm2d(mid_channels)\n",
    "        self.act1 = Swish()\n",
    "\n",
    "        # squeeze and excitation phase\n",
    "        if self.has_se:\n",
    "            self.se = SEBlock(mid_channels, reduction)\n",
    "\n",
    "        # output phase\n",
    "        self.project = nn.Conv2d(mid_channels, out_channels, 1, bias=False)\n",
    "        self.bn2 = nn.BatchNorm2d(out_channels)\n",
    "\n",
    "    def drop_connect(self, x: torch.Tensor) -> torch.Tensor:\n",
    "        if not self.training:\n",
    "            return x\n",
    "        keep_prob = 1 - self.drop_connect_rate\n",
    "        random_tensor = keep_prob + torch.rand((x.shape[0], 1, 1, 1), dtype=x.dtype, device=x.device)\n",
    "        binary_tensor = torch.floor(random_tensor)\n",
    "        return x / keep_prob * binary_tensor\n",
    "\n",
    "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n",
    "        identity = x\n",
    "\n",
    "        if hasattr(self, \"expand\"):\n",
    "            x = self.expand(x)\n",
    "            x = self.bn0(x)\n",
    "            x = self.act0(x)\n",
    "\n",
    "        x = self.depthwise(x)\n",
    "        x = self.bn1(x)\n",
    "        x = self.act1(x)\n",
    "\n",
    "        if self.has_se:\n",
    "            x = self.se(x)\n",
    "\n",
    "        x = self.project(x)\n",
    "        x = self.bn2(x)\n",
    "\n",
    "        if self.stride == 1 and x.shape == identity.shape:\n",
    "            if self.drop_connect_rate:\n",
    "                x = self.drop_connect(x)\n",
    "            x += identity\n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CustomEfficientNet(nn.Module):\n",
    "    def __init__(\n",
    "        self,\n",
    "        width_coefficient: float,\n",
    "        depth_coefficient: float,\n",
    "        dropout_rate: float = 0.2,\n",
    "        num_classes: int = 1000,\n",
    "    ):\n",
    "        super().__init__()\n",
    "        self.dropout_rate = dropout_rate\n",
    "\n",
    "        def round_filters(filters: int, divisor: int = 8) -> int:\n",
    "            filters *= width_coefficient\n",
    "            new_filters = max(divisor, int(filters + divisor / 2) // divisor * divisor)\n",
    "            if new_filters < 0.9 * filters:\n",
    "                new_filters += divisor\n",
    "            return int(new_filters)\n",
    "\n",
    "        def round_repeats(repeats: int) -> int:\n",
    "            return int(torch.ceil(torch.tensor(depth_coefficient * repeats)).item())\n",
    "\n",
    "        # stem: the initial convolutional layer that processes the input image before it enters the main network architecture\n",
    "        out_channels = round_filters(32)\n",
    "        self.stem = nn.Sequential(nn.Conv2d(3, out_channels, 3, stride=2, padding=1, bias=False), nn.BatchNorm2d(out_channels), Swish())\n",
    "\n",
    "        # Blocks\n",
    "        self.blocks = nn.ModuleList([])\n",
    "        self.blocks.append(MBConvBlock(in_channels=out_channels, out_channels=round_filters(16), expand_ratio=1, kernel_size=3, stride=1))\n",
    "\n",
    "        in_channels = round_filters(16)\n",
    "        block_args = [\n",
    "            # (out_channels, num_repeats, kernel_size, stride, expand_ratio)\n",
    "            (24, 2, 3, 2, 6),\n",
    "            (40, 2, 5, 2, 6),\n",
    "            (80, 3, 3, 2, 6),\n",
    "            (112, 3, 5, 1, 6),\n",
    "            (192, 4, 5, 2, 6),\n",
    "            (320, 1, 3, 1, 6),\n",
    "        ]\n",
    "        for out_channels, num_repeats, kernel_size, stride, expand_ratio in block_args:\n",
    "            out_channels = round_filters(out_channels)\n",
    "            repeats = round_repeats(num_repeats)\n",
    "            for i in range(repeats):\n",
    "                if i == 0:\n",
    "                    self.blocks.append(\n",
    "                        MBConvBlock(\n",
    "                            in_channels=in_channels,\n",
    "                            out_channels=out_channels,\n",
    "                            expand_ratio=expand_ratio,\n",
    "                            kernel_size=kernel_size,\n",
    "                            stride=stride,\n",
    "                        )\n",
    "                    )\n",
    "                else:\n",
    "                    self.blocks.append(\n",
    "                        MBConvBlock(\n",
    "                            in_channels=in_channels,\n",
    "                            out_channels=out_channels,\n",
    "                            expand_ratio=expand_ratio,\n",
    "                            kernel_size=kernel_size,\n",
    "                            stride=1,\n",
    "                        )\n",
    "                    )\n",
    "                in_channels = out_channels\n",
    "\n",
    "        # head: the final layers of the network that process the high-level features extracted by previous layers and produce the final output\n",
    "        out_channels = round_filters(1280)\n",
    "        self.head = nn.Sequential(\n",
    "            nn.Conv2d(in_channels, out_channels, 1, bias=False),\n",
    "            nn.BatchNorm2d(out_channels),\n",
    "            Swish(),\n",
    "            nn.AdaptiveAvgPool2d(1),\n",
    "            nn.Flatten(),\n",
    "            nn.Dropout(dropout_rate),\n",
    "            nn.Linear(out_channels, num_classes),\n",
    "        )\n",
    "\n",
    "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n",
    "        x = self.stem(x)\n",
    "        for block in self.blocks:\n",
    "            x = block(x)\n",
    "        x = self.head(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_1_1_'></a>[EfficientNet-B0](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_1_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b0_1 = CustomEfficientNet(width_coefficient=1.0, depth_coefficient=1.0, dropout_rate=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b0_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_1_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b0_1, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_1_2_'></a>[EfficientNet-B1](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_2_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b1_1 = CustomEfficientNet(width_coefficient=1.0, depth_coefficient=1.1, dropout_rate=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b1_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_2_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b1_1, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_1_3_'></a>[EfficientNet-B2](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_3_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b2_1 = CustomEfficientNet(width_coefficient=1.1, depth_coefficient=1.2, dropout_rate=0.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b2_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_3_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b2_1, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_1_4_'></a>[EfficientNet-B3](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_4_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b3_1 = CustomEfficientNet(width_coefficient=1.2, depth_coefficient=1.4, dropout_rate=0.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b3_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_4_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b3_1, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_1_5_'></a>[EfficientNet-B4](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_5_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b4_1 = CustomEfficientNet(width_coefficient=1.4, depth_coefficient=1.8, dropout_rate=0.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b4_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_5_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b4_1, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_1_6_'></a>[EfficientNet-B5](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_6_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b5_1 = CustomEfficientNet(width_coefficient=1.6, depth_coefficient=2.2, dropout_rate=0.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b5_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_6_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b5_1, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_1_7_'></a>[EfficientNet-B6](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_7_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b6_1 = CustomEfficientNet(width_coefficient=1.8, depth_coefficient=2.6, dropout_rate=0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b6_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_7_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b6_1, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_1_8_'></a>[EfficientNet-B7](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_8_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b7_1 = CustomEfficientNet(width_coefficient=2.0, depth_coefficient=3.1, dropout_rate=0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b7_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_1_8_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b7_1, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id='toc2_2_'></a>[PyTorch EfficientNet](#toc0_)\n",
    "\n",
    "- EfficientNet is available in PyTorch: [pytorch.org/vision/stable/models/efficientnet.html](https://pytorch.org/vision/stable/models/efficientnet.html)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_2_1_'></a>[EfficientNet-B0](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_1_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b0_2 = efficientnet_b0()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b0_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_1_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b0_2, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_2_2_'></a>[EfficientNet-B1](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_2_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b1_2 = efficientnet_b1()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b1_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_2_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b1_2, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_2_3_'></a>[EfficientNet-B2](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_3_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b2_2 = efficientnet_b2()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b2_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_3_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b2_2, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_2_4_'></a>[EfficientNet-B3](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_4_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b3_2 = efficientnet_b3()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b3_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_4_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b3_2, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_2_5_'></a>[EfficientNet-B4](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_5_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b4_2 = efficientnet_b4()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b4_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_5_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b4_2, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_2_6_'></a>[EfficientNet-B5](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_6_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b5_2 = efficientnet_b5()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b5_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_6_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b5_2, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_2_7_'></a>[EfficientNet-B6](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_7_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b6_2 = efficientnet_b6()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b6_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_7_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b6_2, (1, 3, 224, 224), device=\"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id='toc2_2_8_'></a>[EfficientNet-B7](#toc0_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_8_1_'></a>[Initialize the Model](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b7_2 = efficientnet_b7()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "efficientnet_b7_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id='toc2_2_8_2_'></a>[Model Summary](#toc0_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(efficientnet_b7_2, (1, 3, 224, 224), device=\"cpu\")"
   ]
  }
 ],
 "metadata": {
  "author_email": "AmirhosseinHeydari78@gmail.com",
  "author_github": "https://github.com/mr-pylin",
  "author_name": "Amirhossein Heydari",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  },
  "origin_repo": "https://github.com/mr-pylin/pytorch-workshop"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}