model zoo v1.6.0

benchmarks/README.md

@@ -6,7 +6,7 @@ Training and inference scripts with Intel-optimized MKL
 
 The model scripts can be run on Linux and require the following
 dependencies to be installed:
-* [Docker](https://docs.docker.com/install/) also support bare metal run
+* [Docker](https://docs.docker.com/install/)
 * [Python](https://www.python.org/downloads/) 3.5 or later
 * [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
 * `wget` for downloading pre-trained models
@@ -15,29 +15,21 @@ dependencies to be installed:
 
 | Use Case               | Framework     | Model               | Mode      | Instructions    |
 | -----------------------| --------------| ------------------- | --------- |------------------------------|
-| Content Creation       | TensorFlow    | [DRAW](https://arxiv.org/pdf/1502.04623.pdf)               | Inference | [FP32](content_creation/tensorflow/draw/README.md#fp32-inference-instructions) |
-| Face Detection and Alignment | TensorFlow    | [MTCC](https://arxiv.org/pdf/1604.02878.pdf)               | Inference | [FP32](face_detection_and_alignment/tensorflow/mtcc/README.md#fp32-inference-instructions) |
 | Image Recognition      | TensorFlow    | [DenseNet169](https://arxiv.org/pdf/1608.06993.pdf)         | Inference | [FP32](image_recognition/tensorflow/densenet169/README.md#fp32-inference-instructions) |
-| Image Recognition      | TensorFlow    | [Inception ResNet V2](https://arxiv.org/pdf/1602.07261.pdf) | Inference | [Int8](image_recognition/tensorflow/inception_resnet_v2/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/inception_resnet_v2/README.md#fp32-inference-instructions) |
 | Image Recognition      | TensorFlow    | [Inception V3](https://arxiv.org/pdf/1512.00567.pdf)        | Inference | [Int8](image_recognition/tensorflow/inceptionv3/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/inceptionv3/README.md#fp32-inference-instructions) |
 | Image Recognition      | TensorFlow    | [Inception V4](https://arxiv.org/pdf/1602.07261.pdf)        | Inference | [Int8](image_recognition/tensorflow/inceptionv4/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/inceptionv4/README.md#fp32-inference-instructions) |
 | Image Recognition      | TensorFlow    | [MobileNet V1*](https://arxiv.org/pdf/1704.04861.pdf)        | Inference | [Int8](image_recognition/tensorflow/mobilenet_v1/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/mobilenet_v1/README.md#fp32-inference-instructions) |
 | Image Recognition      | TensorFlow    | [ResNet 101](https://arxiv.org/pdf/1512.03385.pdf)          | Inference | [Int8](image_recognition/tensorflow/resnet101/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/resnet101/README.md#fp32-inference-instructions) |
 | Image Recognition      | TensorFlow    | [ResNet 50](https://arxiv.org/pdf/1512.03385.pdf)           | Inference | [Int8](image_recognition/tensorflow/resnet50/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/resnet50/README.md#fp32-inference-instructions) |
 | Image Recognition      | TensorFlow    | [ResNet 50v1.5*](https://github.com/tensorflow/models/tree/master/official/resnet) | Inference | [Int8](image_recognition/tensorflow/resnet50v1_5/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/resnet50v1_5/README.md#fp32-inference-instructions) |
-| Image Segmentation     | TensorFlow    | [Mask R-CNN](https://arxiv.org/pdf/1703.06870.pdf)          | Inference | [FP32](image_segmentation/tensorflow/maskrcnn/README.md#fp32-inference-instructions) |
-| Language Modeling      | TensorFlow    | [LM-1B](https://arxiv.org/pdf/1602.02410.pdf)               | Inference | [FP32](language_modeling/tensorflow/lm-1b/README.md#fp32-inference-instructions) |
-| Language Translation   | TensorFlow    | [GNMT](https://arxiv.org/pdf/1609.08144.pdf)                | Inference | [FP32](language_translation/tensorflow/gnmt/README.md#fp32-inference-instructions) |
-| Language Translation   | TensorFlow    | [GNMT](https://arxiv.org/pdf/1609.08144.pdf)                | Training  | [FP32](language_translation/tensorflow/gnmt/README.md#fp32-training-instructions) |
+| Image Recognition      | TensorFlow    | [ResNet 50v1.5*](https://github.com/tensorflow/models/tree/master/official/resnet) | Training | [FP32](image_recognition/tensorflow/resnet50v1_5/README.md#fp32-training-instructions) |
+| Reinforcement          | TensorFlow    | [MiniGo](https://arxiv.org/abs/1712.01815.pdf)              | Training  | [FP32](reinforcement/tensorflow/minigo/README.md#fp32-training-instructions)|
+| Language Translation   | TensorFlow    | [GNMT*](https://arxiv.org/pdf/1609.08144.pdf)                | Inference | [FP32](language_translation/tensorflow/mlperf_gnmt/README.md#fp32-inference-instructions) |
+| Language Translation   | TensorFlow    | [Transformer_LT_Official ](https://arxiv.org/pdf/1706.03762.pdf)| Inference | [FP32](language_translation/tensorflow/transformer_lt_official/README.md#fp32-inference-instructions) |
 | Object Detection       | TensorFlow    | [R-FCN](https://arxiv.org/pdf/1605.06409.pdf)               | Inference | [Int8](object_detection/tensorflow/rfcn/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/rfcn/README.md#fp32-inference-instructions) |
-| Object Detection       | TensorFlow    | [Faster R-CNN](https://arxiv.org/pdf/1506.01497.pdf)        | Inference | [Int8](object_detection/tensorflow/faster_rcnn/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/faster_rcnn/README.md#fp32-inference-instructions) |
 | Object Detection       | TensorFlow    | [SSD-MobileNet*](https://arxiv.org/pdf/1704.04861.pdf)       | Inference | [Int8](object_detection/tensorflow/ssd-mobilenet/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd-mobilenet/README.md#fp32-inference-instructions) |
 | Object Detection       | TensorFlow    | [SSD-ResNet34*](https://arxiv.org/pdf/1512.02325.pdf)        | Inference | [Int8](object_detection/tensorflow/ssd-resnet34/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd-resnet34/README.md#fp32-inference-instructions) |
-| Object Detection       | TensorFlow    | [SSD-ResNet34*](https://arxiv.org/pdf/1512.02325.pdf)        | Training  | [FP32](object_detection/tensorflow/ssd-resnet34/README.md#fp32-training-instructions) |
-| Object Detection       | TensorFlow    | [SSD-VGG16](https://arxiv.org/pdf/1512.02325.pdf)           | Inference | [Int8](object_detection/tensorflow/ssd_vgg16/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd_vgg16/README.md#fp32-inference-instructions) |
-| Recommendation         | TensorFlow    | [NCF](https://arxiv.org/pdf/1708.05031.pdf)                 | Inference | [FP32](recommendation/tensorflow/ncf/README.md#fp32-inference-instructions) |
 | Recommendation         | TensorFlow    | [Wide & Deep Large Dataset](https://arxiv.org/pdf/1606.07792.pdf)	| Inference | [Int8](recommendation/tensorflow/wide_deep_large_ds/README.md#int8-inference-instructions) [FP32](recommendation/tensorflow/wide_deep_large_ds/README.md#fp32-inference-instructions) |
-| Recommendation         | TensorFlow    | [Wide & Deep Large Dataset](https://arxiv.org/pdf/1606.07792.pdf)	| Training | [FP32](recommendation/tensorflow/wide_deep_large_ds/README.md#fp32-training-instructions) |
 | Recommendation         | TensorFlow    | [Wide & Deep](https://arxiv.org/pdf/1606.07792.pdf)         | Inference | [FP32](recommendation/tensorflow/wide_deep/README.md#fp32-inference-instructions) |
 
 *Means the model is belong to [MLPerf](https://mlperf.org/) models, will long term support. 

benchmarks/common/base_benchmark_util.py

@@ -1,7 +1,7 @@
 #
 # -*- coding: utf-8 -*-
 #
-# Copyright (c) 2018 Intel Corporation
+# Copyright (c) 2018-2019 Intel Corporation
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -67,8 +67,8 @@ def _define_args(self):
 
         self._common_arg_parser.add_argument(
             "-p", "--precision",
-            help="Specify the model precision to use: fp32, int8, or bfloat16",
-            required=required_arg, choices=["fp32", "int8", "bfloat16"],
+            help="Specify the model precision to use: fp32, int8",
+            required=required_arg, choices=["fp32", "int8"],
             dest="precision")
 
         self._common_arg_parser.add_argument(
@@ -87,6 +87,16 @@ def _define_args(self):
             dest="batch_size", default=-1,
             type=check_positive_number_or_equal_to_negative_one)
 
+        self._common_arg_parser.add_argument(
+            "--mpi_num_processes", type=check_positive_number,
+            help="The number of MPI processes",
+            dest="mpi", default=None)
+
+        self._common_arg_parser.add_argument(
+            "--mpi_num_processes_per_socket", type=check_positive_number,
+            help="Specify how many MPI processes to launch per socket",
+            dest="num_mpi", default=1)
+
         self._common_arg_parser.add_argument(
             "-d", "--data-location",
             help="Specify the location of the data. If this parameter is not "
@@ -107,6 +117,11 @@ def _define_args(self):
                  " specified or is -1, all cores will be used.",
             dest="num_cores", type=int, default=-1)
 
+        self._common_arg_parser.add_argument(
+            "--num-instances", type=check_positive_number,
+            help="Specify the number of instances to run.",
+            dest="num_instances", default=1)
+
         self._common_arg_parser.add_argument(
             "-a", "--num-intra-threads", type=check_positive_number,
             help="Specify the number of threads within the layer",
@@ -117,23 +132,6 @@ def _define_args(self):
             help="Specify the number threads between layers",
             dest="num_inter_threads", default=None)
 
-        self._common_arg_parser.add_argument(
-            "-np", "--num-processes", type=check_positive_number,
-            help="Specify the number of processes to run on as mpirun '-np' "
-                 "input for multi-instance execution. ",
-            dest="num_processes", default=1)
-
-        self._common_arg_parser.add_argument(
-            "-ppn", "--num-processes-per-node", type=check_positive_number,
-            help="Specify the number of processes per node as mpirun '-ppn' "
-                 "input for multi-instance execution. ",
-            dest="num_processes_per_node", default=1)
-
-        self._common_arg_parser.add_argument(
-            "-ts", "--num-train-steps", type=check_positive_number,
-            help="Specify the number of training steps ",
-            dest="num_train_steps", default=1)
-
         self._common_arg_parser.add_argument(
             "--data-num-intra-threads", type=check_positive_number,
             help="The number intra op threads for the data layer config",
@@ -158,6 +156,7 @@ def _define_args(self):
             "-g", "--in-graph", help="Full path to the input graph ",
             dest="input_graph", default=None, type=check_valid_filename)
 
+
         self._common_arg_parser.add_argument(
             "-k", "--benchmark-only",
             help="For benchmark measurement only. If neither --benchmark-only "
@@ -221,11 +220,11 @@ def _define_args(self):
     def _validate_args(self):
         """validate the args and initializes platform_util"""
         # check if socket id is in socket number range
-        num_numas = self._platform_util.num_numa_nodes
+        num_sockets = self._platform_util.num_cpu_sockets
         args = self.args
-        if not -1 <= args.socket_id < num_numas:
-            raise ValueError("Socket id must be within NUMA number range: "
-                             "[0, {}].".format(num_numas - 1))
+        if not -1 <= args.socket_id < num_sockets:
+            raise ValueError("Socket id must be within socket number range: "
+                             "[0, {}].".format(num_sockets - 1))
 
         # check number of cores
         num_logical_cores_per_socket = \

benchmarks/common/base_model_init.py

@@ -57,6 +57,18 @@ def __init__(self, args, custom_args=[], platform_util=None):
         if not platform_util:
             raise ValueError("Did not find any platform info.")
 
+        # Invoke mpirun if mpi_num_processes env is not None
+        if os.environ["MPI_NUM_PROCESSES"] != "None":
+            if os.environ["MPI_NUM_PROCESSES_PER_SOCKET"] == "1":
+              # Map by socket using OpenMPI by default (PPS=1).
+              self.python_exe = "mpirun --allow-run-as-root -n " + os.environ["MPI_NUM_PROCESSES"] + " --map-by socket " + self.python_exe
+            else:
+              # number of processes per socket (pps)
+              pps = int(os.environ["MPI_NUM_PROCESSES_PER_SOCKET"])
+              split_a_socket = str(platform_util.num_cores_per_socket // pps)
+              # Launch pps MPI processes over one socket
+              self.python_exe = "mpirun --allow-run-as-root -n " + os.environ["MPI_NUM_PROCESSES"] + " --map-by ppr:" + str(pps) + ":socket:pe=" + split_a_socket + " --cpus-per-proc " + split_a_socket + " " + self.python_exe
+
     def run_command(self, cmd):
         """
         Prints debug messages when verbose is enabled, and then runs the
@@ -75,6 +87,8 @@ def get_command_prefix(self, socket_id, numactl=True):
         Returns the command prefix with:
          - LD_PRELOAD for int8 models (if tcmalloc is not disabled)
          - The numactl command with --cpunodebind and --membind set to the specified socket_id (if numactl=True)
+
+        Should be used only for single instance.
         """
         command = ""
 
@@ -97,36 +111,6 @@ def get_command_prefix(self, socket_id, numactl=True):
 
         return command
 
-    def get_multi_instance_train_prefix(self, option_list=None):
-        """
-        Returns the multi-instance train command prefix with:
-         - Define the number of processes.
-         - Define the processes per each node.
-         - Set the other parameters with the format of parameter list "option:value", such as environment variable
-           "-genv:I_MPI_ASYNC_PROGRESS=1".
-        """
-        command = "mpirun "
-        if self.args.num_processes:
-            if self.args.num_processes > 0:
-                command += "-n {} ".format(self.args.num_processes)
-            else:
-                print("Warning: {} is not a valid value.".format(self.args.num_processes))
-
-        if self.args.num_processes_per_node:
-            if self.args.num_processes_per_node > 0:
-                command += "-ppn {} ".format(self.args.num_processes_per_node)
-            else:
-                print("Warning: {} is not a valid value.".format(self.args.num_processes_per_node))
-
-        if option_list:
-            for item in option_list:
-                if item.count(':') != 1:
-                    print("Warning: {} does not follow the option_list definition.".format(item))
-                else:
-                    option, value = item.split(':')
-                    command += "{} {} ".format(option, value)
-        return command
-
     def add_args_to_command(self, command, arg_list):
         """
         Add args that are specified in the arg list to the command.  batch_size

benchmarks/common/platform_util.py

@@ -23,9 +23,9 @@
 from __future__ import print_function
 
 import os
+import re
 import platform as system_platform
 import subprocess
-import shlex
 import sys
 
 NUMA_NODES_STR_ = "NUMA node(s)"
@@ -35,6 +35,145 @@
 LOGICAL_CPUS_STR_ = "CPU(s)"
 
 
+class CPUInfo():
+    """CPU information class."""
+
+    def __init__(self):
+        """Initialize CPU information class."""
+        self._binding_data = CPUInfo._sort_membind_info(self._get_core_membind_info())
+
+    @staticmethod
+    def _get_core_membind_info():
+        """
+        Return sorted information about cores and memory binding.
+        E.g.
+        CPU ID, Socket ID, Node ID, HT CPU ID,
+        0  ,     0    ,    0   ,     0
+        1  ,     0    ,    0   ,     1
+        :return: list with cpu, sockets, ht core and memory binding information
+        :rtype: List[List[str, Any]]
+        """
+        args = ["lscpu", "--parse=CPU,Core,Socket,Node"]
+        process_lscpu = subprocess.check_output(args, universal_newlines=True).split("\n")
+
+        # Get information about core, node, socket and cpu
+        bind_info = []
+        for line in process_lscpu:
+            pattern = r"^([\d]+,[\d]+,[\d]+,[\d]+)"
+            regex_out = re.search(pattern, line)
+            if regex_out:
+                bind_info.append(regex_out.group(1).strip().split(","))
+
+        return bind_info
+
+    @staticmethod
+    def _sort_membind_info(membind_bind_info):
+        """
+        Sore membind info data.
+        :param membind_bind_info: raw membind info data
+        :type membind_bind_info: List[List[str]]
+        :return: sorted membind info
+        :rtype: List[List[Dict[str, int]]]
+        """
+        membind_cpu_list = []
+        nodes_count = int(max(element[2] for element in membind_bind_info)) + 1
+        # Sort list by Node id
+        for node_number in range(nodes_count):
+            node_core_list = []
+            core_info = {}
+            for entry in membind_bind_info:
+                cpu_id = int(entry[0])
+                core_id = int(entry[1])
+                node_id = int(entry[2])
+                socket_id = int(entry[3])
+
+                # Skip nodes other than current node number
+                if node_number != node_id:
+                    continue
+
+                # Add core info
+                if cpu_id == core_id:
+                    core_info.update({
+                        core_id: {
+                            "cpu_id": cpu_id,
+                            "node_id": node_id,
+                            "socket_id": socket_id,
+                        },
+                    })
+                else:
+                    # Add information about Hyper Threading
+                    core_info[core_id]["ht_cpu_id"] = cpu_id
+
+            # Change dict of dicts to list of dicts
+            for iterator in range(len(core_info)):
+                curr_core_id = len(core_info) * node_number + iterator
+                single_core_info = core_info.get(curr_core_id)
+                if single_core_info:
+                    node_core_list.append(single_core_info)
+
+            membind_cpu_list.append(node_core_list)
+
+        return membind_cpu_list
+
+    @property
+    def sockets(self):
+        """
+        Return count of sockets available on server.
+        :return: available cores
+        :rtype: int
+        """
+        available_sockets = len(self._binding_data)
+        return int(available_sockets)
+
+    @property
+    def cores(self):
+        """
+        Return amount of cores available on server.
+        :return: amount of cores
+        :rtype: int
+        """
+        available_cores = self.cores_per_socket * self.sockets
+        return int(available_cores)  # type: ignore
+
+    @property
+    def cores_per_socket(self):
+        """
+        Return amount of available cores per socket.
+        :return: amount of cores
+        :rtype: int
+        """
+        available_cores_per_socket = len(self._binding_data[0])
+        return available_cores_per_socket
+
+    @property
+    def binding_information(self):
+        """
+        Return information about cores and memory binding.
+        Format:
+        [
+            [ # socket 0
+                { # Core 0
+                    "cpu_id": 0,
+                    "node_id": 0,
+                    "socket_id": 0,
+                    "ht_cpu_id": 56
+                }
+            ],
+            [ # socket 1
+                { # Core 0
+                    "cpu_id": 28,
+                    "node_id": 1,
+                    "socket_id": 1,
+                    "ht_cpu_id": 84
+                }
+            ]
+        ]
+        :return: dict with cpu, sockets, ht core and memory binding information
+        :rtype: List[List[Dict[str, int]]]
+        """
+        return self._binding_data
+
+
 class PlatformUtil:
     '''
     This module implements a platform utility that exposes functions that