您好，请问训练过后如何在benchmark数据库上检测训练结果的准确率？

[lilmangolil]

如题，请问是否是运行/tools/eval.py? 请问第297行fr1 = open('predict_dict.pkl', 'r') 与第298行fr2 = open('gtboxes_dict.pkl', 'r')，这两个pkl文件是指什么呢？如何生成这两个文件？望您回复，感谢！

[yanghedada]

这两个文件就是用eva.py生成的文件。你可以查看 https://www.jianshu.com/p/6e5f7925f207

[lilmangolil]

这两个文件就是用eva.py生成的文件。你可以查看 https://www.jianshu.com/p/6e5f7925f207

谢谢您的分享~暂时还没运行出结果

[yangxue0827]

不建议用这个代码的评估函数，是有问题的，请参考https://github.com/DetectionTeamUCAS/Faster-RCNN_Tensorflow/blob/master/tools/eval.py @lilmangolil @yanghedada

[yanghedada]

谢谢提醒，感谢分享@yangxue0827
我遇到一个新的问题：改了评分的函数之后，这里的分数就变了。
这是新版的评分：

这是旧版的评分：

这是faster rcnn输出的坐标，这里有问题吗？？：

  ymin, xmin, ymax, xmax = _fast_rcnn_decode_boxes[:, 0], _fast_rcnn_decode_boxes[:, 1], \
                                 _fast_rcnn_decode_boxes[:, 2], _fast_rcnn_decode_boxes[:, 3]

修改之后的代码如下：

#!/usr/bin/python
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function

import argparse
import os
import pickle
import sys
import time
import io
import numpy as np
import tensorflow as tf

sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='iso-8859-1')
from data.io.image_preprocess import short_side_resize_for_inference_data
from data.io.read_tfrecord import next_batch
from help_utils.tools import *
from libs.fast_rcnn import build_fast_rcnn
from libs.label_name_dict.label_dict import *
from libs.networks.network_factory import get_flags_byname, get_network_byname
from libs.rpn import build_rpn
from libs.val_libs import voc_eval
from tools import restore_model
import numpy as np
import cv2
FLAGS = get_flags_byname(cfgs.NET_NAME)

def eval_dict_convert(args):
  with tf.Graph().as_default():
    img_name_batch, img_batch, gtboxes_and_label_batch, num_objects_batch = \
        next_batch(dataset_name=cfgs.DATASET_NAME,
                   batch_size=cfgs.BATCH_SIZE,
                   shortside_len=cfgs.SHORT_SIDE_LEN,
                   is_training=False)

    # ***********************************************************************************************
    # *                                         share net                                           *
    # ***********************************************************************************************
    _, share_net = get_network_byname(net_name=cfgs.NET_NAME,
                                      inputs=img_batch,
                                      num_classes=None,
                                      is_training=True,
                                      output_stride=None,
                                      global_pool=False,
                                      spatial_squeeze=False)

    # ***********************************************************************************************
    # *                                            RPN                                              *
    # ***********************************************************************************************
    rpn = build_rpn.RPN(net_name=cfgs.NET_NAME,
                        inputs=img_batch,
                        gtboxes_and_label=None,
                        is_training=False,
                        share_head=True,
                        share_net=share_net,
                        stride=cfgs.STRIDE,
                        anchor_ratios=cfgs.ANCHOR_RATIOS,
                        anchor_scales=cfgs.ANCHOR_SCALES,
                        scale_factors=cfgs.SCALE_FACTORS,
                        base_anchor_size_list=cfgs.BASE_ANCHOR_SIZE_LIST,  # P2, P3, P4, P5, P6
                        level=cfgs.LEVEL,
                        top_k_nms=cfgs.RPN_TOP_K_NMS,
                        rpn_nms_iou_threshold=cfgs.RPN_NMS_IOU_THRESHOLD,
                        max_proposals_num=cfgs.MAX_PROPOSAL_NUM,
                        rpn_iou_positive_threshold=cfgs.RPN_IOU_POSITIVE_THRESHOLD,
                        rpn_iou_negative_threshold=cfgs.RPN_IOU_NEGATIVE_THRESHOLD,
                        rpn_mini_batch_size=cfgs.RPN_MINIBATCH_SIZE,
                        rpn_positives_ratio=cfgs.RPN_POSITIVE_RATE,
                        remove_outside_anchors=False,  # whether remove anchors outside
                        rpn_weight_decay=cfgs.WEIGHT_DECAY[cfgs.NET_NAME])

    # rpn predict proposals
    rpn_proposals_boxes, rpn_proposals_scores = rpn.rpn_proposals()  # rpn_score shape: [300, ]

    # ***********************************************************************************************
    # *                                         Fast RCNN                                           *
    # ***********************************************************************************************
    fast_rcnn = build_fast_rcnn.FastRCNN(img_batch=img_batch,
                                         feature_pyramid=rpn.feature_pyramid,
                                         rpn_proposals_boxes=rpn_proposals_boxes,
                                         rpn_proposals_scores=rpn_proposals_scores,
                                         img_shape=tf.shape(img_batch),
                                         roi_size=cfgs.ROI_SIZE,
                                         scale_factors=cfgs.SCALE_FACTORS,
                                         roi_pool_kernel_size=cfgs.ROI_POOL_KERNEL_SIZE,
                                         gtboxes_and_label=None,
                                         fast_rcnn_nms_iou_threshold=cfgs.FAST_RCNN_NMS_IOU_THRESHOLD,
                                         fast_rcnn_maximum_boxes_per_img=100,
                                         fast_rcnn_nms_max_boxes_per_class=cfgs.FAST_RCNN_NMS_MAX_BOXES_PER_CLASS,
                                         show_detections_score_threshold=cfgs.FINAL_SCORE_THRESHOLD,  # show detections which score >= 0.6
                                         num_classes=cfgs.CLASS_NUM,
                                         fast_rcnn_minibatch_size=cfgs.FAST_RCNN_MINIBATCH_SIZE,
                                         fast_rcnn_positives_ratio=cfgs.FAST_RCNN_POSITIVE_RATE,
                                         fast_rcnn_positives_iou_threshold=cfgs.FAST_RCNN_IOU_POSITIVE_THRESHOLD,
                                         use_dropout=False,
                                         weight_decay=cfgs.WEIGHT_DECAY[cfgs.NET_NAME],
                                         is_training=False,
                                         level=cfgs.LEVEL)

    fast_rcnn_decode_boxes, fast_rcnn_score, num_of_objects, detection_category = \
        fast_rcnn.fast_rcnn_predict()

    # train
    init_op = tf.group(
        tf.global_variables_initializer(),
        tf.local_variables_initializer()
    )

    restorer, restore_ckpt = restore_model.get_restorer(checkpoint_path=args.weights)

    config = tf.ConfigProto()
    # config.gpu_options.per_process_gpu_memory_fraction = 0.5
    config.gpu_options.allow_growth = True
    with tf.Session(config=config) as sess:
      sess.run(init_op)
      if not restorer is None:
        restorer.restore(sess, restore_ckpt)
        print('restore model')

      coord = tf.train.Coordinator()
      threads = tf.train.start_queue_runners(sess, coord)
      img_name_batchs  = []
      all_boxes = []
      for i in range(args.img_num):

        start = time.time()

        _img_name_batch, _img_batch, _gtboxes_and_label_batch, _fast_rcnn_decode_boxes, \
            _fast_rcnn_score, _detection_category \
            = sess.run([img_name_batch, img_batch, gtboxes_and_label_batch, fast_rcnn_decode_boxes,
                        fast_rcnn_score, detection_category])
        end = time.time()
        ymin, xmin, ymax, xmax = _fast_rcnn_decode_boxes[:, 0], _fast_rcnn_decode_boxes[:, 1], \
                                 _fast_rcnn_decode_boxes[:, 2], _fast_rcnn_decode_boxes[:, 3]

        boxes = np.transpose(np.stack([xmin, ymin, xmax, ymax]))
        dets = np.hstack((_detection_category.reshape(-1, 1),
                          _fast_rcnn_score.reshape(-1, 1),
                          boxes))
        img_name_batchs.append(_img_name_batch[:1][0])
        all_boxes.append(dets)
        view_bar('{} image cost {}s'.format(
            str(_img_name_batch[0]), (end - start)), i + 1, args.img_num)
      fw1 = open( 'detections.pkl', 'wb')
      pickle.dump(all_boxes, fw1)
      fw1.close()
      coord.request_stop()
      coord.join(threads)

  return img_name_batchs

def eval(args):
  print('Called with args:')
  img_name_batchs = eval_dict_convert(args)
  test_imgname_list = [img.decode('utf-8','ignore') for img in img_name_batchs]
  with open('detections.pkl','rb') as f:
        all_boxes = pickle.load(f,encoding='iso-8859-1')
  voc_eval.voc_evaluate_detections(all_boxes=all_boxes,
                                     test_annotation_path=args.annotation_dir,
                                     test_imgid_list=test_imgname_list)

def parse_args():
  """
  Parse input arguments
  """
  parser = argparse.ArgumentParser(description='Evaluate a trained FPN model')
  parser.add_argument('--eval_imgs', dest='eval_imgs',
                        help='evaluate imgs dir ',
                        default='../data/io/VOCdevkit_test/JPEGImages', type=str)
  parser.add_argument('--annotation_dir', dest='annotation_dir',
                    help='the dir save annotations',
                    default='../data/io/VOCdevkit_test/Annotations', type=str)
  parser.add_argument('--weights', dest='weights',
                      help='model path',
                      default='../output/airplane/res101_trained_weights/v3_airplane/airplane_115500model.ckpt',
                      type=str)
  parser.add_argument('--img_num', dest='img_num',
                      help='image numbers',
                      default=100, type=int)

  args = parser.parse_args()
  return args


if __name__ == "__main__":
    args = parse_args()
    eval(args)

[yangxue0827]

你这个代码有问题，首先用的是训练集在评估，其次图片是经过缩放的，检测结果需要映射到原图才行。 @yanghedada

[yangxue0827]

fpn的框存放形式就是ymin xmin ymax xmax ，所以你这里是对的 @yanghedada

[yanghedada]

谢谢指点，感谢你分享。@yangxue0827

在进行评估的时候is_training=False应该就是test集数据了。

我上次修改的时候忽略了图片的预处理。现在修改如下：

        _img_name_batch, _img_batch, _gtboxes_and_label_batch, _fast_rcnn_decode_boxes, \
            _fast_rcnn_score, _detection_category \
            = sess.run([img_name_batch, img_batch, gtboxes_and_label_batch, fast_rcnn_decode_boxes,
                        fast_rcnn_score, detection_category])
        end = time.time()
        raw_img = cv2.imread(os.path.join(args.eval_imgs, _img_name_batch[0].decode('utf-8','ignore')))
        raw_h, raw_w = raw_img.shape[0], raw_img.shape[1]

        ymin, xmin, ymax, xmax = _fast_rcnn_decode_boxes[:, 0], _fast_rcnn_decode_boxes[:, 1], \
                                 _fast_rcnn_decode_boxes[:, 2], _fast_rcnn_decode_boxes[:, 3]

        resized_h, resized_w = _img_batch.shape[1], _img_batch.shape[2]

        xmin = xmin * raw_w / resized_w
        xmax = xmax * raw_w / resized_w
        ymin = ymin * raw_h / resized_h
        ymax = ymax * raw_h / resized_h
        boxes = np.transpose(np.stack([xmin, ymin, xmax, ymax]))
        dets = np.hstack((_detection_category.reshape(-1, 1),
                          _fast_rcnn_score.reshape(-1, 1),
                          boxes))

[yangxue0827]

所以结果是多少了呢 @yanghedada

[yanghedada]

谢谢作者指导，感谢作者提供这么流畅的代码。@yangxue0827
我的训练结果有点雷人。可能是我得训练方式不对。
我只针对airplane进行训练，但是这里的结果不是很优美。如下：
我在voc_eval.py添加了一个参数FAST_RCNN_IOU_MAP。

当FAST_RCNN_IOU_MAP=.01，会给一个安慰结果。

当FAST_RCNN_IOU_MAP=.5，无语了

预测图：

我的config：

这里会不会还有其他问题？：

#!/usr/bin/python
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function

import argparse
import os
import pickle
import sys
import time
import io
import numpy as np
import tensorflow as tf

sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='iso-8859-1')
from data.io.image_preprocess import short_side_resize_for_inference_data
from data.io.read_tfrecord import next_batch
from data.io.divide_data import mkdir
from help_utils.tools import *
from libs.fast_rcnn import build_fast_rcnn
from libs.label_name_dict.label_dict import *
from libs.box_utils import draw_box_in_img
from libs.networks.network_factory import get_flags_byname, get_network_byname
from libs.rpn import build_rpn
from libs.val_libs import voc_eval
from tools import restore_model
import numpy as np
import cv2
FLAGS = get_flags_byname(cfgs.NET_NAME)

def eval_dict_convert(args, draw_imgs=True):
  with tf.Graph().as_default():
    '''
    这里就是读取图片的操作，
    '''
    img_name_batch, img_batch, gtboxes_and_label_batch, num_objects_batch = \
        next_batch(dataset_name=cfgs.DATASET_NAME,
                   batch_size=cfgs.BATCH_SIZE,
                   shortside_len=cfgs.SHORT_SIDE_LEN,
                   is_training=False)

    # ***********************************************************************************************
    # *                                         share net                                           *
    # ***********************************************************************************************
    _, share_net = get_network_byname(net_name=cfgs.NET_NAME,
                                      inputs=img_batch,
                                      num_classes=None,
                                      is_training=True,
                                      output_stride=None,
                                      global_pool=False,
                                      spatial_squeeze=False)

    # ***********************************************************************************************
    # *                                            RPN                                              *
    # ***********************************************************************************************
    rpn = build_rpn.RPN(net_name=cfgs.NET_NAME,
                        inputs=img_batch,
                        gtboxes_and_label=None,
                        is_training=False,
                        share_head=True,
                        share_net=share_net,
                        stride=cfgs.STRIDE,
                        anchor_ratios=cfgs.ANCHOR_RATIOS,
                        anchor_scales=cfgs.ANCHOR_SCALES,
                        scale_factors=cfgs.SCALE_FACTORS,
                        base_anchor_size_list=cfgs.BASE_ANCHOR_SIZE_LIST,  # P2, P3, P4, P5, P6
                        level=cfgs.LEVEL,
                        top_k_nms=cfgs.RPN_TOP_K_NMS,
                        rpn_nms_iou_threshold=cfgs.RPN_NMS_IOU_THRESHOLD,
                        max_proposals_num=cfgs.MAX_PROPOSAL_NUM,
                        rpn_iou_positive_threshold=cfgs.RPN_IOU_POSITIVE_THRESHOLD,
                        rpn_iou_negative_threshold=cfgs.RPN_IOU_NEGATIVE_THRESHOLD,
                        rpn_mini_batch_size=cfgs.RPN_MINIBATCH_SIZE,
                        rpn_positives_ratio=cfgs.RPN_POSITIVE_RATE,
                        remove_outside_anchors=False,  # whether remove anchors outside
                        rpn_weight_decay=cfgs.WEIGHT_DECAY[cfgs.NET_NAME])

    # rpn predict proposals
    rpn_proposals_boxes, rpn_proposals_scores = rpn.rpn_proposals()  # rpn_score shape: [300, ]

    # ***********************************************************************************************
    # *                                         Fast RCNN                                           *
    # ***********************************************************************************************
    fast_rcnn = build_fast_rcnn.FastRCNN(img_batch=img_batch,
                                         feature_pyramid=rpn.feature_pyramid,
                                         rpn_proposals_boxes=rpn_proposals_boxes,
                                         rpn_proposals_scores=rpn_proposals_scores,
                                         img_shape=tf.shape(img_batch),
                                         roi_size=cfgs.ROI_SIZE,
                                         scale_factors=cfgs.SCALE_FACTORS,
                                         roi_pool_kernel_size=cfgs.ROI_POOL_KERNEL_SIZE,
                                         gtboxes_and_label=None,
                                         fast_rcnn_nms_iou_threshold=cfgs.FAST_RCNN_NMS_IOU_THRESHOLD,
                                         fast_rcnn_maximum_boxes_per_img=100,
                                         fast_rcnn_nms_max_boxes_per_class=cfgs.FAST_RCNN_NMS_MAX_BOXES_PER_CLASS,
                                         show_detections_score_threshold=cfgs.FINAL_SCORE_THRESHOLD,  # show detections which score >= 0.6
                                         num_classes=cfgs.CLASS_NUM,
                                         fast_rcnn_minibatch_size=cfgs.FAST_RCNN_MINIBATCH_SIZE,
                                         fast_rcnn_positives_ratio=cfgs.FAST_RCNN_POSITIVE_RATE,
                                         fast_rcnn_positives_iou_threshold=cfgs.FAST_RCNN_IOU_POSITIVE_THRESHOLD,
                                         use_dropout=False,
                                         weight_decay=cfgs.WEIGHT_DECAY[cfgs.NET_NAME],
                                         is_training=False,
                                         level=cfgs.LEVEL)

    fast_rcnn_decode_boxes, fast_rcnn_score, num_of_objects, detection_category = \
        fast_rcnn.fast_rcnn_predict()

    # train
    init_op = tf.group(
        tf.global_variables_initializer(),
        tf.local_variables_initializer()
    )

    restorer, restore_ckpt = restore_model.get_restorer(checkpoint_path=args.weights)

    config = tf.ConfigProto()
    # config.gpu_options.per_process_gpu_memory_fraction = 0.5
    config.gpu_options.allow_growth = True
    with tf.Session(config=config) as sess:
      sess.run(init_op)
      if not restorer is None:
        restorer.restore(sess, restore_ckpt)
        print('restore model')

      coord = tf.train.Coordinator()
      threads = tf.train.start_queue_runners(sess, coord)
      img_name_batchs  = []
      all_boxes = []
      for i in range(args.img_num):

        start = time.time()

        _img_name_batch, _img_batch, _gtboxes_and_label_batch, _fast_rcnn_decode_boxes, \
            _fast_rcnn_score, _detection_category \
            = sess.run([img_name_batch, img_batch, gtboxes_and_label_batch, fast_rcnn_decode_boxes,
                        fast_rcnn_score, detection_category])
        end = time.time()
        raw_img = cv2.imread(os.path.join(args.eval_imgs, _img_name_batch[0].decode('utf-8','ignore')))
        raw_h, raw_w = raw_img.shape[0], raw_img.shape[1]

        ymin, xmin, ymax, xmax = _fast_rcnn_decode_boxes[:, 0], _fast_rcnn_decode_boxes[:, 1], \
                                 _fast_rcnn_decode_boxes[:, 2], _fast_rcnn_decode_boxes[:, 3]

        resized_h, resized_w = _img_batch.shape[1], _img_batch.shape[2]

        xmin = xmin * raw_w / resized_w
        xmax = xmax * raw_w / resized_w
        ymin = ymin * raw_h / resized_h
        ymax = ymax * raw_h / resized_h

        if  len(_detection_category) != 0:
            if draw_imgs:
                show_indices = _fast_rcnn_score >= cfgs.FINAL_SCORE_THRESHOLD
                show_scores = _fast_rcnn_score[show_indices]
                show_boxes = _fast_rcnn_decode_boxes[show_indices]
                show_categories = _detection_category[show_indices]

                pre_ymin, pre_xmin, pre_ymax, pre_xmax = _fast_rcnn_decode_boxes[:, 0], _fast_rcnn_decode_boxes[:, 1], \
                                         _fast_rcnn_decode_boxes[:, 2], _fast_rcnn_decode_boxes[:, 3]

                pre_box = np.stack([pre_xmin, pre_ymin, pre_xmax, pre_ymax], axis=1)
                final_detections = draw_box_in_img.draw_boxes_with_label_and_scores(np.squeeze(_img_batch, 0),
                                                                                    boxes=pre_box,
                                                                                    labels=show_categories,
                                                                                    scores=show_scores)

                gt_ymin, gt_xmin, gt_ymax, gt_xmax = _gtboxes_and_label_batch[0][:, 0], _gtboxes_and_label_batch[0][:, 1], \
                                         _gtboxes_and_label_batch[0][:, 2], _gtboxes_and_label_batch[0][:, 3],
                gt_box = np.stack([gt_xmin, gt_ymin, gt_xmax, gt_ymax], axis=1)
                final_detectionsgt = draw_box_in_img.draw_boxes_with_label_and_scores(np.squeeze(_img_batch, 0),
                                                                                    boxes=gt_box,
                                                                                    labels=_gtboxes_and_label_batch[0][:,-1],
                                                                                    scores=[1]*len(_gtboxes_and_label_batch))

                cv2.imwrite('./eval/images/' + _img_name_batch[0].decode('utf-8','ignore'),
                            final_detections[:, :, ::-1])
                cv2.imwrite('./eval/images/'  + _img_name_batch[0].decode('utf-8', 'ignore').split('.jpg')[0] + 'gt.jpg',
                            final_detectionsgt[:, :, ::-1])

            boxes = np.transpose(np.stack([xmin, ymin, xmax, ymax]))
            dets = np.hstack((_detection_category.reshape(-1, 1),
                              _fast_rcnn_score.reshape(-1, 1),
                              boxes))

            img_name_batchs.append(_img_name_batch[0])
            all_boxes.append(dets)
        view_bar('{} image cost {}s'.format(
            str(_img_name_batch[0]), (end - start)), i + 1, args.img_num)

      fw1 = open( 'detections.pkl', 'wb')
      pickle.dump(all_boxes, fw1)
      fw1.close()
      coord.request_stop()
      coord.join(threads)

  return img_name_batchs

def eval(args):
  print('Called with args:')
  img_name_batchs = eval_dict_convert(args)
  test_imgname_list = [img.decode('utf-8','ignore') for img in img_name_batchs]
  with open('detections.pkl','rb') as f:
        all_boxes = pickle.load(f,encoding='iso-8859-1')
  voc_eval.voc_evaluate_detections(all_boxes=all_boxes,
                                     test_annotation_path=args.annotation_dir,
                                     test_imgid_list=test_imgname_list)

def parse_args():
  """
  Parse input arguments
  """
  parser = argparse.ArgumentParser(description='Evaluate a trained FPN model')
  parser.add_argument('--eval_imgs', dest='eval_imgs',
                        help='evaluate imgs dir ',
                        default='../data/io/VOCdevkit_test/JPEGImages', type=str)
  parser.add_argument('--annotation_dir', dest='annotation_dir',
                    help='the dir save annotations',
                    default='../data/io/VOCdevkit_test/Annotations', type=str)
  parser.add_argument('--weights', dest='weights',
                      help='model path',
                      default='../output/airplane/res101_trained_weights/v3_airplane/airplane_115500model.ckpt',
                      type=str)
  parser.add_argument('--img_num', dest='img_num',
                      help='image numbers',
                      default=300, type=int)

  args = parser.parse_args()
  return args


if __name__ == "__main__":
    args = parse_args()
    eval(args)

[yangxue0827]

你可以试试这个代码，用法差不多，复现效果比较好。最近FPN也在重新复现，效果好的话会放出来。
PS：个人感觉效果不会这么差，不至于0，应该哪里有问题，你看看tensorboard里的训练情况，然后把训练集评估一下，如果还是很差应该是写的有问题，训练集一般不会差的。 @yanghedada

[yangxue0827]

Recommend improved code: https://github.com/DetectionTeamUCAS/FPN_Tensorflow. @lilmangolil @yanghedada

[yanghedada]

谢谢作者解答。@yangxue0827
你好，我用了你的faster rcnn，但是我的mAP效果不佳。。。。

FAST_LOSS：

RPN_LOSS：

在voc_242002model.ckpt：

cls : aeroplane|| Recall: 0.7380952380952381 || Precison: 0.0023791250959324635|| AP: 0.6294789028578562
____________________
cls : bicycle|| Recall: 0.8823529411764706 || Precison: 0.0011015642211940956|| AP: 0.578044344103218
____________________
cls : bird|| Recall: 0.782608695652174 || Precison: 0.0012719949120203519|| AP: 0.6370069069309825
____________________
cls : boat|| Recall: 0.7857142857142857 || Precison: 0.0007327471356248335|| AP: 0.4043851369097608
____________________
cls : bottle|| Recall: 0.525 || Precison: 0.001454192922927775|| AP: 0.23564146671995506
____________________
cls : bus|| Recall: 1.0 || Precison: 0.001471955376510691|| AP: 0.6344413779384931
____________________
cls : car|| Recall: 0.8 || Precison: 0.00344180225281602|| AP: 0.6395417155970032
____________________
cls : cat|| Recall: 0.9565217391304348 || Precison: 0.0017038413878562577|| AP: 0.7523934973006534
____________________
cls : chair|| Recall: 0.7241379310344828 || Precison: 0.003131057104517668|| AP: 0.22768151791782928
____________________
cls : cow|| Recall: 0.6666666666666666 || Precison: 0.0005633406098162102|| AP: 0.39731236120468105
____________________
cls : diningtable|| Recall: 0.7 || Precison: 0.0010634257500949488|| AP: 0.252669762431736
____________________
cls : dog|| Recall: 0.9230769230769231 || Precison: 0.003489640130861505|| AP: 0.8175730502373146
____________________
cls : horse|| Recall: 0.6923076923076923 || Precison: 0.0005416140097490522|| AP: 0.5775718257645969
____________________
cls : motorbike|| Recall: 0.75 || Precison: 0.0008656759486365604|| AP: 0.4705122412093328
____________________
cls : person|| Recall: 0.7853881278538812 || Precison: 0.012140890802569351|| AP: 0.6721500009894374
____________________
cls : pottedplant|| Recall: 0.4 || Precison: 0.0005585812037424941|| AP: 0.17587795929412015
____________________
cls : sheep|| Recall: 0.7619047619047619 || Precison: 0.0011026118117290333|| AP: 0.5402947800914372
____________________
cls : sofa|| Recall: 1.0 || Precison: 0.001028504260946224|| AP: 0.4510309424789766
____________________
cls : train|| Recall: 0.9375 || Precison: 0.0011174849139536616|| AP: 0.8021723415874062
____________________
cls : tvmonitor|| Recall: 0.7368421052631579 || Precison: 0.0010598031794095382|| AP: 0.5245479991293179
____________________
mAP is : 0.5210164065347055

config:

# ------------------------------------------------
VERSION = 'FasterRCNN_pascal_v2'
NET_NAME = 'resnet_v1_101' #'MobilenetV2'
ADD_BOX_IN_TENSORBOARD = True

# ---------------------------------------- System_config
ROOT_PATH = os.path.abspath('../')
print(20*"++--")
print(ROOT_PATH)
GPU_GROUP = "0"
SHOW_TRAIN_INFO_INTE = 50
SMRY_ITER = 100
SAVE_WEIGHTS_INTE = 500
FAST_RCNN_IOU_MAP=0.5

SUMMARY_PATH = ROOT_PATH + '/output/summary'
TEST_SAVE_PATH = ROOT_PATH + '/tools/test_result'
# INFERENCE_IMAGE_PATH = ROOT_PATH + '/tools/inference_image'
# INFERENCE_SAVE_PATH = ROOT_PATH + '/tools/inference_results'

if NET_NAME.startswith("resnet"):
    weights_name = NET_NAME
elif NET_NAME.startswith("MobilenetV2"):
    weights_name = "mobilenet/mobilenet_v2_1.0_224"
else:
    raise Exception('net name must in [resnet_v1_101, resnet_v1_50, MobilenetV2]')

PRETRAINED_CKPT = ROOT_PATH + '/data/pretrained_weights/resnet_v1_101_2016_08_28/' + weights_name + '.ckpt'
TRAINED_CKPT = os.path.join(ROOT_PATH, 'output/trained_weights')
EVALUATE_DIR = ROOT_PATH + '/output/evaluate_result_pickle/'

# ------------------------------------------ Train config
RESTORE_FROM_RPN = False
IS_FILTER_OUTSIDE_BOXES = True
FIXED_BLOCKS = 1  # allow 0~3

RPN_LOCATION_LOSS_WEIGHT = 1.
RPN_CLASSIFICATION_LOSS_WEIGHT = 1.

FAST_RCNN_LOCATION_LOSS_WEIGHT = 1.
FAST_RCNN_CLASSIFICATION_LOSS_WEIGHT = 1.
RPN_SIGMA = 3.0
FASTRCNN_SIGMA = 1.0

MUTILPY_BIAS_GRADIENT = None   # 2.0  # if None, will not multipy
GRADIENT_CLIPPING_BY_NORM = None   # 10.0  if None, will not clip

EPSILON = 1e-5
MOMENTUM = 0.9
LR = 0.001 # 0.001  # 0.0003
DECAY_STEP = [5000, 10000, 50000, 100000]  # 50000, 70000
MAX_ITERATION = 200000

# -------------------------------------------- Data_preprocess_config
DATASET_NAME = 'pascal'  # 'ship', 'spacenet', 'pascal', 'coco' airplane
PIXEL_MEAN = [123.68, 116.779, 103.939]  # R, G, B. In tf, channel is RGB. In openCV, channel is BGR
IMG_SHORT_SIDE_LEN = 600
IMG_MAX_LENGTH = 1000
CLASS_NUM = 20

# --------------------------------------------- Network_config
BATCH_SIZE = 1
INITIALIZER = tf.random_normal_initializer(mean=0.0, stddev=0.01)
BBOX_INITIALIZER = tf.random_normal_initializer(mean=0.0, stddev=0.001)
WEIGHT_DECAY = 0.00004 if NET_NAME.startswith('Mobilenet') else 0.0001

# ---------------------------------------------Anchor config
BASE_ANCHOR_SIZE_LIST = [64, 256]  # can be modified
ANCHOR_STRIDE = [16]  # can not be modified in most situations
ANCHOR_SCALES = [0.5, 1., 2.0]  # [4, 8, 16, 32]
ANCHOR_RATIOS = [0.5, 1., 1.5, 2.0]
ROI_SCALE_FACTORS = [10., 10., 5.0, 5.0]
ANCHOR_SCALE_FACTORS = None


# --------------------------------------------RPN config
KERNEL_SIZE = 3
RPN_IOU_POSITIVE_THRESHOLD = 0.7
RPN_IOU_NEGATIVE_THRESHOLD = 0.3
TRAIN_RPN_CLOOBER_POSITIVES = False

RPN_MINIBATCH_SIZE = 256
RPN_POSITIVE_RATE = 0.5
RPN_NMS_IOU_THRESHOLD = 0.7
RPN_TOP_K_NMS_TRAIN = 12000
RPN_MAXIMUM_PROPOSAL_TARIN = 2000

RPN_TOP_K_NMS_TEST = 6000  # 5000
RPN_MAXIMUM_PROPOSAL_TEST = 300  # 300


# -------------------------------------------Fast-RCNN config
ROI_SIZE = 14
ROI_POOL_KERNEL_SIZE = 2
USE_DROPOUT = False
KEEP_PROB = 1.0
SHOW_SCORE_THRSHOLD = 0.5  # only show in tensorboard

FAST_RCNN_NMS_IOU_THRESHOLD = 0.3  # 0.6
FAST_RCNN_NMS_MAX_BOXES_PER_CLASS = 100
FAST_RCNN_IOU_POSITIVE_THRESHOLD = 0.5
FAST_RCNN_IOU_NEGATIVE_THRESHOLD = 0.0   # 0.1 < IOU < 0.5 is negative
FAST_RCNN_MINIBATCH_SIZE = 256  # if is -1, that is train with OHEM
FAST_RCNN_POSITIVE_RATE = 0.25

ADD_GTBOXES_TO_TRAIN = False

20万次之后的mAP并没有到达你的效果。我这个有config什么问题吗？？

再次感谢作者。。。

[yangxue0827]

FPN这个代码没有复现好，在多类效果差，我在readme已经说明了。请使用新的代码 https://github.com/DetectionTeamUCAS/FPN_Tensorflow。 @yanghedada
l另外，你使用faster-rcnn的配置有错误，anchor设置过少，导致采样不足，请参考默认的配置文件。

@yanghedada

[lilmangolil]

FPN这个代码没有复现好，在多类效果差，我在readme已经说明了。请使用新的代码 https://github.com/DetectionTeamUCAS/FPN_Tensorflow。 @yanghedada
l另外，你使用faster-rcnn的配置有错误，anchor设置过少，导致采样不足，请参考默认的配置文件。

@yanghedada

您好，请问anchor size的设定的依据是什么呢，是该层的感受野大小吗？感谢！

[yanghedada]

就是按照感受野大小设定的。＠lilmangolil

[lilmangolil]

谢谢作者指导，感谢作者提供这么流畅的代码。@yangxue0827
我的训练结果有点雷人。可能是我得训练方式不对。
我只针对airplane进行训练，但是这里的结果不是很优美。如下：
我在voc_eval.py添加了一个参数FAST_RCNN_IOU_MAP。

当FAST_RCNN_IOU_MAP=.01，会给一个安慰结果。

当FAST_RCNN_IOU_MAP=.5，无语了

预测图：

我的config：

这里会不会还有其他问题？：

#!/usr/bin/python
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function

import argparse
import os
import pickle
import sys
import time
import io
import numpy as np
import tensorflow as tf

sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='iso-8859-1')
from data.io.image_preprocess import short_side_resize_for_inference_data
from data.io.read_tfrecord import next_batch
from data.io.divide_data import mkdir
from help_utils.tools import *
from libs.fast_rcnn import build_fast_rcnn
from libs.label_name_dict.label_dict import *
from libs.box_utils import draw_box_in_img
from libs.networks.network_factory import get_flags_byname, get_network_byname
from libs.rpn import build_rpn
from libs.val_libs import voc_eval
from tools import restore_model
import numpy as np
import cv2
FLAGS = get_flags_byname(cfgs.NET_NAME)

def eval_dict_convert(args, draw_imgs=True):
  with tf.Graph().as_default():
    '''
    这里就是读取图片的操作，
    '''
    img_name_batch, img_batch, gtboxes_and_label_batch, num_objects_batch = \
        next_batch(dataset_name=cfgs.DATASET_NAME,
                   batch_size=cfgs.BATCH_SIZE,
                   shortside_len=cfgs.SHORT_SIDE_LEN,
                   is_training=False)

    # ***********************************************************************************************
    # *                                         share net                                           *
    # ***********************************************************************************************
    _, share_net = get_network_byname(net_name=cfgs.NET_NAME,
                                      inputs=img_batch,
                                      num_classes=None,
                                      is_training=True,
                                      output_stride=None,
                                      global_pool=False,
                                      spatial_squeeze=False)

    # ***********************************************************************************************
    # *                                            RPN                                              *
    # ***********************************************************************************************
    rpn = build_rpn.RPN(net_name=cfgs.NET_NAME,
                        inputs=img_batch,
                        gtboxes_and_label=None,
                        is_training=False,
                        share_head=True,
                        share_net=share_net,
                        stride=cfgs.STRIDE,
                        anchor_ratios=cfgs.ANCHOR_RATIOS,
                        anchor_scales=cfgs.ANCHOR_SCALES,
                        scale_factors=cfgs.SCALE_FACTORS,
                        base_anchor_size_list=cfgs.BASE_ANCHOR_SIZE_LIST,  # P2, P3, P4, P5, P6
                        level=cfgs.LEVEL,
                        top_k_nms=cfgs.RPN_TOP_K_NMS,
                        rpn_nms_iou_threshold=cfgs.RPN_NMS_IOU_THRESHOLD,
                        max_proposals_num=cfgs.MAX_PROPOSAL_NUM,
                        rpn_iou_positive_threshold=cfgs.RPN_IOU_POSITIVE_THRESHOLD,
                        rpn_iou_negative_threshold=cfgs.RPN_IOU_NEGATIVE_THRESHOLD,
                        rpn_mini_batch_size=cfgs.RPN_MINIBATCH_SIZE,
                        rpn_positives_ratio=cfgs.RPN_POSITIVE_RATE,
                        remove_outside_anchors=False,  # whether remove anchors outside
                        rpn_weight_decay=cfgs.WEIGHT_DECAY[cfgs.NET_NAME])

    # rpn predict proposals
    rpn_proposals_boxes, rpn_proposals_scores = rpn.rpn_proposals()  # rpn_score shape: [300, ]

    # ***********************************************************************************************
    # *                                         Fast RCNN                                           *
    # ***********************************************************************************************
    fast_rcnn = build_fast_rcnn.FastRCNN(img_batch=img_batch,
                                         feature_pyramid=rpn.feature_pyramid,
                                         rpn_proposals_boxes=rpn_proposals_boxes,
                                         rpn_proposals_scores=rpn_proposals_scores,
                                         img_shape=tf.shape(img_batch),
                                         roi_size=cfgs.ROI_SIZE,
                                         scale_factors=cfgs.SCALE_FACTORS,
                                         roi_pool_kernel_size=cfgs.ROI_POOL_KERNEL_SIZE,
                                         gtboxes_and_label=None,
                                         fast_rcnn_nms_iou_threshold=cfgs.FAST_RCNN_NMS_IOU_THRESHOLD,
                                         fast_rcnn_maximum_boxes_per_img=100,
                                         fast_rcnn_nms_max_boxes_per_class=cfgs.FAST_RCNN_NMS_MAX_BOXES_PER_CLASS,
                                         show_detections_score_threshold=cfgs.FINAL_SCORE_THRESHOLD,  # show detections which score >= 0.6
                                         num_classes=cfgs.CLASS_NUM,
                                         fast_rcnn_minibatch_size=cfgs.FAST_RCNN_MINIBATCH_SIZE,
                                         fast_rcnn_positives_ratio=cfgs.FAST_RCNN_POSITIVE_RATE,
                                         fast_rcnn_positives_iou_threshold=cfgs.FAST_RCNN_IOU_POSITIVE_THRESHOLD,
                                         use_dropout=False,
                                         weight_decay=cfgs.WEIGHT_DECAY[cfgs.NET_NAME],
                                         is_training=False,
                                         level=cfgs.LEVEL)

    fast_rcnn_decode_boxes, fast_rcnn_score, num_of_objects, detection_category = \
        fast_rcnn.fast_rcnn_predict()

    # train
    init_op = tf.group(
        tf.global_variables_initializer(),
        tf.local_variables_initializer()
    )

    restorer, restore_ckpt = restore_model.get_restorer(checkpoint_path=args.weights)

    config = tf.ConfigProto()
    # config.gpu_options.per_process_gpu_memory_fraction = 0.5
    config.gpu_options.allow_growth = True
    with tf.Session(config=config) as sess:
      sess.run(init_op)
      if not restorer is None:
        restorer.restore(sess, restore_ckpt)
        print('restore model')

      coord = tf.train.Coordinator()
      threads = tf.train.start_queue_runners(sess, coord)
      img_name_batchs  = []
      all_boxes = []
      for i in range(args.img_num):

        start = time.time()

        _img_name_batch, _img_batch, _gtboxes_and_label_batch, _fast_rcnn_decode_boxes, \
            _fast_rcnn_score, _detection_category \
            = sess.run([img_name_batch, img_batch, gtboxes_and_label_batch, fast_rcnn_decode_boxes,
                        fast_rcnn_score, detection_category])
        end = time.time()
        raw_img = cv2.imread(os.path.join(args.eval_imgs, _img_name_batch[0].decode('utf-8','ignore')))
        raw_h, raw_w = raw_img.shape[0], raw_img.shape[1]

        ymin, xmin, ymax, xmax = _fast_rcnn_decode_boxes[:, 0], _fast_rcnn_decode_boxes[:, 1], \
                                 _fast_rcnn_decode_boxes[:, 2], _fast_rcnn_decode_boxes[:, 3]

        resized_h, resized_w = _img_batch.shape[1], _img_batch.shape[2]

        xmin = xmin * raw_w / resized_w
        xmax = xmax * raw_w / resized_w
        ymin = ymin * raw_h / resized_h
        ymax = ymax * raw_h / resized_h

        if  len(_detection_category) != 0:
            if draw_imgs:
                show_indices = _fast_rcnn_score >= cfgs.FINAL_SCORE_THRESHOLD
                show_scores = _fast_rcnn_score[show_indices]
                show_boxes = _fast_rcnn_decode_boxes[show_indices]
                show_categories = _detection_category[show_indices]

                pre_ymin, pre_xmin, pre_ymax, pre_xmax = _fast_rcnn_decode_boxes[:, 0], _fast_rcnn_decode_boxes[:, 1], \
                                         _fast_rcnn_decode_boxes[:, 2], _fast_rcnn_decode_boxes[:, 3]

                pre_box = np.stack([pre_xmin, pre_ymin, pre_xmax, pre_ymax], axis=1)
                final_detections = draw_box_in_img.draw_boxes_with_label_and_scores(np.squeeze(_img_batch, 0),
                                                                                    boxes=pre_box,
                                                                                    labels=show_categories,
                                                                                    scores=show_scores)

                gt_ymin, gt_xmin, gt_ymax, gt_xmax = _gtboxes_and_label_batch[0][:, 0], _gtboxes_and_label_batch[0][:, 1], \
                                         _gtboxes_and_label_batch[0][:, 2], _gtboxes_and_label_batch[0][:, 3],
                gt_box = np.stack([gt_xmin, gt_ymin, gt_xmax, gt_ymax], axis=1)
                final_detectionsgt = draw_box_in_img.draw_boxes_with_label_and_scores(np.squeeze(_img_batch, 0),
                                                                                    boxes=gt_box,
                                                                                    labels=_gtboxes_and_label_batch[0][:,-1],
                                                                                    scores=[1]*len(_gtboxes_and_label_batch))

                cv2.imwrite('./eval/images/' + _img_name_batch[0].decode('utf-8','ignore'),
                            final_detections[:, :, ::-1])
                cv2.imwrite('./eval/images/'  + _img_name_batch[0].decode('utf-8', 'ignore').split('.jpg')[0] + 'gt.jpg',
                            final_detectionsgt[:, :, ::-1])

            boxes = np.transpose(np.stack([xmin, ymin, xmax, ymax]))
            dets = np.hstack((_detection_category.reshape(-1, 1),
                              _fast_rcnn_score.reshape(-1, 1),
                              boxes))

            img_name_batchs.append(_img_name_batch[0])
            all_boxes.append(dets)
        view_bar('{} image cost {}s'.format(
            str(_img_name_batch[0]), (end - start)), i + 1, args.img_num)

      fw1 = open( 'detections.pkl', 'wb')
      pickle.dump(all_boxes, fw1)
      fw1.close()
      coord.request_stop()
      coord.join(threads)

  return img_name_batchs

def eval(args):
  print('Called with args:')
  img_name_batchs = eval_dict_convert(args)
  test_imgname_list = [img.decode('utf-8','ignore') for img in img_name_batchs]
  with open('detections.pkl','rb') as f:
        all_boxes = pickle.load(f,encoding='iso-8859-1')
  voc_eval.voc_evaluate_detections(all_boxes=all_boxes,
                                     test_annotation_path=args.annotation_dir,
                                     test_imgid_list=test_imgname_list)

def parse_args():
  """
  Parse input arguments
  """
  parser = argparse.ArgumentParser(description='Evaluate a trained FPN model')
  parser.add_argument('--eval_imgs', dest='eval_imgs',
                        help='evaluate imgs dir ',
                        default='../data/io/VOCdevkit_test/JPEGImages', type=str)
  parser.add_argument('--annotation_dir', dest='annotation_dir',
                    help='the dir save annotations',
                    default='../data/io/VOCdevkit_test/Annotations', type=str)
  parser.add_argument('--weights', dest='weights',
                      help='model path',
                      default='../output/airplane/res101_trained_weights/v3_airplane/airplane_115500model.ckpt',
                      type=str)
  parser.add_argument('--img_num', dest='img_num',
                      help='image numbers',
                      default=300, type=int)

  args = parser.parse_args()
  return args


if __name__ == "__main__":
    args = parse_args()
    eval(args)

您好，请问您所提供的这段代码是否有完整的工程呢？因为里面好多import的文件原始文件里是没有的。谢谢啦(#^.^#)

[yanghedada]

你好@lilmangolil。有一些代码是从点这里拷的。但我建议使用DetectionTeamUCAS/FPN_Tensorflow的代码，这是最完整的代码，两者的项目代码风格不一样。而且，训练出来的模型的mAP的准确度和提交在VOC服务器上mAP相差不大，在1~10%以内。感谢作者的代码。

[yangxue0827]

10%？ @yanghedada 会有这么大吗？

[yanghedada]

差不多吧@yangxue0827。在2007上测试为80%，提交一个2012的结果为74%

[yangxue0827]

你是指提交官网后，训练2007数据集的结果是80，训练2012数据集是74吗 @yanghedada

[yanghedada]

2007我是在电脑上测试的4952张图片80%。我只提交了2012的数据，2007没看到提交选项。@yangxue0827

[yangxue0827]

我没有训练和测试过2012数据集，readme里是2007数据集在07和12评价标准下测的结果。你的74是训练的2012数据集吗，然后用2012标准测试？
ps 你2007数据集能测试到80？我最好78.5 @yanghedada

[yanghedada]

@yangxue0827，应该是测试2000张图片是80%. 4952张图片如下，现场跑的：

model restore from : /home/yanghe/YangHe_MyCode/FPN_Tensorflow-master/output/trained_weights/FPN_Res101_v1/voc_80000model.ckpt
2018-12-11 22:16:40.830512: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:897] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-12-11 22:16:40.830848: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1392] Found device 0 with properties: 
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.683
pciBusID: 0000:01:00.0
totalMemory: 10.92GiB freeMemory: 10.36GiB
2018-12-11 22:16:40.830859: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-12-11 22:16:41.007018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-12-11 22:16:41.007044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958]      0 
2018-12-11 22:16:41.007049: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   N 
2018-12-11 22:16:41.007183: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10017 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
restore model
003607.jpg image cost 0.10984158515930176s:[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>]100%	4952/4952Writing aeroplane VOC resutls file
Writing bicycle VOC resutls file
Writing bird VOC resutls file
Writing boat VOC resutls file
Writing bottle VOC resutls file
Writing bus VOC resutls file
Writing car VOC resutls file
Writing cat VOC resutls file
Writing chair VOC resutls file
Writing cow VOC resutls file
Writing diningtable VOC resutls file
Writing dog VOC resutls file
Writing horse VOC resutls file
Writing motorbike VOC resutls file
Writing person VOC resutls file
Writing pottedplant VOC resutls file
Writing sheep VOC resutls file
Writing sofa VOC resutls file
Writing train VOC resutls file
Writing tvmonitor VOC resutls file
cls : aeroplane|| Recall: 0.9368421052631579 || Precison: 0.0006888900356055524|| AP: 0.8316195907441402
____________________
cls : bicycle|| Recall: 0.9317507418397626 || Precison: 0.0008097061591820421|| AP: 0.8428989611072188
____________________
cls : bird|| Recall: 0.9564270152505446 || Precison: 0.0011153908695475427|| AP: 0.8564636847169369
____________________
cls : boat|| Recall: 0.9239543726235742 || Precison: 0.0006345525621038943|| AP: 0.6681021871004866
____________________
cls : bottle|| Recall: 0.9040511727078892 || Precison: 0.0010303391127905422|| AP: 0.6836585228090871
____________________
cls : bus|| Recall: 0.971830985915493 || Precison: 0.000545292179140335|| AP: 0.893992976333262
____________________
cls : car|| Recall: 0.9592006661115737 || Precison: 0.002975037575344376|| AP: 0.8971617581902198
____________________
cls : cat|| Recall: 0.9776536312849162 || Precison: 0.0008884534056957478|| AP: 0.9199375930964409
____________________
cls : chair|| Recall: 0.8783068783068783 || Precison: 0.001715261435291504|| AP: 0.6108782073953818
____________________
cls : cow|| Recall: 0.9754098360655737 || Precison: 0.000614017662037455|| AP: 0.7791710473975842
____________________
cls : diningtable|| Recall: 0.9320388349514563 || Precison: 0.00048727625264258543|| AP: 0.6986196556015922
____________________
cls : dog|| Recall: 0.983640081799591 || Precison: 0.001226562081636504|| AP: 0.9059106898861442
____________________
cls : horse|| Recall: 0.9683908045977011 || Precison: 0.0008725303120137326|| AP: 0.8889594199400792
____________________
cls : motorbike|| Recall: 0.9384615384615385 || Precison: 0.0007925062685946655|| AP: 0.8489529517343617
____________________
cls : person|| Recall: 0.9293286219081273 || Precison: 0.010406388256212797|| AP: 0.8462024604502763
____________________
cls : pottedplant|| Recall: 0.8541666666666666 || Precison: 0.00103803512609595|| AP: 0.5071455263152422
____________________
cls : sheep|| Recall: 0.9504132231404959 || Precison: 0.0005689267073985208|| AP: 0.8354602699815161
____________________
cls : sofa|| Recall: 0.9623430962343096 || Precison: 0.0006337869043091998|| AP: 0.786707445700414
____________________
cls : train|| Recall: 0.9432624113475178 || Precison: 0.000724355500608622|| AP: 0.865695670375713
____________________
cls : tvmonitor|| Recall: 0.961038961038961 || Precison: 0.0007615832698680609|| AP: 0.7962795325532337
____________________
mAP is : 0.7981909075714665

我使用2007和2012的训练集和验证集进行训练，使用了预训练模型。

提交结点这里

[yangxue0827]

那效果还可以的 @yanghedada

[yangxue0827]

刚看到提交的结果，你自己测的结果和提交官网的结果是同一个测试集的结果吗？那样的话差的有点多啊 @yanghedada

[yanghedada]

我自己的测试结果是2007的测试集数据，提交的结果是2012的测试集数据 来自 魅蓝 note 2

…

-------- 原始邮件 -------- 发件人：yangxue <notifications@github.com> 时间：周二 12月11日 23:19 收件人：yangxue0827/FPN_Tensorflow <FPN_Tensorflow@noreply.github.com> 抄送：yanghedada <y_yanghe@163.com>,Mention <mention@noreply.github.com> 主题：Re: [yangxue0827/FPN_Tensorflow] 您好，请问训练过后如何在benchmark数据库上检测训练结果的准确率？ (#46)

刚看到提交的结果，你自己测的结果和提交官网的结果是同一个测试集的结果吗？那样的话差的有点多啊 @yanghedada — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/yangxue0827/FPN_Tensorflow","title":"yangxue0827/FPN_Tensorflow","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in ***@***.*** in #46: 刚看到提交的结果，你自己测的结果和提交官网的结果是同一个测试集的结果吗？那样的话差的有点多啊 @yanghedada "}],"action":{"name":"View Issue","url":"#46 (comment)"}}} [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#46 (comment)", "url": "#46 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } }, { ***@***.***": "MessageCard", ***@***.***": "http://schema.org/extensions", "hideOriginalBody": "false", "originator": "AF6C5A86-E920-430C-9C59-A73278B5EFEB", "title": "Re: [yangxue0827/FPN_Tensorflow] 您好，请问训练过后如何在benchmark数据库上检测训练结果的准确率？ (#46)", "sections": [ { "text": "", "activityTitle": "**yangxue**", "activityImage": "https://assets-cdn.github.com/images/email/message_cards/avatar.png", "activitySubtitle": ***@***.***", "facts": [ ] } ], "potentialAction": [ { "name": "Add a comment", ***@***.***": "ActionCard", "inputs": [ { "isMultiLine": true, ***@***.***": "TextInput", "id": "IssueComment", "isRequired": false } ], "actions": [ { "name": "Comment", ***@***.***": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"IssueComment\",\n\"repositoryFullName\": \"yangxue0827/FPN_Tensorflow\",\n\"issueId\": 46,\n\"IssueComment\": \"{{IssueComment.value}}\"\n}" } ] }, { "name": "Close issue", ***@***.***": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"IssueClose\",\n\"repositoryFullName\": \"yangxue0827/FPN_Tensorflow\",\n\"issueId\": 46\n}" }, { "targets": [ { "os": "default", "uri": "#46 (comment)" } ], ***@***.***": "OpenUri", "name": "View on GitHub" }, { "name": "Unsubscribe", ***@***.***": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"MuteNotification\",\n\"threadId\": 416984855\n}" } ], "themeColor": "26292E" } ]

[yangxue0827]

那没什么问题了，谢了。 @yanghedada

[memeda2232]

我观察你放出来的loss图总是振荡不收敛，我也有相似的情况，请问你是如何解决的？

[memeda2232]

@yanghedada

[yanghedada]

我的策略只能是调整学习率和rpn的batch_size，BASE_ANCHOR_SIZE_LIST ，但也没有更好的收敛。使用更多的epoch，也没法达到最后的收敛。readme中，只训练用了8k个循环，看起来还不错，但是8k之后可能也会是震荡，收敛比较困难。@memeda2232

[chenzui1113]

我自己的测试结果是2007的测试集数据，提交的结果是2012的测试集数据 来自 魅蓝 note 2
…
-------- 原始邮件 -------- 发件人：yangxue notifications@github.com 时间：周二 12月11日 23:19 收件人：yangxue0827/FPN_Tensorflow FPN_Tensorflow@noreply.github.com 抄送：yanghedada y_yanghe@163.com,Mention mention@noreply.github.com 主题：Re: [yangxue0827/FPN_Tensorflow] 您好，请问训练过后如何在benchmark数据库上检测训练结果的准确率？ (#46)
刚看到提交的结果，你自己测的结果和提交官网的结果是同一个测试集的结果吗？那样的话差的有点多啊 @yanghedada — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/yangxue0827/FPN_Tensorflow","title":"yangxue0827/FPN_Tensorflow","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in @.*** in #46: 刚看到提交的结果，你自己测的结果和提交官网的结果是同一个测试集的结果吗？那样的话差的有点多啊 @yanghedada "}],"action":{"name":"View Issue","url":"#46 (comment)"}}} [ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "#46 (comment)", "url": "#46 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.": "Organization", "name": "GitHub", "url": "https://github.com" } }, { @.": "MessageCard", @.": "http://schema.org/extensions", "hideOriginalBody": "false", "originator": "AF6C5A86-E920-430C-9C59-A73278B5EFEB", "title": "Re: [yangxue0827/FPN_Tensorflow] 您好，请问训练过后如何在benchmark数据库上检测训练结果的准确率？ (#46)", "sections": [ { "text": "", "activityTitle": "yangxue", "activityImage": "https://assets-cdn.github.com/images/email/message_cards/avatar.png", "activitySubtitle": @.", "facts": [ ] } ], "potentialAction": [ { "name": "Add a comment", @.": "ActionCard", "inputs": [ { "isMultiLine": true, @.": "TextInput", "id": "IssueComment", "isRequired": false } ], "actions": [ { "name": "Comment", @.": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "IssueComment",\n"repositoryFullName": "yangxue0827/FPN_Tensorflow",\n"issueId": 46,\n"IssueComment": "{{IssueComment.value}}"\n}" } ] }, { "name": "Close issue", @.": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "IssueClose",\n"repositoryFullName": "yangxue0827/FPN_Tensorflow",\n"issueId": 46\n}" }, { "targets": [ { "os": "default", "uri": "#46 (comment)" } ], @.": "OpenUri", "name": "View on GitHub" }, { "name": "Unsubscribe", @.***": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "MuteNotification",\n"threadId": 416984855\n}" } ], "themeColor": "26292E" } ]

请问这个提交到voc2012上

@yangxue0827，应该是测试2000张图片是80%. 4952张图片如下，现场跑的：

model restore from : /home/yanghe/YangHe_MyCode/FPN_Tensorflow-master/output/trained_weights/FPN_Res101_v1/voc_80000model.ckpt
2018-12-11 22:16:40.830512: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:897] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-12-11 22:16:40.830848: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1392] Found device 0 with properties: 
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.683
pciBusID: 0000:01:00.0
totalMemory: 10.92GiB freeMemory: 10.36GiB
2018-12-11 22:16:40.830859: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-12-11 22:16:41.007018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-12-11 22:16:41.007044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958]      0 
2018-12-11 22:16:41.007049: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   N 
2018-12-11 22:16:41.007183: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10017 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
restore model
003607.jpg image cost 0.10984158515930176s:[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>]100%	4952/4952Writing aeroplane VOC resutls file
Writing bicycle VOC resutls file
Writing bird VOC resutls file
Writing boat VOC resutls file
Writing bottle VOC resutls file
Writing bus VOC resutls file
Writing car VOC resutls file
Writing cat VOC resutls file
Writing chair VOC resutls file
Writing cow VOC resutls file
Writing diningtable VOC resutls file
Writing dog VOC resutls file
Writing horse VOC resutls file
Writing motorbike VOC resutls file
Writing person VOC resutls file
Writing pottedplant VOC resutls file
Writing sheep VOC resutls file
Writing sofa VOC resutls file
Writing train VOC resutls file
Writing tvmonitor VOC resutls file
cls : aeroplane|| Recall: 0.9368421052631579 || Precison: 0.0006888900356055524|| AP: 0.8316195907441402
____________________
cls : bicycle|| Recall: 0.9317507418397626 || Precison: 0.0008097061591820421|| AP: 0.8428989611072188
____________________
cls : bird|| Recall: 0.9564270152505446 || Precison: 0.0011153908695475427|| AP: 0.8564636847169369
____________________
cls : boat|| Recall: 0.9239543726235742 || Precison: 0.0006345525621038943|| AP: 0.6681021871004866
____________________
cls : bottle|| Recall: 0.9040511727078892 || Precison: 0.0010303391127905422|| AP: 0.6836585228090871
____________________
cls : bus|| Recall: 0.971830985915493 || Precison: 0.000545292179140335|| AP: 0.893992976333262
____________________
cls : car|| Recall: 0.9592006661115737 || Precison: 0.002975037575344376|| AP: 0.8971617581902198
____________________
cls : cat|| Recall: 0.9776536312849162 || Precison: 0.0008884534056957478|| AP: 0.9199375930964409
____________________
cls : chair|| Recall: 0.8783068783068783 || Precison: 0.001715261435291504|| AP: 0.6108782073953818
____________________
cls : cow|| Recall: 0.9754098360655737 || Precison: 0.000614017662037455|| AP: 0.7791710473975842
____________________
cls : diningtable|| Recall: 0.9320388349514563 || Precison: 0.00048727625264258543|| AP: 0.6986196556015922
____________________
cls : dog|| Recall: 0.983640081799591 || Precison: 0.001226562081636504|| AP: 0.9059106898861442
____________________
cls : horse|| Recall: 0.9683908045977011 || Precison: 0.0008725303120137326|| AP: 0.8889594199400792
____________________
cls : motorbike|| Recall: 0.9384615384615385 || Precison: 0.0007925062685946655|| AP: 0.8489529517343617
____________________
cls : person|| Recall: 0.9293286219081273 || Precison: 0.010406388256212797|| AP: 0.8462024604502763
____________________
cls : pottedplant|| Recall: 0.8541666666666666 || Precison: 0.00103803512609595|| AP: 0.5071455263152422
____________________
cls : sheep|| Recall: 0.9504132231404959 || Precison: 0.0005689267073985208|| AP: 0.8354602699815161
____________________
cls : sofa|| Recall: 0.9623430962343096 || Precison: 0.0006337869043091998|| AP: 0.786707445700414
____________________
cls : train|| Recall: 0.9432624113475178 || Precison: 0.000724355500608622|| AP: 0.865695670375713
____________________
cls : tvmonitor|| Recall: 0.961038961038961 || Precison: 0.0007615832698680609|| AP: 0.7962795325532337
____________________
mAP is : 0.7981909075714665

我使用2007和2012的训练集和验证集进行训练，使用了预训练模型。

提交结点这里

我自己的测试结果是2007的测试集数据，提交的结果是2012的测试集数据 来自 魅蓝 note 2
…
-------- 原始邮件 -------- 发件人：yangxue notifications@github.com 时间：周二 12月11日 23:19 收件人：yangxue0827/FPN_Tensorflow FPN_Tensorflow@noreply.github.com 抄送：yanghedada y_yanghe@163.com,Mention mention@noreply.github.com 主题：Re: [yangxue0827/FPN_Tensorflow] 您好，请问训练过后如何在benchmark数据库上检测训练结果的准确率？ (#46)
刚看到提交的结果，你自己测的结果和提交官网的结果是同一个测试集的结果吗？那样的话差的有点多啊 @yanghedada — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/yangxue0827/FPN_Tensorflow","title":"yangxue0827/FPN_Tensorflow","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in @.*** in #46: 刚看到提交的结果，你自己测的结果和提交官网的结果是同一个测试集的结果吗？那样的话差的有点多啊 @yanghedada "}],"action":{"name":"View Issue","url":"#46 (comment)"}}} [ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "#46 (comment)", "url": "#46 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.": "Organization", "name": "GitHub", "url": "https://github.com" } }, { @.": "MessageCard", @.": "http://schema.org/extensions", "hideOriginalBody": "false", "originator": "AF6C5A86-E920-430C-9C59-A73278B5EFEB", "title": "Re: [yangxue0827/FPN_Tensorflow] 您好，请问训练过后如何在benchmark数据库上检测训练结果的准确率？ (#46)", "sections": [ { "text": "", "activityTitle": "yangxue", "activityImage": "https://assets-cdn.github.com/images/email/message_cards/avatar.png", "activitySubtitle": @.", "facts": [ ] } ], "potentialAction": [ { "name": "Add a comment", @.": "ActionCard", "inputs": [ { "isMultiLine": true, @.": "TextInput", "id": "IssueComment", "isRequired": false } ], "actions": [ { "name": "Comment", @.": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "IssueComment",\n"repositoryFullName": "yangxue0827/FPN_Tensorflow",\n"issueId": 46,\n"IssueComment": "{{IssueComment.value}}"\n}" } ] }, { "name": "Close issue", @.": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "IssueClose",\n"repositoryFullName": "yangxue0827/FPN_Tensorflow",\n"issueId": 46\n}" }, { "targets": [ { "os": "default", "uri": "#46 (comment)" } ], @.": "OpenUri", "name": "View on GitHub" }, { "name": "Unsubscribe", @.***": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "MuteNotification",\n"threadId": 416984855\n}" } ], "themeColor": "26292E" } ]

@yangxue0827，应该是测试2000张图片是80%. 4952张图片如下，现场跑的：

model restore from : /home/yanghe/YangHe_MyCode/FPN_Tensorflow-master/output/trained_weights/FPN_Res101_v1/voc_80000model.ckpt
2018-12-11 22:16:40.830512: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:897] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-12-11 22:16:40.830848: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1392] Found device 0 with properties: 
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.683
pciBusID: 0000:01:00.0
totalMemory: 10.92GiB freeMemory: 10.36GiB
2018-12-11 22:16:40.830859: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-12-11 22:16:41.007018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-12-11 22:16:41.007044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958]      0 
2018-12-11 22:16:41.007049: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   N 
2018-12-11 22:16:41.007183: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10017 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
restore model
003607.jpg image cost 0.10984158515930176s:[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>]100%	4952/4952Writing aeroplane VOC resutls file
Writing bicycle VOC resutls file
Writing bird VOC resutls file
Writing boat VOC resutls file
Writing bottle VOC resutls file
Writing bus VOC resutls file
Writing car VOC resutls file
Writing cat VOC resutls file
Writing chair VOC resutls file
Writing cow VOC resutls file
Writing diningtable VOC resutls file
Writing dog VOC resutls file
Writing horse VOC resutls file
Writing motorbike VOC resutls file
Writing person VOC resutls file
Writing pottedplant VOC resutls file
Writing sheep VOC resutls file
Writing sofa VOC resutls file
Writing train VOC resutls file
Writing tvmonitor VOC resutls file
cls : aeroplane|| Recall: 0.9368421052631579 || Precison: 0.0006888900356055524|| AP: 0.8316195907441402
____________________
cls : bicycle|| Recall: 0.9317507418397626 || Precison: 0.0008097061591820421|| AP: 0.8428989611072188
____________________
cls : bird|| Recall: 0.9564270152505446 || Precison: 0.0011153908695475427|| AP: 0.8564636847169369
____________________
cls : boat|| Recall: 0.9239543726235742 || Precison: 0.0006345525621038943|| AP: 0.6681021871004866
____________________
cls : bottle|| Recall: 0.9040511727078892 || Precison: 0.0010303391127905422|| AP: 0.6836585228090871
____________________
cls : bus|| Recall: 0.971830985915493 || Precison: 0.000545292179140335|| AP: 0.893992976333262
____________________
cls : car|| Recall: 0.9592006661115737 || Precison: 0.002975037575344376|| AP: 0.8971617581902198
____________________
cls : cat|| Recall: 0.9776536312849162 || Precison: 0.0008884534056957478|| AP: 0.9199375930964409
____________________
cls : chair|| Recall: 0.8783068783068783 || Precison: 0.001715261435291504|| AP: 0.6108782073953818
____________________
cls : cow|| Recall: 0.9754098360655737 || Precison: 0.000614017662037455|| AP: 0.7791710473975842
____________________
cls : diningtable|| Recall: 0.9320388349514563 || Precison: 0.00048727625264258543|| AP: 0.6986196556015922
____________________
cls : dog|| Recall: 0.983640081799591 || Precison: 0.001226562081636504|| AP: 0.9059106898861442
____________________
cls : horse|| Recall: 0.9683908045977011 || Precison: 0.0008725303120137326|| AP: 0.8889594199400792
____________________
cls : motorbike|| Recall: 0.9384615384615385 || Precison: 0.0007925062685946655|| AP: 0.8489529517343617
____________________
cls : person|| Recall: 0.9293286219081273 || Precison: 0.010406388256212797|| AP: 0.8462024604502763
____________________
cls : pottedplant|| Recall: 0.8541666666666666 || Precison: 0.00103803512609595|| AP: 0.5071455263152422
____________________
cls : sheep|| Recall: 0.9504132231404959 || Precison: 0.0005689267073985208|| AP: 0.8354602699815161
____________________
cls : sofa|| Recall: 0.9623430962343096 || Precison: 0.0006337869043091998|| AP: 0.786707445700414
____________________
cls : train|| Recall: 0.9432624113475178 || Precison: 0.000724355500608622|| AP: 0.865695670375713
____________________
cls : tvmonitor|| Recall: 0.961038961038961 || Precison: 0.0007615832698680609|| AP: 0.7962795325532337
____________________
mAP is : 0.7981909075714665

我使用2007和2012的训练集和验证集进行训练，使用了预训练模型。

提交结点这里

您好，请问您所提交的VOC2012的结果有没有修改什么东西？我提交到VOC2012上测试一直没有出结果，想问下您可能是什么原因？
http://host.robots.ox.ac.uk:8080/anonymous/OSHIRQ.html