文章目錄

• • 一、Detectron2 操作介紹
  • • 1.1 訓練
    • 1.2 測試
    • 1.3 數據及格式要求
    • 1.4 Load/Save model
    • 1.5 模型輸入形式
    • 1.6 模型輸出
    • 1.7 config usage
  • 二、Detectron2 代碼結構介紹
  • • 2.1 數據
    • 2.2 模型
    • 2.3 訓練類的實現
    • 2.4 訓練
    • 2.5 推理
    • 2.6 模型的加載和保存
    • 2.7 性能評估
    • 2.8 日志存儲

一、Detectron2 操作介紹

Detectron2代碼鏈接：https://github.com/facebookresearch/detectron2

Detectron2說明文檔：https://detectron2.readthedocs.io/index.html

安裝之后要編譯：

# 編譯 python setup.py build develop

1.1 訓練

1、訓練有兩個腳本， tools/plain_train_net.py 提供的默認參數更少

- tools/plain_train_net.py - tools/train_net.py

2、訓練之前要設置對應數據集

https://github.com/facebookresearch/detectron2/blob/master/datasets/README.md

3、訓練

# 單GPU cd toos/ ./train_net.py \--config-file ../configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_1x.yaml \--num-gpus 1 SOLVER.IMS_PER_BATCH 2 SOLVER.BASE_LR 0.0025 # 多GPU cd tools/ ./train_net.py --num-gpus 8 \--config-file ../configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_1x.yaml

4、評估模型性能

./train_net.py \--config-file ../configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_1x.yaml \--eval-only MODEL.WEIGHTS /path/to/checkpoint_file

更多的信息可以使用下面的命令查看：

./train_net.py -h

1.2 測試

1、從 model zoo 下載官方訓好的模型

2、測試demo

# demo測試 cd demo/ python demo.py --config-file ../configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml \--input input1.jpg input2.jpg \[--other-options]--opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl # 可修改細節 - To run on your webcam, replace --input files with --webcam. - To run on a video, replace --input files with --video-input video.mp4. - To run on cpu, add MODEL.DEVICE cpu after --opts. - To save outputs to a directory (for images) or a file (for webcam or video), use --output.

1.3 數據及格式要求

https://detectron2.readthedocs.io/en/latest/tutorials/builtin_datasets.html

1.4 Load/Save model

1、detectron2 的 Models (和其他 sub-models) 以如下形式建立：

build_model, build_backbone, build_roi_heads: from detectron2.modeling import build_model model = build_model(cfg) # returns a torch.nn.Module

2、Load/Save checkpoint：

from detectron2.checkpoint import DetectionCheckpointer DetectionCheckpointer(model).load(file_path_or_url) # load a file, usually from cfg.MODEL.WEIGHTScheckpointer = DetectionCheckpointer(model, save_dir="output") checkpointer.save("model_999") # save to output/model_999.pth

Detectron2 的 checkpointer 將模型以 .pth 和 .pkl 的形式保存，可以使用 torch.load / torch.save 來處理前者，使用pickle.dump / pickle.load 來處理后者。

1.5 模型輸入形式

outputs = model(inputs) # inputs is a list[dict] The dict may contain the following keys:- “image”: Tensor in (C, H, W) format. The meaning of channels are defined by cfg.INPUT.FORMAT. Image normalization, if any, will be performed inside the model using cfg.MODEL.PIXEL_{MEAN,STD}.- “height”, “width”: the desired output height and width, which is not necessarily the same as the height or width of the image field. For example, the image field contains the resized image, if resize is used as a preprocessing step. But you may want the outputs to be in original resolution. If provided, the model will produce output in this resolution, rather than in the resolution of the image as input into the model. This is more efficient and accurate.- “instances”: an Instances object for training, with the following fields:- “gt_boxes”: a Boxes object storing N boxes, one for each instance.- “gt_classes”: Tensor of long type, a vector of N labels, in range [0, num_categories).- “gt_masks”: a PolygonMasks or BitMasks object storing N masks, one for each instance.- “gt_keypoints”: a Keypoints object storing N keypoint sets, one for each instance.- “sem_seg”: Tensor[int] in (H, W) format. The semantic segmentation ground truth for training. Values represent category labels starting from 0.- “proposals”: an Instances object used only in Fast R-CNN style models, with the following fields:- “proposal_boxes”: a Boxes object storing P proposal boxes.- “objectness_logits”: Tensor, a vector of P scores, one for each proposal.For inference of builtin models, only “image” key is required, and “width/height” are optional.

1.6 模型輸出

訓練模式：a dict[str->ScalarTensor] with all the losses.

推理模式： a list[dict], one dict for each image.

每個dict包含內容如下:

- “instances”: Instances object with the following fields:- “pred_boxes”: Boxes object storing N boxes, one for each detected instance.- “scores”: Tensor, a vector of N confidence scores.- “pred_classes”: Tensor, a vector of N labels in range [0, num_categories).- “pred_masks”: a Tensor of shape (N, H, W), masks for each detected instance.- “pred_keypoints”: a Tensor of shape (N, num_keypoint, 3). Each row in the last dimension is (x, y, score). Confidence scores are larger than 0.- “sem_seg”: Tensor of (num_categories, H, W), the semantic segmentation prediction.- “proposals”: Instances object with the following fields:- “proposal_boxes”: Boxes object storing N boxes.- “objectness_logits”: a torch vector of N confidence scores.- “panoptic_seg”: A tuple of (pred: Tensor, segments_info: Optional[list[dict]]). The pred tensor has shape (H, W), containing the segment id of each pixel.If segments_info exists, each dict describes one segment id in pred and has the following fields:- “id”: the segment id- “isthing”: whether the segment is a thing or stuff- “category_id”: the category id of this segment.If a pixel’s id does not exist in segments_info, it is considered to be void label defined in Panoptic Segmentation.If segments_info is None, all pixel values in pred must be ≥ -1. Pixels with value -1 are assigned void labels. Otherwise, the category id of each pixel is obtained by category_id = pixel // metadata.label_divisor.

1.7 config usage

from detectron2.config import get_cfg cfg = get_cfg() # obtain detectron2's default config cfg.xxx = yyy # add new configs for your own custom components cfg.merge_from_file("my_cfg.yaml") # load values from a filecfg.merge_from_list(["MODEL.WEIGHTS", "weights.pth"]) # can also load values from a list of str print(cfg.dump()) # print formatted configs

二、Detectron2 代碼結構介紹

API地址

• engine：整合數據和model的過程，實現訓練、測試
• data：model的輸入
• modeling：solver（optimizer）的輸入
• solver：優化器
• layers：構成modeling的基本層
• evaluation：評估
• config：讀取配置文件
• projects：工程示例
• checkpoint：存儲和加載模型權重

2.1 數據

./detectron2/data/

1、讀取圖片

./data/common.py

2、數據增強

./data/common.py

3、轉化為batch

./data/build.py

4、修改數據路徑

./data/datasets/

• pascal voc

• coco (register_coco.py & coco.py )
  
  修改超參數：

./configs/Base-RCNN-FPN.yaml

2.2 模型

./detectron2/modeling/

1、backbone

# backbone的抽象基類 # ./modeling/backbone/backbone.py

抽象類：

• 類, 是從一堆對象中抽象出來的, 比如貓類,狗類,人類

• 抽象類, 是從一堆類中抽象出來的, 比如上面的三個類可以抽取出動物類

• 抽象類的特點是不能給實例化, 只能被子類繼承, 由子類實現了父類的抽象方法后, 子類才能被實例化

• Python的abc提供了@abstractmethod裝飾器實現抽象方法

build backbone

./modeling/backbone/build.py

backbone/resnet.py 中繼承了 CNNBlockBase，定義了不同的Block，ResNet繼承Backbone，并使用定義的block實現ResNet的backbone

當使用 from 模塊名 import * 時，想要有一些變量不被調用，可以借助模塊提供的 __all__ 變量：

• 該變量的值是一個列表，存儲的是當前模塊中一些成員（變量、函數或者類）的名稱。通過在模塊文件中設置 __all__ 變量，當其它文件以 “from 模塊名 import * ”的形式導入該模塊時，該文件中只能使用 __all__ 列表中指定的成員。
• 也就是說，只有以“from 模塊名 import *”形式導入的模塊，當該模塊設有 __all__ 變量時，只能導入該變量指定的成員，未指定的成員是無法導入的。

注冊與調用：

定義：

@BACKBONE_REGISTRY.register() def build_resnet_backbone(cfg, input_shape):return ResNet(stem, stages, out_features=out_features).freeze(freeze_at)

調用：./build.py中根據配置文件名調用之前Register好的backbone

backbone = BACKBONE_REGISTRY.get(backbone_name)(cfg, input_shape)

./modeling/backbone/fpn.py 文件又把build_resnet_backbone生產的resnet作為子結構輸入，擴展了不同的FPN的backbone:

# resnet @BACKBONE_REGISTRY.register() def build_resnet_fpn_backbone(cfg, input_shape: ShapeSpec): # retinanet @BACKBONE_REGISTRY.register() def build_resnet_fpn_backbone(cfg, input_shape: ShapeSpec):

2、proposal 生成

./modeling/proposal_generator/build.py

根據配置文件調用相應的 proposal generator ：

PROPOSAL_GENERATOR_REGISTRY.get(name)(cfg, input_shape) # 1 @RPN_HEAD_REGISTRY.register() class StandardRPNHead(nn.Module): # 2 @PROPOSAL_GENERATOR_REGISTRY.register() class RPN(nn.Module):

3、RoI Heads

接口：

./modeling/roi_heads/roi_heads.py

實現：

# 1 @ROI_HEADS_REGISTRY.register() class Res5ROIHeads(ROIHeads): # 2 @ROI_HEADS_REGISTRY.register() class StandardROIHeads(ROIHeads):

4、mask head

def build_mask_head(cfg, input_shape):name = cfg.MODEL.ROI_MASK_HEAD.NAMEreturn ROI_MASK_HEAD_REGISTRY.get(name)(cfg, input_shape)

5、keypoint head

def build_keypoint_head(cfg, input_shape):name = cfg.MODEL.ROI_KEYPOINT_HEAD.NAMEreturn ROI_KEYPOINT_HEAD_REGISTRY.get(name)(cfg, input_shape)

6、執行流程

./modeling/meta_arch/

• batch_input 進行預處理
• 輸入backbone進行特征提取
• 將feature和img輸入給proposal_generator
• 將proposal 結果給到 RoI Heads

def forward(self, batched_inputs):if not self.training:return self.inference(batched_inputs)images = self.preprocess_image(batched_inputs)if "instances" in batched_inputs[0]:gt_instances = [x["instances"].to(self.device) for x in batched_inputs]else:gt_instances = Nonefeatures = self.backbone(images.tensor)if self.proposal_generator:proposals, proposal_losses = self.proposal_generator(images, features, gt_instances)else:assert "proposals" in batched_inputs[0]proposals = [x["proposals"].to(self.device) for x in batched_inputs]proposal_losses = {}_, detector_losses = self.roi_heads(images, features, proposals, gt_instances)if self.vis_period > 0:storage = get_event_storage()if storage.iter % self.vis_period == 0:self.visualize_training(batched_inputs, proposals)losses = {}losses.update(detector_losses)losses.update(proposal_losses)return losses

2.3 訓練類的實現

./detectron2/engine/train_loop.py

1、HookBase 定義了四個階段：

• before_train
• after_train
• before_step
• after_step

2、TrainerBase 對 hook 靈活調用，使用各個功能

3、SimpleTrainer （./train_loop.py) 繼承自TrainerBase，對TrainerBase中預留接口的訓練核心部分的方法def run_step(self)做了具體實現，包括推理計算loss以及backward：

4、DefaultTrainer（./defaults.py）繼承自SimpleTrainer，實現了訓練流程，包括創建model, optimizer, scheduler, dataloader，根據配置文件增加了輔助功能hooks類中的功能

2.4 訓練

./tools/

可見GPU號修改：

os.environ['CUDA_VISIBLE_DEVIES'] = '0, 1, 2'

train_net.py中層層抽象，在之前TrainBase$\to$ SimpleTrainer$\to$DefaultTrainer上又增加了一層抽象，添加evaluation模塊的功能，以及inference with test-time augmentation功能

2.5 推理

./detectron2/engine/defaults.py

2.6 模型的加載和保存

1、兩種保存模型的方法：

• 僅保存權重

# 保存 torch.save(model.state_dict(), path) # 加載 model = Model() model.load_state_dict(torch.load(path)) model.eval()

• 保存整個模型和對應權重

# 保存 torch.save(model, path) # 加載 model = torch.load(path) model.eval()

2、保存 checkpoint 的方法

完整的checkpoint一般保存了模型的 state_dict、優化器的state_dict、epoch等

• 保存checkpoint

torch.save({'epoch': epoch,'model_state_dict': model.state_dict(),'optimizer_state_dict': optimizer.state_dict(),'loss': loss}, path)

• 加載checkpoint

checkpoitn = torch.load(path) model.load_state_dict(checkpoint['model_state_dict']) optimizer.load_state_dict(checkpoint['optimizer_state_dict']) epoch = checkpoint['epoch'] loss = checkpoint['loss'] model.eval()

2.7 性能評估

./detectron2/evaluation/evaluator.py

2.8 日志存儲

./engine/hooks.py

日志存儲是通過 hook 來控制的，hooks.py 中的 after_step() 方法調用 writer.write() 進行日志的寫入。

總結

以上是生活随笔為你收集整理的Detectron2学习笔记的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。