You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
Merge `develop` branch into `master` (#3518) * update ci-testing.yml (#3322) * update ci-testing.yml * update greetings.yml * bring back os matrix * update ci-testing.yml (#3322) * update ci-testing.yml * update greetings.yml * bring back os matrix * Enable direct `--weights URL` definition (#3373) * Enable direct `--weights URL` definition @KalenMike this PR will enable direct --weights URL definition. Example use case: ``` python train.py --weights https://storage.googleapis.com/bucket/dir/model.pt ``` * cleanup * bug fixes * weights = attempt_download(weights) * Update experimental.py * Update hubconf.py * return bug fix * comment mirror * min_bytes * Update tutorial.ipynb (#3368) add Open in Kaggle badge * `cv2.imread(img, -1)` for IMREAD_UNCHANGED (#3379) * Update datasets.py * comment Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * COCO evolution fix (#3388) * COCO evolution fix * cleanup * update print * print fix * Create `is_pip()` function (#3391) Returns `True` if file is part of pip package. Useful for contextual behavior modification. ```python def is_pip(): # Is file in a pip package? return 'site-packages' in Path(__file__).absolute().parts ``` * Revert "`cv2.imread(img, -1)` for IMREAD_UNCHANGED (#3379)" (#3395) This reverts commit 21a9607e00f1365b21d8c4bd81bdbf5fc0efea24. * Update FLOPs description (#3422) * Update README.md * Changing FLOPS to FLOPs. Co-authored-by: BuildTools <unconfigured@null.spigotmc.org> * Parse URL authentication (#3424) * Parse URL authentication * urllib.parse.unquote() * improved error handling * improved error handling * remove %3F * update check_file() * Add FLOPs title to table (#3453) * Suppress jit trace warning + graph once (#3454) * Suppress jit trace warning + graph once Suppress harmless jit trace warning on TensorBoard add_graph call. Also fix multiple add_graph() calls bug, now only on batch 0. * Update train.py * Update MixUp augmentation `alpha=beta=32.0` (#3455) Per VOC empirical results https://github.com/ultralytics/yolov5/issues/3380#issuecomment-853001307 by @developer0hye * Add `timeout()` class (#3460) * Add `timeout()` class * rearrange order * Faster HSV augmentation (#3462) remove datatype conversion process that can be skipped * Add `check_git_status()` 5 second timeout (#3464) * Add check_git_status() 5 second timeout This should prevent the SSH Git bug that we were discussing @KalenMike * cleanup * replace timeout with check_output built-in timeout * Improved `check_requirements()` offline-handling (#3466) Improve robustness of `check_requirements()` function to offline environments (do not attempt pip installs when offline). * Add `output_names` argument for ONNX export with dynamic axes (#3456) * Add output names & dynamic axes for onnx export Add output_names and dynamic_axes names for all outputs in torch.onnx.export. The first four outputs of the model will have names output0, output1, output2, output3 * use first output only + cleanup Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Revert FP16 `test.py` and `detect.py` inference to FP32 default (#3423) * fixed inference bug ,while use half precision * replace --use-half with --half * replace space and PEP8 in detect.py * PEP8 detect.py * update --half help comment * Update test.py * revert space Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add additional links/resources to stale.yml message (#3467) * Update stale.yml * cleanup * Update stale.yml * reformat * Update stale.yml HUB URL (#3468) * Stale `github.actor` bug fix (#3483) * Explicit `model.eval()` call `if opt.train=False` (#3475) * call model.eval() when opt.train is False call model.eval() when opt.train is False * single-line if statement * cleanup Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * check_requirements() exclude `opencv-python` (#3495) Fix for 3rd party or contrib versions of installed OpenCV as in https://github.com/ultralytics/yolov5/issues/3494. * Earlier `assert` for cpu and half option (#3508) * early assert for cpu and half option early assert for cpu and half option * Modified comment Modified comment * Update tutorial.ipynb (#3510) * Reduce test.py results spacing (#3511) * Update README.md (#3512) * Update README.md Minor modifications * 850 width * Update greetings.yml revert greeting change as PRs will now merge to master. Co-authored-by: Piotr Skalski <SkalskiP@users.noreply.github.com> Co-authored-by: SkalskiP <piotr.skalski92@gmail.com> Co-authored-by: Peretz Cohen <pizzaz93@users.noreply.github.com> Co-authored-by: tudoulei <34886368+tudoulei@users.noreply.github.com> Co-authored-by: chocosaj <chocosaj@users.noreply.github.com> Co-authored-by: BuildTools <unconfigured@null.spigotmc.org> Co-authored-by: Yonghye Kwon <developer.0hye@gmail.com> Co-authored-by: Sam_S <SamSamhuns@users.noreply.github.com> Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai> Co-authored-by: edificewang <609552430@qq.com>
3 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
Merge `develop` branch into `master` (#3518) * update ci-testing.yml (#3322) * update ci-testing.yml * update greetings.yml * bring back os matrix * update ci-testing.yml (#3322) * update ci-testing.yml * update greetings.yml * bring back os matrix * Enable direct `--weights URL` definition (#3373) * Enable direct `--weights URL` definition @KalenMike this PR will enable direct --weights URL definition. Example use case: ``` python train.py --weights https://storage.googleapis.com/bucket/dir/model.pt ``` * cleanup * bug fixes * weights = attempt_download(weights) * Update experimental.py * Update hubconf.py * return bug fix * comment mirror * min_bytes * Update tutorial.ipynb (#3368) add Open in Kaggle badge * `cv2.imread(img, -1)` for IMREAD_UNCHANGED (#3379) * Update datasets.py * comment Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * COCO evolution fix (#3388) * COCO evolution fix * cleanup * update print * print fix * Create `is_pip()` function (#3391) Returns `True` if file is part of pip package. Useful for contextual behavior modification. ```python def is_pip(): # Is file in a pip package? return 'site-packages' in Path(__file__).absolute().parts ``` * Revert "`cv2.imread(img, -1)` for IMREAD_UNCHANGED (#3379)" (#3395) This reverts commit 21a9607e00f1365b21d8c4bd81bdbf5fc0efea24. * Update FLOPs description (#3422) * Update README.md * Changing FLOPS to FLOPs. Co-authored-by: BuildTools <unconfigured@null.spigotmc.org> * Parse URL authentication (#3424) * Parse URL authentication * urllib.parse.unquote() * improved error handling * improved error handling * remove %3F * update check_file() * Add FLOPs title to table (#3453) * Suppress jit trace warning + graph once (#3454) * Suppress jit trace warning + graph once Suppress harmless jit trace warning on TensorBoard add_graph call. Also fix multiple add_graph() calls bug, now only on batch 0. * Update train.py * Update MixUp augmentation `alpha=beta=32.0` (#3455) Per VOC empirical results https://github.com/ultralytics/yolov5/issues/3380#issuecomment-853001307 by @developer0hye * Add `timeout()` class (#3460) * Add `timeout()` class * rearrange order * Faster HSV augmentation (#3462) remove datatype conversion process that can be skipped * Add `check_git_status()` 5 second timeout (#3464) * Add check_git_status() 5 second timeout This should prevent the SSH Git bug that we were discussing @KalenMike * cleanup * replace timeout with check_output built-in timeout * Improved `check_requirements()` offline-handling (#3466) Improve robustness of `check_requirements()` function to offline environments (do not attempt pip installs when offline). * Add `output_names` argument for ONNX export with dynamic axes (#3456) * Add output names & dynamic axes for onnx export Add output_names and dynamic_axes names for all outputs in torch.onnx.export. The first four outputs of the model will have names output0, output1, output2, output3 * use first output only + cleanup Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Revert FP16 `test.py` and `detect.py` inference to FP32 default (#3423) * fixed inference bug ,while use half precision * replace --use-half with --half * replace space and PEP8 in detect.py * PEP8 detect.py * update --half help comment * Update test.py * revert space Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add additional links/resources to stale.yml message (#3467) * Update stale.yml * cleanup * Update stale.yml * reformat * Update stale.yml HUB URL (#3468) * Stale `github.actor` bug fix (#3483) * Explicit `model.eval()` call `if opt.train=False` (#3475) * call model.eval() when opt.train is False call model.eval() when opt.train is False * single-line if statement * cleanup Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * check_requirements() exclude `opencv-python` (#3495) Fix for 3rd party or contrib versions of installed OpenCV as in https://github.com/ultralytics/yolov5/issues/3494. * Earlier `assert` for cpu and half option (#3508) * early assert for cpu and half option early assert for cpu and half option * Modified comment Modified comment * Update tutorial.ipynb (#3510) * Reduce test.py results spacing (#3511) * Update README.md (#3512) * Update README.md Minor modifications * 850 width * Update greetings.yml revert greeting change as PRs will now merge to master. Co-authored-by: Piotr Skalski <SkalskiP@users.noreply.github.com> Co-authored-by: SkalskiP <piotr.skalski92@gmail.com> Co-authored-by: Peretz Cohen <pizzaz93@users.noreply.github.com> Co-authored-by: tudoulei <34886368+tudoulei@users.noreply.github.com> Co-authored-by: chocosaj <chocosaj@users.noreply.github.com> Co-authored-by: BuildTools <unconfigured@null.spigotmc.org> Co-authored-by: Yonghye Kwon <developer.0hye@gmail.com> Co-authored-by: Sam_S <SamSamhuns@users.noreply.github.com> Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai> Co-authored-by: edificewang <609552430@qq.com>
3 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
4 年之前
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356
  1. # YOLOv5 🚀 by Ultralytics, GPL-3.0 license
  2. """
  3. Validate a trained YOLOv5 model accuracy on a custom dataset
  4. Usage:
  5. $ python path/to/val.py --data coco128.yaml --weights yolov5s.pt --img 640
  6. """
  7. import argparse
  8. import json
  9. import os
  10. import sys
  11. from pathlib import Path
  12. from threading import Thread
  13. import numpy as np
  14. import torch
  15. from tqdm import tqdm
  16. FILE = Path(__file__).absolute()
  17. sys.path.append(FILE.parents[0].as_posix()) # add yolov5/ to path
  18. from models.experimental import attempt_load
  19. from utils.datasets import create_dataloader
  20. from utils.general import coco80_to_coco91_class, check_dataset, check_img_size, check_requirements, \
  21. check_suffix, check_yaml, box_iou, non_max_suppression, scale_coords, xyxy2xywh, xywh2xyxy, set_logging, \
  22. increment_path, colorstr
  23. from utils.metrics import ap_per_class, ConfusionMatrix
  24. from utils.plots import plot_images, output_to_target, plot_study_txt
  25. from utils.torch_utils import select_device, time_sync
  26. from utils.callbacks import Callbacks
  27. def save_one_txt(predn, save_conf, shape, file):
  28. # Save one txt result
  29. gn = torch.tensor(shape)[[1, 0, 1, 0]] # normalization gain whwh
  30. for *xyxy, conf, cls in predn.tolist():
  31. xywh = (xyxy2xywh(torch.tensor(xyxy).view(1, 4)) / gn).view(-1).tolist() # normalized xywh
  32. line = (cls, *xywh, conf) if save_conf else (cls, *xywh) # label format
  33. with open(file, 'a') as f:
  34. f.write(('%g ' * len(line)).rstrip() % line + '\n')
  35. def save_one_json(predn, jdict, path, class_map):
  36. # Save one JSON result {"image_id": 42, "category_id": 18, "bbox": [258.15, 41.29, 348.26, 243.78], "score": 0.236}
  37. image_id = int(path.stem) if path.stem.isnumeric() else path.stem
  38. box = xyxy2xywh(predn[:, :4]) # xywh
  39. box[:, :2] -= box[:, 2:] / 2 # xy center to top-left corner
  40. for p, b in zip(predn.tolist(), box.tolist()):
  41. jdict.append({'image_id': image_id,
  42. 'category_id': class_map[int(p[5])],
  43. 'bbox': [round(x, 3) for x in b],
  44. 'score': round(p[4], 5)})
  45. def process_batch(detections, labels, iouv):
  46. """
  47. Return correct predictions matrix. Both sets of boxes are in (x1, y1, x2, y2) format.
  48. Arguments:
  49. detections (Array[N, 6]), x1, y1, x2, y2, conf, class
  50. labels (Array[M, 5]), class, x1, y1, x2, y2
  51. Returns:
  52. correct (Array[N, 10]), for 10 IoU levels
  53. """
  54. correct = torch.zeros(detections.shape[0], iouv.shape[0], dtype=torch.bool, device=iouv.device)
  55. iou = box_iou(labels[:, 1:], detections[:, :4])
  56. x = torch.where((iou >= iouv[0]) & (labels[:, 0:1] == detections[:, 5])) # IoU above threshold and classes match
  57. if x[0].shape[0]:
  58. matches = torch.cat((torch.stack(x, 1), iou[x[0], x[1]][:, None]), 1).cpu().numpy() # [label, detection, iou]
  59. if x[0].shape[0] > 1:
  60. matches = matches[matches[:, 2].argsort()[::-1]]
  61. matches = matches[np.unique(matches[:, 1], return_index=True)[1]]
  62. # matches = matches[matches[:, 2].argsort()[::-1]]
  63. matches = matches[np.unique(matches[:, 0], return_index=True)[1]]
  64. matches = torch.Tensor(matches).to(iouv.device)
  65. correct[matches[:, 1].long()] = matches[:, 2:3] >= iouv
  66. return correct
  67. @torch.no_grad()
  68. def run(data,
  69. weights=None, # model.pt path(s)
  70. batch_size=32, # batch size
  71. imgsz=640, # inference size (pixels)
  72. conf_thres=0.001, # confidence threshold
  73. iou_thres=0.6, # NMS IoU threshold
  74. task='val', # train, val, test, speed or study
  75. device='', # cuda device, i.e. 0 or 0,1,2,3 or cpu
  76. single_cls=False, # treat as single-class dataset
  77. augment=False, # augmented inference
  78. verbose=False, # verbose output
  79. save_txt=False, # save results to *.txt
  80. save_hybrid=False, # save label+prediction hybrid results to *.txt
  81. save_conf=False, # save confidences in --save-txt labels
  82. save_json=False, # save a COCO-JSON results file
  83. project='runs/val', # save to project/name
  84. name='exp', # save to project/name
  85. exist_ok=False, # existing project/name ok, do not increment
  86. half=True, # use FP16 half-precision inference
  87. model=None,
  88. dataloader=None,
  89. save_dir=Path(''),
  90. plots=True,
  91. callbacks=Callbacks(),
  92. compute_loss=None,
  93. ):
  94. # Initialize/load model and set device
  95. training = model is not None
  96. if training: # called by train.py
  97. device = next(model.parameters()).device # get model device
  98. else: # called directly
  99. device = select_device(device, batch_size=batch_size)
  100. # Directories
  101. save_dir = increment_path(Path(project) / name, exist_ok=exist_ok) # increment run
  102. (save_dir / 'labels' if save_txt else save_dir).mkdir(parents=True, exist_ok=True) # make dir
  103. # Load model
  104. check_suffix(weights, '.pt')
  105. model = attempt_load(weights, map_location=device) # load FP32 model
  106. gs = max(int(model.stride.max()), 32) # grid size (max stride)
  107. imgsz = check_img_size(imgsz, s=gs) # check image size
  108. # Multi-GPU disabled, incompatible with .half() https://github.com/ultralytics/yolov5/issues/99
  109. # if device.type != 'cpu' and torch.cuda.device_count() > 1:
  110. # model = nn.DataParallel(model)
  111. # Data
  112. data = check_dataset(data) # check
  113. # Half
  114. half &= device.type != 'cpu' # half precision only supported on CUDA
  115. if half:
  116. model.half()
  117. # Configure
  118. model.eval()
  119. is_coco = isinstance(data.get('val'), str) and data['val'].endswith('coco/val2017.txt') # COCO dataset
  120. nc = 1 if single_cls else int(data['nc']) # number of classes
  121. iouv = torch.linspace(0.5, 0.95, 10).to(device) # iou vector for mAP@0.5:0.95
  122. niou = iouv.numel()
  123. # Dataloader
  124. if not training:
  125. if device.type != 'cpu':
  126. model(torch.zeros(1, 3, imgsz, imgsz).to(device).type_as(next(model.parameters()))) # run once
  127. task = task if task in ('train', 'val', 'test') else 'val' # path to train/val/test images
  128. dataloader = create_dataloader(data[task], imgsz, batch_size, gs, single_cls, pad=0.5, rect=True,
  129. prefix=colorstr(f'{task}: '))[0]
  130. seen = 0
  131. confusion_matrix = ConfusionMatrix(nc=nc)
  132. names = {k: v for k, v in enumerate(model.names if hasattr(model, 'names') else model.module.names)}
  133. class_map = coco80_to_coco91_class() if is_coco else list(range(1000))
  134. s = ('%20s' + '%11s' * 6) % ('Class', 'Images', 'Labels', 'P', 'R', 'mAP@.5', 'mAP@.5:.95')
  135. p, r, f1, mp, mr, map50, map, t0, t1, t2 = 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.
  136. loss = torch.zeros(3, device=device)
  137. jdict, stats, ap, ap_class = [], [], [], []
  138. for batch_i, (img, targets, paths, shapes) in enumerate(tqdm(dataloader, desc=s)):
  139. t_ = time_sync()
  140. img = img.to(device, non_blocking=True)
  141. img = img.half() if half else img.float() # uint8 to fp16/32
  142. img /= 255.0 # 0 - 255 to 0.0 - 1.0
  143. targets = targets.to(device)
  144. nb, _, height, width = img.shape # batch size, channels, height, width
  145. t = time_sync()
  146. t0 += t - t_
  147. # Run model
  148. out, train_out = model(img, augment=augment) # inference and training outputs
  149. t1 += time_sync() - t
  150. # Compute loss
  151. if compute_loss:
  152. loss += compute_loss([x.float() for x in train_out], targets)[1] # box, obj, cls
  153. # Run NMS
  154. targets[:, 2:] *= torch.Tensor([width, height, width, height]).to(device) # to pixels
  155. lb = [targets[targets[:, 0] == i, 1:] for i in range(nb)] if save_hybrid else [] # for autolabelling
  156. t = time_sync()
  157. out = non_max_suppression(out, conf_thres, iou_thres, labels=lb, multi_label=True, agnostic=single_cls)
  158. t2 += time_sync() - t
  159. # Statistics per image
  160. for si, pred in enumerate(out):
  161. labels = targets[targets[:, 0] == si, 1:]
  162. nl = len(labels)
  163. tcls = labels[:, 0].tolist() if nl else [] # target class
  164. path, shape = Path(paths[si]), shapes[si][0]
  165. seen += 1
  166. if len(pred) == 0:
  167. if nl:
  168. stats.append((torch.zeros(0, niou, dtype=torch.bool), torch.Tensor(), torch.Tensor(), tcls))
  169. continue
  170. # Predictions
  171. if single_cls:
  172. pred[:, 5] = 0
  173. predn = pred.clone()
  174. scale_coords(img[si].shape[1:], predn[:, :4], shape, shapes[si][1]) # native-space pred
  175. # Evaluate
  176. if nl:
  177. tbox = xywh2xyxy(labels[:, 1:5]) # target boxes
  178. scale_coords(img[si].shape[1:], tbox, shape, shapes[si][1]) # native-space labels
  179. labelsn = torch.cat((labels[:, 0:1], tbox), 1) # native-space labels
  180. correct = process_batch(predn, labelsn, iouv)
  181. if plots:
  182. confusion_matrix.process_batch(predn, labelsn)
  183. else:
  184. correct = torch.zeros(pred.shape[0], niou, dtype=torch.bool)
  185. stats.append((correct.cpu(), pred[:, 4].cpu(), pred[:, 5].cpu(), tcls)) # (correct, conf, pcls, tcls)
  186. # Save/log
  187. if save_txt:
  188. save_one_txt(predn, save_conf, shape, file=save_dir / 'labels' / (path.stem + '.txt'))
  189. if save_json:
  190. save_one_json(predn, jdict, path, class_map) # append to COCO-JSON dictionary
  191. callbacks.run('on_val_image_end', pred, predn, path, names, img[si])
  192. # Plot images
  193. if plots and batch_i < 3:
  194. f = save_dir / f'val_batch{batch_i}_labels.jpg' # labels
  195. Thread(target=plot_images, args=(img, targets, paths, f, names), daemon=True).start()
  196. f = save_dir / f'val_batch{batch_i}_pred.jpg' # predictions
  197. Thread(target=plot_images, args=(img, output_to_target(out), paths, f, names), daemon=True).start()
  198. # Compute statistics
  199. stats = [np.concatenate(x, 0) for x in zip(*stats)] # to numpy
  200. if len(stats) and stats[0].any():
  201. p, r, ap, f1, ap_class = ap_per_class(*stats, plot=plots, save_dir=save_dir, names=names)
  202. ap50, ap = ap[:, 0], ap.mean(1) # AP@0.5, AP@0.5:0.95
  203. mp, mr, map50, map = p.mean(), r.mean(), ap50.mean(), ap.mean()
  204. nt = np.bincount(stats[3].astype(np.int64), minlength=nc) # number of targets per class
  205. else:
  206. nt = torch.zeros(1)
  207. # Print results
  208. pf = '%20s' + '%11i' * 2 + '%11.3g' * 4 # print format
  209. print(pf % ('all', seen, nt.sum(), mp, mr, map50, map))
  210. # Print results per class
  211. if (verbose or (nc < 50 and not training)) and nc > 1 and len(stats):
  212. for i, c in enumerate(ap_class):
  213. print(pf % (names[c], seen, nt[c], p[i], r[i], ap50[i], ap[i]))
  214. # Print speeds
  215. t = tuple(x / seen * 1E3 for x in (t0, t1, t2)) # speeds per image
  216. if not training:
  217. shape = (batch_size, 3, imgsz, imgsz)
  218. print(f'Speed: %.1fms pre-process, %.1fms inference, %.1fms NMS per image at shape {shape}' % t)
  219. # Plots
  220. if plots:
  221. confusion_matrix.plot(save_dir=save_dir, names=list(names.values()))
  222. callbacks.run('on_val_end')
  223. # Save JSON
  224. if save_json and len(jdict):
  225. w = Path(weights[0] if isinstance(weights, list) else weights).stem if weights is not None else '' # weights
  226. anno_json = str(Path(data.get('path', '../coco')) / 'annotations/instances_val2017.json') # annotations json
  227. pred_json = str(save_dir / f"{w}_predictions.json") # predictions json
  228. print(f'\nEvaluating pycocotools mAP... saving {pred_json}...')
  229. with open(pred_json, 'w') as f:
  230. json.dump(jdict, f)
  231. try: # https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocoEvalDemo.ipynb
  232. check_requirements(['pycocotools'])
  233. from pycocotools.coco import COCO
  234. from pycocotools.cocoeval import COCOeval
  235. anno = COCO(anno_json) # init annotations api
  236. pred = anno.loadRes(pred_json) # init predictions api
  237. eval = COCOeval(anno, pred, 'bbox')
  238. if is_coco:
  239. eval.params.imgIds = [int(Path(x).stem) for x in dataloader.dataset.img_files] # image IDs to evaluate
  240. eval.evaluate()
  241. eval.accumulate()
  242. eval.summarize()
  243. map, map50 = eval.stats[:2] # update results (mAP@0.5:0.95, mAP@0.5)
  244. except Exception as e:
  245. print(f'pycocotools unable to run: {e}')
  246. # Return results
  247. model.float() # for training
  248. if not training:
  249. s = f"\n{len(list(save_dir.glob('labels/*.txt')))} labels saved to {save_dir / 'labels'}" if save_txt else ''
  250. print(f"Results saved to {colorstr('bold', save_dir)}{s}")
  251. maps = np.zeros(nc) + map
  252. for i, c in enumerate(ap_class):
  253. maps[c] = ap[i]
  254. return (mp, mr, map50, map, *(loss.cpu() / len(dataloader)).tolist()), maps, t
  255. def parse_opt():
  256. parser = argparse.ArgumentParser(prog='val.py')
  257. parser.add_argument('--data', type=str, default='data/coco128.yaml', help='dataset.yaml path')
  258. parser.add_argument('--weights', nargs='+', type=str, default='yolov5s.pt', help='model.pt path(s)')
  259. parser.add_argument('--batch-size', type=int, default=32, help='batch size')
  260. parser.add_argument('--imgsz', '--img', '--img-size', type=int, default=640, help='inference size (pixels)')
  261. parser.add_argument('--conf-thres', type=float, default=0.001, help='confidence threshold')
  262. parser.add_argument('--iou-thres', type=float, default=0.6, help='NMS IoU threshold')
  263. parser.add_argument('--task', default='val', help='train, val, test, speed or study')
  264. parser.add_argument('--device', default='', help='cuda device, i.e. 0 or 0,1,2,3 or cpu')
  265. parser.add_argument('--single-cls', action='store_true', help='treat as single-class dataset')
  266. parser.add_argument('--augment', action='store_true', help='augmented inference')
  267. parser.add_argument('--verbose', action='store_true', help='report mAP by class')
  268. parser.add_argument('--save-txt', action='store_true', help='save results to *.txt')
  269. parser.add_argument('--save-hybrid', action='store_true', help='save label+prediction hybrid results to *.txt')
  270. parser.add_argument('--save-conf', action='store_true', help='save confidences in --save-txt labels')
  271. parser.add_argument('--save-json', action='store_true', help='save a COCO-JSON results file')
  272. parser.add_argument('--project', default='runs/val', help='save to project/name')
  273. parser.add_argument('--name', default='exp', help='save to project/name')
  274. parser.add_argument('--exist-ok', action='store_true', help='existing project/name ok, do not increment')
  275. parser.add_argument('--half', action='store_true', help='use FP16 half-precision inference')
  276. opt = parser.parse_args()
  277. opt.save_json |= opt.data.endswith('coco.yaml')
  278. opt.save_txt |= opt.save_hybrid
  279. opt.data = check_yaml(opt.data) # check YAML
  280. return opt
  281. def main(opt):
  282. set_logging()
  283. print(colorstr('val: ') + ', '.join(f'{k}={v}' for k, v in vars(opt).items()))
  284. check_requirements(requirements=FILE.parent / 'requirements.txt', exclude=('tensorboard', 'thop'))
  285. if opt.task in ('train', 'val', 'test'): # run normally
  286. run(**vars(opt))
  287. elif opt.task == 'speed': # speed benchmarks
  288. for w in opt.weights if isinstance(opt.weights, list) else [opt.weights]:
  289. run(opt.data, weights=w, batch_size=opt.batch_size, imgsz=opt.imgsz, conf_thres=.25, iou_thres=.45,
  290. save_json=False, plots=False)
  291. elif opt.task == 'study': # run over a range of settings and save/plot
  292. # python val.py --task study --data coco.yaml --iou 0.7 --weights yolov5s.pt yolov5m.pt yolov5l.pt yolov5x.pt
  293. x = list(range(256, 1536 + 128, 128)) # x axis (image sizes)
  294. for w in opt.weights if isinstance(opt.weights, list) else [opt.weights]:
  295. f = f'study_{Path(opt.data).stem}_{Path(w).stem}.txt' # filename to save to
  296. y = [] # y axis
  297. for i in x: # img-size
  298. print(f'\nRunning {f} point {i}...')
  299. r, _, t = run(opt.data, weights=w, batch_size=opt.batch_size, imgsz=i, conf_thres=opt.conf_thres,
  300. iou_thres=opt.iou_thres, save_json=opt.save_json, plots=False)
  301. y.append(r + t) # results and times
  302. np.savetxt(f, y, fmt='%10.4g') # save
  303. os.system('zip -r study.zip study_*.txt')
  304. plot_study_txt(x=x) # plot
  305. if __name__ == "__main__":
  306. opt = parse_opt()
  307. main(opt)