Commit Graph

544 Commits

Author SHA1 Message Date
Glenn Jocher 4bad914761
Update resume.py (#4115) 2021-07-22 17:22:11 +02:00
Glenn Jocher 2c073cd207
Add train.py ``--img-size` floor (#4099) 2021-07-21 16:50:47 +02:00
Ayush Chaurasia 0cc7c58787
W&B: fix refactor bugs (#4069) 2021-07-19 13:57:13 +02:00
Glenn Jocher 7fdcc77bf4
Albumentations >= 1.0.3 (#4068) 2021-07-19 13:23:19 +02:00
Glenn Jocher c8a98cb7cb
Missing `nc` and `names` handling in check_dataset() (#4066) 2021-07-19 13:10:21 +02:00
Glenn Jocher f7d8562060
`val.py` refactor (#4053)
* val.py refactor

* cleanup

* cleanup

* cleanup

* cleanup

* save after eval

* opt.imgsz bug fix

* wandb refactor

* dataloader to train_loader

* capitalize global variables

* runs/hub/exp to runs/detect/exp

* refactor wandb logging

* Refactor wandb operations (#4061)

Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com>
2021-07-19 10:43:01 +02:00
Ayush Chaurasia 62409eea08
W&B sweeps support (#3938)
* Add support for W&B Sweeps

* Update and reformat

* Update search space

* reformat

* reformat sweep.py

* Update sweep.py

* Move sweeps files to wandb dir

* Remove print

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-07-14 19:13:02 +02:00
Glenn Jocher 720aaa65c8
Rename `test.py` to `val.py` (#4000) 2021-07-14 15:43:54 +02:00
Glenn Jocher b3dabdcc38
Update `probability` to `p` (#3980) 2021-07-12 15:54:43 +02:00
Glenn Jocher 90e60b403d
Update `np.random.random()` to `random.random()` (#3967) 2021-07-11 16:56:36 +02:00
Glenn Jocher 8298ce5e88
Update `albumentations>=1.0.2` (#3966) 2021-07-11 15:49:30 +02:00
Glenn Jocher 80299a57e2
Numerical stability fix for Albumentations (#3958) 2021-07-10 19:50:53 +02:00
Glenn Jocher 443af8b25a
Cache v0.4 update (#3954) 2021-07-10 14:18:46 +02:00
Glenn Jocher a26e7de2bf
Update augmentations.py (#3948) 2021-07-09 16:45:04 +02:00
Glenn Jocher 248504cf13
Feature visualization improvements 32 (#3947) 2021-07-09 15:23:02 +02:00
Glenn Jocher 8c6f9e15bf
Update `dataset_stats()` for zipped datasets (#3926)
* Update `dataset_stats()` for zipped datasets

@KalenMike

* cleanup
2021-07-08 11:42:30 +02:00
Glenn Jocher 850970e081
`feature_visualization()` CUDA fix (#3925) 2021-07-07 16:23:31 +02:00
Glenn Jocher 87b094bcbc
Feature visualization update (#3920)
* Feature visualization update

* Save to jpg (faster)

* Save to png
2021-07-07 15:41:58 +02:00
Glenn Jocher 33202b7f0b
YOLOv5 + Albumentations integration (#3882)
* Albumentations integration

* ToGray p=0.01

* print confirmation

* create instance in dataloader init method

* improved version handling

* transform not defined fix

* assert string update

* create check_version()

* add spaces

* update class comment
2021-07-05 18:01:54 +02:00
Glenn Jocher 6a3ee7cf03
Hub models `map_location=device` (#3894)
* Hub models `map_location=device`

* cleanup
2021-07-05 16:20:46 +02:00
Glenn Jocher 3c3f8fbd5d
Improved BGR2RGB speeds (#3880)
* Update BGR2RGB ops

* speed improvements

* cleanup
2021-07-04 20:12:32 +02:00
Glenn Jocher 9e8fb9fd0b
Create `utils/augmentations.py` (#3877)
* Create `utils/augmentations.py`

* cleanup
2021-07-04 18:14:04 +02:00
ketan-b 9d86b54eb3
Add multi-stream saving feature (#3864)
* Added the recording feature for multiple streams

Thanks for the very cool repo!!
I was trying to record multiple feeds at the same time, but the current version of the detector only had one video writer and one vid_path!
So the streams were not being saved and only were initialized with one frame and this process didn't record the whole thing.

Fix:
I made a list of `vid_writer` and `vid_path` and the `i` from the loop over the `pred` took care of the writer which need to work!

I hope this helps, Thanks!

* Cleanup list lengths

* batch size variable

* Update datasets.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-07-04 12:55:57 +02:00
Valentin Aliferov 831773f5a2
Add EXIF rotation to YOLOv5 Hub inference (#3852)
* rotating an image according to its exif tag

* Update common.py

* Update datasets.py

* Update datasets.py

faster

* delete extraneous gpg file

* Update common.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-07-02 13:25:54 +02:00
Glenn Jocher c6c88dc601
Copy-Paste augmentation for YOLOv5 (#3845)
* Copy-paste augmentation initial commit

* if any segments

* Add obscuration rejection

* Add copy_paste hyperparameter

* Update comments
2021-07-01 00:35:04 +02:00
Feras Oughali 7d6af69638
Fix `LoadStreams()` dataloader frame skip issue (#3833)
* Update datasets.py to read every 4th frame of streams

* Update datasets.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-06-30 12:11:29 +02:00
Glenn Jocher 5ea771d93d
Move IoU functions to metrics.py (#3820) 2021-06-29 13:18:13 +02:00
Glenn Jocher 3213d8713f
Fix for `dataset_stats()` with updated data.yaml (#3819)
@KalenMike
2021-06-29 12:44:59 +02:00
Glenn Jocher 02719dde52
Update `feature_visualization()` (#3807)
* Update `feature_visualization()`

Only plot for data with height, width > 1

* cleanup

* Cleanup
2021-06-28 13:48:14 +02:00
Zigarss 20d45aa4f1
Add feature map visualization (#3804)
* Add feature map visualization

Add a feature_visualization function to visualize the mid feature map of the model.

* Update yolo.py

* remove boolean from forward and reorder if statement

* remove print from forward

* General cleanup

* Indent

* Update plots.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-06-28 13:18:45 +02:00
Glenn Jocher 8e7f285051
Fix datasets for aws and get_coco.sh (#3788)
* merge master

* Update get_coco.sh
2021-06-26 15:29:16 +02:00
Glenn Jocher 8035b61682
Update objectness IoU sort (#3786) 2021-06-26 14:52:18 +02:00
Glenn Jocher 157aa2f886
Objectness IoU Sort (#3610)
Co-authored-by: U-LAPTOP-5N89P8V7\banhu <ban.huang@foxmail.com>
2021-06-26 14:45:53 +02:00
Glenn Jocher f89941711c
NGA xView 2018 Dataset Auto-Download (#3775)
* update clip_coords for numpy

* uncomment

* cleanup

* Add autosplits

* fix

* cleanup
2021-06-26 00:49:05 +02:00
Ayush Chaurasia ffb6e11050
W&B: Update Tables API and comply with new dataset_check (#3772)
* Update tables API and windows path fix

* update dataset check
2021-06-25 20:45:33 +02:00
Yonghye Kwon 374957317a
Add `xyxy2xywhn()` (#3765)
* Edit Comments for numpy2torch tensor process

Edit Comments for numpy2torch tensor process

* add xyxy2xywhn

add xyxy2xywhn

* add xyxy2xywhn

* formatting

* pass arguments

pass arguments

* edit comment for xyxy2xywhn()

edit comment for xyxy2xywhn()

* cleanup datasets.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-06-25 11:47:46 +02:00
Glenn Jocher f79d7479da
Add optional dataset.yaml `path` attribute (#3753)
* Add optional dataset.yaml `path` attribute

@KalenMike

* pass locals to python scripts

* handle lists

* update coco128.yaml

* Capitalize first letter

* add test key

* finalize GlobalWheat2020.yaml

* finalize objects365.yaml

* finalize SKU-110K.yaml

* finalize SKU-110K.yaml

* finalize VisDrone.yaml

* NoneType fix

* update download comment

* voc to VOC

* update

* update VOC.yaml

* update VOC.yaml

* remove dashes

* delete get_voc.sh

* force coco and coco128 to ../datasets

* Capitalize Argoverse_HD.yaml

* Capitalize Objects365.yaml

* update Argoverse_HD.yaml

* coco segments fix

* VOC single-thread

* update Argoverse_HD.yaml

* update data_dict in test handling

* create root
2021-06-25 01:25:03 +02:00
Yonghye Kwon 417a2f425c
Edit comment (#3759)
edit comment
2021-06-24 15:57:27 +02:00
Glenn Jocher 63060910a6
Update `check_datasets()` for dynamic unzip path (#3732)
@KalenMike
2021-06-22 16:05:38 +02:00
Glenn Jocher 9ac7d388a9
Backwards compatible cache version checks (#3730) 2021-06-22 13:50:47 +02:00
Glenn Jocher b83e1a4adc
Fix `img2label_paths()` order (#3720)
* Fix `img2label_paths()` order

* fix, 1
2021-06-21 22:50:56 +02:00
Ayush Chaurasia 75c0ff43af
[x]W&B: Don't resume transfer learning runs (#3604)
* Allow config cahnge

* Allow val change in wandb config

* Don't resume transfer learning runs

* Add entity in log dataset
2021-06-21 14:00:25 +02:00
Glenn Jocher fad27c0046
Update DDP for `torch.distributed.run` with `gloo` backend (#3680)
* Update DDP for `torch.distributed.run`

* Add LOCAL_RANK

* remove opt.local_rank

* backend="gloo|nccl"

* print

* print

* debug

* debug

* os.getenv

* gloo

* gloo

* gloo

* cleanup

* fix getenv

* cleanup

* cleanup destroy

* try nccl

* return opt

* add --local_rank

* add timeout

* add init_method

* gloo

* move destroy

* move destroy

* move print(opt) under if RANK

* destroy only RANK 0

* move destroy inside train()

* restore destroy outside train()

* update print(opt)

* cleanup

* nccl

* gloo with 60 second timeout

* update namespace printing
2021-06-19 16:30:25 +02:00
Mai Thanh Minh bf209f6fe9
Skip HSV augmentation when hyperparameters are [0, 0, 0] (#3686)
* Create shortcircuit in augment_hsv when hyperparameter are zero

* implement faster opt-in

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-06-19 11:51:21 +02:00
Glenn Jocher 814806c61d
Update cache check (#3691)
Swapped order of operations for faster first per f527704cd3 (r52362419)
2021-06-19 11:22:09 +02:00
Glenn Jocher f527704cd3
Cache v0.3: improved corrupt image/label reporting (#3676)
* Cache v0.3: improved corrupt image/label reporting

Fix for https://github.com/ultralytics/yolov5/issues/3656#issuecomment-863660899

* cleanup
2021-06-18 10:21:47 +02:00
Glenn Jocher 9b6dba6207
Update `dataset_stats()` to list of dicts (#3657)
* Update `dataset_stats()` to list of dicts

@KalenMike

* Update datasets.py
2021-06-17 13:59:52 +02:00
xiaowk5516 d808855f77
Assert non-premature end of JPEG images (#3638)
* premature end of JPEG images

* PEP8 reformat

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-06-16 13:31:26 +02:00
Glenn Jocher 6c0e1d9fd7
Update `verify_image_label()` (#3635) 2021-06-16 11:12:15 +02:00
Glenn Jocher 7d3686a686
Update `check_file()` (#3622)
* Update `check_file()`

* Update datasets.py
2021-06-15 13:21:04 +02:00
Ayush Chaurasia daab682b06
Suppress wandb images size mismatch warning (#3611)
* supress wandb images size mismatch warning

* supress wandb images size mismatch warning

* PEP8 reformat and optimize imports

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-06-14 18:54:58 +02:00
Glenn Jocher 239a11c197
Download COCO and VOC by default (#3608) 2021-06-14 13:58:54 +02:00
Glenn Jocher f8adee156d merge develop 2021-06-14 10:45:15 +02:00
Glenn Jocher 7a565f130a
Update `dataset_stats()` (#3593)
@KalenMike this is a PR to add image filenames and labels to our stats dictionary and to save the dictionary to JSON. Save location is next to the train labels.cache file. The single JSON contains all stats for entire dataset.

Usage example:
```python
from utils.datasets import *

dataset_stats('coco128.yaml', verbose=True)
```
2021-06-12 13:26:41 +02:00
Ayush Chaurasia 53d4fc2e26
W&B: Allow changed in config variable #3588 2021-06-11 23:18:57 +02:00
Glenn Jocher e8c5237403
ConfusionMatrix `normalize=True` fix (#3587) 2021-06-11 11:46:05 +02:00
Glenn Jocher ec2da4a82c
Add ConfusionMatrix `normalize=True` flag (#3586) 2021-06-11 11:37:08 +02:00
Glenn Jocher 095197bd4a
Ignore Seaborn plot warnings (#3576)
* Ignore Seaborn plot warnings

* Update plots.py

* Update metrics.py
2021-06-10 13:51:29 +02:00
dependabot[bot] c6deb73a89
Bump pip from 18.1 to 19.2 in /utils/google_app_engine (#3561)
Bumps [pip](https://github.com/pypa/pip) from 18.1 to 19.2.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst)
- [Commits](https://github.com/pypa/pip/compare/18.1...19.2)

---
updated-dependencies:
- dependency-name: pip
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-09 21:14:09 +02:00
Glenn Jocher 5948f20a3d
Update test.py profiling (#3555)
* Update test.py profiling

* half_precision to half

* inplace
2021-06-09 16:25:17 +02:00
Glenn Jocher 8b5086c21b
Multi-GPU default to single device 0 (#3554)
* Multi-GPU default to single device 0

* Multi-GPU default to single device 0

* add space
2021-06-09 15:31:47 +02:00
Glenn Jocher 958ab92dc1
Remove `opt` from `create_dataloader()`` (#3552) 2021-06-09 13:14:56 +02:00
Glenn Jocher f8ec71e1c2
Update `check_python(minimum=3.6.2)` (#3548) 2021-06-09 11:34:15 +02:00
Glenn Jocher 1b5edb6f8e
Update `dataset_stats()` for HUB (#3536)
* Update `dataset_stats()` for HUB 

Cleanup of b6fdd2e

* autodownload flag

* Update general.py

* cleanup
2021-06-09 10:56:11 +02:00
Glenn Jocher b6fdd2e5e5
Create `dataset_stats()` for HUB 2021-06-08 23:09:45 +02:00
Glenn Jocher 8d52c1c5c5
Update datasets.py (#3531)
Minor updates to https://github.com/ultralytics/yolov5/pull/3505, inplace accumulation.
2021-06-08 18:36:40 +02:00
Dean Mark 28bff22df8
Use multi-threading in cache_labels (#3505)
* Use multi threading in cache_labels

* PEP8 reformat

* Add num_threads

* changed ThreadPool.imap_unordered to Pool.imap_unordered

* Remove inplace additions

* Update datasets.py

refactor initial desc

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-06-08 18:00:21 +02:00
Glenn Jocher 317f2ccc9d
Improved `check_requirements()` offline-handling (#3466)
Improve robustness of `check_requirements()` function to offline environments (do not attempt pip installs when offline).
2021-06-04 21:34:36 +02:00
Glenn Jocher 563ea9475a
Add `check_git_status()` 5 second timeout (#3464)
* Add check_git_status() 5 second timeout

This should prevent the SSH Git bug that we were discussing @KalenMike

* cleanup

* replace timeout with check_output built-in timeout
2021-06-04 21:17:36 +02:00
Yonghye Kwon c37f072ba7
Faster HSV augmentation (#3462)
remove datatype conversion process that can be skipped
2021-06-04 20:02:20 +02:00
Glenn Jocher d40481acc5
Add `timeout()` class (#3460)
* Add `timeout()` class

* rearrange order
2021-06-04 15:23:33 +02:00
Glenn Jocher 8e3b4a0bf3
Update MixUp augmentation `alpha=beta=32.0` (#3455)
Per VOC empirical results https://github.com/ultralytics/yolov5/issues/3380#issuecomment-853001307 by @developer0hye
2021-06-04 12:47:53 +02:00
Glenn Jocher f8651c388f
Parse URL authentication (#3424)
* Parse URL authentication

* urllib.parse.unquote()

* improved error handling

* improved error handling

* remove %3F

* update check_file()
2021-06-03 18:44:58 +02:00
chocosaj 3cb9ad4fc4
Update FLOPs description (#3422)
* Update README.md

* Changing FLOPS to FLOPs.

Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
2021-06-03 12:31:51 +02:00
Glenn Jocher fdbe527dc0
Revert "`cv2.imread(img, -1)` for IMREAD_UNCHANGED (#3379)" (#3395)
This reverts commit 21a9607e00.
2021-05-31 10:39:00 +02:00
Glenn Jocher d833ab3d25
Create `is_pip()` function (#3391)
Returns `True` if file is part of pip package. Useful for contextual behavior modification.

```python
def is_pip():
    # Is file in a pip package?
    return 'site-packages' in Path(__file__).absolute().parts
```
2021-05-30 20:52:42 +02:00
tudoulei 21a9607e00
`cv2.imread(img, -1)` for IMREAD_UNCHANGED (#3379)
* Update datasets.py

* comment

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-05-29 21:12:01 +02:00
Glenn Jocher ba6f3f974b
Enable direct `--weights URL` definition (#3373)
* Enable direct `--weights URL` definition

@KalenMike this PR will enable direct --weights URL definition. Example use case:
```
python train.py --weights https://storage.googleapis.com/bucket/dir/model.pt
```

* cleanup

* bug fixes

* weights = attempt_download(weights)

* Update experimental.py

* Update hubconf.py

* return bug fix

* comment mirror

* min_bytes
2021-05-28 15:18:44 +02:00
Glenn Jocher 4d4a2b0520
Ignore blank lines in `*.txt` labels (#3366)
Fix for https://github.com/ultralytics/yolov5/issues/958#issuecomment-849512083
2021-05-27 14:31:26 +02:00
Glenn Jocher 2435bfe896
Add URL download to check_file() (#3330)
* Add URL file download to check_file()

* cleanup

* pathlib bug fix
2021-05-26 15:51:49 +02:00
Glenn Jocher c6b5bfca85
Updated cache v0.2 with `hashlib` (#3350)
* Update cache v0.2 to include parent hash

Possible fix for https://github.com/ultralytics/yolov5/issues/3349

* Update datasets.py
2021-05-26 14:26:52 +02:00
Glenn Jocher aad99b63d6
TensorBoard DP/DDP graph fix (#3325) 2021-05-25 11:45:24 +02:00
Glenn Jocher 73a92dc1b6
Explicit `git clone` master (#3311) 2021-05-24 12:42:36 +02:00
Glenn Jocher 9f3a388cea
Improved check_requirements() robustness (#3298)
Add try: except clause on missing requirements install to catch install failures. Prompted by Ultralytics YOLOv5 API failure on Cython.
2021-05-23 18:00:23 +02:00
yeric1789 a039b7796d
PyTorch Hub `crops = results.crop()` return values (#3282)
* Changing save_one_box 

Made to work with other changes to common.py

* PEP8 and single line BGR

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-05-23 16:00:55 +02:00
Glenn Jocher 0e2f2cbb51
Update LoadStreams init fallbacks (#3295) 2021-05-23 14:55:42 +02:00
Charles Frye 19100ba007
Improves docs and handling of entities and resuming by WandbLogger (#3264)
* adds latest tag to match wandb defaults

* adds entity handling, 'last' tag

* fixes bug causing finished runs to resume

* removes redundant "last" tag for wandb artifact
2021-05-21 23:42:53 +02:00
Glenn Jocher 683cefead4
YouTube stream ending fix (#3277)
* YouTube stream ending fix

Properly terminates YouTube streams on video end. Should resolve issues #2769 and #3220.

* Update datasets.py
2021-05-21 16:51:07 +02:00
Glenn Jocher f3402353fb
check `batch_size % utilized_device_count` (#3276)
Bug fix to check batch_size divisibility of utilized CUDA device count vs total system CUDA device count.
2021-05-21 15:20:03 +02:00
yeric1789 7b0eb952ca
`plot_one_box()` default `color=(128, 128, 128)` (#3240)
* Color can be none by default

* `plot_one_box()` default `color=(128, 128, 128)`

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-05-19 19:47:36 +02:00
Adrian Holovaty 3f74cd9ed1
Parameterize max_det + inference default at 1000 (#3215)
* Added max_det parameters in various places

* 120 character line

* PEP8

* 120 character line

* Update inference default to 1000 instances

* Update inference default to 1000 instances

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-05-17 22:47:44 +02:00
Glenn Jocher 13a1c72699
Update datasets.py (#3216) 2021-05-17 22:24:26 +02:00
Glenn Jocher ffb47ffbeb
Silent Patches Bug Fix (#3214) 2021-05-17 17:47:30 +02:00
Glenn Jocher 36b0a9e7d0
Add CAP_PROP_FRAME_COUNT for YouTube sources (#3193) 2021-05-17 12:27:40 +02:00
Glenn Jocher ae53f50ae7
Plot labels histogram colors (#3192) 2021-05-17 11:44:22 +02:00
Glenn Jocher c1c7eb023f
Update JSON response (#3139) 2021-05-12 18:23:50 +02:00
Glenn Jocher 52c05707d6
Update Colors() (#3046)
* Update Colors()

* update colors

* update colors
2021-05-12 17:41:11 +02:00
Glenn Jocher abfcf9eb79
Replace print() with logging.info() in trainloader (#3103)
Might indirectly help #3095 by providing better visibility on source of corruption.
2021-05-10 17:07:16 +02:00
Glenn Jocher 57b0d3a605
Add check_python() (#3088)
* Add check_python()

Checks python version against minimum version of 3.7.0.

* remove packaging dependency

* refactor import
2021-05-09 15:33:38 +02:00
Ayush Chaurasia d2a17289c9
Explicitly convert artifact path to posix_path (#3067)
* Explicitly convert artifact path to posix_path

* Remove redudant str() casting
2021-05-07 19:53:56 +02:00