Commit Graph

367 Commits

Author SHA1 Message Date
Glenn Jocher 7b833e37bf
LoadStreams() frame loss bug fix (#2222) 2021-02-15 11:02:20 -08:00
Glenn Jocher bdd88e1ed7
YOLOv5 Segmentation Dataloader Updates (#2188)
* Update C3 module

* Update C3 module

* Update C3 module

* Update C3 module

* update

* update

* update

* update

* update

* update

* update

* update

* update

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update datasets

* update

* update

* update

* update attempt_downlaod()

* merge

* merge

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* parameterize eps

* comments

* gs-multiple

* update

* max_nms implemented

* Create one_cycle() function

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* GitHub API rate limit fix

* update

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* astuple

* epochs

* update

* update

* ComputeLoss()

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* merge

* merge

* merge

* merge

* update

* update

* update

* update

* commit=tag == tags[-1]

* Update cudnn.benchmark

* update

* update

* update

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* update

* mosaic9

* update

* update

* update

* update

* update

* update

* institute cache versioning

* only display on existing cache

* reverse cache exists booleans
2021-02-11 21:22:45 -08:00
NanoCode012 a5d5f9262d
Changed socket port and added timeout (#2176) 2021-02-10 13:01:48 -08:00
Transigent c9bda112ae
Update utils/datasets.py to support .webp files (#2174)
Simply added 'webp' as an image format to the img_formats array so that webp image files can be used as training data.
2021-02-10 00:16:38 -08:00
Glenn Jocher 86897e3663
Update train.py test batch_size (#2148)
* Update train.py

* Update loss.py
2021-02-06 10:29:32 -08:00
train255 e9b3de463a
Unique *.cache filenames fix (#2134)
* fix #2121

* Update test.py

* Update train.py

* Update autoanchor.py

* Update datasets.py

* Update log_dataset.py

* Update datasets.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-02-05 11:12:43 -08:00
Glenn Jocher 4bdc5a397e
LoadImages() pathlib update (#2140) 2021-02-05 09:06:23 -08:00
Ayush Chaurasia 73a0669930
Start setup for improved W&B integration (#1948)
* Add helper functions for wandb and artifacts

* cleanup

* Reorganize files

* Update wandb_utils.py

* Update log_dataset.py

We can remove this code, as the giou hyp has been deprecated for a while now.

* Reorganize and update dataloader call

* yaml.SafeLoader

* PEP8 reformat

* remove redundant checks

* Add helper functions for wandb and artifacts

* cleanup

* Reorganize files

* Update wandb_utils.py

* Update log_dataset.py

We can remove this code, as the giou hyp has been deprecated for a while now.

* Reorganize and update dataloader call

* yaml.SafeLoader

* PEP8 reformat

* remove redundant checks

* Update util files

* Update wandb_utils.py

* Remove word size

* Change path of labels.zip

* remove unused imports

* remove --rect

* log_dataset.py cleanup

* log_dataset.py cleanup2

* wandb_utils.py cleanup

* remove redundant id_count

* wandb_utils.py cleanup2

* rename cls

* use pathlib for zip

* rename dataloader to dataset

* Change import order

* Remove redundant code

* remove unused import

* remove unused imports

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-02-01 21:38:41 -08:00
Glenn Jocher 9646ca438a
Update plot_study() (#2112) 2021-02-01 13:51:11 -08:00
Glenn Jocher be9edffded
Update test.py --task speed and study (#2099)
* Add --speed benchmark

* test range 256 - 1536

* update

* update

* update

* update
2021-01-31 13:58:49 -08:00
Glenn Jocher a18efc3a73
Add variable-stride inference support (#2091) 2021-01-30 13:47:23 -08:00
Glenn Jocher 630ec06af6
Update to colors.TABLEAU_COLORS (#2069) 2021-01-28 12:37:21 -08:00
Glenn Jocher f639e14e4d
Metric-Confidence plots feature addition (#2057)
* Metric-Confidence plots feature addition

* cleanup

* Metric-Confidence plots feature addition

* cleanup

* Update run-once lines

* cleanup

* save all 4 curves to wandb
2021-01-27 17:10:53 -08:00
Glenn Jocher f59f80114c
Add 'exclude' tuple to check_requirements() (#2041) 2021-01-26 21:17:36 -08:00
Glenn Jocher 59c21c7bcb
W&B log epoch (#1946)
* W&B log epoch

* capitalize

* W&B log epoch

* capitalize

* Update train.py

New try using https://docs.wandb.ai/library/log#incremental-logging

* Update train.py

* Update test.py

* Update train.py

* Update plots.py

* Update train.py

* Update train.py

* label plot step -1

* update

* update

* update

* update

* update

* update

* Update train.py

* Update train.py
2021-01-26 21:16:01 -08:00
Glenn Jocher 08d3119e09
Add histogram equalization fcn (#2049) 2021-01-26 17:30:42 -08:00
Glenn Jocher 8fcbe43423
Check im.format during dataset caching (#2042)
* Check im.format during dataset caching

* Update datasets.py
2021-01-25 20:55:35 -08:00
Glenn Jocher 3b7feeafdf
Update general.py check_git_status() fix (#2020) 2021-01-22 15:59:01 -08:00
Glenn Jocher aac33f87ad
Update inference multiple-counting (#2019)
* Update inference multiple-counting

* update github check
2021-01-22 15:39:08 -08:00
Glenn Jocher 77fd83225f
Update google_utils.py (#2017) 2021-01-22 15:08:42 -08:00
Glenn Jocher 1445ab2b27
check_git_status() Windows fix (#2015)
* check_git_status() Windows fix

* Update general.py

* Update general.py

* Update general.py

* Update general.py

* Update general.py

* Update general.py
2021-01-22 12:37:12 -08:00
Glenn Jocher d9212140b3
Add xywhn2xyxy() (#1983) 2021-01-19 13:33:52 -08:00
Glenn Jocher b1cf25dd9a
check_git_status() asserts (#1977) 2021-01-18 10:49:28 -08:00
Abhiram V 17751b9891
Update autoanchor.py with yaml.SafeLoader (#1971) 2021-01-18 10:47:42 -08:00
Abhiram V 35400dc7b7
Update plots.py with yaml.SafeLoader (#1968) 2021-01-18 10:46:20 -08:00
Glenn Jocher b26a2f6242
check_git_status() when not exist /workspace (#1966) 2021-01-17 11:55:25 -08:00
Glenn Jocher e8a41e8916
prevent check_git_status() in docker images (#1951)
* prevent check_git_status() running docker images

* Update general.py
2021-01-16 01:15:38 -08:00
Glenn Jocher ca9babb8e6
Add ComputeLoss() class (#1950) 2021-01-15 13:50:24 -08:00
Glenn Jocher b75c432ea0
GitHub API rate limit fallback (#1930) 2021-01-13 21:09:50 -08:00
Glenn Jocher 051e9e80de
check_git_status() bug fix (#1925) 2021-01-13 10:25:53 -08:00
Glenn Jocher 509dd51aca
check_git_status() improvements (#1916)
* check_online()

* Update general.py

* update check_git_status()

* reverse rev-parse order

* fetch

* improved responsiveness

* comment

* comment

* remove hyp['giou'] compat warning
2021-01-12 21:51:49 -08:00
Glenn Jocher dd03b20ba5
colorstr() updates (#1909)
* W&B ImportError message fix

* colorstr() updates

* colorstr() updates

* colorstr() default to 'blue', 'bold'

* train: magenta

* train: blue
2021-01-12 10:33:15 -08:00
Glenn Jocher 1d1c0567a4
PyTorch Hub results.render() (#1897) 2021-01-10 20:06:11 -08:00
Glenn Jocher ffef77124e
Update torch_utils.py (#1895) 2021-01-10 12:26:29 -08:00
Glenn Jocher 0f11aaf551
GitHub API rate limit newline (#1894) 2021-01-10 11:47:00 -08:00
Glenn Jocher d06ad3bc3d
GitHub API rate limit fix (#1894) 2021-01-10 11:42:45 -08:00
Glenn Jocher 6ab589583c
Add colorstr() (#1887)
* Add colorful()

* update

* newline fix

* add git description

* --always

* update loss scaling

* update loss scaling 2

* rename to colorstr()
2021-01-09 15:24:18 -08:00
Glenn Jocher fea9c9b80c
nn.SiLU() citation correction (#1713) 2021-01-06 20:58:41 -08:00
Glenn Jocher e77c77f580
Add check_requirements() (#1853)
* Add check_requirements()

* add import

* parameterize filename

* add to detect, test
2021-01-06 16:35:40 -08:00
Tommy in Tongji 135ec5c5ce
W&B ID reset on training completion (#1852)
* Update train.py

Fix the bug of always the same W&B ID  and continue  overwrite with the old logging.
BUG report
https://github.com/ultralytics/yolov5/issues/1851

* Fix the bug of duplicate W&B ID

fix the bug of https://github.com/ultralytics/yolov5/issues/1851
If we had trained on yolov5s.pt, the program will generate a new unique W&B ID.
If we hadn't, the program will keep the old code, we can still use --resume aug.

* Update general.py

* revert train.py changes

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-01-06 16:27:22 -08:00
Glenn Jocher 69be8e738f
YOLOv5 v4.0 Release (#1837)
* Update C3 module

* Update C3 module

* Update C3 module

* Update C3 module

* update

* update

* update

* update

* update

* update

* update

* update

* update

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update datasets

* update

* update

* update

* update attempt_downlaod()

* merge

* merge

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* parameterize eps

* comments

* gs-multiple

* update

* max_nms implemented

* Create one_cycle() function

* update

* update

* update

* update

* update

* update

* update

* update study.png

* update study.png

* Update datasets.py
2021-01-04 19:54:09 -08:00
Glenn Jocher 0e341c5660
Create one_cycle() function (#1836) 2021-01-04 15:49:08 -08:00
Glenn Jocher 9f5a18bb80
Torch CUDA synchronize update (#1826)
* torch.cuda.synchronize() update

* torch.cuda.synchronize() update

* torch.cuda.synchronize() update

* newline
2021-01-03 11:23:12 -08:00
Glenn Jocher c0ffcdf998
Display correct CUDA devices (#1776)
* Display correct CUDA devices

* cleanup
2020-12-24 13:01:35 -08:00
Glenn Jocher 3004fb5bc1
Automatic m.half() profile on x.half() 2020-12-21 15:20:33 -08:00
Glenn Jocher 0bd9c48609
Update torch_utils.py
FLOPS to GFLOPS
2020-12-21 13:29:52 -08:00
Glenn Jocher 394d1c89f3
Input channel yaml['ch'] addition (#1741) 2020-12-19 10:54:01 -08:00
Glenn Jocher 685d601308
Increase plot_labels() speed (#1736) 2020-12-18 18:05:38 -08:00
Glenn Jocher 49abc722fc
Update profile_idetection() (#1727) 2020-12-18 01:19:17 -08:00
Glenn Jocher d5289b54c4
clean_str() function addition (#1674)
* clean_str() function addition

* cleanup

* add euro symbol €

* add closing exclamation (spanish)

* cleanup
2020-12-17 17:20:20 -08:00
Glenn Jocher 6bd5e8bca7
nn.SiLU() export support (#1713) 2020-12-16 17:55:57 -08:00
Polydefkis Gkagkos 1fc9d42a64
NMS --classes 0 bug fix (#1710) 2020-12-16 08:58:51 -08:00
Glenn Jocher 8bc0027afc
Update loss criteria constructor (#1711) 2020-12-16 08:39:35 -08:00
NanoCode012 035ac82ed0
Fix torch multi-GPU --device error (#1701)
* Fix torch GPU error

* Update torch_utils.py

single-line device =

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-12-15 20:42:14 -08:00
Glenn Jocher 69ea70cd3b
Add idetection_profile() function to plots.py (#1700) 2020-12-15 18:35:47 -08:00
Glenn Jocher 54043a9fa4
Streaming --save-txt bug fix (#1672)
* Streaming --save-txt bug fix

* cleanup
2020-12-11 15:45:32 -08:00
Glenn Jocher ada90e3901
Profile() feature addition (#1673)
* Profile() feature addition

* cleanup
2020-12-11 09:30:39 -08:00
Glenn Jocher 94a7f55c4e
FReLU bias=False bug fix (#1666) 2020-12-10 13:06:15 -08:00
Glenn Jocher 2e8e02745b
vast.ai compatability updates (#1657) 2020-12-09 19:01:08 -08:00
Glenn Jocher fa8f1fb0e9
Simplify autoshape() post-process (#1653)
* Simplify autoshape() post-process

* cleanup

* cleanup
2020-12-09 07:44:06 -08:00
Glenn Jocher 84f9bb5d92
Normalized mosaic plotting bug fix (#1647) 2020-12-08 18:44:13 -08:00
Glenn Jocher 0bb43953eb
Reinstate PR curve sentinel values (#1645) 2020-12-08 17:40:49 -08:00
Glenn Jocher 68e6ab668b
Hub device mismatch bug fix (#1619) 2020-12-06 17:53:38 +01:00
Glenn Jocher 791dadb51c
Pycocotools best.pt after COCO train (#1616)
* Pycocotools best.pt after COCO train

* cleanup
2020-12-06 14:58:33 +01:00
Glenn Jocher 8918e63476
Increase FLOPS robustness (#1608) 2020-12-05 11:41:34 +01:00
Glenn Jocher f010147578
Update matplotlib.use('Agg') tight (#1583)
* Update matplotlib tight_layout=True

* udpate

* udpate

* update

* png to ps

* update

* update
2020-12-02 15:53:16 +01:00
Glenn Jocher 784feae30a
Update matplotlib svg backend (#1580) 2020-12-02 14:05:12 +01:00
Hu Ye 577f298d9b
plot_images() scale bug fix (#1566)
fix bugs in plot_images
2020-12-01 11:29:59 +01:00
Glenn Jocher b6ed1104a6
Daemon thread plotting (#1561)
* Daemon thread plotting

* remove process_batch

* plot after print
2020-11-30 16:44:14 +01:00
Glenn Jocher cff9263490
f.read().strip() (#1551) 2020-11-29 11:59:52 +01:00
Glenn Jocher 9fa7f9f598 f.read().strip() 2020-11-29 11:58:14 +01:00
Glenn Jocher 96a84468b9
Update labels_to_image_weights() (#1545) 2020-11-28 12:25:45 +01:00
Glenn Jocher c9798ae0e1
Update plot_study_txt() (#1533) 2020-11-26 22:18:17 +01:00
Glenn Jocher 0f2057ed33
Targets scaling bug fix (#1529) 2020-11-26 18:33:28 +01:00
Glenn Jocher 2c3efa430b
Mosaic plots bug fix (#1526) 2020-11-26 14:02:22 +01:00
Glenn Jocher 12499f1c01 --image_weights bug fix (#1524) 2020-11-26 13:25:51 +01:00
Glenn Jocher 9728e2b8ae
--image_weights bug fix (#1524) 2020-11-26 11:49:01 +01:00
Glenn Jocher e9a0ae6f19
Cache bug fix (#1513)
* Caching bug fix #1508

* np.zeros((0,5)) x2
2020-11-25 20:33:14 +01:00
yxNONG b3ceffb513
Add QFocalLoss() (#1482)
* Update loss.py

implement the quality focal loss which is a more general case of focal loss
more detail in https://arxiv.org/abs/2006.04388 

In the obj loss (or the case cls loss with label smooth), the targets is no long barely be 0 or 1 (can be 0.7), in this case, the normal focal loss is not work accurately
quality focal loss in behave the same as focal loss when the target is equal to 0 or 1, and work accurately when targets in (0, 1)

example:

targets:
tensor([[0.6225, 0.0000, 0.0000],
        [0.9000, 0.0000, 0.0000],
        [1.0000, 0.0000, 0.0000]])
___________________________
pred_prob:
tensor([[0.6225, 0.2689, 0.1192],
        [0.7773, 0.5000, 0.2227],
        [0.8176, 0.8808, 0.1978]])
____________________________
focal_loss
tensor([[0.0937, 0.0328, 0.0039],
        [0.0166, 0.1838, 0.0199],
        [0.0039, 1.3186, 0.0145]])
______________
qfocal_loss
tensor([[7.5373e-08, 3.2768e-02, 3.9179e-03],
        [4.8601e-03, 1.8380e-01, 1.9857e-02],
        [3.9233e-03, 1.3186e+00, 1.4545e-02]])
 
we can see that targets[0][0] = 0.6255 is almost the same as pred_prob[0][0] = 0.6225, 
the targets[1][0] = 0.9 is greater then pred_prob[1][0] = 0.7773 by 0.1227
however, the focal loss[0][0] = 0.0937 larger then focal loss[1][0] = 0.0166 (which against the purpose of focal loss)

for the quality focal loss , it implement the case of targets not equal to 0 or 1

* Update loss.py
2020-11-25 19:32:27 +01:00
Glenn Jocher 2026d4c5eb Update caching (#1496) 2020-11-24 16:25:21 +01:00
Glenn Jocher bde5d9aaaa Update caching (#1496) 2020-11-24 16:23:00 +01:00
Glenn Jocher 0822cda781 Update caching (#1496) 2020-11-24 16:22:02 +01:00
Glenn Jocher 89c7a5b8dc
Update caching (#1496) 2020-11-24 16:13:04 +01:00
igornishka 44f42b1589
changed prints to logging in utils/datasets (#1315)
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-11-24 16:03:19 +01:00
Glenn Jocher 7aeef2dca5
Prevent PR plotting (#1489) 2020-11-24 00:46:01 +01:00
Glenn Jocher 354109c54c Autosplit (#1488) 2020-11-23 18:35:25 +01:00
Glenn Jocher 4798e66fdf
Autosplit (#1488) 2020-11-23 17:18:21 +01:00
Glenn Jocher 0a3ff71ae0
Confusion matrix (#1474)
* initial commit

* add plotting

* matrix to cpu

* bug fix

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* seaborn pandas to requirements.txt

* seaborn pandas to requirements.txt

* update wandb plotting

* remove pandas

* if plots

* if plots

* if plots

* if plots

* if plots

* initial commit

* add plotting

* matrix to cpu

* bug fix

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* seaborn pandas to requirements.txt

* seaborn pandas to requirements.txt

* update wandb plotting

* remove pandas

* if plots

* if plots

* if plots

* if plots

* if plots

* Cat apriori to autolabels

* cleanup
2020-11-23 15:27:14 +01:00
Glenn Jocher 95fa65339f
Cat apriori to autolabels (#1484) 2020-11-23 13:38:47 +01:00
Glenn Jocher 201bafc7cf
Sync train and test iou_thresh (#1465)
* Sync train and test iou_thresh

* Sync train and test iou_thresh

* weights names .lower()

* Notebook update
2020-11-21 12:38:35 +01:00
Glenn Jocher a70e554fc3
Remove redundant downloads mirror (#1461) 2020-11-20 13:10:56 +01:00
Glenn Jocher 394131c2aa
Use torchvision.ops.nms (#1460) 2020-11-20 11:23:36 +01:00
yujun 05a955a3f6
FLOPS computation device bug fix (#1447)
* Update torch_utils.py

fix issue#113 , inputs device should be same with model parameters' device

* Update torch_utils.py

* Update torch_utils.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-11-19 12:56:20 +01:00
Glenn Jocher 64bce3c822 Update labels.png with rectangles fix (#1432) 2020-11-18 12:27:30 +01:00
Glenn Jocher 1542ccadb2
Update labels.png with rectangles (#1432) 2020-11-18 00:17:33 +01:00
Glenn Jocher 4250f84dfb
Update PR curve (#1428)
* Update PR curve

* legend outside

* list(Path().glob())
2020-11-17 17:29:55 +01:00
Jacobsolawetz 8d2d6d2349
Update img2label_paths() (#1414) 2020-11-17 13:26:06 +01:00
Glenn Jocher f5429260ca
PyTorch Hub and autoShape update (#1415)
* PyTorch Hub and autoShape update

* comment x for imgs

* reduce comment
2020-11-16 23:09:55 +01:00
Glenn Jocher 92c9b72832
Matplotlib AGG backend (#1413) 2020-11-16 16:34:07 +01:00
Glenn Jocher 548a98a391
Recursive directories dataset capability (#1408)
* Recursive directories dataset capability

* x.split('.')[-1]

* f += glob.glob(f"{p}{os.sep}**.*", recursive=True)

* f += glob.glob(str(p / '**.*'), recursive=True)

* remove . from image and vid formats

* .txt to txt

* str(p / '**' / '*.*')
2020-11-16 16:24:57 +01:00