Commit Graph

252 Commits

Author SHA1 Message Date
Jan Hajek efa4946d15
W&B entity support (#2298)
* W&B entity support

* shorten wandb_entity to entity

Co-authored-by: Jan Hajek <jan.hajek@gmail.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-02-25 19:18:19 -08:00
Glenn Jocher 71dd2768f2
Improved model+EMA checkpointing 2 (#2295) 2021-02-24 21:03:21 -08:00
Glenn Jocher ec1d8496ba
Improved model+EMA checkpointing (#2292)
* Enhanced model+EMA checkpointing

* update

* bug fix

* bug fix 2

* always save optimizer

* ema half

* remove model.float()

* model half

* carry ema/model in fp32

* rm model.float()

* both to float always

* cleanup

* cleanup
2021-02-24 18:26:46 -08:00
Glenn Jocher ca5b10b759
Update train.py (#2290)
* Update train.py

* Update train.py

* Update train.py

* Update train.py

* Create train.py
2021-02-24 13:31:20 -08:00
Glenn Jocher e27ca0d845
Update minimum stride to 32 (#2266) 2021-02-21 21:46:42 -08:00
Glenn Jocher 6b634c6b87
Linear LR scheduler option (#2150)
* Linear LR scheduler option

* Update train.py
2021-02-06 11:26:54 -08:00
Glenn Jocher ad839eda38
Update train.py (#2149) 2021-02-06 11:21:04 -08:00
Glenn Jocher 86897e3663
Update train.py test batch_size (#2148)
* Update train.py

* Update loss.py
2021-02-06 10:29:32 -08:00
Glenn Jocher 2acbe9699a
W&B epoch logging update (#2073) 2021-01-28 18:57:24 -08:00
Glenn Jocher f639e14e4d
Metric-Confidence plots feature addition (#2057)
* Metric-Confidence plots feature addition

* cleanup

* Metric-Confidence plots feature addition

* cleanup

* Update run-once lines

* cleanup

* save all 4 curves to wandb
2021-01-27 17:10:53 -08:00
Glenn Jocher 59c21c7bcb
W&B log epoch (#1946)
* W&B log epoch

* capitalize

* W&B log epoch

* capitalize

* Update train.py

New try using https://docs.wandb.ai/library/log#incremental-logging

* Update train.py

* Update test.py

* Update train.py

* Update plots.py

* Update train.py

* Update train.py

* label plot step -1

* update

* update

* update

* update

* update

* update

* Update train.py

* Update train.py
2021-01-26 21:16:01 -08:00
Glenn Jocher 046c37e465
verbose on final_epoch (#1997) 2021-01-20 18:27:38 -08:00
Abhiram V e9941d50fa
Update train.py with yaml.SafeLoader (#1972) 2021-01-18 10:49:08 -08:00
huntr.dev | the place to protect open source b5d851d653
Security Fix for Arbitrary Code Execution - huntr.dev (#1962)
Co-authored-by: Anon-Artist <61599526+Anon-Artist@users.noreply.github.com>
Co-authored-by: Jamie Slome <jamie@418sec.com>
2021-01-17 14:10:16 -08:00
Glenn Jocher ca9babb8e6
Add ComputeLoss() class (#1950) 2021-01-15 13:50:24 -08:00
Glenn Jocher f4a78e1b40
W&B mosaic log bug fix (#1949) 2021-01-15 12:09:53 -08:00
Glenn Jocher 777032efbc
Daemon thread mosaic plots fix (#1943) 2021-01-14 13:16:48 -08:00
NanoCode012 3a56cac414
Fix batch-size on resume for multi-gpu (#1942) 2021-01-14 09:53:13 -08:00
Glenn Jocher 509dd51aca
check_git_status() improvements (#1916)
* check_online()

* Update general.py

* update check_git_status()

* reverse rev-parse order

* fetch

* improved responsiveness

* comment

* comment

* remove hyp['giou'] compat warning
2021-01-12 21:51:49 -08:00
Glenn Jocher dd03b20ba5
colorstr() updates (#1909)
* W&B ImportError message fix

* colorstr() updates

* colorstr() updates

* colorstr() default to 'blue', 'bold'

* train: magenta

* train: blue
2021-01-12 10:33:15 -08:00
Glenn Jocher 1d1c0567a4
PyTorch Hub results.render() (#1897) 2021-01-10 20:06:11 -08:00
Glenn Jocher 6ab589583c
Add colorstr() (#1887)
* Add colorful()

* update

* newline fix

* add git description

* --always

* update loss scaling

* update loss scaling 2

* rename to colorstr()
2021-01-09 15:24:18 -08:00
Glenn Jocher e77c77f580
Add check_requirements() (#1853)
* Add check_requirements()

* add import

* parameterize filename

* add to detect, test
2021-01-06 16:35:40 -08:00
Glenn Jocher 69be8e738f
YOLOv5 v4.0 Release (#1837)
* Update C3 module

* Update C3 module

* Update C3 module

* Update C3 module

* update

* update

* update

* update

* update

* update

* update

* update

* update

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update datasets

* update

* update

* update

* update attempt_downlaod()

* merge

* merge

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* parameterize eps

* comments

* gs-multiple

* update

* max_nms implemented

* Create one_cycle() function

* update

* update

* update

* update

* update

* update

* update

* update study.png

* update study.png

* Update datasets.py
2021-01-04 19:54:09 -08:00
Glenn Jocher 0e341c5660
Create one_cycle() function (#1836) 2021-01-04 15:49:08 -08:00
Glenn Jocher 7180b22e2d
DDP Multi-GPU --resume bug fix (#1810) 2020-12-30 12:40:45 -08:00
Glenn Jocher 685d601308
Increase plot_labels() speed (#1736) 2020-12-18 18:05:38 -08:00
Glenn Jocher 7e161d9774
Single class train update (#1719) 2020-12-17 12:02:03 -08:00
Glenn Jocher c923fbff90
W&B artifacts feature addition (#1712)
* Log artifacts

* cleanup
2020-12-16 17:52:12 -08:00
Glenn Jocher 8bc0027afc
Update loss criteria constructor (#1711) 2020-12-16 08:39:35 -08:00
Glenn Jocher 7947c86b57
Update COCO train postprocessing (#1702) 2020-12-15 21:50:28 -08:00
Glenn Jocher 791dadb51c
Pycocotools best.pt after COCO train (#1616)
* Pycocotools best.pt after COCO train

* cleanup
2020-12-06 14:58:33 +01:00
Glenn Jocher b6ed1104a6
Daemon thread plotting (#1561)
* Daemon thread plotting

* remove process_batch

* plot after print
2020-11-30 16:44:14 +01:00
Glenn Jocher 9728e2b8ae
--image_weights bug fix (#1524) 2020-11-26 11:49:01 +01:00
Glenn Jocher bde5d9aaaa Update caching (#1496) 2020-11-24 16:23:00 +01:00
Glenn Jocher 0822cda781 Update caching (#1496) 2020-11-24 16:22:02 +01:00
Glenn Jocher 0a3ff71ae0
Confusion matrix (#1474)
* initial commit

* add plotting

* matrix to cpu

* bug fix

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* seaborn pandas to requirements.txt

* seaborn pandas to requirements.txt

* update wandb plotting

* remove pandas

* if plots

* if plots

* if plots

* if plots

* if plots

* initial commit

* add plotting

* matrix to cpu

* bug fix

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* update plot

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* cleanup

* seaborn pandas to requirements.txt

* seaborn pandas to requirements.txt

* update wandb plotting

* remove pandas

* if plots

* if plots

* if plots

* if plots

* if plots

* Cat apriori to autolabels

* cleanup
2020-11-23 15:27:14 +01:00
asim266 9ef4760b74
Rename precision_recall_curve.png bug fix (#1431)
* fix error in train.py line 400, changed precision-recall_curve.png to precision_recall_curve.png and fix error in test.py of {wandb.init() before wandb.log()} by importing wand and initializing it before wand.log()

* Update test.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-11-17 23:47:27 +01:00
Glenn Jocher 9c91aeae10
W&B logging add hyperparameters (#1399)
* W&B logging add hyperparameters

* hyp bug fix and image logging updates

* if plots and wandb:

* cleanup

* wandb/ gitignore add

* cleanup 2

* cleanup 3

* move wandb import to top of file

* wandb evolve

* update import

* wandb.run.finish()

* default anchors: 3
2020-11-16 13:35:34 +01:00
Glenn Jocher 0c26c4e831 Increase pycocotools robustness (#1396) 2020-11-14 13:53:58 +01:00
Glenn Jocher fe341fa44d
Utils reorganization (#1392)
* Utils reorganization

* Add new utils files

* cleanup

* simplify

* reduce datasets.py

* remove evolve.sh

* loadWebcam cleanup
2020-11-14 11:50:32 +01:00
Glenn Jocher 379396e896
Yaml constructor posixpath --resume bug fix (#1390)
* resume fix for yaml constructor posixpath error

* fix update

* remove weights/ dir backup
2020-11-13 19:03:02 +01:00
Glenn Jocher c4addd7761
Unified '/project/name' results saving (#1377)
* Project/name update

* Update ci-testing.yml

* address project with path separator failure mode

* Project/name update

* address project with path separator failure mode

* Update ci-testing.yml

* detect.py default --name bug fix

* missing rstrip PR

* train/exp0 to train/exp
2020-11-12 23:37:46 +01:00
Glenn Jocher ce3872f86c
W&B evolution bug fix (#1373) 2020-11-12 12:55:52 +01:00
Khiem Doan 1c8464e199
Use pathlib instead of low-level module (#1329)
* Use pathlib instead of low-level module

* Use pathlib instead of low-level module

* Update detect.py

* Update test.py

* reformat

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-11-10 13:01:19 +01:00
Glenn Jocher 4821d076e2
Increment train, test, detect runs/ (#1322)
* Increment train, test, detect runs/

* Update ci-testing.yml

* inference/images to data/images

* move images

* runs/exp to runs/train/exp

* update 'results saved to %s' str
2020-11-08 19:39:05 +01:00
Junghoon Kim 187f7c2ed1
Change optimizer parameters group method (#1239)
* Change optimizer parameters group method

* Add torch nn

* Change isinstance method(torch.Tensor to nn.Parameter)

* parameter freeze fix, PEP8 reformat

* freeze bug fix

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-11-02 00:08:36 +01:00
Ayush Chaurasia 96fcde40b8
W&B feature improvements (#1258)
* W&B feature improvements

This PR add:
* Class to id labels. Now, the caption of bounding boxes will display the class name and the class confidence score.
* The project name is set to "Yolov5" and the run name will be set to opt.logdir

* cleanup

* remove parenthesis on caption

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-11-01 20:54:29 +01:00
Ayush Chaurasia ca290dca24
Weights & Biases (W&B) Feature Addition (#1235)
* Add wandb metric logging and bounding box debugging

* Improve formatting, readability

* Remove mutliple path for init, improve formatting

* Add wandb params

* Remove typecasting in bbox coordinates and reformat

* Cleanup

* add wandb to requirements.txt

* minor updates to test.py

* general reorg

* reduce --log-imgs to 10

* clean wandb import

* reverse wandb import assert

* add except AssertionError to try import

* move wandb init to all global ranks

* replace print() with logger.info()

* replace print() with logger.info()

* move wandb.init() bug fix

* project PosixPath to basename bug fix

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-10-31 20:16:35 +01:00
Glenn Jocher 453acdec67 Update tensorboard logging 2020-10-26 01:08:33 +01:00
Jirka Borovec c67e72200e
fix compatibility for hyper config (#1146)
* fix/hyper

* Hyp giou check to train.py

* restore general.py

* train.py overwrite fix

* restore general.py and pep8 update

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-10-15 15:05:58 +02:00
Glenn Jocher 4d3680c81d
Minor import and spelling updates (#1133) 2020-10-13 17:24:27 +02:00
Jirka Borovec 00917a6225
update expt name comment and folder parsing for training (#978)
* comment

* fix parsing

* fix evolve

* folder

* tqdm

* Update train.py

* Update train.py

* reinstate anchors into meta dict

anchor evolution is working correctly now

* reinstate logger

prefer the single line readout for concise logging, which helps simplify notebook and tutorials etc.

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-10-13 14:10:21 +02:00
Glenn Jocher 0ada058f63
Generalized regression criterion renaming (#1120) 2020-10-11 17:25:17 +02:00
Glenn Jocher 5fac5ad165
Precision-Recall Curve Feature Addition (#1107)
* initial commit

* Update general.py

Indent update

* Update general.py

refactor duplicate code

* 200 dpi
2020-10-09 14:50:59 +02:00
Glenn Jocher 66676eb039 init_torch_seeds >> init_seeds bug fix 2020-10-06 15:00:47 +02:00
Glenn Jocher f1c63e2784 add mosaic and warmup to hyperparameters (#931) 2020-09-13 14:03:54 -07:00
Glenn Jocher a62a45b2dd prevent testloader caching on --notest 2020-09-11 16:59:13 -07:00
Glenn Jocher c8e51812a5 hyp evolution force-autoanchor fix 2020-09-04 13:13:10 -07:00
Glenn Jocher c687d5c129 reorganize train initialization steps 2020-09-04 12:25:53 -07:00
Glenn Jocher 44cdcc7e0b hyp['anchors'] evolution update 2020-09-03 12:54:22 -07:00
NanoCode012 d8274d0434
Fix results_file not renaming (#903) 2020-09-03 00:47:50 -07:00
Glenn Jocher 281d78c105
Update train.py (#902)
* Update train.py with simplified ckpt names

* Return default hyps to hyp.scratch.yaml

Leave line commented for future use once mystery of best finetuning hyps to apply becomes clearer.

* Force test_batch*_pred.jpg replot on final epoch

This will allow you to see predictions final testing run after training completes in runs/exp0
2020-09-02 15:08:43 -07:00
Naman Gupta 6f3db5e662
Remove autoanchor and class checks on resumed training (#889)
* Class frequency not calculated on resuming training

Calculation of class frequency is not needed when resuming training.
Anchors can still be recalculated whether resuming or not.

* Check rank for autoanchor

* Update train.py

no autoanchor checks on resume

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-09-02 09:05:31 -07:00
Glenn Jocher f06e2d518c opt.image_weights bug fix (#885) 2020-08-31 11:05:37 -07:00
Glenn Jocher 69ff781ca5 opt.img_weights bug fix (#885) 2020-08-31 10:33:07 -07:00
Glenn Jocher 08e97a2f88 Update hyperparameters to add lrf, anchors 2020-08-28 14:58:43 -07:00
Glenn Jocher a21bd0687c Update train.py forward simplification 2020-08-25 13:48:03 -07:00
Glenn Jocher 09402a2174 torch.from_tensor() bug fix 2020-08-25 03:14:17 -07:00
Glenn Jocher 83dc540b1d remove ema.ema hasattr(ema, 'module') check 2020-08-22 15:18:39 -07:00
Glenn Jocher 4447f4b937
--resume to same runs/exp directory (#765)
* initial commit

* add weight backup dir on resume
2020-08-20 18:24:33 -07:00
NanoCode012 fb4fc8cd02
Fix ema attribute error in DDP mode (#775)
* Fix ema error in DDP mode

* Update train.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-08-18 09:52:21 -07:00
Glenn Jocher ebafd1ead5
single command --resume (#756)
* single command --resume

* else check files, remove TODO

* argparse.Namespace()

* tensorboard lr

* bug fix in get_latest_run()
2020-08-17 16:28:43 -07:00
Glenn Jocher 916d4aad9a
v3.0 Release (#725)
* initial commit

* remove yolov3-spp from test.py study

* update study --img range

* update mAP

* cleanup and speed updates

* update README plot
2020-08-13 14:25:05 -07:00
NanoCode012 0892c44bc4
Fix Logging (#719)
* Add logging setup

* Fix fusing layers message

* Fix logging does not have end

* Add logging

* Change logging to use logger

* Update yolo.py

I tried this in a cloned branch, and everything seems to work fine

* Update yolo.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-08-12 14:18:19 -07:00
Marc a925f283a7
max workers for dataloader (#722) 2020-08-12 13:57:36 -07:00
NanoCode012 4949401a94
Fix redundant outputs via Logging in DDP training (#500)
* Change print to logging

* Clean function set_logging

* Add line spacing

* Change leftover prints to log

* Fix scanning labels output

* Fix rank naming

* Change leftover print to logging

* Reorganized DDP variables

* Fix type error

* Make quotes consistent

* Fix spelling

* Clean function call

* Add line spacing

* Update datasets.py

* Update train.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-08-11 11:18:45 -07:00
Glenn Jocher e71fd0ec0b Model freeze capability (#679) 2020-08-10 22:49:43 -07:00
Glenn Jocher 8e5c66579b update train.py remove save_json final_epoch 2020-08-09 21:24:40 -07:00
Glenn Jocher 41523e2c91
Dataset autodownload feature addition (#685)
* initial commit

* move download scripts into data/scripts

* new check_dataset() function in general.py

* move check_dataset() out of with context

* Update general.py

* DDP update

* Update general.py
2020-08-09 20:52:57 -07:00
NanoCode012 3d8ed0a76b
Fix missing model.stride in DP and DDP mode (#683) 2020-08-09 11:01:36 -07:00
Glenn Jocher a0ac5adb7b Single-source training update (#680) 2020-08-09 02:27:35 -07:00
Glenn Jocher 3c6e2f7668
Single-source training (#680)
* Single-source training

* Extract hyperparameters into seperate files

* weight decay scientific notation yaml reader bug fix

* remove import glob

* intersect_dicts() implementation

* 'or' bug fix

* .to(device) bug fix
2020-08-09 02:12:44 -07:00
NanoCode012 d7cfbc47ab
Fix unrecognized local rank argument (#676) 2020-08-08 16:40:10 -07:00
Glenn Jocher 93684531c6
train.py --logdir argparser addition (#660)
* train.py --logdir argparser addition

* train.py --logdir argparser addition
2020-08-06 22:26:38 -07:00
NanoCode012 886b9841c8
Add Multi-Node support for DDP Training (#504)
* Add support for multi-node DDP

* Remove local_rank confusion

* Fix spacing
2020-08-06 11:15:24 -07:00
lorenzomammana 728efa6576
Fix missing imports (#627)
* Fix missing imports

* Update detect.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-08-04 12:22:15 -07:00
Glenn Jocher eb99dff9ef import random bug fix (#614) 2020-08-03 10:52:59 -07:00
Jirka Borovec d5b6416c87
Explicit Imports (#498)
* expand imports

* optimize

* miss

* fix
2020-08-02 15:47:36 -07:00
Glenn Jocher f1096b2cf7 hyperparameter evolution update (#566) 2020-08-02 10:32:04 -07:00
Glenn Jocher c1a2a7a411 hyperparameter evolution bug fix (#566) 2020-08-01 23:00:10 -07:00
Glenn Jocher 8074745908 hyperparameter evolution bug fix (#566) 2020-08-01 19:24:14 -07:00
Glenn Jocher e32abb5fb9 hyperparameter evolution bug fix (#566) 2020-08-01 19:15:48 -07:00
Glenn Jocher 8056fe2db8 hyperparameter evolution bug fix (#566) 2020-08-01 15:07:40 -07:00
Glenn Jocher 127cbeb3f5 hyperparameter expansion to flips, perspective, mixup 2020-08-01 13:47:54 -07:00
Glenn Jocher bcd452c482 replace random_affine() with random_perspective()
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-07-31 15:53:52 -07:00
Liu Changyu c020875b17
PyTorch 1.6.0 update with native AMP (#573)
* PyTorch have Automatic Mixed Precision (AMP) Training.

* Fixed the problem of inconsistent code length indentation

* Fixed the problem of inconsistent code length indentation

* Mixed precision training is turned on by default
2020-07-31 10:52:45 -07:00
Laughing 4e2b9ecc7e
LR --resume repeat bug fix (#565) 2020-07-30 10:48:20 -07:00
AlexWang1900 a209a32019
Fix bug #541 #542 (#545)
* fix #541 #542

* Update train.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-07-28 18:31:01 -07:00
NanoCode012 7f8471eaeb
--notest bug fix (#518)
* Fix missing results_file and fi when notest passed

* Update train.py

reverting previous changes and  removing functionality from 'if not opt.notest or final_epoch:  # Calculate mAP' loop.

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2020-07-25 10:24:39 -07:00