Resolves https://github.com/ultralytics/yolov5/issues/388#issuecomment-1077121821

May resolve https://github.com/ultralytics/yolov5/issues/6947

* Assert engine precision #6777

* Default to FP32 inputs for TensorRT engines

* Default to FP16 TensorRT exports #6777

* Remove wrong line #6777

* Automatically adjust detect.py input precision #6777

* Automatically adjust val.py input precision #6777

* Add missing colon

* Cleanup

* Cleanup

* Remove default trt_fp16_input definition

* Experiment

* Reorder detect.py if statement to after half checks

* Update common.py

* Update export.py

* Cleanup

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Fix CoreML P6 inference

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix exporting saved_model: pb exceeds 2GB

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Replace TF v1.x API with TF v2.x API for saved_model export

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Clean up

* Remove lambda in tf.function()

* Revert "Remove lambda in tf.function()" to be compatible with TF v2.4

This reverts commit 46c7931f11.

* Fix for pre-commit.ci

* Cleanup1

* Cleanup2

* Backwards compatibility update

* Update common.py

* Update common.py

* Cleanup3

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Add benchmarks.py

* Update

* Add requirements

* Updates

* Updates

* Updates

* Updates

* Updates

* Updates

* dataset autodownload from root

* Update

* Redirect to /dev/null

* sudo --help

* Cleanup

* Add exports pd df

* Updates

* Updates

* Updates

* Cleanup

* dir handling fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cleanup

* Cleanup2

* Cleanup3

* Cleanup model_type

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Edge TPU TF imports fix

Fix for https://github.com/ultralytics/yolov5/issues/6535#issuecomment-1030631526

* Update common.py

* Edge TPU `tf.lite.experimental.load_delegate` fix

Fix attempt for #6535

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add models/tf.py for TensorFlow and TFLite export

* Set auto=False for int8 calibration

* Update requirements.txt for TensorFlow and TFLite export

* Read anchors directly from PyTorch weights

* Add --tf-nms to append NMS in TensorFlow SavedModel and GraphDef export

* Remove check_anchor_order, check_file, set_logging from import

* Reformat code and optimize imports

* Autodownload model and check cfg

* update --source path, img-size to 320, single output

* Adjust representative_dataset

* Put representative dataset in tfl_int8 block

* detect.py TF inference

* weights to string

* weights to string

* cleanup tf.py

* Add --dynamic-batch-size

* Add xywh normalization to reduce calibration error

* Update requirements.txt

TensorFlow 2.3.1 -> 2.4.0 to avoid int8 quantization error

* Fix imports

Move C3 from models.experimental to models.common

* Add models/tf.py for TensorFlow and TFLite export

* Set auto=False for int8 calibration

* Update requirements.txt for TensorFlow and TFLite export

* Read anchors directly from PyTorch weights

* Add --tf-nms to append NMS in TensorFlow SavedModel and GraphDef export

* Remove check_anchor_order, check_file, set_logging from import

* Reformat code and optimize imports

* Autodownload model and check cfg

* update --source path, img-size to 320, single output

* Adjust representative_dataset

* detect.py TF inference

* Put representative dataset in tfl_int8 block

* weights to string

* weights to string

* cleanup tf.py

* Add --dynamic-batch-size

* Add xywh normalization to reduce calibration error

* Update requirements.txt

TensorFlow 2.3.1 -> 2.4.0 to avoid int8 quantization error

* Fix imports

Move C3 from models.experimental to models.common

* implement C3() and SiLU()

* Add TensorFlow and TFLite Detection

* Add --tfl-detect for TFLite Detection

* Add int8 quantized TFLite inference in detect.py

* Add --edgetpu for Edge TPU detection

* Fix --img-size to add rectangle TensorFlow and TFLite input

* Add --no-tf-nms to detect objects using models combined with TensorFlow NMS

* Fix --img-size list type input

* Update README.md

* Add Android project for TFLite inference

* Upgrade TensorFlow v2.3.1 -> v2.4.0

* Disable normalization of xywh

* Rewrite names init in detect.py

* Change input resolution 640 -> 320 on Android

* Disable NNAPI

* Update README.me --img 640 -> 320

* Update README.me for Edge TPU

* Update README.md

* Fix reshape dim to support dynamic batching

* Fix reshape dim to support dynamic batching

* Add epsilon argument in tf_BN, which is different between TF and PT

* Set stride to None if not using PyTorch, and do not warmup without PyTorch

* Add list support in check_img_size()

* Add list input support in detect.py

* sys.path.append('./') to run from yolov5/

* Add int8 quantization support for TensorFlow 2.5

* Add get_coco128.sh

* Remove --no-tfl-detect in models/tf.py (Use tf-android-tfl-detect branch for EdgeTPU)

* Update requirements.txt

* Replace torch.load() with attempt_load()

* Update requirements.txt

* Add --tf-raw-resize to set half_pixel_centers=False

* Remove android directory

* Update README.md

* Update README.md

* Add multiple OS support for EdgeTPU detection

* Fix export and detect

* Export 3 YOLO heads with Edge TPU models

* Remove xywh denormalization with Edge TPU models in detect.py

* Fix saved_model and pb detect error

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix pre-commit.ci failure

* Add edgetpu in export.py docstring

* Fix Edge TPU model detection exported by TF 2.7

* Add class names for TF/TFLite in DetectMultibackend

* Fix assignment with nl in TFLite Detection

* Add check when getting Edge TPU compiler version

* Add UTF-8 encoding in opening --data file for Windows

* Remove redundant TensorFlow import

* Add Edge TPU in export.py's docstring

* Add the detect layer in Edge TPU model conversion

* Default `dnn=False`

* Cleanup data.yaml loading

* Update detect.py

* Update val.py

* Comments and generalize data.yaml names

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: unknown <fangjiacong@ut.cn>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Check TensorRT>=8.0.0 version

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix tolist() to add the file for each Detection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix PEP8 requirement for 2 spaces before an inline comment

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cleanup

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Add ONNX inference providers

Fix for https://github.com/ultralytics/yolov5/issues/5916

* Update common.py

* tensorflow or tflite exclusively as interpreter

As per bug report https://github.com/ultralytics/yolov5/issues/5709 I think there should be only one attempt to assign interpreter, and it appears tflite is only ever needed for the case of edgetpu model.

* Scope imports

* Nested definition line fix

* Update common.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Export and detect with TensorRT engine file

* Resolve `isort`

* Make validation works with TensorRT engine

* feat: update export docstring

* feat: change suffix from *.trt to *.engine

* feat: get rid of pycuda

* feat: make compatiable with val.py

* feat: support detect with fp16 engine

* Add Lite to Edge TPU string

* Remove *.trt comment

* Revert to standard success logger.info string

* Fix Deprecation Warning

```
export.py:310: DeprecationWarning: Use build_serialized_network instead.
  with builder.build_engine(network, config) as engine, open(f, 'wb') as t:
```

* Revert deprecation warning fix

@imyhxy it seems we can't apply the deprecation warning fix because then export fails, so I'm reverting my previous change here.

* Update export.py

* Update export.py

* Update common.py

* export onnx to file before building TensorRT engine file

* feat: triger ONNX export failed early

* feat: load ONNX model from file

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* New `DetectMultiBackend()` class

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pb to pt fix

* Cleanup

* explicit apply_classifier path

* Cleanup2

* Cleanup3

* Cleanup4

* Cleanup5

* Cleanup6

* val.py MultiBackend inference

* warmup fix

* to device fix

* pt fix

* device fix

* Val cleanup

* COCO128 URL to assets

* half fix

* detect fix

* detect fix 2

* remove half from DetectMultiBackend

* training half handling

* training half handling 2

* training half handling 3

* Cleanup

* Fix CI error

* Add torchscript _extra_files

* Add TorchScript

* Add CoreML

* CoreML cleanup

* New `DetectMultiBackend()` class

* pb to pt fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cleanup

* explicit apply_classifier path

* Cleanup2

* Cleanup3

* Cleanup4

* Cleanup5

* Cleanup6

* val.py MultiBackend inference

* warmup fix

* to device fix

* pt fix

* device fix

* Val cleanup

* COCO128 URL to assets

* half fix

* detect fix

* detect fix 2

* remove half from DetectMultiBackend

* training half handling

* training half handling 2

* training half handling 3

* Cleanup

* Fix CI error

* Add torchscript _extra_files

* Add TorchScript

* Add CoreML

* CoreML cleanup

* revert default to pt

* Add Usage examples

* Cleanup val

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Logger consolidation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix `save_one_box()`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* precommit: isort

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update isort config

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update name

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>