From 039ab1ccd759314cc8c11e68478348bc069ec2a5 Mon Sep 17 00:00:00 2001 From: Blake Blackshear Date: Sat, 5 Oct 2024 14:51:05 -0500 Subject: [PATCH 1/5] add docs for yolonas plus models (#14161) * add docs for yolonas plus models * typo --- docs/docs/plus/first_model.md | 4 ++-- docs/docs/plus/improving_model.md | 31 +++++++++++++++---------------- docs/docs/plus/index.md | 25 ++++++++++++++++++++++--- 3 files changed, 39 insertions(+), 21 deletions(-) diff --git a/docs/docs/plus/first_model.md b/docs/docs/plus/first_model.md index bbaf9cacb..6978bb491 100644 --- a/docs/docs/plus/first_model.md +++ b/docs/docs/plus/first_model.md @@ -5,7 +5,7 @@ title: Requesting your first model ## Step 1: Upload and annotate your images -Before requesting your first model, you will need to upload at least 10 images to Frigate+. But for the best results, you should provide at least 100 verified images per camera. Keep in mind that varying conditions should be included. You will want images from cloudy days, sunny days, dawn, dusk, and night. Refer to the [integration docs](../integrations/plus.md#generate-an-api-key) for instructions on how to easily submit images to Frigate+ directly from Frigate. +Before requesting your first model, you will need to upload and verify at least 1 image to Frigate+. The more images you upload, annotate, and verify the better your results will be. Most users start to see very good results once they have at least 100 verified images per camera. Keep in mind that varying conditions should be included. You will want images from cloudy days, sunny days, dawn, dusk, and night. Refer to the [integration docs](../integrations/plus.md#generate-an-api-key) for instructions on how to easily submit images to Frigate+ directly from Frigate. It is recommended to submit **both** true positives and false positives. This will help the model differentiate between what is and isn't correct. You should aim for a target of 80% true positive submissions and 20% false positives across all of your images. If you are experiencing false positives in a specific area, submitting true positives for any object type near that area in similar lighting conditions will help teach the model what that area looks like when no objects are present. @@ -13,7 +13,7 @@ For more detailed recommendations, you can refer to the docs on [improving your ## Step 2: Submit a model request -Once you have an initial set of verified images, you can request a model on the Models page. Each model request requires 1 of the 12 trainings that you receive with your annual subscription. This model will support all [label types available](./index.md#available-label-types) even if you do not submit any examples for those labels. Model creation can take up to 36 hours. +Once you have an initial set of verified images, you can request a model on the Models page. For guidance on choosing a model type, refer to [this part of the documentation](./index.md#available-model-types). Each model request requires 1 of the 12 trainings that you receive with your annual subscription. This model will support all [label types available](./index.md#available-label-types) even if you do not submit any examples for those labels. Model creation can take up to 36 hours. ![Plus Models Page](/img/plus/plus-models.jpg) ## Step 3: Set your model id in the config diff --git a/docs/docs/plus/improving_model.md b/docs/docs/plus/improving_model.md index 0e97b21a5..37a765994 100644 --- a/docs/docs/plus/improving_model.md +++ b/docs/docs/plus/improving_model.md @@ -3,7 +3,7 @@ id: improving_model title: Improving your model --- -You may find that Frigate+ models result in more false positives initially, but by submitting true and false positives, the model will improve. Because a limited number of users submitted images to Frigate+ prior to this launch, you may need to submit several hundred images per camera to see good results. With all the new images now being submitted, future base models will improve as more and more users (including you) submit examples to Frigate+. Note that only verified images will be used when training your model. Submitting an image from Frigate as a true or false positive will not verify the image. You still must verify the image in Frigate+ in order for it to be used in training. +You may find that Frigate+ models result in more false positives initially, but by submitting true and false positives, the model will improve. With all the new images now being submitted by subscribers, future base models will improve as more and more examples are incorporated. Note that only images with at least one verified label will be used when training your model. Submitting an image from Frigate as a true or false positive will not verify the image. You still must verify the image in Frigate+ in order for it to be used in training. - **Submit both true positives and false positives**. This will help the model differentiate between what is and isn't correct. You should aim for a target of 80% true positive submissions and 20% false positives across all of your images. If you are experiencing false positives in a specific area, submitting true positives for any object type near that area in similar lighting conditions will help teach the model what that area looks like when no objects are present. - **Lower your thresholds a little in order to generate more false/true positives near the threshold value**. For example, if you have some false positives that are scoring at 68% and some true positives scoring at 72%, you can try lowering your threshold to 65% and submitting both true and false positives within that range. This will help the model learn and widen the gap between true and false positive scores. @@ -36,18 +36,17 @@ Misidentified objects should have a correct label added. For example, if a perso ## Shortcuts for a faster workflow -|Shortcut Key|Description| -|-----|--------| -|`?`|Show all keyboard shortcuts| -|`w`|Add box| -|`d`|Toggle difficult| -|`s`|Switch to the next label| -|`tab`|Select next largest box| -|`del`|Delete current box| -|`esc`|Deselect/Cancel| -|`← ↑ → ↓`|Move box| -|`Shift + ← ↑ → ↓`|Resize box| -|`-`|Zoom out| -|`=`|Zoom in| -|`f`|Hide/show all but current box| -|`spacebar`|Verify and save| +| Shortcut Key | Description | +| ----------------- | ----------------------------- | +| `?` | Show all keyboard shortcuts | +| `w` | Add box | +| `d` | Toggle difficult | +| `s` | Switch to the next label | +| `tab` | Select next largest box | +| `del` | Delete current box | +| `esc` | Deselect/Cancel | +| `← ↑ → ↓` | Move box | +| `Shift + ← ↑ → ↓` | Resize box | +| `scrollwheel` | Zoom in/out | +| `f` | Hide/show all but current box | +| `spacebar` | Verify and save | diff --git a/docs/docs/plus/index.md b/docs/docs/plus/index.md index 35189ed3f..b05f4f306 100644 --- a/docs/docs/plus/index.md +++ b/docs/docs/plus/index.md @@ -15,17 +15,36 @@ With a subscription, 12 model trainings per year are included. If you cancel you Information on how to integrate Frigate+ with Frigate can be found in the [integration docs](../integrations/plus.md). +## Available model types + +There are two model types offered in Frigate+: `mobiledet` and `yolonas`. Both of these models are object detection models and are trained to detect the same set of labels [listed below](#available-label-types). + +Not all model types are supported by all detectors, so it's important to choose a model type to match your detector as shown in the table under [supported detector types](#supported-detector-types). + +| Model Type | Description | +| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------- | +| `mobiledet` | Based on the same architecture as the default model included with Frigate. Runs on Google Coral devices and CPUs. | +| `yolonas` | A newer architecture that offers slightly higher accuracy and improved detection of small objects. Runs on Intel, NVidia GPUs, and AMD GPUs. | + ## Supported detector types +Currently, Frigate+ models support CPU (`cpu`), Google Coral (`edgetpu`), OpenVino (`openvino`), ONNX (`onnx`), and ROCm (`rocm`) detectors. + :::warning -Frigate+ models are not supported for TensorRT or OpenVino yet. +Using Frigate+ models with `onnx` and `rocm` is only available with Frigate 0.15, which is still under development. ::: -Currently, Frigate+ models only support CPU (`cpu`) and Coral (`edgetpu`) models. OpenVino is next in line to gain support. +| Hardware | Recommended Detector Type | Recommended Model Type | +| ---------------------------------------------------------------------------------------------------------------------------- | ------------------------- | ---------------------- | +| [CPU](/configuration/object_detectors.md#cpu-detector-not-recommended) | `cpu` | `mobiledet` | +| [Coral (all form factors)](/configuration/object_detectors.md#edge-tpu-detector) | `edgetpu` | `mobiledet` | +| [Intel](/configuration/object_detectors.md#openvino-detector) | `openvino` | `yolonas` | +| [NVidia GPU](https://deploy-preview-13787--frigate-docs.netlify.app/configuration/object_detectors#onnx)\* | `onnx` | `yolonas` | +| [AMD ROCm GPU](https://deploy-preview-13787--frigate-docs.netlify.app/configuration/object_detectors#amdrocm-gpu-detector)\* | `rocm` | `yolonas` | -The models are created using the same MobileDet architecture as the default model. Additional architectures will be added in future releases as needed. +_\* Requires Frigate 0.15_ ## Available label types From 2a15b95f18dd7d21f3a3d2bf260be12c5f3cc6a9 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Mon, 7 Oct 2024 14:28:24 -0600 Subject: [PATCH 2/5] Docs updates (#14202) * Clarify live docs * Link out to common config examples in getting started guide * Add tip for go2rtc name configuration * direct link --- docs/docs/configuration/reference.md | 8 +++++--- docs/docs/guides/configuring_go2rtc.md | 10 +++++++++- docs/docs/guides/getting_started.md | 4 +++- 3 files changed, 17 insertions(+), 5 deletions(-) diff --git a/docs/docs/configuration/reference.md b/docs/docs/configuration/reference.md index 76a9e6158..e9b1d605f 100644 --- a/docs/docs/configuration/reference.md +++ b/docs/docs/configuration/reference.md @@ -480,10 +480,12 @@ snapshots: # Uses https://github.com/AlexxIT/go2rtc (v1.9.2) go2rtc: -# Optional: jsmpeg stream configuration for WebUI +# Optional: Live stream configuration for WebUI. +# NOTE: Can be overridden at the camera level live: - # Optional: Set the name of the stream that should be used for live view - # in frigate WebUI. (default: name of camera) + # Optional: Set the name of the stream configured in go2rtc + # that should be used for live view in frigate WebUI. (default: name of camera) + # NOTE: In most cases this should be set at the camera level only. stream_name: camera_name # Optional: Set the height of the jsmpeg stream. (default: 720) # This must be less than or equal to the height of the detect stream. Lower resolutions diff --git a/docs/docs/guides/configuring_go2rtc.md b/docs/docs/guides/configuring_go2rtc.md index 8316376f2..5c85f7d11 100644 --- a/docs/docs/guides/configuring_go2rtc.md +++ b/docs/docs/guides/configuring_go2rtc.md @@ -13,7 +13,15 @@ Use of the bundled go2rtc is optional. You can still configure FFmpeg to connect # Setup a go2rtc stream -First, you will want to configure go2rtc to connect to your camera stream by adding the stream you want to use for live view in your Frigate config file. For the best experience, you should set the stream name under go2rtc to match the name of your camera so that Frigate will automatically map it and be able to use better live view options for the camera. Avoid changing any other parts of your config at this step. Note that go2rtc supports [many different stream types](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#module-streams), not just rtsp. +First, you will want to configure go2rtc to connect to your camera stream by adding the stream you want to use for live view in your Frigate config file. Avoid changing any other parts of your config at this step. Note that go2rtc supports [many different stream types](https://github.com/AlexxIT/go2rtc/tree/v1.9.4#module-streams), not just rtsp. + +:::tip + +For the best experience, you should set the stream name under `go2rtc` to match the name of your camera so that Frigate will automatically map it and be able to use better live view options for the camera. + +See [the live view docs](../configuration/live.md#setting-stream-for-live-ui) for more information. + +::: ```yaml go2rtc: diff --git a/docs/docs/guides/getting_started.md b/docs/docs/guides/getting_started.md index e2a4420a3..79082970b 100644 --- a/docs/docs/guides/getting_started.md +++ b/docs/docs/guides/getting_started.md @@ -306,7 +306,9 @@ By default, Frigate will retain video of all events for 10 days. The full set of ### Step 7: Complete config -At this point you have a complete config with basic functionality. You can see the [full config reference](../configuration/reference.md) for a complete list of configuration options. +At this point you have a complete config with basic functionality. +- View [common configuration examples](../configuration/index.md#common-configuration-examples) for a list of common configuration examples. +- View [full config reference](../configuration/reference.md) for a complete list of configuration options. ### Follow up From f86957e5e19b321fd76579b39609410bc49fc782 Mon Sep 17 00:00:00 2001 From: JC Date: Wed, 9 Oct 2024 01:15:10 +0100 Subject: [PATCH 3/5] Improve docs on exports API endpoints (#14224) * Add (optional) export name to the create-export API endpoint docs * Add the exports list endpoint to the docs --- docs/docs/integrations/api.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/docs/docs/integrations/api.md b/docs/docs/integrations/api.md index 0c48c0c82..f0a844888 100644 --- a/docs/docs/integrations/api.md +++ b/docs/docs/integrations/api.md @@ -411,17 +411,37 @@ HTTP Live Streaming Video on Demand URL for the specified event. Can be viewed i HTTP Live Streaming Video on Demand URL for the camera with the specified time range. Can be viewed in an application like VLC. +### `GET /api/exports` + +Fetch a list of all export recordings + +Sample response: +```json +[ + { + "camera": "doorbell", + "date": 12800057, + "id": "doorbell_pjis54", + "in_progress": false, + "name": "2024-10-04 fox visit", + "thumb_path": "/media/frigate/clips/export/doorbell_pjis54.webp", + "video_path": "/media/frigate/exports/doorbell_pjis54.mp4" + } +] +``` + ### `POST /api/export//start//end/` Export recordings from `start-timestamp` to `end-timestamp` for `camera` as a single mp4 file. These recordings will be exported to the `/media/frigate/exports` folder. -It is also possible to export this recording as a time-lapse. +It is also possible to export this recording as a time-lapse using the "playback" key in the json body, or specify a custom export filename, using the "name" key. **Optional Body:** ```json { - "playback": "realtime" // playback factor: realtime or timelapse_25x + "playback": "realtime", // playback factor: realtime or timelapse_25x + "name": "custom export name" // override the default export filename with a custom name } ``` From 51509760e31b1986291b0b6df4a04b749f434013 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Sat, 12 Oct 2024 06:13:00 -0600 Subject: [PATCH 4/5] Update object docs (#14295) --- docs/docs/configuration/objects.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration/objects.md b/docs/docs/configuration/objects.md index 1a93f9704..796d31258 100644 --- a/docs/docs/configuration/objects.md +++ b/docs/docs/configuration/objects.md @@ -5,7 +5,7 @@ title: Available Objects import labels from "../../../labelmap.txt"; -Frigate includes the object models listed below from the Google Coral test data. +Frigate includes the object labels listed below from the Google Coral test data. Please note: From af844ea9d5c0ec54166df98c88b763affe81dc95 Mon Sep 17 00:00:00 2001 From: Nicolas Mowen Date: Tue, 15 Oct 2024 09:39:31 -0600 Subject: [PATCH 5/5] Update coral troubleshooting docs (#14370) * Update coral docs for latest ubuntu * capitalization Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> --------- Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> --- docs/docs/troubleshooting/edgetpu.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/docs/troubleshooting/edgetpu.md b/docs/docs/troubleshooting/edgetpu.md index 8c917fa2f..33e00f11a 100644 --- a/docs/docs/troubleshooting/edgetpu.md +++ b/docs/docs/troubleshooting/edgetpu.md @@ -49,7 +49,10 @@ The USB Coral can become stuck and need to be restarted, this can happen for a n ## PCIe Coral Not Detected -The most common reason for the PCIe coral not being detected is that the driver has not been installed. See [the coral docs](https://coral.ai/docs/m2/get-started/#2-install-the-pcie-driver-and-edge-tpu-runtime) for how to install the driver for the PCIe based coral. +The most common reason for the PCIe Coral not being detected is that the driver has not been installed. This process varies based on what OS and kernel that is being run. + +- In most cases [the Coral docs](https://coral.ai/docs/m2/get-started/#2-install-the-pcie-driver-and-edge-tpu-runtime) show how to install the driver for the PCIe based Coral. +- For Ubuntu 22.04+ https://github.com/jnicolson/gasket-builder can be used to build and install the latest version of the driver. ## Only One PCIe Coral Is Detected With Coral Dual EdgeTPU