Mirrors/blakeblackshear.frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2024-12-19 19:06:16 +01:00
Code Issues Projects Releases Wiki Activity

Genai descriptions are not generated until tracked objects end (#15561)

Browse Source
This commit is contained in:
Josh Hawkins 2024-12-17 17:33:04 -06:00 committed by GitHub
parent d9ef8fa206
commit 3dc26e78ef
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 84 additions and 52 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/docs/configuration/genai.md
View File

@ -5,6 +5,8 @@ title: Generative AI
Generative AI can be used to automatically generate descriptive text based on the thumbnails of your tracked objects. This helps with [Semantic Search](/configuration/semantic_search) in Frigate to provide more context about your tracked objects. Descriptions are accessed via the _Explore_ view in the Frigate UI by clicking on a tracked object's thumbnail.
Requests for a description are sent off automatically to your AI provider at the end of the tracked object's lifecycle. Descriptions can also be regenerated manually via the Frigate UI.
:::info
Semantic Search must be enabled to use Generative AI.

134
web/src/components/overlay/detail/SearchDetailDialog.tsx
View File

@ -469,60 +469,90 @@ function ObjectDetailsTab({
</div>
</div>
<div className="flex flex-col gap-1.5">
<div className="text-sm text-primary/40">Description</div>
<Textarea
className="h-64"
placeholder="Description of the tracked object"
value={desc}
onChange={(e) => setDesc(e.target.value)}
/>
<div className="flex w-full flex-row justify-end gap-2">
{config?.cameras[search.camera].genai.enabled && (
<div className="flex items-center">
<Button
className="rounded-r-none border-r-0"
aria-label="Regenerate tracked object description"
onClick={() => regenerateDescription("thumbnails")}
>
Regenerate
</Button>
{search.has_snapshot && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
className="rounded-l-none border-l-0 px-2"
aria-label="Expand regeneration menu"
>
<FaChevronDown className="size-3" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem
className="cursor-pointer"
aria-label="Regenerate from snapshot"
onClick={() => regenerateDescription("snapshot")}
>
Regenerate from Snapshot
</DropdownMenuItem>
<DropdownMenuItem
className="cursor-pointer"
aria-label="Regenerate from thumbnails"
onClick={() => regenerateDescription("thumbnails")}
>
Regenerate from Thumbnails
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
)}
{config?.cameras[search.camera].genai.enabled &&
!search.end_time &&
(config.cameras[search.camera].genai.required_zones.length === 0 ||
search.zones.some((zone) =>
config.cameras[search.camera].genai.required_zones.includes(zone),
)) &&
(config.cameras[search.camera].genai.objects.length === 0 ||
config.cameras[search.camera].genai.objects.includes(
search.label,
)) ? (
<>
<div className="text-sm text-primary/40">Description</div>
<div className="flex h-64 flex-col items-center justify-center gap-3 border p-4 text-sm text-primary/40">
<div className="flex">
<ActivityIndicator />
</div>
<div className="flex">
Frigate will not request a description from your Generative AI
provider until the tracked object's lifecycle has ended.
</div>
</div>
</>
) : (
<>
<div className="text-sm text-primary/40">Description</div>
<Textarea
className="h-64"
placeholder="Description of the tracked object"
value={desc}
onChange={(e) => setDesc(e.target.value)}
/>
</>
)}
<div className="flex w-full flex-row justify-end gap-2">
{config?.cameras[search.camera].genai.enabled && search.end_time && (
<>
<div className="flex items-start">
<Button
className="rounded-r-none border-r-0"
aria-label="Regenerate tracked object description"
onClick={() => regenerateDescription("thumbnails")}
>
Regenerate
</Button>
{search.has_snapshot && (
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button
className="rounded-l-none border-l-0 px-2"
aria-label="Expand regeneration menu"
>
<FaChevronDown className="size-3" />
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem
className="cursor-pointer"
aria-label="Regenerate from snapshot"
onClick={() => regenerateDescription("snapshot")}
>
Regenerate from Snapshot
</DropdownMenuItem>
<DropdownMenuItem
className="cursor-pointer"
aria-label="Regenerate from thumbnails"
onClick={() => regenerateDescription("thumbnails")}
>
Regenerate from Thumbnails
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
)}
</div>
<Button
variant="select"
aria-label="Save"
onClick={updateDescription}
>
Save
</Button>
</>
)}
<Button
variant="select"
aria-label="Save"
onClick={updateDescription}
>
Save
</Button>
</div>
</div>
</div>