1
0
mirror of https://github.com/Unleash/unleash.git synced 2025-10-13 11:17:26 +02:00
Commit Graph

330 Commits

Author SHA1 Message Date
Jaanus Sellin
5b879d9331
feat: add origin column for consumption based tables (#10494)
We will start separating traffic, when it comes from different origins.
Also will add it as primary key so merging will work correctly.

There is a problem with using normal down migration, is that if we split
data by origin, when down migrating and removing column, and re adding
the old migration, we will have conflicts and it will fail.

This means before we down migrate, we need to aggregate/sum the data and
only then we can reapply old schema.
2025-08-13 16:16:05 +03:00
Jaanus Sellin
9fe638b0b7
fix: week range for lifecycle backfill (#10493)
It is using almost same logic as
https://github.com/Unleash/unleash/pull/10476, but week range is
correct.
2025-08-13 13:43:52 +03:00
Jaanus Sellin
04a57e502e
fix: now lifecycle trends will be added for flags that have official flag type (#10482)
We have instances that have fake flag types in features table. This
ensures, we only count in flags have official types.
2025-08-08 13:23:35 +03:00
Jaanus Sellin
75df390d0b
fix: backfilling lifecycle trends based on new logic (#10476)
Previously lifecycle trends were taking one snapshot per flag, now they
take one per flag per stage it moved.
2025-08-07 15:05:04 +03:00
Gastón Fournier
629624fd1c
docs: update references and names to new SDK nomenclature (#10431)
## About the changes
Follow up on https://github.com/Unleash/unleash/pull/10430 this PR
adapts documentation.

---------

Co-authored-by: Melinda Fekete <melinda.fekete@getunleash.io>
2025-07-30 14:35:37 +02:00
Gastón Fournier
902845bf82
chore: amend user-created-missing events (#10333)
## About the changes
Users could have been created in Unleash without a corresponding event
(a.k.a. audit log), due to a non transactional user insert
([fix](https://github.com/Unleash/unleash/pull/10327)). This could have
happened because of providing the wrong role id or some other causes
we're not aware of.

This amends the situation by inserting an event for each user that
exists in the instance (not deleted) and doesn't have it's corresponding
user-created event.

The event is inserted as already announced because this happened in the
past.

The event log will look like this (simulated the situation in local
dev):
```json
{
  "id": 11,
  "type": "user-created",
  "createdBy": "unleash_system_user",
  "createdAt": "2025-07-08T16:06:17.428Z",
  "createdByUserId": null,
  "data": {
    "id": "6",
    "email": "xyz@three.com"
  },
  "preData": null,
  "tags": [],
  "featureName": null,
  "project": null,
  "environment": null,
  "label": "User created",
  "summary": "**unleash_system_user** created user ****"
}
```

The main problem is we can't create the event in the past, so this will
have to do it
2025-07-09 07:19:25 +00:00
Nuno Góis
43a6166673
chore: unknown flags with environment (#10325)
https://linear.app/unleash/issue/2-3681/add-environment-to-unknown-flag-reports

This does a few things:
 - Adds environment to our unknown flag reports
 - Increases our max unknown flag reports from 10 to 100
 - Reduces our clean up interval from 24 to 2 hours
- Flattens our response (easier to get started with and display in a
table, we can later decide if we want to group this data)

<img width="496" alt="image"
src="https://github.com/user-attachments/assets/11350639-9f7f-4011-8b39-b135c820ca21"
/>
2025-07-08 10:56:33 +01:00
Jaanus Sellin
2d2ba4ae25
feat: start storing event group type and id (#10233)
Simple 2 columns for event type and event id.

I thought about adding check for type, but decided to handle checking in
backend code.

Currently the types will be

1. transaction
2. change-request

And ID is ulid
2025-06-30 12:50:51 +03:00
Christopher Kolstad
6aecc3f93e
task: added notified at to change request requested approvals (#10196)
Adding this should allow us to only notify users that haven't been
notified before.

Necessary because the change_added event does not include a preData that
allowed us to diff requestedApprovers based on the event alone.

Also added a index on this column, since we're going to be using it to
filter.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-23 10:51:47 +00:00
Christopher Kolstad
ce8d49be10
task: added table for requested approvers for CRs (#10159)
As part of the task to make it possible to send notifications to
approvers for a CR, this PR adds a table that can link users to CRs
they've been requested to make an approval for.
2025-06-18 11:52:29 +02:00
Christopher Kolstad
b70f862f93
fix: md5 is deprecated and fails FEDRAMP. Replace with sha256 (#10125)
#10121 points out that we're using md5 functions still. This PR updates
our migrations to no longer use md5 at all (so if you haven't run the
migrations, you won't get email hashes until you get to the included
migration with this PR). If you've already run the migrations, we'll
drop the existing `email_hash varchar(32)` column and replace it with a
`email_hash TEXT` column.

We're also replacing the md5 function with `encode(sha256(email),
'hex')`. encode has been supported since PG10, sha256 came with PG11.

Do we want an index on the email_hash? I wasn't sure, but if we want to
do lookup we probably should have an index on it (though not a unique
one)
2025-06-13 09:41:40 +02:00
Gastón Fournier
bdb763c9d5
chore!: remove deprecated default env from new installs (#10080)
**BREAKING CHANGE**: DEFAULT_ENV changed from `default` (should not be
used anymore) to `development`

## About the changes
- Only delete default env if the install is fresh new.
- Consider development the new default. The main consequence of this
change is that the default is no longer considered `type=production`
environment but also for frontend tokens due to this assumption:
724c4b78a2/src/lib/schema/api-token-schema.test.ts (L54-L59)
(I believe this is mostly due to the [support for admin
tokens](https://github.com/Unleash/unleash/pull/10080#discussion_r2126871567))
- `feature_toggle_update_total` metric reports `n/a` in environment and
environment type as it's not environment specific
2025-06-06 12:02:21 +02:00
Jaanus Sellin
cf87f8cfe1
feat: make lifecycle trends more detailed (#10079)
##  PR Summary: Update `lifecycle_trends` Table Schema

###  What Changed

- **Updated `stage` column constraint**
- **Old values:** `'initial', 'develop', 'production', 'cleanup',
'archived'`
- **New values:** `'initial', 'pre-live', 'live', 'completed',
'archived'`

- **Replaced `flag_type` CHECK constraint with a foreign key**
  - Removed hardcoded values: `'experimental', 'release', 'permanent'`
  - Added foreign key:
    ```sql
FOREIGN KEY (flag_type) REFERENCES feature_types(id) ON DELETE CASCADE
    ```

- **Added down migration** to:
  - Restore original `stage` constraint
- Drop the foreign key on `flag_type` and re-add the original CHECK
constraint

### 💡 Why

- Align `stage` values with lifecycle model
- Use the `feature_types` table to ensure referential integrity
2025-06-04 10:32:32 +03:00
Gastón Fournier
5019f4fcbc
chore!: removing userId strategy for new installations of Unleash (#9800)
This removes a strategy that was already deprecated, but only for new
installations.

I tested starting with an installation with this strategy being used and
then updating, and I was still able to edit the strategy, so this should
not impact current users.

On a fresh install the strategy is no longer available.

---------

Co-authored-by: Nuno Góis <github@nunogois.com>
2025-06-04 09:30:13 +02:00
Jaanus Sellin
e474abb946
feat: lifecycle trends migration (#10066)
Adding a single table to capture all lifecycle trends data.

One field presents a challenge: `median_time_in_stage_days`. This value
is calculated per `stage`, not per `flag_type`. As a result, we would
need to duplicate the total median time across each flag type. This
isn’t a major issue.

An alternative would be to create a separate table solely for the median
values, but that seems like overkill.
2025-06-03 08:55:28 +03:00
Jaanus Sellin
d8c83fb824
fix: do not allow creating cr for same environment (#10010)
If there are two concurrent requests to create or edit change requests,
two separate ones may be created in parallel. The UI does not currently
handle this scenario, and if additional changes are made, they might be
added to a random existing change request—resulting in a messy and
unpredictable state.

This PR adds a unique index to the `change_requests` table 
```
on (created_by, project, environment)
WHERE state NOT IN ('Applied', 'Cancelled', 'Rejected', 'Scheduled').
```

In the extremely rare case where such conflicting data already exists in
a database, the migration will automatically cancel one of the
conflicting change requests.
2025-05-30 08:20:11 +03:00
Gastón Fournier
76b201e40e
feat: add migration for cdn_tokens (#10021) 2025-05-21 14:20:04 +00:00
Gastón Fournier
abe160eb7d
feat: Unleash v7 ESM migration (#9877)
We're migrating to ESM, which will allow us to import the latest
versions of our dependencies.

Co-Authored-By: Christopher Kolstad <chriswk@getunleash.io>
2025-05-14 09:47:12 +02:00
Tymoteusz Czech
499ee1e099
migration: project settings - external link templates (#9933) 2025-05-08 13:51:21 +02:00
Mateusz Kwasniewski
9ca44e6188
feat: add domain to links (#9930) 2025-05-08 13:20:47 +02:00
Nuno Góis
eb238f502a
chore: unknown flags (#9837)
https://linear.app/unleash/issue/2-3406/hold-unknown-flags-in-memory-and-show-them-in-the-ui-somehow

This PR introduces a suggestion for a “unknown flags” feature.

When clients report metrics for flags that don’t exist in Unleash (e.g.
due to typos), we now track a limited set of these unknown flag names
along with the appnames that reported them. The goal is to help users
identify and clean up incorrect flag usage across their apps.

We store up to 10 unknown flag + appName combinations, keeping only the
most recent reports. Data is collected in-memory and flushed
periodically to the DB, with deduplication and merging to ensure we
don’t exceed the cap even across pods.

We were especially careful to make this implementation defensive, as
unknown flags could be reported in very high volumes. Writes are
batched, deduplicated, and hard-capped to avoid DB pressure.

No UI has been added yet — this is backend-only for now and intended as
a step toward better visibility into client misconfigurations.

I would suggest starting with a simple banner that opens a dialog
showing the list of unknown flags and which apps reported them.

<img width="497" alt="image"
src="https://github.com/user-attachments/assets/b7348e0d-0163-4be4-a7f8-c072e8464331"
/>
2025-05-07 11:48:36 +01:00
Mateusz Kwasniewski
bb82b6920b
feat: feature link migration (#9900) 2025-05-06 11:21:20 +02:00
Mateusz Kwasniewski
085c62c99a
feat: client instances sdk type (#9844) 2025-04-25 12:20:48 +02:00
Mateusz Kwasniewski
497cbcdef2
feat: environment required approvals migration (#9612) 2025-03-25 15:33:58 +01:00
Gastón Fournier
92a13c4c55
fix: all users have a root role and warning if not (#9584)
## About the changes
SCIM provisioned users ended up without a root role. Unleash was
assigning them the Viewer role by code but some queries using the db to
resolve the role did not have the same logic leading to weird behaviors.

This amends the situation by assigning the Viewer role to those users
following the least privilege principle.

Also adds a warning when assuming the Viewer role. That should never
happen but we want to be confident before removing it.

Depends on
https://github.com/bricks-software/unleash-enterprise/pull/164
2025-03-20 13:59:37 +01:00
Fredrik Strand Oseberg
9bdad5ee44
Feat/tag colors migration (#9560)
Adds a color field to the tag types table
2025-03-18 15:01:59 +01:00
Jaanus Sellin
482443f373
feat: frontend consumption table (#9523)
Similar to backend traffic, we have table for frontend traffic. We might
want to rename the backend on to align with this one.
2025-03-12 16:44:01 +02:00
Vignesh S
ffb17d43ba
fix: features table migrations in 20220603081324-add-archive-at-to-fea… (#9518)
Changes to migration file -
20220603081324-add-archive-at-to-feature-toggle.js

1. Fixes the migration script where features table is updated with
archived_at column with the latest date instead of taking the date from
the events table.
2. Also it fails for the latest toggle which was archived and then
revived but after migration it updates the toggle as archived toggle.
3. Also because of the buggy reference to the outer join's events table
`e` its taking a huge time to run the migration.

This PR fixes all the above issues.

## About the changes

We have faced an issue during migration of unleash-server from 4.8.2 to
6.6.0 where one of our toggle which was archived and unarchived once
before migration and that was the latest toggle in terms of the archived
date, but after the migration was ran it was in archived status.

Upon further debugging and running the SQL command in the features table
migration file we noticed that we should not be referencing the outer
join's events table `e` for the feature_name check and additionally we
should add the features table's archived toggle check instead.

### Important files

The change in only in this file.


[20220603081324-add-archive-at-to-feature-toggle.js](https://github.com/Unleash/unleash/pull/9518/files#diff-b91c299b96edc46ca3a1963bf54966aa777c9fa107f3bd8b45f5fb54dc57460e)

## Discussion points

Let me know if any further details is required.
2025-03-12 10:28:04 +01:00
Mateusz Kwasniewski
dfed9c0773
feat: create connection count consumption table (#9444) 2025-03-10 12:51:11 +01:00
David Leek
2ae447cd40
chore: migration for new archive columns on release defintion (#9412) 2025-03-04 14:36:30 +01:00
Christopher Kolstad
edd03d02dd
task: add status_code to edge traffic table to store 304s as well (#9312) 2025-02-14 14:52:45 +01:00
Christopher Kolstad
1dad28fdb5
task: add edge observability tables (#9307)
Adds two stat tables to store edge observability data.
2025-02-14 08:48:13 +00:00
David Leek
5921f0ea09
chore: add migration that backfills scim user email hashes (#9295) 2025-02-12 09:05:36 +01:00
David Leek
9a8607b07e
chore: clear scim fields when deleting user + migration for existing cases (#9217) 2025-02-05 15:45:51 +01:00
Nuno Góis
138410eafd
chore: drop release plan template view permissions (#9195)
https://linear.app/unleash/issue/2-3225/drop-the-release-template-view-permissions

Drops the release plan template view permissions in favor of an "open by
default" approach.

Should merge the Enterprise PR first.
2025-02-03 16:24:25 +00:00
Tymoteusz Czech
af1b6c8c37
migration: cors root role permission (#9080)
after https://github.com/Unleash/unleash/pull/8996
2025-01-13 09:19:27 +01:00
Mateusz Kwasniewski
67068afb13
chore: fix migration file (#9079) 2025-01-10 13:41:07 +01:00
Mateusz Kwasniewski
f47de55d67
chore: fix migration file (#9078) 2025-01-10 13:36:34 +01:00
Mateusz Kwasniewski
5a1f1a00ba
feat: migration for unique connections (#9076) 2025-01-10 13:25:09 +01:00
Tymoteusz Czech
608a3986af
migration: read logs permission (#9049)
continuation of https://github.com/Unleash/unleash/pull/8996
2025-01-08 10:52:41 +01:00
Fredrik Strand Oseberg
2363d99fe7
feat: add migration (#8891) 2025-01-06 14:38:56 +01:00
Nuno Góis
adaf91a791
chore: remove Unleash AI (#9010)
https://linear.app/unleash/issue/2-3071/finish-experiment

Removes Unleash AI.

Also removes other related changes made during the experiment
development.
2024-12-20 11:02:49 +00:00
Tymoteusz Czech
4fb5f6a96c
migration: add auth config permission (#8988)
after https://github.com/Unleash/unleash/pull/8987
2024-12-17 10:33:14 +01:00
Tymoteusz Czech
67864e7008
migration: add permissions for instance maintenance (#8885)
after #8875
2024-12-12 16:53:46 +01:00
Jaanus Sellin
17d3b5c2fb
fix: make project ui query optimized (#8961)
From 13 seconds to 0.1 seconds.

1. Joining 1 million events to projects/features is slow. **Solved by
using CTE.**
2. Running grouping on 1 million rows is slow. **Solved by adding
index.**
2024-12-12 08:41:10 +02:00
Nuno Góis
c9110224a5
chore: filter out milestone strategies in features_view (#8883)
https://linear.app/unleash/issue/2-3033/filter-out-release-plan-milestone-strategies-from-admin-feature

Filters out milestone strategies from our `features_view`.

This felt like the most elegant way to address this, since it seems we
only rely on this view to fetch the feature strategies in the admin UI.


### No more milestone strategies showing up on our UI when we have a
running milestone


![image](https://github.com/user-attachments/assets/02bac5a5-7ddb-4bde-b487-8b6bca0923b5)

### However they're still part of the client features response

```json
{
	"name": "r-plan",
	"type": "release",
	"enabled": false,
	"project": "default",
	"stale": false,
	"strategies": [
		{
			"name": "flexibleRollout",
			"constraints": [
				{
					"values": [
						"Portugal"
					],
					"inverted": false,
					"operator": "IN",
					"contextName": "country",
					"caseInsensitive": false
				},
				{
					"values": [
						"Portugal",
						"Norway"
					],
					"inverted": false,
					"operator": "IN",
					"contextName": "country",
					"caseInsensitive": false
				}
			],
			"parameters": {
				"groupId": "newOverview",
				"rollout": "100",
				"stickiness": "default"
			},
			"variants": [
				{
					"name": "A",
					"weight": 500,
					"stickiness": "default",
					"weightType": "variable"
				},
				{
					"name": "B",
					"weight": 500,
					"stickiness": "default",
					"weightType": "variable"
				}
			]
		},
		{
			"name": "flexibleRollout",
			"constraints": [],
			"parameters": {
				"groupId": "much_feature",
				"rollout": "25",
				"stickiness": "default"
			},
			"variants": []
		},
		{
			"name": "flexibleRollout",
			"constraints": [],
			"parameters": {
				"groupId": "r-plan",
				"rollout": "100",
				"stickiness": "default"
			},
			"variants": []
		}
	],
	"variants": [],
	"description": null,
	"impressionData": false
},
```
2024-12-02 08:17:17 +00:00
David Leek
db02918a35
chore: make milestone_strategies.title nullable (#8864) 2024-11-27 09:16:24 +01:00
Christopher Kolstad
0f76a8a5a6
task: added unique index for release plan templates (#8846)
We want to prevent our users from defining multiple templates with the
same name. So this adds a unique index on the name column when
discriminator is template.
2024-11-25 11:25:55 +01:00
Jaanus Sellin
4234020b8d
feat: backfill licensed users (#8791)
**This migration introduces a query that calculates the licensed user
counts and inserts them into the licensed_users table.**

**The logic ensures that:**

1. All users created up to a specific date are included as active users
until they are explicitly deleted.
2. Deleted users are excluded after their deletion date, except when
their deletion date falls within the last 30 days or before their
creation date.
3. The migration avoids duplicating data by ensuring records are only
inserted if they don’t already exist in the licensed_users table.


**Logic Breakdown:**

**Identify User Events (user_events):** Extracts email addresses from
user-related events (user-created and user-deleted) and tracks the type
and timestamp of the event. This step ensures the ability to
differentiate between user creation and deletion activities.
**Generate a Date Range (dates):** Creates a continuous range of dates
spanning from the earliest recorded event up to the current date. This
ensures we analyze every date, even those without events.
**Determine Active Users (active_emails):** Links dates with user events
to calculate the status of each email address (active or deleted) on a
given day. This step handles:

- The user's creation date.
- The user's deletion date (if applicable).

**Calculate Daily Active User Counts (result):** 
For each date, counts the distinct email addresses that are active based
on the conditions:

- The user has no deletion date.
- The user's deletion date is within the last 30 days relative to the
current date.
- The user's creation date is before the deletion date.
2024-11-20 09:10:07 +02:00
Thomas Heartman
82e752be45
1-3131: db migration to make potentially stale non-nullable (#8796)
This change adds a db migration to make the potentially_stale column
non-nullable. It'll set any NULL values to `false`.

In the down-migration, make the column nullable again.
2024-11-19 12:27:22 +01:00