1
0
mirror of https://github.com/Unleash/unleash.git synced 2024-12-22 19:07:54 +01:00
Commit Graph

1821 Commits

Author SHA1 Message Date
Jaanus Sellin
7baed29c07
feat: application overview schema (#6295) 2024-02-21 12:59:55 +02:00
David Leek
869e33138b
chore: adds killswitch to scheduled per sec caches (#6291)
## About the changes

Adds killswitches to update revision id and publish unpublished events
2024-02-21 10:12:23 +01:00
Mateusz Kwasniewski
e5c07f00cb
feat: rate limit password reset attempts (#6257) 2024-02-21 08:49:54 +01:00
Nuno Góis
17c0d7137f
chore: add action state db indexes (#6283)
Adds some relevant indexes to `action_states`, useful for our new
"action events" query.
2024-02-20 15:45:33 +00:00
Christopher Kolstad
7350c91a1e
fix: display all roles if we can't get project roles 2024-02-20 16:23:25 +01:00
Christopher Kolstad
e9d9db17fe
feat: Adding Project access requires same role (#6270)
In order to prevent users from being able to assign roles/permissions
they don't have, this PR adds a check that the user performing the
action either is Admin, Project owner or has the same role they are
trying to grant/add.

This addAccess method is only used from Enterprise, so there will be a
separate PR there, updating how we return the roles list for a user, so
that our frontend can only present the roles a user is actually allowed
to grant.

This adds the validation to the backend to ensure that even if the
frontend thinks we're allowed to add any role to any user here, the
backend can be smart enough to stop it.

We should still update frontend as well, so that it doesn't look like we
can add roles we won't be allowed to.
2024-02-20 15:56:53 +01:00
Gastón Fournier
9c156ac31f
chore: avoid printing out warnings from known frontend proxies (#6271)
## About the changes
Our frontend API creates new instances of unleash-client-proxy. Because
this is by-design, we don't want to log a warning that was designed to
warn users about potential misconfiguration of Unleash Proxy.

As an extra, I'm renaming ProxyController to FrontendAPIController to
better reflect the intent of this controller.
2024-02-20 11:27:21 +01:00
Jaanus Sellin
03929e3031
feat: project applications UI (#6260)
![image](https://github.com/Unleash/unleash/assets/964450/a1129857-820c-4e93-ac59-ef5f4743d774)
2024-02-19 09:50:53 +02:00
Nuno Góis
5f781b4c8f
refactor: better prom metric helper types (#6261)
Improves typing in our Prometheus metric helpers.
2024-02-16 15:11:29 +00:00
Tymoteusz Czech
b02f8005f1
migration: flag-trends - add users column (#6254)
Add `users` column to per-project trends.
2024-02-16 15:42:47 +01:00
Gastón Fournier
7a48fb57a6
feat: permission matrix (PoC) (#6223)
## About the changes
This is a rough initial version as a PoC for a permission matrix. 

This is only available after enabling the flag `userAccessUIEnabled`
that is set to true by default in local development.

The access was added to the users' admin page but could be embedded in
different contexts (e.g. when assigning a role to a user):

![image](https://github.com/Unleash/unleash/assets/455064/3f541f46-99bb-409b-a0fe-13f5d3f9572a)


This is how the matrix looks like

![screencapture-localhost-3000-admin-users-3-access-2024-02-13-12_15_44](https://github.com/Unleash/unleash/assets/455064/183deeb6-a0dc-470f-924c-f435c6196407)

---------

Co-authored-by: Nuno Góis <github@nunogois.com>
2024-02-16 14:31:33 +01:00
Ivar Conradi Østhus
4a81f0932f
fix: Allow AuthType None to use valid API tokens (#6247)
Fixes ##5799 and #5785

When you do not provide a token we should resolve to the "default"
environment to maintain backward compatibility. If you actually provide
a token we should prefer that and even block the request if it is not
valid.

An interesting fact is that "default" environment is not available on a
fresh installation of Unleash. This means that you need to provide a
token to actually get access to toggle configurations.


---------

Co-authored-by: Thomas Heartman <thomas@getunleash.io>
2024-02-16 08:24:56 +00:00
Mateusz Kwasniewski
f0d5c8e3d1
fix: include first day of range in the extended metrics (#6245) 2024-02-16 08:36:25 +01:00
Nuno Góis
e4649e6421
chore: add call incoming webhook rate limit metric (#6252)
Follow up to https://github.com/Unleash/unleash/pull/6248 - Add this
rate limit to metrics.


![image](https://github.com/Unleash/unleash/assets/14320932/6b458f5d-2e13-4ac4-8331-7950e7784585)
2024-02-15 17:05:52 +00:00
David Leek
cb53df6176
chore: emit PROXY_REPOSITORY_CREATED event when creating new repo (#6251)
## About the changes

- Emits a new event on the eventBus when Proxy-service creates a new
repository for a frontend token
- Adds a prometheus metrics counter for created proxy-repositories


![image](https://github.com/Unleash/unleash/assets/707867/85a84fa7-4f03-4dc1-b0ba-3ffd2477045b)
2024-02-15 14:58:48 +01:00
Nuno Góis
ef8d2edcc0
chore: rate limit calling inc webhooks (#6248)
https://linear.app/unleash/issue/2-1942/rate-limit-incoming-webhooks-call-endpoint

Adds a configurable rate limit to calling incoming webhooks. We're
setting a 1RPS limit for now, but I'm open to suggestions.
2024-02-15 10:25:32 +00:00
Jaanus Sellin
8dc27204d1
feat: add gen:api:clean for clean orval schemas (#6244)
Created a build script that generates orval schemas with automatic
cleanup. Also generating new ones.

1. yarn gen:api **(generates schemas)**
2. rm -rf src/openapi/apis **(remove apis)**
3.  sed -i '1q' src/openapi/index.ts **(remove all rows except first)**
2024-02-15 11:45:35 +02:00
Mateusz Kwasniewski
2999f8df2d
fix: remove jitter from time sensitive scheduled jobs (#6240) 2024-02-14 15:10:44 +01:00
Jaanus Sellin
3d77825493
feat: project applications server side paging and sorting and filtering (#6236)
Uses exactly same pattern as search-store. Nothing too crazy here.
Most code is in tests.
2024-02-14 13:03:44 +02:00
Thomas Heartman
7eb9a01bda
chore: extract and export type used by CR emails (#6234)
This change takes the (now rather involved) type used to send CR
schedule suspension emails and extracts it into a proper exported type.

This will allow us to import it in enterprise as well instead of
redefining it.
2024-02-14 10:38:13 +09:00
Thomas Heartman
e366e48b4c
feat: support environment variant updates in email service (#6183)
This change updates the email service's `sendScheduleSuspendedEmail`
method to support environment variants being changed.
2024-02-14 10:11:39 +09:00
Jaanus Sellin
eb5d7a3788
feat: sdk reporting flag and e2e test (#6216)
1. Add flag
2. Add e2e test with more complete example
3. Some bug fixes
2024-02-13 14:13:21 +02:00
Nicolae Socaciu
12d2a1ba63
fix-css-update-for-CR-email-template (#6186)
Fix: removing a css line that was blocking some style
Improvement: clean up some unnecessary comments 

Before the fix:

![image](https://github.com/Unleash/unleash/assets/103567375/7a94c20d-ea7c-40cd-b207-128ab5674e97)


after:

![image](https://github.com/Unleash/unleash/assets/103567375/11ebb46e-c724-4b38-91f6-f2f3e4aff9e6)
2024-02-13 10:16:25 +09:00
Jaanus Sellin
5a75093cbc
feat: project applications e2e PoC (#6189)
1. Adding store layer
2. Updating schemas
3. Refactoring project files that I touched into feature oriented
architecture

Next steps E2E tests.
2024-02-12 16:00:59 +02:00
David Leek
ccd2fee4ee
feat: implement a store for stat_traffic_data (#6190)
## About the changes

Implements a new store for collected traffic data usage that connects to
the new table `stat_traffic_data` primary key'd on [day, trafficGroup,
status_code_series].

Day being a date
Traffic group being which endpoint is being counted for, ie /api/admin,
/api/frontend etc
Status code series grouping 2xx status responses and 304 into their
respective 200 / 300 series.

No service here, this is for pro/enterprise
2024-02-12 08:39:51 +01:00
Fredrik Strand Oseberg
260ef70309
Feat/UI error observability (#6169)
This PR adds an endpoint to Unleash that accepts an error message and
option error stack and logs it as an error. This allows us to leverage
errors in logs observability to catch UI errors consistently.

Considered a test, but this endpoint only accepts and logs input, so I'm
not sure how useful it would be.
2024-02-09 13:07:44 +01:00
Jaanus Sellin
4972b9686c
feat: project applications controller/service layer (#6184)
Just adding controller/service layer, connecting with schema.
Next PR will implement store and e2e tests.
2024-02-09 13:18:26 +02:00
David Leek
1b1bde8aec
chore: add migration for traffic data collection (#6171)
## About the changes

Adds migration for creating table `stat_traffic_usage`.
This table primary-keys on day, traffic_group, and status_code_series.
Adds individual indexes for day, traffic_group, and status_code_series.

Traffic group is the grouping for API endpoints for which traffic is
counted.
status_code_series is 200/202 etc = 200, 304 etc = 300
2024-02-09 09:58:58 +01:00
Nuno Góis
4c1dfbefa7
chore: clean up customRootRolesKillSwitch (#6173)
https://linear.app/unleash/issue/2-1308/remove-customrootroleskillswitch-flag

Cleans up the `customRootRolesKillSwitch` flag.
2024-02-09 08:41:40 +00:00
Gastón Fournier
fa3352786a
chore: reimplementation of app stats (#6155)
## About the changes
App stats is mainly used to cap the number of applications reported to
Unleash based on the last 7 days information:
cc2ccb1134/src/lib/middleware/response-time-metrics.ts (L24-L28)

Instead of getting all stats, just calculate appCount statistics

Use scheduler service instead of setInterval
2024-02-08 17:15:42 +01:00
Jaanus Sellin
4a4196c66a
feat: preliminary project applications schema (#6152)
Currently keeping flat structure and separate out into multiple
requests(schemas) if needed.
In future, we will also add dates.
2024-02-08 16:16:55 +02:00
David Leek
bb2f88980c
chore: add feature flag for request counting inside Unleash (#6164)
## About the changes

Adds the feature flag for use in request traffic counting
2024-02-08 14:46:54 +01:00
Jaanus Sellin
ecc200bf8a
feat: add index on events for faster revision id (#6170)
On all pods and instances, we run the same revision update query every
second. It is relatively fast when the application has started. This is
the single most ran query in unleash.
Benchmarks:

1. Running pod with existing revisionID:
  - old 5.5ms
  - new 0.028ms

2. New pod without existing revisionID
 - old 9.329ms
 - new 0.033ms



This query is getting optimized


7e66a79f9f/src/lib/features/events/event-store.ts (L161)
2024-02-08 14:11:58 +02:00
Jaanus Sellin
bc7d4b8edb
feat: add partial index on client_applications annouced FALSE (#6166)
We have customers with tens or hundreds of thousands of applications,
and we have a scheduler running that sets application fields to
`announced` as true. However, every time it runs, it queries the entire
table, which is slow and causes database connection acquisition issues.
To make it faster, we added a partial index to the table.
2024-02-08 10:57:10 +02:00
Thomas Heartman
c43d7c0653
chore: update schedule emails to handle suspensions and segment updates (#6145)
This PR updates the change request email sending method to handle the
recent changes we have made. That means that the email now:
- says that change requests have been suspended instead of saying that
application will fail.
- handles cases where segments or strategies have been updated causing
potential conflicts.

I have updated the email templates and made some adjustments to the
email sending method. To make the transition from one to the other
easier, I have kept the original method as an interim solution until
enterprise has switched over.
2024-02-08 17:30:42 +09:00
andreas-unleash
59a736f56b
chore: add inMemoryScheduledChangeRequests flag (#6151)
What it says on the box

Closes #
[1-2030](https://linear.app/unleash/issue/1-2030/create-inmemoryscheduledchangerequests-flag)

---------

Signed-off-by: andreas-unleash <andreas@getunleash.ai>
2024-02-07 16:35:14 +02:00
Jaanus Sellin
b9a8280177
fix: now archived features are not marked as stale anymore (#6149) 2024-02-07 12:53:25 +02:00
Mateusz Kwasniewski
7e9958d8b6
feat: sdk with double start prevention (#6146) 2024-02-07 10:01:05 +01:00
Gastón Fournier
c69f41e4ce
fix: error logging properly (#6144) 2024-02-06 16:39:57 +01:00
Jaanus Sellin
b45f7be680
fix: default featureSearchFeedback to false (#6137)
We need to turn it false, because if it is true, it will not get any
variants.

This solution is hacky, but good for now.
2024-02-06 15:10:27 +00:00
Mateusz Kwasniewski
cc060b7a4c
fix: proxy repository error handling (#6142) 2024-02-06 15:44:25 +01:00
Gastón Fournier
067d130a8b
chore: memoizee active tokens (#6135)
## About the changes
getAllActive from api-tokens store is the second most frequent query

![image](https://github.com/Unleash/unleash/assets/455064/63c5ae76-bb62-41b2-95b4-82aca59a7c16)

To prevent starving our db connections, we can cache this data that
rarely changes and clear the cache when we see changes. Because we will
only clear changes in the node receiving the change we're only caching
the data for 1 minute.

This should give us some room to test if this solution will work

---------

Co-authored-by: Nuno Góis <github@nunogois.com>
2024-02-06 15:14:08 +01:00
Mateusz Kwasniewski
6d94036683
fix: disable double start (#6141) 2024-02-06 13:39:57 +01:00
Christopher Kolstad
3bea6bbd20
fix: filter out service and system users from inactive users list (#6134) 2024-02-05 16:41:51 +01:00
Christopher Kolstad
1da59abb2d
fix: inactive users query was too wide (#6133)
Knex wasn't formatting the query like I expected. This changes the query
to use more AND, less ORs
2024-02-05 14:31:04 +00:00
Christopher Kolstad
ea38877b0c
feat: List and delete inactive users
Adds a new Inactive Users list component to admin/users for easier cleanup of users that are counted as inactive: No sign of activity (logins or api token usage) in the last 180 days.

---------

Co-authored-by: David Leek <david@getunleash.io>
2024-02-05 14:07:38 +01:00
Ivar Conradi Østhus
897500dd54
fix: version-service should not use process.nextTick (#6124)
In the beginning we used process.nextTick() as a trick to load some data
initally in the constructor of a service.

This is a bad pattern and we should generally avoid any async operations
in the constructor. Today we have two alternatives:

1. Defer loading until data is needed (wrap it in async)
2. Use the schdule-service.
2024-02-05 11:24:37 +01:00
Ivar Conradi Østhus
354b88383c
fix: schduler-service graceful shutdown (#6125)
When a stop signal is sent to Unleash the scheduler-service should
cancel any scheduled jobs. This also applies to the job scheduled for
initial execution with jitter.


We observed that initial jobs was executed after the database
connections are terminated. This appears after v5.9.0 of Unleash.

```
Error: aborted
    at Object.queryBuilder (/unleash/node_modules/knex/lib/knex-builder/make-knex.js:112:26)
    at createQueryBuilder (/unleash/node_modules/knex/lib/knex-builder/make-knex.js:320:26)
    at EventStore.knex [as db] (/unleash/node_modules/knex/lib/knex-builder/make-knex.js:101:12)
    at EventStore.setUnannouncedToAnnounced (/unleash/node_modules/unleash-server/dist/lib/features/events/event-store.js:286:33)
    at EventStore.publishUnannouncedEvents (/unleash/node_modules/unleash-server/dist/lib/features/events/event-store.js:293:35)
    at EventAnnouncer.publishUnannouncedEvents (/unleash/node_modules/unleash-server/dist/lib/services/event-announcer-service.js:9:32)
    at runScheduledFunctionWithEvent (/unleash/node_modules/unleash-server/dist/lib/features/scheduler/scheduler-service.js:30:23)
    at Timeout.<anonymous> (/unleash/node_modules/unleash-server/dist/lib/features/scheduler/scheduler-service.js:50:27)
    at runNextTicks (node:internal/process/task_queues:60:5)
    at process.processTimers (node:internal/timers:509:9)
```
2024-02-05 11:04:13 +01:00
Gastón Fournier
1d18187f7d
chore: split metrics by request type (#6119)
## About the changes
Queries on client-feature-toggle store have many purposes depending on
the requestType, making the query more complex or not depending on the
use case. Also, each use case has different frequencies (i.e. playground
is expected to be used rarely).

The name for the store metrics was wrong, copy&pasted from:

7b04db0547/src/lib/features/feature-toggle/feature-toggle-store.ts (L107)

Which was also present in feature-tag metrics:

7b04db0547/src/lib/db/feature-tag-store.ts (L37)

With this, we'll have more granularity to understand the execution time
and frequency of each
2024-02-05 09:01:42 +01:00
Ivar Conradi Østhus
77b7cb03e9
fix: maintenance mode should assume disable if db call fails. (#6120)
Usually maintenance mode is disabled. If the call throws, which we see a
lot of when a unleash instance is in terminating state, we should return
a default value.

By having it throw inside of the memoizee function, the response is not
cached, and it will trigger new calls until it return a cachable result.
2024-02-03 09:17:11 +01:00