## Why
Currently AWS API Gateway doesn't have compression enabled by default,
this PR will make it easier to for example deploy Unleash over to AWS
Lambda without further configuration in API Gateway, frameworks like
Serverless requires a bit more work to set up compression and some times
one might not need compression at all.
## How
Create a new config flag called `disableCompression` which will not
include `compression` middleware in express' instance when set as true.
### What
This PR makes the rate limit for user creation and simple login (our
password based login) configurable in the same way you can do
metricsRateLimiting.
### Worth noting
In addition this PR adds a `rate_limit{endpoint, method}` prometheus
gauge, which gets the data from the UnleashConfig.
### What
The heaviest requests we serve are the register and metrics POSTs from
our SDKs/clients.
This PR adds ratelimiting to /api/client/register, /api/client/metrics,
/api/frontend/register and /api/frontend/metrics with a default set to
6000 requests per minute (or 100 rps) for each of the endpoints.
It will be overrideable by the environment variables documented.
### Points of discussion
@kwasniew already suggested using featuretoggles with variants to
control the rate per clientId. I struggled to see if we could
dynamically update the middleware after initialisation, so this attempt
will need a restart of the pod to update the request limit.
This commit changes our linter/formatter to biome (https://biomejs.dev/)
Causing our prehook to run almost instantly, and our "yarn lint" task to
run in sub 100ms.
Some trade-offs:
* Biome isn't quite as well established as ESLint
* Are we ready to install a different vscode plugin (the biome plugin)
instead of the prettier plugin
The configuration set for biome also has a set of recommended rules,
this is turned on by default, in order to get to something that was
mergeable I have turned off a couple the rules we seemed to violate the
most, that we also explicitly told eslint to ignore.
Fix issues uncovered when reviewing integrations list and form.
- YouTube CSP
- Text content and formatting
- Margins
- Update old integration icons
- Fix headers in dark theme
As part of the move to a unified domain this PR updates the default
EMAIL_SENDER to noreply@getunleash.io . Should not be merged/deployed
until we've verified DMARC, DKIM for the new domain.
As requested in
[Linear](https://linear.app/unleash/issue/2-1147/unleash-cloud-make-keepalive-configurable)
this PR makes the serverKeepAliveTimeout configurable via the
SERVER_KEEPALIVE_TIMEOUT environment variable. This was already
configurable when starting Unleash programmatically, but it's nice to
have as an env variable as well
<!-- Thanks for creating a PR! To make it easier for reviewers and
everyone else to understand what your changes relate to, please add some
relevant content to the headings below. Feel free to ignore or delete
sections that you don't think are relevant. Thank you! ❤️ -->
## About the changes
<!-- Describe the changes introduced. What are they and why are they
being introduced? Feel free to also add screenshots or steps to view the
changes if they're visual. -->
Adds an environment variable for switching off feature telemetry in
version check
After a Team Retro, one of our squads felt like we needed more data on
our test suites. This is the first effort to make our test results
easier to grab. It uses the test-reporter action to add a github check
to our main build and PR builds with our test results.
This at least should make it easier to parse which tests are failing.
However, it does not give us trends. So it does not yet make it easier
to decide which tests are flaky just from a quick view.
---------
Co-authored-by: Gastón Fournier <gaston@getunleash.io>
This PR reuses the revision Id information from the "optimal 304 for
server SDKs" to improve the freshness of the frontend API config data.
In addition it allows us to reduce the polling (and eventually remove it
when we are confident).
---------
Co-authored-by: Gastón Fournier <gaston@getunleash.io>
## About the changes
This PR removes the optimal304 flag after being tested in production.
We're keeping the existing configuration that allows users to disable
cache mainly because it's useful for testing.
<!-- Thanks for creating a PR! To make it easier for reviewers and
everyone else to understand what your changes relate to, please add some
relevant content to the headings below. Feel free to ignore or delete
sections that you don't think are relevant. Thank you! ❤️ -->
## About the changes
<!-- Describe the changes introduced. What are they and why are they
being introduced? Feel free to also add screenshots or steps to view the
changes if they're visual. -->
This deprecates the `username` properties on api-token schemas, and adds
a `tokenName` property.
DB field `username` has been renamed to `token_name`, migration added
for the rename.
Both `username` and `tokenName` can be used when consuming the service,
but only one of them.
## Discussion points
<!-- Anything about the PR you'd like to discuss before it gets merged?
Got any questions or doubts? -->
There's a couple of things I'd like to get opinions on and discuss:
- Frontend still uses the deprecated `username` property
- ApiTokenSchema is used both for input and output of `Create`
controller endpoints and should be split out into separate schemas. I'll
set up a task for this
---------
Co-authored-by: Thomas Heartman <thomas@getunleash.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
### What
We've had this marked as deprecated through our v4, this PR removes it.
### Worth noting
This updates the deprecation notices with removal notices in the
documentation as well.
### Considerations
The tags API is still located under
/api/admin/features/{featureName}/tags. It should be moved to
/api/admin/projects/{project}/features/{featureName}/tags. I vote we do
that in a separate PR, we'd probably also need to deprecate the existing
tags endpoints for v5 and remove in v6. We could use 308s to signify
that they are moved.
---------
Co-authored-by: Thomas Heartman <thomas@getunleash.ai>
## About the changes
Ref:
https://docs.getunleash.io/reference/deploy/configuring-unleash#further-customization
> **eventHook** (`function(event, data)`) - (_deprecated in Unleash 4.3_
in favor of the [Webhook addon](../addons/webhook.md)) If provided, this
function will be invoked whenever a feature is mutated. The possible
values for `event` are `'feature-created'`, `'feature-archived'` and
`'feature-revived'`. The `data` argument contains information about the
mutation. Its fields are `type` (string) - the event type (same as
`event`); `createdBy` (string) - the user who performed the mutation;
`data` - the contents of the change. The contents in `data` differs
based on the event type; For `'feature-archived'` and
`'feature-revived'`, the only field will be `name` - the name of the
feature. For `'feature-created'` the data follows a schema defined in
the code
[here](7b7f0b84e8/src/lib/schema/feature-schema.ts (L77)).
See an [api here](/reference/api/legacy/unleash/admin/events).
Related to: https://github.com/Unleash/unleash/issues/1265
## About the changes
This connects our backend with Prometheus (or compatible) metrics
service, and exposes raw data (i.e. acting as a proxy)
Co-authored-by: Christopher Kolstad <chriswk@getunleash.ai>
Is protected behind a config option called `enableHeapSnapshotEnpoint`.
When API endpoint is triggered it will write a heapSnapshot to disk.
Steps to create heap snapshot:
1. Add env variable `ENABLE_HEAP_SNAPSHOT_ENPOINT=true`.
2. Call this endpoint: `/internal-backstage/heapSnapshot`
3. Download the created heapdump from the server.
The snapshot can be loaded in the chrome dev-tool to analysis.
* This PR adds a configurable maxAge header to the CORS middleware. This
allows the preflight request to be cached so that we can reduce the
request load on our end for the frontend clients starting to utilise the
frontend api.
From Thomas and mine testing. BaseUriPath can't be set as an environment
variable because we override it when trying to format the URI from the
server config. This PR makes sure we only format if the custom server
options actually have baseUriPath set.
Co-authored-by: Thomas Heartman <thomas@getunleash.ai>
* fix: remove unused exp flag
* fix: remove unused flag
* fix: add support for external flag resolver
* fix: rename flagsresolver to flagresolver
* fix: disable external flag resolver
* fix: refactor a bit
* fix: stop using unleash in server-dev
* fix: remove userGroups flag
* fix: revert bumping frontend
* refactor: batch feature metric inserts
* fix: use startOfHour when collapsing metrics
* refactor: avoid extra loop to sum yes/no values
* refactor: add experimental flag for batching metrics
* feat: add support for handling non standard postgres dates
* docs: update some http docs links to point to their respective https versions
* chore: refactor non standard date handling callback code to be a little clearer
* Update website/docs/deploy/configuring-unleash-v3.md
Co-authored-by: Thomas Heartman <thomas@getunleash.ai>
Co-authored-by: Thomas Heartman <thomas@getunleash.ai>
* implemented changes to resolve issue 1170
* added applicationName to the list of db options in the documentation'
Co-authored-by: Daniele Casal <daniele.casal@lloydsbanking.com>
Co-authored-by: Sukhvinder Panesar <79143027+esspee-lbg@users.noreply.github.com>
This adds support for multi project tokens to be created. Backward compatibility is handled at 3 different layers here:
- The API is made backwards compatible though a permissive data type that accepts either a project?: string or projects?: string[] property, validation is done through JOI here, which ensures that projects and project are not set together. In the case of neither, this defaults to the previous default of ALL_PROJECTS
- The service layer method to handle adding tokens has been made tolerant to either of the above case and has been deprecated, a new method supporting only the new structure of using projects has been added
- Existing compatibility for consumers of Unleash as a library should not be affected either, the ApiUser constructor is now tolerant to the the first input and will internally map to the new cleaned structure
* wip: environment for permissions
* fix: add migration for roles
* fix: connect environment with access service
* feat: add tests
* chore: Implement scaffolding for new rbac
* fix: add fake store
* feat: Add api endpoints for roles and permissions list
* feat: Add ability to provide permissions when creating a role and rename environmentName to name in the list permissions datastructure
* fix: Make project roles resolve correctly against new environments permissions structure
* fix: Patch migration to also populate permission names
* fix: Make permissions actually work with new environments
* fix: Add back to get permissions working for editor role
* fix: Removed ability to set role type through api during creation - it's now always custom
* feat: Return permissions on get role endpoint
* feat: Add in support for updating roles
* fix: Get a bunch of tests working and delete a few that make no sense anymore
* chore: A few small cleanups - remove logging and restore default on dev server config
* chore: Refactor role/access stores into more logical domains
* feat: Add in validation for roles
* feat: Patch db migration to handle old stucture
* fix: migration for project roles
* fix: patch a few broken tests
* fix: add permissions to editor
* fix: update test name
* fix: update user permission mapping
* fix: create new user
* fix: update root role test
* fix: update tests
* feat: Validation now works when updating a role
* fix: Add in very barebones down migration for rbac so that tests work
* fix: Improve responses from role resolution - getting a non existant role will throw a NotFound error
* fix: remove unused permissions
* fix: add test for connecting roles and deleting project
* fix: add test for adding a project member with a custom role
* fix: add test for changing user role
* fix: add guard for deleting role if the role is in use
* fix: alter migration
* chore: Minor code cleanups
* chore: Small code cleanups
* chore: More minor cleanups of code
* chore: Trim some dead code to make the linter happy
* feat: Schema validation for roles
* fix: setup permission for variant
* fix: remove unused import
* feat: Add cascading delete for role_permissions when deleting a role
* feat: add configuration option for disabling legacy api
* chore: update frontend to beta version
* 4.6.0-beta.0
* fix: export default project constant
* fix: update snapshot
* fix: module pattern ../../lib
* fix: move DEFAULT_PROJECT to types
* fix: remove debug logging
* fix: remove debug log state
* fix: Change permission descriptions
* fix: roles should have unique name
* fix: root roles should be connected to the default project
* fix: typo in role-schema.ts
* fix: Role permission empty string for non environment type
* feat: new permission for moving project
* fix: add event for changeProject
* fix: Removing a user from a project will now check to see if that project has an owner, rather than checking if any project has an owner
* fix: add tests for move project
* fix: Add in missing create/delete tag permissions
* fix: Removed duplicate impl caused by multiple good samaritans putting it back in!
* fix: Trim out add tag permissions, for now at least
* chore: Trim out new add and delete tag permissions - we're going with update feature instead
* chore: update frontend
* 4.6.0-beta.1
* feat: Prevent editing of built in roles
* fix: Patch an issue where permissions for variants/environments didn't match the front end
* fix: lint
Co-authored-by: Ivar Conradi Østhus <ivarconr@gmail.com>
Co-authored-by: Fredrik Oseberg <fredrik.no@gmail.com>
