We're getting security advisories against the node version we're using.
This PR bumps everything to 18.18.2/18.x again. And we'll trust our
excellent monitoring on detecting performance regressions again
## About the changes
This generates the changelog in the branch taking into account latest
release (it's conditional to wether the release ends with .0 or not)
## About the changes
Add partial index on events by announced. This should help avoid `Seq
Scan on events` when the majority of events are announced=true
---
Co-authored-by: Ivar Østhus <ivar@getunleash.io>
Co-authored-by: Gard Rimestad <gard@getunleash.io>
We are pinning node to version 18.17.1 as we have seen some performance
degregation on node 18.18.0 on arm64. We will investigate this further
at a later point. This is to mitigate the issue.
Our next step in pinpointing the issue will be to compare between
running on musl vs libc on arm64.
## What
This adds a workflow which automatically triggers on published releases
to update the version number for our version checker. In addition it
adds a workflow dispatch, in case the version number ended up wrong
after an automatic run (for instance when patching an earlier released
minor).
## Observations
Currently the version checker which receives the update only validates
that the new version is also valid semver, it does not however
check/verify that we're not suddenly telling it an older version is the
newest (improvement available in the version updater repo).
This PR adds some troubleshooting information to the website readme and
also makes it so that we get more readable errors in the build logs when
something goes wrong.
### What
To reduce the work needed, this PR adds paths-ignore to avoid building
unnecessary workflows.
Co-authored-by: Gastón Fournier <gaston@getunleash.io>
## About the changes
Avoids code injection via the committer name or email
Also removes pushing to s3 on `main` folder as it's not used and we're
already pushing to `commits` folder
## About the changes
When we do releases on the release branch we should not sync that back
to master on unleash-enterprise repo. At most we can trigger a different
process to automate the release of unleash-enterprise, but that'd be a
different story.
## About the changes
Reduce the build time of OSS docker image from
[~30m](https://github.com/Unleash/unleash/actions/workflows/docker_publish.yaml)
to [under
15m](https://github.com/Unleash/unleash/actions/runs/5222180536/jobs/9427342758)
1. Build frontend outside docker multiplatform.
2. Allow `frontend/build` to be copied to the image by removing this
from `.dockerignore`
3. Run with `--ignore-scripts` to avoid building the frontend on the
`prepare` script, but this requires us to run all the prepare scripts
manually (except the frontend build).
**Note:** we need to build frontend in the `prepare` script to be able
to have source code dependencies
## Manual Testing
Manually downloaded from
https://hub.docker.com/r/unleashorg/unleash-server/tags?page=1 and
compared both `unleash` folders from main and the version built with the
new process
https://github.com/Unleash/unleash/actions/runs/5223078089/jobs/9429430190#step:5:48
No major difference was spotted (only expected changes due to
development done in main)
**Command used to extract the contents:**
```
cd /tmp
mkdir main && cd main
docker pull unleashorg/unleash-server:main-edge-18-alpine
docker export $(docker create unleashorg/unleash-server:main-edge-18-alpine) > container.tar && tar xvf container.tar
mkdir ../new-process && cd ../new-process
docker pull unleashorg/unleash-server:sha-ccac902-18-alpine
docker export $(docker create unleashorg/unleash-server:sha-ccac902-18-alpine) > container.tar && tar xvf container.tar
meld ./unleash ../main/unleash
```
## About the changes
Running `yarn install` without the `dist` folder will trigger a `yarn
build` automatically as part of the `prepare script`
As we can see here:
https://github.com/Unleash/unleash/actions/runs/5200272004/jobs/9378770279
```
[build:frontend] ✓ built in 41.10s
// more output
[build:frontend] ✓ built in 39.81s
```
## About the changes
Previous PR https://github.com/Unleash/unleash/pull/3871 we were
supposed to change this for PRs but the change was made on
`release.yaml` file. This fixes the issue
After a Team Retro, one of our squads felt like we needed more data on
our test suites. This is the first effort to make our test results
easier to grab. It uses the test-reporter action to add a github check
to our main build and PR builds with our test results.
This at least should make it easier to parse which tests are failing.
However, it does not give us trends. So it does not yet make it easier
to decide which tests are flaky just from a quick view.
---------
Co-authored-by: Gastón Fournier <gaston@getunleash.io>
## About the changes
Delay static asset generation to speed up the CI/CD pipeline.
Next, we should add a validation step before deployment to validate that
the static assets were properly published
# Simplify package scripts
This PR's purpose is to raise a discussion surrounding our current
package scripts.
It includes some suggestions that aim to simplify the scripts and
hopefully bring a much more straightforward approach to developing and
contributing to Unleash.
Building (prod) should only happen **explicitly** and when needed.
## Before PR (current behavior)
- Clone the project;
- Open 2 terminals: One for `unleash` and another for
`unleash/frontend`;
- On `unleash`:
- Run `yarn` (which will also build, for some reason?);
- Run `yarn start:dev` to start backend in dev mode (`tsc-watch`);
- On `unleash/frontend`:
- Run `yarn` (which will also build, for some reason?);
- Run `yarn start` to start frontend in dev mode (`vite`);
So it seems to me like we build unnecessarily every time we install
dependencies. Neither dev scripts need to build the project, as backend
uses `tsc-watch` and frontend uses `vite`. I'm unsure why this is the
case, as building can take a very long time.
There's also some complexity in the way we need to split the terminal to
`cd` into `frontend` and treat it as a different project. The fact that
we have different script names is also confusing (`yarn start`, `yarn
start:dev`, etc).
## After PR
- Clone the project;
- Run `yarn` to install all dependencies;
- Run `yarn dev` to get started developing Unleash;
Running `yarn` should take care of everything needed to start
developing. This includes installing dependencies for frontend as well.
It should not build projects if we are not being explicit about it,
especially since we don't need to build them at this stage.
Running `yarn dev` should start the project in dev mode. This means
running both projects in `dev` mode, which for `backend` means running
`tsc-watch` and for `frontend` means running `vite`.
Here this PR attempts to provide a better DX by using
[concurrently](https://www.npmjs.com/package/concurrently) and
[wait-on](https://www.npmjs.com/package/wait-on) - This means both tasks
are ran simultaneously, stdout is labeled accordingly, and are stopped
together. It also means that `frontend` waits for `backend` to be
serving at `4242` before starting, since `frontend` starts pretty much
immediately with `vite` and `backend` takes a bit longer. Of course,
when the `backend` is hot-reloading you may still find some
`ECONNREFUSED`s on `frontend` stdout while it recompiles.
No more splitting your terminal and treating `frontend` as a separate
project.
## Discussion points
Maybe there's a better alternative to `tsc-watch`? I briefly explored
some alternatives and while they had a much faster starting speed,
hot-reload was sometimes slower. IMO we should aspire to run
`src/server-dev.ts` directly and only compile when needed.
Running `dev:backend` still serves a version of the frontend (at 4242).
**Why? Can we remove that behavior?**
I can't imagine a scenario in dev where we wouldn't want to run the
latest version of the frontend with `vite`.
~~**Note:** This PR removes all other out-of-scope scripts to focus on
this revamp. If we decide to merge it, we should evaluate what other
existing scripts we still want to include. May be a good opportunity to
clean up unused ones and only include the ones we really use. This
includes scripts that our GH actions rely on.~~
**Update:** In an effort to minimize impact surface of this PR and make
it a bit more ready for merging:
- It updates some docs in
2a4ff805e8
and
1bbc488251
to reflect our new simplified flow;
- It includes the old package scripts for now in
039bc04699;
- It updates some of our GH actions to reflect the new scripts in
7782cb9b12;
Given its current status I'll promote the PR to "ready for review".
I still think we should have a second look at our existing scripts and
GH actions to see what we really need and/or should adapt, but it should
be a team effort so we have a broader context. Maybe on a follow-up PR.
Does this require any changes to related projects (e.g. Enterprise)?
---------
Co-authored-by: Gastón Fournier <gaston@getunleash.io>
This PR revamps e2e tests, while adding a new one for the interactive
demo guide:
- Bumps Cypress from `9.7.0` to `12.11.0`;
- Bumps Cypress GH action from `v2` to `v5`;
- Makes any adjustments needed;
- Fixes a lot of issues identified with existing tests;
- Adds new `demo.spec.ts` e2e test that covers the entire demo guide
flow;
**Note:** Currently does not include `demo.spec.ts` in the GH action, as
it
[fails](https://github.com/Unleash/unleash/actions/runs/4896839575/jobs/8744137231?pr=3656)
on step 2.13 (last step of "user-specific" topic). It runs perfectly
fine locally, though.
Might be placebo, but in general tests seem less flaky now and they may
even be faster (especially when not adding the `demo` one, which would
always take a long time).
I've copied most of the descriptions from what we did for batch metrics
under the Edge tag, however there are a couple of top level descriptions
that are "fine" but I'm definitely open to suggestions.
---------
Co-authored-by: Thomas Heartman <thomas@getunleash.ai>
