2021-06-04 11:17:15 +02:00
|
|
|
{
|
|
|
|
|
"name": "websitev-2",
|
|
|
|
|
"version": "0.0.0",
|
|
|
|
|
"private": true,
|
2022-07-19 07:57:03 +02:00
|
|
|
"engines": {
|
|
|
|
|
"node": ">=16.14"
|
|
|
|
|
},
|
2021-06-04 11:17:15 +02:00
|
|
|
"scripts": {
|
|
|
|
|
"docusaurus": "docusaurus",
|
|
|
|
|
"start": "docusaurus start",
|
#1391: add generated doc cleaning script (#2077)
* #1391: add generated doc cleaning script
## What
The cleaning script replaces all references to the Unleash ushosted instance in the
generated OpenAPI docs. It removes extra path segments (such as leading
`/ushosted` instances) and replaces the ushosted base url with something
user-agnostic.
## Why
When we host the OpenAPI docs in our official documentation, the generated
docs shouldn't necessarily point at _one specific instance_, and especially
not one that the reader is unlikely to ever use. Instead, we can remove all
the bits that are specific to the generation source we use, and make the docs
easier to use. In particular, removing the leading `/ushosted` is likely to
save us loooots of questions.
* #1391: change env var used for generating openapi from localhost
Using NODE_ENV=development doesn't necessarily make sense, so adding
an extra variable sounds reasonable to me.
* #1391: ensure that all generation commands also clean docs
* #1391: change <your-unleash-instance-url> to <your-unleash-url>
* #1391: fix ushosted replacement: not all paths start with /api
* #1391: chore: remove potential `ushosted` ending of api url
In the event that we change the base URL of OpenAPI, so that paths
don't start with `/ushosted/`, the script should still work, changing
those paths into <your-unleash-url> too.
Additionally, remove all instances of `/ushosted` that we find. In the
event that some things switch around or whatever.
2022-09-20 12:43:39 +02:00
|
|
|
"build": "yarn generate && docusaurus build",
|
2021-06-04 11:17:15 +02:00
|
|
|
"swizzle": "docusaurus swizzle",
|
#1391: add generated doc cleaning script (#2077)
* #1391: add generated doc cleaning script
## What
The cleaning script replaces all references to the Unleash ushosted instance in the
generated OpenAPI docs. It removes extra path segments (such as leading
`/ushosted` instances) and replaces the ushosted base url with something
user-agnostic.
## Why
When we host the OpenAPI docs in our official documentation, the generated
docs shouldn't necessarily point at _one specific instance_, and especially
not one that the reader is unlikely to ever use. Instead, we can remove all
the bits that are specific to the generation source we use, and make the docs
easier to use. In particular, removing the leading `/ushosted` is likely to
save us loooots of questions.
* #1391: change env var used for generating openapi from localhost
Using NODE_ENV=development doesn't necessarily make sense, so adding
an extra variable sounds reasonable to me.
* #1391: ensure that all generation commands also clean docs
* #1391: change <your-unleash-instance-url> to <your-unleash-url>
* #1391: fix ushosted replacement: not all paths start with /api
* #1391: chore: remove potential `ushosted` ending of api url
In the event that we change the base URL of OpenAPI, so that paths
don't start with `/ushosted/`, the script should still work, changing
those paths into <your-unleash-url> too.
Additionally, remove all instances of `/ushosted` that we find. In the
event that some things switch around or whatever.
2022-09-20 12:43:39 +02:00
|
|
|
"generate": "docusaurus gen-api-docs all && node clean-generated-docs.js",
|
|
|
|
|
"deploy": "yarn generate && docusaurus deploy",
|
2021-06-04 11:17:15 +02:00
|
|
|
"clear": "docusaurus clear",
|
|
|
|
|
"serve": "docusaurus serve",
|
|
|
|
|
"write-translations": "docusaurus write-translations",
|
2022-02-17 12:30:49 +01:00
|
|
|
"write-heading-ids": "docusaurus write-heading-ids",
|
|
|
|
|
"storybook": "start-storybook -p 6006",
|
|
|
|
|
"build-storybook": "build-storybook"
|
2021-06-04 11:17:15 +02:00
|
|
|
},
|
|
|
|
|
"dependencies": {
|
chore: fix broken docs build / remove unused tag files (#2402)
## Issue
So, we've got an issue with our docs build not working. When building
for production, we get an error that looks a little bit like this:
```
$ docusaurus build
[INFO] [en] Creating an optimized production build...
✔ Client
✖ Server
Compiled with some errors in 40.85s
TypeError: source_default(...).bold is not a function
TypeError: source_default(...).bold is not a function
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.
```
Which isn't very helpful at all. If you go into
`/node_modules/@docusaurus/core/lib/client/serverEntry.js` and modify
the `render` function to log the actual error and remove anything
chalk-related, you get this instead:
```
$ docusaurus build
[INFO] [en] Creating an optimized production build...
✔ Client
✖ Server
Compiled with some errors in 44.62s
Actual error: Error: Unexpected: cant find current sidebar in context
at useCurrentSidebarCategory (main:11618:247)
at MDXContent (main:38139:1593)
at Fb (main:149154:44)
at Ib (main:149156:254)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
Actual error: Error: Unexpected: cant find current sidebar in context
at useCurrentSidebarCategory (main:11618:247)
at MDXContent (main:38513:1469)
at Fb (main:149154:44)
at Ib (main:149156:254)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
Error: Unexpected: cant find current sidebar in context
Error: Unexpected: cant find current sidebar in context
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.
```
That's better, but it's still not very clear.
## Getting more info
We've had problems with a similar error message before. Last time it was
caused by an empty file that docusaurus couldn't process.
A similar issue has also been described in [ this docusaurus GitHub
issue ](https://github.com/facebook/docusaurus/issues/7686). That's also
what gave me the idea of changing the logging in the dependency.
I'm currently unsure whether this is caused by the openapi docs or
something else. I've been in touch with the [openapi plugin
maintainer](https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/issues/323)
and he has been able to see the same error when building for prod
locally, but it was due to some old generated files.
Worth noting: this only seems to affect the prod build. Building for dev
(`yarn docusaurus start`) works just fine. It also fails locally **and**
in CI, so it _is_ an issue.
## Updating the logging
To get better logging, you can go into the
`/node_modules/@docusaurus/core/lib/client/serverEntry.js` file and
update the `render` function from
```js
export default async function render(locals) {
try {
return await doRender(locals);
}
catch (err) {
// We are not using logger in this file, because it seems to fail with some
// compilers / some polyfill methods. This is very likely a bug, but in the
// long term, when we output native ES modules in SSR, the bug will be gone.
// prettier-ignore
console.error(chalk.red(`${chalk.bold('[ERROR]')} Docusaurus server-side rendering could not render static page with path ${chalk.cyan.underline(locals.path)}.`));
const isNotDefinedErrorRegex = /(?:window|document|localStorage|navigator|alert|location|buffer|self) is not defined/i;
if (isNotDefinedErrorRegex.test(err.message)) {
// prettier-ignore
console.info(`${chalk.cyan.bold('[INFO]')} It looks like you are using code that should run on the client-side only.
To get around it, try using ${chalk.cyan('`<BrowserOnly>`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#browseronly')}) or ${chalk.cyan('`ExecutionEnvironment`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#executionenvironment')}).
It might also require to wrap your client code in ${chalk.cyan('`useEffect`')} hook and/or import a third-party library dynamically (if any).`);
}
throw err;
}
}
```
to
```js
export default async function render(locals) {
try {
return await doRender(locals);
}
catch (err) {
console.error(err)
throw err;
}
}
```
That'll yield the errors about the current sidebar in context.
## Root cause
Found the issue! 🙋🏼 It's explained in [this comment on the openapi docs
integration](https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/issues/323#issuecomment-1311549864)
for now, but in short: we have tags defined that we don't use. They're
being picked up by docusaurus, but don't have the proper context. That's
causing this.
The previously mentioned comment is included here for easy finding in
the future:
### Root cause explanation
The OpenAPI spec we use to generate the docs has a number of tags listed
at the root level. This is necessary for this plugin to pick up tag
categories and is, as far as I can tell, also how it _should_ be done.
However, not all of those tags are in use. Specifically, there's 2 tags
that are not.
When the plugin generates docs from the spec, it generates pages for all
endpoints and all tags and groups them by tag. However, it seems likely
that if a tag doesn't have any associated endpoints, then it won't get
added to the sidebar because there's no endpoint that references it.
But the doc files for these tags do end up lying around in the directory
regardless, and when docusaurus tries to pick up the files in the
generated directory, it also tries to pick up the unused tag files. But
because they're not part of a sidebar, they end up throwing errors
because they can't find the sidebar context.
### How I found it
The fact that I got more instances of the error message without the
sidebar ref than with it made me pay more attention to the number of
errors. I decided to check how many files were in the generated
directory and how many files were referenced from the generated sidebar.
Turns out the difference there was **2**: there were two generated files
in the directory that the sidebar didn't reference.
At this point, it was easy enough to try and delete those files before
rebuilding, and wouldn't you know: it worked!
### Our use case
Now, why do we have tags that are unused in the root spec? Can't we just
remove them?
That's a good question with a bit of a complicated answer. Unleash uses
an open core model and the OpenAPI integration is part of that open
core. The closed-source parts of Unleash are located in another repo and
extend the open-source distribution.
Because the OpenAPI spec is configured in the open-source part,
enterprise-only tags etc also need to be configured there. Then, when
the changes are absorbed into enterprise, we can use the tags there.
It gets more complicated because we use an enterprise instance to
generate the docs (because we want enterprise-endpoints to be listed
too). The instance uses the latest released instance of Unleash to have
the docs most accurately reflect the current state of things.
So, in this case, the tags have been added, but not yet used by any
endpoints, which suddenly causes this build failure. We can add the tags
to the enterprise-version, but the spec wouldn't be updated before the
next release regardless, which will probably be in a week or so.
This isn't an ideal setup, but .. it is what it is.
## Solutions and workarounds
As mentioned in the previous section, the reason the build was failing
was that there were unused tag files that docusaurus tried to include in
the build. Because they don't belong to a sidebar, the compilation
failed.
I've reported the issue to the openapi plugin maintainers and am waiting
for a response.
However, it seems that having unused root tags declared is invalid
according to the spec, so it's something we should look into fixing in
the future.
### Current workaround: cleaning script
The current workaround is to extend the api cleaning script to manually
remove the unused tag files.
### Ideal solution: filter root-level tags
Ideally, we shouldn't list unused OpenAPI tags on the root level at all.
However, because of the way we add root-level tags (as a predefined,
static lists, refer to `src/lib/openapi/index.ts`) and endpoints
(dynamically added at runtime) today, we don't really have a clear way
to filter the list of tags. This gets even more complicated when taking
the enterprise functionality and the potential extra tags they must
have.
This is, however, something that should definitely be looked into.
Working with OpenAPI across multipile repos is already troublesome, so
this is just yet another thing to look into.
2022-11-11 14:01:54 +01:00
|
|
|
"@docusaurus/core": "2.2.0",
|
|
|
|
|
"@docusaurus/plugin-client-redirects": "2.2.0",
|
|
|
|
|
"@docusaurus/plugin-google-analytics": "2.2.0",
|
|
|
|
|
"@docusaurus/preset-classic": "2.2.0",
|
|
|
|
|
"@docusaurus/remark-plugin-npm2yarn": "2.2.0",
|
Source proxy and Edge docs from GitHub (#3122)
## What
The main purpose of this PR is to
1. Delete the proxy docs in this repo and replace them with the proxy's
GitHub readme.
2. Add the docs for Unleash Edge.
### Detailed change description
This PR contains a lot of small changes in a large number of files. To
make it easier to get an overview, here's a detailed description of what
happens where:
#### In the `website/docs`directory
Except for the deletion of the proxy doc, all changes in this directory
are rewriting internal links, so that they point to the newly generated
document instead.
#### `package.json` and `yarn.lock`
When including the documentation for Edge, we also want to render the
mermaid diagrams it uses. Docusaurus supports this via a plugin. All
changes in these files are related to installing that plugin.
#### `docusaurus.config.js`
There's two types of changes in this file:
1. Mermaid-related changes: we ask docusaurus to render mermaid in
markdown files and add the plugin
2. Document generation. There's some rewrites to the sdk doc generation
plus an entirely new section that generates docs for Edge and the proxy
#### `sidebars.js`
Two things:
1. Add the edge docs
2. Move both the Edge and the proxy docs up a level, so that they're
directly under "reference docs" instead of nested inside "unleash
concepts".
#### In the `website/remote-content` directory
These are the remote content files. Previously, all of this lived only
in a `readme-fns.js` file, but with the introduction of Edge and proxy
docs, this has been moved into its own directory and refactored into
three files (`shared`, `sdks`, `edge-proxy`).
#### `custom.css`
Style updates to center mermaid diagrams and provide more space around
them.
#### In `static/img`
The image files that were included in the proxy doc and that have been
deleted.
## Why
For two reasons:
1. Reduce duplication for the proxy. Have one source of truth.
2. Add docs for edge.
## Discussion points and review wishes
This is a big PR, and I don't expect anyone to do a line-by-line review
of it, nor do I think that is particularly useful. Instead, I'd like to
ask reviewers to:
1. Visit the [documentation
preview](https://unleash-docs-git-docs-source-proxy-gh-unleash-team.vercel.app/reference/unleash-proxy)
and have a look at both the proxy docs and the Edge docs. Potentially
have a look at the SDK docs too to verify that everything still works.
2. Consider whether they think moving the proxy and edge docs up a level
(in the sidebar) makes sense.
3. Let me know what slug they'd prefer for the Edge docs. I've gone with
`unleash-edge` for now (so that it's
`docs.getunleash.io/reference/unleash-edge`), but we could potentially
also just use `edge`. WDYT?
4. Read through the detailed changes section.
5. Let me know if they have any other concerns or questions.
## Screenies
The new proxy doc:
The new edge doc:
2023-02-16 13:36:28 +01:00
|
|
|
"@docusaurus/theme-mermaid": "2.2.0",
|
2021-09-27 09:09:49 +02:00
|
|
|
"@mdx-js/react": "1.6.22",
|
2022-10-28 14:45:24 +02:00
|
|
|
"@svgr/webpack": "6.5.1",
|
chore: fix broken docs build / remove unused tag files (#2402)
## Issue
So, we've got an issue with our docs build not working. When building
for production, we get an error that looks a little bit like this:
```
$ docusaurus build
[INFO] [en] Creating an optimized production build...
✔ Client
✖ Server
Compiled with some errors in 40.85s
TypeError: source_default(...).bold is not a function
TypeError: source_default(...).bold is not a function
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.
```
Which isn't very helpful at all. If you go into
`/node_modules/@docusaurus/core/lib/client/serverEntry.js` and modify
the `render` function to log the actual error and remove anything
chalk-related, you get this instead:
```
$ docusaurus build
[INFO] [en] Creating an optimized production build...
✔ Client
✖ Server
Compiled with some errors in 44.62s
Actual error: Error: Unexpected: cant find current sidebar in context
at useCurrentSidebarCategory (main:11618:247)
at MDXContent (main:38139:1593)
at Fb (main:149154:44)
at Ib (main:149156:254)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
Actual error: Error: Unexpected: cant find current sidebar in context
at useCurrentSidebarCategory (main:11618:247)
at MDXContent (main:38513:1469)
at Fb (main:149154:44)
at Ib (main:149156:254)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
Error: Unexpected: cant find current sidebar in context
Error: Unexpected: cant find current sidebar in context
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.
```
That's better, but it's still not very clear.
## Getting more info
We've had problems with a similar error message before. Last time it was
caused by an empty file that docusaurus couldn't process.
A similar issue has also been described in [ this docusaurus GitHub
issue ](https://github.com/facebook/docusaurus/issues/7686). That's also
what gave me the idea of changing the logging in the dependency.
I'm currently unsure whether this is caused by the openapi docs or
something else. I've been in touch with the [openapi plugin
maintainer](https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/issues/323)
and he has been able to see the same error when building for prod
locally, but it was due to some old generated files.
Worth noting: this only seems to affect the prod build. Building for dev
(`yarn docusaurus start`) works just fine. It also fails locally **and**
in CI, so it _is_ an issue.
## Updating the logging
To get better logging, you can go into the
`/node_modules/@docusaurus/core/lib/client/serverEntry.js` file and
update the `render` function from
```js
export default async function render(locals) {
try {
return await doRender(locals);
}
catch (err) {
// We are not using logger in this file, because it seems to fail with some
// compilers / some polyfill methods. This is very likely a bug, but in the
// long term, when we output native ES modules in SSR, the bug will be gone.
// prettier-ignore
console.error(chalk.red(`${chalk.bold('[ERROR]')} Docusaurus server-side rendering could not render static page with path ${chalk.cyan.underline(locals.path)}.`));
const isNotDefinedErrorRegex = /(?:window|document|localStorage|navigator|alert|location|buffer|self) is not defined/i;
if (isNotDefinedErrorRegex.test(err.message)) {
// prettier-ignore
console.info(`${chalk.cyan.bold('[INFO]')} It looks like you are using code that should run on the client-side only.
To get around it, try using ${chalk.cyan('`<BrowserOnly>`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#browseronly')}) or ${chalk.cyan('`ExecutionEnvironment`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#executionenvironment')}).
It might also require to wrap your client code in ${chalk.cyan('`useEffect`')} hook and/or import a third-party library dynamically (if any).`);
}
throw err;
}
}
```
to
```js
export default async function render(locals) {
try {
return await doRender(locals);
}
catch (err) {
console.error(err)
throw err;
}
}
```
That'll yield the errors about the current sidebar in context.
## Root cause
Found the issue! 🙋🏼 It's explained in [this comment on the openapi docs
integration](https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/issues/323#issuecomment-1311549864)
for now, but in short: we have tags defined that we don't use. They're
being picked up by docusaurus, but don't have the proper context. That's
causing this.
The previously mentioned comment is included here for easy finding in
the future:
### Root cause explanation
The OpenAPI spec we use to generate the docs has a number of tags listed
at the root level. This is necessary for this plugin to pick up tag
categories and is, as far as I can tell, also how it _should_ be done.
However, not all of those tags are in use. Specifically, there's 2 tags
that are not.
When the plugin generates docs from the spec, it generates pages for all
endpoints and all tags and groups them by tag. However, it seems likely
that if a tag doesn't have any associated endpoints, then it won't get
added to the sidebar because there's no endpoint that references it.
But the doc files for these tags do end up lying around in the directory
regardless, and when docusaurus tries to pick up the files in the
generated directory, it also tries to pick up the unused tag files. But
because they're not part of a sidebar, they end up throwing errors
because they can't find the sidebar context.
### How I found it
The fact that I got more instances of the error message without the
sidebar ref than with it made me pay more attention to the number of
errors. I decided to check how many files were in the generated
directory and how many files were referenced from the generated sidebar.
Turns out the difference there was **2**: there were two generated files
in the directory that the sidebar didn't reference.
At this point, it was easy enough to try and delete those files before
rebuilding, and wouldn't you know: it worked!
### Our use case
Now, why do we have tags that are unused in the root spec? Can't we just
remove them?
That's a good question with a bit of a complicated answer. Unleash uses
an open core model and the OpenAPI integration is part of that open
core. The closed-source parts of Unleash are located in another repo and
extend the open-source distribution.
Because the OpenAPI spec is configured in the open-source part,
enterprise-only tags etc also need to be configured there. Then, when
the changes are absorbed into enterprise, we can use the tags there.
It gets more complicated because we use an enterprise instance to
generate the docs (because we want enterprise-endpoints to be listed
too). The instance uses the latest released instance of Unleash to have
the docs most accurately reflect the current state of things.
So, in this case, the tags have been added, but not yet used by any
endpoints, which suddenly causes this build failure. We can add the tags
to the enterprise-version, but the spec wouldn't be updated before the
next release regardless, which will probably be in a week or so.
This isn't an ideal setup, but .. it is what it is.
## Solutions and workarounds
As mentioned in the previous section, the reason the build was failing
was that there were unused tag files that docusaurus tried to include in
the build. Because they don't belong to a sidebar, the compilation
failed.
I've reported the issue to the openapi plugin maintainers and am waiting
for a response.
However, it seems that having unused root tags declared is invalid
according to the spec, so it's something we should look into fixing in
the future.
### Current workaround: cleaning script
The current workaround is to extend the api cleaning script to manually
remove the unused tag files.
### Ideal solution: filter root-level tags
Ideally, we shouldn't list unused OpenAPI tags on the root level at all.
However, because of the way we add root-level tags (as a predefined,
static lists, refer to `src/lib/openapi/index.ts`) and endpoints
(dynamically added at runtime) today, we don't really have a clear way
to filter the list of tags. This gets even more complicated when taking
the enterprise functionality and the potential extra tags they must
have.
This is, however, something that should definitely be looked into.
Working with OpenAPI across multipile repos is already troublesome, so
this is just yet another thing to look into.
2022-11-11 14:01:54 +01:00
|
|
|
"browserslist": "^4.16.5",
|
2022-07-11 23:57:11 +02:00
|
|
|
"clsx": "1.2.1",
|
2023-02-14 03:01:25 +01:00
|
|
|
"docusaurus-plugin-openapi-docs": "1.5.2",
|
docs: Use Go readme (#2816)
# PR 1: add remote content plugin and rust readme
## What
This PR does a few connected things:
1. It adds the ["docusaurus-plugin-remote-content" package](https://github.com/rdilweb/docusaurus-plugin-remote-content).
2. It adds configuration to make it work with Readmes found on GitHub.
3. It adds the Rust SDK's readme (replacing the link we used to have) as a proof of concept on how to do it.
## Why
With documentation split between GitHub readmes and the official docs, it's hard to keep everything up to date and in sync. It's also quite confusing that some information is only available in some places, but not in others.
We've talked about auto-including readmes from GitHub for a while, so here's a proof of concept (finally) 🥳
The intention is to get this merged and then to migrate the other SDK docs one by one, ensuring that everything in the documentation is also in the readme (so that no info is lost).
## Discussion points
### Generation directory
The current generation method generates the files into `/reference/sdks/<sdk name>`. I think this works for now, but it means it adds auto-generated files into a directory that you can't ignore (at least not yet).
We could instead generate them into `/generated/sdks` and update the slugs so that they still match the expected pattern.
However, this would make the sidebar a little harder to work with (for now). That said, there may be ways around it. It's worth exploring.
### Generation method
By default, this plugin will generate files whenever you build. That (probably) means that you need an internet connection _and_ that you'll end up with a bunch of untracked files.
An option is to only generate the files "manually" and commit them to the repo. That would allow you to build the project without an internet connection and would also remove the need for ignoring the files. We could automate the generation if we wanted to.
## Preview / Screenies
Visit [/reference/sdks/rust](https://unleash-docs-git-docs-include-sdk-readmes-unleash-team.vercel.app/reference/sdks/rust) in the preview to see what it looks like live.
# PR 2: add go readme
This PR changes the docs generation to use the Go SDK's GitHub readme
for the SDK docs instead of a separate document.
## What
The changes in this PR are:
- Delete the existing Go SDK documentation. All the content in this
guide already exists in the Go readme.
- Add the Go SDK to the list of auto-generated readme docs
- Move the readme-related code into a separate module, `readme-fns.js`
(I'm not bullish about the file name: we can change it if you have
suggestions)
- Add a note to the top of all generated readmes saying you'll need an
API url and an API token. The note also links you to the relevant
reference and how-to docs.
## Why
Having two different bits of documentation for the same SDK is
troublesome. By only having the data in one place, we can avoid it going
out of sync and getting stale.
2023-01-05 10:47:49 +01:00
|
|
|
"docusaurus-plugin-remote-content": "^3.1.0",
|
2023-02-14 06:35:55 +01:00
|
|
|
"docusaurus-theme-openapi-docs": "1.5.2",
|
2021-09-27 09:09:49 +02:00
|
|
|
"file-loader": "6.2.0",
|
chore: fix broken docs build / remove unused tag files (#2402)
## Issue
So, we've got an issue with our docs build not working. When building
for production, we get an error that looks a little bit like this:
```
$ docusaurus build
[INFO] [en] Creating an optimized production build...
✔ Client
✖ Server
Compiled with some errors in 40.85s
TypeError: source_default(...).bold is not a function
TypeError: source_default(...).bold is not a function
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.
```
Which isn't very helpful at all. If you go into
`/node_modules/@docusaurus/core/lib/client/serverEntry.js` and modify
the `render` function to log the actual error and remove anything
chalk-related, you get this instead:
```
$ docusaurus build
[INFO] [en] Creating an optimized production build...
✔ Client
✖ Server
Compiled with some errors in 44.62s
Actual error: Error: Unexpected: cant find current sidebar in context
at useCurrentSidebarCategory (main:11618:247)
at MDXContent (main:38139:1593)
at Fb (main:149154:44)
at Ib (main:149156:254)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
Actual error: Error: Unexpected: cant find current sidebar in context
at useCurrentSidebarCategory (main:11618:247)
at MDXContent (main:38513:1469)
at Fb (main:149154:44)
at Ib (main:149156:254)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
Error: Unexpected: cant find current sidebar in context
Error: Unexpected: cant find current sidebar in context
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.
```
That's better, but it's still not very clear.
## Getting more info
We've had problems with a similar error message before. Last time it was
caused by an empty file that docusaurus couldn't process.
A similar issue has also been described in [ this docusaurus GitHub
issue ](https://github.com/facebook/docusaurus/issues/7686). That's also
what gave me the idea of changing the logging in the dependency.
I'm currently unsure whether this is caused by the openapi docs or
something else. I've been in touch with the [openapi plugin
maintainer](https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/issues/323)
and he has been able to see the same error when building for prod
locally, but it was due to some old generated files.
Worth noting: this only seems to affect the prod build. Building for dev
(`yarn docusaurus start`) works just fine. It also fails locally **and**
in CI, so it _is_ an issue.
## Updating the logging
To get better logging, you can go into the
`/node_modules/@docusaurus/core/lib/client/serverEntry.js` file and
update the `render` function from
```js
export default async function render(locals) {
try {
return await doRender(locals);
}
catch (err) {
// We are not using logger in this file, because it seems to fail with some
// compilers / some polyfill methods. This is very likely a bug, but in the
// long term, when we output native ES modules in SSR, the bug will be gone.
// prettier-ignore
console.error(chalk.red(`${chalk.bold('[ERROR]')} Docusaurus server-side rendering could not render static page with path ${chalk.cyan.underline(locals.path)}.`));
const isNotDefinedErrorRegex = /(?:window|document|localStorage|navigator|alert|location|buffer|self) is not defined/i;
if (isNotDefinedErrorRegex.test(err.message)) {
// prettier-ignore
console.info(`${chalk.cyan.bold('[INFO]')} It looks like you are using code that should run on the client-side only.
To get around it, try using ${chalk.cyan('`<BrowserOnly>`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#browseronly')}) or ${chalk.cyan('`ExecutionEnvironment`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#executionenvironment')}).
It might also require to wrap your client code in ${chalk.cyan('`useEffect`')} hook and/or import a third-party library dynamically (if any).`);
}
throw err;
}
}
```
to
```js
export default async function render(locals) {
try {
return await doRender(locals);
}
catch (err) {
console.error(err)
throw err;
}
}
```
That'll yield the errors about the current sidebar in context.
## Root cause
Found the issue! 🙋🏼 It's explained in [this comment on the openapi docs
integration](https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/issues/323#issuecomment-1311549864)
for now, but in short: we have tags defined that we don't use. They're
being picked up by docusaurus, but don't have the proper context. That's
causing this.
The previously mentioned comment is included here for easy finding in
the future:
### Root cause explanation
The OpenAPI spec we use to generate the docs has a number of tags listed
at the root level. This is necessary for this plugin to pick up tag
categories and is, as far as I can tell, also how it _should_ be done.
However, not all of those tags are in use. Specifically, there's 2 tags
that are not.
When the plugin generates docs from the spec, it generates pages for all
endpoints and all tags and groups them by tag. However, it seems likely
that if a tag doesn't have any associated endpoints, then it won't get
added to the sidebar because there's no endpoint that references it.
But the doc files for these tags do end up lying around in the directory
regardless, and when docusaurus tries to pick up the files in the
generated directory, it also tries to pick up the unused tag files. But
because they're not part of a sidebar, they end up throwing errors
because they can't find the sidebar context.
### How I found it
The fact that I got more instances of the error message without the
sidebar ref than with it made me pay more attention to the number of
errors. I decided to check how many files were in the generated
directory and how many files were referenced from the generated sidebar.
Turns out the difference there was **2**: there were two generated files
in the directory that the sidebar didn't reference.
At this point, it was easy enough to try and delete those files before
rebuilding, and wouldn't you know: it worked!
### Our use case
Now, why do we have tags that are unused in the root spec? Can't we just
remove them?
That's a good question with a bit of a complicated answer. Unleash uses
an open core model and the OpenAPI integration is part of that open
core. The closed-source parts of Unleash are located in another repo and
extend the open-source distribution.
Because the OpenAPI spec is configured in the open-source part,
enterprise-only tags etc also need to be configured there. Then, when
the changes are absorbed into enterprise, we can use the tags there.
It gets more complicated because we use an enterprise instance to
generate the docs (because we want enterprise-endpoints to be listed
too). The instance uses the latest released instance of Unleash to have
the docs most accurately reflect the current state of things.
So, in this case, the tags have been added, but not yet used by any
endpoints, which suddenly causes this build failure. We can add the tags
to the enterprise-version, but the spec wouldn't be updated before the
next release regardless, which will probably be in a week or so.
This isn't an ideal setup, but .. it is what it is.
## Solutions and workarounds
As mentioned in the previous section, the reason the build was failing
was that there were unused tag files that docusaurus tried to include in
the build. Because they don't belong to a sidebar, the compilation
failed.
I've reported the issue to the openapi plugin maintainers and am waiting
for a response.
However, it seems that having unused root tags declared is invalid
according to the spec, so it's something we should look into fixing in
the future.
### Current workaround: cleaning script
The current workaround is to extend the api cleaning script to manually
remove the unused tag files.
### Ideal solution: filter root-level tags
Ideally, we shouldn't list unused OpenAPI tags on the root level at all.
However, because of the way we add root-level tags (as a predefined,
static lists, refer to `src/lib/openapi/index.ts`) and endpoints
(dynamically added at runtime) today, we don't really have a clear way
to filter the list of tags. This gets even more complicated when taking
the enterprise functionality and the potential extra tags they must
have.
This is, however, something that should definitely be looked into.
Working with OpenAPI across multipile repos is already troublesome, so
this is just yet another thing to look into.
2022-11-11 14:01:54 +01:00
|
|
|
"immer": "^9.0.6",
|
2022-06-15 00:02:22 +02:00
|
|
|
"react": "18.2.0",
|
|
|
|
|
"react-dom": "18.2.0",
|
2023-02-14 17:52:29 +01:00
|
|
|
"unleash-proxy-client": "2.4.2",
|
2021-09-27 09:09:49 +02:00
|
|
|
"url-loader": "4.1.1"
|
2021-06-04 11:17:15 +02:00
|
|
|
},
|
2021-06-04 22:57:42 +02:00
|
|
|
"resolutions": {
|
2022-08-26 10:27:30 +02:00
|
|
|
"async": "^3.2.4",
|
2021-09-27 12:13:43 +02:00
|
|
|
"trim": "^1.0.0",
|
2022-08-30 09:52:54 +02:00
|
|
|
"got": "^11.8.5",
|
2021-09-27 22:38:59 +02:00
|
|
|
"glob-parent": "^6.0.0",
|
2021-09-20 12:37:53 +02:00
|
|
|
"browserslist": "^4.16.5",
|
|
|
|
|
"set-value": "^4.0.1",
|
2021-09-27 13:26:18 +02:00
|
|
|
"immer": "^9.0.6",
|
|
|
|
|
"ansi-regex": "^5.0.1",
|
2022-08-26 10:27:30 +02:00
|
|
|
"nth-check": "^2.0.1",
|
|
|
|
|
"shelljs": "^0.8.5",
|
2022-11-03 14:43:36 +01:00
|
|
|
"trim-newlines": "^4.0.0",
|
2022-12-06 09:46:22 +01:00
|
|
|
"minimatch": "^5.0.0",
|
2022-12-28 02:40:24 +01:00
|
|
|
"decode-uri-component": "^0.4.0",
|
2022-12-08 11:32:22 +01:00
|
|
|
"qs": "^6.9.7"
|
2021-06-04 22:57:42 +02:00
|
|
|
},
|
2021-06-04 11:17:15 +02:00
|
|
|
"browserslist": {
|
|
|
|
|
"production": [
|
|
|
|
|
">0.5%",
|
|
|
|
|
"not dead",
|
|
|
|
|
"not op_mini all"
|
|
|
|
|
],
|
|
|
|
|
"development": [
|
|
|
|
|
"last 1 chrome version",
|
|
|
|
|
"last 1 firefox version",
|
|
|
|
|
"last 1 safari version"
|
|
|
|
|
]
|
2022-02-17 12:30:49 +01:00
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2023-01-04 23:44:01 +01:00
|
|
|
"@babel/core": "7.20.12",
|
chore: fix broken docs build / remove unused tag files (#2402)
## Issue
So, we've got an issue with our docs build not working. When building
for production, we get an error that looks a little bit like this:
```
$ docusaurus build
[INFO] [en] Creating an optimized production build...
✔ Client
✖ Server
Compiled with some errors in 40.85s
TypeError: source_default(...).bold is not a function
TypeError: source_default(...).bold is not a function
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.
```
Which isn't very helpful at all. If you go into
`/node_modules/@docusaurus/core/lib/client/serverEntry.js` and modify
the `render` function to log the actual error and remove anything
chalk-related, you get this instead:
```
$ docusaurus build
[INFO] [en] Creating an optimized production build...
✔ Client
✖ Server
Compiled with some errors in 44.62s
Actual error: Error: Unexpected: cant find current sidebar in context
at useCurrentSidebarCategory (main:11618:247)
at MDXContent (main:38139:1593)
at Fb (main:149154:44)
at Ib (main:149156:254)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
Actual error: Error: Unexpected: cant find current sidebar in context
at useCurrentSidebarCategory (main:11618:247)
at MDXContent (main:38513:1469)
at Fb (main:149154:44)
at Ib (main:149156:254)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
at W (main:149162:89)
at Jb (main:149165:98)
at Ib (main:149157:145)
Error: Unexpected: cant find current sidebar in context
Error: Unexpected: cant find current sidebar in context
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
at /Users/thomas/projects/work/unleash/website/node_modules/@docusaurus/core/lib/webpack/utils.js:180:24
at /Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:554:14
at processQueueWorker (/Users/thomas/projects/work/unleash/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
at processTicksAndRejections (node:internal/process/task_queues:78:11)
[INFO] Docusaurus version: 2.2.0
Node version: v16.14.0
error Command failed with exit code 1.
```
That's better, but it's still not very clear.
## Getting more info
We've had problems with a similar error message before. Last time it was
caused by an empty file that docusaurus couldn't process.
A similar issue has also been described in [ this docusaurus GitHub
issue ](https://github.com/facebook/docusaurus/issues/7686). That's also
what gave me the idea of changing the logging in the dependency.
I'm currently unsure whether this is caused by the openapi docs or
something else. I've been in touch with the [openapi plugin
maintainer](https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/issues/323)
and he has been able to see the same error when building for prod
locally, but it was due to some old generated files.
Worth noting: this only seems to affect the prod build. Building for dev
(`yarn docusaurus start`) works just fine. It also fails locally **and**
in CI, so it _is_ an issue.
## Updating the logging
To get better logging, you can go into the
`/node_modules/@docusaurus/core/lib/client/serverEntry.js` file and
update the `render` function from
```js
export default async function render(locals) {
try {
return await doRender(locals);
}
catch (err) {
// We are not using logger in this file, because it seems to fail with some
// compilers / some polyfill methods. This is very likely a bug, but in the
// long term, when we output native ES modules in SSR, the bug will be gone.
// prettier-ignore
console.error(chalk.red(`${chalk.bold('[ERROR]')} Docusaurus server-side rendering could not render static page with path ${chalk.cyan.underline(locals.path)}.`));
const isNotDefinedErrorRegex = /(?:window|document|localStorage|navigator|alert|location|buffer|self) is not defined/i;
if (isNotDefinedErrorRegex.test(err.message)) {
// prettier-ignore
console.info(`${chalk.cyan.bold('[INFO]')} It looks like you are using code that should run on the client-side only.
To get around it, try using ${chalk.cyan('`<BrowserOnly>`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#browseronly')}) or ${chalk.cyan('`ExecutionEnvironment`')} (${chalk.cyan.underline('https://docusaurus.io/docs/docusaurus-core/#executionenvironment')}).
It might also require to wrap your client code in ${chalk.cyan('`useEffect`')} hook and/or import a third-party library dynamically (if any).`);
}
throw err;
}
}
```
to
```js
export default async function render(locals) {
try {
return await doRender(locals);
}
catch (err) {
console.error(err)
throw err;
}
}
```
That'll yield the errors about the current sidebar in context.
## Root cause
Found the issue! 🙋🏼 It's explained in [this comment on the openapi docs
integration](https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/issues/323#issuecomment-1311549864)
for now, but in short: we have tags defined that we don't use. They're
being picked up by docusaurus, but don't have the proper context. That's
causing this.
The previously mentioned comment is included here for easy finding in
the future:
### Root cause explanation
The OpenAPI spec we use to generate the docs has a number of tags listed
at the root level. This is necessary for this plugin to pick up tag
categories and is, as far as I can tell, also how it _should_ be done.
However, not all of those tags are in use. Specifically, there's 2 tags
that are not.
When the plugin generates docs from the spec, it generates pages for all
endpoints and all tags and groups them by tag. However, it seems likely
that if a tag doesn't have any associated endpoints, then it won't get
added to the sidebar because there's no endpoint that references it.
But the doc files for these tags do end up lying around in the directory
regardless, and when docusaurus tries to pick up the files in the
generated directory, it also tries to pick up the unused tag files. But
because they're not part of a sidebar, they end up throwing errors
because they can't find the sidebar context.
### How I found it
The fact that I got more instances of the error message without the
sidebar ref than with it made me pay more attention to the number of
errors. I decided to check how many files were in the generated
directory and how many files were referenced from the generated sidebar.
Turns out the difference there was **2**: there were two generated files
in the directory that the sidebar didn't reference.
At this point, it was easy enough to try and delete those files before
rebuilding, and wouldn't you know: it worked!
### Our use case
Now, why do we have tags that are unused in the root spec? Can't we just
remove them?
That's a good question with a bit of a complicated answer. Unleash uses
an open core model and the OpenAPI integration is part of that open
core. The closed-source parts of Unleash are located in another repo and
extend the open-source distribution.
Because the OpenAPI spec is configured in the open-source part,
enterprise-only tags etc also need to be configured there. Then, when
the changes are absorbed into enterprise, we can use the tags there.
It gets more complicated because we use an enterprise instance to
generate the docs (because we want enterprise-endpoints to be listed
too). The instance uses the latest released instance of Unleash to have
the docs most accurately reflect the current state of things.
So, in this case, the tags have been added, but not yet used by any
endpoints, which suddenly causes this build failure. We can add the tags
to the enterprise-version, but the spec wouldn't be updated before the
next release regardless, which will probably be in a week or so.
This isn't an ideal setup, but .. it is what it is.
## Solutions and workarounds
As mentioned in the previous section, the reason the build was failing
was that there were unused tag files that docusaurus tried to include in
the build. Because they don't belong to a sidebar, the compilation
failed.
I've reported the issue to the openapi plugin maintainers and am waiting
for a response.
However, it seems that having unused root tags declared is invalid
according to the spec, so it's something we should look into fixing in
the future.
### Current workaround: cleaning script
The current workaround is to extend the api cleaning script to manually
remove the unused tag files.
### Ideal solution: filter root-level tags
Ideally, we shouldn't list unused OpenAPI tags on the root level at all.
However, because of the way we add root-level tags (as a predefined,
static lists, refer to `src/lib/openapi/index.ts`) and endpoints
(dynamically added at runtime) today, we don't really have a clear way
to filter the list of tags. This gets even more complicated when taking
the enterprise functionality and the potential extra tags they must
have.
This is, however, something that should definitely be looked into.
Working with OpenAPI across multipile repos is already troublesome, so
this is just yet another thing to look into.
2022-11-11 14:01:54 +01:00
|
|
|
"@docusaurus/module-type-aliases": "2.2.0",
|
2023-02-13 13:23:59 +01:00
|
|
|
"@storybook/addon-actions": "6.5.16",
|
|
|
|
|
"@storybook/addon-essentials": "6.5.16",
|
|
|
|
|
"@storybook/addon-interactions": "6.5.16",
|
|
|
|
|
"@storybook/addon-links": "6.5.16",
|
|
|
|
|
"@storybook/react": "6.5.16",
|
2022-06-13 18:59:59 +02:00
|
|
|
"@storybook/testing-library": "0.0.13",
|
2022-06-11 07:40:07 +02:00
|
|
|
"@tsconfig/docusaurus": "1.0.6",
|
2023-01-05 02:53:41 +01:00
|
|
|
"babel-loader": "9.1.2",
|
2022-11-23 22:57:46 +01:00
|
|
|
"enhanced-resolve": "5.12.0",
|
2023-01-16 23:15:37 +01:00
|
|
|
"react-router": "6.6.2",
|
2022-10-05 23:33:51 +02:00
|
|
|
"replace-in-file": "6.3.5",
|
2022-04-06 21:42:26 +02:00
|
|
|
"storybook-addon-root-attribute": "1.0.2",
|
2022-10-07 19:54:29 +02:00
|
|
|
"typescript": "4.8.4"
|
2021-06-04 11:17:15 +02:00
|
|
|
}
|
|
|
|
|
}
|
