Mirrors/audiobookshelf
Mirrors/audiobookshelf
1
0
Fork 0
mirror of https://github.com/advplyr/audiobookshelf.git synced 2024-12-20 19:06:06 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,298 Commits 13 Branches 130 Tags 62 MiB
a43b93d796
Commit Graph

245 Commits

Author SHA1 Message Date
advplyr
9ebe4b55dd
Merge pull request #1431 from lkiesow/x-accel 
Implement X-Accel Redirect
 2023-01-23 17:27:23 -06:00
advplyr
f7a3dbf209 Fix:Embed metadata tool embeds cover image 2023-01-22 18:02:57 -06:00
advplyr
d900093976 Fix:Make m4b embed cover image 2023-01-22 18:00:35 -06:00
Lars Kiesow
08250e266e
Implement X-Accel Redirect 
This patch implements [X-Accel](https://www.nginx.com/resources/wiki/start/topics/examples/x-accel/)
redirect headers as an optional way for offloading static file delivery
from Express to Nginx, which is far better optimized for static file
delivery.

This provides a really easy to configure way for getting a huge
performance boost over delivering all files through Audiobookshelf.

How it works
------------

The way this works is basically that Audiobookshelf gets an HTTP request
for delivering a static file (let's say an audiobook). It will first
check the user is authorized and then convert the API path to a local
file path.

Now, instead of reading and delivering the file, Audiobookshelf will
return just the HTTP header with an additional `X-Accel-Redirect`
pointing to the file location on the file syste.

This header is picked up by Nginx which will then deliver the file.

Configuration
-------------

The configuration for this is very simple. You need to run Nginx as
reverse proxy and it must have access to your Audiobookshelf data
folder.

You then configure Audiobookshelf to use X-Accel by setting
`USE_X_ACCEL=/protected`. The path is the internal redirect path used by
Nginx.

In the Nginx configuration you then configure this location and map it
to the storage area to serve like this:

```
location /protected/ {
  internal;
  alias /;
}
```

That's all.

Impact
------

I just did a very simple performance test, downloading a 1170620819
bytes large audiobook file from another machine on the same network
like this, using `time -p` to measure how log the process took:

```sh
URL='https://url to audiobook…'

for i in `seq 1 50`
do
  echo "$i"
  curl -s -o /dev/null "${URL}"
done
```

This sequential test with 50 iterations and without x-accel resulted in:

```
real 413.42
user 197.11
sys 82.04
```

That is an average download speed of about 1080 MBit/s.

With X-Accel enabled, serving the files through Nginx, the same test
yielded the following results:

```
real 200.37
user 86.95
sys 29.79
```

That is an average download speed of about 2229 MBit/s, more than
doubling the previous speed.

I have also run the same test with 4 parallel processes and 25 downloads
each. Without x-accel, that test resulted in:

```
real 364.89
user 273.09
sys 112.75
```

That is an average speed of about 2448 MBit/s.

With X-Accel enabled, the parallel test also shows a significant
speedup:

```
real 167.19
user 195.62
sys 78.61
```

That is an average speed of about 5342 MBit/s.

While doing that, I also peaked at the system load which was a bit lower
when using X-Accel. Even though the system was delivering far more data.
But I just looked at the `load1` values and did not build a proper test
for that. That means, I cant provide any definitive data.

Supported Media
---------------

The current implementation works for audio files and book covers. There
are other media files which would benefit from this mechanism like feed
covers or author pictures.

But that's something for a future developer ;-)
 2023-01-23 00:02:27 +01:00
advplyr
6dec750d3e Fix:Close open playback session on server when local playback session syncing from mobile 2023-01-15 15:00:18 -06:00
advplyr
62aec63d1d Fix:Backups to not backup temp db files 2023-01-08 09:59:24 -06:00
advplyr
08d16ce7c2 Silence remove invalid sessions debug log 2023-01-08 09:15:11 -06:00
advplyr
2cb3808326 Fix:Loading backups catching failed backups 2023-01-08 09:11:55 -06:00
advplyr
bdb6f0c0aa Update:Sync session API endpoint to not respond with a payload 2023-01-07 17:33:05 -06:00
advplyr
5a26704c32 Add:Option to disable backup of audio files in embed metadata tool #1370 2023-01-07 15:16:52 -06:00
advplyr
9a85ad1f6b Fix:Check if Windows before cleaning file path for POSIX separators #1254 2023-01-05 17:45:27 -06:00
advplyr
9de7be1cb4 Update scanner, music meta tags and fix issue with force update 2023-01-02 16:35:39 -06:00
advplyr
44363f05ac Start of new epub reader 2023-01-01 18:09:00 -06:00
advplyr
70ba2f7850 Add:RSS feed for series & cleanup empty series from db #1265 2022-12-31 16:58:19 -06:00
advplyr
c6763dee2d Remove invalid RSS feeds on init and remove feeds when associated entity is removed 2022-12-31 14:08:34 -06:00
advplyr
315c83e4c3 RSS feed for collection to update when any item in the collection is updated #606 2022-12-28 18:08:03 -06:00
advplyr
308ccf470f Add:Open RSS feed for collection #606 #1265 2022-12-27 18:03:31 -06:00
advplyr
061695f922 Add:API endpoint for opening RSS feed for collection #606 #1265 2022-12-26 17:48:39 -06:00
advplyr
e803dcd325 Update:RSS feed API routes 2022-12-26 16:58:36 -06:00
advplyr
0717ae39db Fix music fine file with inode 2022-12-24 11:12:39 -06:00
advplyr
33e183b802 Merge branch 'master' into socket-fixes 2022-12-23 07:27:14 -06:00
advplyr
b884f8fe11 Laying the groundwork for music media type #964 2022-12-22 16:38:55 -06:00
Paul Nettleton
2cba83f1dd Server socket event fixes 2022-12-22 16:26:11 -06:00
advplyr
616ffb8f79 Add:M4b tool configurable options bitrate/channels/codec #1029 #1257 2022-12-19 17:13:04 -06:00
advplyr
bc771a3a44 Delete DownloadManager.js 2022-12-19 16:20:18 -06:00
advplyr
d130dd6d5e Fix:Setting file ownership for /config and /metadata/logs #584 2022-12-15 17:30:45 -06:00
advplyr
41e192c6a5 Update more vars 2022-12-12 17:52:20 -06:00
advplyr
6b654d3c2d Update:Starting session for finished item sets the user start time back to 0 2022-12-12 17:18:56 -06:00
advplyr
67db41a525 Update:Get item cover API endpoint to allow for returning the raw cover image 2022-12-04 16:23:15 -06:00
advplyr
180293ebc1 Update:Cleanup socket usage & add func for emitting events to admin users 2022-11-24 16:35:26 -06:00
advplyr
e2af33e136 Update:Refactor socket connection management into SocketAuthority 2022-11-24 15:53:58 -06:00
advplyr
64a8a046c1 Update:Backups API endpoints, add get all backups route, update socket init event payload 2022-11-24 13:14:29 -06:00
advplyr
a2f1723642 Update log level for RSS feed requests 2022-11-21 06:39:32 -06:00
Lars Kiesow
c307b1e6fb No feed log level 
This patch drops the log level for logging requests to non-existing
feeds from error to debug. The reasoning behind this is that this is a
client error and a client error is returned and can be handled by the
client.

Otherwise anyone can easily spam the logs with error messages by just
requesting non-existing feeds.
 2022-11-21 01:54:25 +01:00
advplyr
e3ba739db5 Update:Encode & embed metadata API endpoints, separate cache & search endpoints into controllers 2022-11-19 13:28:06 -06:00
advplyr
2e5e02472c Update:Playback session sync local status codes 2022-11-17 17:00:37 -06:00
advplyr
c7794e00f6 Update:Author image from cache API status codes 2022-11-16 15:32:32 -06:00
advplyr
7d569e1e3e Update:Some incorrect status codes returned from API 2022-11-12 09:36:00 -06:00
advplyr
1118b8b782 Metadata embed and m4b merge fixes and cleanup 2022-11-05 13:13:52 -05:00
Keagan Hilliard
3824154c15 Forgot to update the merge 2022-11-03 10:20:32 -06:00
Keagan Hilliard
586c8a550a Removed a noisy log and limit chapter embedding to items with only 1 audiofile 2022-11-03 10:09:49 -06:00
Keagan Hilliard
d57effe97c Fixed a couple of issues, should be working well now 2022-11-03 09:32:50 -06:00
Keagan Hilliard
c1938f78c2 Added json file support 2022-11-02 19:40:50 -06:00
advplyr
efecf7ed82 Update:Podcast episode auto download schedule setting for max new episodes to download #1091 2022-10-26 16:55:16 -05:00
advplyr
142205f060 Add:Purge items cache button and api endpoint 2022-10-02 14:46:48 -05:00
advplyr
02d997897c Add:Cancel m4b merge button #1008 2022-10-02 14:31:04 -05:00
advplyr
39979ff8a3 Add:Tasks widget in appbar for merging m4bs & remove old m4b merge routes 2022-10-02 14:16:17 -05:00
advplyr
6701551289 Fix:Ensure podcast library item folder exists before downloading episodes #1019 2022-09-30 16:55:31 -05:00
advplyr
f36a5eae6d Update:Audiobook merge to set metadata with tone and replace m4b in library item #594 2022-09-26 18:07:31 -05:00
advplyr
97da73baf3 Update:Experimental metadata embed tool to use tone 2022-09-25 15:56:06 -05:00
advplyr
7aa89f16c9 Add:Notification system queueing and queue limit #996 2022-09-25 10:19:44 -05:00
advplyr
88726bed86 Update:Notification system descriptions #996 2022-09-25 09:46:45 -05:00
advplyr
0ef2a2e4b6 Update:Notifications onTest for testing and parse title/body template #996 2022-09-24 16:15:16 -05:00
advplyr
37a3fdb606 Notifications UI update and delete endpoint 2022-09-23 18:10:03 -05:00
advplyr
b08ad8785e Notification create/update events UI 2022-09-22 18:12:48 -05:00
advplyr
ff04eb8d5e Add:Notification settings, notification manager trigger #996 2022-09-21 18:01:10 -05:00
advplyr
9a7503cde2 Start adding notification manager 2022-09-20 18:08:41 -05:00
advplyr
70ef09f451 Add:Podcast quickmatch attempts quick matching unmatched episodes #983 2022-09-15 18:35:56 -05:00
advplyr
3dc848a106 Update:Podcast episodes look for new episodes after this date add input to set the max # of episodes to download 2022-09-03 08:06:52 -05:00
advplyr
575ec9d00b Fix:Update library item RSS feed if item was updated #939 2022-08-28 15:41:51 -05:00
Ron Heft
6b02779e0f
Fix currentTime not updating on the local session 2022-08-26 20:28:41 -04:00
advplyr
e611d7a8fd Update:Local session sync lock to prevent duplicate inserts 2022-08-23 18:10:06 -05:00
advplyr
0aadf579f3 Update:Backups include author images #781 2022-08-20 15:10:31 -05:00
advplyr
46668854ad Add:Schedule podcast new episode checks 2022-08-19 18:41:58 -05:00
advplyr
2304f37cbe Add:Schedule periodic library scans #655 2022-08-17 18:44:21 -05:00
advplyr
7a69afdcd9 Add:Podcast auto-download option to delete an episode if it exceeds X max episodes to keep #903 2022-08-15 17:35:13 -05:00
advplyr
24a142e718 Add:RSS feed icon over library item covers when feed is open #893 2022-08-05 19:23:18 -05:00
advplyr
d93d4f3236 Update:Auto-download new podcast episode check max failed attempts to 24 2022-07-31 14:00:17 -05:00
advplyr
516c5c3308 Add:Podcast episode match tab and find episode by title api route 2022-07-31 13:12:37 -05:00
advplyr
8416f2d6be Fix:Remove invalid playback sessions on server start #868 2022-07-29 17:13:46 -05:00
advplyr
18123664ee Fix:RSS Feed cover, Update:Remove experimental scanner 2022-07-18 18:39:51 -05:00
advplyr
ab08d83c04 Remove archiver dependency 2022-07-06 20:12:14 -05:00
advplyr
415e0a7b5a Remove dependency date-and-time 2022-07-06 19:18:27 -05:00
advplyr
d0af1c3c9a Remove fs-extra dependency 2022-07-05 19:53:01 -05:00
advplyr
d3dd13eae5 Remove node-stream-zip dependency 2022-07-05 19:24:16 -05:00
mcdinner
ebb2c5f791 Remove cachePathExists property (Issue #800) 
Remove cachePathsExist property to ensure missing cache directories are recreated when EnsureCachePaths() called.
 2022-07-03 16:35:12 +02:00
advplyr
140b718592
Merge pull request #699 from jmt-gh/698_metadata_downloads_not_created 
Update some instances of mkdir to ensureDir (#698)
 2022-06-08 16:05:19 -05:00
advplyr
b7e546f2f5 Update:Remove node-cron dependency 2022-06-07 20:04:51 -05:00
advplyr
26ef275ab4 Update:Remove image-type dependency 2022-06-07 19:53:05 -05:00
advplyr
416db7c981 Update:Remove read-chunk dependency 2022-06-07 19:44:38 -05:00
advplyr
03bffb725a Update:Remove rss feed dependencies add node-xml lib 2022-06-07 19:25:14 -05:00
advplyr
fbbceaa642 Add:Persist RSS feeds in db #696, Update:RSS feed data model 2022-06-07 18:29:43 -05:00
jmt-gh
f5aae25cc8 remove random character 2022-06-06 18:52:08 -07:00
jmt-gh
8d03943acb remove extra debug log 2022-06-06 18:51:49 -07:00
jmt-gh
853513b926 update approach for ensuring download directory always exists 2022-06-06 18:51:08 -07:00
jmt-gh
c606a41314 update Cache folder creation to leverage ensureDirs 2022-06-06 08:18:15 -07:00
jmt-gh
35f29ca22b Use ensureDir instead of mkdir to fix 698 
This commit updates the mkdir for creating the download location to
ensureDir, which is an alias for mkdirs and mkdirp, meaning they will
create the entire path of the directory if it does not exist.

https://github.com/jprichardson/node-fs-extra/blob/master/docs/ensureDir.md
 2022-06-06 08:12:58 -07:00
jmt-gh
8809c7b900 Handle null and delete cover cases 
This commit adds in supporting if a cover path is null. If this is the
case, we completely remove the video stream from the file, as the user
either:

a) uploaded a file with no video stream (so removing it is a no-op)
b) removed the cover in ABS, so we should respect that on merge
 2022-06-05 10:36:42 -07:00
jmt-gh
9b6fa8fe8c
Merge branch 'advplyr:master' into update_cover_on_merge 2022-06-04 19:00:41 -07:00
jmt-gh
d6c02ebb2c Support embedding cover art metadata 
Added support for chapter metadata in #678, but completely missed that
coverart wasn't getting embedded in the embed metadata tool. This commit
adds that in
 2022-06-04 18:56:55 -07:00
jmt-gh
3bc3914fd9 Fix cover art not being generated for m4b export 
This commit fixes an issue where cover art was not being generated
properly when creating an M4B audiobook.

More context can be found in discord:
https://discord.com/channels/942908292873723984/981321213882282035/982777444631195681
 2022-06-04 17:50:26 -07:00
jmt-gh
ab0094a53b Support embedding updated chapter metadata (676) 
This commit resolves issue #676. The embed metadata tool was missing the
flag that tells ffmpeg to not only update the "top" metadata, but also
the chapter metadata.
 2022-06-04 10:17:42 -07:00
advplyr
09beb6a2ae Merge branch 'master' of https://github.com/advplyr/audiobookshelf 2022-06-02 16:32:42 -05:00
Selfhost Alt
1350a91fba Handle another type of corrupted backup file 2022-05-30 23:53:00 -07:00
advplyr
acf22ca4fa Testing video media type 2022-05-30 19:26:53 -05:00
advplyr
514893646a Add:OPML Upload for bulk adding podcasts #588 2022-05-29 11:46:45 -05:00
advplyr
f002532c1e Add:User listening sessions page, Update:Listening sessions to save media times and device info 2022-05-26 19:09:46 -05:00
advplyr
2a235b8324 Add:RSS feeds for audiobooks #606 2022-05-19 18:51:58 -05:00
advplyr
09dc95f560 Fix:Create cache dirs on server init 2022-05-15 11:19:04 -05:00
advplyr
853858825b Fix:File permissions on cache dirs and cache images, Fix:Db delete read stream closing before write stream resulting in deletes sometimes not happening 2022-05-15 09:51:08 -05:00
advplyr
c962090c3a Update:No longer creating initial root user and initial library, add init root user page, web app works with no libraries 2022-05-14 17:23:22 -05:00
advplyr
17d15144eb Update:Podcast new episode check cronjob to use last episode pub date if exists otherwise fallback to using last check date 2022-05-11 18:55:19 -05:00
advplyr
c6323f8ad9 Fix:Local playback session store date/dayOfWeek string to be used in stats 2022-05-11 17:35:04 -05:00
advplyr
d285845e04 Fix:Crash when mobile sends invalid library item to sync with session 2022-05-11 17:07:41 -05:00
advplyr
195a30096f Update:Experimental RSS feed setting custom slugs with default to library item id #553 2022-05-03 18:52:34 -05:00
Selfhost Alt
5ac342defd Handle corrupted backups gracefully and continue loading other backups 2022-05-02 22:47:16 -07:00
advplyr
9b9de84740 Add:Experimental embed audio metadata page 2022-05-02 18:48:00 -05:00
advplyr
2059c9f14a Fix:Podcast RSS feed require fs 2022-05-02 17:21:16 -05:00
advplyr
0e16a9c8de Update:Many more debug logs for auto-download podcasts, add timeout for feed request, use anonymous function in cron job 2022-05-02 17:17:26 -05:00
advplyr
678dceefed Add:Experimental generate podcast RSS feed #553 2022-05-02 16:42:30 -05:00
advplyr
8b38dda229 Add:experimental generate podcast feed for testing 2022-05-02 14:41:59 -05:00
advplyr
7373c7159b Add additional logs during podcast episode checks and allow up to 3 failed feed requests 2022-05-01 19:54:33 -05:00
advplyr
84c12a6e7e Add:Experimental embed metadata in audio files #141 2022-05-01 18:33:46 -05:00
advplyr
4185807da4 Add:Check for new episodes manual check and update last check time, Update:Adding new podcasts and downloading podcast episodes restricted to admin users 2022-04-29 16:42:40 -05:00
advplyr
33dfb764fa Add:Support for openaudible folder structure (subject to change), add support for treating single audio files in the root directory as library items #401 2022-04-27 19:42:34 -05:00
advplyr
dcd4f69383 Fix: set downloaded/uploaded cover owner and permissions and if creating intitial config/metadata directories at startup then set owner of those #394 2022-04-24 19:12:00 -05:00
advplyr
21bdd9f9ec Fix set invalid flag to false when adding first episode to an empty podcast library item, dont show podcast errors on episode cards 2022-04-24 17:03:43 -05:00
advplyr
034d858f18 Change new podcast modal to remove episode download list #494, Fix error when importing many episodes (set max size to 5MB) #493, show podcast episodes downloading and in queue on podcast landing page 2022-04-23 19:41:06 -05:00
advplyr
c5a9c2bf5a
Merge pull request #489 from selfhost-alt/configurable-backup-size 
Make maximum backup size configurable
 2022-04-23 17:06:59 -05:00
advplyr
b2d299dba6 Remove open playback sessions for user when starting a new playback session 2022-04-23 16:18:34 -05:00
Selfhost Alt
cb5d9a8287 Add explicit byte conversion variable to make code more self-documenting 2022-04-23 10:26:37 -07:00
Selfhost Alt
7c7e8285a4 Make maximum backup size configurable 2022-04-23 10:19:31 -07:00
advplyr
ad3fbe7abf Add back in m4b merge downloader in experimental #478 2022-04-21 18:52:28 -05:00
advplyr
1c6cd7499b Remove old cover method make sure cover filename is an actual image 2022-04-20 17:34:20 -05:00
jflattery
28c99cf17f Increase readability of logs 
Add podcast title to log output when autodownload fails
 2022-04-20 17:35:15 +00:00
advplyr
5d305c96ad Add support for WMA and AIFF audio files #449, add remove orphan streams, clean up audio mime type logic 2022-04-16 12:37:10 -05:00
Rasmus Krämer
cd274e0844
Merge branch 'master' of https://github.com/rasmuslos/audiobookshelf 2022-04-15 12:59:45 +02:00
Rasmus Krämer
e9249430c3
Parse current time as float 2022-04-15 12:59:42 +02:00
advplyr
8bbeae4873 Fix check podcast episodes cronjob 2022-04-14 10:15:42 -05:00
advplyr
05dff2583a Backups to store server version in zip details and check and show alert for old backups created before version 2.0.0 2022-04-13 18:51:06 -05:00
advplyr
4edba20e9e Update podcast search page to support manually entering podcast RSS feed 2022-04-13 16:55:48 -05:00
advplyr
b0b1d2707d Add podcast episode date picker for pubDate 2022-04-12 17:32:27 -05:00
advplyr
10d9e11387 Update abmetadata file for new data model, add chapter and description section parser 2022-04-12 16:05:16 -05:00
advplyr
2a386ca2a9 Add sync local media progress routes for offline mobile playback session support 2022-04-09 17:56:51 -05:00
advplyr
ac3fa31d1e Update Podcast Episode add libraryItemId, expanded returns audioTrack object 2022-04-05 19:40:40 -05:00
advplyr
6e5e638076 Update Book.js to return array of AudioTrack objects on json expand 2022-04-03 16:01:59 -05:00
advplyr
c201e2aa98 Add mediaPlayer to playback session 2022-04-02 11:19:57 -05:00
advplyr
c75895d711 Fix:Podcast scanner get embedded cover art 2022-03-28 20:23:16 -05:00
advplyr
0dd219f303 Add podcast episode auto download new episodes cron 2022-03-26 19:58:59 -05:00
advplyr
12027b9a76 Podcast episode player fixes, episode table ui updates 2022-03-26 18:23:33 -05:00
advplyr
0e665e2091 Add playing podcast episodes, episode progress, podcast page, podcast home page shelves 2022-03-26 17:41:26 -05:00
advplyr
5446aea910 Add Scanner support for podcasts 2022-03-26 14:29:49 -05:00
advplyr
3150822117 New data model removing media entity for books 2022-03-26 11:59:34 -05:00
advplyr
920ca683b9 Podcast episode downloader, update podcast data model 2022-03-21 19:24:38 -05:00
advplyr
e1e6b46456 Create podcast manager and re-organize managers 2022-03-20 16:41:06 -05:00