Mirrors/audiobookshelf
Mirrors/audiobookshelf
1
0
Fork 0
mirror of https://github.com/advplyr/audiobookshelf.git synced 2025-10-27 11:18:14 +01:00
Code Issues Packages Projects Releases Wiki Activity
5,922 Commits 83 Branches 155 Tags 117 MiB
4725a466da
Commit Graph

283 Commits

Author SHA1 Message Date
lukeIam
dd9a3858d7 Merge remote-tracking branch 'origin/master' into auth_passportjs 2023-08-12 16:44:44 +02:00
advplyr
56e3449db6 Remove media progress for podcast episodes when episode is removed 2023-08-06 14:18:51 -05:00
advplyr
354e16e462 Update:Only load Users when needed 2023-07-22 15:32:20 -05:00
advplyr
1d974375a0 Update:Only load libraries from db when needed 2023-07-22 14:25:20 -05:00
advplyr
80aea0c82d Fix:Save metadata files when updating library items #1952 2023-07-22 07:50:47 -05:00
advplyr
8cff68ca64 Fix purge metadata/items paths 2023-07-10 17:00:31 -05:00
advplyr
f73a0cce72 Update Dockerfile for sqlite3, update models for cascade delete, fix backup schedule 2023-07-09 11:39:15 -05:00
advplyr
254ba1f089 Migrate backups manager 2023-07-08 14:40:49 -05:00
advplyr
b5d4c11f6f Fix RSS feeds to use slug instead of id 2023-07-06 17:07:10 -05:00
advplyr
cf7fd315b6 Init sqlite take 2 2023-07-04 18:14:44 -05:00
advplyr
aeba7674f8 Add new api route for downloading backup, remove static metadata route 2023-06-27 16:41:32 -05:00
advplyr
5b0d105e21 Remove deprecated /s/ and /ebook/ api routes 2023-06-27 15:56:33 -05:00
advplyr
5a01be1ee3 Add tempFileDir for uploads 2023-06-06 15:40:52 -05:00
Coda
2fd3238576
Fix: useTempFiles=true, upload use tmp instead of ram 2023-06-04 15:56:41 -07:00
Coda
a3642b204d
Fix: decode filename as utf8 on upload 2023-06-03 21:44:13 -07:00
advplyr
05ce9c6eda Add:Email smtp config & send ebooks to devices #1474 2023-05-29 17:38:38 -05:00
advplyr
019063e6f4 Update:New API routes for library files and downloads 2023-05-28 12:34:22 -05:00
advplyr
4f75a89633 Update:New EBook API endpoint 2023-05-28 10:47:28 -05:00
advplyr
056da0ef70 Fix:Static ebook route 2023-05-28 08:39:41 -05:00
advplyr
b52e240025 Add:Batch re-scan #1754 2023-05-27 14:51:03 -05:00
advplyr
4359ca28df Fix XAccel issue 2023-04-29 16:05:05 -05:00
advplyr
7010a13648 Fixes for passport local and allow empty password 2023-04-16 10:08:13 -05:00
lukeIam
812395b21b Merge remote-tracking branch 'origin/master' into auth_passportjs 2023-04-14 20:27:43 +02:00
advplyr
05b102722b Remove unused ebook routes 2023-04-03 17:33:02 -05:00
advplyr
ef954ee68f Remove downloads folder in metadata dir 2023-04-03 17:28:55 -05:00
lukeIam
be53b31712 Merge remote-tracking branch 'origin/master' into auth_passportjs 2023-03-24 18:23:08 +01:00
lukeIam
e1ddb95250 Inital passportjs integration 2023-03-24 18:21:25 +01:00
mfcar
61c759e0c4
Add tasks queue dropdown 2023-03-05 11:15:36 +00:00
Lars Kiesow
08250e266e
Implement X-Accel Redirect 
This patch implements [X-Accel](https://www.nginx.com/resources/wiki/start/topics/examples/x-accel/)
redirect headers as an optional way for offloading static file delivery
from Express to Nginx, which is far better optimized for static file
delivery.

This provides a really easy to configure way for getting a huge
performance boost over delivering all files through Audiobookshelf.

How it works
------------

The way this works is basically that Audiobookshelf gets an HTTP request
for delivering a static file (let's say an audiobook). It will first
check the user is authorized and then convert the API path to a local
file path.

Now, instead of reading and delivering the file, Audiobookshelf will
return just the HTTP header with an additional `X-Accel-Redirect`
pointing to the file location on the file syste.

This header is picked up by Nginx which will then deliver the file.

Configuration
-------------

The configuration for this is very simple. You need to run Nginx as
reverse proxy and it must have access to your Audiobookshelf data
folder.

You then configure Audiobookshelf to use X-Accel by setting
`USE_X_ACCEL=/protected`. The path is the internal redirect path used by
Nginx.

In the Nginx configuration you then configure this location and map it
to the storage area to serve like this:

```
location /protected/ {
  internal;
  alias /;
}
```

That's all.

Impact
------

I just did a very simple performance test, downloading a 1170620819
bytes large audiobook file from another machine on the same network
like this, using `time -p` to measure how log the process took:

```sh
URL='https://url to audiobook…'

for i in `seq 1 50`
do
  echo "$i"
  curl -s -o /dev/null "${URL}"
done
```

This sequential test with 50 iterations and without x-accel resulted in:

```
real 413.42
user 197.11
sys 82.04
```

That is an average download speed of about 1080 MBit/s.

With X-Accel enabled, serving the files through Nginx, the same test
yielded the following results:

```
real 200.37
user 86.95
sys 29.79
```

That is an average download speed of about 2229 MBit/s, more than
doubling the previous speed.

I have also run the same test with 4 parallel processes and 25 downloads
each. Without x-accel, that test resulted in:

```
real 364.89
user 273.09
sys 112.75
```

That is an average speed of about 2448 MBit/s.

With X-Accel enabled, the parallel test also shows a significant
speedup:

```
real 167.19
user 195.62
sys 78.61
```

That is an average speed of about 5342 MBit/s.

While doing that, I also peaked at the system load which was a bit lower
when using X-Accel. Even though the system was delivering far more data.
But I just looked at the `load1` values and did not build a proper test
for that. That means, I cant provide any definitive data.

Supported Media
---------------

The current implementation works for audio files and book covers. There
are other media files which would benefit from this mechanism like feed
covers or author pictures.

But that's something for a future developer ;-)
 2023-01-23 00:02:27 +01:00
advplyr
10677d6fb0
Merge pull request #1421 from lkiesow/fingerprinting 
Reduce Fingerprinting
 2023-01-22 08:03:17 -06:00
Lars Kiesow
274b0e48be
Skip AUDIOBOOKSHELF_UID/GID if undefined 
This patch slightly changes the behavior of the `AUDIOBOOKSHELF_UID` and
`AUDIOBOOKSHELF_GID` options. Instead of defining a default user and
group, trying to modify files and silently failing if the filesystem
mode cannot be changed, this patch will just skip the entire process in
the first place.

If these options are defined, Audiobookshelf should behave exactly as
before. If they are not defined, Audiobookshelf will now cause fewer
file modifications (or less failures when trying to modify files).

If this patch gets applied, it should probably be highlighted in the
release notes. This usually shouldn't cause problems for migrations
since the Docker guides explicitly configure the options and the
package installations do not seem to use this at all, but there is still
a change that it will and users should be aware of that.

If a problem arises, users can easily fix the problem by either setting
the permissions once manually to the audiobookshelf user or by simply
defining the `AUDIOBOOKSHELF_UID/GID` options.
 2023-01-22 12:30:36 +01:00
Lars Kiesow
a1b49f5fcf
Reduce Fingerprinting 
As DieselTech#6997 pointed out in Matrix, it is a good idea to reduce
fingerprinting by removing the `X-Powered-By` response header as pointed
out by the Express security best practices:

See http://expressjs.com/en/advanced/best-practice-security.html#reduce-fingerprinting
 2023-01-21 23:18:06 +01:00
advplyr
9a85ad1f6b Fix:Check if Windows before cleaning file path for POSIX separators #1254 2023-01-05 17:45:27 -06:00
advplyr
3426832f2b Fix for windows, update regex to only include line number, move to end of log 2023-01-05 16:44:34 -06:00
advplyr
44363f05ac Start of new epub reader 2023-01-01 18:09:00 -06:00
advplyr
70ba2f7850 Add:RSS feed for series & cleanup empty series from db #1265 2022-12-31 16:58:19 -06:00
advplyr
0e6b0d3eff Update:Remove RSS feeds from login response payload and include feeds from library items request 2022-12-31 10:59:12 -06:00
advplyr
4d93e39fa9 Add:Item metadata utils config page for managing tags #1163 2022-12-18 14:17:52 -06:00
advplyr
10295b000a Update:Remove HOST default to allow for ipv6 #1256 2022-12-17 15:55:53 -06:00
advplyr
0fcf978ffe Add /playlist/:id to dynamic routes 2022-11-27 14:23:28 -06:00
advplyr
180293ebc1 Update:Cleanup socket usage & add func for emitting events to admin users 2022-11-24 16:35:26 -06:00
advplyr
e2af33e136 Update:Refactor socket connection management into SocketAuthority 2022-11-24 15:53:58 -06:00
advplyr
77a86d92f4 Update:Socket event for getting online users & test event for messaging all online users 2022-11-24 13:51:41 -06:00
advplyr
64a8a046c1 Update:Backups API endpoints, add get all backups route, update socket init event payload 2022-11-24 13:14:29 -06:00
advplyr
fdf67e17a0 Add:API endpoint to get users online and open listening sessions #1125 2022-11-10 17:42:20 -06:00
advplyr
6e064eeafb Add:Server setting for default language #1103 2022-11-08 18:09:07 -06:00
advplyr
39979ff8a3 Add:Tasks widget in appbar for merging m4bs & remove old m4b merge routes 2022-10-02 14:16:17 -05:00
advplyr
441b8c5bb7 Update:M4b Merge tool moved to manage page 2022-10-02 11:53:53 -05:00
advplyr
cd04533eea Update:Setting up paths to eventually support subdirectory 2022-10-01 16:07:30 -05:00
advplyr
ac30a971c5 Fix:Clean user data on server start removing invalid media progress items 2022-09-28 17:12:27 -05:00
advplyr
0ef2a2e4b6 Update:Notifications onTest for testing and parse title/body template #996 2022-09-24 16:15:16 -05:00
advplyr
b08ad8785e Notification create/update events UI 2022-09-22 18:12:48 -05:00
advplyr
9a7503cde2 Start adding notification manager 2022-09-20 18:08:41 -05:00
advplyr
ae4ac392c6 Add:Podcasts latest episodes page 2022-09-17 15:23:33 -05:00
advplyr
be28b9899e Update:Audio player does not open on load 2022-08-27 17:27:55 -05:00
advplyr
46668854ad Add:Schedule podcast new episode checks 2022-08-19 18:41:58 -05:00
advplyr
2304f37cbe Add:Schedule periodic library scans #655 2022-08-17 18:44:21 -05:00
advplyr
24a142e718 Add:RSS feed icon over library item covers when feed is open #893 2022-08-05 19:23:18 -05:00
advplyr
ed23feaf3f Fix typo Received Ping 2022-07-30 08:37:35 -05:00
advplyr
8416f2d6be Fix:Remove invalid playback sessions on server start #868 2022-07-29 17:13:46 -05:00
advplyr
9d7ffdfcd0 Update docker file healthcheck to use /healthcheck instead of /ping 2022-07-24 15:46:19 -05:00
advplyr
9e7b84f289 Update:JWT signing 2022-07-18 17:19:16 -05:00
advplyr
bca2cfda13 Update:Remove log listener for root user & only set when on logger config page 2022-07-07 17:25:52 -05:00
advplyr
d301c12acd Remove dependency express-rate-limit 2022-07-06 19:14:47 -05:00
advplyr
7aa7e662b2 Remove dependency express-fileupload 2022-07-06 19:10:25 -05:00
advplyr
d0af1c3c9a Remove fs-extra dependency 2022-07-05 19:53:01 -05:00
advplyr
19f020e7a6 Fix:Open playback session on init #743 2022-06-18 13:11:15 -05:00
advplyr
140b718592
Merge pull request #699 from jmt-gh/698_metadata_downloads_not_created 
Update some instances of mkdir to ensureDir (#698)
 2022-06-08 16:05:19 -05:00
advplyr
fbbceaa642 Add:Persist RSS feeds in db #696, Update:RSS feed data model 2022-06-07 18:29:43 -05:00
jmt-gh
853513b926 update approach for ensuring download directory always exists 2022-06-06 18:51:08 -07:00
advplyr
c88bbf1ce4 Fix:Authors landing page available on refresh #659 2022-06-01 16:29:29 -05:00
advplyr
93b8e11378 Fix:Mark media as finished if less than 5 seconds remain on a sync and call progress sync again when last track ends #635 2022-05-29 12:55:14 -05:00
advplyr
8beac53f5f Update:Send source back with auth request 2022-05-21 11:21:03 -05:00
advplyr
09dc95f560 Fix:Create cache dirs on server init 2022-05-15 11:19:04 -05:00
advplyr
c962090c3a Update:No longer creating initial root user and initial library, add init root user page, web app works with no libraries 2022-05-14 17:23:22 -05:00
advplyr
63a8e2433e Fix:Manage and chapters item page available on refresh 2022-05-14 13:08:56 -05:00
advplyr
678dceefed Add:Experimental generate podcast RSS feed #553 2022-05-02 16:42:30 -05:00
advplyr
8b38dda229 Add:experimental generate podcast feed for testing 2022-05-02 14:41:59 -05:00
advplyr
84c12a6e7e Add:Experimental embed metadata in audio files #141 2022-05-01 18:33:46 -05:00
advplyr
220bbc3d2d Fix:Series covers on home page not spread out correctly #505, Update:Server settings are now returned with auth requests 2022-04-29 17:43:46 -05:00
advplyr
dcd4f69383 Fix: set downloaded/uploaded cover owner and permissions and if creating intitial config/metadata directories at startup then set owner of those #394 2022-04-24 19:12:00 -05:00
advplyr
034d858f18 Change new podcast modal to remove episode download list #494, Fix error when importing many episodes (set max size to 5MB) #493, show podcast episodes downloading and in queue on podcast landing page 2022-04-23 19:41:06 -05:00
advplyr
ad3fbe7abf Add back in m4b merge downloader in experimental #478 2022-04-21 18:52:28 -05:00
advplyr
584e754eae Remove db log from testing 2022-04-20 08:38:24 -05:00
advplyr
68cf748e77 Fix previous version check for db migration to v2 2022-04-20 08:31:57 -05:00
advplyr
e83df2bf4b Update migration version 2022-04-19 20:55:40 -05:00
advplyr
5d305c96ad Add support for WMA and AIFF audio files #449, add remove orphan streams, clean up audio mime type logic 2022-04-16 12:37:10 -05:00
advplyr
79a82df914 Remove NFO metadata and save metadata button 2022-04-13 18:23:44 -05:00
advplyr
10d9e11387 Update abmetadata file for new data model, add chapter and description section parser 2022-04-12 16:05:16 -05:00
advplyr
ac097862fc Update sorting and filtering for podcasts, add title ignore prefix to podcast metadata, check user permissions for podcast episode row UI 2022-04-11 19:42:09 -05:00
advplyr
c60807f998 Removing remaining legacy objects, remove njodb error for fileExists 2022-04-10 10:05:05 -05:00
Rasmus Krämer
5c91c1e2c7 Added select all option to the episode selector 2022-04-09 10:25:24 +02:00
advplyr
0dd219f303 Add podcast episode auto download new episodes cron 2022-03-26 19:58:59 -05:00
advplyr
0e665e2091 Add playing podcast episodes, episode progress, podcast page, podcast home page shelves 2022-03-26 17:41:26 -05:00
advplyr
3150822117 New data model removing media entity for books 2022-03-26 11:59:34 -05:00
advplyr
920ca683b9 Podcast episode downloader, update podcast data model 2022-03-21 19:24:38 -05:00
advplyr
e1e6b46456 Create podcast manager and re-organize managers 2022-03-20 16:41:06 -05:00
advplyr
f8d0384155 Migration change metadata folder from /books to /items, podcast data model updates, add podcast routes 2022-03-19 10:13:10 -05:00
advplyr
c9ea5dd2d7 New data model backups and move backups to API endpoints 2022-03-18 13:44:29 -05:00
advplyr
57399bb79e Clean up ApiRouter adding MiscController, move upload and scan to api endpoints 2022-03-18 11:51:55 -05:00
advplyr
14a8f84446 New data model update bookmarks and bookmark routes to use API 2022-03-17 20:28:04 -05:00
advplyr
099ae7c776 New data model play media entity, PlaybackSessionManager 2022-03-17 19:10:47 -05:00
advplyr
1cf9e85272 New data model update MeController user progress routes 2022-03-17 13:33:22 -05:00
advplyr
c4eeb1cfb7 New data model Book media type contains Audiobooks updates 2022-03-17 12:25:12 -05:00
Quietus
d561a48229 Allowed the configuration of a "HOST" parameter to enable ipv6 support. 2022-03-17 11:06:52 +00:00
advplyr
0af6ad63c1 New data model start of PlaybackSessionManager to replace StreamManager, remove podcast & ip npm package 2022-03-15 19:28:54 -05:00
advplyr
68b13ae45f New data model migration for users, bookmarks and playback sessions 2022-03-15 18:57:15 -05:00
advplyr
7d66f1eec9 New data model edit tracks page, match, quick match, clean out old files 2022-03-13 19:34:31 -05:00
advplyr
73257188f6 New data model save covers, scanner, new api routes 2022-03-12 17:45:32 -06:00
advplyr
f2be3bc95e Add multi select dropdown with query from server 2022-03-10 19:13:19 -06:00
advplyr
2a30cc428f New api routes, updating web client pages, audiobooks to libraryItem migration 2022-03-10 18:45:02 -06:00
advplyr
b97ed953f7 Add db migration file to change audiobooks to library items with new data model 2022-03-09 19:23:17 -06:00
advplyr
295c6b0c74 Add:Generate book metadata file when book details are changed,Add:Server setting for storing book metadata in book folder 2022-02-27 14:28:18 -06:00
advplyr
aa50cc2d81 Change:Main dir paths moved to global vars, server settings stored in globals vars 2022-02-27 13:47:52 -06:00
advplyr
3a34daef7c Fix:Client emitter users not found no longer is an error 2022-02-26 16:35:40 -06:00
advplyr
aa675422a9 Add:Support multiple book upload #248 2022-02-26 16:19:22 -06:00
advplyr
7f1fc1901b Add:Server setting to disable folder watcher #378 2022-02-23 17:52:21 -06:00
advplyr
89f498f31a Add:Chromecast support in experimental #367, Change:Audio player model for direct play 2022-02-22 17:33:55 -06:00
advplyr
11be49a535 Add:Library match all books #359 2022-02-15 18:33:33 -06:00
advplyr
c0c4b1fa23 Fix:FFprobe path for node-ffprobe 2022-02-06 17:14:55 -06:00
advplyr
590dbe3991 Fix:Downloader sets file permission and owner on files #286 2022-02-03 18:08:39 -06:00
advplyr
a17348f916 Change:Fallback to audio stream tags if probe format has no tags and remove old scanner #256 2021-12-24 18:06:17 -06:00
advplyr
1b03af4041 Add:Api route to sync stream data for use in android native 2021-12-19 18:45:56 -06:00
advplyr
ddf0fa72e8 Add:Cover image cache, resize & use webp image #223 2021-12-12 17:15:37 -06:00
Igor Kaldowski
b68c181631 Add: Force re-scan library 2021-12-04 23:57:47 +00:00
advplyr
8225d35022 Fix:Cover aspect ratio for search match,Fix:Dynamic route 404s 2021-12-02 19:40:42 -06:00
advplyr
1ef9a689bc Lazy bookshelf finalized 2021-12-01 19:07:03 -06:00
advplyr
24d2e09724 Fix:Cancel library scan 2021-11-25 20:25:44 -06:00
advplyr
a5fc382cad Add:New scanner and scanner server settings 2021-11-25 18:39:02 -06:00
advplyr
bf11d266dc New scanner updates with library scan data 2021-11-24 20:15:50 -06:00
advplyr
72f9732b67 Fix: book id length & check duplicate ids, Change: library to lazy load book cards 2021-11-15 20:09:42 -06:00
advplyr
91e44bc2f9 Add: User listening sessions and user listening stats #167 2021-11-12 19:43:16 -06:00
advplyr
b0ea2f8008 Fix: daily log rewrite, Add: purge user audiobook data for audiobooks that no longer exist 2021-11-04 07:59:28 -05:00
advplyr
8ca6c62a03 Fix: Daily log file not adding newlines to logs, Add: Config log page loads last 5000 logs for the current day #72 2021-10-31 19:10:45 -05:00
advplyr
75ba884932 Add: Daily file logs and log server settings 2021-10-31 17:55:28 -05:00
advplyr
ff92fbb849 Change: AudiobookProgress becomes UserAudiobookData, Change: Delete audiobook progress route 2021-10-28 17:19:09 -05:00
advplyr
9f66054a72 Add: Experimental bookmarks edit and delete #115 2021-10-26 20:09:04 -05:00
advplyr
8b31c6555a Fix: audiobook progress emitter emit to all user sockets #145, Fix: save user after progress update 2021-10-25 21:14:54 -05:00
advplyr
c5eafdfa8a Change: audio player shows time remaining and percent #122, Add: experimental bookmarks start #115 2021-10-24 18:25:44 -05:00
advplyr
335bbac81d Add: Filter and side rail button to show books with issues #132, Add: realtime updates for audiobook progress 2021-10-24 15:53:51 -05:00
advplyr
ff1eeda468 Change: config page to multiple pages, Add: user permissions for accessible libraries #120, Add: map genre metadata tag #114, Add: experimental audio player keyboard controls #121, Add: view user audiobook progress list 2021-10-22 20:08:02 -05:00
advplyr
cd6e99b4c3 Update users table info #94, Reorder libraries in config #95, Use dropdown for library menu #96, update mobi reader 2021-10-12 20:07:42 -05:00
advplyr
e80ec10e8a Fix ebook url #75, download other files #75, fix book icon disappearing #88, backups #87 2021-10-08 17:30:20 -05:00
advplyr
f752c19418 narrator filter, no series filter, full paths toggle, book landing page details, new sans font, update query string on filter/sort, persist experimental feature flag, batch edit redirect bug, upload file permissions and owner 2021-10-06 21:08:52 -05:00
advplyr
75aede914f Fix URL encoding, fix download m4b cover art 2021-10-06 13:00:12 -05:00
advplyr
19dcb6173e Fix download requests, v1.4.0 rollout 2021-10-06 07:23:32 -05:00
advplyr
d9d34e87e0 Support multi library 1.4.0 2021-10-05 21:10:49 -05:00
advplyr
577f3bead9 Support for libraries and folder mapping, updating static cover path, detect reader.txt 2021-10-04 22:11:42 -05:00
advplyr
a590e795e3 Fix ebook url, add opus to uploader, set chapters after track updates 2021-10-02 15:36:33 -05:00
advplyr
bd336345ee Support for opus audio type, experimental features toggle, epub reader starting point 2021-10-02 13:50:39 -05:00