Mirrors/audiobookshelf
Mirrors/audiobookshelf
1
0
Fork 0
mirror of https://github.com/advplyr/audiobookshelf.git synced 2024-12-20 19:06:06 +01:00
Code Issues Packages Projects Releases Wiki Activity
4,746 Commits 13 Branches 130 Tags 62 MiB
e50bd93958
Commit Graph

345 Commits

Author SHA1 Message Date
advplyr
d9e7f5d133 Update BinaryManager JSDocs, move validVersions to required binary objects 2024-02-17 17:40:33 -06:00
advplyr
a119b05d85 Merge branch 'master' into ffmpeg-latest 2024-02-17 17:05:51 -06:00
advplyr
af7cb2432b Update:Log uncaught exceptions to crash_logs.txt #706 & cleanup logger 2024-02-15 16:46:19 -06:00
mikiher
d43a1109c8 Modify BinaryManager to download version 6.1 and remove old dowloaded versions 2024-01-25 17:51:06 +02:00
advplyr
69e23ef9f2 Add:Epub metadata parser and cover extractor #1479 2024-01-07 17:51:07 -06:00
mikiher
3051b963ef
Merge branch 'advplyr:master' into binary-manager 2023-12-27 06:44:22 +02:00
advplyr
fae383a045 Fix:RSS feeds for collections not updating #2414 2023-12-14 15:45:34 -06:00
mikiher
8f7a420cca Fix directory writable check (fs.access not working on Windows) 2023-12-14 09:47:18 +02:00
advplyr
6f6395bad7 Only log update binary env path if it was updated 2023-12-07 17:32:06 -06:00
mikiher
898b072e68
Merge branch 'advplyr:master' into binary-manager 2023-12-06 09:27:17 +02:00
advplyr
34156af403 Fix:Updating media progress not clearing cache #2392 2023-12-05 17:58:54 -06:00
advplyr
1ce1904c89 Add ffbinaries lib 2023-12-05 17:35:15 -06:00
mikiher
2e989fbe83 Add BinaryManager 2023-12-05 21:19:17 +02:00
mikiher
3ff41f2b43 Cache HTTP headers and status 2023-11-25 23:49:56 +02:00
advplyr
572fb0993c Rename ApiCacheManager to add .js file extension 2023-11-24 14:20:14 -06:00
mikiher
5e1e748c71 Add ApiCacheManager unit test 2023-11-23 09:53:52 +02:00
mikiher
107b4b83c1 Add cache middleware to most /libraries get requests 2023-11-22 18:40:42 +02:00
mikiher
4dec8c265d Add ApiCacheManager 2023-11-17 08:47:40 +02:00
advplyr
5220361151 Fix:Podcast episode cron not adding/removing library items correctly #2277 2023-11-03 07:07:58 -05:00
advplyr
bef6549805 Update:Replace library scan toast with task manager #1279 2023-10-20 17:46:18 -05:00
advplyr
6f65350269 Update:JSDocs for task manager 2023-10-20 16:39:32 -05:00
advplyr
0d5792405f Fix:Podcast episodes store RSS feed guid so they can be matched if the RSS feed changes the episode URL #2207 2023-10-16 17:47:44 -05:00
advplyr
c98fac30b6 Update:Validate image URI content-type before writing image file 2023-10-14 10:52:56 -05:00
advplyr
290a377ef9 Update:Remove local cover path input & replace with url from web input, include SSRF request filter 2023-10-13 16:33:47 -05:00
advplyr
e687a3403e Fix:Cleaning up orphan streams on server init #2209 2023-10-11 17:05:56 -05:00
advplyr
c2c8cf919e Fix:Bad backup causing other backup files to not be displayed #1961 2023-09-30 16:01:10 -05:00
advplyr
102c90c4e8
Merge pull request #2133 from mfcar/mf/backup 
Add more information to the backup page
 2023-09-22 16:56:12 -05:00
advplyr
5e976c08af Update cover API endpoint to only load necessary data from DB #2073 2023-09-21 16:57:48 -05:00
mfcar
bfa87a2131
Add a way to see the backup location 2023-09-20 22:33:58 +01:00
advplyr
604f52762b
Merge pull request #2120 from itzexor/x-accel-encode 
[server] x-accel: encode all paths to URIs
 2023-09-18 17:51:53 -05:00
advplyr
b5a27226cc Fix:Misleading log on cover manager 2023-09-18 16:45:30 -05:00
James Ross
207ba7ec8e x-accel: encode all paths to URIs 
updates util function  encodeUriPath to use node:url with a file:// path
prefix, and updates all instances x-accel redirection to use this helper
util instead of sending unencoded paths into the header.
 2023-09-18 13:08:19 -07:00
advplyr
0aae672e19 Fix:Scanner purge cover cache when extracting from audio file 2023-09-17 14:53:25 -05:00
advplyr
efae529fac Add cover finder to new book scanner 2023-09-06 17:48:50 -05:00
advplyr
f02992dd4d Remove the setting of file permissions #2057 2023-09-06 07:12:11 -05:00
advplyr
1dd1fe8994 Update match all books to load items from DB, remove library items loading to memory on init 2023-09-04 16:33:55 -05:00
advplyr
b9da3fa30e Add new podcast scanner and remove old scanner 2023-09-04 11:50:55 -05:00
advplyr
9123dcb365 Remove series search api endpoint, update authors and series to load from db 2023-09-03 10:49:02 -05:00
advplyr
2df95c1712 Updates for new book scanner 2023-09-02 17:49:28 -05:00
advplyr
0ecfdab463 Update new library scanner for scanning in new books 2023-09-01 18:01:17 -05:00
advplyr
75276f5a44 Fix:Server crash when updating cover to a directory #2007 2023-08-30 18:05:52 -05:00
Shawn Hoffman
24989e73ae
Merge branch 'master' into shawn/rss-feeds 2023-08-22 10:30:16 -07:00
Shawn Hoffman
13427b9f70 Add RSS feeds config page 2023-08-22 10:11:10 -07:00
advplyr
6c1b4e3a36 Update db model references 2023-08-20 13:34:03 -05:00
advplyr
ff0d6326d3 Update OPML api route to load podcasts from db 2023-08-19 15:19:27 -05:00
advplyr
3651fffbee Update library filter data to load from db and cache, update rss feed routes to load library items from db 2023-08-13 15:10:26 -05:00
advplyr
f465193b9c Update User.toJSONForPublic to remove mostRecent key and session key only includes the PlaybackSession 2023-08-12 16:11:58 -05:00
advplyr
f1130eb63a Update MeController api endpoints to load library items from DB 2023-08-12 15:52:09 -05:00
advplyr
db80cec168 Update collection API routes to load libraryItems from DB 2023-08-12 15:01:27 -05:00
advplyr
43a5296dd7
Update server/managers/BackupManager.js 2023-08-07 17:14:47 -05:00
Niclas Haderer
409c5f7b75 fix: the server does not crash any more when an invalid backup file is uploaded 2023-08-06 10:05:53 +02:00
advplyr
58da095bcf Update query for continue series shelf 2023-08-03 18:14:25 -05:00
advplyr
5a9eed0a5a Update:Only load collections when needed 2023-07-22 16:18:55 -05:00
advplyr
1d974375a0 Update:Only load libraries from db when needed 2023-07-22 14:25:20 -05:00
advplyr
daa8c4cd67 Update:Remove sort index from podcast episodes 2023-07-22 09:24:46 -05:00
advplyr
22323f606d Fix:RSS feed covers #1948 2023-07-21 16:59:00 -05:00
advplyr
6814adffcc Update:Only load feeds when needed 2023-07-17 16:48:46 -05:00
advplyr
5b6878e5de Fix:Crash on local playback sessions #1912 2023-07-17 13:58:19 -05:00
advplyr
d99b2c25e8 Fixes for db migration & local playback sessions 2023-07-16 15:05:51 -05:00
advplyr
c881bcbe59 Update logs for cache purge 2023-07-14 15:04:27 -05:00
advplyr
c5a4f63670 Update Backup to use key to check for old backups no longer supported 2023-07-14 14:20:35 -05:00
advplyr
d56e3a3617 Merge branch 'master' into sqlite_2 2023-07-11 17:07:13 -05:00
advplyr
995cf51ae3 Update:Default m4b encoding bitrate to 128k #1892 2023-07-11 16:57:30 -05:00
advplyr
f73a0cce72 Update Dockerfile for sqlite3, update models for cascade delete, fix backup schedule 2023-07-09 11:39:15 -05:00
advplyr
254ba1f089 Migrate backups manager 2023-07-08 14:40:49 -05:00
advplyr
b5d4c11f6f Fix RSS feeds to use slug instead of id 2023-07-06 17:07:10 -05:00
advplyr
a0bc959850 Add feed migration and cleanup 2023-07-05 18:18:37 -05:00
advplyr
cf7fd315b6 Init sqlite take 2 2023-07-04 18:14:44 -05:00
advplyr
aeba7674f8 Add new api route for downloading backup, remove static metadata route 2023-06-27 16:41:32 -05:00
advplyr
d748d43efc Fallback to using from address if test address is not set, add reset button when form has changes 2023-06-12 17:12:52 -05:00
daVinci2793
d54edb93d6 Updates to Email settings/manager to include test email 2023-06-12 04:53:51 +00:00
advplyr
05ce9c6eda Add:Email smtp config & send ebooks to devices #1474 2023-05-29 17:38:38 -05:00
advplyr
15aaf2863c Add:OPML Export #1260 2023-05-28 15:10:34 -05:00
advplyr
ea79948122 Fix:Podcast episode downloads where RSS feed uses the same title #1802 2023-05-28 11:24:51 -05:00
advplyr
53c96b2540 Update:Handle multiple sessions open, sync when paused, show alert of multiple sessions open when both are playing #1660 2023-05-27 17:21:43 -05:00
advplyr
b52e240025 Add:Batch re-scan #1754 2023-05-27 14:51:03 -05:00
advplyr
2cc23b6d6b Update:Auto update home page shelves when new episode is added #716 2023-05-27 09:13:44 -05:00
advplyr
1da471e136 Fix:Embed metadata tool embed ASIN, SERIES and SERIESPART #1794 2023-05-26 17:57:56 -05:00
advplyr
2ab287e2a9 Fix:Podcast cron filter out failed library item 2023-05-08 16:20:09 -04:00
advplyr
3dc9416da6 Add:Chapters to podcast episodes #1646 2023-04-09 14:32:51 -05:00
advplyr
25ca950dd0 Update listening sessions per device and show open sessions 2023-04-08 18:01:24 -05:00
advplyr
05b102722b Remove unused ebook routes 2023-04-03 17:33:02 -05:00
advplyr
ef954ee68f Remove downloads folder in metadata dir 2023-04-03 17:28:55 -05:00
advplyr
034b8956a2 Add:Batch embed metadata and queue system for metadata embedding #700 2023-04-02 16:13:18 -05:00
advplyr
1a3f0e332e Fix download podcast episode that is not mp3 2023-04-01 16:31:04 -05:00
advplyr
704fbaced8 Update:Download podcast episodes and embed meta tags #1488 2023-03-29 18:05:53 -05:00
advplyr
022bf9d0ef Show current episode download on init and download queue page updates 2023-03-05 10:35:34 -06:00
mfcar
61c759e0c4
Add tasks queue dropdown 2023-03-05 11:15:36 +00:00
mfcar
34ac972130
Add download queue 2023-02-27 02:56:07 +00:00
advplyr
caacf461ab Open rss feed metadataDetails optional 2023-02-25 08:53:09 -06:00
mfcar
0d3e6b1d0a
Add rss details configuration 2023-02-25 13:20:26 +00:00
advplyr
4039dc7968 Podcast episode download notification adding variables for mediaTags, podcastAuthor, podcastDescription, podcastGenres, episodeTitle, episodeSubtitle, episodeDescription 2023-02-15 15:57:04 -06:00
Philipp Rintz
e345c4cc9e
Correct the libraryTags variable 2023-02-15 00:00:34 +01:00
Philipp Rintz
7207efb4da
Add library tags variable to podcast notifications 2023-02-14 16:41:58 +01:00
advplyr
23f2c8a251 Fix:Replacing item cover remove old covers case insensitive #1391 2023-02-11 15:56:18 -06:00
advplyr
2ad03bcb9a Fix:Bad backup files and backing up playlists, feeds #1485 2023-02-10 15:33:42 -06:00
advplyr
f9e6655359 Update:API endpoint for syncing multiple local sessions. New API endpoint to get current user. Deprecate /me/sync-local-progress endpoint 2023-02-05 16:52:17 -06:00
advplyr
3383ec2046 Add logs to playback session manager 2023-02-04 13:23:13 -06:00
advplyr
337cf90c4b Add debug logs to playback sessions 2023-02-02 16:24:34 -06:00
advplyr
8908aa7a82 Fix:Podcast RSS feeds update on new/updated episodes #1435 2023-01-28 14:58:10 -06:00
advplyr
9ebe4b55dd
Merge pull request #1431 from lkiesow/x-accel 
Implement X-Accel Redirect
 2023-01-23 17:27:23 -06:00
advplyr
f7a3dbf209 Fix:Embed metadata tool embeds cover image 2023-01-22 18:02:57 -06:00
advplyr
d900093976 Fix:Make m4b embed cover image 2023-01-22 18:00:35 -06:00
Lars Kiesow
08250e266e
Implement X-Accel Redirect 
This patch implements [X-Accel](https://www.nginx.com/resources/wiki/start/topics/examples/x-accel/)
redirect headers as an optional way for offloading static file delivery
from Express to Nginx, which is far better optimized for static file
delivery.

This provides a really easy to configure way for getting a huge
performance boost over delivering all files through Audiobookshelf.

How it works
------------

The way this works is basically that Audiobookshelf gets an HTTP request
for delivering a static file (let's say an audiobook). It will first
check the user is authorized and then convert the API path to a local
file path.

Now, instead of reading and delivering the file, Audiobookshelf will
return just the HTTP header with an additional `X-Accel-Redirect`
pointing to the file location on the file syste.

This header is picked up by Nginx which will then deliver the file.

Configuration
-------------

The configuration for this is very simple. You need to run Nginx as
reverse proxy and it must have access to your Audiobookshelf data
folder.

You then configure Audiobookshelf to use X-Accel by setting
`USE_X_ACCEL=/protected`. The path is the internal redirect path used by
Nginx.

In the Nginx configuration you then configure this location and map it
to the storage area to serve like this:

```
location /protected/ {
  internal;
  alias /;
}
```

That's all.

Impact
------

I just did a very simple performance test, downloading a 1170620819
bytes large audiobook file from another machine on the same network
like this, using `time -p` to measure how log the process took:

```sh
URL='https://url to audiobook…'

for i in `seq 1 50`
do
  echo "$i"
  curl -s -o /dev/null "${URL}"
done
```

This sequential test with 50 iterations and without x-accel resulted in:

```
real 413.42
user 197.11
sys 82.04
```

That is an average download speed of about 1080 MBit/s.

With X-Accel enabled, serving the files through Nginx, the same test
yielded the following results:

```
real 200.37
user 86.95
sys 29.79
```

That is an average download speed of about 2229 MBit/s, more than
doubling the previous speed.

I have also run the same test with 4 parallel processes and 25 downloads
each. Without x-accel, that test resulted in:

```
real 364.89
user 273.09
sys 112.75
```

That is an average speed of about 2448 MBit/s.

With X-Accel enabled, the parallel test also shows a significant
speedup:

```
real 167.19
user 195.62
sys 78.61
```

That is an average speed of about 5342 MBit/s.

While doing that, I also peaked at the system load which was a bit lower
when using X-Accel. Even though the system was delivering far more data.
But I just looked at the `load1` values and did not build a proper test
for that. That means, I cant provide any definitive data.

Supported Media
---------------

The current implementation works for audio files and book covers. There
are other media files which would benefit from this mechanism like feed
covers or author pictures.

But that's something for a future developer ;-)
 2023-01-23 00:02:27 +01:00
advplyr
6dec750d3e Fix:Close open playback session on server when local playback session syncing from mobile 2023-01-15 15:00:18 -06:00
advplyr
62aec63d1d Fix:Backups to not backup temp db files 2023-01-08 09:59:24 -06:00
advplyr
08d16ce7c2 Silence remove invalid sessions debug log 2023-01-08 09:15:11 -06:00
advplyr
2cb3808326 Fix:Loading backups catching failed backups 2023-01-08 09:11:55 -06:00
advplyr
bdb6f0c0aa Update:Sync session API endpoint to not respond with a payload 2023-01-07 17:33:05 -06:00
advplyr
5a26704c32 Add:Option to disable backup of audio files in embed metadata tool #1370 2023-01-07 15:16:52 -06:00
advplyr
9a85ad1f6b Fix:Check if Windows before cleaning file path for POSIX separators #1254 2023-01-05 17:45:27 -06:00
advplyr
9de7be1cb4 Update scanner, music meta tags and fix issue with force update 2023-01-02 16:35:39 -06:00
advplyr
44363f05ac Start of new epub reader 2023-01-01 18:09:00 -06:00
advplyr
70ba2f7850 Add:RSS feed for series & cleanup empty series from db #1265 2022-12-31 16:58:19 -06:00
advplyr
c6763dee2d Remove invalid RSS feeds on init and remove feeds when associated entity is removed 2022-12-31 14:08:34 -06:00
advplyr
315c83e4c3 RSS feed for collection to update when any item in the collection is updated #606 2022-12-28 18:08:03 -06:00
advplyr
308ccf470f Add:Open RSS feed for collection #606 #1265 2022-12-27 18:03:31 -06:00
advplyr
061695f922 Add:API endpoint for opening RSS feed for collection #606 #1265 2022-12-26 17:48:39 -06:00
advplyr
e803dcd325 Update:RSS feed API routes 2022-12-26 16:58:36 -06:00
advplyr
0717ae39db Fix music fine file with inode 2022-12-24 11:12:39 -06:00
advplyr
33e183b802 Merge branch 'master' into socket-fixes 2022-12-23 07:27:14 -06:00
advplyr
b884f8fe11 Laying the groundwork for music media type #964 2022-12-22 16:38:55 -06:00
Paul Nettleton
2cba83f1dd Server socket event fixes 2022-12-22 16:26:11 -06:00
advplyr
616ffb8f79 Add:M4b tool configurable options bitrate/channels/codec #1029 #1257 2022-12-19 17:13:04 -06:00
advplyr
bc771a3a44 Delete DownloadManager.js 2022-12-19 16:20:18 -06:00
advplyr
d130dd6d5e Fix:Setting file ownership for /config and /metadata/logs #584 2022-12-15 17:30:45 -06:00
advplyr
41e192c6a5 Update more vars 2022-12-12 17:52:20 -06:00
advplyr
6b654d3c2d Update:Starting session for finished item sets the user start time back to 0 2022-12-12 17:18:56 -06:00
advplyr
67db41a525 Update:Get item cover API endpoint to allow for returning the raw cover image 2022-12-04 16:23:15 -06:00
advplyr
180293ebc1 Update:Cleanup socket usage & add func for emitting events to admin users 2022-11-24 16:35:26 -06:00
advplyr
e2af33e136 Update:Refactor socket connection management into SocketAuthority 2022-11-24 15:53:58 -06:00
advplyr
64a8a046c1 Update:Backups API endpoints, add get all backups route, update socket init event payload 2022-11-24 13:14:29 -06:00
advplyr
a2f1723642 Update log level for RSS feed requests 2022-11-21 06:39:32 -06:00
Lars Kiesow
c307b1e6fb No feed log level 
This patch drops the log level for logging requests to non-existing
feeds from error to debug. The reasoning behind this is that this is a
client error and a client error is returned and can be handled by the
client.

Otherwise anyone can easily spam the logs with error messages by just
requesting non-existing feeds.
 2022-11-21 01:54:25 +01:00
advplyr
e3ba739db5 Update:Encode & embed metadata API endpoints, separate cache & search endpoints into controllers 2022-11-19 13:28:06 -06:00
advplyr
2e5e02472c Update:Playback session sync local status codes 2022-11-17 17:00:37 -06:00
advplyr
c7794e00f6 Update:Author image from cache API status codes 2022-11-16 15:32:32 -06:00
advplyr
7d569e1e3e Update:Some incorrect status codes returned from API 2022-11-12 09:36:00 -06:00
advplyr
1118b8b782 Metadata embed and m4b merge fixes and cleanup 2022-11-05 13:13:52 -05:00
Keagan Hilliard
3824154c15 Forgot to update the merge 2022-11-03 10:20:32 -06:00
Keagan Hilliard
586c8a550a Removed a noisy log and limit chapter embedding to items with only 1 audiofile 2022-11-03 10:09:49 -06:00
Keagan Hilliard
d57effe97c Fixed a couple of issues, should be working well now 2022-11-03 09:32:50 -06:00
Keagan Hilliard
c1938f78c2 Added json file support 2022-11-02 19:40:50 -06:00
advplyr
efecf7ed82 Update:Podcast episode auto download schedule setting for max new episodes to download #1091 2022-10-26 16:55:16 -05:00
advplyr
142205f060 Add:Purge items cache button and api endpoint 2022-10-02 14:46:48 -05:00
advplyr
02d997897c Add:Cancel m4b merge button #1008 2022-10-02 14:31:04 -05:00
advplyr
39979ff8a3 Add:Tasks widget in appbar for merging m4bs & remove old m4b merge routes 2022-10-02 14:16:17 -05:00
advplyr
6701551289 Fix:Ensure podcast library item folder exists before downloading episodes #1019 2022-09-30 16:55:31 -05:00
advplyr
f36a5eae6d Update:Audiobook merge to set metadata with tone and replace m4b in library item #594 2022-09-26 18:07:31 -05:00
advplyr
97da73baf3 Update:Experimental metadata embed tool to use tone 2022-09-25 15:56:06 -05:00