tubearchivist
/
tubearchivist
mirror of https://github.com/tubearchivist/tubearchivist.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,005 Commits 2 Branches 38 Tags 17 MiB
Commit Graph

746 Commits

Author SHA1 Message Date
simon f94bbec672
add cookie wiki link 2022-05-07 19:00:40 +07:00
simon 3722f11a65
cleanup cookie file after task 2022-05-07 09:34:51 +07:00
simon 087043811f
avoid unneeded cookie open 2022-05-07 09:34:33 +07:00
simon 160c23c36c
conform import path style 2022-05-06 08:09:19 +07:00
simon 94fa191fb8
skip premium videos, clean exit for empty bulk_list, #237 2022-05-05 23:38:10 +07:00
simon 9a2b29c2ae
implement channle name change in reindex, #211 2022-05-05 22:59:56 +07:00
simon 7a51368955
fix skip deleting none existing subtitles, add update_media_url 2022-05-05 22:56:22 +07:00
simon 7fa0d0f331
untangle delete subtitles from delete media file, #195 2022-05-05 09:11:37 +07:00
Nathan DeTar 4eba36f7a0
Cookie Validation Button (#242) 
* Added cookie validate button & message placeholder

* Added handleCookieValidate(), postCookieValidate()

* show validation only if enbled, add processing message for immediate feedback

Co-authored-by: simon <simobilleter@gmail.com>
 2022-05-04 17:07:01 +07:00
simon 1613b71942
hide cookie after rescan and download process 2022-05-03 17:32:32 +07:00
simon 83beb53860
simplify _add_single to call update_status in PendingInteract directly 2022-05-02 21:38:03 +07:00
simon 8a4c50779a
standardize prints for PostData mapper 2022-05-02 18:39:54 +07:00
simon ca5b00a373
refresh for PendingInteract delete to avoid race condition, #217 2022-05-02 18:20:56 +07:00
simon 34a1fe9e8e
fix thumb re-embed task, #231 2022-05-02 17:38:20 +07:00
simon 34c18e75f5
add LL and WL as valid playlists with cookie 2022-05-02 11:30:38 +07:00
simon 6fbac12314
fix channel_validate in PlaylistSubscription, #223 2022-05-02 11:27:50 +07:00
Nathan DeTar c4107fc370
Fix to prevent holding onto SB info. (#236) 2022-05-02 10:41:42 +07:00
simon ca3ea20283
add socket_timeout when passing check_formats to yt-dlp 2022-05-02 09:48:52 +07:00
simon cd139dfc1c
API: add cookie validation view 2022-04-30 19:13:49 +07:00
simon bc7d90f1f4
raise FileNotFoundError when trying to use unavailable cookie 2022-04-30 19:09:19 +07:00
simon 3a21690309
implement post_process_updated for settings, import cookie 2022-04-30 18:35:04 +07:00
simon 487c81d50b
fix FileNotFoundError in cookie hide 2022-04-30 18:34:28 +07:00
simon 088e87bccf
add cookiefile to PendingList and VideoDownloader 2022-04-30 17:27:57 +07:00
simon 6cc5fd8890
add cookiefile to yt_obs in YouTubeItem baseclass 2022-04-30 16:40:36 +07:00
simon 4e3d71e0e3
simplify, un-userspaced 2022-04-30 16:38:23 +07:00
simon 9b107b9940
implement cookie validator 2022-04-30 13:05:01 +07:00
simon 43eec5aac0
move CookieHandler to separate module 2022-04-30 12:05:41 +07:00
simon 430fdb6f1c
add cookie import to frontend 2022-04-29 18:38:43 +07:00
simon 1fa26cdc44
added basic cookie import functionality 2022-04-23 21:59:59 +07:00
simon eb7313fe6b
API: add run task view 2022-04-23 20:50:38 +07:00
simon 71b3654942
add localhost:3000 to allowed 2022-04-23 20:17:31 +07:00
simon 3f99f7edff
add localhost to allowed origin, optional cors disable 2022-04-23 20:16:28 +07:00
simon b76f38e0bc
API: fix downloads list sort 2022-04-21 05:45:55 +07:00
simon 1477370376
init data instead of class attribute 2022-04-20 22:51:30 +07:00
simon 04fc6ed26a
API: add pagination 2022-04-20 22:43:07 +07:00
simon 3147df20da
skip subtitle segments without duration, take 2 2022-04-18 11:52:13 +07:00
simon d086f63861
API: sort and query filter download view, delete by filter 2022-04-17 20:10:49 +07:00
simon 40bb3e880e
API: implement status update and delete of item in queue 2022-04-17 19:15:40 +07:00
simon eb6d6be3b9
handle 404 in channel-video and playlist-video api view 2022-04-17 10:08:24 +07:00
simon b8ca324aaf
add playlist-video and channel-video api views 2022-04-17 09:58:18 +07:00
simon 7d45d23767
process ta_download search results 2022-04-17 05:21:35 +07:00
simon 9224696e33 Merge branch 'api-fix' into testing 2022-04-17 05:03:05 +07:00
simon 50f4e0e5b7
skip base64 generator, hotfix 2022-04-16 13:08:32 +07:00
simon fcdb31e848
bump release number 2022-04-16 09:30:40 +07:00
simon 59f1c111aa
extend timeout for sleep interval 2022-04-16 08:58:35 +07:00
Nathan DeTar 1cf82bdbbd
Cleanup & Prevent repeat of skip message. (#227) 
* Cleanup & Prevent repeat of skip message.

* skip sponsorblock segments if not indexed

* disable onVideoProgress sb check if not indexed

Co-authored-by: simon <simobilleter@gmail.com>
 2022-04-16 08:53:15 +07:00
simon ef803a157a
standard api key processor for list and single views 2022-04-16 00:37:56 +07:00
simon 51ceffd58f
update all links to tubearchivist organization 2022-04-15 22:35:13 +07:00
simon fe610fdaca
add log output for backup progress 2022-04-15 20:41:38 +07:00
simon c84fbdfb9c
fix _check_get_sb for per channel deactivate 2022-04-15 20:04:01 +07:00
simon 7231b5d245
implement unset of sb per channel overwrite 2022-04-15 20:03:15 +07:00
Ainsey11 dc29c6718c
Fix: #210 - Download UI Box being removed when file is still processing (#219) 
* Fix: https://github.com/bbilly1/tubearchivist/issues/210 - Download UI element is removed when the file has finished downloading, but is still being moved on disk to the storage destination. This means the user is presented with nothing in the UI yet the file is still being processed.

* FIX: Resolving linting issue

* FIX: Set moving message to never expire and then a second message after the move with a 4 second expiry timer.

Co-authored-by: Rob Ainsworth <roba@immjsystems.com>
 2022-04-15 15:21:24 +07:00
simon d4b1d97f5c
add mappings for sponsorblock segments 2022-04-15 15:16:31 +07:00
simon f71644c8bc
guarantee stats in mapping for sort 2022-04-15 15:03:43 +07:00
simon 4853a8b12b
limit video_daily to below 10k in daily refresh 2022-04-14 16:18:08 +07:00
simon fd00369859
fix last page pagination link building error, #221 2022-04-14 16:04:21 +07:00
simon a8a7edb93e
skip subtilte events without duration, #196 2022-04-14 15:53:10 +07:00
simon c59c1fc007
improved sponsorblock indexing, #build 
Changed:
- improved more flexible sb timestamps indexing
- fixed per channel sb activate/deactivate
- improved Dockerfile with multi stage build
- added playlist API list view
 2022-04-13 15:56:05 +07:00
simon 3007e02fe5
fix per channel sb integration 2022-04-13 15:53:00 +07:00
Nathan DeTar 39c4bd8883
Adjust to API changes for SB integration. (#222) 2022-04-13 10:30:32 +07:00
simon cd3e9dd024
add playlist API list view 2022-04-13 09:51:15 +07:00
simon 8edde732b6
improved sponsorblock key with additional metadata 2022-04-12 17:47:13 +07:00
simon 3df1df8b5a
set explicit mappings for sponsorblock key 2022-04-12 17:45:06 +07:00
simon 385f53372c
bump django 2022-04-11 17:57:29 +07:00
Ainsey11 a8ee0da2b8
Addiing clarification on when the speed limit change takes effect (#218) 
Relates to https://github.com/bbilly1/tubearchivist/issues/216 

Speed limits work as designed, however the wording in the tooltip is a little unclear to newer users about when the speed limit takes effect on their downloads
 2022-04-10 22:33:00 +07:00
Nathan DeTar c316d05549
Added sponsorblock skipping (#208) 
* Added sponsorblock skipping.

* Basic framework for sending SB timestamps

* Sponsorblock send timestamp UI improvements

* Added Sponsorblock Icons

* Minor UI tweaks

* Revert UI changes, implement in new UI

* Added notification when sponsor segment is skipped

* Add formatting for notifications & SB messages

* Added SB messages to JS player

* Added SB skip notifcation to videos page.

* Added SB messages to video page

* Change SB messages.

* Check channel_overwrites

* Check Per Channel Settings.

* Cleanup
 2022-04-10 16:20:58 +07:00
simon 31378ac756
better generic schedule validator for impossible input, #209 2022-04-10 15:58:11 +07:00
simon 10385b1414
bump bs4 2022-04-10 15:57:15 +07:00
simon d8c7b3df0b
better error message for wrong es version, #197 2022-04-09 14:12:03 +07:00
simon 99e0c1c90e
bump yt-dlp 2022-04-08 23:15:58 +07:00
simon 236215cc4c
add config var to video.html template 2022-04-08 22:10:16 +07:00
simon 552636d882
handle 404 in video item API view 2022-04-08 17:55:24 +07:00
simon 9d73dbc45a
implement video overwrites index for sponsorblock 2022-04-08 17:19:25 +07:00
simon 859bf2a28d
add target _blank to external links 2022-04-08 17:18:33 +07:00
simon 365ebf53a5
implement per channel sponsorblock 2022-04-08 17:17:39 +07:00
simon fe3e3cfacc
fix filenotfounderror for missing subtitles when deleting video 2022-04-08 15:56:34 +07:00
simon efa240440b
fix userspace for vide base config 2022-04-08 15:15:59 +07:00
simon e2f4dd124a
bump beautifulsoup4 2022-04-08 15:15:37 +07:00
simon a07d789e66
add base64 blur video thumb 2022-04-08 00:29:09 +07:00
simon 6a1cb15114
validate hours to not be greater than 23, #209 2022-04-07 23:02:07 +07:00
simon b1a7a6a148
use cleaned_data for config form parser 2022-04-07 22:30:20 +07:00
simon a84e657577
add documentation for sponsorblock api endpoints 2022-04-05 22:27:55 +07:00
simon 03dd25cff4
implement vote on sponsorblock segments api 2022-04-05 22:25:40 +07:00
simon 286b1cf9b6
simulate sponsorblock post request 2022-04-05 21:51:45 +07:00
simon bace7d41af
add random string generator helper function 2022-04-05 21:51:10 +07:00
simon 738b083a7f
create dedicated SponsorBlock class to handle integration 2022-04-05 21:50:40 +07:00
simon 1ac92254ad
add global vars for version and url 2022-04-05 18:43:15 +07:00
simon e404bac7c3
bump celery 2022-04-05 18:35:06 +07:00
simon a8b62e5cdc better error message on sponsorblock fail 2022-04-04 23:28:32 +07:00
simon 730d12212f implement basic sponsblock indexing, #119 2022-04-04 23:28:32 +07:00
simon 29f8e148da bump libraries 2022-04-04 23:28:32 +07:00
simon cae00b032b return better data for ping api, add to readme 2022-04-01 16:24:40 +07:00
simon 9ab6955125
add connection test api view 2022-03-31 21:13:21 +07:00
simon 46622d9ab2
bump redis lib 2022-03-31 21:13:04 +07:00
simon 6f5969f520
add video list api view 2022-03-29 17:05:22 +07:00
simon 0c8fa8f49e
clean up prints 2022-03-29 17:03:34 +07:00
simon 80af255f25
reinvent SearchProcess class for API endpoints 2022-03-29 16:48:04 +07:00
simon cdc617c382
add datestr generator 2022-03-29 16:47:21 +07:00
simon f1fb504874
fix spacing 2022-03-29 10:33:43 +07:00
simon 5dce2441fa
add API login view 2022-03-29 10:17:47 +07:00
simon 5d8e314983
be more explicit what you are searching now 2022-03-28 12:43:57 +07:00
simon 358194468e
increase version to v0.1.3 2022-03-26 19:47:50 +07:00
simon f15bb50cfe
fix playlist reindex after PendingList refactor 2022-03-26 18:31:00 +07:00
simon fcadb5ead8
add random headers for requests outside of yt-dlp 2022-03-26 11:49:53 +07:00
simon a0ed88580f
fix index validate_mappings for new nested object 2022-03-26 10:52:57 +07:00
simon 40d7e6da37
add secondary sort for channel and title, #193 2022-03-25 17:28:17 +07:00
simon bcf9185bd3
implement es version check at startup 2022-03-25 15:33:09 +07:00
simon 337b373628
improve build to install debug tools in testing 2022-03-24 19:54:47 +07:00
simon 59d58f8866
bump redis version 2022-03-24 19:51:45 +07:00
simon d0f826485f
move and rename configure button 2022-03-24 16:10:40 +07:00
simon d5cd90eb34
style and toggle hide overwrite form 2022-03-24 15:43:15 +07:00
simon 8d1d09e698
extend IndexPaginate to optionally return _source key 2022-03-23 17:17:42 +07:00
simon fb4c11bd38
refactor ElasticIndex and ElasticBackup to use ElasticWrap and IndexPaginate 2022-03-23 17:17:17 +07:00
simon bfcc538ed1
fix concurrency issue with subtitle indexing 2022-03-23 16:20:33 +07:00
simon 912c19f6cf
use ElasticWrap in FilesystemScanner 2022-03-23 15:56:53 +07:00
simon fda520ad44
refactor and consolidate Reindex class 2022-03-23 15:48:38 +07:00
simon 1f7d6871cf
use ElasticWrap to delete item from pending 2022-03-23 11:09:21 +07:00
simon 5b1c9c64de
cleanup, use ElasticWrap 2022-03-23 09:44:31 +07:00
simon f802c4d596
remove logging 2022-03-22 18:43:29 +07:00
simon 192d379a3e
refacter api views to use ElasticWrap 2022-03-22 18:43:16 +07:00
simon 9f5c9b17a5
refactor redis connection, fix sync_redis_state task setup issue 2022-03-22 17:50:54 +07:00
simon fa25a56126
fail open for in continue watching videos building 2022-03-22 13:07:22 +07:00
simon 75cd9d382d
fix duration builder on reindex by ignoring vtt files 2022-03-22 12:59:39 +07:00
simon fd4f15ab66
refactor StartupCheck into class, lock if already run for multithreading 2022-03-22 10:35:36 +07:00
simon 1f1dfcb54f
better playlist scan message, private methods in YoutubeChannel class 2022-03-22 10:07:33 +07:00
simon 3d451e47e5
full playlist refresh and index for channel overwrites 2022-03-22 09:42:41 +07:00
simon fc10fa62e8
remove now redundant find playlist botton 2022-03-21 19:07:58 +07:00
simon 875c601f4e
implement index channel playlist from form 2022-03-21 18:27:00 +07:00
simon 91a9477bba
refactor index_channel_playlists task, move to YoutubeChannel class 2022-03-21 17:59:21 +07:00
simon fc311a7b7e
refactor validate_playlists and move into DownloadPostProcess class 2022-03-21 11:14:44 +07:00
simon f803c5298b
fix missing update_status method call in ignore video 2022-03-21 11:09:16 +07:00
simon e67d576d79
fix missing video list in index_channel_playlists 2022-03-21 10:47:15 +07:00
simon 7c6041d686
refactor VideoDownloader and implement per channel delete older than 2022-03-19 17:37:28 +07:00
simon 1411262793
remember channel overwrites on rescan, remove unused rescape_all_channels method 2022-03-19 15:12:29 +07:00
simon 9fd7e2b11a
rename channel_overwrites attribute 2022-03-19 15:03:38 +07:00
simon 3ba6e0478d
fix filesysem rescan function 2022-03-19 14:36:46 +07:00
simon b68a00073f
fix thumbnail validation with new PendingList class 2022-03-19 14:27:37 +07:00
simon 89428297c9
add download format channel overwrite per video 2022-03-18 22:22:41 +07:00
simon a7945e30e3
fix building missing vid ids list 2022-03-18 22:11:41 +07:00
simon ac9df4e082
map channel overwrite to video id for later efficient mapping 2022-03-18 21:39:33 +07:00
simon a6937db5fd
use the refactored PendingList class 2022-03-18 18:27:25 +07:00
simon 05b8dbc02f
refacor PendingList class into subclasses 2022-03-18 17:19:21 +07:00
simon 1498fadf27
implement channel playlist index overwrite 2022-03-16 12:32:42 +07:00
simon f0e82caebb
add bool true to channel overwrite form parser 2022-03-16 12:32:02 +07:00
simon 2ef8823c2d
better channel overwrite update form 2022-03-16 12:09:50 +07:00
simon dc67293052
add fallback channel overwrite fallback to frontend 2022-03-15 17:45:43 +07:00
simon 544d842a4a
rename channel format overwrite field 2022-03-15 17:45:15 +07:00
simon 774220ab1f
add channel overwrite form to frontend, store in es 2022-03-15 12:00:48 +07:00
simon 105d5bf3f7
add auto label to subtitle track 2022-03-13 22:56:20 +07:00
simon 026cc378fe
add get_overwrites and set_overwrites methods for channel 2022-03-12 22:19:42 +07:00
simon 8170d4913b
add fallback total video length if ffmpeg extraction failes 2022-03-12 21:53:03 +07:00
simon 7595e7501f
sort continue watching videos 2022-03-12 20:29:26 +07:00
simon f6950a2ca5
list all in progress videos 2022-03-12 17:29:34 +07:00
simon aff0cfb794
fix retiming issue for auto subtitles 2022-03-11 17:47:04 +07:00
simon d3e9646fb6
private methods for YoutubeSubtitle and SubtitleParser 2022-03-10 20:45:13 +07:00
Nathan DeTar 6e3df21f8c
Continue Watching Section (#188) 
* Replaced isWatched() function.

* Switched to `updateVideoWatchStatus()` function

* Updated Onclick to `updateVideoWatchStatus(this)`

* Handle `this` input in `updateVideoWatchStatus()`
 2022-03-10 20:20:23 +07:00
simon 1ce832b846
fix autocaption extraction, flatten words, #180 2022-03-10 19:58:13 +07:00
simon 6a6c8fa5d8
bump yt-dlp version 2022-03-10 17:39:35 +07:00
simon c186798e78
rewrite SubtitleParser, #180 2022-03-09 00:25:44 +07:00
simon 40c8e6d146
standardize template jinja indentation 2022-03-07 20:58:17 +07:00
simon fb12a32d4f
use generic data-id and data-status attributes for watched checkbox 2022-03-07 20:48:45 +07:00
simon 437e83b2ae
add continue watching section to home page 2022-03-07 20:26:37 +07:00
simon 25bc66ae80
bump django version 2022-03-07 20:23:23 +07:00
Nathan DeTar ad5e74cb27
Prevent setting progress bar on player close if video is watched. (#182) 
* Prevent setting progress bar on close if watched.
 2022-02-26 18:11:09 +07:00
simon b28905bbae
bump footer version v0.1.2 2022-02-25 12:27:16 +07:00
simon 36a2996cda
clean exit of progress builder when no videos in channel 2022-02-25 12:20:22 +07:00
simon b98ccd9dab
handle error if subscribed channel not available anymore, #175 2022-02-25 11:21:29 +07:00
simon 060f0d575e
change color of progress bar to dark 2022-02-25 10:51:37 +07:00
Nathan DeTar 99c97a703f
Reduce API Calls (#181) 
* Reduce API calls

* Fix video id

* Updated `createVideoTag()` description.

* Fixed URL used for cast integration

* Check video duration

* Updates progress bar on watched and close.

* Set progress bar width to 0% by default

* Cleanup, function descriptions

* Cleanup console logging

* Update progress bar on cast progress every 10s

* Catch short <30s videos and mark as watched
 2022-02-25 10:39:33 +07:00
simon 806448624d
return if no in progress videos 2022-02-24 21:25:12 +07:00
simon df777104af
add video progress bar to channel_id and playlist_id views 2022-02-24 19:30:12 +07:00
simon d88d6d6a61
add video progress bar 2022-02-24 18:58:26 +07:00
simon 6078d8d276
add youtube_id to progress api key 2022-02-24 18:55:52 +07:00
simon c4d6bb35a3
add list_items for wildcard matching 2022-02-24 18:55:18 +07:00
simon e0db73543e Merge branch 'master' into testing 2022-02-24 09:39:56 +07:00
Nathan DeTar 4812b8da55
Save Video Progress (#179) 
* Added cast integration docs.

* Changed header sizes.

* Added more space above Requirements

* Added cast integration docs.

* Removed separate cast integration docs

* Further indented quote from Google

* Switch to HTML based video position.

* Ground work for API changes and video progress

* Added onpause attribute to video.

* Added save video progress feature.

* Added API check for subtitle status.

* Switch method to DELETE if position is 0

* Added `createVideoTag()` function

* Added `InsertVideoTag()` function

* Switch to JS generated video tag, add on page load

* Removed extra data from DELETE request

* Removed unused code

* Reduced duplicate code

* Cleanup & groundwork cast pull metadata from API

* Minor bug fix

* Fix saving video progress on player close.

* Only send video progress when unwatched

* Cleanup

* Added `getURL()` function

* Cast use API & save progress/mark as watched

* Added cast progress checks

* Changed thresholds for marking videos as watched

* Added `watchedThreshold()` function
 2022-02-24 09:36:31 +07:00
simon b8b95f9d79
inheirt VideoProgressView from base class to get auth 2022-02-21 20:59:04 +07:00
simon ec00568008
fix auto reset of playlist_subscribed 2022-02-20 19:22:11 +07:00
simon 241d8326f7
add video progress API endpoints 2022-02-17 18:20:30 +07:00
simon 4d83af7c14
bump redis client 2022-02-17 18:19:52 +07:00
simon 265795f4a9
add config key to api response 2022-02-17 16:59:59 +07:00
simon 047fc1f3f1
bump release version 2022-02-13 10:58:58 +07:00
simon 8fce31b983
fix reindex and deactivate error 2022-02-13 10:05:08 +07:00
simon fef866bf0b
indicate max page size on downloads page, add note to wiki, #156 2022-02-12 19:42:07 +07:00
simon 428cc315e4
Subtitle improvements, #build 
Changes:
- merges subtitle suport for JS video player
- merges hint what to do when no videos found
- merges better indexing and error handeling of subtitles
 2022-02-12 19:11:47 +07:00
Nathan DeTar 385d6bace8
Fix cast support to handle new video tag format. (#169) 
* Added subtitle support to JS player.

* Move `video-item` id to source tag.

* Move `video-item` id to source tag.

* Fix cast support to handle new video tag format

* Add subtitle support to cast integration, WIP

* Replace `&amp` with `&` in video titles.

* Check if the video is already marked as watched

* Switch to HTML watched check.
 2022-02-12 19:08:19 +07:00
simon 78720b33b7
clean exit when auto subtitle extract fails 2022-02-12 18:52:03 +07:00
simon ce4fa8ee61
delete existing before of reindexing subtitles 2022-02-12 18:22:01 +07:00
simon a5f6be16e1
switch subtitle source order 2022-02-12 17:09:05 +07:00
simon b494fc10af
cover edge cases where end timestamp is after start timestamp of new cue 2022-02-12 16:03:04 +07:00
simon 09500557c8
open fail on subtitle line parse error 2022-02-11 18:47:29 +07:00
simon 7524691b79
implement API token revoke 2022-02-11 18:19:10 +07:00
simon f1de8db4f3
hide API token by default 2022-02-11 17:18:37 +07:00
simon 08f5248e7a
make subtitle index optional 2022-02-11 13:36:36 +07:00
Stephen Herbein 4467e97f3c
When no videos are found, add a hint to start a search/download (#167) 2022-02-11 12:52:26 +07:00
simon af8e01cd8c
fix clean exit if no subtitles selected 2022-02-11 09:03:56 +07:00
Nathan DeTar 656a0c7327
Added subtitle support to JS player. (#168) 2022-02-11 08:53:09 +07:00
simon 16f33feda0
process subtitle media url paths 2022-02-10 19:45:22 +07:00
simon 3ea5e9c537
bump dependencies 2022-02-10 19:27:05 +07:00
simon 4d30bed3cc
extend delete video to also delete subtitles 2022-02-10 19:09:07 +07:00