tubearchivist
/
tubearchivist
mirror of https://github.com/tubearchivist/tubearchivist.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

documenting main pages

simon 2021-09-28 22:23:03 +07:00
parent 50cfabea6b
commit ac8d158960
5 changed files with 148 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
Channels.md

@ -1 +1,23 @@
# Channels Overview and Channel Detail Page
The channels are organized on two different levels:
## Channels Overview
Accessible at `/channel/` of your Tube Archivist, the **Overview Page** shows a list of all channels you have indexed.
- You can filter that list to show or hide subscribed channels from the drop down menu. Clicking on the channel banner or the channel name will direct you to the *Channel Detail Page*.
- If you are subscribed to a channel a *Unsubscribe* button will show.
The **Subscribe to Channels** button <img src="../tubearchivist/static/img/icon-add.svg?raw=true" alt="add icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);margin:0 5px;"> opens a text field to subscribe to a channel. You have a few options:
- Enter the YouTube channel ID, a 25 character alphanumeric string. For example *UCBa659QWEk1AI4Tg--mrJ2A*
- Enter the URL to the channel page on YouTube. For example *https://www.youtube.com/channel/UCBa659QWEk1AI4Tg--mrJ2A*
- Enter the video URL for any video and let Tube Archivist extract the channel ID for you. For example *https://www.youtube.com/watch?v=2tdiKTSdE9Y*
- Add one per line.
- **Note**: Adding a link to a YouTube username will not work, for example: *https://www.youtube.com/c/TomScottGo* will fail, because a user can have multiple channels. On YouTube same as on **Tube Archivist**, you can only subscribe to a channel and *not* a user.
The search icon <img src="../tubearchivist/static/img/icon-search.svg?raw=true" alt="search icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);margin:0 5px;"> opens a text box to search for indexed channel names. Possible matches will show as you type.
## Channel Detail
Each channel will get a dedicated channel detail page accessible at `/channel/<channel-id>/` of your Tube Archivist. This page shows all the videos you have downloaded from this channel plus additional metadata.
- If you are subscribed to the channel, an *Unsubscribe* button will show.
- You can *Show* the channel description, that matches with the *About* tab on YouTube.
- The **Mark as Watched** button will mark all videos of this channel as watched.

30
Downloads.md

@ -1 +1,29 @@
# Downloads Page Functionality
# Downloads Page
Accessible at `/downloads/` of your Tube Archivist, this page handles all the download functionality.
## Rescan Subscriptions
The **Rescan subscriptions** icon <img src="../tubearchivist/static/img/icon-rescan.svg?raw=true" alt="rescan icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);margin:0 5px;"> will start a background task to look for new videos from the channels you are subscribed to. You can define the channel page size on the [settings page](Settings.md#subscriptions). With the default channel page size, expect this process to take around 2-3 seconds for each channel you are subscribed to. A status message will show the progress.
Then for every video found, **Tube Archivist** will skip the video if it has already been downloaded or if you added it to the *ignored* list before. All the other videos will get added to the download queue. Expect this to take around 1 second for each video as **Tube Archivist** needs to grab some additional metadata. New videos will get added at the bottom of the download queue.
## Download Queue
The **Download Queue** icon <img src="../tubearchivist/static/img/icon-download.svg?raw=true" alt="download icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);margin:0 5px;"> will start the download process starting from the top of the queue. Take a look at the relevant settings on the [Settings Page](Settings.md#downloads). Once the process started, a progress message will show with additional details and controls:
- The stop icon <img src="../tubearchivist/static/img/icon-stop.svg?raw=true" alt="stop icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);margin:0 5px;"> will gracefully stop the download process, once the current video has been finished successfully.
- The cancel icon <img src="../tubearchivist/static/img/icon-close.svg?raw=true" alt="close icon" width="20px" style="filter:invert(16%) sepia(60%) saturate(3717%) hue-rotate(349deg) brightness(86%) contrast(120%);margin:0 5px;"> is equivalent to killing the process and will stop the download immediately. Any leftover files will get deleted, the canceled video will still be available in the download queue.
## Add to Download Queue
The **Add to Download Queue** icon <img src="../tubearchivist/static/img/icon-add.svg?raw=true" alt="add icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);margin:0 5px;"> opens a text field to manually add videos to the download queue. You have a few options:
- Add a link to a YouTube video. For example *https://www.youtube.com/watch?v=2tdiKTSdE9Y*.
- Add a YouTube video ID. For example *2tdiKTSdE9Y*.
- Add a link to a YouTube video by providing the shortened URL, for example *https://youtu.be/2tdiKTSdE9Y*.
- Add a Channel ID or Channel URL to add every available video to the download queue. This will ignore the channel page size as described before and is meant for an initial download of the whole channel. You can still ignore selected videos before starting the download.
- Add a playlist ID or URL to add every available video in the list to the download queue, for example *https://www.youtube.com/playlist?list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha* or *PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha*. Note that when you add a link to a video in a playlist, Tube Archivist assumes you want to download only the specific video and not the whole playlist, for example *https://www.youtube.com/watch?v=CINVwWHlzTY&list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha* will only add one video *CINVwWHlzTY* to the queue.
- Add one link per line.
## The Download Queue
Below the three buttons you find the download queue. New items will get added at the bottom of the queue, the next video to download once you click on **Download queue** will be the first in the list.
Every video has two buttons:
- **Ignore**: This will remove that video from the download queue and this video will not get added again, even when you **Rescan Subscriptions**.
- **Download now**: This will give priority to this video. If the download process is already running, the prioritized video will get downloaded as soon as the current video is finished. If there is no download process running, this will start downloading this single video and stop after that.

29
Home.md

@ -1,10 +1,29 @@
# Tube Archivist Wiki
*Documentation of user functionality*
**WIP**: This is work in progress!
Welcome to the official Tube Archivist Wiki. This is an up-to-date documentation of user functionality.
Table of contents:
* [Main](Main): Tube Archivist landing page
* [Channels](Channels): Browse your channels, handle subscriptions
* [Downloads](Downloads): Scanning subscriptions, handle download queue
* [Settings](Settings): All the configuration options
* [Main](Main.md): Tube Archivist landing page
* [Channels](Channels.md): Browse your channels, handle subscriptions
* [Downloads](Downloads.md): Scanning subscriptions, handle download queue
* [Settings](Settings.md): All the configuration options
## Getting Started
1. [Subscribe](Channels.md#channels-overview) to some of your favourite YouTube channels.
2. [Scan](Downloads.md#rescan-subscriptions) subscriptions to add the latest videos to the download queue.
3. [Add](Downloads.md#add-to-download-queue) additional videos, channels or playlist - ignore the ones you don't want to download.
4. [Download](Downloads.md#download-queue) and let **Tube Archivist** do it's thing.
5. Sit back and enjoy your archived and indexed collection!
## General Navigation
* Clicking on the channel name or the channel icon brings you to the dedicated channel page to show videos from to that channel.
* Clicking on a video title brings you to the dedicated video page and shows additional details.
* Clicking on a video thumbnail opens the video player and starts streaming the selected video.
* Hover over the playing video to show additional control options.
Videos you *haven't* marked as **watched** show a empty checkbox, click on it to mark it as watched:
<img src="../tubearchivist/static/img/icon-unseen.svg?raw=true" alt="unseen icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);">
Videos you *have* marked as watched, will show as a filled checkbox:
<img src="../tubearchivist/static/img/icon-seen.svg?raw=true" alt="seen icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);">

9
Main.md

@ -1 +1,10 @@
# Tube Archivist Home Page Functionality
This is the landing page, when you first open **Tube Archivist**. You have a few options to sort and filter that view:
- With the **Sort Order** you can select how the "Recent Videos" are sorted:
- **Date Published**: Sorts the list by date when the video was published on YouTube, newest on top.
- **Date Downloaded**: Sorts the list based on when you have downloaded the video to your archive, newest on top.
- With **Hide Watched** you can filter out videos you have already marked as watched to only show unwatched videos.
- You can use those two options together to for example filter the list to *Hide Watched* videos **and** sort by date downloaded.
Additionally the search icon <img src="../tubearchivist/static/img/icon-search.svg?raw=true" alt="search icon" width="20px" style="filter:invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);margin:0 5px;"> opens a text field to search your collection.

65
Settings.md

@ -1 +1,64 @@
# Settings Page Functionality
# Settings Page
Accessible at `/settings/` of your **Tube Archivist**, this page holds all the configurations and additional functionality related to the database.
Click on **Update Settings** at the bottom of the form to apply your configurations.
## Color scheme
Switch between the easy on the eyes dark theme and the burning bright theme.
## Archive View
These default values will get applied on container restart.
- **Default Sort**: This defines the default sort order as described on the [Main](Main.md) page.
- **Default Hide Watched**: Hide watched videos by default.
- **Show Subscribed Channels Only**: This controls the filter on the *Channel Overview Page*.
- **Page Size**: Defines how many results get displayed on a given page. Same value goes for all archive views.
## Subscriptions
Settings related to the channel management.
- **Channel Page Size**: Defines how many pages will get analyzed by **Tube Archivist** each time you click on *Rescan Subscriptions*. The default page size used by yt-dlp is **50**, that's also the recommended value to set here. Any value higher will slow down the rescan process, for example if you set the value to 51, that means yt-dlp will have to go through 2 pages of results instead of 1 and by that doubling the time that process takes.
## Downloads
Settings related to the download process.
- **Download Limit**: Stop the download process after downloading the set quantity of videos.
- **Download Speed Limit**: Set your download speed limit in KB/s. This will pass the option `--limit-rate` to yt-dlp.
- **Sleep Interval**: Time in seconds to sleep between requests to YouTube. It's a good idea to set this to **3** seconds. Might be necessary to avoid throttling.
## Download Format
Additional settings passed to yt-dlp.
- **Format**: This controls which streams get downloaded and is equivalent to passing `--format` to yt-dlp. Use one of the recommended one or look at the documentation of [yt-dlp](https://github.com/yt-dlp/yt-dlp#format-selection). Please note: The option `--merge-output-format mp4` is automatically passed to yt-dlp to guarantee browser compatibility.
- **Embed Metadata**: This saves the available tags directly into the media file by passing `--embed-metadata` to yt-dlp.
# Actions
Additional database functionality.
## Manual Media Files Import
So far this depends on the video you are trying to import to be still available on YouTube to get the metadata. Add the files you like to import to the */cache/import* folder. Then start the process from the settings page *Manual Media Files Import*. Make sure to follow one of the two methods below.
### Method 1:
Add a matching *.json* file with the media file. Both files need to have the same base name, for example:
- For the media file: \<base-name>.mp4
- For the JSON file: \<base-name>.info.json
- Alternate JSON file: \<base-name>.json
**Tube Archivist** then looks for the 'id' key within the JSON file to identify the video.
### Method 2:
Detect the YouTube ID from filename, this accepts the default yt-dlp naming convention for file names like:
- \<base-name>[\<youtube-id>].mp4
- The YouTube ID in square brackets at the end of the filename is the crucial part.
### Some notes:
- This will **consume** the files you put into the import folder: Files will get converted to mp4 if needed (this might take a long time...) and moved to the archive, *.json* files will get deleted upon completion to avoid having duplicates on the next run.
- Maybe start with a subset of your files to import to make sure everything goes well...
- Follow the logs to monitor progress and errors: `docker-compose logs -f tubearchivist`.
## Backup Database
This will backup your metadata into a zip file. The file will get stored at *cache/backup* and will contain the necessary files to restore the Elasticsearch index formatted **nd-json** files plus a complete export of the index in a set of conventional **json** files.
BE AWARE: This will **not** backup any media files, just the metadata from the Elasticsearch.
## Restore From Backup
The restore functionality will expect the same zip file in *cache/backup* as created from the **Backup database** function. This will recreate the index from the snapshot. If there are multiple backup files in the folder, the newest one will take priority.
BE AWARE: This will **replace** your current index with the one from the backup file. This won't restore any media files.