tubearchivist
/
docs
mirror of https://github.com/tubearchivist/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #33 from lamusmaser/batch-editing 

Fix formatting issues.
This commit is contained in:
lamusmaser 2024-04-19 19:58:33 -06:00 committed by GitHub
parent 49dfca8c03 b17bdbc63e
commit 74b3849e17
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 76 additions and 68 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
mkdocs/docs/advanced.md
View File

@ -147,16 +147,16 @@ This is obviously particularly likely to create problems. Also note that the `--
## Erase errors from download queue
Sometimes the download queue might have some videos that have errored due to rate limits or other issues during the download process. Videos that have errors won't be retried in a future download queue re-run unless you individually click "Download now" for each individual video or until the container is fully restarted. In order to bulk clear the errors from the download queue without restarting the container, you need to execute the following command from within the TA container:
???+ bug "Erase Errors"
```bash
curl -X POST "$ES_URL/ta_download/_update_by_query?pretty" -u elastic:$ELASTIC_PASSWORD -H 'Content-Type: application/json' -d'
{
"script": {
"source": "ctx._source.message = null",
"lang": "painless"
},
"query": {
"match_all": {}
}
}
'
```
```bash
curl -X POST "$ES_URL/ta_download/_update_by_query?pretty" -u elastic:$ELASTIC_PASSWORD -H 'Content-Type: application/json' -d'
{
"script": {
"source": "ctx._source.message = null",
"lang": "painless"
},
"query": {
"match_all": {}
}
}
'
```

1
mkdocs/docs/configuration/forward-auth.md
View File

@ -7,6 +7,7 @@ Check the documentation of your auth proxy and your reverse proxy on how to corr
Note that this automatically creates new users in the database if they do not already exist.
| Environment Variable | Example | Description |
| :------------------- | :------ | :---------- |
| `TA_ENABLE_AUTH_PROXY` | `true` | Set to anything besides empty string to use forward proxy authentication. |
| `TA_AUTH_PROXY_USERNAME_HEADER` | `HTTP_REMOTE_USER` | The name of the request header that the auth proxy passes to the proxied application (**Tube Archivist** in this case), so that the application can identify the user. Check the documentation of your auth proxy to get this information.
??? note

1
mkdocs/docs/faq.md
View File

@ -40,6 +40,7 @@ Docker is the only supported installation method. If you don't have any experien
## How do I install this on Windows?
The only supported methodology is through Docker on Windows. We don't currently have installation instructions provided, however it has been reported that some users have been successful in getting it to work with modification of the base `docker-compose.yml`. Try following the [Docker Compose](installation/docker-compose.md) instructions.
<!-- This question uses &lt; for `<` and &gt; for `>` because Makedocs doesn't handle the characters properly. -->
## Where is the .exe/.rpm/.pkg/.msi/&lt;insert installer here&gt;?
As noted [here](faq.md#how-do-i-install-this-natively), this project is largely focused on container-based applications. We are not looking to support cross-platform for a small user base. It is not on our roadmap to support any additional installation types or be included as part of additional installation package manager solutions. However, this is an open source project and we won't stop the community from providing those packages. If you do decide to do so, please contribute to our installation instructions and point to your created resources. Pay attention to any licensing requirements for dependencies.

90
mkdocs/docs/installation/synology.md
View File

@ -2,10 +2,10 @@
These are beginner's guides/installation instructions for additional platforms generously provided by users of these platforms. When in doubt, verify the details with the [project README](https://github.com/tubearchivist/tubearchivist#installing). If you see any issues here while using these instructions, please contribute.
There are several different methods to install **Tube Archivist** on Synology platforms. This will focus on the available `docker` package manager implementation for Synology 7.1 and prior.<!-| and `docker-compose` implementations. -->
There are several different methods to install **Tube Archivist** on Synology platforms. This will focus on the available `docker` package manager implementation for Synology 7.1 and prior.<!-- and `docker-compose` implementations. -->
!!! note
Synology Package Manager for 7.2 and later should be able to follow the instructions below or can create a project using the [docker compose installation instructions.](docker-compose.md)
Synology Package Manager for 7.2 and later should be able to follow the instructions below or can create a project using the [docker compose installation instructions.](docker-compose.md)
### Prepare Directories/Folders
@ -20,7 +20,7 @@ Before we setup **Tube Archivist**, we need to setup the directories/folders. Yo
> !!! note
By default, this will be where all persistent data is stored. Change the folders as best meets your requirements.
6. Select the appropriate options from the remaining checkbox configurations.
![Synology | Create Docker Folder](../assets/Synology_0.2.0_Docker-Folder-Create.png)
![Synology - Create Docker Folder](../assets/Synology_0.2.0_Docker-Folder-Create.png)
7. Click the **Next** button.
8. If you are going to **Encrypt** your folder, check the appropriate box and provide the Encryption Key and its confirmation.
9. Click the **Next** button.
@ -61,39 +61,39 @@ Before we setup **Tube Archivist**, we need to setup the directories/folders. Yo
5. **Name** the folder "media".
#### 7. Confirm Folder Structure
Once all of the folders have been created, it should have a folder structure within `Docker\TubeArchivist` that includes "`cache`", "`es`", "`media`", and "`redis`" folders.
![Synology | Docker Folder Structure](../assets/Synology_0.2.0_Docker-Folder-Structure.png)
![Synology - Docker Folder Structure](../assets/Synology_0.2.0_Docker-Folder-Structure.png)
#### 8. Change Permissions | CLI Required
!!! failure
#### 8. Change Permissions - CLI Required
!!! failure CLI Required
If you do not have SSH access enabled for CLI, [enable it](https://kb.synology.com/en-sg/DSM/tutorial/How_to_login_to_DSM_with_root_permission_via_SSH_Telnet) before continuing.
1. Open an SSH connection to the Synology. Login as your primary `Admin` user, or the user that was enabled for SSH access.
2. Elevate your access to `root`. Steps are provided [here](https://kb.synology.com/en-sg/DSM/tutorial/How_to_login_to_DSM_with_root_permission_via_SSH_Telnet).
3. Change directories to the **Volume** where the "Docker" folder resides.
!!! example
`cd /volume1`
!!! example
`cd /volume1`
4. Change directories to the "Docker" folder.
!!! example
`cd Docker`
!!! example
`cd Docker`
5. Change directories to the "TubeArchivist" folder.
!!! example
`cd TubeArchivist`
!!! example
`cd TubeArchivist`
6. Change the owner of the "redis" folder. *If correct, this does not have an output.*
!!! example
`chown 999:100 redis`
!!! example
`chown 999:100 redis`
7. Change the owner of the "es" folder. *If correct, this does not have an output.*
!!! example
`chown 1000:0 es`
!!! example
`chown 1000:0 es`
8. Confirm that the folders have the correct permissions.
!!! example
`ls -hl`
![Synology | Docker Folder Permissions Command](../assets/Synology_0.3.6_Docker-Folder-Permissions-Commands.png)
!!! example
`ls -hl`
![Synology - Docker Folder Permissions Command](../assets/Synology_0.3.6_Docker-Folder-Permissions-Commands.png)
9. Logout from root.
!!! example
`logout`
!!! example
`logout`
10. Disconnect from the SSH connection.
!!! example
`exit`
!!! example
`exit`
### Synology Docker Setup
@ -106,26 +106,26 @@ Once all of the folders have been created, it should have a folder structure wit
3. Search for `Docker`.
4. Click `Install`.
![Synology | Install Docker Utility](../assets/Synology_0.2.0_Docker-Install.png)
![Synology - Install Docker Utility](../assets/Synology_0.2.0_Docker-Install.png)
#### 2. Download the Docker images
2. After `Docker` is installed, open the `Docker` utility.
3. Go to the `Registry` tab.
4. Search for the following `images` and download them. Follow the recommended versions for each of the images.
| `redis/redis-stack-server`
![Synology | Redis Image Search](../assets/Synology_0.3.6_Docker-Redis-Search.png)
| `bbilly1/tubearchivist-es`
![Synology | ElasticSearch Image Search](../assets/Synology_0.2.0_Docker-ES-Search.png)
| `bbilly1/tubearchivist`
![Synology | Tube Archivist Image Search](../assets/Synology_0.2.0_Docker-TA-Search.png)
`redis/redis-stack-server`
![Synology - Redis Image Search](../assets/Synology_0.3.6_Docker-Redis-Search.png)
`bbilly1/tubearchivist-es`
![Synology - ElasticSearch Image Search](../assets/Synology_0.2.0_Docker-ES-Search.png)
`bbilly1/tubearchivist`
![Synology - Tube Archivist Image Search](../assets/Synology_0.2.0_Docker-TA-Search.png)
> !!! info
Upgrades in Synology require use of the `latest` tag.
5. Go to the `Image` tab. From here, create a container based on each image with the associated configurations below.
#### 3. Configure ElasticSearch
| ElasticSearch
- ElasticSearch
1. Select the associated image.
2. Click the **Launch** button in the top.
3. Edit the **Container Name** to be "tubearchivist-es".
@ -138,6 +138,7 @@ Once all of the folders have been created, it should have a folder structure wit
10. The **Links** tab does not require configuration for this container.
11. In the **Environment** tab, add in the following ElasticSearch specific environment variables that may apply.
| Environment Variable | Setting |
| :------------------- | :------ |
| `discovery.type | single-node` |
| `ES_JAVA_OPTS | -Xms512m -Xmx512m` |
| `UID | 1000` |
@ -146,15 +147,15 @@ Once all of the folders have been created, it should have a folder structure wit
| `ELASTIC_PASSWORD | verysecret` |
| `path.repo | /usr/share/elasticsearch/data/snapshot` |
> !!! danger "BE AWARE"
| Do not use the default password, as it is very insecure.
| Activating snapshots for backups should only be done *after* setting the `path.repo` setting.
![Synology | ElasticSearch Environment Configurations](../assets/Synology_0.2.0_Docker-ES-Env-Conf.png)
- Do not use the default password, as it is very insecure.
- Activating snapshots for backups should only be done *after* setting the `path.repo` setting.
![Synology - ElasticSearch Environment Configurations](../assets/Synology_0.2.0_Docker-ES-Env-Conf.png)
12. Click on the **Apply** button.
13. Back on the **Create Container** screen, click the **Next** button.
14. Review the settings to confirm, then click the **Apply** button.
#### 4. Configure Redis
| Redis
- Redis
1. Select the associated image.
2. Click the **Launch** button in the top.
3. Edit the **Container Name** to be "tubearchivist-redis".
@ -170,7 +171,7 @@ Once all of the folders have been created, it should have a folder structure wit
13. Review the settings to confirm, then click the **Apply** button.
#### 5. Configure Tube Archivist
| Tube Archivist
- Tube Archivist
1. Select the associated image.
2. Click the **Launch** button in the top.
3. Edit the **Container Name** to be "tubearchivist".
@ -184,6 +185,7 @@ Once all of the folders have been created, it should have a folder structure wit
11. In the **Links** tab, select the `tubearchivist-redis` container from the **Container Name** dropdown and provide it the same alias, "tubearchivist-redis".
12. In the **Environment** tab, add in the following **Tube Archivist** specific environment variables that may apply. **Change the variables as is appropriate to your use case. Follow the [README section](https://github.com/tubearchivist/tubearchivist#installing) for details on what to set each variable.**
| Environment Variable | Setting |
| :------------------- | :------ |
| `TA_HOST | synology.local` |
| `ES_URL | http://tubearchivist-es:9200` |
| `REDIS_HOST | tubearchivist-redis` |
@ -194,9 +196,9 @@ Once all of the folders have been created, it should have a folder structure wit
| `ELASTIC_PASSWORD | verysecret` |
| `TZ | America/New_York` |
> !!! danger "BE AWARE"
| Do not use the default password as it is very insecure.
| Ensure that ELASTIC_PASSWORD matches the password used on the `tubearchivist-es` container.
![Synology | Tube Archivist Environment Configurations](../assets/Synology_0.2.0_Docker-TA-Env-Conf.png)
- Do not use the default password as it is very insecure.
- Ensure that ELASTIC_PASSWORD matches the password used on the `tubearchivist-es` container.
![Synology - Tube Archivist Environment Configurations](../assets/Synology_0.2.0_Docker-TA-Env-Conf.png)
13. Click on the **Apply** button.
14. Back on the **Create Container** screen, click the **Next** button.
@ -209,17 +211,17 @@ Once all of the folders have been created, it should have a folder structure wit
> !!! note
Synology Docker presents the logs in a paginated format, showing in historical order (oldest first). If you are not seeing the logs update, check if there are additional pages.
8. After it has started, go to the location provided in the `TA_HOST`. This should give you the standard **Tube Archivist** login screen.
<!-|
<!--
### Docker-Compose Setup -->
<!-| This section is a Work In Progress -->
<!-- This section is a Work In Progress -->
**From there, you should be able to start up your containers and you're good to go!**
### Synology Docker Upgrade
When a new version of the image is available, you can use the following steps to more easily upgrade your previous instance.
!!! failure
If you did not use the `latest` tag, you may have some variances in your upgrade steps. Those are detailed below these instructions.
!!! failure `latest` Tag Required
If you did not use the `latest` tag, you may have some variances in your upgrade steps. Those are detailed below these instructions.
1. Go to the Registry Tab and download the newest instance of the `:latest` tag, as seen in the Installation Instructions earlier.
2. Go to Image Tab and confirm that you have the newer version available.
3. Stop the running `tubearchivist` container.
@ -235,7 +237,7 @@ When a new version of the image is available, you can use the following steps to
4. After the image is now running and the upgrade of the backend files occurs, shut down the new container. Rename or delete the old container. Rename the new container to have the intended name.
!!! info
Links are incredibly important if you upgrade or change the ES or Redis container images. You will either need to remove the links, create the new containers, then re-add the links or rebuild all of the images with the same instructions as Installation, starting at Step 3.
Links are incredibly important if you upgrade or change the ES or Redis container images. You will either need to remove the links, create the new containers, then re-add the links or rebuild all of the images with the same instructions as Installation, starting at Step 3.
### Support

26
mkdocs/docs/urls.md
View File

@ -5,26 +5,28 @@ description: How URLs from YouTube get parsed
# URLs
This document describes how **Tube Archivist** identifies and treats links from YouTube.
!!! note
!!! info
Application logic of **Tube Archivist** is tied only to the IDs, not the names.
## Video
A video ID is **11** characters long, e.g. `2tdiKTSdE9Y`.
Urls can have several forms:
- Watch URL: Regular URLs you will see while browsing YouTube, with the path */watch* and a *v* parameter, e.g. `https://www.youtube.com/watch?v=2tdiKTSdE9Y`
- Share URL: Link you will get when you click on *share* on a video, e.g. `https://youtu.be/2tdiKTSdE9Y`
- Shorts URL: e.g. `https://www.youtube.com/shorts/U80grnZJm_8`
| URL Type | Example | Description |
| :------- | :------ | :---------- |
| Watch URL | `https://www.youtube.com/watch?v=2tdiKTSdE9Y` | Regular URLs you will see while browsing YouTube, with the path */watch* and a *v* parameter |
| Share URL | `https://youtu.be/2tdiKTSdE9Y` | Link you will get when you click on *share* on a video |
| Shorts URL | `https://www.youtube.com/shorts/U80grnZJm_8` | Video links for YouTube shorts |
## Channel
A channel ID is **24** characters long, e.g. `UCBa659QWEk1AI4Tg--mrJ2A`.
Channel URLs can have these forms, all will get translated to the ID:
- ID URL: With a *channel* path, e.g. `https://www.youtube.com/channel/UCBa659QWEk1AI4Tg--mrJ2A`
- Channel Handle: Starting with a `@` this handle is personal and unique, e.g. `@TomScottGo`
- Alias URL: Based off the channel handle, e.g. `https://www.youtube.com/@TomScottGo`
| URL Type | Example | Description |
| :------- | :------ | :---------- |
| ID URL | `https://www.youtube.com/channel/UCBa659QWEk1AI4Tg--mrJ2A` | With a *channel* path |
| Channel Handle | `@TomScottGo` | Starting with a `@` this handle is personal and unique |
| Alias URL | `https://www.youtube.com/@TomScottGo` | Based off the channel handle |
### Channel subpages
**Tube Archivist** can differentiate between the primary subpages:
@ -37,7 +39,9 @@ Channel URLs can have these forms, all will get translated to the ID:
## Playlist
A playlist ID can be `34`, `26` or `18` characters long, e.g. `PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha`
- Playlist URLs start with a *playlist* path and has a *list* parameter, e.g. `https://www.youtube.com/playlist?list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha`
| URL Type | Example | Description |
| :------- | :------ | :---------- |
| Playlist URL | `https://www.youtube.com/playlist?list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha` | Playlist URLs start with a *playlist* path and have a *list* parameter |
### Playlist vs Video URLs
While browsing YouTube videos in Playlists, you might encounter URLs looking like that: `https://www.youtube.com/watch?v=QPZ0pIK_wsc&list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha`. As established above, based on the */watch* path and the *v* parameter, **Tube Archivist** will treat this as a video with the ID `QPZ0pIK_wsc` and **not** as a playlist. If you mean the playlist, you can easily grab the correct ID from the URL, e.g. `PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha`.
While browsing YouTube videos in Playlists, you might encounter URLs looking like this: `https://www.youtube.com/watch?v=QPZ0pIK_wsc&list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha`. As established above, based on the */watch* path and the *v* parameter, **Tube Archivist** will treat this as a video with the ID `QPZ0pIK_wsc` and **not** as a playlist. If you mean the playlist, you can easily grab the correct ID from the URL, e.g. `PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha`.