nex
1d8797889d
All checks were successful
Build and Publish drop-in-url-previews / build_and_publish (push) Successful in 53s
98 lines
6.5 KiB
Markdown
98 lines
6.5 KiB
Markdown
# Drop In URL previews server
|
|
|
|
*aka DIP / Drop in previews*
|
|
|
|
A simple python server that handles /_matrix/media/*/preview_url requests, for servers like Dendrite.
|
|
|
|
You may also want to replace your homeserver's URL preview generator with this one (in case this offers more features).
|
|
|
|
## Features
|
|
|
|
DIP is complete with the following features:
|
|
|
|
* **Full** [`OG/OpenGraph`](https://ogp.me/) tag support
|
|
* Partial [`Twitter`](https://developer.twitter.com/en/docs/twitter-for-websites/cards/overview/markup) card support
|
|
* Supports rendering previews for image files
|
|
* Proxying requests through a HTTP/HTTPS/SOCKS4/SOCKS5 proxy
|
|
* Custom user agent for requests
|
|
* Caching previews to prevent repeated lookups
|
|
* Built-in media duplication prevention
|
|
|
|
## Installation
|
|
Just use docker.
|
|
|
|
Or you can run it yourself, but that's your choice.
|
|
```bash
|
|
pip install -r requirements.txt
|
|
PREVIEW_HOMESERVER=https://example.com python3 server.py
|
|
```
|
|
|
|
### With a reverse proxy
|
|
You should use a reverse proxy to handle routing the request to this server. An example with caddy
|
|
(ripped from my config file for matrix.nexy7574.co.uk)is below:
|
|
|
|
```caddy
|
|
matrix.example.com {
|
|
# Add cache control, CSP, and CORP:
|
|
header /_matrix/media/*/thumbnail/*/* Cache-Control 'max-age=172800,stale-while-revalidate=86400'
|
|
header /_matrix/media/*/download/*/* Cache-Control 'max-age=172800,stale-while-revalidate=86400'
|
|
header /_matrix/media/* Content-Security-Policy "sandbox; default-src 'none'; script-src 'none'; plugin-types application/pdf; style-src 'unsafe-inline'; object-src 'self';"
|
|
header /_matrix/media/* Cross-Origin-Resource-Policy 'cross-origin'
|
|
|
|
# Send all requests for URL previews to the DIP server
|
|
rewrite /_matrix/media/*/preview_url /preview_url
|
|
reverse_proxy /preview_url localhost:2226
|
|
|
|
# Send all other requests to the homeserver
|
|
reverse_proxy localhost:8008
|
|
}
|
|
```
|
|
|
|
You can most likely figure out how to do this with your chosen reverse proxy server if you aren't using Caddy.
|
|
|
|
### Example docker-compose.yml
|
|
```yaml
|
|
version: "3"
|
|
services:
|
|
url_previews:
|
|
environment:
|
|
- "PREVIEW_HOMESERVER=https://matrix.nexy7574.co.uk"
|
|
- "FORWARDED_ALLOW_IPS=*"
|
|
ports:
|
|
- "2226:2226/tcp"
|
|
restart: "unless-stopped"
|
|
container_name: "dendrite-url-previews"
|
|
volumes:
|
|
- url_previews:/app/data/
|
|
image: git.i-am.nexus/nex/drop-in-url-previews:master
|
|
|
|
volumes:
|
|
url_previews:
|
|
```
|
|
|
|
## Configuration
|
|
|
|
| Environment Variable | Description | Example | Default |
|
|
|------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|-----------------------------------|
|
|
| `PREVIEW_HOMESERVER` | The homeserver to use for the previews. | `https://matrix.nexy7574.co.uk` | The host name of the request URL. |
|
|
| `PREVIEW_HOST` | The host IP/Name to listen to. | `192.168.0.2` | `0.0.0.0` |
|
|
| `PREVIEW_PORT` | The port to listen to. | `8080` | `2226` |
|
|
| `PREVIEW_PROXY` | A HTTP/HTTPS/SOCKS4/SOCKS5 proxy to use for all network requests. | `http://localhost:1080` | null |
|
|
| `PREVIEW_USER_AGENT` | The user agent to use for all network requests. Must be one of `google`, `bing`, `duckduckgo`, `firefox`, `chrome`, `twitter`, `facebook`, `honest` (uses a unique user agent) | `firefox` | `google` |
|
|
| `PREVIEW_MAX_MEDIA_MB` | The maximum size of media to proxy in megabytes. Media larger than this downloaded from sites will not be re-uploaded to the homeserver's media repo, and as such cannot be used in the preview response. | `10` | `50` |
|
|
| `PREVIEW_DATABASE_URL` | The sqlite://, postgres://, or mysql:// URL to use for the database. | `postgres://user:pass@localhost:5432/dip` | `sqlite:///app/data/db.sqltie3` |
|
|
| `FORWARDED_ALLOW_IPS` | The list of reverse proxy IPs to trust. See [Uvicorn docs](https://www.uvicorn.org/settings/#http) | * | `127.0.0.1` |
|
|
| `LOG_LEVEL` | The log level to use. One of `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`. | `INFO` | `INFO` |
|
|
| `LOG_DEBUG_TIDY` | When `LOG_LEVEL` is `DEBUG`, silences some really noisy loggers (like HTTP request loggers) to help you debug this program, not a dependency). | `true` | `false` |
|
|
|
|
## How does it work?
|
|
|
|
Given that this server is meant to be a drop-in solution, there's a few differences to how you would expect
|
|
something this deeply integrated with a homeserver to work.
|
|
|
|
Here is a basic flow for how a preview request goes:
|
|
|
|
1. The user's client sends a request for a preview
|
|
2. The server will take the provided access token, and checks that it is valid with the homeserver.
|
|
3. If the token is invalid, M_INVALID_TOKEN is returned.
|
|
4. The server then checks if it has a cached entry
|