nex
13e0280132
Some checks failed
Build and Publish drop-in-url-previews / build_and_publish (push) Failing after 12s
Signed-off-by: nex <git@nexy7574.co.uk> |
||
---|---|---|
.gitea/workflows | ||
src | ||
.gitignore | ||
Dockerfile | ||
LICENSE | ||
README.md | ||
requirements.txt |
Drop In URL previews server
aka DIP / Drop in previews
A simple python server that handles /_matrix/media/*/preview_url requests, for servers like Dendrite.
You may also want to replace your homeserver's URL preview generator with this one (in case this offers more features).
Features
DIP is complete with the following features:
- Full
OG/OpenGraph
tag support - Partial
Twitter
card support - Supports rendering previews for image files
- Proxying requests through a HTTP/HTTPS/SOCKS4/SOCKS5 proxy
- Custom user agent for requests
- Caching previews to prevent repeated lookups
- Built-in media duplication prevention
Installation
Just use docker.
Or you can run it yourself, but that's your choice.
pip install -r requirements.txt
PREVIEW_HOMESERVER=https://example.com python3 server.py
With a reverse proxy
You should use a reverse proxy to handle routing the request to this server. An example with caddy (ripped from my config file for matrix.nexy7574.co.uk)is below:
matrix.example.com {
# Add cache control, CSP, and CORP:
header /_matrix/media/*/thumbnail/*/* Cache-Control 'max-age=172800,stale-while-revalidate=86400'
header /_matrix/media/*/download/*/* Cache-Control 'max-age=172800,stale-while-revalidate=86400'
header /_matrix/media/* Content-Security-Policy "sandbox; default-src 'none'; script-src 'none'; plugin-types application/pdf; style-src 'unsafe-inline'; object-src 'self';"
header /_matrix/media/* Cross-Origin-Resource-Policy 'cross-origin'
# Send all requests for URL previews to the DIP server
rewrite /_matrix/media/*/preview_url /preview_url
reverse_proxy /preview_url localhost:2226
# Send all other requests to the homeserver
reverse_proxy localhost:8008
}
You can most likely figure out how to do this with your chosen reverse proxy server if you aren't using Caddy.
Example docker-compose.yml
version: "3"
services:
url_previews:
environment:
- "PREVIEW_HOMESERVER=https://matrix.nexy7574.co.uk"
- "FORWARDED_ALLOW_IPS=*"
ports:
- "2226:2226/tcp"
restart: "unless-stopped"
container_name: "dendrite-url-previews"
volumes:
- url_previews:/app/data/
build: . # there's no pre-built image
volumes:
url_previews:
Configuration
Environment Variable | Description | Example | Default |
---|---|---|---|
PREVIEW_HOMESERVER |
The homeserver to use for the previews. | https://matrix.nexy7574.co.uk |
The host name of the request URL. |
PREVIEW_HOST |
The host IP/Name to listen to. | 192.168.0.2 |
0.0.0.0 |
PREVIEW_PORT |
The port to listen to. | 8080 |
2226 |
PREVIEW_PROXY |
A HTTP/HTTPS/SOCKS4/SOCKS5 proxy to use for all network requests. | http://localhost:1080 |
null |
PREVIEW_USER_AGENT |
The user agent to use for all network requests. Must be one of google , bing , duckduckgo , firefox , chrome , twitter , facebook , honest (uses a unique user agent) |
firefox |
google |
PREVIEW_MAX_MEDIA_MB |
The maximum size of media to proxy in megabytes. Media larger than this downloaded from sites will not be re-uploaded to the homeserver's media repo, and as such cannot be used in the preview response. | 10 |
50 |
PREVIEW_DATABASE_URL |
The sqlite://, postgres://, or mysql:// URL to use for the database. | postgres://user:pass@localhost:5432/dip |
sqlite:///app/data/db.sqltie3 |
FORWARDED_ALLOW_IPS |
The list of reverse proxy IPs to trust. See Uvicorn docs | * | 127.0.0.1 |
LOG_LEVEL |
The log level to use. One of DEBUG , INFO , WARNING , ERROR , CRITICAL . |
INFO |
INFO |
LOG_DEBUG_TIDY |
When LOG_LEVEL is DEBUG , silences some really noisy loggers (like HTTP request loggers) to help you debug this program, not a dependency). |
true |
false |
How does it work?
Given that this server is meant to be a drop-in solution, there's a few differences to how you would expect something this deeply integrated with a homeserver to work.
Here is a basic flow for how a preview request goes:
- The user's client sends a request for a preview
- The server will take the provided access token, and checks that it is valid with the homeserver.
- If the token is invalid, M_INVALID_TOKEN is returned.
- The server then checks if it has a cached entry