Drop-in server for Matrix URL previews, for servers like Dendrite that don't support them natively.
Go to file
Nexus a24a020fc7
All checks were successful
Build and Publish drop-in-url-previews / build_and_publish (push) Successful in 31s
add docker container repo push
2024-04-22 16:45:41 +01:00
.gitea/workflows add docker container repo push 2024-04-22 16:45:41 +01:00
src Fix stupid ass internal error with peewee 2024-03-14 19:00:54 +00:00
.gitignore Initial commit 2024-02-09 22:37:15 +00:00
Dockerfile 0.2.0a1 2024-02-25 22:09:51 +00:00
LICENSE Initial commit 2024-02-09 22:37:15 +00:00
README.md Properly implement authentication 2024-02-28 18:09:17 +00:00
requirements.txt 0.2.0a1 2024-02-25 22:09:51 +00:00

Drop In URL previews server

aka DIP / Drop in previews

A simple python server that handles /_matrix/media/*/preview_url requests, for servers like Dendrite.

You may also want to replace your homeserver's URL preview generator with this one (in case this offers more features).

Features

DIP is complete with the following features:

  • Full OG/OpenGraph tag support
  • Partial Twitter card support
  • Supports rendering previews for image files
  • Proxying requests through a HTTP/HTTPS/SOCKS4/SOCKS5 proxy
  • Custom user agent for requests
  • Caching previews to prevent repeated lookups
  • Built-in media duplication prevention

Installation

Just use docker.

Or you can run it yourself, but that's your choice.

pip install -r requirements.txt
PREVIEW_HOMESERVER=https://example.com python3 server.py

With a reverse proxy

You should use a reverse proxy to handle routing the request to this server. An example with caddy (ripped from my config file for matrix.nexy7574.co.uk)is below:

matrix.example.com {
    # Add cache control, CSP, and CORP:
    header /_matrix/media/*/thumbnail/*/* Cache-Control 'max-age=172800,stale-while-revalidate=86400'
    header /_matrix/media/*/download/*/* Cache-Control 'max-age=172800,stale-while-revalidate=86400'
    header /_matrix/media/* Content-Security-Policy "sandbox; default-src 'none'; script-src 'none'; plugin-types application/pdf; style-src 'unsafe-inline'; object-src 'self';"
    header /_matrix/media/* Cross-Origin-Resource-Policy 'cross-origin'

    # Send all requests for URL previews to the DIP server
    rewrite /_matrix/media/*/preview_url /preview_url
    reverse_proxy /preview_url localhost:2226
    
    # Send all other requests to the homeserver
    reverse_proxy localhost:8008
}

You can most likely figure out how to do this with your chosen reverse proxy server if you aren't using Caddy.

Example docker-compose.yml

version: "3"
services:
  url_previews:
    environment:
      - "PREVIEW_HOMESERVER=https://matrix.nexy7574.co.uk"
      - "FORWARDED_ALLOW_IPS=*"
    ports:
      - "2226:2226/tcp"
    restart: "unless-stopped"
    container_name: "dendrite-url-previews"
    volumes:
      - url_previews:/app/data/
    build: .  # there's no pre-built image

volumes:
  url_previews:

Configuration

Environment Variable Description Example Default
PREVIEW_HOMESERVER The homeserver to use for the previews. https://matrix.nexy7574.co.uk The host name of the request URL.
PREVIEW_HOST The host IP/Name to listen to. 192.168.0.2 0.0.0.0
PREVIEW_PORT The port to listen to. 8080 2226
PREVIEW_PROXY A HTTP/HTTPS/SOCKS4/SOCKS5 proxy to use for all network requests. http://localhost:1080 null
PREVIEW_USER_AGENT The user agent to use for all network requests. Must be one of google, bing, duckduckgo, firefox, chrome, twitter, facebook, honest (uses a unique user agent) firefox google
PREVIEW_MAX_MEDIA_MB The maximum size of media to proxy in megabytes. Media larger than this downloaded from sites will not be re-uploaded to the homeserver's media repo, and as such cannot be used in the preview response. 10 50
PREVIEW_DATABASE_URL The sqlite://, postgres://, or mysql:// URL to use for the database. postgres://user:pass@localhost:5432/dip sqlite:///app/data/db.sqltie3
FORWARDED_ALLOW_IPS The list of reverse proxy IPs to trust. See Uvicorn docs * 127.0.0.1
LOG_LEVEL The log level to use. One of DEBUG, INFO, WARNING, ERROR, CRITICAL. INFO INFO
LOG_DEBUG_TIDY When LOG_LEVEL is DEBUG, silences some really noisy loggers (like HTTP request loggers) to help you debug this program, not a dependency). true false

How does it work?

Given that this server is meant to be a drop-in solution, there's a few differences to how you would expect something this deeply integrated with a homeserver to work.

Here is a basic flow for how a preview request goes:

  1. The user's client sends a request for a preview
  2. The server will take the provided access token, and checks that it is valid with the homeserver.
  3. If the token is invalid, M_INVALID_TOKEN is returned.
  4. The server then checks if it has a cached entry