# Drop In URL previews server

*aka DIP / Drop in previews*

A simple python server that handles /_matrix/media/*/preview_url requests, for servers like Dendrite.

You may also want to replace your homeserver's URL preview generator with this one (in case this offers more features).

## Features

DIP is complete with the following features:

* **Full** [`OG/OpenGraph`](https://ogp.me/) tag support
* Partial [`Twitter`](https://developer.twitter.com/en/docs/twitter-for-websites/cards/overview/markup) card support
* Supports rendering previews for image files
* Proxying requests through a HTTP/HTTPS/SOCKS4/SOCKS5 proxy
* Custom user agent for requests
* Caching previews to prevent repeated lookups
* Built-in media duplication prevention

## Installation
Just use docker.

Or you can run it yourself, but that's your choice.
```bash
pip install -r requirements.txt
PREVIEW_HOMESERVER=https://example.com python3 server.py
```

### With a reverse proxy
You should use a reverse proxy to handle routing the request to this server. An example with caddy
(ripped from my config file for matrix.nexy7574.co.uk)is below:

```caddy
matrix.example.com {
    # Add cache control, CSP, and CORP:
    header /_matrix/media/*/thumbnail/*/* Cache-Control 'max-age=172800,stale-while-revalidate=86400'
    header /_matrix/media/*/download/*/* Cache-Control 'max-age=172800,stale-while-revalidate=86400'
    header /_matrix/media/* Content-Security-Policy "sandbox; default-src 'none'; script-src 'none'; plugin-types application/pdf; style-src 'unsafe-inline'; object-src 'self';"
    header /_matrix/media/* Cross-Origin-Resource-Policy 'cross-origin'

    # Send all requests for URL previews to the DIP server
    rewrite /_matrix/media/*/preview_url /preview_url
    reverse_proxy /preview_url localhost:2226
    
    # Send all other requests to the homeserver
    reverse_proxy localhost:8008
}
```

You can most likely figure out how to do this with your chosen reverse proxy server if you aren't using Caddy.

### Example docker-compose.yml
```yaml
version: "3"
services:
  url_previews:
    environment:
      - "PREVIEW_HOMESERVER=https://matrix.nexy7574.co.uk"
      - "FORWARDED_ALLOW_IPS=*"
    ports:
      - "2226:2226/tcp"
    restart: "unless-stopped"
    container_name: "dendrite-url-previews"
    volumes:
      - url_previews:/app/data/
    image: git.i-am.nexus/nex/drop-in-url-previews:master

volumes:
  url_previews:
```

## Configuration

| Environment Variable   | Description                                                                                                                                                                                               | Example                                   | Default                           |
|------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|-----------------------------------|
| `PREVIEW_HOMESERVER`   | The homeserver to use for the previews.                                                                                                                                                                   | `https://matrix.nexy7574.co.uk`           | The host name of the request URL. |
| `PREVIEW_HOST`         | The host IP/Name to listen to.                                                                                                                                                                            | `192.168.0.2`                             | `0.0.0.0`                         |
| `PREVIEW_PORT`         | The port to listen to.                                                                                                                                                                                    | `8080`                                    | `2226`                            |
| `PREVIEW_PROXY`        | A HTTP/HTTPS/SOCKS4/SOCKS5 proxy to use for all network requests.                                                                                                                                         | `http://localhost:1080`                   | null                              |
| `PREVIEW_USER_AGENT`   | The user agent to use for all network requests. Must be one of `google`, `bing`, `duckduckgo`, `firefox`, `chrome`, `twitter`, `facebook`, `honest` (uses a unique user agent)                            | `firefox`                                 | `google`                          |
| `PREVIEW_MAX_MEDIA_MB` | The maximum size of media to proxy in megabytes. Media larger than this downloaded from sites will not be re-uploaded to the homeserver's media repo, and as such cannot be used in the preview response. | `10`                                      | `50`                              |
| `PREVIEW_DATABASE_URL` | The sqlite://, postgres://, or mysql:// URL to use for the database.                                                                                                                                      | `postgres://user:pass@localhost:5432/dip` | `sqlite:///app/data/db.sqltie3`   |
| `FORWARDED_ALLOW_IPS`  | The list of reverse proxy IPs to trust. See [Uvicorn docs](https://www.uvicorn.org/settings/#http)                                                                                                        | *                                         | `127.0.0.1`                       |
| `LOG_LEVEL`            | The log level to use. One of `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`.                                                                                                                             | `INFO`                                    | `INFO`                            |
| `LOG_DEBUG_TIDY`       | When `LOG_LEVEL` is `DEBUG`, silences some really noisy loggers (like HTTP request loggers) to help you debug this program, not a dependency).                                                            | `true`                                    | `false`                           |  

## How does it work?

Given that this server is meant to be a drop-in solution, there's a few differences to how you would expect
something this deeply integrated with a homeserver to work.

Here is a basic flow for how a preview request goes:

1. The user's client sends a request for a preview
2. The server will take the provided access token, and checks that it is valid with the homeserver.
3. If the token is invalid, M_INVALID_TOKEN is returned.
4. The server then checks if it has a cached entry