# Audiences

Audience detection has chosen the response body since version 0.9.0. One URL has three representations:

- **Browser** gets rendered HTML.
- **AI crawler** gets Markdown when `Accept` includes `text/markdown`, or when the User-Agent matches `[crawlers].bots`.
- **AI agent** gets JSON when `Accept` includes `application/json` and does not include `text/html`. JSON includes the page body and, when set, the front-matter `since` version.
- **Blocked** matches `[crawlers].blocked` first and does not receive the real page.

```toml
[crawlers]
intent = "allow-all"
disallow = ["/private/**"]

[[crawlers.bots]]
name = "GPTBot"

[[crawlers.blocked]]
name = "BadBot"
```

`intent` is `allow-all`, `block-ai`, or `block-all`. It controls `robots.txt`. That file is not `llms.txt`. `llms.txt` is a map of pages for language models. `[llms]` customizes its title and description. When `[llms]` is absent, the file still uses the site title.

```toml
[llms]
title = "Benson"
description = "Documentation for the Benson site engine."
```

`sitemap.xml` lists public pages. Drafts, hidden pages, and private pages are left out. `[crawlers].disallow` is the robots disallow list.

`[headers.security]` sets HSTS, CSP, `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy`, and `Permissions-Policy`. Omitted fields are not sent.