# indexnowkit for PHP Tell search engines (Bing, Yandex, Naver, Seznam, Yep, and the other participants of the IndexNow registry) about new, changed and deleted pages the moment a model is committed. One attribute on the model, one key, done. Google does not participate; IndexNow is a notification, not indexing. | Package | Framework | Install | |---|---|---| | [Symfony bundle](symfony-bundle/index.md) | Symfony 6.4 \| 7 \| 8 + Doctrine | `composer require indexnowkit/symfony-bundle indexnowkit/doctrine` | | [Laravel](laravel/index.md) | Laravel 12 \| 13 | `composer require indexnowkit/laravel` | | [Yii2](yii2/index.md) | Yii 2.0.45+ | `composer require indexnowkit/yii2` | | [Yii3](yii3/index.md) | yiisoft/active-record 1.x, yiisoft/db 2.x | `composer require indexnowkit/yii3` | | [Doctrine](doctrine/index.md) | Doctrine ORM without Symfony | `composer require indexnowkit/doctrine` | | [CLI](cli/index.md) | any site: cron on Bitrix, WordPress, MODX, OpenCart; static sites on deploy (PHAR, Docker image, GitHub Action) | `composer global require indexnowkit/cli` | | [Core](core/index.md) | plain PHP, any PSR-18 client | `composer require indexnowkit/core` | | [Sitemap](sitemap/index.md) | the `sitemap` command of every adapter | `composer require indexnowkit/sitemap` | | [Verify](verify/index.md) | a pre-flight GET before every submission, `check --sample` | `composer require indexnowkit/verify` | | [History](history/index.md) | the `history` and `status` commands, PSR-16 and PDO stores | `composer require indexnowkit/history` | Start with the page of your framework, then the [attribute reference](core/attribute-reference.md), the [configuration](core/configuration.md) (one concept, four keys: Symfony, Laravel, Yii2, Yii3) and the [operations guide](core/operations.md) with its production checklist. Machine-readable: [llms.txt](llms.txt), [llms-full.txt](llms-full.txt). Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php). # Symfony IndexNow bundle — `indexnowkit/symfony-bundle` Tell search engines about new, changed and deleted pages the moment a Doctrine entity is committed. One attribute on the entity, one env variable, done. [](https://packagist.org/packages/indexnowkit/symfony-bundle) [](https://packagist.org/packages/indexnowkit/symfony-bundle) [](https://github.com/indexnowkit/php/actions) [](https://github.com/indexnowkit/spec)    [](https://github.com/indexnowkit/php/blob/main/packages/symfony-bundle/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/symfony-bundle/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Who gets notified **Yandex, Bing (and DuckDuckGo via Bing), Naver, Seznam, Yep, Internet Archive, Amazon** — every engine in the [IndexNow](https://www.indexnow.org) [registry](https://www.indexnow.org/searchengines.json). One request to the shared endpoint reaches all of them; name engines explicitly only to reach a single one. **Google: no.** Google does not support IndexNow, its sitemap ping endpoint is gone (404) and the Indexing API is restricted to `JobPosting` / `BroadcastEvent`. This bundle will not pretend otherwise. **Notification, not indexing.** IndexNow tells an engine that a URL changed; whether and when the page is crawled and indexed is the engine's decision. See the result in Bing Webmaster Tools (IndexNow Insights) and Yandex.Webmaster (Indexing → Reindex pages); a useful metric is the share of submitted URLs in the index after a few days. Deleted pages: answer 410 (gone for good) or 404 (temporarily); for a move answer 301 and submit both URLs; a soft-404 or a redirect to the home page does harm. Bing's URL Submission API and Google's Indexing API are different protocols and not covered here. ## Why this over X Most IndexNow packages are a thin HTTP client: you collect the URLs, you call it, you read the answer. This family does the part that goes wrong in practice: - **Declared on the model** (`#[IndexNow]`) and submitted from the ORM hooks — no controller code to forget. - **After the commit**, not on flush: a rolled-back transaction announces nothing. - **Debounce** (10 minutes per URL, shared through your cache), **batches** of up to 10 000 URLs, one key per host from env. - **Answers handled**: 202 (key pending), 422, 429 with `Retry-After` back-off and a retry through your queue, 403 escalation. - **`check` before the first submission** says what is wrong (key file, engines, queue, cache, environment); `explain` says why a URL was or was not sent. - **One core** under the Symfony, Laravel, Yii2, Yii3 and Doctrine adapters with a shared conformance suite: the same behaviour everywhere, documented once. ## Install ```bash composer require indexnowkit/symfony-bundle composer require symfony/http-client nyholm/psr7 # any PSR-18 client works; this pair is auto-configured composer require indexnowkit/doctrine # for automatic submission when entities change composer require indexnowkit/sitemap # optional: the indexnow:sitemap command bin/console indexnow:key:generate --write-env # adds INDEXNOW_KEY to .env.local ``` The Flex recipe (a contrib recipe: `composer config extra.symfony.allow-contrib true` once) registers the bundle, creates `config/packages/indexnowkit.yaml`, imports the key file route and adds `INDEXNOW_KEY` (empty: dry run outside production until `indexnow:key:generate --write-env` fills `.env.local`) and `INDEXNOW_BASE_URL` to `.env`. Without Flex, add `IndexNowKit\SymfonyBundle\IndexNowKitBundle` to `config/bundles.php` and import `@IndexNowKitBundle/config/routes.php` from `config/routes.yaml`. ```yaml # config/packages/indexnowkit.yaml indexnowkit: key: '%env(INDEXNOW_KEY)%' base_url: '%env(INDEXNOW_BASE_URL)%' # used by console commands and Messenger workers ``` Entity hooks need `indexnowkit/doctrine` **and** `doctrine/doctrine-bundle`. Without them the bundle still works for manual submission, and `indexnow:check` says so instead of failing silently. ## Declare what has a public page `#[IndexNow]` is repeatable: one attribute per family of public URLs the entity has. ```php use Doctrine\ORM\Mapping as ORM; use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; #[ORM\Entity] #[IndexNowDefaults(when: 'isPublished', fields: ['slug', 'title', 'body', 'published'])] #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] #[IndexNow(route: 'post_amp', params: ['slug' => 'slug'], when: 'hasAmp')] #[IndexNow(via: 'category')] // a changed post also refreshes its category page #[IndexNow(urls: ['/'])] // and the homepage class Post { #[ORM\Id, ORM\GeneratedValue, ORM\Column] public ?int $id = null; #[ORM\ManyToOne] public ?Category $category = null; public function __construct( #[ORM\Column(unique: true)] public string $slug, #[ORM\Column] public string $title = '', #[ORM\Column(type: 'text')] public string $body = '', #[ORM\Column] public bool $published = true, #[ORM\Column] public bool $amp = false, ) {} public function isPublished(): bool { return $this->published; } public function hasAmp(): bool { return $this->amp; } } ``` | Option | Meaning | |---|---| | `route` / `params` | route name and `param => property, getter, "self", dotted.path` or a typed `Param\*` value | | `resolver` | a `UrlResolverInterface` service id or class for anything custom | | `via` | an accessor to a related object or collection whose pages are resubmitted | | `url` / `urls` | an accessor returning the URL(s), or literal URLs | | `when` / `whenFields` | bool accessor; unpublished entities are skipped and `published → draft` is sent as a deletion | | `fields` | for updates, submit only when one of these fields changed | | `events` | subset of `created`, `updated`, `deleted` | | `locales` | `current` (default), `all` (every `framework.enabled_locales`), or a list | | `host` | generate this rule's URLs on another host (multi-domain) | | `name` | stable rule id for logs, `indexnow:explain` and overriding in a subclass | Full model, typed parameters, inheritance and the semantics table: [core attribute reference](../core/attribute-reference.md). ## Verify ```bash bin/console indexnow:check # config, key file reachable, engines, dispatch, Doctrine hooks bin/console indexnow:check --live # also sends a real probe request to every engine ``` Run it after every key rotation and after every deployment that touches the configuration. It is the command that answers most "it does not work" reports on its own. ## How it works - URLs are collected in `onFlush` / `postFlush` and handed over **only after the outermost transaction commits** (a DBAL driver middleware watches the real COMMIT). Rolled-back changes are never submitted. - Every rule of an entity is classified separately: the article page can be an update while the AMP page of the same entity is a deletion, in the same flush. - Everything collected during one HTTP request, console command or Messenger message is sent as **one batch** after the response was sent (`kernel.terminate`), never inside your request. - `dispatch: auto` uses **Messenger** when a transport is configured, otherwise sends synchronously after the response. `sync` always sends on terminate. `none` collects and never sends, for applications that drain the collector themselves. - The same URL is not re-sent within **10 minutes** (`debounce.per_url`, stored in `cache.app`), batches are split at **10 000 URLs**, hosts are grouped, `202` is a success, `403` means the key file is wrong. - Failures are logged on the `indexnow` Monolog channel and never break your request. `http.timeout` (10 s) and `throttle.max_requests_per_minute` (60, per process) apply to the HTTP client the bundle builds on first use. ## Manual submission ```php public function __construct(private readonly IndexNowKit\IndexNowKit $indexNow) {} $this->indexNow->submit(['/posts/hello', 'https://www.example.com/about']); $this->indexNow->submitEntity($post); $this->indexNow->explain($post, IndexNowKit\Event::Updated); // which rule produced which URL ``` ## Commands | Command | Options | |---|---| | `indexnow:check` | `--live` send a real probe · `--host` check one host only · `--probe-url` page to probe when the root redirects | | `indexnow:submit ` | `-f, --force` ignore the debounce store · `--dry-run` · `--json` | | `indexnow:submit-entity [ids...]` | `--event=updated`, `created` or `deleted` · `--limit` (default 1000, when no ids) · `--explain` show rule → URL and send nothing · `-f, --force` · `--dry-run` · `--json` | | `indexnow:explain ` | `--event=updated`, `created` or `deleted` | | `indexnow:sitemap [sitemap]` | `--changed-since="1 day"` · `--allow-foreign-hosts` follow CDN-hosted parts · `-f, --force` · `--dry-run` list only · `--json` · `--no-verify` | | `indexnow:history` | `--host` · `--status=ok|pending|failed|skipped` · `--url` · `--since=2h|3d|2026-09-01` · `--limit` (default 50) · `--json` · `--purge[=days]` | | `indexnow:status` | `--json` | | `indexnow:key:generate` | `-l, --length` (8-128, default 32) · `--alphanumeric` · `--write-env[=FILE]` (default `.env.local`) · `--force` rotate an existing key | `` accepts an FQCN or a short `App\Entity` name. `indexnow:submit-entity` and `indexnow:explain` need Doctrine. ### Sitemaps `composer require indexnowkit/sitemap # optional: the indexnow:sitemap command` `indexnow:sitemap` with no argument reads `sitemap.url`, else `/sitemap.xml`; a local path or `file://` URL reads the file without the web server. XML and text sitemaps, indexes and gzip are handled by the [`indexnowkit/sitemap`](../sitemap/index.md) package; the command streams and submits every `batch.max_urls` URLs, so size is not a concern. `sitemap.enabled: false` removes the command; decorating `indexnowkit.sitemap_reader` shapes what it submits ([docs/extending.md](extending.md)). Without the package everything else works unchanged: `indexnow:sitemap` says `indexnowkit/sitemap is not installed: composer require indexnowkit/sitemap` and exits 1, `indexnow:check` prints `sitemap: not installed (…)`, a `sitemap` block left in the yaml still compiles and is ignored. Nothing is logged about it. ### History `composer require indexnowkit/history # optional: what was submitted, when, with what answer` ```yaml indexnowkit: history: store: pdo # null (default, nothing kept) | psr16 (the debounce cache pool) | pdo pdo: { service: default } # a Doctrine connection name or service id — or dsn: 'sqlite:%kernel.project_dir%/var/indexnow.sqlite' ``` Every `Result` the submitter produces — the sync flush, the Messenger worker, the commands, a URL skipped by `indexnowkit/verify` — is recorded: normalized URLs, host, engine, status, reason, HTTP code, the error message (never the response body or the key). `bin/console indexnow:history` lists them newest first (`--host`, `--status`, `--url`, `--since`, `--json`); `indexnow:history --purge` removes what is older than `history.retention_days` (a cron line); `bin/console indexnow:status` prints the switches, the dispatch mode with the Messenger transport, the debounce store, the 403 counter of every host, the last successful submission and the history size (`--json` for machines). The profiler panel gets a "Recent submissions" table. `pdo` needs the table: the migration is in the package's [docs/migrations.md](../history/migrations.md) (`Schema::sql()`); until it exists `indexnow:check` prints a `history.store` error and the submitter logs the failure without breaking the request. `psr16` is a ring buffer of `history.limit` records for one process and small sites. Your own `Submission\SubmissionStoreInterface` registered as `indexnowkit.submission_store` takes precedence over either. Without the package `indexnow:history` and `indexnow:status` say `indexnowkit/history is not installed: composer require indexnowkit/history` and exit 1, `indexnow:check` prints `history: not installed (…)`. ## Configuration The full annotated tree, every default and every compile-time validation: [docs/configuration.md](configuration.md). | Topic | | |---|---| | Multiple domains | [docs/multi-domain.md](multi-domain.md) | | Async delivery and retries | [docs/messenger.md](messenger.md) | | HTTP client, proxy, scoped clients | [docs/http-client.md](http-client.md) | | Doctrine details, priorities, connections | [docs/doctrine.md](doctrine.md) | | Custom resolvers | [docs/custom-resolvers.md](custom-resolvers.md) | | Extending: what is replaceable, decorating services | [docs/extending.md](extending.md) | | Testing your integration | [docs/testing.md](testing.md) | | Troubleshooting | [docs/troubleshooting.md](troubleshooting.md) | ## Operations - [Production checklist](../core/operations.md#production-checklist) — key and base URL, `check` in the deploy pipeline, `strict_hosts`, a shared debounce store, a monitored queue, staging that cannot submit, the three lines to alert on. - [Monitoring rules and the Sentry filter](../core/operations.md#monitoring-rules), [deleted pages](../core/operations.md#deleted-pages-what-your-site-must-return), [what not to submit](../core/operations.md#what-not-to-submit). - [Multi-domain: hosts, www and apex, hreflang](multi-domain.md) · [troubleshooting](troubleshooting.md). ## Debugging Three tools, in the order you should reach for them. 1. **`bin/console indexnow:explain 'App\Entity\Post' 42`** walks the whole decision path for one entity — rules, event subscription, `when` guard, `fields` filter, resolved URLs, normalization, host and key, key file, debounce — and sends nothing. 2. **The Web Profiler panel** shows what the request collected, what was actually sent, and the HTTP outcome per engine, alongside the dispatch mode, the key file URL per host and the debounce window. 3. **The `indexnow` Monolog channel** carries everything. Set it to `debug` while diagnosing: the reason a rule decided *not* to produce a URL is logged there. Message texts and levels are listed in the [operations guide](../core/operations.md). An invalid configuration does not throw from a flush: IndexNow is disabled, one `critical` line is logged, and `indexnow:check` prints the exact error. ## Limitations - DQL and QueryBuilder bulk `UPDATE` / `DELETE` bypass the unit of work: use `indexnow:submit` or `$indexNow->submit()`. - Sub-domains are separate hosts: give each its own key with the `hosts` map, and set `strict_hosts: true` so a host you did not configure is skipped rather than announced under the default key. - `dispatch: sync` depends on `kernel.terminate` actually firing. An early `exit()`, a fatal error, or a worker runtime whose bridge does not dispatch it per request will discard the batch — with a warning. Under Swoole, RoadRunner or FrankenPHP prefer `dispatch: messenger`. - Long-running custom commands should call `$indexNow->flush()` periodically instead of accumulating URLs for the whole process lifetime. - Outside production (`production_environments`, default `prod`/`production`), a missing `INDEXNOW_KEY` switches `dry_run` on instead of failing, so dev and test never hit the real API. - A renamed page (changed slug) announces its old URL as deleted and the new one as updated in the same flush; an entity whose slug is a `readonly` property only gets the new URL (logged at `debug`). ## Compatibility Public API of the bundle: configuration nodes, command names and options, service ids and aliases listed in [docs/extending.md](extending.md), the `Console\*Interface`s of `indexnowkit/console` and the core's `Adapter\SubmitterFactoryInterface` they are aliased to, the Messenger message and handler, and the container parameters listed in [docs/configuration.md](configuration.md). `DependencyInjection\*` is wiring, not API. The core's rules apply, including the "may grow" interfaces: [bc.md](../core/bc.md); what this package itself keeps stable: [docs/bc.md](bc.md). Before 1.0 a minor version may break; every break is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/symfony-bundle/CHANGELOG.md) with the migration. ## Notes for AI assistants - Composer package `indexnowkit/symfony-bundle` (Symfony 6.4 | 7 | 8, on `indexnowkit/core`); entity hooks need `indexnowkit/doctrine` + `doctrine/doctrine-bundle`; the `indexnow:sitemap` command needs `indexnowkit/sitemap`; pre-flight checks need `indexnowkit/verify`; `indexnow:history` / `indexnow:status` need `indexnowkit/history` (`history.store: psr16|pdo`). Configuration: `config/packages/indexnowkit.yaml`, root key `indexnowkit`. `bin/console indexnow:key:generate --env-file=.env.local` writes a fresh `INDEXNOW_KEY`; `bin/console indexnow:submit …` submits URLs by hand, `bin/console indexnow:explain ` shows why a URL is or is not produced. - Minimal complete snippet (every `use` included): ```php use Doctrine\ORM\Mapping as ORM; use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; #[ORM\Entity] #[IndexNowDefaults(when: 'isPublished', fields: ['slug', 'title', 'published'])] #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] #[IndexNow(urls: ['/'])] class Post { /* ORM columns, isPublished() */ } ``` - Verify: `bin/console indexnow:check` (exit 1 on any error; `--strict` fails on warnings too, `--json` for machines), `bin/console indexnow:config --json` (the effective configuration, keys masked: paste it into a bug report), `bin/console indexnow:explain 'App\\Entity\\Post' 1` (why a URL was or was not produced), `bin/console indexnow:submit-entity 'App\\Entity\\Post' 1 --dry-run`. - Pitfalls: - `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`); Yii3 has `sync` | `none` only. - Locales: `router.locales` in Laravel, Yii2 and Yii3, `framework.enabled_locales` in Symfony; `locales: 'all'` on a rule uses that list. - `url:` names an accessor (method or property) that returns the URL; `urls:` is a list of literal URLs. Never put a literal in `url:`. - A string in `when:` is an accessor read as truthy (`published`, `isPublished`). A status string needs `Equals`: `when: new Equals('status', 'published')` (`IndexNowKit\Attribute\Param\Equals`). - Manual submission is `submitEntity()` in Symfony, `submitModel()` in Laravel, `submitRecord()` in Yii2 and Yii3; the commands are `indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record` (Yii2), `indexnow:submit-record` (Yii3). Bulk queries (`update()`, `DB::table()`, `updateAll()`) fire no hooks: submit afterwards with those. - Laravel has two classes called `IndexNowKit`: the facade `IndexNowKit\Laravel\Facades\IndexNowKit` and the core service `IndexNowKit\IndexNowKit` (inject by type). Yii2 exposes the core through `Yii::$app->indexnow->kit()`; Yii3 defines `IndexNowKit\IndexNowKit` in the container. - Outside production a configured key with `dry_run` unset makes `check` fail (a staging copy would submit real URLs): set `dry_run: true` there, or `dry_run: false` explicitly when it submits on purpose. - Unknown configuration keys are warned about at boot (typos such as debounce.per_urls); the key list is `Config::OPTIONS` plus the adapter's own keys. ## Other frameworks | | | |---|---| | PHP | [core](../core/index.md), [doctrine](../doctrine/index.md), [laravel](../laravel/index.md), [yii2](../yii2/index.md), [yii3](../yii3/index.md) | | JS/TS | @indexnowkit/core, next, prisma (soon) | | Python | indexnowkit, indexnowkit-django (soon) | Design rationale: [docs/spec](https://github.com/indexnowkit/spec). Changelog: [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/symfony-bundle/CHANGELOG.md). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Laravel IndexNow package — `indexnowkit/laravel` Tell search engines about new, changed and deleted pages the moment an Eloquent model is committed. One attribute on the model, one env variable, done. [](https://packagist.org/packages/indexnowkit/laravel) [](https://packagist.org/packages/indexnowkit/laravel) [](https://github.com/indexnowkit/php/actions) [](https://github.com/indexnowkit/spec)    [](https://github.com/indexnowkit/php/blob/main/packages/laravel/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/laravel/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Who gets notified **Yandex, Bing (and DuckDuckGo via Bing), Naver, Seznam, Yep, Internet Archive, Amazon** — every engine in the [IndexNow](https://www.indexnow.org) [registry](https://www.indexnow.org/searchengines.json). One request to the shared endpoint reaches all of them; name engines explicitly only to reach a single one. **Google: no.** Google does not support IndexNow, its sitemap ping endpoint is gone (404) and the Indexing API is restricted to `JobPosting` / `BroadcastEvent`. This package will not pretend otherwise. **Notification, not indexing.** IndexNow tells an engine that a URL changed; whether and when the page is crawled and indexed is the engine's decision. See the result in Bing Webmaster Tools (IndexNow Insights) and Yandex.Webmaster (Indexing → Reindex pages); a useful metric is the share of submitted URLs in the index after a few days. Deleted pages: answer 410 (gone for good) or 404 (temporarily); for a move answer 301 and submit both URLs; a soft-404 or a redirect to the home page does harm. Bing's URL Submission API and Google's Indexing API are different protocols and not covered here. ## Why this over X Most IndexNow packages are a thin HTTP client: you collect the URLs, you call it, you read the answer. This family does the part that goes wrong in practice: - **Declared on the model** (`#[IndexNow]`) and submitted from the ORM hooks — no controller code to forget. - **After the commit**, not on flush: a rolled-back transaction announces nothing. - **Debounce** (10 minutes per URL, shared through your cache), **batches** of up to 10 000 URLs, one key per host from env. - **Answers handled**: 202 (key pending), 422, 429 with `Retry-After` back-off and a retry through your queue, 403 escalation. - **`check` before the first submission** says what is wrong (key file, engines, queue, cache, environment); `explain` says why a URL was or was not sent. - **One core** under the Symfony, Laravel, Yii2, Yii3 and Doctrine adapters with a shared conformance suite: the same behaviour everywhere, documented once. ## Install ```bash composer require indexnowkit/laravel composer require indexnowkit/sitemap # optional: the indexnow:sitemap command php artisan vendor:publish --tag=indexnow-config # config/indexnow.php (optional, every key has a default) php artisan indexnow:key:generate --write-env # adds INDEXNOW_KEY to .env php artisan indexnow:check # config, key file reachable, queue, cache ``` The service provider is auto-discovered. Laravel ships Guzzle, which is the PSR-18 client the package discovers; any other PSR-18 client works too (`indexnow.http.client`). ```dotenv INDEXNOW_KEY=... # from key:generate INDEXNOW_BASE_URL=https://www.example.com # defaults to APP_URL; used by artisan and queue workers ``` ## Declare what has a public page `#[IndexNow]` is repeatable: one attribute per family of public URLs the model has. `IndexNowable` registers the observer. ```php use Illuminate\Database\Eloquent\Model; use Illuminate\Database\Eloquent\Relations\BelongsTo; use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; use IndexNowKit\Laravel\Eloquent\IndexNowable; /** * @property string $slug * @property bool $published * @property bool $amp */ #[IndexNowDefaults(when: 'isPublished', fields: ['slug', 'title', 'body', 'published'])] #[IndexNow(route: 'posts.show', params: ['post' => 'self'])] // route model binding #[IndexNow(route: 'posts.amp', params: ['slug' => 'slug'], when: 'hasAmp')] #[IndexNow(via: 'category')] // a changed post also refreshes its category page #[IndexNow(urls: ['/'])] // and the homepage class Post extends Model { use IndexNowable; /** @var array */ protected $casts = ['published' => 'bool', 'amp' => 'bool']; public function isPublished(): bool { return $this->published; } public function hasAmp(): bool { return $this->amp; } /** @return BelongsTo */ public function category(): BelongsTo { return $this->belongsTo(Category::class); } } ``` | Option | Meaning | |---|---| | `route` / `params` | route name and `param => attribute, method, "self", dotted.path` or a typed `Param\*` value | | `resolver` | a `UrlResolverInterface` class or container binding for anything custom | | `via` | a relation (or dotted path) whose pages are resubmitted | | `url` / `urls` | a method returning the URL(s), or literal URLs | | `when` / `whenFields` | bool attribute or method; drafts are skipped and `published → draft` is sent as a deletion | | `fields` | for updates, submit only when one of these attributes changed | | `events` | subset of `created`, `updated`, `deleted` | | `locales` | `current` (default), `all` (`indexnow.router.locales`), or a list | | `host` | generate this rule's URLs on another host (multi-domain) | | `name` | stable rule id for logs, `indexnow:explain` and overriding in a subclass | Accessors read Eloquent attributes, casts, accessors and relations (`category.slug`) and fall back to methods (`isPublished()`). `params: ['post' => 'self']` passes the model to `route()`, so `{post}` and `{post:slug}` both work. A `when` attribute that only has a **database** default is not on the model right after `create()`: give it a model default (`protected $attributes = ['published' => false]`). Full model, typed parameters, inheritance and the semantics table: [core attribute reference](../core/attribute-reference.md). ### Models you cannot annotate ```php // AppServiceProvider::boot() use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults, RuleSet}; use IndexNowKit\Laravel\Facades\IndexNowKit; IndexNowKit::observe(Product::class, [new IndexNow(route: 'products.show', params: ['product' => 'self'])], new IndexNowDefaults(when: 'is_active')); IndexNowKit::rules()->registerFor(Page::class, fn (Page $page): ?RuleSet => ...); // decided per object ``` Two classes are called `IndexNowKit`. The **facade** `IndexNowKit\Laravel\Facades\IndexNowKit` (above) proxies the `IndexNowManager` of this package: `observe()`, `rules()`, `submitModel()`, `submitModels()`, `submit()`, `collect()`, `flush()`, `explain()`. The **core** `IndexNowKit\IndexNowKit` is the same service without the Eloquent-specific parts; inject it by type (`public function __construct(private IndexNowKit $indexNow)`) or take it from the facade with `IndexNowKit::kit()`. Import one of them per file, or alias the other. ## Verify ```bash php artisan indexnow:check # config, key file reachable, engines, queue connection, cache store, spool php artisan indexnow:check --live # also sends a real probe request to every engine ``` Run it after every key rotation and after every deployment that touches the configuration. ## How it works - Observer callbacks resolve URLs **while the old state is still live** (`getOriginal()` in `updated`, the row in `deleting`) and hand them over through `Connection::afterCommit()`: nothing leaves before the outermost transaction commits, a rolled-back transaction (or savepoint) discards them. `DB::transaction()` nesting is handled by Laravel's transaction manager. - Every rule is classified separately: the article page can be an update while the AMP page of the same model is a deletion, in the same request. - Everything collected during one request, artisan command or queue job is sent as **one batch** in `app()->terminating()` (or after each handled job), never inside your request. - `dispatch: queue` (the default) pushes a `SubmitUrlsJob`; 429 and 5xx are retried with backoff, `Retry-After` wins, 403/422 fail the job so a broken key file shows up in `failed_jobs`. `QUEUE_CONNECTION=sync` runs it inline. - `SoftDeletes`: soft delete is a deletion, `restore()` a creation, `forceDelete()` a deletion. - A renamed page (changed slug, or a changed route key behind `self`) announces its old URL as deleted and the new one as updated, in the same batch. - Nothing thrown from a rule, a resolver or the HTTP layer reaches your application: it is logged, the save succeeds. ## Commands | Command | Options | |---|---| | `indexnow:check` | `--live` real probe · `--host=` one host · `--probe-url=` page for the probe | | `indexnow:submit ` | `-f, --force` ignore debounce · `--dry-run` · `--json` | | `indexnow:submit-model [ids...]` | `--event=` · `--limit=` · `--explain` · `-f, --force` · `--dry-run` · `--json` | | `indexnow:explain ` | `--event=` — rules, `when`, URLs, key, debounce; sends nothing | | `indexnow:sitemap [sitemap]` | `--changed-since="1 day"` · `--allow-foreign-hosts` · `-f, --force` · `--dry-run` · `--json` · `--no-verify` | | `indexnow:history` | `--host=` · `--status=ok|pending|failed|skipped` · `--url=` · `--since=2h|3d|2026-09-01` · `--limit=` (default 50) · `--json` · `--purge[=days]` | | `indexnow:status` | `--json` | | `indexnow:key:generate` | `-l, --length` · `--alphanumeric` · `--write-env[=FILE]` (default `.env`) · `--force` rotate | `` accepts an FQCN or a short `App\Models` name. ### Sitemaps `composer require indexnowkit/sitemap # optional: the indexnow:sitemap command` `indexnow:sitemap` with no argument reads `indexnow.sitemap.url`, else `/sitemap.xml`; a local path works too. Schedule it: `Schedule::command('indexnow:sitemap --changed-since="1 day"')->daily()`. Without the package everything else works unchanged: `indexnow:sitemap` says `indexnowkit/sitemap is not installed: composer require indexnowkit/sitemap` and exits 1, `indexnow:check` prints `sitemap: not installed (…)`, the `sitemap` block of `config/indexnow.php` is ignored. Nothing is logged about it. Details: [docs/sitemap.md](sitemap.md). ### History `composer require indexnowkit/history # optional: what was submitted, when, with what answer` ```php // config/indexnow.php 'history' => [ 'store' => env('INDEXNOW_HISTORY_STORE'), // null (default, nothing kept) | psr16 (the debounce cache store) | pdo 'pdo' => ['service' => null], // a connection of config/database.php (null = default) — or 'dsn' => 'sqlite:/var/data/indexnow.sqlite' ], ``` Every `Result` the submitter produces — the flush in `app()->terminating()`, the queue job, the commands, a URL skipped by `indexnowkit/verify` — is recorded: normalized URLs, host, engine, status, reason, HTTP code, the error message (never the response body or the key). `php artisan indexnow:history` lists them newest first (`--host`, `--status`, `--url`, `--since`, `--json`); `indexnow:history --purge` removes what is older than `history.retention_days` (`Schedule::command('indexnow:history --purge')->daily()`); `php artisan indexnow:status` prints the switches, the dispatch mode with the queue connection and queue, the debounce store, the 403 counter of every host, the last successful submission and the history size (`--json` for machines); `php artisan about` gets a `History` line. `pdo` needs the table: the migration is in the package's [docs/migrations.md](../history/migrations.md) (`Schema::sql()`); until it exists `indexnow:check` prints a `history.store` error and the submitter logs the failure without breaking the flush. `psr16` is a ring buffer of `history.limit` records for one process and small sites. A `Submission\SubmissionStoreInterface` you bind yourself takes precedence over either. Without the package `indexnow:history` and `indexnow:status` say `indexnowkit/history is not installed: composer require indexnowkit/history` and exit 1, `indexnow:check` prints `history: not installed (…)`. ## Configuration Every key of `config/indexnow.php`, its default and what it does: [docs/configuration.md](configuration.md). | Topic | | |---|---| | Queue, retries, Horizon | [docs/queue.md](queue.md) | | Multiple domains and locales | [docs/multi-domain.md](multi-domain.md) | | Sitemaps | [docs/sitemap.md](sitemap.md) | | Extending: bindings you can replace, custom resolvers, checks | [docs/extending.md](extending.md) | | Testing your integration | [docs/testing.md](testing.md) | | Troubleshooting | [docs/troubleshooting.md](troubleshooting.md) | ## Operations - [Production checklist](../core/operations.md#production-checklist) — key and base URL, `check` in the deploy pipeline, `strict_hosts`, a shared debounce store, a monitored queue, staging that cannot submit, the three lines to alert on. - [Monitoring rules and the Sentry filter](../core/operations.md#monitoring-rules), [deleted pages](../core/operations.md#deleted-pages-what-your-site-must-return), [what not to submit](../core/operations.md#what-not-to-submit). - [Multi-domain: hosts, www and apex, hreflang](multi-domain.md) · [queue](queue.md) · [troubleshooting](troubleshooting.md). ## Debugging 1. **`php artisan indexnow:explain "App\Models\Post" 42`** walks the decision path for one model — rules, event subscription, `when`, `fields`, resolved URLs, normalization, host and key, debounce — and sends nothing. 2. **The log channel** (`indexnow.logging.channel`, default channel otherwise) carries everything; at `debug` it also says why a rule decided *not* to produce a URL. Messages and levels: [operations guide](../core/operations.md). 3. **`failed_jobs`** holds batches an engine rejected permanently (403: key file not reachable). An invalid configuration does not throw from a save: IndexNow is disabled, one `critical` line is logged, and `indexnow:check` prints the exact error. ## Limitations - `Model::query()->update()`, `delete()`, `insert()`, `upsert()` and `DB::table()` fire no model events (conformance A13): call `IndexNowKit::submitModels($query->get())` or `php artisan indexnow:submit-model` afterwards. - `attach()` / `detach()` / `sync()` on a pivot fire no events on the owner. Put `$touches = ['posts']` on the related model: the owner's `updated` (only `updated_at` changed) reaches a rule without a `fields` filter. - `dispatch: sync` depends on `terminating` firing. Under Octane it does; an early `exit()` or a fatal error discards the batch with a warning. Prefer the default `queue`. - Sub-domains are separate hosts: give each its own key with the `hosts` map, and set `strict_hosts: true`. - Outside production (`production_environments`, default `prod`/`production`), a missing `INDEXNOW_KEY` switches `dry_run` on instead of failing. ## Compatibility Public API: `config/indexnow.php` keys, command names and options, the container bindings listed in [docs/extending.md](extending.md), `Facades\IndexNowKit` / `IndexNowManager`, `Eloquent\IndexNowable`, `Queue\SubmitUrlsJob`. The core's rules apply, including the "may grow" interfaces: [bc.md](../core/bc.md); what this package itself keeps stable: [docs/bc.md](bc.md). Before 1.0 a minor version may break; every break is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/laravel/CHANGELOG.md) with the migration. Laravel 12 and 13, PHP 8.2–8.5 (Laravel 13 needs PHP 8.3). ## Notes for AI assistants - Composer package `indexnowkit/laravel` (Laravel 12 | 13, on `indexnowkit/core`); the `indexnow:sitemap` command needs `indexnowkit/sitemap`; pre-flight checks need `indexnowkit/verify`; `indexnow:history` / `indexnow:status` need `indexnowkit/history` (`history.store: psr16|pdo`). Configuration: `config/indexnow.php` and `INDEXNOW_*` env variables; `php artisan indexnow:key:generate --env-file=.env` writes a fresh `INDEXNOW_KEY`; `php artisan indexnow:submit …` submits URLs by hand, `php artisan indexnow:explain ` shows why a URL is or is not produced. - Minimal complete snippet (every `use` included): ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; use IndexNowKit\Laravel\Eloquent\IndexNowable; #[IndexNowDefaults(when: 'isPublished', fields: ['slug', 'title', 'published'])] #[IndexNow(route: 'posts.show', params: ['post' => 'self'])] // route model binding #[IndexNow(urls: ['/'])] class Post extends Model { use IndexNowable; public function isPublished(): bool { return (bool) $this->published; } } ``` - Verify: `php artisan indexnow:check` (exit 1 on any error; `--strict` fails on warnings too, `--json` for machines), `php artisan indexnow:config --json` (the effective configuration, keys masked: paste it into a bug report), `php artisan indexnow:explain 'App\\Models\\Post' 1` (why a URL was or was not produced), `php artisan indexnow:submit-model 'App\\Models\\Post' 1 --dry-run`. - Pitfalls: - `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`); Yii3 has `sync` | `none` only. - Locales: `router.locales` in Laravel, Yii2 and Yii3, `framework.enabled_locales` in Symfony; `locales: 'all'` on a rule uses that list. - `url:` names an accessor (method or property) that returns the URL; `urls:` is a list of literal URLs. Never put a literal in `url:`. - A string in `when:` is an accessor read as truthy (`published`, `isPublished`). A status string needs `Equals`: `when: new Equals('status', 'published')` (`IndexNowKit\Attribute\Param\Equals`). - Manual submission is `submitEntity()` in Symfony, `submitModel()` in Laravel, `submitRecord()` in Yii2 and Yii3; the commands are `indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record` (Yii2), `indexnow:submit-record` (Yii3). Bulk queries (`update()`, `DB::table()`, `updateAll()`) fire no hooks: submit afterwards with those. - Laravel has two classes called `IndexNowKit`: the facade `IndexNowKit\Laravel\Facades\IndexNowKit` and the core service `IndexNowKit\IndexNowKit` (inject by type). Yii2 exposes the core through `Yii::$app->indexnow->kit()`; Yii3 defines `IndexNowKit\IndexNowKit` in the container. - Outside production a configured key with `dry_run` unset makes `check` fail (a staging copy would submit real URLs): set `dry_run: true` there, or `dry_run: false` explicitly when it submits on purpose. - Unknown configuration keys are warned about at boot (typos such as debounce.per_urls); the key list is `Config::OPTIONS` plus the adapter's own keys. ## Other frameworks | | | |---|---| | PHP | [core](../core/index.md), [symfony-bundle](../symfony-bundle/index.md), [doctrine](../doctrine/index.md), [laravel](./index.md), [yii2](../yii2/index.md), [yii3](../yii3/index.md) | | JS/TS | @indexnowkit/core, next, prisma (soon) | | Python | indexnowkit, indexnowkit-django (soon) | Design rationale: [docs/spec](https://github.com/indexnowkit/spec). Changelog: [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/laravel/CHANGELOG.md). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Yii2 IndexNow extension — `indexnowkit/yii2` Tell search engines about new, changed and deleted pages the moment an ActiveRecord row is committed. One attribute on the model, one component, done. [](https://packagist.org/packages/indexnowkit/yii2) [](https://packagist.org/packages/indexnowkit/yii2) [](https://github.com/indexnowkit/php/actions) [](https://github.com/indexnowkit/spec)    [](https://github.com/indexnowkit/php/blob/main/packages/yii2/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/yii2/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Who gets notified **Yandex, Bing (and DuckDuckGo via Bing), Naver, Seznam, Yep, Internet Archive, Amazon** — every engine in the [IndexNow](https://www.indexnow.org) [registry](https://www.indexnow.org/searchengines.json). One request to the shared endpoint reaches all of them; name engines explicitly only to reach a single one. **Google: no.** Google does not support IndexNow; this package will not pretend otherwise. **Notification, not indexing.** IndexNow tells an engine that a URL changed; whether and when the page is crawled and indexed is the engine's decision. See the result in Bing Webmaster Tools (IndexNow Insights) and Yandex.Webmaster (Indexing → Reindex pages); a useful metric is the share of submitted URLs in the index after a few days. Deleted pages: answer 410 (gone for good) or 404 (temporarily); for a move answer 301 and submit both URLs; a soft-404 or a redirect to the home page does harm. Bing's URL Submission API and Google's Indexing API are different protocols and not covered here. ## Why this over X Most IndexNow packages are a thin HTTP client: you collect the URLs, you call it, you read the answer. This family does the part that goes wrong in practice: - **Declared on the model** (`#[IndexNow]`) and submitted from the ORM hooks — no controller code to forget. - **After the commit**, not on flush: a rolled-back transaction announces nothing. - **Debounce** (10 minutes per URL, shared through your cache), **batches** of up to 10 000 URLs, one key per host from env. - **Answers handled**: 202 (key pending), 422, 429 with `Retry-After` back-off and a retry through your queue, 403 escalation. - **`check` before the first submission** says what is wrong (key file, engines, queue, cache, environment); `explain` says why a URL was or was not sent. - **One core** under the Symfony, Laravel, Yii2, Yii3 and Doctrine adapters with a shared conformance suite: the same behaviour everywhere, documented once. ## Install ```bash composer require indexnowkit/yii2 symfony/http-client nyholm/psr7 # any PSR-18 client + PSR-17 factories work composer require indexnowkit/sitemap # optional: the indexnow/sitemap command ``` ```php // config/web.php and config/console.php 'bootstrap' => ['indexnow'], // registers the console controller and the key file route 'components' => [ 'indexnow' => [ 'class' => \IndexNowKit\Yii2\IndexNowComponent::class, 'options' => [ 'key' => getenv('INDEXNOW_KEY'), 'base_url' => 'https://www.example.com', // used by console commands and queue workers 'dry_run' => YII_ENV_DEV, // dev/staging: log the request, send nothing (check fails when this is unset outside production) ], ], ], ``` ```bash php yii indexnow/key-generate --write-env # writes INDEXNOW_KEY=… to .env (or prints the key) php yii indexnow/check # options, key file reachable, queue, cache, URL rules ``` Yii2 does not read `.env` by itself: export the variable (`export INDEXNOW_KEY=…`), put it in the web server or container environment, or load the file with `vlucas/phpdotenv` before `config/*.php` runs — `getenv('INDEXNOW_KEY')` returns `false` until one of these is done, and `check` says `no key configured`. In `yii2-app-basic`, `config/web.php` and `config/console.php` are independent: configure the `indexnow` component **and** `urlManager` (pretty URLs, rules) in both, or `check`, `explain` and `submit-record` see a different setup than the web application. Pretty URLs (`urlManager.enablePrettyUrl`) are required for the key file route `/.txt`. The package needs a PSR-18 client (`symfony/http-client` + `nyholm/psr7` as above, or Guzzle); it discovers one, or takes the component/class named in `http.client`. ## Declare what has a public page `#[IndexNow]` is repeatable: one attribute per family of public URLs. `IndexNowBehavior` registers the hooks. Save the example as `models/Post.php` under `namespace app\models;` — it reads the columns `slug`, `title`, `body`, `published`, `amp` (the AMP page exists while it is true) and `category_id`; `Category` is a record of your own with its own `#[IndexNow]` rule (drop the `via: 'category'` line if you have none). ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; use IndexNowKit\Yii2\ActiveRecord\IndexNowBehavior; use yii\db\ActiveQuery; use yii\db\ActiveRecord; #[IndexNowDefaults(when: 'published', fields: ['slug', 'title', 'body', 'published'])] #[IndexNow(route: 'post/view', params: ['slug' => 'slug'])] #[IndexNow(route: 'post/amp', params: ['slug' => 'slug'], when: 'amp')] #[IndexNow(via: 'category')] // a changed post also refreshes its category page #[IndexNow(urls: ['/'])] // and the homepage final class Post extends ActiveRecord { public static function tableName(): string { return 'posts'; } public function init(): void { parent::init(); $this->loadDefaultValues(); // `published` has a database default: make it visible before the first save } public function behaviors(): array { return [IndexNowBehavior::class]; } public function getCategory(): ActiveQuery { return $this->hasOne(Category::class, ['id' => 'category_id']); } } ``` | Option | Meaning | |---|---| | `route` / `params` | a Yii route (`controller/action`) and `param => attribute, method, "self", dotted.path` (`self` = the primary key) | | `resolver` | a `UrlResolverInterface` class or component id for anything custom | | `via` | a relation (or dotted path) whose pages are resubmitted | | `url` / `urls` | a method returning the URL(s), or literal URLs | | `when` / `whenFields` | bool attribute or method; drafts are skipped and `published → draft` is sent as a deletion | | `fields` | for updates, submit only when one of these attributes changed | | `events`, `locales`, `host`, `name` | subset of events; `current`/`all`/list (`router.locales`); another host; stable rule id | Accessors read ActiveRecord attributes and relations (`category.slug`) and fall back to methods. A `when` column that only has a **database** default is null on a fresh record: call `$this->loadDefaultValues()` in `init()` or set the attribute before `save()`. Classes you cannot annotate: `'active_record' => ['models' => [Product::class]]` in the options, or `Yii::$app->indexnow->observe(Product::class, [new IndexNow(...)])` at runtime. Full model, typed parameters, inheritance and the semantics table: [core attribute reference](../core/attribute-reference.md). ## Verify ```bash php yii indexnow/check # config, key file reachable, engines, queue component, cache component, spool php yii indexnow/check --live # also sends a real probe request to every engine ``` Run it after every key rotation and after every deployment that touches the configuration. ## How it works - URLs are resolved **in the ActiveRecord event**, while the old state is live (`changedAttributes` on `afterUpdate`, the row and its relations in `beforeDelete`). A renamed page announces its old URL as deleted. - Outside a transaction they go to the request collector right away. Inside one, Yii2 gives no savepoint events, so they are held with a verifier and **re-read by primary key when the transaction commits**: a change the row does not show (an inner `beginTransaction()` that rolled back) is dropped with every URL it produced. A rollback drops everything. One `SELECT` per changed record, only inside explicit transactions. Details: [docs/commit-safety.md](commit-safety.md). - Everything collected during one request is sent **after the response** (`Response::EVENT_AFTER_SEND`), in one batch; console commands flush when they end, queue workers after every job. - `dispatch: auto` (default) pushes a `SubmitUrlsJob` to the `queue` component when `yiisoft/yii2-queue` is configured (429/5xx re-pushed with the delay of `retry.*`, `Retry-After` honoured), else sends synchronously. Details: [docs/queue.md](queue.md). - Nothing thrown from a rule, a resolver or the HTTP layer reaches your application: it is logged under the `indexnow` category, the save succeeds. An invalid configuration disables IndexNow with one `critical` line; `php yii indexnow/check` prints the exact error. ## Commands | Command | Options | |---|---| | `indexnow/check` | `--live` real probe · `--host=` one host · `--probe-url=` page for the probe | | `indexnow/submit ` | `--force` ignore debounce · `--dry-run` · `--json` | | `indexnow/submit-record [ids...]` | `--event=` · `--limit=` · `--explain` · `--force` · `--dry-run` · `--json` | | `indexnow/explain ` | `--event=` — rules, `when`, URLs, key, debounce; sends nothing | | `indexnow/sitemap [sitemap]` | `--changed-since="1 day"` · `--allow-foreign-hosts` · `--force` · `--dry-run` · `--json` · `--no-verify` | | `indexnow/history` | `--host=` · `--status=ok|pending|failed|skipped` · `--url=` · `--since=2h|3d|2026-09-01` · `--limit=` (default 50) · `--json` · `--purge[=days]` | | `indexnow/status` | `--json` | | `indexnow/key-generate` | `--length` · `--alphanumeric` · `--write-env[=FILE]` · `--force` rotate | `` is an FQCN or a short name under `app\models`. Ids are space- or comma-separated. ### Sitemaps `composer require indexnowkit/sitemap # optional: the indexnow/sitemap command` `indexnow/sitemap` with no argument reads `sitemap.url`, else `/sitemap.xml`; a local path works too. Without the package everything else works unchanged: `indexnow/sitemap` says `indexnowkit/sitemap is not installed: composer require indexnowkit/sitemap` and exits 1, `indexnow/check` prints `sitemap: not installed (…)`, a `sitemap` block in the options is ignored, `sitemapConfig()` / `sitemapSource()` throw a `LogicException` with the same sentence. Nothing is logged about it. ### History `composer require indexnowkit/history # optional: what was submitted, when, with what answer` ```php 'indexnow' => ['class' => IndexNowComponent::class, 'options' => [ // ... 'history' => [ 'store' => 'pdo', // null (default, nothing kept) | psr16 (the debounce cache component) | pdo 'pdo' => ['service' => 'db'], // the db component holding the table — or 'dsn' => 'sqlite:/var/data/indexnow.sqlite' ], ]], ``` Every `Result` the submitter produces — the flush after the response, the yii2-queue job, the commands, a URL skipped by `indexnowkit/verify` — is recorded: normalized URLs, host, engine, status, reason, HTTP code, the error message (never the response body or the key). `php yii indexnow/history` lists them newest first (`--host`, `--status`, `--url`, `--since`, `--json`); `indexnow/history --purge` removes what is older than `history.retention_days` (a cron line); `php yii indexnow/status` prints the switches, the dispatch mode with the queue component, the debounce store, the 403 counter of every host, the last successful submission and the history size (`--json` for machines). `pdo` needs the table: the migration is in the package's [docs/migrations.md](../history/migrations.md) (`Schema::sql()`); until it exists `indexnow/check` prints a `history.store` error and the submitter logs the failure without breaking the flush. `psr16` is a ring buffer of `history.limit` records for one process and small sites. The `submissionStore` property of the component takes precedence over either. Without the package `indexnow/history` and `indexnow/status` say `indexnowkit/history is not installed: composer require indexnowkit/history` and exit 1, `indexnow/check` prints `history: not installed (…)`, `historyConfig()` throws a `LogicException` with the same sentence. ## Configuration and docs Every option, its default and what it does: [docs/configuration.md](configuration.md). Commit safety: [docs/commit-safety.md](commit-safety.md). Replacing pieces, custom resolvers, checks: [docs/extending.md](extending.md). Queue, retries, failures: [docs/queue.md](queue.md). Several hosts, www and apex, languages: [docs/multi-domain.md](multi-domain.md). Testing your integration: [docs/testing.md](testing.md). ## Operations - [Production checklist](../core/operations.md#production-checklist) — key and base URL, `check` in the deploy pipeline, `strict_hosts`, a shared debounce store, a monitored queue, staging that cannot submit, the three lines to alert on. - [Monitoring rules and the Sentry filter](../core/operations.md#monitoring-rules), [deleted pages](../core/operations.md#deleted-pages-what-your-site-must-return), [what not to submit](../core/operations.md#what-not-to-submit). - [Multi-domain: hosts, www and apex, languages](multi-domain.md) · [queue](queue.md) · [commit safety](commit-safety.md) · [troubleshooting](troubleshooting.md). ## Debugging `php yii indexnow/check` validates the options, fetches the key file and reports how submissions are wired (queue, cache, pretty URLs, ActiveRecord hooks, sitemap spool); `php yii indexnow/explain 'app\models\Post' 1` shows the rules, guards and URLs of one record without sending anything; the `indexnow` log category at `debug` tells why a URL was or was not submitted. Symptoms and fixes: [docs/troubleshooting.md](troubleshooting.md). ## Limitations - `updateAll()`, `deleteAll()`, `updateAttributes()`, `updateCounters()` fire no events (conformance A13): call `Yii::$app->indexnow->submitRecords(Post::find()->where(...)->all())` or `php yii indexnow/submit-record` afterwards. - `link()` / `unlink()` write the junction row with a plain command, no event on the owner: save the owner with a bumped timestamp afterwards (`$post->updated_at = time(); $post->save(false)`), or call `submitRecord($post)`. - The sync driver of `yii2-queue` ignores the delay between attempts: 429/5xx attempts run back-to-back (development only, `check` warns). - Without pretty URLs the key file cannot be routed: enable them, or serve `/.txt` as a static file and set `key_file.enabled: false`. ## Compatibility Public API: the `options` tree, command names and options, `IndexNowComponent` methods and properties, `ActiveRecord\IndexNowBehavior`, `Queue\SubmitUrlsJob`. The core's rules apply: [bc.md](../core/bc.md); what this package itself keeps stable: [docs/bc.md](bc.md). Before 1.0 a minor version may break; every break is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/yii2/CHANGELOG.md). Yii 2.0.45+, PHP 8.2–8.5. ## Notes for AI assistants - Composer package `indexnowkit/yii2` (Yii 2.0.45+, on `indexnowkit/core`); the `indexnow/sitemap` command needs `indexnowkit/sitemap`; pre-flight checks need `indexnowkit/verify`; `indexnow/history` / `indexnow/status` need `indexnowkit/history` (`history.store: psr16|pdo`). Configuration: the `indexnow` application component (`options` array), `'bootstrap' => ['indexnow']`. `php yii indexnow/key-generate --write-env` writes a fresh `INDEXNOW_KEY`; `php yii indexnow/submit …` submits URLs by hand, `php yii indexnow/explain ` shows why a URL is or is not produced. - Minimal complete snippet (every `use` included): ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; use IndexNowKit\Yii2\ActiveRecord\IndexNowBehavior; #[IndexNowDefaults(when: 'published', fields: ['slug', 'title', 'published'])] #[IndexNow(route: 'post/view', params: ['slug' => 'slug'])] #[IndexNow(urls: ['/'])] final class Post extends ActiveRecord { public function behaviors(): array { return [IndexNowBehavior::class]; } } ``` - Verify: `php yii indexnow/check` (exit 1 on any error; `--strict` fails on warnings too, `--json` for machines), `php yii indexnow/config --json` (the effective configuration, keys masked: paste it into a bug report), `php yii indexnow/explain 'app\\models\\Post' 1` (why a URL was or was not produced), `php yii indexnow/submit-record 'app\\models\\Post' 1 --dry-run`. - Pitfalls: - `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`); Yii3 has `sync` | `none` only. - Locales: `router.locales` in Laravel, Yii2 and Yii3, `framework.enabled_locales` in Symfony; `locales: 'all'` on a rule uses that list. - `url:` names an accessor (method or property) that returns the URL; `urls:` is a list of literal URLs. Never put a literal in `url:`. - A string in `when:` is an accessor read as truthy (`published`, `isPublished`). A status string needs `Equals`: `when: new Equals('status', 'published')` (`IndexNowKit\Attribute\Param\Equals`). - Manual submission is `submitEntity()` in Symfony, `submitModel()` in Laravel, `submitRecord()` in Yii2 and Yii3; the commands are `indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record` (Yii2), `indexnow:submit-record` (Yii3). Bulk queries (`update()`, `DB::table()`, `updateAll()`) fire no hooks: submit afterwards with those. - Laravel has two classes called `IndexNowKit`: the facade `IndexNowKit\Laravel\Facades\IndexNowKit` and the core service `IndexNowKit\IndexNowKit` (inject by type). Yii2 exposes the core through `Yii::$app->indexnow->kit()`; Yii3 defines `IndexNowKit\IndexNowKit` in the container. - Outside production a configured key with `dry_run` unset makes `check` fail (a staging copy would submit real URLs): set `dry_run: true` there, or `dry_run: false` explicitly when it submits on purpose. - Unknown configuration keys are warned about at boot (typos such as debounce.per_urls); the key list is `Config::OPTIONS` plus the adapter's own keys. ## Other frameworks | | | |---|---| | PHP | [core](../core/index.md), [symfony-bundle](../symfony-bundle/index.md), [doctrine](../doctrine/index.md), [laravel](../laravel/index.md) | | JS/TS | @indexnowkit/core, next, prisma (soon) | | Python | indexnowkit, indexnowkit-django (soon) | MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Yii3 IndexNow package — `indexnowkit/yii3` Tell search engines about new, changed and deleted pages the moment an ActiveRecord row is committed. One attribute on the model, one `composer require`, done. [](https://packagist.org/packages/indexnowkit/yii3) [](https://packagist.org/packages/indexnowkit/yii3) [](https://github.com/indexnowkit/php/actions) [](https://github.com/indexnowkit/spec)    [](https://github.com/indexnowkit/php/blob/main/packages/yii3/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/yii3/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Who gets notified **Yandex, Bing (and DuckDuckGo via Bing), Naver, Seznam, Yep, Internet Archive, Amazon** — every engine in the [IndexNow](https://www.indexnow.org) [registry](https://www.indexnow.org/searchengines.json). One request to the shared endpoint reaches all of them; name engines explicitly only to reach a single one. **Google: no.** Google does not support IndexNow; this package will not pretend otherwise. **Notification, not indexing.** IndexNow tells an engine that a URL changed; whether and when the page is crawled and indexed is the engine's decision. See the result in Bing Webmaster Tools (IndexNow Insights) and Yandex.Webmaster (Indexing → Reindex pages); a useful metric is the share of submitted URLs in the index after a few days. Deleted pages: answer 410 (gone for good) or 404 (temporarily); for a move answer 301 and submit both URLs; a soft-404 or a redirect to the home page does harm. Bing's URL Submission API and Google's Indexing API are different protocols and not covered here. ## Why this over X Most IndexNow packages are a thin HTTP client: you collect the URLs, you call it, you read the answer. This family does the part that goes wrong in practice: - **Declared on the model** (`#[IndexNow]`) and submitted from the ActiveRecord events — no controller code to forget. - **After the commit**, not on save: a rolled-back transaction announces nothing. - **Debounce** (10 minutes per URL, shared through your cache), **batches** of up to 10 000 URLs, one key per host from env. - **Answers handled**: 202 (key pending), 422, 429 with `Retry-After` back-off, 403 escalation. - **`check` before the first submission** says what is wrong (key file, engines, cache, environment, the route, the hook); `explain` says why a URL was or was not sent. - **One core** under the Symfony, Laravel, Yii2, Yii3 and Doctrine adapters with a shared conformance suite: the same behaviour everywhere, documented once. ## Install ```bash composer require indexnowkit/yii3 symfony/http-client nyholm/psr7 # any PSR-18 client + PSR-17 factories work composer require indexnowkit/sitemap # optional: the indexnow:sitemap command ``` An application on `yiisoft/app` (or any application that loads the `yiisoft/config` groups `params`, `di`, `di-web`, `di-console`, `params-console`, `events-web`, `events-console`, `routes` and `bootstrap` through its runner) is wired by the package's `config/*.php`: the container definitions, the key file route, the console commands, the flush after the response and the ActiveRecord observer. Configure it in your params: ```php // config/common/params.php 'indexnowkit/yii3' => [ 'key' => $_ENV['INDEXNOW_KEY'] ?? null, // or leave it: the package reads INDEXNOW_KEY itself 'base_url' => 'https://www.example.com', // used by console commands (no request to take the host from) // dev/staging: log the request, send nothing. Compare with the same list `production_environments` holds, // or `YII_ENV=production` would switch dry-run on in production (check fails when this is unset outside it). 'dry_run' => !in_array($_ENV['YII_ENV'] ?? null, ['prod', 'production'], true), ], ``` ```bash ./yii indexnow:key:generate --write-env # writes INDEXNOW_KEY=… to .env (or prints the key) ./yii indexnow:check # params, key file reachable, route, hook, cache, dispatch ``` The package reads `INDEXNOW_KEY`, `INDEXNOW_PREVIOUS_KEY`, `INDEXNOW_BASE_URL` and `INDEXNOW_DRY_RUN` from `$_ENV`, `$_SERVER` and `getenv()` (`vlucas/phpdotenv` of the application template fills `$_ENV`), and takes the environment name from `YII_ENV`. The key file route needs `yiisoft/router` with `yiisoft/router-fastroute` (the application template has them); URL generation for `route:` rules goes through the container's `UrlGeneratorInterface`. The package needs a PSR-18 client (`symfony/http-client` + `nyholm/psr7` as above, or Guzzle): it discovers one, or takes the container id named in `http.client`. The `params` merge of your application must be recursive for the block (`RecursiveMerge::groups('params', ...)` in `configuration.php`, as the template does); without it the package's defaults still apply to what your block leaves out. ## Declare what has a public page `#[IndexNow]` is repeatable: one attribute per family of public URLs. `#[IndexNowEvents]` registers the hooks, and `EventsTrait` of yiisoft/active-record is what makes the record dispatch events at all. Save the example as `src/Model/Post.php` under `namespace App\Model;` — it reads the columns `slug`, `title`, `body`, `published`, `amp` (the AMP page exists while it is true) and `category_id`; `Category` is a record of your own with its own `#[IndexNow]` rule (drop the `via: 'category'` line if you have none). `route:` names a route of your `routes.php` (`Route::get('/posts/{slug}')->name('post/view')`). ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; use IndexNowKit\Yii3\ActiveRecord\IndexNowEvents; use Yiisoft\ActiveRecord\ActiveQueryInterface; use Yiisoft\ActiveRecord\ActiveRecord; use Yiisoft\ActiveRecord\Trait\EventsTrait; use Yiisoft\ActiveRecord\Trait\MagicRelationsTrait; #[IndexNowDefaults(when: 'published', fields: ['slug', 'title', 'body', 'published'])] #[IndexNow(route: 'post/view', params: ['slug' => 'slug'])] #[IndexNow(route: 'post/amp', params: ['slug' => 'slug'], when: 'amp')] #[IndexNow(via: 'category')] // a changed post also refreshes its category page #[IndexNow(urls: ['/'])] // and the homepage #[IndexNowEvents] // the hook: yiisoft/active-record dispatches events only with EventsTrait final class Post extends ActiveRecord { use EventsTrait; use MagicRelationsTrait; public ?int $id = null; public string $slug = ''; public string $title = ''; public ?string $body = null; public bool $published = true; public bool $amp = false; public ?int $category_id = null; public function tableName(): string { return 'posts'; } public function getCategoryQuery(): ActiveQueryInterface { return $this->hasOne(Category::class, ['id' => 'category_id']); } } ``` | Option | Meaning | |---|---| | `route` / `params` | a route name and `argument => property, method, "self", dotted.path` (`self` = the primary key) | | `resolver` | a `UrlResolverInterface` class or container id for anything custom | | `via` | a relation (or dotted path) whose pages are resubmitted | | `url` / `urls` | a method returning the URL(s), or literal URLs | | `when` / `whenFields` | bool property or method; drafts are skipped and `published → draft` is sent as a deletion | | `fields` | for updates, submit only when one of these properties changed | | `events`, `locales`, `host`, `name` | subset of events; `current`/`all`/list (`router.locales`); another host; stable rule id | Accessors read ActiveRecord properties and relations (`category.slug`, a `getQuery()` relation) and fall back to methods. **`via:` and any dotted path through a relation need `MagicRelationsTrait`** next to `EventsTrait`, as in the model above: yiisoft/active-record reads a `getQuery()` relation by name only through that trait, and without it the rule fails with an error from the core instead of yielding the related pages. A `when` column that only has a **database** default is null on a fresh record until `loadDefaultValues()`: give the typed property a default (`public bool $published = true;`) as above. Classes you cannot annotate: `'active_record' => ['models' => [Product::class]]` in the params (the class still needs `EventsTrait`), or `$indexNow->observe(Product::class, [new IndexNow(...)])` at runtime. Full model, typed parameters, inheritance and the semantics table: [core attribute reference](../core/attribute-reference.md). ## Verify ```bash ./yii indexnow:check # params, key file reachable, engines, route, hook, cache, dispatch, spool ./yii indexnow:check --live # also sends a real probe request to every engine ``` Run it after every key rotation and after every deployment that touches the configuration. ## How it works - URLs are resolved **in the ActiveRecord event**, while the old state is live (`BeforeUpdate` keeps the old values for `AfterUpdate`, `BeforeDelete` still sees the row and its relations). A renamed page announces its old URL as deleted. - Outside a transaction they go to the request collector right away. Inside one, yiisoft/db gives no commit or rollback events at all, so they are held with a verifier and **re-read by primary key at the end of the request** (or the command): a change the row does not show (a rolled-back transaction, an inner `beginTransaction()` that rolled back to its savepoint) is dropped with every URL it produced. One `SELECT` per changed record, only inside explicit transactions. Details: [docs/commit-safety.md](commit-safety.md). - Everything collected during one request is sent **after the response** (`AfterEmit` of yiisoft/yii-http), in one batch; a console command flushes when it ends (`ApplicationShutdown`); a long-running command calls `$indexNow->flush()` between its units of work. - `dispatch: sync` (default) sends inline after the response; `none` collects and never sends. There is no queue mode until `yiisoft/queue` has a stable release: replace `DispatcherInterface` in your `di/` with a dispatcher over the queue you run ([docs/extending.md](extending.md)). - Nothing thrown from a rule, a resolver or the HTTP layer reaches your application: it is logged under the `indexnow` category (`logging.category`), the save succeeds. An invalid configuration disables IndexNow with one `critical` line; `./yii indexnow:check` prints the exact error. ## Commands | Command | Options | |---|---| | `indexnow:check` | `--live` real probe · `--host=` one host (repeatable) · `--probe-url=` page for the probe · `--json` · `--strict` · `--sample=` / `--sample-class=` (needs `indexnowkit/verify`) | | `indexnow:config` | `--json` — the effective configuration, keys and DSNs masked | | `indexnow:submit ` | `--force` ignore debounce · `--dry-run` · `--json` | | `indexnow:submit-record [ids...]` | `--event=` · `--limit=` · `--explain` · `--force` · `--dry-run` · `--json` | | `indexnow:explain ` | `--event=` · `--json` — rules, `when`, URLs, key, debounce; sends nothing | | `indexnow:sitemap [sitemap]` | `--changed-since="1 day"` · `--allow-foreign-hosts` · `--force` · `--dry-run` · `--json` · `--no-verify` | | `indexnow:history` | `--host=` · `--status=ok|pending|failed|skipped` · `--url=` · `--since=2h|3d|2026-09-01` · `--limit=` (default 50) · `--json` · `--purge[=days]` | | `indexnow:status` | `--json` | | `indexnow:key:generate` | `--length` · `--alphanumeric` · `--write-env[=FILE]` · `--force` rotate · `--no-previous` · `--yes` | `` is an FQCN or a short name under `active_record.namespaces` (`App\Model`, `App\Entity` by default). ### Sitemaps `composer require indexnowkit/sitemap # optional: the indexnow:sitemap command` `indexnow:sitemap` with no argument reads `sitemap.url`, else `/sitemap.xml`; a local path works too. Without the package everything else works unchanged: `indexnow:sitemap` says `indexnowkit/sitemap is not installed: composer require indexnowkit/sitemap` and exits 1, `indexnow:check` prints `sitemap: not installed (…)`, a `sitemap` block in the params is ignored, `sitemapConfig()` / `sitemapSource()` throw a `LogicException` with the same sentence. Nothing is logged about it. ### Verify `composer require indexnowkit/verify # optional: one GET before every submission` With `'verify' => ['enabled' => true]` every URL is fetched before it is submitted: `noindex`, `robots.txt`, a canonical pointing elsewhere, a redirect or an origin error skip it with a logged reason, and `indexnow:check --sample=` / `--sample-class=` report what an engine would see. With `dispatch: sync` the GETs run inside the web request after the response was sent; `check` warns about it. Without the package a `verify` block is ignored and `check` prints `verify: not installed (…)`. ### History `composer require indexnowkit/history # optional: what was submitted, when, with what answer` ```php 'indexnowkit/yii3' => [ // ... 'history' => [ 'store' => 'pdo', // null (default, nothing kept) | psr16 (the debounce cache) | pdo 'pdo' => ['service' => ConnectionInterface::class], // the yiisoft/db connection id holding the table — or 'dsn' => 'sqlite:/var/data/indexnow.sqlite' ], ], ``` Every `Result` the submitter produces — the flush after the response, the commands, a URL skipped by `indexnowkit/verify` — is recorded: normalized URLs, host, engine, status, reason, HTTP code, the error message (never the response body or the key). `./yii indexnow:history` lists them newest first (`--host`, `--status`, `--url`, `--since`, `--json`); `indexnow:history --purge` removes what is older than `history.retention_days` (a cron line); `./yii indexnow:status` prints the switches, the dispatch mode, the debounce store, the 403 counter of every host, the last successful submission and the history size (`--json` for machines). `pdo` needs the table: the migration is in the package's [docs/migrations.md](../history/migrations.md) (`Schema::sql()`); until it exists `indexnow:check` prints a `history.store` error and the submitter logs the failure without breaking the flush. `psr16` is a ring buffer of `history.limit` records for one process and small sites. A `SubmissionStoreInterface` definition of your own in `di/` takes precedence over either. Without the package `indexnow:history` and `indexnow:status` say `indexnowkit/history is not installed: composer require indexnowkit/history` and exit 1, `indexnow:check` prints `history: not installed (…)`, `historyConfig()` throws a `LogicException` with the same sentence. ## Configuration and docs Every option, its default and what it does: [docs/configuration.md](configuration.md). Commit safety: [docs/commit-safety.md](commit-safety.md). Replacing pieces in the container, custom resolvers, checks, a queue: [docs/extending.md](extending.md). Several hosts, www and apex, locales: [docs/multi-domain.md](multi-domain.md). Testing your integration: [docs/testing.md](testing.md). ## Operations - [Production checklist](../core/operations.md#production-checklist) — key and base URL, `check` in the deploy pipeline, `strict_hosts`, a shared debounce store, staging that cannot submit, the three lines to alert on. - [Monitoring rules and the Sentry filter](../core/operations.md#monitoring-rules), [deleted pages](../core/operations.md#deleted-pages-what-your-site-must-return), [what not to submit](../core/operations.md#what-not-to-submit). - [Multi-domain: hosts, www and apex, locales](multi-domain.md) · [commit safety](commit-safety.md) · [troubleshooting](troubleshooting.md). ## Debugging `./yii indexnow:check` validates the params, fetches the key file and reports how submissions are wired (dispatch, cache, the route, the ActiveRecord hook, sitemap spool); `./yii indexnow:explain 'App\Model\Post' 1` shows the rules, guards and URLs of one record without sending anything; the `indexnow` log category at `debug` tells why a URL was or was not submitted. Symptoms and fixes: [docs/troubleshooting.md](troubleshooting.md). ## Limitations - `updateAll()`, `deleteAll()`, `updateCounters()` fire no events (conformance A13): call `$indexNow->submitRecords(Post::query()->where(...)->all())` or `./yii indexnow:submit-record` afterwards. - `upsert()` does fire an event (`AfterUpsert`), but the data layer does not say whether the row was inserted or updated, so it is announced as an update: a rule limited to `events: [Created]` does not fire on an upsert. - `link()` / `unlink()` write the junction row with a plain command, no event on the owner: save the owner with a bumped timestamp afterwards (`$post->updated_at = time(); $post->save();`), or call `submitRecord($post)`. - A record without `EventsTrait` dispatches no events at all: the attribute alone hooks nothing. - A transaction still open when the request ends delivers nothing (the verifier would read uncommitted data): the package logs a warning naming the count; close the transaction. - No `dispatch: queue` until `yiisoft/queue` is released; a queue of your own replaces `DispatcherInterface`. ## Compatibility Public API: the `indexnowkit/yii3` params block, command names and options, the container definitions of `config/di.php`, `IndexNow` methods, `ActiveRecord\IndexNowEvents`. The core's rules apply: [bc.md](../core/bc.md); what this package itself keeps stable: [docs/bc.md](bc.md). Before 1.0 a minor version may break; every break is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/yii3/CHANGELOG.md). `yiisoft/active-record ^1.0`, `yiisoft/db ^2.0`, `yiisoft/router ^4.0`, PHP 8.2–8.5. ## Notes for AI assistants - Composer package `indexnowkit/yii3` (Yii3: `yiisoft/active-record ^1.0`, `yiisoft/db ^2.0`, `yiisoft/router ^4.0`, on `indexnowkit/core`); the `indexnow:sitemap` command needs `indexnowkit/sitemap`; pre-flight checks need `indexnowkit/verify`; `indexnow:history` / `indexnow:status` need `indexnowkit/history` (`history.store: psr16|pdo`). Configuration: the `indexnowkit/yii3` params block; the package's `config/*.php` are picked up by `yiisoft/config` (di, routes, events, bootstrap, commands). `./yii indexnow:key:generate --write-env` writes a fresh `INDEXNOW_KEY`; `./yii indexnow:submit …` submits URLs by hand, `./yii indexnow:explain ` shows why a URL is or is not produced, `./yii indexnow:config --json` prints the effective configuration with the keys masked. - Minimal complete snippet (every `use` included): ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; use IndexNowKit\Yii3\ActiveRecord\IndexNowEvents; use Yiisoft\ActiveRecord\ActiveRecord; use Yiisoft\ActiveRecord\Trait\EventsTrait; use Yiisoft\ActiveRecord\Trait\MagicRelationsTrait; #[IndexNowDefaults(when: 'published', fields: ['slug', 'title', 'published'])] #[IndexNow(route: 'post/view', params: ['slug' => 'slug'])] #[IndexNow(via: 'category')] // needs MagicRelationsTrait, like any dotted path through a relation #[IndexNow(urls: ['/'])] #[IndexNowEvents] final class Post extends ActiveRecord { use EventsTrait; use MagicRelationsTrait; public ?int $id = null; public string $slug = ''; public string $title = ''; public bool $published = true; public ?int $category_id = null; public function tableName(): string { return 'posts'; } public function getCategoryQuery(): \Yiisoft\ActiveRecord\ActiveQueryInterface { return $this->hasOne(Category::class, ['id' => 'category_id']); } } ``` - Verify: `./yii indexnow:check` (exit 1 on any error; `--strict` fails on warnings too, `--json` for machines), `./yii indexnow:config --json` (paste it into a bug report), `./yii indexnow:explain 'App\\Model\\Post' 1` (why a URL was or was not produced), `./yii indexnow:submit-record 'App\\Model\\Post' 1 --dry-run`. - Pitfalls: - The record needs **both** `#[IndexNowEvents]` and `use EventsTrait;`: yiisoft/active-record dispatches events only through the trait, and the attribute only provides the handlers. `via:` and dotted paths through a relation additionally need `use MagicRelationsTrait;`. - `upsert()` raises `AfterUpsert`, not `AfterInsert`/`AfterUpdate`: it is announced as an update, so a rule limited to `events: [Created]` does not fire on it. `updateAll()`, `deleteAll()` and `updateCounters()` fire no events at all. - `dispatch` is `sync` or `none` in Yii3 (no queue until `yiisoft/queue` is stable; replace `DispatcherInterface` in `di/` for one); `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`). - Locales: `router.locales` in Laravel, Yii2 and Yii3 (`router.locale_parameter` names the route argument, `_language` in Yii3), `framework.enabled_locales` in Symfony; `locales: 'all'` on a rule uses that list. - `route:` is the **name** of a route (`->name('post/view')`), not its pattern; `route: 'post/view'` needs `Route::get('/posts/{slug}')->name('post/view')` in the routes configuration. - `url:` names an accessor (method or property) that returns the URL; `urls:` is a list of literal URLs. Never put a literal in `url:`. - A string in `when:` is an accessor read as truthy (`published`, `isPublished`). A status string needs `Equals`: `when: new Equals('status', 'published')` (`IndexNowKit\Attribute\Param\Equals`). - Manual submission is `submitEntity()` in Symfony, `submitModel()` in Laravel, `submitRecord()` in Yii2 and Yii3 (inject `IndexNowKit\Yii3\IndexNow`); the commands are `indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record` (Yii2), `indexnow:submit-record` (Yii3). Bulk queries (`update()`, `DB::table()`, `updateAll()`) fire no hooks: submit afterwards with those. - Laravel has two classes called `IndexNowKit`: the facade `IndexNowKit\Laravel\Facades\IndexNowKit` and the core service `IndexNowKit\IndexNowKit` (inject by type). Yii2 exposes the core through `Yii::$app->indexnow->kit()`; Yii3 defines `IndexNowKit\IndexNowKit` and every core interface in the container. - Outside production a configured key with `dry_run` unset makes `check` fail (a staging copy would submit real URLs): set `dry_run: true` there, or `dry_run: false` explicitly when it submits on purpose. - Unknown configuration keys are warned about at boot (typos such as debounce.per_urls); the key list is `Config::OPTIONS` plus the adapter's own keys. ## Other frameworks | | | |---|---| | PHP | [core](../core/index.md), [symfony-bundle](../symfony-bundle/index.md), [doctrine](../doctrine/index.md), [laravel](../laravel/index.md), [yii2](../yii2/index.md) | | JS/TS | @indexnowkit/core, next, prisma (soon) | | Python | indexnowkit, indexnowkit-django (soon) | MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # IndexNow command line — `indexnowkit/cli` Tell Yandex, Bing, Naver and Seznam which URLs changed, from any host that has PHP and from any CI — no framework. One binary, `indexnow`: check the key file, submit URLs or a whole sitemap (only what is new or changed since the last run), keep the history. Cron on Bitrix, WordPress, MODX, OpenCart or Joomla; a deploy of Hugo, Astro or Jekyll; "just send these ten URLs". Three packagings of the same thing: a Composer package, `indexnow.phar`, the Docker image `ghcr.io/indexnowkit/indexnow` — and a GitHub Action on top of the image. [](https://packagist.org/packages/indexnowkit/cli) [](https://packagist.org/packages/indexnowkit/cli) [](https://github.com/indexnowkit/php/actions)   [](https://github.com/indexnowkit/php/blob/main/packages/cli/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/cli/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Who gets notified **Yandex, Bing (and DuckDuckGo via Bing), Naver, Seznam, Yep, Internet Archive, Amazon** — every engine in the [IndexNow](https://www.indexnow.org) [registry](https://www.indexnow.org/searchengines.json). One request to the shared endpoint reaches all of them; name engines explicitly (`INDEXNOW_ENGINES=yandex,bing`) only to reach a single one. **Google: no.** Google does not support IndexNow; this tool will not pretend otherwise. IndexNow is a notification, not indexing: the engine decides whether and when to crawl. ## Install ```bash composer global require indexnowkit/cli # ~/.composer/vendor/bin/indexnow, or vendor/bin/indexnow in a project curl -LO https://github.com/indexnowkit/php-cli/releases/latest/download/indexnow.phar && php indexnow.phar --version docker run --rm ghcr.io/indexnowkit/indexnow --version ``` PHP 8.2+ with `pdo_sqlite` and `xmlreader` (the PHAR checks that before it runs; shared hosting without `pdo_sqlite` cannot keep the state file — `--state memory` runs without one). No `ext-intl` needed. ## Quick start ```bash cd /var/www/site # the .env and the state file live in the working directory indexnow key:generate --write-env # INDEXNOW_KEY=… into .env (mode 0600) echo 'INDEXNOW_BASE_URL=https://www.example.com' >> .env indexnow key:file /var/www/site/public # writes public/.txt: the file the engines verify indexnow check --live # the configuration, the key file over HTTP, one real probe per engine indexnow sitemap # the whole sitemap once… ``` …then one line in crontab — only what changed since the last run, whatever `` says: ```cron */30 * * * * cd /var/www/site && indexnow sitemap --new-only --json >> /var/log/indexnow.log 2>&1 ``` Or ten URLs by hand: `indexnow submit https://www.example.com/a /b /c`. ## Any CMS: Bitrix, WordPress, MODX, OpenCart, Joomla The CLI needs two things every site has: the document root (for `key:file`) and a sitemap. Bitrix generates `sitemap.xml` and the index per information block with its own "Search engine optimization" module (since version 14); WordPress has one at `/wp-sitemap.xml` (5.5+) or from an SEO plugin; MODX (`pdoTools`, `SEO Suite`), OpenCart (a sitemap feed) and Joomla (an extension) likewise. Point `INDEXNOW_SITEMAP_URL` at it when it is not `/sitemap.xml`, or give the file: `indexnow sitemap /var/www/site/public/sitemap.xml` reads it from disk. No PHP of the CMS runs; the CLI is a separate process on the same host. ### Bitrix, step by step On a BitrixVM the site lives in `/home/bitrix/www` (the other sites of a multi-site setup in `/home/bitrix/ext_www/`) and the "Search engine optimization" module writes `sitemap.xml` right into that document root (Marketing → Search engine optimization → sitemap.xml settings; regenerate it there or by its agent). The CLI needs PHP 8.2+ on the command line — the PHP the VM menu selected. ```bash mkdir -p /home/bitrix/indexnow && cd /home/bitrix/indexnow # outside the document root: the .env and the state file indexnow key:generate --write-env echo 'INDEXNOW_BASE_URL=https://www.example.com' >> .env indexnow key:file /home/bitrix/www # /home/bitrix/www/.txt, one file per configured host indexnow check --live ``` ```cron */30 * * * * cd /home/bitrix/indexnow && indexnow sitemap /home/bitrix/www/sitemap.xml --new-only --json >> /var/log/indexnow.log 2>&1 ``` The sitemap is read from disk, so nothing depends on the web server or on caching; only URLs the state file has not seen with this `` go out. Nothing is installed into Bitrix, no module, no agent: the CLI is a process of the `bitrix` user next to the site. ## Static sites: on deploy The GitHub Action `indexnowkit/indexnow-action` runs `check` and then `sitemap --new-only` from the image; the state file lives in `.indexnow/` and is cached between runs: ```yaml - uses: actions/cache@v4 with: { path: .indexnow, key: indexnow-${{ github.ref_name }} } - uses: indexnowkit/indexnow-action@v1 with: key: ${{ secrets.INDEXNOW_KEY }} base-url: https://www.example.com sitemap: dist/sitemap.xml # the file the build wrote, or a URL; default /sitemap.xml new-only: 'true' ``` Inputs, outputs and the step summary: [docs/action.md](action.md). Any other CI runs the image the same way: `docker run --rm -v "$PWD:/work" -e INDEXNOW_KEY -e INDEXNOW_BASE_URL ghcr.io/indexnowkit/indexnow sitemap --new-only` ([docs/docker.md](docker.md)). ## The state file `.indexnow/state.sqlite` in the working directory (`--state`, `INDEXNOW_STATE`), one sqlite file, created on first use in a `0700` directory: the debounce window (`debounce.per_url`, 10 minutes: a URL announced twice within it is sent once — `--force` overrides), the 403 counters per host, the submission history (`history`, `status`) and the fingerprints of the sitemap URLs announced so far (`sitemap --new-only`). Everything that makes a scheduled run idempotent, in one file that fits `actions/cache`; no backup needed (losing it costs one full re-announcement). `--state memory` keeps nothing between runs — a read-only container, a one-off `--dry-run`. `check` prints the `state:` line; details in [docs/state.md](state.md). ## Configuration Environment first, file second. Every option of the core and of the three packages is one variable: `INDEXNOW_` for the core (`INDEXNOW_KEY`, `INDEXNOW_BASE_URL`, `INDEXNOW_ENGINES`, `INDEXNOW_DEBOUNCE_PER_URL`, …), `INDEXNOW__` for the blocks (`INDEXNOW_SITEMAP_URL`, `INDEXNOW_SITEMAP_MAX_DEPTH`, `INDEXNOW_VERIFY_ENABLED`, `INDEXNOW_HISTORY_PDO_DSN`). A `.env` in the working directory is read when it is there (`--env-file`, `--no-env-file`); the process environment wins over it. For the parts that are not one value (a hosts map with per-host key locations) a JSON file in the shape of `Config::fromArray()`: `--config indexnow.json` or `INDEXNOW_CONFIG`; the variables win over the file, `INDEXNOW_HOSTS` replaces its hosts map whole. Precedence: the command's options, the process environment, `.env`, `--config`, the defaults. Two defaults are the CLI's: `debounce.store` is `state` (the state file; `memory` and `none` as everywhere) and `history.store` is `pdo` over the same file (`INDEXNOW_HISTORY_STORE=none` switches it off). `dispatch` is `sync` or `none`: a process has no queue. `check` warns about an `INDEXNOW_*` variable nothing reads. The tables: [docs/configuration.md](configuration.md). ## Commands | Command | | |---|---| | `check [--live] [--host=…] [--json] [--strict] [--sample=]` | the configuration, the key file of every host over HTTP, the state file, the debounce store, sitemap spool, verify, history; `--live` sends one probe per engine; `--sample` fetches a page the way the pre-flight would ([console](../console/index.md), codes in the core's `docs/check-codes.md`) | | `config [--json]` | the effective configuration, keys masked, plus the `cli` block (which file, which state); what a bug report pastes | | `submit … [--force] [--dry-run] [--json]` | URLs, or paths under `base_url`, now | | `sitemap [url|file] [--new-only] [--changed-since=…] [--dry-run] [--no-verify] [--json]` | the sitemap (index, gzip, text) in batches; `--new-only` only what the state file has not seen with this `lastmod` ([sitemap](../sitemap/index.md)) | | `key:generate [--write-env[=file]] [--force]` | a key; `--write-env` puts `INDEXNOW_KEY=` into `.env` (mode 0600; `--force` rotates, keeping the old key as `INDEXNOW_PREVIOUS_KEY`) | | `key:file [--host=…] [--dry-run]` | `/.txt` for every configured host (and the previous key during a rotation), or the path `key_location` names; the one command that exists only here | | `history [--host=…] [--status=…] [--since=…] [--json] [--purge]` | what was sent, when, with what answer ([history](../history/index.md)) | | `status [--json]` | switches, debounce store, 403 counters per host, the last successful submission, the history size | Global options: `--env-file`, `--no-env-file`, `--config`, `--state`, `-v` (the log on stderr). Exit codes: 0, 1 (an engine or the source failed), 2 (bad arguments; `--new-only` without a state, `sitemap.enabled: false`). `--json` keeps stdout machine-readable; the notes go to stderr. ## Docker `ghcr.io/indexnowkit/indexnow:` (`0.1`, `latest`), `php:8.3-cli-alpine` plus the PHAR, runs as the user `indexnow` (uid 1000) in `/work`: mount the directory that holds `.indexnow/` and the sitemap files there. The `-action` tag is the same image without a user for the GitHub Action. [docs/docker.md](docker.md). ## Limitations - No `explain`, no `submit-`: they need the ORM of a framework adapter. `check --sample ` says what an engine would see of a page (`indexnowkit/verify` is built in; `verify.enabled` switches the pre-flight on). - `debounce.store` and `history.pdo.service` cannot name a container id: there is no container. `state`, `memory`, `none`; `history.pdo.dsn` for a database of your own. - The image has no `ext-intl`: international host names go through the core's pure PHP Punycode. - Bulk changes in a CMS fire nothing: run `sitemap --new-only` on a schedule, that is the point. ## Other packages | Package | Framework | |---|---| | [`indexnowkit/symfony-bundle`](../symfony-bundle/index.md) | Symfony + Doctrine | | [`indexnowkit/laravel`](../laravel/index.md) | Laravel | | [`indexnowkit/yii2`](../yii2/index.md), [`indexnowkit/yii3`](../yii3/index.md) | Yii | | [`indexnowkit/core`](../core/index.md) | plain PHP, the library under all of them | A framework adapter submits the moment a model is committed; this CLI submits when it is run. On a host with a framework, prefer the adapter and use the CLI for what it adds (`key:file` on a bare host, a sitemap in cron). ## Notes for AI assistants - Composer package `indexnowkit/cli`, binary `indexnow` (also indexnow.phar from the GitHub releases of `indexnowkit/php-cli` and the image `ghcr.io/indexnowkit/indexnow`); no framework, no ORM. Configuration: `INDEXNOW_*` variables (`.env` in the working directory) or `--config FILE` (JSON); state in the `.indexnow` directory (state.sqlite). - Minimal complete setup (no PHP code: the CLI is the program): ```bash indexnow key:generate --write-env && echo 'INDEXNOW_BASE_URL=https://www.example.com' >> .env indexnow key:file /var/www/site/public && indexnow check --live indexnow sitemap --new-only --json # cron ``` ```php // the same graph in PHP, when the CLI is embedded (IndexNowKit\Cli\Wiring is a composition root over the core) use IndexNowKit\Cli\Application; exit((new Application())->run()); ``` - Commands: `check`, `config`, `submit`, `sitemap`, `key:generate`, `key:file`, `history`, `status` — the family's `indexnow:check`, `indexnow:config`, `indexnow:submit`, `indexnow:sitemap`, `indexnow:key:generate`, `indexnow:history`, `indexnow:status` without the prefix (the binary is the prefix). Verify: `indexnow check --live`. - Pitfalls: - The key file must be served by the site (`key:file ` writes it; `check` fetches it); a 403 from every engine means the key file is wrong or cached stale. - Outside production (`INDEXNOW_ENV` not in `prod, production`) a configured key with `dry_run` unset makes `check` fail; set `INDEXNOW_DRY_RUN=true` there or `INDEXNOW_ENV=prod`. - `sitemap --new-only` needs the state file between runs (`actions/cache` in CI); `--state memory` makes every run the first. `--changed-since` and `--new-only` add up. - `debounce.store` accepts `state`, `memory`, `none` only; `http.client` cannot be set (no container). `history.store` is `pdo` over the state file by default; `INDEXNOW_HISTORY_STORE=none` switches the history off. - Unknown `INDEXNOW_*` variables are warned about by `check` (code config.unknown); the key list is `Config::OPTIONS` plus `sitemap.*`, `verify.*`, `history.*` as `INDEXNOW__`. ## Versioning SemVer; until 1.0 minor versions may contain breaking changes, listed in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/cli/CHANGELOG.md). What the compatibility promise covers — the commands, their options and the variables, not the classes: [docs/bc.md](bc.md). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # IndexNow client for PHP — `indexnowkit/core` Tell Yandex, Bing and the other [IndexNow](https://www.indexnow.org) engines which URLs changed, from any PHP application. Batching, debounce, throttling, retry policy, key file handling and the `#[IndexNow]` rule model, on top of PSR-18 / PSR-17 / PSR-3 / PSR-16 only. The framework adapters ([Symfony](../symfony-bundle/index.md), [Doctrine](../doctrine/index.md), [Laravel](../laravel/index.md), [Yii2](../yii2/index.md), [Yii3](../yii3/index.md)) and the add-on packages build on it; use it directly in plain PHP, a CMS plugin or a custom framework. [](https://packagist.org/packages/indexnowkit/core) [](https://packagist.org/packages/indexnowkit/core) [](https://github.com/indexnowkit/php/actions) [](https://github.com/indexnowkit/spec)    [](https://github.com/indexnowkit/php/blob/main/packages/core/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/core/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Who gets notified **Yandex, Bing (and DuckDuckGo via Bing), Naver, Seznam, Yep, Internet Archive, Amazon** — every engine in the IndexNow [registry](https://www.indexnow.org/searchengines.json). One request to the shared endpoint `api.indexnow.org` reaches all of them; name engines explicitly (`engines: [yandex, bing]`) only to reach a single one. Internet Archive has no working direct endpoint at the time of writing — it is reached through `api`. **Google: no.** Google does not support IndexNow, its sitemap ping endpoint is gone and the Indexing API is limited to `JobPosting` / `BroadcastEvent`. Keep your sitemap for Google; this library will not pretend otherwise. **Notification, not indexing.** IndexNow tells an engine that a URL changed; whether and when the page is crawled and indexed is the engine's decision. See the result in Bing Webmaster Tools (IndexNow Insights) and Yandex.Webmaster (Indexing → Reindex pages); a useful metric is the share of submitted URLs in the index after a few days. Deleted pages: answer 410 (gone for good) or 404 (temporarily); for a move answer 301 and submit both URLs; a soft-404 or a redirect to the home page does harm. Bing's URL Submission API and Google's Indexing API are different protocols and not covered here. ## Why this over X Most IndexNow packages are a thin HTTP client: you collect the URLs, you call it, you read the answer. This family does the part that goes wrong in practice: - **Declared on the model** (`#[IndexNow]`) and submitted from the ORM hooks — no controller code to forget. - **After the commit**, not on flush: a rolled-back transaction announces nothing. - **Debounce** (10 minutes per URL, shared through your cache), **batches** of up to 10 000 URLs, one key per host from env. - **Answers handled**: 202 (key pending), 422, 429 with `Retry-After` back-off and a retry through your queue, 403 escalation. - **`check` before the first submission** says what is wrong (key file, engines, queue, cache, environment); `explain` says why a URL was or was not sent. - **One core** under the Symfony, Laravel, Yii2, Yii3 and Doctrine adapters with a shared conformance suite: the same behaviour everywhere, documented once. ## Install ```bash composer require indexnowkit/core symfony/http-client nyholm/psr7 # any PSR-18 client + PSR-17 factories work ``` If you use a framework, prefer its adapter: it wires everything below through your container and hooks into entity changes. The family: | Package | What | |---|---| | `indexnowkit/core` | this package: protocol client, rules, key file, the adapter kit | | [`indexnowkit/doctrine`](../doctrine/index.md) | Doctrine ORM listener plus a DBAL middleware, commit-safe | | [`indexnowkit/symfony-bundle`](../symfony-bundle/index.md) | Symfony: config, Messenger, key file route, commands, profiler panel | | [`indexnowkit/laravel`](../laravel/index.md) | Laravel: Eloquent observer, queue, key file route, artisan commands | | [`indexnowkit/yii2`](../yii2/index.md) | Yii2: ActiveRecord events with verify-on-commit, yii2-queue, console controller | | [`indexnowkit/yii3`](../yii3/index.md) | Yii3: `#[IndexNowEvents]` on yiisoft/active-record with verify-on-commit, a yiisoft/config plugin, console commands | | [`indexnowkit/sitemap`](../sitemap/index.md) | reads a sitemap (index, gzip, text) and submits its URLs; the `sitemap` command of every adapter | | [`indexnowkit/verify`](../verify/index.md) | one GET before every submission: noindex, robots.txt, canonical, redirects, origin errors; `check --sample` | | [`indexnowkit/history`](../history/index.md) | what was submitted, when, with what answer: PSR-16 and PDO stores, the `history` and `status` commands | | [`indexnowkit/console`](../console/index.md) | the `check`, `config`, `submit`, `submit-`, `explain`, `key:generate` commands (`symfony/console` classes, their bodies and their definitions); every adapter requires it | | [`indexnowkit/cli`](../cli/index.md) | no framework: the `indexnow` binary (Composer, PHAR, Docker image, GitHub Action) — `check`, `submit`, `sitemap --new-only`, `key:file`, `history`, `status` over `INDEXNOW_*` variables and a state file; cron on any CMS (Bitrix, WordPress, MODX, OpenCart), static sites on deploy | | [`indexnowkit/testing`](../testing/index.md) | `require-dev`: the conformance kits (C01–C22, A01–A21), the H01–H06 assertions, the mock IndexNow server | ## Quick start ```php use IndexNowKit\Config; use IndexNowKit\IndexNowKit; $indexNow = IndexNowKit::create(Config::fromEnv()); // INDEXNOW_KEY, INDEXNOW_BASE_URL, ... foreach ($indexNow->submit(['/posts/hello', 'https://www.example.com/about']) as $result) { printf("%s %s %d %s\n", $result->engine, $result->status->value, $result->httpCode ?? 0, $result->error ?? ''); } ``` ```dotenv INDEXNOW_KEY=6f3c9a... # 8-128 characters, [A-Za-z0-9-] INDEXNOW_BASE_URL=https://www.example.com ``` `submit()` never throws for remote problems: every engine × host × batch yields a `Result` and a log line, and URLs that were not sent (debounced, disabled, dry-run, unknown host) yield a `skipped` result that says why. ## The key file Search engines verify ownership by fetching `https://{host}/{key}.txt`, whose body must be exactly the key. ```php $key = IndexNowKit\Key\KeyGenerator::generate(); // 32 hex characters, CSPRNG file_put_contents("public/$key.txt", $key); // or answer the request yourself: $body = (new KeyFileResponder($indexNow->keys))->bodyForPath($path, $host); // null -> 404 ``` Serve it with `200 OK` and `text/plain`, without redirects; `KeyFileResponder::headers()` has the right headers. A key file elsewhere on the host is fine with `key_location`. `Check\Checker` validates the configuration, fetches every key file and, with `liveProbe: true`, sends a real probe. `403` always means the key file is wrong; rotation guidance is in [docs/operations.md](operations.md). ## What happens to a URL 1. **Normalize** — relative paths resolved against `base_url`, scheme and host lower-cased, IDN hosts to punycode, default ports and fragments removed, dot-segments resolved. Anything that is not a public `http(s)` URL is dropped with a warning. 2. **De-duplicate** within the call, then **debounce**: URLs sent successfully in the last `debounce.per_url` seconds are skipped. A failing store never blocks delivery, it just stops de-duplicating and logs a warning. 3. **Group by host** and look up the key. Hosts without a key are `skipped` and never sent under another host's key. 4. **Chunk** into at most `batch.max_urls` URLs, **throttle** one token per HTTP request, and **POST** one batch per endpoint: `{"host", "key", "keyLocation"?, "urlList"}` as `application/json; charset=utf-8`. 5. **Interpret** the answer into a `Result` and mark successful URLs in the debounce store. ## Results | `status` | HTTP | `reason` | `retryable` | Meaning | |---|---|---|---|---| | `ok` | 200 | — | no | accepted | | `pending` | 202 | — | no | accepted, key verification pending; counts as success | | `failed` | 400 | `invalid_request` | no | malformed request (bug: please report) | | `failed` | 403 | `invalid_key` | no | key file not reachable or does not match | | `failed` | 422 | `unprocessable` | no | URLs do not belong to the host / `keyLocation` invalid | | `failed` | 429 | `rate_limited` | yes | `retryAfter` filled when the engine said so | | `failed` | 5xx | `server_error` | yes | | | `failed` | — | `transport` | yes | network failure or timeout | | `failed` | — or other | `unexpected` | see below | a misbehaving HTTP client (retryable) or a status no engine should return (not) | | `skipped` | — | `disabled` `dry_run` `debounced` `no_key` `invalid_url` | no | nothing was sent | `Reason` is the stable identifier for metrics and alerts, `Result::$error` the human sentence; `Reason::translationKey()` (`indexnowkit.reason.`) names the message for a UI. Decide whether to retry from `Result::$retryable`, not from the reason. `Result` also carries `engine`, `endpoint`, `host`, `urls`, `httpCode` and `metricLabels()`; `Result::retryableUrls($results)` collects what is worth retrying. ```php $indexNow->submitter->addListener(fn (IndexNowKit\Result $r) => $metrics->increment('indexnow_results_total', $r->metricLabels())); ``` Log lines go to the PSR-3 logger you pass to `IndexNowKit::create()`. See [docs/operations.md](operations.md) for the levels, the exact messages and a "my URL was not submitted" checklist. ## Declaring pages: `#[IndexNow]` `#[IndexNow]` is **repeatable**: one attribute per family of public URLs the object has. Exactly one source per rule — `route`, `resolver`, `via`, `url` or `urls`. Class-wide policy goes to `#[IndexNowDefaults]`, whose `when` is ANDed with each rule's own `when` (a draft page is never public, whatever the rule says). ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults, IndexNowUrl}; use IndexNowKit\Attribute\Param\{Accessor, Call, Formatted, Placeholder, Value}; #[IndexNowDefaults(when: 'isPublished', fields: ['slug', 'title', 'body', 'published'])] #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] // the article page #[IndexNow(route: 'post_amp', params: ['slug' => 'slug'], when: 'hasAmp', whenFields: ['ampEnabled'])] #[IndexNow(via: 'category')] // resubmit the category page #[IndexNow(via: 'tags')] // and every tag page #[IndexNow(urls: ['/', '/blog'])] // and two literal URLs class Post {} ``` Typed parameter sources, next to the plain accessor string (property, getter, `is`/`has` method, `dotted.path`, `self`): ```php #[IndexNow(route: 'post_show', params: [ 'year' => new Formatted('publishedAt', 'Y'), // DateTimeInterface::format() 'cat' => 'category.slug', // dotted path through a relation 'section' => new Value('blog'), // a constant 'slug' => new Call('slugFor', Placeholder::Locale), // a method call, one URL per locale ])] ``` Other shapes, all real cases: ```php #[IndexNow(url: 'publicUrl')] // a property or method returning string|iterable|null #[IndexNow(resolver: SyliusChannelUrls::class)] // a UrlResolverInterface class or service id #[IndexNow(route: 'page_show', params: ['slug' => 'slug'], host: new Accessor('tenant.domain'))] // multi-domain #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], locales: 'all')] // localized routes class Page {} class Offer { #[IndexNowUrl(when: 'isLive')] // the get_absolute_url() convention public function getPublicUrl(): string { return '/offers/' . $this->code; } } ``` Rules are inherited from parent classes and identified by `name` (derived from the source, or given explicitly): a subclass rule whose name repeats an ancestor's **replaces** it, a new name **adds** a page. ### Deletion semantics Visibility (`when`) is evaluated per rule, before and after a change. `true → false` submits that rule's URLs as a **deletion** so engines recrawl the 404; `false → true` is a creation; no transition is an update filtered by `fields`. Deleting an object whose rule does not apply submits nothing: the page was never public. `when` is often a getter (`isPublished`) while the ORM change set holds the field (`published`). The convention `isPublished → published`/`is_published` and `getStatus → status` is applied automatically; when the names are unrelated, name the backing fields with `whenFields`. A status string or enum is not a boolean: use `when: new Equals('status', 'published')` (`IndexNowKit\Attribute\Param\Equals`); rules registered at runtime may pass a closure. Full model, semantics table and the adapter-facing types (`UrlRule`, `RuleSet`, `RuleRegistry`): [docs/attribute-reference.md](attribute-reference.md). ```php $indexNow = IndexNowKit::create($config, resolver: new AttributeUrlResolver(new AttributeReader(), ParamExtractor::plain(), $router, $locator)); $indexNow->submitEntity($post, IndexNowKit\Event::Updated); $indexNow->submitEntities($posts); // many objects, de-duplicated, one request per host and batch $urls = $indexNow->urlsFor($post, Event::Deleted); // resolve without sending $rows = $indexNow->explain($post, Event::Updated); // ResolvedUrl: which rule produced which URL ``` `urlsFor()`, `explain()` and `submitEntity()` go through `GuardedUrlResolver`, which never throws: an invalid attribute is logged and yields no URLs, so a typo cannot break a flush. ## Configuration | Option | Env | Default | Meaning | |---|---|---|---| | `enabled` | `INDEXNOW_ENABLED` | `true` | `false` drops every submission (logged at `info`) | | `key` | `INDEXNOW_KEY` | — | default key, used for every host not listed in `hosts` | | `hosts` | `INDEXNOW_HOSTS` (`a.com=KEY1,b.com=KEY2`) | `[]` | per-host `{key, key_location, base_url}` | | `strict_hosts` | `INDEXNOW_STRICT_HOSTS` | `false` | apply the default key only to the `base_url` host | | `base_url` | `INDEXNOW_BASE_URL` | `null` | resolves relative URLs; required outside HTTP requests | | `engines` | `INDEXNOW_ENGINES` | `['api']` | engine names or custom `https://` endpoints | | `dispatch` | `INDEXNOW_DISPATCH` | `sync` | adapter-defined delivery mode; the core only reports it | | `batch.max_urls` | `INDEXNOW_BATCH_MAX_URLS` | `10000` | URLs per request: the protocol's ceiling, not a target | | `debounce.per_url` | `INDEXNOW_DEBOUNCE_PER_URL` | `600` | seconds before the same URL is sent again (`0` = off) | | `throttle.max_requests_per_minute` | `INDEXNOW_THROTTLE_PER_MINUTE` | `60` | per-process request rate (`0` = unlimited) | | `http.timeout` | `INDEXNOW_HTTP_TIMEOUT` | `10.0` | seconds, applied to clients created by discovery | | `dry_run` | `INDEXNOW_DRY_RUN` | `false` | log the request instead of sending it | | `environment` | `INDEXNOW_ENV` / `APP_ENV` | — | anything but `prod`/`production` without a key turns `dry_run` on | Also `key_file.enabled`, `http.user_agent` and `key_location`. Every value is validated at construction, so a bad setup fails at boot, not at the first submission. Full reference, per-host overrides, `Config::with()`, `Config::OPTIONS` and `unknownOptions()`: [docs/configuration.md](configuration.md). ## Retries, queues and bulk No retries inside a web request: `429`/`5xx` come back as `retryable` results. Use `RetryingSubmitter` in CLI, cron and workers, or re-enqueue `Result::retryableUrls($results)` after `(new RetryPolicy())->delayAfter($results, $attempt)` seconds. Collect during a unit of work, deliver once: ```php $indexNow->collect(['/posts/1', '/posts/2']); // anywhere during the request $indexNow->flush(); // at the end of the unit of work ``` See [docs/retries-and-queues.md](retries-and-queues.md) for the worker recipe and bulk/migration guidance. Re-announcing a bulk change from the site's own URL list is the job of the add-on package in the family table (Install); `$kit->transport` is the transport such consumers read through. Adapters prove their wiring with `Testing\Conformance\CoreConformanceTestCase`: extend it, return the facade your container built and its `FakeTransport`, and the protocol scenarios of the spec run against it. ## Testing `IndexNowKit\Testing` is part of the published package: `FakeTransport` (records POSTs, answers queued responses), `ArrayLogger`, `FrozenClock`, `RecordingDispatcher`. ```php $transport = new FakeTransport(); $indexNow = IndexNowKit::create($config, transport: $transport, debounce: new NullDebounceStore()); $indexNow->submitEntity($post); self::assertSame(['https://www.example.com/posts/hello'], $transport->posts[0]['body']['urlList']); ``` More recipes in [docs/testing.md](testing.md). ## Extension points | Interface | Default | Replace it to | |---|---|---| | `Http\TransportInterface` | `Psr18Transport::discover()` | use your own HTTP stack (`LazyTransport` defers building it) | | `Key\KeyProviderInterface` | `StaticKeyProvider` | keys from a database, per tenant | | `Url\UrlNormalizerInterface` | `UrlNormalizer` | strip tracking parameters, enforce trailing slashes, map hosts | | `Url\UrlResolverInterface` | `NullUrlResolver` — build an `AttributeUrlResolver` and pass it as `resolver:` | turn objects into URLs your way | | `Url\RouteUrlResolverInterface` | — (adapter-provided) | bridge your framework's router | | `Attribute\AttributeReaderInterface` | `AttributeReader` | `RuleRegistry` for runtime rules, or your own metadata source | | `Collector\CollectorInterface` | `Collector` | a durable outbox, a per-tenant buffer | | `Debounce\DebounceStoreInterface` | `MemoryDebounceStore` | `Psr16DebounceStore`, or your own | | `Throttle\ThrottleInterface` | `TokenBucket` | `NullThrottle`, a shared limiter | | `Dispatch\DispatcherInterface` | `SyncDispatcher` | `CallableDispatcher` for a queue, `NullDispatcher` | | `SubmitterInterface` | `Submitter` | decorate (`RetryingSubmitter`), record, mock | Pass any of them to `IndexNowKit::create()` by name, or assemble the graph by hand: `Client` → `Submitter` → `Collector` + `DispatcherInterface` → `IndexNowKit`. The pieces a framework adapter wires from its configuration have factories with one source of error texts — `Http\TransportFactory::lazy()` (`http.client`), `Debounce\DebounceStoreFactory::fromConfig()` (`debounce.store`), `Dispatch\DispatcherFactory::fromConfig()` (`dispatch`), `fromConfig()` on `Collector`, `TokenBucket`, `AttributeUrlResolver` and `KeyFileResponder` — and `Adapter\ConfigFactory` turns a raw framework array into a `Config` without ever throwing from a hook. A container that assembles at runtime describes the whole graph once with `Adapter\ServicesBuilder` and gets it lazily from `Adapter\Services`; ORM hooks share `Hook\ObserverHelper`, queue jobs `Retry\WorkerOutcome`, commands the runners and `Console\Definitions` of `indexnowkit/console`. Writing an adapter? [docs/adapters.md](adapters.md). ## Exceptions All exceptions implement `IndexNowKit\Exception\IndexNowException`: `ConfigurationException` (invalid `Config`, attribute or resolver setup), `InvalidUrlException` (a URL that cannot be submitted, caught by `Submitter` and dropped with a warning), `InvalidArgumentException` (programming errors) and `Http\Exception\TransportException` (network failure, turned into a retryable `Result` by `Client`; consumers reading documents through the transport see it; `Checker` turns it into an error line). Nothing throws out of a lifecycle hook — see the error contract in [docs/adapters.md](adapters.md). ## Limitations - The same URL is not re-sent within `debounce.per_url` (10 minutes by default): that is what Yandex asks for. - No retries inside a web request; `TokenBucket` throttles per process. Multi-process limits belong to your queue. - Only `http(s)` URLs on hosts you hold a key for. Sub-domains are separate hosts, each with its own key file. - Bulk ORM operations bypass entity hooks in every adapter: submit those URLs yourself. - Google is not reachable through IndexNow. ## Requirements PHP 8.2+, `ext-json`, `ext-filter`, a PSR-18 client with PSR-17 factories (`symfony/http-client` and Guzzle are configured automatically with the timeout and no redirects; other clients are used as is). Optional: `ext-intl` (IDN via UTS #46, otherwise a pure-PHP punycode encoder). ## Versioning SemVer. Before 1.0, minor versions may contain breaking changes; they are listed in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/core/CHANGELOG.md). What is covered by the promise and what is not: [docs/bc.md](bc.md). ## Notes for AI assistants - Composer package `indexnowkit/core` (PHP ^8.2, PSR-18 client + PSR-17 factories needed). Framework users want an adapter instead: `indexnowkit/symfony-bundle`, `indexnowkit/laravel`, `indexnowkit/yii2`, `indexnowkit/yii3`, `indexnowkit/doctrine`. - Minimal complete snippet (every `use` included): ```php use IndexNowKit\Config; use IndexNowKit\IndexNowKit; $indexNow = IndexNowKit::create(Config::fromEnv()); // INDEXNOW_KEY, INDEXNOW_BASE_URL, INDEXNOW_DRY_RUN $results = $indexNow->submit(['/posts/hello']); // list, never throws for remote errors ``` - Verify: `(new IndexNowKit\Check\Checker($config, $indexNow->keys, $indexNow->transport))->run()` is what the adapters' `check` command runs; `$indexNow->explain($object)` shows the rule behind every URL; every remote outcome is a `Result` with `status`/`reason`, nothing throws. - Pitfalls: - `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`); Yii3 has `sync` | `none` only. - Locales: `router.locales` in Laravel, Yii2 and Yii3, `framework.enabled_locales` in Symfony; `locales: 'all'` on a rule uses that list. - `url:` names an accessor (method or property) that returns the URL; `urls:` is a list of literal URLs. Never put a literal in `url:`. - A string in `when:` is an accessor read as truthy (`published`, `isPublished`). A status string needs `Equals`: `when: new Equals('status', 'published')` (`IndexNowKit\Attribute\Param\Equals`). - Manual submission is `submitEntity()` in Symfony, `submitModel()` in Laravel, `submitRecord()` in Yii2 and Yii3; the commands are `indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record` (Yii2), `indexnow:submit-record` (Yii3). Bulk queries (`update()`, `DB::table()`, `updateAll()`) fire no hooks: submit afterwards with those. - Laravel has two classes called `IndexNowKit`: the facade `IndexNowKit\Laravel\Facades\IndexNowKit` and the core service `IndexNowKit\IndexNowKit` (inject by type). Yii2 exposes the core through `Yii::$app->indexnow->kit()`; Yii3 defines `IndexNowKit\IndexNowKit` in the container. - Outside production a configured key with `dry_run` unset makes `check` fail (a staging copy would submit real URLs): set `dry_run: true` there, or `dry_run: false` explicitly when it submits on purpose. - Unknown configuration keys are warned about at boot (typos such as debounce.per_urls); the key list is `Config::OPTIONS` plus the adapter's own keys. ## Other packages | | | |---|---| | PHP | the family table under [Install](#install) | | JS/TS | `@indexnowkit/core`, `next`, `prisma` (planned) | | Python | `indexnowkit`, `indexnowkit-django` (planned) | Design rationale and the cross-language model: [docs/spec](https://github.com/indexnowkit/spec). Conformance suite: [indexnowkit/spec](https://github.com/indexnowkit/spec). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Attribute reference [Русская версия](attribute-reference.ru.md) A class declares a **list of rules**, one per family of public URLs it has. PHP writes them as attributes; every other language in the family writes decorators or config objects. All of them compile down to the same `IndexNowKit\Attribute\UrlRule`, and everything downstream — event classification, guards, locales, `via` delegation, deduplication, `explain` output — consumes only that. ## The three attributes | Attribute | Target | Purpose | |---|---|---| | `#[IndexNow]` | class, **repeatable** | one URL rule | | `#[IndexNowDefaults]` | class | policy shared by every rule of the class and its subclasses | | `#[IndexNowUrl]` | public method | the method's return value is a URL family (the `get_absolute_url()` convention) | ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults, IndexNowUrl}; #[IndexNowDefaults(when: 'isPublished', fields: ['slug', 'title', 'body', 'published'])] #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] #[IndexNow(route: 'post_amp', params: ['slug' => 'slug'], when: 'hasAmp', whenFields: ['ampEnabled'])] class Post {} ``` ## Sources Exactly one source per `#[IndexNow]`. Zero sources, or two, throws `ConfigurationException` at compile time with a message naming the offenders. | Source | Value | Produces | |---|---|---| | `route` | framework route name | one URL per locale, generated by the adapter's `RouteUrlResolverInterface` | | `resolver` | `UrlResolverInterface` class name or service id | whatever the resolver returns | | `via` | accessor to a related object or collection | the related objects' own URLs, resolved as updates | | `url` | accessor returning `string`, `iterable` or `null` | those URLs | | `urls` | list of literal URLs | those URLs, absolute or `base_url`-relative | `url` and `urls` are easy to swap, so both are checked: `url: '/about'` and `urls: ['aboutUrl']` are rejected with a message telling you which one you meant. `resolver` needs a `ResolverLocatorInterface`. In plain PHP that is `ArrayResolverLocator`, which also instantiates a class name on demand as long as its constructor takes no required arguments. Framework adapters look the id up in the container. ## Parameters `params` maps a route parameter name to a source. A plain string is the accessor DSL; anything else is one of four typed `Param\ParamValue` objects. ### The accessor DSL Resolved in this order, on the object itself: 1. `'self'` — the object (route model binding: `params: ['post' => 'self']`); 2. a dotted path — each segment resolved recursively (`'category.slug'`); a non-object segment throws; 3. a method with that exact name; 4. `get`, `is` or `has` plus the capitalised name (`'published'` finds `getPublished()`, then `isPublished()`, then `hasPublished()`); 5. a property, including a private one. Nothing matched throws `ConfigurationException` naming the accessor and the class. ### Typed sources | Class | Example | Meaning | |---|---|---| | `Param\Accessor` | `new Accessor('category.slug')` | the explicit form of a plain accessor string | | `Param\Value` | `new Value('html')` | a constant | | `Param\Formatted` | `new Formatted('publishedAt', 'Y')` | `DateTimeInterface::format()` of the accessor's value | | `Param\Call` | `new Call('slugFor', Placeholder::Locale)` | a method call; extra arguments are passed as given | `Param\Placeholder::Locale` and `Param\Placeholder::Host` are substituted per generated URL, so a `Call` can return a per-locale slug or a per-tenant path. Extraction runs once per URL, not once per rule. ### Coercion Route parameters must be usable in a URL. The extractor accepts `null` and scalars as they are, unwraps a `BackedEnum` to its `value`, casts a `Stringable` value object to string, and passes plain objects through for route model binding. A bare `DateTimeInterface` is rejected with a message pointing at `new Formatted(...)`, because formatting a date implicitly is how a URL silently changes shape. Anything else throws. ## Rule options | Option | Type | Default | Meaning | |---|---|---|---| | `when` | accessor name, a `Condition` (`new Equals(path, value)` or your own), or a closure `fn(object): bool` (runtime rules only) | inherit | the page exists only while the condition holds | | `whenFields` | list of field names | `[]` | fields backing this rule's own `when` when its name does not match the field (a class-level `when` has its own `whenFields` in `#[IndexNowDefaults]`) | | `fields` | list of field names, or `null` | inherit, then `[]` | for updates only: submit when one of these changed; `[]` = any field | | `events` | subset of `created`, `updated`, `deleted` (strings or `Event` cases), or `null` | inherit, then all three | which lifecycle events the rule listens to | | `locales` | `'current'`, `'all'` or a list, or `null` | inherit, then `'current'` | locale expansion for localized routes | | `host` | string or `ParamValue` | `null` | generate this rule's URLs on that host (multi-domain) | | `name` | string | derived | stable rule id for logs, `explain` output and subclass overrides | `when` is a **conjunction**: the class-level `when` and the rule's own `when` must both hold. `fields`, `events` and `locales` are defaults a rule overrides; `null` means inherit, `[]` means "no filter". An accessor string is checked for truthiness, which is right for booleans and wrong for a status string (`'draft'` is truthy). For string or enum states use `Equals`, which also gives exact old-state detection from the ORM change set: ```php use IndexNowKit\Attribute\Param\Equals; #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], when: new Equals('status', 'published'))] #[IndexNow(route: 'job_show', params: ['id' => 'id'], when: new Equals('state', JobState::Open))] // BackedEnum or its value ``` Rules registered at runtime (`RuleRegistry`) may pass a closure: `when: fn (WP_Post $p): bool => $p->post_status === 'publish'`. A closure's old value cannot be reconstructed, so list the fields it reads in `whenFields`; a change of one of them is treated as a visibility flip (see the semantics table). ### Your own conditions Two interfaces, by what the condition reads. `Attribute\Param\FieldCondition` (`field()`, `heldFor(mixed $oldValue)`) reads one field: the core reads it through the graph's `ParamExtractor` (its readers see Eloquent attributes) and asks `heldFor()`, for the current value and, from the ORM change set, for the old one — `Equals` is the shipped one. `Attribute\Param\Condition` (`evaluate(object $subject): bool`) looks at the whole object itself, for what one field cannot say. Either goes in `when` (an attribute argument must be a constant expression, so a condition class with a constructor of scalars, not a closure): ```php use IndexNowKit\Attribute\Param\Condition; use IndexNowKit\Attribute\Param\FieldCondition; final readonly class OneOf implements FieldCondition // reads one field: the classifier sees the old state { /** @param list $values */ public function __construct(private string $path, private array $values) {} public function field(): string { return $this->path; } public function heldFor(mixed $oldValue): bool { return in_array($oldValue, $this->values, true); } } final readonly class Sellable implements Condition // three fields at once: reads the object itself { public function evaluate(object $subject): bool { return $subject instanceof Offer && $subject->stock > 0 && $subject->price !== null && !$subject->hidden; } } #[IndexNow(route: 'offer_show', params: ['id' => 'id'], when: new OneOf('state', ['open', 'reserved']))] ``` A `Condition` has no old value: `ChangeClassifier` evaluates it on the current object, so `open → closed` is classified as a plain update, not as the deletion it is — unless `whenFields` names the fields the condition reads (then a change of one of them counts as a flip). Implement `FieldCondition` when the condition reads one field, and the change set gives the exact old state, as it does for `Equals`. Neither interface is evaluated by hand: the graph's extractor does it (`$indexNow->extractor->condition($subject, new Equals('status', 'published'))`): a `FieldCondition` as `heldFor()` of the value the extractor reads for `field()`, so it sees Eloquent and Active Record attributes through the adapter's readers; a `Condition` as its own `evaluate()`. `Condition` and `FieldCondition` are in the Implement tier of [bc.md](bc.md), with the pre-1.0 caveat that they are new in 0.8 (and siblings since 0.12: a field condition has no `evaluate()`). `Equals` is a condition, not a value source: `params: ['status' => new Equals(...)]` is a type error, and `ParamExtractor` names the fix. `explain` prints every condition with the value it read (`when: status ("draft") -> true — a non-empty string is truthy; use new Equals('status', "draft")`), and `explain --json` gives the same walk as a document. An unknown event name throws `ConfigurationException` naming the attribute and the value. ### Rule names Derived from the source when not given: the route name; `resolver:`; `via:`; `url:` (and `url:` for `#[IndexNowUrl]`); `urls:`. Two rules of the same class that would derive the same name get `#2`, `#3` appended in declaration order. Give an explicit `name` whenever you intend a subclass to override a specific rule, or whenever the derived name would be unstable. ## Class defaults and inheritance The compiler walks the class hierarchy **root first**, then the leaf. - `#[IndexNowDefaults]` merges field by field, the nearest declaration wins. A declaration that sets its own `when` also replaces the inherited `whenFields`; one that does not adds to them. - Rules accumulate. A rule whose name repeats an ancestor's **replaces** it; a new name **adds** a page. That is how a subclass changes one page without restating the others. - `#[IndexNowUrl]` is read on public methods declared by each class in the chain, so an override in a subclass wins. The method must not require arguments. - Interfaces and traits are **not** scanned: PHP does not inherit class attributes through them, and Doctrine mapping behaves the same way. ```php #[IndexNowDefaults(when: 'isPublished')] #[IndexNow(route: 'content_show', params: ['slug' => 'slug'])] abstract class Content {} #[IndexNow(route: 'content_show', params: ['slug' => 'slug', 'section' => new Value('news')])] // replaces #[IndexNow(route: 'news_amp', params: ['slug' => 'slug'])] // adds class News extends Content {} ``` A hierarchy where only some subclasses have public pages should carry no rules on the base class: what is not declared is not inherited. ## Semantics: event, before, after Visibility is evaluated per rule. `W` is the conjunction of the class `when` and the rule's `when`; `W_before` is reconstructed from the ORM change set. | ORM event | `W_before` | `W_after` | `fields` match | Rule event | State the URL is built from | |---|---|---|---|---|---| | insert | — | true | — | `Created` if subscribed | new state, after the write (ids assigned) | | insert | — | false | — | none | — | | update | true | true | yes | `Updated` if subscribed | new state, after the write | | update | true | true | no | none | — | | update | true | false | ignored | **`Deleted`** if subscribed | current state, before the write | | update | false | true | ignored | `Created` if subscribed | new state, after the write | | update | false | false | — | none | — | | delete | — | true | — | `Deleted` if subscribed | pre-delete state | | delete | — | false | — | none | — | | `via` target, any event | — | true | per target rule | target resolved as `Updated` | the target's own rules and guards | | `via` target, any event | — | false | — | none | — | Two consequences worth stating out loud. `fields` never suppresses a visibility transition, only a plain update: a page that just went dark is announced whichever field did it. And deleting an object whose rule does not apply submits nothing — that page was never public, so purging drafts stays quiet. ### Reconstructing `W_before` `ChangeClassifier::classify(UrlRule $rule, object $subject, ParamExtractor $extractor, array $changedFields, array $changeSet = [])` returns the `Event` a rule cares about, or `null`. Old-state visibility is best effort, in three tiers: 1. A `when` accessor whose backing field is present in the change set is evaluated **exactly** from the old value. The backing field is found by name, then by convention: `isPublished → published → is_published`, `hasAmp → amp → has_amp`, `getStatus → status`. `UrlRule::fieldCandidates()` exposes that list. 2. An accessor with no change-set entry, but a field it depends on (its candidates, or a declared `whenFields` entry) among the changed fields, is assumed to have **flipped**. A false positive costs one request; a false negative leaves a dead page in the index. 3. Otherwise the accessor keeps its current value. Name the fields with `whenFields` when the accessor is a method whose name has nothing to do with the column, for example `when: 'isVisibleToPublic', whenFields: ['status', 'visibleFrom']`. ## `via` `via` resubmits a related object's pages: a changed comment refreshes its post, a changed product refreshes its category. Targets are always resolved as `Updated`, because their page exists regardless of what happened to the source. Depth is capped at 3 and fan-out at 100 related objects per rule (constructor arguments `maxViaDepth` and `maxViaFanout` of `AttributeUrlResolver`); exceeding the depth throws, exceeding the fan-out logs a warning and stops. A target rule that delegates back through the same accessor name is skipped, so `A -> B -> A` terminates. Resulting URLs keep the whole chain in their rule name: `via:category -> category_show`. ## Field names `fields` and `whenFields` are **model field names** as the developer writes them, never database columns. Doctrine's `getEntityChangeSet()` gives exactly those. A declared field matches a changed one when they are equal or one is a dotted prefix of the other, so `fields: ['address']` catches an embeddable change reported as `address.city`. ## Types adapters consume ```php final readonly class UrlRule { public string $name; public RuleSource $source; // Route|Resolver|Via|Url|Urls public ?string $route; public array $params; public ?string $resolver; public ?string $via; public ?string $url; public array $urls; public array $when; public array $whenFields; public array $fields; public array $events; public array|string $locales; public string|ParamValue|null $host; public function listensTo(Event $event): bool; public function caresAbout(array $changedFields): bool; public function appliesTo(object $subject): bool; // every `when` accessor is truthy public function whenDependsOn(string $field): bool; public static function fieldCandidates(string $accessor): array; } ``` `RuleSet` is every rule of one class in declaration order (parents first). It is `Countable` and iterable, empty for classes without rules so callers never branch on null, and offers `isEmpty()`, `get(string $name)` and `listensTo(Event $event)` as a cheap pre-filter for ORM hooks. `AttributeReaderInterface::rules(string|object $classOrObject): RuleSet` is the lookup. The default `AttributeReader` compiles attributes through `RuleCompiler` and caches per class for the process lifetime. It throws `ConfigurationException` on a malformed declaration — ORM hooks must read through `ObjectChangeHandler` or `GuardedUrlResolver`, which log instead. `ResolvedUrl` carries provenance for `explain` output, logs and profiler panels: `url`, `rule`, `class`, `event`, `locale`, plus `source()` (`App\Entity\Post#post_amp`) and `ResolvedUrl::urls()` to flatten a list to deduplicated strings. ## Rules registered at runtime Models that cannot carry attributes — CMS post types, classes you do not own, a closure API — use `RuleRegistry`, which implements `AttributeReaderInterface` on top of an inner reader (attributes by default). ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults, RuleRegistry}; $registry = new RuleRegistry(); $registry->register(Post::class, [ new IndexNow(route: 'posts.show', params: ['post' => 'self']), new IndexNow(urls: ['/']), ], new IndexNowDefaults(when: 'isPublished')); $registry->register(WP_Post::class, [new IndexNow(resolver: 'wp_permalink')], new IndexNowDefaults( when: fn (WP_Post $post): bool => $post->post_status === 'publish', // or: new Equals('post_status', 'publish') whenFields: ['post_status'], )); $registry->registerFor(CmsPage::class, fn (CmsPage $page): ?RuleSet => $page->isSystem() ? null : $rulesFor($page)); $indexNow = IndexNowKit::create($config, attributes: $registry); ``` `register()` compiles attribute instances built in code, with no reflection. `registerFor()` decides per object and may return `null` to fall through to the inner reader. Registered rules replace whatever the inner reader would return for that class, and subclasses inherit them. ## Resolving without an ORM `AttributeUrlResolver` resolves every rule of a class through its source; `GuardedUrlResolver` wraps it so nothing throws. `ObjectChangeHandler` is the piece ORM hooks build on: it classifies a created, updated or deleted object per rule and resolves the URLs, logging every silent outcome. See [adapters.md](adapters.md). ## Anti-patterns Five declarations that compile, run, and submit the wrong thing. **1. A literal URL in `url:`.** `url:` names an accessor; `urls:` lists literals. ```php #[IndexNow(url: '/')] // wrong: reads a property or method called "/" #[IndexNow(urls: ['/'])] // right #[IndexNow(url: 'canonicalUrl')] // right: $post->canonicalUrl() or ->canonicalUrl ``` **2. A status string in `when:`.** A string is an accessor read as truthy, so `'published'` means "the attribute `published` is truthy" — a `status` column holding `'draft'` is truthy too. `explain` shows the value it read and says so. ```php #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], when: 'status')] // wrong: 'draft' is truthy #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], when: new Equals('status', 'published'))] // right #[IndexNow(route: 'post_show', params: ['slug' => 'slug', 'v' => new Equals('status', 'published')])] // wrong: a condition is not a param value (type error) ``` **2b. A custom `Condition` on a field that flips.** `when: new Published()` that reads `status` internally cannot tell the classifier what the old state was: `published → draft` is an update, the dead page stays indexed. Implement `FieldCondition`, or name the field in `whenFields`. **3. A rule on a page the engine must not index.** A preview, an admin page, a page with `noindex` or a `robots.txt` disallow: the engine fetches it, finds it unindexable, and counts a mistake against the key. The [`indexnowkit/verify`](../verify/index.md) add-on catches what the rule lets through (one GET before submission; `check --sample` for a dry report) — the rule is still the right place. ```php #[IndexNow(route: 'post_preview', params: ['slug' => 'slug'])] // wrong: preview pages carry noindex #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], when: 'isPublished')] // right: the public page, guarded ``` **4. Non-canonical URLs.** Filter and sort variants, tracking parameters, the apex next to `www`, `http` next to `https`: submit the canonical page once. Generate URLs through the router with `base_url` on the canonical origin; do not build them by string concatenation from the request. ```php #[IndexNow(urls: ['/products?sort=price&utm_source=indexnow'])] // wrong: a variant, and a tracking parameter #[IndexNow(route: 'products_index')] // right: the canonical listing ``` **5. No `when` on a model that has drafts.** Without a guard every save submits, drafts included, and a page taken down is announced as an update, not a deletion. ```php #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] // wrong when Post has a draft state #[IndexNowDefaults(when: 'isPublished')] // right: drafts are skipped, #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] // published → draft is a deletion ``` What the library checks for you: the URL is absolute `http(s)`, has no credentials, fragment or control characters, and belongs to a host you hold a key for (`strict_hosts`). What it cannot check: `noindex`, `robots.txt`, a canonical pointing elsewhere, the status code the page answers — that is the rule author's job today, and the job of the `verify` add-on (`check --sample`) in a later release. # Configuration [Русская версия](configuration.ru.md) `IndexNowKit\Config` is an immutable value object shared by every adapter. It is built in one of three ways and validated in the constructor, so a broken setup fails at boot instead of at the first submission. ```php use IndexNowKit\Config; $config = Config::fromArray([...]); // framework config files $config = Config::fromEnv(); // INDEXNOW_* environment variables $config = new Config(key: '...', baseUrl: '...'); // named arguments $config = $config->with(dryRun: true); // immutable copy ``` ## Options `fromArray()` takes the nested shape below; it is the canonical schema every language adapter mirrors. ```php Config::fromArray([ 'enabled' => true, 'key' => $_ENV['INDEXNOW_KEY'], 'hosts' => [ 'www.example.com' => 'KEY-FOR-EXAMPLE', 'shop.example.com' => [ 'key' => 'KEY-FOR-SHOP', 'key_location' => 'https://shop.example.com/keys/indexnow.txt', 'base_url' => 'https://shop.example.com', ], ], 'strict_hosts' => true, 'key_location' => null, 'base_url' => 'https://www.example.com', 'engines' => ['api'], 'dispatch' => 'sync', 'batch' => ['max_urls' => 10000], 'debounce' => ['per_url' => 600], 'throttle' => ['max_requests_per_minute' => 60], 'http' => ['timeout' => 10.0, 'user_agent' => null], 'serve_key_file' => true, 'dry_run' => false, 'environment' => $_ENV['APP_ENV'] ?? null, ]); ``` | Option | Constructor argument | Default | Meaning | |---|---|---|---| | `enabled` | `enabled` | `true` | `false` drops every submission; the URLs come back as `skipped` results with reason `disabled`, logged at `info` | | `key` | `key` | `null` | default key, 8-128 characters of `[A-Za-z0-9-]`, used for every host not listed in `hosts` | | `hosts` | `hosts` | `[]` | `host => key`, or `host => {key, key_location?, base_url?}` | | `strict_hosts` | `strictHosts` | `false` | apply the default key **only** to the `base_url` host; every other host needs a `hosts` entry or its URLs are skipped | | `key_location` | `keyLocation` | `null` | absolute URL of the key file when it is not `https://{host}/{key}.txt` | | `base_url` | `baseUrl` | `null` | absolute site URL; resolves relative URLs and is required outside HTTP requests | | `engines` | `engines` | `['api']` | engine names (`api`, `yandex`, `bing`, `naver`, `seznam`, `yep`, `internetarchive`, `amazon`) or full endpoint URLs | | `dispatch` | `dispatch` | `'sync'` | delivery mode defined by the adapter; the core validates the identifier and reports it | | `batch.max_urls` | `batchMaxUrls` | `10000` | URLs per request; `Config::MAX_BATCH_URLS` is the protocol maximum — a ceiling, not a target: smaller batches are accepted just as well | | `debounce.per_url` | `debouncePerUrl` | `600` | seconds during which the same URL is not re-sent; `0` disables debouncing | | `throttle.max_requests_per_minute` | `throttleMaxRequestsPerMinute` | `60` | outgoing requests per minute, per process; `0` = unlimited | | `http.timeout` | `httpTimeout` | `10.0` | seconds, applied only to clients the library creates itself | | `http.user_agent` | `userAgent` | `null` | overrides `indexnowkit-php/ (+https://github.com/indexnowkit/php)` | | `key_file.enabled` | `serveKeyFile` | `true` | whether an adapter should answer `GET /{key}.txt`; `serve_key_file` is the deprecated name and wins when both are set | | `key_file.cache_max_age` | `keyFileMaxAge` / `keyFileHeaders()` | `300` | `Cache-Control: max-age` of the key file response; short on purpose, a cached old file turns every submission into a 403 after a rotation. `keyFileHeaders()` adds `Vary: Host` whenever the body depends on the host — a `hosts` map, **or** `strict_hosts`, where the default key is served for the base host only and every other host gets a 404; without the header a shared cache would keep whichever of the two answers came first | | `debounce.store` | `debounceStore` | `null` | `memory` (per process), `none`, or an id the adapter resolves to its shared cache; `null` = the adapter's default (Laravel `cache`, bundle `cache.app`, Yii2 `cache`, Yii3 the container id `Psr\SimpleCache\CacheInterface`, plain PHP `memory`) | | `http.client` | `httpClient` | `null` | id or class of a PSR-18 client the adapter resolves; `null` = discovery. It carries the application's own settings, so `check` warns when it is set (`http.client`): a client that follows redirects internally turns a 30x to a catch-all page into a 200 for the key file check. The pre-flight of `indexnowkit/verify` does **not** use it — a followed redirect would hide exactly the 3xx the pre-flight exists to see, so verify builds its own client from `verify.timeout` and `verify.max_redirects` | | `dry_run` | `dryRun` | `false` | log the request instead of sending it | | `environment` | `environment` | `null` | application environment; drives the non-production safety net below | | `production_environments` | `productionEnvironments` | `['prod', 'production']` | environment names (case-insensitive) that count as production; replaces the default list | | `previous_key` | `previousKey` | `null` | the key before a rotation: still accepted by the key file, never submitted; also `hosts..previous_key` | | `hosts..engines` | `hostEngines` / `endpointsFor()` | inherit `engines` | engines for one host only | | `engine_aliases` | `engineAliases` / `resolveEngine()` | `{}` | short names for custom endpoints, usable wherever an engine is named | | `locale_hosts` | `localeHosts` / `hostForLocale()` | `{}` | locale => host; rules with `locales` and no `host` generate each locale on its host | | `logging.max_body` | `logBody` | `300` | bytes of an engine response body kept in a failure log line | | `max_url_length` | `maxUrlLength` | `2048` | URLs above it are skipped as `invalid_url` | | `debounce.key_prefix` | `debounceKeyPrefix` | `'indexnowkit_'` | cache key prefix of a shared debounce store | | `logging.max_urls` | `logUrls` / `logSample()` | `20` | URLs listed in one log line; `0` = counts only | | `logging.forbidden_escalation` | `forbiddenEscalation` | `5` | consecutive 403s per host before the log escalates to `critical` | | `logging.levels` | `logLevels` / `logLevel()` | `{}` | per-outcome PSR-3 level overrides; events and defaults in `Config::LOG_EVENTS` | | `retry.max_attempts`, `retry.base_delay`, `retry.multiplier`, `retry.max_delay`, `retry.server_error_delay` | `retryPolicy()` | `3`, `60`, `2.0`, `3600`, `5` | the `RetryPolicy` for queue handlers and `RetryingSubmitter` | | `resolver.max_via_depth`, `resolver.max_via_fanout` | `resolverMaxViaDepth`, `resolverMaxViaFanout` | `3`, `100` | limits of `via:` traversal in `AttributeUrlResolver`. `IndexNowKit::create()` does not build that resolver: the adapter that does passes `resolverMaxViaDepth`, `resolverMaxViaFanout` and `localeHosts` to it | | `collector.max_urls` | `collectorMaxUrls` | `0` | `IndexNowKit::collect()` flushes early at this size; `0` = only on `flush()` | | `collector.detect_leaks` | `collectorDetectLeaks` | `true` | shutdown warning about collected, never flushed URLs | | `normalizer.strip_tracking_params` | `normalizerStripTrackingParams` | `true` | drop `utm_*`, `gclid`, `fbclid`, `yclid`, … (`Url\CanonicalUrlNormalizer::TRACKING_PARAMS`, a growing list) from the query before de-duplication, debounce and submission: external traffic sources append them, routing never generates them | | `normalizer.tracking_params` | `normalizerTrackingParams` | `[]` | more query parameters to drop: names (`ref`) or prefixes (`mtm_*`), case-insensitive | | `normalizer.trailing_slash` | `normalizerTrailingSlash` | `'keep'` | `keep` submits the path as generated; `add` ends every path without an extension with `/`; `strip` removes the trailing `/` except on the root. Only when the site has a canonical form: the two forms are different pages otherwise | | `normalizer.sort_query` | `normalizerSortQuery` | `false` | order the query parameters by name (stable), so `?b=1&a=2` and `?a=2&b=1` are one URL | The `normalizer.*` options are applied by `Url\UrlNormalizerFactory::fromConfig()`, which every adapter and `IndexNowKit::create()` use to build the normalizer: `Url\UrlNormalizer` (absolute URL, host, port, dot-segments) wrapped in `Url\CanonicalUrlNormalizer`. Turning `strip_tracking_params` on or off changes the debounce keys of URLs that carried such parameters once. What `Url\UrlNormalizer` does unconditionally, with no option behind it: the scheme and the host are lower-cased and an internationalized host becomes punycode, the default port is dropped, dot-segments are removed, the fragment is cut, and the percent-encoding of the path and the query is brought to the canonical form of RFC 3986 §6.2.2 — a percent-escape of an unreserved character (`A-Za-z0-9-._~`) becomes the character, every other escape is upper-cased, so `/%7Euser/a%2Db` and `/~user/a-b` are one URL and are debounced, submitted and recorded once. A URL with credentials, control characters or a non-`http(s)` scheme is rejected with `InvalidUrlException` instead. Constants worth referencing instead of hard-coding: `Config::MAX_BATCH_URLS` (10000), `Config::DEFAULT_BATCH_MAX_URLS`, `Config::DEFAULT_DEBOUNCE_PER_URL` (600), `Config::DEFAULT_THROTTLE_PER_MINUTE` (60), `Config::DEFAULT_HTTP_TIMEOUT` (10.0), `Config::PRODUCTION_ENVIRONMENTS` (`['prod', 'production']`), `Config::DEFAULT_MAX_URL_LENGTH`, `Config::DEFAULT_LOG_URLS`, `Config::DEFAULT_FORBIDDEN_ESCALATION`, `Config::DEFAULT_RETRY_*`, `Config::DEFAULT_RESOLVER_MAX_VIA_*`, `Config::LOG_EVENTS`. ## One concept, four keys The adapters share the core keys under the same names and add a few of their own; some concepts have a different key (or a different value set) per framework. The tables below are generated from the code (`bin/config-table`) and checked in CI, so they are the current truth; the prose of each adapter's `docs/configuration.md` explains the semantics. _Generated by `bin/config-table` from `Config::OPTIONS`, `SitemapConfig::OPTIONS`, the bundle configuration tree, `ConfigFactory::LARAVEL_OPTIONS`, `ConfigFactory::YII_OPTIONS` and `ConfigFactory::YII3_OPTIONS`; do not edit by hand._ ### Core keys: the same name in every adapter Every key of `Config::OPTIONS` is accepted under this name by the Symfony bundle (`indexnowkit:`), the Laravel package (`config/indexnow.php`), the Yii2 component (`options`) and the Yii3 params block (`indexnowkit/yii3`). The default column is the one the core ships, as the bundle declares it in its configuration tree (`—` = unset); the two exceptions are in the synonyms table: `dispatch` (`auto` in Symfony and Yii2, `queue` in Laravel, `sync` in Yii3) and `debounce.store` (`cache.app` / `cache` / `cache` / the PSR-16 `CacheInterface` of the container). `environment` comes from `kernel.environment` / `APP_ENV` / `YII_ENV` unless set. | Key | Default | |---|---| | `enabled` | `true` | | `key` | — | | `hosts` | `[]` | | `key_location` | — | | `base_url` | — | | `engines` | `[api]` | | `dispatch` | `auto` | | `serve_key_file` | deprecated alias of `key_file.enabled` | | `dry_run` | `false` | | `strict_hosts` | `false` | | `environment` | — | | `production_environments` | `[prod, production]` | | `max_url_length` | `2048` | | `previous_key` | — | | `key_file.enabled` | `true` | | `key_file.cache_max_age` | `300` | | `batch.max_urls` | `10000` | | `debounce.per_url` | `600` | | `debounce.key_prefix` | `indexnowkit_` | | `debounce.store` | `cache.app` | | `throttle.max_requests_per_minute` | `60` | | `http.timeout` | `10` | | `http.user_agent` | — | | `http.client` | — | | `logging.max_urls` | `20` | | `logging.forbidden_escalation` | `5` | | `logging.levels` | `[]` | | `logging.max_body` | `300` | | `engine_aliases` | `[]` | | `locale_hosts` | `[]` | | `retry.max_attempts` | `3` | | `retry.base_delay` | `60` | | `retry.multiplier` | `2` | | `retry.max_delay` | `3600` | | `retry.server_error_delay` | `5` | | `resolver.max_via_depth` | `3` | | `resolver.max_via_fanout` | `100` | | `collector.max_urls` | `0` | | `collector.detect_leaks` | `true` | | `normalizer.strip_tracking_params` | `true` | | `normalizer.tracking_params` | `[]` | | `normalizer.trailing_slash` | `keep` | | `normalizer.sort_query` | `false` | `hosts` (per-host keys, `hosts..{key, key_location, base_url, engines, previous_key}`) is accepted everywhere too. ### Sitemap keys (`indexnowkit/sitemap`) The `sitemap` block is the same in the four adapters and is owned by the sitemap package: `sitemap.enabled`, `sitemap.url`, `sitemap.max_depth`, `sitemap.max_sitemaps`, `sitemap.max_bytes`, `sitemap.allow_foreign_hosts`, `sitemap.spool`, `sitemap.spool_dir`, `sitemap.fetch_retries`. ### Verify keys (`indexnowkit/verify`) The `verify` block is the same in the four adapters and is owned by the verify package (its `docs/configuration.md` has the table): `verify.enabled`, `verify.redirect`, `verify.non_canonical`, `verify.origin_error`, `verify.delay`, `verify.timeout`, `verify.max_redirects`, `verify.max_batch`, `verify.time_budget`, `verify.robots_cache_ttl`, `verify.user_agent`. ### History keys (`indexnowkit/history`) The `history` block is the same in the four adapters and is owned by the history package (its `docs/configuration.md` has the table): `history.store`, `history.limit`, `history.key_prefix`, `history.pdo.dsn`, `history.pdo.service`, `history.pdo.table`, `history.retention_days`. ### One concept, four keys | Concept | Symfony (`indexnowkit:`) | Laravel (`config/indexnow.php`) | Yii2 (`options`) | Yii3 (`indexnowkit/yii3` params) | Notes | |---|---|---|---|---|---| | Delivery mode | `dispatch` | `dispatch` | `dispatch` | `dispatch` | `auto` (Messenger when a transport is set, else `sync`), `messenger`, `sync`, `none` — Symfony; `queue` (default), `sync`, `none` — Laravel, no `auto`; `auto` (default: `queue` when the queue component exists, else `sync`), `queue`, `sync`, `none` — Yii2; `sync` (default), `none` — Yii3 (a queue is a replaced `DispatcherInterface`) | | Queue / transport | `messenger.transport` | `queue.connection` | `queue.component` | — | Symfony: a `framework.messenger.transports` name (the bundle routes `SubmitUrlsMessage` to it); Laravel: a `queue.connections` name (default: the app default); Yii2: the yii2-queue component id (default `queue`); Yii3: none until yiisoft/queue is released | | Queue delay / extras | `messenger.delay` | `queue.delay` | `queue.delay` | — | Symfony also `messenger.stamps`, `messenger.bus`; Laravel also `queue.queue`; Yii2 also `queue.ttr`, `queue.priority` | | Locales for `locales: all` | `framework.enabled_locales` | `router.locales` | `router.locales` | `router.locales` | Symfony reads the framework setting; Laravel, Yii2 and Yii3 list them in the package configuration (`router.locale_parameter` names the route parameter, `_language` in Yii3; `router.set_app_locale` switches the application locale while generating in Laravel and Yii2; Yii2 read `router.languages` / `language_parameter` / `set_app_language` before 0.12 and still accepts them) | | ORM hook switch | `doctrine.enabled` | `eloquent.enabled` | `active_record.enabled` | `active_record.enabled` | Symfony also `doctrine.listener_priority`, `doctrine.connections`; Yii2 and Yii3 also `active_record.models` (classes you cannot annotate); Yii3 also `active_record.namespaces` (short class names of the commands) | | Key file route | `key_file.path` | `key_file.path` | `key_file.pattern` | `key_file.pattern` | Symfony/Laravel: a path with `{key}` (default `/{key}.txt`); Yii2: a URL rule pattern (default `.txt`); Yii3: a yiisoft/router pattern (default `/{key:[A-Za-z0-9-]{8,128}}.txt`); all four: `key_file.enabled`, `key_file.cache_max_age`; Symfony/Laravel also `key_file.host`, `key_file.route_name`; Laravel also `key_file.middleware` | | Log destination | `logging.channel` | `logging.channel` | `logging.category` | `logging.category` | Monolog channel (Symfony, default `indexnow`), log channel name (Laravel), Yii log category (Yii2 and Yii3, default `indexnow`) | | Debounce store | `debounce.store` | `debounce.store` | `debounce.store` | `debounce.store` | Same key, different values: a PSR-6 pool service id (Symfony, default `cache.app`), a cache store name (Laravel, default `cache` = the default store), a cache component id (Yii2, default `cache`), a PSR-16 container id (Yii3, default `Psr\SimpleCache\CacheInterface`); `memory` and `none` everywhere | | HTTP client | `http.client` | `http.client` | `http.client` | `http.client` | Same key: a service id (PSR-18 or symfony/http-client) in Symfony, a container binding or class in Laravel, a component id or class in Yii2, a container id in Yii3; unset = PSR-18 discovery | ### Adapter-only keys | Adapter | Keys | |---|---| | Symfony | `messenger.bus`, `messenger.transport`, `messenger.delay`, `messenger.stamps`, `key_file.path`, `key_file.host`, `key_file.route_name`, `logging.channel`, `flush.priority`, `flush.console_priority`, `profiler.enabled`, `doctrine.enabled`, `doctrine.listener_priority`, `doctrine.connections` | | Laravel | `queue.connection`, `queue.queue`, `queue.delay`, `key_file.path`, `key_file.host`, `key_file.route_name`, `key_file.middleware`, `router.locales`, `router.locale_parameter`, `router.set_app_locale`, `eloquent.enabled`, `logging.channel` | | Yii2 | `queue.component`, `queue.ttr`, `queue.delay`, `queue.priority`, `key_file.pattern`, `router.locales`, `router.locale_parameter`, `router.set_app_locale`, `router.languages`, `router.language_parameter`, `router.set_app_language`, `active_record.enabled`, `active_record.models`, `logging.category` | | Yii3 | `key_file.pattern`, `router.locales`, `router.locale_parameter`, `active_record.enabled`, `active_record.namespaces`, `active_record.models`, `logging.category`, `checks` | ## Environment variables `Config::fromEnv()` reads `getenv()` merged with `$_SERVER` and `$_ENV`. Pass your own array as the first argument to read from somewhere else, and a second argument to change the `INDEXNOW_` prefix. Empty strings count as unset. **Environment over a file: `Config::arrayFromEnv()`.** The same variables as the nested array `fromArray()` takes, holding only the variables that are set (values as strings; `fromArray()` coerces them), so an application without a framework merges them over its configuration file with the environment winning: `Config::fromArray(array_replace_recursive($file, Config::arrayFromEnv()))`. `toArray()` is not the tool for that merge — it carries every default. `Config::fromArray(Config::arrayFromEnv($env))` equals `Config::fromEnv($env)`. The `indexnow` CLI of [`indexnowkit/cli`](../cli/index.md) reads its configuration this way, and extends the rule to the blocks of the optional packages (`INDEXNOW__`: `INDEXNOW_SITEMAP_MAX_DEPTH`, `INDEXNOW_HISTORY_PDO_DSN`). **Booleans are parsed, not cast.** Every boolean option — `enabled`, `dry_run`, `strict_hosts`, `key_file.enabled`, `serve_key_file`, `collector.detect_leaks`, `normalizer.strip_tracking_params`, `normalizer.sort_query` — goes through the same parser (`filter_var` with the boolean filter) in `fromEnv()` **and** in `fromArray()`, so the strings `false`, `0`, `no` and `off` mean false and `true`, `1`, `yes`, `on` mean true. An empty string is "not set" and falls back to the default; a non-scalar is a `ConfigurationException` naming the key. This matters wherever an adapter hands an environment variable straight to `fromArray()` without a cast of its own — Yii3's params block does, and a plain `(bool)` there would read `INDEXNOW_DRY_RUN=false` as true and quietly submit nothing while `check` reported it as a deliberate choice. | Variable | Option | |---|---| | `INDEXNOW_ENABLED` | `enabled` (any boolean literal `filter_var` accepts) | | `INDEXNOW_KEY` | `key` | | `INDEXNOW_PREVIOUS_KEY` | `previous_key`: the key before a rotation, still served and accepted by the key file, never submitted | | `INDEXNOW_HOSTS` | `hosts`, as `host=key,host2=key2`; per-host `key_location`/`base_url` need `fromArray()` | | `INDEXNOW_STRICT_HOSTS` | `strict_hosts` | | `INDEXNOW_KEY_LOCATION` | `key_location` | | `INDEXNOW_BASE_URL` | `base_url` | | `INDEXNOW_ENGINES` | `engines`, comma-separated (`api` or `yandex,bing`) | | `INDEXNOW_DISPATCH` | `dispatch` | | `INDEXNOW_BATCH_MAX_URLS` | `batch.max_urls` | | `INDEXNOW_DEBOUNCE_PER_URL` | `debounce.per_url` | | `INDEXNOW_THROTTLE_PER_MINUTE` | `throttle.max_requests_per_minute` | | `INDEXNOW_HTTP_TIMEOUT` | `http.timeout` | | `INDEXNOW_USER_AGENT` | `http.user_agent` | | `INDEXNOW_KEY_FILE_ENABLED` (`INDEXNOW_SERVE_KEY_FILE` still wins) | `key_file.enabled` | | `INDEXNOW_KEY_FILE_CACHE_MAX_AGE` | `key_file.cache_max_age` | | `INDEXNOW_DEBOUNCE_STORE` | `debounce.store` | | `INDEXNOW_HTTP_CLIENT` | `http.client` | | `INDEXNOW_DRY_RUN` | `dry_run` | | `INDEXNOW_ENV`, else `APP_ENV` | `environment` | | `INDEXNOW_PRODUCTION_ENVIRONMENTS` | `production_environments`, comma-separated | | `INDEXNOW_MAX_URL_LENGTH` | `max_url_length` | | `INDEXNOW_LOG_URLS`, `INDEXNOW_FORBIDDEN_ESCALATION` | `logging.max_urls`, `logging.forbidden_escalation` | | `INDEXNOW_RETRY_MAX_ATTEMPTS`, `INDEXNOW_RETRY_BASE_DELAY`, `INDEXNOW_RETRY_MULTIPLIER`, `INDEXNOW_RETRY_MAX_DELAY`, `INDEXNOW_RETRY_SERVER_ERROR_DELAY` | `retry.*` | ## Hosts, keys and `strict_hosts` Sub-domains are separate hosts for IndexNow: each needs its own key file. Three layouts: - **One site.** Set `key` and `base_url`. Every host you submit uses that key. - **Several sites, one key each.** Fill `hosts`. Hosts missing from the map still fall back to `key`. - **Several sites, nothing else.** Set `strict_hosts: true`. The default key then applies only to the `base_url` host; URLs of any other unlisted host are skipped with reason `no_key` instead of being announced under someone else's key. Recommended whenever URLs can come from user input or from a multi-tenant database. `hosts..key_location` overrides the key file URL for that host only, and must be on that host. `hosts..base_url` gives the host its own absolute base for URL generation outside a request — a console command or a queue worker has no request context, so without it every site would be generated on the single global `base_url`. `Config::baseUrlFor($host)` returns that per-host base, falling back to `base_url` when the host is the base host, and `null` otherwise. Keys can be enumerated with `Config::$hosts`, `Config::$keyLocations` and `Config::$hostBaseUrls` (all lower-cased host maps). To load keys from a database or a tenant registry, implement `Key\KeyProviderInterface` instead. ## The dry-run safety net `Config::fromArray()` switches `dry_run` on by itself when **all** of these hold: no `key`, no `hosts`, an `environment` is given, and it is not in `production_environments` (default `Config::PRODUCTION_ENVIRONMENTS`). A developer who never sets `INDEXNOW_KEY` locally therefore gets logging instead of a boot failure, and never reaches the real API. The reverse case is worth alerting on: `dry_run` on while `environment` says production means nothing is being submitted at all. `Config::isProduction()` reports it, and `Check\Checker` raises it as an **error** rather than a warning in that combination. ## Validation The constructor throws `Exception\ConfigurationException` for: - `enabled` without `key`, `hosts` or `dry_run`; - a `key` (or any host key) outside `[A-Za-z0-9-]{8,128}`; - a `hosts` key that is not a bare host name (scheme, port or path present); - `base_url` that is not an absolute `http(s)` URL, or carries credentials; - `key_location` that is not an absolute `http(s)` URL with a path, or is not on the `base_url` host — engines only accept a key file served from the submitted host; - `hosts..key_location` or `hosts..base_url` pointing at a different host; - `batch.max_urls` outside `1..10000`, negative `debounce.per_url` or `throttle.max_requests_per_minute`, `http.timeout` at or below zero, an empty `engines` list; - a `dispatch` value that is not a short identifier, a `http.user_agent` containing line breaks; - `strict_hosts` without any known host; - an engine name that is neither a known engine nor an `https` endpoint (plain `http` is allowed only on loopback hosts, for mock servers). `Config::fromArray()` additionally rejects non-numeric values for numeric options rather than silently falling back to the default. ## Deriving configurations `with()` takes constructor argument names and returns a validated copy; an unknown name throws. ```php $probe = $config->with(dryRun: false, engines: ['yandex']); $config->withDryRun(true); // shorthand $config->userAgent(); // the effective User-Agent string $config->baseHost(); // lower-cased host of base_url, or null ``` ## Detecting typos in adapter config `Config::OPTIONS` lists every key `fromArray()` understands, in dotted form. `Config::unknownOptions($data, $allowed)` returns the keys of an array that are neither core options nor listed in `$allowed`, so an adapter can warn about `debounce.per_urls` instead of silently ignoring it. List nested keys as `block.key`, never as a bare `block`: a bare name stops the check from looking inside the block. Adapters get this through `Adapter\ConfigFactory::load()` (`ownedOptions:`), which also merges the adapter's defaults, resolves `dispatch: auto` and turns an invalid value into a `critical` log line and a disabled `Config` instead of an exception. ```php $unknown = Config::unknownOptions($userConfig, ['messenger', 'messenger.bus', 'doctrine.enabled']); if ($unknown !== []) { $logger->warning('indexnow: unknown option(s): {options}', ['options' => implode(', ', $unknown)]); } ``` Nested arrays are checked one level deep by dotted path; `hosts` is always accepted because its keys are host names. Naming a block in `$allowed` (for example `messenger`) allows the whole block, so an adapter lists either the block name or the individual dotted paths it owns. # Operations Everything here is about the question an operator actually asks: *my page changed, why was nothing submitted?* — and, before that, about not shipping a setup that submits the wrong thing. ## Production checklist Before the first real submission, and again after every deployment that touches the configuration: 1. **Key and base URL.** `INDEXNOW_KEY` (8–128 characters of `[A-Za-z0-9-]`) and `base_url` are set; every host you submit serves `https:///.txt` with `200`, `text/plain`, the key as the body and no redirect. 2. **`check --strict` is green** in the environment that submits (`bin/console indexnow:check --strict`, `php artisan indexnow:check --strict`, `php yii indexnow/check --strict` in Yii2, `./yii indexnow:check --strict` in Yii3): exit code 0. Put it in the deploy pipeline; it exits 1 on any error and, with `--strict`, on any warning. `check --json` (schema `docs/check.schema.json` of `indexnowkit/console`, codes in [check-codes.md](check-codes.md)) is the form for monitoring: alert on `status` and on the codes, never on the texts. `config --json` is what to paste into a bug report. With `indexnowkit/verify`, `check --sample=` (or `--sample-class=`) fetches a few of your own pages and reports noindex, robots.txt, canonical and redirects as warnings; with `indexnowkit/history`, `status` prints the switches, the 403 counters and the last successful submission ([Status and history](#status-and-history)). 3. **`strict_hosts: true`** whenever a `hosts` map exists or the application answers under more than one hostname (a staging copy, an internal name, the apex next to `www`). 4. **A shared debounce store.** `debounce.store` is a cache that web requests and workers share, not `memory`. 5. **The queue is monitored.** `dispatch: queue` / `messenger` runs a worker; failed jobs are visible; the 403 "rejected permanently" line has an owner. 6. **Staging cannot submit.** Outside production set `INDEXNOW_DRY_RUN=1` (or `INDEXNOW_ENABLED=0`) and `key_file.enabled: false`, so the staging host neither sends nor serves the production key. Since core 0.6, `check` fails on a staging copy that has a key and no `dry_run` setting; a preview environment that submits on purpose says `dry_run: false` explicitly. 7. **Alerts on three lines**: the 403 escalation (`critical`), `invalid configuration, IndexNow is disabled` (`critical`), and `collected URL(s) discarded` (`warning`). The monitoring rules below say how. 8. **Short key-file caching.** `key_file.cache_max_age` ≤ 300 and the CDN honours it: after a rotation the old file must not be served for a day. 9. **`previous_key` removed** once every engine answers 200 for the new key (`check --live`). 10. **Someone looks at the result**: Bing Webmaster Tools → IndexNow Insights, Yandex.Webmaster → Indexing → Reindex pages. IndexNow is a notification; the share of submitted URLs that are in the index after a few days is the number that says whether the setup works. ## What IndexNow is, and is not A submission tells an engine that a URL changed. Whether and when the page is crawled and indexed is the engine's decision; a `200` from the endpoint means "received", nothing more. Google does not participate. The Bing URL Submission API and Google's Indexing API are different protocols with their own quotas and are not covered by this library. Where to see the result: Bing Webmaster Tools (IndexNow Insights: received URLs, crawl outcome, errors per key) and Yandex.Webmaster (Indexing → Reindex pages, and the crawl statistics). A useful success metric is the share of submitted URLs present in the index after a few days, and the time between a change and the updated snippet. ## Deleted pages: what your site must return An engine that receives a URL fetches it. The response decides what happens to the page in the index: | Situation | Return | Effect | |---|---|---| | Gone for good | `410 Gone` | the fastest removal; `404` works too but is treated as "maybe temporary" | | Temporarily unavailable | `404` (or `503` with `Retry-After` for maintenance) | the page stays indexed for a while | | Moved | `301` to the new URL, and submit **both** URLs (the old one is resolved as a deletion, the new one as an update — the ORM adapters do this on a slug change) | the index follows the redirect | | A "not found" page that answers `200` (soft 404) | do not: fix it to `404`/`410` | the engine keeps a useless page and trusts the site less | | Redirect to the home page | do not: `410` or `301` to the closest equivalent | same as a soft 404 | The library sends the URL of a deleted object exactly once; the site's answer does the rest. The pre-flight of `indexnowkit/verify` never blocks a deletion: a URL that answers `404` or `410` is submitted as is (log line `indexnow verify: {url} gone (HTTP 410), submitted as a deletion`); with `verify.redirect: follow` a `301`/`308` submits both the old and the new URL, a `302`/`303`/`307` only the original. ## What not to submit The engines fetch what you submit, and a URL that is not meant to be indexed costs trust and quota: - pages with `` or an `X-Robots-Tag: noindex` header; - paths that `robots.txt` disallows (the engine cannot fetch them; some count it as an error against the key); - non-canonical URLs: tracking parameters, sort/filter variants, session ids, `http://` next to `https://`, the apex next to `www` — submit the `` target only; - URLs that answer `3xx`, `4xx` or `5xx` (except the deletions above); - drafts, previews, unpublished or access-restricted pages. What protects you today: the URL normalizer accepts only absolute `http(s)` URLs, strips fragments and default ports, and rejects URLs with credentials or control characters; `strict_hosts` keeps foreign hosts out; the `when` guard of a rule keeps drafts out (`when: 'isPublished'`), and a `published → draft` change is submitted as a deletion. What it cannot see: a `noindex` tag, a `robots.txt` rule, a canonical pointing elsewhere. Those are the job of the rule (do not declare a rule on such a model, or guard it with `when`) — and of [`indexnowkit/verify`](../verify/index.md): with `verify.enabled: true` every URL gets one GET before submission and a page with `noindex` (meta or `X-Robots-Tag`), a path `robots.txt` disallows, a page whose canonical is another URL (`non_canonical: skip|replace`), a redirect (`redirect: skip|follow`) or an origin error (`401`/`403`/`5xx`, `origin_error: skip|send`) is skipped with a `Result` of the matching `Reason`; `404`/`410` pass as deletions. Off by default; with `dispatch: sync` the GETs run inside the web request, so use a queue. `check --sample=` / `--sample-class=` reports the same signals for a few pages without submitting anything (warnings at most, so a CI run against an unreachable production stays green). ## Log channel and levels Every message starts with `indexnow: ` and goes to the PSR-3 logger you inject. Framework adapters put it on a dedicated channel — `indexnow` in the Symfony bundle — so `tail -f var/log/prod.indexnow.log` shows the whole story. ### Delivery outcomes (`Client`) | Level | Message | |---|---| | `debug` | `indexnow: {engine} accepted {count} URL(s) for {host}` | | `info` | `indexnow: {engine} accepted {count} URL(s) for {host}, key verification pending (202)` | | `info` | `indexnow: dry-run POST {endpoint} {body}` | | `warning` | `indexnow: skipping {count} URL(s) for unmanaged host {host}: no key configured (add it to "hosts" or set base_url)` | | `warning` | `indexnow: {engine} could not process URLs for {host} (422): URLs do not belong to the host or keyLocation is invalid` | | `warning` | `indexnow: {engine} rate limited (429) for {host}, retry after {retry_after}s` | | `warning` | `indexnow: {engine} server error {status} for {host}` | | `warning` | `indexnow: {engine} transport error for {host}: {error}` | | `error` | `indexnow: {engine} rejected the key for {host} (403). Check that https://{host}/{key}.txt is reachable and contains the key (run the check command of your adapter, e.g. indexnow:check).` | | `error` | `indexnow: {engine} rejected the request as malformed (400): {body}` | | `error` | `indexnow: {engine} unexpected status {status} for {host}: {body}` | | `error` | `indexnow: {engine} HTTP client failure for {host}: {error}` | | `error` | `indexnow: cannot encode {count} URL(s) for {host} as JSON: {error}` | | `error` | `indexnow: throttle failed, sending without rate limiting: {error}` | | `critical` | the 403 message plus `{consecutive} consecutive failures: submissions for this host are not being indexed.` | The 403 escalation is the one line to page on. `logging.forbidden_escalation` is 5 by default: the fifth consecutive 403 for a host is logged once at `critical`, further ones drop back to `warning` so they do not spam, and any non-403 response resets the counter. Since core 0.8 the counter lives in the cache behind `debounce.store` (the adapters pass it to `Client` as the PSR-16 "failure cache"; plain PHP: `IndexNowKit::create(..., failureCache: $cache)`), so PHP-FPM workers and queue workers count together and the fleet writes the `critical` line once per streak: the keys are `403.` and `…_escalated`, kept for an hour after the last 403. With `debounce.store: memory` or `none` the counter stays in the process, where every worker counts its own 403s and pages on its own fifth failure — alert on the `warning` rate of `reason=invalid_key` as well there. A cache that throws is logged once (`failure cache unavailable, counting 403s per process`) and the process counts on. Every other level in these tables is the default of `logging.levels` (`Config::LOG_EVENTS`) and can be raised or lowered per outcome; `logging.max_urls` decides how many URLs a line lists (0 for PII-sensitive logs). Keys are masked everywhere, including inside response bodies and exception messages. ### Configuration (`Adapter\ConfigFactory`, adapters) | Level | Message | |---|---| | `warning` | `indexnow: unknown option(s) in the indexnow configuration: {options}` (dotted keys, the typo check) | | `critical` | `indexnow: invalid configuration, IndexNow is disabled until it is fixed: {error} (run "{check}")` — nothing is sent until the value is fixed | ### Submission pipeline (`Submitter`) | Level | Message | |---|---| | `info` | `indexnow: disabled (enabled: false), dropping {count} URL(s)` | | `warning` | `indexnow: dropping URL: {error}` | | `warning` | `indexnow: debounce store unavailable, submitting without de-duplication: {error}` | | `warning` | `indexnow: debounce store failed after a successful submission, URLs may be re-sent within {ttl}s: {error}` | | `debug` | `indexnow: debounced {count} URL(s) submitted within the last {ttl}s` | | `error` | `indexnow: result listener {listener} failed: {error}` / `indexnow: result event listener failed: {error}` | `disabled` is at `info` on purpose: it is the most common "nothing is happening at all" state, and `debug` is filtered out in most production setups. ### Resolution (`GuardedUrlResolver`, `ObjectChangeHandler`) | Level | Message | |---|---| | `debug` | ``indexnow: {class} rule "{rule}" skipped for {event}: `when` is false`` | | `debug` | ``indexnow: {class} rule "{rule}" ignores this update (fields {changed} vs filter {fields}, or `when` unchanged and false)`` | | `debug` | ``indexnow: no URLs for {class} ({event}): no rule applies (no #[IndexNow], event not subscribed, or `when` is false)`` | | `debug` | `indexnow: {class} does not subscribe to {event}` | | `warning` | `indexnow: #[IndexNow(via: "{via}")] on {class} stops after {max} related objects` | | `error` | `indexnow: invalid #[IndexNow] on {class}: {error}` | | `error` | ``indexnow: cannot evaluate `when` of {class} rule "{rule}": {error}`` | | `error` | `indexnow: cannot classify the change of {class} for rule "{rule}": {error}` | | `error` | `indexnow: cannot resolve URLs for {class} rule "{rule}" ({event}): {error}` | Turn the `indexnow` channel to `debug` while diagnosing: the four debug lines above are the difference between "nothing happened" and "the rule decided not to". ### ORM hooks (`Hook\ObserverHelper`, the observers of every adapter) | Level | Message | |---|---| | `debug` | `indexnow: {source} ({event}) -> {url}` — one line per resolved URL, with the rule that produced it | | `error` | `indexnow: cannot resolve the URLs of {class}: {error}` — the hook went on, the object was not submitted | | `error` | `indexnow: cannot collect {count} URL(s): {error}` | ### Queue workers (`Retry\WorkerOutcome`, the jobs of every adapter) | Level | Message | |---|---| | `info` | `indexnow: {count} URL(s) of job {id} will be retried{delay}{attempt}` — `{delay}` is ` in {n}s` where the job sets the delay (Laravel), `{attempt}` is ` (attempt {n})` where the job knows it | | `error` | `indexnow: giving up on {count} URL(s) of job {id} after {attempt} attempt(s)` (Laravel and yii2-queue; Messenger reports exhausted retries itself) | | `error` | `indexnow: {count} URL(s) of job {id} rejected permanently ({reasons}); run "{check}"` — `{reasons}` lists ` `: `api 403`, `yandex 422` | ### Delivery hand-off | Level | Message | |---|---| | `warning` | `indexnow: {count} collected URL(s) discarded: the unit of work ended without flush() (request end hook not run?)` | | `debug` | `indexnow: discarding {count} staged URL(s), transaction rolled back` / `..., savepoint rolled back` | | `debug` | `indexnow: throttle limit of {per_minute} requests/min reached, waiting {wait_ms} ms` | | `error` | `indexnow: sync dispatch of {count} URL(s) failed, they are lost: {error}` / `indexnow: dispatch of {count} URL(s) failed, they are lost: {error}` | ## Metrics `Result::metricLabels()` returns low-cardinality labels ready for a counter: `status`, `engine`, `reason`, `http_code`, `retryable`. The host is deliberately absent because it is unbounded in multi-tenant setups; add `$result->host` yourself if your cardinality budget allows. ```php $indexNow->submitter->addListener(function (IndexNowKit\Result $result) use ($metrics): void { $metrics->counter('indexnow_results_total', $result->metricLabels())->inc(); $metrics->counter('indexnow_urls_total', $result->metricLabels())->incBy($result->urlCount()); }); ``` A listener that throws is logged and ignored; delivery is never affected. A decorator around `SubmitterInterface` must forward `addListener()`, or listeners registered on the outer object never fire. Alert on: `reason=invalid_key` (the key file broke), a sustained `reason=rate_limited`, `status=failed` with `retryable=false`, and the collector-discard warning above. ## Status and history Two read-only commands come with [`indexnowkit/history`](../history/index.md) (`composer require indexnowkit/history`; `indexnow:history` / `indexnow:status` in Symfony, Laravel and Yii3, `indexnow/history` / `indexnow/status` in Yii2): - **`status`** prints the switches (`enabled`, `dry_run`, environment), the dispatch mode with what the adapter knows about its queue (Messenger transport and bus, Laravel connection and queue, the Yii2 queue component; Yii3 has no queue mode, so it adds nothing there and names the container id of the debounce store instead), the debounce window and store, the engines, the **403 counter of every configured host with its escalation flag** (`Retry\ForbiddenCounter`, the same cache the client counts in), the last successful submission ("3 min ago, 2 URLs, api"), the history size and the core version. `--json` follows `status.schema.json` of the package: alert on `hosts[].escalated` and on `history.error`. Nothing is fetched. - **`history`** lists what the submitter recorded, newest first: `at`, status, reason, engine, HTTP code, URLs (`--host`, `--status=ok|pending|failed|skipped`, `--url` exact after normalization, `--since=2h|3d|2026-09-01`, `--limit`, `--json`). `history --purge` removes what is older than `history.retention_days` (`--purge=30` for 30 days) and prints one line — a cron entry. Both read the `Submission\SubmissionStoreInterface` of the adapter (see [submission-store.md](submission-store.md)): the package's `psr16` ring buffer (one process, development, small sites) or `pdo` table (production; the migration is in the package's `docs/migrations.md`), set by `history.store`, or a store of your own. What is recorded: the normalized URLs (tracking parameters already stripped), host, engine, status, reason, HTTP code, `retryable`, endpoint and the error sentence of the `Result` — never a response body, a header or the key. `check` adds `history.store` (the configured store, or an error with the migration hint when the table is missing) and `history.records` (`history: 1 240 records, last 3 min ago`). ## Monitoring rules Four rules cover what goes wrong in production; the first two page, the other two open a ticket. | # | Signal | Threshold | Meaning and action | |---|---|---|---| | 1 | `critical` on the `indexnow` channel | any | the key file broke (403 ×5) or the configuration is invalid and IndexNow is off: run `check`, fix, redeploy | | 2 | results with `status=failed`, `retryable=false` (403, 422, 400) | > 0 in 15 min | permanent rejections: the key file, URLs of a foreign host, or a bug — `explain` one of the URLs | | 3 | results with `reason=rate_limited` | sustained for 10 min | the engine throttles you: lower `throttle.max_requests_per_minute`, raise `batch.max_urls` usage, or wait; retries follow `Retry-After` | | 4 | `warning: … collected URL(s) discarded` | any | a request or job ended without `flush()`: the runtime skipped the terminate hook (early `exit()`, fatal error, long-running runtime) — prefer a queued dispatch there | Everything else the library logs at `warning` is per request and self-healing (a cache blip, a 5xx that the queue retries): count it, do not page on it. A `debug`-level channel in production is fine volume-wise only with `logging.max_urls: 0`. **Sentry filter.** The library logs at `warning` for outcomes the queue retries; forwarding every one of them to Sentry turns a rate-limited hour into hundreds of events. Keep `error` and above from the `indexnow` channel, drop the rest: ```php // sentry.php / config/sentry.php — keep errors, drop the per-request warnings of the library 'before_send' => static function (\Sentry\Event $event): ?\Sentry\Event { $level = (string) $event->getLevel(); if ($event->getLogger() === 'indexnow' && !\in_array($level, ['error', 'fatal'], true)) { return null; } return $event; }, ``` (Symfony: the channel name is `logging.channel`, default `indexnow`; Laravel: the log channel of `indexnow.logging.channel`; Yii2 and Yii3: the category of `logging.category`, default `indexnow` — Yii's Sentry targets pass it as the logger.) ## "My URL was not submitted" Walk it in this order. Each step names the reason or log line that proves it. 1. **Is IndexNow on?** `enabled: false` yields `skipped` / `disabled` and one `info` line per call. 2. **Is it dry-run?** `dry_run` yields `skipped` / `dry_run` and an `info` line with the full body. Outside production a missing key turns this on automatically — that is the intended dev behaviour and a bug in prod. `Checker` reports it as an error when `environment` says production. 3. **Did the rule fire at all?** With an ORM, the `debug` lines above say whether a rule was skipped by `when`, by `events`, or by `fields`. No lines at all means no rules were found: check that the class really carries `#[IndexNow]` and that nothing logged `invalid #[IndexNow] on {class}`. 4. **Did the URL survive normalization?** `warning: indexnow: dropping URL` and `skipped` / `invalid_url`. The usual cause is a relative URL with no `base_url`, in a console command or a worker. 5. **Is there a key for that host?** `warning: skipping ... unmanaged host` and `skipped` / `no_key`. With `strict_hosts` this fires for every host outside `base_url` and the `hosts` map. 6. **Was it debounced?** `skipped` / `debounced`. The same URL is not re-sent within `debounce.per_url`. The debug line reports the count. 7. **Did the engine reject it?** `failed` with reason `invalid_key` (403, key file), `unprocessable` (422, URLs on another host or a bad `keyLocation`), `invalid_request` (400, please report), `rate_limited` or `server_error`. 8. **Did anything get collected but never flushed?** See the next section. ## The collector and units of work `Collector` buffers normalized URLs and is drained once by `IndexNowKit::flush()`. Nothing sends until then. `Collector::reset()` empties the buffer **without delivering**, for long-running runtimes that recycle services between requests. It logs at `warning` when the buffer was not empty. That line means a unit of work ended without a flush and those URLs are gone; it is nearly always the smoking gun for "the entity saved and nothing arrived". Under Symfony, `flush()` runs on `kernel.terminate`, `console.terminate` and `WorkerMessageHandledEvent`. `kernel.terminate` fires only when the SAPI lets it: an early `exit()`, a fatal error before termination, or a reverse-proxy setup that never releases the request can skip it. Under Swoole, RoadRunner or FrankenPHP the behaviour depends on the runtime bridge. In those environments prefer a queue-backed dispatch, where the batch is durably enqueued before the worker moves on, and treat the collector-discard warning as a monitored signal. Long-running custom commands should call `flush()` periodically instead of accumulating for the life of the process. ## Debounce and cache outages The debounce store fails **open**. If `filterRecent()` throws, the submission proceeds without deduplication and logs `debounce store unavailable`; if `markSubmitted()` throws afterwards, the window is not recorded and the URLs may be re-sent within the TTL. Both are warnings, one per `submit()` call, so the noise is bounded by request volume rather than URL volume. The visible symptom of a Redis blip is therefore a burst of duplicate submissions, not lost ones. That is the right trade: a missed submission leaves stale content in the index, a duplicate costs one request. `MemoryDebounceStore` is per process and bounded to 50 000 entries. It is right for CLI runs, tests and single workers; a web application should use `Psr16DebounceStore` on a shared cache so the window survives across processes. ## Throttling in web requests versus workers `TokenBucket` blocks with `usleep()` and counts one token per outgoing HTTP request, per process. Inside a web request it only engages when a single request produces more batches than the limit, so keep `throttle.max_requests_per_minute` comfortably above that, or install `NullThrottle` there and rate-limit in the worker. A throttle that throws never blocks delivery: the request goes out unlimited and an `error` is logged. ## Key rotation Rotating a key breaks submissions until the new key file is reachable, because engines answer 403 for a key whose file they cannot verify. 1. Serve the **new** key file first, alongside the old one if your setup allows it. With the shipped key file route, `previous_key` (`INDEXNOW_PREVIOUS_KEY`) does exactly that: the route answers for both keys, submissions use the new one only. 2. Keep `Cache-Control` short. `KeyFileResponder::DEFAULT_MAX_AGE` is 300 seconds for exactly this reason: a CDN holding the old file for a day means a day of 403s. 3. Switch the configured key. `key:generate --write-env --force` does the whole step in the env file: the new key goes to `INDEXNOW_KEY`, the old one to `INDEXNOW_PREVIOUS_KEY`. It refuses to rotate while `INDEXNOW_PREVIOUS_KEY` still holds the key of an earlier rotation (engines may still verify against it): remove the variable first, or pass `--no-previous` to drop the old key on purpose, or `--yes` to overwrite it. 4. Run the check command. `Checker` fetches every key file over HTTP and compares the body, its `Content-Type` and its `Cache-Control`/`Age` against `key_file.cache_max_age`, and `robots.txt`; `--live` sends a real probe to every endpoint even when `dry_run` is on. With `previous_key` set, the old key file is fetched too: `previous key file OK … rotation window open` (`key_file.previous`) means both keys are served; a warning means the old file is already gone while engines may still verify against it. 5. Watch for the 403 escalation. Five consecutive failures for a host means nothing is being indexed. 6. Remove `previous_key` once `check --live` is green for every host: the line goes away with it. If the key file cannot live at `/{key}.txt`, set `key_location` to its absolute URL on the same host. A `key_location` on a different host is rejected at configuration time, because engines answer 422 for it. Behind a proxy or a CDN on a PSR-15 stack, put `Key\KeyFileRequestHandler` in the middleware pipeline **before** the router (and before any authentication or maintenance-mode middleware): the key file is then served whatever the application does with the rest of its routes, and every other request goes on untouched. The key travels in the JSON body of every submission and in the key file, nowhere else: the library never uses the GET form of the protocol (`?url=…&key=…`), so the key does not end up in access logs, proxy logs or referrers. Logs and exception messages of the library mask it to four characters. # Retries, queues and bulk submissions The core never retries inside a web request. `submit()` returns one `Result` per endpoint × host × batch, and the ones worth trying again carry `retryable: true` (429, 5xx, network failures and unexpected client errors). What you do with them is a deployment decision, not a library one. ## RetryPolicy `Retry\RetryPolicy` decides how long to wait, identically in every adapter. ```php use IndexNowKit\Retry\RetryPolicy; $policy = new RetryPolicy( maxAttempts: 3, // total attempts including the first baseDelay: 60, // seconds before the second attempt after a 429 without Retry-After multiplier: 2.0, maxDelay: 3600, serverErrorDelay: 5, // seconds before the second attempt after 5xx or a network failure ); $delay = $policy->delayAfter($results, $attempt); // null = stop ``` `delayAfter()` returns `null` when the attempt number has reached `maxAttempts` or nothing in the batch is retryable. Otherwise it honours the largest `Retry-After` any result reported, and falls back to `base × multiplier^(attempt-1)`, clamped to `maxDelay`. The base is 60 seconds after a 429, because the engine explicitly asked you to slow down, and 5 seconds after a 5xx or a network blip, which is usually transient. ## In-process retries `Retry\RetryingSubmitter` decorates any `SubmitterInterface` and re-submits the retryable URLs in place. The delay is a blocking `sleep()`, so this belongs in CLI commands, cron jobs and queue workers, never in a web request. ```php use IndexNowKit\Retry\{RetryPolicy, RetryingSubmitter}; $submitter = new RetryingSubmitter($indexNow->submitter, new RetryPolicy(maxAttempts: 3)); $results = $submitter->submit($urls); ``` The returned list holds the last outcome for each URL: results that were retried replace their earlier failure, and results that were never retryable are carried through unchanged. Pass a `$sleeper` callable as the fourth argument to make the retries instant in tests. `RetryingSubmitter` forwards `addListener()` to the inner submitter, so profilers and metrics keep working. Any decorator you write must do the same, or every listener registered on the outer object is silently dropped. ## Queue workers Enqueue the URL list, submit in the worker, re-enqueue what came back retryable. ```php // producer $indexNow->collect($urls); // during the unit of work $indexNow->flush(); // hands the batch to the DispatcherInterface // dispatcher, enqueuing instead of sending use IndexNowKit\Dispatch\CallableDispatcher; $dispatcher = new CallableDispatcher(fn (array $urls) => $queue->push(new SubmitUrls($urls, attempt: 1)), $logger); // worker $results = $indexNow->submit($message->urls); $retry = IndexNowKit\Result::retryableUrls($results); $delay = (new RetryPolicy())->delayAfter($results, $message->attempt); if ($retry !== [] && $delay !== null) { $queue->later($delay, new SubmitUrls($retry, attempt: $message->attempt + 1)); } ``` `Result::retryableUrls()` deduplicates and keeps first-occurrence order. `Result::allUrls()` and `Result::urlsWhere($results, $predicate)` cover the other selections (`Result::urlsOf()`, deprecated since 0.2.0, is gone in 0.4). A worker has no request context, so `base_url` must be configured or every relative URL is dropped as invalid. A dispatcher must never throw into user code: `SyncDispatcher` and `CallableDispatcher` log and swallow. ## Which failures are worth retrying | Outcome | Retry | Why | |---|---|---| | 429 `rate_limited` | yes, after `Retry-After` | the engine will accept it later | | 5xx `server_error` | yes | transient on the engine's side | | network / timeout `transport` | yes | transient on yours | | `unexpected` | check `retryable` | an ill-behaved HTTP client is retryable; a status no engine should return is not | | 403 `invalid_key` | **no** | the key file is wrong; retrying changes nothing, fix it and resubmit | | 422 `unprocessable` | **no** | the URLs do not belong to the host, or `keyLocation` is invalid | | 400 `invalid_request` | **no** | a bug in the library; please report it | | `skipped` (any reason) | **no** | nothing was sent on purpose | ## Bulk imports and migrations A migration that touches 50 000 rows is the one case where the defaults are wrong. - **Do not call `submit()` for 50 000 URLs inside a web request.** Chunk into `Config::MAX_BATCH_URLS`-sized submissions from a CLI command or a worker, with a `RetryingSubmitter` around them. - **Prefer the site's own URL list.** The add-on package in the README family table streams it and filters by modification date, so re-announcing yesterday's changes is one command, not a script. - **Watch the debounce store.** `MemoryDebounceStore` is bounded to 50 000 entries and evicts expired entries first, then the oldest. A run larger than that which also re-touches earlier URLs silently gets a shorter effective debounce window. Use `Psr16DebounceStore` on a shared cache for long runs. - **Throttle in the worker, not in the request.** `TokenBucket` blocks with `usleep()` and counts per process. In a web request keep `throttle.max_requests_per_minute` well above the number of batches one request can produce, or use `NullThrottle` there and rate-limit in the queue instead. - **Rule fan-out is smaller than it looks.** Four rules on a class plus `via: 'category'` means one imported row touches six URLs, but the collector deduplicates within the unit of work and `debounce.per_url` deduplicates across them: a homepage rule costs one submission per debounce window, not one per row. ## Collecting and flushing `Collector` is the per-unit-of-work buffer: `add()`, `all()`, `count()`, `drain()`, `reset()`. `IndexNowKit::flush()` drains it into the dispatcher and does nothing when it is empty. Call it once at the end of the HTTP request, the console command or the queue message. `reset()` empties the buffer **without** delivering, for long-running runtimes that recycle services between requests. It logs a warning when the buffer was not empty, because that means a unit of work ended without a flush and the URLs are gone. Alert on that line. Replace `CollectorInterface` when you need a durable outbox instead. # Testing `IndexNowKit\Testing` is part of the published package, not a dev-only helper: application and adapter test suites are expected to use it. Four doubles, no framework, no HTTP. | Double | Replaces | Gives you | |---|---|---| | `FakeTransport` | `Http\TransportInterface` | recorded POSTs with the decoded body, queued responses and failures | | `ArrayLogger` | `Psr\Log\LoggerInterface` | every record, plus `messages()` with the context interpolated | | `FrozenClock` | `Psr\Clock\ClockInterface` | a clock that only moves when you call `advance()` | | `RecordingDispatcher` | `Dispatch\DispatcherInterface` | the batches handed over, without sending them | ## Asserting what would be submitted ```php use IndexNowKit\{Config, IndexNowKit}; use IndexNowKit\Debounce\NullDebounceStore; use IndexNowKit\Testing\{ArrayLogger, FakeTransport}; $transport = new FakeTransport(); $logger = new ArrayLogger(); $indexNow = IndexNowKit::create( new Config(key: 'test-key-1234', baseUrl: 'https://www.example.com'), transport: $transport, logger: $logger, debounce: new NullDebounceStore(), ); $results = $indexNow->submit(['/posts/hello', '/posts/hello', '/about']); self::assertCount(1, $transport->posts); self::assertSame('https://api.indexnow.org/indexnow', $transport->posts[0]['url']); self::assertSame( ['https://www.example.com/posts/hello', 'https://www.example.com/about'], $transport->posts[0]['body']['urlList'], ); self::assertTrue($results[0]->isSuccess()); ``` Every entry of `$transport->posts` is `['url' => ..., 'json' => ..., 'headers' => ..., 'body' => ...]`, where `body` is the decoded payload, so you assert on `host`, `key`, `keyLocation` and `urlList` directly. `NullDebounceStore` keeps a test from depending on the debounce window. Use `MemoryDebounceStore` with a `FrozenClock` instead when the window is what you are testing. ## Entities and rules ```php $urls = $indexNow->urlsFor($post, IndexNowKit\Event::Updated); self::assertSame(['https://www.example.com/posts/hello'], $urls); foreach ($indexNow->explain($post, IndexNowKit\Event::Updated) as $resolved) { // $resolved->rule, ->class, ->event, ->locale, ->url, ->source() } ``` `urlsFor()` and `explain()` never throw, so a test that expects a broken attribute to be reported asserts on the log instead: ```php self::assertStringContainsString( 'invalid #[IndexNow] on ' . Broken::class, implode("\n", $logger->messages('error')), ); ``` ## Engine responses and failures `willRespond()` queues responses in order; anything beyond the queue gets the constructor default. Queue a `Throwable` to simulate a network failure. ```php use IndexNowKit\Http\Response; use IndexNowKit\Testing\FakeTransport; $transport = (new FakeTransport())->willRespond( new Response(429, '', 30), // rate limited, Retry-After: 30 new Response(200), ); $results = $indexNow->submit(['/a']); self::assertTrue($results[0]->retryable); self::assertSame(30, $results[0]->retryAfter); self::assertSame(IndexNowKit\Reason::RateLimited, $results[0]->reason); $transport->willRespond(FakeTransport::failing('connection refused')); // TransportException on the next POST ``` `FakeTransport::failing()` returns a ready-made `TransportException`; `Response::parseRetryAfter()` is what a real transport uses to turn the header into seconds, and takes a `$now` argument so HTTP-date values are testable. ## Retries without waiting `RetryingSubmitter` takes a sleeper, so a retry test runs instantly and can assert on the delay. Continuing the queue above (429 with `Retry-After: 30`, then 200): ```php use IndexNowKit\Retry\{RetryPolicy, RetryingSubmitter}; $slept = []; $submitter = new RetryingSubmitter( $indexNow->submitter, new RetryPolicy(maxAttempts: 3, baseDelay: 60), $logger, static function (int $seconds) use (&$slept): void { $slept[] = $seconds; }, ); $submitter->submit(['/a']); self::assertSame([30], $slept); // Retry-After won over the exponential base ``` ## Debounce windows ```php use IndexNowKit\Debounce\MemoryDebounceStore; use IndexNowKit\Testing\FrozenClock; $clock = new FrozenClock('2026-01-01 00:00:00'); $indexNow = IndexNowKit::create($config, transport: $transport, debounce: new MemoryDebounceStore($clock)); $indexNow->submit(['/a']); $indexNow->submit(['/a']); self::assertCount(1, $transport->posts); // second call debounced $clock->advance(601); $indexNow->submit(['/a']); self::assertCount(2, $transport->posts); ``` `TokenBucket` takes the same clock plus its own sleeper, so throttling is testable the same way. ## Collecting without sending ```php use IndexNowKit\Testing\RecordingDispatcher; $dispatcher = new RecordingDispatcher(); $indexNow = IndexNowKit::create($config, transport: $transport, dispatcher: $dispatcher); $indexNow->collect(['/a', '/b']); self::assertSame(2, $indexNow->collector->count()); $indexNow->flush(); self::assertSame(['https://www.example.com/a', 'https://www.example.com/b'], $dispatcher->urls()); self::assertCount(1, $dispatcher->batches); self::assertTrue($indexNow->collector->isEmpty()); ``` This is the right double for adapter tests: it proves the unit-of-work hook fired without involving HTTP at all. ## The key file ```php $transport->onGet('https://www.example.com/test-key-1234.txt', new Response(200, 'test-key-1234')); $report = (new IndexNowKit\Check\Checker($config, $indexNow->keys, $transport))->run(); self::assertFalse($report->hasErrors()); ``` Unregistered GET URLs answer `404`, which is what a "key file missing" test wants. ## Dry run `dry_run` exercises the whole pipeline — normalization, deduplication, grouping, key lookup — and stops before the POST. Results come back as `skipped` with reason `dry_run`, and the body is in the `info` log line. ```php $indexNow = IndexNowKit::create($config->with(dryRun: true), transport: $transport); self::assertSame([], $transport->posts); ``` Prefer it in application test suites where you care that a change *would* have been announced; prefer `FakeTransport` where you care about the exact payload. ## Assertions for an adapter's HTTP and command tests The conformance scenarios H01–H06 are the same in every framework, only the way a response or a command output is captured differs. Two static helpers of [`indexnowkit/testing`](../testing/index.md) (`composer require --dev indexnowkit/testing`) hold the assertions, so an adapter test parses its framework's objects and asserts once: ```php use IndexNowKit\Testing\Conformance\CheckOutputAssertions; use IndexNowKit\Testing\Conformance\KeyFileAssertions; // H01: 200, text/plain, the key as the body, Cache-Control with public and max-age, Vary: Host exactly when the // body depends on the host — a hosts map or strict_hosts, which is what Config::keyFileHeaders() decides KeyFileAssertions::assertKeyFileResponse($response->getStatusCode(), $response->headers->all(), $response->getContent(), $key, maxAge: 300, expectVaryHost: true); // H02/H03: an unknown key, another host's key, key_file.enabled: false KeyFileAssertions::assertNotServed($response->getStatusCode()); // H04/H05: the check command CheckOutputAssertions::assertExitCode(0, $exitCode, $output); // the output is the failure message CheckOutputAssertions::assertReady($output, 'www.example.com'); // ": key file OK" and the closing line CheckOutputAssertions::assertKeyFileHint($output, 403); // the status and the hint about what the engines do ``` `Cache-Control` is compared by directive (frameworks order them differently), header names in any case, values as a string or a list. ## Conformance kits for adapters Two abstract PHPUnit cases of `indexnowkit/testing` turn docs/spec/03 into runnable scenarios against *your* wiring (the package is `require-dev`; the core itself ships no PHPUnit code): - `Testing\Conformance\CoreConformanceTestCase` (C01, C03, C04, C06, C09–C12, C14, C19, C20): return the facade your container built and the `FakeTransport` it is wired to; optionally a second configured host for C04. - `Testing\Conformance\OrmConformanceTestCase` (A01–A21, plus A05b/A05c): implement the driver — the transaction verbs of your data layer (`begin()`, `commit()`, `rollback()`), the end of a unit of work (`flush()`, `collectedCount()`), and fixtures with fixed rule shapes (`createPost()`, `createMultiPost()`, `createCategorizedPost()`, `createTag()`, `attachTag()`, `bulkUpdateTitle()`, …). The docblock of the class lists the rules every fixture must carry; the URL conventions (`postUrl()`, `ampUrl()`, `categoryUrl()`, `homeUrl()`) are overridable. `indexnowkit/doctrine` (`tests/OrmConformanceTest.php`) and `indexnowkit/laravel` (`tests/Conformance/`) are the reference drivers. A scenario that does not apply to your framework is documented in your README, not skipped silently. ## Notes for adapter authors - Assert on rules and events through `ObjectChangeHandler::createdEvents()`, `updatedEvents()` and `deletedEvents()` before resolving, so an ORM test does not need URLs to verify classification. - `IndexNowKit::create()` rejects combining a custom `submitter:` with `transport:`, `debounce:`, `throttle:` or `normalizer:`, because a custom submitter builds its own pipeline. Pass those to your submitter instead. - `indexnowkit/testing` ships a mock IndexNow server for end-to-end runs through a real PSR-18 client: `php -S 127.0.0.1:8089 vendor/indexnowkit/testing/resources/mock-server/router.php`, with scenarios selected by an `X-Mock-Scenario` header (`ok200`, `pending202`, `forbidden403`, `ratelimit429`, …), `MOCK_KEYS` for the key files it serves and a request log at `GET /_mock/requests`. The core's own `Psr18TransportTest` runs against a private copy of the same router (`tests/Support/mock-server/`), because the core cannot depend on `testing`. ## How the family itself is verified Three floors, one rule each, all in the monorepo CI (`.github/workflows/ci.yml`, `taint.yml`): | Floor | Where | Tool | Rule | |---|---|---|---| | Line coverage | `packages//tests/coverage-floor.txt`, every package | PHPUnit + pcov, `bin/coverage-floor` | the coverage measured when the floor was set; raising it is a normal commit, lowering it a separate commit with the reason | | Mutation score (MSI) | `packages//tests/msi-floor.txt` — core, verify, sitemap, history, console | [Infection](https://infection.github.io) over the whole `src`, `bin/mutation ` | the same ratchet; the `mutation` job is non-blocking until every floor is a CI measurement that held three weekly runs, the `mutation / changed lines` job of a pull request mutates only the lines it changes | | Taint | `packages//psalm.xml` — the same five packages | [Psalm](https://psalm.dev/docs/security_analysis/) `--taint-analysis`, `bin/taint ` | blocking; a flow that is the feature (a file path from the command line) is suppressed in `psalm.xml` with the reason next to it | Coverage says a line ran; the mutation score says a test would notice if the line were wrong (`<` for `<=`, a dropped `return`, an off-by-one in a limit), which is the difference between "covered" and "checked". Log texts and exception messages are not API ([bc.md](bc.md)), so the mutants that only reword them are ignored in `infection.json5` by regex, not by annotations in the code. Infection and Psalm are tools of the monorepo (`tools/infection`, `tools/psalm`, with their locks), not dev dependencies of the packages: Infection needs PHP 8.3 while the packages support 8.2, and Psalm is used for the taint analysis only — phpstan level 9 with strict rules is the type checker. A library has no taint source of its own, so each package in the taint matrix carries `tests/Taint/entrypoints.php`: its public API called with request data (`$_GET`, `$_POST`, `php://input`), the entry points Psalm follows into the sinks it knows (`PDO`, `file_put_contents()`, `header()`, `echo`). What Psalm cannot see here, and the tests cover instead: a URL that went through `Url\UrlNormalizer` is clean to Psalm (`parse_url()` ends the flow), so the SSRF class — redirects, `canonical`, nested sitemaps — is proven by the allow-list tests of verify and sitemap, not by the taint job. The framework adapters are outside the matrix: their inputs are the frameworks' request objects, which Psalm does not treat as sources without a plugin per framework. # Writing an adapter For someone who has never read this package's source and wants a working framework adapter by the end of the day. Every section names the core types involved and the conformance scenarios from [docs/spec/03-conformance.md](https://github.com/indexnowkit/spec/blob/main/03-conformance.md) it satisfies. ## 1. Is an adapter the right thing? You do not need a package to use IndexNow. `IndexNowKit::create()` plus a `CallableUrlResolver` covers a single application: ```php $locator = new ArrayResolverLocator(['post' => fn (Post $p) => '/posts/' . $p->slug]); $indexNow = IndexNowKit::create($config, resolver: new AttributeUrlResolver(new AttributeReader(), ParamExtractor::plain(), null, $locator)); ``` An adapter is warranted when other people's applications should get the same behaviour without wiring it. Three shapes exist, and most packages are one of them: - **ORM hook** — the framework has a unit of work and a commit boundary (`indexnowkit/doctrine`). - **CMS hook** — models cannot carry attributes; rules are registered at runtime (WordPress post types, Drupal). - **Framework glue** — container wiring, config, commands, a key-file route (`indexnowkit/symfony-bundle`). ## 2. The 20-minute adapter Everything a minimal adapter needs, in one file, on layer 2 of the kit: `Adapter\ServicesBuilder` describes the graph (your container's pieces as closures, everything else from the core's factories), `Adapter\Services` builds it lazily, `Hook\ObserverHelper` is the never-throwing part of the model hooks. It passes A01, A04, A07 and H01–H03; the core keeps this exact class under test (`tests/Unit/Adapter/TwentyMinuteAdapterTest.php`). ```php final class IndexNowIntegration { public readonly Services $services; private readonly ObserverHelper $hooks; /** @param array $frameworkConfig the raw config array, your own blocks included */ public function __construct(array $frameworkConfig, ?string $environment, LoggerInterface $logger, ?RouteUrlResolverInterface $router = null) { // Never throws: an invalid value is one critical log line and a disabled Config until it is fixed. $config = (new ConfigFactory(ownedOptions: ['myfw.route_prefix'], checkCommand: 'myfw indexnow:check'))->load($frameworkConfig, $environment, $logger); $builder = (new ServicesBuilder($config, $logger)) ->httpClientLocator(fn (string $id): object => $this->service($id) ?? throw new RuntimeException($id)) ->debounceStore(fn (Services $s): DebounceStoreInterface => DebounceStoreFactory::fromConfig($s->config, fn (string $id) => $this->cache($id))) ->resolverLocator(new ArrayResolverLocator([], locate: fn (string $id) => $this->service($id), hint: 'a service id')); if ($router !== null) { $builder->router($router); } $this->services = $builder->build(); // no IO: nothing is built before it is used $this->hooks = ObserverHelper::forChanges($this->services->changes(), fn(array $urls) => $this->services->kit()->collect($urls), $logger); // the hook resolves without building the client } /** Model save hook. */ public function onSaved(object $model, array $changedFields): void { $urls = $this->hooks->guard($model, static fn (ObjectChangeHandler $changes): array => $changes->updated($model, $changedFields)); $this->hooks->deliver($urls ?? []); } /** Model delete hook, before the row disappears: resolve now, deliver once it is gone. */ public function onDeleting(object $model): void { $urls = $this->hooks->guard($model, static fn (ObjectChangeHandler $changes): array => $changes->deleted($model)); $this->hooks->rememberDeletion($model, $urls ?? []); } public function onDeleted(object $model): void { $this->hooks->deliver($this->hooks->takeDeletion($model) ?? []); } /** End of the unit of work: after the response was sent, if the platform allows it. Never throws. */ public function onShutdown(): void { $this->services->flushIfCollected(); } /** GET /{key}.txt */ public function keyFileResponse(string $path, string $host): ?array { $body = $this->services->keyFileResponder()->bodyForPath($path, $host); return $body === null ? null : [$body, $this->services->config->keyFileHeaders()]; } /** How your framework resolves `debounce.store` to a PSR-16 cache and `#[IndexNow(resolver: ...)]` / `http.client` ids to services. */ private function cache(string $id): CacheInterface { /* your container */ } private function service(string $id): ?object { /* your container; null = unknown */ } } ``` Everything below is refinement of those six methods. What the builder did not get comes from the factories of layer 1: `Http\TransportFactory::lazy()` (`http.client`, through your locator), `Debounce\DebounceStoreFactory`, `Dispatch\DispatcherFactory` (`dispatch`; give `queueFactory()` a closure for your queue), `fromConfig()` on `Collector`, `TokenBucket`, `AttributeUrlResolver` and `KeyFileResponder`. Override any node with `transport()`, `submitter()`, `dispatcher()`, `urlResolver()`, … and every dependent node uses the replacement; `build()` throws `ConfigurationException` for what is statically wrong (a `debounce.store` id without a store, a queue mode without a queue). `Services` also gives you `checker()` (add your lines with `checks()`), `submitterFactory()` for the commands, `rules()` for rules registered at runtime, and `hasCollected()`/`flushIfCollected()` for the request-end hook. The parity between the two layers is a test in the core (`ServicesParityTest`). Three accessors exist because two adapters wrote the same workaround before them. `router()` and `resolverLocator()` return `null` when the graph has no such node, which is right for the core and wrong for a framework adapter that always sets one: writing `?? throw` after every call is a dead branch with a message nobody reads. Use **`requireRouter()`** and **`requireResolverLocator()`** instead — same node, a `ConfigurationException` naming the missing node when it is really absent. And the `events` closure **may return `null`**, like the nullable nodes: an adapter whose container may or may not hold a `Psr\EventDispatcher\EventDispatcherInterface` answers that question lazily inside the closure rather than asking the container during `build()`, which is meant to do no IO at all. `Hook\ObserverHelper` has two named constructors for the same reason: `forChanges($changes, $sink, $logger)` for a hook that must not build the client, and `forKit($kit, $logger)` for the plain case, so nobody has to work out what the union-typed constructor wants. A container that describes services (Symfony, Laravel) stays on layer 1 and calls the same factories service by service: its service ids and bindings are its public API, and a builder would hide them. `IndexNowKit::create()` is the plain-PHP form of the same graph. ### Optional packages `indexnowkit/sitemap`, `indexnowkit/verify` and `indexnowkit/history` are `suggest`ed, not required: an adapter must work without them and say so where the user looks. The recipe, the same in the three reference adapters (shown for sitemap; verify uses `PageSignals::class` as the marker and decorates the submitter and the command factory when `verify.enabled`, history uses `HistoryConfig::class` and puts its store into the submission-store slot when `history.store` is set — an interface cannot be a marker, `class_exists()` is false for it): - **One predicate per adapter, `Adapter\OptionalPackage`**: `OptionalPackage::sitemap($installed)` (`verify()`, `history()`: the Composer name, the marker class and the feature word live in the core, the markers as strings) — `installed()` is `class_exists()` of the marker unless the adapter passes an override (`null` = detect; the bundle's `sitemapInstalled` constructor argument, a Laravel container binding under `IndexNowKitServiceProvider::SITEMAP_PACKAGE`, the Yii2 component's `sitemapInstalled` property, the Yii3 service's `sitemapInstalled` argument). No statics: the override travels with the adapter's own configuration. `notInstalledMessage()`, `checkLine()`, `checkLevel()` and `check()` are the three texts below, written once. **Ask the core, not the package**: `Sitemap\Adapter\SitemapServices::package()` returns the same object but lives in `indexnowkit/sitemap` — an adapter that called it to find out whether the package is installed booted fine in its test suite (the package in `require-dev`, `installed: false` passed by hand) and was a `Class not found` in an application without the package. The CI job `optional-packages-absent` removes the three packages and boots every adapter with detection; `Testing\Conformance\OptionalPackageAssertions::assertDetected()` is the assertion of that test. - **Separate classes behind it**: every file with a `use IndexNowKit\Sitemap\*` is instantiated only when the predicate holds (`\Console\SitemapCommand`, and whatever registers the reader, the spool check and the runner). A `::class` constant on an absent class is safe; `SitemapConfig::OPTIONS`, `SitemapReader::MAX_*` or `Sitemap\Console\Definitions` in a file that is loaded without the package are a fatal. - **The package wires itself**: `Sitemap\Adapter\SitemapServices`, `Verify\Adapter\VerifyServices` and `History\Adapter\HistoryServices` hold what every adapter needs from the package — `options()`, `config()`, the reader / transport / robots cache / stores, the decorators, the check lines with their texts, the runners — as plain static functions over the pieces, plus `*For()` twins over `Adapter\Services` for a runtime graph. Call them; do not copy the constructions or the texts. What stays in the adapter is what the framework decides: where the block comes from, how a cache, a connection or `http.client` is looked up, the queue facts of `status`, the sample check over the ORM, when a request is a web request. - **A stub command with the same name** (`SitemapNotInstalledCommand`, or the Yii action) that ignores its arguments, prints `indexnowkit/sitemap is not installed: composer require indexnowkit/sitemap` and exits `ExitCode::FAILURE`: a cron that ran `sitemap` before the package went optional gets a sentence, not "command not found". - **`OptionalPackage::check($block, $defaults)`** in the checker (a `Check\StaticCheck`): `sitemap: not installed (composer require indexnowkit/sitemap)` at level ok when the block is absent or equal to the defaults the adapter ships, or `sitemap: not installed, the sitemap block in the configuration is ignored (composer require indexnowkit/sitemap)` at level warning when the application configured a block nothing reads. That line is the only place the absence is mentioned: no log line at boot or on a request. - **`Check\SampleGateCheck` over `Check\SampleOptions`**, both in the core, for `check --sample` / `--sample-class`. `verify` is the one optional package with a command option in front of it, so the check that decides what `--sample` means has to load **without** the package — which is why the gate lives in the core and not in verify, and why every adapter used to carry a byte-identical copy of it together with its two user-facing sentences. Do not copy it again. `SampleOptions` is one mutable holder per graph that the `check` command fills with what it parsed (`urls`, `classes`, and the `sampler` closure that turns a class into URLs through your record loader — the only part that is yours). `SampleGateCheck::withPackage($options, $factory)` hands them to the package's `Verify\Check\SampleCheck`; `::withoutPackage($options, $package, $block, $defaults)` writes the "not installed" line of the predicate with ` — pre-flight checks off`, or an error `check --sample needs indexnowkit/verify` when a sample was given anyway. Every line it writes carries the code `verify.installed` (`SampleGateCheck::CODE`), never `verify.sample` — that code exists only while the package does ([check-codes.md](check-codes.md)). - **`SitemapConfig::loadOrDisabled($block, $logger, $checkCommand)`** (from `indexnowkit/sitemap`) builds the sitemap configuration at runtime: an invalid block is one `critical` line naming the error and your check command, and a disabled configuration — nothing throws from the container. - **`ConfigFactory(ignoreBlocks: ['sitemap'])`** without the package (and `...SitemapConfig::OPTIONS` in `ownedOptions` with it), so a configuration written for the package does not warn as "unknown option" once the package is gone. `ownedOptions` stays dotted: a bare `sitemap` in it would hide every typo inside the block. ## 3. The component graph ``` Transport -> Client -> Submitter -> Collector + Dispatcher -> IndexNowKit ^ ^ KeyProvider UrlResolver <- AttributeReader ``` `IndexNowKit::create()` builds all of it with sensible defaults; every argument is optional and named, and parameter **names** are part of the compatibility promise, so always pass them by name. In a container, build the same graph service by service — that is exactly what the Symfony bundle does, and nothing in the library requires the facade. Two rules when substituting pieces. A custom `submitter:` brings its own pipeline, so combining it with `transport:`, `debounce:`, `throttle:` or `normalizer:` is rejected instead of silently ignored. And wrap the transport in `Http\LazyTransport` so a request that submits nothing never pays for client discovery and never fails on a missing PSR-18 client: ```php $transport = new LazyTransport(fn () => Psr18Transport::discover(timeout: $config->httpTimeout)); ``` ## 4. Configuration Map your framework's config file onto `Config::fromArray()`. `Config::OPTIONS` is the canonical list of keys the core owns; `Config::unknownOptions($data, $allowed)` reports typos in the rest, so `debounce.per_urls` does not pass silently. Strip your own blocks before handing the array over. `dispatch` is a free identifier the core validates and reports but never acts on — the adapter decides what `sync`, `queue` or `messenger` mean. Feed `environment` from your framework's environment name to get the non-production dry-run safety net. Full details in [configuration.md](configuration.md). If your config can only be validated at runtime (environment placeholders), do not let a bad value throw from a save hook. `Adapter\ConfigFactory` is that path, declared once per adapter: ```php $factory = new ConfigFactory( ownedOptions: ['queue.connection', 'queue.delay', 'key_file.path', ...SitemapConfig::OPTIONS], // dotted keys only dispatchModes: ['queue', 'sync', 'none'], // [0] is what `dispatch: auto` may not be: see autoDispatch autoDispatch: static fn (): string => $queueExists ? 'queue' : 'sync', needBaseUrl: ['queue'], // a worker has no request to take the host from defaults: ['dispatch' => 'auto', 'debounce' => ['store' => 'cache']], // scalars and blocks of scalars, never lists validate: static fn (Config $c): ?string => $c->dispatch === 'queue' && !$queueExists ? 'the queue component is not configured' : null, checkCommand: 'myfw indexnow:check', ); $config = $factory->load($raw, $environment, $logger); // runtime: warning on unknown keys, critical + disabled on an error $config = $factory->build($raw, $environment); // check command, tests: throws ConfigurationException ``` The merge is deliberate: a top-level raw key replaces the default, the known blocks (`http`, `debounce`, `key_file`, `throttle`, `retry`, `batch`, `logging`) and your owned blocks merge key by key, lists (`engines`, `hosts`) come from the raw array untouched. `key_file.enabled`, `key_file.cache_max_age`, `debounce.store` and `http.client` are core options: read them from the `Config`, do not carve them out. ### Names The vocabulary is the core's, in configuration keys and method names alike, so a reader moving between adapters meets one word per concept. `locale` (the attribute's `locales`, `ResolvedUrl::$locale`, `locale_hosts`, `router.locales`, `router.locale_parameter`, `router.set_app_locale`) — not "language", even where the framework says so (Yii2 renamed its keys back in 0.12). Methods follow `submitX` / `submitXs` with the framework's word for an object: `submitEntity()` / `submitEntities()` in the core and Doctrine, `submitModel()` / `submitModels()` in Laravel, `submitRecord()` / `submitRecords()` in Yii2 and Yii3 — and the command is `submit-` (`indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record`, `indexnow:submit-record`). Framework-native differences stay where the framework's own vocabulary is the point (`queue.connection` / `queue.component`, `logging.channel` / `logging.category`, `eloquent.*` / `active_record.*`). ## 5. How your framework says "this object has a public page" | Model | Core piece | |---|---| | PHP attributes on the class | `Attribute\AttributeReader` (the default) | | rules registered in code, per class or per object | `Attribute\RuleRegistry` | | your own metadata source | implement `Attribute\AttributeReaderInterface` | | the object knows its own URL | `#[IndexNowUrl]` on the method, or `Url\CallableUrlResolver` | | attributes behind `__get()` / an array (Eloquent, CMS records) | implement `Attribute\SubjectReaderInterface`, give it to the graph's `ParamExtractor` (`ServicesBuilder::paramExtractor(new ParamExtractor(new MyReader()))`, `IndexNowKit::create(extractor:)`) | `RuleRegistry` decorates any reader, so a CMS adapter keeps attribute support for free: ```php $registry = new RuleRegistry(); // wraps AttributeReader by default $registry->register(Post::class, [new IndexNow(route: 'posts.show', params: ['post' => 'self'])], new IndexNowDefaults(when: 'isPublished')); $registry->registerFor(CmsPage::class, fn (CmsPage $p): ?RuleSet => $rulesFor($p)); // null = fall through ``` Whatever the source, every path should end at `GuardedUrlResolver`, which is the only never-throwing entry point. ## 6. URLs Implement `Url\RouteUrlResolverInterface` for your router. It is deliberately two methods, so the core can re-extract parameters per locale and pin a host per rule: ```php public function locales(array|string $locales): array; // list; [null] = no locale dimension public function generate(string $route, array $params, ?string $locale = null, ?string $host = null): string; ``` - `locales('current')` returns `[null]`; `'all'` returns every locale your framework has enabled; an explicit list is returned as given. An empty list means "no locale dimension" and must become `[null]`. - `generate()` returns an **absolute** URL. Outside a request there is no host to inherit, so fall back to `$config->baseUrl`; when `$host` is given, prefer `$config->baseUrlFor($host)` and fall back to `https://$host`. - Wrap your router's exceptions in `ConfigurationException` with the route name in the message. A missing parameter is the most common attribute mistake and the message is what the user will see. - Parameters arrive already extracted and coerced. An object value means route model binding (`params: ['post' => 'self']`); decide in the bridge how your router consumes it. Without a router, use `Url\ArrayResolverLocator` to serve `#[IndexNow(resolver: ...)]`, or let models expose `url:`/`urls:` rules. Replace `Url\UrlNormalizerInterface` only to change canonical form — stripping tracking parameters, enforcing a trailing-slash policy, mapping hosts. Implementations must throw `InvalidUrlException` and nothing else, or they break the never-throw contract of `submit()`. ## 7. Hooking model changes `Url\ObjectChangeHandler` is the piece to build on. It combines rule lookup, per-rule event classification and guarded resolution, and never throws: an invalid rule set or a failing resolver is logged and yields nothing. ```php $changes = $indexNow->changes(); // or new ObjectChangeHandler($reader, $guarded, $extractor, $logger) $guarded = $indexNow->resolver(); // the GuardedUrlResolver behind it, for explain() and resolveRule() $changes->created($model); // list $changes->updated($model, $changedFields, $changeSet); $changes->deleted($model); ``` Two levels exist because ORMs differ. Hooks that run **before** the write, where ids do not exist yet, collect `RuleEvent`s first and resolve them later: ```php $events = $changes->updatedEvents($model, array_keys($changeSet), $changeSet); // list // ... the write happens ... foreach ($events as $ruleEvent) { $urls = $changes->resolve($model, $ruleEvent); } ``` Hooks that run **after** the write (observers, save hooks) call `created()` / `updated()` / `deleted()` directly. Three things decide correctness: - **Deletions must be resolved while the object still has its identifiers and old state.** That includes a rule whose `when` just turned false: `updatedEvents()` returns it as `Event::Deleted`, and it must be resolved before the write, not after. - **Supply both `$changedFields` and `$changeSet` when you have them.** The change set (`field => [old, new]`) is what makes the old-state visibility exact instead of heuristic. See [attribute-reference.md](attribute-reference.md#reconstructing-w_before). - **Never let a hook throw into the host application.** A typo in an attribute must not break a checkout. Satisfies A03, A04, A07, A08, A09, A10, A12. ### Example: an Eloquent-style observer ```php final class IndexNowObserver { public function __construct(private readonly IndexNowKit $indexNow) {} public function created(Model $model): void { $this->collect($this->indexNow->changes()->created($model)); } public function updated(Model $model): void { $changeSet = []; foreach ($model->getChanges() as $field => $new) { $changeSet[$field] = [$model->getOriginal($field), $new]; } $this->collect($this->indexNow->changes()->updated($model, array_keys($changeSet), $changeSet)); } public function deleting(Model $model): void // before the row disappears { $this->collect($this->indexNow->changes()->deleted($model)); } private function collect(array $resolved): void { $this->indexNow->collect(ResolvedUrl::urls($resolved)); } } ``` Register it so its URLs are handed over only **after** the surrounding transaction commits: resolve synchronously (the old state is live), hand off through the framework's after-commit hook (`Connection::afterCommit()` in Laravel). If your framework has no such hook, use the next section. ## 8. Commit safety URLs must not leave before the outermost transaction commits, or a rolled-back write is announced to search engines. `Transaction\TransactionStaging` lives in the core precisely so every adapter solves this the same way. ```php $staging = new TransactionStaging(sink: fn (array $urls) => $indexNow->collect($urls), logger: $logger); $staging->stage($scope, $urls); // inside an open transaction $staging->commit($scope); // real COMMIT: hands the URLs to the sink $staging->discard($scope); // ROLLBACK, or a commit that threw: drops them, logged at debug ``` `$scope` is any object whose identity outlives the transaction — the native database connection is the usual choice. Entries are held in a `WeakMap`, so a forgotten scope does not leak. `hasPending()` and `pendingCount()` are there for diagnostics. Where the real commit signal comes from differs per framework: a DBAL driver middleware (Doctrine), `Connection::afterCommit()` (Laravel — its transaction manager already drops callbacks of a rolled-back savepoint, so the Laravel adapter needs no staging of its own; `ShouldHandleEventsAfterCommit` is *not* used, because a deferred `updated` handler runs after `syncOriginal()` and loses the old values), `transaction.on_commit` (Django). When the framework offers none, use `Transaction\VerifyingStaging` instead of guessing. Satisfies A01, A02, A05. ### 8a. No commit signal at all: verify on commit Yii2 fires commit/rollback events only for the outermost transaction and nothing for savepoints; Yii3's `yiisoft/db` fires nothing. `Transaction\VerifyingStaging` holds URLs together with a *verifier*, a closure that re-reads the row by primary key and says whether the change actually landed (created/updated: the row exists with the new values, `VerifyingStaging::rowMatches($row, $expected)`; deleted: no row): ```php $staging = new VerifyingStaging($logger); // in the ORM event, when a transaction is open. The last argument is the subject key: the identity of the row // within this transaction, so a second change of the same row merges into the first entry instead of adding one. $subject = Post::class . '#' . $id; $staging->stage($connection, fn (): bool => $this->rowMatches($record, $written), $urls, $subject, key: $subject); // when the data layer says the transaction ended (commit event), or at the end of the request when it says nothing: $indexNow->collect($staging->flush($connection)); // runs the verifiers, drops what did not land (logged at debug) $staging->discard($connection); // on a rollback event: nothing to verify ``` One primary-key lookup per staged subject, only for changes inside an explicit transaction (autocommitted changes go straight to the collector). A change that did not land drops every URL it produced, including `via` pages and the old URL of a renamed page: announcing "deleted" for a page that still exists is the one outcome to avoid. A verifier that throws counts as landed (a stale URL costs one crawl, a lost one costs the update) and is logged at warning. Two rules decide what to put in `$expected`, and getting them wrong is silent — every URL of the change disappears at `debug` level: - **An insert is verified by existence, nothing else.** Pass an empty `$expected`: an insert that reached the commit landed by definition, and the row being there is the whole answer. Handing `rowMatches()` every non-null property of a new record is how a single DECIMAL or `timestamptz` column silences the announcement of every new page of a class. A delete asks for `null` the same way. - **For an update, only unambiguous values are compared.** The row comes back raw from the driver — a string for an integer, `'19.90'` for what the application wrote as `19.9`, a zone suffix on a `timestamptz`, the database's own spelling of a JSON document. `rowMatches()` therefore compares integers, strings, booleans, backed enums and null, and skips everything else (floats, dates, arrays, objects) as if the row did not carry the column. A row with nothing left to compare counts as matching. Do not filter `$expected` yourself and do not add a comparison of your own on the skipped types: the question the verifier answers is "did the transaction commit", not "is the row byte for byte what I wrote". Pass the subject key whenever a row can be written more than once inside one transaction — two `save()` calls, a create followed by an update, a rename in two steps. Without it each write is staged on its own, every verifier runs at the end against the **last** state of the row, and the earlier ones compare against values that are no longer there. With it the entry is merged: the URLs of both writes are kept and the verifier of the later write is the one that runs. Satisfies A02, A05, A05b, A05c without touching the connection configuration; the Yii adapters are the reference (Yii2 flushes on the commit event of the outermost transaction, Yii3 at the end of the request, keeping what is still inside an open transaction). ## 9. The unit of work `Collector\CollectorInterface` buffers normalized URLs for one HTTP request, console command or queue message, and `IndexNowKit::flush()` drains it into the dispatcher exactly once. Call `flush()`: - after the response has been sent, where the platform allows it (`kernel.terminate`, `fastcgi_finish_request`); - at the end of a console command; - after each handled queue message. In long-running runtimes call `CollectorInterface::reset()` between requests. The default `Collector` logs a `warning` when a reset discards a non-empty buffer, which is the signal that a unit of work ended without a flush. Do not swallow it. Replace the interface for a durable outbox or a per-tenant buffer. Satisfies A06, H06. ## 10. Delivery `Dispatch\DispatcherInterface` has one method and must never throw into user code. `SyncDispatcher` sends inline, `CallableDispatcher` hands the list to any queue, `NullDispatcher` drops it (`dispatch: none` — collect, never send). The worker recipe is the same everywhere: `submit()`, then `Result::retryableUrls($results)`, then `RetryPolicy::delayAfter($results, $attempt)`, then re-enqueue. Which statuses are final and which are retryable is in [retries-and-queues.md](retries-and-queues.md). Satisfies A14, C13. A worker has no request context, so `base_url` must be set or every relative URL is dropped. ## 11. Keys and the key file `Key\KeyProviderInterface` has four methods and is called on the submission path, so implementations must be cheap and must never throw for an unknown host: ```php public function keyFor(string $host): ?string; // null = unmanaged, URLs are skipped public function keyLocationFor(string $host): ?string; public function isKnownKey(string $key, ?string $host = null): bool; // serve /{key}.txt? public function managedHosts(): array; // diagnostics; empty when unknown ``` `StaticKeyProvider::fromConfig($config)` covers config-backed setups including `strict_hosts`. For a database-backed multi-tenant install, implement the interface and cache per request. **Honour the `$host` argument of `isKnownKey()`**: without it, tenant A's key file is served on tenant B's host, which lets one tenant claim ownership signals on another's domain. `null` means "any managed host" and is only for single-site adapters and CLI diagnostics. Serving the file on a PSR-7 / PSR-15 stack (Slim, Mezzio, Laminas, Yii3, a plain PSR-15 pipeline) is `Key\KeyFileRequestHandler`, the first recipe for a new adapter — no class of your own: ```php $handler = KeyFileRequestHandler::fromConfig($config, $keys, $responseFactory, $streamFactory); // PSR-17 factories of the application $app->get('/{key:[A-Za-z0-9-]{8,128}}.txt', $handler); // a route: handle() reads the key off the path, 404 otherwise $app->add($handler); // or a middleware before the router: a request it does not serve goes on return $handler->respond($route->getArgument('key'), $request); // or the key your router already extracted ``` It answers 200 with the key and `Config::keyFileHeaders()` for a key of the requested host (the URI host), 404 otherwise; as a middleware it hands everything else — a path that is not a key file, an unknown key, another host's key — to the next handler untouched. The core requires only the PSR interfaces (`psr/http-server-handler`, `psr/http-server-middleware`); tests use `nyholm/psr7`. Behind it, and for a stack that is not PSR-7 (HttpFoundation, Illuminate), `Key\KeyFileResponder` decides so no adapter reimplements the matching: ```php $responder = new KeyFileResponder($keys, $config->serveKeyFile); $body = $responder->bodyForPath($request->getPath(), $request->getHost()); // or bodyForKey() if your router if ($body === null) { return $this->notFound(); } // extracted {key} already return $this->response($body, 200, $config->keyFileHeaders()); ``` `KeyFileResponder::PATH_PATTERN` is the request-path regex (group 1 is the key) for routers that match by pattern, `CONTENT_TYPE` is `text/plain; charset=utf-8`, and `DEFAULT_MAX_AGE` is 300 seconds — short on purpose, because a cached old key file turns every submission into a 403 after a rotation. Serve 200 with no redirect, 404 otherwise. Which of the two a stack gets, in one table — the recipe is decided by whether the framework's request is PSR-7, not by taste: a bridge (`symfony/psr-http-message-bridge` plus a PSR-17 implementation) costs two dependencies in `require` for thirty lines, so the HttpFoundation and Illuminate adapters keep a controller of their own. | Stack | Recipe | |---|---| | Yii3, Slim, Mezzio, Laminas, any PSR-15 pipeline | `Key\KeyFileRequestHandler` as a route handler or a middleware (above); the Yii3 package delegates to it | | Symfony (HttpFoundation) | a controller over `Key\KeyFileResponder` (`SymfonyBundle\Controller\KeyFileController`) | | Laravel (Illuminate, HttpFoundation underneath) | a controller over `Key\KeyFileResponder` (`Laravel\Http\KeyFileController`) | | Yii2 | a controller over `Key\KeyFileResponder` and `Config::keyFileHeaders()` | | Bitrix, a plain `index.php` | `KeyFileResponder::bodyForPath($_SERVER['REQUEST_URI'], $_SERVER['HTTP_HOST'])` and `header()` per `Config::keyFileHeaders()` | `Key\KeyGenerator::generate($length, $hex)` produces CSPRNG keys, 32 hex characters by default; pass `hex: false` for the full `[A-Za-z0-9]` alphabet. A `key:generate --write-env` style command is the first thing users run. Satisfies H01–H03. ## 12. Transport `Http\TransportInterface` is two methods over any HTTP stack — `wp_remote_post()`, a framework client, raw curl: ```php public function post(string $url, string $json, array $headers = []): Response; public function get(string $url): Response; ``` Rules: never throw on an HTTP status code, throw `Http\Exception\TransportException` for network failures and timeouts, cap the body you read (`Psr18Transport` uses 2 KiB for POST diagnostics and 50 MiB for GET, a generous cap for the largest documents consumers of the transport read). Parse `Retry-After` with `Response::parseRetryAfter($header)` so every adapter interprets delta-seconds and HTTP-dates identically and applies the same clamp. Configure no redirects and a timeout. Redirects deserve a sentence, because PSR-18 says nothing about them: the standard leaves following a 3xx to the client, and clients differ (Guzzle follows five by default, Symfony's twenty, a hand-built curl none). `Psr18Transport` switches redirects off on the clients it builds itself (`max_redirects: 0`, `allow_redirects: false`); a client the application hands in through `http.client` keeps its own defaults, and `check` warns about it (`http.client`), because a key file that redirects to a catch-all page then looks like a 200 — the conformance scenario H02 ("the key file answers without a redirect") cannot be verified honestly on such a client. When an application must pass its own client, pass one configured without redirects. On a PSR-7 stack, hand the transport the application's PSR-17 factories as well (`Psr18Transport::discover(requestFactory:, streamFactory:)`, `TransportFactory::lazy(…, requestFactory:, streamFactory:)`), so that a request is built by the same implementation the rest of the application uses and `php-http/discovery` is never consulted. Implement `Http\StreamingTransportInterface` too when your stack can read a response body in chunks (`download(string $url, $sink): Response` writes the body to a stream resource and returns an empty-bodied `Response`). Consumers that read large documents (the add-on packages) then never hold a document in memory; with a plain `TransportInterface` they buffer each document once through `get()`. `LazyTransport` and `Testing\FakeTransport` implement both. ## 13. Debounce, throttle, clock `MemoryDebounceStore` is per process and bounded; `Psr16DebounceStore` shares the window across processes through any PSR-16 cache; `NullDebounceStore` disables it. Wire your framework's cache to the PSR-16 one by default for web applications, and memory for CLI and tests. A debounce store may throw: the submitter treats a failing read as "nothing is recent" and a failing write as "window not recorded", logs a warning, and delivers anyway. Preserve that fail-open behaviour in your own store. The core takes PSR-16 and only PSR-16. A stack whose cache is a PSR-6 pool (Symfony's `cache.app`, a Laminas `StorageInterface`) wraps it in the PSR-16 view its own ecosystem ships — `Symfony\Component\Cache\Psr16Cache`, `Laminas\Cache\Psr\SimpleCache\SimpleCacheDecorator` — and hands that to `Psr16DebounceStore`; the bundle does exactly this. A `Psr6DebounceStore` of the core would be a second store to keep in step for nothing the view does not give. Two questions every adapter answers the same way: `Debounce\DebounceStoreFactory::isShared($store)` says whether `debounce.store` names a cache shared by every process (the 403 counter, the robots cache of verify and the `psr16` history store then share it; `memory` and `none` keep them in the process), and the probe of `Check\DebounceStoreCheck` writes `DebounceStoreCheck::PROBE_KEY` — a key without the characters PSR-16 reserves, so a strict cache does not refuse it and report a working store as broken. `TokenBucket` blocks with `usleep()` per process. In a web request `NullThrottle` is often the better default, with the real rate limiting in the queue worker. Both take a `Psr\Clock\ClockInterface`, so tests use `FrozenClock`. **Open the clock as a replaceable service or binding of its own**, under `Psr\Clock\ClockInterface`, and pass it to all three places that read time: `Throttle\TokenBucket::fromConfig($config, $logger, clock: …)`, `Debounce\DebounceStoreFactory::fromConfig(…, clock: …)` and the submitter (`Submitter`'s last parameter, and `Adapter\SubmitterFactory`'s, which is what the commands build their submitter with). `Testing\FrozenClock` is useless otherwise: an application that binds it still gets wall-clock debounce windows, wall-clock throttle waits and wall-clock timestamps in the submission history — and the records written by a command end up on a different clock from the records written by the application. On layer 2 this is the `clock` node (`Adapter\ServicesBuilder::clock()`, `Adapter\Services::clock()`), which every other node already reads. ## 14. Diagnostics users will ask for Ship six commands of your own. They are what turns "it does not work" into a self-service answer, and their bodies are the `indexnowkit/console` package (`Console\*Runner`, rendering to a `Symfony\Component\Console\Style\SymfonyStyle`; Laravel's `OutputStyle` is one; require it — the core itself does not depend on `symfony/console`). A framework command parses its arguments and calls the runner; every framework prints the same thing. **The command lives in the package, the adapter registers it.** On `symfony/console` the commands are classes already: `Console\Command\*` of `indexnowkit/console` (`SubmitCommand`, `SubmitSubjectsCommand`, `ExplainCommand`, `CheckCommand`, `ConfigCommand`, `KeyGenerateCommand`, the three `*NotInstalledCommand` stubs), `Sitemap\Console\SitemapCommand`, `History\Console\HistoryCommand` and `StatusCommand`. An adapter on symfony/console (the bundle, Yii3, a plain `Symfony\Component\Console\Application`) registers those classes and hands them what varies by constructor — the runners, a `Console\Vocabulary`, a `Console\ConfigSourceInterface` for `check` and `config`, the `.env` file of `key:generate`, a `Check\SampleOptions` with the ORM sampler already inside — and writes no command of its own. The table below is what a command **not** on symfony/console (artisan, a Yii2 controller) parses and hands to the runner. | Command | Runner | What the adapter supplies | |---|---|---| | `check` | `Console\CheckRunner` | a closure that builds `Config` from the raw configuration (throws `ConfigurationException`); `Check\CheckInterface` services for adapter wiring (is the ORM hook active? is the queue routed?) and the checks of the add-on packages. `--sample` / `--sample-class` go into the graph's `Check\SampleOptions`, which `Check\SampleGateCheck` reads (§2 "Optional packages"); the ORM sampler behind it is yours | | `config` | `Console\ConfigRunner` | the closure that builds the `Config`, the raw configuration array as the framework read it (its own blocks included, for the adapter-only keys) and the effective block of every installed optional package. The runner prints the effective values with the keys masked, the adapter-only keys next to them, and masks secrets in both. `config --json` is what a bug report pastes | | `submit ...` | `Console\SubmitRunner` | — | | `submit- [ids]` | `Console\SubmitSubjectsRunner` + `SubmitSubjectsOptions` | a `Console\SubjectLoaderInterface`: class resolution (FQCN or the framework's short name), objects by id, first N objects; `byIds()` / `all()` receive the `Event` so `deleted` can include soft-deleted rows | | `explain ` | `Console\ExplainRunner` | the same loader | | `key:generate` | `Console\KeyGenerateRunner` | the default env file path | Three more commands come from the optional packages, so a complete adapter registers nine: `sitemap` submits the site's own URL list (`indexnowkit/sitemap`), and `history` and `status` read what was submitted (`indexnowkit/history`: `History\Console\Definitions::history()` / `::status()` for the inputs, `History\Adapter\HistoryServices::historyRunnerFor()` / `::statusRunnerFor()` for the bodies; `status` also takes the queue facts only the adapter knows). Each of the three is behind the package predicate of §2, with a stub command of the same name when the package is absent. `Console\Definitions` and the packages' own `Definitions` are the single source of every command name, option, shortcut and description — an adapter that writes its own option descriptions has already drifted. Shared by all of them: `Adapter\SubmitterFactoryInterface` (`SubmitterFactory`: the separate submitter `--force` and `--dry-run` build, `SubmitterFactory::choose()` picks it or the application's), `Console\ResultFormatterInterface` (`ResultRenderer`: table or `--json`; an application replaces it to match its own CLI), `Submission\ResultSummary` (a run that submits in many batches folds results into counts) and `Console\Vocabulary` (the words that differ: "entity" / "model", `bin/console` / `php artisan`, where the configuration lives). Expose the three interfaces under stable service ids so an application can decorate them, and expose the runners too: a tenant loop over `SubmitSubjectsRunner` is a ten-line application command. `Checker` never throws and covers configuration, key files and a live probe; `CheckRunner` prints its report. The adapter-specific lines are `CheckInterface` services tagged for the checker, not special cases in the command. Result listeners (`SubmitterInterface::addListener()`) feed an admin log or a profiler panel. Register the listener on the same submitter instance the application uses, and forward `addListener()` in any decorator. ## 15. The error contract | Situation | Behaviour | |---|---| | invalid `Config` | throws `ConfigurationException` at construction | | invalid rule declaration read through `AttributeReaderInterface` | throws `ConfigurationException` | | invalid rule declaration read through `ObjectChangeHandler` / `GuardedUrlResolver` | logged at `error`, yields no URLs | | resolver failure (missing accessor, router error) in a hook | logged at `error`, yields no URLs | | URL that cannot be submitted | `InvalidUrlException` inside the normalizer, caught by `Submitter`, `warning` + `skipped` result | | HTTP status of any kind | never throws; a `Result` with a `Reason` | | network failure | `TransportException` inside the transport, converted to a retryable `failed` result | | debounce store, throttle, listener or dispatcher failure | logged, delivery continues | | programming errors (empty batch, bad key length) | `InvalidArgumentException` | The golden rule: **nothing reaching a lifecycle hook may throw into the host application.** ### What your adapter throws The core's own promise is that every exception implements `Exception\IndexNowException`, so `catch (IndexNowException $e)` is the stable form ([bc.md](bc.md#exceptions)). An adapter that throws its framework's exception class for a situation the core has a class for breaks that catch for its users, and the family has drifted here before: the same "`history.store` is set but no store could be built" was `yii\base\InvalidConfigException` in one adapter and `Exception\ConfigurationException` in another. One rule for all of them: | Situation | Throw | |---|---| | anything wrong with the configuration the application wrote (an unknown service id, a `debounce.store` that is not a cache, a store that cannot be built) | `Exception\ConfigurationException` — never the framework's own configuration exception, even where the framework has one | | a delegate of an optional package that is not installed (a stub command's collaborator, an accessor behind the predicate of §2) | `\LogicException` with `Adapter\OptionalPackage::notInstalledMessage()` as the message, so the sentence names the `composer require` line once | | an invariant of your own code that the application cannot cause (a node that must exist by construction, an unreachable branch) | the native class (`\LogicException`, `\RuntimeException`); it is a bug report, not a user error | `Adapter\Services::requireRouter()` and `requireResolverLocator()` exist so the second row does not become a third copy of `?? throw` in every adapter: they throw a `ConfigurationException` naming the missing node. And nothing in this table applies inside a lifecycle hook — there, the golden rule wins and everything is logged instead. ## 16. Testing your adapter Use `IndexNowKit\Testing` (`FakeTransport`, `ArrayLogger`, `FrozenClock`, `RecordingDispatcher`) — see [testing.md](testing.md). Assert classification through `ObjectChangeHandler::*Events()` before any URL exists, and delivery through `RecordingDispatcher`. For the HTTP and command scenarios, parse your framework's response or output and hand it to `Testing\Conformance\KeyFileAssertions` (H01–H03: status, content type, `Cache-Control` by directive, `Vary: Host` exactly when `Config::keyFileHeaders()` adds it — a hosts map or `strict_hosts`) and `Testing\Conformance\CheckOutputAssertions` (H04–H05: exit code with the output as the failure message, the ready line, the key file hint), so your tests do not carry a copy of the core's phrases. Both, the conformance kits and the mock server are the `indexnowkit/testing` package (`require-dev`); the core ships only the four PHPUnit-free doubles. Then work through the conformance scenarios with the kits of `indexnowkit/testing` (`Testing\Conformance\CoreConformanceTestCase`, `OrmConformanceTestCase`, see [testing.md](testing.md)): C01–C22 for anything that talks to the protocol, A01–A21 for an ORM adapter, H01–H06 for a framework adapter. Declare in your README which scenarios do not apply to your framework and why — A13 (bulk operations bypass hooks) is a documented limitation everywhere, not a failure. ## 17. Packaging Name it `indexnowkit/`, require `indexnowkit/core ^0.5`, keep the framework itself in `require` and the optional pieces in `suggest` (`indexnowkit/sitemap ^0.1.1` for the `sitemap` command and its `Definitions`, wired as in §2 "Optional packages"; keep it in `require-dev` so the tests cover both states). Run a version matrix in CI over the framework's supported majors and LTS releases, static analysis at the maximum level, and publish EN plus RU READMEs following the family table used here. The Definition of Done is in [docs/spec/91-roadmap.md](https://github.com/indexnowkit/spec/blob/main/91-roadmap.md). ## 18. Reference adapters The bundle and the Laravel package sit on layer 1 (the static factories and `Adapter\ConfigFactory`, one service or binding per node, because those ids are their public API); the Yii2 component sits on layer 2 (`Adapter\ServicesBuilder`, the graph described once, the pieces exposed as delegates); the Yii3 package sits on layer 2 too, with every node of the graph a definition of the container and the core's factory as the default of each. All of them share `Hook\ObserverHelper` in the observers, `Retry\WorkerOutcome` in the queue jobs and `Console\Definitions` (`indexnowkit/console`) in the commands. | Section | `doctrine` | `symfony-bundle` | `laravel` | `yii2` | `yii3` | |---|---|---|---|---|---| | layer | — | 1 (services) | 1 (bindings) | 2 (`ServicesBuilder`) | 2 (`ServicesBuilder`, every node a container definition) | | component graph | `src/IndexNowDoctrine.php` | `src/DependencyInjection/IndexNowKitLoader.php` | `src/IndexNowKitServiceProvider.php` | `src/IndexNowComponent.php` (`services()`) | `src/Wiring.php`, `config/di.php` | | configuration | — | `src/DependencyInjection/{IndexNowKitConfiguration,ConfigFactory}.php` | `config/indexnow.php`, `src/Config/ConfigFactory.php` | `src/Config/ConfigFactory.php` | `config/params.php`, `src/Config/ConfigFactory.php` | | router bridge | — | `src/Url/SymfonyRouteUrlResolver.php` | `src/Url/LaravelRouteUrlResolver.php` | `src/Url/YiiRouteUrlResolver.php` | `src/Url/YiiRouteUrlResolver.php` (`UrlGeneratorInterface`) | | resolver lookup | — | `src/Url/ResolverLocatorFactory.php` (core `ArrayResolverLocator`) | in the provider (core `ArrayResolverLocator`) | in the component (core `ArrayResolverLocator`) | in `Wiring` (core `ArrayResolverLocator` over the container) | | model change hooks | `src/IndexNowListener.php` | via the Doctrine package | `src/Eloquent/IndexNowObserver.php` (`ObserverHelper` + `afterCommit()`) | `src/ActiveRecord/IndexNowObserver.php` (`ObserverHelper` + staging), `IndexNowBehavior.php` | `src/ActiveRecord/IndexNowObserver.php` (`ObserverHelper` + staging), `IndexNowEvents.php` (attribute handlers) | | commit safety | `src/Middleware/*` | `src/Doctrine/StagingSink.php` | Laravel's `afterCommit()` | core `VerifyingStaging` | core `VerifyingStaging`, verified at the end of the request | | unit of work | — | `src/EventListener/FlushListener.php` | `terminating()`, `JobProcessed` | `EVENT_AFTER_SEND`, `EVENT_AFTER_REQUEST` | `src/Event/FlushListener.php` (`AfterEmit`, `ApplicationShutdown`) | | delivery | — | `src/Messenger/*` (`WorkerOutcome`) | `src/Queue/*` (`WorkerOutcome`) | `src/Queue/*` (yii2-queue, `WorkerOutcome`) | `sync` / `none`; a replaced `DispatcherInterface` | | key file | — | `src/Controller/KeyFileController.php`, `config/routes.php` | `src/Http/KeyFileController.php` | `src/Http/KeyFileController.php` | `src/Http/KeyFileHandler.php` over the core's `Key\KeyFileRequestHandler` (PSR-15), `config/routes.php` | | diagnostics | — | the command classes of `indexnowkit/console`, `sitemap` and `history`, registered in `src/DependencyInjection/IndexNowKitLoader.php` (`loadConsole()`); `src/DataCollector/*` | `src/Console/*` (`Definitions`), `src/Check/*` | `src/Console/IndexNowController.php` (`Definitions`), `src/Check/*` | the same command classes, mapped in `config/params-console.php`, wired in `config/di-console.php`; `src/Check/*` | | subject reader | — | — | `src/Eloquent/EloquentSubjectReader.php` | `src/ActiveRecord/ActiveRecordSubjectReader.php` | `src/ActiveRecord/ActiveRecordSubjectReader.php` | ## 19. Compatibility What the core guarantees, what is excluded, and how to ask for a new extension point instead of reaching into `@internal`: [bc.md](bc.md). ## 20. Definition of Done for an adapter - [ ] `Adapter\ConfigFactory` declared with dotted `ownedOptions` (plus `SitemapConfig::OPTIONS` when the package is installed, `ignoreBlocks: ['sitemap']` when it is not); a regression test that a typo inside an owned block (`key_file.enabld`) is warned about; an invalid runtime value disables IndexNow with one `critical` line and never throws from a hook. - [ ] The graph is built through the factories (`Http\TransportFactory`, `Debounce\DebounceStoreFactory`, `Dispatch\DispatcherFactory`, `fromConfig()`); no copied `match` over `debounce.store`, no own "not a PSR-18 client" text, no own class-name resolution (`Console\ClassNameResolver`). - [ ] `#[IndexNow(resolver: ...)]` through `ArrayResolverLocator(locate:, hint:)`; the resolver is `GuardedUrlResolver`. - [ ] Hooks over `Hook\ObserverHelper` (guard, deliver, remembered deletions; no own `WeakMap`, no own "cannot resolve" text); deletions resolved before the row disappears; a commit boundary (`afterCommit`, `TransactionStaging`, `VerifyingStaging`). - [ ] A queue job over `Retry\WorkerOutcome` (retryable vs final, the three log lines) plus your framework's action; or a runtime-assembled container over `Adapter\ServicesBuilder` with `queueFactory()`. - [ ] Flush at the end of every unit of work (request, command, queue message); `Collector::reset()` in long-running runtimes. - [ ] `Key\KeyFileRequestHandler` on a PSR-15 stack, else `KeyFileResponder::fromConfig()` + `Config::keyFileHeaders()`, on a route without session or CSRF; H01–H03 green. - [ ] Nine commands: on symfony/console the classes of `indexnowkit/console` (`Console\Command\*`), `indexnowkit/sitemap` (`Sitemap\Console\SitemapCommand`) and `indexnowkit/history` (`History\Console\HistoryCommand`, `StatusCommand`), registered with their runners, a `Vocabulary`, a `ConfigSourceInterface` and a `SampleOptions`; on another console layer six commands over the runners of `indexnowkit/console` (`check`, `config`, `submit`, `submit-`, `explain`, `key:generate`), `sitemap` from `indexnowkit/sitemap`, `history` and `status` from `indexnowkit/history`; their inputs from `Console\Definitions` / `Sitemap\Console\Definitions` / `History\Console\Definitions` (no own option descriptions), `check` with your `CheckInterface` lines plus `Check\DebounceStoreCheck` (with a probe), `Check\SampleGateCheck` and `Sitemap\Check\SitemapSpoolCheck`. - [ ] `indexnowkit/sitemap` in `suggest` and `require-dev`, behind one predicate (§2 "Optional packages"): without it the `sitemap` command is a stub that explains what to install and exits 1, `check` prints the `StaticCheck` line, a `sitemap` block in the configuration warns about nothing, every other command works, and nothing is logged at boot; a test set with the predicate forced to false. - [ ] `indexnowkit/testing` in `require-dev`; conformance kits green (C01–C22, A01–A21 for an ORM, H01–H06 through `Testing\Conformance\KeyFileAssertions` and `CheckOutputAssertions`); undocumented scenarios named in the README. - [ ] CI matrix over the framework's supported majors, phpstan level 9 on every flavour, EN + RU README with the family table, `docs/troubleshooting.md`, a changelog with migration notes. # Backward compatibility `indexnowkit/core` follows SemVer. **Before 1.0, minor versions may contain breaking changes**; every one is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/core/CHANGELOG.md) with the migration. After 1.0 the rules below become the promise. This page exists because "public API" is ambiguous for a library whose main audience is other library authors. Which PHP and framework versions a release supports, and the rule for dropping one, is [compatibility.md](compatibility.md); **raising the minimum PHP version is not a breaking change** under this promise (Composer does not offer the new minor to an application on the old PHP). ## Three tiers | Tier | What it means | Examples | |---|---|---| | **Call** | You call it. Signatures do not change incompatibly; new parameters are only appended with defaults. | `IndexNowKit`, `Config` (including the static `serveKeyFileFrom()`, `fromEnv()` and `arrayFromEnv()`), `Submitter`, `Client`, `Result`, `Checker`, `KeyGenerator`, `KeyFileResponder`, `Key\KeyFileRequestHandler` (the PSR-15 handler and middleware over it; the constructor and `fromConfig()` take named arguments), `RetryPolicy`, `ObjectChangeHandler`, `GuardedUrlResolver`, `RuleRegistry`, `Transaction\VerifyingStaging`, `Adapter\SubmitterFactory`, `Submission\ResultSummary`, `Adapter\ConfigFactory`, `Adapter\ServicesBuilder`, `Adapter\Services`, `Adapter\OptionalPackage` (including the static `sitemap()`, `verify()`, `history()`), the factories (`Http\TransportFactory`, `Debounce\DebounceStoreFactory`, `Dispatch\DispatcherFactory`, every `fromConfig()`), `Check\DebounceStoreCheck`, `Check\StaticCheck`, `Check\LocalesCheck`, `Check\DispatchLine`, the writers of `Check\CheckReport`, `Dispatch\BatchingDispatcher` (including the static `newJobId()`), `Url\RouteOrigin` (static helpers of the router bridges), `Hook\ObserverHelper`, `Retry\WorkerOutcome`, `Retry\ForbiddenCounter`, `Submission\NullSubmissionStore`, the four test doubles of `Testing\` | | **Implement** | You implement it, and the core calls you. Methods are not added without a major version. | `TransportInterface`, `StreamingTransportInterface`, `Url\RuleAwareUrlResolverInterface` (until 1.0 a method may still be appended in a minor), `Url\ParamExtractorAwareInterface`, `Url\RouteUrlResolverInterface` (one implementation per framework adapter) and `Url\ResolverLocatorInterface` (one shipped implementation, `Url\ArrayResolverLocator`, which every adapter configures with closures rather than replacing); a capability the core needs later comes as a new interface that extends them, the way `History\HistoryStoreInterface` extends `SubmissionStoreInterface`, `Check\CheckInterface`, `KeyProviderInterface`, `UrlNormalizerInterface`, `UrlResolverInterface`, `DebounceStoreInterface`, `ThrottleInterface`, `DispatcherInterface`, `Attribute\SubjectReaderInterface`, `Adapter\SubmitterFactoryInterface`, `Submission\SubmissionStoreInterface` (new in 0.8, see [submission-store.md](submission-store.md)), `Attribute\Param\Condition` and `FieldCondition` (new in 0.8, the `when` guards); the three new interfaces live through one minor unchanged before 1.0 | | **May grow** | Interfaces the core also implements for you, where a new method may appear in a minor. Extend the shipped class rather than implementing the interface from scratch. | `ClientInterface`, `Check\CheckerInterface`, `SubmitterInterface`, `CollectorInterface`, `AttributeReaderInterface` | | **Sealed** | Closed sets the core switches over. Do not implement them: an unknown implementation is a configuration error, or worse, a silent miss. | `Attribute\Param\ParamValue` (`Accessor`, `Value`, `Formatted`, `Call` are the set: a param source of your own is a resolver, `#[IndexNow(resolver: …)]`) | The "may grow" tier is the honest label for interfaces that are still learning what adapters need. If you implement one directly, pin `^0.8.0` rather than `^0.8` and read the changelog before upgrading. Decorating a shipped implementation (`RetryingSubmitter` decorates `Submitter`, `RuleRegistry` decorates `AttributeReader`) is safe in both directions. `RouteUrlResolverInterface` and `ResolverLocatorInterface` used to be listed here; they are "Implement" since 0.11, for two different reasons. `RouteUrlResolverInterface` has no shipped implementation to decorate — there is one per framework adapter — so a method cannot be added to it in a minor. `ResolverLocatorInterface` does have one, `ArrayResolverLocator`, but no adapter decorates or replaces it: all four configure the shipped class with the `locate:` and `hint:` closures, so nothing would gain from the interface growing, and the stricter tier is what the four call sites actually rely on. ## Named arguments `IndexNowKit::create()` takes only named optional parameters after `$config`, and will take more; the list is the signature, and that is the one place to read it. **Parameter names are part of the promise; the order is not.** New parameters are appended, never inserted, and every call should use named arguments: ```php IndexNowKit::create($config, transport: $transport, logger: $logger, resolver: $resolver); ``` The same holds for the constructors of `Config`, `Client`, `Submitter`, `AttributeUrlResolver`, `GuardedUrlResolver`, `TransactionStaging`, `VerifyingStaging`, `RetryPolicy`, `TokenBucket`, `Collector` and `Psr18Transport`: pass anything past the first argument by name. `RuleCompiler` (`compile()`, `fromAttributes()`) is a public static helper in the same "call" tier: adapters call it to compile their own declarations; its signatures only grow by appended optional parameters. `Attribute\ParamExtractor` is an object of the same tier: `new ParamExtractor(...$readers)` takes the `SubjectReaderInterface`s of the graph, `extract()`, `read()`, `resolve()`, `condition()` read with them, `with()`/`fromReaders()`/`readers()` compose. One instance per graph — `IndexNowKit::create(extractor:)`, `Adapter\ServicesBuilder::paramExtractor()`, the adapters' `ParamExtractor` binding or service — shared by `AttributeUrlResolver`, `ObjectChangeHandler` and the `explain` command; the constructors and `fromConfig()` of those take it as a required parameter (since 0.12: a graph never falls back to the DSL by omission; `ParamExtractor::plain()` says so when the DSL alone is meant). `IndexNowKit` derives its extractor from the resolver it is given (`Url\ParamExtractorAwareInterface`, which `AttributeUrlResolver` implements), so the change handler and `explain` read exactly what the resolver reads; a custom resolver without the interface falls back to the plain DSL. `ObjectChangeHandler::renamed(object $subject, array $changeSet, ?object $previous = null, array $selfFields = [])` is the contract for the "old URLs of a renamed object": the core rebuilds the previous state by reflection from the change set, and an adapter whose objects cannot be reset that way (Eloquent attributes) passes `$previous`, a copy of the object as it was. That stays the design: `Attribute\SubjectReaderInterface` is read-only (`supports()`, `has()`, `read()`) and gets no `write()` — writing into an ORM object (dirty tracking, model events, casts) is the adapter's business, and the adapter knows how to produce a before-image (`replicate()->setRawAttributes(getOriginal())` in Eloquent) better than the core. The shipped default implementations are in the "call" tier as well: construct them with named arguments and their public methods stay. That is `Http\LazyTransport` (the default `IndexNowKit::$transport`), `Http\Psr18Transport`, `Key\StaticKeyProvider`, `Url\UrlNormalizer`, `Url\ArrayResolverLocator`, `Url\CallableUrlResolver`, `Url\NullUrlResolver`, `Attribute\AttributeReader`, `Attribute\ChangeClassifier`, `Collector\Collector`, `Debounce\{MemoryDebounceStore, Psr16DebounceStore, NullDebounceStore}`, `Throttle\NullThrottle`, `Dispatch\{SyncDispatcher, CallableDispatcher, NullDispatcher}` and `Clock\SystemClock`. `Config::with()` takes constructor parameter names as keys and rejects unknown ones with a message listing what it accepts. Renaming a `Config` property is therefore a breaking change and appears in the changelog. ## Value objects and enums `Result`, `ResolvedUrl`, `UrlRule`, `RuleSet`, `RuleEvent`, `Http\Response`, `Check\CheckItem`, `Retry\WorkerOutcome`, `Submission\SubmissionRecord` and the attribute classes are `final readonly`. Their properties are read-only public API: reading them is safe, constructing them is safe, and new properties are only appended with defaults. Prefer the named constructors (`Result::ok()`, `Result::skipped()`, `Result::failed()`) over the constructor, so an appended parameter never reaches your call sites. Enums are a special case: **adding a case is not a breaking change** in this library, because the wire protocol and the failure taxonomy grow. | Enum | Adding cases? | |---|---| | `Reason` | yes — always handle unknown cases with a `default` arm | | `Engine` | yes — new IndexNow participants get added | | `Attribute\RuleSource` | yes | | `Event`, `ResultStatus`, `Check\CheckLevel` | no; these are closed sets | A `match` over `Reason` or `Engine` without a `default` will fatal on a new case. Write the default arm. The `Reason` cases and what they mean for a `Result` (`isSkip()`: nothing was sent; `isRetryable()`: a later attempt may succeed by itself): | Case | `isSkip()` | `isRetryable()` | Produced by | |---|---|---|---| | `disabled`, `dry_run`, `debounced`, `no_key`, `invalid_url` | yes | no | the core pipeline | | `noindex`, `robots_disallowed`, `non_canonical`, `redirected` | yes | no | the `verify` package's pre-flight (cases reserved in core 0.8) | | `origin_error` | yes | yes | `verify`: the page could not be fetched | | `invalid_request` (400), `invalid_key` (403), `unprocessable` (422), `unexpected` | no | no | the engine's answer | | `rate_limited` (429), `server_error` (5xx), `transport` | no | yes | the engine's answer or the network | ## Constants These are the values to reference instead of hard-coding, and they are covered by the promise: `Config::MAX_BATCH_URLS`, `Config::DEFAULT_BATCH_MAX_URLS`, `Config::DEFAULT_DEBOUNCE_PER_URL`, `Config::DEFAULT_THROTTLE_PER_MINUTE`, `Config::DEFAULT_HTTP_TIMEOUT`, `Config::PRODUCTION_ENVIRONMENTS`, `Config::OPTIONS`, `Result::NO_ENGINE`, `Client::FORBIDDEN_ESCALATION`, `Client::FAILURE_CACHE_TTL` (the lifetime of the 403 counter in the failure cache; `History\Adapter\HistoryServices::forbiddenCounter()` needs it), `Check\DebounceStoreCheck::PROBE_KEY` (the key every adapter's cache probe writes), `Check\LocalesCheck::CODE`, `KeyValidator::MIN_LENGTH`, `KeyValidator::MAX_LENGTH`, `KeyValidator::ALPHABET`, `KeyValidator::PATTERN`, `KeyFileResponder::PATH_PATTERN`, `KeyFileResponder::CONTENT_TYPE`, `KeyFileResponder::DEFAULT_MAX_AGE`, `Http\Response::MAX_RETRY_AFTER`, `Psr18Transport::POST_BODY_LIMIT`, `Psr18Transport::GET_BODY_LIMIT`, `UrlNormalizer::MAX_URL_LENGTH`, `UrlNormalizer::MAX_HOST_LENGTH`, `UrlNormalizer::MAX_LABEL_LENGTH`, `ParamExtractor::SELF`, `Version::VERSION`. The names an adapter writes instead of a string literal: the node names of `Adapter\Services::*` (`TRANSPORT`, `KEYS`, `NORMALIZER`, `THROTTLE`, `DEBOUNCE_STORE`, `CLIENT`, `SUBMITTER`, `COLLECTOR`, `DISPATCHER`, `READER`, `ROUTER`, `RESOLVER_LOCATOR`, `URL_RESOLVER`, `PARAM_EXTRACTOR`, `FAILURE_CACHE`, `SUBMISSION_STORE`, `CHANGES`, `CLOCK` — a container adapter maps them to its definitions), the dispatch modes the core itself knows (`Dispatch\DispatcherFactory::SYNC`, `::NONE`) and the two reserved values of `debounce.store` (`Debounce\DebounceStoreFactory::MEMORY`, `::NONE`). A node name is added when a node is added, which is a minor; none is renamed or removed before 1.0 without a "Changed" entry. Enums (`ResultStatus`, `Reason`, `Event`, `Engine`, `Check\CheckLevel`, `Attribute\RuleSource`, `Attribute\Param\Placeholder`) and the value objects of the rule model (`Attribute\UrlRule`, `RuleSet`, `RuleEvent`, `Attribute\Param\{Accessor, Value, Formatted, Call}` and the condition `Attribute\Param\Equals`, `Url\ResolvedUrl`) are public API: their public properties are read by adapters and their constructors only grow by appended optional parameters. Their **values** may change in a minor when the protocol or a safety limit changes; the constants themselves will not disappear. ## Exceptions Every exception implements `Exception\IndexNowException`, so `catch (IndexNowException $e)` is the stable form. `ConfigurationException`, `InvalidUrlException`, `InvalidArgumentException` and `Http\Exception\TransportException` keep their meanings. `ConfigurationException` and `InvalidUrlException` extend `Exception\InvalidArgumentException`, which extends PHP's `\InvalidArgumentException`, so both `catch (Exception\InvalidArgumentException)` and `catch (\InvalidArgumentException)` see them. Exception **messages** are not API: they are written for humans and get improved. Match on the class, or on `Result::$reason`, never on message text. ## What is not covered - Anything marked `@internal` in a docblock. Today that is `Config\ConfigParser` and `Config\ConfigNormalizer` (the readers and the normalisation behind `Config`; call `Config::fromArray()` / `fromEnv()`), `Url\Punycode`, `Transaction\StagingFrame`, `Attribute\IndexNow::normalizeEvents()`, `Collector::reportLeak()` and the constructor of `Adapter\Services` (built by `ServicesBuilder::build()`). - Private and protected members of `final` classes, which is all of them: the library has no inheritance points by design, only interfaces. - Log message texts. They are documented in [operations.md](operations.md) so you can grep them, and they are improved between versions. Alert on `Reason` values and log **levels**, not on wording. - Anything under `tests/`, including fixtures and the mock server copy. The published test doubles live in `IndexNowKit\Testing` and **are** covered. The conformance kits (`Testing\Conformance\CoreConformanceTestCase`, `OrmConformanceTestCase`) and the assertion helpers are the `indexnowkit/testing` package since 0.7.0, with their own [bc.md](../testing/bc.md): driver methods only grow by appended methods with a default implementation, a scenario is only added, never removed, in a minor. - The exact set of `Result` objects a single `submit()` call returns. Grouping by host and batching are implementation details of throughput; use `Result::allUrls()`, `Result::retryableUrls()` and `Result::urlsWhere()` instead of indexing into the list. ## Deprecations A deprecated member keeps working for at least one minor version, carries a `@deprecated` tag naming the replacement, and is listed in the changelog. Currently deprecated: | Since | Member | Use instead | |---|---|---| | 0.4.0 | `serve_key_file` (`Config::fromArray()`, `fromEnv()`: `INDEXNOW_SERVE_KEY_FILE`) | `key_file.enabled` / `INDEXNOW_KEY_FILE_ENABLED`; the explicit `serve_key_file` still wins while both exist | Removed after their deprecation window: `Result::urlsOf()` (deprecated 0.2.0, removed 0.4.0). Moved out of the core without a deprecation window (the pre-1.0 rule): in 0.4.0 `IndexNowKit::sitemap()` and everything under `Sitemap\`, now the `indexnowkit/sitemap` package; in 0.7.0 `Testing\Conformance\*` and the assertion helpers (`Testing\KeyFileAssertions`, `CheckOutputAssertions`, `ReadmeAssertions`, now `Testing\Conformance\*` in `indexnowkit/testing`) and everything under `Console\` except `SubmitterFactory*` (now `Adapter\`) and `ResultSummary` (now `Submission\`), now the `indexnowkit/console` package with the FQCN unchanged and its own [bc.md](../console/bc.md). ## Before 1.0 Minor versions may break. The changes made in 0.2.0, 0.4.0, 0.7.0 and 0.8.0 are listed in the changelog (0.5.0 and 0.6.0 were additive); the shape of the breakage to expect is the same: renamed classes as the namespace layout settles, and signatures on the "may grow" interfaces as more adapters land. Application code that only uses the facade, `Config`, the attributes and `Result` has been stable since 0.1 and is expected to stay so. If you need an extension point that does not exist, open an issue rather than reaching into `@internal` or copying a final class. Adapter-driven interface changes are exactly what the pre-1.0 window is for. # Compatibility What each `indexnowkit/*` package runs on, and how long that is meant to last. The constraints are the ones in the packages' `composer.json`; the dates are the upstream end-of-life dates at the time of writing (2026-09), linked so you can check them. The promise itself (what may change in a minor, what may not) is [bc.md](bc.md). ## Policy - **PHP.** Every package requires the PHP versions that still receive security fixes from php.net, and CI runs the whole matrix (8.2, 8.3, 8.4, 8.5) against the lowest and the highest dependency set. The minimum is raised in the first minor after the previous version leaves security support: `^8.2` becomes `^8.3` in the first minor released after 2026-12-31. **Raising the minimum PHP is not a breaking change** of the library: Composer does not offer the new minor to an application on the old PHP, and the code the application sees does not change. None of the packages declares an upper bound of its own; where the matrix stops short of the newest PHP, the bound comes from a dependency — the `yiisoft/*` packages of `indexnowkit/yii3` are the case today (they declare `8.1 - 8.5`), so an application on a newer PHP waits for them, not for us. - **Frameworks.** An adapter supports the framework versions that are in bug-fix or security support upstream, and drops a version in the first minor after its security support ends. A new major of a framework is added in a minor of the adapter when the test suite passes on it, without a release of the core. - **Between the packages.** Every package pins the core to one minor; the adapters pin `console`, `sitemap`, `verify`, `history` and `doctrine` the same way. A release wave moves the constraints together (see [the family changelog](https://github.com/indexnowkit/php/blob/main/CHANGELOG.md)), so `composer update indexnowkit/*` is the upgrade. ## Matrix | Package | PHP | Framework / library | Upstream support ends | |---|---|---|---| | `indexnowkit/core` | `^8.2` | PSR-18 client of your choice (`php-http/discovery`), PSR-3, PSR-16; PSR-7/PSR-17 and PSR-15 (`psr/http-server-handler`, `psr/http-server-middleware`) for `Key\KeyFileRequestHandler` — interfaces only, no implementation is required | — | | `indexnowkit/console` | `^8.2` | `symfony/console ^6.4 \|\| ^7.0 \|\| ^8.0` | 6.4: security fixes to 2027-11; 7.4 LTS: 2029-11 | | `indexnowkit/testing` | `^8.2` | PHPUnit `^11.5 \|\| ^12.0 \|\| ^13.0` (the conformance kits) | per [phpunit.de](https://phpunit.de/supported-versions.html); PHPUnit 11 left bug-fix support on 2026-02-06. The Laravel adapter's own suite stays on PHPUnit 11: `laravel/framework` 12/13 and PHPUnit 12.5 disagree on the error handler | | `indexnowkit/sitemap` | `^8.2` | `symfony/console ^6.4 \|\| ^7.0 \|\| ^8.0` for the command | as console | | `indexnowkit/verify` | `^8.2` | the core's transport; `symfony/console` for `check --sample` (suggested) | as console | | `indexnowkit/history` | `^8.2` | `ext-pdo` (sqlite, mysql, pgsql schemas), PSR-16; `symfony/console` for the commands | as console | | `indexnowkit/doctrine` | `^8.2` | `doctrine/orm ^2.19 \|\| ^3.0`, `doctrine/dbal ^3.8 \|\| ^4.0` | ORM 2.x / DBAL 3.x: security fixes only, see [doctrine-project.org](https://www.doctrine-project.org/projects.html) | | `indexnowkit/symfony-bundle` | `^8.2` | Symfony `^6.4 \|\| ^7.0 \|\| ^8.0` (`framework-bundle`, `http-kernel ^6.4.13`; Symfony 8 needs PHP 8.4), `doctrine/doctrine-bundle ^2.13 \|\| ^3.0` with `indexnowkit/doctrine` | 6.4 LTS: bug fixes to 2026-11, security to 2027-11; 7.4 LTS: 2028-11 / 2029-11 ([symfony.com/releases](https://symfony.com/releases)) | | `indexnowkit/laravel` | `^8.2` | `illuminate/support ^12.0 \|\| ^13.0` (Laravel 12, 13) | 12: bug fixes to 2026-08, security to 2027-02; 13: 2027-08 / 2028-02 ([laravel.com/docs/releases](https://laravel.com/docs/releases)) | | `indexnowkit/yii2` | `^8.2` | `yiisoft/yii2 ^2.0.45`; `yiisoft/yii2-queue ^2.3` for `dispatch: queue` | 2.0.x maintained, no end date announced ([yiiframework.com](https://www.yiiframework.com/release-cycle)) | | `indexnowkit/yii3` | `^8.2`, in practice up to 8.5 | `yiisoft/active-record ^1.0`, `yiisoft/db ^2.0`, `yiisoft/router ^4.0`; `yiisoft/config`, `yii-http`, `yii-console`, `yii-event` of the application read its config groups | Yii3 packages follow their own SemVer lines ([github.com/yiisoft](https://github.com/yiisoft)); every `yiisoft/*` in `require` declares `8.1 - 8.5`, so that is the real PHP ceiling of this adapter; no queue mode until `yiisoft/queue` is released | PHP itself: 8.2 security fixes to 2026-12-31, 8.3 to 2027-12-31, 8.4 to 2028-12-31, 8.5 to 2029-12-31 ([php.net/supported-versions](https://www.php.net/supported-versions.php)). Symfony 8 is a target of the bundle since 0.13 (CI: PHP 8.4 with `framework-bundle ^8.0`); `console`, `sitemap`, `history`, `yii2` and `yii3` accept `symfony/console ^8.0`. Laravel 11 and Symfony 6.3 and below are not supported. ## Flavours in CI `bin/ci ` runs the same install the workflow does: `highest` and `lowest` for every package, `dbal3` for `doctrine` (DBAL 3 / ORM 2), `symfony64` and `symfony8` for `symfony-bundle` (Symfony 6.4, and Symfony 8 on PHP 8.4, each with the highest of everything else). A change that passes `highest` but not `lowest` is a constraint bug, not a code bug: the fix is the constraint. # Codes of `check` Every line the check command prints carries a stable code (`Check\CheckItem::$code`). The code is what `check --json` consumers, deploy pipelines and alert rules match on; the **text is not API** and gets improved between versions, the same way `Reason` is the identifier of a `Result` and `Result::$error` the sentence. A code names the check, not the outcome: `key_file.status` is `ok` when the key file answers 200 with the right body and `error` when it does not, so a rule written as "fail the deploy when `key_file.status` is not ok" survives a rewording. Codes are added in minor versions when a check is added (a new line in the table below), and never renamed or removed before 1.0 without an entry under "Changed" in the changelog. A code is a dotted lower-case identifier; the first segment is the area. Lines about one host carry it in `CheckItem::$host` (`"host"` in the JSON), the global lines have `null` there. ## Core (`Check\Checker`) | Code | Levels | Line | |---|---|---| | `config.enabled` | warning | `enabled: false`: nothing will be submitted | | `config.dry_run` | warning, error | `dry_run` is on (error when the environment is production) | | `environment.name` | ok, warning | the `environment: …` line; warning outside production when real requests leave | | `environment.non_production_submits` | warning, error | a non-production environment with a key and `dry_run` off: error when `dry_run` was left unset, warning when it says `false` explicitly | | `config.strict_hosts` | ok, warning | `strict_hosts` on; or off next to a `hosts` map / in production | | `config.base_url` | ok, warning | `base_url` set or missing | | `config.engines` | ok | the resolved engine list | | `config.delivery` | ok | dispatch, debounce window, batch size, throttle, timeout | | `config.hosts` | error | no host to check at all (no `base_url`, no `hosts`) | | `http.client` | warning | a custom `http.client` fetches the key files: if it follows redirects, a 30x to a catch-all page looks like a 200 | | `key.missing` (host) | error | no key for the host | | `key.invalid` (host) | error | the key fails `KeyValidator` | | `key_file.location` (host) | error | `key_location` points to another host (engines answer 422) | | `key_file.served_externally` (host) | warning | `key_file.enabled: false` and no `key_location`: the web server must serve the file | | `key_file.status` (host) | ok, error | `GET /.txt`: ok on 200 with the key as body; error on any other status | | `key_file.body` (host) | error | 200 with a body that is not the key (a catch-all route) | | `key_file.fetch` (host) | error | the key file could not be fetched (network error, no HTTP client) | | `key_file.content_type` (host) | ok, warning, error | after a matching key file: `text/plain` ok; no `Content-Type` header warning; another type error; one neutral ok line when the transport exposes no headers | | `key_file.cache_control` (host) | ok, warning | after a matching key file: `Cache-Control` lifetime (`s-maxage`, else `max-age`) or `Age` above `key_file.cache_max_age` is a warning (a rotation would serve the old key for that long); absent header: no line | | `key_file.robots` (host) | ok, warning | `robots.txt` (when it answers 200): a `Disallow` covering the key file path for every bot or an engine's bot is a warning | | `key_file.previous` (host) | ok, warning | `previous_key` set: the old key file still answers 200 with the old key (ok: rotation window open), or not (warning) | | `probe.config` (host) | error | `--live`: the live configuration cannot be built | | `probe.response` (host) | ok, warning, error | `--live`: one line per engine: 200 ok, 202 warning (verification pending), anything else error | | `check.failed` | error | a registered `CheckInterface` threw; the line names the class | | `debounce.store` | ok, warning, error | `Check\DebounceStoreCheck`: off, `none`, `memory` (warning), a shared store probed ok, or unusable (error) | | `.installed` | ok, warning | `Adapter\OptionalPackage`: an optional package of the family is not installed (`sitemap.installed`, `verify.installed`, `history.installed`); warning when its block is configured and ignored. `verify.installed` also carries the `--sample` gate and is an **error** there, see "Optional packages" below | ## Adapters | Code | Package | Levels | Line | |---|---|---|---| | `wiring.messenger` | symfony-bundle | warning | `dispatch: messenger` without a routed transport | | `wiring.doctrine` | symfony-bundle | ok, warning | entity hooks active or not | | `queue.dispatch` | laravel, yii2 | ok | `dispatch` is not `queue`: what happens instead | | `dispatch.mode` | yii3 | ok, error | `sync` / `none`, or the `DispatcherInterface` the application replaced in the container; error when the dispatcher cannot be built | | `queue.connection` | laravel | error | the queue connection is not defined | | `queue.component` | yii2 | error | the yii2-queue component does not exist | | `queue.driver` | laravel, yii2 | ok, warning | the queue driver: `sync` (warning, nothing is retried) or a real one | | `eloquent.enabled` | laravel | ok, warning | model observers active or not | | `active_record.enabled` | yii2, yii3 | ok, warning, error | ActiveRecord hooks active or not; Yii3: error when the observer is not installed (the package's bootstrap did not run) | | `url_manager.key_file` | yii2 | ok, error | the key file is not served by the application, or `key_file` is misconfigured | | `url_manager.pretty_url` | yii2 | error | `enablePrettyUrl` is off, `/.txt` cannot be routed | | `url_manager.rule` | yii2 | ok, error | the key file URL rule is registered, or missing (component not in `bootstrap`) | | `router.key_file` | yii3 | ok, error | the key file is not served by the application (`key_file.enabled: false`), or `key_file` is misconfigured | | `router.route` | yii3 | ok, error | the route `indexnow/key-file` is in the route collection (or the console says the web application serves it), or missing (the `routes` group of the package is not merged) | | `router.locales` | symfony-bundle, laravel, yii2 (the core's `Check\LocalesCheck`) | ok, warning | a rule asks for `locales: 'all'` while the locale list of the application is empty, so one URL in the current locale is generated instead of one per locale (`framework.enabled_locales` in Symfony, `router.locales` in Laravel and Yii2): one text, the option and the classes named. The `ok` line naming the configured locales and the route parameter is written by the adapters that know the parameter (Laravel, Yii2); the bundle prints the warning only | ## Optional packages The line of an optional package that is not installed is `.installed` (above). With the package installed: | Code | Package | Levels | Line | |---|---|---|---| | `sitemap.spool` | sitemap | ok, warning, error | where sitemap documents are spooled; error when `spool: disk` has no writable directory | | `verify.installed` | verify, and the core's `Check\SampleGateCheck` | ok, warning, error | with the package: `verify: installed, disabled (verify.enabled: false)` or `verify: enabled (redirect: …, non_canonical: …, origin_error: …)`. Without the package the line comes from the gate the core ships in front of it: the `.installed` line above plus ` — pre-flight checks off` (warning when a `verify` block is configured and ignored), and **error** `check --sample needs indexnowkit/verify (composer require indexnowkit/verify)` when `--sample` or `--sample-class` was given. Every adapter used to carry a copy of that gate; it is one class in the core now | | `verify.dispatch` | verify | warning | `verify.enabled` with `dispatch: sync`: the pre-flight GETs run inside the web request; use a queue | | `verify.transport` | verify | ok | `verify.enabled` with an `http.client` of the application: the line says that the pre-flight does **not** use it — it builds its own PSR-18 client with `verify.timeout` and no redirects, because a client that follows redirects internally would hide the 3xx the pre-flight exists to see. `http.client` still sends the submissions | | `verify.sample` (host) | verify | ok, warning, error | one line per `--sample` / `--sample-class` URL: `verify sample {url}: HTTP 200, index, canonical: self, robots: allowed`; noindex, disallow, a foreign canonical, a redirect, a 4xx/5xx or a transport failure are **warnings**, never errors; with the package and no sample: ok `no sample given`. Without the package a sample is an error under `verify.installed`, not under this code — the code exists only while the package does | | `history.store` | history | ok, error | the configured store (`history: pdo store (indexnow_submissions)`, `history: psr16 store (500 records kept)`, `history: custom store ()`, `history: installed, no store configured (history.store)`); error with the exception and the migration hint when the store fails (a missing table) | | `history.records` | history | ok | `history: 1 240 records, last 3 min ago`, or `history: no records yet` | Application checks (`CheckInterface` implementations you register) choose their own codes; leave the core areas (`config`, `environment`, `key`, `key_file`, `probe`, `debounce`) to the core. A line without a code is allowed but appears as `"code": null` in the JSON. # Submission store `Submission\SubmissionStoreInterface` is where the `Submitter` remembers what it did: one record per `Result` after every `submit()`, written after the listeners and the PSR-14 event. The core ships the interface, the value object `Submission\SubmissionRecord` (`urls`, `result`, `at`) and `Submission\NullSubmissionStore`, which keeps nothing and is what every adapter wires by default. [`indexnowkit/history`](../history/index.md) brings the two stores every adapter can wire with one option (`history.store: psr16` — a ring buffer in the debounce cache, `history.store: pdo` — a table, migration in the package's `docs/migrations.md`), the `history` and `status` commands (`--json`), the `history.store` / `history.records` lines of `check` and, in Symfony, a "Recent submissions" table in the profiler; its `HistoryStoreInterface` extends this one with `count()`, `last()` and `purge()`. A store of your own still plugs into the same point and the commands work with it (`recent()` only). ## Wiring | Where | How | |---|---| | plain PHP | `IndexNowKit::create($config, submissionStore: $store)` or `new Submitter(..., store: $store, clock: $clock)` | | `Adapter\ServicesBuilder` | `->submissionStore($store)` (an instance or a `Closure(Services): SubmissionStoreInterface`) | | Symfony bundle | replace the service `indexnowkit.submission_store` (alias `Submission\SubmissionStoreInterface`) | | Laravel | `$this->app->singleton(SubmissionStoreInterface::class, MyStore::class)` after the provider | | Yii2 | component property `submissionStore` (instance, class name, configuration array or component id) | The console submitters (`submit --force`, `--dry-run`, the sitemap command) record through the same store. ## What becomes a record | Situation | Records | |---|---| | one URL, `engines: ['api']`, 200 | 1 record, `status: ok`, `engine: api` | | one URL, `engines: ['api', 'yandex']` | 2 records, one per engine; `lastFor($url)` returns the later one, whatever its status | | 10 000 + 1 URLs, one engine | 2 records (one per batch of `batch.max_urls`) | | a URL of another host next to a URL of `base_url` | 2 records (one per host) | | `dry_run: true` | 1 record per engine × batch, `status: skipped`, `reason: dry_run`, the engine it would have reached | | `enabled: false`, a debounced URL, an unmanaged host, an invalid URL | 1 record per host (per URL for an invalid one), `status: skipped`, `engine: none` (`Result::NO_ENGINE`) | | 429 / 5xx / network failure | 1 record, `status: failed`, `retryable: true`; the retry of a queue job writes its own record later | | a listener throws | nothing changes: listeners are called before the store and are isolated | Every record gets the same `at`: the Submitter's clock (`Psr\Clock\ClockInterface`, `Clock\SystemClock` by default) read once per `submit()`. ## Contract for an implementation - `record()` must not throw. If it does, the Submitter logs `indexnow: submission store failed, {count} result(s) not recorded: {error}` once for the call and delivery is not affected. - `recent()` returns the newest records first; `host` and `status` are filters, both optional. - `lastFor($url)` matches the URL as it is stored in `Result::$urls`: normalized for everything that reached the pipeline, as given for an invalid URL (`reason: invalid_url`). An index from URL to record is the store's job; a linear scan is fine for a ring buffer of a few hundred entries. - Tier Implement ([bc.md](bc.md)): the core calls you, methods are not added in a minor. Before 1.0 the interface is new and may still move; it has to live through one minor unchanged before 1.0 is tagged. # IndexNow console runners — `indexnowkit/console` The bodies of the `check`, `submit`, `submit-`, `explain` and `key:generate` commands every framework adapter of the family ships (`bin/console indexnow:check`, `php artisan indexnow:check`, `php yii indexnow/check`), and the one declaration of their arguments and options. An adapter's command is input parsing over a runner from this package; every framework prints the same thing, and an application reuses a runner from its own command (a tenant loop over `SubmitSubjectsRunner` is a ten-line command). Split out of [`indexnowkit/core`](../core/index.md) in core 0.7 so the core no longer imports `symfony/console`; the FQCN (`IndexNowKit\Console\*`) are unchanged. [](https://packagist.org/packages/indexnowkit/console) [](https://packagist.org/packages/indexnowkit/console) [](https://github.com/indexnowkit/php/actions)   [](https://github.com/indexnowkit/php/blob/main/packages/console/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/console/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Install ```bash composer require indexnowkit/console # brings indexnowkit/core and symfony/console ^6.4 || ^7.0 || ^8.0 ``` With a framework adapter you install nothing: `indexnowkit/symfony-bundle`, `indexnowkit/laravel` and `indexnowkit/yii2` and `indexnowkit/yii3` require this package and register the commands. `indexnowkit/sitemap` builds its `sitemap` command on it too. ## What is inside | Command | Runner | What the adapter supplies | |---|---|---| | `check` | `Console\CheckRunner` | a closure that builds `Config` from the raw configuration (throws `ConfigurationException`); `Check\CheckInterface` services for adapter wiring and the add-on packages | | `submit ...` | `Console\SubmitRunner` | — | | `submit- [ids]` | `Console\SubmitSubjectsRunner` + `SubmitSubjectsOptions` | a `Console\SubjectLoaderInterface`: class resolution (FQCN or the framework's short name), objects by id, first N objects | | `explain ` | `Console\ExplainRunner` | the same loader | | `key:generate` | `Console\KeyGenerateRunner` | the default env file path | Every runner renders to a `Symfony\Component\Console\Style\SymfonyStyle` (Laravel's `OutputStyle` is one) and returns a `Console\ExitCode`. Shared by all of them: `Console\Definitions` (the arguments and options of every command, declared once — `CommandDefinition`, `ArgumentDefinition`, `OptionDefinition` — and rendered by the adapter into its framework's command), `Console\ResultFormatterInterface` (`ResultRenderer`: the table or `--json`; an application replaces it to match its own CLI), `Console\Vocabulary` (the words that differ between frameworks: "entity" / "model" / "record", `bin/console` / `php artisan` / `php yii`, where the configuration lives), `Console\ClassNameResolver` (a short class name to a FQCN, with the error texts). The submitters the commands use for `--force` / `--dry-run` (`Adapter\SubmitterFactory`) and the aggregate of a batched run (`Submission\ResultSummary`) stay in the core: they are not CLI concerns. ## Commands: how an application on symfony/console registers them Since 0.5.0 the commands are classes of this package (`IndexNowKit\Console\Command\*`), the ones the Symfony bundle and the Yii3 package register. An application with a `Symfony\Component\Console\Application` of its own registers them the same way: build the runners, hand each command what varies by constructor — nothing here knows a framework or a container. ```php use IndexNowKit\Check\Checker; use IndexNowKit\Check\SampleOptions; use IndexNowKit\Config; use IndexNowKit\Console\CheckRunner; use IndexNowKit\Console\Command\CheckCommand; use IndexNowKit\Console\Command\ConfigCommand; use IndexNowKit\Console\Command\KeyGenerateCommand; use IndexNowKit\Console\Command\SubmitCommand; use IndexNowKit\Console\ConfigRunner; use IndexNowKit\Console\ConfigSourceInterface; use IndexNowKit\Console\KeyGenerateRunner; use IndexNowKit\Console\SubmitRunner; use IndexNowKit\Console\Vocabulary; use IndexNowKit\IndexNowKit; use Symfony\Component\Console\Application; final class EnvConfigSource implements ConfigSourceInterface // what check and config read { public function raw(): array { return Config::fromEnv()->toArray(); } public function build(): Config { return Config::fromEnv(); } // throws ConfigurationException when invalid public function packages(): array { return []; } // the blocks of the installed optional packages } $indexNow = IndexNowKit::create(Config::fromEnv()); $words = new Vocabulary(cli: 'bin/indexnow', configLocation: 'the INDEXNOW_* env vars'); $submitters = $indexNow->submitterFactory(); // --force / --dry-run build their own submitter $application = new Application('indexnow'); $application->addCommands([ new CheckCommand(new CheckRunner(new Checker($indexNow->config, $indexNow->keys, $indexNow->transport), $words), new EnvConfigSource(), new SampleOptions()), new ConfigCommand(new ConfigRunner($words), new EnvConfigSource()), new SubmitCommand(new SubmitRunner($indexNow, $submitters)), new KeyGenerateCommand(new KeyGenerateRunner($words), envFileName: '.env'), // --write-env without a value: /.env ]); $application->run(); ``` `SubmitSubjectsCommand` (`indexnow:submit-`, its name is `Vocabulary::$submitSubjects`) and `ExplainCommand` need a `SubjectLoaderInterface` — the ORM of the application — and are registered by the adapters that have one. `indexnow:sitemap` is `IndexNowKit\Sitemap\Console\SitemapCommand` of `indexnowkit/sitemap`, `indexnow:history` and `indexnow:status` are `IndexNowKit\History\Console\HistoryCommand` / `StatusCommand` of `indexnowkit/history`; without the package, `Command\SitemapNotInstalledCommand`, `HistoryNotInstalledCommand` and `StatusNotInstalledCommand` stand in under the same names with the install line and exit 1. In a Symfony container `SubmitSubjectsCommand` is registered lazily with the `command` and `description` attributes of the `console.command` tag (its name is not an attribute of the class); every other command carries `#[AsCommand]` and is lazy on its own. ## Plain PHP ```php use IndexNowKit\Check\Checker; use IndexNowKit\Config; use IndexNowKit\Console\CheckRunner; use IndexNowKit\Console\Vocabulary; use IndexNowKit\IndexNowKit; use Symfony\Component\Console\Input\ArrayInput; use Symfony\Component\Console\Output\ConsoleOutput; use Symfony\Component\Console\Style\SymfonyStyle; $indexNow = IndexNowKit::create(Config::fromEnv()); $io = new SymfonyStyle(new ArrayInput([]), new ConsoleOutput()); $runner = new CheckRunner(new Checker($indexNow->config, $indexNow->keys, $indexNow->transport), new Vocabulary()); exit($runner->run($io, static fn(): Config => Config::fromEnv(), live: true)); ``` No framework at all? Install the CLI instead of writing this: [`indexnowkit/cli`](../cli/index.md) (`composer global require indexnowkit/cli`, a PHAR, a Docker image) registers these classes over `INDEXNOW_*` variables and a state file. Writing an adapter? [core/docs/adapters.md §14](../core/adapters.md) walks through the six commands; the bundle, the Laravel package, the Yii2 component and the Yii3 package are the reference wirings. ## Requirements PHP 8.2+, `indexnowkit/core ^0.13`, `symfony/console ^6.4 || ^7.0 || ^8.0`. ## Notes for AI assistants - Composer package `indexnowkit/console`: the command bodies (`IndexNowKit\Console\*Runner`), the command definitions (`IndexNowKit\Console\Definitions`) and, since 0.5.0, the symfony/console command classes themselves (`IndexNowKit\Console\Command\*`: `check`, `config`, `submit`, `submit-entity` / `submit-record`, `explain`, `key:generate`, the three "not installed" stubs) that the Symfony bundle, the Laravel package (artisan runs any symfony/console command; `submit-model` through a `LazyCommand`, the class argument named `model`) and the Yii3 package register; Yii2 (a `yii\console\Controller`) builds its actions on the runners. Since 0.5.0 also `Console\SubjectSampler` (the `--sample-class` sampler) and `Console\AbstractSubjectLoader` (the skeleton of an ORM loader). Framework users install an adapter, not this package. - Minimal complete snippet (every `use` included) — an application command over a runner: ```php use IndexNowKit\Console\SubmitRunner; use IndexNowKit\IndexNowKit; use Symfony\Component\Console\Style\SymfonyStyle; final class ReannounceCommand { public function __construct(private SubmitRunner $runner, private IndexNowKit $indexNow) {} public function run(SymfonyStyle $io): int { return $this->runner->run($io, ['https://www.example.com/pricing'], force: true, dryRun: false, json: false); } } ``` - Verify: the adapter's `check` command (`bin/console indexnow:check`, `php artisan indexnow:check`, `php yii indexnow/check`) is `CheckRunner`; every runner returns an `ExitCode` (`SUCCESS` 0, `FAILURE` 1, `INVALID` 2 for bad input) and never throws for remote errors. - Pitfalls: - Before core 0.7 these classes lived in `indexnowkit/core` with the same FQCN; only `Console\SubmitterFactory` (now `IndexNowKit\Adapter\SubmitterFactory`) and `Console\ResultSummary` (now `IndexNowKit\Submission\ResultSummary`) changed their namespace. - Option and argument names come from `Definitions` (`--force`, `--dry-run`, `--json`, `--live`, `--host`, `--probe-url`, `--limit`, `--event`, `--write-env`, `--length`): an adapter's command must not declare its own copies. - `--force` re-announces URLs inside the debounce window; `--dry-run` logs the request instead of sending it (`dry_run` in the configuration does the same for every submission). - Manual submission is `submitEntity()` in Symfony, `submitModel()` in Laravel, `submitRecord()` in Yii2 and Yii3; the commands are `indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record` (Yii2), `indexnow:submit-record` (Yii3). - `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`); Yii3 has `sync` | `none` only. ## Versioning SemVer; until 1.0 minor versions may contain breaking changes, listed in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/console/CHANGELOG.md). What the compatibility promise covers: [docs/bc.md](bc.md). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Backward compatibility `indexnowkit/console` follows SemVer and the tiers of the core's [docs/bc.md](../core/bc.md). **Before 1.0, minor versions may contain breaking changes**, listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/console/CHANGELOG.md). | Tier | Members | |---|---| | **Call** — signatures only grow by appended, defaulted parameters; pass anything past the first argument by name | `CheckRunner`, `ConfigRunner`, `SubmitRunner`, `SubmitSubjectsRunner`, `ExplainRunner`, `KeyGenerateRunner` (constructors and `run()`), `ResultRenderer`, `Vocabulary` (constructor: named arguments), `ClassNameResolver`, `SubjectSampler` (the `--sample-class` sampler over the adapter's loader; `PER_CLASS`), `Definitions::*` | | **Commands** — `final` classes over the runners, registered by an adapter on symfony/console; constructors take named arguments and grow only by appended, defaulted parameters | `Command\SubmitCommand`, `Command\SubmitSubjectsCommand`, `Command\ExplainCommand`, `Command\CheckCommand`, `Command\ConfigCommand`, `Command\KeyGenerateCommand`, `Command\SitemapNotInstalledCommand`, `Command\HistoryNotInstalledCommand`, `Command\StatusNotInstalledCommand`. The **name** of each is a contract (`#[AsCommand]`, or `Vocabulary::$submitSubjects` for `SubmitSubjectsCommand`); the description is not. `SubmitSubjectsCommand` and `ExplainCommand` take `string $classArgument = 'class'` (since 0.5.0): the name of the class argument as the adapter's command always called it (`model` in Laravel) — positional on the command line either way. `Command\NotInstalledCommand` is the one abstract class of the package: extend it for a stub of your own optional command, the message is its only constructor argument | | **Implement** — methods are not added without a major version | `SubjectLoaderInterface`, `ResultFormatterInterface`, `ConfigSourceInterface` (new in 0.5.0: before 1.0 a method may still be appended in a minor, listed under "Changed"), `AbstractSubjectLoader` (new in 0.5.0: the skeleton of an ORM loader — `findOne()` and `findMany()` are what you implement, `resolveClass()`, `byIds()`, `all()` and `guard()` are final; a protected method is not added without a major version) | | **Value objects** — `final readonly`, properties only appended with defaults | `CommandDefinition`, `ArgumentDefinition`, `OptionDefinition`, `SubmitSubjectsOptions` | | **Constants** — referenced, not hard-coded | `ExitCode::SUCCESS`, `FAILURE`, `INVALID`, `OptionDefinition::FLAG`, `VALUE`, `OPTIONAL_VALUE`, `LIST`, `CheckRunner::CONFIG_INVALID` | | **Documents** — the shape only grows by optional members | `docs/check.schema.json`, the JSON of `check --json` (`status`, `environment`, `items[].{level, code, message, host}`); the codes are the core's `docs/check-codes.md` | **Command surface.** The argument and option names, defaults and descriptions in `Definitions` are what the adapters render into their commands, so they are the public API of every adapter's CLI: an option is renamed only with a deprecation window on the adapter side. `CommandDefinition::laravelSignature()` is gone in 0.5.0 (artisan registers the command classes of this package since Laravel 0.15; the renderer had no other consumer). Descriptions and the printed texts of the runners are not API (they are written for humans and get improved); exit codes are. Not covered: log and exception message texts, anything under `tests/`. The package pins `indexnowkit/core ^0.13`: the runners take the core's `Config`, `Checker`, `Adapter\SubmitterFactoryInterface` and `Submission\ResultSummary`, so a core minor that changes them ships with a `console` minor. # IndexNow test kit — `indexnowkit/testing` The test suite every part of the family shares, as a `require-dev` package: the conformance scenarios of the specification as abstract PHPUnit cases you extend against *your* wiring (C01–C22 for anything that talks to the protocol, A01–A21 for an ORM adapter), the assertions of the HTTP and command scenarios (H01–H05) so a framework test parses its own response object and asserts once, an assertion for the README section AI assistants read, and the mock IndexNow server for end-to-end runs. It is what `indexnowkit/doctrine`, `indexnowkit/symfony-bundle`, `indexnowkit/laravel`, `indexnowkit/yii2` and `indexnowkit/yii3` test themselves with; an adapter for another framework starts here. The four test doubles (`FakeTransport`, `ArrayLogger`, `FrozenClock`, `RecordingDispatcher`) stay in [`indexnowkit/core`](../core/index.md) under `IndexNowKit\Testing`: they implement core interfaces and need no PHPUnit, so an application test suite gets them without this package. [](https://packagist.org/packages/indexnowkit/testing) [](https://packagist.org/packages/indexnowkit/testing) [](https://github.com/indexnowkit/php/actions)   [](https://github.com/indexnowkit/php/blob/main/packages/testing/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/testing/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Install ```bash composer require --dev indexnowkit/testing # brings indexnowkit/core; PHPUnit 11 is expected in your require-dev ``` Everything lives under `IndexNowKit\Testing\Conformance\`. ## Conformance kits Three abstract test cases turn [docs/spec/03](https://github.com/indexnowkit/spec/blob/main/03-conformance.md) into runnable scenarios against the facade your container built: ```php use IndexNowKit\IndexNowKit; use IndexNowKit\Testing\Conformance\CoreConformanceTestCase; use IndexNowKit\Testing\FakeTransport; final class CoreConformanceTest extends CoreConformanceTestCase { protected function kit(): IndexNowKit { return $this->container()->get(IndexNowKit::class); } protected function transport(): FakeTransport { return $this->container()->get(FakeTransport::class); } protected function secondHost(): ?string { return 'example.de'; } // a second entry of `hosts`, or null to skip C04 } ``` - `CoreConformanceTestCase` (C01, C03, C04, C06, C09–C12, C14, C19, C20): return the facade and the `FakeTransport` it is wired to; the scenarios use fresh URLs, so the debounce window is irrelevant. - `SubmissionStoreConformanceTestCase` (S01–S08): return a fresh `Submission\SubmissionStoreInterface` from `createStore()` (`supportsPurge()` when it has the `purge()` of `indexnowkit/history`): S01 record and read back, S02 newest first, S03 host filter, S04 status filter, S05 `lastFor()`, S06 limit, S07 several URLs of one Result, S08 empty store and `purge()`. - `OrmConformanceTestCase` (A01–A21, plus A05b/A05c): implement the driver — the transaction verbs of your data layer (`begin()`, `commit()`, `rollback()`), the end of a unit of work (`flush()`, `collectedCount()`), and fixtures with fixed rule shapes (`createPost()`, `createMultiPost()`, `createCategorizedPost()`, `createTag()`, `attachTag()`, `bulkUpdateTitle()`, …). The docblock of the class lists the rules every fixture must carry; the URL conventions (`postUrl()`, `ampUrl()`, `categoryUrl()`, `homeUrl()`) are overridable. `indexnowkit/doctrine` (`tests/OrmConformanceTest.php`) and `indexnowkit/laravel` (`tests/Conformance/`) are the reference drivers. A scenario that does not apply to your framework is documented in your README, not skipped silently. The scenario identifiers are a cross-language contract: a scenario is added, never renumbered. ## Assertions for HTTP and command tests The scenarios H01–H05 are the same in every framework, only the way a response or a command output is captured differs. Parse your framework's objects, assert here: ```php use IndexNowKit\Testing\Conformance\CheckOutputAssertions; use IndexNowKit\Testing\Conformance\KeyFileAssertions; // H01: 200, text/plain, the key as the body, Cache-Control with public and max-age, Vary: Host only with a hosts map KeyFileAssertions::assertKeyFileResponse($response->getStatusCode(), $response->headers->all(), $response->getContent(), $key, maxAge: 300, expectVaryHost: true); // H02/H03: an unknown key, another host's key, key_file.enabled: false KeyFileAssertions::assertNotServed($response->getStatusCode()); // H04/H05: the check command CheckOutputAssertions::assertExitCode(0, $exitCode, $output); // the output is the failure message CheckOutputAssertions::assertReady($output, 'www.example.com'); // ": key file OK" and the closing line CheckOutputAssertions::assertKeyFileHint($output, 403); // the status and the hint about what the engines do ``` `Cache-Control` is compared by directive (frameworks order them differently), header names in any case, values as a string or a list. The phrases are the ones the core's `Checker` and the `check` command print, so your test does not carry a copy of them. `ReadmeAssertions::assertAiNotes($packageDir, $commands, $optionKeys)` checks the "Notes for AI assistants" section of a package README (EN and RU): present, with a PHP snippet that carries its `use` lines, naming only commands of the family and configuration keys the package accepts. Every package of the family runs it; an adapter of yours can too. `OptionalPackageAssertions::assertDetected($checkOutput)` checks what `check` prints about `indexnowkit/sitemap`, `indexnowkit/verify` and `indexnowkit/history` when the adapter leaves the predicates to detection: a `: not installed` line with the install command for every package that is really absent, and no such line for one that is present. Under `INDEXNOWKIT_OPTIONAL_PACKAGES=absent` (the `optional-packages-absent` CI job, which physically removes the three packages) the absence itself becomes an assertion — an adapter that loads a class of the package to decide whether it is installed passes with the package in `vendor/` and is a fatal error without it. ## The mock IndexNow server For end-to-end runs through a real PSR-18 client, without touching the engines: ```bash php -S 127.0.0.1:8089 vendor/indexnowkit/testing/resources/mock-server/router.php ``` Point `engines` at `http://127.0.0.1:8089/indexnow` (plain HTTP is accepted on loopback hosts only) and pick the behaviour with the `X-Mock-Scenario` header or `?scenario=`: `ok200` (default), `pending202`, `bad400`, `forbidden403`, `unprocessable422`, `ratelimit429` (`Retry-After: 2`), `ratelimit429-then-ok` and `flaky500-then-ok` (`?n=` failures first), `timeout`. The server validates the body like the real endpoint (host, key, `urlList`, at most 10 000 URLs, every URL on the declared host → 422 otherwise), serves `GET /{key}.txt` for the keys listed in the `MOCK_KEYS` environment variable (comma separated), and logs every request: `GET /_mock/requests` returns the log as JSON, `DELETE /_mock/requests` clears it. Start it from a test with `proc_open` on a free port, as the core's `Psr18TransportTest` does. ## Requirements PHP 8.2+, `indexnowkit/core ^0.7`, PHPUnit 11 in your `require-dev` (the test cases extend `PHPUnit\Framework\TestCase`). ## Notes for AI assistants - Composer package `indexnowkit/testing`, `require-dev` only: conformance test cases and assertions for a test suite that uses `indexnowkit/core` or one of its adapters; nothing here runs in an application. - Minimal complete snippet (every `use` included) — an adapter's conformance test: ```php use IndexNowKit\IndexNowKit; use IndexNowKit\Testing\Conformance\CoreConformanceTestCase; use IndexNowKit\Testing\FakeTransport; final class CoreConformanceTest extends CoreConformanceTestCase { protected function kit(): IndexNowKit { return $this->app->get(IndexNowKit::class); } // the facade the container built protected function transport(): FakeTransport { return $this->app->get(FakeTransport::class); } // the transport it is wired to } ``` - Verify: `vendor/bin/phpunit` runs the scenarios; a red C-scenario is a wiring problem in the adapter, not in the kit. - Pitfalls: - The test doubles (`FakeTransport`, `ArrayLogger`, `FrozenClock`, `RecordingDispatcher`) are `IndexNowKit\Testing\*` in the core; the kits and assertions are `IndexNowKit\Testing\Conformance\*` here. Before core 0.7 the assertions lived in the core under `IndexNowKit\Testing\*`. - `assertKeyFileResponse()` expects `Vary: Host` only when the application serves several hosts (a `hosts` map) and refuses it otherwise. - `CheckOutputAssertions::assertExitCode()` takes the whole output as its third argument so a failing test shows what the command printed. - The mock server accepts plain HTTP only on loopback hosts; `engines` must name the full endpoint (`http://127.0.0.1:8089/indexnow`). - `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`): a conformance test of an adapter runs with `dispatch: sync`. ## Versioning SemVer; until 1.0 minor versions may contain breaking changes, listed in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/testing/CHANGELOG.md). What the compatibility promise covers: [docs/bc.md](bc.md). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Backward compatibility `indexnowkit/testing` follows SemVer and the tiers of the core's [docs/bc.md](../core/bc.md). **Before 1.0, minor versions may contain breaking changes**, listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/testing/CHANGELOG.md). | Tier | Members | |---|---| | **Call** — signatures only grow by appended, defaulted parameters; pass anything past the first argument by name | `KeyFileAssertions::*`, `CheckOutputAssertions::*`, `ReadmeAssertions::*` and their constants (`SECTION_EN`, `SECTION_RU`, `FAMILY_COMMANDS`, `CROSS_ADAPTER_KEYS` grow, never shrink) | | **Extend** — the abstract driver methods of a kit only grow by appended methods with a default implementation; a scenario is only added, never removed or renumbered, in a minor | `CoreConformanceTestCase`, `OrmConformanceTestCase`, `SubmissionStoreConformanceTestCase` | | **Resource** — the file path and the scenario names are stable; scenarios are added, not renamed | `resources/mock-server/router.php` (`X-Mock-Scenario`: `ok200`, `pending202`, `bad400`, `forbidden403`, `unprocessable422`, `ratelimit429`, `ratelimit429-then-ok`, `flaky500-then-ok`, `timeout`; `GET /_mock/requests`, `DELETE /_mock/requests`; `MOCK_KEYS`) | The conformance identifiers (C01–C22, A01–A21, H01–H06) are a cross-language contract of the specification: they are frozen at 1.0 of the family. What an assertion *accepts* may become stricter in a minor when the specification does (listed in the changelog); what it *rejects* never becomes accepted silently. Not covered: the failure-message texts of the assertions, anything under `tests/`. The package pins `indexnowkit/core ^0.7`: it reads `Config::OPTIONS` and the test doubles of the core, so a core minor that renames them ships with a `testing` minor. # IndexNow sitemap reader — `indexnowkit/sitemap` Re-announce a site's URLs to Yandex, Bing and the other [IndexNow](https://www.indexnow.org) engines from its own sitemap: a sitemap index, gzip-compressed and text sitemaps are streamed entry by entry and submitted in batches, so a million-URL sitemap never lives in memory. The `sitemap` command of every framework adapter of the family (`indexnowkit/symfony-bundle`, `laravel`, `yii2`, `yii3`) is this package; in plain PHP it is three lines over [`indexnowkit/core`](../core/index.md). **Google: no.** Google does not support IndexNow, its sitemap ping endpoint is gone and the Indexing API is limited to `JobPosting` / `BroadcastEvent`. Keep your sitemap for Google; this package announces it to the IndexNow engines only. IndexNow is a notification, not indexing: the engine decides whether and when to crawl. A run without `--changed-since` re-announces the whole sitemap: do that once, then schedule `--changed-since "1 day"`. `--changed-since` relies on ``; a generator that writes `lastmod = now()` for every URL turns every run into a full run, and entries without `lastmod` are skipped when the option is set. [](https://packagist.org/packages/indexnowkit/sitemap) [](https://packagist.org/packages/indexnowkit/sitemap) [](https://github.com/indexnowkit/php/actions)    [](https://github.com/indexnowkit/php/blob/main/packages/sitemap/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/sitemap/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Install ```bash composer require indexnowkit/sitemap # brings indexnowkit/core; needs ext-xmlreader, ext-zlib for .gz ``` With a framework adapter you install nothing: the adapter requires this package and registers the command (`bin/console indexnow:sitemap`, `php artisan indexnow:sitemap`, `php yii indexnow/sitemap`) with its `sitemap` configuration block. ## Plain PHP ```php use IndexNowKit\Config; use IndexNowKit\IndexNowKit; use IndexNowKit\Sitemap\SitemapConfig; use IndexNowKit\Sitemap\SitemapReader; $kit = IndexNowKit::create(Config::fromEnv()); $reader = SitemapReader::fromConfig(SitemapConfig::fromArray(['spool' => 'auto']), $kit->transport); $batch = []; foreach ($reader->read('https://www.example.com/sitemap.xml', new DateTimeImmutable('-1 day')) as $entry) { $batch[] = $entry->url; if (\count($batch) === $kit->config->batchMaxUrls) { $kit->submit($batch); $batch = []; } } $kit->submit($batch); ``` `read()` yields `SitemapEntry` objects (`url`, `lastmod`), optionally only those whose `` is newer than `$changedSince` (entries without `lastmod` are then skipped). The root may be an http(s) URL, a local path or a `file://` URL; nested sitemaps of an index are fetched over the transport you pass (the one the facade submits through, so `http.client` and `http.timeout` apply). `$kit->transport` is `null` when the facade was built around a custom submitter: use `Http\TransportFactory::lazy($kit->config)` then. ## Configuration `SitemapConfig::fromArray()` reads the `sitemap` block every adapter exposes; `SitemapConfig::OPTIONS` lists its dotted keys for `Config::unknownOptions()`. | Key | Default | | |---|---|---| | `sitemap.enabled` | `true` | `false`: the adapter registers no command and no reader | | `sitemap.url` | `null` | sitemap read when the command gets no argument; `null` = `/sitemap.xml` | | `sitemap.max_depth` | `3` | levels of `` followed below the root (`0` = the root only) | | `sitemap.max_sitemaps` | `1000` | documents fetched per run, root included | | `sitemap.max_bytes` | `52428800` | size cap of one uncompressed document (50 MiB, the protocol maximum; at least 1024) | | `sitemap.allow_foreign_hosts` | `false` | follow nested sitemaps on other origins (CDN-hosted parts); `--allow-foreign-hosts` enables it for one run | | `sitemap.spool` | `auto` | where a document is kept while parsing: `auto` = temp file, memory when the temp dir is not writable; `disk` = temp file or fail; `memory` | | `sitemap.spool_dir` | `null` | directory of the temp files (`sys_get_temp_dir()`); point it at a writable volume on a read-only filesystem | | `sitemap.fetch_retries` | `2` | extra attempts (1 s, 2 s, 4 s apart) after a network failure or 5xx while fetching a document; 4xx and broken documents are never retried | ## How it stays safe and small Memory stays flat whatever the sitemap size: every document is spooled (`Sitemap\Spool`: a temp file, or memory on a read-only filesystem; straight from the socket when the transport implements `Http\StreamingTransportInterface`, as `Psr18Transport` does), gzip is inflated chunk by chunk into a second spool, and `XMLReader` walks the spool through the `indexnowkit-spool://` wrapper with a few KiB of buffers. Nested sitemaps must live on the origin of the root unless `allow_foreign_hosts` says otherwise; recursion depth, document count and document size (before and after gunzip) are capped; external entities and network access are disabled in the XML parser. A failing nested sitemap is logged and skipped; a failing root throws `Http\Exception\TransportException`, and a response shorter than its `Content-Length` is a truncated download, never a document. Details in [SECURITY.md](https://github.com/indexnowkit/php/blob/main/packages/sitemap/SECURITY.md). A `` may name any host, so the command keeps only the entries whose host this site has a key for (`base_url` plus the `hosts` map — `KeyProviderInterface::managedHosts()`); the rest are dropped before anything fetches or submits them, and one line says how many and on which hosts. This matters most with [`indexnowkit/verify`](../verify/index.md): its pre-flight GETs every URL of the batch, so without the filter a sitemap **built from user content** (or a swapped one) could walk internal addresses. **When the sitemap is built from user content, set `strict_hosts: true`** as well, so a host that slips through is refused a key rather than sent under the default one. With no key at all (neither `base_url` nor a `hosts` map) there is nothing to compare against: the entries go as they are, with a warning. ## The command `Sitemap\Console\SitemapRunner` is the body of `sitemap [url]` (`--changed-since "1 day"`, `--allow-foreign-hosts`, `--force`, `--dry-run`, `--json`, `--no-verify`, `--new-only`); it streams, submits every `batch.max_urls` URLs, and submits the pending batch before reporting a mid-run failure (the re-run is idempotent, what was read is still worth announcing). `--new-only` submits only the URLs that are new or changed since the last run: the runner keeps the fingerprint of every entry it announced (URL plus ``, or none) in a `Sitemap\SeenStoreInterface` the application provides — the [`indexnow` CLI](../cli/index.md) keeps one in its state file, so a sitemap without `lastmod` still gets each change announced once; a framework adapter has no such store yet and answers `--new-only needs a store of seen URLs` with exit 2. Every run that is not `--dry-run` teaches the store (a failed batch excepted), so "the whole sitemap once, then `--new-only` in cron" announces each change exactly once. `--no-verify` submits past the pre-flight of [`indexnowkit/verify`](../verify/index.md) when the adapter has it enabled (a sitemap is the site's own list of its URLs, and a batch above `verify.max_batch` would go unverified anyway, with a warning); without the package the flag is accepted and changes nothing. `--force` ignores the debounce window (URLs announced within the last `debounce.per_url` seconds are sent again): for a deliberate one-off re-run, never in a schedule. `batch.max_urls` (10 000) is the protocol's ceiling, not a target: smaller batches are accepted just as well, and a scheduled run with `--changed-since` normally sends a few. The `check` command of every adapter carries `Sitemap\Check\SitemapSpoolCheck`: where documents are spooled, and whether that directory is writable — the kind of thing that otherwise only shows up on the first scheduled run. An application decorates the source (filter, rewrite) or replaces it (another format, a database) by implementing `Sitemap\SitemapSourceInterface` and binding it under the adapter's alias. Writing an adapter? [docs/adapters.md](adapters.md). ## For adapter authors `Sitemap\Adapter\SitemapServices` is what a framework adapter wires for this package, in one place: the predicate (`package()`), the owned options, the validated block, the reader, the spool check, the body of the `sitemap` command (`runner()`) and the command itself (`command()`: `Sitemap\Console\SitemapCommand`, the class an adapter on symfony/console registers instead of writing one) — as static functions over the pieces, with `*For()` twins over the core's `Adapter\Services` for a runtime graph. The Symfony bundle, the Laravel, the Yii2 and the Yii3 adapters build on it; see [adapters.md](../core/adapters.md) of the core. ## Requirements PHP 8.2+, `ext-xmlreader`, `indexnowkit/core ^0.13`, `indexnowkit/console ^0.5` (the command and its body render through `symfony/console`); `ext-zlib` for gzip-compressed sitemaps. ## Notes for AI assistants - Composer package `indexnowkit/sitemap` (add-on of `indexnowkit/core`; `ext-xmlreader`, `ext-zlib` for gzip). With a framework adapter it is the `sitemap` command — nothing to wire. - Minimal complete snippet (every `use` included): ```php use IndexNowKit\Config; use IndexNowKit\IndexNowKit; use IndexNowKit\Sitemap\SitemapConfig; use IndexNowKit\Sitemap\SitemapReader; $indexNow = IndexNowKit::create(Config::fromEnv()); $reader = SitemapReader::fromConfig(SitemapConfig::fromArray(['spool' => 'auto']), $indexNow->transport); foreach ($reader->read('https://www.example.com/sitemap.xml') as $entry) { $indexNow->collect([$entry->url]); } $indexNow->flush(); // batches of batch.max_urls, debounced ``` - Verify: the adapter's `check` command prints the `sitemap:` spool line; `bin/console indexnow:sitemap --dry-run`, `php artisan indexnow:sitemap --dry-run`, `php yii indexnow/sitemap --dry-run`. - Pitfalls: - `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`); Yii3 has `sync` | `none` only. - Locales: `router.locales` in Laravel, Yii2 and Yii3, `framework.enabled_locales` in Symfony; `locales: 'all'` on a rule uses that list. - `url:` names an accessor (method or property) that returns the URL; `urls:` is a list of literal URLs. Never put a literal in `url:`. - A string in `when:` is an accessor read as truthy (`published`, `isPublished`). A status string needs `Equals`: `when: new Equals('status', 'published')` (`IndexNowKit\Attribute\Param\Equals`). - Manual submission is `submitEntity()` in Symfony, `submitModel()` in Laravel, `submitRecord()` in Yii2 and Yii3; the commands are `indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record` (Yii2), `indexnow:submit-record` (Yii3). Bulk queries (`update()`, `DB::table()`, `updateAll()`) fire no hooks: submit afterwards with those. - Laravel has two classes called `IndexNowKit`: the facade `IndexNowKit\Laravel\Facades\IndexNowKit` and the core service `IndexNowKit\IndexNowKit` (inject by type). Yii2 exposes the core through `Yii::$app->indexnow->kit()`; Yii3 defines `IndexNowKit\IndexNowKit` in the container. - Outside production a configured key with `dry_run` unset makes `check` fail (a staging copy would submit real URLs): set `dry_run: true` there, or `dry_run: false` explicitly when it submits on purpose. - Unknown configuration keys are warned about at boot (typos such as debounce.per_urls); the key list is `Config::OPTIONS` plus the adapter's own keys. ## Versioning SemVer; until 1.0 minor versions may contain breaking changes, listed in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/sitemap/CHANGELOG.md). What the compatibility promise covers: [docs/bc.md](bc.md). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Wiring the `sitemap` command into an adapter The command body, the reader and the check live here; an adapter parses its own input and binds three objects. Everything reads one `SitemapConfig`, built from the raw `sitemap` block of the adapter's configuration. ```php use IndexNowKit\Sitemap\Check\SitemapSpoolCheck; use IndexNowKit\Sitemap\Console\SitemapOptions; use IndexNowKit\Sitemap\Console\SitemapRunner; use IndexNowKit\Sitemap\SitemapConfig; use IndexNowKit\Sitemap\SitemapReader; $sitemap = SitemapConfig::fromArray($raw['sitemap'] ?? []); // throws ConfigurationException naming the key $reader = SitemapReader::fromConfig($sitemap, $kit->transport ?? TransportFactory::lazy($kit->config), $logger); $check = new SitemapSpoolCheck($sitemap); // add it to the checks of your `check` command $runner = new SitemapRunner($kit, $reader, $submitterFactory, $sitemap->url, $formatter, sitemapUrlOption: 'myfw.sitemap.url'); $exit = $runner->run($io, new SitemapOptions($argument, $changedSince, $allowForeignHosts, $force, $dryRun, $json)); ``` - **Configuration.** Add `SitemapConfig::OPTIONS` to the keys your `Adapter\ConfigFactory` owns (`ownedOptions: [...MY_OPTIONS, ...SitemapConfig::OPTIONS]`), so a typo inside the block is warned about. Do not list a bare `sitemap` key: it would stop `Config::unknownOptions()` from looking inside the block. At runtime build the block with `SitemapConfig::loadOrDisabled($block, $logger, 'php artisan indexnow:check')`: an invalid block is one `critical` line and `SitemapConfig::disabled()`, the way the core `ConfigFactory::load()` treats the core options (a DI container that validates at compile time, like the bundle, uses `fromArray()` and lets the build fail); when `enabled` is false, register no command (bundle) or refuse to run it with `sitemap.enabled is false.` and exit `INVALID` (Laravel, Yii2). - **Log lines.** The one line the wiring above adds, so operators can grep for it (the core's `docs/operations.md` lists the rest): | Level | Message | |---|---| | `critical` | `indexnow: invalid sitemap configuration, the sitemap command is disabled until it is fixed: {error} (run "{check}")` — `loadOrDisabled()`, `{check}` is the adapter's check command (Laravel, Yii2) | - **Transport.** The reader fetches over the transport the facade submits through, so `http.client` and `http.timeout` apply and nothing is discovered twice. `$kit->transport` is `null` only when the facade was built around a custom submitter; `Http\TransportFactory::lazy($kit->config)` covers that. - **Source.** Type the command against `SitemapSourceInterface` and expose the reader under an alias of it, so an application can decorate the source (filter, rewrite) or replace it. `--allow-foreign-hosts` only reaches the shipped `SitemapReader`; the runner warns when the configured source is something else. Whatever the source, the runner keeps only the entries on hosts `$kit->keys->managedHosts()` names, and counts the rest in one line: a replacement source does not have to filter hosts itself, and cannot opt out of it. - **Output.** The runner streams, submits every `batch.max_urls` URLs through `Adapter\SubmitterFactory::choose()` (`--force`/`--dry-run` get a separate submitter), folds results into `Submission\ResultSummary`, and submits the pending batch before reporting a mid-run failure; `--json` keeps stdout machine-readable (the error goes to stderr). Exit codes are `Console\ExitCode` of `indexnowkit/console`. - **Words.** The only framework-specific string is `sitemapUrlOption`, printed in `Give a sitemap URL, or configure or base_url.` when no sitemap is known. - **`--new-only` and the store of seen URLs.** The option is declared for every adapter (`Definitions::sitemap()`), the behaviour needs a `SeenStoreInterface` (`unseen()` streams the entries whose fingerprint — URL plus ``, or none — is unknown or changed; `remember()` records a batch). Pass one as the named `seen:` argument of `SitemapRunner` / `SitemapServices::runner()`; without it the option answers `SitemapRunner::NO_SEEN_STORE` with exit 2, and nothing is read. The runner remembers every batch whose results did not fail, with or without the option, never with `--dry-run`. The `indexnow` CLI keeps the store in its sqlite state file (`indexnow_sitemap_seen`: `url`, `fingerprint`, `seen_at`); the framework adapters do not ship one yet — a PSR-16 store would lose fingerprints at TTL, so it will be a table over the adapter's connection, when asked for. The reference wiring: `IndexNowKitLoader` of the Symfony bundle (services `indexnowkit.sitemap_config`, `indexnowkit.sitemap_reader`, `indexnowkit.console.sitemap`), `IndexNowKitServiceProvider::registerDiagnostics()` in Laravel, `IndexNowComponent::sitemapConfig()` / `sitemapSource()` in Yii2. # Backward compatibility `indexnowkit/sitemap` follows SemVer and the tiers of the core's [docs/bc.md](../core/bc.md). **Before 1.0, minor versions may contain breaking changes**, listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/sitemap/CHANGELOG.md). | Tier | Members | |---|---| | **Call** — signatures only grow by appended, defaulted parameters; pass anything past the first argument by name | `SitemapReader` (constructor, `fromConfig()`, `read()`, `parse()`), `SitemapConfig` (constructor, `fromArray()`, `disabled()`), `Console\SitemapRunner` (constructor and the static `changedSince()`: the clock, `$now` and the store of seen URLs are appended, optional), `Console\SitemapCommand` (the `indexnow:sitemap` name is a contract, the description is not), `Console\SitemapOptions`, `Check\SitemapSpoolCheck`, `Adapter\SitemapServices` (`runner()` grows by the named `$seen`), `Spool::create()`, `probeDisk()`, `uri()`, `close()` | | **Implement** — methods are not added without a major version | `SitemapSourceInterface`, `SeenStoreInterface` (new in 0.9: `unseen()`, `remember()`; a capability the runner needs later comes as a new interface) | | **Value objects** — `final readonly`, properties only appended with defaults | `SitemapEntry`, `SitemapConfig` | | **Constants** — referenced, not hard-coded; values may change in a minor | `SitemapReader::MAX_XML_BYTES`, `MAX_SITEMAPS`, `SitemapConfig::OPTIONS`, `DEFAULT_MAX_DEPTH`, `DEFAULT_FETCH_RETRIES`, `MIN_MAX_BYTES`, `Spool::SCHEME` | | **Enum** — closed set | `SpoolMode` | Not covered: `Spool`'s `stream_*` methods (the PHP stream-wrapper protocol, `@internal`), log and exception message texts, anything under `tests/`. Adapters pin `indexnowkit/sitemap`, `indexnowkit/core` and `indexnowkit/console` together: a `SitemapReader` of this package over an older core is refused by Composer, on purpose (0.1 clashed with the classes core 0.3 still shipped; 0.3 needs the `Adapter\SubmitterFactory` of core 0.7 and the runners of `indexnowkit/console`). # IndexNow pre-flight check — `indexnowkit/verify` One GET of every URL before it is sent to Yandex, Bing and the other [IndexNow](https://www.indexnow.org) engines. A page that says `noindex`, that `robots.txt` disallows, that names another canonical URL, that redirects, or that the origin does not serve, is not submitted; each becomes a skipped `Result` with a stable `Reason` (`noindex`, `robots_disallowed`, `non_canonical`, `redirected`, `origin_error`) in the log, the listeners, the PSR-14 events and the submission store. The engines see only URLs worth crawling — announcing a `noindex` page or a redirect is at best wasted quota and at worst a reason for an engine to trust the key less. **What it never blocks: 404 and 410.** A URL that answers 404 or 410 is submitted as is — that is the deletion signal IndexNow accepts, and the reason to submit a removed page in the first place. IndexNow is a notification, not indexing: the engine decides whether and when to crawl. **Off by default.** Installing the package changes nothing until `verify.enabled: true`. [](https://packagist.org/packages/indexnowkit/verify) [](https://packagist.org/packages/indexnowkit/verify) [](https://github.com/indexnowkit/php/actions)    [](https://github.com/indexnowkit/php/blob/main/packages/verify/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/verify/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Install ```bash composer require indexnowkit/verify # brings indexnowkit/core; nothing else ``` With a framework adapter (`indexnowkit/symfony-bundle`, `laravel`, `yii2`, `yii3`) that is all: the adapter finds the package, adds the `verify` block to its configuration and, once `verify.enabled` is `true`, decorates its submitter — the queue, Messenger and yii2-queue workers verify too, because they get the same submitter. `check` prints one line about it either way (`verify: installed, disabled (verify.enabled: false)`). ```yaml # Symfony: config/packages/indexnowkit.yaml # Laravel: config/indexnow.php 'verify' => [...] indexnowkit: # Yii2: 'verify' => [...] of the component verify: enabled: true redirect: skip # skip | follow non_canonical: skip # skip | replace origin_error: skip # skip | send ``` ## What one GET decides | The URL answers | Decision | |---|---| | `200` with `X-Robots-Tag: noindex` / `none` (no bot prefix, or an IndexNow engine's bot; `googlebot:` is another engine) or `` in `` | skipped, `Reason::Noindex` | | `200` and `robots.txt` of the host disallows the path for `*` or an engine's bot (checked before the GET) | skipped, `Reason::RobotsDisallowed` | | `200` with `Link: <…>; rel="canonical"` or `` naming another URL (after the core's normalization: tracking parameters, trailing slash) | `non_canonical: skip` → skipped, `Reason::NonCanonical`; `replace` → the canonical is submitted instead, when its host is one of yours | | `3xx` | `redirect: skip` → skipped, `Reason::Redirected`; `follow` → the chain is followed (`max_redirects`, http(s), your hosts only) and the target verified; after a `301`/`308` both URLs are submitted (the page moved), after a `302`/`303`/`307` only the original | | `404`, `410` | **submitted** — a deletion | | `401`, `403`, `5xx`, any other `4xx`, a timeout or connection failure | `origin_error: skip` → skipped, `Reason::OriginError` (retryable: a queue may try again); `send` → submitted with a warning. A `401`/`403` on a protected environment means: set `verify.enabled: false` there | The GET is a real GET (not HEAD: many origins answer HEAD differently), without following redirects, with `verify.timeout` (5 s) and its own `User-Agent` (`verify.user_agent`). Only the `` of the first 256 KiB is read for the signals; a body that is not `text/html` / `application/xhtml+xml` contributes headers only. Comments, attribute order and case do not matter; a relative `` resolves against the document's `` when it declares one, as a crawler resolves it. The pre-flight uses its **own** PSR-18 client, discovered by the core with `max_redirects: 0` — not the application's `http.client`, even when one is configured. It has to see the 3xx answers itself (`verify.redirect`, `verify.max_redirects`, the host check on redirect targets), and PSR-18 cannot tell a client someone else built to stop following redirects. `http.client` keeps carrying the POST submissions; `check` prints the split (`verify.transport`). `robots.txt` is fetched once per host and process and kept in the PSR-16 cache behind `debounce.store` (`verify.robots_cache_ttl`, an hour). A `robots.txt` that cannot be fetched (not `200`, not `404`) blocks nothing: one warning per host, every path is treated as allowed — an unreachable `robots.txt` is no reason to keep quiet. **Time.** One blocking GET per URL, sequentially, `verify.time_budget` (60 s) for the whole batch: past it the rest is sent unverified with one warning, so a queue job never outlives its visibility timeout and gets a second worker. **Batches.** A batch above `verify.max_batch` (100) URLs is sent unverified with one warning: that is the `sitemap` command, where the site itself lists its URLs (`indexnow:sitemap --no-verify` says so explicitly). `submit --dry-run` runs the pre-flight (a GET is harmless) so it shows what would be skipped. ## `dispatch: sync` and `verify.delay` With `dispatch: sync` the pre-flight GETs run inside the web request that changed the page: N URLs cost N round trips to your own origin, and on a single-process development server (`symfony server:start` without workers, `php artisan serve`, `php yii serve`) a request to yourself hangs until the timeout. `check` warns about it (`verify.dispatch`). Use `dispatch: queue` / `messenger`, or keep `verify.enabled: false` outside production. `verify.delay` (seconds, at most 30) waits before the first GET of a batch, so a queue worker that runs right after the commit does not fetch the page from a cache that still holds the old version; it applies only outside a web request and is ignored with `dispatch: sync`. ## `check --sample` ```bash bin/console indexnow:check --sample=https://www.example.com/blog/post-1 --sample-class='App\Entity\Post' php artisan indexnow:check --sample=https://www.example.com/blog/post-1 --sample-class='App\Models\Post:42' php yii indexnow/check --sample=https://www.example.com/blog/post-1,https://www.example.com/ # Yii2: comma-separated, no comma inside a URL ``` Every sample is fetched and reported as one line (`verify sample https://…: HTTP 200, index, canonical: self, robots: allowed`, code `verify.sample`, the host of the URL). `--sample-class` takes a class annotated with `#[IndexNow]` (`` — up to three objects of it, `:` — that one) and resolves their URLs the way the submission does. A `noindex`, a disallow, another canonical, a redirect, a `4xx`/`5xx` or an unreachable origin is a **warning, never an error**: production may be unreachable from CI, and the report must not fail a deploy over it. Without the package `--sample` is an error (`check --sample needs indexnowkit/verify`), `check` without samples prints a `no sample given` line (code `verify.sample`). ## Plain PHP ```php use IndexNowKit\Config; use IndexNowKit\Http\TransportFactory; use IndexNowKit\IndexNowKit; use IndexNowKit\Key\StaticKeyProvider; use IndexNowKit\Url\UrlNormalizerFactory; use IndexNowKit\Verify\VerifyConfig; use IndexNowKit\Verify\VerifyingSubmitter; $core = Config::fromEnv(); $verify = VerifyConfig::fromArray(['enabled' => true, 'redirect' => 'follow']); $indexNow = IndexNowKit::create($core); $submitter = new VerifyingSubmitter( $indexNow->submitter, TransportFactory::lazy($verify->transportConfig($core), null, ['User-Agent' => $verify->userAgent()]), $verify, StaticKeyProvider::fromConfig($core), UrlNormalizerFactory::fromConfig($core), ); $results = $submitter->submit(['https://www.example.com/blog/post-1']); ``` `VerifyingSubmitter` implements `SubmitterInterface`; listeners registered on it see every result (the inner ones and the skipped ones) exactly once; pass the same PSR-14 dispatcher and submission store you gave the inner submitter so the skipped results reach them too. The log lines are fixed strings operators can grep ([docs/operations.md](operations.md)); the options are in [docs/configuration.md](configuration.md). ## For adapter authors `Verify\Adapter\VerifyServices` is what a framework adapter wires for this package, in one place: the predicate (`package()`), the owned options, the validated block, the pre-flight transport, the robots cache, the decorated submitter and command submitter factory, the `check` lines — as static functions over the pieces, with `*For()` twins over the core's `Adapter\Services` for a runtime graph. The Symfony bundle, the Laravel, the Yii2 and the Yii3 adapters build on it; see [adapters.md](../core/adapters.md) of the core. ## Requirements PHP 8.2+, `indexnowkit/core ^0.11`; `indexnowkit/console ^0.4` for the `--sample` options of the `check` command (every adapter has it). ## Notes for AI assistants - Composer package `indexnowkit/verify` (add-on of `indexnowkit/core`). With a framework adapter nothing is wired by hand: `verify.enabled: true` in the adapter's configuration decorates the submitter; the workers of the queue verify too. - Minimal complete snippet (every `use` included): ```php use IndexNowKit\Config; use IndexNowKit\Http\TransportFactory; use IndexNowKit\IndexNowKit; use IndexNowKit\Key\StaticKeyProvider; use IndexNowKit\Url\UrlNormalizerFactory; use IndexNowKit\Verify\VerifyConfig; use IndexNowKit\Verify\VerifyingSubmitter; $core = Config::fromEnv(); $verify = VerifyConfig::fromArray(['enabled' => true]); $indexNow = IndexNowKit::create($core); $submitter = new VerifyingSubmitter($indexNow->submitter, TransportFactory::lazy($verify->transportConfig($core), null, ['User-Agent' => $verify->userAgent()]), $verify, StaticKeyProvider::fromConfig($core), UrlNormalizerFactory::fromConfig($core)); $submitter->submit(['https://www.example.com/page']); // skipped Results carry Reason::Noindex, RobotsDisallowed, NonCanonical, Redirected, OriginError ``` - Verify: `bin/console indexnow:check --sample=https://www.example.com/page`, `php artisan indexnow:check --sample=…`, `php yii indexnow/check --sample=…` — one line per sample, warnings only; `check` alone prints the `verify:` line (installed / disabled / not installed) and, with the package, a "no sample given" line. - Pitfalls: - `verify.enabled` is `false` by default; installing changes nothing. With `dispatch: sync` the GETs run inside the web request (`check` warns): use a queue. - 404 and 410 pass on purpose (deletions); `verify.redirect: follow` submits both URLs after a 301/308, only the original after a 302/303/307. - A canonical or a redirect target on a host without a key in `hosts` / `base_url` is a skip, never a submission: a page cannot make you announce another site. - robots.txt unavailable (500, timeout) allows everything with one warning; 404 is the normal "no robots.txt". - Batches above `verify.max_batch` (100) go unverified with a warning — that is the `sitemap` command; `indexnow:sitemap --no-verify` says it explicitly. - `indexnow:check --sample` needs this package; without it the option is an error with the install line. - `dispatch: auto` exists in Symfony and Yii2, **not** in Laravel (Yii3: `sync` | `none`); locales are `router.locales` (Laravel, Yii2, Yii3), `framework.enabled_locales` (Symfony). ## Versioning SemVer; until 1.0 minor versions may contain breaking changes, listed in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/verify/CHANGELOG.md). What the compatibility promise covers: [docs/bc.md](bc.md). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Backward compatibility `indexnowkit/verify` follows SemVer and the tiers of the core's [docs/bc.md](../core/bc.md). **Before 1.0, minor versions may contain breaking changes**, listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/verify/CHANGELOG.md). | Tier | Members | |---|---| | **Call** — signatures only grow by appended, defaulted parameters; pass anything past the first argument by name | `VerifyingSubmitter` (constructor, `submit()`, `prepare()`, `addListener()`), `VerifyConfig` (constructor, `fromArray()`, `disabled()`, `loadOrDisabled()`, `toArray()`, `transportConfig()`, `userAgent()`, `defaultUserAgent()`), `PageSignals::fromResponse()`, `UrlReference::resolve()`, `RobotsCache` (constructor, `disallows()`, `key()`), `VerifyingSubmitterFactory` (constructor, `create()`, `inner()`), `Check\SampleCheck` | | **Value objects** — `final readonly`, properties only appended with defaults | `PageSignals`, `VerifyConfig` | | **Constants** — referenced, not hard-coded; values may change in a minor | `VerifyConfig::OPTIONS`, `DEFAULT_*`, `MAX_DELAY`, `PageSignals::MAX_BYTES`, `SOURCE_*`, `SampleCheck::CODE` | | **Enum** — closed sets | `RedirectPolicy`, `NonCanonicalPolicy`, `OriginErrorPolicy` | | **Texts that are a contract** | the log-line prefixes of [operations.md](operations.md) (operators grep them); the `Reason` cases of the skipped results (`Noindex`, `RobotsDisallowed`, `NonCanonical`, `Redirected`, `OriginError`) and the check codes `verify.installed`, `verify.dispatch`, `verify.sample` | Not covered: `Verdict` (`@internal`), the `error` sentences of the results and the exception messages, anything under `tests/`. The package pins `indexnowkit/core ^0.11`: the `Reason` cases it uses and `Retry\ForbiddenCounter`'s sibling `Http\TransportFactory::lazy(..., $extraHeaders)` appeared there. # Configuration — the `verify` block The block lives next to the core options in the adapter's configuration (`indexnowkit.verify` in the bundle, `'verify' => [...]` in `config/indexnow.php`, `'verify' => [...]` of the Yii2 component); `VerifyConfig::OPTIONS` lists its keys in dotted form and the adapters add them to the keys they accept, so a typo warns at boot like a core typo. In plain PHP: `VerifyConfig::fromArray($block)`. | Key | Default | Meaning | |---|---|---| | `verify.enabled` | `false` | Decorate the submitter with the pre-flight. Off: nothing is fetched, nothing changes. | | `verify.redirect` | `skip` | `skip`: a 3xx is skipped (`Reason::Redirected`). `follow`: the chain is followed (`max_redirects` hops, http(s), hosts with a key only) and the target verified; after a 301/308 both URLs are submitted, after a 302/303/307 only the original. | | `verify.non_canonical` | `skip` | `skip`: a page whose canonical is another URL is skipped (`Reason::NonCanonical`). `replace`: the canonical is submitted instead, when its host has a key; a foreign canonical is a skip. | | `verify.origin_error` | `skip` | `skip`: 401, 403, 5xx, any other 4xx and a transport failure are skipped (`Reason::OriginError`, retryable). `send`: submitted anyway, with a warning. | | `verify.delay` | `0` | Seconds (0–30) to wait before the first GET of a batch, outside a web request only (a queue worker right after the commit, when a cache may still hold the old page). Ignored with `dispatch: sync`. | | `verify.timeout` | `5` | Seconds one pre-flight GET may take (`http.timeout` of the pre-flight transport). | | `verify.max_redirects` | `3` | Hops followed under `redirect: follow`; more is a skip. | | `verify.max_batch` | `100` | Largest batch verified. A larger one (the `sitemap` command) is sent unverified with one warning; `sitemap --no-verify` says it explicitly. Each URL costs one blocking GET (plus one per redirect hop and one `robots.txt` per origin), sequentially: with `dispatch: sync` that latency lands on the web response. | | `verify.time_budget` | `60` | Seconds the pre-flight of one batch may take in total. A queue job has a visibility timeout (yii2-queue `ttr`, SQS `visibility_timeout`, Messenger `redeliver_timeout`); a pre-flight running past it is handed to a second worker and the batch is sent twice. When the budget runs out the remaining URLs are sent unverified with one warning. `0` = no budget. | | `verify.robots_cache_ttl` | `3600` | Seconds a fetched `robots.txt` is kept in the PSR-16 cache behind `debounce.store` under `robots.` (`robots.__` for anything but https on 443: staging on another origin has its own file); `0` = per process only. | | `verify.user_agent` | `indexnowkit-verify/ (+https://github.com/indexnowkit/php)` | `User-Agent` of the pre-flight GETs. Allow it in your WAF, or it will see 403s. | **The pre-flight always uses its own PSR-18 client**, discovered by the core, with `max_redirects: 0` and a 1 MiB body limit (the signals live in the first 256 KiB). It does **not** use the application's `http.client`, even when one is configured: the pre-flight has to see the 3xx answers itself — `verify.redirect`, `verify.max_redirects` and the host check on redirect targets all read them — and PSR-18 has no way to tell a client someone else built not to follow redirects. `http.client` still carries the POST submissions of the inner submitter, so a corporate proxy or an instrumented client keeps applying where it matters; `check` prints which client does what (`verify.transport`). `verify.timeout` replaces `http.timeout` for that client (`VerifyConfig::transportConfig(Config $core)`). Every other core option (`hosts`, `base_url`, `normalizer.*`, `strict_hosts`) applies as it does to the submission: the pre-flight verifies the normalized URL, and a canonical or a redirect target is "one of your hosts" when the key provider has a key for it. An invalid block is one `critical` log line and the pre-flight is off (`VerifyConfig::loadOrDisabled()`); `check` prints the error (`verify.config`). # Operations ## Log lines Every line of the pre-flight starts with `indexnow verify: `; the prefixes below are fixed (operators grep them, and `docs/bc.md` lists them as a contract), the values in `{}` are PSR-3 context. | Level | Line | When | |---|---|---| | info | `indexnow verify: skipped {url}: noindex ({source})` | `X-Robots-Tag` or `meta robots` says `noindex` / `none` | | info | `indexnow verify: skipped {url}: robots.txt disallows ({rule})` | the host's `robots.txt` disallows the path (`{rule}` = `Disallow: /path`) | | info | `indexnow verify: skipped {url}: canonical is {canonical}` | the page names another canonical, `non_canonical: skip` | | info | `indexnow verify: skipped {url}: canonical is {canonical}: canonical points to a foreign host` | `non_canonical: replace`, but the canonical's host has no key | | info | `indexnow verify: replaced {url} with canonical {canonical}` | `non_canonical: replace`: the canonical is submitted instead | | info | `indexnow verify: skipped {url}: redirected to {location} ({status})` | a 3xx under `redirect: skip`, or without `Location`; with `: redirect leaves the configured hosts`, `: redirect loop`, `: more than {max} redirects` under `follow` | | info | `indexnow verify: {url} gone (HTTP {status}), submitted as a deletion` | 404 / 410 | | warning | `indexnow verify: skipped {url}: origin error ({error})` | 401, 403, 5xx, any other 4xx or a transport failure, `origin_error: skip`; `{error}` is `HTTP 503` or the transport message; 401/403 add `the origin refuses the pre-flight GET; on a protected environment set verify.enabled: false` | | warning | `indexnow verify: {url} origin error ({error}), sent anyway (verify.origin_error: send)` | the same under `origin_error: send` | | warning | `indexnow verify: batch of {count} URLs exceeds verify.max_batch ({max}), sent unverified; use sitemap --no-verify or raise the limit` | a batch above `verify.max_batch` | | warning | `indexnow verify: robots.txt of {host} unavailable ({error})`, `… unavailable (HTTP {status}), URLs are treated as allowed` | `robots.txt` not 200 / 404 or a transport failure — once per host and process | | warning | `indexnow verify: robots cache unavailable, fetching robots.txt per process: {error}` | the PSR-16 cache threw — once per process | | critical | `indexnow verify: invalid verify configuration, pre-flight checks are off until it is fixed: {error} (run "{check}")` | the `verify` block does not validate | The skipped results carry the same facts for code: `Result::$reason` (`noindex`, `robots_disallowed`, `non_canonical`, `redirected`, `origin_error`), `Result::$error` (the sentence after `skipped {url}: `), `engine` `none`, `retryable` true for `origin_error` only. Metrics by `Result::metricLabels()` count them under `status: skipped, reason: …`. ## What to alert on - `origin error` lines in a row for one host: the origin refuses the pre-flight (WAF, auth, maintenance) and nothing reaches the engines until it is fixed or `verify.origin_error: send` / `verify.enabled: false` is set. - `exceeds verify.max_batch` outside a scheduled `sitemap` run: something submits thousands of URLs at once. - `robots.txt of {host} unavailable`: the pre-flight allows everything; the robots.txt itself is the problem. ## Checks `indexnow:check` prints, with the package: `verify: installed, disabled (verify.enabled: false)` (ok, `verify.installed`) or `verify: enabled (redirect: skip, non_canonical: skip, origin_error: skip)` (an invalid block is not a check line: `VerifyConfig::loadOrDisabled()` logs it at `critical` and the pre-flight is off, the bundle rejects it at compile time); `verify.dispatch` (warning) for `verify.enabled` with `dispatch: sync`; one `verify.sample` line per `--sample` / `--sample-class` (ok or warning, never error), or `verify sample: no sample given (check --sample=)` (ok). Without the package: `verify: not installed (composer require indexnowkit/verify) — pre-flight checks off` (ok), and `--sample` is an error with the install line. ## Staging and CI Outside production the core already refuses to submit without an explicit `dry_run: false`; the pre-flight runs under `dry_run` too (a GET is harmless) so `submit --dry-run` and `check --sample` show what would be skipped. A staging behind basic auth answers 401 to the pre-flight: `verify.enabled: false` there, or `verify.origin_error: send`. # IndexNow submission history — `indexnowkit/history` What was announced to Yandex, Bing and the other [IndexNow](https://www.indexnow.org) engines, when, and with what answer: the two shipped implementations of the core's `Submission\SubmissionStoreInterface` — a PSR-16 ring buffer for development and small sites, a database table (PDO) for production — plus the `history` and `status` commands of every adapter and the "recent submissions" list of the Symfony profiler panel. IndexNow is a notification, not indexing: the history tells you what your side did; whether a page was crawled is in Bing Webmaster Tools and Yandex.Webmaster. **Nothing is recorded until a store is named** (`history.store: null` by default). [](https://packagist.org/packages/indexnowkit/history) [](https://packagist.org/packages/indexnowkit/history) [](https://github.com/indexnowkit/php/actions)    [](https://github.com/indexnowkit/php/blob/main/packages/history/LICENSE) [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/history/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) ## Install ```bash composer require indexnowkit/history # brings indexnowkit/core ``` ```yaml # Symfony: config/packages/indexnowkit.yaml # Laravel: config/indexnow.php 'history' => [...] indexnowkit: # Yii2: 'history' => [...] of the component history: store: pdo # null | psr16 | pdo pdo: service: ~ # the adapter's connection (default connection when null), or dsn: 'mysql:…' table: indexnow_submissions retention_days: 90 ``` With `store: pdo` create the table first: [docs/migrations.md](migrations.md) has a Doctrine Migrations, a Laravel and a Yii2 snippet, each one line over `History\Pdo\Schema::sql($driver)`. `check` prints `history: 1 240 records, last 3 min ago` (or the exception with the migration hint when the table is missing). ## What is recorded One record per `Result` — one batch of URLs at one engine — with the URLs **after the core's normalization** (tracking parameters already stripped), the engine, host, status (`ok`, `pending`, `failed`, `skipped`), the `Reason`, the HTTP code, and `Result::$error` cut to 1000 characters. Skipped results (debounced, dry-run, the pre-flight of `indexnowkit/verify`) are records too. **Never** a response body, a header, or the key. Personal data enters the history only when a URL carries it. | Store | Where | For | |---|---|---| | `psr16` | the adapter's cache behind `debounce.store`, a ring buffer of `history.limit` (500) records under `history.`, the slot from an atomic `increment()` when the cache has one | one process, development, small sites: on a cache without `increment()` (a plain PSR-16 array/file cache) workers recording at the same moment lose all but one of their records, an eviction empties the history | | `pdo` | the table `history.pdo.table` (one row per URL, the rows of one Result share a `batch`; one transaction of multi-row INSERTs per Result), indexes on `url`, `at`, `(host, at)` | production; `history --purge` removes what is older than `history.retention_days`. With `dispatch: sync` inside an application transaction the record shares that transaction (a rollback erases the record of a request that has left): give the history its own connection with `history.pdo.dsn` when that matters | Any other `SubmissionStoreInterface` you bind works with the commands too (`recent()` is enough); counting, "last submission" and `--purge` need `History\HistoryStoreInterface` (`count()`, `last()`, `purge()`), which both shipped stores implement. ## Commands ```bash bin/console indexnow:history --host=www.example.com --status=failed --since=2h --limit=20 php artisan indexnow:history --url=https://www.example.com/post-1 # exact URL after normalization php yii indexnow/history --json # {"records":[{at,status,reason,engine,http_status,error,urls[]}]} bin/console indexnow:history --purge # older than retention_days; --purge=30 for 30 days; one line for cron bin/console indexnow:status --json # enabled, dry_run, environment, dispatch, debounce store, 403 counters per host, last successful submission, history size, core version ``` `indexnow:status` is read-only: nothing is fetched, nothing is sent. Its JSON follows [docs/status.schema.json](https://github.com/indexnowkit/php/blob/main/packages/history/docs/status.schema.json). ## Plain PHP ```php use IndexNowKit\Config; use IndexNowKit\History\Pdo\PdoSubmissionStore; use IndexNowKit\IndexNowKit; $store = new PdoSubmissionStore(new PDO('sqlite:var/indexnow.sqlite')); $store->createTable(); // or the migration of docs/migrations.md $indexNow = IndexNowKit::create(Config::fromEnv(), submissionStore: $store); $indexNow->submit(['https://www.example.com/post-1']); foreach ($store->recent(10) as $record) { echo $record->at->format(DATE_ATOM), ' ', $record->result->status->value, ' ', implode(' ', $record->urls), PHP_EOL; } ``` ## For adapter authors `History\Adapter\HistoryServices` is what a framework adapter wires for this package, in one place: the predicate (`package()`), the owned options, the validated block, the stores over a PDO or a PSR-16 cache, the `check` line, the bodies of `history` and `status` (`historyRunner()`, `statusRunner()`; the commands themselves are `History\Console\HistoryCommand` and `StatusCommand`, the classes an adapter on symfony/console registers over those runners), the store description — as static functions over the pieces, with `*For()` twins over the core's `Adapter\Services` for a runtime graph. The Symfony bundle, the Laravel, the Yii2 and the Yii3 adapters build on it; see [adapters.md](../core/adapters.md) of the core. ## Requirements PHP 8.2+, `indexnowkit/core ^0.11`; `ext-pdo` with the driver of your database for `store: pdo` (`sqlite`, `mysql`, `pgsql` schemas shipped); `indexnowkit/console ^0.5` for the commands (every adapter has it). ## Notes for AI assistants - Composer package `indexnowkit/history` (add-on of `indexnowkit/core`). With a framework adapter nothing is wired by hand: `history.store: pdo` (or `psr16`) in the adapter's configuration binds the store; `history.pdo.table` needs the migration of `docs/migrations.md` first. - Minimal complete snippet (every `use` included): ```php use IndexNowKit\Config; use IndexNowKit\History\Pdo\PdoSubmissionStore; use IndexNowKit\IndexNowKit; $store = new PdoSubmissionStore(new PDO('sqlite:var/indexnow.sqlite')); $store->createTable(); $indexNow = IndexNowKit::create(Config::fromEnv(), submissionStore: $store); $indexNow->submit(['https://www.example.com/post-1']); $last = $store->lastFor('https://www.example.com/post-1'); // ?SubmissionRecord: urls, result (status, reason, httpCode, error), at ``` - Verify: `bin/console indexnow:history`, `php artisan indexnow:history`, `php yii indexnow/history` list the records; `indexnow:status` / `indexnow/status` print the wiring and the last successful submission; `indexnow:check` / `indexnow/check` print the `history:` line. - Pitfalls: - `history.store` is `null` by default: installing records nothing. `psr16` loses records when the cache is evicted or two workers write at once; production uses `pdo`. - `--purge` and the record counts need the shipped stores (`History\HistoryStoreInterface`); a custom `SubmissionStoreInterface` still lists with `indexnow:history`. - `--url` matches the URL after normalization (`normalizer.strip_tracking_params` is on by default): pass the URL without `utm_*`. - `history.key_prefix` and the PDO table name must not carry `{}()/\@:` (PSR-6) or anything but `[A-Za-z0-9_]` (SQL identifier). - `dispatch: auto` exists in Symfony and Yii2, **not** in Laravel (Yii3: `sync` | `none`); locales are `router.locales` (Laravel, Yii2, Yii3), `framework.enabled_locales` (Symfony). ## Versioning SemVer; until 1.0 minor versions may contain breaking changes, listed in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/history/CHANGELOG.md). What the compatibility promise covers: [docs/bc.md](bc.md). MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Backward compatibility `indexnowkit/history` follows SemVer and the tiers of the core's [docs/bc.md](../core/bc.md). **Before 1.0, minor versions may contain breaking changes**, listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/history/CHANGELOG.md). | Tier | Members | |---|---| | **Call** — signatures only grow by appended, defaulted parameters | `Psr16SubmissionStore`, `Pdo\PdoSubmissionStore` (constructors, `createTable()`), `Pdo\Schema::sql()`, `assertTable()`, `HistoryConfig` (constructor, `fromArray()`, `disabled()`, `loadOrDisabled()`, `toArray()`), `Check\HistoryCheck`, `Console\HistoryRunner`, `Console\StatusRunner`, `Console\HistoryCommand` and `Console\StatusCommand` (the names `indexnow:history` / `indexnow:status` are a contract, the descriptions are not), `Console\HistoryOptions`, `Console\Definitions::*` | | **Implement** — methods are not added without a major version | `HistoryStoreInterface` | | **Value objects** — `final readonly`, properties only appended with defaults | `HistoryConfig` | | **Constants** — referenced, not hard-coded | `HistoryConfig::OPTIONS`, `STORES`, `DEFAULT_*`, `Schema::DRIVERS`, `HistoryCheck::CODE_*` | | **Documents** — the shape only grows by optional members | `docs/status.schema.json`, the JSON of `history --json`; the table of `Pdo\Schema` (a column is only added, nullable or with a default) | Not covered: `RecordCodec` (`@internal`), the printed texts, anything under `tests/`. The cache keys of the PSR-16 store (`history.index`, `history.`) and the `batch` format are implementation details: a minor may change them, and the ring buffer then starts over. # Configuration — the `history` block The block lives next to the core options in the adapter's configuration (`indexnowkit.history` in the bundle, `'history' => [...]` in `config/indexnow.php`, `'history' => [...]` of the Yii2 component); `HistoryConfig::OPTIONS` lists its keys in dotted form and the adapters add them to the keys they accept. In plain PHP the stores are constructed directly (`new PdoSubmissionStore($pdo)`, `new Psr16SubmissionStore($cache, $prefix, $limit)`). | Key | Default | Meaning | |---|---|---| | `history.store` | `null` | `null`: nothing is recorded (`Submission\NullSubmissionStore`). `psr16`: a ring buffer in the cache behind `debounce.store`. `pdo`: a database table. | | `history.limit` | `500` | Records the `psr16` ring buffer keeps; the oldest is overwritten. | | `history.key_prefix` | `debounce.key_prefix` | Cache key prefix of the `psr16` store (`history.index`, `history.`); no `{}()/\@:`. | | `history.pdo.service` | `null` | Which connection of the adapter gives the PDO: the bundle's `doctrine.dbal._connection` (default connection when null), a Laravel connection name (`DB::connection(name)`), a Yii2 `db` component id. | | `history.pdo.dsn` | `null` | A PDO DSN when the history has its own database (`sqlite:var/indexnow.sqlite`, `mysql:host=…;dbname=…`); ignored when `pdo.service` resolves. Credentials go in the adapter's own PDO options, not here. | | `history.pdo.table` | `indexnow_submissions` | The table; `[A-Za-z_][A-Za-z0-9_]*` — it is the one identifier that reaches SQL from the configuration. Create it with [migrations.md](migrations.md). | | `history.retention_days` | `90` | What `indexnow:history --purge` removes beyond. | An invalid block is one `critical` log line and nothing is recorded (`HistoryConfig::loadOrDisabled()`); `check` prints the error (`history.store`). # Migrations for `history.store: pdo` `History\Pdo\Schema::sql(string $driver, string $table = 'indexnow_submissions'): list` is the source of truth: the `CREATE TABLE` and the three indexes (`url`, `at`, `(host, at)`) for `sqlite`, `mysql` and `pgsql`. Every snippet below runs it; a schema for another driver is yours to write after the sqlite one. Columns: `id` (autoincrement), `batch` (26 chars, groups the URLs of one Result), `url` (2048), `host` (255), `engine` (32), `status` (16), `reason` (32, null), `http_status` (smallint, null), `error` (text, at most 1000 characters), `retryable` (bool), `endpoint` (255), `at` (datetime, UTC). ## Doctrine Migrations (Symfony) ```php use Doctrine\DBAL\Schema\Schema as DbalSchema; use Doctrine\Migrations\AbstractMigration; use IndexNowKit\History\Pdo\Schema; final class Version20260906000000 extends AbstractMigration { public function up(DbalSchema $schema): void { foreach (Schema::sql($this->connection->getDatabasePlatform() instanceof \Doctrine\DBAL\Platforms\PostgreSQLPlatform ? 'pgsql' : ($this->connection->getDatabasePlatform() instanceof \Doctrine\DBAL\Platforms\SqlitePlatform ? 'sqlite' : 'mysql')) as $sql) { $this->addSql($sql); } } public function down(DbalSchema $schema): void { $this->addSql('DROP TABLE indexnow_submissions'); } } ``` ## Laravel ```php use Illuminate\Database\Migrations\Migration; use Illuminate\Support\Facades\DB; use IndexNowKit\History\Pdo\Schema; return new class extends Migration { public function up(): void { foreach (Schema::sql(DB::connection()->getDriverName()) as $sql) { // 'sqlite' | 'mysql' | 'pgsql' DB::statement($sql); } } public function down(): void { DB::statement('DROP TABLE indexnow_submissions'); } }; ``` ## Yii2 ```php use IndexNowKit\History\Pdo\Schema; use yii\db\Migration; final class m260906_000000_indexnow_submissions extends Migration { public function safeUp(): void { foreach (Schema::sql($this->db->getDriverName()) as $sql) { // 'sqlite' | 'mysql' | 'pgsql' $this->execute($sql); } } public function safeDown(): void { $this->dropTable('indexnow_submissions'); } } ``` ## Retention `indexnow:history --purge` runs one `DELETE … WHERE at < ?` for everything older than `history.retention_days` (`--purge=30` for 30 days). On a big table that is one long statement and a long lock: schedule it in a quiet hour, and keep `retention_days` at what you read back (90 days answers "what did we send last quarter"). Partitioning by `at` is the database's job, not the package's. # Operations ## `indexnow:history` | Option | Meaning | |---|---| | `--host=` | records of one host | | `--status=ok\|pending\|failed\|skipped` | records of one status | | `--url=` | the records naming this URL, newest first (exact match after the core's normalization) | | `--since=` | an ISO date (`2026-09-01`, `2026-09-01T10:00:00+00:00`) or a relative interval (`2h`, `3d`, `45m`, `1w`) | | `--limit=` | at most this many records (50) | | `--json` | `{"records":[{"at":"…","status":"ok","reason":null,"engine":"api","http_status":200,"error":null,"urls":["…"]}]}` | | `--purge[=]` | remove the records older than `history.retention_days` (or the given days); prints `purged {n} records older than {date}` — one line for cron; refuses on a store without `purge()` | The table prints `at | status | reason | engine | http | url`, one URL per line, the newest record first. ## `indexnow:status` Read-only (no GET, no POST): enabled / dry_run, the environment, dispatch (and the queue / transport the adapter knows), the debounce store, the 403 counter per configured host with its escalation flag (from `Retry\ForbiddenCounter` of the core, the shared cache when `debounce.store` is a cache), the last successful submission (`last successful submission 3 min ago (2 URLs, api)`), the history size, the core version. `--json` follows [status.schema.json](https://github.com/indexnowkit/php/blob/main/packages/history/docs/status.schema.json). ## `check` `history: installed, no store configured (history.store)` (ok, `history.store`) · `history: pdo store (indexnow_submissions)` / `history: psr16 store (500 records kept)` (ok) · `history: custom store ()` (ok) · `history: the pdo store failed: . Run the migration of docs/migrations.md (table …) or fix history.pdo.*` (error) · `history: 1 240 records, last 3 min ago` / `history: no records yet` (ok, `history.records`). ## Cron ``` 15 3 * * * bin/console indexnow:history --purge >> var/log/indexnow-purge.log ``` ## Log lines `indexnow history: invalid history configuration, nothing is recorded until it is fixed: {error} (run "{check}")` (critical). A store that throws while recording is the core's line `indexnow: submission store failed, {count} result(s) not recorded: {error}` (error) — the submission itself went through. # The GitHub Action `indexnowkit/indexnow-action` A Docker action over `ghcr.io/indexnowkit/indexnow:-action`: `indexnow check --json` (the configuration and the key file of the **live** site — nothing is announced before the engines can verify the key), then `indexnow sitemap --json` (or `submit --json` when `urls` is given), the outputs and a step summary. The source is [`action/`](https://github.com/indexnowkit/php/tree/main/packages/cli/action) of this package, mirrored into the repository `indexnowkit/indexnow-action` (`action.yml` at its root, as the Marketplace wants); `v1` moves within the major, `v1.0.1` is pinned. ```yaml name: IndexNow on: push: { branches: [main] } # after the deploy job, when the site is live jobs: indexnow: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/cache@v4 # the state file: what was announced already with: { path: .indexnow, key: indexnow-${{ github.ref_name }}-${{ github.run_id }}, restore-keys: indexnow-${{ github.ref_name }}- } - uses: indexnowkit/indexnow-action@v1 with: key: ${{ secrets.INDEXNOW_KEY }} base-url: https://www.example.com sitemap: dist/sitemap.xml # a file of the checkout, or a URL; default /sitemap.xml new-only: 'true' ``` | Input | Default | | |---|---|---| | `key` | required | the IndexNow key; `/.txt` must be served by the site | | `base-url` | required | the site; the host of the key file and of every URL | | `key-location` | | the URL of the key file when it is not `/.txt` | | `engines` | `api` | `api` (the shared endpoint) or `yandex,bing,…` | | `sitemap` | `/sitemap.xml` | a URL, or a file in the checkout | | `urls` | | one URL per line; then `submit` runs instead of `sitemap` | | `changed-since` | | only entries with `lastmod` after this (`1 day`, `2026-09-01`) | | `new-only` | `false` | only what the state file has not seen with this `lastmod`; needs the cache above | | `state-path` | `.indexnow` | the directory of the state file, relative to the workspace | | `verify` | `false` | fetch every URL before submitting it (noindex, canonical, redirects, robots.txt are skipped) | | `dry-run` | `false` | report what would be submitted, send nothing | | `fail-on-error` | `true` | `false`: a failed check or submission is a notice, the step stays green | Outputs: `submitted`, `skipped`, `failed` (URL counts), `summary-json` (a file with the check report and the results). The step summary shows the result table and the check lines. What sets it apart from the other IndexNow actions: every engine in one run (or a list), the key file checked before anything is sent, `new-only` over a state file instead of a `lastmod` window (a sitemap without `lastmod` still gets each change announced once), the pre-flight, the history in the state file. Inputs arrive as `INPUT_` with the dashes kept (`INPUT_BASE-URL`), which is why the entrypoint (`docker/indexnow-action` in the image) is PHP. # Backward compatibility `indexnowkit/cli` follows SemVer and the tiers of the core's [docs/bc.md](../core/bc.md). **Before 1.0, minor versions may contain breaking changes**, listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/cli/CHANGELOG.md). The public contract of a command line is what a cron line, a CI step and a runbook depend on: | Tier | Members | |---|---| | **Commands** — names, arguments, options, exit codes; an option is renamed only with a deprecation window | `check`, `config`, `submit`, `sitemap`, `key:generate`, `history`, `status` (the definitions of `indexnowkit/console`, `sitemap` and `history`), `key:file` (`Definitions::keyFile()`); the global options `--env-file`, `--no-env-file`, `--config`, `--state`; the exit codes of `Console\ExitCode` | | **Variables and files** | `INDEXNOW_*` (the core's list plus `INDEXNOW__` for `sitemap`, `verify`, `history`), `INDEXNOW_CONFIG`, `INDEXNOW_STATE`; the `.env` file; the JSON file in the shape of `Config::fromArray()`; the defaults `debounce.store = state`, `history.store = pdo` | | **Packagings** | the PHAR (`indexnow.phar` and `indexnow.phar.sha256` in the GitHub release of `php-cli`), the image `ghcr.io/indexnowkit/indexnow` (tags ``, ``, `latest`, `-action`; the user `indexnow`, `WORKDIR /work`, `ENTRYPOINT indexnow`), the inputs and outputs of `action.yml` of `indexnowkit/indexnow-action` (`v1` moves within the major) | | **Documents** | the JSON of `check --json`, `config --json`, `history --json`, `status --json`, `submit --json`, `sitemap --json` as the packages define them; `summary.json` of the action (`check`, `results`) | | **Call** — signatures only grow by appended, defaulted parameters | `Application` (constructor, `wiring()`), `Wiring` (the accessors), `Commands`, `GlobalOptions`, `Env\Dotenv`, `Config\EnvBlocks`, `Config\EnvConfigSource`, `KeyFileRunner`, `Command\KeyFileCommand`, `Definitions::keyFile()`, `Version::current()` — for an application that embeds the CLI | | **Internal until 1.0** | everything under `State\` (`State`, `SqliteCache`, `SqliteSeenStore`, the tables `indexnow_cache`, `indexnow_sitemap_seen`), `LazyChecker`, `Check\*`, `Command\ConfigurationErrorCommand`: the shape of the state file may change in a minor (a run then starts as if the file were new) | Not covered: the printed texts (they are for humans and get improved; the codes of `check` are), log lines, anything under `tests/`, `docker/indexnow-action` (the entrypoint is reached only through `action.yml`). The package pins `indexnowkit/core ^0.13.1`, `console ^0.5`, `sitemap ^0.9`, `verify ^0.4`, `history ^0.4`: a minor of any of them that changes what the CLI wires ships with a `cli` minor. # Configuration — the variables, the `.env`, the JSON file Environment first, file second. The precedence, highest first: the options of the command (`--state`, `--dry-run`, `--force`), the process environment, the `.env` of the working directory (`--env-file` names another, `--no-env-file` reads none; the process wins over it), the JSON file (`--config`, `INDEXNOW_CONFIG`), the defaults. Nothing is written into the process environment: the file is parsed (`symfony/dotenv`: quotes, `${VAR}`, `export`), then merged. ## The core: `INDEXNOW_` The variables of `Config::fromEnv()` — the table is in the core's [configuration.md](../core/configuration.md#environment-variables): `INDEXNOW_KEY`, `INDEXNOW_PREVIOUS_KEY`, `INDEXNOW_HOSTS` (`host=key,host2=key2`), `INDEXNOW_KEY_LOCATION`, `INDEXNOW_BASE_URL`, `INDEXNOW_ENGINES`, `INDEXNOW_DISPATCH` (`sync` or `none` here), `INDEXNOW_DRY_RUN`, `INDEXNOW_STRICT_HOSTS`, `INDEXNOW_ENV` (else `APP_ENV`), `INDEXNOW_PRODUCTION_ENVIRONMENTS`, `INDEXNOW_BATCH_MAX_URLS`, `INDEXNOW_DEBOUNCE_PER_URL`, `INDEXNOW_DEBOUNCE_STORE` (`state` — the default —, `memory`, `none`), `INDEXNOW_THROTTLE_PER_MINUTE`, `INDEXNOW_HTTP_TIMEOUT`, `INDEXNOW_USER_AGENT`, `INDEXNOW_KEY_FILE_ENABLED`, `INDEXNOW_KEY_FILE_CACHE_MAX_AGE`, `INDEXNOW_MAX_URL_LENGTH`, `INDEXNOW_LOG_URLS`, `INDEXNOW_FORBIDDEN_ESCALATION`, `INDEXNOW_RETRY_*`. `INDEXNOW_HTTP_CLIENT` is refused: there is no container to resolve it in. ## The packages: `INDEXNOW__` The dotted key of the option, upper-cased, `.` as `_` (`Cli\Config\EnvBlocks`). Values as strings; the package coerces (`"3"`, `"true"`). | Block | Variables | |---|---| | `sitemap` ([sitemap](../sitemap/index.md#configuration)) | `INDEXNOW_SITEMAP_ENABLED`, `INDEXNOW_SITEMAP_URL` (the default sitemap; else `/sitemap.xml`), `INDEXNOW_SITEMAP_MAX_DEPTH`, `INDEXNOW_SITEMAP_MAX_SITEMAPS`, `INDEXNOW_SITEMAP_MAX_BYTES`, `INDEXNOW_SITEMAP_ALLOW_FOREIGN_HOSTS`, `INDEXNOW_SITEMAP_SPOOL`, `INDEXNOW_SITEMAP_SPOOL_DIR`, `INDEXNOW_SITEMAP_FETCH_RETRIES` | | `verify` ([verify](../verify/configuration.md)) | `INDEXNOW_VERIFY_ENABLED` (the pre-flight GET before every submission; off by default), `INDEXNOW_VERIFY_REDIRECT`, `INDEXNOW_VERIFY_NON_CANONICAL`, `INDEXNOW_VERIFY_ORIGIN_ERROR`, `INDEXNOW_VERIFY_DELAY`, `INDEXNOW_VERIFY_TIMEOUT`, `INDEXNOW_VERIFY_MAX_REDIRECTS`, `INDEXNOW_VERIFY_MAX_BATCH`, `INDEXNOW_VERIFY_TIME_BUDGET`, `INDEXNOW_VERIFY_ROBOTS_CACHE_TTL`, `INDEXNOW_VERIFY_USER_AGENT` | | `history` ([history](../history/configuration.md)) | `INDEXNOW_HISTORY_STORE` (`pdo` — the default, over the state file —, `psr16` — over the cache of the state file —, `none` / empty: off), `INDEXNOW_HISTORY_LIMIT`, `INDEXNOW_HISTORY_KEY_PREFIX`, `INDEXNOW_HISTORY_PDO_DSN` (a database of your own; a `sqlite:` file is created like the state file, any other needs the migration of the history package), `INDEXNOW_HISTORY_PDO_TABLE`, `INDEXNOW_HISTORY_RETENTION_DAYS`; `INDEXNOW_HISTORY_PDO_SERVICE` is refused (a framework connection) | The CLI's own: `INDEXNOW_CONFIG` (the JSON file), `INDEXNOW_STATE` (the state file, or `memory`). ## The JSON file The nested shape of `Config::fromArray()` plus the `sitemap`, `verify` and `history` blocks — what a framework adapter's configuration looks like, as JSON (no `include`: it is PHAR-safe): ```json { "key": "abcdef1234567890abcdef1234567890", "base_url": "https://www.example.com", "hosts": {"shop.example.com": {"key": "…", "key_location": "https://shop.example.com/keys/….txt"}}, "strict_hosts": true, "sitemap": {"url": "https://www.example.com/sitemap-index.xml", "max_depth": 2}, "verify": {"enabled": true, "redirect": "follow"}, "history": {"retention_days": 30} } ``` The variables win over it key by key (`array_replace_recursive`); `INDEXNOW_HOSTS` replaces the hosts map whole. A key neither the core nor the packages know is a `config.unknown` warning of `check`. `config --json` prints the effective values (keys masked) and the `cli` block: `config_file`, `env_file`, `state`. # The Docker image `ghcr.io/indexnowkit/indexnow` `php:8.3-cli-alpine` plus `indexnow.phar` as `/usr/local/bin/indexnow`; `pdo_sqlite`, `xmlreader`, `zlib`, `curl` and `openssl` come with the base image, `intl` does not (international host names go through the core's pure PHP Punycode). Tags: `` (`0.1.0`), `` (`0.1`), `latest`; `-action` is the same image without a `USER`, for the GitHub Action only. Multi-platform: `linux/amd64`, `linux/arm64`. The image runs as the user `indexnow` (uid 1000) with `WORKDIR /work` and `ENTRYPOINT ["indexnow"]`: mount the directory that holds `.indexnow/` (the state file) and the sitemap files on `/work`, give the variables: ```bash docker run --rm -v "$PWD:/work" -e INDEXNOW_KEY -e INDEXNOW_BASE_URL ghcr.io/indexnowkit/indexnow check --live docker run --rm -v "$PWD:/work" --env-file .env ghcr.io/indexnowkit/indexnow sitemap --new-only --json docker run --rm -e INDEXNOW_KEY=… -e INDEXNOW_BASE_URL=… -e INDEXNOW_STATE=memory ghcr.io/indexnowkit/indexnow submit https://www.example.com/a ``` A mounted directory owned by another uid: `--user "$(id -u):$(id -g)"`, or `INDEXNOW_STATE=/tmp/state.sqlite` (`memory` when nothing may persist). `.env` inside `/work` is read like on a host; `--env-file` of Docker hands the variables in without a file in the container. GitLab CI, the same image: ```yaml indexnow: image: { name: ghcr.io/indexnowkit/indexnow:0.1, entrypoint: [""] } cache: { key: indexnow, paths: [.indexnow/] } script: - indexnow check --json - indexnow sitemap --new-only --json public/sitemap.xml variables: { INDEXNOW_BASE_URL: https://www.example.com } # INDEXNOW_KEY: a masked CI variable ``` How it is built: the PHAR first (`bin/phar` in the monorepo, the `phar` job of the release workflow of `php-cli`), then `docker build --target cli|action packages/cli` copies it in — the Dockerfile compiles nothing, so the image holds exactly the PHAR of the release. The first push of a version creates the GHCR package; the organization made it public once. # The state file `indexnow` keeps what makes scheduled runs idempotent in one sqlite file: `.indexnow/state.sqlite` in the working directory, `--state ` or `INDEXNOW_STATE` elsewhere, `memory` for a run that keeps nothing (a read-only container, a one-off `--dry-run`). Nothing opens it before a store is asked for: `list`, `--version`, `help` and a configuration that does not build never touch it. It is created on first use, its directory with mode `0700`, in WAL mode (a cron run and a manual one do not lock each other out), the tables created if missing. | Table | What | Who reads it | |---|---|---| | `indexnow_cache` | a PSR-16 cache: the debounce window (`debounce.store = state`, keys `indexnowkit_`, TTL `debounce.per_url`), the consecutive-403 counters per host, the robots.txt cache of the pre-flight | `submit`, `sitemap`, `check` (the probe), `status` | | `indexnow_submissions` | the history: one row per URL of a result, the rows of one result share a `batch` (the schema of the history package; `history.pdo.table` renames it) | `history`, `status`, `check` | | `indexnow_sitemap_seen` | the fingerprint of every sitemap URL announced so far (URL plus ``, or none) and when | `sitemap --new-only` | What it means: a URL announced within the last `debounce.per_url` seconds (600) is not sent again by the next run — `--force` overrides; a sitemap URL announced once is not sent again by `--new-only` until its `lastmod` changes; `history --purge` removes records older than `history.retention_days`. No keys are stored. No backup needed: losing the file costs one full re-announcement (the engines de-duplicate). Do not put it under the document root. `check` prints `state: (writable | read-only | not created yet)` (`cli.state`, a warning when it cannot be written) and probes the debounce store through it (`debounce.store`). In CI, cache the directory between runs (`actions/cache` with `path: .indexnow`); in Docker, mount the working directory on `/work`. The classes behind it (`IndexNowKit\Cli\State\*`) are internal until 1.0: the shape of the file may change in a minor, and a run then starts as if the file were new. # `indexnowkit/doctrine` — commit-safe IndexNow for Doctrine ORM Listens to `onFlush` / `postFlush`, resolves the URLs of entities that declare `#[IndexNow]` rules, and hands them over **only after the outermost transaction really committed**, using a DBAL driver middleware. Rolled-back flushes submit nothing. Deletions are resolved before the row disappears. Doctrine ORM 2.19+ and 3.x, DBAL 3.x and 4.x, PHP 8.2+. [Русская версия](https://github.com/indexnowkit/php/blob/main/packages/doctrine/README.ru.md) · Issues and pull requests: [github.com/indexnowkit/php](https://github.com/indexnowkit/php/issues) (the `php-*` repositories are read-only splits) [](https://packagist.org/packages/indexnowkit/doctrine) [](https://packagist.org/packages/indexnowkit/doctrine) [](https://github.com/indexnowkit/php/actions) [](https://github.com/indexnowkit/spec)   [](https://github.com/indexnowkit/php/blob/main/packages/doctrine/LICENSE) **Symfony users: take [`indexnowkit/symfony-bundle`](../symfony-bundle/index.md)** — it wires all of this, adds the router bridge, the commands and the profiler panel. This package is for Doctrine without Symfony. ## Why this over X Most IndexNow packages are a thin HTTP client: you collect the URLs, you call it, you read the answer. This family does the part that goes wrong in practice: - **Declared on the model** (`#[IndexNow]`) and submitted from the ORM hooks — no controller code to forget. - **After the commit**, not on flush: a rolled-back transaction announces nothing. - **Debounce** (10 minutes per URL, shared through your cache), **batches** of up to 10 000 URLs, one key per host from env. - **Answers handled**: 202 (key pending), 422, 429 with `Retry-After` back-off and a retry through your queue, 403 escalation. - **`check` and `explain`** in the Symfony bundle say what is wrong before the first submission and why a URL was or was not sent. - **One core** under the Symfony, Laravel, Yii2, Yii3 and Doctrine adapters with a shared conformance suite: the same behaviour everywhere, documented once. ## Install ```bash composer require indexnowkit/doctrine ``` ## Standalone wiring ```php use Doctrine\DBAL\DriverManager; use Doctrine\ORM\{EntityManager, ORMSetup}; use IndexNowKit\{Config, IndexNowKit}; use IndexNowKit\Doctrine\IndexNowDoctrine; use IndexNowKit\Url\{ArrayResolverLocator, AttributeUrlResolver}; $indexNow = IndexNowKit::create(Config::fromEnv(), logger: $logger); $resolver = new AttributeUrlResolver( $indexNow->attributes, $indexNow->extractor, // the DSL alone: Doctrine entities are plain objects router: null, // no framework router: see "Routes" below locator: new ArrayResolverLocator([ 'post_url' => fn (Post $post): string => '/posts/' . $post->slug, // #[IndexNow(resolver: 'post_url')] ]), logger: $logger, ); $wiring = new IndexNowDoctrine($indexNow, $resolver, $logger, autoFlush: true); $ormConfiguration = ORMSetup::createAttributeMetadataConfiguration([__DIR__ . '/src/Entity'], isDevMode: false); $wiring->registerMiddleware($ormConfiguration); // BEFORE DriverManager::getConnection() $connection = DriverManager::getConnection(['driver' => 'pdo_sqlite', 'path' => __DIR__ . '/var/app.db'], $ormConfiguration); $entityManager = new EntityManager($connection, $ormConfiguration); $wiring->registerListener($entityManager); ``` `registerMiddleware()` must run before the connection is created, because DBAL middlewares wrap the driver at connect time. In a typical bootstrap that means: build the ORM `Configuration`, call `registerMiddleware()`, then create the `EntityManager`, then call `registerListener()`. `$autoFlush: true` submits as soon as the URLs are handed over, which is what a script or a CLI process wants. Pass `false` and call `$indexNow->flush()` yourself at the end of the unit of work when you control the request cycle. `IndexNowDoctrine` exposes the three pieces it builds — `$wiring->staging`, `$wiring->listener`, `$wiring->middleware` — so a container can register them individually instead. A container that builds its graph lazily (`Adapter\Services` of the core) can hand the listener a closure and a sink instead of the facade, so registering the listener — and a request that writes nothing — builds no submitter, client or transport: ```php $listener = new IndexNowListener( fn(): ObjectChangeHandler => $services->changes(), // built on the first flush that has something to classify null, $staging, $logger, sink: fn(array $urls) => $services->kit()->collect($urls), ); ``` ## Declaring pages The `#[IndexNow]` attribute comes from the core and is repeatable: one rule per family of public URLs. ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; #[ORM\Entity] #[IndexNowDefaults(when: 'isPublished', fields: ['slug', 'title', 'body', 'published'])] #[IndexNow(resolver: 'post_url')] #[IndexNow(via: 'category')] // a changed post also refreshes its category page #[IndexNow(urls: ['/'])] // and the homepage class Post { #[ORM\Column] private bool $published = false; public function isPublished(): bool { return $this->published; } } ``` Full model — sources, typed parameters, `when` / `whenFields` / `fields` / `events` / `locales` / `host`, inheritance and the semantics table — is in the core's [attribute reference](../core/attribute-reference.md). ### Routes `#[IndexNow(route: ...)]` needs a `RouteUrlResolverInterface` bridge to a framework router. Standalone Doctrine has none, so a rule using `route:` fails at resolution time with *"no router bridge is configured"* (logged, never thrown into your flush). Use `url:`, `urls:` or `resolver:` instead, or implement the two-method `RouteUrlResolverInterface` for your own router and pass it as the `router:` argument above. ## What the listener does In `onFlush`, every scheduled insertion, update, deletion and changed collection is classified **per rule** through the core's `ObjectChangeHandler`: - **Insertions** produce `created` events. Their URLs are resolved in `postFlush`, once identifiers are assigned. - **Updates** are classified per rule from `UnitOfWork::getEntityChangeSet()`. A rule whose `when` turned false becomes a **deletion** and is resolved immediately in `onFlush`, while the old state is still live; a rule whose `when` turned true becomes a creation; otherwise it is an update, filtered by the rule's `fields`. One entity can therefore produce an update for one page and a deletion for another in the same flush. - **Changed to-many associations** are not part of the owner's change set, so a scheduled collection update or deletion re-classifies its owner with the association's field name as the changed field. Changing `post.tags` resubmits the post's pages. - **Deletions** are resolved in `onFlush`, before the row disappears. A rule that does not apply — a draft that was never public — submits nothing. In `postFlush` the deferred rules are resolved, every URL is logged at `debug` with the rule that produced it (`indexnow: App\Entity\Post#post_amp (updated) -> https://example.com/amp/hello`), and the batch is handed off. Nothing here throws into your application. An invalid attribute, an unreadable `when` accessor or a failing resolver is logged on the `indexnow` channel and yields no URLs. ## Renamed pages When a field a route parameter reads changes — the slug, the category the path goes through — the old URL now answers 404. On an update the listener resolves the rule against the **previous** values of the change set and announces those URLs as deleted, next to the new URLs as updated, in the same flush (`ObjectChangeHandler::renamed()`, scenario A21). Route rules only; the old page must have been public (`when` true before the change); a field the URL depends on that cannot be written back (`readonly`, uninitialized) skips the old URL with a `debug` line. Nothing in this path throws into `flush()`. ## Commit safety `postFlush` runs before the outer `COMMIT` whenever `flush()` is wrapped in `wrapInTransaction()` or a manual transaction, and Doctrine has no after-commit event. So: - if the connection has an open transaction, the URLs are staged against its **native** connection object; - the DBAL driver middleware sees the real `commit()` and `rollBack()` — nesting level 0, identically in DBAL 3 and 4 (`Middleware\IndexNowConnection` / `IndexNowConnectionV3`, picked by `IndexNowDriver` at connect time) — and either releases the staged URLs or discards them; - a `commit()` that itself throws discards them too, so a pooled connection never delivers them later; - a nested transaction rolled back to its savepoint (`ROLLBACK TO SAVEPOINT`, what DBAL issues for an inner `rollBack()`) drops the URLs staged inside it; the outer `COMMIT` delivers the rest; - outside a transaction the URLs are handed over immediately. If the driver exposes no native connection object, the listener logs a warning and submits inside the open transaction rather than losing the URLs. ## Limitations - DQL and QueryBuilder bulk `UPDATE` / `DELETE`, and `Connection::executeStatement()`, bypass the unit of work and are not detected. Submit those URLs with `$indexNow->submit()`. - `route:` needs a router bridge (see above). - Entities inserted through `INSERT ... SELECT` never reach `postFlush`. - Attributes are not read from interfaces or traits: PHP does not inherit class attributes through them, and Doctrine mapping behaves the same way. ## Compatibility with other listeners Register the listener **after** anything that computes values the URLs depend on. With Gedmo Sluggable the slug is written in `onFlush`, so the IndexNow listener must run later; the Symfony bundle uses priority `-100` for exactly this reason. ## Compatibility Public API of this package: the classes named in the changelog and the README, their constructor parameter names (pass optional arguments by name), and the DBAL middleware classes. The core's rules apply, including the "may grow" interfaces: [bc.md](../core/bc.md); what this package itself keeps stable: [docs/bc.md](bc.md). Before 1.0 a minor version may break; every break is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/doctrine/CHANGELOG.md) with the migration. ## Notes for AI assistants - Composer package `indexnowkit/doctrine` (Doctrine ORM 2.19+/3, DBAL 3/4, on `indexnowkit/core`). In a Symfony application use `indexnowkit/symfony-bundle`, which wires this package. - Minimal complete snippet (every `use` included): ```php use Doctrine\ORM\Mapping as ORM; use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults}; #[ORM\Entity] #[IndexNowDefaults(when: 'isPublished')] #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] // a Symfony route, or resolver: 'post_url' without a router class Post { /* ORM columns, isPublished() */ } ``` - Verify: with the bundle, `bin/console indexnow:check` and `bin/console indexnow:explain 'App\\Entity\\Post' 1`; standalone, `$indexNow->explain($post)` returns the resolved URLs with the rule that produced each. - Pitfalls: - `dispatch: auto` exists in Symfony (`auto` | `messenger` | `sync` | `none`) and Yii2 (`auto` | `queue` | `sync` | `none`), **not** in Laravel (`queue` | `sync` | `none`); Yii3 has `sync` | `none` only. - Locales: `router.locales` in Laravel, Yii2 and Yii3, `framework.enabled_locales` in Symfony; `locales: 'all'` on a rule uses that list. - `url:` names an accessor (method or property) that returns the URL; `urls:` is a list of literal URLs. Never put a literal in `url:`. - A string in `when:` is an accessor read as truthy (`published`, `isPublished`). A status string needs `Equals`: `when: new Equals('status', 'published')` (`IndexNowKit\Attribute\Param\Equals`). - Manual submission is `submitEntity()` in Symfony, `submitModel()` in Laravel, `submitRecord()` in Yii2 and Yii3; the commands are `indexnow:submit-entity`, `indexnow:submit-model`, `indexnow/submit-record` (Yii2), `indexnow:submit-record` (Yii3). Bulk queries (`update()`, `DB::table()`, `updateAll()`) fire no hooks: submit afterwards with those. - Laravel has two classes called `IndexNowKit`: the facade `IndexNowKit\Laravel\Facades\IndexNowKit` and the core service `IndexNowKit\IndexNowKit` (inject by type). Yii2 exposes the core through `Yii::$app->indexnow->kit()`; Yii3 defines `IndexNowKit\IndexNowKit` in the container. - Outside production a configured key with `dry_run` unset makes `check` fail (a staging copy would submit real URLs): set `dry_run: true` there, or `dry_run: false` explicitly when it submits on purpose. - Unknown configuration keys are warned about at boot (typos such as debounce.per_urls); the key list is `Config::OPTIONS` plus the adapter's own keys. ## Documentation | | | |---|---| | Attribute reference | [core/docs/attribute-reference.md](../core/attribute-reference.md) | | Configuration | [core/docs/configuration.md](../core/configuration.md) | | Operations and logging | [core/docs/operations.md](../core/operations.md) | | Testing | [core/docs/testing.md](../core/testing.md) | | Writing your own adapter | [core/docs/adapters.md](../core/adapters.md) | | Design rationale | [docs/spec](https://github.com/indexnowkit/spec) | Changelog: [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/doctrine/CHANGELOG.md). Versioning: SemVer; before 1.0 minor versions may break. MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org. # Backward compatibility `indexnowkit/doctrine` follows SemVer. **Before 1.0, minor versions may contain breaking changes**; every one is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/doctrine/CHANGELOG.md) with the migration. After 1.0 the rules below become the promise. The core's tiers ("call", "implement", "may grow") apply to every core class you touch through this package: [core bc.md](../core/bc.md). ## What the package keeps stable | Surface | Promise | |---|---| | **`IndexNowDoctrine`** (`registerMiddleware()`, `registerListener()`, the constructor's named parameters) | Method and parameter names stay; new parameters are appended with defaults. | | **`IndexNowListener`** (the `onFlush` / `postFlush` subscriber) and **`Middleware\IndexNowMiddleware`** | Their constructors' named parameters and the Doctrine events they subscribe to stay. They are `final`: put your own logic on top of the core `ObjectChangeHandler`, not on a subclass. | | **Behaviour** described in the README: URLs resolved in `onFlush` while the old state is live, handed over only after the outermost transaction committed, nothing on rollback, deletions resolved before the row disappears, savepoints honoured | A change to any of these is a breaking change and appears in the changelog with the conformance ids (A01–A21) it affects. | | **DBAL 3 and 4, ORM 2.19+ and 3** | Dropping a major of either is a breaking change of this package. | Not a contract: `Middleware\IndexNowDriver`, `Middleware\IndexNowConnection`, `Middleware\IndexNowConnectionV3`, `Middleware\SavepointStatement` (the DBAL wrapping, which follows DBAL's own API), and log message texts (their `context` keys are). ## Pinning `composer require indexnowkit/doctrine:^0.5` gets every 0.5.x patch. Read the changelog before a minor. # Backward compatibility `indexnowkit/symfony-bundle` follows SemVer. **Before 1.0, minor versions may contain breaking changes**; every one is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/symfony-bundle/CHANGELOG.md) with the migration. After 1.0 the rules below become the promise. The core's tiers ("call", "implement", "may grow") apply to every core class you touch through the bundle: [core bc.md](../core/bc.md). ## What the bundle keeps stable | Surface | Promise | |---|---| | **Configuration keys** of `indexnowkit:` ([configuration.md](configuration.md)) | Keys and their meaning stay; new keys are only added with a default. A rename ships the old key as deprecated for one minor (`serve_key_file` → `key_file.enabled` is the example) and is listed in the changelog. | | **Environment variables** the recipe reads (`INDEXNOW_KEY`, `INDEXNOW_BASE_URL`, …) | Names stay; new ones are only added. | | **Command names and options** (`indexnow:check`, `indexnow:key:generate`, `indexnow:submit`, `indexnow:submit-entity`, `indexnow:explain`, `indexnow:sitemap`) | Names, arguments and options come from the core `Console\Definitions`; new options are only added. Output is not a contract except the exit codes and the `--json` shape of the core formatter. | | **Service ids and aliases** listed in [extending.md](extending.md) (`indexnowkit`, `indexnowkit.config`, `indexnowkit.submitter`, `indexnowkit.transport`, the `Interface` aliases, the `indexnowkit.check` tag) | Ids and the interface each one implements stay; decorating them stays possible. Ids not in that table (`indexnowkit.check.debounce_store.probe`, `indexnowkit.console.*` internals) are implementation details. | | **Container parameters** `indexnowkit.dispatch`, `indexnowkit.messenger_routed`, `indexnowkit.doctrine_hooked` | Names and types stay. | | **Public classes** `IndexNowKitBundle`, `Command\EntityLoader` (since 0.15.0 over `Console\AbstractSubjectLoader` of `indexnowkit/console`; the constructor is unchanged), `DependencyInjection\ConsoleConfigSource`, `Controller\KeyFileController`, `Routing\KeyFileRouteLoader`, `EventListener\FlushListener` (constructor since 0.15.0: `(CollectorInterface, Closure(): IndexNowKit)` — a `service_closure('indexnowkit')`, no PSR-11 locator), `Check\WiringCheck`, `Check\CacheProbe`, `Check\MappedClasses`, `Messenger\SubmitUrlsMessage`, `DataCollector\IndexNowDataCollector` | Constructor parameters are passed by name; new ones are appended with defaults. They are `final`: extend by decoration, not inheritance. `Check\LocalesCheck` and `Check\EntitySampler` of the bundle are gone in 0.15.0: the line is the core's `Check\LocalesCheck` (same code `router.locales`, same service id `indexnowkit.check.locales`), the sampler is `Console\SubjectSampler` of `indexnowkit/console` (same service id `indexnowkit.check.entity_sampler.callable`). The command classes are not the bundle's since 0.15.0: `indexnow:*` are `IndexNowKit\Console\Command\*` of `indexnowkit/console`, `Sitemap\Console\SitemapCommand` and `History\Console\HistoryCommand` / `StatusCommand` of their packages, registered under their class names as service ids ([extending.md](extending.md) "Replacing a command"). | | **Route** `indexnowkit_key_file` (or `key_file.route_name`) | The route name and the `{key}.txt` shape stay. | Not a contract: the Twig templates of the profiler panel, log message texts (their `context` keys are), the exact wording `indexnow:check` prints (its exit code and the ok/warning/error levels are), and the compiled container shape the bundle's own tests pin. ## Pinning `composer require indexnowkit/symfony-bundle:^0.8` gets every 0.8.x patch. Read the changelog before a minor. # Configuration reference Every node of the `indexnowkit` extension, with its default and what it does. Anything that is not an environment placeholder is validated when the container is compiled, so a typo fails at `cache:clear`, not at the first submission. ```yaml # config/packages/indexnowkit.yaml indexnowkit: # Master switch. false = collect nothing, submit nothing, register no Doctrine listener. # Prefer dry_run if you still want the logs and the profiler panel. enabled: true # Default IndexNow key, 8-128 characters of [A-Za-z0-9-]. # Generate: bin/console indexnow:key:generate --write-env key: null # '%env(INDEXNOW_KEY)%' # Absolute site URL. Needed outside HTTP requests (console commands, Messenger workers) and to # resolve relative URLs. Required when dispatch is "messenger", and whenever `hosts` is used. base_url: null # '%env(INDEXNOW_BASE_URL)%' # Absolute URL of the key file when it is NOT served at https:///.txt. # Must be on the host of base_url. key_location: null # The key before a rotation: /.txt keeps being served while engines re-verify, # nothing is ever submitted under it. Remove it once the rotation settled. previous_key: null # Multi-domain: one entry per additional host. Hosts not listed here use the default key, # unless strict_hosts is on. An array node cannot come from a single env var, so use # per-entry %env(...)% placeholders. hosts: # example.de: # key: '%env(INDEXNOW_KEY_DE)%' # key_location: 'https://example.de/keys/indexnow.txt' # optional # base_url: 'https://example.de' # console/worker URL generation # engines: [yandex] # this host only; default: `engines` # previous_key: '%env(INDEXNOW_KEY_DE_OLD)%' # rotation window # Refuse URLs of hosts that are neither base_url nor listed in `hosts`, instead of announcing # them under the default key. Recommended for multi-domain setups, and for any production # app that is also reachable under a staging or internal hostname (indexnow:check warns). strict_hosts: false # Environment names treated as production: no dry-run safety net there, and indexnow:check # flags dry_run. Replace (not extend) the list when yours is not prod/production. production_environments: [prod, production] # URLs longer than this many bytes are skipped as invalid_url. The protocol sets no limit. max_url_length: 2048 # api = api.indexnow.org, the shared endpoint reaching Yandex, Bing, Naver, Seznam, Yep, Internet Archive and Amazon. # Name a single engine only to target it, or give a full https endpoint URL (or an alias below). engines: [api] # Short names for custom endpoints, usable in engines and hosts..engines. engine_aliases: {} # { corp: 'https://index.corp.example/indexnow' } # locale => host. A rule with `locales` and no `host` generates each locale on its host # (en on www.example.com, de on example.de). List those hosts in `hosts` with their keys. locale_hosts: {} # { en: www.example.com, de: example.de } # auto = messenger when a Messenger transport is configured, otherwise sync. # sync = submit after the response was sent (kernel.terminate). none = collect, never send. dispatch: auto # auto | sync | messenger | none messenger: # Bus service id. It needs the dispatch_after_current_bus middleware; default buses have it. bus: messenger.default_bus # Transport name from framework.messenger.transports. When set, the bundle adds the routing # for SubmitUrlsMessage itself, so messenger.yaml needs no edit. transport: null # e.g. 'async' # Milliseconds: a DelayStamp on every SubmitUrlsMessage. Needs a transport that supports # delays (doctrine, amqp, redis, sqs); in-memory and sync transports ignore it. delay: 0 # Service ids of extra stamps added to every message (a FIFO group id, a priority). stamps: [] batch: # URLs per request. The protocol maximum is 10000; larger sets are split. max_urls: 10000 debounce: # Seconds during which the same URL is not re-submitted. Yandex accepts the same URL at most # once per 10 minutes. 0 disables debouncing. per_url: 600 # 'memory' (per process: CLI, tests), 'none', or a PSR-6 cache pool service id shared by all # processes. store: cache.app # Cache key prefix. Give each application sharing one pool its own. key_prefix: indexnowkit_ throttle: # Outgoing requests per minute, per process: N workers get N buckets. For a site-wide limit replace # indexnowkit.throttle (ThrottleInterface) with a shared limiter, e.g. on symfony/rate-limiter + Redis. max_requests_per_minute: 60 http: # Seconds. Applied to the client the bundle creates itself. timeout: 10.0 # Override the indexnowkit-php/ User-Agent. user_agent: null # Service id of a PSR-18 client OR of a symfony/http-client (including # framework.http_client.scoped_clients, wrapped automatically). Default: auto-discovery. # Use a scoped client for proxy, retries or extra headers. client: null key_file: # Serve the key file so engines can verify the key. enabled: true # Route path. {key} is required and constrained to the key format. path: '/{key}.txt' # Restrict the route to this host pattern (a Symfony route host requirement). Default: any host. host: null # Name of the route; rename it when it clashes with an existing one. route_name: indexnowkit_key_file # Cache-Control max-age in seconds. Keep it short so a key rotation propagates quickly. cache_max_age: 300 # Deprecated alias of key_file.enabled. serve_key_file: null # Log the request instead of sending it. Switched on automatically outside prod when no key is set. dry_run: false logging: # Monolog channel every bundle service logs to. channel: indexnow # URLs listed in one log line (the count is always logged). 0 = no URLs in logs (PII policies). max_urls: 20 # Consecutive 403s for one host before the log level escalates to critical (the line to page on). forbidden_escalation: 5 # Bytes of an engine response body kept in a failure log line. max_body: 300 # Override the level of an outcome. Events and their defaults: ok (debug), pending (info), # invalid_request (error), unprocessable (warning), rate_limited (warning), server_error (warning), # unexpected (error), transport (warning), no_key (warning), dry_run (info), disabled (info), # debounced (debug), invalid_url (warning). levels: {} # e.g. { debounced: info, rate_limited: error } resolver: # How many "via:" hops a rule may follow (Comment -> Post -> Author). max_via_depth: 3 # How many related objects one "via:" hop may yield; the rest is dropped with a warning. max_via_fanout: 100 collector: # Flush as soon as this many URLs were collected in one request or command (0 = only at the # end). Bounds memory in long imports; the flush goes through the normal dispatcher. max_urls: 0 # Warn at shutdown about collected URLs that were never flushed. detect_leaks: true profiler: # Register the profiler panel when WebProfilerBundle is present. enabled: true flush: # Listener priority of the kernel.terminate flush. Default -1000: before the profiler (-1024), # so results land in the panel. Raise or lower it to order against your own terminate listeners. priority: -1000 # Same for console.terminate and the Messenger WorkerMessageHandledEvent flush. console_priority: -1024 # Needs indexnowkit/sitemap (composer require indexnowkit/sitemap). Without the package the block is # accepted as is and ignored; indexnow:check says so. sitemap: # Register indexnow:sitemap and the sitemap reader. false = the command does not exist; nothing # else in the bundle reads sitemaps. enabled: true # Sitemap read by indexnow:sitemap when no argument is given. Default: /sitemap.xml. url: null # Levels of followed below the root (0 = the root only). max_depth: 3 # Documents fetched per run, root included. max_sitemaps: 1000 # Size cap of one uncompressed sitemap document (protocol maximum 50 MiB). Documents are # spooled to temp files, never held in memory, so this bounds disk and time rather than RAM. max_bytes: 52428800 # Follow nested sitemaps on other origins (CDN-hosted parts). Off by default: a sitemap then # decides which hosts this server fetches from. --allow-foreign-hosts enables it for one run. allow_foreign_hosts: false # Where a document is kept while parsing. auto = a temp file, or memory when the temp dir is not # writable (read-only container; logged once). disk = a temp file or fail. memory = never touch # the disk (at most max_bytes per document, so memory stays bounded either way). spool: auto # Directory for the temp files. Default: sys_get_temp_dir(), i.e. TMPDIR / sys_temp_dir. On a # readOnlyRootFilesystem point it at an emptyDir or tmpfs mount. spool_dir: null # Extra attempts (1 s, 2 s, 4 s apart) when fetching a document fails on the network or with a # 5xx. 4xx and broken documents are never retried. fetch_retries: 2 # Needs indexnowkit/history (composer require indexnowkit/history). Without the package the block is # accepted as is and ignored; indexnow:check says so. history: # Where every submission Result is recorded: null = nothing is kept; psr16 = a ring buffer of `limit` # records in the debounce cache pool (one process, development, small sites); pdo = a database table # (production). Literal, not an env placeholder: the store is wired at compile time. store: null # Records the psr16 store keeps; the oldest is overwritten. limit: 500 # Cache key prefix of the psr16 store. Default: debounce.key_prefix. key_prefix: null pdo: # PDO DSN of the pdo store, when the connection is not a Doctrine one. Not both dsn and service. dsn: null # The Doctrine DBAL connection the pdo store uses: a connection name ("default" = # doctrine.dbal.default_connection) or a service id (a DBAL Connection or a PDO). # Default when dsn is null: the default connection. service: null # Table of the pdo store; the package's docs/migrations.md creates it. [A-Za-z_][A-Za-z0-9_]* only. table: indexnow_submissions # What `indexnow:history --purge` removes beyond. retention_days: 90 doctrine: # Hook Doctrine ORM. Needs indexnowkit/doctrine + doctrine/doctrine-bundle. enabled: true # Lower than Gedmo so slugs exist before URLs are resolved. listener_priority: -100 # Restrict the listener and the commit-safety middleware to these DBAL connection names. # Empty = all connections. connections: [] ``` ## Compile-time validation The container fails to build when: | Rule | Message | |---|---| | `dispatch: messenger` without `base_url` | a Messenger worker has no request context and would generate `http://localhost/...` URLs | | a `hosts` map without `base_url` | the default host would be unknown | | `strict_hosts: true` with neither `base_url` nor `hosts` | there would be no known host at all | | `engines: []` | at least one engine is required | | a literal `key` outside `[A-Za-z0-9-]{8,128}` | | | a literal `base_url` that is not an absolute http(s) URL | | | an `engines` entry that is neither a known engine nor an `http(s)` URL | | | `key_file.path` not starting with `/` or without `{key}` | | | a literal `key_location` that is not an absolute http(s) URL | | | a literal `previous_key` outside `[A-Za-z0-9-]{8,128}` | | | a literal `http.user_agent` containing a line break | | | `logging.levels` with an unknown event | the message lists the known events | | literal `sitemap.url` that is not an absolute http(s) URL | | | `history.pdo.dsn` and `history.pdo.service` together, a literal `history.store` other than `psr16`/`pdo`, a `history.pdo.table` that is not an identifier, a `history.key_prefix` with `{}()/\@:` | | | `dispatch: messenger` without `symfony/messenger` installed | install it, or use `dispatch: sync` | | a number outside its range: `max_url_length` ≥ 64, `http.timeout` ≥ 0.1, `batch.max_urls` 1–10000, `sitemap.max_bytes` ≥ 1024, `sitemap.max_sitemaps` ≥ 1, `key_file.cache_max_age` ≥ 0, `resolver.max_via_fanout` ≥ 1, `logging.forbidden_escalation` ≥ 1 | the node's own message | Literal values only. A `%env(...)%` placeholder is resolved at runtime, so it is skipped here and validated by the core's `Config` instead — see the next section. Core options with no node here: `retry.*` (Symfony retries through Messenger's `retry_strategy`, so the core `RetryPolicy` is not used by the bundle) and `serve_key_file`, replaced by `key_file.enabled`. ## Environment placeholders and runtime failures Anything read from an environment variable is only known when the container runs. A bad value there — an empty `INDEXNOW_KEY` in production, a malformed `INDEXNOW_BASE_URL` — used to surface as an exception thrown from a Doctrine flush or from `kernel.terminate`. Instead, the bundle logs one `critical` line and runs disabled until the value is fixed: ``` indexnow: invalid configuration, IndexNow is disabled until it is fixed: (run "bin/console indexnow:check") ``` `bin/console indexnow:check` prints the same error and exits non-zero, so it belongs in your deployment smoke tests. ## The `hosts` node and environment variables `hosts` is an array node. Symfony cannot populate a whole array node from a single environment variable, so this does **not** work: ```yaml indexnowkit: hosts: '%env(json:INDEXNOW_HOSTS)%' # not supported ``` List the hosts in YAML and put a placeholder on each key instead: ```yaml indexnowkit: base_url: '%env(INDEXNOW_BASE_URL)%' hosts: example.de: key: '%env(INDEXNOW_KEY_DE)%' base_url: 'https://example.de' ``` ## Per-environment defaults The Flex recipe sets `dry_run: true` in `dev` and `test`. Independently of that, the core turns `dry_run` on by itself whenever no key is configured and `kernel.environment` is not `prod` or `production`, so a developer who never sets `INDEXNOW_KEY` gets logging instead of a boot failure. The reverse is worth an alert: `dry_run` on in production means nothing is being submitted at all. `indexnow:check` reports that combination as an error, not a warning. ## Service aliases Every replaceable piece is a service with an interface alias, so an application can decorate it. | Interface | Service id | |---|---| | `IndexNowKit\IndexNowKit` (facade) | `indexnowkit` | | `Config` | `indexnowkit.config` | | `Http\TransportInterface` | `indexnowkit.transport` (lazy) wrapping `indexnowkit.transport.real` | | `Key\KeyProviderInterface` | `indexnowkit.key_provider` | | `Key\KeyFileResponder` | `indexnowkit.key_file_responder` | | `Url\UrlNormalizerInterface` | `indexnowkit.url_normalizer` | | `Url\UrlResolverInterface` | `indexnowkit.url_resolver` | | `Url\GuardedUrlResolver` | `indexnowkit.guarded_url_resolver` | | `Url\ObjectChangeHandler` | `indexnowkit.change_handler` | | `Url\RouteUrlResolverInterface` | `indexnowkit.route_url_resolver` | | `Url\ResolverLocatorInterface` | `indexnowkit.resolver_locator` | | `Attribute\AttributeReaderInterface` | `indexnowkit.attribute_reader` | | `ClientInterface` | `indexnowkit.client` | | `SubmitterInterface` | `indexnowkit.submitter` | | `Collector\CollectorInterface` | `indexnowkit.collector` | | `Debounce\DebounceStoreInterface` | `indexnowkit.debounce_store` | | `Throttle\ThrottleInterface` | `indexnowkit.throttle` | | `Dispatch\DispatcherInterface` | `indexnowkit.dispatcher` | | `Sitemap\SitemapSourceInterface` (and `Sitemap\SitemapReader`) | `indexnowkit.sitemap_reader` (only with `sitemap.enabled`) | | `Console\SubjectLoaderInterface` (`indexnowkit/console`) | `indexnowkit.entity_loader` (only with Doctrine) | | `Check\CheckerInterface` | `indexnowkit.checker` (runs every `Check\CheckInterface` service, autoconfigured with the `indexnowkit.check` tag) | | `Adapter\SubmitterFactoryInterface` (core) | `indexnowkit.command_submitter_factory` | | `Console\ResultFormatterInterface` (`indexnowkit/console`) | `indexnowkit.result_formatter` | | `Submission\SubmissionStoreInterface` | `indexnowkit.submission_store` (the null store; with `indexnowkit/history` and `history.store` the package's store, alias `indexnowkit.history.store`; your own service under this id replaces either) | | `Console\Vocabulary`, `Console\*Runner` (`indexnowkit/console`) | `indexnowkit.console.vocabulary`, `indexnowkit.console.{check,submit,submit_entity,explain,sitemap,key_generate}`, with `indexnowkit/history` `indexnowkit.console.{history,status}` | | `Check\WiringCheck`, core `Check\SitemapSpoolCheck`, history `Check\HistoryCheck` | `indexnowkit.check.wiring`, `indexnowkit.check.sitemap_spool`, `indexnowkit.check.history` (tagged `indexnowkit.check`) | | `History\HistoryConfig`, `Retry\ForbiddenCounter` (with `indexnowkit/history`) | `indexnowkit.history_config`, `indexnowkit.forbidden_counter`; the PDO / PSR-16 of the store: `indexnowkit.history.pdo` / `indexnowkit.history.cache` | | `Routing\KeyFileRouteLoader` | `indexnowkit.key_file_routes` | Only `indexnowkit`, `IndexNowKit\IndexNowKit` and the key file controller are public; inject the rest by type where you need them. How to decorate or replace each one: [extending.md](extending.md). ## Container parameters `indexnowkit.dispatch` (the resolved mode, after `auto`), `indexnowkit.messenger.transport`, `indexnowkit.messenger_routed`, `indexnowkit.doctrine_hooked`, `indexnowkit.key_file.path`, `indexnowkit.key_file.host`, `indexnowkit.key_file.route_name`, `indexnowkit.log_channel`. `indexnow:check` prints the first, third and fourth. # Custom resolvers `route:` covers pages that map to a named route. Anything else — a headless front-end, a URL assembled from several sources, one entity that publishes a different page per sales channel — is a resolver. ## Write one ```php namespace App\IndexNow; use IndexNowKit\Event; use IndexNowKit\Url\UrlResolverInterface; final class ProductUrls implements UrlResolverInterface { public function __construct(private readonly ChannelRepository $channels) {} /** @return iterable */ public function resolve(object $subject, Event $event): iterable { \assert($subject instanceof Product); foreach ($this->channels->forProduct($subject) as $channel) { yield $channel->baseUrl . '/p/' . $subject->getSlug(); } } } ``` Return absolute or `base_url`-relative URLs, or nothing at all when the object has no public page. The `$event` argument lets a resolver behave differently for a deletion, which is rarely needed but occasionally is. ## Reference it ```php use IndexNowKit\Attribute\IndexNow; #[IndexNow(resolver: ProductUrls::class)] class Product {} ``` With Symfony's default `services.yaml` (`autoconfigure: true` for everything under `App\`), the class is already a service whose id **is** its FQCN, and the bundle auto-tags every `UrlResolverInterface` service with `indexnowkit.url_resolver`. Nothing else to declare. Reference a service id instead when the resolver is registered under a name: ```yaml # config/services.yaml services: app.product_urls: class: App\IndexNow\ProductUrls arguments: ['@App\Repository\ChannelRepository'] ``` ```php #[IndexNow(resolver: 'app.product_urls')] ``` ## The pitfall: constructor dependencies Lookup happens in this order: 1. the tagged-service locator, by the exact string you wrote; 2. failing that, if the string is a class implementing `UrlResolverInterface`, it is instantiated with `new`. Step 2 only works when the constructor takes no required arguments. A resolver with dependencies that is **not** registered under the id you wrote fails with: > IndexNow resolver App\IndexNow\ProductUrls has constructor dependencies but is not registered as a service under > that id. Register it in services.yaml (autoconfigure: true tags it automatically) or reference the service id in > #[IndexNow(resolver: ...)]. Three ways to hit it, all fixable in one line: - the class lives outside the autoconfigured namespace — register it explicitly; - the service was registered under a different id — reference that id in the attribute; - `autoconfigure: false` in your services file — add the `indexnowkit.url_resolver` tag by hand. A resolver referenced by a string that is neither a tagged service nor an instantiable class fails with *"is neither a UrlResolverInterface service nor an instantiable class"*. Both errors are logged, not thrown: the flush completes and no URLs are produced. Verify the wiring without saving anything: ```bash bin/console debug:container --tag=indexnowkit.url_resolver bin/console indexnow:explain 'App\Entity\Product' 42 ``` ## Resolvers and rule guards A resolver rule is a rule like any other, so it keeps `when`, `fields`, `events`, `via` and `name`: ```php #[IndexNow(resolver: ProductUrls::class, when: 'isListed', fields: ['slug', 'listed', 'channels'])] ``` The guards are applied **before** the resolver runs. A resolver never sees an object whose `when` is false, and never runs for an event the rule does not subscribe to. ## Replacing URL resolution entirely `#[IndexNow(resolver: ...)]` replaces the URLs of one rule. To replace the whole mechanism — no attributes at all, metadata from your own source — decorate the `indexnowkit.url_resolver` service: ```yaml services: App\IndexNow\EverythingResolver: decorates: indexnowkit.url_resolver ``` A hand-written top-level resolver has no rules, so the per-rule guards do not apply to it. `GuardedUrlResolver` keeps the class-level event-subscription check for that case, and still catches everything it throws. If your objects cannot carry attributes but you do want rules, register them at runtime with `IndexNowKit\Attribute\RuleRegistry` and decorate `indexnowkit.attribute_reader` instead. That keeps the whole rule model — guards, per-rule deletions, `via`, locales — and is almost always the better choice. See the [core attribute reference](../core/attribute-reference.md#rules-registered-at-runtime). # Doctrine integration Entity hooks are the reason most people install this bundle: change an entity, commit, and the URLs go out. This page covers the wiring, its limits, and the settings you may need to touch. ## What must be installed ```bash composer require indexnowkit/doctrine doctrine/doctrine-bundle ``` Hooks activate only when **all** of these hold: - `indexnowkit/doctrine` is installed (`IndexNowKit\Doctrine\IndexNowListener` exists); - `doctrine/doctrine-bundle` is registered, or a `doctrine` extension is present in the container; - `indexnowkit.doctrine.enabled` is `true` (the default); - `indexnowkit.enabled` is `true`. Otherwise the bundle still works for manual submission and the commands, and `indexnow:check` says so: ``` ! doctrine: entity hooks are NOT active (needs indexnowkit/doctrine + doctrine/doctrine-bundle, doctrine.enabled: true and enabled: true); use indexnow:submit or $indexNow->submit() ``` `indexnow:submit-entity` and `indexnow:explain` are registered only when Doctrine is available, because they load entities through the manager registry. ## What gets registered | Service | Tag | Purpose | |---|---|---| | `indexnowkit.doctrine.listener` | `doctrine.event_listener` on `onFlush` and `postFlush` | classifies changes per rule, resolves URLs | | `indexnowkit.doctrine.middleware` | `doctrine.middleware` | watches the real COMMIT and ROLLBACK | | `indexnowkit.doctrine.staging` | — | holds URLs until the commit | | `indexnowkit.doctrine.sink` | — | hands committed URLs to the request collector | ## Renamed pages When a field a route parameter reads changes (the slug, the category the path goes through), the old URL now answers 404. The listener resolves the rule against the **previous** values of the change set and announces those URLs as deleted, next to the new URLs as updated, in the same flush (`ObjectChangeHandler::renamed()`, scenario A21). Only route rules; only fields that are writable properties (a `readonly` slug is logged at `debug` and skipped); the old page must have been public (`when` true before the change). ## Listener priority ```yaml indexnowkit: doctrine: listener_priority: -100 ``` The default is `-100`, low enough to run **after** listeners that compute the values URLs depend on. Gedmo Sluggable writes the slug in `onFlush`; if the IndexNow listener ran first it would resolve a route with an empty slug. Lower the value further if you have your own `onFlush` listener that must also run first. Raise it only if you know nothing computes URL-relevant data during the flush. ## Connections and entity managers ```yaml indexnowkit: doctrine: connections: ['default'] # empty = every connection ``` The list scopes both the event listener and the commit-safety middleware to those DBAL connection names. Leave it empty in a single-connection application. Set it when a second connection points at a read replica or a third-party database whose writes have nothing to do with your public pages, so its flushes do not pay for the listener. Names are DBAL connection names as configured under `doctrine.dbal.connections`, and they apply to both tags at once: restricting the listener without the middleware would leave URLs staged and never released. ## Commit safety Doctrine has no after-commit event, and `postFlush` runs **before** the outer `COMMIT` whenever `flush()` is wrapped in `wrapInTransaction()` or a manual transaction. So the listener does not deliver, it stages: 1. `postFlush` resolves the URLs and checks the connection's transaction nesting level. 2. Inside a transaction, the URLs are staged against the **native** connection object. 3. The DBAL driver middleware sees the real `commit()` (nesting level 0, identically in DBAL 3 and 4) and releases them to the collector, or sees `rollBack()` and discards them with a `debug` line. 4. A `commit()` that itself throws also discards, so a pooled connection never delivers them later. 5. Outside a transaction the URLs go to the collector immediately. 6. `kernel.terminate` flushes the collector once, after the response was sent. If a driver exposes no native connection object — an unusual wrapped or custom driver — the listener logs `indexnow: driver has no native connection object; submitting inside an open transaction` and delivers anyway, rather than losing the URLs. ## Version support | | Supported | |---|---| | Doctrine ORM | 2.19+ and 3.x | | DBAL | 3.x and 4.x | | DoctrineBundle | 2.13+ and 3.x | DBAL 4 changed `commit()` and `rollBack()` to return `void` where DBAL 3 returns `bool`; the middleware detects which is present and installs the matching connection wrapper. Nothing to configure. ORM 2 and 3 differ in how a collection reports its mapping (an array versus an object); the listener reads the field name from both. ## What bypasses the unit of work These never reach `onFlush`, so nothing is submitted: - DQL and QueryBuilder bulk `UPDATE` and `DELETE`; - `Connection::executeStatement()` and raw SQL; - `INSERT ... SELECT`; - anything written by another process, a migration, or a database trigger. That is conformance scenario A13 and it is a documented limitation, not a bug: an ORM cannot report changes it never saw. Submit those URLs yourself: ```bash bin/console indexnow:submit-entity 'App\Entity\Post' 42 43 bin/console indexnow:submit-entity 'App\Entity\Post' # every entity, up to --limit bin/console indexnow:submit /posts/hello /posts/world ``` A large migration is better served by regenerating the sitemap and running `indexnow:sitemap --changed-since`. ## Long-running processes A Messenger worker flushes after each handled message, because `FlushListener` also listens to `WorkerMessageHandledEvent`. A console command flushes on `console.terminate`. A custom long-running command — an importer looping over 100 000 rows — gets neither until it exits. The collector grows for the whole run and the batch goes out at the very end, or is lost if the process is killed. Flush periodically: ```php foreach ($rows as $i => $row) { $this->em->persist($this->toEntity($row)); if ($i % 500 === 0) { $this->em->flush(); $this->em->clear(); $this->indexNow->flush(); // hand the collected URLs over } } $this->em->flush(); $this->indexNow->flush(); ``` Also pick the debounce store deliberately for such runs: `cache.app` (the default, shared across processes) keeps the window meaningful, while `memory` is per process and bounded at 50 000 entries. ## Multiple rules on one entity Every `#[IndexNow]` rule of a changed entity is classified separately. During one flush a `Post` can produce an update for its article page and a deletion for its AMP page, because `hasAmp` turned false while `isPublished` stayed true. The listener resolves the deletion in `onFlush`, while the old state is still readable, and the update in `postFlush`. Changed to-many associations are handled too: `post.tags` is not part of the post's change set, so a scheduled collection update re-classifies the owner with the association's field name as the changed field. Use `bin/console indexnow:explain 'App\Entity\Post' 42` to see which rule decided what. # Extending and replacing pieces Every piece of the pipeline is a container service with an interface alias. Three ways to change behaviour, in the order you should try them: 1. **Configuration.** Most knobs are in `indexnowkit.yaml` ([configuration.md](configuration.md)). Prefer it: the value is validated at compile time and shows up in `indexnow:check`. 2. **Decorate.** Keep the shipped implementation, wrap it. Symfony's `#[AsDecorator]` on the service id below is enough; the bundle sees your class through the alias. 3. **Replace.** Register your own implementation of the interface and alias the service id to it. Do this when the shipped implementation is the wrong model, not when you need to tweak it. ## What is replaceable, and how | Piece | Service id | Interface | Config knob | Decorate / replace | |---|---|---|---|---| | HTTP client | `indexnowkit.transport.real` (built by `TransportFactory`) | PSR-18 `ClientInterface` or symfony `HttpClientInterface` | `http.client`, `http.timeout`, `http.user_agent` | point `http.client` at any client service: scoped client, proxy, `RetryableHttpClient`, extra headers | | Transport | `indexnowkit.transport` (lazy, wraps the real one) | `Http\TransportInterface`; `Http\StreamingTransportInterface` for streamed GET | | decorate; see the streaming note below | | Key provider | `indexnowkit.key_provider` | `Key\KeyProviderInterface` | `key`, `hosts`, `key_location`, `previous_key`, `strict_hosts` | replace for keys from a database or a vault; a rotation needs only `previous_key` | | Key file endpoint | `indexnowkit.key_file_responder` + `KeyFileController` | `Key\KeyFileResponder` (final class) | `key_file.*` | disable it and serve the file yourself | | URL normalizer | `indexnowkit.url_normalizer` | `Url\UrlNormalizerInterface` | `base_url`, `max_url_length` | decorate to strip tracking parameters, force a canonical host, or allow only some paths (throw `InvalidUrlException`: the URL becomes a `skipped/invalid_url` result). This is the one hook every path shares, including `--force` / `--dry-run` commands | | URL resolver | `indexnowkit.url_resolver` | `Url\UrlResolverInterface` | `#[IndexNow]` rules, `resolver.max_via_depth`, `resolver.max_via_fanout` | decorate, or add resolvers referenced by `#[IndexNow(resolver:)]` ([custom-resolvers.md](custom-resolvers.md)) | | Attribute reader | `indexnowkit.attribute_reader` | `Attribute\AttributeReaderInterface` | | decorate with `RuleRegistry` to register rules at runtime | | Guarded resolver / change handler | `indexnowkit.guarded_url_resolver`, `indexnowkit.change_handler` | `Url\GuardedUrlResolver`, `Url\ObjectChangeHandler` (final) | | consume them; the ORM listener and the commands do | | Client (HTTP half) | `indexnowkit.client` | `ClientInterface` | `engines`, `hosts..engines`, `batch.max_urls`, `logging.*` | decorate for per-host policy or metrics at request level | | Submitter | `indexnowkit.submitter` | `SubmitterInterface` | `debounce.*`, `dry_run`, `enabled` | decorate (`RetryingSubmitter` is an example); listeners via PSR-14 events. Note: `--force` / `--dry-run` commands build their own `Submitter` through `SubmitterFactory` and bypass a decorator here (they do fire the PSR-14 events) | | Collector | `indexnowkit.collector` | `Collector\CollectorInterface` | `collector.max_urls`, `collector.detect_leaks` | replace for a different per-request store | | Dispatcher | `indexnowkit.dispatcher` | `Dispatch\DispatcherInterface` | `dispatch`, `messenger.transport`, `messenger.delay`, `messenger.stamps` | replace for another queue (`dispatch: none` + your own drain of the collector). A decorator cannot add stamps after the fact: use `messenger.stamps` | | Debounce store | `indexnowkit.debounce_store` | `Debounce\DebounceStoreInterface` | `debounce.store` (any PSR-6 pool, `memory`, `none`), `debounce.key_prefix` | replace | | Throttle | `indexnowkit.throttle` | `Throttle\ThrottleInterface` | `throttle.max_requests_per_minute` | replace for a shared (Redis) limiter | | Clock | `indexnowkit.clock` | PSR-20 `Psr\Clock\ClockInterface` (default `Clock\SystemClock`) | | replace to move every time the graph reads at once: the throttle window, the `memory` debounce window, the time a submission record gets and the pre-flight decorator. `IndexNowKit\Testing\FrozenClock` in tests ([testing.md](testing.md)) | | Sitemap source | `indexnowkit.sitemap_reader` | `Sitemap\SitemapSourceInterface` | `sitemap.*` | decorate to filter or rewrite entries; replace to read from another place or format. Registered only with `indexnowkit/sitemap` installed and `sitemap.enabled: true` (the default) | | Entity loader (commands) | `indexnowkit.entity_loader` | `indexnowkit/console` `Console\SubjectLoaderInterface` | | decorate for soft deletes, tenant scoping, another id format (`byIds()` / `all()` receive the `Event`). Registered only when the Doctrine integration is active ([doctrine.md](doctrine.md)) | | Command submitter (`--force`, `--dry-run`) | `indexnowkit.command_submitter_factory` | core `Adapter\SubmitterFactoryInterface` | | decorate to wrap what the manual commands submit through | | Command output | `indexnowkit.result_formatter` | `indexnowkit/console` `Console\ResultFormatterInterface` | | replace to match your CLI's JSON envelope or table style | | Command bodies | `indexnowkit.console.*` | `indexnowkit/console` `Console\*Runner` | | the commands (the classes of `indexnowkit/console`, `sitemap` and `history`, see "Replacing a command") are input parsing over these; reuse a runner from your own command (a tenant loop over `SubmitSubjectsRunner`) | | Configuration source of `check` / `config` | `indexnowkit.console.config_source` | `indexnowkit/console` `Console\ConfigSourceInterface` (`DependencyInjection\ConsoleConfigSource`) | | what the two commands read: the processed tree, its strict build, the package blocks | | `indexnow:check` | `indexnowkit.checker` | `Check\CheckerInterface`; add lines with `Check\CheckInterface` services (autoconfigured) | | add checks rather than replacing the checker | | Key file route | `indexnowkit.key_file_routes` | `Routing\KeyFileRouteLoader` | `key_file.path`, `key_file.host`, `key_file.route_name` | do not import `config/routes.php` and register your own route to `KeyFileController` | | Doctrine listener | `indexnowkit.doctrine.listener` | `IndexNowListener` (final) | `doctrine.*` | disable and write your own on top of `ObjectChangeHandler` (`created()`, `updated()`, `deleted()`, `renamed()`); skip a namespace by decorating the attribute reader to return an empty `RuleSet` | | Flush timing | `indexnowkit.flush_listener` | `EventListener\FlushListener` | `flush.priority`, `flush.console_priority` | order against your own terminate listeners | | Logging | every service, channel `logging.channel` | PSR-3 | `logging.channel`, `logging.levels`, `logging.max_urls`, `logging.forbidden_escalation` | route the channel in `monolog.yaml`; per-outcome levels need no code | Not replaceable on purpose: the spool the sitemap reader parses through (`Sitemap\Spool`), the rule compiler and the result/reason value objects. They are the parts a wrong replacement would silently break, and they have no behaviour worth swapping: configure the spool (`sitemap.spool`, `sitemap.spool_dir`) instead. ## Decorating the sitemap source Keep the shipped reader (fetching, gzip, indexes, retries, spooling) and shape what comes out of it: ```php use IndexNowKit\Sitemap\SitemapSourceInterface; use Symfony\Component\DependencyInjection\Attribute\AsDecorator; use Symfony\Component\DependencyInjection\Attribute\AutowireDecorated; #[AsDecorator('indexnowkit.sitemap_reader')] final class PublicOnlySitemapSource implements SitemapSourceInterface { public function __construct(#[AutowireDecorated] private readonly SitemapSourceInterface $inner) {} public function read(string $sitemap, ?\DateTimeImmutable $changedSince = null): iterable { foreach ($this->inner->read($sitemap, $changedSince) as $entry) { if (!str_starts_with($entry->url, 'https://www.example.com/private/')) { yield $entry; } } } } ``` Yield, do not collect: the command submits every `batch.max_urls` entries while your generator is still running. A replacement that reads from a database or a search index implements the same interface; `$sitemap` is then whatever the command was given (or `sitemap.url`), free for you to interpret. `--allow-foreign-hosts` reaches only the shipped reader, the command warns when it is passed to another source. The shipped reader also accepts a local path or `file://` URL: a sitemap the application writes into `public/` can be read without the web server (`bin/console indexnow:sitemap /var/www/public/sitemap.xml`). Its index parts must then be local files too, or the run needs `--allow-foreign-hosts` to fetch them by URL. ## Decorating the transport ```php use IndexNowKit\Http\Response; use IndexNowKit\Http\StreamingTransportInterface; use Symfony\Component\DependencyInjection\Attribute\AsDecorator; use Symfony\Component\DependencyInjection\Attribute\AutowireDecorated; #[AsDecorator('indexnowkit.transport')] final class MeteredTransport implements StreamingTransportInterface { public function __construct(#[AutowireDecorated] private readonly StreamingTransportInterface $inner, private readonly Meter $meter) {} public function post(string $url, string $json, array $headers = []): Response { $this->meter->count('indexnow.post'); return $this->inner->post($url, $json, $headers); } public function get(string $url): Response { return $this->inner->get($url); } public function download(string $url, $sink): Response { return $this->inner->download($url, $sink); } } ``` Implement `StreamingTransportInterface`, not only `TransportInterface`: a decorator without `download()` still works, but the sitemap reader then falls back to `get()` and buffers each document once (up to 50 MiB) before spooling it. Most needs (proxy, retries, headers, timeouts, mTLS) belong in `http.client` rather than in a transport decorator. ## Replacing the key provider ```php use IndexNowKit\Key\KeyProviderInterface; final class VaultKeyProvider implements KeyProviderInterface { /* keyFor(), keyLocationFor(), isKnownKey(), managedHosts() */ } ``` ```yaml services: App\IndexNow\VaultKeyProvider: ~ indexnowkit.key_provider: '@App\IndexNow\VaultKeyProvider' ``` `indexnow:check`, the key file controller, the profiler panel and `explain` all go through the alias, so they report your keys. ## Submitting from your own domain events Nothing here needs Doctrine. Call the facade from the service that knows the page changed (a CMS publish action, a MongoDB document listener, an import): `$indexNow->submitEntity($page, Event::Updated)` resolves the URLs from the `#[IndexNow]` rules of the object and submits at once; `$indexNow->collect($urls)` + the request-end flush delivers through the configured dispatcher instead. Objects that cannot carry attributes get their rules from `RuleRegistry::registerFor()` (decorate `indexnowkit.attribute_reader`). ## Adding lines to `indexnow:check` A service implementing `Check\CheckInterface` is tagged `indexnowkit.check` by autoconfiguration and runs after the built-in checks; it adds lines, it cannot throw (an exception becomes an error line naming the class): ```php use IndexNowKit\Check\CheckInterface; use IndexNowKit\Check\CheckReport; final class TenantKeysCheck implements CheckInterface { public function __construct(private readonly TenantRepository $tenants) {} public function check(CheckReport $report): void { $missing = $this->tenants->withoutIndexNowKey(); $missing === [] ? $report->ok('tenants: every active tenant has a key') : $report->error(\sprintf('tenants without a key: %s', implode(', ', $missing))); } } ``` ## Replacing a command The `indexnow:*` commands are classes of the packages, not of the bundle (since 0.15.0): `IndexNowKit\Console\Command\*` of `indexnowkit/console` (`check`, `config`, `submit`, `submit-entity` as `SubmitSubjectsCommand`, `explain`, `key:generate`, the three "not installed" stubs), `IndexNowKit\Sitemap\Console\SitemapCommand`, `IndexNowKit\History\Console\HistoryCommand` and `StatusCommand`. The bundle registers each under its class name as the service id, with the runner (`indexnowkit.console.*`), the vocabulary, `DependencyInjection\ConsoleConfigSource` (what `check` and `config` read), `%kernel.project_dir%/.env.local` for `key:generate` and `indexnowkit.check.samples` as arguments. To change what a command does, decorate the runner, not the command — that is the body: ```yaml services: App\IndexNow\TenantAwareSubmitEntity: decorates: indexnowkit.console.submit_entity # a SubmitSubjectsRunner that loops over the tenants arguments: ['@.inner'] ``` To replace a command outright, register your own under the same name; the last `console.command` registered for a name wins, and `IndexNowKit\Console\Definitions` gives it the same arguments and options: ```php #[AsCommand(name: 'indexnow:submit')] final class SubmitThroughGatewayCommand extends Command { protected function configure(): void { Definitions::submit()->applyTo($this); } } ``` `SubmitSubjectsCommand` carries no `#[AsCommand]` (its name is the vocabulary's, `indexnow:submit-entity` here), so the bundle registers it with the `command` and `description` attributes of the tag; a replacement of that one names itself the same way. Every other command is lazy through its attribute; the bundle's own test keeps every `console.command` lazy. ## Conformance tests for your integration `IndexNowKit\Testing\Conformance\CoreConformanceTestCase` (shipped by the core, needs PHPUnit) runs the protocol scenarios of the spec (C01, C03, C04, C06, C09 to C12, C14, C19, C20) against the facade *your* container built. The bundle's own `tests/Functional/CoreConformanceTest.php` is the reference: boot your kernel, return `indexnowkit` and the `FakeTransport` aliased as `indexnowkit.transport`, optionally a second configured host. Run it in the application that decorates or replaces bundle services to prove the pipeline still conforms. ## Listening instead of replacing `Submitter` dispatches every `Result` as a PSR-14 event through `event_dispatcher` when it exists; a listener on `IndexNowKit\Result` gets each outcome for metrics or alerting without touching the pipeline ([operations guide](../core/operations.md)). ## What is the core's `SubmitUrlsHandler` is `Retry\WorkerOutcome` (which URLs to retry, which were rejected for good, the log lines) plus `RecoverableMessageHandlingException` with the engine's Retry-After. The commands configure their arguments and options from `Console\Definitions` and `Sitemap\Console\Definitions` (`CommandDefinition::applyTo($command)`), so `bin/console indexnow:submit-entity --help` matches artisan and Yii2; a custom command over a core runner can call the same `applyTo()`. # HTTP client ## The default With no `http.client` configured, the bundle discovers a PSR-18 client. If `symfony/http-client` is installed it is used and configured with `http.timeout` (10 seconds by default), `max_duration` at twice that, and **no redirects**; Guzzle gets the equivalent treatment. Any other discovered client keeps its own defaults, including its own timeout, because the library will not reconfigure a client it did not create. Redirects are disabled on purpose. An engine endpoint that redirects is a misconfiguration, and following it would hide the real response. Install the pieces if discovery fails: ```bash composer require symfony/http-client nyholm/psr7 ``` The failure is explicit: *"No PSR-18 HTTP client / PSR-17 factories found."* It is raised when the transport is first used, not at boot. ## Built on first use The transport service is a `LazyTransport` wrapping a closure. Discovery and client construction happen at the first outgoing request, so: - a request that submits nothing pays nothing; - a `dry_run` setup never builds a client at all; - `indexnow:check` can report a missing client as a check error instead of crashing. ## Using your own client Point `http.client` at a service id. Two kinds are accepted: ```yaml indexnowkit: http: client: 'app.psr18_client' # a Psr\Http\Client\ClientInterface service ``` ```yaml indexnowkit: http: client: 'my_scoped.client' # a symfony/http-client service, wrapped in Psr18Client automatically ``` Anything else fails at runtime with *"indexnowkit.http.client must be a PSR-18 client or a symfony/http-client service"*. A client you supply is used as it is: `http.timeout` is **not** applied to it, because its own configuration wins. Set the timeout on the client instead. ## Scoped clients The most useful case is a `framework.http_client` scoped client, which gives you proxy settings, retries, extra headers and per-service timeouts without touching the rest of the application. ```yaml # config/packages/framework.yaml framework: http_client: scoped_clients: indexnow.client: scope: 'https://(api\.indexnow\.org|yandex\.com|www\.bing\.com)' timeout: 10 max_duration: 20 max_redirects: 0 headers: Accept: 'application/json' proxy: '%env(default::HTTP_PROXY)%' retry_failed: max_retries: 2 delay: 1000 ``` ```yaml # config/packages/indexnowkit.yaml indexnowkit: http: client: 'indexnow.client' ``` Keep `max_redirects: 0`. And be deliberate about `retry_failed`: it retries transport-level failures underneath the library, which then never sees them. That is fine for connection resets, but leave 429 and 5xx to the library so `Retry-After` and the `Result` taxonomy still apply — Symfony's default retry status codes include 429 and 5xx, so narrow `http_codes` if you enable it. ## Custom endpoints `engines` accepts full endpoint URLs next to the known names, which is how you point at a staging or mock server: ```yaml indexnowkit: engines: ['https://api.indexnow.org/indexnow'] ``` Custom endpoints must use `https`, because the key travels in the request body. Plain `http` is accepted only for loopback hosts (`localhost`, `127.0.0.1`, `[::1]`), which is what test suites use. ## User-Agent The default is `indexnowkit-php/ (+https://github.com/indexnowkit/php)`. Override it with `http.user_agent` when an engine or your own proxy needs to identify the traffic. Line breaks are rejected. ## Timeouts and what they mean `http.timeout` is a per-request timeout applied to the client the bundle creates. A timeout surfaces as a `failed` result with reason `transport`, `retryable: true`, and a warning on the `indexnow` channel. It is never an exception in your application. Ten seconds is generous for an endpoint whose entire job is to accept a JSON list. Lower it if your worker throughput matters more than the occasional slow acceptance; raise it only if you see `transport` failures that disappear on retry. # Async delivery with Messenger Synchronous dispatch sends after the response has been sent, which is already off the critical path. Messenger buys you one thing more: **retries**. A 429 or a 5xx from an engine is retried with back-off by the transport instead of being logged and forgotten. ## The one-line setup Name a transport and the bundle does the routing: ```yaml # config/packages/indexnowkit.yaml indexnowkit: key: '%env(INDEXNOW_KEY)%' base_url: '%env(INDEXNOW_BASE_URL)%' messenger: transport: async ``` That adds `framework.messenger.routing` for `IndexNowKit\SymfonyBundle\Messenger\SubmitUrlsMessage` pointing at the `async` transport, so `messenger.yaml` needs no edit. The transport itself must exist: ```yaml # config/packages/messenger.yaml framework: messenger: failure_transport: failed transports: async: dsn: '%env(MESSENGER_TRANSPORT_DSN)%' retry_strategy: max_retries: 3 delay: 5000 multiplier: 3 failed: 'doctrine://default?queue_name=failed' ``` Then run a worker: `bin/console messenger:consume async`. If you prefer to route it yourself, leave `indexnowkit.messenger.transport` unset and add the entry by hand: ```yaml framework: messenger: routing: 'IndexNowKit\SymfonyBundle\Messenger\SubmitUrlsMessage': async ``` ## `dispatch: auto` `auto` resolves to `messenger` when `symfony/messenger` is installed **and** `framework.messenger.transports` is non-empty; otherwise to `sync`. `enabled: false` forces `none`. That detection is about transports, not about routing. A message that is dispatched but not routed to any transport is handled **synchronously** by the bus, inside `kernel.terminate` — it works, but nothing is retried. That is why `indexnow:check` prints a warning when the mode is `messenger` and `SubmitUrlsMessage` is not routed: ``` ! dispatch is "messenger" but SubmitUrlsMessage is not routed to a transport: it is handled synchronously, 429/5xx are not retried. Set indexnowkit.messenger.transport or add framework.messenger.routing. ``` Set `dispatch: messenger` explicitly in production so the requirement is stated rather than inferred. ## `base_url` is mandatory A worker has no request context. Without `base_url` the router would generate `http://localhost/...` and every relative URL handed to `submit()` would be dropped as invalid. The container refuses to compile with `dispatch: messenger` and no `base_url`. The same applies to per-host `base_url` in multi-domain setups; see [multi-domain.md](multi-domain.md). ## The bus and transactions The default bus is `messenger.default_bus`; change it with `indexnowkit.messenger.bus`. Whatever bus you name must carry the `dispatch_after_current_bus` middleware, which the default buses do. The message is dispatched with `DispatchAfterCurrentBusStamp`, so when the collector is flushed while another message is being handled, the submission is only dispatched after that handler finished. Combined with the `doctrine_transaction` middleware this keeps the guarantee that nothing is announced before its transaction commits. A failure to dispatch never breaks the request: it is logged as `indexnow: cannot queue {count} URL(s) (message {id}), they are lost: {error}` on the `indexnow` channel (the line of the core's `Dispatch\BatchingDispatcher`, the same in every adapter; `{noun}` is `message` here). ## Retry semantics `SubmitUrlsHandler` submits the batch and then decides: - Nothing retryable — the handler returns. Failures that are final (400, 403, 422) are logged once more at `error` as `indexnow: {count} URL(s) of job {id} rejected permanently ({reasons}); run "bin/console indexnow:check"`; the message is acknowledged and not retried, because retrying a bad key changes nothing. - Something retryable (429, 5xx, network) — it logs `indexnow: {count} URL(s) of job {id} will be retried` at `info` and throws `RecoverableMessageHandlingException`, which hands the decision to the transport's `retry_strategy`. On Symfony 7.2 and later, when the engine sent a `Retry-After`, the largest value in the batch is passed to that exception as an explicit retry delay in milliseconds, so the transport waits as long as the engine asked instead of applying its own back-off. On older versions the transport's `retry_strategy` decides alone. Note what is retried: the **whole batch**, not just the failed URLs. Debouncing keeps that cheap — URLs that succeeded on the first attempt are inside their debounce window and come back as `skipped` rather than being sent again — as long as `debounce.per_url` is larger than your retry delays, which the 600-second default comfortably is. ## When retries are exhausted The bundle adds no dead-letter machinery of its own; it relies on Messenger's. Configure `framework.messenger.failure_transport` and use the standard commands: ```bash bin/console messenger:failed:show bin/console messenger:failed:show -vv # the IndexNow exception message and the URL list bin/console messenger:failed:retry bin/console messenger:failed:remove ``` Without a failure transport an exhausted message is discarded and the URLs are lost with only the worker's log line to show for it. Set one. For a manual resubmission the URLs are in the failed message; `bin/console indexnow:submit ...` sends them directly, and `--force` bypasses the debounce window if the failure is recent. ## The worker flushes too `FlushListener` listens to `WorkerMessageHandledEvent`, so anything a message handler collects — an entity written by a queue job, for example — is submitted after that message is handled, not at the end of the worker process. Long-running custom commands that are not Messenger workers get no such event. Call `$indexNow->flush()` yourself, periodically, or the collector grows for the life of the process and the URLs go out in one batch at the end. ## Monitoring | Signal | Where | |---|---| | `indexnow: {count} URL(s) of job {id} will be retried` | `indexnow` channel, `info`, in the worker | | `indexnow: {count} URL(s) of job {id} rejected permanently ({reasons}); run "bin/console indexnow:check"` | `indexnow` channel, `error`, in the worker | | `indexnow: cannot queue {count} URL(s) (message {id}), they are lost` | `indexnow` channel, `error`, in the web request | | exhausted retries | Messenger's failure transport, default channel | | the URLs a request handed over | Web Profiler panel, which also says results appear in the worker's log | The profiler panel of the dispatching request cannot show HTTP outcomes: by design the request is over before the worker runs. # Multiple domains For IndexNow every host is a separate site with its own key and its own key file. Sub-domains included: `example.com` and `blog.example.com` are two sites, not one. This page works through one application serving three of them: `example.com`, `example.de` and `blog.example.com`. ## Configuration ```yaml # config/packages/indexnowkit.yaml indexnowkit: key: '%env(INDEXNOW_KEY)%' # the key for the base_url host base_url: 'https://example.com' hosts: example.de: key: '%env(INDEXNOW_KEY_DE)%' base_url: 'https://example.de' blog.example.com: key: '%env(INDEXNOW_KEY_BLOG)%' base_url: 'https://blog.example.com' strict_hosts: true ``` Three things to notice. **`base_url` is still required.** It names the default host, the one the top-level `key` belongs to. A `hosts` map without it fails at compile time. **Each host gets its own `base_url`.** A console command or a Messenger worker has no request to inherit a host from. Without a per-host `base_url`, every URL would be generated on the single global one and the German pages would be announced as `https://example.com/...`, which the engine rejects with 422. **`strict_hosts: true` is the safety net.** Without it the default key is used for *any* host you submit, including one that arrived from user input or from a stale database row. With it, a host that is neither `base_url` nor in the map is skipped with reason `no_key` and a warning, instead of being announced under someone else's key. Add `key_location` under a host when its key file cannot live at `/{key}.txt`: ```yaml shop.example.com: key: '%env(INDEXNOW_KEY_SHOP)%' base_url: 'https://shop.example.com' key_location: 'https://shop.example.com/keys/indexnow.txt' ``` It must be on that same host. A `key_location` pointing elsewhere is rejected at configuration time, because engines answer 422 for it. ## Routes with a host Symfony routes can pin a host, and the router bridge honours it: ```php $routes->add('de_article_show', '/artikel/{slug}') ->controller([ArticleController::class, 'show']) ->host('example.de'); ``` Generating that route produces an `example.de` URL, and the client groups by host and picks the German key. Nothing else is needed for this case. ## Rules that pin a host When one entity class serves several domains, pin the host on the rule instead of on the route: ```php use IndexNowKit\Attribute\IndexNow; use IndexNowKit\Attribute\Param\Accessor; #[IndexNow(route: 'page_show', params: ['slug' => 'slug'], host: 'example.de')] // literal #[IndexNow(route: 'page_show', params: ['slug' => 'slug'], host: new Accessor('tenant.domain'))] // per row class Page {} ``` A literal host is used as given. An `Accessor` (or any `Param\*` value) is read from the object, so a multi-tenant `Page` announces itself on its tenant's domain. The bridge then generates on `hosts..base_url` if configured, and on `https://` otherwise. Two rules that differ only by host need different `name` values, or the second overwrites the first: ```php #[IndexNow(route: 'page_show', params: ['slug' => 'slug'], host: 'example.com', name: 'page_com')] #[IndexNow(route: 'page_show', params: ['slug' => 'slug'], host: 'example.de', name: 'page_de')] ``` ## Key files Every host must serve its own key file, and only its own. The bundle's controller reads the request host and asks the key provider whether that key belongs to it, so `https://example.de/.txt` returns 200 while `https://example.de/.txt` returns 404. One tenant cannot serve another tenant's key file, which would otherwise let them claim ownership signals on a domain they do not control. Nothing extra is needed if all three domains reach the same Symfony application. If they do not — a static site on one of them, a CDN in front of another — serve that host's key file there by hand, with `200`, `text/plain` and no redirect. Restrict the route to a single host pattern with `key_file.host` when only one domain should answer it. ## Verifying ```bash bin/console indexnow:check # every managed host bin/console indexnow:check --host example.de # one host bin/console indexnow:check --host example.de --live ``` For each host the report fetches the key file over HTTP and compares the body against the configured key, so a missing file, a redirect, an HTML error page or a stale key after a rotation all surface as an error naming the host and the URL (with the key masked). `--live` additionally POSTs the site root to every engine, even when `dry_run` is on, and reports what each answered. The report also confirms `strict_hosts` is active, which is the line to look for after adding it. ## What a wrong host looks like | Symptom | Cause | |---|---| | `skipped` / `no_key`, warning `skipping N URL(s) for unmanaged host` | the host is not in `hosts` and is not the `base_url` host, with `strict_hosts` on | | `failed` / `unprocessable` (422) | the URLs do not belong to the host they were submitted under, usually a missing per-host `base_url` in a console command or worker | | `failed` / `invalid_key` (403) | that host's key file is not reachable or does not match | | URLs generated as `http://localhost/...` | no `base_url` at all, outside a request | `bin/console indexnow:explain App\Entity\Page 42` prints the host and the key file for every URL it would submit, which is usually faster than reading logs. ## www and apex `example.com` and `www.example.com` are two hosts to IndexNow: each needs its own key file, and a URL submitted under the other one's key answers 422. Pick the canonical one (the one your pages link to and `` names), put it in `base_url`, redirect the other with `301`, and do not list it in `hosts` — listing both would announce two copies of every page. With `strict_hosts: true` a request that reached the application under the non-canonical name submits nothing instead of announcing duplicates. ## hreflang clusters Localized pages that point at each other with `hreflang` are one cluster to the engines: when one changes, announce the cluster. A rule with `locales: 'all'` does that for the locales of one model; for locales living on other hosts `locale_hosts` sends each locale to its host under that host's key. When translations are separate objects, `via:` walks to them: ```php #[IndexNow(route: 'article_show', params: ['slug' => 'slug'], locales: 'all')] // every locale of this article #[IndexNow(via: 'translations')] // or: the sibling objects' own rules ``` `locales: 'all'` expands to `framework.enabled_locales`. An application that never filled that list gets one URL without a locale per rule, silently — `indexnow:check` says so (`router.locales`, one warning naming the entities that ask for every locale). Either list the locales in `framework.enabled_locales` or name them on the rule (`locales: ['en', 'de']`). # Testing your integration Two questions are worth testing: *would this change be announced?* and *what exactly would be sent?* The first is answered by `dry_run`, the second by swapping the transport for a fake. ## Option 1: dry run The Flex recipe already sets `dry_run: true` in `test`. The whole pipeline runs — rules, guards, URL generation, normalization, host grouping, key lookup, debouncing — and stops before the POST. Results come back as `skipped` with reason `dry_run`. ```php $results = self::getContainer()->get(IndexNowKit::class)->submitEntity($post); self::assertSame(IndexNowKit\Reason::DryRun, $results[0]->reason); self::assertSame(['https://www.example.com/posts/hello'], $results[0]->urls); ``` Good enough for "the rule fires for this entity". It never touches the network and needs no container changes. ## Option 2: fake the transport Alias the transport service to `IndexNowKit\Testing\FakeTransport` in a compiler pass, and the recorded POSTs become assertable. This is the pattern the bundle's own functional tests use. ```php // tests/TestKernel.php protected function build(ContainerBuilder $container): void { $container->addCompilerPass(new class implements CompilerPassInterface { public function process(ContainerBuilder $container): void { $container->setAlias('indexnowkit.transport', FakeTransport::class)->setPublic(true); $container->getDefinition('indexnowkit')->setPublic(true); } }); } protected function configureContainer(ContainerConfigurator $container): void { $container->services()->set(FakeTransport::class)->public(); $container->extension('indexnowkit', [ 'key' => '0123456789abcdef0123456789abcdef', 'base_url' => 'https://www.example.com', 'dispatch' => 'sync', 'debounce' => ['per_url' => 0], ]); } ``` Aliasing `indexnowkit.transport` replaces the lazy wrapper, so discovery never runs and no HTTP client is needed at all in the test environment. Then: ```php public function testPublishingAnArticleSubmitsItsUrl(): void { $client = self::createClient(); $transport = self::getContainer()->get(FakeTransport::class); $client->request('POST', '/articles', ['title' => 'Hello', 'published' => '1']); self::assertResponseIsSuccessful(); self::assertCount(1, $transport->posts); self::assertSame( ['https://www.example.com/en/articles/hello'], $transport->posts[0]['body']['urlList'], ); self::assertSame('www.example.com', $transport->posts[0]['body']['host']); } ``` Set `dry_run: false` for this variant, or nothing is ever posted. ## Making the test deterministic - **Anything that reads the time** — the debounce window, the throttle window, the time a submission record gets — comes from one service, `indexnowkit.clock`. Replace it in the compiler pass and move time instead of sleeping: ```php // build(), in the compiler pass $container->register('indexnowkit.clock', IndexNowKit\Testing\FrozenClock::class)->setPublic(true); // the test $clock = self::getContainer()->get('indexnowkit.clock'); $indexNow->submit([$url]); // sent $indexNow->submit([$url]); // debounced $clock->advance(601); $indexNow->submit([$url]); // sent again ``` Register it (do not alias it) so the definition really replaces the default `Clock\SystemClock`; see the core's [testing guide](../core/testing.md). - **`debounce: {per_url: 0}`** or `debounce: {store: memory}`. With the default `cache.app` and a real pool, a second test submitting the same URL is silently debounced and the assertion fails for the wrong reason. - **`dispatch: sync`**. Sending happens on `kernel.terminate`, which the test client fires; with `messenger` you would assert on the transport's message queue instead. - **A separate cache directory per kernel variant**, if you boot kernels with different configurations in one suite. Symfony caches the compiled container by environment name, and two variants sharing it will silently reuse the wrong wiring. ## Asserting on the collector instead When you care that URLs were *collected* but not how they were sent, read the collector before terminate, or swap the dispatcher: ```php // configureContainer() $container->services()->set(RecordingDispatcher::class)->public(); // build(), in the compiler pass $container->setAlias('indexnowkit.dispatcher', RecordingDispatcher::class)->setPublic(true); // the test $dispatcher = self::getContainer()->get(RecordingDispatcher::class); self::assertSame(['https://www.example.com/en/articles/hello'], $dispatcher->urls()); ``` `IndexNowKit\Testing\RecordingDispatcher` records every batch handed over and sends nothing, which isolates "the entity hook fired and the transaction committed" from "the HTTP call worked". ## Simulating engine failures ```php $transport->willRespond( new IndexNowKit\Http\Response(429, '', 30), new IndexNowKit\Http\Response(200), ); $transport->willRespond(FakeTransport::failing('connection refused')); // a TransportException ``` Queued responses are consumed in order; anything beyond the queue gets the default `200`. Assert on `$result->reason`, `$result->retryable` and `$result->retryAfter` rather than on message text. For the key file, `$transport->onGet($url, $response)` answers a specific GET; unregistered ones return 404. ## Asserting on logs `IndexNowKit\Testing\ArrayLogger` is the simplest way to check that a rule was skipped for the reason you expect: ```php self::assertStringContainsString('rule "post_amp" skipped', implode("\n", $logger->messages('debug'))); ``` In a full kernel test, the Monolog test handler on the `indexnow` channel does the same job. ## Multi-domain tests Give each host its own key and `base_url`, turn `strict_hosts` on, and add a host-scoped route. Then assert on `$transport->posts[N]['body']['host']` and `['key']`: the client groups URLs by host, so a wrong per-host `base_url` shows up as URLs landing under the wrong key rather than as an exception. ## What not to test The protocol itself is covered by the core's conformance suite (C01–C22) and the bundle's own functional tests (A01–A14, H01–H06). Your suite should cover **your rules**: which entity changes produce which URLs, and that the guards match your publishing workflow. `bin/console indexnow:explain` is the interactive version of the same check. # Troubleshooting [Русская версия](troubleshooting.ru.md) Start here: ```bash bin/console indexnow:check # configuration, key files, engines, wiring bin/console indexnow:explain 'App\Entity\Post' 42 # one entity, the whole decision path, sends nothing ``` Between them they answer most reports without reading a single log line. ## Nothing is sent at all Work down this list; each item names the evidence that confirms it. | Cause | Evidence | |---|---| | `enabled: false` | `skipped` / `disabled`, and one `info` line per call | | `dry_run: true` | `skipped` / `dry_run`, an `info` line with the full request body | | an invalid configuration disabled IndexNow | one `critical` line at boot, and `indexnow:check` exits non-zero | | the entity has no matching rule | `indexnow:explain` says which rule was skipped and why | | the collector was never flushed | `warning: N collected URL(s) discarded: the unit of work ended without flush()` | | Doctrine hooks are not active | `indexnow:check` prints `doctrine: entity hooks are NOT active` | | Messenger is configured but no worker runs | the profiler panel shows URLs dispatched; nothing in the worker log | **Invalid configuration** is the sneaky one. An environment placeholder cannot be validated at compile time, so an empty `INDEXNOW_KEY` in production does not break the container — it logs ``` indexnow: invalid configuration, IndexNow is disabled until it is fixed: (run "bin/console indexnow:check") ``` at `critical` and runs disabled. Alert on that line, and put `indexnow:check` in your deployment smoke tests. ## 403 — the key file By far the most common failure, and it always means the same thing: the engine could not verify `https:///.txt`. ```bash bin/console indexnow:check --host www.example.com curl -i https://www.example.com/.txt ``` The response must be `200`, `text/plain`, body exactly the key, **no redirect**. Frequent causes: - the bundle routes were never imported (no Flex): add `@IndexNowKitBundle/config/routes.php` to `config/routes.yaml`; - `key_file.enabled: false` (or the deprecated `serve_key_file: false`) without a `key_location`; - a CDN or reverse proxy serving a cached old file after a key rotation — `key_file.cache_max_age` defaults to 300 seconds for exactly this reason; - a catch-all route or a firewall matching `/{key}.txt` first; - `http → https` or `www` redirects on the key file; - the request reaching a different host than the one the key belongs to, which the controller answers with 404 by design. The fifth consecutive 403 for a host is logged once at `critical` with *"submissions for this host are not being indexed"*. That is the line to page on. ## 422 — unprocessable The URLs do not belong to the host they were submitted under, or `keyLocation` is invalid. In practice: - a console command or worker generated URLs on the wrong host because a per-host `base_url` is missing — see [multi-domain.md](multi-domain.md); - a rule pinned a `host:` whose key is not configured; - `key_location` points at another host. The configuration layer rejects that, so this only appears with a custom `KeyProviderInterface`. `indexnow:explain` prints the host and key file for every URL it would send, which settles it immediately. ## 429 — rate limited The engine asked you to slow down. The result is `retryable` with `retryAfter` filled when the engine said so. - With Messenger, the transport retries and honours `Retry-After` on Symfony 7.2+. Nothing to do. - With `dispatch: sync`, nothing is retried: the batch is logged and dropped. Switch to Messenger if you see this regularly. - Lower `throttle.max_requests_per_minute` if your own traffic is causing it. Remember it counts per process. - A migration or import announcing tens of thousands of URLs at once is the usual trigger; batch it from a worker. ## A specific URL is never submitted ```bash bin/console indexnow:explain 'App\Entity\Post' 42 --event=updated ``` The output walks the rules in order and prints, per rule, the event subscription, the `when` result, the `fields` filter and the resolved URLs; then per URL the normalized form, the host, the masked key, the key file URL and the debounce state. Common findings: - **`when: false`** — the entity is not published, so the page does not exist. Correct behaviour. - **`fields` filter** — you changed a field the rule does not care about. Add it to `fields`, or drop `fields` to submit on any change. - **not subscribed** — the rule's `events` does not include this event. - **`debounced`** — sent within the last `debounce.per_url` seconds. `--force` on `indexnow:submit` bypasses it. - **`no key for host`** — the host is not in `hosts` and is not the `base_url` host. - **rule errors** — a missing accessor or a route that cannot be generated is printed in red, and logged at `error`. ## Publishing and unpublishing behave oddly `when` is evaluated before and after the change. `true → false` submits a deletion so engines recrawl the 404; `false → true` a creation. Two things to check: - **The backing field must be in the change set.** `when: 'isPublished'` is matched to the `published` field by convention (`isPublished` → `published` → `is_published`, `getStatus` → `status`). When the accessor name has nothing to do with the column, name the fields explicitly with `whenFields: ['status', 'visibleFrom']`. - **Deleting a draft submits nothing**, by design. The page was never public. ## The profiler panel `enabled`, `dry_run`, `dispatch` and `strict_hosts` at the top tell you the mode. Then: - **Collected** — URLs the request buffered. Zero here means no rule produced anything: the problem is upstream, in the rules or the guards. - **Results** — what was actually sent, per engine and host, with the HTTP code and the `reason`. Populated even for synchronous dispatch, which happens after the response. - With Messenger, the results table is empty by design and the panel says HTTP results appear in the worker log. - **Key files** — the URL each managed host must serve, with the key masked. Open one in a browser as a first check. - **base_url** — flagged when unset, because relative URLs and console or worker generation then fail. ## Logs Everything is on the `indexnow` Monolog channel and every message starts with `indexnow: `. ```yaml # config/packages/dev/monolog.yaml monolog: handlers: indexnow: type: stream path: '%kernel.logs_dir%/indexnow.log' level: debug channels: ['indexnow'] ``` Set it to `debug` while diagnosing: the reason a rule decided **not** to produce a URL is only logged at that level. Every message text and level is listed in the [operations guide](../core/operations.md). Keys are masked in every log line, including inside response bodies and exception messages, so these files are safe to attach to a bug report. ## Staging submitted its URLs | Symptom | Cause | Fix | |---|---|---| | Bing/Yandex report URLs of `staging.example.com`, or `failed` / `unprocessable` (422) for them in the log | the staging copy runs with the production key and no `dry_run`; its URLs were generated on its own host | outside production set `INDEXNOW_DRY_RUN=1` (or `INDEXNOW_ENABLED=0`); `check` fails on such a copy since core 0.6 | | the staging host serves the production key file | `key_file.enabled` is on everywhere | `key_file.enabled: false` outside production, so no engine can verify the key on that host | | the engines indexed staging pages | the staging host answered `200` for them and served the key | return `410` (or `noindex` + block in `robots.txt`) on staging, and rotate the key if it was exposed | | a preview environment must submit on purpose | — | say `dry_run: false` explicitly in that environment; `check` then warns instead of failing | ## Duplicates with `memory` and several workers | Symptom | Cause | Fix | |---|---|---| | the same URL is submitted by every worker within minutes | `debounce.store: memory` is per process; each web worker and queue worker keeps its own window | `debounce.store` = a shared cache; `check` warns about `memory` | | duplicates right after a cache outage | the store fails open: no deduplication while the cache is down | expected and bounded (one request per URL); watch the `debounce store unavailable` warning rate | | duplicates after a deploy | the shared cache was flushed, or `debounce.key_prefix` changed | harmless once; keep the prefix stable per application | ## Duplicate submissions after a cache outage The debounce store fails open. If Redis or your cache pool is unavailable, deduplication is skipped and a warning is logged (`debounce store unavailable, submitting without de-duplication`). The visible symptom is a burst of repeated submissions, not lost ones, which is the right trade: a duplicate costs one request, a miss leaves stale content indexed. `indexnow:check` reads a test key through the configured pool and prints `debounce: 600s per URL, shared through cache pool "cache.app" (RedisAdapter)`, or an error naming the pool when it is not usable; `debounce.store: memory` is flagged as per-process. ## `indexnow:sitemap` in a read-only container `SitemapReader` keeps each document in a temp file while parsing. With `readOnlyRootFilesystem` and no writable `/tmp`, the default `sitemap.spool: auto` falls back to memory and logs one warning per run; memory use is bounded by `sitemap.max_bytes` per document (50 MiB), so nothing breaks, it just costs RAM. Mount an `emptyDir` on `/tmp` (or set `TMPDIR` / `sitemap.spool_dir` to a writable volume) to get the temp file back, or set `spool: memory` to make the choice explicit. `indexnow:check` prints where documents are spooled and why the temp dir is unusable. A sitemap that ends midway ("ends early (truncated download or broken sitemap)") or a connection lost during the download is retried `sitemap.fetch_retries` times; when it still fails, the URLs read so far are submitted, the command exits with 1, and a re-run is safe (IndexNow is idempotent, the debounce store absorbs the repeats). For a daily cron with `--changed-since`, use a window wider than the interval (`"2 days"`) so a failed run leaves no gap. ## Getting help Include the output of `bin/console indexnow:check`, the relevant `indexnow` log lines, and the `bin/console indexnow:explain` output for one affected entity. Those three cover almost every question a maintainer would ask. # Backward compatibility `indexnowkit/laravel` follows SemVer. **Before 1.0, minor versions may contain breaking changes**; every one is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/laravel/CHANGELOG.md) with the migration. After 1.0 the rules below become the promise. The core's tiers ("call", "implement", "may grow") apply to every core class you touch through this package: [core bc.md](../core/bc.md). ## What the package keeps stable | Surface | Promise | |---|---| | **`config/indexnow.php` keys** ([configuration.md](configuration.md)) | Keys and their meaning stay; new keys are only added with a default. A published config file from an older minor keeps working; a rename ships the old key as deprecated for one minor and is listed in the changelog. | | **Environment variables** the config file reads (`INDEXNOW_KEY`, `INDEXNOW_BASE_URL`, `INDEXNOW_DRY_RUN`, …) | Names stay; new ones are only added. | | **Artisan commands and options** (`indexnow:check`, `indexnow:key:generate`, `indexnow:submit`, `indexnow:submit-model`, `indexnow:explain`, `indexnow:sitemap`, `indexnow:history`, `indexnow:status`) | Names, arguments and options come from `Console\Definitions` of `indexnowkit/console` (and the sitemap and history packages); new options are only added. The `model` argument of `submit-model` and `explain` stays. Output is not a contract except the exit codes and the `--json` shape of the core formatter. The classes are the packages' since 0.15.0 — `IndexNowKit\Console\Command\*`, `Sitemap\Console\SitemapCommand`, `History\Console\{History,Status}Command` — each bound in the container under its class name ([extending.md](extending.md) "Replacing a command"). | | **Container bindings** listed in [extending.md](extending.md) | Each abstract stays bound to an implementation of the same interface; rebinding and decorating stay possible. | | **Facade** `Facades\IndexNowKit` and **`IndexNowManager`** (`kit()`, `rules()`, `observe()`, `submit()`, `submitModel()`, `submitModels()`, `urlsFor()`, `urlsForAll()`, `explain()`, `collect()`, `flush()`) | Method names and parameter names stay; new parameters are appended with defaults. | | **Trait `Eloquent\IndexNowable`**, **`Eloquent\IndexNowObserver`**, **`Eloquent\RouteBindingFieldsInterface`** | The trait stays a drop-in; the observer's public hooks keep their names; the interface gets no new method without a major. | | **Queue job** `Queue\SubmitUrlsJob` | Its constructor and the serialized shape stay (new parameters are appended with a default), so jobs queued before an upgrade still run after it. | | **Route** `indexnow.key_file` (or `key_file.route_name`, `{key}.txt`) | Name and shape stay. | | **Check classes** `Check\QueueCheck`, `Check\EloquentCheck`, `Check\CacheStoreProbe` and the `indexnowkit.check` tag | Names stay; adding a tagged `CheckInterface` keeps working. A tagged service of another type is a `ConfigurationException`. The `router.locales` line is the core's `Check\LocalesCheck` since 0.15.0 (`Check\RouterCheck` of this package is gone: same code, same texts). | | **Exceptions** | Configuration mistakes are `IndexNowKit\Exception\ConfigurationException`, a bad command argument is `IndexNowKit\Exception\InvalidArgumentException` — both from the core, both `IndexNowException`. | Not a contract: log message texts (their `context` keys are), the exact wording the commands print (exit codes and levels are), and the service provider's private methods. ## Pinning `composer require indexnowkit/laravel:^0.9` gets every 0.9.x patch. Read the changelog before a minor. # Configuration `config/indexnow.php` (publish it with `php artisan vendor:publish --tag=indexnow-config`; without the file every key below uses its default). Keys shared with every indexnowkit adapter are described in the [core configuration reference](../core/configuration.md); this page lists all of them briefly and the Laravel-only blocks in full. Values come from `env()`, so they are validated at runtime: an invalid value never throws from a save or from `terminating`. It is logged once at `critical` (`indexnow: invalid configuration, IndexNow is disabled until it is fixed: ...`), IndexNow runs disabled, and `php artisan indexnow:check` prints the exact error. Unknown keys (typos) are logged at `warning`. ## Core keys | Key | Default | Meaning | |---|---|---| | `enabled` | `INDEXNOW_ENABLED`, `true` | kill switch; `false` = nothing is sent, changes are logged at debug | | `key` | `INDEXNOW_KEY` | the IndexNow key, `[A-Za-z0-9-]{8,128}` | | `previous_key` | `INDEXNOW_PREVIOUS_KEY` | the key before a rotation: its file is still served, nothing is submitted under it | | `key_location` | `INDEXNOW_KEY_LOCATION` | full URL of the key file when it is not `https:///.txt` | | `base_url` | `INDEXNOW_BASE_URL`, then `APP_URL` | origin for URLs generated outside HTTP requests (artisan, workers); required with `dispatch: queue` | | `hosts` | `[]` | `host => key` or `host => ['key', 'key_location', 'base_url', 'engines', 'previous_key']` | | `strict_hosts` | `INDEXNOW_STRICT_HOSTS`, `false` | skip URLs of hosts outside `base_url` / `hosts` instead of sending them under the default key | | `environment` | `app()->environment()` | feeds the non-production dry-run safety net | | `production_environments` | `['prod', 'production']` | environments where a missing key is an error, not dry-run | | `max_url_length` | `2048` | longer URLs are `invalid_url` | | `engines` | `INDEXNOW_ENGINES`, `api` | `api`, `yandex`, `bing`, `naver`, `seznam`, `yep`, `internetarchive`, `amazon`, an endpoint URL or an alias | | `engine_aliases` | `[]` | `alias => endpoint URL` | | `locale_hosts` | `[]` | `locale => host` for rules with `locales` and no `host` | | `dispatch` | `INDEXNOW_DISPATCH`, `queue` | `queue`, `sync`, `none` (see below) | | `dry_run` | `INDEXNOW_DRY_RUN`, `false` | log the request instead of sending it | | `batch.max_urls` | `10000` | URLs per request (protocol maximum) | | `debounce.per_url` | `600` | seconds a URL is not resubmitted; `0` = off | | `debounce.key_prefix` | `indexnowkit_` | cache key prefix of the shared window | | `throttle.max_requests_per_minute` | `60` | token bucket per process | | `http.timeout` | `10` | seconds | | `http.user_agent` | `null` | override the `indexnowkit-php/x.y.z` agent | | `retry.*` | `3 / 60 / 2.0 / 3600 / 5` | `max_attempts`, `base_delay`, `multiplier`, `max_delay`, `server_error_delay` of `SubmitUrlsJob` | | `resolver.max_via_depth` / `max_via_fanout` | `3` / `100` | limits of `via:` | | `collector.max_urls` / `detect_leaks` | `0` / `true` | early flush threshold; warn at shutdown about unflushed URLs | | `logging.max_urls` / `forbidden_escalation` / `max_body` / `levels` | `20` / `5` / `300` / `[]` | log line shaping; see the core operations guide | ## Laravel keys ### `dispatch` | Value | Behaviour | |---|---| | `queue` (default) | each flushed batch becomes a `SubmitUrlsJob` on `queue.connection` / `queue.queue`, delayed by `queue.delay` seconds; retries follow `retry.*`. `QUEUE_CONNECTION=sync` runs it inline. | | `sync` | the batch is sent from `app()->terminating()`, after the response; 429/5xx are not retried | | `none` | URLs are collected but never sent (drain `IndexNowKit::kit()->collector` yourself) | ```php 'queue' => [ 'connection' => env('INDEXNOW_QUEUE_CONNECTION'), // null = queue.default 'queue' => env('INDEXNOW_QUEUE'), // null = the connection's default queue 'delay' => 0, // seconds before the first attempt ], ``` Details, Horizon and failure handling: [queue.md](queue.md). ### `debounce.store` | Value | Store | |---|---| | `cache` (default) | `Cache::store()` — the default cache store, shared by web requests and workers | | a store name (`redis`, `array`) | that store | | `memory` | per process; fine for CLI scripts and tests, warned about by `indexnow:check` | | `none` | no debounce | The window is best effort and fails open: an unusable store logs a warning and the URLs are still sent. ### `http.client` `null` discovers a PSR-18 client (Guzzle in a default Laravel application, otherwise php-http/discovery). A class name or container binding of a `Psr\Http\Client\ClientInterface` uses that client (proxy, extra headers, retries of your own). The transport is built on first use, so a request that submits nothing never touches it. ### `key_file` ```php 'key_file' => [ 'enabled' => true, // false = the route is not registered (serve the file yourself or set key_location) 'path' => '/{key}.txt', // must contain {key} 'host' => null, // restrict the route to a host (Route::domain()) 'cache_max_age' => 300, // Cache-Control max-age; short on purpose: a cached old file means 403 after a rotation 'route_name' => 'indexnow.key_file', 'middleware' => [], // no "web" group by default: no session, no CSRF, no cookies ], ``` The controller serves only a key that belongs to the requested host (404 otherwise) and adds `Vary: Host` when a `hosts` map is configured. `serve_key_file` (core name) is accepted as an alias of `key_file.enabled`. The route is compatible with `route:cache`. ### `router` ```php 'router' => [ 'locales' => [], // locales generated for rules with locales: 'all' (empty = current only) 'locale_parameter' => 'locale',// added to the route parameters only when the route declares {locale} 'set_app_locale' => true, // App::setLocale() while generating a locale's URL (localized slugs), restored after ], ``` Inside an HTTP request URLs keep the host Laravel generated them on; in the console (artisan, queue workers) they are rebased onto `base_url`; a rule with `host:` uses `hosts..base_url`, else `https://`. Routes with their own `Route::domain()` keep it. Details: [multi-domain.md](multi-domain.md). ### `eloquent` `eloquent.enabled: false` keeps the observer registered but inert: models using `IndexNowable` submit nothing, manual submission still works. `indexnow:check` says so. ### `logging.channel` Log channel name (`config/logging.php`); `null` = the default channel. Every service of the package logs there, with messages prefixed `indexnow:`. ### `sitemap` Needs `indexnowkit/sitemap` (`composer require indexnowkit/sitemap`); without the package the block is ignored and `indexnow:check` says so. ```php 'sitemap' => [ 'enabled' => true, // false = no indexnow:sitemap command 'url' => null, // default argument (null = /sitemap.xml) 'max_depth' => 3, // nested sitemap indexes 'max_sitemaps' => 1000, 'max_bytes' => 52428800, // per document 'allow_foreign_hosts' => false,// follow index entries on another origin (CDN) 'spool' => 'auto', // auto | disk | memory: where a document is kept while parsing 'spool_dir' => null, // temp directory (null = sys_get_temp_dir()) 'fetch_retries' => 2, // after a network failure or 5xx ], ``` Details: [sitemap.md](sitemap.md). ### `history` Needs `indexnowkit/history` (`composer require indexnowkit/history`); without the package the block is ignored and `indexnow:check` says so. Nothing is kept until `store` is set. ```php 'history' => [ 'store' => env('INDEXNOW_HISTORY_STORE'), // null | psr16 (ring buffer in the debounce.store cache) | pdo (a table) 'limit' => 500, // records the psr16 store keeps 'key_prefix' => null, // cache key prefix of the psr16 store (null = debounce.key_prefix) 'pdo' => [ 'dsn' => env('INDEXNOW_HISTORY_PDO_DSN'), // a PDO DSN when the table is not on a Laravel connection; not both dsn and service 'service' => null, // the config/database.php connection holding the table (null = default) 'table' => 'indexnow_submissions', // [A-Za-z_][A-Za-z0-9_]*; the package's docs/migrations.md creates it ], 'retention_days' => 90, // what indexnow:history --purge removes beyond ], ``` `psr16` uses the cache store of `debounce.store` (the default store with `memory`/`none`); `pdo` takes `DB::connection(service)->getPdo()` or a PDO built from `dsn`. The table is never created by the package. ## Environment variables `INDEXNOW_KEY`, `INDEXNOW_PREVIOUS_KEY`, `INDEXNOW_KEY_LOCATION`, `INDEXNOW_BASE_URL`, `INDEXNOW_ENABLED`, `INDEXNOW_STRICT_HOSTS`, `INDEXNOW_ENGINES` (comma-separated), `INDEXNOW_DISPATCH`, `INDEXNOW_DRY_RUN`, `INDEXNOW_QUEUE_CONNECTION`, `INDEXNOW_QUEUE`, `INDEXNOW_DEBOUNCE_STORE`, `INDEXNOW_LOG_CHANNEL`, `INDEXNOW_VERIFY`, `INDEXNOW_HISTORY_STORE`, `INDEXNOW_HISTORY_PDO_DSN` are read by the shipped config file. Anything else goes through your published copy. ## Startup checks `php artisan indexnow:check` validates the configuration (exact error on failure), fetches every key file over HTTP, and prints the Laravel wiring: the queue connection `SubmitUrlsJob` goes to (or that `sync` retries nothing), whether the debounce cache store is usable, where sitemap documents are spooled, whether observers are active, and the locales `locales: 'all'` expands to (`router.locales`; a rule asking for every locale with an empty list is one warning). `--live` adds a real probe request per engine. # Extending The service provider registers one container binding per core interface. Replace any of them with `$this->app->bind()` / `singleton()` / `extend()` in your own provider (register it after the package, or bind in `register()` — the package resolves everything lazily). ## Bindings | Abstract | Default | Replace it to | |---|---|---| | `IndexNowKit\Config` | `ConfigFactory::create(config('indexnow'))` | — (edit the config) | | `IndexNowKit\Key\KeyProviderInterface` | `StaticKeyProvider::fromConfig()` | keys from a database (multi-tenant); honour `$host` in `isKnownKey()` | | `IndexNowKit\Http\TransportInterface` | `LazyTransport` over a discovered PSR-18 client | your own HTTP stack (or set `http.client`) | | `IndexNowKit\Url\UrlNormalizerInterface` | `UrlNormalizer` | canonical form: strip tracking parameters, trailing-slash policy | | `IndexNowKit\Throttle\ThrottleInterface` | `TokenBucket` | a shared rate limiter | | `IndexNowKit\ClientInterface` | `Client` | — | | `IndexNowKit\Debounce\DebounceStoreInterface` | by `debounce.store` | another window store | | `IndexNowKit\SubmitterInterface` | `Submitter` | wrap with `RetryingSubmitter`, add listeners for metrics | | `IndexNowKit\Collector\CollectorInterface` (scoped) | `Collector` | a durable outbox, a per-tenant buffer | | `IndexNowKit\Attribute\AttributeReaderInterface` / `RuleRegistry` | `RuleRegistry` over `AttributeReader` | your own rule source | | `IndexNowKit\Url\RouteUrlResolverInterface` / `LaravelRouteUrlResolver` | the router bridge | another URL scheme | | `IndexNowKit\Url\ResolverLocatorInterface` | `ArrayResolverLocator` over the container | — | | `IndexNowKit\Url\UrlResolverInterface` | `AttributeUrlResolver` | replace the whole "object → URLs" step | | `IndexNowKit\Url\GuardedUrlResolver` | over the URL resolver | — | | `IndexNowKit\Url\ObjectChangeHandler` | over the rules, the resolver and the extractor | what an insert / update / delete means for URLs; this is what the Eloquent hooks resolve through | | `Psr\Clock\ClockInterface` | `IndexNowKit\Clock\SystemClock` | the time the throttle, the debounce window and the submission records read (`IndexNowKit\Testing\FrozenClock` in tests) | | `IndexNowKit\Dispatch\DispatcherInterface` | by `dispatch` | another delivery (an outbox table, a bus) | | `IndexNowKit\IndexNowKit` | the core facade | — | | `IndexNowKit\Key\KeyFileResponder` | over the key provider | — | | `IndexNowKit\Check\CheckerInterface` | `Checker` with the tagged checks | — | | `IndexNowKit\Sitemap\SitemapSourceInterface` | `SitemapReader` | filter, rewrite or replace the sitemap source (bound only with `indexnowkit/sitemap` installed) | | `IndexNowKit\Laravel\IndexNowManager` | facade root | — | | `IndexNowKit\Laravel\Eloquent\IndexNowObserver` (singleton) | the observer | — | | `IndexNowKit\Laravel\Eloquent\RouteBindingFieldsInterface` | `LaravelRouteUrlResolver` | which model field a `{post:slug}` parameter binds to (drives rename detection) when routes come from elsewhere | | `IndexNowKit\Console\SubjectLoaderInterface` | `Console\ModelLoader` | model lookup (tenant scoping, another id format) | | `IndexNowKit\Console\ResultFormatterInterface` | `ResultRenderer` | command output (your JSON envelope or table style) | | `IndexNowKit\Adapter\SubmitterFactoryInterface` | `SubmitterFactory` | what `--force` / `--dry-run` submit through | | `IndexNowKit\Console\Vocabulary`, `Console\*Runner` | Laravel words; the command bodies | reuse a runner from your own command (a tenant loop over `SubmitSubjectsRunner`) | | `IndexNowKit\Console\ConfigSourceInterface` | `Console\ConfigSource` | what `indexnow:check` and `indexnow:config` read (the config repository, its strict build, the package blocks) | | `IndexNowKit\Console\Command\*`, `Sitemap\Console\SitemapCommand`, `History\Console\{History,Status}Command` | the command classes of the packages, one binding each | replace one command ("Replacing a command" below) | | `indexnowkit.logger` | `Log::channel(logging.channel)` | a PSR-3 logger of your own (tests: `ArrayLogger`) | ## Custom resolvers ```php #[IndexNow(resolver: ProductUrlResolver::class)] class Product extends Model { use IndexNowable; } final class ProductUrlResolver implements UrlResolverInterface { public function __construct(private readonly UrlGenerator $urls) {} // constructor dependencies are injected public function resolve(object $subject, Event $event): array { return [$this->urls->route('products.show', ['product' => $subject], true)]; } } ``` The class is built by the container; a binding id works the same (`resolver: 'shop.product_urls'`). An unknown id is a logged configuration error, not an exception. ## Rules registered at runtime ```php IndexNowKit::observe(Vendor\Package\Post::class, [new IndexNow(route: 'posts.show', params: ['post' => 'self'])], new IndexNowDefaults(when: 'published')); IndexNowKit::observe(Vendor\Package\Page::class); // the class carries its own attributes, only the observer is missing IndexNowKit::rules()->registerFor(Node::class, fn (Node $node): ?RuleSet => $node->type === 'page' ? RuleCompiler::compile(...) : null); ``` ## Extra checks Anything implementing `IndexNowKit\Check\CheckInterface` and tagged `indexnowkit.check` is printed by `indexnow:check`: ```php $this->app->singleton(CdnPurgeCheck::class); $this->app->tag([CdnPurgeCheck::class], IndexNowKitServiceProvider::CHECK_TAG); ``` Add lines to the report; never throw — a failing check is an error line. A tagged service that is not a `CheckInterface` is a `ConfigurationException` naming the tag when the checker is built, not a fatal in the middle of `indexnow:check`. ## Submission results Every `Result` (engine, host, status, reason, HTTP code, URL count) is dispatched as a Laravel event, the object itself: `Event::listen(Result::class, fn (Result $result) => ...)` receives it, Telescope's events watcher lists it, `Event::fake()` catches it in tests. The submitter and the command submitters (`--force`, `--dry-run`, the sitemap command) publish through the same PSR-14 bridge (`IndexNowKitServiceProvider::EVENTS`). The lower-level `IndexNowKit::kit()->submitter->addListener(fn (Result $result) => ...)` still works and fires first. `php artisan about` has an `IndexNow` section: core version, enabled/dry-run, environment, base URL, the masked key, engines, dispatch, debounce — the first things a support request needs. `php artisan indexnow:config --json` is the full effective configuration. ## Reading model attributes `#[IndexNow]` accessors on Eloquent models go through `Eloquent\EloquentSubjectReader`, the reader of the `IndexNowKit\Attribute\ParamExtractor` the provider binds (`ParamExtractor::class`, shared by the resolver, the change handler and `indexnow:explain`). It claims attributes, casts, accessors and relations (methods with a declared `Relation` return type, or already loaded); anything else falls to the core DSL (methods, properties). An accessor that matches nothing is a `ConfigurationException` logged at `error`, not a silent `null`. Objects neither can see into (a CMS record behind `get_field()`) get a reader of their own: ```php $this->app->extend(ParamExtractor::class, fn(ParamExtractor $extractor) => $extractor->with(new CmsFieldReader())); ``` ## What is the core's `IndexNowObserver` keeps only what is Eloquent's (the change set from `getChanges()`/`getOriginal()`, the previous state from `getRawOriginal()`, `Connection::afterCommit()`); guarding, logging and the URLs of a row about to be deleted are the core's `Hook\ObserverHelper` (`forChanges()`). The hooks read the `ObjectChangeHandler` binding and nothing else, so a `save()` that announces no URL never builds the submitter, the client, the transport, the throttle or the debounce store; the facade is made in the sink, once there are URLs to collect. `IndexNowObserver::forKit()` builds one over an existing facade. `SubmitUrlsJob` is `Retry\WorkerOutcome` plus `release()`/`fail()` with the delay the `RetryPolicy` computes, and it carries the attempts the batch already spent (`spentAttempts`) so `retry.max_attempts` bounds the batch and not each re-queued job. The artisan commands are the command classes of `indexnowkit/console`, `indexnowkit/sitemap` and `indexnowkit/history` themselves — artisan runs any symfony/console command — so `php artisan indexnow:submit-model --help` is byte for byte what the bundle and Yii3 print. A custom command over a core runner extends `Symfony\Component\Console\Command\Command` and calls `Definitions::check()->applyTo($this)` in `configure()`, or extends `Illuminate\Console\Command` and declares its own `$signature`. ## Replacing a command Every command is a container binding under its class name, so the container's own tools apply: ```php // decorate: the same name, your behaviour around the package's $this->app->extend(\IndexNowKit\Console\Command\CheckCommand::class, fn ($command, $app) => new MyCheckCommand($command)); // or register a command of your own with the same name after the package: the later registration wins in artisan $this->commands([MyCheckCommand::class]); ``` `indexnow:submit-model` is registered through a `Symfony\Component\Console\Command\LazyCommand` (its name comes from the vocabulary, not from `#[AsCommand]`): replacing the `SubmitSubjectsCommand` binding is enough, the lazy wrapper resolves it on first run. # Multiple domains and locales ## One application, several hosts Every host gets its own key (engines verify `https:///.txt` on the submitted host): ```php 'key' => env('INDEXNOW_KEY'), // www.example.com, the base_url host 'base_url' => 'https://www.example.com', 'hosts' => [ 'example.de' => env('INDEXNOW_KEY_DE'), 'shop.example.com' => [ 'key' => env('INDEXNOW_KEY_SHOP'), 'base_url' => 'https://shop.example.com', // origin for this host's URLs outside requests 'engines' => ['yandex', 'bing'], // per-host engine list ], ], 'strict_hosts' => true, // hosts not listed are skipped, not sent under the default key ``` The key file route serves each host's own key only (a request for `example.de`'s key on `www.example.com` is 404) and answers with `Vary: Host`, so a shared CDN never caches one host's file for another. `indexnow:check` fetches every host's key file; `--host=example.de` limits it to one. ## Rules on another host ```php #[IndexNow(route: 'products.show', params: ['product' => 'self'], host: 'shop.example.com')] ``` `host` can also be an accessor (`host: 'tenant.domain'`) for multi-tenant models. The URL is generated through the route and rebased onto `hosts..base_url`, else `https://`. A route with its own `Route::domain()` keeps that domain. ## Locales ```php 'router' => ['locales' => ['en', 'de'], 'locale_parameter' => 'locale', 'set_app_locale' => true], 'locale_hosts' => ['de' => 'example.de'], // optional: one host per locale ``` ```php #[IndexNow(route: 'articles.show', params: ['article' => 'self'], locales: 'all')] ``` - `locales: 'current'` (default) generates one URL; `'all'` one per `router.locales`; a list as given. `'all'` with an empty `router.locales` falls back to the current locale alone — one warning in the log and one `indexnow:check` line (`router.locales`), not silence. - The locale is added as the `router.locale_parameter` route parameter **only when the route declares it** (`/{locale}/articles/{article}`), so no query string sneaks in. With `set_app_locale`, `App::setLocale()` is switched for the duration of the generation and restored, which is what packages with localized slugs read. - With `locale_hosts`, a rule without `host` generates each locale on that locale's host and under that host's key. ## Origin of generated URLs | Context | Origin | |---|---| | HTTP request | whatever Laravel generated (the request host) | | artisan, queue worker | `base_url` (Laravel's console request is `APP_URL`; the package rebases scheme, host and port) | | rule with `host:` | `hosts..base_url`, else `https://` | | route with `Route::domain()` | the route's domain, always | A staging copy reached under another hostname would otherwise submit its URLs under the production key; that is what `strict_hosts: true` prevents, and why `indexnow:check` warns when it is off in production. ## www and apex `example.com` and `www.example.com` are two hosts to IndexNow: each needs its own key file, and a URL submitted under the other one's key answers 422. Pick the canonical one (the one your pages link to and `` names), put it in `base_url`, redirect the other with `301`, and do not list it in `hosts` — listing both would announce two copies of every page. With `strict_hosts: true` a request that reached the application under the non-canonical name submits nothing instead of announcing duplicates. ## hreflang clusters Localized pages that point at each other with `hreflang` are one cluster to the engines: when one changes, announce the cluster. A rule with `locales: 'all'` does that for the locales of one model; for locales living on other hosts `locale_hosts` sends each locale to its host under that host's key. When translations are separate objects, `via:` walks to them: ```php #[IndexNow(route: 'article_show', params: ['slug' => 'slug'], locales: 'all')] // every locale of this article #[IndexNow(via: 'translations')] // or: the sibling objects' own rules ``` # Queue, retries, failures `dispatch: queue` is the default. Every batch the collector flushes (end of a request, artisan command or handled job) becomes one `IndexNowKit\Laravel\Queue\SubmitUrlsJob` carrying the normalized URLs. ```php 'dispatch' => 'queue', 'queue' => ['connection' => null, 'queue' => null, 'delay' => 0], 'retry' => ['max_attempts' => 3, 'base_delay' => 60, 'multiplier' => 2.0, 'max_delay' => 3600, 'server_error_delay' => 5], ``` ## What the worker does 1. `SubmitterInterface::submit($urls)`: debounce, group by host, split by `batch.max_urls`, one request per engine. 2. Retryable outcomes (429, 5xx, network failure) → `release($delay)`. The delay is `Retry-After` when the engine sent one, else `base_delay × multiplier^(attempt-1)` for 429 and `server_error_delay × multiplier^(attempt-1)` for 5xx/network, capped at `max_delay`. After `max_attempts` the job **fails** with an error naming the count. 3. Final rejections (400, 403, 422) → the job **fails** immediately with the engine answer in the message and a hint to run `indexnow:check`. 403 means the key file is not reachable under `https:///.txt`. 4. Anything else finishes the job. URLs that were accepted are recorded in the debounce store, so a released job only resends what was rejected. A batch that was accepted in part comes back as a *new* job with the rejected URLs only (`release()` would replay the whole payload). `max_attempts` bounds the batch, not the queue message: the attempts already spent travel with the new job in `$job->spentAttempts`, `$job->attempt()` is the batch-wide attempt number, and `$job->tries` is what is left of `retry.max_attempts`. `backoff()` continues the same curve for the case the job throws instead of releasing. ## Why failures are failures A 403 is not transient. Recording it in `failed_jobs` is the one place ops already watch; retrying it would hide the broken key file. `php artisan queue:retry ` after fixing the key file re-sends the batch (the URLs are in the payload). ## Horizon, workers, Octane - The job is a plain `ShouldQueue` job: Horizon shows it under its connection and queue, tags are the default ones. - Long-running workers flush the collector after every handled job (`JobProcessed`), so a job that saves models submits their URLs in a follow-up batch on the same connection. - Under Octane the collector is a scoped binding (reset per request) and `terminating` runs per request. ## Synchronous delivery `dispatch: sync` sends from `app()->terminating()`, after the response was written. No retries: a 429 is logged and the URLs are lost until the next change. `QUEUE_CONNECTION=sync` with `dispatch: queue` runs the job inline at flush time; a `release()` on the sync driver retries nothing, so it is a development setting. ## Checking the wiring `php artisan indexnow:check` prints the connection and queue the job goes to, warns when the connection driver is `sync`, and fails when the connection does not exist in `config/queue.php`. # Sitemaps ```bash composer require indexnowkit/sitemap # optional: the indexnow:sitemap command ``` The reader lives in the optional [`indexnowkit/sitemap`](../sitemap/index.md) package. Without it `indexnow:sitemap` exists as a stub that prints `indexnowkit/sitemap is not installed: composer require indexnowkit/sitemap` and exits 1 (a scheduled run gets a sentence, not "command not found"), `indexnow:check` prints `sitemap: not installed (…)`, the `sitemap` block of the config is ignored, and nothing else changes or is logged. `php artisan indexnow:sitemap` submits every URL of a sitemap, a sitemap index, a text sitemap or a gzipped one, reading it as a stream and sending every `batch.max_urls` URLs while it is still being read. A million-URL index never has to fit in memory. ```bash php artisan indexnow:sitemap # sitemap.url, else /sitemap.xml php artisan indexnow:sitemap https://www.example.com/news.xml php artisan indexnow:sitemap storage/app/sitemap.xml # local file, no web server involved php artisan indexnow:sitemap --changed-since="1 day" # only newer than that (entries without lastmod are skipped) php artisan indexnow:sitemap --dry-run # list, send nothing php artisan indexnow:sitemap --json # one row per engine/host/status with url_count and batches ``` Schedule it once a day as a safety net for changes that bypassed the hooks: ```php // routes/console.php Schedule::command('indexnow:sitemap --changed-since="1 day"')->daily(); ``` ## Options | Option | | |---|---| | `--changed-since=` | `"1 day"`, `"2 hours"`, `"2026-09-01"` | | `--allow-foreign-hosts` | follow index entries hosted on another origin (a CDN) for this run; `sitemap.allow_foreign_hosts` makes it permanent | | `-f, --force` | ignore the debounce window | | `--dry-run` | print the entries instead of submitting | | `--json` | machine-readable summary; with `--dry-run` a JSON array of URLs | When the sitemap breaks midway (network, 5xx after `fetch_retries`), the batch read so far is still submitted, the command exits 1 and says how much went out; the re-run is idempotent thanks to the debounce window. ## Configuration ```php 'sitemap' => [ 'enabled' => true, 'url' => null, 'max_depth' => 3, 'max_sitemaps' => 1000, 'max_bytes' => 52428800, 'allow_foreign_hosts' => false, 'spool' => 'auto', 'spool_dir' => null, 'fetch_retries' => 2, ], ``` `spool` decides where a document is kept while parsing: `auto` uses a temp file and falls back to memory when the temp dir is not writable (read-only containers), `disk` fails instead, `memory` never touches the disk (bounded by `max_bytes`). `indexnow:check` reports the location, or why the temp dir is unusable. ## Your own source The command depends on `IndexNowKit\Sitemap\SitemapSourceInterface`. Bind another implementation to read from a database, filter entries or rewrite URLs: ```php $this->app->singleton(SitemapSourceInterface::class, fn ($app) => new FilteringSource($app->make(SitemapReader::class))); ``` `--allow-foreign-hosts` only reaches the shipped `SitemapReader`; a custom source decides on its own. `spatie/laravel-sitemap` users: point `sitemap.url` at the generated file (`public/sitemap.xml`) or pass the path as the argument; the package reads it locally. The reader behind the command is the [`indexnowkit/sitemap`](../sitemap/index.md) package (required by this one); its README documents the `sitemap` block and the safety rules. # Testing your integration The core ships test doubles in `IndexNowKit\Testing` (part of the package, not dev-only): `FakeTransport`, `ArrayLogger`, `FrozenClock`, `RecordingDispatcher`. Bind them into the container before the first submission. ```php use IndexNowKit\Http\TransportInterface; use IndexNowKit\Laravel\IndexNowKitServiceProvider; use IndexNowKit\Testing\{ArrayLogger, FakeTransport}; final class PostPublishingTest extends TestCase { private FakeTransport $transport; protected function setUp(): void { parent::setUp(); $this->app->instance(TransportInterface::class, $this->transport = new FakeTransport()); $this->app->instance(IndexNowKitServiceProvider::LOGGER, new ArrayLogger()); config(['indexnow.dispatch' => 'sync', 'indexnow.debounce.store' => 'memory', 'indexnow.debounce.per_url' => 0]); } public function test_publishing_a_post_notifies_the_engines(): void { $this->post('/posts', ['title' => 'Hello'])->assertCreated(); // the batch leaves on terminate, inside ->post() self::assertSame(['https://www.example.com/posts/hello'], $this->transport->posts[0]['body']['urlList']); self::assertSame('www.example.com', $this->transport->posts[0]['body']['host']); } } ``` Every entry of `$transport->posts` is `['url' => ..., 'json' => ..., 'headers' => ..., 'body' => ...]`, so you assert on `host`, `key`, `keyLocation` and `urlList` directly. `FakeTransport::onGet($url, $response)` stubs key file fetches for `indexnow:check`; `willRespond(new Response(429, '', 30))` queues engine answers. ## Time `$this->app->instance(ClockInterface::class, $clock = new FrozenClock('2026-09-07 08:30:00'))` fixes the time the throttle, the debounce window and the submission records of `indexnowkit/history` read. `$clock->advance(601)` walks past `debounce.per_url` without sleeping. ## Without HTTP at all `IndexNowKit::urlsFor($post)` and `IndexNowKit::explain($post)` resolve the rules of one model without sending; `ObjectChangeHandler::updatedEvents()` (via `IndexNowKit::kit()->changes()`) tells which rule classified an update as created / updated / deleted before any URL exists. ## Queue `Queue::fake()` then `Queue::assertPushed(SubmitUrlsJob::class, fn ($job) => $job->urls === [...])`. To run the worker logic in a test, call `$job->handle($submitter, $logger)` with a mocked `Illuminate\Contracts\Queue\Job` set through `$job->setJob($mock)`; `release()` and `fail()` land on the mock. ## Transactions in tests `RefreshDatabase` / `DatabaseTransactions` wrap each test in a transaction and swap Laravel's transaction manager for the testing one, which runs `afterCommit` callbacks at the test's wrapping level. Submissions therefore behave as in production: they leave when your code's own transaction commits. ## dry_run Outside `production_environments` a missing `INDEXNOW_KEY` switches `dry_run` on: the whole pipeline runs (normalization, deduplication, grouping, key lookup) and stops before the POST, with the body in the `info` log. Set `INDEXNOW_KEY` and `INDEXNOW_DRY_RUN=false` in `phpunit.xml` to exercise the transport in tests. ## Conformance The package's own suite runs the core conformance kits (`IndexNowKit\Testing\Conformance\CoreConformanceTestCase`, `OrmConformanceTestCase`) against the container on `orchestra/testbench`; `tests/Conformance/*` shows the driver an adapter implements. # Troubleshooting [Русская версия](troubleshooting.ru.md) Start with `php artisan indexnow:check`, then `php artisan indexnow:explain "App\Models\Post" `, then the log channel at `debug`. ## Nothing is sent | Symptom | Cause | Fix | |---|---|---| | `check`: `configuration: ...` and exit 1 | an `env()` value is invalid; IndexNow runs disabled | fix the value; the exact error is printed | | `explain`: `when: published -> false` right after `create()` | the `when` attribute only has a database default, so the fresh model does not have it | `protected $attributes = ['published' => false]` on the model, or set it explicitly | | `explain`: `no #[IndexNow] rule` | the model has no attribute and was not registered | add the attribute, or `IndexNowKit::observe()` | | URLs resolved but no POST | `dispatch: queue` and no worker | `php artisan queue:work`, or `dispatch: sync` | | log: `rule "..." ignores this update (fields ...)` | `fields` filter did not match the changed attributes | add the attribute to `fields`, or drop the filter | | log: `Cannot generate route "posts.show": Missing required parameter` | the rule's `params` do not match the route | `params: ['post' => 'self']` for route model binding, or name every parameter | | log: `Cannot read "foo" on App\Models\Post: no method foo(), getFoo(), isFoo() or hasFoo(), no property "foo"…` | typo in an accessor | fix the accessor; attributes, casts, accessors, relations and methods are all valid | | a mass `update()` changed nothing in the index | bulk statements fire no events (A13) | `IndexNowKit::submitModels($query->get())` or `indexnow:submit-model` | | `attach()` on a pivot changed nothing | pivot operations fire no owner events | `$touches = ['posts']` on the related model, rule without `fields` filter | ## Sent, but the engine answers | Answer | Meaning | Fix | |---|---|---| | 403 (`invalid_key`, job failed) | `https:///.txt` is not reachable or has another body | `indexnow:check`; a CDN may cache the old file (`key_file.cache_max_age`) | | 422 (`unprocessable`) | URLs of another host than `host`, or key file on another host | one key per host (`hosts`), `strict_hosts: true` | | 429 (`rate_limited`) | too many requests | the job releases with `Retry-After`; lower `throttle.max_requests_per_minute` | | 202 (`pending`) | accepted, key verification pending | normal for a new key; `check --live` later answers 200 | ## Duplicates, timing - The same URL is not resubmitted within `debounce.per_url` (600 s). `--force` bypasses it; `debounce.store: cache` shares the window between requests and workers, `memory` does not. - Everything from one request leaves as one batch on `terminating`; a job saving models flushes after the job. - A rolled-back transaction submits nothing; a savepoint rollback inside `DB::transaction()` drops only the inner URLs. ## Staging submitted its URLs | Symptom | Cause | Fix | |---|---|---| | Bing/Yandex report URLs of `staging.example.com`, or `failed` / `unprocessable` (422) for them in the log | the staging copy runs with the production key and no `dry_run`; its URLs were generated on its own host | outside production set `INDEXNOW_DRY_RUN=1` (or `INDEXNOW_ENABLED=0`); `check` fails on such a copy since core 0.6 | | the staging host serves the production key file | `key_file.enabled` is on everywhere | `key_file.enabled: false` outside production, so no engine can verify the key on that host | | the engines indexed staging pages | the staging host answered `200` for them and served the key | return `410` (or `noindex` + block in `robots.txt`) on staging, and rotate the key if it was exposed | | a preview environment must submit on purpose | — | say `dry_run: false` explicitly in that environment; `check` then warns instead of failing | ## Duplicates with `memory` and several workers | Symptom | Cause | Fix | |---|---|---| | the same URL is submitted by every worker within minutes | `debounce.store: memory` is per process; each web worker and queue worker keeps its own window | `debounce.store` = a shared cache; `check` warns about `memory` | | duplicates right after a cache outage | the store fails open: no deduplication while the cache is down | expected and bounded (one request per URL); watch the `debounce store unavailable` warning rate | | duplicates after a deploy | the shared cache was flushed, or `debounce.key_prefix` changed | harmless once; keep the prefix stable per application | ## Testing environments Outside `production_environments` a missing key enables `dry_run`: requests are logged, not sent. `check` warns about it; in a production environment `dry_run` is an error. ## Where things are logged Channel `indexnow.logging.channel` (default channel otherwise). Levels: success `debug`, 202 `info`, 403/400 `error`, 422/429/5xx `warning`, invalid configuration `critical`, silent decisions (`when` false, `fields` mismatch) `debug`. The full list is in the [core operations guide](../core/operations.md). # Backward compatibility `indexnowkit/yii2` follows SemVer. **Before 1.0, minor versions may contain breaking changes**; every one is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/yii2/CHANGELOG.md) with the migration. After 1.0 the rules below become the promise. The core's tiers ("call", "implement", "may grow") apply to every core class you touch through this package: [core bc.md](../core/bc.md). ## What the package keeps stable | Surface | Promise | |---|---| | **Component options** (`'options' => [...]` of `IndexNowComponent`, [configuration.md](configuration.md)) | Keys and their meaning stay; new keys are only added with a default. A rename ships the old key as deprecated for one minor and is listed in the changelog. | | **Component properties** `options`, `transport`, `debounceStore`, `dispatcher`, `urlResolver`, `logger`, `clock`, `checks`, `environment`, `sitemapInstalled` and the **accessors** `config()`, `kit()`, `rules()`, `keys()`, `transport()`, `submitter()`, `collector()`, `dispatcher()`, `debounceStore()`, `staging()`, `services()`, `sitemapPackage()`, `sitemapInstalled()` | Names and types stay; new properties and accessors are only added. | | **Console actions and options** (`indexnow/check`, `indexnow/key-generate`, `indexnow/submit`, `indexnow/submit-record`, `indexnow/explain`, `indexnow/sitemap`) | Names, arguments and options come from the core `Console\Definitions`; new options are only added. Output is not a contract except the exit codes and the `--json` shape of the core formatter. | | **Bootstrap** (`'bootstrap' => ['indexnow']`), the controller id `indexnow`, the key file controller `indexnow-key-file` and its URL rule `.txt` | Ids and the rule stay. | | **Behavior `ActiveRecord\IndexNowBehavior`**, **`ActiveRecord\IndexNowObserver`** | The behavior stays a drop-in; the observer's public hooks keep their names. | | **Queue job** `Queue\SubmitUrlsJob` | Its public properties and the serialized shape stay, so jobs pushed before an upgrade still run after it. | | **Check classes** `Check\QueueCheck`, `Check\UrlManagerCheck`, `Check\ActiveRecordCheck`, `Check\CacheProbe` and the `checks` property | Names stay; adding a `CheckInterface` keeps working. The `router.locales` line is the core's `Check\LocalesCheck` since 0.14.0 (`Check\RouterCheck` of this package is gone: same code, one text for the family). | | **Exceptions** | A mistake in the `indexnow` configuration is `IndexNowKit\Exception\ConfigurationException`, whatever part of the adapter finds it. Yii's `InvalidConfigException` is left to what Yii itself throws. | Not a contract: log message texts (their `context` keys are), the exact wording the actions print (exit codes and levels are), and the `Wiring` / `References` helpers the component builds its graph with. ## Pinning `composer require indexnowkit/yii2:^0.7` gets every 0.7.x patch. Read the changelog before a minor. # Commit safety A URL must not reach the engines before the transaction that changed the row commits, or a rolled-back write is announced. Yii2 makes this harder than Laravel or Doctrine: - `yii\db\Connection` fires `EVENT_COMMIT_TRANSACTION` / `EVENT_ROLLBACK_TRANSACTION` **only for the outermost** transaction. Nested `beginTransaction()` calls are savepoints and fire nothing. - The classes involved (`Transaction`, `Schema::createSavepoint()`) are not designed for interception without replacing the connection's command classes in your configuration. ## What the package does 1. In the ActiveRecord event the URLs are resolved while the old state is live (that is why the observer is synchronous, see the core's [adapters guide](../core/adapters.md)). 2. `$db->getTransaction()` is `null`: the change is autocommitted, the URLs go to the collector right away. 3. A transaction is open: the URLs are staged (core `Transaction\VerifyingStaging`) together with a **verifier**, a closure that re-reads the row by primary key with a plain `Query` (bypassing `find()` scopes) and answers whether the change landed: - insert / update / rename: the row exists and carries the written values. Only the values whose text form every driver agrees on are compared (integers, strings, booleans, backed enums, null); a DECIMAL the driver pads to the scale of the column, a timestamp it hands back with a zone suffix and a JSON document it re-spells are skipped like a column the row does not carry, so they never discard the announcement. - delete: no row. An update or a delete is staged under the subject (`Post#7`), so a second change of the same record in the same transaction joins the first instead of replacing it: the verifiers run once, at the end, and the first change's expected values are no longer in the row. An insert is staged on its own, because a savepoint rollback frees its primary key and the next insert takes it back. 4. `EVENT_COMMIT_TRANSACTION`: the verifiers run, the URLs of the changes that landed go to the collector, the rest is dropped with a `debug` line (`discarding N staged URL(s) of Post#7, change not committed`). A change that did not land drops **every** URL it produced, including `via` pages and the old URL of a renamed page: announcing "deleted" for a page that still exists is the one outcome to avoid. 5. `EVENT_ROLLBACK_TRANSACTION`: everything staged on that connection is dropped without a query. Cost: one `SELECT ... WHERE = ?` per changed record, only for changes made inside an explicit transaction. Conformance A02, A05, A05b, A05c pass without touching the connection configuration. ## What it cannot tell - An update inside a rolled-back savepoint whose written values happen to equal the row's current values (`UPDATE ... SET title = title`) passes verification and is sent as a harmless refresh of an existing page. - A verifier that throws (connection gone) counts as landed and is logged at `warning`: a stale URL costs one crawl, a lost one costs the update. - Records without a primary key cannot be verified; their URLs are sent (one `warning` per change inside a transaction, nothing at all outside one — there the URLs never go through a verifier). ## Long-running commands Staged URLs are delivered on the commit event, so an import that commits every 1000 rows sends as it goes. Without transactions the collector fills until the command ends (`collector.max_urls` caps it); call `Yii::$app->indexnow->flush()` between batches if you prefer smaller requests. # Configuration Everything lives in the `options` array of the `indexnow` component. Keys mirror the family-wide schema of `indexnowkit/core` ([configuration.md there](../core/configuration.md)); the blocks marked **Yii** are handled by this package. Verify with `php yii indexnow/check`. ## Core keys Values usually come from `getenv()` in your config file; nothing is read from the environment by the package itself. | Key | Default | Meaning | |---|---|---| | `enabled` | `true` | kill switch; `false` = nothing is sent, changes are logged at debug | | `key` | — | the IndexNow key, `[A-Za-z0-9-]{8,128}` (`php yii indexnow/key-generate`) | | `previous_key` | — | the key before a rotation: its file is still served, nothing is submitted under it | | `key_location` | — | full URL of the key file when it is not `https:///.txt` | | `base_url` | — | origin for URLs generated outside web requests (console, queue workers); required with `dispatch: queue` | | `hosts` | `[]` | `host => key` or `host => ['key', 'key_location', 'base_url', 'engines', 'previous_key']` | | `strict_hosts` | `false` | skip URLs of hosts outside `base_url` / `hosts` instead of sending them under the default key | | `environment` | `YII_ENV` (component property `environment` overrides) | feeds the non-production dry-run safety net and the `check` staging error | | `production_environments` | `['prod', 'production']` | environments where a missing key is an error, not dry-run | | `max_url_length` | `2048` | longer URLs are `invalid_url` | | `engines` | `['api']` | `api`, `yandex`, `bing`, `naver`, `seznam`, `yep`, `internetarchive`, `amazon`, an endpoint URL or an alias | | `engine_aliases` | `[]` | `alias => endpoint URL` | | `locale_hosts` | `[]` | `language => host` for rules with `locales` and no `host` | | `dispatch` | `auto` | `auto`, `queue`, `sync`, `none` (see below) | | `dry_run` | unset | log the request instead of sending it; unset outside production makes `check` fail when a key is configured | | `batch.max_urls` | `10000` | URLs per request (protocol ceiling, not a target) | | `debounce.per_url` | `600` | seconds a URL is not resubmitted; `0` = off | | `debounce.key_prefix` | `indexnowkit_` | cache key prefix of the shared window | | `throttle.max_requests_per_minute` | `60` | token bucket per process | | `http.timeout` | `10` | seconds | | `http.user_agent` | `null` | override the `indexnowkit-php/x.y.z` agent | | `retry.*` | `3 / 60 / 2.0 / 3600 / 5` | `max_attempts`, `base_delay`, `multiplier`, `max_delay`, `server_error_delay` of `SubmitUrlsJob` | | `resolver.max_via_depth` / `max_via_fanout` | `3` / `100` | limits of `via:` | | `collector.max_urls` / `detect_leaks` | `0` / `true` | early flush threshold; warn at shutdown about unflushed URLs | | `logging.max_urls` / `forbidden_escalation` / `max_body` / `levels` | `20` / `5` / `300` / `[]` | log line shaping; see the core operations guide | The full semantics of every core key, and the same table for the other adapters, are in the [core configuration reference](../core/configuration.md) ("One concept, three keys"). ## Yii blocks | Key | Default | Meaning | |---|---|---| | `dispatch` | `auto` | `auto` = `queue` when the queue component exists, else `sync`; or `sync`, `queue`, `none` | | `queue.component` | `queue` | id of the `yiisoft/yii2-queue` component `SubmitUrlsJob` is pushed to | | `queue.ttr` | `300` | time-to-reserve of the job (seconds) | | `queue.delay` | `0` | delay before the first execution (seconds) | | `queue.priority` | `null` | driver priority | | `debounce.store` | `cache` | id of a `yii\caching\CacheInterface` component, `memory` (per process) or `none` | | `http.client` | `null` | component id or class of a PSR-18 client (`null` = discover one) | | `key_file.enabled` | `true` | register the URL rule and controller for `/.txt` | | `key_file.pattern` | `.txt` | the URL rule pattern | | `key_file.cache_max_age` | `300` | `Cache-Control: max-age` of the key file | | `router.locales` | `[]` | locales generated for rules with `locales: 'all'` | | `router.locale_parameter` | `language` | GET parameter that carries the locale (added whenever a rule has a locale) | | `router.set_app_locale` | `true` | switch `Yii::$app->language` while generating a locale's URL, restored afterwards | Before 0.12.0 the three were `router.languages`, `router.language_parameter` and `router.set_app_language`; the old spellings are still read (with a deprecation warning in the log) until the next minor. The vocabulary is the core's: `locales` on the attribute, `ResolvedUrl::$locale`, `locale_hosts`. With `router.locales` left empty a rule written `locales: 'all'` yields one URL in the current locale; `indexnow/check` says so on the `router.locales` line, and warns when a hooked model actually carries such a rule. | `active_record.enabled` | `true` | `false` = `IndexNowBehavior` and the `models` list are inert | | `active_record.models` | `[]` | ActiveRecord classes hooked through class-level events (no behavior needed) | | `sitemap.*` | | needs `indexnowkit/sitemap` (`composer require indexnowkit/sitemap`), else the block is ignored and `indexnow/check` says so: `enabled`, `url`, `max_depth`, `max_sitemaps`, `max_bytes`, `allow_foreign_hosts`, `spool` (auto\|disk\|memory), `spool_dir`, `fetch_retries` | | `logging.category` | `indexnow` | category of the lines written to `Yii::getLogger()`; route it in `log.targets` | | `history.*` | | needs `indexnowkit/history` (`composer require indexnowkit/history`), else the block is ignored and `indexnow/check` says so: `store` (null\|psr16\|pdo; null = nothing kept), `limit` (500, the psr16 ring buffer), `key_prefix` (null = `debounce.key_prefix`), `pdo.dsn` (a PDO DSN; not together with `pdo.service`), `pdo.service` (the `db` component holding the table, default `db`), `pdo.table` (`indexnow_submissions`; the package's `docs/migrations.md` creates it), `retention_days` (90, what `indexnow/history --purge` removes beyond) | ## Component properties Besides `options`, the component accepts replacements as instances, config arrays, class names or component ids (`Instance::ensure`): `transport` (`TransportInterface`), `debounceStore`, `dispatcher`, `urlResolver` (replaces the attribute resolver entirely), `logger` (PSR-3), `checks` (extra `CheckInterface`s for `indexnow/check`), `environment`, `sitemapInstalled` (`null` detects `indexnowkit/sitemap`; `false` runs as if the package were absent — tests, or a deployment that must not read sitemaps), `verifyInstalled` and `historyInstalled` (the same for `indexnowkit/verify` and `indexnowkit/history`), `verifyTransport` (the transport of the pre-flight GETs), `submissionStore` (your own `SubmissionStoreInterface`; it wins over `history.store`), `clock` (PSR-20; the throttle, the debounce window and the submission timestamps read it, so `IndexNowKit\Testing\FrozenClock` makes a whole flush deterministic). A replacement that cannot be resolved (an id that names nothing, an object of another type) is an `IndexNowKit\Exception\ConfigurationException` naming the value. ## Console controller properties `controllerMap['indexnow']` may configure `loader` (`SubjectLoaderInterface`: how `submit-record`/`explain` find records), `formatter` (`ResultFormatterInterface`: table/JSON rendering), `submitters` (`SubmitterFactoryInterface`), `modelNamespaces` (default `['app\models']`). ## Invalid configuration Values come from `getenv()`, so they are only known at runtime. A broken value does not throw from a save or the response: one `critical` line (`indexnow: invalid configuration, IndexNow is disabled until it is fixed: ...`) and IndexNow runs disabled. `php yii indexnow/check` prints the exact error. Unknown keys are logged at `warning`. # Extending ## Replacing pieces Yii2 has no application-wide interface bindings, so the component builds the core graph itself and exposes the seams as properties (an instance, a config array, a class name or a component id): | Property | Interface | Use it for | |---|---|---| | `transport` | `IndexNowKit\Http\TransportInterface` | a recording transport in tests, a proxy | | `debounceStore` | `IndexNowKit\Debounce\DebounceStoreInterface` | a shared store the cache component cannot express | | `dispatcher` | `IndexNowKit\Dispatch\DispatcherInterface` | another queue, an outbox table | | `urlResolver` | `IndexNowKit\Url\UrlResolverInterface` | replace the whole "object → URLs" step | | `logger` | PSR-3 | anything but Yii's logger | | `clock` | PSR-20 `Psr\Clock\ClockInterface` | one clock for the throttle, the debounce window and the submission timestamps (`IndexNowKit\Testing\FrozenClock` in tests) | | `checks` | list of `IndexNowKit\Check\CheckInterface` | extra lines in `indexnow/check` (a CDN purge, a tenant table) | A value none of these can resolve (a component id that names nothing, a class that is not of the interface) is an `IndexNowKit\Exception\ConfigurationException` naming the value, as everywhere else in the family. Underneath, `Wiring` describes the graph once with the core's `Adapter\ServicesBuilder` (the properties above are its overrides, resolved by `References`; the Yii pieces are closures) and `services()` returns the lazy `Adapter\Services`; nothing is built before it is used, and a request that collects nothing builds nothing. Everything built is also readable, as delegates: `kit()`, `config()`, `submitter()`, `collector()`, `keys()`, `transport()`, `debounceStore()`, `routeResolver()`, `observer()`, `staging()`, `checker()`, `rules()`, and `sitemapConfig()` / `sitemapSource()` with the optional `indexnowkit/sitemap` installed (`sitemapInstalled()`; without it they throw a `LogicException` naming the package). ## Custom resolvers ```php #[IndexNow(resolver: ProductUrlResolver::class)] // a class Yii::$container can build #[IndexNow(resolver: 'productUrls')] // or an application component id ``` The class implements `IndexNowKit\Url\UrlResolverInterface`; constructor dependencies come from `Yii::$container`. ## Rules at runtime ```php Yii::$app->indexnow->observe(Product::class, [new IndexNow(route: 'product/view', params: ['id' => 'self'])], new IndexNowDefaults(when: 'active')); Yii::$app->indexnow->rules()->registerFor(Page::class, fn (Page $page): ?RuleSet => ...); // decided per object ``` `observe()` hooks the class through class-level events (`yii\base\Event::on`), the same mechanism as the `active_record.models` list. ## Manual submissions `submit(iterable $urls)`, `submitRecord($record, Event $event)`, `submitRecords(iterable $records)` (one request for many), `urlsFor()`, `explain()` return `Result`s; `collect()` parks URLs in the request collector, `flush()` sends them now. Listen to results the Yii way: `Yii::$app->indexnow->on(IndexNowComponent::EVENT_RESULT, fn (ResultEvent $e) => ...)` (`Event\ResultEvent::$result` is the `Result`; the submitter and the command submitters raise it for every result), or lower-level `Yii::$app->indexnow->submitter()->addListener(fn (Result $r) => ...)`. ## What is the core's The observer keeps only what is Yii's (the change set from `changedAttributes`, the previous state, the verify-on-commit staging); guarding, logging and the URLs of a row about to be deleted are the core's `Hook\ObserverHelper`. `SubmitUrlsJob` is `Retry\WorkerOutcome` plus a delayed re-push of the rejected URLs (the delay of `Retry\RetryPolicy` from `retry.*`, `Retry-After` first; yii2-queue's `canRetry()` only concerns exceptions, see [queue.md](queue.md)). The options of every `indexnow/` come from `Console\Definitions` and `Sitemap\Console\Definitions`, names, shortcuts, descriptions and defaults (`getActionOptionsHelp()` reads them from the definitions, not from the property docblocks), so `php yii help indexnow/submit-record` matches the bundle and artisan. ## Console `controllerMap['indexnow']` accepts `loader` (`IndexNowKit\Console\SubjectLoaderInterface`, tenant scoping or another id format), `formatter` (`ResultFormatterInterface`, your JSON envelope) and `submitters` (`SubmitterFactoryInterface`). The command bodies are the `IndexNowKit\Console\*Runner` of `indexnowkit/console`; a tenant loop over `SubmitSubjectsRunner` is a ten-line controller action. # Multiple domains and languages ## One application, several hosts Every host gets its own key (engines verify `https:///.txt` on the submitted host): ```php 'options' => [ 'key' => getenv('INDEXNOW_KEY'), // www.example.com, the base_url host 'base_url' => 'https://www.example.com', 'hosts' => [ 'example.de' => getenv('INDEXNOW_KEY_DE'), 'shop.example.com' => [ 'key' => getenv('INDEXNOW_KEY_SHOP'), 'base_url' => 'https://shop.example.com', // origin for this host's URLs outside requests 'engines' => ['yandex', 'bing'], // per-host engine list ], ], 'strict_hosts' => true, // hosts not listed are skipped, not sent under the default key ], ``` The key file action serves each host's own key only (a request for `example.de`'s key on `www.example.com` is 404) and answers with `Vary: Host`, so a shared CDN never caches one host's file for another. `php yii indexnow/check` fetches every host's key file; `--host=example.de` limits it to one. ## Rules on another host ```php #[IndexNow(route: 'product/view', params: ['slug' => 'slug'], host: 'shop.example.com')] ``` `host` can also be an accessor (`host: 'tenant.domain'`) for multi-tenant records. The URL is generated through `urlManager` and rebased onto `hosts..base_url`, else `https://`. A URL rule with its own host (`'http://shop.example.com/products/' => 'product/view'`) keeps it. ## Languages ```php 'router' => ['locales' => ['en', 'de'], 'locale_parameter' => 'language', 'set_app_locale' => true], 'locale_hosts' => ['de' => 'example.de'], // optional: one host per language ``` ```php #[IndexNow(route: 'article/view', params: ['slug' => 'slug'], locales: 'all')] ``` - `locales: 'current'` (default) generates one URL; `'all'` one per `router.locales`; a list as given. - The locale is passed as the `router.locale_parameter` parameter (`language` by default), so a URL rule that declares it (`'/articles/' => 'article/view'`) puts it in the path; without such a rule it becomes a query parameter — declare the rule. - With `set_app_locale`, `Yii::$app->language` is switched for the duration of the generation and restored, which is what localized slugs and `Url::to()` helpers read. - With `locale_hosts`, a rule without `host` generates each language on that language's host and under that host's key. ## Origin of generated URLs | Context | Origin | |---|---| | web request | whatever `urlManager` generated (the request host) | | console command, queue worker | `base_url` (the console `urlManager` has no host; the package rebases scheme, host and port) | | rule with `host:` | `hosts..base_url`, else `https://` | | URL rule with its own host | the rule's host, always | A staging copy reached under another hostname would otherwise submit its URLs under the production key; that is what `strict_hosts: true` prevents, and why `indexnow/check` warns when it is off in production. ## www and apex `example.com` and `www.example.com` are two hosts to IndexNow: each needs its own key file, and a URL submitted under the other one's key answers 422. Pick the canonical one (the one your pages link to and `` names), put it in `base_url`, redirect the other with `301`, and do not list it in `hosts` — listing both would announce two copies of every page. With `strict_hosts: true` a request that reached the application under the non-canonical name submits nothing instead of announcing duplicates. ## hreflang clusters Localized pages that point at each other with `hreflang` are one cluster to the engines: when one changes, announce the cluster. A rule with `locales: 'all'` does that for the locales of one model; for locales living on other hosts `locale_hosts` sends each locale to its host under that host's key. When translations are separate objects, `via:` walks to them: ```php #[IndexNow(route: 'article/view', params: ['slug' => 'slug'], locales: 'all')] // every language of this article #[IndexNow(via: 'translations')] // or: the sibling objects' own rules ``` # Queue, retries, failures `dispatch: auto` (the default) is `queue` when the `queue` component exists. Every batch the collector flushes (end of a request, a console command or a handled job) becomes one `IndexNowKit\Yii2\Queue\SubmitUrlsJob` carrying the normalized URLs, pushed with `queue.ttr`, `queue.delay` and `queue.priority`. ```php 'dispatch' => 'queue', 'queue' => ['component' => 'queue', 'ttr' => 300, 'delay' => 0, 'priority' => null], 'retry' => ['max_attempts' => 3, 'base_delay' => 60, 'multiplier' => 2.0, 'max_delay' => 3600, 'server_error_delay' => 5], ``` ## What the worker does 1. `SubmitterInterface::submit($urls)`: debounce, group by host, split by `batch.max_urls`, one request per engine. 2. Retryable outcomes (429, 5xx, network failure): the job **re-pushes the rejected URLs itself** as a new `SubmitUrlsJob` with the same `id` (the log lines stay connected), `attempt + 1`, the same `ttr`, and a delay, then ends successfully. The delay is `Retry-After` when the engine sent one, else `base_delay × multiplier^(attempt-1)` for 429 and `server_error_delay × multiplier^(attempt-1)` for 5xx/network, capped at `max_delay` (the core's `Retry\RetryPolicy`, from the `retry.*` options of the component). At `retry.max_attempts` the job logs `indexnow: giving up on N URL(s) of job after N attempt(s)` at error and ends. 3. Final rejections (400, 403, 422): `indexnow: N URL(s) of job rejected permanently (...); run "php yii indexnow/check"` at error, the job ends. 403 means the key file is not reachable under `https:///.txt`. 4. Anything else ends the job. URLs that were accepted are recorded in the debounce store, so a re-pushed job only resends what was rejected. The queue's own retry settings (`attempts` of the driver, `ttr`) do not apply to 429/5xx: the job never throws for them. `SubmitUrlsJob::canRetry()` (`RetryableJobInterface`) only concerns exceptions: the core's client never throws on HTTP or network failures, but a custom transport or submitter may throw `Queue\RetryableSubmissionException` to get the driver's retry within `maxAttempts`; any other exception is not retried. ## The sync driver `yii\queue\sync\Queue` stores the pushes and runs them in `run()` (with `handle: true`, at the end of the request), ignoring the delay: the re-pushed attempts run back-to-back in the same `run()`, up to `retry.max_attempts`. Fine for development; `php yii indexnow/check` warns about it. Use db, redis, amqp or another real driver in production. ## Workers Long-running workers flush the collector after every handled job (`Queue::EVENT_AFTER_EXEC` and `EVENT_AFTER_ERROR`), so a job that saves records submits their URLs in a follow-up batch on the same queue. ## Synchronous delivery `dispatch: sync` sends after the response has been sent (or when the command ends). No retries: a 429 is logged and the URLs are lost until the next change. ## Checking the wiring `php yii indexnow/check` prints the dispatch mode and the queue component the job goes to, warns when the component is the sync driver, and fails when `dispatch: queue` names a component that does not exist. # Testing your integration Configure the component with the core's test doubles and read what left: ```php use IndexNowKit\Testing\{ArrayLogger, FakeTransport}; $transport = new FakeTransport(); $logger = new ArrayLogger(); $config['components']['indexnow'] = [ 'class' => \IndexNowKit\Yii2\IndexNowComponent::class, 'options' => ['key' => str_repeat('a', 32), 'base_url' => 'https://www.example.com', 'dispatch' => 'sync', 'debounce' => ['per_url' => 0, 'store' => 'memory']], 'transport' => $transport, 'logger' => $logger, ]; $post = new Post(['slug' => 'hello']); $post->save(false); Yii::$app->indexnow->flush(); // in a request this happens after the response is sent $transport->posts[0]['body']['urlList']; // ['https://www.example.com/posts/hello'] $logger->messages('error'); // [] ``` `FakeTransport::willRespond(new Response(429))` and `onGet($url, new Response(200, $key))` script the engines and the key file; `ArrayLogger::messages($level)` returns interpolated lines. ## A deterministic clock `'clock' => new \IndexNowKit\Testing\FrozenClock('2026-09-07 08:00:00')` on the component replaces the clock of the whole graph: the throttle, the debounce window and the timestamps the submitter records in `history.store` all read it, so `advance(601)` is how a test reopens a ten-minute debounce window without sleeping. The package's own suite (`tests/`) runs the core conformance kits (`OrmConformanceTestCase`, `CoreConformanceTestCase`) through a web and a console application in memory; `Yii2TestCase` is a template for an application test case: `Fixtures::webApp()` / `consoleApp()` build the application, `Fixtures::destroy()` unregisters Yii's error handler and class-level events between tests. ## Without HTTP at all `Yii::$app->indexnow->kit()->urlsFor($post)` (or `explain($post)`) returns the URLs a record would announce, with the rule that produced each — the assertion for a rule test that should not build a transport. ## Transactions and verify-on-commit Inside `Yii::$app->db->transaction()` (or an explicit `beginTransaction()`), URLs are held with a verifier and re-read by primary key when the transaction commits; a rollback drops everything, an inner rollback the rows it undid. In a test: ```php $tx = Yii::$app->db->beginTransaction(); $post = new Post(['slug' => 'held']); $post->save(false); Yii::$app->indexnow->flush(); self::assertSame([], $transport->posts); // nothing until the commit $tx->commit(); Yii::$app->indexnow->flush(); self::assertCount(1, $transport->posts); // now it left ``` A test that wraps every case in a transaction it rolls back at the end therefore never sees a submission: assert on `urlsFor()` there, or commit. Details: [commit-safety.md](commit-safety.md). ## Queue With `dispatch: queue` (or `auto` with a queue component) the batch becomes a `SubmitUrlsJob`; run it inline with `yii\queue\sync\Queue` (`'queue' => ['class' => \yii\queue\sync\Queue::class, 'handle' => true]`) and assert on the transport after `flush()`, or assert on the pushed job with a driver that records (`handle => false` and read `Yii::$app->queue`). 429/5xx re-pushes are described in [queue.md](queue.md). ## dry_run Outside `production_environments` a missing key enables `dry_run`: the whole pipeline runs — rules, guards, URL generation, host and key selection — and the request is logged (`ArrayLogger::messages('info')` contains the body) instead of sent. Set `dry_run: false` for a test that must reach the transport. ## Conformance `tests/Conformance/` runs the core kits against this package (`CoreConformanceTestCase`: C01–C22 through the component; `OrmConformanceTestCase`: A01–A21 through `IndexNowBehavior`; the H01–H06 assertions `KeyFileAssertions` / `CheckOutputAssertions` through the key file action and the console). An application that replaces a piece (`urlResolver`, `dispatcher`) can extend the same abstract cases to prove nothing regressed. # Troubleshooting [Русская версия](troubleshooting.ru.md) Start with `php yii indexnow/check`, then `php yii indexnow/explain 'app\models\Post' `, then the `indexnow` log category at `debug` (`log.targets[].categories = ['indexnow']`, `levels = ['error', 'warning', 'info', 'trace']`). ## Nothing is sent | Symptom | Cause | Fix | |---|---|---| | `check`: `configuration: ...` and exit 1 | an `options` value (usually from `getenv()`) is invalid; IndexNow runs disabled | fix the value; the exact error is printed and logged once at `critical` | | log: `unknown option(s) in the indexnow configuration: ...` | a typo in `options` (`debounce.per_urls`, `key_file.enabld`) | the dotted path names the key | | `explain`: `when: published -> false` right after `save()` | the `when` attribute only has a database default | set it explicitly before `save()`, or give it a default in `init()` | | `explain`: `no #[IndexNow] rule` | the class has no attribute and was not registered | add `#[IndexNow]`, `IndexNowBehavior`, `active_record.models` or `observe()` | | `explain` yields URLs, the log says nothing on save | the behavior is not attached, or `active_record.enabled` / `enabled` is false | `check` prints `active_record: ... hooked` or the reason | | `debug` log: `did not land on commit` | the change was rolled back with a savepoint, or the verifier could not see the row | expected for a rollback; for a verifier problem see [commit-safety.md](commit-safety.md) | | `debug` log: `debounced` | the URL was sent within `debounce.per_url` | `--force` on a command, or lower the window | | `warning`: `skipping ... unmanaged host` | the URL's host is neither `base_url` nor in `hosts` | add the host to `hosts`, or fix `base_url` | | console: `base_url is not set` | URLs are relative and there is no request | set `base_url`; required with `dispatch: queue` | ## The key file | Symptom | Cause | Fix | |---|---|---| | `GET /.txt` is 404 | pretty URLs are off, or `key_file.enabled` is false, or the key differs | `check` prints the rule; `urlManager.enablePrettyUrl = true` | | the engines answer 403 | the served body is not the submitted key, a redirect, or a cached old file after a rotation | `curl -i https://host/.txt`; `key_file.cache_max_age` is 300 s on purpose | | `check`: `key file ... returned 200` but 403 persists | `hosts` and the submitted host differ (www vs apex) | list every host you submit under `hosts`, set `strict_hosts` | ## Queue | Symptom | Cause | Fix | |---|---|---| | `check`: `queue component "queue" is not configured` | `dispatch: queue` (or `auto` resolved to it) without yii2-queue | install `yiisoft/yii2-queue` and configure the component, or `dispatch: sync` | | 429/5xx attempts run back-to-back, no delay | `yii\queue\sync\Queue` ignores the delay of the re-push | a real driver (db, redis, amqp); `check` warns about `SyncQueue` | | the queue's `attempts` does not limit 429/5xx | the job re-pushes them itself, up to `retry.max_attempts` | set `retry.max_attempts` (and `retry.*` delays) in the component options, see [queue.md](queue.md) | | `ttr` exceeded on large batches | `queue.ttr` (300 s) is too small for the batch count | raise `queue.ttr`, or lower `batch.max_urls` | ## Sitemap `php yii indexnow/sitemap --dry-run` lists what would be sent; `sitemap.enabled is false.` means the block is off or invalid (the log has the reason). `check` prints where documents are spooled; on a read-only filesystem set `sitemap.spool_dir` or `sitemap.spool: memory`. The reader belongs to [`indexnowkit/sitemap`](../sitemap/index.md). ## Sent, but the engine answers | Answer | Meaning | Fix | |---|---|---| | 403 (`invalid_key`, job rejected permanently) | `https:///.txt` is not reachable or has another body | `indexnow/check`; a CDN may cache the old file (`key_file.cache_max_age`) | | 422 (`unprocessable`) | URLs of another host than `host`, or the key file on another host | one key per host (`hosts`), `strict_hosts: true`; console URLs need `base_url` on the right host | | 429 (`rate_limited`) | too many requests | the job re-pushes itself with `Retry-After`; lower `throttle.max_requests_per_minute` | | 202 (`pending`) | accepted, key verification pending | normal for a new key; `check --live` later answers 200 | The 403 counter that escalates to `critical` is per process: several workers each count their own five. ## Duplicates, timing - The same URL is not resubmitted within `debounce.per_url` (600 s). `--force` bypasses it; `debounce.store: cache` shares the window between requests and workers, `memory` does not. - Everything from one request leaves as one batch after the response (`Response::EVENT_AFTER_SEND`); a console command flushes when it ends, a queue worker after every job. - A rolled-back transaction submits nothing; a change re-read on commit that the row does not show is dropped ([commit-safety.md](commit-safety.md)). ## Staging submitted its URLs | Symptom | Cause | Fix | |---|---|---| | Bing/Yandex report URLs of `staging.example.com`, or `failed` / `unprocessable` (422) for them in the log | the staging copy runs with the production key and no `dry_run`; its URLs were generated on its own host | outside production set `INDEXNOW_DRY_RUN=1` (or `INDEXNOW_ENABLED=0`); `check` fails on such a copy since core 0.6 | | the staging host serves the production key file | `key_file.enabled` is on everywhere | `key_file.enabled: false` outside production, so no engine can verify the key on that host | | the engines indexed staging pages | the staging host answered `200` for them and served the key | return `410` (or `noindex` + block in `robots.txt`) on staging, and rotate the key if it was exposed | | a preview environment must submit on purpose | — | say `dry_run: false` explicitly in that environment; `check` then warns instead of failing | ## Duplicates with `memory` and several workers | Symptom | Cause | Fix | |---|---|---| | the same URL is submitted by every worker within minutes | `debounce.store: memory` is per process; each web worker and queue worker keeps its own window | `debounce.store` = a shared cache; `check` warns about `memory` | | duplicates right after a cache outage | the store fails open: no deduplication while the cache is down | expected and bounded (one request per URL); watch the `debounce store unavailable` warning rate | | duplicates after a deploy | the shared cache was flushed, or `debounce.key_prefix` changed | harmless once; keep the prefix stable per application | # Backward compatibility `indexnowkit/yii3` follows SemVer. **Before 1.0, minor versions may contain breaking changes**; every one is listed under "Changed" in [CHANGELOG.md](https://github.com/indexnowkit/php/blob/main/packages/yii3/CHANGELOG.md) with the migration. After 1.0 the rules below become the promise. The core's tiers ("call", "implement", "may grow") apply to every core class you touch through this package: [core bc.md](../core/bc.md). ## What the package keeps stable | Surface | Promise | |---|---| | **The params block** `indexnowkit/yii3` ([configuration.md](configuration.md)) | Keys and their meaning stay; new keys are only added with a default. A rename ships the old key as deprecated for one minor and is listed in the changelog. | | **Container definitions** of `config/di.php`: every core interface, `Services`, `Config`, `IndexNowKit`, `IndexNow`, `Wiring`, `VerifyingStaging`, `IndexNowObserver`, `SubjectLoaderInterface`, `Vocabulary`, `ResultFormatterInterface`, `SubmitterFactoryInterface`, `SampleOptions`, `IndexNow::VERIFY_TRANSPORT`; of `config/di-console.php`: the `Console\*Runner` of `indexnowkit/console`, `ConfigSourceInterface`, `SitemapRunner`, `HistoryRunner`, `StatusRunner` (with the package), the command classes; of `config/di-web.php`: `Key\KeyFileRequestHandler`, `Http\KeyFileHandler` | Ids stay; replacing one in the application's `di/` keeps working; new ids are only added. | | **`IndexNow`** methods `config()`, `services()`, `kit()`, `rules()`, `keys()`, `staging()`, `observer()`, `submit()`, `submitRecord()`, `submitRecords()`, `urlsFor()`, `urlsForAll()`, `explain()`, `collect()`, `flush()`, `flushIfCollected()`, `observe()`, the `*Package()` / `*Installed()` / `*Config()` accessors | Names and types stay; new methods are only added. | | **Console commands and options** (`indexnow:check`, `indexnow:config`, `indexnow:submit`, `indexnow:submit-record`, `indexnow:explain`, `indexnow:key:generate`, `indexnow:sitemap`, `indexnow:history`, `indexnow:status`) | Names, arguments and options come from the `Definitions` of `indexnowkit/console` and the optional packages; new options are only added. Output is not a contract except the exit codes and the `--json` shapes. The classes are the packages' (`IndexNowKit\Console\Command\*`, `Sitemap\Console\SitemapCommand`, `History\Console\HistoryCommand` / `StatusCommand`), mapped in `params-console.php`; this package ships no command class of its own, only `Console\ConfigSource`. | | **The config groups** the package ships (`params`, `params-console`, `di`, `di-web`, `di-console`, `events-web`, `events-console`, `routes`, `bootstrap`), the route name `indexnow/key-file` and the default pattern | Stay. | | **`ActiveRecord\IndexNowEvents`**, **`ActiveRecord\ObserverProvider`** (`set($observer, ?LoggerInterface $logger = null)`, `get()`, `isSet()`), **`ActiveRecord\IndexNowObserver`** public hooks | The attribute stays a drop-in; the observer's public hooks keep their names; `set()` grows by appended, defaulted parameters only. | | **Check codes** `dispatch.mode`, `debounce.store`, `router.key_file`, `router.route`, `router.base_url`, `active_record.enabled`, `key_file.status` and the `checks` option | Stay ([core check-codes.md](../core/check-codes.md)). | `ObserverProvider::reset()` is **test support**, marked `@internal`: it undoes the package's bootstrap in a process that builds a second container. An application never calls it, and it may change or disappear in a minor version. The codes of the optional packages follow those packages, not this one: `verify.installed`, `history.store`, `history.records`, `sitemap.spool` and the rest come from `indexnowkit/verify`, `indexnowkit/history` and `indexnowkit/sitemap` (and, without the package, from the core's `Adapter\OptionalPackage`, whose code is `.installed`). This package prints them; it does not own them. Not a contract: log message texts (their `context` keys are), the exact wording the commands print (exit codes and levels are), the `Wiring` helper's private graph construction, `Env`. `Wiring::NODES` maps every node of the core's `Adapter\Services` to the container id it is read from, and the package's own test keeps that map complete: the one node with no definition of its own is `failureCache`, which is derived from `debounce.store` rather than replaced on its own. ## Pinning `composer require indexnowkit/yii3:^0.1` gets every 0.1.x patch. Read the changelog before a minor. # Commit safety URLs must not leave before the transaction that produced them commits, or a rolled-back write is announced to the engines. `yiisoft/db` gives no signal at all — no commit event, no rollback event, and its `Command`, `Transaction` and `Connection` are `final`, so nothing can be intercepted. The package therefore **verifies instead of listening**: the core's `Transaction\VerifyingStaging`, the same mechanism `indexnowkit/yii2` uses for savepoints. ## What happens on a save 1. The `#[IndexNowEvents]` handler runs inside the ActiveRecord event (`AfterInsert`, `AfterUpdate`, `AfterDelete`; `BeforeUpdate` and `BeforeDelete` keep the old state) and resolves the URLs while the old values are live. 2. `ConnectionInterface::getTransaction()` says whether a transaction is open. - **No transaction**: autocommit already happened, the URLs go to the request collector. - **A transaction**: the URLs are staged on the connection together with a *verifier*, a closure that re-reads the row by primary key and answers whether the change landed (created / updated / renamed: the row exists and the written columns carry the written values; deleted: no row). 3. At the end of the unit of work — `AfterEmit` of yiisoft/yii-http after the response was sent, `ApplicationShutdown` of yiisoft/yii-console when the command ends, or an explicit `$indexNow->flush()` — the verifiers run and the URLs of the changes that landed go to the collector, which is then drained into the dispatcher. A change that did not land drops **every** URL it produced, including `via` pages and the old URL of a renamed page: announcing a page as deleted when it still exists is the one outcome to avoid. The discard is logged at `debug`. One `SELECT ... WHERE pk = ?` per staged record, only for changes inside explicit transactions. ## Nested transactions `beginTransaction()` inside a transaction is a savepoint in yiisoft/db. The re-read handles every combination without a hook: inner rollback + outer commit drops the inner changes and keeps the outer ones (A05c), inner commit + outer rollback drops everything (A05), both commit delivers everything in one batch (A05b). ## A transaction still open at the end If a transaction is still open when the flush runs (a request that forgot to commit, a long command flushing between batches inside a transaction), the staged URLs are **kept**, not verified: the verifier reads through the same connection and would see uncommitted data. The package logs one `warning` naming the count. They leave at the next flush after the transaction ended; at the end of a request they are lost with the connection — close the transaction. ## The compromise An update inside a rolled-back transaction whose written values happen to equal the row's current values (`UPDATE ... SET title = title`) passes the verification and is submitted as an update of an existing page: harmless, one extra crawl. A verifier that throws (no primary key declared, a connection error) counts as landed and is logged at `warning`: a stale URL costs one crawl, a lost one costs the update. ## Records without a primary key The verifier needs a primary key. A record whose table declares none submits unverified with a `warning` naming the class; declare a primary key or override `primaryKey()`. # Configuration Everything lives in the `indexnowkit/yii3` block of your params (`config/common/params.php`), merged over the package's [config/params.php](https://github.com/indexnowkit/php/blob/main/packages/yii3/config/params.php). Keys mirror the family-wide schema of `indexnowkit/core` ([configuration.md there](../core/configuration.md)); the blocks marked **Yii3** are handled by this package. Verify with `./yii indexnow:check`; `./yii indexnow:config` prints the effective result with the keys masked. ## Core keys The package reads four variables itself (`$_ENV`, `$_SERVER`, `getenv()`): `INDEXNOW_KEY`, `INDEXNOW_PREVIOUS_KEY`, `INDEXNOW_BASE_URL`, `INDEXNOW_DRY_RUN`. Anything else is a literal in the block. | Key | Default | Meaning | |---|---|---| | `enabled` | `true` | kill switch; `false` = nothing is sent, changes are logged at debug | | `key` | `INDEXNOW_KEY` | the IndexNow key, `[A-Za-z0-9-]{8,128}` (`./yii indexnow:key:generate`) | | `previous_key` | `INDEXNOW_PREVIOUS_KEY` | the key before a rotation: its file is still served, nothing is submitted under it | | `key_location` | — | full URL of the key file when it is not `https:///.txt` | | `base_url` | `INDEXNOW_BASE_URL` | origin for URLs generated outside web requests (console commands) | | `hosts` | `[]` | `host => key` or `host => ['key', 'key_location', 'base_url', 'engines', 'previous_key']` | | `strict_hosts` | `false` | skip URLs of hosts outside `base_url` / `hosts` instead of sending them under the default key | | `environment` | `YII_ENV` (then `APP_ENV`) | feeds the non-production dry-run safety net and the `check` staging error | | `production_environments` | `['prod', 'production']` | environments where a missing key is an error, not dry-run | | `dry_run` | `INDEXNOW_DRY_RUN`, else auto | log the request instead of sending it; unset outside production with a key = `check` fails | | `max_url_length` | `2048` | longer URLs are `invalid_url` | | `engines` | `['api']` | `api`, `yandex`, `bing`, `naver`, `seznam`, `yep`, `internetarchive`, `amazon`, an endpoint URL or an alias | | `engine_aliases` | `[]` | `alias => endpoint URL` | | `locale_hosts` | `[]` | `locale => host` for rules with `locales` and no `host` | | `dispatch` | `sync` | `sync` (after the response), `none` (collect, never send); no `queue` until `yiisoft/queue` is stable | | `debounce.per_url` | `600` | seconds a URL is not resubmitted; `0` = off | | `debounce.store` | `Psr\SimpleCache\CacheInterface` | a container id of a PSR-16 cache, `memory`, `none` | | `debounce.key_prefix` | `indexnowkit_` | cache key prefix (also the 403 counter and the robots cache of verify) | | `throttle.max_requests_per_minute` | `60` | per process | | `http.timeout` | `10` | seconds | | `http.client` | `null` | a container id of a PSR-18 client; `null` = discovery | | `key_file.enabled` | `true` | serve `/.txt` through the package's route | | `key_file.cache_max_age` | `300` | `Cache-Control: max-age` of the key file | | `retry.*`, `batch.max_urls`, `logging.*`, `normalizer.*`, `collector.*`, `resolver.*`, `forbidden_escalation` | core defaults | see the core reference | ## Yii3 keys | Key | Default | Meaning | |---|---|---| | `key_file.pattern` | `/{key:[A-Za-z0-9-]{8,128}}.txt` | the route pattern of the key file (`yiisoft/router`); the `key` argument is the key | | `router.locales` | `[]` | the list `locales: 'all'` on a rule expands to | | `router.locale_parameter` | `_language` | the route argument the locale is passed as; a route whose pattern lacks it gets it as a query parameter | | `active_record.enabled` | `true` | `false` = the `#[IndexNowEvents]` hooks and the `models` list are inert; `submit()` and the commands still work | | `active_record.namespaces` | `['App\Model', 'App\Entity']` | where a short class name of `indexnow:submit-record` / `indexnow:explain` is looked up | | `active_record.models` | `[]` | classes hooked without `#[IndexNowEvents]` (they still need `EventsTrait`); the package's bootstrap calls `IndexNow::observe()` for each | | `logging.category` | `indexnow` | the `category` context of every log line (what yiisoft/log targets filter on) | | `checks` | `[]` | container ids of `Check\CheckInterface` implementations to append to `indexnow:check` | | `sitemap.*` | — | the block of `indexnowkit/sitemap` (its [configuration](../sitemap/index.md)); ignored without the package | | `verify.*` | — | the block of `indexnowkit/verify` (the pre-flight GET before every submission); ignored without the package | | `history.*` | — | the block of `indexnowkit/history`; `history.pdo.service` names a `ConnectionInterface` id of the container, `history.pdo.dsn` a database of its own; ignored without the package | ## What is not an option Replacing a piece of the core (the transport, the debounce store, the dispatcher, the URL resolver, the submission store, the clock) is a container definition, not a params key: every interface is defined by the package's `config/di.php` and the application's `di/` wins ([extending.md](extending.md)). ## Invalid values The block is validated at runtime (values come from `.env`): an unknown key is a `warning` naming the dotted path (`debounce.per_urls`), an invalid value is one `critical` line naming `./yii indexnow:check` and IndexNow runs disabled until it is fixed. Nothing throws from a save or a listener. `indexnow:check` and `indexnow:config` use the strict path and print the exact error. ## Merging `yiisoft/config` replaces a whole top-level key of a group unless the application lists the group in `RecursiveMerge` (`configuration.php` of the template does for `params`). Without the recursive merge your block replaces the package's entirely; the package then applies its own defaults (`dispatch`, `key_file.pattern`, `router.locale_parameter`, `logging.category`) to what your block leaves out, so a minimal block works either way. # Extending ## Replacing pieces Every piece of the core is a definition of the container (`config/di.php` of the package), keyed by its interface. A definition with the same key in your application's `di/` wins, and the package builds every dependent piece over yours: ```php // config/common/di/indexnow.php use IndexNowKit\Dispatch\DispatcherInterface; use IndexNowKit\Http\TransportInterface; return [ TransportInterface::class => App\IndexNow\RecordingTransport::class, // a proxy, a recorder in tests DispatcherInterface::class => App\IndexNow\QueueDispatcher::class, // your queue: see below ]; ``` | Definition | Default | Replace it for | |---|---|---| | `IndexNowKit\Http\TransportInterface` | `http.client` through the core's lazy transport, else PSR-18 discovery | a proxy, a recording transport | | `IndexNowKit\Debounce\DebounceStoreInterface` | `debounce.store` (the container's PSR-16 cache) | a store the cache cannot express | | `IndexNowKit\Dispatch\DispatcherInterface` | `sync` / `none` of `dispatch` | a queue (see below) | | `IndexNowKit\Url\UrlResolverInterface` | the attribute resolver over the router bridge | replacing the whole "object → URLs" step | | `IndexNowKit\Url\RouteUrlResolverInterface` | `Url\YiiRouteUrlResolver` over `UrlGeneratorInterface` | another URL generator | | `IndexNowKit\Submission\SubmissionStoreInterface` | the store of `history.store`, else the null store | your own submission log | | `IndexNowKit\Attribute\ParamExtractor` | `new ParamExtractor(new ActiveRecordSubjectReader())` | more `SubjectReaderInterface`s (`->with()`) | | `IndexNowKit\Attribute\AttributeReaderInterface` | `RuleRegistry` over `AttributeReader` | rules from another source | | `Psr\Clock\ClockInterface` | the system clock | `FrozenClock` in tests | | `IndexNowKit\Key\KeyProviderInterface`, `UrlNormalizerInterface`, `ThrottleInterface`, `ClientInterface`, `SubmitterInterface`, `CollectorInterface`, `ResolverLocatorInterface`, `ObjectChangeHandler` | the core's factories | rarely | How it works: `Wiring` describes the graph with the core's `Adapter\ServicesBuilder`. The graph the application runs on (`Adapter\Services`, also a definition) reads every node from the container; the default of each node (`Wiring::transport()`, `Wiring::client()`, …) is the core's factory for that node over a graph whose *other* nodes come from the container. Replace nothing and you get the graph `IndexNowKit::create()` builds; replace one thing and it reaches every dependent piece, the checker and the command submitters included. Injectable as they are: `IndexNowKit\IndexNowKit` (the core facade), `IndexNowKit\Yii3\IndexNow` (the service with `submitRecord()` and the package accessors), `Config`, `RuleRegistry`, `GuardedUrlResolver`, `KeyFileResponder`, `CheckerInterface`, `SubmitterFactoryInterface`, `VerifyingStaging`, `IndexNowObserver`, `SubjectLoaderInterface`, `Vocabulary`, `ResultFormatterInterface`, the runners of `indexnowkit/console` (`di-console.php`). ## A queue `yiisoft/queue` has no stable release, so the package ships no queue mode. A dispatcher over the queue you run is one class and one definition: ```php final class QueueDispatcher implements IndexNowKit\Dispatch\DispatcherInterface { public function __construct(private readonly MyQueue $queue) {} /** @param list $urls */ public function dispatch(array $urls): void { $this->queue->push(new SubmitUrlsJob($urls, attempt: 1)); // never throw into the request } } ``` The worker is the recipe of [retries-and-queues.md](../core/retries-and-queues.md): `SubmitterInterface::submit()` (inject it, it is the graph's submitter with the pre-flight of verify when enabled), then `Retry\WorkerOutcome` decides what to re-enqueue and with what delay. Keep `dispatch: sync` in the params (it is what `check` reports; the line names your class) and set `base_url`: a worker has no request to take the host from. ## Custom resolvers ```php #[IndexNow(resolver: ProductUrlResolver::class)] // a class the container can build (autowired) #[IndexNow(resolver: 'app.product_urls')] // or a container id ``` The class implements `IndexNowKit\Url\UrlResolverInterface`; constructor dependencies come from the container. ## Rules at runtime ```php $indexNow->observe(Product::class, [new IndexNow(route: 'product/view', params: ['id' => 'self'])], new IndexNowDefaults(when: 'active')); $indexNow->rules()->registerFor(Page::class, fn (Page $page): ?RuleSet => ...); // decided per object ``` `observe()` appends the observer to the class's event dispatcher of yiisoft/active-record (`Event\ObservedDispatcher`: the class's own attribute handlers run first), the same mechanism as the `active_record.models` list. The class still needs `EventsTrait`. ## Manual submissions | Method | Returns | |---|---| | `IndexNow::submit(iterable $urls)`, `submitRecord($record, Event $event)`, `submitRecords(iterable $records)` (one request for many) | `list` — what the engines answered | | `urlsFor($record, Event $event)`, `urlsForAll(iterable $records, Event $event)` | `list` — the URLs the rules yield, nothing sent | | `explain($record, Event $event)` | `list` — the same URLs with the rule that produced each | | `collect(iterable $urls)`, `flush()` | nothing: `collect()` parks URLs in the request collector, `flush()` verifies the staged changes and sends now | Every `Result` is also dispatched to the container's PSR-14 `EventDispatcherInterface`: listen to `IndexNowKit\Result` in your `events.php`. ## Checks `'checks' => [App\IndexNow\CdnPurgeCheck::class]` in the params appends your `Check\CheckInterface` lines to `indexnow:check`; the ids are resolved through the container. ## Console `SubjectLoaderInterface` (how `submit-record` and `explain` find records: tenant scoping, another id format), `ResultFormatterInterface` (your JSON envelope) and `SubmitterFactoryInterface` are definitions; the command bodies are the `IndexNowKit\Console\*Runner` of `indexnowkit/console` (`di-console.php`), so a tenant loop over `SubmitSubjectsRunner` is a ten-line command of your own. The commands themselves are classes of the packages, not of this one: `IndexNowKit\Console\Command\*` (`indexnowkit/console`), `IndexNowKit\Sitemap\Console\SitemapCommand`, `IndexNowKit\History\Console\HistoryCommand` and `StatusCommand` — `params-console.php` maps the names to them, `di-console.php` gives them their runners, the `ConfigSourceInterface` (`Console\ConfigSource` over the facade) and the `.env` of `key:generate`. To change what a command does, replace its runner definition in your `di/` (`SubmitSubjectsRunner::class => MyTenantLoop::class`); to replace a command outright, map the name to your class in your own `params-console.php` (the application's params win) and give it the same inputs with `Definitions::submit()->applyTo($this)`. The key file route is the core's `Key\KeyFileRequestHandler` behind `Http\KeyFileHandler` (`di-web.php`): replace the former to serve the file from another key source, the route and the `{key}` pattern stay. ## What is the core's The observer keeps only what is Yii3's: the change set from the old-value snapshot of `BeforeUpdate`, the previous state for renamed pages, the verify-on-commit staging keyed by the connection. Guarding, logging and the URLs of a row about to be deleted are the core's `Hook\ObserverHelper`; the inputs of every command come from `Console\Definitions` of `indexnowkit/console` and the definitions of the optional packages, so `./yii indexnow:submit-record --help` matches the bundle and artisan. # Multiple domains and locales ## One application, several hosts Every host gets its own key (engines verify `https:///.txt` on the submitted host): ```php 'indexnowkit/yii3' => [ 'key' => $_ENV['INDEXNOW_KEY'] ?? null, // www.example.com, the base_url host 'base_url' => 'https://www.example.com', 'hosts' => [ 'example.de' => $_ENV['INDEXNOW_KEY_DE'] ?? null, 'shop.example.com' => [ 'key' => $_ENV['INDEXNOW_KEY_SHOP'] ?? null, 'base_url' => 'https://shop.example.com', // origin for this host's URLs outside requests 'engines' => ['yandex', 'bing'], // per-host engine list ], ], 'strict_hosts' => true, // hosts not listed are skipped, not sent under the default key ], ``` The key file handler serves each host's own key only (a request for `example.de`'s key on `www.example.com` is 404) and answers with `Vary: Host`, so a shared CDN never caches one host's file for another. `./yii indexnow:check` fetches every host's key file; `--host=example.de` limits it to one. ### Behind a proxy or a CDN The handler takes the host from the request URI of the PSR-7 request the application hands it — the host the application itself believes it is answering under. **Making that host trustworthy is the application's middleware, not this package**: behind a load balancer or a CDN the origin sees the proxy's `Host` (or the client's, forged) until a trusted-proxy middleware has rewritten the request from `X-Forwarded-Host` / `Forwarded`. In a Yii3 application that is `Yiisoft\ProxyMiddleware\TrustedHostsNetworkResolver`, and it belongs **before** the router in the middleware stack, so every downstream handler — the key file route included — sees the same host. Two consequences worth knowing: - Without such a middleware behind a proxy, every host reaches the handler as the proxy's, so only that host's key file is served and the others answer 404. `./yii indexnow:check` fetches the files over HTTP from outside and therefore reports exactly what an engine will see. - The same host is what URLs generated during a web request are built on (see the table below), so a proxy that is not resolved makes a request announce its pages under the proxy's hostname. `indexnow:check` run inside a web request warns when that host and `base_url` differ. ## Rules on another host ```php #[IndexNow(route: 'product/view', params: ['slug' => 'slug'], host: 'shop.example.com')] ``` `host` can also be an accessor (`host: 'tenant.domain'`) for multi-tenant records. The URL is generated through `UrlGeneratorInterface::generateAbsolute()` with the scheme and host of `hosts..base_url`, else `https://`. ## Locales ```php 'router' => ['locales' => ['en', 'de'], 'locale_parameter' => '_language'], 'locale_hosts' => ['de' => 'example.de'], // optional: one host per locale ``` ```php #[IndexNow(route: 'article/view', params: ['slug' => 'slug'], locales: 'all')] ``` - `locales: 'current'` (default) generates one URL; `'all'` one per `router.locales`; a list as given. - The locale is passed as the `router.locale_parameter` argument (`_language`, the convention of yii-demo), so a route whose pattern declares it (`Route::get('/{_language:en|de}/articles/{slug}')->name('article/view')`) puts it in the path; without such a route it becomes a query parameter — declare the argument in the route. - With `locale_hosts`, a rule without `host` generates each locale on that locale's host and under that host's key. ## Origin of generated URLs | Context | Origin | |---|---| | web request | the request's scheme and host (`CurrentRoute` carries the URI) | | console command | `base_url` (no request to take the host from; `check` says so when it is unset) | | rule with `host:` | `hosts..base_url`, else `https://` | A staging copy reached under another hostname would otherwise submit its URLs under the production key; that is what `strict_hosts: true` prevents, and why `indexnow:check` warns when it is off in production. ## www and apex `example.com` and `www.example.com` are two hosts to IndexNow: each needs its own key file, and a URL submitted under the other one's key answers 422. Pick the canonical one (the one your pages link to and `` names), put it in `base_url`, redirect the other with `301`, and do not list it in `hosts` — listing both would announce two copies of every page. With `strict_hosts: true` a request that reached the application under the non-canonical name submits nothing instead of announcing duplicates. ## hreflang clusters Localized pages that point at each other with `hreflang` are one cluster to the engines: when one changes, announce the cluster. A rule with `locales: 'all'` does that for the locales of one record; for locales living on other hosts `locale_hosts` sends each locale to its host under that host's key. When translations are separate records, `via:` walks to them: ```php #[IndexNow(route: 'article/view', params: ['slug' => 'slug'], locales: 'all')] // every locale of this article #[IndexNow(via: 'translations')] // or: the sibling records' own rules ``` # Testing your integration Replace the transport and the logger in the test container and read what left: ```php use IndexNowKit\Http\TransportInterface; use IndexNowKit\Testing\{ArrayLogger, FakeTransport}; use IndexNowKit\Yii3\IndexNow; use Psr\Log\LoggerInterface; $transport = new FakeTransport(); $logger = new ArrayLogger(); $container = new Container(ContainerConfig::create()->withDefinitions([ ...$config->get('di'), // the application's definitions, the package's included TransportInterface::class => $transport, LoggerInterface::class => $logger, ])); // run the bootstrap group the way the runner does: ObserverProvider::set() lives there foreach ($config->get('bootstrap') as $callback) { $callback($container); } $post = new Post(); $post->slug = 'hello'; $post->save(); $container->get(IndexNow::class)->flush(); // in a request this happens after the response is sent $transport->posts[0]['body']['urlList']; // ['https://www.example.com/posts/hello'] $logger->messages('error'); // [] ``` `FakeTransport::willRespond(new Response(429))` and `onGet($url, new Response(200, $key))` script the engines and the key file; `ArrayLogger::messages($level)` returns interpolated lines. Put `'dry_run' => false` in the test params: `YII_ENV` of a test run is not production, and without a key `dry_run` switches on. The package's own suite (`tests/`) runs the core conformance kits (`OrmConformanceTestCase`, `CoreConformanceTestCase`) through a container built from the package's `config/*.php`; `Yii3TestCase` and `Support\Fixtures` are a template for an application test case: `Fixtures::container()` builds the container (sqlite in memory, the routes, the PSR-14 dispatcher from `events-web.php`), `Fixtures::destroy()` resets `ObserverProvider`, `EventDispatcherProvider` and `ConnectionProvider` between tests — three static providers of the process. ## Several containers in one process `ObserverProvider` and `EventDispatcherProvider` of yiisoft/active-record are process-wide, so a test suite (or a per-request container in a worker) that builds a second container hands the same classes a second observer. Two things keep that honest, and both are automatic: - `IndexNow::observe()` and the `active_record.models` list **replace** the observed dispatcher of a class instead of wrapping another one around it, so a class is never observed twice and the old container is not held alive by the static provider. - `ObserverProvider::reset()` (test support, `@internal`) gives every class its own dispatcher back before it drops the observer. Call it between tests — `Fixtures::destroy()` does. Without the reset, the classes of the previous test keep pointing at the previous container's observer, and a save in the next test submits through a graph whose transport you already stopped looking at. ## Without HTTP at all `$container->get(IndexNow::class)->urlsFor($post)` (or `explain($post)`) returns the URLs a record would announce, with the rule that produced each — the assertion for a rule test that should not build a transport. ## Transactions and verify-on-commit Inside `$db->transaction()` (or an explicit `beginTransaction()`), URLs are held with a verifier and re-read by primary key at the next flush after the transaction ended; a rollback leaves no row and drops them. In a test: ```php $tx = $db->beginTransaction(); $post = new Post(); $post->slug = 'held'; $post->save(); $indexNow->flush(); self::assertSame([], $transport->posts); // nothing until the commit (a warning says the URLs wait) $tx->commit(); $indexNow->flush(); self::assertCount(1, $transport->posts); // now it left ``` A test that wraps every case in a transaction it rolls back at the end therefore never sees a submission: assert on `urlsFor()` there, or commit. Details: [commit-safety.md](commit-safety.md). ## The key file Send a request through the router (`Yiisoft\Router\Middleware\Router` over the `UrlMatcherInterface` of the container) and hand the response to `Testing\Conformance\KeyFileAssertions` — `tests/Support/Web.php` of the package shows the ten lines. ## Commands `new CommandTester($container->get(CheckCommand::class))` runs a command the way `./yii` does; the package's `tests/Feature/CommandsTest.php` covers every one. ## dry_run Outside `production_environments` a missing key enables `dry_run`: the whole pipeline runs — rules, guards, URL generation, host and key selection — and the request is logged (`ArrayLogger::messages('info')` contains the body) instead of sent. Set `dry_run: false` for a test that must reach the transport. ## Conformance `tests/Conformance/` runs the core kits against this package (`CoreConformanceTestCase`: C01–C22 through the container; `OrmConformanceTestCase`: A01–A21 through `#[IndexNowEvents]`; the H01–H06 assertions `KeyFileAssertions` / `CheckOutputAssertions` through the key file handler and the commands). An application that replaces a piece (`UrlResolverInterface`, `DispatcherInterface`) can extend the same abstract cases to prove nothing regressed. # Troubleshooting [Русская версия](troubleshooting.ru.md) Start with `./yii indexnow:check`, then `./yii indexnow:explain 'App\Model\Post' `, then the `indexnow` log category at `debug` (a yiisoft/log target with `categories: ['indexnow']`, `levels: ['error', 'warning', 'info', 'debug']`). ## Nothing is sent | Symptom | Cause | Fix | |---|---|---| | `check`: `configuration: ...` and exit 1 | a value of the block (usually from `.env`) is invalid; IndexNow runs disabled | fix the value; the exact error is printed and logged once at `critical` | | log: `unknown option(s) in the indexnow configuration: ...` | a typo in the block (`debounce.per_urls`, `key_file.enabld`) | the dotted path names the key | | `check`: `active record: the observer is not installed` | the package's `config/bootstrap.php` did not run | the runner must load the `bootstrap` group (`bootstrap-web` / `bootstrap-console` reference `$bootstrap` in the template) | | PHP warning `a record with #[IndexNowEvents] was saved but the observer is not installed` | the same, seen from a save | the same | | `explain` yields URLs, the log says nothing on save | the record has no `EventsTrait` (no events at all), or `active_record.enabled` / `enabled` is false | `use EventsTrait;` on the record; `check` prints the hook line | | `explain`: `when: published -> false` right after `save()` | the `when` column only has a database default | give the typed property a default, or set it before `save()` | | `explain`: `no #[IndexNow] rule` | the class has no attribute and was not registered | add `#[IndexNow]`, `active_record.models` or `observe()` | | `debug` log: `change not committed` | the change was rolled back, or the verifier could not see the row | expected for a rollback; for a verifier problem see [commit-safety.md](commit-safety.md) | | `warning` log: `staged URL(s) wait for a transaction that is still open` | the flush happened inside a transaction; the verifier would read uncommitted data | commit before the response is sent, or flush after the commit in a long command. The URLs go out at the first flush after the transaction ends; a request that ends inside it never reaches one | | `warning` log: `dropping ... staged URL(s) of a connection whose transaction was still open` | the same connection was still inside a transaction at three flushes in a row (a worker that never closes it) | close the transaction; the URLs are dropped rather than carried into another request | | `explain` yields URLs, `via:` or a dotted path through a relation is an error | the record has no `MagicRelationsTrait` | `use MagicRelationsTrait;` next to `EventsTrait`: relations are read through it | | `debug` log: `debounced` | the URL was sent within `debounce.per_url` | `--force` on a command, or lower the window | | `warning`: `skipping ... unmanaged host` | the URL's host is neither `base_url` nor in `hosts` | add the host to `hosts`, or fix `base_url` | | console: `set base_url` in a `ConfigurationException` | URLs are relative and there is no request | set `base_url` | | `Cannot generate route "post/view"` | the route name is unknown to the router, or an argument is missing | `->name('post/view')` on the route; `params` must cover its arguments | ## The key file | Symptom | Cause | Fix | |---|---|---| | `GET /.txt` is 404 | `key_file.enabled` is false, the `routes` group of the package is not merged, or the key differs | `check` prints the route line; `key_file.pattern` must end in `.txt` with a `key` argument | | the engines answer 403 | the served body is not the submitted key, a redirect, or a cached old file after a rotation | `curl -i https://host/.txt`; `key_file.cache_max_age` is 300 s on purpose | | `check`: `key file ... returned 200` but 403 persists | `hosts` and the submitted host differ (www vs apex) | list every host you submit under `hosts`, set `strict_hosts` | | only one host's key file is served, the others answer 404 behind a proxy | every request reaches the origin under the proxy's `Host` | a trusted-proxy middleware before the router ([multi-domain.md](multi-domain.md#behind-a-proxy-or-a-cdn)) | ## Dispatch | Symptom | Cause | Fix | |---|---|---| | `check`: `"dispatch" must be one of sync, none` | `dispatch: queue` from another adapter's configuration | `sync`, and a `DispatcherInterface` definition for your queue ([extending.md](extending.md)) | | the pre-flight of verify slows requests down | `verify.enabled` with `dispatch: sync` fetches pages after the response | expected; a queue dispatcher moves it to a worker | ## Sitemap `./yii indexnow:sitemap --dry-run` lists what would be sent; `sitemap.enabled is false.` means the block is off or invalid (the log has the reason). `check` prints where documents are spooled; on a read-only filesystem set `sitemap.spool_dir` or `sitemap.spool: memory`. The reader belongs to [`indexnowkit/sitemap`](../sitemap/index.md). ## Sent, but the engine answers | Answer | Meaning | Fix | |---|---|---| | 403 (`invalid_key`) | `https:///.txt` is not reachable or has another body | `indexnow:check`; a CDN may cache the old file (`key_file.cache_max_age`) | | 422 (`unprocessable`) | URLs of another host than `host`, or the key file on another host | one key per host (`hosts`), `strict_hosts: true`; console URLs need `base_url` on the right host | | 429 (`rate_limited`) | too many requests | lower `throttle.max_requests_per_minute`; a queue dispatcher retries with `Retry-After` | | 202 (`pending`) | accepted, key verification pending | normal for a new key; `check --live` later answers 200 | The 403 counter that escalates to `critical` is shared through the cache behind `debounce.store`; with `memory` it is per process. ## Duplicates, timing - The same URL is not resubmitted within `debounce.per_url` (600 s). `--force` bypasses it; the container's PSR-16 cache (the default `debounce.store`) shares the window between requests and workers, `memory` does not. - Everything from one request leaves as one batch after the response (`AfterEmit`); a console command flushes when it ends (`ApplicationShutdown`). - A rolled-back transaction submits nothing; a change re-read at the flush that the row does not show is dropped ([commit-safety.md](commit-safety.md)). ## Staging submitted its URLs | Symptom | Cause | Fix | |---|---|---| | Bing/Yandex report URLs of `staging.example.com`, or `failed` / `unprocessable` (422) for them in the log | the staging copy runs with the production key and no `dry_run`; its URLs were generated on its own host | outside production set `INDEXNOW_DRY_RUN=1` (or `enabled: false`); `check` fails on such a copy | | the staging host serves the production key file | `key_file.enabled` is on everywhere | `key_file.enabled: false` outside production, so no engine can verify the key on that host | | the engines indexed staging pages | the staging host answered `200` for them and served the key | return `410` (or `noindex` + block in `robots.txt`) on staging, and rotate the key if it was exposed | | a preview environment must submit on purpose | — | say `dry_run: false` explicitly in that environment; `check` then warns instead of failing | # Справочник по атрибутам [English version](attribute-reference.md) Класс объявляет **список правил** — по одному на семейство публичных URL, которые у него есть. В PHP это атрибуты; в других языках семейства — декораторы или конфиг-объекты. Все они компилируются в один и тот же `IndexNowKit\Attribute\UrlRule`, и всё ниже по течению — классификация событий, guard'ы, локали, делегирование `via`, дедупликация, вывод `explain` — работает только с ним. ## Три атрибута | Атрибут | Цель | Назначение | |---|---|---| | `#[IndexNow]` | класс, **повторяемый** | одно правило URL | | `#[IndexNowDefaults]` | класс | политика, общая для всех правил класса и его потомков | | `#[IndexNowUrl]` | публичный метод | возвращаемое значение метода — семейство URL (конвенция `get_absolute_url()`) | ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults, IndexNowUrl}; #[IndexNowDefaults(when: 'isPublished', fields: ['slug', 'title', 'body', 'published'])] #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] #[IndexNow(route: 'post_amp', params: ['slug' => 'slug'], when: 'hasAmp', whenFields: ['ampEnabled'])] class Post {} ``` ## Источники Ровно один источник на `#[IndexNow]`. Ноль источников или два — `ConfigurationException` при компиляции с сообщением, называющим виновников. | Источник | Значение | Даёт | |---|---|---| | `route` | имя маршрута фреймворка | один URL на локаль, сгенерированный `RouteUrlResolverInterface` адаптера | | `resolver` | класс `UrlResolverInterface` или id сервиса | то, что вернёт резолвер | | `via` | аксессор к связанному объекту или коллекции | собственные URL связанных объектов, разрешённые как обновления | | `url` | аксессор, возвращающий `string`, `iterable` или `null` | эти URL | | `urls` | список литеральных URL | эти URL, абсолютные или относительно `base_url` | `url` и `urls` легко перепутать, поэтому проверяются оба: `url: '/about'` и `urls: ['aboutUrl']` отклоняются с сообщением, какой из них вы имели в виду. `resolver` требует `ResolverLocatorInterface`. В чистом PHP это `ArrayResolverLocator`, который также создаёт класс по имени, если у его конструктора нет обязательных аргументов. Адаптеры фреймворков ищут id в контейнере. ## Параметры `params` сопоставляет имя параметра маршрута источнику. Простая строка — DSL аксессоров; всё остальное — один из четырёх типизированных объектов `Param\ParamValue`. ### DSL аксессоров Разрешается в этом порядке, на самом объекте: 1. `'self'` — сам объект (route model binding: `params: ['post' => 'self']`); 2. путь через точку — каждый сегмент разрешается рекурсивно (`'category.slug'`); не-объект в середине бросает исключение; 3. метод с точно таким именем; 4. `get`, `is` или `has` плюс имя с заглавной (`'published'` находит `getPublished()`, затем `isPublished()`, затем `hasPublished()`); 5. свойство, в том числе приватное. Ничего не подошло — `ConfigurationException` с именем аксессора и класса. ### Типизированные источники | Класс | Пример | Смысл | |---|---|---| | `Param\Accessor` | `new Accessor('category.slug')` | явная форма строкового аксессора | | `Param\Value` | `new Value('html')` | константа | | `Param\Formatted` | `new Formatted('publishedAt', 'Y')` | `DateTimeInterface::format()` значения аксессора | | `Param\Call` | `new Call('slugFor', Placeholder::Locale)` | вызов метода; дополнительные аргументы передаются как есть | `Param\Placeholder::Locale` и `Param\Placeholder::Host` подставляются для каждого генерируемого URL, так что `Call` может вернуть slug для локали или путь для тенанта. Извлечение выполняется один раз на URL, а не на правило. ### Приведение типов Параметры маршрута должны годиться для URL. Экстрактор принимает `null` и скаляры как есть, разворачивает `BackedEnum` в его `value`, приводит `Stringable` value object к строке и пропускает обычные объекты для route model binding. Голый `DateTimeInterface` отклоняется с указанием на `new Formatted(...)`: неявное форматирование даты — так URL незаметно меняет форму. Всё остальное бросает исключение. ## Опции правила | Опция | Тип | Дефолт | Смысл | |---|---|---|---| | `when` | имя аксессора, `Condition` (`new Equals(path, value)` или свой класс) или замыкание `fn(object): bool` (только правила времени выполнения) | наследуется | страница существует, пока условие истинно | | `whenFields` | список имён полей | `[]` | поля, стоящие за `when` этого правила, когда имя не совпадает с полем (у `when` уровня класса свои `whenFields` в `#[IndexNowDefaults]`) | | `fields` | список имён полей или `null` | наследуется, затем `[]` | только для обновлений: отправлять, когда изменилось одно из них; `[]` = любое поле | | `events` | подмножество `created`, `updated`, `deleted` (строки или case'ы `Event`) или `null` | наследуется, затем все три | какие события жизненного цикла слушает правило | | `locales` | `'current'`, `'all'` или список, или `null` | наследуется, затем `'current'` | разворачивание локалей для локализованных маршрутов | | `host` | строка или `ParamValue` | `null` | генерировать URL этого правила на этом хосте (мультидомен) | | `name` | строка | выводится | стабильный id правила для логов, `explain` и переопределения в подклассе | `when` — **конъюнкция**: `when` класса и `when` правила должны выполняться оба. `fields`, `events` и `locales` — дефолты, которые правило переопределяет; `null` — наследовать, `[]` — «без фильтра». Строковый аксессор проверяется на truthy, что верно для булевых и неверно для строки статуса (`'draft'` — truthy). Для строковых и enum-состояний используйте `Equals`; он же даёт точное определение старого состояния из change set ORM: ```php use IndexNowKit\Attribute\Param\Equals; #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], when: new Equals('status', 'published'))] #[IndexNow(route: 'job_show', params: ['id' => 'id'], when: new Equals('state', JobState::Open))] // BackedEnum или его value ``` Два интерфейса — по тому, что условие читает. `Attribute\Param\FieldCondition` (`field()`, `heldFor($oldValue)`) читает одно поле: ядро читает его через `ParamExtractor` графа (ридеры видят атрибуты Eloquent/ActiveRecord) и спрашивает `heldFor()` — для текущего значения и, из change set ORM, для старого; `Equals` — поставляемая реализация. `Attribute\Param\Condition` (`evaluate(object $subject): bool`) смотрит на объект целиком — для того, что одним полем не сказать. Оба годятся в `when`. `Condition` не знает старого значения: `ChangeClassifier` вычисляет его на текущем объекте, и переход `open → closed` считается обновлением, а не удалением — если только `whenFields` не называет поля, которые условие читает. Вручную условие вычисляет экстрактор графа: `$indexNow->extractor->condition($subject, new Equals(...))` (с 0.12 у `FieldCondition` нет `evaluate()`). `Equals` в `params` — ошибка типов (условие — не источник значения). `explain` печатает каждое условие со считанным значением (`when: status ("draft") -> true — a non-empty string is truthy; use new Equals('status', "draft")`), `explain --json` — то же одним документом. Подробнее — в английской версии. Правила, зарегистрированные во время выполнения (`RuleRegistry`), могут передать замыкание: `when: fn (WP_Post $p): bool => $p->post_status === 'publish'`. Старое значение замыкания восстановить нельзя, поэтому перечислите читаемые им поля в `whenFields`; изменение одного из них считается сменой видимости (см. таблицу семантики). Неизвестное имя события — `ConfigurationException` с именем атрибута и значением. ### Имена правил Выводятся из источника, если не заданы: имя маршрута; `resolver:`; `via:`; `url:` (и `url:` для `#[IndexNowUrl]`); `urls:<первые два литерала>`. Два правила одного класса с одинаковым выведенным именем получают `#2`, `#3` в порядке объявления. Задавайте явное `name`, когда подкласс должен переопределить конкретное правило или когда выведенное имя нестабильно. ## Дефолты класса и наследование Компилятор обходит иерархию **от корня** к листу. - `#[IndexNowDefaults]` сливается поле за полем, ближайшее объявление побеждает. Объявление со своим `when` заменяет и унаследованные `whenFields`; без `when` — добавляет к ним. - Правила накапливаются. Правило, чьё имя повторяет предка, **заменяет** его; новое имя **добавляет** страницу. Так подкласс меняет одну страницу, не повторяя остальные. - `#[IndexNowUrl]` читается на публичных методах, объявленных каждым классом цепочки, поэтому переопределение в подклассе побеждает. Метод не должен требовать аргументов. - Интерфейсы и трейты **не** сканируются: PHP не наследует атрибуты класса через них, и Doctrine-маппинг ведёт себя так же. ```php #[IndexNowDefaults(when: 'isPublished')] #[IndexNow(route: 'content_show', params: ['slug' => 'slug'])] abstract class Content {} #[IndexNow(route: 'content_show', params: ['slug' => 'slug', 'section' => new Value('news')])] // заменяет #[IndexNow(route: 'news_amp', params: ['slug' => 'slug'])] // добавляет class News extends Content {} ``` В иерархии, где публичные страницы есть лишь у части подклассов, базовый класс не должен нести правил: что не объявлено, то не наследуется. ## Семантика: событие, до, после Видимость оценивается для каждого правила. `W` — конъюнкция `when` класса и `when` правила; `W_before` восстанавливается из change set ORM. | Событие ORM | `W_before` | `W_after` | `fields` совпали | Событие правила | Состояние, из которого строится URL | |---|---|---|---|---|---| | insert | — | true | — | `Created`, если подписано | новое, после записи (id назначены) | | insert | — | false | — | нет | — | | update | true | true | да | `Updated`, если подписано | новое, после записи | | update | true | true | нет | нет | — | | update | true | false | не важно | **`Deleted`**, если подписано | текущее, до записи | | update | false | true | не важно | `Created`, если подписано | новое, после записи | | update | false | false | — | нет | — | | delete | — | true | — | `Deleted`, если подписано | состояние до удаления | | delete | — | false | — | нет | — | | цель `via`, любое событие | — | true | по правилу цели | цель разрешается как `Updated` | собственные правила и guard'ы цели | | цель `via`, любое событие | — | false | — | нет | — | Два следствия стоит проговорить. `fields` никогда не подавляет смену видимости, только обычное обновление: страница, которая только что погасла, объявляется независимо от того, какое поле это сделало. И удаление объекта, чьё правило не применяется, ничего не отправляет — страница никогда не была публичной, чистка черновиков молчит. ### Восстановление `W_before` `ChangeClassifier::classify(UrlRule $rule, object $subject, ParamExtractor $extractor, array $changedFields, array $changeSet = [])` возвращает `Event`, важное правилу, или `null`. Видимость старого состояния — best effort, в три яруса: 1. Аксессор `when`, чьё поле есть в change set, вычисляется **точно** из старого значения. Поле ищется по имени, затем по конвенции: `isPublished → published → is_published`, `hasAmp → amp → has_amp`, `getStatus → status`. `UrlRule::fieldCandidates()` отдаёт этот список. 2. Аксессор без записи в change set, но с зависимым полем (его кандидаты или объявленное `whenFields`) среди изменённых считается **перевернувшимся**. Ложное срабатывание стоит одного запроса; пропуск оставляет мёртвую страницу в индексе. 3. Иначе аксессор сохраняет текущее значение. Называйте поля через `whenFields`, когда аксессор — метод, имя которого не связано с колонкой, например `when: 'isVisibleToPublic', whenFields: ['status', 'visibleFrom']`. ## `via` `via` переотправляет страницы связанного объекта: изменившийся комментарий обновляет свой пост, изменившийся товар — категорию. Цели всегда разрешаются как `Updated`: их страница существует независимо от того, что случилось с источником. Глубина ограничена 3, fan-out — 100 связанных объектов на правило (аргументы конструктора `maxViaDepth` и `maxViaFanout` у `AttributeUrlResolver`); превышение глубины бросает исключение, превышение fan-out пишет warning и останавливается. Правило цели, делегирующее обратно через тот же аксессор, пропускается, так что `A -> B -> A` завершается. Полученные URL сохраняют всю цепочку в имени правила: `via:category -> category_show`. ## Имена полей `fields` и `whenFields` — **имена полей модели**, как их пишет разработчик, никогда не колонки БД. Doctrine `getEntityChangeSet()` даёт ровно их. Объявленное поле совпадает с изменённым, когда они равны или одно — префикс другого через точку, так что `fields: ['address']` ловит изменение embeddable, отражённое как `address.city`. ## Типы, которые потребляют адаптеры ```php final readonly class UrlRule { public string $name; public RuleSource $source; // Route|Resolver|Via|Url|Urls public ?string $route; public array $params; public ?string $resolver; public ?string $via; public ?string $url; public array $urls; public array $when; public array $whenFields; public array $fields; public array $events; public array|string $locales; public string|ParamValue|null $host; public function listensTo(Event $event): bool; public function caresAbout(array $changedFields): bool; public function appliesTo(object $subject): bool; // каждый аксессор `when` truthy public function whenDependsOn(string $field): bool; public static function fieldCandidates(string $accessor): array; } ``` `RuleSet` — все правила одного класса в порядке объявления (сначала родители). Он `Countable` и итерируемый, пуст для классов без правил (вызывающим не нужно проверять null), и даёт `isEmpty()`, `get(string $name)` и `listensTo(Event $event)` как дешёвый предфильтр для ORM-хуков. `AttributeReaderInterface::rules(string|object $classOrObject): RuleSet` — точка поиска. Дефолтный `AttributeReader` компилирует атрибуты через `RuleCompiler` и кэширует по классу на время процесса. На испорченном объявлении он бросает `ConfigurationException` — ORM-хуки должны читать через `ObjectChangeHandler` или `GuardedUrlResolver`, которые логируют. `ResolvedUrl` несёт происхождение для `explain`, логов и панели профайлера: `url`, `rule`, `class`, `event`, `locale`, плюс `source()` (`App\Entity\Post#post_amp`) и `ResolvedUrl::urls()` для сплющивания списка в уникальные строки. ## Правила, регистрируемые во время выполнения Модели, которые не могут нести атрибуты — типы записей CMS, чужие классы, API на замыканиях — используют `RuleRegistry`, реализующий `AttributeReaderInterface` поверх внутреннего ридера (по умолчанию атрибуты). ```php use IndexNowKit\Attribute\{IndexNow, IndexNowDefaults, RuleRegistry}; $registry = new RuleRegistry(); $registry->register(Post::class, [ new IndexNow(route: 'posts.show', params: ['post' => 'self']), new IndexNow(urls: ['/']), ], new IndexNowDefaults(when: 'isPublished')); $registry->register(WP_Post::class, [new IndexNow(resolver: 'wp_permalink')], new IndexNowDefaults( when: fn (WP_Post $post): bool => $post->post_status === 'publish', // или: new Equals('post_status', 'publish') whenFields: ['post_status'], )); $registry->registerFor(CmsPage::class, fn (CmsPage $page): ?RuleSet => $page->isSystem() ? null : $rulesFor($page)); $indexNow = IndexNowKit::create($config, attributes: $registry); ``` `register()` компилирует экземпляры атрибутов, созданные в коде, без рефлексии. `registerFor()` решает по объекту и может вернуть `null`, чтобы передать слово внутреннему ридеру. Зарегистрированные правила заменяют то, что вернул бы внутренний ридер для этого класса, и наследуются подклассами. ## Разрешение без ORM `AttributeUrlResolver` разрешает каждое правило класса через его источник; `GuardedUrlResolver` оборачивает его, чтобы ничего не бросалось. `ObjectChangeHandler` — то, на чём строятся ORM-хуки: классифицирует созданный, изменённый или удалённый объект по правилам и разрешает URL, логируя каждый молчаливый исход. См. [adapters.md](adapters.md). ## Антипаттерны Пять объявлений, которые компилируются, работают и отправляют не то. **1. Литеральный URL в `url:`.** `url:` называет аксессор; `urls:` перечисляет литералы. ```php #[IndexNow(url: '/')] // неверно: читает свойство или метод с именем "/" #[IndexNow(urls: ['/'])] // верно #[IndexNow(url: 'canonicalUrl')] // верно: $post->canonicalUrl() или ->canonicalUrl ``` **2. Строка статуса в `when:`.** Строка — аксессор, читаемый как truthy: `'published'` значит «атрибут `published` truthy», а колонка `status` со значением `'draft'` тоже truthy. ```php #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], when: 'status')] // неверно: 'draft' — truthy #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], when: new Equals('status', 'published'))] // верно ``` **3. Правило на странице, которую нельзя индексировать.** Превью, админка, страница с `noindex` или запрет в `robots.txt`: поисковик её запросит, найдёт неиндексируемой и запишет ошибку в счёт ключа. ```php #[IndexNow(route: 'post_preview', params: ['slug' => 'slug'])] // неверно: превью несёт noindex #[IndexNow(route: 'post_show', params: ['slug' => 'slug'], when: 'isPublished')] // верно: публичная страница под guard'ом ``` **4. Неканонические URL.** Варианты фильтров и сортировок, трекинг-параметры, apex рядом с `www`, `http` рядом с `https`: отправляйте каноническую страницу один раз. Генерируйте URL через роутер с `base_url` на каноническом origin; не склеивайте их строками из запроса. ```php #[IndexNow(urls: ['/products?sort=price&utm_source=indexnow'])] // неверно: вариант и трекинг-параметр #[IndexNow(route: 'products_index')] // верно: каноническая выдача ``` **5. Нет `when` у модели с черновиками.** Без guard'а отправляет каждое сохранение, черновики включительно, а снятая страница объявляется обновлением, не удалением. ```php #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] // неверно, если у Post есть черновики #[IndexNowDefaults(when: 'isPublished')] // верно: черновики пропускаются, #[IndexNow(route: 'post_show', params: ['slug' => 'slug'])] // published → draft — удаление ``` Что библиотека проверяет за вас: URL абсолютный `http(s)`, без credentials, фрагмента и управляющих символов, и принадлежит хосту, для которого у вас есть ключ (`strict_hosts`). Что проверить не может: `noindex`, `robots.txt`, canonical на другой адрес, код ответа страницы — сегодня это забота автора правила, а в следующих релизах — дополнения `verify` (`check --sample`). # Конфигурация [English version](configuration.md) — там же генерируемая таблица «One concept, four keys» (ключи четырёх адаптеров). `IndexNowKit\Config` — неизменяемый value object, общий для всех адаптеров. Строится одним из трёх способов и валидируется в конструкторе, поэтому сломанная настройка падает при загрузке, а не при первой отправке. ```php use IndexNowKit\Config; $config = Config::fromArray([...]); // конфиг-файлы фреймворка $config = Config::fromEnv(); // переменные окружения INDEXNOW_* $config = new Config(key: '...', baseUrl: '...'); // именованные аргументы $config = $config->with(dryRun: true); // неизменяемая копия ``` ## Опции `fromArray()` принимает вложенную форму ниже; это каноническая схема, которую повторяет каждый языковой адаптер. ```php Config::fromArray([ 'enabled' => true, 'key' => $_ENV['INDEXNOW_KEY'], 'hosts' => [ 'www.example.com' => 'KEY-FOR-EXAMPLE', 'shop.example.com' => [ 'key' => 'KEY-FOR-SHOP', 'key_location' => 'https://shop.example.com/keys/indexnow.txt', 'base_url' => 'https://shop.example.com', ], ], 'strict_hosts' => true, 'key_location' => null, 'base_url' => 'https://www.example.com', 'engines' => ['api'], 'dispatch' => 'sync', 'batch' => ['max_urls' => 10000], 'debounce' => ['per_url' => 600], 'throttle' => ['max_requests_per_minute' => 60], 'http' => ['timeout' => 10.0, 'user_agent' => null], 'key_file' => ['enabled' => true], 'dry_run' => false, 'environment' => $_ENV['APP_ENV'] ?? null, ]); ``` | Опция | Аргумент конструктора | Дефолт | Смысл | |---|---|---|---| | `enabled` | `enabled` | `true` | `false` отбрасывает каждую отправку; URL возвращаются как `skipped` с причиной `disabled`, лог на `info` | | `key` | `key` | `null` | ключ по умолчанию, 8–128 символов `[A-Za-z0-9-]`, для каждого хоста, не перечисленного в `hosts` | | `hosts` | `hosts` | `[]` | `host => key` или `host => {key, key_location?, base_url?, engines?, previous_key?}` | | `strict_hosts` | `strictHosts` | `false` | применять ключ по умолчанию **только** к хосту `base_url`; любому другому хосту нужна запись в `hosts`, иначе его URL пропускаются | | `key_location` | `keyLocation` | `null` | абсолютный URL файла ключа, если он не `https://{host}/{key}.txt` | | `base_url` | `baseUrl` | `null` | абсолютный URL сайта; разрешает относительные URL, обязателен вне HTTP-запроса | | `engines` | `engines` | `['api']` | имена движков (`api`, `yandex`, `bing`, `naver`, `seznam`, `yep`, `internetarchive`, `amazon`) или полные URL endpoint'ов | | `dispatch` | `dispatch` | `'sync'` | режим доставки, определяемый адаптером; ядро проверяет идентификатор и печатает его | | `batch.max_urls` | `batchMaxUrls` | `10000` | URL на запрос; `Config::MAX_BATCH_URLS` — максимум протокола, потолок, а не цель: меньшие батчи принимаются так же | | `debounce.per_url` | `debouncePerUrl` | `600` | секунды, в течение которых тот же URL не отправляется повторно; `0` выключает дебаунс | | `throttle.max_requests_per_minute` | `throttleMaxRequestsPerMinute` | `60` | исходящих запросов в минуту на процесс; `0` = без лимита | | `http.timeout` | `httpTimeout` | `10.0` | секунды, только для клиентов, которые библиотека создаёт сама | | `http.user_agent` | `userAgent` | `null` | заменяет `indexnowkit-php/ (+https://github.com/indexnowkit/php)` | | `key_file.enabled` | `serveKeyFile` | `true` | должен ли адаптер отвечать на `GET /{key}.txt`; `serve_key_file` — устаревшее имя, побеждает, если заданы оба | | `key_file.cache_max_age` | `keyFileMaxAge` / `keyFileHeaders()` | `300` | `Cache-Control: max-age` ответа с файлом ключа; короткий нарочно — закэшированный старый файл после ротации превращает каждую отправку в 403. `keyFileHeaders()` добавляет `Vary: Host`, когда тело зависит от хоста: есть карта `hosts` **или** включён `strict_hosts` (ключ по умолчанию отдаётся только базовому хосту, остальные получают 404, и общий кэш без `Vary` запомнил бы тот ответ, который пришёл первым) | | `debounce.store` | `debounceStore` | `null` | `memory` (на процесс), `none` или id, который адаптер разрешает в свой общий кэш; `null` = дефолт адаптера (Laravel `cache`, бандл `cache.app`, Yii2 `cache`, Yii3 — id контейнера `Psr\SimpleCache\CacheInterface`, чистый PHP `memory`) | | `http.client` | `httpClient` | `null` | id или класс PSR-18 клиента, который разрешает адаптер; `null` = discovery. Клиент несёт настройки приложения, поэтому `check` предупреждает, когда он задан (`http.client`): клиент, который сам ходит по редиректам, превращает 30x на страницу-заглушку в 200 при проверке файла ключа. Pre-flight пакета `indexnowkit/verify` его **не** использует — пройденный редирект скрыл бы ровно тот 3xx, ради которого pre-flight и существует, — и строит собственный клиент по `verify.timeout` и `verify.max_redirects` | | `dry_run` | `dryRun` | `false` | логировать запрос вместо отправки; вне production незаданный `dry_run` при настроенном ключе делает `check` красным | | `environment` | `environment` | `null` | окружение приложения; управляет страховкой ниже | | `production_environments` | `productionEnvironments` | `['prod', 'production']` | имена окружений (без учёта регистра), считающихся production; заменяет список по умолчанию | | `previous_key` | `previousKey` | `null` | ключ до ротации: файл ключа его ещё принимает, отправки под ним не идут; также `hosts..previous_key` | | `hosts..engines` | `hostEngines` / `endpointsFor()` | наследует `engines` | движки только для одного хоста | | `engine_aliases` | `engineAliases` / `resolveEngine()` | `{}` | короткие имена для своих endpoint'ов, применимы везде, где называется движок | | `locale_hosts` | `localeHosts` / `hostForLocale()` | `{}` | локаль => хост; правила с `locales` и без `host` генерируют каждую локаль на её хосте | | `logging.max_body` | `logBody` | `300` | байт тела ответа движка, сохраняемых в строке лога об ошибке | | `max_url_length` | `maxUrlLength` | `2048` | URL длиннее пропускаются как `invalid_url` | | `debounce.key_prefix` | `debounceKeyPrefix` | `'indexnowkit_'` | префикс ключей общего debounce store | | `logging.max_urls` | `logUrls` / `logSample()` | `20` | URL, перечисляемых в одной строке лога; `0` = только счётчики | | `logging.forbidden_escalation` | `forbiddenEscalation` | `5` | подряд идущих 403 на хост до эскалации лога в `critical` | | `logging.levels` | `logLevels` / `logLevel()` | `{}` | переопределение PSR-3 уровня по исходу; события и дефолты в `Config::LOG_EVENTS` | | `retry.max_attempts`, `retry.base_delay`, `retry.multiplier`, `retry.max_delay`, `retry.server_error_delay` | `retryPolicy()` | `3`, `60`, `2.0`, `3600`, `5` | `RetryPolicy` для обработчиков очередей и `RetryingSubmitter` | | `resolver.max_via_depth`, `resolver.max_via_fanout` | `resolverMaxViaDepth`, `resolverMaxViaFanout` | `3`, `100` | лимиты обхода `via:` в `AttributeUrlResolver`; `IndexNowKit::create()` этот резолвер не строит — адаптер, который строит, передаёт их вместе с `localeHosts` | | `collector.max_urls` | `collectorMaxUrls` | `0` | `IndexNowKit::collect()` сбрасывает буфер досрочно при этом размере; `0` = только на `flush()` | | `collector.detect_leaks` | `collectorDetectLeaks` | `true` | warning при завершении о собранных, но не сброшенных URL | Что `Url\UrlNormalizer` делает всегда, без опции: приводит схему и хост к нижнему регистру, интернационализированный хост — к punycode, убирает порт по умолчанию, разрешает dot-сегменты, срезает фрагмент и приводит percent-кодирование пути и запроса к канонической форме RFC 3986 §6.2.2 — экранированный незарезервированный символ (`A-Za-z0-9-._~`) становится символом, остальные экраны переводятся в верхний регистр, так что `/%7Euser/a%2Db` и `/~user/a-b` — один URL и дебаунсятся, отправляются и записываются один раз. URL с учётными данными, управляющими символами или схемой не `http(s)` отвергается через `InvalidUrlException`. Константы, на которые стоит ссылаться вместо чисел: `Config::MAX_BATCH_URLS` (10000), `Config::DEFAULT_BATCH_MAX_URLS`, `Config::DEFAULT_DEBOUNCE_PER_URL` (600), `Config::DEFAULT_THROTTLE_PER_MINUTE` (60), `Config::DEFAULT_HTTP_TIMEOUT` (10.0), `Config::PRODUCTION_ENVIRONMENTS` (`['prod', 'production']`), `Config::DEFAULT_MAX_URL_LENGTH`, `Config::DEFAULT_LOG_URLS`, `Config::DEFAULT_FORBIDDEN_ESCALATION`, `Config::DEFAULT_RETRY_*`, `Config::DEFAULT_RESOLVER_MAX_VIA_*`, `Config::LOG_EVENTS`. ## Одно понятие — четыре ключа Адаптеры разделяют ключи ядра под теми же именами и добавляют несколько своих; у некоторых понятий ключ (или набор значений) свой в каждом фреймворке. Таблицы генерируются из кода (`bin/config-table`) и проверяются в CI — см. раздел «One concept, four keys» в [английской версии](configuration.md#one-concept-four-keys). Коротко: | Понятие | Symfony | Laravel | Yii2 | Yii3 | |---|---|---|---|---| | Режим доставки `dispatch` | `auto` \| `messenger` \| `sync` \| `none` | `queue` \| `sync` \| `none` (без `auto`) | `auto` \| `queue` \| `sync` \| `none` | `sync` \| `none` (очередь — заменённый `DispatcherInterface`) | | Очередь / транспорт | `messenger.transport` | `queue.connection` | `queue.component` | — | | Локали для `locales: all` | `framework.enabled_locales` | `router.locales` | `router.locales` (до 0.12 — `router.languages`) | `router.locales` | | Переключатель ORM-хуков | `doctrine.enabled` | `eloquent.enabled` | `active_record.enabled` | `active_record.enabled` | | Маршрут файла ключа | `key_file.path` | `key_file.path` | `key_file.pattern` | `key_file.pattern` | | Куда логировать | `logging.channel` | `logging.channel` | `logging.category` | `logging.category` | ## Переменные окружения `Config::fromEnv()` читает `getenv()`, слитый с `$_SERVER` и `$_ENV`. Первый аргумент — свой массив вместо них, второй — другой префикс вместо `INDEXNOW_`. Пустые строки считаются незаданными. **Булевы значения разбираются, а не приводятся.** Каждая булева опция — `enabled`, `dry_run`, `strict_hosts`, `key_file.enabled`, `serve_key_file`, `collector.detect_leaks`, `normalizer.strip_tracking_params`, `normalizer.sort_query` — проходит через один и тот же разбор (`filter_var` с булевым фильтром) и в `fromEnv()`, и в `fromArray()`: строки `false`, `0`, `no`, `off` означают ложь, `true`, `1`, `yes`, `on` — истину. Пустая строка — «не задано» и даёт дефолт, не-скаляр — `ConfigurationException` с именем ключа. Это важно там, где адаптер отдаёт переменную окружения прямо в `fromArray()` без собственного приведения — так делает блок params Yii3, и обычный `(bool)` прочитал бы `INDEXNOW_DRY_RUN=false` как истину: продакшен молча не отправлял бы ничего, а `check` называл бы это осознанным выбором. | Переменная | Опция | |---|---| | `INDEXNOW_ENABLED` | `enabled` (любой булев литерал, который принимает `filter_var`) | | `INDEXNOW_KEY` | `key` | | `INDEXNOW_PREVIOUS_KEY` | `previous_key` | | `INDEXNOW_HOSTS` | `hosts` в виде `host=key,host2=key2`; `key_location`/`base_url` по хостам — только через `fromArray()` | | `INDEXNOW_STRICT_HOSTS` | `strict_hosts` | | `INDEXNOW_KEY_LOCATION` | `key_location` | | `INDEXNOW_BASE_URL` | `base_url` | | `INDEXNOW_ENGINES` | `engines`, через запятую (`api` или `yandex,bing`) | | `INDEXNOW_DISPATCH` | `dispatch` | | `INDEXNOW_BATCH_MAX_URLS` | `batch.max_urls` | | `INDEXNOW_DEBOUNCE_PER_URL` | `debounce.per_url` | | `INDEXNOW_THROTTLE_PER_MINUTE` | `throttle.max_requests_per_minute` | | `INDEXNOW_HTTP_TIMEOUT` | `http.timeout` | | `INDEXNOW_USER_AGENT` | `http.user_agent` | | `INDEXNOW_KEY_FILE_ENABLED` (`INDEXNOW_SERVE_KEY_FILE` всё ещё побеждает) | `key_file.enabled` | | `INDEXNOW_KEY_FILE_CACHE_MAX_AGE` | `key_file.cache_max_age` | | `INDEXNOW_DEBOUNCE_STORE` | `debounce.store` | | `INDEXNOW_HTTP_CLIENT` | `http.client` | | `INDEXNOW_DRY_RUN` | `dry_run` | | `INDEXNOW_ENV`, иначе `APP_ENV` | `environment` | | `INDEXNOW_PRODUCTION_ENVIRONMENTS` | `production_environments`, через запятую | | `INDEXNOW_MAX_URL_LENGTH` | `max_url_length` | | `INDEXNOW_LOG_URLS`, `INDEXNOW_FORBIDDEN_ESCALATION` | `logging.max_urls`, `logging.forbidden_escalation` | | `INDEXNOW_RETRY_MAX_ATTEMPTS`, `INDEXNOW_RETRY_BASE_DELAY`, `INDEXNOW_RETRY_MULTIPLIER`, `INDEXNOW_RETRY_MAX_DELAY`, `INDEXNOW_RETRY_SERVER_ERROR_DELAY` | `retry.*` | ## Хосты, ключи и `strict_hosts` Поддомены для IndexNow — отдельные хосты: каждому нужен свой файл ключа. Три схемы: - **Один сайт.** Задайте `key` и `base_url`. Каждый отправляемый хост использует этот ключ. - **Несколько сайтов, у каждого ключ.** Заполните `hosts`. Хосты вне карты всё ещё падают на `key`. - **Несколько сайтов и ничего лишнего.** Задайте `strict_hosts: true`. Ключ по умолчанию тогда применяется только к хосту `base_url`; URL любого другого неперечисленного хоста пропускаются с причиной `no_key`, а не объявляются под чужим ключом. Рекомендуется всегда, когда URL могут прийти из пользовательского ввода или мультитенантной базы. `hosts..key_location` переопределяет URL файла ключа только для этого хоста и должен быть на нём. `hosts..base_url` даёт хосту свою абсолютную базу для генерации URL вне запроса — у консольной команды или воркера очереди нет контекста запроса, и без него все сайты генерировались бы на едином `base_url`. `Config::baseUrlFor($host)` возвращает эту базу, падает на `base_url`, когда хост — базовый, и `null` иначе. Ключи перечисляются через `Config::$hosts`, `Config::$keyLocations` и `Config::$hostBaseUrls` (карты по хостам в нижнем регистре). Чтобы брать ключи из базы или реестра тенантов, реализуйте `Key\KeyProviderInterface`. ## Страховка dry-run `Config::fromArray()` сам включает `dry_run`, когда выполняется **всё** из: нет `key`, нет `hosts`, задан `environment`, и он не в `production_environments` (по умолчанию `Config::PRODUCTION_ENVIRONMENTS`). Разработчик, не задавший `INDEXNOW_KEY` локально, получает логирование вместо падения при загрузке и никогда не доходит до реального API. Обратный случай стоит алерта: `dry_run` включён при `environment` production — не отправляется ничего. `Config::isProduction()` сообщает об этом, а `Check\Checker` поднимает это как **ошибку**, а не warning. И зеркально: вне production настроенный ключ и незаданный `dry_run` — ошибка `check` (стейджинг отправлял бы боевые URL); явный `dry_run: false` говорит, что окружение отправляет нарочно, и даёт warning. ## Валидация Конструктор бросает `Exception\ConfigurationException` при: - `enabled` без `key`, `hosts` или `dry_run`; - `key` (или ключе хоста) вне `[A-Za-z0-9-]{8,128}`; - ключе `hosts`, который не голое имя хоста (есть схема, порт или путь); - `base_url`, который не абсолютный `http(s)` URL или несёт credentials; - `key_location`, который не абсолютный `http(s)` URL с путём или не на хосте `base_url` — движки принимают файл ключа только с отправляемого хоста; - `hosts..key_location` или `hosts..base_url`, указывающих на другой хост; - `batch.max_urls` вне `1..10000`, отрицательных `debounce.per_url` или `throttle.max_requests_per_minute`, `http.timeout` не больше нуля, пустом списке `engines`; - `dispatch`, который не короткий идентификатор, `http.user_agent` с переводами строк; - `strict_hosts` без единого известного хоста; - имени движка, которое не известный движок, не алиас и не `https` endpoint (простой `http` разрешён только на loopback-хостах, для mock-серверов). Каждое сообщение называет факт, допустимое и как исправить. `Config::fromArray()` дополнительно отклоняет нечисловые значения числовых опций вместо тихого дефолта. ## Производные конфигурации `with()` принимает имена аргументов конструктора и возвращает проверенную копию; неизвестное имя бросает исключение. Смена `dryRun` делает копию «явной» (`$dryRunExplicit`), прочие изменения флаг сохраняют. ```php $probe = $config->with(dryRun: false, engines: ['yandex']); $config->withDryRun(true); // сокращение $config->userAgent(); // действующая строка User-Agent $config->baseHost(); // хост base_url в нижнем регистре или null ``` ## Опечатки в конфигурации адаптера `Config::OPTIONS` перечисляет каждый ключ, который понимает `fromArray()`, в форме через точку. `Config::unknownOptions($data, $allowed)` возвращает ключи массива, которые ни опции ядра, ни в `$allowed`, так что адаптер предупреждает о `debounce.per_urls`, а не молча игнорирует. Вложенные ключи перечисляйте как `block.key`, никогда как голый `block`: голое имя останавливает проверку внутри блока. Адаптеры получают это через `Adapter\ConfigFactory::load()` (`ownedOptions:`), который также сливает дефолты адаптера, разрешает `dispatch: auto` и превращает невалидное значение в `critical`-строку лога и выключенный `Config` вместо исключения. ```php $unknown = Config::unknownOptions($userConfig, ['messenger', 'messenger.bus', 'doctrine.enabled']); if ($unknown !== []) { $logger->warning('indexnow: unknown option(s): {options}', ['options' => implode(', ', $unknown)]); } ``` Вложенные массивы проверяются на один уровень по пути через точку; `hosts` принимается всегда, потому что его ключи — имена хостов. Имя блока в `$allowed` (например `messenger`) разрешает весь блок, так что адаптер перечисляет либо имя блока, либо отдельные пути, которыми владеет. # Диагностика [English version](troubleshooting.md) Начните с `php artisan indexnow:check`, затем `php artisan indexnow:explain "App\Models\Post" `, затем канал лога на `debug`. ## Не отправляется ничего | Симптом | Причина | Исправление | |---|---|---| | `check`: `configuration: ...` и exit 1 | значение из `env()` невалидно; IndexNow работает выключенным | исправьте значение; точная ошибка напечатана | | `explain`: `when: published -> false` сразу после `create()` | у атрибута `when` дефолт только в базе, свежая модель его не имеет | `protected $attributes = ['published' => false]` на модели, или задайте явно | | `explain`: `no #[IndexNow] rule` | у модели нет атрибута и она не зарегистрирована | добавьте атрибут или `IndexNowKit::observe()` | | URL разрешены, POST нет | `dispatch: queue` и нет воркера | `php artisan queue:work` или `dispatch: sync` | | лог: `rule "..." ignores this update (fields ...)` | фильтр `fields` не совпал с изменёнными атрибутами | добавьте атрибут в `fields` или уберите фильтр | | лог: `Cannot generate route "posts.show": Missing required parameter` | `params` правила не совпадают с маршрутом | `params: ['post' => 'self']` для route model binding, либо назовите каждый параметр | | лог: `Cannot read "foo" on App\Models\Post: no method foo(), getFoo(), isFoo() or hasFoo(), no property "foo"…` | опечатка в аксессоре | исправьте аксессор; годятся атрибуты, casts, accessors, relations и методы | | массовый `update()` ничего не изменил в индексе | bulk-запросы не вызывают событий (A13) | `IndexNowKit::submitModels($query->get())` или `indexnow:submit-model` | | `attach()` на pivot ничего не изменил | операции с pivot не дают событий владельцу | `$touches = ['posts']` на связанной модели, правило без фильтра `fields` | ## Отправлено, но движок отвечает | Ответ | Смысл | Исправление | |---|---|---| | 403 (`invalid_key`, job failed) | `https:///.txt` недоступен или с другим телом | `indexnow:check`; CDN может кэшировать старый файл (`key_file.cache_max_age`) | | 422 (`unprocessable`) | URL другого хоста, чем `host`, или файл ключа на другом хосте | ключ на каждый хост (`hosts`), `strict_hosts: true` | | 429 (`rate_limited`) | слишком много запросов | job освобождается с `Retry-After`; понизьте `throttle.max_requests_per_minute` | | 202 (`pending`) | принято, проверка ключа в процессе | нормально для нового ключа; `check --live` позже ответит 200 | Счётчик 403, эскалирующий в `critical`, — на процесс: несколько воркеров считают каждый свои пять. ## Дубли, тайминги - Тот же URL не отправляется повторно в течение `debounce.per_url` (600 с). `--force` обходит это; `debounce.store: cache` делит окно между запросами и воркерами, `memory` — нет. - Всё из одного запроса уходит одним батчем на `terminating`; job, сохраняющий модели, сбрасывает после job. - Откатившаяся транзакция ничего не отправляет; откат savepoint внутри `DB::transaction()` отбрасывает только внутренние URL. ## Стейджинг отправил свои URL | Симптом | Причина | Исправление | |---|---|---| | Bing/Яндекс показывают URL `staging.example.com`, или в логе `failed` / `unprocessable` (422) по ним | стейджинг работает с боевым ключом и без `INDEXNOW_DRY_RUN`; URL сгенерированы на его хосте | вне production задайте `INDEXNOW_DRY_RUN=1` (или `INDEXNOW_ENABLED=0`); с core 0.6 `check` на такой копии падает | | стейджинг отдаёт боевой файл ключа | `key_file.enabled` включён везде | `key_file.enabled: false` вне production | | движки проиндексировали страницы стейджинга | хост стейджинга ответил `200` и отдал ключ | отдавайте `410` (или `noindex` + запрет в `robots.txt`) и ротируйте ключ, если он утёк | | preview-окружение должно отправлять нарочно | — | `INDEXNOW_DRY_RUN=0` явно; `check` тогда предупреждает, а не падает | ## Дубли с `memory` и несколькими воркерами | Симптом | Причина | Исправление | |---|---|---| | один и тот же URL отправляет каждый воркер | `debounce.store: memory` — на процесс | `debounce.store` = общий store (`cache`); `check` предупреждает о `memory` | | дубли сразу после сбоя кэша | store fails open | ожидаемо и ограничено; следите за warning `debounce store unavailable` | | дубли после деплоя | кэш сброшен или изменился `debounce.key_prefix` | безвредно один раз; держите префикс стабильным | ## Тестовые окружения Вне `production_environments` отсутствующий ключ включает `dry_run`: запросы логируются, не отправляются. `check` об этом предупреждает; в production `dry_run` — ошибка. Настроенный ключ без `INDEXNOW_DRY_RUN` вне production — ошибка `check` (стейджинг отправлял бы боевые URL). ## Где логи Канал `indexnow.logging.channel` (иначе канал по умолчанию). Уровни: успех `debug`, 202 `info`, 403/400 `error`, 422/429/5xx `warning`, невалидная конфигурация `critical`, молчаливые решения (`when` false, несовпадение `fields`) `debug`. Полный список — в [руководстве по эксплуатации ядра](../core/operations.md). # Диагностика [English version](troubleshooting.md) Начните отсюда: ```bash bin/console indexnow:check # конфигурация, файлы ключей, движки, проводка bin/console indexnow:explain 'App\Entity\Post' 42 # одна сущность, весь путь решения, ничего не отправляет ``` Вдвоём они закрывают большинство обращений без единой строки лога. ## Не отправляется ничего Идите по списку сверху вниз; каждый пункт называет улику, которая его подтверждает. | Причина | Улика | |---|---| | `enabled: false` | `skipped` / `disabled` и одна `info`-строка на вызов | | `dry_run: true` | `skipped` / `dry_run`, `info`-строка с полным телом запроса | | невалидная конфигурация выключила IndexNow | одна `critical`-строка при загрузке, `indexnow:check` завершается ненулевым кодом | | у сущности нет подходящего правила | `indexnow:explain` говорит, какое правило пропущено и почему | | коллектор ни разу не сброшен | `warning: N collected URL(s) discarded: the unit of work ended without flush()` | | хуки Doctrine не активны | `indexnow:check` печатает `doctrine: entity hooks are NOT active` | | Messenger настроен, воркер не запущен | панель профайлера показывает отправленные в очередь URL; в логе воркера пусто | **Невалидная конфигурация** — самая коварная. Плейсхолдер окружения нельзя проверить при компиляции, поэтому пустой `INDEXNOW_KEY` в production не ломает контейнер — он логирует ``` indexnow: invalid configuration, IndexNow is disabled until it is fixed: (run "bin/console indexnow:check") ``` на уровне `critical` и работает выключенным. Поставьте алерт на эту строку и включите `indexnow:check` в smoke-тесты деплоя. ## 403 — файл ключа Самая частая ошибка, и означает всегда одно: движок не смог проверить `https:///.txt`. ```bash bin/console indexnow:check --host www.example.com curl -i https://www.example.com/.txt ``` Ответ должен быть `200`, `text/plain`, тело — ровно ключ, **без редиректа**. Частые причины: - маршруты бандла не импортированы (без Flex): добавьте `@IndexNowKitBundle/config/routes.php` в `config/routes.yaml`; - `key_file.enabled: false` (или устаревший `serve_key_file: false`) без `key_location`; - CDN или reverse proxy отдаёт закэшированный старый файл после ротации ключа — `key_file.cache_max_age` по умолчанию 300 секунд именно поэтому; - catch-all маршрут или firewall перехватывает `/{key}.txt` раньше; - редиректы `http → https` или на `www` на файле ключа; - запрос пришёл на другой хост, чем тот, которому принадлежит ключ, — контроллер отвечает 404 намеренно. Пятый подряд 403 для хоста логируется один раз на `critical` с текстом *«submissions for this host are not being indexed»*. Это строка для пейджинга. Счётчик живёт в процессе: несколько воркеров считают каждый свои пять. ## 422 — unprocessable URL не принадлежат хосту, под которым отправлены, или `keyLocation` невалиден. На практике: - консольная команда или воркер сгенерировали URL на неверном хосте, потому что нет `base_url` для хоста — см. [multi-domain.md](multi-domain.md); - правило закрепило `host:`, ключ которого не настроен; - `key_location` указывает на другой хост. Слой конфигурации это отклоняет, так что такое бывает только с собственным `KeyProviderInterface`. `indexnow:explain` печатает хост и файл ключа для каждого URL, который отправил бы, — это решает вопрос сразу. ## 429 — rate limited Движок попросил сбавить темп. Результат `retryable`, `retryAfter` заполнен, если движок его назвал. - С Messenger транспорт повторяет и учитывает `Retry-After` на Symfony 7.2+. Делать ничего не нужно. - С `dispatch: sync` повторов нет: батч логируется и отбрасывается. Переходите на Messenger, если видите это регулярно. - Понизьте `throttle.max_requests_per_minute`, если причина — ваш собственный трафик. Он считается на процесс. - Обычный триггер — миграция или импорт, объявляющий десятки тысяч URL разом; делайте это порциями из воркера. ## Конкретный URL так и не отправляется ```bash bin/console indexnow:explain 'App\Entity\Post' 42 --event=updated ``` Вывод идёт по правилам по порядку и печатает для каждого подписку на событие, результат `when`, фильтр `fields` и разрешённые URL; затем для каждого URL — нормализованную форму, хост, замаскированный ключ, URL файла ключа и состояние дебаунса. Типичные находки: - **`when: false`** — сущность не опубликована, страницы нет. Правильное поведение. - **фильтр `fields`** — изменилось поле, которое правилу не важно. Добавьте его в `fields` или уберите `fields`, чтобы отправлять при любом изменении. - **не подписано** — `events` правила не включает это событие. - **`debounced`** — отправлено в последние `debounce.per_url` секунд. `--force` у `indexnow:submit` обходит это. - **`no key for host`** — хост не в `hosts` и не хост `base_url`. - **ошибки правила** — отсутствующий аксессор или маршрут, который не генерируется, печатаются красным и логируются на `error`. ## Публикация и снятие ведут себя странно `when` вычисляется до и после изменения. `true → false` отправляет удаление, чтобы движки переобошли 404; `false → true` — создание. Две вещи для проверки: - **Поле за `when` должно быть в change set.** `when: 'isPublished'` сопоставляется полю `published` по конвенции (`isPublished` → `published` → `is_published`, `getStatus` → `status`). Когда имя аксессора не связано с колонкой, назовите поля явно: `whenFields: ['status', 'visibleFrom']`. - **Удаление черновика ничего не отправляет** — намеренно. Страница никогда не была публичной. ## Панель профайлера `enabled`, `dry_run`, `dispatch` и `strict_hosts` наверху говорят о режиме. Затем: - **Collected** — URL, которые запрос буферизовал. Ноль здесь значит, что ни одно правило ничего не дало: проблема выше, в правилах или guard'ах. - **Results** — что реально отправлено, по движку и хосту, с HTTP-кодом и `reason`. Заполняется и при синхронной доставке, которая происходит после ответа. - С Messenger таблица результатов пуста намеренно, панель говорит, что HTTP-результаты в логе воркера. - **Key files** — URL, который должен отдавать каждый управляемый хост, с замаскированным ключом. Откройте один в браузере как первую проверку. - **base_url** — помечен, если не задан: тогда относительные URL и генерация в консоли или воркере не работают. ## Логи Всё идёт в Monolog-канал `indexnow`, каждое сообщение начинается с `indexnow: `. ```yaml # config/packages/dev/monolog.yaml monolog: handlers: indexnow: type: stream path: '%kernel.logs_dir%/indexnow.log' level: debug channels: ['indexnow'] ``` На время диагностики ставьте `debug`: причина, по которой правило решило **не** давать URL, логируется только на этом уровне. Все тексты и уровни — в [руководстве по эксплуатации](../core/operations.md). Ключи замаскированы в каждой строке, включая тела ответов и сообщения исключений, — эти файлы безопасно прикладывать к багрепорту. ## Стейджинг отправил свои URL | Симптом | Причина | Исправление | |---|---|---| | Bing/Яндекс показывают URL `staging.example.com`, или в логе `failed` / `unprocessable` (422) по ним | стейджинг работает с боевым ключом и без `dry_run`; URL сгенерированы на его хосте | вне production задайте `INDEXNOW_DRY_RUN=1` (или `INDEXNOW_ENABLED=0`); с core 0.6 `check` на такой копии падает | | стейджинг отдаёт боевой файл ключа | `key_file.enabled` включён везде | `key_file.enabled: false` вне production — ни один движок не сможет проверить ключ на этом хосте | | движки проиндексировали страницы стейджинга | хост стейджинга ответил на них `200` и отдал ключ | отдавайте на стейджинге `410` (или `noindex` + запрет в `robots.txt`) и ротируйте ключ, если он утёк | | preview-окружение должно отправлять нарочно | — | задайте там явный `dry_run: false`; `check` тогда предупреждает, а не падает | ## Дубли с `memory` и несколькими воркерами | Симптом | Причина | Исправление | |---|---|---| | один и тот же URL отправляет каждый воркер в течение минут | `debounce.store: memory` — на процесс; у каждого веб- и queue-воркера своё окно | `debounce.store` = общий кэш; `check` предупреждает о `memory` | | дубли сразу после сбоя кэша | store fails open: пока кэш лежит, дедупликации нет | ожидаемо и ограничено (один запрос на URL); следите за частотой warning `debounce store unavailable` | | дубли после деплоя | общий кэш сброшен или изменился `debounce.key_prefix` | безвредно один раз; держите префикс стабильным для приложения | ## Дубли после сбоя кэша Debounce store fails open. Если Redis или пул кэша недоступен, дедупликация пропускается и пишется warning (`debounce store unavailable, submitting without de-duplication`). Видимый симптом — всплеск повторных отправок, а не потерянные, и это правильный размен: дубль стоит один запрос, пропуск оставляет устаревший контент в индексе. `indexnow:check` читает тестовый ключ через настроенный пул и печатает `debounce: 600s per URL, shared through cache pool "cache.app" (RedisAdapter)` либо ошибку с именем пула, если он непригоден; `debounce.store: memory` помечается как «на процесс». ## `indexnow:sitemap` в read-only контейнере `SitemapReader` держит каждый документ во временном файле во время разбора. При `readOnlyRootFilesystem` и без записываемого `/tmp` дефолтный `sitemap.spool: auto` падает на память и пишет один warning на прогон; память ограничена `sitemap.max_bytes` на документ (50 МиБ), так что ничего не ломается, просто стоит RAM. Смонтируйте `emptyDir` на `/tmp` (или задайте `TMPDIR` / `sitemap.spool_dir` на записываемый том), чтобы вернуть временный файл, или задайте `spool: memory` явно. `indexnow:check` печатает, куда spool'ятся документы и почему temp dir непригоден. Sitemap, обрывающийся на середине («ends early (truncated download or broken sitemap)»), или потерянное соединение повторяются `sitemap.fetch_retries` раз; если всё равно не вышло, прочитанные URL отправляются, команда завершается с 1, и повторный запуск безопасен (IndexNow идемпотентен, debounce store поглощает повторы). Для ежедневного cron с `--changed-since` берите окно шире интервала (`"2 days"`), чтобы упавший прогон не оставил дыры. ## Как просить помощи Приложите вывод `bin/console indexnow:check`, релевантные строки лога `indexnow` и вывод `bin/console indexnow:explain` для одной затронутой сущности. Эти три закрывают почти каждый вопрос мейнтейнера. # Диагностика [English version](troubleshooting.md) Начните с `php yii indexnow/check`, затем `php yii indexnow/explain 'app\models\Post' `, затем категория лога `indexnow` на `debug` (`log.targets[].categories = ['indexnow']`, `levels = ['error', 'warning', 'info', 'trace']`). ## Не отправляется ничего | Симптом | Причина | Исправление | |---|---|---| | `check`: `configuration: ...` и exit 1 | значение в `options` (обычно из `getenv()`) невалидно; IndexNow работает выключенным | исправьте значение; точная ошибка напечатана и залогирована один раз на `critical` | | лог: `unknown option(s) in the indexnow configuration: ...` | опечатка в `options` (`debounce.per_urls`, `key_file.enabld`) | путь через точку называет ключ | | `explain`: `when: published -> false` сразу после `save()` | у атрибута `when` дефолт только в базе | задайте его до `save()` или вызовите `loadDefaultValues()` в `init()` | | `explain`: `no #[IndexNow] rule` | у класса нет атрибута и он не зарегистрирован | добавьте `#[IndexNow]`, `IndexNowBehavior`, `active_record.models` или `observe()` | | `explain` даёт URL, лог на save молчит | behavior не подключён, либо `active_record.enabled` / `enabled` выключены | `check` печатает `active_record: ... hooked` или причину | | `debug`-лог: `did not land on commit` | изменение откатилось savepoint'ом, или verifier не увидел строку | ожидаемо для отката; про verifier см. [commit-safety.md](commit-safety.md) | | `debug`-лог: `debounced` | URL отправлен в пределах `debounce.per_url` | `--force` у команды, или уменьшите окно | | `warning`: `skipping ... unmanaged host` | хост URL — ни `base_url`, ни в `hosts` | добавьте хост в `hosts` или исправьте `base_url` | | консоль: `base_url is not set` | URL относительные, а запроса нет | задайте `base_url`; обязателен при `dispatch: queue` | ## Файл ключа | Симптом | Причина | Исправление | |---|---|---| | `GET /.txt` — 404 | pretty URL выключены, или `key_file.enabled` false, или ключ другой | `check` печатает правило; `urlManager.enablePrettyUrl = true` | | движки отвечают 403 | отдаваемое тело — не отправляемый ключ, редирект, или закэшированный старый файл после ротации | `curl -i https://host/.txt`; `key_file.cache_max_age` — 300 с нарочно | | `check`: `key file ... returned 200`, но 403 остаётся | `hosts` и отправляемый хост различаются (www vs apex) | перечислите каждый отправляемый хост в `hosts`, включите `strict_hosts` | ## Очередь | Симптом | Причина | Исправление | |---|---|---| | `check`: `queue component "queue" is not configured` | `dispatch: queue` (или `auto`, разрешённый в него) без yii2-queue | установите `yiisoft/yii2-queue` и настройте компонент, либо `dispatch: sync` | | попытки 429/5xx идут подряд без задержки | `yii\queue\sync\Queue` игнорирует задержку повторного push | реальный драйвер (db, redis, amqp); `check` предупреждает о `SyncQueue` | | `attempts` очереди не ограничивает 429/5xx | job перепушивает их сам, до `retry.max_attempts` | задайте `retry.max_attempts` (и задержки `retry.*`) в опциях компонента, см. [queue.md](queue.md) | | `ttr` превышен на больших батчах | `queue.ttr` (300 с) мал для числа батчей | поднимите `queue.ttr` или понизьте `batch.max_urls` | ## Sitemap `php yii indexnow/sitemap --dry-run` перечисляет, что было бы отправлено; `sitemap.enabled is false.` значит, что блок выключен или невалиден (причина в логе). `check` печатает, куда spool'ятся документы; на read-only ФС задайте `sitemap.spool_dir` или `sitemap.spool: memory`. Ридер — в [`indexnowkit/sitemap`](../sitemap/index.md). ## Отправлено, но движок отвечает | Ответ | Смысл | Исправление | |---|---|---| | 403 (`invalid_key`, job rejected permanently) | `https:///.txt` недоступен или с другим телом | `indexnow/check`; CDN может кэшировать старый файл (`key_file.cache_max_age`) | | 422 (`unprocessable`) | URL другого хоста, чем `host`, или файл ключа на другом хосте | ключ на каждый хост (`hosts`), `strict_hosts: true`; консольным URL нужен `base_url` на нужном хосте | | 429 (`rate_limited`) | слишком много запросов | job перепушивает себя с `Retry-After`; понизьте `throttle.max_requests_per_minute` | | 202 (`pending`) | принято, проверка ключа в процессе | нормально для нового ключа; `check --live` позже ответит 200 | Счётчик 403, эскалирующий в `critical`, — на процесс: несколько воркеров считают каждый свои пять. ## Дубли, тайминги - Тот же URL не отправляется повторно в течение `debounce.per_url` (600 с). `--force` обходит это; `debounce.store: cache` делит окно между запросами и воркерами, `memory` — нет. - Всё из одного запроса уходит одним батчем после ответа (`Response::EVENT_AFTER_SEND`); консольная команда сбрасывает при завершении, воркер очереди — после каждого job. - Откатившаяся транзакция ничего не отправляет; изменение, перечитанное на commit и не подтверждённое строкой, отбрасывается ([commit-safety.md](commit-safety.md)). ## Стейджинг отправил свои URL | Симптом | Причина | Исправление | |---|---|---| | Bing/Яндекс показывают URL `staging.example.com`, или в логе `failed` / `unprocessable` (422) по ним | стейджинг работает с боевым ключом и без `dry_run`; URL сгенерированы на его хосте | вне production задайте `'dry_run' => true` (или `'enabled' => false`); с core 0.6 `check` на такой копии падает | | стейджинг отдаёт боевой файл ключа | `key_file.enabled` включён везде | `key_file.enabled: false` вне production | | движки проиндексировали страницы стейджинга | хост стейджинга ответил `200` и отдал ключ | отдавайте `410` (или `noindex` + запрет в `robots.txt`) и ротируйте ключ, если он утёк | | preview-окружение должно отправлять нарочно | — | явный `'dry_run' => false`; `check` тогда предупреждает, а не падает | ## Дубли с `memory` и несколькими воркерами | Симптом | Причина | Исправление | |---|---|---| | один и тот же URL отправляет каждый воркер | `debounce.store: memory` — на процесс | `debounce.store` = компонент кэша (`cache`); `check` предупреждает о `memory` | | дубли сразу после сбоя кэша | store fails open | ожидаемо и ограничено; следите за warning `debounce store unavailable` | | дубли после деплоя | кэш сброшен или изменился `debounce.key_prefix` | безвредно один раз; держите префикс стабильным | # Диагностика [English version](troubleshooting.md) Начните с `./yii indexnow:check`, затем `./yii indexnow:explain 'App\Model\Post' `, затем категория лога `indexnow` на уровне `debug` (target yiisoft/log с `categories: ['indexnow']`, `levels: ['error', 'warning', 'info', 'debug']`). ## Ничего не отправляется | Симптом | Причина | Что делать | |---|---|---| | `check`: `configuration: ...` и код возврата 1 | значение блока (обычно из `.env`) недопустимо; IndexNow работает выключенным | исправьте значение; точная ошибка печатается и один раз логируется на уровне `critical` | | лог: `unknown option(s) in the indexnow configuration: ...` | опечатка в блоке (`debounce.per_urls`, `key_file.enabld`) | путь через точку называет ключ | | `check`: `active record: the observer is not installed` | `config/bootstrap.php` пакета не выполнился | раннер должен грузить группу `bootstrap` (`bootstrap-web` / `bootstrap-console` ссылаются на `$bootstrap` в шаблоне) | | PHP warning `a record with #[IndexNowEvents] was saved but the observer is not installed` | то же самое, увиденное со стороны сохранения | то же самое | | `explain` отдаёт URL, а в логе на сохранении тишина | у записи нет `EventsTrait` (событий нет вовсе) либо `active_record.enabled` / `enabled` равно false | `use EventsTrait;` на записи; `check` печатает строку хука | | `explain`: `when: published -> false` сразу после `save()` | у колонки `when` есть только умолчание в базе | задайте умолчание типизированному свойству или установите его до `save()` | | `explain`: `no #[IndexNow] rule` | у класса нет атрибута и он не зарегистрирован | добавьте `#[IndexNow]`, `active_record.models` или `observe()` | | `explain` отдаёт URL, а `via:` или путь через точку по отношению даёт ошибку | у записи нет `MagicRelationsTrait` | `use MagicRelationsTrait;` рядом с `EventsTrait`: отношения читаются через него | | лог `debug`: `change not committed` | изменение откатили либо проверка не увидела строку | для отката это ожидаемо; про проблему проверки — [commit-safety.md](commit-safety.md) | | лог `warning`: `staged URL(s) wait for a transaction that is still open` | flush произошёл внутри транзакции; проверка прочитала бы незакоммиченные данные | закоммитьте до отправки ответа либо делайте flush после коммита в длинной команде. URL уходят на первом flush после конца транзакции; запрос, который закончился внутри неё, до такого flush не доживает | | лог `warning`: `dropping ... staged URL(s) of a connection whose transaction was still open` | одно и то же соединение было внутри транзакции на трёх flush подряд (воркер, который её не закрывает) | закройте транзакцию; URL отбрасываются, а не переносятся в чужой запрос | | лог `debug`: `debounced` | URL уже отправляли внутри окна `debounce.per_url` | `--force` у команды либо уменьшите окно | | `warning`: `skipping ... unmanaged host` | хост URL не совпадает ни с `base_url`, ни с `hosts` | добавьте хост в `hosts` либо исправьте `base_url` | | консоль: `set base_url` в `ConfigurationException` | URL относительные, а запроса нет | задайте `base_url` | | `Cannot generate route "post/view"` | имя маршрута неизвестно роутеру либо не хватает аргумента | `->name('post/view')` на маршруте; `params` должны покрывать его аргументы | ## Файл ключа | Симптом | Причина | Что делать | |---|---|---| | `GET /.txt` отвечает 404 | `key_file.enabled` равно false, группа `routes` пакета не смёржена, либо ключ другой | `check` печатает строку маршрута; `key_file.pattern` должен заканчиваться на `.txt` и содержать аргумент `key` | | движки отвечают 403 | отдаётся не тот ключ, редирект либо закешированный старый файл после ротации | `curl -i https://host/.txt`; `key_file.cache_max_age` равен 300 с намеренно | | `check`: `key file ... returned 200`, но 403 остаётся | `hosts` и хост отправки различаются (www против apex) | перечислите в `hosts` каждый хост, под которым отправляете, включите `strict_hosts` | | за прокси отдаётся файл ключа только одного хоста, остальные отвечают 404 | до origin каждый запрос доходит с `Host` прокси | middleware доверенных прокси перед роутером ([multi-domain.md](multi-domain.md#behind-a-proxy-or-a-cdn)) | ## Dispatch | Симптом | Причина | Что делать | |---|---|---| | `check`: `"dispatch" must be one of sync, none` | `dispatch: queue` из конфигурации другого адаптера | `sync` плюс определение `DispatcherInterface` для вашей очереди ([extending.md](extending.md)) | | pre-flight пакета verify замедляет запросы | `verify.enabled` вместе с `dispatch: sync` тянет страницы после ответа | это ожидаемо; диспетчер очереди переносит работу в воркер | ## Sitemap `./yii indexnow:sitemap --dry-run` печатает, что было бы отправлено; `sitemap.enabled is false.` означает, что блок выключен или недопустим (причина в логе). `check` печатает, куда спулятся документы; на файловой системе только для чтения задайте `sitemap.spool_dir` либо `sitemap.spool: memory`. Сам ридер живёт в [`indexnowkit/sitemap`](../sitemap/index.md). ## Отправили, а движок отвечает | Ответ | Смысл | Что делать | |---|---|---| | 403 (`invalid_key`) | `https:///.txt` недоступен или отдаёт другое тело | `indexnow:check`; CDN может кешировать старый файл (`key_file.cache_max_age`) | | 422 (`unprocessable`) | URL хоста, отличного от `host`, либо файл ключа на другом хосте | один ключ на хост (`hosts`), `strict_hosts: true`; консольным URL нужен `base_url` нужного хоста | | 429 (`rate_limited`) | слишком много запросов | уменьшите `throttle.max_requests_per_minute`; диспетчер очереди повторяет с учётом `Retry-After` | | 202 (`pending`) | принято, проверка ключа впереди | нормально для нового ключа; `check --live` позже ответит 200 | Счётчик 403, эскалирующий до `critical`, общий через кэш за `debounce.store`; с `memory` он живёт в одном процессе. ## Дубли и тайминг - Один и тот же URL не отправляется повторно внутри `debounce.per_url` (600 с). `--force` обходит окно; PSR-16-кэш контейнера (умолчание `debounce.store`) делит окно между запросами и воркерами, `memory` — нет. - Всё, что собрано за один запрос, уходит одним батчем после ответа (`AfterEmit`); консольная команда делает flush, когда заканчивается (`ApplicationShutdown`). - Откаченная транзакция не отправляет ничего; изменение, перечитанное на flush и не найденное в строке, отбрасывается ([commit-safety.md](commit-safety.md)). ## Staging отправил свои URL | Симптом | Причина | Что делать | |---|---|---| | Bing/Yandex сообщают про URL `staging.example.com` либо `failed` / `unprocessable` (422) по ним в логе | staging-копия работает с продовым ключом и без `dry_run`; её URL сгенерированы на её собственном хосте | вне production задайте `INDEXNOW_DRY_RUN=1` (или `enabled: false`); `check` на такой копии падает | | staging-хост отдаёт продовый файл ключа | `key_file.enabled` включён везде | `key_file.enabled: false` вне production, чтобы ни один движок не смог проверить ключ на этом хосте | | движки проиндексировали staging-страницы | staging-хост ответил `200` по ним и отдал ключ | отвечайте `410` (или `noindex` плюс запрет в `robots.txt`) на staging и ротируйте ключ, если он засветился | | preview-окружение должно отправлять намеренно | — | скажите `dry_run: false` явно в этом окружении; тогда `check` предупреждает, а не падает |