Google Indexing API for Magento 2 - installation and configuration
1. Important information before implementation
The module integrates Magento 2 with Google Indexing API and lets you add URLs to a central submission queue. The queue is processed by Magento cron, and each submission is validated, deduplicated, limited, and logged.
According to Google documentation, Indexing API is officially intended primarily for pages with structured data:
JobPosting,BroadcastEventembedded inVideoObject.
Using the API for products, categories, CMS pages, or blog posts does not guarantee indexing or rankings in search results. The module should be treated as a technical tool for submitting URLs, not as a replacement for XML sitemap, proper canonicals, robots, hreflang, internal linking, and overall SEO quality.
Official Google materials:
- https://developers.google.com/search/apis/indexing-api/v3/quickstart
- https://developers.google.com/search/apis/indexing-api/v3/prereqs
- https://developers.google.com/search/apis/indexing-api/v3/using-api
- https://developers.google.com/search/apis/indexing-api/v3/quota-pricing
2. Requirements
Before installation, make sure the environment meets the requirements:
- Magento 2.4.x,
- PHP 8.1 or newer,
- Composer,
- a working Magento cron,
- the ability to install the
google/apiclientpackage, - administrative access to Magento,
- a Google Cloud project with Indexing API enabled,
- a Google Search Console property verified for the store domain,
- a Google service account added as an owner in Google Search Console.
The module requires the package:
google/apiclient:^2.16The package is declared in the module composer.json, so Composer should install it automatically.
3. Module installation
3.1. Installation via Composer from a VCS repository
If the module is installed from a private or public Git repository, add the repository to the Magento project:
composer config repositories.kowal.google.indexing.api vcs https://github.com/kowalco/google-indexing-apiIf the repository is private, configure the GitHub token:
composer config --global --auth github-oauth.github.com Install the package:
composer require kowal/module-google-indexing-apiEnable the module:
bin/magento module:enable Kowal_GoogleIndexingApiRun the database schema update:
bin/magento setup:upgradeFlush cache:
bin/magento cache:flushIn production mode also run:
bin/magento setup:di:compilebin/magento setup:static-content:deploybin/magento cache:flush3.2. Local installation in app/code
If the module is installed without Composer as local code, place it in the directory:
app/code/Kowal/GoogleIndexingApiThen install the Google API Client dependency in the Magento project:
composer require google/apiclient:^2.16Enable the module and run the standard Magento commands:
bin/magento module:enable Kowal_GoogleIndexingApibin/magento setup:upgradebin/magento cache:flushFor production:
bin/magento setup:di:compilebin/magento setup:static-content:deploybin/magento cache:flush3.3. Installation verification
Check whether the module is active:
bin/magento module:status Kowal_GoogleIndexingApiAfter successful installation, the following should be available in the admin panel:
Stores > Configuration > Kowal > Google Indexing API,- admin menu
Google Indexing API > Indexing Queue, Google Indexing API > Import URLs,Google Indexing API > API Logs,Google Indexing API > Setup Assistant.
The following tables should be created in the database:
kowal_google_indexing_queue,kowal_google_indexing_api_log.
4. Preparing Google Cloud and Search Console
4.1. Creating a Google Cloud project
- Go to Google Cloud Console.
- Create a new project or select an existing project used for the store.
- Enable the API:
Indexing APIWithout the API enabled, the module will not be able to send submissions correctly.
4.2. Creating a service account
- In Google Cloud, go to
IAM & Admin > Service Accounts. - Create a new service account.
- Generate a key in JSON format.
- Download the JSON file and store it securely.
The module requires the JSON to contain at least the fields:
type,project_id,private_key,client_email.
The type field must have the value:
service_account4.3. Adding the service account as an owner in Search Console
- Open Google Search Console.
- Select the property corresponding to the store domain.
- Make sure the property is verified.
- Add the
client_emailaddress from the JSON file as a property owner.
Example service account address:
my-service-account@project-name.iam.gserviceaccount.comIf the service account is not an owner of the Search Console property, Google may return permission errors, for example a lack of URL ownership confirmation.
5. Module configuration in Magento
The configuration is located at:
Stores > Configuration > Kowal > Google Indexing APIThe configuration supports Magento scopes:
- Default Config,
- Website,
- Store View.
This makes it possible to have separate settings for different stores or store views if the project requires it.
6. General section
Enable
Default:
NoEnables or disables the module.
When the field value is No, cron does not process the queue. Addresses may exist in the database, but the queue processor will not send them to Google.
Recommendation:
- during the first configuration, set
Noor leaveDry Run = Yes, - after successful tests, set
Yes.
Dry Run
Default:
YesTest mode. When Dry Run is enabled, the module processes queue records, saves statuses and logs, but does not send a real submission to Google.
This is the safest mode for the first launch, configuration testing, and checking whether URLs are entering the queue as expected.
Recommendation:
- always perform the first tests with
Dry Run = Yes, - disable
Dry Runonly after checking credentials, allowed hosts, the queue, and logs.
7. Google Access Credentials section
Credentials Source
Default:
Encrypted configuration valueDetermines where the module retrieves the Google service account JSON from.
Available options:
| Option | Technical value | Description |
|---|---|---|
Encrypted configuration value | config | The JSON is pasted into the Magento configuration and stored as an encrypted sensitive config value. |
Uploaded JSON file | file | The JSON is uploaded as a file and stored outside the pub directory, in var/google-indexing. |
Recommendation:
- for simple implementations, an encrypted configuration value can be used,
- for environments with file access control, uploading a JSON file may be more convenient.
Service Account JSON
Visible when Credentials Source = Encrypted configuration value.
Paste the full contents of the Google service account JSON file into this field.
The module validates the JSON before saving. It checks:
- correct JSON format,
- presence of the fields
type,project_id,private_key,client_email, - the value
type = service_account.
The value is stored as encrypted Magento configuration.
Service Account JSON File
Visible when Credentials Source = Uploaded JSON file.
Allows you to upload the Google service account JSON file.
The module:
- accepts only files with the
.jsonextension, - validates the file contents,
- checks the required fields
type,project_id,private_key,client_email, - saves the file outside the public directory, in
var/google-indexing, - tries to set file permissions to
0600if the file driver allows it.
The file is saved with a name that depends on the configuration scope, for example for the global scope:
var/google-indexing/service-account-default-0.jsonGoogle Cloud Project ID
Text field for the Google Cloud project identifier.
In the current module implementation, the main authorization is based on the data from the service account JSON. The Google Cloud Project ID field has an informational role and helps organize the configuration, especially when the store uses multiple environments or several Google Cloud projects.
Recommendation:
- enter the
project_idvalue from the JSON file, - use separate Google Cloud projects for production and test environments if such separation is adopted in the project.
8. Queue and Limits section
Daily Publish Limit
Default:
200Defines the maximum number of publish submissions the module can perform during the day.
The limiter counts request types:
publish,publish_dry_run.
If the limit is used up, the queue processor will not fetch additional records for sending until the next daily window.
Recommendation:
- leave
200if the project uses the default Google onboarding limit, - increase the value only when the Google Cloud project has an approved higher limit,
- setting
0blocks sending because the available number of slots will be0.
Requests Per Minute Limit
Default:
60Defines the maximum number of publish requests per minute.
The module compares this value with the number of requests recorded in the logs during the last minute. If the per-minute limit is used up, cron will not process additional records in that run.
Recommendation:
- for a typical implementation, keep the default value,
- reduce the value if you want to load the API more conservatively,
- do not set
0unless you want to temporarily stop sending.
Cron Batch Size
Default:
20Defines the maximum number of queue records processed in one cron run.
The actual number of processed records is also limited by:
Daily Publish Limit,Requests Per Minute Limit,- the number of records ready to be sent,
- the status and date of
scheduled_at.
Recommendation:
20is a safe starting value,- for large queues, the value can be increased, but only with Google limits in mind.
Default Indexing Lag minutes
Default:
15Defines the default delay between adding a URL to the queue and the moment when it can be sent.
The delay helps:
- reduce duplicates,
- avoid sending after every minor change,
- give the administrator time to correct content,
- manage the API limit better.
In the current implementation, this setting is used when the submission does not have its own delay.
Manual Form Indexing Lag minutes
Default:
15Defines the delay for URLs added through the form:
Google Indexing API > Import URLsIf the administrator pastes a list of URLs manually, each valid address will be scheduled with this delay.
Recommendation:
- set
0if manual import should enter the queue immediately, - leave
15if you want to keep a buffer for deduplication and submission control.
Mass Action Indexing Lag minutes
Default:
15Defines the delay for URLs added through:
- product mass actions,
- CMS page mass actions,
- indexing submission buttons on the product, category, and CMS page edit screens.
Recommendation:
- for bulk operations, leave the value at
15or higher, - for small stores and manual admin work, a lower value may be considered.
Max Attempts
Default:
5Defines the maximum number of attempts to send one queue record.
If Google returns a temporary error, the module will set the status to retry as long as the number of attempts is lower than Max Attempts. After the limit is exceeded, the record will receive the failed_permanent status.
Errors treated as temporary:
408,409,412,429,500,502,503,504.
Retry Delay minutes
Default:
15Base delay before the next send attempt after a temporary error.
Cron uses an increasing delay. The multiplier depends on the number of attempts and is capped at 24. This prevents subsequent attempts from being made too aggressively.
Example for the value 15:
| Attempt | Approximate delay |
|---|---|
| 1 | 15 minutes |
| 2 | 30 minutes |
| 3 | 60 minutes |
| 4 | 120 minutes |
Allowed Source Types
Default:
manual,product,category,cms_page,amasty_blog_postComma-separated list of allowed source types.
Supported values:
| Value | Meaning |
|---|---|
manual | URL added manually through the import form. |
product | Product URL. |
category | Category URL. |
cms_page | CMS page URL. |
amasty_blog_post | Amasty Blog post URL if a separate integration module is used. |
If the source type is not on the list, the scheduler will skip the submission and mark it as skipped.
Recommendation:
- leave the default values if the store uses all standard sources,
- remove the sources you do not want to allow into the queue.
Allowed URL Hosts
Default:
emptyComma-separated list of allowed URL hosts.
Example:
example.com,www.example.comIf the list is filled in, the module will accept only addresses belonging to the specified hosts. If a URL has a different host, validation will return the error:
host_not_allowedRecommendation:
- in production, always fill in this list,
- add all hosts used by the store, for example the main domain, the
wwwversion, store view domains, and language domains, - do not add test domains to the production configuration.
Require HTTPS URLs
Default:
YesRequires submitted URLs to use the https scheme.
If the field is enabled, an address with http will be rejected with the error:
https_requiredRecommendation:
- for production stores, leave
Yes, - use
Noonly in exceptional test environments.
9. Auto-Indexing section
Enable Auto-Indexing
Default:
NoThis field is prepared for automatic integrations of entity save or delete events and additional URL providers.
In the current module scope, manual and administrative mechanisms for adding URLs to the queue are available, including URL import, Mass Actions, and buttons on edit forms.
Recommendation:
- leave
Noif auto-indexing has not been implemented in the project, - enable it only when the project includes support for automatic entity events using this configuration.
10. Logs section
API Log Retention Days
Default:
90Defines the number of days API logs are retained.
The log cleanup cron runs daily at:
03:15It removes entries older than the number of days set in this field.
Recommendation:
90days is a reasonable diagnostic value,- with a large number of requests, retention can be reduced,
- for SEO audits, retention can be increased, keeping the log table size in mind.
11. Installation and configuration assistant
The assistant is located at:
Google Indexing API > Setup AssistantIts purpose is to quickly verify whether the Magento and Google configuration is ready for the first test.
11.1. Current Status section
The assistant shows the current state of the most important elements:
| Field | Meaning |
|---|---|
Module | Indicates whether the module is enabled in the configuration. |
Dry Run | Indicates whether test mode without real sending to Google is active. |
Credentials | Shows whether the module can read and parse Google credentials. |
Service Account Email | Displays the client_email from the service account JSON. This address should be added as an owner in Search Console. |
Allowed Hosts | Shows the list of hosts allowed in the configuration. |
Queue | Shows the number of records in statuses scheduled, pending, retry, and failed_permanent. |
If Credentials has the status Missing or invalid, go back to the configuration and correct the JSON or credentials file.
If Allowed Hosts shows Not configured, the module does not restrict hosts. Technically this may work, but in production it is recommended to explicitly enter the store hosts.
11.2. Setup Steps section
The assistant displays a list of steps needed before the first real request:
- Create or select a Google Cloud project.
- Enable Indexing API and create a JSON key for the service account.
- Paste the JSON or upload the JSON file in the Magento configuration.
- Add the service account email as an owner in Google Search Console.
- Run tests, then import one URL with
Dry Runenabled.
The assistant contains links to:
- Google Cloud for Indexing API,
- the module configuration in Magento,
- Google Search Console.
11.3. Test Google Credentials
Button:
Test Google Credentialschecks whether Magento can use the service account data to obtain an OAuth token for the scope:
https://www.googleapis.com/auth/indexingA positive result means that:
- the JSON is valid,
- the private key can be used,
- Google issued an OAuth token.
A negative result may indicate:
- invalid JSON,
- an incorrect or damaged
private_key, - a missing required field in the JSON,
- a communication issue with Google,
- use of a key that is not a service account key.
This test does not yet confirm that the service account has owner access to the domain in Search Console. For that, a URL metadata test or a real submission of a test URL is required.
11.4. Test URL Metadata
Form:
Test URL Metadatalets you enter a public URL from an allowed host and perform a metadata request to Google Indexing API.
Before sending the request, the module:
- normalizes the URL,
- checks whether the URL is absolute,
- checks the
httporhttpsscheme, - when
Require HTTPS URLsis enabled, requireshttps, - checks
Allowed URL Hostsif they have been configured.
Possible results:
| Result | Meaning |
|---|---|
| HTTP 2xx success | Google returned metadata for the URL. |
| HTTP 404 | Often means the URL has not yet had a previous successful submission through Indexing API. This does not necessarily mean the configuration is incorrect. |
| Validation error before the request | The URL does not meet the module conditions, for example wrong host, missing HTTPS, or a non-absolute address. |
| HTTP error other than 404 | You should check the Google message, Search Console permissions, credentials, and limits. |
The metadata test does not create a new publish submission. It is used for connection and URL status diagnostics.
12. First launch step by step
Recommended order for the first launch:
- Install the module and run
setup:upgrade. - In Magento, go to
Stores > Configuration > Kowal > Google Indexing API. - Set
Enable = Yes. - Leave
Dry Run = Yes. - Select the credentials source.
- Paste the service account JSON or upload the JSON file.
- Fill in
Google Cloud Project IDwith theproject_idvalue from the JSON. - Fill in
Allowed URL Hosts, for exampleexample.com,www.example.com. - Leave the default limits if Google has not approved higher limits.
- Save the configuration and flush cache.
- Go to
Google Indexing API > Setup Assistant. - Check whether the assistant shows the credentials as ready.
- Click
Test Google Credentials. - Add the service account email as an owner in Google Search Console if this has not already been done.
- Perform
Test URL Metadatafor a public URL from an allowed host. - Go to
Google Indexing API > Import URLs. - Add one test URL with the action
URL_UPDATED. - Run Magento cron:
bin/magento cron:run- Check
Google Indexing API > Indexing Queue. - Check
Google Indexing API > API Logs. - If everything works correctly, disable
Dry Run. - Add one test URL again and check the real Google response.
13. Importing URLs from the admin panel
Manual import is located at:
Google Indexing API > Import URLsThe form contains the fields:
| Field | Description |
|---|---|
Action | Type of submission to Google: add/update or delete. |
Store View | The store view to which the submission should be assigned. A global option Use global/no store is also available. |
URLs | List of absolute URLs, one address per line. |
Available actions:
| Form action | API value | Meaning |
|---|---|---|
Submit URLs for indexing | URL_UPDATED | Informs Google that the URL was added or updated. |
Delete URLs from indexing | URL_DELETED | Informs Google that the URL was removed and may be removed from the index. |
After submitting the form, the module will display a summary:
- added,
- updated,
- deduplicated,
- invalid,
- skipped.
The first 10 validation messages are shown as notices in the admin panel.
14. Indexing queue
The queue is located at:
Google Indexing API > Indexing QueueEach queue record contains, among others:
- URL,
- URL hash,
- store ID,
- website ID,
- source type,
- source entity ID,
- request origin,
- the
URL_UPDATEDorURL_DELETEDaction, - status,
- priority,
- number of attempts,
- maximum number of attempts,
- scheduled send date,
- processed date,
- last error code,
- last error reason,
- last error message,
- Google response,
created_byinformation.
Queue statuses
| Status | Meaning |
|---|---|
scheduled | The URL is scheduled but is still waiting for the scheduled_at date. |
pending | The URL is ready to be processed by cron. |
processing | The URL is currently being processed. |
success | Google returned a successful response. |
retry | A temporary error occurred and the record is waiting for another attempt. |
failed_permanent | Sending failed permanently or the maximum number of attempts was exceeded. |
cancelled | The record was manually canceled. |
Actions on queue records
| Action | Operation |
|---|---|
Transmit Now | Marks the record as urgent, locks it, sends it immediately through the Google client, and saves the result. |
Retry | Sets the record to pending, clears the lock, and schedules another attempt immediately. |
Cancel | Sets the status to cancelled and clears the lock. |
Note: Transmit Now performs a real request if Dry Run = No. With Dry Run = Yes, a dry-run log will be saved without a real send to Google.
15. Cron
The module adds two cron jobs in the default group.
Queue processing
kowal_google_indexing_process_queueSchedule:
*/5 * * * *The job runs the queue processor every 5 minutes.
The processor:
- Checks whether the module is enabled.
- Releases old locks for
processingrecords older than 30 minutes. - Moves
scheduledrecords topendingifscheduled_at <= now. - Checks available slots in daily and per-minute limits.
- Fetches
pendingandretryrecords. - Sorts them by priority and scheduled send date.
- Sends the request to Google or performs a dry run.
- Saves the response, status, and API log.
Log cleanup
kowal_google_indexing_cleanup_logsSchedule:
15 3 * * *The job removes API logs older than the number of days set in the field:
API Log Retention Days16. API logs
The logs are available at:
Google Indexing API > API LogsThe log includes:
- queue record ID,
- store ID,
- request type,
- endpoint URL,
- payload,
- HTTP status,
- response body,
- request duration,
- log creation date.
Request types:
| Type | Meaning |
|---|---|
publish | Real submission of a URL to Google. |
publish_dry_run | Processing in dry-run mode without a real request to Google. |
metadata | Metadata test for a URL. |
17. Mass Actions and admin buttons
The module adds mechanisms for submitting URLs from the admin panel.
Products
A mass action is available in the product grid to add product URLs to the queue.
The module skips products that are:
- disabled,
- not individually visible.
URLs are generated based on URL rewrites for active store views.
CMS pages
A mass action is available in the CMS pages grid to add page URLs to the queue.
The module skips inactive pages.
If a CMS page is assigned to all store views, the module resolves URLs for all store views.
Product, category, and CMS page - buttons on the edit form
The module adds indexing submission buttons on the edit screens of:
- product,
- category,
- CMS page.
The button resolves entity URLs for store views and adds them to the queue as URL_UPDATED.
18. Recommended initial configuration
For the first production implementation:
| Field | Recommendation |
|---|---|
Enable | Yes after saving credentials and allowed hosts. |
Dry Run | Yes during testing, later No. |
Credentials Source | Encrypted configuration value or Uploaded JSON file, according to project policy. |
Google Cloud Project ID | project_id from JSON. |
Daily Publish Limit | 200, unless Google approved a higher limit. |
Requests Per Minute Limit | 60 or less for a cautious implementation. |
Cron Batch Size | 20. |
Default Indexing Lag | 15. |
Manual Form Indexing Lag | 0-15, depending on how administrators work. |
Mass Action Indexing Lag | 15. |
Max Attempts | 5. |
Retry Delay | 15. |
Allowed Source Types | manual,product,category,cms_page,amasty_blog_post. |
Allowed URL Hosts | All production store hosts. |
Require HTTPS URLs | Yes. |
Enable Auto-Indexing | No, unless the project implements automatic providers. |
API Log Retention Days | 90. |
19. Common issues and diagnostics
Credentials test failed
Check:
- whether the JSON is valid,
- whether the JSON comes from a service account,
- whether it contains
private_keyandclient_email, - whether the
typefield has the valueservice_account, - whether the JSON file was not damaged during pasting.
Google returns a permission error
Check:
- whether the domain is verified in Search Console,
- whether the service account
client_emailis added as an owner, - whether the tested URL belongs to the same Search Console property,
- whether you are using the correct Google Cloud project and the correct JSON.
The URL is rejected before sending
Check the validation message:
| Message | Cause |
|---|---|
empty_url | Empty URL. |
url_too_long | The URL has more than 2048 characters. |
url_not_absolute | The URL has no scheme or host. |
https_required | The HTTPS requirement is enabled and the URL uses HTTP. |
invalid_scheme | The scheme is neither http nor https. |
host_not_allowed | The URL host is not included in Allowed URL Hosts. |
The queue is not being processed
Check:
- whether
Enable = Yes, - whether Magento cron is working,
- whether the records have the
pendingorretrystatus, - whether
scheduled_atis not in the future, - whether daily or per-minute limits are not exhausted,
- whether
Daily Publish LimitandRequests Per Minute Limitare not set to0.
Records are going to retry
Check:
- the HTTP status in the queue record,
- the API log,
- the Google response,
- whether a
429limit error is occurring, - whether there are temporary
5xxerrors, - whether
Max AttemptsandRetry Delayare set as expected.
20. Optional integration with Amasty Blog
Integration with Amasty Blog is предусмотрено as a separate module:
Kowal_GoogleIndexingApiAmastyBlogPackage:
kowal/module-google-indexing-api-amasty-blogThis module is not required for the main integration to work. It should be installed only in projects that use amasty/blog and need a mass action for blog posts.




















