Kowal AI Product Feed for OpenAI Vector Store and LLMs
Kowal AI Product Feed is a Magento 2 module for building an AI data layer that prepares store content in a format ready to be used by OpenAI Vector Store, AI assistants, semantic search engines, RAG pipelines, and public content discovery mechanisms such as llms.txt and NDJSON feeds.
This is not a standard product catalog export. The module organizes, normalizes, and publishes store knowledge in a way that allows it to safely power AI systems while maintaining control over data freshness, content scope, and document structure.
As a result, Magento can become a real source of knowledge for:
- product chatbots and copilots,
- intelligent customer service,
- semantic search,
- AI-generated answers,
- content automation,
- external AI integrations based on public or private feeds.
What the module does in practice
The module retrieves data from Magento and builds structured knowledge documents from it. It supports not only product content, but also additional content channels that are important from both AI and sales perspectives.
It can currently work with content such as:
product.core- core product data, descriptions, URL, attributes, and categories,product.faq- product questions and answers,product.docs- documentation and Markdown files,blog.article- blog posts,category.content- category content,cms.page- CMS pages.
Each document is saved locally, compared by checksum, and synchronized only when it has actually changed. This reduces redundant uploads, stabilizes the synchronization process, and provides greater control over costs and data quality.
Public AI and LLM endpoints
In the new version, the module can also publish data under public frontend endpoints:
/llms.txt/ai-feed/{storeCode}/index.json/ai-feed/{storeCode}/{contentType}.ndjson
This means the store can not only synchronize data to OpenAI Vector Store, but also provide a controlled content layer for external AI agents, crawlers, and integration tools.
The module can additionally:
- add a signal to the page
headthrough a link tollms.txt, - optionally display a link to
llms.txtin the footer, - publish feeds per store view and per language,
- maintain consistency between store content, the AI layer, and the public content discovery layer.
Why this matters for business
In many stores, the data needed to implement AI is scattered: some is in product descriptions, some in FAQ, some in the blog, and some in separate technical materials. Without an integration layer, it is difficult to build stable AI solutions on top of that.
Kowal AI Product Feed solves this problem because it:
- centralizes store knowledge in a single export and synchronization architecture,
- organizes data for use by language models,
- reduces the cost of building additional AI integrations,
- shortens the implementation time for chats, assistants, and semantic search engines,
- makes it possible to develop an
AI SEO,AEO, andLLM discoverabilitystrategy, - provides a foundation for multilingual AI implementations in Magento 2.
This means faster rollout of new features, greater consistency of AI-generated answers, and better use of the content the store already has.
Who this module is for
The module is designed for:
- Magento 2 stores implementing AI chat or an AI assistant,
- companies building semantic search engines and RAG on e-commerce data,
- technical teams integrating Magento with
OpenAI, - stores developing multilingual product knowledge bases,
- agencies and software houses implementing AI in Magento,
- brands that want to prepare their store for future LLM content discovery channels.
How other modules use it
Other modules can use Kowal AI Product Feed as a central knowledge source and integration layer instead of building separate exports, custom data serialization, and their own synchronization processes.
Example use cases:
- an AI chat module can ask questions to the
Vector Storeand answer based on current product knowledge, - an FAQ module can enrich the shared knowledge base with additional questions and answers,
- a documentation module can feed AI with instructional and technical content,
- a customer service module can use the same data for contextual answers,
- a recommendation module can use structured content for better product matching,
- content modules can extend the knowledge layer with blog, CMS, and additional document types.
Most importantly, other modules do not need to know the technical details of synchronization, checksums, file export, or upload to OpenAI. They receive a ready-to-use, consistent, and extensible data layer.
Key benefits
- a ready AI data layer for Magento 2,
- support for
OpenAI Vector Store, - public
llms.txtandNDJSONfeeds, - synchronization of changed content only,
- support for multiple content channels,
- prepared for
AI SEO,AEO, and LLM integrations, - compatible with Magento store view architecture and language versions,
- a solid foundation for further AI automation development in the store.
Kowal AI Product Feed - installation and configuration
Document purpose
This document describes the complete process of installing, launching, and configuring the Kowal_AiProductFeed module in Magento 2. It has been prepared so you can go from technical implementation to the first working export and the publication of llms.txt.
Module functional scope
The module enables:
- exporting store content to
JSONandJSONLfiles, - synchronizing data to
OpenAI Vector Store, - support for multiple content channels such as product, blog, category, and CMS,
- publishing public endpoints:
/llms.txt/ai-feed/{storeCode}/index.json/ai-feed/{storeCode}/{contentType}.ndjson
- adding a link to
llms.txtin theheadsection, - optionally adding a link to
llms.txtin the store footer.
llms.txt works per current Store View. This means each language version can return its own manifest and its own feed links.
Prerequisites
Before installation, make sure that:
- Magento 2 is working correctly,
- you have SSH access to the Magento root directory,
Composeris available in the environment,- the PHP process can write to the
var/directory, - you have an active
OpenAI API Keyif you want to use synchronization toOpenAI Vector Store.
Module installation
1. Adding the Composer repository
If the module is installed from a Git repository:
composer config repositories.ai.product.feed vcs https://github.com/kowalco/module-ai-product-feedIf the repository is private:
composer config --global --auth github-oauth.github.com 2. Package installation
composer require kowal/module-ai-product-feed3. Enabling the module
bin/magento module:enable Kowal_AiProductFeed4. Updating Magento
bin/magento setup:upgrade5. Compilation and cache
In a production environment, also run:
bin/magento setup:di:compilebin/magento cache:flushIn a development environment, it is usually enough to run:
bin/magento cache:flushIf there is an additional cache layer in front of Magento, also perform a purge on the side of:
VarnishCloudflareNginx fastcgi/proxy cache- another reverse proxy or CDN
6. Confirming module status
bin/magento module:status Kowal_AiProductFeedConfiguration location
The module configuration is available in the admin panel:
Stores -> Configuration -> General -> Kowal AI Feed
Recommended configuration model
The best practice is to configure at the Store View level, especially if the store operates in multiple languages.
Recommendation:
- one
Store View= one language, - one
Store View= oneVector Store ID, - one
Store View= a separate public AI feed layer.
Example:
PLstore view -> PolishVector Store IDENstore view -> EnglishVector Store ID
Field configuration
Enable Module
Enables the module for the selected scope.
Recommendation:
- set
Yesfor thoseStore Viewentries that should generate AI data
OpenAI API Key
Global API key used to communicate with OpenAI.
Required if:
- you want to synchronize files to
OpenAI Vector Store
Not required if:
- you only want to use local exports,
- you only want to use
llms.txtand public feeds.
Vector Store ID
The OpenAI Vector Store identifier for a specific Store View.
Recommendation:
- configure at the
Store Viewlevel, - set a separate identifier for each language.
Feed Location
Relative path to the directory where export files will be saved.
Default value:
var/ai-feeds
Recommendation:
- keep the directory in
var/, - do not expose this directory directly through the web server.
Enable Public AI Feed
Enables public endpoints:
/llms.txt/ai-feed/{storeCode}/index.json/ai-feed/{storeCode}/{contentType}.ndjson
Enable this option only for those store views whose content may be publicly available.
Important:
/llms.txtis not a global index of all stores- each language version of the store returns its own manifest
/llms.txtitself does not require generated files to exist in order to return a valid response
Show LLMs.txt Link in Footer
Adds a visible link to /llms.txt in the store footer.
Recommendation:
Noby default,Yesif you want to explicitly communicate the availability of the AI layer to partners, integrators, or bots.
Docs Location
Relative path to the directory with Markdown product documentation for product.docs.
Examples:
var/ai-docsvar/ai-docs/pl
Supported files include:
{docsLocation}/{SKU}.md{docsLocation}/{SKU}.markdown{docsLocation}/{storeCode}/{SKU}.md{docsLocation}/{storeCode}/{SKU}.markdown{docsLocation}/{SKU}/*.md{docsLocation}/{SKU}/*.markdown{docsLocation}/{storeCode}/{SKU}/*.md{docsLocation}/{storeCode}/{SKU}/*.markdown
Max Products Per Batch
Batch size for product export.
Starting value:
500
Reduce the value if:
- the catalog is very large,
- the environment has limited memory,
- additional AI operations increase the load.
Sync Product Core
Enables the product.core content type.
Recommendation:
Yes
Sync Product FAQ
Enables product.faq.
Use if the store has data in the kowal_zapytajoprodukt_pytania table.
Sync Product Docs
Enables product.docs.
Use if the Markdown documentation is already prepared.
Sync Blog Content
Enables the blog channel and the blog.article type.
Sync Category Content
Enables the category channel and the category.content type.
Sync CMS Pages
Enables the cms_page channel and the cms.page type.
Max Retries
Maximum number of API operation attempts.
Default:
3
Polling Timeout
Maximum wait time for file processing to finish on the OpenAI side.
Default:
600
Polling Interval
Interval between subsequent status checks.
Default:
5
HTTP Timeout
Maximum duration of a single HTTP request to OpenAI.
Default:
60
Recommended configuration order
After installation, configure the module in this order:
- Enable the module for the target
Store View. - Add the
OpenAI API Keyif you plan to synchronize withOpenAI. - Set the
Vector Store IDfor each language. - Verify the
Feed Location. - Set the
Docs Locationif you use Markdown documentation. - Enable the required content types and channels.
- Enable
Enable Public AI Feedif the endpoints should be publicly available. - Optionally enable
Show LLMs.txt Link in Footer. - Clear the Magento cache.
bin/magento cache:flushFirst launch
Dry run
First perform a dry run for one Store View:
bin/magento kowal:ai-export:sync --channel=product --store=1 --dry-runActual export
Then run the actual synchronization:
bin/magento kowal:ai-export:sync --channel=product --store=1Status check
bin/magento kowal:ai-export:status --store=1Verification of public endpoints
If Enable Public AI Feed is enabled, check:
curl -i https://twoja-domena.pl/llms.txtcurl -i https://twoja-domena.pl/ai-feed/pl/index.jsoncurl -i https://twoja-domena.pl/ai-feed/pl/product.core.ndjsonIf the store has separate language versions in the URL, also check:
curl -i https://twoja-domena.pl/en/llms.txtcurl -i https://twoja-domena.pl/de/llms.txtExpected behavior:
/llms.txtreturns a manifest only for the currentStore View/llms.txtworks even when feed files have not yet been generated/ai-feed/{storeCode}/index.jsonshows only existing feeds for the given store/ai-feed/{storeCode}/{contentType}.ndjsonworks only when the relevant source files exist
Cache header verification:
curl -I https://twoja-domena.pl/llms.txtcurl -I https://twoja-domena.pl/ai-feed/pl/index.jsonYou should see:
- for
/llms.txt, headers that practically disable cache, - for
index.json, a short cache lifetime and forced revalidation.
HTML verification:
- the page source should contain a link in the
headto/llms.txt, - if you activated the footer option, an
LLMs.txtlink should appear in the footer.
CLI commands
Synchronization
bin/magento kowal:ai-export:sync --channel=productbin/magento kowal:ai-export:sync --channel=product --store=1bin/magento kowal:ai-export:sync --channel=product --store=1 --content-type=product.core --identifier=example-sku --dry-runbin/magento kowal:ai-export:sync --channel=blog --store=1 --source=amasty_blogbin/magento kowal:ai-export:sync --channel=category --store=1bin/magento kowal:ai-export:sync --channel=cms_page --store=1Status
bin/magento kowal:ai-export:statusbin/magento kowal:ai-export:status --store=1bin/magento kowal:ai-export:status --channel=blog --store=1bin/magento kowal:ai-export:status --channel=blog --store=1 --source=amasty_blogImport blog sources
bin/magento kowal:ai-blog-source:import --source=amasty_blog --store=1Legacy product alias
Still available for backward compatibility:
bin/magento kowal:ai-feed:generatebin/magento kowal:ai-feed:generate --store=1 --content-type=product.core --sku=example-sku --dry-runCron
The module uses three cron jobs:
kowal_ai_blog_source_importkowal_ai_feed_generatekowal_ai_feed_cleanup
Make sure Magento cron is working correctly in the environment.
Most common issues
/llms.txt returns 404
Check:
- whether the current module version with frontend routing is deployed,
- whether you ran
bin/magento setup:upgrade, - whether in production mode you ran
bin/magento setup:di:compile, - whether you performed the standard static content deployment according to the project process,
- whether you cleared the cache,
- whether you purged the CDN / reverse proxy layer,
- whether
Enable Public AI Feedis enabled.
If https://twoja-domena.pl/llms.txt?v=123 works but https://twoja-domena.pl/llms.txt does not, this usually means an old shared cache is still active somewhere in the chain.
/llms.txt works, but the feeds are empty
This usually means that:
- the export has not been run yet,
- there is no active content for the given channel,
- the relevant synchronization flags are disabled.
This behavior is correct. The manifest itself can work before the actual feeds do.
No files in var/ai-feeds
Check:
- whether the module is active for the correct
Store View, - whether the required content types are enabled,
- whether Magento has write permissions to
var/, - whether the synchronization process has been started.
Problems with synchronization to OpenAI
Check:
- the correctness of the
OpenAI API Key, - the correctness of the
Vector Store ID, - the availability of outgoing HTTP connections,
- timeout and retry values.
Summary
After proper installation and configuration, the module provides Magento 2 with a ready AI data layer:
- local,
- multilingual,
- capable of synchronization with
OpenAI, - ready for publishing through
llms.txtandNDJSONfeeds, - extensible for additional integrations and AI modules.



















