Skip to main content

​
Purpose

A data source is content the chatbot (and AI Search) may learn from: your TYPO3 pages, a sitemap, PDFs, questions and answers, or plain text. The chatbot only answers from the content you add here. You find it in AI Universe → AI Chatbot/Search → Data Source.

​
Adding a data source

  1. Click + Add Source.
  2. Choose the Source Type:
    • Sitemap XML – the address of your sitemap, for example https://example.com/sitemap.xml.
    • PDF Documents – a folder with PDF files. Click Browse to pick it, or Upload PDF (max. 25 MB).
    • TYPO3 Pages – page IDs, separated by commas. Also choose Recursive (how many sub-page levels: Infinite levels, Provided pages only, or 1 to 4 levels) and Index mode (see the tip below).
    • Web Pages – a website address. Add /* at the end to include only one part, for example https://example.com/blog/*. Turn on Single URL to read only this one page.
    • Q&A Pairs – type a Question and an Answer. Click Add Q&A pair for more.
    • Text – paste any text the AI should know.
    • Indexed Search, ke_search and Solr – only shown if that extension is installed.
  3. Enter a Name (for example Main Website) and, if you like, a Description.
  4. Choose Used by: Both (Chatbot & Search), Chatbot or Search.
  5. Choose how often the content is read again (Sync Interval): Hourly, Daily, Weekly or Custom cron expression (your own schedule, for example 0 0 * * 0 = every Sunday at midnight).
  6. Choose the Source groups. Global is always included.
  7. Turn on Enabled.
  8. Click Save.
Data Source tab with the source list and the buttons Manage source groups, Sync All and Add Source
Index mode for TYPO3 Pages: Frontend (rendered page HTML) reads the page as visitors see it and is recommended on TYPO3 13.4 and newer. Legacy (database columns only) reads only the text stored in the page records.
What happens next? The source is marked for reading. The background task T3CS Training (Site N) is created for your website if it doesn’t exist yet – you don’t need to create it yourself. See Scheduler.
There is only one training task per site. Its schedule comes from the data source you saved last. If your sources use different intervals, save the one with the interval you want last.

​
Editing or deleting a source

Use the buttons next to each data source to edit or delete it.
Deleting a data source also removes its training queue and embedded data for that source.

​
Source Groups | Page-Level Source Configuration

Source groups let you choose which content the chatbot and AI Search may use on which page. Example: on product pages, only product content is used; on HR pages, only HR content.

​
Backend Configuration

​
Manage Source Groups

  1. Go to Data Source → Source Groups.
  2. Create, edit or delete groups.
Manage source groups dialog
The Global Source Group is a system group and cannot be edited or deleted.

​
Assign Source Groups to Data Sources

When you add or edit a data source, choose one or more Source groups. Global is always added. Used by (Both (Chatbot & Search), Chatbot or Search) decides which tool may use the source.

​
Page-Level Configuration (T3AS / T3AC)

  1. Open the page in the page module (Web → Page, on TYPO3 v14 Content → Layout).
  2. Click Edit page properties.
  3. Open the tab AI Chatbot / Search (or AI Chatbot if only AI Chatbot is installed).
  4. In Source groups, choose the groups this page may use. Global uses all sources.
  5. Click Save.
Page properties, tab AI Chatbot / Search with the Source groups field The choice also applies to all sub-pages, for both AI Search and AI Chatbot.

​
Practical Example

A company has three groups: Products, HR and Internal. Product pages use only Products, so answers are about products. HR pages use only HR.
Source groups only filter which content is used for answers. They don’t change training or the background task.

​
Header / Footer for Sitemap and Web-Page Source Types

Most websites repeat the same header and footer on every page. With these switches, that text is learned only once instead of on every page. Add Source dialog for Sitemap XML with Index site header and Index site footer
  • Index site header – learn the page header once (useful if it contains helpful text, for example your services).
  • Index site footer – learn the page footer once (useful for contact details, company info or policy links).
Turn them on in the data source form for Sitemap XML or Web Pages sources.

​
Sync

Sync Now (one source) and Sync All (all sources) mark the content to be read again. The actual reading and learning happens when the background task runs (see below).

​
Scheduler

The Scheduler is TYPO3’s tool for background tasks. Each website has one task called T3CS Training (Site N) (N = the ID of the website’s start page). It is created for you when you save your first data source. When it runs, it:
  1. Reads the sources marked for sync again.
  2. Trains all items with status Pending – the AI learns the content.
  3. Cleans up old Failed items after the number of days in Retention Days. Learned (Completed) items are kept.
Don’t want to wait? Click Run All on the Dashboard or Run Task Now in the Training Center.
The Scheduler must run on your server (a cron job). If nothing is ever trained, ask your hosting provider or developer to set it up.

​
Settings used by training

Most websites can keep the defaults. You find them in AI Universe → AI Foundation → AI Features, on the Training and Rate Limiting cards (badge AI Chatbot/Search).
Batch Size and Retention Days are also written to the Scheduler task as --batch-size and --retention-days.
The archive folder for cleanup CSV files is var/log/t3cs by default. There is no backend field for it.
You can start the training yourself on the command line instead of waiting for the Scheduler, for example to test it or to train everything again.
  • nst3af:training <rootPageId> – train the content of one site (<rootPageId> = ID of the site root page).
  • --dry-run – only show what would happen. --source=ID – only one data source. --force – train everything again.
  • scheduler:run --task=<taskUid> – run the Scheduler task itself (the task UID is shown in the Scheduler module).
  • See all options with typo3 nst3af:training --help.
More AI Foundation background tasks are listed under AI Foundation → Scheduler & CLI in the backend.

​
History cleanup

Delete old visitor conversations and searches automatically, for example everything older than 90 days. This helps you follow your privacy rules. It is separate from the training cleanup above.
  1. Go to the Scheduler module (System → Scheduler on TYPO3 v12 and v13, Administration → Scheduler on TYPO3 v14).
  2. Create a new task.
  3. Choose Execute console commands.
  4. Choose t3af:history:cleanup.
  5. Set days (how old entries must be before they are deleted; default 90).
  6. Set how often it runs on the Timing tab.
  7. Save the task.
Scheduler task t3af:history:cleanup with days argument set to 3
You can delete old usage history on the command line, for example in your own cron job.
  • t3af:history:cleanup – deletes history older than 90 days.
  • t3af:history:cleanup 3 – deletes history older than 3 days (use any number of days).