AI Glossary

A prompting technique that improves LLM accuracy on complex tasks by guiding the model to show its reasoning step-by-step before arriving at a final answer.

Crawlability is whether — and how easily — automated agents (search crawlers, AI bots) can fetch and read the content of a website, governed by robots.txt, response codes, JavaScript rendering, paywalls, login walls, and rate limits.

A prompting technique where you include a few examples of the desired input-output format in your prompt, helping the model understand exactly what you want without any fine-tuning.

GPTBot is OpenAI's web crawler — one of a growing list of AI-specific crawlers (PerplexityBot, ClaudeBot, Google-Extended, Applebot-Extended, etc.) that fetch content for training and live retrieval by AI assistants.

llms.txt is an emerging standard for a plain-text file at the root of a website that summarises the site for AI assistants — analogous to robots.txt for crawlers or sitemap.xml for search engines.

The set of practices and tools for deploying, monitoring, and maintaining machine learning models in production — essentially DevOps principles applied to the ML lifecycle.

An open standard for connecting AI assistants to external data sources and tools through a unified, composable interface.

The practice of designing and refining inputs to language models to elicit more accurate, useful, and consistent outputs.

Schema markup is structured metadata embedded in a webpage (using the schema.org vocabulary, typically as JSON-LD) that tells search engines and AI assistants exactly what a page is about — product, article, FAQ, organisation, person, recipe, event, etc.

Structured data is any machine-readable representation of information on a webpage — most commonly schema.org JSON-LD, but also microdata, RDFa, sitemaps, OpenGraph tags, Twitter cards, and (increasingly) llms.txt files.

A technique for constraining a language model's output to follow a specific format like JSON, XML, or a defined schema, ensuring the response can be reliably parsed by downstream code.

The foundational instruction set given to an LLM that defines its role, behaviour, tone, and constraints for a particular application — set once at the application level and shaping all subsequent user interactions.

A parameter that controls how random or deterministic an LLM's output is — lower values produce more predictable, focused responses while higher values increase creativity and variation.