Use a Prompt in Code

Your code pulls a prompt with get_prompt (npm: getPrompt). Both are in Python 0.11.0 and npm 0.14.0 and later. The call sends GET /v1/prompts/resolve to the api host with the key of init(). The key needs the scope prompts:read, which an ingest key and a secret key have. This is the only prompt route that an API key can reach. The SDK pulls nothing at import time or at init().

Pull a prompt and call the provider

Give label or version, not both. With neither, the label is production. A bad argument raises ValueError in Python and rejects with TypeError in npm. In Python, get_prompt is a plain function: the first call for a key can wait up to timeout_s.

The Prompt object

repr(prompt) in Python, and String(prompt) and JSON.stringify(prompt) in npm, show the identity and not the text.

The cache and the fallback

• The cache is in memory, one for each process, with at most 500 entries and no disk. A version pin never expires.
• A label is fresh for cache_ttl_s (default 60 seconds, 5 to 3,600) or cacheTtlMs (default 60,000, 5,000 to 3,600,000). After that, the next call gets the cached value at once, and one background refresh asks with If-None-Match. The api answers 304 when the label did not move.
• Only the first call for a key waits for the network. It waits at most timeout_s (default 1, 0.1 to 10) or timeoutMs (default 1,000, 100 to 10,000).
• A refresh that fails (a network error, a 5xx, a timeout, 401, 403, 404 or 410) keeps the last good value. source becomes stale, and the SDK writes one WARNING for each key each minute.
• If the first pull fails and nothing is cached, the call returns the fallback. With no fallback, Python raises risicare.PromptUnavailable and npm rejects with PromptUnavailable. The error has a reason and the HTTP status. The SDK does not ask for a failed key again inside one cache time.
• RISICARE_PROMPT_CACHE_TTL_S (5 to 3,600) and RISICARE_PROMPT_TIMEOUT_S (0.1 to 10) set the defaults, in seconds in both SDKs. The SDK ignores a value out of range and writes one WARNING.

A fallback is a PromptContent with the keys of a stored version: messages, and optionally model, parameters, tools and output_schema. version is the number of the stored version that the content copies. The SDK computes its hash. A fallback with the same content as a stored version, key for key, has the hash of that version, and its calls count as that version. Other content shows as Unknown hash. A fallback has a hash and no label. Archived prompts, and labels that hold no version, make a pull fail, so your code uses the last good value or the fallback.

Fill in the variables

prompt.compile(**values) (npm: prompt.compile(values)) gives the messages with the values put in. A placeholder is {{name}} or {{ name }}. Any other {{ is plain text. A value that is not a string goes in as compact JSON. A missing value raises risicare.MissingVariable (npm: throws MissingVariable). A value that you do not use is ignored. The SDK never reads a value again as a template.

prompt.openai_body(**values) (npm: openaiBody(values)) gives a dict or an object for client.chat.completions.create. It always has messages. It has model, temperature, top_p, max_completion_tokens, stop, seed, tools and response_format only when the prompt sets them. The prompt value Max output tokens goes in as max_completion_tokens. A server that takes only max_tokens gets the key from you: rename it.

prompt.anthropic_body(**values) (npm: anthropicBody(values)) gives a dict or an object for client.messages.create. It always has messages (the messages that are not system messages). It has model, max_tokens, system (the system messages, joined by a blank line), tools and stop_sequences only when the prompt sets them. It adds no default for max_tokens, and the Anthropic API needs one, so set it when the prompt does not. It never puts temperature or top_p in the body, and it leaves out seed and the output schema. When the prompt sets one of them, the helper writes one WARNING, once for each process, that names what it left out.

In npm, the helpers are typed any, so the result fits the type of the provider client.

The prompt keys on LLM spans

When the messages of an OpenAI or Anthropic call come from compile(), openai_body() or anthropic_body() (npm: openaiBody() or anthropicBody()), the span has the four keys below. When there is a version, it also has llm.prompt_template.version, with the same text as risicare.prompt.version. The SDK writes them with content capture on or off. No template text and no variable value goes on a span.

The list that the helpers return carries a hidden reference to the prompt, and the OpenAI and Anthropic instrumentation reads it. In Python the list is a list subclass. In npm it is a plain array with a hidden property. It compares equal to a plain list, and its JSON does not change. A copy does not keep the reference: list(messages), a slice or a new list in Python; a spread, slice, map or structuredClone in npm. For such a path, use a block.

Blocks

with prompt.context(): (Python only) and prompt.run(fn) (both SDKs; Python also takes *args and **kwargs) make every LLM span that starts inside carry the name, the version, the hash and the label of the prompt.

For a prompt that you did not pull, name it yourself with prompt_context(name, version, hash=None, label=None) (npm: withPrompt(name, version, fn, { hash, label })). The span then gets risicare.prompt.name and risicare.prompt.version, and llm.prompt_template.version with the same text as the version. It gets risicare.prompt.hash and risicare.prompt.label when you give them.

Rules of a block:

• A value is a non-empty string or an integer. A longer text is cut to 200 characters. A boolean, a float or an empty value is left out and writes one WARNING.
• The scope is one call, not the trace. A nested block replaces the outer block. An agent, tool, function or embeddings span gets no key. A key that you set with attributes is not overwritten. A stream that you read after the block ends keeps the keys.
• A list from compile() replaces an enclosing block as a whole. All keys of the span come from the list, and none from the block.
• The spans of the LangChain callback (npm: handler) and the spans that the OpenTelemetry bridge converts get no keys from a list. Use prompt.context() or prompt.run(fn) for those.

How the dashboard uses the keys

• Prompt chip: a span that carries a prompt shows a chip after the model badge: name@version. For a stored version it reads support@4 · a1b2c3d4 and links to that version. When the hash of the call differs from the stored version, it reads support@4 · other hash. A small Calls link opens the trace list for the prompt. See Traces.
• Filter keys: prompt: matches the name, prompt_version: the version, and prompt_hash: the first 8 to 64 characters of the hash. prompt:support prompt_version:4 matches one span. See Filtering and Search.
• Prompt column: the trace list can show the first prompt pair of a trace as support@4, and "+1" when the trace has more. A click on it adds the filter. The column is on by default when the project has a stored prompt.
• Results by version counts the calls by the name, the version and the hash. See Versions.
• Runs: evals.run(..., prompt=prompt) (npm: runDataset({ ..., prompt })) sends the name, the version and the hash with the run, and runs each trial inside prompt.context() (npm: prompt.run()). The api binds the run to the stored version when the hash equals its content hash. See Evaluations.

Next Steps