Rollback
Move production back to the version it held before, by hand or from a rollback proposal of an alert rule.
A rollback moves production back to the version that it held before the last move. A rollback is a label move. It does not change or delete a version, and no control restores a version. To go back to the version that the label held just before, roll back the label. To use any other older version, open it and promote it; the gate checks that promotion. A save with content that equals a stored version is refused.
Risicare never rolls back by itself. A person always decides.
Roll back by hand
A member or higher can roll back.
Open the dialog
Open Actions for the label production on the card and click Roll back. Or open the Label history tab and click Roll back to v3 in the newest production row.
Read the facts
The dialog is titled "Roll back production to v3". It says that the gate does not check a rollback, because production ran v3 until the time of the last move. It also shows the calls and the share with errors of both versions in the last 7 days.
Give a reason
Write a Reason (required, at least 20 characters), up to 1,000 characters. The reason goes to the label history with your name and the evidence.
Confirm
Click Roll back to v3. The label history records the kind Rollback.
Code that pulls production gets the old version after its next refresh. With the default cache time of 60 seconds, the first call after those 60 seconds starts the refresh, and the calls after the refresh get the old version.
A rollback goes only to the version that the label held just before. If a person put the target version on the label with an override, only an owner or an admin can roll back to it.
If the rule of the label has A rollback needs approval too, the first click on Roll back to v3 does not move the label. The dialog says "This label asks for a second person to approve a rollback too. Ask for the rollback; another person approves it." and the button then reads Request the rollback. Another person approves it. The card then reads "Rollback of production to v3 waits for approval". See Gate and approvals.
Rollback proposals
An alert rule can watch a prompt label. When the rule fires, Risicare opens a rollback proposal. A proposal is data. Nothing moves until a person applies it.
Make a proposal rule
Open the rule form
On the Alerts page, click New rule. See Cost Alerts for the fields of a rule.
Choose a metric
A rule can watch a label for the metrics Error rate, Latency p95, Cost, Score mean and Run verdict.
Watch a label
In the section Watch a prompt label, choose a Prompt and a Label. The rule then follows the version that the label holds. When the label moves, the rule follows it. The page names the version that the label holds now. Do not watch a label turns the watch off.
The label must hold a version that replaced another one. If it does not, the form says "This label has no previous version, so a rollback has nowhere to go." A proposal needs at least 30 calls of the version.
For Error rate, the form reads the error rate of the previous version in the 7 days before the move. It suggests twice that value as the threshold. You can keep your own value. With fewer than 30 calls, or with no error, it suggests nothing and says why.
Review a proposal
A banner on the prompt page, between the label cards and the tabs, reads for example "Rollback proposed: the rule 'support errors' fired. 4.8 % of the calls of v4 had errors since the move 12 minutes ago. v3 had 1.0 %." The Prompts list shows the mark Rollback proposed, and the Show menu has a filter with the same name. A member or higher sees Review the proposal.
The dialog is titled "Roll back production to v3". It shows:
- An Evidence table: Metric, Value, Threshold, Window, Calls in the window, and for an error rate Errors since the move and Errors in the 7 days before. A figure the alert did not give shows "—" with the reason.
- Two links: Open the calls of v4 and Compare v4 with v3.
- A Reason for the rollback (required, at least 20 characters), filled in from the proposal. Change it if you know more.
Then choose:
- Apply the rollback moves the label back. It is a normal rollback that carries the proposal. The gate does not check it. If the label asks for approval of rollbacks, the first click shows that sentence and the button then reads Request the rollback.
- Dismiss with a reason asks for a Reason to dismiss (required, at least 20 characters). Click Dismiss the proposal. The label does not move, and the same rule stays quiet for 24 hours.
If the label moved while the dialog was open, the dialog says the proposal is out of date. Close it and read the page again.
What happens to a proposal
A proposal ends after 7 days with no decision, or when a move of the label makes it out of date. The Rollback proposals table on the Label history tab lists every proposal with the columns Proposed, Move, Why, Outcome and Reason.
| Outcome | Meaning |
|---|---|
| Waiting for a decision | The proposal is open. |
| Rolled back | A person applied it. |
| Dismissed | A person dismissed it. |
| Superseded | The label moved before a decision. |
| Expired | 7 days passed with no decision. |
The Why column says "A rule on the production error rate fired" or "The verdict alert fired". The alert History names the label, the prompt and the version that the label held when the rule fired. A proposal sends the event prompt.rollback_proposed. See Webhooks.