Engineering
Commit message from a diff
Write a commit message from a diff: a subject line that states what was found, then prose explaining why the change was made.
Add in the app
Engineering · For your bots
Triage a failing build
Triage a red CI run: find the first real failure, attribute it to a change, and say whether the code is wrong or the runner was.
In Lobstack: Skills › Library › Add
Download LobstackWhen to use it
A build went red and somebody has to decide whether to fix code, fix a test, or fix the pipeline.
What your bot will do
- 01
Find the FIRST failure in the log, not the last. A failing run usually ends in a cascade of steps that failed because an earlier one did, so the error nearest the bottom is the least informative line in the file.
- 02
Read the assertion rather than the summary. "1 failed" says nothing; the expected-versus-actual says what the code believes about itself.
- 03
Decide which of three things happened before proposing anything, because they have different owners:
- the change under test is wrong;
- the change is right and the test is stale;
- nothing is wrong and the environment failed — a refused download, a full disk, an unset secret, a port already bound.
- 04
Attribute it. Find the last green run and name the change in between that touched what broke. If the same job has failed on unrelated changes, it is flaky, and then the test is the defect.
- 05
Report the cause and the smallest fix for it. Never re-run a job to get it green without saying why it was red: a re-run that passes has not answered the question, it has only stopped asking it.
The file
---name: failing-builddescription: "Triage a red CI run: find the first real failure, attribute it to a change, and say whether the code is wrong or the runner was."metadata: title: "Triage a failing build" category: engineering tags: [ci, debugging]--- ## When to use this A build went red and somebody has to decide whether to fix code, fix a test, orfix the pipeline. ## How 1. Find the FIRST failure in the log, not the last. A failing run usually ends in a cascade of steps that failed because an earlier one did, so the error nearest the bottom is the least informative line in the file.2. Read the assertion rather than the summary. "1 failed" says nothing; the expected-versus-actual says what the code believes about itself.3. Decide which of three things happened before proposing anything, because they have different owners: - the change under test is wrong; - the change is right and the test is stale; - nothing is wrong and the environment failed -- a refused download, a full disk, an unset secret, a port already bound.4. Attribute it. Find the last green run and name the change in between that touched what broke. If the same job has failed on unrelated changes, it is flaky, and then the test is the defect.5. Report the cause and the smallest fix for it. Never re-run a job to get it green without saying why it was red: a re-run that passes has not answered the question, it has only stopped asking it.