Question 1

Is the default 0.80 threshold justified, or just a round number someone picked?

Accepted Answer

It was measured. The benchmark is 800 closed issues from fourteen repositories across four ecosystems, each carrying exactly one label its own maintainers applied. At 0.80 the action fires on 649 of 800 with 78% precision; at 0.70 it covers 701 at 76%, and at 0.95 it covers 521 at 83%. The curve is shallow, so raising the threshold trades a lot of coverage for a little precision.

Question 2

Which labels does it get wrong, and does that matter for my repository?

Accepted Answer

The disagreement concentrates in one class: bug 97%, feature 86%, docs 82%, question 40%. The README's explanation is that maintainers rarely treat question as a category — they use it as a workflow state meaning this is support, not their bug tracker. Excluding it, the figure is 88% across 508 decisions, and it holds between 85% and 90% across JavaScript, Rust, Go and Python.

Question 3

What stops it writing to my repository before I trust it?

Accepted Answer

Start with apply: false, which is the default. Every run writes a table into the job summary showing what it would have done without touching anything, and the CLI equivalent, npx hush-triage owner/repo, reads issues and writes nothing — there is no --apply flag on it at all. Once a week of summaries looks right, set apply: true. Applying anything needs issues: write on the token.

Question 4

Can it handle my own labels, and what does it refuse to touch?

Accepted Answer

Your labels are the sharp instrument here. The defaults are bug, feature, docs and question, replaceable with a JSON object of name to meaning, and the README reports that sharpening those descriptions to say what each label is not moved the benchmark three points and bug to 97%. It applies a label only if it already exists in the repository, and it never closes, locks, deletes or edits anything, never removes a label a person set, and does nothing at all while apply is false.

Question 5

How much does a triage run cost?

Accepted Answer

One request per issue at about $0.00002, so ten thousand issues cost roughly twenty cents. All four questions travel in that single request, which is why it finishes in 202 to 530 ms per issue, before the page reloads. The CLI prints the average latency and the total spent, as in the remotion-dev example: 237 ms average and $0.00033 total for six open issues.

Question 6

Does it handle pull requests, or only issues?

Accepted Answer

Both. Pointed at pull_request_target it asks a different set: kind across fix, feature, docs, refactor, chore and test; risky for migrations, auth, permissions, payments, deletion paths, public API or deploy config; untested when behaviour changed with no test in the diff; and undescribed when a reviewer would have to read the diff. It sees the title, description, file list with per-file line counts, commit count and whether it is a first contribution, but never the diff body.

Question 7

What can I tune, and what does the action refuse to do?

Accepted Answer

Inputs are typesafe-api-key, github-token, apply, comment, labels, check-duplicates and five thresholds: label 0.80, label-confidence 0.60, spam 0.90, needs-info 0.85 and duplicate 0.85. It never closes, locks, deletes or edits anything, never applies a label that does not already exist, never removes or overwrites a label a person set, and does nothing at all while apply is false. MIT licensed, and npm test exercises the decision layer offline.

Hush

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev community