How it works
Comment resolution rate measures what percentage of suggestion comments the PR author addressed, either with a later commit or, when no later commit addresses the comment, with a clear reply in the thread. This metric indicates how actionable and effective code review feedback is, and how responsive authors are to reviewer suggestions. Higher resolution rates indicate:- Actionable, specific review feedback
- Authors responsive to reviewer suggestions
- Effective collaboration during code review
- Aligned understanding of code quality standards
Calculation
The rate is calculated by comparing addressed suggestions to total suggestions. Formula:What counts as actionable
Only comments that could reasonably result in code changes:- Suggestions: Proposed improvements or changes, including minor style or naming feedback
What’s excluded
Comments that don’t require code changes:- Questions: Clarification requests
- Praise: Positive feedback
- Discussion: General comments or context
How resolution is detected
Weave uses AI to determine if a comment was resolved:- Identify comment: Extract the reviewer’s feedback
- Find subsequent commits: Look at commits after the comment
- Analyze changes: Check if commits address the specific feedback
- Determine resolution: AI evaluates if the concern was fully addressed
- Fall back to the conversation: If no later commit addresses the comment, Weave checks whether the thread clearly addressed, deferred, or dismissed the concern
Resolution is based on code changes and the conversation, not a thread’s resolved status. Marking a thread resolved on the pull request doesn’t count as addressed on its own.
Why it matters
Comment resolution rate reflects:- Review effectiveness: Are reviews producing helpful, actionable feedback?
- Author responsiveness: Do authors address reviewer concerns?
- Team dynamics: Is there good collaboration and trust?
- Quality standards: Are expectations aligned?
Interpreting the numbers
80-100% (excellent):- High-quality, actionable review feedback
- Authors addressing nearly all suggestions
- Strong collaboration and trust
- Most feedback is addressed
- Some comments may be discussed and deferred
- Healthy balance of standards and pragmatism
- Significant feedback not addressed
- Possible communication gaps
- May indicate unclear expectations
- Most feedback ignored or not acted upon
- Review process may not be effective
- Possible trust or communication issues
Common patterns
High resolution rate (90%+)
Positive interpretation:- Excellent, actionable feedback
- Authors committed to addressing concerns
- Strong code quality culture
- Reviewers only commenting on trivial issues
- Authors over-complying without pushback
- Missing opportunities for design discussions
Low resolution rate (50%)
Possible causes:- Reviewers commenting on out-of-scope items
- Authors deferring changes to future PRs
- Philosophical disagreements
- Comments are questions, not suggestions
- Complex PRs with design tradeoff discussions
- Experimental or exploratory code
- Comments suggesting future improvements
Improving resolution rate
For reviewers
-
Make feedback actionable:
- Be specific about what to change
- Provide examples or suggestions
- Distinguish between blocking and non-blocking comments
-
Scope feedback appropriately:
- Focus on the PR’s stated goal
- Don’t request unrelated improvements
- Tag “out of scope” comments clearly
-
Use comment intent tags:
- Mark suggestions vs. questions
- Indicate blocking vs. non-blocking feedback
- Be clear about expectations
For authors
-
Respond to all comments:
- Address each piece of feedback
- Explain if you disagree or defer changes
- Reply in the thread when you handle feedback without a code change
-
Update code based on feedback:
- Make requested changes in follow-up commits
- Reference comments in commit messages
- Push all fixes together when possible
-
Communicate deferments:
- Explain why changes are deferred
- Create follow-up tickets for future work
- Get reviewer agreement on deferments
For teams
-
Align on standards:
- Document coding conventions
- Define “blocking” vs. “nice to have”
- Create PR templates and checklists
-
Foster healthy communication:
- Encourage constructive feedback
- Make it safe to disagree
- Focus on learning and improvement
-
Review the review process:
- Track resolution rate trends
- Discuss patterns in retrospectives
- Adjust practices based on data
Resolution rate should be balanced with other metrics. Very high rates (99%+) might indicate reviewers avoiding meaningful feedback. Very low rates (30%) suggest broken review process.
Technical details
How does AI detect comment resolution?
How does AI detect comment resolution?
The AI resolution detector:
- Analyzes comment text: Understands the reviewer’s specific concern
- Examines commits: Reviews all commits made after the comment
- Compares changes: Checks if file/line changes address the feedback
- Assigns confidence: Rates resolution as very high, high, medium, or low confidence
- Considers context: Looks at thread replies and discussion
Which comments are included?
Which comments are included?
Included:
- Suggestion comments (proposed improvements)
- Comments on merged PRs only
- Question comments
- Praise comments
- Comments on abandoned/closed PRs
- Comments from AI code review tools, unless you group by AI tool (each group then shows only that tool’s comments)
Why does a comment I resolved on the pull request show as not addressed?
Why does a comment I resolved on the pull request show as not addressed?
A thread’s resolved status isn’t used. A suggestion counts as addressed when a later commit makes the requested change, or when no later commit addresses it and the thread clearly addresses, defers, or dismisses it.Changes to files Weave excludes from analysis aren’t examined, so a fix made only in those files isn’t detected. This includes generated code, lockfiles, and some docs and config files (for example
.md, .yml, .json, and .tfvars).If a verdict is wrong, users with edit access can override it from the chart’s drill-down using the Addressed toggle.How is the rate aggregated?
How is the rate aggregated?
Resolution rate is calculated as a percentage across all comments:
- Per PR: Percentage of comments resolved in that PR
- Per engineer: Percentage of comments resolved across all their PRs
- Per team/org: Percentage of all comments resolved
What data sources are used?
What data sources are used?
Comment resolution rate uses:
- Git repository data: PR reviews, comments, and commits
- AI analysis: LLM-powered resolution detection
- Supported platforms: GitHub, GitLab, Bitbucket
Related metrics
- Comment resolution: Detailed drill-down of which comments were resolved
- Review cycles: Number of review rounds per PR
- Review quality: How thorough and helpful reviews are
- Code review efficiency index: Composite review effectiveness metric