How it works
Tracks whether review feedback results in code changes. Weave analyzes comments left during review and checks if subsequent commits address them.Calculation method
Weave uses AI analysis to determine if a comment was resolved:- Comment selection — Only analyzes comments from merged PRs. Focuses on “suggestion” comments (proposed improvements), excluding questions and praise that don’t require code changes.
- Commit analysis — For each comment, examines all commits made after the comment was created but before the PR was merged. Analyzes the commit’s file changes, diff content, and code modifications.
-
Resolution detection — Uses AI to determine if a commit directly and completely addresses the specific feedback in the comment. The commit must:
- Address the exact concern raised in the comment
- Modify the specific file and lines referenced
- Fully resolve all points mentioned (not partial fixes)
- Confidence levels — Each resolution is assigned a confidence level (very high, high, medium, low) based on how clearly the commit addresses the comment.
- Context consideration — Incorporates thread replies and broader PR conversation context to better understand if feedback was acknowledged or addressed.
- No later commits — If no commits were made after the comment, Weave judges resolution from the thread alone: a clear acknowledgment, deferral, or reasoned dismissal counts as addressed. Weave also judges the thread replies this way when later commits exist but none of them addresses the comment.
Resolution is based on code changes and the conversation, not a thread’s resolved status. Marking a thread resolved on the pull request doesn’t count as addressed on its own.
Comment intents
Comments are categorized by intent to focus analysis on actionable feedback:- Suggestion: Proposed improvements (expected to result in changes)
- Question: Clarification requests (may or may not result in changes)
- Praise: Positive feedback (no changes expected)
- Other: General discussion, FYI notes, or context that fits none of the above (no changes expected)