Better Code Review

Understanding the Problem

Reviewing code is time consuming and tedious for managers and developers. However, it's also necessary for maintaining quality and helping developers improve. Volume of commits and line-by-line changes make it hard to know where to focus review time. Lots of time spent looking for and reviewing code that didn't have a big impact

[Image]

Identifying User Outcomes

As a manager, I want to:

• Understand how commits relate to bigger projects and initiatives

• Spend less time reviewing code by focusing on the most important changes

As a developer, I want to:

• Understand how my work is being evaluated

• Ensure that scores are accurate and understandable

Designing Solutions

After we built the dashboards, our team used the tool internally for about a year. They offered feedback about challenges they were having related to code review.

Commit Groups

• Too much noise in the commit feed

• Developers have different habits and timing around commits

• Bill came up with the idea and logic for Commit Groups

• These collect related commits, combine LI score, and associate them with a task in Jira

• Grouping them made it easier to see what happened over a recent timeframe vs. minute-by-minute updates

• Helped organize commits and reduce the time needed to review by half

• Reviewers could also view all changes in a single diff viewer screen, rather than having to open a dozen or more individual commits

Scoring Transparency

• Devs (including those who worked on SO), struggled to understand why a commit received a certain score

• We had built a basic diff viewer, and reframed it to focus on understanding scores

• Expanding panel with total score, score for each line, explanation about why line was scored as it was

• Managers could adjust scores and use advanced settings to make measurements more accurate for their team's work (e.g. ignore file type, set scores for each language)

Code Changes

• More detail about changes to the code

• As we adjusted the diff viewer for scoring, we also looked at ways to make review faster and easier

• Most diff viewers show lines added and removed. This doesn't give much insight

• Move, updated, find and replace, copy and paste, and more

• Color and iconography, connection between moved lines

• Offers a more complete picture of what happened, and allows reviewers to pinpoint the change they're looking for

Better annotation and collaboration

• Comment on lines or on the whole commit

• Considered a full-baked PR tool, but moved away from it

• Too much overlap with GitHub, hard to identify exactly what problem we needed to solve

Measuring and Learning

Results

Reduced time for reviewing commits.

Better understanding of big picture – how commits related to projects, what the impact was

Full transparency about scoring

Insights

• Many are skeptical of the accuracy and reliability of scores. Offering transparency and control seems to help address this.

• Both managers and developers need to understand why commits are scored the way they are. And they need to be able to adjust or calibrate to align with how they measure their developers.

• Users seem more comfortable if they think of us as a complement to GitHub rather than a replacement. If our tools can enhance existing workflows instead of introducing new ones, that seems to alleviate some concern about additional overhead.

• Key workflows (e.g. reviewing Pull Requests) remain a challenge, but we're not sure if building a full PR replacement for GitHub is a good idea. Need to better understand the work flow among different teams before investing time there.