A new Charm Review Queue

Recently, our review queue for Juju charms has been overrun with new contributions and updates to our charm ecosystem. As of late, the charm queue is upwards of four weeks backlogged. While this is a “great” problem to have, it’s a pretty poor experience for users who have taken the time to create a new charm or contribute a change. After weeks of waiting, you finally get a review, quickly fix the items in the review and must now wait additional time for it to be re-reviewed.

When we were first starting out the review queue was fantastic. It was a means for us to track and manage incoming contributions and typically response was within 24 hours! As the community has grown, so has the need for a more robust and modern review queue. I’ve been working on a change for the past few weeks on and off, taking the feedback from people who have lived through the process as well as incorporating my own opinions to help produce a better product.

The new review queue, makes strides to resolve some of the pain points in the current review queue and turn an eye to the future of the charm review process. One of the biggest points of feedback from users and authors alike was knowing where their review was in the process. The review queue now historically tracks all reviews ever submitted to the charm store, including a lot of old charms from Ubuntu series long past!

The review queue now, compared to it’s previous version is quite a different. For starters, it’s about twice the size. This is because of some new rules put in place for ingesting reviews. Since we’re pulling in all reviews from all time we’ve found quite a few that have fallen off the radar prematurely. This also means that users can get their reviews in the queue without having to know what button to click in Launchpad or some other weird workflow nonsense.

The Solutions team will be spending the next week going through these and following up when required. Along with this new queue we’ve also enacted a new policy on the ecosystem team that makes us all do reviews every single week. Users can also find all the reviews they’ve ever submitted and all the reviews they’ve ever reviewed by going to http://review.juju.solutions/user/<LP-USER>.

To summarize here are a list of the current features:

  • Mirror of entire charm review
  • Show number of comments of each review, this helps identify outliers or charms with heavy discussion.
  • Display progress of a review and outline next steps for reviewer and author
  • Decoupled from charm world and charm store

These are features which will be landing shortly:

  • Integrate automated charm testing
  • Ask Ubuntu question integration
  • Merges for tooling
  • Analytic of charm review process
  • Filtering and user preferences

The code is currently hosted on Github and pull requests are welcome!