So, it's been more than a year on since I wrote this review, so what has changed ?
Well. The first thing to say is that we (that is, a large multi-national financial services company) continue to use Sonarqube, indeed it has become mandatory for all projects (new and existing). We have introduced an aggregation portal which takes metrics from SonarQube via its API along with other sources, to provide a cross project and somewhat sanitised view for upwards reporting. It's important, we feel, not to try and hide issues, but at the same time not to 'set hares running' by exposing more senior management to metrics 'in the raw'. So instead, we gather all the evidence that we have, and add to that some constructive assessment from the lead solution designers, scrum masters and others, to provide more balanced and reasoned view. As we all know, there are a whole multitude of metrics baying for our attention, and it is not always obvious which are critical and which are less important (and that is often a factor of timing and priority).
One thing we did do this year is consider other complementary products, particularly in the area of identifying security vulnerabilities with both our own code bases and the open source 3rd party libraries that are routinely packaged with a released application. The latter category can often account for 80%+ of the actual app, so it's an important area not to neglect. Sonarqube does provide some support here i.r.o the integration of OWASP top 10, but it clearly isn't an area of strength when compared to more dedicated products. We did an RFP and have now selected two further products that will bolster this aspect considerably.
We have also moved forward with SonarQube in 3 important ways. First, we have upgraded our implementation to version 5.4 (prev. 4.5.x). This was important to many of our teams because some plugin support require the later version. The second change is that we have moved our implementation of the Sonarqube server into docker. Sonarsource provide an OOTB image on DockerHub which is a good starting point. We have enhanced it in a couple of ways to reduce the size and attack surface and also to add our specific config, but it was pretty easy to do so, so good job from Sonarsource here. The third difference is we have moved some of our install to use the Professional version rather than OSS. There were a couple of reasons, one was to access some commercial plugins which come bundled as part of the product and it made more sense (funding-wise). Another was to provide better support for a central SQ service. When I said 'some' of our installs, that was deliberate. We don't only provide SQ as a central PaaS, but also allow distributed DevOps teams to spin up their own, as long as they fully understand that operational support becomes their problem too of course (no free lunch here !). This works well for teams who want to manage more of their delivery pipeline rather than be part of a change control process where other participants might need to be consulted and perhaps engage in regression testing when changes are requested.
One significant change in v5.x is the movement of the database update to the server. This has a couple of important consequences. The first is that the build-breaker plugin is no longer useful since its harder to synchronise the fact that a build has failed with the update of the analysis outcome visible on the server. We use that plugin a lot, so it was a bit of a PITA. There is a compatible approach that SonarSource have documented, but personally I'm not a great fan because it increases the number of moving parts and thus the opportunity for something else to fail. But, with any upgrade there are always 'swings and roundabouts', and on the whole the positives outweigh the negatives (decoupling the client-side analysis from database update *is* on the whole a good thing). SQ v5 also comes with a bunch of new 'runners', now called 'scanners'. We have used the basic one, the Maven one and the MSBuild one, and all work fine. It's another change that you need to consider as part of migration, but not a massive one. Security controls have been enhanced in v5 and it's now easier to apply more granular access controls than in v4. For companies that outsource development work that's likely to be quite important (it is for us).
Licensing in the 'immutable server' world, whether that's docker or native Cloud remains unresolved. SonarSource seem a little behind the curve here, but we are talking to them. The key point is that we no longer stand up environments (including CI/CD pipelines) with any intention that they will have a 'shelf life' beyond their immediate use. Creating environments for specific use cases then tearing them down frequently (often this can be measured in minutes or hours) has become common-place for use and has tremendous advantages over previously used 'convergence' approaches using config management tools like Pupper, Chef or Ansible. Many vendors recognise this and have adjusted licensing arrangements, SonarSource aren't quite there yet (but they are willing to talk about it).
Anyway, that's probably enough of an update. I hope you find this, and the previous review helpful ?
Original Review (circa: 2014/15)
Moving to a largely evidence-based assessment is hugely beneficial, especially if you are managing out-sourced resources. It provides a clear definition of what acceptable quality actually means, and supports the decision of when you can stop, as well as what is not as there are no arguments based on an opinion. That said, metrics only take you so far, you still need smart people who can interpret and see beyond the base facts provided.
The ability to integrate analysis of software engineering metrics directly into the Development Lifecycle (DLC) in much the same way as any other practice such as Unit Testing. Specifically, developers run SonarQube analysis frequently and don’t commit changes to SCM when build breaker issues remain.
Early warning via CI build pipeline and especially setting up ‘Build Breakers’ based on a team or code base specific quality gate - a set of rules/thresholds that determine the most important measures for a particular code base.
Targeted improvement allows a team to identify specific areas of threat (e.g. TD) and then set purposeful goals to improve in those areas (rather than trying to improve everything).
The SonarQube community is very active which often means that finding solutions is a blog post away from other like-minded organisations. Community plugins are a staple for this product and have tremendous breadth and depth.
It would be utterly impossible to contemplate Continuous Delivery without including a major focus on ensuring affordable software quality. SonarQube plays a key role in this endeavour and provides Senior Management oversight across multiple project teams and business deliveries. Fits in very well with existing Continuous Integration build pipeline workflows. As we move towards Continuous Delivery ensuring a ‘no surprises’ release management.
Our software quality assessment at an affordable cost (licensing, time and effort). Previous attempts have failed to win the support of the development community (typically overly complex and intrusive and/or not sufficiently timely) without which the initiative will be doomed to failure.
