What is our primary use case?
We're collecting code quality metrics.
How has it helped my organization?
We have the software metrics that SonarQube gives us, which is something we did not have before. This helps us work towards aiming coding standards to empower us to move in the direction of better code quality. SonarQube provides targets and metrics for that.
What is most valuable?
I like the dashboard it shows by default, where you can see things at a glance. At the same time, you can also drill way down and see a lot of stuff about your code, like complexity metrics, and things like that. It gives you a nice dashboard where you can just look at a birds-eye view.
What needs improvement?
We've been using the Community Edition, which means that we get to use it at our leisure, and they're kind enough to literally give it to us. However, it takes a fair amount of effort to figure out how to get everything up and running. Since we didn't go with the professional paid version, we're not entitled to support. Of course, that could be self-correcting if we were to make the step to buy into this and really use it. Then their technical support would be available to us to make strides for using it better.
On the other hand, there are published books available. However, the one problem I ran into is they were a little bit out of date. They're still very helpful, but we had to kind of translate from the previous version that was covered in the published books to what's actually available now.
An improvement I would like to see would be on the part of the authors to come out with a new edition or revision that covers some of the newer features of SonarQube and newer configurations. I'd buy a copy.
In terms of additional features, it's actually a very complete solution from what we have seen. Again, I would like the authors to revise their books. I think even ordinary people that are using the licensed model with direct support could walk through some different use cases, just from having been around the block a few times. There are enough things that the software does that this could be very beneficial. Even beyond the technical issues of installation, there are further use cases that could be helpful. For instance, how to get the big bang from the buck out of it.
For how long have I used the solution?
We've been using SonarQube for around eight months.
What do I think about the stability of the solution?
We use C++ and a lot of Python. Another group in our company is using Java. SonarQube is more directly suited for Java, being almost built into it, whereas C++ requires some extensions. The Java group is using a newer version. We were kind of hoping to piggyback on theirs but SonarQube did not create newer versions of the C++ interfaces as open source. It's starts costing money so we haven't crossed that threshold yet. We haven't established a clear path.
What do I think about the scalability of the solution?
I think if you're going to get the paid model, I get the impression it would do pretty much everything you need as far as metrics go.
A colleague of mine did some work looking at some plugins for Visual Studio and things like that, but they weren't going to work out, so we did take a look at some other options where they could have everything done on the desktop. Our solution in place now requires an infrastructure where it doesn't look at your code, but rather the code that you last checked in, which takes some levels of complexity that we've kind of built-in anyway. It's a little less intuitive how it works to the casual observer. It's set up now to where they don't have to know how it works, they can just go to the web interface and see it.
There are about eight programmers in our section of the solution. So we're kind of a smaller shop compared to some, but larger than many.
Certainly right now I think SonarQube is being underutilized, just because old habits die hard. If I had any say I would like to change that. We had coding standards in place, but they were written documents, whereas SonarQube takes that to another level and you had to look at the specification to see what you said you were going to do. It also tells you what the industry norms are, and whether or not you're meeting them. We have had some discussions about which we want to do. If we want it to happen automatically or if we want to go look for it again ourselves. I cast my vote in the automatic way because the research has already been done by the SonarQube community to come up with these roles, rules, coding standards, etc.
It wasn't done in a vacuum. The agile community has been beating on issues like this for a long time, and they're getting to a point that it's becoming a self-sustaining method.
How are customer service and technical support?
They do have a lot of information on their website for the parts that they're offering free. We don't have licensing but there is a lot of information, it's just a matter of digging for it and you have to infer a few things. With the proper amount of agony we've managed to get there. There are some subtleties as far as configuration parameters. It does it one way, but we'd really like to do it a different way. Finding that magic incantation to flip that switch is not always in bold print so to speak.
Even for the freebie community which we're in, they haven't held back information. The information is out there to do some amazing stuff with it, but you've got to get your shovel and go dig it up.
We do have some other licensed software and when you look for information on their product, all roads lead to them and when you get there, you log in with your account that costs tens of thousands of dollars. SonarQube isn't like that. They don't hold the information back but you just have to go find it on their website by yourself.
Which solution did I use previously and why did I switch?
We didn't have a previous solution other than paper systems that we never got in the habit of going back to referring to. We didn't switch, we started fresh.
How was the initial setup?
The initial setup was complex because we were using the Community Edition. We did have some issues with the compatibility of the different components. For example, there is the server itself, but then you can plug in different packages, like the C++ package. We've also experimented a little bit with Python metrics, but unfortunately we don't have a project that's really under that control yet, to really get a feel for how that works.
Configuration issues were pretty complicated, but once we got things up and running, it's been extremely stable, it was kind of maintenance-free, now, although we have a time issue. Of the scans that it does, it could be somewhat time-consuming, so originally some of the developers would say, "Well we want to be able to do that on our desktop." I told them, "I don't think you know what you're asking for, here." But as an alternative, we have it set up with our continuous integration server, which we use in TeamCity by the way. In the middle of the night, it automatically runs a scan for them, while they're in bed at home asleep so their results will be ready the next morning. This way, whatever they have most recently checked in, they can see the results right there. And then it runs in the background so it doesn't matter how long it takes per se, it gets it done by the next time they come in. That's part of what continuous integration does, it does things for you that years ago people would do themselves, and never get around to it.
What about the implementation team?
We spent a couple of weeks getting things figured out. I worked with an apprentice, who was kind of going through the motions.
We chose to use a Red Hat operating system for the base. It's running on a Red Hat 7 server which contributes to the stability from the foundation, then installed the actual SonarQube server on Red Hat. That's when we had the compatibility issues and so on when we started installing the scan engines on top of that. That's when things were not compatible with each other and we had to fall back and figure out why things weren't plugging and playing. However, they did have on their website a sheet that had a little chart that showed the compatibility between the different versions and once we discovered that I was able to see which version can work with which.
We didn't have to change the OS or the SonarQube's service itself, but the C++ extension. The version of the C++ extension we were using was not compatible with the Community Edition we had.
We've had a consultant at one point, not to look specifically at SonarQube, but rather at our firmer development processes as a whole. He's the one that played us towards SonarQube being a reasonable option. In fact, he was the one that helped us in finding the compatibility chart.
It's been mostly me doing the implementation on my own. I haven't been full time on it, but about half of my time is devoted to this. I do take some breaks and write some code and do some refactoring on occasion.
As far as time on SonarQube itself, only about a tenth of a person is devoted to this. It's part of an infrastructure. I have a whole family of virtual machines that do different things: build, test, etc..
Which other solutions did I evaluate?
We had looked at other code quality systems. We had looked at a number of them. I don't remember them all, but Clockwork was on that list. I think it comes down to picking one and getting used to how it works because they all do mostly the same thing. Some of them focus more on Java, some more on C++. I think Java seems to be the favorite. As far as what they can really do for you, there didn't seem to be any one of them that does ten times what another does. There were some differences, but not no show-stoppers that I recall. I guess the advice would be that one of several tools could do a good job for you, but you still have to manage it and manage the behavior that goes along with it.
What other advice do I have?
I would rate SonarQube as a nine out of ten.
Once you start drilling down through the menus, it tells you a lot of stuff about your code in one view. That's really quite neat. That shows you a view of maintainability. They have a maintainability view that shows bubbles for all the different code modules, and yours is beside the bubble. This represents the amount of "code smells," which is actually kind of a common definition. The bigger the bubble, the more your code smells. This shows where more attention is needed or it's a bubble that's kind of drifting out of control.
I have one graph here where there are probably 50 bubbles. There's one axis that shows technical death, meaning the amount of work that it's going to take to get the smells under control. The other axis is lines of code, which is obviously a very common thing to look at. On this particular graph, there are a whole bunch of bubbles down in the lower-left corner, which means you have a lot of small manageable things.
If you hover over the bubble, it tells you what module it is. How many lines of code. Technical death and manpower estimate, things like that.
Disclosure: I am a real user, and this review is based on my own experience and opinions.