What is our primary use case?
I work in a consulting firm responsible for adding, managing, and deploying government projects. We are using Microsoft Azure DevOps in one of the projects for backlog management, test planning, test execution, sprint planning, bug fixes, and enhancement requests. We use the solution for anything related to development testing.
What is most valuable?
The solution's most valuable features are backlog management, build release pipeline, and testing. They're easy, intuitive, and increase productivity. Usually, if you don't use such a solution, you end up using Excel. Then, you won't have shared documents, and there'll be no single source of growth. Everybody will keep a different document somewhere, and you will spend a lot of effort reconciling the latest status.
Using Microsoft Azure DevOps makes it really easy for us. Anytime you can see how many bugs are open, you can directly get it out of the tool. The solution's reporting is really easy. You can create ad hoc reports based on management requirements. If you are sitting in a meeting and somebody asks you the number of chain requests, bugs, or enhancements, you can create quick queries and show them the status. I think this directly affects productivity.
What needs improvement?
Microsoft Azure DevOps doesn't have an ITSM tool compared to its competitors. We also use Jira for another project, and Jira supports ITSM or ticketing. Since Microsoft Azure DevOps doesn't have this feature, we have to depend on another solution for service request management for support tickets.
The solution should include ITSM tools and security. DevSecOps are third-party security plug-ins that you can integrate with DevOps. Azure DevOps itself doesn't have anything out of the box. Enabling security so that the solution automatically starts checking things would be a really handy feature.
For how long have I used the solution?
I have been using Microsoft Azure DevOps for three years.
What do I think about the stability of the solution?
We haven’t faced any issues with the solution’s stability.
I rate the solution a nine or ten out of ten for stability.
What do I think about the scalability of the solution?
Since it's a SaaS solution, we haven't faced any scalability or performance issues, and we haven't struggled when we had a lot of users. We have gone through a curve. We started with around ten users. At the peak of the project, we had almost 50 users. Since we are in maintenance, we have come down to 10 to 15 users.
We use 100% of Microsoft Azure DevOps for our project. Everything is within Azure DevOps. If anybody says that we need to work on a feature, the first thing we do is create a DevOps item. So, we don't do anything outside DevOps.
The tool provides the features, but we haven't been able to onboard end users. We are a consultancy firm that works with system integrators and also engages with the end client. We have been able to onboard the system integrators, and we are also using it.
However, the end users still prefer sending emails and documents. If you send them a link to run a test script, they won't do it. So, the end users still prefer the old ways, such as emails and documents.
I rate the solution's scalability a nine or ten out of ten.
How are customer service and support?
So far, we haven't faced any issues in terms of technical support. There is good documentation available if you are looking for support for configuration. So, you usually end up resolving your issues yourself. Since this tool is widely used, you can find help online. People are writing content about this solution, and Microsoft itself has good documentation.
How was the initial setup?
On a scale from one to ten, where one is difficult and ten is easy, I rate the solution's initial setup a nine out of ten. The solution's initial setup is pretty easy, and the rollout is pretty quick. You can enable it and then keep on modifying and updating it.
What about the implementation team?
It took us less than a week to deploy Azure DevOps. Since we were using a cloud environment, there was no infrastructure requirement. We went on Azure DevOps, created an organization, and then created a project. Inside the project, we selected the type of project.
There are different templates that you can follow, including the CMM-level approach or the basic approach. We selected one of the templates and copied the template. We made some modifications to the template for the project because that template is used for governing steps.
Then, we created depositories, which is pretty quick. In a week's time, we were up and running with backlog management. It took a couple of weeks to complete the automated build and deployment pipelines.
We needed one person to set up the project and one knowledgeable about the build and deployment pipelines. If you have a person who knows how to do the pipelines, you can also configure the project. So, one person is good enough to set up the entire project.
What was our ROI?
We have seen a return on investment with Microsoft Azure DevOps in terms of productivity because it really helps with the amount of time you need to consolidate reporting and planning. The status is always up to date, and the deployment is very streamlined. You can do the entire thing in Excel, but the overhead would be too much, and you would lose out on things. So, team synchronization and productivity are the return on investment with the solution.
I rate the solution’s return on investment a nine out of ten.
What's my experience with pricing, setup cost, and licensing?
The solution's pricing is pretty cheap. The best part of the Azure DevOps and SaaS model is that there's no upfront cost. The tool has a per-user license. It's free for five users, and there is a price above five users. The solution's deployment and licensing costs are very cheap compared to those of its competitors.
The solution's pricing is not fixed. The solution's testing license is $50 per user. It's $15 for normal users who use backlog management. We have two people from the test team and seven from the other team. This is in maintenance.
Since we had a big testing team, we had 15 people in testing and 30 people in backlog management during peak time. You can say it has a 70:30 ratio. Most of the cost is in testing, and the backlog management is really cheap.
On a scale from one to ten, where one is cheap and ten is expensive, I rate the solution's pricing a three out of ten.
Which other solutions did I evaluate?
Before choosing Microsoft Azure DevOps, we evaluated other options like Jira and HP ALM. Jira is good at ITSM and backlog management, but it is dependent on third-party tools for pipeline deployment.
It's too complex to do product management with HP ALM. It's a good ITSM tool, but the process it follows for product management is very stringent, which is not very flexible for sprint planning. There is too much overhead in HP ALM to do quick sprints.
What other advice do I have?
We are working with the SaaS (Software as a Service) version of the solution, which is on the cloud. Since Microsoft provides the latest upgrades and patches, it should be the latest version.
We start by creating backlog items. Whenever we get a requirement, we log it into Azure DevOps and plan the backlog. The backlog includes what features we need to develop and what tasks we need to assign to each developer. Each developer is part of the DevOps. Once you have created that backlog, we assign it to different developers based on a sprint.
Suppose we are going to run a four-week development cycle. So, we plan the development cycle, pick a few items from the backlog, assign them to that sprint, assign them to the developer, and then manage the execution of that development cycle. Once that's completed, we will transfer it to the test team so they can test it in Azure DevOps.
They have test scripts that are documented in Azure DevOps. They run tests, record videos, and capture screenshots in Azure DevOps. After the test verification, we deploy the solution. In addition to backlog management and product management, we use Azure DevOps for build and release deployment. We don't manually go and build the software.
Our code repository is also part of DevOps. As soon as we check in the new code, Azure DevOps automatically builds the solution and then deploys it in the development environment. Once it's confirmed, the same is deployed to quality and production. We use the solution to do everything end to end, other than ITSM.
Specifically, Azure DevOps is integrated with deployment for us. When we manually deploy a solution, it's prone to errors. We use Azure for website deployment and Azure DevOps for Apple app or Google app deployment.
As soon as the approval is done in Azure DevOps, apps are automatically published. It will publish an app on the Google Play Store, Apple Play Store, and Azure, which we use for web hosting. So, it is integrated with web hosting, Apple Store, and Google Play Store.
The solution does not really need any maintenance. Once you enable the testing solution, you can start creating your test plans and test scripts directly. Every time you do a deployment, you just need to run those test scripts, which is pretty easy. It's more about creating your test script than configuring the tool. Even if I do it in Excel, I need to spend time on that.
The solution's analytics and reporting are pretty easy. We use them very often on an ad hoc basis whenever we discuss and plan what to deploy and what the next steps are. It's pretty easy, and we haven't faced an issue where we weren't able to take out any reports just by doing it on an ad hoc basis. It's pretty easy, and you don't need to write code or anything.
The tool is pretty flexible and easy to use. I suggest starting with the cloud version because you can create your project easily. Since it's free for five users, organizations with budget constraints can start playing with limited users. I would say start with the cloud-based version and start playing with it. Once you get comfortable with it, you can expand it for other projects. The tool serves a wide variety of use cases.
The biggest key trend these days is fast deployments or quick releases. Given how competitive the market has become, you need to keep on adding features to your product. Azure DevOps supports the sprint methodology, which supports fast deployment.
On top of that, it supports automated build release deployment. That was a headache when I started working. Sometimes, you forget a file when deploying in production, and your system will go down. The solution's features support the latest fast or quick deployment trend.
Overall, I rate the solution a seven out of ten.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.