Try our new research platform with insights from 80,000+ expert users
Nelson Hernandez Guerra - PeerSpot reviewer
Developer Senior Genexus 16 Analyst at Migrate Brasil
Real User
An open source automation server that helps you build anything
Pros and Cons
  • "I love Jenkins. I like that you work on anything, and you make anything. Jenkins is very important for my team. I am satisfied with the product."
  • "Jenkins can be improved, but it's difficult for me to explain. The initial setup could be more straightforward. If you connect Jenkins with bookings and lockouts, it can be challenging."

What is our primary use case?

We use Jenkins for deploying with GenX and work on the pipeline too. We engage in many activities using both Jenkins and GenX. 

What is most valuable?

I love Jenkins. I like that you work on anything, and you make anything. Jenkins is very important for my team. I am satisfied with the product.

What needs improvement?

Jenkins can be improved, but it's difficult for me to explain. The initial setup could be more straightforward. If you connect Jenkins with bookings and lockouts, it can be challenging.

For how long have I used the solution?

I have been using Jenkins for three years.

Buyer's Guide
Jenkins
October 2024
Learn what your peers think about Jenkins. Get advice and tips from experienced pros sharing their opinions. Updated: October 2024.
814,649 professionals have used our research since 2012.

What do I think about the stability of the solution?

Jenkins is a stable product.

What do I think about the scalability of the solution?

Jenkins is a scalable solution. We have ten people using this product right now.

How are customer service and support?

Technical support was very good.

On a scale from one to five, I would give technical support a five.

How would you rate customer service and support?

Positive

How was the initial setup?

The initial setup depends on the project and related activities.

What about the implementation team?

I have implemented this solution myself, and I have also used a consultant on other occasions.

What's my experience with pricing, setup cost, and licensing?

Jenkins is not expensive and reasonably priced.

What other advice do I have?

I would recommend Jenkins to potential users. You can use Jenkins with other products and make anything you like. 

On a scale from one to ten, I would give Jenkins a ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Hisham Shoukathali - PeerSpot reviewer
Automation Technical Lead at a tech vendor with 10,001+ employees
Real User
Effective continuous deployment, simple multi-cluster implementation, and one-click setup
Pros and Cons
  • "The most valuable feature of Jenkins is its continuous deployment. We can deploy to multi-cluster and multi-regions in the cloud."
  • "Jenkins could improve by allowing more scripting languages. We need to use Groovy scripting and it is difficult to debug and it is not ideal for creating file scripts. We tried to search for assistance but we did not find much help."

What is our primary use case?

We are using Jenkins for running our test jobs, and multi-cluster deployments in the cloud.

What is most valuable?

The most valuable feature of Jenkins is its continuous deployment. We can deploy to multi-cluster and multi-regions in the cloud.

What needs improvement?

Jenkins could improve by allowing more scripting languages. We need to use Groovy scripting and it is difficult to debug and it is not ideal for creating file scripts. We tried to search for assistance but we did not find much help.

For how long have I used the solution?

I have been working with Jenkins for approximately three years.

What do I think about the stability of the solution?

Jenkins is not always stable. We have encountered approximately 20 percent downtime.

What do I think about the scalability of the solution?

The scalability of Jenkins could improve. If we are running a lot of jobs, it is not scaling up or down very well.

We have multiple jobs running and they can be between 50 to 100 at a time.

Which solution did I use previously and why did I switch?

I have not used another solution prior to Jenkins.

How was the initial setup?

The initial setup of Jenkins is a one-click deployment to multiple regions which is helpful. Additionally, it is easy to configure, and it is straightforward.

What about the implementation team?

Jenkins is easy to maintain.

What other advice do I have?

My advice to others is they should use Jenkins in the cloud. If they try to access the solution outside of the cloud environment, you need to configure whitelists and other configurations and keep an eye on them.

I rate Jenkins a seven out of ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: PeerSpot has made contact with the reviewer to validate that the person is a real user. The information in the posting is based upon a vendor-supplied case study, but the reviewer has confirmed the content's accuracy.
PeerSpot user
Buyer's Guide
Jenkins
October 2024
Learn what your peers think about Jenkins. Get advice and tips from experienced pros sharing their opinions. Updated: October 2024.
814,649 professionals have used our research since 2012.
Senior Team Lead at Peristent Systems
Real User
Simple setup, high availability, and scalable
Pros and Cons
  • "The most valuable features of Jenkins are creating builds, and connecting them with Sonar for Sonar analysis. Additionally, we connect it with other vulnerability tools, such as WhiteSource which is useful."
  • "Jenkins could improve the integration with other platforms."

What is our primary use case?

We use Jenkins for management purposes.

What is most valuable?

The most valuable features of Jenkins are creating builds, and connecting them with Sonar for Sonar analysis. Additionally, we connect it with other vulnerability tools, such as WhiteSource which is useful.

What needs improvement?

Jenkins could improve the integration with other platforms.

For how long have I used the solution?

I have been using Jenkins for approximately two years.

What do I think about the stability of the solution?

Jenkins is stable.

What do I think about the scalability of the solution?

I have found Jenkins to be scalable.

We have different teams within our company using this solution with different types of setups. We have approximately 25 users using the solution.

How are customer service and support?

I have not contacted the support from Jenkins.

How was the initial setup?

The initial implementation of Jenkins is simple.

What about the implementation team?

The installation of Jenkins is done by our team.

What other advice do I have?

I would recommend this solution to others.

I rate Jenkins an eight out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Sherief Shawky - PeerSpot reviewer
Software Development Manager at Intellisc
Real User
Open-source with a short learning curve but cloud repositories can't trigger on-prem Jenkins systems
Pros and Cons
  • "It has a lot of community posts and support."
  • "There is no way for the cloud repositories to trigger Jenkins."

What is our primary use case?

We use the solution for the whole automation cycle for the deployments. We are using Jenkins and pipelines for once commits or push commits on Bitbucket or directories. Jenkins is listening for those changes and is applying (or triggered by) the repository changes to deploy and run the test cases, automate test cases, and deploy them on servers for the deployment, testing, or production.

What is most valuable?

It's open-source and free to use.

The learning curve for Jenkins is not a big deal. It has a lot of community posts and support. 

The initial setup is simple. 

We have found the solution to be stable.

It is my understanding that the solution can scale. 

What needs improvement?

Jenkins is on-premise (on our infrastructure) and Bitbucket or Azure directories are on the cloud. Therefore, triggering from the repositories to the on-premise, Jenkins is not applicable. We are trying to reach them now, and we are currently using a plan or a process to listen to the repositories every once in a while to know if there are no new changes applied. It triggers the automation for the deployment and the running test cases, and therefore it may take two minutes or three minutes to have the deployment done after the latest commit. This is due to the fact that we are using on-premise Jenkins for on-premise deployment, yet have the repositories on the cloud. There is no way for the cloud repositories to trigger Jenkins. We are trying to research now how to have the Jenkins over a public IP, so the repositories can trigger it.

For how long have I used the solution?

We have been dealing with Jenkins for around three years now.

What do I think about the stability of the solution?

The product is stable. There are no bugs or glitches. It doesn't crash or freeze. It's reliable. 

What do I think about the scalability of the solution?

Although it is my understanding the solution can scale, we don't have much information about scalability for the Jenkins. We didn't investigate scaling yet.

How are customer service and support?

We tend to search for solutions online. I've never reached out to technical support. I rely more on the community. 

How was the initial setup?

It's pretty straightforward to set up the product. The DevOps team just took around two weeks or three weeks for the first deployment, for automation for the first deployment using Jenkins. It fulfilled our requirements. DevOps is not a target by itself, DevOps is an operation to remove any pain areas, or time-consuming tasks, or to automate it to have it in seconds. It fulfills our requirements.

What's my experience with pricing, setup cost, and licensing?

The product is open-source.

What other advice do I have?

For the development environment, we are using the on-premise infrastructure. For some customers we are also using on-premise; for other customers, we are using the cloud.

We have branches in Egypt and branches in Dubai that are using Jenkins for the whole automation process and we're really enjoying using it.

I would recommend the solution to others.

I'd give it a rating of seven out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Sanjeeb Pandey - PeerSpot reviewer
DevOps Architect at a tech vendor with 10,001+ employees
Real User
Plenty of plugins, lightweight installation, and effective third-part tool integration
Pros and Cons
  • "We are using the open-source version and there is a lot of plugins and features that are available and it works on agents for free. In other solutions, it will cost extra to use them with the agent."
  • "The UI of Jenkins could improve."

What is our primary use case?

We use Jenkins for building our applications, deploying our applications, and some automation tasks.

What is most valuable?

We are using the open-source version and there is a lot of plugins and features that are available and it works on agents for free. In other solutions, it will cost extra to use them with the agent.

What needs improvement?

The UI of Jenkins could improve.

For how long have I used the solution?

I have been using Jenkins for approximately six years.

What do I think about the stability of the solution?

Jenkins is stable. It provides all the required features for stability, such as backups.

What do I think about the scalability of the solution?

Jenkins is scalable because it is open source and it integrates with other third-party vendor tools which are currently in the market, such as Microsoft Azure or Amazon AWS. It gets very well integrated with all the new tools, it doesn't remain isolated.

We have multiple projects that are using this solution and each project has multiple users. In one project we could have 50 users or in another 10 users are using it.

How are customer service and support?

We didn't face any issues to escalate to Jenkins for technical help.

Which solution did I use previously and why did I switch?

We have previously used Bamboo. I use both Jenkins and Bamboo per our project requirements. Jenkins is more suitable for commercial projects and is more scalable and flexible as compared to other tools because its core focus is on integrating and updating automatically.

How was the initial setup?

The initial setup of Jenkins was straightforward. It's very lightweight and it only requires Java on your system as a requirement.

What about the implementation team?

We did the implementation of the solution ourselves with our team.

What's my experience with pricing, setup cost, and licensing?

We are using the free version of Jenkins. There is not a license required to use the solution because it is open-source.

What other advice do I have?

My advice to others is to explore Jenkins well and it is integrated with the scripting site. Teams should explore the scripting part of the Jenkins because everybody's nowadays is writing pipeline as a code for automating their operations. They should try to utilize the new feature provided to them, such as pipeline as a code. 

It does not matter what solution they are using, such as Microsoft Azure DevOps or Amazon AWS DevOps, Jenkins will integrate with other solutions. They should try and use Jenkins even if they're using some other tool.

I rate Jenkins an eight out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
PeerSpot user
Senior iOS Developer at a media company with 5,001-10,000 employees
Real User
Bamboo vs. Jenkins

A biased and subjective comparison of Bamboo and Jenkins as CI servers for mobile development, based on practical experience with both.

Continuous Integration and Continuous Deployment (Delivery, Distribution) has been around for quite a while. But surprisingly enough on a global scale it pretty much just got into its teen years in regards to mobile development. Well, subjectively, of course.

You can see all levels of mobile CI these days. Some would still install builds from Xcode, others would have a quickly patched up build server under their desk. Xcode Bots meet the needs of yet another group of people. Travis CI is good and for open source projects it’s probably the best option. And guess what, I know few successful iOS development companies that develop apps for enterprise clients, and have no CI at all.

The advanced level of CI would include distributed build systems with multiple build nodes, support for automated unit and UI tests, running tests on simulator and physical devices, automatic deployment to TestFlight, Hockey App, Over the Air, and much more. It becomes not just mobile development, but spans into areas like DevOps and others. Etsy’s blog post is somewhat outdated but still a very good example of where this path can take you.

If you decide to take mobile CI under your total control, you have to pick a build server to start with.

I personally have worked with Bamboo for 1.5 years and I’m dealing with Jenkins right now, so I have few insights and can give some comparison of the two.

Setup & Configuration

Bamboo installation and Jenkins installation tasks are about the same in terms of time and knowledge required. While installing one of the two you’ll climb a certain learning curve which will help you heaps if you ever have to deal with second option.

Both are built using Java. Being Java applications, both will require similar JVM configuration. Default configuration won’t really serve you well. You’ll experience out of memory issues as soon as you add a couple of basic build plans/projects.

And lot of other things are similar: configuration behind proxy, login vs non-login user (Launch Agent vs Daemon), OS X Keychain, iOS Simulator, etc.

GUI

Obviously, this is not a comparison criteria at all. This criteria is as subjective as it could possibly be!

Out of the box Bamboo UI looks much better than Jenkins version.

With Jenkins there are ways to improve your day to day user experience. You can customize theme and even make your custom UI improvements, like adding “Build Now” button where you like it.

You should start by installing Simple Theme Plugin and then configure it with one of the available themes. Not all the themes will look good, it all depends on the Jenkins version you have and browser you use, etc. I tried a bunch of them and ironically enough the only theme that looks good on our production CI box is called “Atlassian”. But I’m dealing with slightly outdated Jenkins version, you could get better results with up to date Jenkins.

There’s multitude of UI, List View and Page Decorator plugins available for Jenkins. Some of them must be good and help you customize you Jenkins look & feel and functionality.

Plugins

This is where the large community plays its role. Subjectively or not, Jenkins has a larger choice of plugins of all kinds, starting from management and organization of build jobs and ending up with reporting.

Via a fancy pie chart diagram we are told that Bamboo has 151 plugins on Atlassian Marketplace.

A quick check on Jenkins Wiki and we get a count of 1,071 plugins.

Reporting is one of the most important plugin categories in my opinion. For example, I prefer to have full control over Xcode build tasks and use makefiles combined with shell commands wrapping xcodebuild rather than use Xcode plugin. Part of the reason is to be able to migrate to another build server with less effort and to be able to run CI tasks on my development laptop. Same goes for other things like uploading to / downloading from S3 bucket, transferring files with rsync or scp, and so on. But in no way I can come up with scripts that will produce good looking reports including HTML formatting, diagrams and images, plus integration into Bamboo or Jenkins UI. For this task I prefer to use plugins.

Jenkins features 127 plugins just for reporting, that’s almost as much as Bamboo can offer in total. Of course, quality of all those plugins deserves a detailed research. Just numbers don’t tell much. But this post is based on some hands on experience, so I’ll compare some reporting plugins I have used over time.

Publish HTML

If you ever used Clang Static Analyzer for your iOS projects, you know then that there’s no proper reporting plugin for this task. You end up with HTML report that needs to be published and made available in build project/plan summary page.

With Bamboo you create a new Shared Artifact, with Jenkins you use HTML Publisher plugin.

Unit Tests

Of course you run unit tests as part of your CI, don’t you? In that case you are well covered by reporting plugin on both Bamboo and Jenkins. Jenkins plugin also includes trending graphs, which is nice.

Cobertura Code Coverage

Of course you don’t just run unit tests, but also gather code coverage data, don’t you?

This is the first time Bamboo falls one plugin short. You can check related discussion for more details.

Jenkins has your back on this one with Cobertura Plugin. You can configure custom threshold values to mark builds as failed if you tests don’t provide enough coverage.

Static Analyzers Reports

OCLint is amazing tool to run another round of static analysis on your code and detect vast number of issues as well as to enforce coding guidelines. OCLint can produce output compatible with PMD output format. So it can then be processed by Jenkins PMD Plugin. You end up with browsable report with issues grouped by priority, category and other criteria. You can navigate all the way down to the line of code causing the warning. Once again, you can configure threshold values to mark builds with lots of warnings as failed.

In fact, the same reporting plugin should be able to pick up output of Clang Static Analyzer which I mentioned before. However I couldn’t figure out the way to make Clang Static Analyzer to spit out correct XML file.

With Bamboo, unfortunately, all you have is publishing HTML report via shared artifact.

P.S. I’m a big fan of Swift programming language, but one thing makes me a bit sad. It will take some time before we get tools like OCLint available for Swift.

Warnings

If you run CI tasks such as build, test, analyze and lint, often you don’t want your build project/plan to stop immediately if it encounters an error, for example test error. You still want your reporting tasks to run and pick up those errors and generate reports for them. This often leads to a problem where you have compilation error but the build is marked as passing. Preferred way to avoid this issue is to have a reporting task which will pick up all the errors and mark build as failed if needed.

Jenkins has Warnings plugin for that. It will scan build logs and detect warnings and errors generated by compiler, and it includes support for clang so xcodebuild is well covered. All you need is to configure thresholds and fail builds if it has compile errors.

I don’t know about Bamboo plugin to do the same job. I remember myself few months ago grepping test logs for errors and then marking builds as failed by doing something like exit 1.

Functional Tests

If you happen to use frameworks like Calabash that produce Cucumber test reports, then Both CI servers have plugins to provide nice reports.

Higher-order Plugins

Forgive me this injection of so popular these days functional slang, but this is how I want to introduce The Mother and The Father of all Jenkins plugins - Jenkins Job DSL Plugin. In short, it allows you to manage your configuration as a code and to generate and update build projects from code. What could be better for a developer?

Check out these resources to find out more.

I plan to write a post about my personal experience with Job DSL plugin.

Jenkins Job DSL plugin was a game changer for me personally. When I started working at new place a while ago, I had year and a half experience with Bamboo and acted as a strong Bamboo advocate initially. Until the moment I discovered unlimited power of Job DSL plugin. I personally can’t imagine going back to Bamboo and dealing with UI to update dozens of similar plans by hand.

That said, I still like Bamboo very much, particularly the way it integrates with the rest of Atlassian products. I hope Atlassian will implement Build Plan DSL plugin or provide API to make it happen. Bamboo Trade Depot Plugin is the only thing that could match Job DSL plugin, but unfortunately it’s not even close.

Build Plan/Project Structure

There is a bit of confusion caused by terminology used by Bamboo and Jenkins.

Bamboo

With Bamboo you start by creating Build Plan.

Each plan consists of one ore more Build Stages. Stages run in sequential order. If one stage fails next stages are never executed. Stages can be configured as manual to be triggered by hand.

Stages contain Build Jobs. Build jobs in one stage can run in parallel if there’s enough build agents for that. Each job can require different capabilities and can be dispatched to run on some designated build agent. Build jobs may produce Build Artifacts and share them with other jobs in consequent stages. Since jobs are parallelized, if one of them fails other jobs will still keep running until they finish on their own. This feature can significantly reduce build time, instead of sequential Build ⟼ Test ⟼ Analyze ⟼ Lint running 10 minutes, you get parallel Build || Test || Analyze || Lint running for about 3 minutes (given that longest job takes around 3 minutes).

Finally each job is made of Build Tasks. Tasks run in order from top to bottom. A task may be a basic shell script or one of the many tasks provided via plugins. Here’s a generalized example of a job and it’s tasks. One large and important group of taks is reporting tasks.

  • Checkout git repository
  • Build
  • Test
  • Deploy
  • Generate test report

A summary of a Build Plan structure

  • Build Stage
    • Build Job
      • Task
      • Task
      • [more tasks]
    • [more jobs]
  • [more stages]

Not So Parallel

Bamboo deserves a special side-note in regards to parallel job execution and iOS build plans.

As I mentioned before, the way you assign build jobs to local or remote agents is via capabilities. An agent defines capabilities it has, a job declares capabilities it wants, and then Bamboo matches the two.

If you are in total control of your CI setup and mobile team is the only one using your particular Bamboo server and all agents, you have all the power to set all the agents’ capabilities and then enforce a requirement that all build jobs created by you or your teammates must explicitly specify which capabilities they need. This way you’ll harness the full power of distributed configuration, all build jobs will run only on the agents they really should run on.

Another situation is when mobile CI is a new addition to company CI setup. There is already a CI server and few dozens of build agents and a lot of other teams using this CI setup. Lots of teams with lots of plans created over the years.

Now imagine that you are adding your specialized Mac build agent to be used for Xcode builds only. You setup and configure remote Mac agent, connect it to the server and… all the other build plans start jumping on your Mac agent! That’s because 99% of those plans declare no capabilities they require, they simply expect that all the agents are identical in terms of capabilities. And that works because all the agents are indeed identical clones. Well were identical clones until a new Mac agent was added.

There’s no easy fix to this problem and you can tackle it in one of 2 ways.

Ideal Solution

Ideally, CI must be done right. All the build plans must be maintained, updated and removed if no longer needed. As a requirement, all build plans must explicitly declare capabilities they require to be able to run. This is something to be enforced at team management level. Company has to have guidelines for creating and managing build plans and there must be a person or even a team (Dev Support team) responsible for keeping guidelines up to date and enforcing them.

Down to Earth Solution

The reality is rough. The number of existing plans is overwhelming, it will take months to chase people responsible for each build plan and communicate the importance of declaring capabilities to them. The whole change has to be made in a safe way so it doesn’t break existing workflows and production deployments.

You don’t have months of waiting allocated in your schedule, you need mobile CI running ASAP. One thing you could do is to setup your own CI server just for mobile and essentially move to “under the desk” setup. This way you get no support from Dev Support team (given that you have one) and all the trouble of setting up, configuring and supporting the server and agents is now yours.

However, you still can do mobile CI as part of company wide CI. There is a plugin that will help you - Bamboo Group Agent plugin. Have a read if you interested, Group Agent plugin offers a solution which is not fully flexible, but will help with original problem.

Jenkins

With Jenkins you start by creating Build Project, which is on occasions called Build Job causing certain confusion. In this post I’ll stick with Project term.

By default all you get is a basic Freestyle project that includes

  • Description
  • Parameters
  • Build Triggers
  • Build Environment
  • Build Steps
  • Post-build Actions

If you draw an analogy to Bamboo, then all you get is a build plan with single stage containing single job and list of tasks (Build Steps and Post-build Actions are nothing but tasks). That’s it. There are no stages and no way to run anything in parallel.

This is where plugins get into play. Mutlijob Plugin does exactly the same thing as Bamboo. Stages are called Phases, phases include Jobs. Jobs run in parallel when possible, while phases execution order is sequential.

One very important distinction is that jobs in multi job project are actually references to existing build projects.

  • Multi Job Build Project
    • Build Phase
      • Build Job 1 ⇢ Build Project 1
      • Build Job 2 ⇢ Build Project 2

In theory you can have a multi job project that includes a job which is a multi job project… You could even unintentionally create build job retain cycles and an infinite build loop.

To satisfy your craving for code Multijob Plugin support is added to Job DSL Plugin.

Once again with a fair bit of work Jenkins can match Bamboo’s default functionality and then with another fair bit of work can surpass it.

I mentioned artifact sharing briefly for Bamboo. For Multijob plugin there is no proper artifact sharing support yet. There’s a number of open tickets (JENKINS-20241, JENKINS-25111, JENKINS-16847, JENKINS-16847) with workarounds available. So the problem can be solved but it’s not part of official Jenkins release yet.

Not So Parallel Either

Jenkins is still prone to the same problem Bamboo is when it comes to adding iOS build nodes to existing infrastructure. Chances are high your Mac build node will be used by all the other build projects if capabilities are not configured properly.

Actually, in Jenkins world there are no capabilities as such, instead labels are used. Semantics are a bit different, but they serve the same purpose after all. Each build node can be labeled with multiple labels and build jobs can use flexible logical expressions to specify target nodes they want to run on. But then, if labels were not used in your corporate CI from day one, the amount of work required for labelling existing build projects can be too big.

At this moment I am not aware of a Jenkins analogue of Bamboo Group Agent.

Branch Management

Branches are an essential part of any source code management workflow. By supporting branches in CI workflow you get a number of benefits.

Timely Feedback

By running CI for each branch you provide developer with earlier feedback on their changes. They will know right away if changes they made are breaking the build, unit or functional tests, introduce static analyzer or linter warnings, etc. Developers can then fix these problems before they create pull request and involve the rest of the team in review process.

Earlier Tests

If you have testable build for each branch, you can make it available to your test team automatically as part of continuous delivery process. This way important and critical changes can be tested before they are merged to upstream branch (doesn’t mean you don’t do integration testing after the merge though). Bugs can be caught earlier and you end up with faster develop-and-test iterations.

Finally, some branches are never destined to be merged upstream. They may contain some experimental feature code, something you still need to build and make available to testers or internal stakeholders. For example have a special build to demo crazy design idea to your UI/UX people.

Branch Naming

I had to mention branch naming in a separate paragraph. For some reason, yet unknown to me, when it comes to naming a branch developers get completely wild. What I mean is that within the same team you can see branch names with and without prefix, using dashes or camel case, dashes and camel case combined, underscores… just anything. Sometimes the same developer would be very inconsistent when naming their branches. The very same developers are very disciplined when it comes to coding, i.e. class, variable and method names, and following coding style guide in general.

The upshot is that you have to have branch naming guidelines in your team, e.g. as a part of Git (other SCM) workflow. If the whole “issue” doesn’t look like an issue to you, wait until you have to manage this branchy havoc in regards to CI.

Bamboo & Branches

Bamboo does a great job with branches. With a single tick of a checkbox you can create branches of a build plan. By the way, you did understand it correctly, you create a branch of a build plan. Essentially, Bamboo clones the original build plan and changes its source repository configuration to point to a different branch. These plans still share same build phases, jobs and tasks, but you can configure some of the branch plan settings differently, that includes notifications, branch merging strategies, etc.

With a standard Java regular expression you can filter branches by their name and instruct Bamboo to ignore branch names that don’t follow guidelines. For example, the regex below will accept only master and development branches, and branches that are prefixed with bugfix/, hotfix/ or feature/ followed by JIRA issue ID and lowercase-with-dashes description.

<code>master|development|((bugfix/|hotfix/|feature/)\w{2,}-\d+(-[\da-z]*)*)

A good Java regex test website is RegexPlanet.

This is a perfect way to indirectly enforce correct branch naming. After missing couple of builds developers will figure out what’s wrong and change their habits. In certain situations you can always add a branch manually via Bamboo UI or CLI tools.

With another simple configuration field you can control aging of the branches. If branch hasn’t been updated for a long time, Bamboo will remove its build plan. Of course you’d want to preserve certain branches forever and there’s yet another checkbox to do that.

I’ve mentioned branch merging strategies and that’s one more feature that Bamboo provides out of the box. Using Branch Updater or Gatekeeper you can configure build plan to do upstream merge before or downstream merge after running the build, and even push merged changes back to the repository. This is a very good way to detect merge conflicts earlier and to keep your git workflow in order.

Bamboo’s branches support also shines when it comes to CI pipelines, more on that later in the post.

Jenkins & Branches

Surprisingly, plugins initially do more harm than good in this case.

On fresh setup Jenkins has no Git support (SCM I get to work with and thus choosing it as an example). You need to install plugin to work with Git and you will most likely install the most popular Git plugin. This plugin is very powerful but of all the things it does not create branches for build projects. Well to be honest it shouldn’t.

When you look for a solution you will think of trying some other plugins first. Thre are a few: Multi-Branch Project, Feature Branch Notifier and others. The common problem with all of them is that they have their own support for SCMs including Git, that means you don’t get to use the Git Plugin and all of its powerful features, instead you end up with a very limited version of it.

I tried Multi-Branch Project plugin, it was OK, but not good enough. I couldn’t get branch filtering to work, it kept picking up all the branches ignoring filters. There’s no option to configure branch aging, no branch merging strategies and so on. Finally, no simple and easy support for branched CI pipelines.

This is what I meant by harm in this case. You spend time trying different plugins and none of them would match Bamboo default functionality.

But all is not lost. Once again you will be saved by Job DSL plugin. Paired with Git Plugin it can work miracles. While Git Plugin can’t branch build projects it can work with branches as such. With one line of Groovy code you can fetch all the branches for your repository including information on their last update. Then with simple string regex match and date comparison you can filter branches just like Bamboo does, and then you can generate build project for each branch, organize them into folders and views to match Bamboo’s UI. Code a bit to generate branch pipelines, same for merging strategies and much more.

Follow this link to find an example of getting branches for GitHub repository.

If you are using Stash there is no direct support for it via Jenkins plugins afaik. But as long as you have API access you can follow Bitbucket example to have basic support for working with branches. If you don’t feel like coding too much, you can wrap some shell scripts in thin layer of Groovy code to get same results.

Yes, it takes time and learning to get used to Job DSL plugin, but the benefits are unmeasurable.

Pipelines

Pipelines is another name for CI/CD workflows. In certain cases single build plan/project is not enough, and that’s when you can create pipelines. Creating a pipeline means you chain build plans together, if first plan (aka parent) is successful it triggers its child plans. Child plans can have children of their own, and so on.

<code>. └── Parent Plan     ├── Child Plan 1     │  ├── Grandchild Plan 1.1     │  └── Grandchild Plan 1.2     ├── Child Plan 2     └── [more children]

A real world example of pipelines for mobile space is UI automation tests, often called Functional or Acceptance tests. UI automation is usually a heavyweight task compared to unit tests. If you run UI automation tests as part of a single build plan it can take too long to complete.

You can create a separate plan for UI automation only, but then you don’t want to trigger it every time there is a commit to SCM. There’s no point running UI automation tests if the build fails. So you add UI automation tests plan as a child plan to the main build plan thus creating pipeline.

This is just one example.

Bamboo Pipelines

Bamboo has support for pipelines out of the box. In parent plan configuration you simply add child plans and configure the way those are triggered, e.g. only when parent is successful.

Child plans don’t have to be configured in any special way, they are completely unaware of being some other plan’s children by default. The very same plan can be included in multiple pipelines acting in a different role.

The real power of Bamboo pipelines is in its support for branches. Child plan will only be triggered if it has the same branch as the parent plan. This means you have to configure branching for both plans to make it work. Normally this also means that both plans are using same repository, but it is not a mandatory requirement. If 2 different repositories have the same branch the feature will work all the same.

When child plan starts it can get artifacts from the parent via Artifact Downloader task. Yet again, the branch of the child plan gets artifacts from the matching branch of the parent plan, all is handled by Bamboo.

Jenkins Pipelines

Jenkins has no pipelines support by default. As usual plugins should be used.

This is something I’m only starting to work with, so this paragraph doesn’t have lots of details.

In general, Jenkins plugins let you match Bamboo functionality with a certain amount of work.

Most popular plugins used for pipelines are

Both are supported by Job DSL allowing you to script complex pipelines in code and change them easily.

Distributed Builds

Both Bamboo and Jenkins have support for distributed builds. Bamboo is using Remote Agents while Jenkins calls them Remote Nodes, sometimes referred as slave nodes or agents.

A side note - both servers support local build agents/nodes as well. Those are running locally (as the name suggests) on the same hardware as the build server.

Back to remote agents/nodes. Complexity of setting them up and configuring doesn’t vary much between the two CI servers.

Both will suffer from issues caused by sitting behind the company proxy, specifically if the server is located somewhere outside your company network (e.g. in AWS cloud).

Why the server is in the cloud and the agent/node is not? - you’d ask. Well, that takes us to next topic.

Mac Support

Right, this whole comparison is focusing on mobile CI after all, meaning you have to deal with one of the most popular mobile platforms these days - that is iOS.

To build an iOS app you must have Xcode, which can run only on OS X (unless you want to follow the path of certain insanity and make it work on other OS).

Hackintosh

Not a very good idea I’d say. The company does iOS development and wants to go with Hackintosh to setup OS X build server. Really?

Cloud (aka OnDemand)

Could be a really good option, but not with industry giants like AWS. AWS or alike is where you’d go if using Bamboo OnDemand feature. I can find posts as old as 2011 discussing this issue: one, two. Apple doesn’t make it easy to run virtual OS X instances to the extent that none of the big cloud providers are brave enough to provide official support for this feature. These days you can go with one of the many Mac Mini colocation services. You either rent a Mac hardware or even provide your own.

Self-hosted

This is also an option. If your company has security concerns about those Mac hosting providers, or doesn’t want to spend money for that, or for any other reason, you can always purchase Mac hardware and host it in your data center.

Whenever you are using self-hosting or remote hosting, you end up dealing with native hardware. I find that Mac Minis with maximized CPU, RAM, and SSD storage are perfect candidates for iOS CI box. The more you have the better.

Next step is to install remote agent/node and get it running. As I mentioned already, installing remote agent for Bamboo is around the same complexity as installing remote node for Jenkins. The problems start popping up when you begin using them.

Bamboo Remote Agent

hings you definitely need to know in regards to Bamboo remote agent.

[The rumor has it that] Atlassian are using Mac OS X to develop their products, including Bamboo, but OS X is never ever listed as officially supported. Indeed, why would you choose to run your JIRA, Stash, Bamboo, whatever, on OS X server? Hopefully with increasing demand for iOS CI Atlassian will put a bit higher priority on fixing Bamboo issues for OS X, and there’s plenty, btw.

Before you even start using remote agent on OS X, you have to experience a bit of pain when trying to set it up.

Major and the most important group of issues is related to Artifacts Sharing.

Whenever your remote agent finishes a build stage it is most often producing build artifacts. It doesn’t matter if those artifacts are shared or not, they must be copied to the server anyway. That is part of distributed build system. Your remote agent can go offline any time, your build plan jobs can run on different agents with different capabilities and artifacts must be passed from one agent to another. All in all, the reality is - the artifacts must be copied to the server.

Problem Proxy

Is your remote agent behind proxy? May I introduce you to HTTP 1.1 Chunked Transfer Encoding then? Well, not to the feature itself, but how it relates to Bamboo: BAM-5182, more.

Bamboo server requires support of HTTP chunked transfer feature of 1.1 protocol version to pick up artifacts. If your proxy doesn’t support this feature, you are in trouble. Strictly speaking, this is not Bamboo’s problem, this is the problem of your company network team. HTTP 1.1 standard was released in 1999! There is a lot of HTTP proxy implementations that support it, nginx for example. However, things move really slow in most of big companies when it comes to changing network infrastructure. If you are so unlucky and you company is still stuck around 1999 in terms of network infrastructure, you will most likely have to find a work-around rather than waiting months and months before you get any progress on proxy upgrade.

Problem Atlassian

But wait then! Too early to take all the blame off the Atlassian’s shoulders!

First of all, Bamboo remote agent JAR totally disrespects JVM flags for HTTP proxy whitelist (nonProxyHosts), upvote! You can find ways around this issue, for example re-routing network calls using tools like SquidMan, but then you will face The Final Blocker: BAM!, BAM!.

Yes, Bamboo remote agent is 27 (twenty seven!) times slower than plain scp (secure copy over ssh) command when it comes to copying a single .zip file which is around few hundred Mb in size.

Imagine that after spending all the time trying to figure out issues around your proxy and enable the agent to share artifacts with the server, you end up facing this beast? It renders all your distributed build setup useless, it takes way more time to copy build artifacts than to produce them. From reports it looks like this issue is specific for Mac OS build agents only.

When I faced this problem I ended up sharing artifacts via Amazon S3 bucket. This is extra work, extra shell scripts to upload and download artifacts, additional expenses for S3 bucket. You become responsible for managing outdated artifacts, you are the one who has to account for multiple build plan branches, and much more. This is really a bit too much of overhead and it is extra annoying when you know this is a core Bamboo functionality and it’s supposed to work right out of the box.

Jenkins Remote Node

Read the Jenkins Remote Node installation post to get initial overview.

As you can see, there are common problems related to running SSH sessions as non-login user and others. I personally ended up running remote node as OS X Launch Daemon. This works, but is not ideal.

I have nothing yet to say about artifact copy speed from slave to master in regards to Jenkins setup, but stay tuned and/or post a comment if you have any knowledge on the matter.

Android

I wanted to mention Android in this last section, after all it’s mobile too.

I wrote one post about building Android - Build Android in The Cloud which already provides some details.

In regards to Bamboo vs Jenkins comparison, almost everything I mentioned for iOS applies to Android, except the very big and troublesome Mac OS X part. You can, and you should build Android on Linux systems, and that’s exactly what AWS instances are running on! It means that if you have 25 build agents/nodes in the cloud, all 25 of them can be used to build Android projects.

I had experience with building Android projects using Bamboo OnDemand setup hosted in AWS cloud. It worked (and what I was last told it still works) flawless and scalability of AWS really pays back. Despite the fact that building Android project was way more slower than building similar iOS project, that wasn’t really a concern given the number of resources available to do the job. I still had to run Android UI Automation tests (usign Calabash) on Mac OS X box, because Linux agents in the cloud didn’t have enough horsepower to run Genymotion or Android Emulator reliably.

Summary

Despite my excitement with Jenkins Job DSL plugin there’s no clear winner for me.

Both solutions can be used for enterprise level mobile CI. Both can be used in pure UI-driven way while Jenkins also offers a code-driven approach with certain trade-offs.

I’d say, whichever server you use, whatever your CI requirements are, just make sure you take it seriously. In some way I’m calling out to all of you to help mobile CI to get out of its teen years.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user272709 - PeerSpot reviewer
it_user272709Project Manager at a tech services company with 10,001+ employees
Real User

really helpful one, I just do not want to be carried away with “Atlassian" stack.

See all 2 comments
it_user181050 - PeerSpot reviewer
Senior Software Automation Engineer with 501-1,000 employees
Vendor
Open source community with many plugins although stability for all but the most popular plugins needs to be improved.

What is most valuable?

Open source community with many plugins & flexibility as an automation platform.

How has it helped my organization?

Rather than send emails and tickets around we can hand out "buttons" to teams so they can do self-service for actions that used to involve manual effort from the Operations group.

What needs improvement?

Polish for the user interface, stability of plugins beyond the very most commonly used ones.

For how long have I used the solution?

Over seven years.

What was my experience with deployment of the solution?

No issues encountered.

What do I think about the stability of the solution?

No, although in some scenarios memory use can bloat over time.

What do I think about the scalability of the solution?

Not directly, but because of our overcomplicated networking setup we had to spread over multiple Jenkins masters each with a set of nodes.

How are customer service and technical support?

Customer Service:

Open source version, but have occasionally asked a quick question in the IRC channel and sometimes get a clue or two.

Technical Support:

Open source version, but have occasionally asked a quick question in the IRC channel and sometimes get a clue or two.

Which solution did I use previously and why did I switch?

We started with Bamboo, but it wasn't flexible enough (at the time) to also be used for general automation (code deployments, application restarts, etc). Really we might have just expected too much of it.

How was the initial setup?

Easy. Drop in .war file, and restart the Tomcat server.

What about the implementation team?

In-house.

What was our ROI?

Hard to say. Although it was free, so infinite.

What's my experience with pricing, setup cost, and licensing?

Free. Takes some work hours of course, but those come back many fold in improved productivity through automation.

Which other solutions did I evaluate?

No, we picked a long time ago before there were a lot of alternative options on the market. The few others available were expensive. Jenkins (well, Hudson at the time) was free!

What other advice do I have?

Do it! There's really no way you can lose. Even if you decide Jenkins isn't for you then you only spent work hours that helped train your staff and create reusable scripts that can be applied in other tools just as well.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Staff Engineer - Product and Platform Engineering at Altimetrik (Deployed at FORD)
Real User
Great open-based framework, but the integration support and reporting could be better
Pros and Cons
  • "Jenkins's open-based framework is very valuable."
  • "It can be improved by including automated mobile reporting integrations."

What is our primary use case?

We use this solution in conjunction with Java which is installed. We have to give it the main part, our desk framework and the GI repository. The solution automatically takes the code from the GI repository and automatically executes it as a face task. It could be done at a set time for minutes, hours, and any day of the week. For example, if we want to get it right every day, it has to be set automatically to take the quote.

What is most valuable?

Jenkins's open-based framework is very valuable. Most of the time, we go open-based and use any test automation, not only for the automation framework but for the developers. They will trigger the jobs also using Jenkins with blue ocean, but there is a cost, and anything you need can be related to Java. For example, if you want to build your application and deploy it, Jenkins takes one day compared with CA or other circles, and in addition, the bamboo Jenkins is a popular solution.

What needs improvement?

The solutions integrations support and reporting could be improved. Currently, Jenkins provides the features automatically. However, if we can trigger the job from our mobile, that would be great. We have done it once, but the next time we tried, it did not work. For example, when I was in India, I tried to execute our Jenkins job but could not. However, when I put the privacy on my data and phone, I connected to the VPN, and it automatically triggered.

For how long have I used the solution?

We have been using this solution as users for over six years and are currently using version 2.23.1.

What do I think about the stability of the solution?

The solution is stable.

What do I think about the scalability of the solution?

The solution is scalable. We can also add users for the particular solution, but that does not apply to the free version. We use the enterprise edition. Enterprise edition means it creates the domain for automation. Public members, can't access this edition, so if you're adding users for your members or groups, you have to go for the land visitation and the maintenance.

In my organization, over 50 users use this solution, specifically developers and QA leads. Not everyone has access to Jenkins because of the use of only one username and password. For example, if I'm developing scripts, my team members also develop them, and we push into the solution. But for Jenkins, it's only one access.

How are customer service and support?

We do not have any experience with customer service and support.

How was the initial setup?

 The initial setup is easy. If you follow the documentation, it only takes a maximum of 20 minutes.

Which other solutions did I evaluate?

We chose this solution because the deployment was right. We have to go further when it comes to the interface edition. Also, it's less when you are competitive with the travel CVA and the bamboo, and we will find the resources using Jenkins easily when it comes to the market level. So that's why we preferred Jenkins.

What other advice do I have?

I rate this solution a seven out of ten. My advice is to go for the proof of concept. Go with the open source and follow this solution because it works. If you get a paid version, you'll have a trial version for some days. If it suits your requirements, then you can purchase it. Otherwise, if you purchase it and it does not meet your needs, then it's a waste of money. See how you can model the integrations, the automation, and the frameworks and then go further into the interface solution.

The solution is good but it can be improved by including automated mobile reporting integrations.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user