Try our new research platform with insights from 80,000+ expert users
PeerSpot user
Mobile QA Developer at a tech vendor
Vendor
It's a good alternative to become parallel infrastructure purely dedicated for testing, but it lacks the ability to control interim status messages.

What is most valuable?

1) Easy to install and setup (including setup to run as windows service)

2) Free and always available alternative to any other build solutions

3) Relatively simple and intuitive UI

4) Big number of pretty useful plugins

5) Easily configurable and scalable

How has it helped my organization?

In some of my projects I had a chance to participate we used to use some commercial solutions like TeamCity or Bamboo. Since we had limited number of build machines and build agents it was really hard to organize automated acceptance and high-level system test runs as it took a while to perform the run.

Even more, since we had restricted number of build agents the huge number of long-running tests could make serious bottle-neck to the entire build process, hence, the feedback is a bit delayed. Jenkins appeared to be a good alternative to become parallel infrastructure purely dedicated for testing.

The bottle-neck problem was mitigated and we were able to make large scale runs on infinite basis. Thus, we could get results 2-3 times a day instead of 1 which brought us more consolidated and consistent picture about application under test state during the day. This trick was helpful for me on many projects

What needs improvement?

A lot of features (even small ones) can be taken from existing analogs. E.g.:

1) Ability to control interim status messages. This feature exists in TeamCity where you can log a message of specific format and it could change the build text and temporary status. In particular it's useful to see the number of tests already passed/failed without waiting for the completion

2) When we need to restart the server while some jobs are running it could be useful to re-run those dropped jobs after restart automatically.

3) There is some set of plugins which is being set up very frequently for many projects. It could be useful to have some pre-defined installations which either contain all necessary plugins by default or install them at the initial run (to decrease the size of initial application package)

Generally, other systems have lots of features which could be useful to see in Jenkins

For how long have I used the solution?

Since 2009. At that time it was called Hudson

Buyer's Guide
Jenkins
November 2024
Learn what your peers think about Jenkins. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
816,406 professionals have used our research since 2012.

What was my experience with deployment of the solution?

No. Deployment easiness is one of the advantages of Jenkins

What do I think about the stability of the solution?

From time to time Jenkins experiences problems after 1-2 weeks of intensive work (where at least 1-2 jobs are running at any point of time).

There used to be some issues when entire configurations could become invisible from the UI (usually that could happen after unexpected system shut down or even simple configuration rename operation) which was a bit painful. But I didn't encounter such problems in the most recent versions. So, it definitely indicates that some work at this direction was definitely done.

What do I think about the scalability of the solution?

No

How are customer service and support?

Customer Service:

Never had a chance to communicate

Technical Support:

Never had a chance to communicate

Which solution did I use previously and why did I switch?

Mainly Jenkins used to be either main system or parallel solution. Major reason for using it was mainly related to:

1) Licensing

2) Limited capacity of build agents

How was the initial setup?

Setup is pretty straightforward

What about the implementation team?

Both in-house and vendor

What's my experience with pricing, setup cost, and licensing?

It's free. There's no advice required

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
PeerSpot user
Software Tester at a tech vendor
Real User
Enables us to automatically build Python binaries into several operating systems
Pros and Cons
  • "We use Jenkins to automatically build Python binaries into several OS's i.e. OS X, Ubuntu, Windows 32-bit and Windows 64-bit."
  • "In our case, we have several products built using Jenkins. It is quite difficult to navigate into the latest stable build in a given OS."

What is most valuable?

We use Jenkins to automatically build Python binaries into several OS's i.e. OS X, Ubuntu, Windows 32-bit and Windows 64-bit.

How has it helped my organization?

We are a company run by remote workers. Using Jenkins really helps us in moving our products forward into a number of different customer segments.

What needs improvement?

I think the UI and the UX can be improved. In our case, we have several products built using Jenkins. It is quite difficult to navigate into the latest stable build in a given OS.

For how long have I used the solution?

Two years.

What do I think about the stability of the solution?

No stability issues.

What do I think about the scalability of the solution?

No scalability issues. As long as the configuration is set correctly, there is nothing difficult in scaling up.

How are customer service and technical support?

Jenkins is a free and open source application. So, StackOverflow is more than enough for us.

Which solution did I use previously and why did I switch?

From the very beginning, we wanted to target OS X, Ubuntu and Windows users. At first, the developer would manually create some builds and put them in Gdrive to be tested. We started to use Jenkins when we had some multiple developers and testers and needed a system to manage and automatically build our products.

How was the initial setup?

In my company, my role is a software tester. I don't know whether the setup is difficult or not.

Which other solutions did I evaluate?

We went directly to Jenkins.

What other advice do I have?

Don't focus on the fact that Jenkins is open source. It is tough as a rock.

This software is ideal for you who work in software development especially those using Agile methodology.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Jenkins
November 2024
Learn what your peers think about Jenkins. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
816,406 professionals have used our research since 2012.
PeerSpot user
Developer at a tech services company with 51-200 employees
Consultant
It’s not the best looking tool. It’s simple but can be complex and it has a load of plugins available for it.

It’s interesting how many places still don’t use continuous integration tools like Jenkins and how many places don’t automate their deployment systems. If that all sounds like
goobledegook to you, or if it makes sense but you’ve never used anything like it then
this post is for you.

Life before Jenkins

Before we talk about Jenkins I think it’s worth talking about my experience of life before
Jenkins, which will be familiar to a lot of people. Here’s what might have happened in an
average working day pre-Jenkins:

  1. I would write some code and commit it
  2. Occasionally a project manager or designer or someone equally focused on interrupting my programming would ask to see the latest version of the project
  3. I’d update my code and show them and then they’d ask if they could have a play with it from their computer
  4. 4. I would then interrupt someone in charge of deployment and would ask them to deploy it somewhere
  5. This person would probably groan (because they’re in the middle of something) and say that they could do it in 15 minutes
  6. The project manager would roll their eyes and sit at their desk reading emails or updating a GANTT chart (or whatever project managers do) for 30 minutes (because the task the developer was doing took twice as long as expected)
  7. The deployer would open their FTP program and copy files up to the dev server, or SSH the files up or whatever, possibly missing a few files the first time and then having to do it a second time
  8. The project manager would complain the work he wanted wasn’t there.
  9. The deployer would realise he hadn’t updated his code and do the process again
  10. Three hours later the same thing the process would start again…

It’s a slow, error prone, painful process and we haven’t even talked about deploying to staging or live environments yet. A major problem is also that a lot of people are unsure what the exact state of the project is because they never know if what they’re looking at is up to date.

 Jenkins to the rescue!

Here’s how that above process would work with Jenkins running.

  1. I write some code and commit it.
  2. Moments later the code is on a development environment without anyone having to do anything.
  3. The project manager looks at the development environment whenever they want to knowing it has the latest work on it
  4. The QA tester checks the release on the development environment and clicks a button in Jenkins to deploy to the staging environment.

Simple! (I even added an extra staging deploy part in there to look extra clever)

So, what is Jenkins?

The Jenkins CI (that’s its proper name) website describes it as “an awardwinning
application that monitors executions of repeated jobs, such as building a software project or jobs run by cron.”

So Jenkins is a tool that can be used to do any kind of repetitive task, but the added bonus of Jenkins is that these repeated tasks can be monitored and report information and make it easy for people to initiate these tasks or have them initiated automatically.

So for example, you might have a .Net project which is the server aspect to a project with iOS and Android apps. Jenkins can build and deploy the .Net project whenever changes are made so the latest version is available. The iOS and Android applications can also be built when code is changed and can be deployed via tools like TestFlight.

The “continuous integration” part comes into play when you have multiple developers working on a project. Jenkins is constantly building the latest version of the project from all developers' code so your work is being integrated (and deployed) continually.

I’m not going to get too deep into technical details since you can get those from the Jenkins website or via the excellent “Jenkins: The Definitive Guide” book but I’ll talk about what Jenkins would for a .Net project with a simple HTML/CSS/JS frontend codenamed
“Gumboots”.

  1. In Jenkins there is a “Gumboots” job which has been configured with the details of a
    GitHub repos containing the code for the project
  2. GitHub has been configured to hook into your “Gumboots” job whenever the code
    changes
  3. A developer pushes some code to the GitHub repos. GitHub notifies Jenkins about the
    new code.
  4. Jenkins gets the new code and runs a build.
  5. Jenkins has been configured to do a few things when the build is run:
           a) It runs a load of automated test cases
           b) If those pass it then builds the project using the MSBuild plugiN
           c) If that works it then runs a Gulp tasks on the CSS and Javascript to minify                   them and concatenate them.
  6. If that is all successful, then it will deploy the new code over SSH to the developmenT server

If any of these steps fail, then Jenkins will send an email with details of the problem to people involved in the project and most importantly to the person who wrote the last piece of code that may have broken the build.

Setting up jobs for Android and iOS applications is very similar and follows the pattern of “get code”, “build code”, “deploy code”.

There’s even a Chuck Norris plugin to give you extra feedback about the state of your jobs. Now really, what more could you want?

What next?

I’ve described a basic, but powerful Jenkins job set up. This will do a lot for you, but it can be taken further. 

The kind of things that you can get Jenkins to do that might be useful for you are:

  • Create a pipeline which pushes a build to dev automatically, then to staging after a
    manual QA check and then to live when a release is required.
  • Integrate Jenkins with an issue tracker like JIRA so that as Jenkins deploys code it tells JIRA what issues have been released
  • Get Jenkins to automatically run migration scripts on your databases when deploying and also get Jenkins to do one click rollbacks of your project if things go wrong
  • Create a parameterised Jenkins job which takes some input and then spins up a new
    server based on that input for you.

There are lots of other tools similar to Jenkins of varying complexity and approaches. These tools are becoming increasingly popular as cloud services which can handle platform
configuration and provisioning as well as the code deployment aspect.

Jenkins has always worked well for me. It’s not the best looking tool, but it has a large
community, it’s simple but can be complex and it has a load of plugins for it.

Hats off to Kohsuke Kawaguchi and the Jenkins team for building us something so useful.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user188790 - PeerSpot reviewer
QA Automation Lead at a tech company with 51-200 employees
Vendor
​Provisioning VMs was an issue during deployment but ​automation in general has been improved.

What is most valuable?

I can build slaves and extensions through plugins.

How has it helped my organization?

Automation in general has been improved.

What needs improvement?

Provisioning of VMs during deployment.

For how long have I used the solution?

Five years.

What was my experience with deployment of the solution?

Provisioning VMs during deployment was an issue.

What do I think about the stability of the solution?

No issues encountered.

What do I think about the scalability of the solution?

No issues encountered.

How are customer service and technical support?

Customer Service:

Not needed.

Technical Support:

Not needed.

Which solution did I use previously and why did I switch?

We previously used Hudson, and CruiseControl.

How was the initial setup?

Initial set-up was straightforward.

What about the implementation team?

We implemented in-house.

Which other solutions did I evaluate?

We compared Jenkins to CruiseControl.

What other advice do I have?

It works.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Fatih Mehmet HARMANCI - PeerSpot reviewer
Software Testing Services Manager at Virgosol
Real User
Top 20
Easy to use, readily available documents online, but time planning could be better
Pros and Cons
  • "The most valuable features of Jenkins are the ease of use and the information about how to use the features is readily available on the internet. Additionally, with the solution, I can use other reporting tools, such as Flow."
  • "Jenkins could improve by adding the ability to edit test automation and make time planning better because it is difficult. It should be easier to do."

What is our primary use case?

Jenkins is used for triggering my test automation. I use Selenium WebDriver for test automation.

What is most valuable?

The most valuable features of Jenkins are the ease of use and the information about how to use the features is readily available on the internet. Additionally, with the solution, I can use other reporting tools, such as Flow.

What needs improvement?

Jenkins could improve by adding the ability to edit test automation and make time planning better because it is difficult. It should be easier to do.

For how long have I used the solution?

I have been using Jenkins for three years.

What do I think about the stability of the solution?

Jenkins has been stable in my usage. However, I have had colleagues say they have had stability problems. They experienced these problems after six months to a year of use.

What do I think about the scalability of the solution?

We have approximately four engineers using Jenkins and they use it as needed. It could be weekly or monthly.

How are customer service and support?

There is not any support for the solution since it is open-source. However, you can find all the information needed online.

Which solution did I use previously and why did I switch?

For triggering I use Jenkins, but for local systems, I used Selenium WebDriver. Additionally, I have used Cypress.

How was the initial setup?

The initial setup of Jenkins is in the medium-difficult range.

What's my experience with pricing, setup cost, and licensing?

Jenkins is a free solution, it is open source.

What other advice do I have?

I rate Jenkins a seven out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Managing Director at Technocure
Real User
Valuable plugins and automation, but the upgrades need improvement
Pros and Cons
  • "This solution has helped us in automating the build and test process, reducing time."
  • "The upgrades need improvement."

What is our primary use case?

We use this solution for continuous integration and deployment, using Groovy Script for pipeline node configuration.

How has it helped my organization?

This solution has helped us in automating the build and test process, reducing time.

What is most valuable?

The most valuable features are plugins that make my server highly available.

What needs improvement?

The upgrades need improvement.

For how long have I used the solution?

Three years.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Ujjwal Gupta - PeerSpot reviewer
Project Lead at Mphasis
Real User
Top 10Leaderboard
Your True Friend when you're out (CI/CD)

What is our primary use case?

I use Jenkins for Continous Integration or Continous Deployment to run test case execution in Nightly build atmosphere. Integrating test scripts to Jenkins is easier and it can run based on the frequency mentioned in settings.

How has it helped my organization?

Jenkins has helped us in executing our test scripts without the Dev team during night time.  It automatically fetches the latest build or codes and execute all your test scripts and share the report with the respective team and stakeholders.

What is most valuable?

I have found the following features extremely helpful.

  • It is open source & user-friendly.
  • It can deploy code instantly & generate test reports. The requirements for continuous integration and continuous deployment can be configured manually.
  • Integration work is automated.
  • It can be integrated with other major tools.

What needs improvement?

They should improve the Version Control tracking system in Jenkins.

For how long have I used the solution?

Three to five years.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
PeerSpot user
Software Engineer at a tech services company with 11-50 employees
Real User
Significantly reduces build times, automates frequent manual tasks, reduces errors
Pros and Cons
  • "Having builds and test tasks triggered on commit helps not to break the product."
  • "We significantly reduced build times of large projects (more than 80k lines of Scala code) using build time on Jenkins as a time sample. It reduced the developer write-test-commit cycle time, and increased productivity."
  • "We have started to integrate Pipelines as a part of a build, and built a library of common functions. It simplified and made our build scripts more readable."
  • "Automation of chores like deployment, frequent manual tasks (like running scripts on test and production systems) reduced the time used and the number of errors made by engineers, freeing them to do meaningful work instead."
  • "Jenkins relies on the old version of interface for configuration management. This needs improvement."
  • "Developer documentation for plugins, plugin development, integrations: Sometimes it’s tricky to do pretty obvious things."
  • "Sometimes you have Jenkins restarting because of OOM errors."

How has it helped my organization?

Most obvious: Having builds and test tasks triggered on commit helps not to break the product.

From my own experience:

We significantly reduced build times of large projects (more than 80k lines of Scala code) using build time on Jenkins as a time sample. It reduced the developer write-test-commit cycle time, and increased productivity.

Integration with GitLab reduced time used for code reviews. Jenkins posted build state and code quality reports into the merge request.

Simplified build scripts: Organisation started to integrate Pipelines as a part of a build, and built a library of common functions. It simplified and made our build scripts more readable. 

Automation of chores like deployment, frequent manual tasks (like running scripts on test and production systems) reduced the time used and the number of errors made by engineers, freeing them to do meaningful work instead.

What is most valuable?

  • Configuration management: It is so easy to configure a Jenkins instance. Migrate configuration to a new environment just by copying XML files and setting up new nodes.
  • Programmable pipelines: In recent versions, Jenkins has a Groovy Sandbox where build scripts execute. I have never seen that powerful a tool in CI solutions yet. On other platforms you can use shell scripts, but Jenkins' solution is much better in terms of readability and portability. And given that you can create your own libraries for the Jenkins Pipelines, it becomes much more powerful and DRYer, simplifying work of DevOps and build engineers.
  • Brand new Blue Ocean UI: Jenkins used to have a pretty outdated UI. Now, you can use the Blue Ocean plugin to make it nice, clean, and modern-looking. Also, it has very good integration with Pipelines (basically it is built to use Pipelines).

What needs improvement?

UI: Jenkins relies on the old version of interface for configuration management.

Developer documentation for plugins, plugin development, integrations: Sometimes it’s tricky to do pretty obvious things.

For how long have I used the solution?

Three to five years.

What do I think about the stability of the solution?

Rarely. I can remember only one time we lost our build info after upgrading Jenkins, somewhere around the 1.6xx versions.

Sometimes you have Jenkins restarting because of OOM errors.

What do I think about the scalability of the solution?

No scalability issues. I used to have up to five worker nodes with one master, and it did not produce any slowdowns. I have never had bigger deployments.

How are customer service and technical support?

I have never used technical support directly. The community, documentation, issue tracker, are pretty good, though not ideal.

Which solution did I use previously and why did I switch?

TeamCity - It’s pretty limited in build runners, mostly targeting enterprise tech (Java, MS Stack, mobile apps) and the price is quite high.

Buildkite - An okay solution, but builds are shell scripts in general. It’s hard to maintain them. Also, I had weird issues with SCM integrations and Github.

GitLab CI - It’s coupled with GitLab too tightly. It’s pretty difficult to configure. It’s slow and requires a lot of resources to run.

How was the initial setup?

As for me, I just start to use it. It runs builds, unless you need something more complicated.

Setup of commonly used plugins is very straightforward, but it can be more difficult to get it running with exotic technologies. Still, it’s much easier than with other solutions.

What's my experience with pricing, setup cost, and licensing?

I used the free OSS version all the time. It was enough for all my needs.

Which other solutions did I evaluate?

I was always choosing between Jenkins, TeamCity, Buildkite, and Bamboo. Most recently, hosted solutions like Codeship and Travis CI added to the list.

For business needs, Jenkins is the most relevant choice because it can be self-hosted, the price is good, it’s robust, and requires almost no effort for maintenance.

For open source projects, Travis CI is standard.

What other advice do I have?

I like it very much, and I'm actively promoting it on my network.

Take your time to get used to the management flows of the application and builds. Jenkins is very powerful when you know how to cook it.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user