What is our primary use case?
Whenever I need to write test cases for my development code, I just give the command to Copilot, and it automatically helps me write them. Then I just need to review them. It also really helps me write the skeleton of the basic design. It won't give me a 100% solution, but it gives me a 75% solution. Then I just need to review it, make some cosmetic changes, and it's ready.
Overall, where I used to spend a whole day on test-related activities, now I just need to spend two to three hours, and I can complete my job.
How has it helped my organization?
It's helped reduce engineering effort. If I can complete my job in half a day, then all the engineers can complete their jobs in half a day. Ultimately, it saves cost and optimizes performance.
From the company perspective, if we needed to spend a million dollars, we can now reduce it to at least a hundred thousand dollars.
What is most valuable?
Copilot's code review, test case generation, and chat features are most valuable to me. When I write code, I feel like I have someone sitting with me to help me. Wherever I'm stuck, I just ask a question, and it provides guidance that I can use.
What needs improvement?
In a few cases, the results aren't correct, so that needs improvement. Also, it would be great if the results could be presented in different formats, not just text. As engineers, it takes time to read through text-based results.
If they could provide diagrams or basic design architectures, it would be easier to understand and take less time to review. So, as of now, image functionality is not provided by GitHub Copilot, and it would be great if they could add that.
So, I would like to see improvements in the result generation, more results, different result formats, and image functionality.
Buyer's Guide
GitHub CoPilot
October 2024
Learn what your peers think about GitHub CoPilot. Get advice and tips from experienced pros sharing their opinions. Updated: October 2024.
814,649 professionals have used our research since 2012.
For how long have I used the solution?
I have been using this solution for four months. I work with the latest version.
What do I think about the stability of the solution?
I would rate the stability a nine out of ten. It is a stable product. Copilot is a good thing when you're writing code. It assists you wherever you need it.
On the other hand, there are a lot of benefits to this feature, like suggesting code before you write it and suggesting the optimal way to write it, which is really helpful.
Apart from that, it helps with writing test cases and suggestions for heavy algorithms where you need to put in a lot of brainpower. So, it makes things easier.
On the other hand, the negative points, Copilot still hasn't reached a point where we can trust it 100%. Sometimes, when we raise a query, it stops working intermittently. So, at that moment, we need to look for other options. It's not, as of now, 100% reliable. Sometimes, it stops working; sometimes, the output has issues. So, we need to be careful with information.
What do I think about the scalability of the solution?
I would rate the scalability an eight out of ten. Sometimes, we face high traffic on Copilot, and it stops responding to our queries. It's happened a few times, maybe three or four times out of a hundred.
It's not 100% reliable. Sometimes, we stop receiving responses. We have around 500 end users in our company. It is used on a daily basis.
How are customer service and support?
The customer service and support are okay.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
We switched to GitHub because it's more appropriate and saves me time. On Google, I need to spend time searching for something, typing it in, and then getting a bunch of solutions out of which I need to pick the best one.
Copilot, on the other hand, gives me the best answer out of all the possibilities with just one query. That's a big benefit.
How was the initial setup?
I would rate my experience with the initial setup of this product a seven out of ten, where one is difficult, and ten is easy to set up.
It is not complex. I gave it a seven because there were a few minor technical challenges with the initial login. It wasn't as smooth as logging in to Gmail or Microsoft Outlook, but I was able to log in after a few tries.
So, it is a bit tricky. These are just minor challenges, not completely smooth.
We use the licensed version, not a cloud version. We purchased a license and installed it.
What about the implementation team?
We did everything ourselves. I was able to do it alone; it didn't require much effort. So, one person can manage both the deployment and maintenance.
It took only one day to deploy it. It's not really a deployment in the traditional sense. We requested a license key from Microsoft, and then we used that key to log in from our laptops. We configure it with our development environment, and it works smoothly.
What was our ROI?
If you invest some amount, it will definitely reduce your production cost. I would rate the ROI an eight out of ten, where one is low ROI and ten is high ROI.
What's my experience with pricing, setup cost, and licensing?
I would rate the pricing a five out of ten, where one is very cheap and ten is very expensive. So it's affordable.
There are no additional costs to the standard license. So the price is fixed.
Which other solutions did I evaluate?
This is the first time I've used a tool like this that assists with code writing. Before that, I used a simple Google search.
Moreover, Google has recently launched its feature, but it's not as mature as Copilot's. They're in the very early phase.
What other advice do I have?
Developers, testers, front-end, back-end, any developer can use it. It's really beneficial for them. On the other hand, testers can also explore test cases with Copilot. It'll help them, too.
Overall, I would rate the solution an eight out of ten.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.