What advice do you have for others considering StreamSets?

We use various tools and alerting systems to notify us of pipeline errors or failures. StreamSets supports data governance and compliance by allowing us to encrypt incoming data based on specified rules. We can easily encrypt columns by providing the column name and hash key. If you're considering using StreamSets for the first time, I would advise first understanding why you want to use it and how it will benefit you. If you're dealing with change tracking or handling large amounts of data, it could be cost-effective compared to services like Amazon. It's easy to schedule and manage tasks with the tool, and you can enhance your skills as an ETL developer. You can easily migrate traditional pipelines built on platforms like Informatica or Talend to StreamSets. I rate the overall solution an eight out of ten.

It's a very good tool. Overall, I would rate the solution an eight out of ten.

To those evaluating StreamSets, I'd advise doing a cost-benefit analysis because the way of using StreamSets differs from person to person. Someone else might have a very different use case, and they may not run into profit using the solution. For us, it was a good solution because we were hiring people for this work. People were doing the job manually. We saved both time and money, so doing a cost-benefit analysis would be the best thing. If you are looking to expand your domain or range of operations, StreamSets is very helpful. If you are just looking for a better data analytics tool that can do bifurcation on data, I believe there are other tools or services available in the market that do not focus on the expansion of operations. They focus on doing better and more complex bifurcations. StreamSets enables you to build data pipelines without knowing how to code. After generating a few responses, you have to enter some basic syntax or code, but generally, one can do a lot of no-code stuff, which was not an important aspect for us because we were operating in the IT space, and our entire team was capable of entering all the syntaxes that were required. It was not an issue for us at any point in time. In fact, in the operations that we were performing, we only used code. When we were testing out our initial datasets, we used some no-code features that were there, but at the later stage, we used only syntaxes. We did not connect to the messaging systems, but we connected some enterprise databases. We were operating with a set of hospitals in the US, and we had to connect with them only the first time. Afterward, it was the data that was passing through the pipeline. Initially, for a completely new user, it's a bit tricky. Some technical expertise is required. It's a bit tough, but because the support team is there, one would be able to do it. Overall, I would rate StreamSets an eight out of ten.

It's a very good tool if you need to access data from a CRM system, Salesforce, etc. However, it can't be used as an end-to-end integration tool because it lacks certain functionality. It could also be very expensive for small enterprises. Overall, I'd rate it a seven out of ten.

The transformation logic is a bit complex when you begin and you may need to read the documentation. When you create logic, you have to be sure of the scenarios in the logic. Any company that is looking for data engineering should use StreamSets because the pricing is quite favorable. I would recommend it.

I give the solution an eight out of ten. We use StreamSets in multiple departments, not just in different locations such as the IT department and the software team. We have ten people using StreamSets in our organization. StreamSets does not require any maintenance. I highly recommend StreamSets as it is a highly customizable streaming application. However, before selecting this application, the user must analyze their data transfer requirements and mode of transfer. For example, StreamSets is expensive. The data combined nodes are good, but they still need to be configured correctly. Data processing and file conversion can slow down the process, so it is important to have enough memory to support the requirements.

I give the solution an eight out of ten. StreamSets still needs to improve the monitoring and visualization before the solution can be a ten out of ten. Since StreamSets is deployed in the cloud, we don't have any maintenance requirements or costs. I highly recommend StreamSets; it is an excellent tool with both batch and streaming capabilities. StreamSets is a great option for anyone to try, though it does require an organization to have the budget to use it.

We frequently use StreamSets' Transformer for Snowflake functionality for large amounts of data. Using it is not that simple but not that difficult. It is moderate. You need some training or an introduction to design pipelines within the transformers. Still, it's a useful feature because we have a large amount of data that we need to transform. Being serverless, Transformer for Snowflake doesn't depend so much on infrastructure or cloud users. That aspect is beneficial for any organization because you don't need to deploy additional resources for it. I highly recommend this solution, if you have the budget for deployment and your requirements are fulfilled by it.

It's cloud-based software, so there are only minimal maintenance requirements. Our IT team takes care of the maintenance of the software, but I don't think much time is required for that. Only regular updates need to be done. It is a minimal task that can be done by one or two personnel. Overall, it provides us a lot with efficiency and increases the effectiveness of our transformation of data sets. The value and increase in revenue it has helped us achieve make it a very good software package. Try the free version and, if the software meets your requirements, I would definitely say get the Enterprise version. It's pretty easy to understand and it generates a great deal of smoothness for your business processes. It's a must-have for every business to improve its efficiency and effectiveness. The major takeaway for me has to be the improvement in the efficiency of our entire process. That stands out for us. StreamSets is a great platform. And the best thing about it is that there are minimal coding requirements. Any person, even someone with a non-technical background, can easily get accustomed to the software and start using it.

Server update maintenance is required, but that is minimal. Any product would require that type of maintenance. I don't think we are investing a lot of time and money in maintenance. The maintenance is just another cost for us. We have only two guys working on the maintenance part of the software. It's a very intuitive product, modern, and very user-friendly in terms of the UI. Almost all our requirements have been met by StreamSets and we don't have any complaints so far. I would recommend starting to use it as soon as possible. No tool is perfect. You have to choose the best of the lot. I certainly believe StreamSets is at the top of the ladder when it comes to similar software. My biggest lesson from using StreamSets is that data integration can be done much more easily now. I only knew that after starting to use StreamSets. When it comes to data integration from multiple sources, and having multiple destinations, people always assume it's a time-consuming, cumbersome project. But once we started using StreamSets, all those assumptions were broken. It's very straightforward and elegant software.

We have been experimenting with Hadoop, but apart from that, we do not use it to establish a connection with other services. As an organization, we have not faced any issues with connectivity using StreamSets. The platform is very stable. Overall, StreamSets is very efficient and effective. It has helped us save a lot of time and also reduced errors a lot. I would definitely rate it very highly. The major reason is that it gives us a single, centralized platform for all our design-pattern requirements and we are able to produce results efficiently. With StreamSets, we are able to transfer or stream data from any source to any destination. It has increased the overall efficiency of our organization. Software AG is constantly improving and evolving the product, and that is something that I like: using a product that is ever-evolving and being upgraded. After deploying StreamSets, I learned a lot about how data planning works and how easy it is to stream from multiple sources to multiple destinations. That is one of my major takeaways. I thought it would be a very complex task, but that myth was broken by StreamSets. The complexity was made very simple for me. My advice is to try the free edition. It's a very user-friendly and intuitive product as well. Try it to get a grasp of what's happening inside the product. Once you try the free edition, you'll definitely go for the Professional edition. I don't have any doubt about that. The product itself will lure you. That is the power of the product.

The ease of using StreamSet to move data into modern analytics platforms, on a scale of one to 10, is about a five. The solution enables you to build data pipelines without knowing how to code if it's the latest, state-of-the-art cloud connecting stuff. If it's for anything structured for Oracle and SQL Server and other data sources, it's difficult. Without knowing how to write code, some of it's easy and some of it is not. My advice to someone who is considering this software is to be very aware that their integrator and data analysis people will need a very specific skill set.

Go through your data integration requirements and compare the other solutions with your requirements. But I hope StreamSets works perfectly with your requirements, because it has all the features that you require. Sometimes we use StreamSets’ Transformer for Snowflake functionality when the data is huge and cannot be integrated through other processes. Transformer for Snowflake is quite good, useful, and easy to set up. But it requires some initial setup training. It is used when you have a large volume of data through your API calls or through IoT devices.

It is very user-friendly, and I promote it big time in my organization among my peers, my juniors, and across different departments. They're growing rapidly. I can see them having a lot of growth based on the features they are bringing. They could capture a lot more market in coming times. They're providing a lot of new features. I love the way they are constantly upgrading and improving the product. They're working on the product, and they're upgrading it to close the gaps. They have developed a data portal recently, and they have made it free. Anyone who doesn't know StreamSets can just create an account and start using that portal. It is a great initiative. I learned directly on the corporate portal license, but if I were to train somebody in my team who doesn't yet have a license, I would just recommend them to go to the free portal, register, and learn how to use StreamSets. It is available for anyone who wants to learn how to work on the tool. We use StreamSets' ability to move data into modern analytics platforms. We use it for Tableau, and we use it for ThoughtSpot. It is quite easy to move data into these analytics platforms. It is not very complicated. The problems that we had were mostly outside of StreamSets. For example, most of our databases were on-prem, and StreamSets was installed on the cloud, such as AWS Cloud. There were some issues with that. It wasn't a drawback because of StreamSets. It was pretty straightforward to plug and play. I have used StreamSets Transformer, but I haven't yet used it with Snowflake. We are planning to use it. We have a couple of use cases we are trying to migrate to Snowflake. I've seen a couple of demos, and I found it to be very easy to use. I didn't see any complications there. It is a great product with the integration of StreamSets Transformer and Snowflake. When we move data from legacy databases to Snowflake, I anticipate there could be a lot of data drift. There could be some column mismatches or table mismatches, but what I saw in the demo was really fantastic because it was creating tables during runtime. It was creating or taking care of the missing columns at runtime. It is a great feature to have, and it will definitely be helpful because we will be migrating our databases to Snowflake on the cloud. It will definitely help us meet our customer goals at a faster pace. I would rate it a nine out of ten. They're improving it a lot, and they need to improve a lot, but it is a great product to use.

Every tool in the market at the moment has some major gaps, especially for large enterprises. It could be the way that the data or pipeline is secured. At present, StreamSets looks like the market leader and is trying to fill that gap. For anyone going through a proof of concept for various tools, StreamSets is almost at the top. I don't think that they need to look any further. We are working only with API, a relational database management system, and our enterprise warehouses at the moment. We are not using any streaming sort of ingestion at the moment. We are not using Snowflake Transformer yet. It just got released. We are using a traditional Snowflake destination stage because our enterprise is huge. We have our own Snowflake architecture. We load the security in the data into our own databases using the destination stage, not Transformer yet. I would rate the solution as 7.5 out of 10.

For people who are starting out, the simple advice is to first try out the cloud login of StreamSets. It is freely available for everyone these days. StreamSets has released its online practice platform to design and create pipelines. Someone simply needs to go to cloud.login.streamsets.com, which is StreamSets official website. It is there that people who are starting out can log into StreamSets cloud and spin up their StreamSets Data Collector machines. Then, they can choose their execution mode. It is all in a Docker-containerized fashion. You don't need to do anything. You simply need to have your laptop ready and step-by-step instructions are given. You just simply spin up your Data Collector, the execution mode, and then you are ready with the canvas. You can design your pipeline, practice, and test there. So, if you want to evaluate StreamSets in basic mode, you can take a look online. This is the easiest way to evaluate StreamSets. It is a drag-and-drop, UI-based approach with a canvas, where you design the pipeline. It is pretty easy to follow. So, once your team feels confident, then they can purchase the StreamSets add-ons, which will provide them end-to-end solutions and vendor support. The best way is to log into their cloud practice platform and create some pipelines. In my current project, there is a requirement to integrate with Snowflake, but I don't have Snowflake experience. I have not integrated Snowflake with StreamSets yet. I personally love working on StreamSets. It is part of my day-to-day activities. I do a lot of work on StreamSets, so I would rate them pretty well as nine out of 10.

If you are looking for something to do batch processing in Java, this is the right solution. We did the exploration when we were trying to implement a batch processing system and decided that StreamSets is the best for that. If you're looking for real-time, you may want to look at another system or the next version of this one. Because of the kind of system that we need to implement with this kind of solution, the most important factors I look at when selecting a vendor are things like latency and real-time processing. I would rate it at nine out of 10. What would make it a 10 would be, as I said, I'd like to have more integration with other kinds of languages or frameworks and also more real-time processing, not batch.