What needs improvement with Microsoft Azure Synapse Analytics?

We will start looking at Microsoft Fabric for further improvement as it brings more value. Also, managing workloads and data transformation using Databricks should be a focus.

I wish the data governance feature could be incorporated without requiring an additional license.

One potential area for improvement could be the availability of an on-premises data lake implementation, as the product is currently only implemented in a cloud environment. Additionally, the possibility of integrating data from multiple sources could be beneficial.

In our company, we always want to improve our data management, including how to manage structured and unstructured data. Every day, we need to do a lot of stuff because we are a bank. We have reports from user accounts and client accounts. Eventually, I want to use machine learning bots, maybe since they can help our company.

Adding more transformations and plugins to the solution is very important.

The security performance and cost are the two things that needs improvement.

One area that could be improved is the schema management. For instance, with Azure Data Lake, I sometimes try to create mappings. In MySQL or MongoDB, I can easily see the datasets and create connections without knowing the exact schema beforehand. I'm not as familiar with that process in Azure Synapse Analytics. It might be possible through tutorials, but it would be helpful to have more integrated tools for data scanning and schema exploration within the studio itself. This could help streamline the workflow and reduce the need to switch between different applications. So, I'd like to have additional tools for data scanning and schema exploration within the Azure Synapse Analytics studio.

The platform is not flexible, and the graphical user interface needs to be improved because the interface makes it hard for the end user to use it. Also, artificial intelligence could be used to improve analytics.

Real-time integration is hard to do in Microsoft Azure Synapse Analytics.

There are some limitations based on the fact that it is a new platform. Some of our users have recommended switching to a different platform they are more familiar with. However, from an infrastructure standpoint, I believe it can be more secure in terms of availability. Currently, even though we have locked down public access, the Synapse can still be viewed with the URL without VPN access. It only allows viewing the configuration and not any sensitive data or pipelines. We are working with Microsoft to address this issue, but it seems like they have recommended that it be open, and there is currently no solution. Perhaps there will be improvements in other aspects in the future.

It's stable, but its stability could be better. However, we understand that it's in production, and new features are getting added and upgraded, so you do get hiccups sometimes.

One area for improvement could be better integration with Power BI, as well as data integration with BW. If we could find a connector or process to export data from BW into Azure, that would be helpful.

Generally, people have differing opinions on whether to use Azure Synapse Analytics or Azure Databricks. These are two different services from Microsoft, and customers often struggle to decide which is best suited for their particular use case. Both services have some features in common and some that are exclusive to one or the other. It is a trade-off and ultimately depends on the use case that best serves the organization. Incorporating the features of both Azure Synapse Analytics and Azure Databricks would meet most use-case scenarios. The cost of the solution has room for improvement. Azure Database has a very good Unity Catalog functionality, which is a form of governance within Azure Databricks. Similarly, for Azure Analytics, having a governance product would be great. However, Microsoft already has Azure Purview, which is meant for governance, but it would be nice to have that functionality built into Azure Synapse Analytics.

The linked services can be improved. We can create dynamic linked services to access a lot of databases but only those of the same type. For example, I can use the same linked services to access 11 SQL databases. However, if I have 11 SQL databases and five Oracle databases, I need two dynamic linked services. I cannot do it with only one linked service. The UI also needs to be improved. When I have used Azure Synapse for programming with PySpark, Scala, or .NET, for example, the UI has been unstable. If I open two notebooks for programming, one notebook will queue the session of the other.

Microsoft Azure Synapse Analytics can improve by adding more flexibility to the reports. Having more visible structures based on the area, region and country would be beneficial.

Microsoft Azure Synapse Analytics could improve the section in the solution where you can implement the Python Spark pipelines, it's not the same as in Databricks which would be better. The data visualization in Microsoft Azure is provided by Power BI, it's not needed to have something in Synapse. The data governance tool is outside Synapse, but there is a data governance tool that is called Purview in Microsoft Azure. They need to improve the Spark part of the solution then it would be complete.

In Azure, when you do the scaling up, it is not totally simple. It takes time to scale up. It actually kind of rebuilds the database behind this when you scale. If I am utilizing 1,000 of what they call the Data Warehouse and you need 1,200, there is downtime required. There is a limit on the number of concurrent queries to around 125 for Azure Synapse.

Microsoft Azure Synapse Analytics could improve in usability. I have found the same issue with all Microsoft solutions.

The product needs a tool that allows for work from a laptop instead of a browser. When working in on-premises environments, it is important to have all tools installed on a laptop rather than relying on internet connectivity which is a big inconvenience. For example, it would be brilliant to add an integration on Visual Studio to create all pipelines in an offline mode. The solution cannot store much data and might require purchasing additional storage. For example if you have 1 TB of data, processing it in the solution will cost ten times more than processing it in Databricks.

Synapse Analytics' performance slows down if you don't get your distribution right because it gets queued and goes into a single node. In addition, the integration with other solutions like Power BI and Azure SQL Database is very slow and leaves a lot to be desired. In the next release, I would like to see a feature to offload Synapse files directly into Gen2.

There are some limitations. The filing can be improved. Some improvement in the filing is necessary. It needs some kind of growth in the filing development in order to reuse some activities.

We'd, of course, always like to pay less for the service if we can.

Synapse Analytics needs to develop an automation framework because now you have to build a cache yourself. You have to build a pipeline in WhereScape, which does end-to-end pipeline automation well. Microsoft should come up with a framework to save people time. If they developed a tool like WhereScape, it would dramatically reduce development time. I'm waiting for Microsoft to implement a feature that automatically detects and processes the copy activity. That feature is available in Databricks, but I hope they will add it to Synapse Analytics' data factory integration run-time.

The only concern for us is the cost part. When it comes to the implementation and the support and maintenance, we see high-cost implications. Of course, it varies from use case to use case. Before we get into the implementation part, we have to validate the pros and cons of the architectural components as part of the design and development. Once that's clear, then we'll go for implementation. We might get into technical glitches, however, there are multiple ways to work around them by putting in the right architectural component, which can solve the problem. There is always a workaround. We've had a couple of interactive sessions with Microsoft already. We have already recommended that they need to strengthen their presence in the data governance part, the data quality part, and then the metadata management, for example, data lineage. We need more data governance to give the flexibility to handle these data quality issues. It would be great if they update their data features.

Microsoft Azure Synapse Analytics could improve its compatibility with Visual Studio. One of the challenges for people moving from an on-premise to a cloud solution, such as Microsoft Azure Synapse Analytics, is that you're constantly working in a browser. There are people that have been working for decades on desktop applications. For them to start working in a browser, it's quite a change. Allowing people to work and do their work inside Visual Studio than in the browser, would be a large advantage.

Microsoft Azure Synapse Analytics can improve by increasing the size of the files that we can load on the platform. We have some files that are too large to be loaded and it would be a benefit to us if the limit was increased. Additionally, the way we use the tool for generating reports can be made better. They should add some drag-and-drop rules without the need of programming these rules using some programming language. It would be helpful if we did not need someone that was technically advanced to be able to do it with, such as someone with no IT background. Having a reporting tool without code would be great.

The only issue that we have run into with the solution's performance is with regard to concurrency. We would also like to see faster response times from support.

I would like to see more ready-to-use products from Synapse. Right now, everything seems a bit futuristic without much modern use.

Its stability is an issue. They have been releasing a version every six months to one year, which means that there are many versions available, and clients are not up to speed on the latest one that they're offering. From a stability point of view, they could do better. They're still upgrading their Synapse Analytics workspace, and it is not that stable. Its scalability can also be better. I would like to see support for cross-DB queries.

An area for improvement would be advanced analytics. The product also comes with a pretty steep learning curve, which could be improved. In the next release, I would like an improvement in internal security, which currently doesn't work at all.

Microsoft Azure Synapse Analytics can improve querying. I have recently used Microsoft Azure Synapse Analytics to connect to the Delta Lake file and I have noticed some issues. It has not been able to read the latest version of the Delta Lake file and I have found this to be a disadvantage that they can improve. In the future, Microsoft Azure Synapse Analytics could improve the performance, there are other solutions that are better, such as Databricks.

Right now, we are really struggling with the performance. it's not as good as we had hoped. I cannot recall coming across any missing features. One problem is that getting the query cost is not easy. I need to use multiple tools. It can be accessed through the web, or we have Azure Data Studio, or SSMS, and I only get the query cost in SSMS, the SQL Server Management Studio. In other tools, I'm not getting it. For example, in the web version or in Azure Data Studio, it is not available. They need to make it accessible in both of those places as well.

When I was trying to link services to an SFTP site it was not able to do all the possible encryption that I needed. They can improve by adding more encryption options.

Indicating what areas need improvement in this solution is a difficult question because the organizations that I am working for are really new in this area. However, an even better more simple interface, or perhaps an extension of a connector app store solution, would be helpful. For example, if your organization, has an electronic client dossier and you want to connect that information system in real-time towards Power BI, you just go to the app store and see if there is an ONS connector, introduce it, set it up, and go from there. In an upcoming release, it would be beneficial to have the ability to have dashboards on mobile devices to receive a better experience for the user.

For improvement, I would like to see more connectors and better integration for Hadoop. There are currently limitations for Hadoop on-prem. I'd like to see better integration and more flexibility.

They should provide a less expensive version with a smaller setup for small businesses. Currently, its price is quite high for entry-level or small businesses. In terms of integration, new connectors are always welcomed.

The solution needs to improve its integration capabilities. It needs better integration with SQL and other databases. The initial setup is complex. The product needs to implement more machine learning and artificial intelligence. If they had more IoT and streaming capabilities, it would be helpful. The product could use more analytics tools. It would be helpful if we could use the product with less coding.

What I would like to see is a possibility for time travel on the data. For instance, a financial department would like to do an analysis of the data set as it was for the first of the month while another department wants to run the analysis about how things went yesterday or last week. They have different starting points for the queries. I've seen Snowflake handle tasks such as these. It's got a feature called time shuffling, which can show me what's there on a certain variety of dates. It's very convenient. It would be great if we could query the data of the day as well as in the data warehouse. They should have scalable memory processor options for the data warehouse. While the solution is flexible, sometimes this works against the user.

We've only recently starting testing this solution and it would be helpful if the documentation was better or if the company could provide some kind of presentation to help us understand how the tool works and what's required to optimize use of the solution.

The macro functions, though useful, are not totally user-friendly. Some people have difficulties in learning them. If the project keeps growing we will need to link some of the documents to the data warehouse. However, at that time, I'm not sure about the possibility of keeping the same systems that we are using right now. We may need some kind of migration. I cannot say for sure, as I am not an expert. We'll need advice on that. The initial setup is not straightforward. The solutions not user-friendly and does require training of some sort. The solution requires a lot of the memory of the different devices.

More integration is needed to improve the product for the future. There needs to be more flexibility when developing applications with this product.

There aren't any features that are really lacking in the solution. We don't really have any issues with it in its current form. When we used the on-premises deployment model, we had data latency issues and suffered from page performance problems. However, since we've moved to the cloud, we haven't had these problems. The initial setup has a bit of a learning curve.

Unfortunately, we have had some issues with the dashboard reporting. Sometimes, the data for specific periods would just appear blank on the dashboard. To investigate this, we worked with a Microsoft incident agent and it turned out to be a result of bugs in the platform when dealing with specific types of queries in Azure Data Factory. We have also encountered some bugs regarding incompatibility with certain versions of Power BI. In the past, Power BI used to integrate nicely with Azure Analysis Services which we used as a bridge between Power BI and Synapse Analytics, but they have since discontinued the features that worked so well for us. Apart from fixing bugs, I would also suggest: * The learning features and documentation could be improved and expanded on. * The calculation and forecasting tools could be made more clear and easy to use. * The price could be lower.

I think that there should be better governance. Actual pipeline level security, if you create a pipeline within the tool, because it's a one-stop-shop for everything, you control who has access to what, and that goes for anything, whether it's an internal or an externally managed table. I think there needs to be an improvement with support for external tables supporting Spark, but further support of Lakehouse to make it a true data warehousing product where you have this true separation of computes and you have a separation of storage. I think it kind of dips its toes in there, but in order to achieve, I think with a vision that Microsoft is doing, trying to get at to replace Bolt, but compete with Snowflake and Databricks at the same time, it needs to have those features on. In the next release, I would like to see Lakehouse and better governance.

The performance and data consistency need to be improved. Support for PSQL functionality should be included, as it is something that is missing right now.

I am pretty sure that there are areas that need improvement but I just can think of them off the top of my head.

I am very sure that there are areas in need of improvement, but I can't recall what they are off the top of my head.

I would like there to be more focus on the Software as a Service (SaaS) product. For example, Snowflake, one of their competitors, is more focused on SaaS. They need to improve a little bit in terms of user-friendliness. I would rate them a three out of five, in terms of how user-friendly the interface is. The auto-scaling capabilities are quite poor, especially compared to Snowflake or Firebolt.

After only using the solution for three months, we haven't yet really gotten into every corner of its capabilities. It's hard to say what might be missing in terms of features. It would be ideal if the solution could be better used intuitively by the staff without having a great deal of training. An organization should be able to adopt and operationalize it throughout their company with minimal training.

I am a researcher. For people to be able to research a solution, there should be at least a free trial. Just advertising a product or saying that this product is better doesn't work. I would strongly recommend providing a lot of free trials and trainings. This will also help Microsoft in having more users or customers. Oracle provides some free trials. You can just go for a free trial and use your database online, which is very good.

Technical support needs to improve. Microsoft improves all of the time, but there are some things where different vendors offer data warehousing and data governance that you can build and workaround. It would be beneficial to take the top vendors and identify some kind of straightforward action to work with them. Instead of having to employ a separate vendor tool to be able to move this, it would be nice to be able to go through Microsoft.

What I would like to see more and more of in Azure is its support for IoT and streaming media information. It needs strong support for social media, internet data, and native support for NoSQL. At this time, it works very well with the structured data Stability could be improved. Technical support is very good, but could also be improved.

Microsoft should develop an interface to make it easier to shift from on-premise to the cloud. There needs to be a lift and shift tool to move on-premise, IMS high-volume data to synopsis sequence data on the cloud.

I would like to see them provide the ingestion of images.

There are features coming int he next few quarters that will be helpful. Soon, Power BI will be directly integrated into Azure. We need to have some Spark tools also available so we can directly select customers and don't need to install everything. There will be features added that relate to application development. There's hopefully going to be more flexibility with the XML. Right now, for example, Data Warehouse is not able to give XML files and your file put is not correct. The feature will hopefully allow us to read XML. The performance needs to improve in future releases. We're hoping that Microsoft will add integration with the Amazon AWS platform.

This solution needs to have query caching so that if the same query is run and the results are available, it will return the data from the cache without having to re-run the query. As it is now, you still have to use compute for each query. We would like to have integration with the data lake so that we can avoid retrieving the data from it, just to store it in Synapse before we can use it. From an architectural perspective, avoiding this replication would be good.

I would like my team to be able to build pipelines that integrate with the Azure Data Factory. Some of the best-in-class ETL products that are available in the market, such as Informatica ETL, are far more mature than this solution.

The major concern is that I have a hard time with having to version control the data warehouse all the time. As it is now, I have to open all of the ports and push everything onto the server. I would like to see version control implemented into the data warehouse. This would make the tool perfect.

The configuration for things like high-availability could be more user-friendly for non-technical users.

So far, with the current SQL that I'm using compared to what we've previously used, I'm happy with it. I wouldn't necessarily point out anything that needs improvement. From my point of view from what we have implemented recently, I'm happy with the solution, I'm happy with the performance. I'm not entirely happy with the billing model. I'm not entirely happy with how the enterprise services are pretty expensive, but that's about it.

With respect to what needs to be improved, concurrent connectivity has some limitations.

There are several specific features and functions available on the on-premises version that do not work with the cloud version. There are many issues regarding migration. They are building it more and more, but if you have the same version on local you cannot just go to the cloud because some of the functionalities don't work yet. You would have to re-code everything, and maybe even do it differently. In terms of documentation, they can make it easier or start with some advising or instruction. As it is now, it's like making a puzzle. Technical support needs some improvement. They will need to include some features for the Cloud version. They can make subsets of the database so you can schedule the marking department to work only between nine to five, but ETL can run at night. With a subset database, you can cut costs, similar to Snowflake whose price is a lot lower.

As far as improvements to the product, I think the Data Factory integration should be enhanced to be properly implemented because right now it is still lagging behind other vendors like Informatica. We ended up having to use Data Factory because we don't have any expertise in Informatica. But I want to see Data Factory get better overall. An enhancement I would like to see is in data loading for Data Factory. Right now we are seeing a little bit of a performance lapse in the cloud server. Specifically, this happens between Azure and our clients maybe because we are connecting to SQL Server Management Studio. There is a slight but noticeable delay. I do not know if it has to do with Azure and whether it is their problem or our problem. I can't complain much about Data Factory, but because I used to use SSIS (SQL Server Integration Services) which was the prior on-prem version, I know what capabilities it is missing in comparison. They changed it over to Data Factory, but by comparison, Data Factory has a long way to go to reach the level of SSIS as a product.

It would be of interest to improve things like the web service integration and availability in terms of being easy to create internal web services in the database.

The initial setup process needs improvement. When you're moving to the cloud it takes a bit of time. It would be great if they could implement something that would make it faster. It takes a bit of time but once you're done, you're done. The simplicity is good, but it should be that anyone can use it. You shouldn't have to be technical because, with the way it is now, only a technical person who has had training can set it up. If you have a dashboard that anyone can use, it would be easier.

There are several things that I would like to see improved with this solution, including: * The price of this solution could be improved. * I would like to see better integration with Active Directory, because we have had problems, and we still do. * I expected that Microsoft would have had something out of the box for Office 365, but they do not. * It is difficult to configure, especially if you have multiple domains. * I would like to see a simpler interface for the Azure portal because it is kind of complicated. * It would be nice to have better monitoring of the usage of the data inside Azure SQL. What we don't use could be achieved in cloud storage. This would allow us to cut down the price. * Better customization would improve this solution.