Azure Data Lake Storage Reviews

We use the solution for storing data but don’t use Synapse to store data directly in it. Instead, Azure Synapse Analytics is utilized to analyze and process data in Data Lake Storage. Data Lake Storage is a large, scalable solution that handles extensive volumes of structured and unstructured data rather than a direct disk storage system.

In Azure Data Lake Storage, the tool we're using, Spark, handles the management, storage, retrieval, and organization of data. Spark employs its algorithms to abstract the underlying complexities. We don’t work with a large amount of data. If we were to handle larger datasets, we would need to focus more on optimizing storage and retrieval processes, as the efficiency of these operations would become more critical.

The version was a bit outdated compared to the newer Microsoft Data Fabric offerings. For instance, the directory services are already available in Data Fabric, so I don't think adding them to Azure Data Lake Storage would be necessary. For example, Snowflake, a cloud data analytics platform, adds its capabilities and optimizations to Azure Data Lake Storage, such as improved performance or easier integration with SQL. Compared to other similar services, Azure Data Lake Storage remains very competitive.

I have been using Azure Data Lake Storage for over a year.

Azure is a stable platform. These interruptions are relatively rare and usually last only a few minutes. It is good for data-oriented applications that don’t require continuous online processing.

These brief outages do not significantly impact the quality of service. We haven’t experienced major stability issues with Azure Storage. 

It is scalable.

Any issues are handled by the team responsible for managing the platform.

We primarily use Azure Synapse, which integrates with Azure Data Lake Storage. Synapse leverages the storage provided by Data Lake Storage, so both are part of the Azure ecosystem but remain distinct services.

Another integration involves SQL Server, which serves data to various consumers as an SQL database. The main consumer is Power BI, which provides extensive reporting capabilities. Additionally, Azure Functions integrates with internal systems at the client’s end.

It is a cost-effective solution.

Using a cloud platform generally allows for flexible capacity management, meaning you can use and pay for resources only when needed. This is particularly useful for our customers, who can run Spark clusters in serverless mode. They only pay for the time they use the service, which is cost-effective since they don’t need constant access to high power and typically run jobs for shorter periods, like half an hour.

It is available continuously and supports data archiving. However, since the current volume of data is not large, the client doesn’t need to focus on archiving or optimization. As their data grows and becomes more historical, they may need to optimize storage and archiving practices.

The other team manages the integration tasks. The process is straightforward as long as the systems, functions, or other components interact with external systems. The ease of integration can depend on the intensity of the integration requirements.

Overall, I rate the solution an eight out of ten.

We leverage its hierarchical storage structure to store pre-generated static data in JSON format. This improves performance significantly. Instead of pulling data from on-premises sources, we pull the pre-processed JSON files from Azure Data Lake Storage.

Our primary use case is storing processed data for direct access by end users. It's a good solution for that.

We have both an education product and several financial products. With our education product, we store pre-generated JSON files in Azure Data Lake Storage, representing different student tests in a hierarchical structure. This allows us to serve pre-made tests to thousands of students without hitting performance bottlenecks.

Similarly, for our financial products, we store the final calculated output in Azure Data Lake Storage for use with Power BI Embedded. Users get their Power BI data directly from the data lake. We’ve offloaded a lot of load from our core Azure SQL Server by using Azure Data Lake Storage.

Overall, we primarily use Azure Data Lake Storage to serve data to end-users, not for complex calculations or analytics.

The security features are great, especially the ability to use SAS tokens.

I also like its speed. It's basically built on Azure Blob Storage, which was already fast. But Azure Data Lake Storage adds the hierarchical structure for even better performance.

That's the biggest benefit of Azure Data Lake Storage – the hierarchical namespace. That structure is what makes it truly suitable for data lake scenarios.

Pricing is always a factor. It could be more affordable.

We've been using Azure Data Lake Storage for about two years now.

The stability is excellent. I'd rate the stability a ten out of ten. 

It is a highly scalable solution. 

We haven't needed to contact support. Everything has worked smoothly.

We used to use Databricks, but our license expired.

The initial setup is straightforward. 

It took less than an hour. The most time-consuming part is deciding your folder structure and how you want to organize data. 

The actual creation of the hierarchical structure and data storage is simple; it shouldn't take more than an hour or two.

Azure Data Lake Storage is cloud-based. We handled the implementation in-house

The performance of Azure Data Lake Storage has had a significant positive impact on the solution's cost management. 

By using Azure Data Lake Storage, we've been able to reduce our reliance on Azure SQL Database, which was running on higher tiers. This has led to cost savings.

We've seen benefits in both cost and performance.

It's a pay-as-you-go model. Your charges are based on the amount of data you store.

There are no extra costs.

It's great for improving the speed and accessibility of your static data.

For my use case, it is a good solution. So, I would rate it a ten out of ten. 

We are restoring external tables and data in Databricks, accessing those tables to read and write data using Azure Data Lake Storage. We use it for data quality purposes and store data to form external tables.

The hierarchical structure allows us to create multiple hierarchies inside, such as storage containers, directories, and subdirectories. It provides multiple edges and access control. We can define who can access which directory and restrict read and write operations.

Version control would be a great improvement. Currently, there is no version control, and if something is deleted, it's permanently gone. The addition of a trash item would help in recovering data deleted by mistake.

We have been using the solution for about three to four years.

There is no downtime, and everything is superior. The SLA is 99.99%.

It's good, so I would rate it as eight or nine out of ten. It handles large amounts of data efficiently.

I would rate technical support ten out of ten.

Positive

The setup is too complex since we handle a large amount of data.

Our team handles everything. We don't have any consultants or third-party integrators.

We did evaluate AWS as it has S3 buckets.

I'd rate the solution eight out of ten.

The most valuable feature of Azure Data Lake Storage is the ability to partition data into various datasets using a directory hierarchy. This folder structure is key for any delivery. Currently, we're not doing much with the data in the tool, but when Databricks comes along, we'll convert it to Parquet format. It's a two-step process: raw data is moved to Parquet, which Databricks can manipulate easily.

One improvement I'd suggest is the out-of-the-box conversion of input data, like spreadsheet or table data, to various formats. We'll be using Parquet, which enables transactional integrity.

I have been using the product for a year. 

Stability is good if you build your Azure Data Lake Storage well in the first place.

Scalability depends on process complexity—it is high for simple processes and low for complex ones. This is due to the architecture of a data lake, but once converted to a data lakehouse, scalability is high across the board. I think Azure Data Lake Storage would suit medium—to large enterprises. 

Microsoft's documentation is superb, and support is good, especially if you have a relevant intermediate supplier.

We haven't compared Azure Data Lake Storage with products from other vendors because we're an Azure shop. We did check that the Azure product was good enough for our needs, and it was, so we didn't explore alternatives like AWS, Google, or Snowflake.

The initial setup is fairly complex, but if you get your data architecture right from the start, it's not a problem. We're using a totally cloud-based deployment with Azure.

Integration capabilities are fairly smooth and comparable to AWS in terms of cloud integration. Some might say it's slightly better, others slightly worse, but I think it's good. I'd rate Azure Data Lake Storage an eight out of ten. However, it's important to note that it's only eventually consistent, so don't expect immediate consistency when changes are made. It works well as a data storage bucket for future use, but it's unsuitable for transactional work. You need to use a data lakehouse like Databricks for transactional processes, which can handle transactional work once the data is in the correct format (like Parquet). The tool is great for storing data you want to put into a data lakehouse, but not for frequent transactions. It's suitable for daily archiving, but anything more frequent than that might cause issues.

It is for data warehousing. We currently work on using Databricks and Hadoop for data warehousing – figuring out initial data migrations is our focus right now.

It's a cloud-based tool within the Microsoft ecosystem, offering many benefits for data handling.

Our focus is on standardization. We're currently analyzing how it could work with Databricks and haven't explored Azure Data Lake Storage for storage extensively.

We've been actively working with big data analytics for the past three years. Initially, we used Microsoft APS (Analytics Platform System).

From an SAP perspective, direct connectivity to SAP systems is an area that could be enhanced. Our landscape heavily relies on SAP, and we find solutions like Snowflake and Databricks more integrated. Azure Data Lake Storage could improve by providing stronger connectivity options for SAP databases.

My company is in the initialization stage. Like, we've only completed the initial setup phase.

So, we're in the analysis phase, working with a sandbox environment at the moment. It has been six months now. 

I would rate the stability an eight out of ten. 

I would rate the scalability a six out of ten. We've primarily encountered issues during the initial migration from Hadoop to Azure Data Lake Storage.

There are more than 30 end users using it. It's more of a real-time kind of setup. We're focusing on continuous data replication.

We had some troubles. 

Neutral

We used different solutions. We switched to Azure because of the increases data volume we needed to handle. 

I would rate the experience with the initial setup a five out of ten, with ten being easy to set up. 

It took us a few weeks to set up. We haven't started that integration process yet. We're currently in the sandbox testing phase.

Our landscape administrator team handles initial setups. We have a separate infra team for the deployment processes. 

There are around ten members in the team. 

It's quite expensive. Compared to other options we've explored, I would rate the pricing a seven out of ten, with ten being expensive. 

Overall, I would rate the solution a seven out of ten. 

We are using Databricks along with some other tools to have an automated process. The data from different sources gets loaded into Data Lake. My use case for Data Lake Storage is as an integration for various sources of data that are processed and loaded into the lake for subsequent analysis.

Using Azure Data Lake Storage has provided us with a cost-effective solution for data storage, which allows us to manage large volumes of data efficiently.

Storage within Azure Data Lake is cheaper, which is one of the reasons we moved to it. Another valuable feature is the flexibility to scale storage up or down as needed.

The documentation could be more user-friendly with better tutorials. While the initial setup is not too complex, it requires understanding various options and their implications. Improving this can help users understand the configuration process better.

I have been using this solution for one to two years.

I would rate the stability an eight out of ten. It is quite stable.

From a scalability point of view, it is easy to scale. The flexibility to expand or reduce capacity according to requirements is well-handled.

I haven't engaged much with the technical support. I cannot provide an accurate mark for it.

Neutral

We moved from Matillion ETL to Snowflake and Azure Data Lake Storage.

The initial setup process is not very complex but requires detailed knowledge of the various configuration options. One needs to understand the implications of each option, such as cost and performance, which can make the setup process somewhat challenging without adequate documentation support.

The pricing is average, not too high and not too cheap.

Depending on your use case, Azure Data Lake Storage can benefit your organization. It is suitable for medium to large scale companies.

I'd rate the solution seven out of ten.

We use the tool for multiple processes. We use it as a storage layer for files coming in from relational systems and data from real-time streaming systems. We also use it as a staging area for data scientists to consume.

The tool offers a big storage layer. The security aspect is quite good. In Azure, there's an option for soft deletes and policy management. This allows us to store only the most up-to-date data while everything else can be policy-managed. This makes handling and management easier.

If I had to nitpick, maybe the throughput could be faster - how quickly you can access data and how fast data can be written onto the Azure Data Lake Storage.

I have been working with the product for five years. 

I rate the tool's stability a nine out of ten. 

The tool is scalable as long as you pay more. 

Support depends on what agreements you have with Microsoft. Many consultants and companies outside Microsoft can also provide expertise in maintaining and managing the Azure environment, especially the Data Lake environment. It doesn't have to be Microsoft. But if you're raising tickets with Microsoft to fix issues, they're pretty reasonable.

Neutral

The tool's deployment is simple. I've worked with Azure Data Lake Storage in different scenarios. It can be on-premises, in the cloud, or a hybrid solution—it depends on the design. I've worked with it in both on-premises and cloud-based scenarios. For the last few years, as everyone's been transitioning to the cloud, we've mainly worked with cloud-based solutions.

Pricing is tricky because it depends on the solution you're building and the type of Data Lake storage you use—hot or cold.

The tool can be used by small and large companies. It's not restricted by price, so it's not just for high-end companies. Especially with cloud options available now, any company can potentially use it. 

For competitors, from a cloud-based provider perspective, you have Amazon, Google, and other cloud providers. If you are building your custom solution, you can use traditional SAN drives on-premise for data lake storage, which becomes expensive. I'd say the main competitors of the cloud options are Microsoft, AWS, and Google. There are potentially other providers like Alibaba, but I haven't used them, so I can't provide more information about them.

I have experience integrating AI solutions with Azure Data Lake Storage and helped design some of them. AI solutions access data similarly to downstream systems like ETL tools. For cloud providers, the connections to AI tools are typically built into their products.

I rate the overall solution a nine out of ten. I definitely recommend Azure Data Lake Storage. I have recommended it for all the solutions I've designed and built for my clients. I would recommend it to anybody considering entering the data space or looking at building warehouses, AI solutions, etc.

At our company, we strategize and design solutions that have an impact on our client's business. Our company is majorly focused on technological projects for development through business strategies. Azure Data Lake Storage is one of the tools that helps our company develop business solutions for our customers. 

The solution is used primarily for storing Big Data and not for any analytics tasks. If the client's system is already on AWS, then we don't recommend Azure Data Lake Storage. 

Azure Data Lake Storage offers high scalability for data storage, whereas some storage systems in the market offer limited data capacity. Using the product, multiple documents can be read simultaneously. In products like Azure Data Lake Storage, storage scalability is a basic requirement to handle Big Data use cases. 

Almost any storage issue with the solution can be solved if you click on specific properties of the configuration.  

The Azure Data Lake Storage should not only be compatible with the Azure platforms but also with other vendor solutions. I have faced a lack of clarity in the migration process when Azure Synapse is being used with the solution to migrate data to Microsoft Fabric.

Traditionally, when our organization uses Azure Data Lake Storage, we also need to use Azure Synapse. But when data needs to be imported to Microsoft Fabric from the solution using Synapse, Microsoft does not provide a clear migration passage.

Thus integrators face difficulty in migrating data from Azure Data Lake Storage to other products from Microsoft. The parent owner of Azure products is focused on launching new products in a span of a few years but is not focusing on how customers will migrate to their latest product from other data storage systems. Microsoft is not developing the older products once they launch new solutions; they are just providing basic support for the former products. 

I am a user of Azure Data Lake Storage. 

Azure Data Lake Storage is one of the rare services with SLAs that show major stability. On the other hand, if I consider GPT and OpenAI services in Azure, half of the SLAs remain locked. 

I would rate the scalability a ten out of ten. I haven't encountered any data use cases so large that Azure Data Lake Storage wasn't able to manage and scale. For instance, when it comes to storing global transactions of Visa, even for such big use cases, Azure Data Lake Storage will be able to handle the data volume. 

At our company, we work with Azure Data Lake Storage for top-level enterprise companies. 

Instead of reaching out to customer support every time an issue occurs with the product, our organization members use the documentation to resolve the problem. The technical specialists of our company are capable of solving most of the issues on their own, leveraging the documentation around Azure Data Lake Storage. 

The initial setup of the product was super easy. There is a very straightforward process that involves visiting the Azure portal, creating accounts, and configuring the needed ratio of Azure Data Lake Storage. The security configuration of the solution is a bit complex and has multiple technicalities compared to the rest of the setup process. 

The solution's setup process can probably be improved by embedding a feature that guides users through the selections necessary to achieve a secure configuration.

Most of the networking configurations are easily available, and to avail of advanced security options, a user needs to visit the advanced setup section. If Azure Data Lake Storage needs to be integrated with a pre-built network and there is a requirement for authenticated user access and the accessible medium, all such sections of security need to be better guided by the solution provider.  

There needs to be additional assistance provided by the solution during the setup process for a highly secure configuration, especially for individuals who are not networking experts.

Azure Data Lake Storage is one of the most affordable products available from the vendor. When the storage capacity offered by the solution is compared with the computation, Azure Data Lake Storage turns out to be a more affordable option than other databases. 

Azure Data Lake Storage can be used not only for Big Data but also for normal-sized data because it's more cost-effective than other database solutions. The solution offers almost the same features as Snowflake but at a lower price. The cost-effectiveness is the major reason why in our company, we use Azure Data Lake Storage instead of Snowflake and Synapse. 

In our organization, we use Azure Data Lake Storage in integration with Spark and Azure Databricks. When you are working with an extremely large volume of data, you should prefer Azure Data Lake Storage over other data warehouses; other solutions may process data and queries faster but wouldn't be able to manage the data size. I would overall rate Azure Data Lake Storage a ten out of ten.