The issue we encountered was the inability to efficiently extract and evaluate our data from existing databases, causing limitations with Tableau, which struggled to handle datasets exceeding a hundred million records. Consequently, we explored alternative systems and initially attempted to access all data from target systems. In pursuit of faster and more effective data management, we considered Exasol as an alternative OLAP system. Ultimately, we opted for SingleStore.
When I was working at Wag, MySQL had trouble scaling. We were using an M4-10X RDS instance and three replicas. These machines are expensive and with the growth that the company was experiencing, it was clear we would not be able to scale properly. There were heavy writes done against MySQL that should not have been there; an event table for example. This table was close to 900M rows, taking more than 300GB. I decided to migrate this table to MemSQL, freeing up both lots of space and write resources on MySQL. After the event table was migrated to MemSQL, using the MemSQL built-in S3 pipeline, the table was compressed down to 30GB.
SingleStore enables organizations to scale from one to one million customers, handling SQL, JSON, full text and vector workloads — all in one unified platform.
The issue we encountered was the inability to efficiently extract and evaluate our data from existing databases, causing limitations with Tableau, which struggled to handle datasets exceeding a hundred million records. Consequently, we explored alternative systems and initially attempted to access all data from target systems. In pursuit of faster and more effective data management, we considered Exasol as an alternative OLAP system. Ultimately, we opted for SingleStore.
The solution is primarily used for storing data.
When I was working at Wag, MySQL had trouble scaling. We were using an M4-10X RDS instance and three replicas. These machines are expensive and with the growth that the company was experiencing, it was clear we would not be able to scale properly. There were heavy writes done against MySQL that should not have been there; an event table for example. This table was close to 900M rows, taking more than 300GB. I decided to migrate this table to MemSQL, freeing up both lots of space and write resources on MySQL. After the event table was migrated to MemSQL, using the MemSQL built-in S3 pipeline, the table was compressed down to 30GB.