Table storage migration to sql
Hello,
We’ve got a table held in table storage that contains a value history dating back many many years. Looking at the analytics side, it holds over 0.5TiB. In this format, it’s useless unless you know the partition key. At which point it’s lighting fast.
Our analytics team now want to start asking various questions about the data and start to shape it to gather more insights.
Naturally azure sql comes to mind.
How best is it to extract this data, and place it into a place where T-SQL can be done?
I’m concerned about egress charges too.
Data factory comes to mind. I know this won’t be quick.
Open to ideas and suggestions
Would Azure synapse be better here?
2
Answers
Why migrating Azure Table Storage to Azure SQL Database?
You just said that if you add a partition key you can query.
If you want to do data visualization you can directly connect Power BI to Azure Table Storage and you are don, you can create dashboard from there.
Plus if you already have 0.5Tb if you want to go to Azure SQL Database you have to go to an S3 which will cost you $250 per month.
Are the data continuously ingested?
If so Azure Data Factory is a must and will cost even more.
Azure Synapse is a product that one day will be retired because no one use it.
…stay where you are, use a partition key and do analytics directly to Azure Table Storage through Power BI.
You could use Azure Search Service to index all data in the Table Storage to improve query performance. Just need to be aware of the potential costs, but it’s probably the quickest solution without migrating your data.