Error while copying Azure Storage table from one Storage to another storage table using Powershell script

esngsrj
October 18, 2024
101 views
0 votes
2 Answers

Facing Error while copying Azure Storage table from one Storage to another storage table using Powershell script "InvalidOperation: Method invocation failed because [System.String] does not contain a method named ‘ExecuteQuerySegmentedAsync’"

#Set Variables to define source and destination values
$sourcestorageaccountname = "stlpttoragetst"
$sourceresourcegroupname = "RG_SUPPLY__ADF_TEST_AUE"
$destinationstorageaccountname = "stlpestorage"
$destinationresourcegroupname = "RG_SUPPLY_DEV_AUE"
$sourcetablename = "DataConfig" 
$desttablename =  "TestTable"

#Get storage account keys
$sourcestorageaccount = Get-AzStorageAccount -ResourceGroupName $sourceresourcegroupname -Name $sourcestorageaccountname
$destinationstorageaccount = Get-AzStorageAccount -ResourceGroupName $destinationresourcegroupname -Name $destinationstorageaccountname
 
$sourcekey = "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$g=="
$destinationkey = "$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$"  

#Function to create CloudTable refernce
function Get-CloudTable {
    param (
        [string]$accountName,
        [string]$accountKey,
        [string]$tableName
    )
    $storageContext = New-AzStorageContext -StorageAccountName $accountName -StorageAccountKey $accountKey

    #Get Cloud Table Refernce
    $table = Get-AzStorageTable -Name $tableName -Context $storageContext

    return $table
}

#Get Source table and Destination table refernce 
$sourceTable = Get-CloudTable -accountName $sourcestorageaccountname -accountKey $sourcekey -tableName $sourceTablename
$destinationTable = Get-CloudTable -accountName $destinationstorageaccountname -accountKey $destinationkey -tableName $desttablename





#Copy Entity from Source Table to Destination table 
$entities = Get-AzTableRowAll -table $sourcetablename

foreach ($entity in $entities){
#     #Insert or replace entity in destination table
       $insertOperation = Add-AzTableRow -Table $desttablename -partitionKey $entity.PartitionKey -RowKey $entity.RowKey -property $entity.Properties

    }
 Write-Output "Table Copied Completed"

Answers

Chosen as BEST ANSWER

We face error while assigning variable to keys so we hardcoded them for the mean while tested on powershell 7

    az login
    az account set --subscription 444444-44445555555555555-55555555555555
    #Set Variables to define source and destination values
    $sourcestorageaccountname = "stsecurestoragetst"
    $sourceresourcegroupname = "RG_PLANTRANSFORM_ADF_TEST_AUE"
    $destinationstorageaccountname = "stsecurestorage"
    $destinationresourcegroupname = "RG_PLANTRANSFORM_ADF_DEV_AUE"
    $sourcetablename = "DataFactoryPipelineConfig"
    $desttablename = "TestTable"
     
    #Get storage account keys
    $sourcestorageaccount = Get-AzStorageAccount -ResourceGroupName $sourceresourcegroupname -Name $sourcestorageaccountname
    $destinationstorageaccount = Get-AzStorageAccount -ResourceGroupName $destinationresourcegroupname -Name $destinationstorageaccountname
     
    $sourcekey = "xxxxx"
    $destinationkey = "zxxx"
    #######
    
     
    function Copy-AzureTable ($sourceresourcegroupname, $sourcestorageaccountname, $sourcetablename, $destinationresourcegroupname, $destinationstorageaccountname, $desttablename, $sourcekey, $destinationkey) {
    
       #$SourceCtx = New-AzStorageContext -StorageAccountName $sourcestorageaccountname -StorageAccountKey $sourcekey
         $SourceCtx = New-AzStorageContext -StorageAccountName "ssecurestoragetst" -StorageAccountKey "KJ7f94f44lF4FZUi7TT7sH2nhuJ/Tfcwr7rPSyjUJKDg=="
         #Write-Host $SourceCtx
        $SourceCloudTable = (Get-AzStorageTable -Name "DataFactoryPipelineConfig" -Context $SourceCtx).CloudTable
        write-Host $SourceCloudTable
     
        $DestinationCtx = New-AzStorageContext -StorageAccountName "stsecurestorage" -StorageAccountKey "OOo5P3Kez9jlEPA/1DC3EYD3yECsnuROW9w=="
     
        # Check if the destination table exists
        try {
            $DestinationCloudTable = (Get-AzStorageTable -Name "TestTable" -Context $DestinationCtx).CloudTable
            Write-Host $DestinationCloudTable
        } catch {
            Write-Error "Destination table '$desttablename' does not exist in the destination storage account."
            return
        }
        $SourceCloudTableContent = Get-AzTableRow -Table $SourceCloudTable
        Write-Host $SourceCloudTableContent
        $SourceCloudTableContent | ForEach-Object -ThrottleLimit 500 -Parallel {
            $DestinationCloudTableUpdate = $using:DestinationCloudTable
            $_ | Update-AzTableRow -Table $DestinationCloudTableUpdate
        }
    }
     
    Copy-AzureTable "sourceresourcegroupname" "sourcestorageaccountname" "sourcetablename" "destinationresourcegroupname" "destinationstorageaccountname" "desttablename" "sourcekey" "destinationkey"

(Edit)

Error while copying Azure Storage table from one Storage to another storage table using Powershell script.

You can use the below script to copy table entities from source to another storage account table using PowerShell 7.

You can see the below entities which I stored in source table storage account.

Portal:

Script:

function Copy-AzureTable ($SourceResourceGroupName, $SourceStorageAccountName, $SourceTable, $DestinationResourceGroupName, $DestinationStorageAccountName, $ExistingDestinationTable, $SourceAccountKey, $DestinationAccountKey) {
   
    $SourceCtx = New-AzStorageContext -StorageAccountName $SourceStorageAccountName -StorageAccountKey $SourceAccountKey
    $SourceCloudTable = (Get-AzStorageTable -Name $SourceTable -Context $SourceCtx).CloudTable

    $DestinationCtx = New-AzStorageContext -StorageAccountName $DestinationStorageAccountName -StorageAccountKey $DestinationAccountKey

    # Check if the destination table exists
    try {
        $DestinationCloudTable = (Get-AzStorageTable -Name $ExistingDestinationTable -Context $DestinationCtx).CloudTable
    } catch {
        Write-Error "Destination table '$ExistingDestinationTable' does not exist in the destination storage account."
        return
    }
    
    $SourceCloudTableContent = Get-AzTableRow -Table $SourceCloudTable
    $SourceCloudTableContent | ForEach-Object -ThrottleLimit 500 -Parallel {
        $DestinationCloudTableUpdate = $using:DestinationCloudTable
        $_ | Update-AzTableRow -Table $DestinationCloudTableUpdate
    }
}

Copy-AzureTable "SourceResourceGroup" "SourceStorageAccountName" "SourceTable" "DestinationResourceGroup" "DestinationStorageAccountName" "ExistingDestinationTable" "SourceAccountKey" "DestinationAccountKey"

Output:

Result         : Microsoft.Azure.Cosmos.Table.DynamicTableEntity
HttpStatusCode : 204
Etag           : W/"datetime'2024-10-17T07%3A29%3A59.4901773Z'"
SessionToken   :
RequestCharge  :
ActivityId     :

Result         : Microsoft.Azure.Cosmos.Table.DynamicTableEntity
HttpStatusCode : 204
Etag           : W/"datetime'2024-10-17T07%3A29%3A59.7052318Z'"
SessionToken   :
RequestCharge  :
ActivityId     :