Azure Question posted in
The official documentation to get you started can be found here.

Azure – Powershell performance degraded when importing Az modules

I’m observing a strange performance hit in one my Powershell scripts when I import some Az modules.

My script is quite complex (4k lines), but in short, it aggregates several data sources and produces a large report on 75k devices. Phase 1 of the script retrieves device data from all data sources in big batches and saves the results in hash tables. Phase 2 is a giant foreach loop that retrieves individual device information from all data sources by indexing into the various hash tables previously created. That way, the number of API calls is kept to a minimum and I can avoid hitting rate limits. This is script version 1.

I recently added a new data source (Azure) to get VDI information for all Azure Virtual Desktop (AVD) session hosts and Azure VMs. To do that, I use several cmdlets from the Az.Accounts, Az.DesktopVirtualization and Az.Compute Powershell modules. Most notably Get-AzWvdSessionHost and Get-AzVM This is script version v2.

The performance degradation is observed during phase 2 (where no API calls happen). V1 used to take roughly 150ms per device. Now, with AVD data, v2 takes 3 times as long: 450ms.

I have since determined that if I simply add the below line at the top of v1 (and change nothing else), v1 becomes as slow as v2:

Import-Module -Name 'Az.Accounts', 'Az.DesktopVirtualization', 'Az.Compute'

This seems to indicate that the Az modules are somehow responsible for the performance degradation. I even tried Remove-Module -Name 'Az.Accounts', 'Az.DesktopVirtualization', 'Az.Compute' right after the import line, but I still observed the performance decline, which seems to indicate that the modules are probably still loaded in the current session somehow, and are having a big impact.

I really can’t explain this strange behaviour and haven’t been able to find any clue online so far.

I guess I could avoid the Az modules completely and query the Azure API endpoints directly with Invoke-RestMethod, but I would prefer avoiding this !

Any suggestions ?

Thanks in advance

EDIT: I have since narrowed down the problem to module Az.Compute. The other 2 Az modules have no impact on the performance of the script.

EDIT2: It looks like this might be linked to assemblies dependencies issues between PowerShell modules Az and Microsoft.Graph. Changing the order of module imports can lead to errors. https://github.com/Azure/azure-powershell/issues/21960#issuecomment-1938831513

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I ended up resolving my performance issue by doing 3 things:

    1. Importing Az modules before Microsoft.Graph modules to prevent assemblies conflicts
    2. Loading and unloading modules just-in-time, i.e. Import-Module just before the first cmdlet that requires that module, then Remove-Module just after the last command that needs the same module.
    3. Optimizing memory usage by filtering out data stored in hash tables, whenever possible.

    I still have no idea why simply importing the Az modules without any other change to my code resulted in such a high performance impact, especially since, although my script used 7GB of RAM, the server still had plenty of free RAM (32GB in total).

    Doing all that, I was able to actually decrease the average time per device to 130ms.


  2. 0

    Considering you’ve added a new Azure data source to get VDI info for All Azure Virtual Desktop (AVD) session hosts and Azure VMs – this might be a rate-limiting scenario. Rate limiting is a mechanism in Azure that prevents API usage spikes on a per subscription basis by limiting the call rate to a specified number per a specified time duration.

    Such ARM induced throttling/limiting happens at two levels i.e. the subscription & tenant, and there’re different limits on each subscription & tenant for such throttling.

    You may read more about Azure Resource Manager throttles requests HERE and plan accordingly.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search