I am currently working on understanding Event Hub along with Azure Function. I have checked out event driven scaling which mentions about the scale controller. But none of the azure documents I referred gave out the logic behind scale controller as in on what basis does it dynamically scale in or out. How does the controller know when to scale in or out and on what mechanism does the controller work?
Can anyone please help me in understanding the logic behind scale controller?
2
Answers
You can find some references to how Azure Functions scales in the host.json.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-host-json-v1?tabs=2x-durable-functions#http
Not an expert answer, but I try. For each trigger there are certain defaults (which of course you can override), if those limits are exceeded a new instance is spawned.
The exact algorithm used by the scale controller is not publicly available but, at a high level, it involves considering metrics over a period of time to understand whether the incoming rate of events is too quick, too slow, or just about right for the rate that events are being processed.
That information is used as part of the computation for an the ideal number of instances, which is then weighed against other factors from configuration and internal to the Functions runtime to vote on whether to add/remove instances.
The metrics themselves, and the associated calculations, are public and can be found in the EventHubScaleMonitor.
In a nutshell, it reads the last enqueued sequence number for a given partition and compares that to the last recorded checkpoint for that partition. The difference between these values is considered the number of events that remain to be processed for that partition (also known as the "event backlog"). There are some corner cases here, such as a sequence number rolling over to 0 once it hits Int64.MaxValue. Generally, though, it is fairly straightforward.
The "Consuming Events" section of Integrate Event Hubs with serverless functions on Azure also provides some high-level context around scaling, with a focus on partitions for the Event Hub. It references some concepts from the legacy Event Hubs SDK package which is no longer used by the extensions, but the high-level details are still accurate.