Technical terminology is its own language. If I speak German to a person who doesn’t understand the words I am saying, it would be the same as saying “hey, I was working on a runtime scaling issue yesterday and it was crazy awesome.” You might get the point that I was working on a runtime scaling issue, but not know what that is specifically and would result in you not really understanding what I am talking about. Well, just in case you are ever in a situation where you need to know about runtime scaling, then you are about to learn it, here and now.
But first, I expect already that you know that Azure Functions comes in 3 delightful flavors which you can read about here. They are: Consumption, Premium (aka Elastic) and Dedicated. How Dedicated scales is discussed here, so that’s all I’ll discuss on that one, because running in that mode isn’t really ‘server-less’. For Consumption and plain vanilla Premium the scaling is managed by the scale controller which is described here. I noticed when I looked at the link, the heading is “Runtime scaling” which is a bit ambiguous because that is what I am actually calling Consumption scaling which is represented by the image at that link showing the scale controller, that is the way both Consumption and “out-of-the-box” Premium scale. OK, you are probably asking why I call out Premium as vanilla and out-of-the-box, well, Runtime Scaling, which I will now call “Runtime Scaling Monitor” applies when your Azure Function trigger exists in a VNET. I have written a few articles about Azure VNET here. As you might, should know that a trigger is what you bind an Azure Function to, so that it gets notified when to process some kind of information, i.e. it gets triggered. Triggers, which currently support VNET integration are Storage, Event Hubs, Service Bus, Cosmos DB and Durable Tasks. I’m going to focus on Event Hubs, but the same applies to all supported trigger types. When you create an Event Hub, as is the case with many other Azure products, there is a globally discoverable endpoint. The endpoint for an Event Hub namespace named csharpguitar-vnet for example is https://csharpguitar-vnet.servicebus.windows.net. You use the endpoint, in combination with the Event Hub name to send messages, there is level of security around that endpoint which uses SAS which means although the endpoint is discoverable, it cannot be used unless you have the key. For some companies or individuals, that isn’t secure enough which is why Service Endpoints and Private Endpoints have been created. Service Endpoints allow you to implement NSGs and Firewall rules on the configured endpoint, however the endpoint remains discoverable. A Private Endpoint does the same as the Service Endpoint, but additionally makes the endpoint invisible, or non globally discoverable. Prior to enabling any VNET integration for the Event Hub I am able to send messages to it, as long as I have the SAS access policy Connection string. I used my AzureFunctionConsumer hosted on GitHub here, you see it in action in Figure 1.
Figure 1, using the Azure Function Consumer to send messages to an Event Hub triggered Azure Function
Event Hubs supports VNET integration which is obvious when you check for the Networking blade in the Azure portal, you will see something like that in Figure 2. Note that the subnet named EVENT-HUB has the Service Endpoint named Microsoft.EventHub enabled.
Figure 2, configure a Service Endpoint with an Azure Event Hub which is bound to an Azure Function
Based on this documentation, when you configure a service endpoint for an Event Hub a denyall rule is added to the IP firewall associated to the virtual network. When I try to use the Azure Function Consumer to now send messages to the endpoint, I receive the following exception.
Exception: Ip has been prevented to connect to the endpoint.
What I can do now is add my IP address and allow access to it, which you see in Figure 2 behind that red rectangle. Once I add my IP, then I can send messages to the Event Hub after the Azure Function is added to the same VNET. I sent the messages to the Event Hub before adding the Azure Function to the VNET, the messages were received successfully by the Event Hub but not processed by the Azure Function. You can see in Figure 3 how I connected the Azure Function (which I’m running in Premium) to the same VNET and subnet as shown in Figure 2.
Figure 3, connecting an Azure Function to an Azure VNET
When I sent new messages to the Event Hub, after the VNET integration, their insertion did trigger the Azure Function, the messages sent before the VNET configuration existed where also processed after the VNET was configured. See Figure 4 that shows the 5 messages I set before the Event Hub had a Service Endpoint, 2 messages that were sent before the Azure Function App was configured into the same VNET and 4 messages which were sent after the VNET was configured for the Azure Function App.
Figure 4, Azure Function Event Hub invocations monitor
So yes, all that is required to know and do before Runtime Scale Monitoring even comes into existence. Access the Function runtime settings on the Configuration blade for the Premium Azure Function and you will see the Runtime Scale Monitoring radio buttons, see Figure 5.
Figure 5, VNET scaling with Azure Function bound triggers
The link in the description is to here.
So, in summary of what you did and the scenario. First, you have a Premium Azure Function which is attached to a VNET, see Figure 3. Remember that Azure Functions in Consumption mode cannot be connected to a VNET, it’s not supported. Then, you attached the Event Hub to the same VNET, which contained a subnet that had the Microsoft.EventHub Service Endpoint enabled. You did that because you wanted to use an NSG or a Firewall as an extra layer of security above the SAS key security mechanism. Good work, but now, the scale controller which is used to monitor the number of messages in the Event Hub to help determine how many instances of your Azure Function are required can no longer connect to your Event Hub…because of that denyall rule that gets created when you created and configured the Service Endpoint. You have effectively blocked the scale controller from monitoring your Event Hub. To get around that, turn On the Runtime Scale Monitoring. The first thing that comes to mind as another possible solution here is Service Tags, I wrote a bit about those in my post where I wrote about the WEBSITE_VNET_ROUTE_ALL app setting. I do not think there is a Service Tag for scale controllers, but perhaps one may exist in the future or there is perhaps a Service Tag that can be used to allow all Azure Resources to connect, or maybe I am missing something. Regardless, now I and hopefully you are too able to know what I mean in case I say to you “hey, I was working on a runtime scaling issue yesterday and it was crazy awesome”, you could then respond to me “well, once you know what the word means, it’s really not that hard right?”, yeah, I guess your right, but if you didn’t know what runtime scaling was you’d think I was something more awesome.