How can we fetch secrets from keyvault in Azure DataBricks if we select "Allow public access from specific virtual networks and IP addresses"

I agree with @Chen Hirsh, Go to azure key vault -> Networking -> + Add Allow public access from specific virtual networks and IP addresses.

With the above Virtual Network access ,you can try to access the key Vault using pyspark, I got required result as below:

Note: Make sure to create scope to connect key vault with azure databricks: Creating Secret Scope -> Go to azure vault -> Enter the Vault URI .(for Example, https://<key_vault_name>.vault.azure.net/) and Resource ID -> Go to Properties tab of an Azure Key Vault in your Azure portal you get both Vault URI and Resource ID.

Sample Code

from azure.keyvault.secrets import SecretClient
from azure.identity import ClientSecretCredential as cs


kv_URI = "Vault_URI"
TENANT_ID = '<Tenent_Id>'
CLIENT_ID = '<Client_Id>'
CLIENT_SECRET = '<Client_Secret>'
credentials = cs(
            tenant_id=TENANT_ID,
            client_id=CLIENT_ID,
            client_secret=CLIENT_SECRET)

def set_secret(secret_name,secret_value):
        print(credentials)
        secret_client = SecretClient(vault_url=kv_URI, credential=credentials)
        secret = secret_client.set_secret(secret_name,secret_value,enabled=True)
        secr_dic={}
        secr_dic['name']=secret.name
        secr_dic['value']=secret.value
        secr_dic['properties']=secret.properties.version
        return secr_dic
#Use keyvalut secret name    
x1=set_secret('demo1','value')
print(x1)