Question 103082 in Azure Question posted in
The official documentation to get you started can be found here.

Set time to live in Azure Blob containers created using BlobServiceClient

I’m currently using the following setup to create containers in an Azure Storage Account, and writing blobs to those containers:

from azure.storage.blob import BlobServiceClient

connstr = "..."
bsc = BlobServiceClient.from_connection_string(connstr)
container_client = bsc.create_container(name="some_container")
blob_client = container_client.upload_blob("some_blob", data="data_item", metadata={})

but nowhere in this flow can I find a way to set a time to live (TTL, or maximum lifecycle time) for these blobs or containers.

From what I’ve understood you can create rules for the storage account using blob storage lifecycle management rules, but this would complicate the script significantly. I’d ideally like to be able to set a different blob-TTL for each container. Am I missing something here?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I ended up going with a combination of the suggestion from @dasari-kamali and other sources. My solution was that I defined a number of standardized rules in the Lifecycle-Management section of my Storage Account:

    enter image description here

    Here I have specific rules for 1 day, 7 days, etc. Each of the specific rules have key/value filters which determine which blobs they apply to. Here for example is the filter value for "specificTtl3":

    enter image description here

    Then you can use the blob tag values (TTL-in-days=7) to determine which rule they should follow. I additionally have a global blob deletion rule which defines a max for all blobs (without tag filtering).

    This workaround isn't perfect as you need to have predetermined rules in place (meaning you can't dynamically set them to a value that isn't configured), but it works nicely for my application. This way you also avoid having a timed function run at regular intervals.


  2. 0

    Below is the Python code snippet for Setting TTL Property in Azure Blob Object :

    Code:-

    import time
from datetime import datetime
from azure.storage.blob import BlobServiceClient

#Code - Initialize Connection String of Azure Storage and Container name using python
blob_client = container_client.upload_blob("sampleblob.txt", data="data_item", metadata={"expiration_time": "2023-05-15T13:54:21Z"})  

def check_and_enforce_ttl(container_client):

    blobs = container_client.list_blobs()
    current_time = int(time.time())

    for blob in blobs:
        blob_properties = blob_client.get_blob_properties()
        metadata = blob_properties.metadata

        ttl = metadata.get("ttl")
        if ttl is not None:
            ttl_seconds = int(ttl)

            creation_time_timestamp = int(blob_properties.creation_time.timestamp())
            delete_time = creation_time_timestamp + ttl_seconds

            if current_time >= delete_time:
                blob_client.delete_blob()
                print(f"Blob '{blob.name}' has expired and has been deleted.")
            else:
                pass
    check_and_enforce_ttl(container_client)

    Output:-

    enter image description here

    For initializing the Azure Blob Storage Connection String and retrieving the blob metadata information using Python Code, refer to this MS Doc1 & Doc2.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search