Using Time To Live feature of CosmosDB

      Another great feature of Cosmos DB is, TTL (Time To Live) support. This is a great option to have if you need a database system with Caching option, or you need to purge your data and you don't want to develop a function to remove data from your dataset. CosmosDB's TTL feature is pretty simple, all you need to do is, turn this feature on and declare when data should be removed from your dataset. Best part about TTL in CosmosDB is, CosmosDB does not charge you when it removes the data from your containers so you can use your Request Units for other transactions!

There are two ways to set Time to Live value. You can set the TTL value on a container or you can set it on a specific item by using CosmosDB SDK. TTL value must be in seconds. TTL timer resets if data gets modified for any reason.

SQL Server vs Cosmos DB

       SQL Server does not support this feature. If you have SQL Server 2017, You have option to use Temporal Tables. This might not be a good option for you depending on the size of your table. Temporal Tables can get pretty large very fast depending on the number of transactions hitting your table. Also you need to convert your table into Temporal Tables, that means schema change! Purging data in SQL Server can be really pain because you need to develop this functionality manually. I am hoping to have TTL feature for Memory-Optimized tables one day! Non-Durable Memory-Optimized tables are great for caching data in SQL Server but you can't convert Memory Optimized tables in to Temporal Tables and Memory Optimized Tables don't support TTL functionality.

How to turn on Time To Live

       Let's look at how to set TTL in container level first. There are two ways to do this, First one is from the Azure Portal. Click on the Data Explorer and pick the Scale & Settings option under your container. You should see the following options.

       All you need to do is, change the Time To Live setting to On(no default) or On. If you like to control TTL in container level, you need to pick On option. If you like to give specific TTL value to items in a container by using SDK, then you need to pick On(no default) option. If TTL value is missing or if you set the value -1, items will not expire!

      You can turn on the TTL from SDK too, following example shows you how to create a container with TTL feature. Cosmos DB team released a new SDK lately,  Following examples shows you how to create container with TTL feature in SDK 2.x and SDK 3.x. In the following example, I create a container and I set the TTL value to 24 hours. If any item does not get modified for 24 hours, CosmosDB removes that item from my dataset for free!

Creating a container with TTL feature using SDK 2.x

DocumentCollection collectionDefinition = new DocumentCollection();

collectionDefinition.Id = "SaleTracker";

collectionDefinition.PartitionKey.Paths.Add("/CustomerId"); collectionDefinition.DefaultTimeToLive = 60 * 60 * 24; // expires in 24 hours 


DocumentCollection ttlCollection = await client.CreateDocumentCollectionAsync( UriFactory.CreateDatabaseUri("OrderTracker"), 

collectionDefinition, 

new RequestOptions { OfferThroughput = 10000 });

Creating a container with TTL feature using SDK 3.x

public async Task<double> CreateContainer(string dbname, string containerId, string partitionId, int ttl)
{ 
var settings = new CosmosContainerSettings(){ 
DefaultTimeToLive = DateTime.Now.TimeOfDay.Add(new TimeSpan(0,1,0)), 
Id = containerId 
}; 
Settings.PartitionKey.Paths.Add(partitionId); 
var response = await cosmosClient.Databases[_dbname].Containers.CreateContainerIfNotExistsAsync(settings); 
return response.RequestCharge; 

     Setting TTL on an item

      You can set a TTL value on an item by using the CosmosDB SDK. Before I show you any code here, I want to explain one important thing. In SQL Server, we have virtual columns. Those are the columns SQL Server uses internally. If you know their names, you can display their values by including them in your queries. CosmosDB has the same concept. CosmosDB has bunch of special keys that holds data for CosmosDB to use internally. One of those keys named _ts , This column/key tells us the modified (created/updated) datetime of an item.

       CosmosDB uses _ts value to expire any documents in a container. To be able to set up the TTL on an item, first you need to be sure TTL settings is On. If TTL feature is off, TTL field will be ignored. To start, first we need to add a new property to the data model. This new field must be serialized to a Json property named "ttl". In the following example, I add a new field named TimeToLive to my Order model. I override its serialized name to ttl by using the JsonProperty annotation.

public class Order { 
 public string id { get; set; } 
 public string OrderId { get; set; } 
 public string CustomerId { get; set; } 
 public DateTime OrderDate { get; set; } 
 public DateTime ExpectedDeliveryDate { get; set; } 
 public Customer CustomerInfo { get; set; } 
 public SalesPerson SalesPersonInfo { get; set; }
 [JsonProperty(PropertyName="ttl", NullValueHandling = NullValueHandling.Ignore)]
 public int? TimeToLive { get;set; }

}

      Now, we are ready to set a value for TimeToLive property. The value must be in seconds. CosmosDB will take care of the rest from this point. Here is two ways to do that. I like to use object extensions for this.

public Order SetTimeToLive(Order currentOrder, int ttl = 60){

 currentOrder.TimeToLive = ttl;

 return currentOrder;

}

// I like this one
public static void SetTimeToLive(this Order currentOrder, int ttl=60){

  currentOrder.TimeToLive = ttl;

}

    In the following scenario, Container TTL value will be applied to this item If there is a default TTL value in container level. Use -1 if you don't want this object to expire.

currentOrder.TimeToLive = null;
//  use -1 to prevent this item from expiring
cuurentOrder.TimeToLive = -1;