Tuesday, June 5, 2018

How to write faster loops in C#

       For and ForEach statements are the most common ways to start a loop in many programming languages. To use them, all you need is a collection of objects. Sounds easy right? Well it is easy until you have a large collection object and you need to loop. Iterations takes place sequentially, one by one in for and foreach loops, they run from a Single Thread. This explains the performance issues with larger collection sets. To handle performance problems, usually first thing comes to our mind is to break the large collection in many pieces and run loop in each of them. This method is very common and it works. That's how Hadoop works. You have a large chunk of data and one computer can't handle so let's break it in to pieces and send these pieces to many computers and sell people as a new technology.
   
       What other options do we have? You can try to write Async statements in your loop. Depending on what you are trying to do in loop, async functions can be a big winner too.You have another option, you can try to run for and foreach loop in parallel way. That means loop will use multiple Threads. To be able to run loops in parallel, you need .Net 4.0 or above frameworks.

Parallel.ForEach()

       Parallel class is in System.Threading.Tasks (mscorlib.dll), You can run a for or foreach loop by using the Parallel class. Here is an example of a parallel foreach loop. I have a large dictionary collection and I want to call the CalculateShipping function to do some calculation for every item of that dictionary in the following example.

public void CalculateLoop(Dictionary<int,SoldItem> sample)
{
    Parallel.ForEach(sample, (t) => { CalculateShipping(t.Value); });
}

// You can use async too
public void CalculateLoop(Dictionary<int,SoldItem> sample)
{
    Parallel.ForEach(sample, async (t) => { await CalculateShipping(t.Value); });
}

       Now here, is a real example with timing information for regular and parallel loops. In the following example, I insert 70k JSON objects into Redis with regular loop first then I flush the Redis and insert the same objects by using Parallel loop.


     Here is the stats. It took 2 seconds to insert 70k JSON objects into Redis by using regular foreach loop. It didn't even take 1 second to insert the same data to Redis when I used Parallel foreach loop.




No comments:

Post a Comment