Sunday, September 16, 2007
Joe Duffy links to a new MSDN Magazine article on PLINQ: "Parallel LINQ - Running Queries on Multiple Processors".

We have been hearing little bits about PLINQ, and a CTP is on the way. PLINQ redefines the query operators from standard LINQ and makes them split the work across multiple processors.

One of the big challenges that we will have as software developers over the coming years is developing for multiple cores. As extra processors and cores are added to our PC's, the clock speed of the machine isn't increasing as rapidly as in the past, so single-threaded applications may actually run slower than on a high-end single processor machine.

PLINQ takes an interesting approach, and redefines the standard LINQ operators that are used for in-memory queries (such as OrderBy, Join, Where etc), and spreads the work over the available CPUs. As LINQ uses a declarative "Tell me what to do, not how to do it" query syntax, there are hardly any changes to the programming model:
  • Reference the System.Concurrency.dll assembly during compilation.
  • Wrap your data source in an IParallelEnumerable<T> with a call to the System.Linq.ParallelEnumerable.AsParallel extension method.
    • i.e. var q = from x in data.AsParallel() ...
  • Optional: choose your pipelining model: pipelined, stop and go, inverted enumeration. You choose this when processing the results.
  • Don't rely on LINQ's default ordering of results. Explicitly order results if it's important to your program.
  • Change the way you handle exceptions. Multiple exceptions may be thrown in the course of one query, and with PLINQ will be wrapped into a MultipleFailuresException.
  • Avoid modifying data in your where clause, or otherwise mutating shared state during the course of your LINQ query. You may open yourself to race conditions.
All of these steps are covered in depth in the article, but on the surface of it, it looks like Joe and the PLINQ team have made the transition from single to parallel LINQ require as little work as possible. I can't wait to try it out when the CTP drops!

Note that PLINQ only applies to LINQ to Objects and LINQ to Xml queries -- SQL Server still does it's own parallel processing. PLINQ parallelises the query execution that occurs within the .NET program.

All comments require the approval of the site owner before being displayed.
Name
E-mail
Home page

Comment (HTML not allowed)