PDC 2009 Day 1: Data-Intensive Computing on Windows HPC Server with the DryadLINQ Framework

Wednesday, November 18, 2009

Dryad is a distributed execution environment and framework (by Microsoft Research) that strives to optimize data flow across nodes in a computation network and to optimize the computation itself across nodes. A Dryad job is a directed acyclic graph (DAG) of inputs, processing vertices and outputs. This is very similar to Unix pipes, only in 2D (scale-out). It’s very natural to attempt to run LINQ queries and operations on a distributed cluster of machines. PLINQ is the first, intra-machine step for parallelizing declarative queries automatically; DryadLINQ is the second, inter-machine step, with full Visual Studio integration,...
no comments