LINQ to HPC (Formerly known as DryadLINQ) Tutorial: Part 2–Data Partitioning (DSC)

יום שלישי, אוגוסט 9, 2011

A new beta has been released since I wrote part 1 of this tutorial. While very little was changed in the product, we have a new name. Another thing held me back personally from publishing this part was the fact that LINQ to HPC is not a part of Windows HPC R2 SP2. So without farther ado I am proud to present the second part of my tutorial about LINQ to HPC. In part 1 of this tutorial we discussed the fundamentals of DSC: how to manually write data to DSC files and how to use the FromEnumerable<T>...

DryadLINQ Tutorial: Part 1 – Distributed Storage Catalog (DSC) Basics

יום שלישי, יוני 14, 2011

One of the most exciting additions to Windows HPC Server 2008 R2 SP2 (currently in beta) is the support for DryadLINQ. DryadLINQ is an API that allows the creation and execution of large scale, data-parallel compute tasks. One of the core capabilities of Dryad (the underlying framework used by DryaLINQ) is the ability to distribute the data over the cluster and maintain data locality by executing the code on the node storing the data. In order to do both, Dryad utilizes a mechanism called The Distributed Storage Catalog (DSC) which I will cover in this post. ...

There's nothing like a job well done

יום חמישי, אפריל 14, 2011

Today @ MIX Ido and I got some very good news. The first part of our work for the HPC Azure Burst training kit was released today for download by Microsoft. Oh and we got a Kinect. The document released provides on overview of the architectural considerations for using Azure compute nodes as a part of your cluster. There is a lot of work still ahead of us and we will kick out some very cool content on MSDN (and cooler stuff on our blogs ) so stay tuned. Yaniv

Understanding HPC SOA application

יום ראשון, אפריל 10, 2011

As server developers, we are used to a certain level of interactivity. Our services get a request and often return some kind of response. Lately I found myself justifying having HPC batch jobs. It’s sometimes hard to grasp that classic HPC programs, like the Human Genome Project, rendering a full feature 3D animation movie, or simply operating a “civilian purposes” nuclear reactor, take a long time. And when you execute such programs for days, weeks, months, etc., interactivity is not even a consideration. Batch jobs are the very essence of classic HPC applications. Still, Microsoft is trying...
tags: , ,
no comments