OpenUp - Transfer gigantic DataTables over WCF / .Net Remoting
Guy Eden has released a new project to OpenUp:
This solution deals with transferring huge DataTables over WCF and .NET Remoting.
When transporting large DataTables between a Server and a remote Client, there are several issues; that stem from .NET serialization. Serialization of a large DataTable is 
memory thirsty. A large enough DataTable will cause the client to get System.OutOfMemoryException, or System.InsufficientMemoryException. Those exceptions cannot be caught at the server side, as they occur in the innards of the framework code that deals with serialization and transport. If the DataTable is really large, the framework will throw the server process all together, with brute force. There is no way around that.
Another issue is throughput, which is not outstanding, and becomes noticeable when the table gets large. The solution at hand circumvents this problem by partitioning the DataTable to chunks and transferring the chunks in a multi-threaded fashion.
Thanks Guy!