Preview SharePoint List Structure

Monday, February 3, 2014

We’re starting a new project where we're collecting data, amongst others, from SharePoint lists from the Municipality’s site.  Though the SharePoint developers recommended we import data through a web service, we found that SSIS has a component for SharePoint as a data source. We started off by downloading and installing the SSIS component for a SharePoint List Source from CodePlex as it does not come with the SSIS itself.  We added the component to our SSIS through the Tools menu >  Choose toolbox items and then chose the relevant components: I needed to predefine for our DBA the tables we...
no comments

Loading 1TB in 10 Minutes in SSIS – SQL Server Scalability on high end hardware

Monday, November 29, 2010

This is a live post from the session of Henk van der Valk  at Tech Ed Israel 2010 . This session has great tips and tricks for you in SSIS and is in English (just in case you're reading this blog and don't understand Hebrew :)) Checking for problems when test loading data:Select * from sys.dm_os_wait_stats Number 1 wait: pageiolatch_upSolution: add more spindles So - just add more hardware?Number 2 pagelatch_upSolution: add more database files Configure 1 to 4 files per filegroup to get 200+ KB writes IOs. Use Soft Numa - assign a specific port for it Use money data type instead...
2 comments

SSIS 2008 improvements over SSIS 2005

Sunday, May 30, 2010

This is continued live blogging from the Donald Farmer seminar from the SQL & BI convention in Tel Aviv of Data Integration in Large Organizations. SSIS 2008 improvements over SSIS 2005 C# scripting can reference all .net assembliesVSA replaced by VSTA Pipeline Scale Up Tuning SSIS through looking at the buffers on the Data Flow.In SSIS 2005 you would add buffers and break up flows SSIS Tip: Tools > Options > BI Designers > IS Designers:Control Flow Data ConnectData Flow Data ConnectCheck them both to connect the components automatically Use Row Sampling component to do performance testing. One buffer structure goes all through the process.We...
5 comments

Live Blogging Data Integration in Large Organizations

This is live blogging from the Donald Farmer seminar from the SQL & BI convention in Tel Aviv. I would really like to thank Hamada who lent me his netstick so I can blog - Thanks Hamada! Business Intelligence is about people not about IT.Your data is always connected to real world and in direct connection to it. It's important to know your business for you to design a good Decision Support system.BI has always been in the middle of the spectrum - tactical decisions. CEO and operational workers will usually look just at your BI reports, they won't create...

Free SSIS SQL Server 2005 Book

Tuesday, January 27, 2009

In celebration of Microsoft Press 25th Anniversary, they are going to offer a monthly free e-book. Till January 28th 2009, they are offering SQL Server 2005 Integration Services Step by Step by Paul Turley, Joe Kasprzak, Scott Cameron, Satoshi Iizuka, and Pablo Guzman for free. Yes, not just a chapter or two, but the entire book! I think you can find the book useful even if you're using SSIS in SQL Server 2008. Though I know they are some changes made to SSIS from SQL Server 2005 to SQL Server 2008, I think that quite a big part of...

Slowly Changing Dimension Type 2 Tips & Tricks

Tuesday, September 9, 2008

Slowly Changing Dimension Type 2 Tips & Tricks I've blogged in the past about Slowly Changing Dimension Type 2, and I see that it's a subject that really interests you. I remember quite distinctly how hard and demanding it was to create it for my project (trust me I do!) and even to do it for two business entities. So, hoping that I may assist you even slightly, I thought I may share with you a few thoughts I have on the matter: Do you really need it? Now I know this is going to sound quite obvious, but before...

BIDS Helper for SQL Server 2008

Sunday, August 17, 2008

I know I had recommended the BIDS Helper for you before, but as a new version of this Visual Studio add – in just got out this Saturday with support for SQL Server 2008, I thought it's a great time to remind everybody they should check it out. Most of the features in this release are available only through their source code, which is there for you to compile and deploy yourself. My personal favorite feature would be the Non-Default Properties Report, which helps you track the settings in your SSIS Package or your SSAS cube and...

BIDS Helper

Thursday, December 6, 2007

I know this means I'm quite late writing about it, but I only came about to learn about this Visual Studio.Net add-in quite recently and I wanted to study it for a bit before writing about it. The BIDS Helper is an excellent extension to the BI Development Studio which comes with SQL Server 2005. Though my main uses for it are for the SSAS part of my BI project, the newly released version also has features for SSIS. Among other features for SSAS, the BIDS Helper enables you to: Validate aggregations – checking whether the calculations you defined...

Microsoft SQL Server 2008 November CTP

Tuesday, November 20, 2007

Yes, it's true, it's finally here! The latest version of Microsoft SQL Server 2008 CTP just got released and it includes all sorts of goodies – especially for us developing on SSAS. The general highlights include:  T – SQL intellisense Geospatial Support Analysis Services MDX Query Optimizer - Block Computation Analysis Services Aggregation Design Lookup Enhancements for SSIS Robust Report Server Platform  Microsoft is urging you to download it and send them feedback so as to refine and enhance product features. The CTP will automatically expire after 180 days. The download is available here. The CTP version of SQL Server 2008 Express Edition is...
no comments