DCSIMG
SqlBulkCopy Bug Workaround - .NET Geek

.NET Geek

"It is upon the Trunk that a gentleman works" - Confucius

SqlBulkCopy Bug Workaround

We are using SqlBulkCopy to import large xml documents into a database. However, we encountered a “minor” bug in the .Net Framework related to table naming. It turns out that if you have a dot “.” in the table name, SqlBulkCopy doesn’t work. The problem has been reported and a KB article is available, but without a workaround besides renaming the table. Our problem wasn’t with the actual name of the table(s), but with the name of the schema. The schema naming convention used in that specific database is [CompanyName.Project].TableName.

The following code failes with the exception below. (Note the assignment to DestinationTable below.)

IDataReader reader = new XmlDataStreamer(/* other stuff here */);
using (SqlBulkCopy sqlBulkCopy = new SqlBulkCopy("Server=(local);Database=ScratchDb;Trusted_Connection=True;", SqlBulkCopyOptions.TableLock))
{
    sqlBulkCopy.DestinationTableName = "[SchemaPart1.Part2].ImportTable";
    sqlBulkCopy.WriteToServer(reader);
}

SqlBulkException

For the sake of argument, let’s just say that it is beyond our control to change the naming of the schema. After Googling around for a solution, it appeared to be more than a few with the same problem. Now what do we do?

My first thought was to try to figure out if there’s a way to manipulate .Net into accepting the table name. After spending some quality time with Reflector it was time to take a walk and think about something else. I discussed the issue with one of the other developers on the team. While describing the issue a solution took form.

If I can’t change the .Net Framework and I can’t change the schema/table name, maybe I can somehow disguise the table name. Being inspired by dynamic languages lately - If it looks like a table and behaves like a table it must be a table. (Not really, but maybe .Net won’t notice the difference)

Solution: Use a view!

  • In the database create a new schema without dots in the name.
  • Create a view over the import table with a one-to-one mapping to the table.

That leaves us with a view named: [CompanyNameProjectImport].ImportTableView

The following works:

IDataReader reader = new XmlDataStreamer(/* other stuff here */);
using (SqlBulkCopy sqlBulkCopy = new SqlBulkCopy("Server=(local);Database=ScratchDb;Trusted_Connection=True;", SqlBulkCopyOptions.TableLock))
{
    sqlBulkCopy.DestinationTableName = "[CompanyNameProjectImport].ImportTableView";
    sqlBulkCopy.WriteToServer(reader);
}

Comments

Bijayani said:

Hi,

I happened to see your post find it quite informative. I would like to share a link where a software engineer has shared a tip on "SqlBulkCopy In .NET".

Here is the link:

www.mindfiresolutions.com/SqlBulkCopy-In-NET-215.php

Hope you find it useful and of assistance.

Thanks,

Bijayani

# January 28, 2010 8:10 AM

Kim said:

@Bijayani - I'm not sure I would use a datatable if high speed imports is what you are looking for. SqlBulkCopy is very efficient indeed, but loading all of the data into a datatable (in memory) just to use SqlBulkCopy to insert the data seems counter intuitive to me.

# January 28, 2010 8:53 PM

Ken Powers said:

I have an alternative work-around.

First, I capture the name of the schema and save it to a String.  Then I remove the dot character from the name and create a new schema.  Then I use the "ChangeSchema" method of the SMO Table Class to move the table to the new schema, and then I bulk copy the data.

Here is the code:

// System.Data cannot handle schema names that contain a "." character

// This code creates a new schema without the "." character

if (objSchema.Name.Contains("."))

{

   // Create String variables to hold the schema names

   String schemaName = objSchema.Name;

   String schemaNameNew = "";

   // Execute a Replace operation on schema name

   schemaNameNew = schemaName.Replace(".", "");

   // Create a new schema with the new name

   Schema objNewSchema = new Schema (destDB, schemaNameNew);

   // You may be loading multiple tables from this schema

   // Check to see if the schema already exists

   // Schema exists, move the Table object to the new scheam

   if (destDB.Schemas.Contains(objNewSchema.Name))

   {

       objTable.ChangeSchema(objNewSchema.Name);

   }

   // Schema does NOT exist.  Create the new schema and then

   // move the Table object to the new schema

   else

   {

       objNewSchema.Create();

       objTable.ChangeSchema(objNewSchema.Name);

   }

}

After the transfer, you can run

"ALTER AUTHORIZATION ON NewSchemaName.TableName TO OriginalSchemaName" for each table.  This is a very fast process and saves you from having to create a view.  You should be able to run ALTER AUTHORIZATION using a SqlCommand object, but I haven't tried that yet.

# April 9, 2010 8:30 PM

Kim said:

How do you handle the case where some other query tries to access the target table with the original schema in the temporary time-frame where the schema is "dot less"?

# April 12, 2010 10:15 AM

Ken Powers said:

In our environment, we don't have to worry about that.  The target database will not be available for queries while the bulk copy operation is running.

However, the current version of my code transfers the table back to the original schema immediately after the SqlBulkCopy operation is completed:

// Now create a SqlDataReader and transfer the data

using (SqlDataReader dr = cmdCopyData.ExecuteReader())

{

 DateTime start = DateTime.Now;                        

 SqlBulkCopy sbc = new SqlBulkCopy(dest.ConnectionString.ToString());

 sbc.BulkCopyTimeout = 6000;

 sbc.BatchSize = 10000;

 sbc.DestinationTableName = "[" + objTable.Schema + "].[" + objTable.Name + "]";

 Console.WriteLine("Beginning Copy ...");

 sbc.WriteToServer(dr);

 Console.WriteLine("Copy completed in {0} seconds.", DateTime.Now.Subtract(start).Seconds);

 sbc.Close();

}

// Now transfer the object back to its original schema, if necessary

if (blnSchemaChanged == true)

{

 Schema objOrigSchema = new Schema(destDB, schemaName);

 objTable.ChangeSchema(objOrigSchema.Name);

}

# April 16, 2010 11:36 PM

Ken Powers said:

Here's an issue I found with my code.  The first time I tested my code, my destination database was "empty".  The database is created at run time, so it has no schemas or tables.  When I ran my code using the empty database as the destination, it worked.  When I used an existing database as the destination, I got an error when it ran the following section of code:

// Schema exists, just move the Table object to the new schema

if (destDB.Schemas.Contains(objNewSchema.Name))

{

 objTable.ChangeSchema(objNewSchema.Name);

}

// Schema does NOT exist.  Create the new schema and then

// move the Table object to the new schema

else

{

 objNewSchema.Create();

 objTable.ChangeSchema(objNewSchema.Name);

 // Add the temporary schema to the collection so we can clean it up later

 tempSchemas.Add(schemaNameNew);

}

The code throws an SmoException when it tries to execute the ChangeSchema method.  The exception says "Schema can only be changed for an existing object."  Any ideas why this is happening?

# April 16, 2010 11:42 PM

icon package said:

 You commit an error. Write to me in PM, we will talk.

<a href="www.hpixel.com/.../a>

# September 24, 2012 10:52 AM
Leave a Comment

(required) 

(required) 

(optional)

(required) 


Enter the numbers above: