May 2007 - Posts
I started to read the "Framework design guidelines" book; it's definitely one of the most interesting books I have ever read. I'll write in my next posts some interesting ideas from this book.
When you start to design your framework you should give simple and powerful tools. Try to match your framework to users with variety skills and abilities that write in different programming language.
Progressive Framework - create a common framework for all situations. Unlike VB and MFS, when you need to develop a complex functionality in your page, it does require a lot of efforts from you and high understanding in MFS.
Fundamental principles of framework design:
# Scenario driven design – first write the scenario and code samples (even in different language) that uses the API's, and only in the end implement it.
# Low barrier to entry – Your API's must be easy to use even by inexperience programmers. It will be easy to find the members and types for basic operations (i.e. divide them to good namespaces: main namespace will contain the common parts like System.Net for example, and sub-namespace will include the extended parts like System.Net.Sockets).
# Trivial usages – just write the commands without initialize some parameters, provide good constructors and override methods (with short and primitive parameters list and intuitive default for them).
Good error message helps to find the wrong use.
# Self documenting object modules – in the simple cases users use the API without any documentation. Although you should write a full documentation for the API for complex case and to give a choice to pre-implementations learning.
Naming: The name is the major factor on the quality of the self-documentation.
Save good names for the classes that really need it. Don't afraid to give a verbose identifier names.
# Exceptions – write explicitly the wrong use in error messages.
# Strong typing – One of the most important subjects is giving intuitive API.
Use User.Name(return string) instead of User.Properties["Name"](return object).
Try to wrap the common properties with a strong type.
# Consistency – Try to be consistent with old API's, it will help with the self-documentation.
# Limiting Abstractions –avoid the use of many abstracts because only an experience architect can implement it (even in the simple cases).
# Layered architecture – divide clearly to a low-level API and a high-level API for high productivity in work. Don't mixes levels in the same namespace try to divide to sub-namespace.(i.e. System.Web contain a low level Http Runtime API's on the other hand, System.Web.UI contains high level page and controls API's).
In SQL server 2005 you have some options to work with large objects. If the data is binary you should use the Binary Large Object (BLOB), and if the data is textual you should use the Character Large Object (CLOB). The BLOB objects and the CLOB objects have a capacity of 2^127 bytes.
When you are dealing with a large object you should ask yourself: Isn’t this object to big to load it from the memory? If the object is too big you should consider using a streaming method. When using a streaming method, the data is being load to the stream by a helper buffer until all the data have arrived (notice: you can use the data only after all the bytes have arrived).
This is an example of using the streaming method:
protected void LoadBlobData()
{
int byteCount;
long currentIndex;
int fileNameColumn = 0;
int dataColumn = 1;
int bufferSize = 1000;
byte[] buffer = new byte[bufferSize];
ConnectionStringSettings connSetting = ConfigurationManager.
ConnectionStrings["MyConnectionString"];
using (SqlConnection cn = new SqlConnection())
{
cn.ConnectionString = connSetting.ConnectionString;
cn.Open();
using (SqlCommand command = new SqlCommand())
{
command.CommandText = "Select file_name,data from
files";
SqlDataReader reader =
command.ExecuteReader
(CommandBehavior.SequentialAccess);
while (reader.Read())
{
string fileName = @"\\MyDirectory\" +
reader.GetString(fileNameColumn);
using (FileStream fs = new FileStream(fileName,
FileMode.OpenOrCreate, FileAccess.Write))
{
currentIndex = 0;
byteCount = (int)reader.GetBytes(dataColumn,
currentIndex, buffer, 0, bufferSize);
while (byteCount != 0)
{
fs.Write(buffer, 0, byteCount);
currentIndex += byteCount;
byteCount =
(int)reader.GetBytes(dataColumn,
currentIndex, buffer, 0, bufferSize);
}
}
}
}
}
}
What is a transaction, in one word?
ACID.
Atomicity: One block that can't be broken into smaller parts.
Consistency: Works on consistent view and leaves the DB in a constituently state.
Isolation: Effects only the transaction.
Durability: Writes the committed transactions to the database.
Single transaction vs. Distributed transactions:
Single transaction – a transaction that works on only one durable resource and on a single open connection (also called local lightweight transaction).These transactions are managed by the Lightweight Transaction Manager (LTM).
Distributed transaction – spans multiple durable resources. Works with two phases committed protocol and has a special manager that is called Distributed Transaction Coordinator (MS-DTC).
The System.Transaction namespace isn't part of the ADO.NET even tough it was develop by the Enterprise Services team in Microsoft. Before the release of the System.Transaction we used the old COM+ hosting model by inheriting from ServicedComponent class (and always using the MS-DTC manager).
The Transaction (in System.Transaction) is fully integrated with SQL server 2005. The transaction automatically promotes a single transaction to a distributed transaction if required.
It's really recommended to use the transaction with the using block. In the end of the block, just assign the complete flag by using the Transaction.Complete() method(This method can be called only once, the second call will throw an InvalidOperationException).
There are three types of transaction options that you can add to the connection:
1. Required (default): If there is an existed transaction - join it, else open a new transaction.
2. Requires new: Always open a new transaction (good for audit operation, when we want to commit the audit statement even though the transaction is rolled back).
3. Suppress: Run out-of-transaction even though there is an active transaction.
A transaction will promote to distribute transaction in three scenarios:
1. The durable resource doesn't implement the IPromotableSinglePhaseNotification.
2. Two durable resources in the same transaction.
3. When there are some application domains to one transaction.
Notice: If you want to create a new transaction manager you just need to implement the IEnlistmentNotification interface (and ISinglePhaseNotification interface for better performance)
On my last post I gave a briefing about what is an aggregate function and now I'm going to implement my own aggregate function.
How to do it?
1. Open a new SqlServer project.
2. Add a new Aggregate template to your project:
3. Implement four methods:
Init: In this method we initialize the fields.
Accumulate: This method will be executed on each row.
Merge: This method will be executed in a situation that the aggregation process runs in multiple threads and the result need to be combined.
Terminate: Returns the result.
4. Enabling the SQLCLR by executing this script:
EXEC sp_configure 'clr enabled', 1 RECONFIGURE WITH OVERRIDE
{To watch the current SQLCLR status runs this script: EXEC sp_configure 'clr enabled'}
For example I implemented my own aggregate function in C# that runs directly in the database.
My aggregate function calculates the multiplication of a selected field in each row.
For example we have this lottery tickets table:
|
Lottery_Code |
Owner |
Chance_To_Win |
|
1 |
Shani |
0.25 |
|
2 |
Doron |
0.25 |
|
3 |
Roni |
0.25 |
|
4 |
Yuval |
0.75 |
What we have to do if we want to implement sql function that calculates the chance that all the lottery tickets will win?
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
[Serializable]
[Microsoft.SqlServer.Server.SqlUserDefinedAggregate(Format.Native,IsNullIfEmpty=true,
IsInvariantToNulls=true)]
public struct MultiSum
{
private SqlInt32 multiSummary;
public void Init()
{
// Init the summary counter
multiSummary = 1;
}
public void Accumulate(SqlInt32 Value)
{
// Business logic
if (Value.IsNull)
return;
multiSummary *= Value;
}
//Calculate the result from multiple threads
public void Merge(MultiSum Group)
{
Accumulate(Group.multiSummary);
}
public SqlString Terminate()
{
if (multiSummary==SqlInt32.Null)
return SqlString.Null;
return multiSummary.ToString();
}
}
SELECT dbo.MultiSum(ChanceToWin) AS Expr1
FROM lottery_tickets
The result is ~0.017
[The chance that everyone will win in the same time is:
The chance that the first win (0.25) multiple the chances that the second (0.25) and so on… all of this is equal to 0.25 * 0.25 * 0.25 * 0.75 = ~0.017]
In SQL server 2005 you can run .Net code within the DB-Server process (code name SQLCLR or integrated CLR).
The SQLCLR family include stored procedure, user defined function, aggregates, triggers and user-define types.
For example, you can create a user define aggregate which execute an operation on each row in the query result (this is a function like others system function: sum, average, count, min and max).
In my next post I'll show you an example to implement on of these functions.