August 2011 - Posts
Build Modern Web Applications with HTML5, CSS3 and JavaScript MSDN Session
HTML5 is making a lot of buzz lately and probably after the BUILD event the buzz will increase. It’s even said that it is going to be a major jump forward in the web and internet world. If you want to find out what is it all about you can attend an open MSDN session that I’ll have at Microsoft Ra’anana. For more details about the session and registration go to this link.
See you there!
Video Manipulation with the Canvas Element
One of the designers in my company asked me to help him with the creation of a transparent video which run on top of an image in HTML. So I did some digging and found a very good resource to do that - Manipulating video using canvas. This post will explain how to combine the video and canvas elements in order to create transparency inside a video during runtime using JavaScript. I’ve based the code on the previous link so they deserve all the credit. You can download the demo code from here.
The Demands
The designer has created a simple movie file with a red square that is floating on top of a black background. He wanted that during runtime the black background will be transparent so the image that he want to show underneath will be visible. Here is a figure that shows how the movie looks like:
The HTML File
<!DOCTYPE HTML>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Working with Canvas</title>
<script type="text/javascript" src="main.js"></script>
<link href="style.css" rel="stylesheet" />
</head>
<body onload="handlePageLoad()">
<video id="video" controls="controls" width="997px" height="587px" autoplay>
<source src="vid.mp4" type="video/mp4" />
</video>
<div>
<canvas id="c1" width="997px" height="587px" />
<canvas id="c2" width="997px" height="587px" />
</div>
</body>
</html>
The HTML include two canvas elements and the video element. I use two canvas elements because one will hold the buffer that I’m working on (and making the black background transparent) and the other will show the movie after the adjustments finished. I also use the following style.css file:
#c1
{
background-image: url(Bg.png);
background-repeat: no-repeat;
}
#c2
{
display: none;
}
#video
{
display: none;
}
In the CSS I’m making sure not to display the buffer canvas and the video element and to show the background image in the first canvas element.
The Real Thing – The JavaScript Code
Here is the implementation of the JavaScript file that will help you to do the job:
var video;
var ctx1;
var ctx2;
var width;
var height;
function timerCallback() {
if (video.paused || video.ended) {
return;
}
generateFrame();
setTimeout(function () {
timerCallback();
}, 0);
}
function handlePageLoad() {
video = document.getElementById("video");
var c1 = document.getElementById("c1");
ctx1 = this.c1.getContext("2d");
var c2 = document.getElementById("c2");
ctx2 = this.c2.getContext("2d");
video.addEventListener("play", function () {
width = this.videoWidth;
height = this.videoHeight;
timerCallback();
}, false);
}
function generateFrame() {
ctx2.drawImage(video, 0, 0, width, height);
var frame = ctx2.getImageData(0, 0, width, height);
var len = frame.data.length / 4;
for (var i = 0; i < len; i++) {
var r = frame.data[i * 4 + 0];
var g = frame.data[i * 4 + 1];
var b = frame.data[i * 4 + 2];
if (g < 10 && r < 10 && b < 10) {
frame.data[i * 4 + 3] = 0;
}
}
ctx1.putImageData(frame, 0, 0);
return;
}
Lets deep dive into the things that the functions are doing:
- The handlePageLoad function is responsible to initialize all the canvas contexts and to add a listener to the play event of the video. When it occurs it will start the timerCallback function.
- The timerCallback function is responsible to stop the generations of frames when the video stops and to generate the current frame using the generateFrame function.
- The generateFrame function is the function that is doing the heavy lifting of drawing on the canvas elements the transparent video. It draw the video frame to the buffer canvas and then extracts that frame. After extracting the frame it is passing on every pixel and check whether it is black. If so it set its opacity to transparent.
The effect will look like:
Summary
In the post I showed how to use a canvas in order to make changes to a running video on the fly. This is only an experiment that I made to show that it is possible to do that. The performance of this implementation isn’t so good and I would have thought about other solutions such a video that includes the background image and the movie.
Profiling JavaScript 101 Using IE9 Developer Tools
Lately I’m involved in a very interesting project which suffer from JavaScript performance issues in old browsers. The main “pain” in the project is the need to support IE8 which is used by many of the client’s users. I’ve got a call to come and help in the process of tuning the application and decided to share some tips of how I profiled the application and found some of the bottlenecks in the application JavaScript execution. Those bottlenecks were handled by the development team very fast and the application’s execution is becoming much better. Since the main problems were mainly in IE8 then I used IE Developer Tools in the process and not FireBug.
Starting a Profiling Session
The first thing that you’ll want to do when you profile JavaScript is to setup your environment. First open the relevant web page in IE and press F12 in order to open the IE Developer Tools. In the developer tools, change the Browser Mode and Document Mode to IE8 and IE8 standards respectfully like in this figure:
Now open the Profiler tab and press the Start profiling button to enable profiling:
Then, press F5 to refresh the page and after the page finish to run press the Stop profiling and get the profiling report.
Reading the Profiling Report
When you have the report here are things that you need to pay attention while figuring out the information you got:
- Count – when the count of function calls is very big (for example the amount of RegExp.exec in the figure) that might indicate that you have a problem. Combining the count with the Exclusive time might help to find bugs in the JavaScript use.
At the client there were a lot of calls to jQuery’s CLASS method. After figuring this out and seeing that the Exclusive execution time was very high I debugged the CLASS method and got the stack call to figure that there was a bug in an application function. Fixing the function helped to improve performance.
There are times that you will find that there are functions that got call in very similar numbers. This might indicate that they were called inside the same function. This is very inaccurate measurement but it might help sometimes.
- Inclusive time – this is the execution time of a method and all of its children. Here you’ll find the methods that executed for very long time. This measurement can help to figure out bottlenecks.
At the client we found out that Telerik’s RadGrid control and its inner LoadSod function are one of the bottlenecks in the application. I’m not saying here that the control has bad performance (on the contrary the control is fast) but only that in the application the control is one of the bottlenecks. I found a way to improve a little the control’s performance but the improvement was very negligible (50 milliseconds) so we dropped it.
- Exclusive time – this is the amount of time that is spent in the function itself. Here you can track very long running functions and then debug them to understand why the run for so long.
At the client we found a bug in one of the functions that was called with setTimeout very often and damaged the performance of the application.
Another way to use the report is the Call tree report that pack all the execution of functions in a tree view:
Here you can drill down the execution stack and figure out using the previous measurements how the application operate.
Tip: pressing double click on a function line in the report can take you to the JavaScript source file (if exists). Then you can set a break point and debug the function call.
Summary
Profiling JavaScript isn’t as simple as it sound. Using profiling tools like in IE Devloper Tools (which I used because IE8 was giving hard times to the client) or FireBug can help to simplify this matter a lot. Knowing how to read the reports will probably help to fine tuned the code you wrote.
Picking a Web Client Side Persistence Mechanism
Yesterday’s evening I got a phone call from a client that wanted to discuss an interesting dilemma – How to pick a web client side persistence mechanism for an offline web application? This is a very important question with all the hype around HTML5 and the new specifications it brings. Many web developers which will build offline applications in the near future will probably find themselves stuck with this dilemma also. So in this post I’ll summarize the options you got and I’ll point you to two new JavaScript libraries that can help you implement your persistence mechanism without the concern what will be the implementation underneath.
Web Client Side Persistence Mechanism Options
With HTML5 specifications there are new options to store data on web client side that add to some older ones (such as cookies). Here are the new options that you face:
- Local Storage -is a key/value dictionary that is stored in the web browser and persists your data even if you close the browser/browser tab. The saved data is local to the web browser and isn’t sent to the server. It include two different type of storages – local storage and session storage. If you like to read more about this mechanism you can read the following post – Using Web Storage in Web Applications.
- IndexedDB – Taken from the W3C specifications – “APIs for a database of records holding simple values and hierarchical objects. Each record consists of a key and some value. Moreover, the database maintains indexes over records it stores.”. I’ll write about this mechanism in a future post.
- Your own custom implementation – that can be based on the new HTML5 File API for example.
So with those new choices what will you pick?
Since indexedDB is currently in specification development (meaning it is very unstable) and local storage is a stable specification you might be tempted to use local storage but you will probably want to use indexedDB in the future. So how can you solve this dilemma?
datajs and AmplifyJS
Lets not solve it at all. You can use a wrapper which will go to indexedDB by default and fallback to local storage or in memory implementation for older browsers. There are two great and brand new libraries that implement such wrappers which I recommend to check when you are going to use a web client side storage:
- datajs – an open source JavaScript library from Microsoft data platform which is described as “new cross-browser JavaScript library that enables data-centric web applications by leveraging modern protocols such as JSON and OData and HTML5-enabled browser features”. datajs supports local storage, indexedDB (currently in experimental support) and a fallback in-memory storage for older browsers. For further reading go to Working with Client-Side Storage Using datajs Store API.
– an open source JavaScript library from .appendto() which is described as “a set of components designed to solve common web application problems with a simplistic API”. It includes a wrapper for various persistent client-side storage systems with the amplify.store wrapper. For further reading go to - Store AmplifyJS API Documentation.
These libraries can make your life easier and remove most of your doubts.
Summary
There are new web client side persistence mechanisms which can help you to create offline web applications. I hope that this post will help you to understand the options you have and some of the libraries that might help you to use those options.
Creating a Many To Many Mapping Using Code First
Today I got a question about how to create a many to many mapping using Entity Framework Code First. The question was how to create the mapping with an existing database. Here is an explanation how to achieve that.
Creating Many To Many Mapping
Sometimes there is a need to create a many to many relation in the database. When we want to achieve that we will create a relation table which will hold the primary key from every table in the many to many relation. Here is an example of a simple many to many relation that can exist in a database:
In Code First, when we want to create such a relation all we have to do is to create the relevant entities which will hold an ICollection in each side of the relation and create the relevant DbContext. Here is an example for a many to many relation in Code First:
public class Person
{
public int PersonId { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public ICollection<Course> CoursesAttending { get; set; }
public Person()
{
CoursesAttending = new HashSet<Course>();
}
}
public class Course
{
public int CourseId { get; set; }
public string Title { get; set; }
public ICollection<Person> Students { get; set; }
public Course()
{
Students = new HashSet<Person>();
}
}
public class SchoolContext : DbContext
{
public DbSet<Course> Courses { get; set; }
public DbSet<Person> People { get; set; }
public SchoolContext()
: base("MyDb")
{
}
}
Simple as that. The problem starts when we have an existing database where the column names or the relation name aren’t generated by Code First convention engine. In the previous figure of the database, Code First will generate the column names – Person_PersonId and Course_CourseId which don’t match to the existing table.
Using Code First Fluent API to Create a Many To Many Relation Mapping
So we have a small problem with existing relation tables. We can solve the problem by overriding the OnModelCreating method and adding the mapping using Code First fluent API. Here is an example of such a mapping to solve the previous model’s problem:
public class SchoolContext : DbContext
{
public DbSet<Course> Courses { get; set; }
public DbSet<Person> People { get; set; }
public SchoolContext()
: base("MyDb")
{
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<Course>().
HasMany(c => c.Students).
WithMany(p => p.CoursesAttending).
Map(
m =>
{
m.MapLeftKey("CourseId");
m.MapRightKey("PersonId");
m.ToTable("PersonCourses");
});
}
}
In the OnModelCreating you will use the HasMany to indicate that course has many students and WithMany to indicate that a person can attend a lot of courses. The Map method will tell Code First how to build the relation table with the relevant name (using ToTable mapping) and with the relevant keys (MapLeftKey and MapRightKey).
Summary
Creating a many to many relation with Code First is easy. You need to get used to the fluent API if you have an existing database but the API is self explanatory so it makes your job easier.