I have found that the biggest obstacle I face when adopting a new language, technology, or framework is using something I don't fully understand how to implement myself. I read hundreds of blog posts every week talking about language extensions to JavaScript, cool new iOS application frameworks, and brand-new SaaS offerings on top of Windows Azure -- just as a small sample. Obviously, just using some piece of technology or adapting a sample to my needs is usually not that hard. The thing is, I can't bring myself to adding code to my arsenal if I don't understand how it works and how it has been implemented. It's somewhat akin to the Not Invented Here syndrome, except I don't actually write all my own frameworks; I just want to
be able to write them. Here are a few recent examples.
In late 2011 I started getting my feet wet with Node.js, and in 2012 implemented several personal and commercial projects on top of Node, Express, and many other Node modules. I was very reluctant to start working with Node until I grasped the fundamental concepts -- the event loop and the asynchronous nature of everything -- which gave me the foundation to implement "something like Node" should I want to. There was even a time when I was considering to implement a Node-like HTTP framework with the new C# support for async/await, but gave up because there are many like it, ASP.NET MVC async controllers notwithstanding.
Next, I always was -- and to some extent, still am -- quite "afraid" of WPF. I can't say that I love client development in general, but nearly everything feels, on a purely psychological level, like a more appealing target for me than XAML-based frameworks. It's not that WPF is very hard for me to use: I understand the fundamental concepts such as data binding, styles, resources, and data templates, enough to implement simple desktop applications or sample Windows 8 and Windows Phone apps. It's the depth and breadth of WPF that always keeps me wondering: Is there a better way to do what I just did? How is this XAML expression working under the hood with this data context and that dependency property? Should I have moved this entire chunk of code into a behavior or a separate control?... I can't say I haven't tried: I read at least three books on WPF, with a total of more than 1500 pages, and they had had zero effect on my state of mind. The result, at least for me, is that on a subconscious level I steer clear of XAML-based frameworks, because I am not sure how to implement my own. The funny thing is that I'm fairly comfortable with several "thick" client technologies, including MFC, Windows Forms, Android, and iOS -- but it's as though XAML, with all its objective advantages, had developed a conditional gag reflex in my mind.
Moving on to something I am very comfortable with, I always felt at home with Reflection-based approaches to virtually anything, from serialization through validation all the way to code generation. Attributes and Reflection have somehow grown very strong roots in my mind ten years ago, when I was first working on a large .NET application, and have been a very powerful tool for me ever since. I attribute this mostly to the fact that I understand how managed types are laid out in memory, how .NET metadata is organized, and thus how Reflection can provide access, at runtime, to this information. In fact, I immediately feel at home with Reflection in other languages and frameworks as well: for instance, my first instinct when working on the Windows Azure Mobile Services unofficial Android SDK was to write my own JSON serializer in Java instead of relying on a third party library. After the fact, it might not have been the best decision, but it took me less than 2 hours to have a working serializer that supports most of the types WAMS requires.
As a final example, I have a very big problem adopting new languages, especially if they're not merely compiled to another language. In other words, I wouldn't mind using something like TypeScript or CoffeeScript, where I can easily see how the source is compiled down to JavaScript, and how a new piece of syntax is merely a mapping or syntactic sugar on top of existing syntax. But a "brand new" language, such as Objective C, poses huge problems for my mind. It's not the brackety syntax I have trouble with; it's the "how" of the language. How are Objective C objects laid out in memory? How are methods dispatched, considering that you can override them and even dispatch them dynamically by name? How does the compiler manage memory automatically (yes, reference counting -- but that's just a buzzword)? The same applies to a language like Python: I can use Python to write scripts, create modules, and even interact with C-style DLLs, but I don't have a clear picture of where attributes are stored and how typing works in a dynamically-typed language.
To conclude, I wish that every tutorial in the world were followed by a "how it works" section that would tell me how I can implement that language, technology, or framework -- on my own. Until such time, I hope this post explains why I tend to over-analyze how stuff works, to the extend of implementing it myself. The root cause of this might be that I'm used to systems, frameworks, and languages that I fully understand; it might have something to do with debugging or performance concerns; but at the end of the day, I need to know how it works: otherwise, I face a psychological barrier that gets more and more painful to overcome every day.
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
During the last few weeks I have been involved in a very interesting project with lots of potential, using Windows Azure Media Services. In a nutshell, Windows Azure Media Services supports encoding, management, and streaming of media from Windows Azure, in a completely hosted solution.
The client requested a proof-of-concept end-to-end solution that involves uploading existing media assets to Windows Azure, encoding them to smooth streaming formats, and delivering them to an audience using Windows computers, iOS devices, and Android devices. (The client’s current solution involves shipping of media files to a third party company, which takes care of the encoding jobs and delivers back URLs for streaming.)
Architecture
The architecture I ended up implementing on top of Windows Azure is the following:
From the client’s perspective, the only on-premise component is the uploader application, which uploads H264 MP4 files to an ASP.NET web application hosted in Windows Azure Web Sites (1). The web application creates a Windows Azure Media Services asset and streams the client’s upload to Windows Azure Blob Storage (2). Metadata about the upload, such as the file name, description, thumbnail URL, and other information is stored in Windows Azure Table Storage (3). When the upload completes, the ASP.NET application puts a message in Windows Azure Queue Storage (4), which a Worker Role dequeues and initiates an encoding job with Windows Azure Media Services (5). When the encoding job completes (6), the Worker Role generates streaming URLs with the Windows Azure Media Services origin servers, and stores these URLs for the web application to consume (8).
This end-to-end process can potentially replace the client’s current solution for media encoding and delivery. Although full pricing information for Windows Azure Media Services has not yet been disclosed (for example, the cost of on-demand streaming units), our estimates indicate a considerable cost savings when moving the whole solution to Windows Azure.
Support for multiple devices
To support as many devices as possible, it was necessary to encode the original MP4 video to two streaming formats: IIS Smooth Streaming, supported easily by Silverlight players, and Apple HLS, supported natively by iOS devices as well as newer Android devices.
On Windows and Mac computers, we could stream the video using a Silverlight player, which I borrowed from CodePlex.
On iOS and Android devices, all we had to do is embed the streaming URL in a <video> element:
<video width="640" height="480" controls>
<source src="..." />
</video>
Finally, for “unsupported” devices, we chose to use a <video> element pointing to the original MP4 file, progressively downloaded from Windows Azure Blob Storage. Although this does not provide the advantages of smooth streaming (adaptive bitrate), it works out of the box on any device that supports the <video> element in a browser.
Implementation highlights
Overall, rolling out the above solution took only a few days. I integrated most of the moving parts on-premises, and then started uploading parts of the system to Windows Azure. First, I uploaded the ASP.NET web application to Windows Azure Web Sites, which was a breeze. Next, I wrapped a console application (that took care of starting and monitoring the encoding process) in a Worker Role. Finally, after sorting out a few remaining issues, the solution was ready to roll on Windows Azure. Below are some of the implementation highlights:
Creating a Windows Azure Media Services asset and uploading the MP4 file to Windows Azure Blob Storage:
Stream inputStream = Request.InputStream;
var blobClient = storageAccount.CreateCloudBlobClient();
var asset = mediaContext.Assets.Create(
assetName, AssetCreationOptions.None);
var writePolicy = mediaContext.AccessPolicies.Create(
"policy for copying", TimeSpan.FromMinutes(30),
AccessPermissions.Write | AccessPermissions.List);
var destination = mediaContext.Locators.CreateSasLocator(
asset, writePolicy, DateTime.UtcNow.AddMinutes(-5));
var destContainer = blobClient.GetContainerReference(
new Uri(destination.Path).Segments[1]);
var destBlob = destContainer.GetBlockBlobReference(file);
dest.UploadFromStream(inputStream);
destBlob.Properties.ContentType = "video/mp4";
destBlob.SetProperties();
Getting a publicly readable URL for the static MP4 file uploaded:
var assetFile = asset.AssetFiles.Create(file);
assetFile.Update();
asset = mediaContext.Assets.Where(
a => a.Id == asset.Id).FirstOrDefault();
var readPolicy = mediaContext.AccessPolicies.Create(
"policy for access", TimeSpan.FromDays(365 * 3),
AccessPermissions.Read | AccessPermissions.List);
var readLocator = mediaContext.Locators.CreateSasLocator(
asset, readPolicy, DateTime.UtcNow.AddMinutes(-5));
string[] parts = readLocator.Path.Split('?');
string staticUrl = parts[0] + "/" + file + "?" + parts[1];
Firing off an encoding job that first encodes the MP4 file to the IIS Smooth Streaming format, and then encodes the result to Apple HLS format suitable for streaming from iOS devices:
var job = mediaContext.Jobs.Create(
"Encoding Job for " + asset.Name);
var encoder = mediaContext.MediaProcessors.Where(
m => m.Name == "Windows Azure Media Encoder").First();
var encodingTask = job.Tasks.AddNew(
"Encoding Task for " + asset.Name, encoder,
"H264 Smooth Streaming 720p", TaskOptions.None);
encodingTask.InputAssets.Add(asset);
var ssOutput = encodingTask.OutputAssets.AddNew(
"output-Silverlight-" + asset.Name,
AssetCreationOptions.None);
var packager = mediaContext.MediaProcessors.Where(
m => m.Name == "Windows Azure Media Packager").First();
var conversionTask = job.Tasks.AddNew(
"Conversion Task for " + asset.Name, packager,
File.ReadAllText("MediaPackager_SmoothToHLS.xml"),
TaskOptions.None);
conversionTask.InputAssets.Add(ssOutput);
conversionTask.OutputAssets.AddNew(
"output-HLS-" + asset.Name, AssetCreationOptions.None);
job.Submit();
What does the future hold?
If this proof-of-concept is successful, we could be talking about petabytes of data hosted in a similar fashion on Windows Azure and streamed to millions of viewers.
We still need to find better solutions for older Android devices, which do not support Apple HLS natively; there’s also live streaming to consider, which Windows Azure Media Services currently does not support.
In closing
This was a project that highlighted the flexibility and ease of use that have been so typical of Windows Azure since the first time I’ve touched it. Whether it’s a Node.js app on Windows Azure Web Sites, a Windows Azure Mobile Service accessed by iOS and Android apps, or a fuller solution like the above, Windows Azure is fun to use and easy to understand.
When I first pitched this solution to the client, I faced lots of anxiety about putting data on Windows Azure or streaming media from a datacenter far away. Later in the project, I was greeted by the client’s less-technical folks humming away on Windows Azure Virtual Machines, configuring firewalls for Windows Azure SQL Database, and experimenting with publishing ASP.NET applications to Windows Azure Web Sites.
Thanks to Maor David and Noam King from Microsoft Israel for their support with this project.
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
I encountered an interesting gotcha today, which I thought would be interesting to share with you. The symptoms were the following: I wrote a simple C++ function exported from a DLL, and tried to invoke it using a P/Invoke wrapper. The C++ function returns a bool, and was declared as follows (the code was simplified for expository purposes):
extern "C" __declspec(dllexport) bool IsPrime(int n) {
if (n <= 1) return false;
if (n == 2) return true;
if (n % 2 == 0) return false;
for (int i = 3; i < n; ++i) {
if (n % i == 0) return false;
}
return true;
}
The managed counterpart was:
[DllImport(@"..\..\..\Debug\NativeDll.dll",
CallingConvention=CallingConvention.Cdecl)]
static extern bool IsPrime(int n);
At runtime, for some values of n, the output would be inconsistent. Specifically, when n=0 or n=1, the managed wrapper would return true – although the C++ function clearly returns false. For other values, however, the wrapper returned the right values.
Next, I disassembled the C++ function, to find the following code responsible for the first branch:
cmp dword ptr[n], 1
jg BranchNotTaken
xor al, al
jmp BailOut
…
BailOut:
…
ret
In other words, to return false, the function clears out the AL register (which is the lowest byte of the EAX register), and then returns. What of the rest of the EAX register? In the debug build, it’s likely to contain 0xCCCCCCCC, because of this part of the function’s prologue:
lea edi,[ebp-0CCh]
mov ecx,33h
mov eax,0CCCCCCCCh
rep stos dword ptr es:[edi]
To conclude, the function returns 0xCCCCCC00 in the EAX register, which is then interpreted as true by the managed wrapper. Why? Because by default, P/Invoke considers bool parameters to be four-byte values, akin to the Win32 BOOL, and then any non-zero value is mapped to true.
To fix this, I had to convince P/Invoke to marshal back the return value as a single byte. There’s no dedicated type for that in the UnmanagedType enumeration, but U1 (unsigned byte) or I1 (signed byte) will do the trick:
[DllImport(@"..\..\..\Debug\NativeDll.dll",
CallingConvention=CallingConvention.Cdecl)]
[return: MarshalAs(UnmanagedType.U1)]
static extern bool IsPrime(int n);
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
I’m very happy to announce our next conference—the biggest one yet—SELA Developer Practice, May 5-9, 2013.
This time we opened the call for papers to local and international speakers, and here are the results, in numbers:
Our last conference was record-breaking in terms of attendance, and we definitely hope to break that record again! If you live in Israel, you can take advantage of early bird registration right now, and save 33%.
I will be delivering three sessions at the SDP:
Improving .NET Performance – the time-tested full-day workshop that teaches you how to measure and improve .NET application performance, based on the Pro .NET Performance book.
Windows Azure Mobile Services – a breakout session on one of the coolest Microsoft technologies, which provides a backend for your mobile apps on all four major platforms: Windows 8, iOS, Android, and Windows Phone.
Attacking Web Applications – a brand-new breakout session covering common attacks on web applications, including XSS, SQLi, CSRF, insecure credential storage, and many others.
In closing, I encourage you to check out the list of sessions and speakers and see for yourself: the SDP is the biggest and best developer conference in Israel.
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
I totally forgot to blog about it, but my unofficial SDK has had authentication support for a few days now. This has been pretty easy to implement, actually, thanks to OAuth and the Mobile Services backend.
If you haven’t gotten started with WAMS authentication yet, you really should try it out. It’s easy as pie, really, and you can set yourself up with 3-4 authentication providers in a matter of several minutes. I’ll leave the rest to this great tutorial on authentication with WAMS.
So, if you’ve got the latest version from GitHub, you can now do this to authenticate:
mobileService.login(
MobileServiceAuthenticationProvider.TWITTER,
new MobileServiceLoginCallback() {
public void errorOccurred(MobileException exception) {
//Invoked if an error occurred
}
public void completedSuccessfully(MobileUser user) {
//Invoked on success -- user.getUserId()
//provides the user id
}
public void cancelled() {
//Invoked if the user cancelled the process
}
});
As a bonus, the client-side SDK will persist the authentication token returned by WAMS after the login flow completes. (If you care about the details, the token is persisted in a preferences file, and is not encrypted; this is something that needs to be taken care of in the future.) On subsequent runs of your app, you won’t have to display the login flow again to the user. Use the MobileService.isLoggedIn method to determine whether the user is currently logged in, and the MobileService.logout method to log out.
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
I'm very happy to announce that the call for papers is now open for the next SELA Developers Practice! Our last conference has been a huge success, with over 1350 registered attendees and a number of workshops that we had to duplicate due to high demand. We want our next conference to be even bigger -- and you can be a part of it!
We are looking for breakout sessions (60-75 minutes) as well as full-day workshops covering any topic related to the Microsoft stack -- from Windows 8 to Windows Phone and Windows Azure, WCF and Entity Framework, ASP.NET and HTML5. The conference will span a whole week starting on May 5, 2013, and will feature two core conference days and three pre- and post-conf workshop days.
To submit your talk proposals, please visit the CFP website. For international speakers, we will be able to cover travel expenses and hotel accommodation, provided that you deliver at least one workshop and one breakout session at the conference.
Looking forward to seeing you at the next SDP!
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
Scheduler scripts are a very recent addition to
Windows Azure Mobile Services. This is a very nice feature that supports periodic background processing in your mobile service. As a reminder, in the original release of Windows Azure Mobile Services, the only opportunity to run any server-side work (in scripts) is when the client performs an operation on a table (select, insert, update, or delete). For any kind of periodic processing, you'd have to use an external solution or have a client ping one of your tables -- a very ugly solution, as opposed to a scheduled script.
When would you use scheduled scripts?
- To perform background processing on data uploaded by your users, or data that you retrieve from some external web services. For example, if you allow free text uploads, you might want to index the text for faster searching, or mine the text to detect interesting patterns.
- To prune unnecessary data from your tables, such as dead push channels (channels that represent devices that are no longer available, or devices where you app is no longer installed).
- To send push notifications at a certain time of day, or when an interesting event occurs at some point in the future. This is helpful when you have users all over the world, and you don't want to send a push notification to European users at 8PM Pacific time...
But another very useful thing you can do with scheduled scripts is test your server-side code. As you write your server-side scripts, the only way to test them previously was to hit one of your tables with a CRUD operation. With scheduled scripts, you can run your script by clicking a button in the Windows Azure Management Portal.
Image borrowed from Scott Guthrie's blog.
When I was working on push notification support in my unofficial Android SDK for Windows Azure Mobile Services, it was very helpful to have a script that sends push notifications to all registered devices, and invoke it whenever I want without hitting a table with a CRUD operation:
function TestPush() {
var reqModule = require('request');
var channelTable = tables.getTable('pushChannel');
channelTable.read({
success: function(channels) {
channels.forEach(function(channel) {
reqModule({
url: 'https://android.googleapis.com/gcm/send',
method: 'POST',
headers: {
'Authorization': 'key=MY_AUTHORIZATION_KEY'
},
json: {
registration_ids: [channel.regId],
data: {
"__builtInType": "notification",
"contentTitle": "Test title",
"contentText": "Test content",
"tickerText": "Test ticker",
"number": "42"
}
}
}, function (err, resp, body) {
if (err || resp.statusCode !== 200) {
console.error('Error sending GCM push: ', err);
}
});
});
}
});
}
With the most recent command-line tool release to manage your Windows Azure Mobile Services, you can list, download, and upload your scheduler scripts from the command-line. Unfortunately, there's no way at this point to invoke the scheduler script from the command-line tool, but you can easily test any script from the portal, as we've seen above.
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
Windows Azure Mobile Services (WAMS) is probably the most exciting Microsoft technology I have encountered in the last 5 years at least. This is a cross-platform framework that serves as a backend for your mobile applications, taking care of concerns such as data access and storage, user authentication, and push notifications. WAMS is the logical next step after PaaS offerings: it is a complete backend-as-a-service, which takes care of most (if not all) your server-side needs.
For a general introduction to Windows Azure Mobile Services, I recommend that you take a look at the Windows Azure Developer Center, or -- a shameless plug -- my Windows Azure Conf talk on WAMS.
At the time of writing, WAMS has full support for Windows 8, Windows Phone 8, and iOS. On these platforms, the WAMS client library provides classes and methods for accessing data tables on the server, for authenticating the user with Microsoft Account, Facebook, Twitter, or Google, and for sending push notifications from the cloud to all connected devices.
There is an obvious gap here: Android. Until such time that the WAMS client library is officially ported to Android, I decided to implement my own unofficial SDK that currently handles data access and push notifications in a fairly seamless fashion. In this post, I wanted to give you a general idea of what it means to develop an Android application that uses WAMS and my unofficial library.
First things first: you have to get the library code from GitHub. What you get is an Eclipse library project that you can add to your workspace and add a reference to it from your Android application.
Before you start writing client-side code to access WAMS, you need to create a mobile service and create a couple of tables. The easiest way to do so today is by using the Azure cross-platform command-line tool, azure, which you can install using 'npm install azure' and 'npm install azure-cli' if you have Node on your development machine.
Once you have configured the azure command-line tool, go ahead and create your first mobile service with a couple of tables:
$ azure mobile create
info: Executing command mobile create
Mobile service name: myfirstmobileservice
SQL administrator user name: user
SQL administrator password: **********
Confirm password: **********
...
$ azure mobile table create myfirstmobileservice message
info: Executing command mobile table create
+ Creating table
info: mobile table create command OK
$ azure mobile table create myfirstmobileservice pushChannel
info: Executing command mobile table create
+ Creating table
info: mobile table create command OK
After creating the service, run the following command to obtain the service URL and application key. You will need these later:
$ azure mobile show myfirstmobileservice
info: Executing command mobile show
+ Getting information
info: Mobile application
data: status Healthy
data: Mobile service name myfirstmobileservice
data: Mobile service status ProvisionConfigured
data: SQL database name myfirstmobileservice_db
data: SQL database status Linked
data: SQL server name YOUR_SERVER_NAME
data: SQL server status Linked
info: Mobile service
data: name myfirstmobileservice
data: state Ready
data: applicationUrl https://myfirstmobileservice.azure-mobile.net/
data: applicationKey YOUR_APPLICATION_KEY
data: masterKey YOUR_MASTER_KEY
data: webspace EASTUSWEBSPACE
data: region East US
data: tables message,pushChannel
info: mobile show command OK
Now you're ready to start working with data and push notifications in your Android application. Suppose that you want to insert new messages into the message table created previously. To do that, you define a class that represents a message, and use annotations from the WAMS client-side library:
@DataTable("message")
public class Message {
@Key public int id;
@DataMember public String from;
@DataMember public String text;
public Message(String from, String text) {
this.from = from;
this.text = text;
}
}
Now, you can connect to the message table and insert a new message, as follows:
MobileService mobileService = new MobileService("http://myfirstmobileservice.azure-mobile.net", "YOUR_APPLICATION_KEY");
MobileTable<Message> messageTable = mobileService.getTable(Message.class);
Message message = new Message("Sasha", "Hello!");
messageTable.insert(message);
There is also support for asynchronous operations, but I'll leave that to you to explore. To verify that the message was inserted, check your table contents with the command-line tool:
$ azure mobile data read myfirstmobileservice message
info: Executing command mobile data read
...
info: mobile data read command OK
To query data from your mobile tables, you can use the query API. The queries you specify are delivered all the way to the Windows Azure SQL Database instance in the cloud, and you receive only the filtered results. For example, here's how you can retrieve the latest 30 messages from the user "Sasha" (additional pagination support is available with the skip method):
List<Message> messages = messageTable.where().equal("user", "Sasha").top(30).select();
Finally, let's move on to the feature I most recently implemented -- push notification support. Android push has become much easier with the introduction of the Google Cloud Messaging (GCM) library. My WAMS client-side wrapper makes it a breeze to interact with GCM. First, you should obtain a sender ID from the GCM console. Once you have it, add the following string resources to your Android app:
<string name="mobileServiceUrl">https://YOURSERVICENAME.azure-mobile.net</string>
<string name="mobileServiceApiKey">YOUR_MOBILE_SERVICE_API_KEY</string>
<string name="mobileServicePushSenderId">YOUR_GCM_SENDER_ID</string>
Now, to register for push on the client, all you need to do is:
mobileService.registerPush();
The library also offers support for a transient push callback that will be invoked when a push notification is received, as well as an intent service you can extend to process push notifications even if your app is not running. For more information on these, take a look at the walkthrough on GitHub.
On the server, to send a push notification to all registered clients that will be displayed in the Android notification area, add a server script resembling the following:
var reqModule = require('request');
var channelTable = tables.getTable('pushChannel');
channelTable.read({
success: function(channels) {
channels.forEach(function(channel) {
reqModule({
url: 'https://android.googleapis.com/gcm/send',
method: 'POST',
headers: {
//TODO: this should be your push API key
'Authorization': 'key=PUSH_API_KEY'
},
json: {
//You could pipe up to 1,000 registration ids here
registration_ids: [channel.regId],
data: {
"__builtInType": "notification",
"contentTitle": "Your notification's title",
"contentText": "Your notification's text",
"tickerText": "Your notification's ticker text",
}
}
}, function (err, resp, body) {
if (err || resp.statusCode !== 200) {
console.error('Error sending GCM push: ', err);
} else {
console.log('Sent GCM push notification successfully to ' + channel.regId);
}
});
});
}
});
Alternatively, if you want to test push notifications from a local app, you can use the following curl command line (without any WAMS intervention):
curl -X POST -H 'Authorization: key=YOUR_PUSH_API_KEY_HERE' -H 'Content-Type: application/json' https://android.googleapis.com/gcm/send --data '{ "registration_ids": ["REG_ID_HERE"], "data": { "__builtInType": "notification", "contentTitle": "This is the title", "contentText": "This is the content text", "tickerText": "This is the ticker text" } }'
This walkthrough does not describe every possible feature the client library currently supports, but it's a good start. For more information, take a look at the docs on GitHub, and stay tuned: I plan to add more features -- such as authentication -- at some point in the future. (Unless the WAMS team beats me to it :-))
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
Windows AzureConf was last month, and it was a blast! This was a virtual conference streamed online on Channel 9 and focused exclusively on Windows Azure, with
Scott Guthrie delivering the keynote and the rest of the talks given by Azure community members -- Azure Insiders and Azure MVPs.
My first talk was about Windows Azure Mobile Services, which is the piece of Microsoft technology I am most excited about in the last 5 years at least. If you haven't seen this technology, Windows Azure Mobile Services provides a backend for your mobile apps on a variety of platforms, currently with native support for Windows 8, Windows Phone 8, and iOS -- and Android support coming later. What does a "mobile backend" entail? Data access through a familiar table model (supported by SQL Azure), push notifications using the appropriate notification platform (WNS, MPNS, APNS), and authentication support for Microsoft Account, Facebook, Twitter, and Google.
To learn more about Windows Azure Mobile Services, tune in to my talk below, or explore the Windows Azure Developer Center.
The presentation, source code, and other auxiliary materials from my talk are available online.
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
The SDP started last Sunday in the shadow of rockets fired on Israel and an alarm and evacuation around 10:30AM. Nonetheless, attendance was almost full and all sessions were delivered as planned. We had three tracks with a total of 18 talks, and mine was concerned with running Node apps on Windows Azure.
This was a similar -- although not identical -- talk to what I did at the Israeli Windows Azure User Group last month. You can find the slides and demos here, but you will have to substitute your account names and keys if you want to run the demos on your machine or upload them to Windows Azure. The talk was recorded, as were all other talks; if you attended the conference you will get an email with details on how to access the recorded talks from all three tracks.
Here's a nice screenshot of some of the messages attendees wrote in my interactive bulletin board demo:
Some of the things we covered:
- Introduction to Node.js, HTTP client, express, nstore
- Windows Azure Portal and the command line 'azure' tool
- Uploading a simple Node app to Windows Azure Web Sites using git integration
- Accessing Azure Table Storage from a Node app
- Using a SQL Azure database from an on-premises Node app
- Connecting a Windows Azure Mobile Service and Windows Store app to the same SQL Azure database
- Using Node with MongoDB in an Ubuntu VM running on Windows Azure
- Using WebMatrix to download and modify a web site from Windows Azure
Thanks for attending, and stay tuned for more updates about next year's SDP. You can save the date already: it will be May 5-9, 2013!
I am posting short links and updates on Twitter as well as on this blog. You can follow me: @goldshtn
The last two talks I went to at BUILD… Spending all my time preparing for the Windows Azure Conference next week, where I have two sessions: one on Node.js and one on Windows Azure Mobile Services.
Herb Sutter – The Future of C++
Herb has been giving this talk for a few years in a row now at major conferences, and there’s something new every time – this really tells you a lot about the excitement in the world of C++ during these years, culminating with last year’s standard, C++11.
Herb shared the plans for an out-of-band Visual Studio update to the C++ compiler, of which a CTP is now available. This update brings the compiler up to par with almost every important C++11 feature, including:
- Variadic templates – definitely my favorite, and the subject of a talk I gave at the SELA C++ Conference in June
Shameless plug – my talk from June :-)
vector<int> v = { 2, 3, 4 };
- Explicit conversion operators
- Raw string literals
- Default template parameters for function templates
template <typename T, typename U = int>
T* make_array(U size) …
Herb also talked a lot about the C++ ISO working group (WG21), which has now been split into numerous study groups (SGs) focusing on exciting subjects including reflection, transactional memory, filesystem APIs, networking, and more.
By Herb’s estimates, we should see a minor standard as early as 2014, with generic lambdas, runtime sized arrays, static-if, thread safe containers, and more… Next, in 2017 we should expect the next large revision to the standard, with primarily standard library additions and changes emanating from the study group work over the next couple of years.
At the end of talk, Herb announced the isocpp.org website, which is designed to be a hub for C++ developers online, and the Standard C++ Foundation, a non-profit organization for improvement of C++ understanding and use.
When all’s said and done, I suggest you keep honing your C++ skills (I have), not for lack of other languages but for the sheer power and elegance of some C++ features, for performance, for easier use of intrinsics and other hardware quirks, and – needless to say – for portability.
Scott Hanselman and Jon Galloway – Bleeding Edge ASP.NET
Frankly, I was pretty exhausted at this point in the day, and coming up with a flu that chased me all the way back to Israel. Jon and Scott pulled a great talk, focusing on the ASP.NET Fall 2012 Update as well as some even newer features. Most of the talk was a huge demo (or set of demos), focusing on:
- Facebook application template for Visual Studio (not just authentication support) + Facebook C# SDK NuGet package
- SPA template
- Mobile application template
- Scaffolding prototype which generates a master page, mobile master page, EF data context, and whatnot
- Web Essentials prototype extension for TypeScript with source maps that allow debugging of TypeScript code, displaying the generated JavaScript code in a tooltip
This funny talk was a great way to round up the conference. Looking forward to next year’s BUILD!
I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
I had barely time to blink, and BUILD is almost behind us. The fourth day was shorter than the rest, but there was one session that made my morning and my day:
Subramanian Ramaswamy – Deep Dive into the Kernel of the .NET Framework on Windows Phone 8
This long title was well-justified, as the .NET Framework on Windows Phone has been revamped as dramatically as the underlying Windows CE –> Windows NT kernel drop-in. This talk was divided into two parts – one about the productivity enhancements for .NET developers on Windows Phone, including easier interoperabiliy with unmanaged code through WinRT components, easier code sharing between platforms with Portable Code Libraries, and async support from C# 5.0. But that’s not what I came for :-)
The second part of the talk focused on CLR internals, and how they were used to improve the startup and execution performance of managed Windows Phone apps. The .NET Framework on Windows Phone 8 is built on top of the CoreCLR, which is the same CLR powering Silverlight applications. Along with CoreCLR, the team has made performance improvements including “compiling in the cloud” to an intermediate device-specific language called MDIL, better optimizations in the native image generation, and introduction of the CoreCLR garbage collector, which is generational and has multicore allocation support.
The “compile in the cloud” initiative involves taking the IL code generated by the C# compiler on the developer’s machine and compiling it with an aggressive optimizing compiler to a device-specific language – MDIL. MDIL files consist almost entirely of ARM assembly instructions, with the exception of specific placeholders that are compiled to ARM assembly only when the app is installed.
To understand the rationale behind MDIL, consider the two current alternatives: JIT and NGEN. Windows Phone 7.5 apps use JIT, which means IL code must be compiled to ARM instructions during startup and runtime. The set of optimizations performed by the JIT is also very limited because of time and resource constraints. Desktop and server apps can use NGEN, which compiles IL code to native assembly instructions (typically during application installation), which improves startup but requires full recompilation when updates to the .NET Framework are applied. In other words, if Windows Phone 8 apps used NGEN, the first compilation could have occurred entirely in the cloud, but any subsequent .NET Framework updates would require a costly recompilation process to be performed on the user’s device. Considering the constrained hardware and power available, recompiling all apps could take hours and noticeably hinder the user experience.
With that said, how does MDIL address this problem? MDIL files contain placeholders, compiled to native instructions only at installation time and recompiled again whenever framework updates are installed. Such placeholders are required whenever your app’s code accesses types from other assemblies (such as the .NET Framework assemblies). For example, when invoking a method, accessing a field, or any other operation that is dependent on object layout, instead of generating the actual ARM assembly instructions, the IL –> MDIL compiler generates a placeholder, logically resembling the following:
; load field ‘n’ from object ref r0
ldr r3, [r0 + “fieldoffset(n)”]
; call method ‘foo’ of object ref r0
add r1, r0, “methodoffset(foo”)
blx r1
Because the placeholders are the exception and not the rule, this compilation process is not as costly and will provide a great user experience. The only thing that needs to be updated when the app is installed or when framework updates are applied are the external placeholders.
Towards the end of the talk, Mani mentioned the CoreCLR garbage collector, which is a weakened version of the workstation GC on desktop CLR. Namely, the collector is generational and allows multiple allocations to proceed at once from multiple cores (I assume with a form of multiple heaps – this needs further investigation). Additionally, the object tracking for roots is less conservative than in the Compact Framework GC – for example, in the following method, the XML document will be collected after the GC.Collect call:
var doc = new XMLDocument();
doc.Load(…);
Console.WriteLine(doc.OuterXml);
GC.Collect();
Console.ReadLine();
All in all, the CoreCLR on Windows Phone offers an exciting perspective on inventiveness and creativity when optimizing for ARM and for less powerful devices.
Clearly, there are some followup questions still waiting for answers. How do you inspect MDIL to determine what the optimizer is doing? Is MDIL coming to the desktop and server as well? Is there still room for JIT optimizations at runtime, such as hot-patching interface dispatch stubs? I hope to find the time to explore this further in the coming weeks :-)
I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
No keynote today, but Scott Hanselman’s 8:30AM talk was a great replacement – he himself called it an “unkeynote”. Also, before the talk Scott looped some hilarious videos, such as MacBook Wheel from The Onion.
Scott Hanselman – One ASP.NET and the Cloud
This was really keynote material in that Scott highlighted some of the recent developments in what the cloud means for developers. He started by showing the Azure PowerShell cmdlets, which give you provisioning of websites or virtual machines by virtue of a few keystrokes. These are the “five computers” somewhere in the world – Azure, Amazon, and other cloud providers – which provide infinite compute capacity at your fingertips.
The next point Scott made is that the browser itself is a virtual machine – running JavaScript – which provides a plugin-less experience for very complex software: Commodore 64 and x86 emulators being a notable example of how the browser becomes a virtual machine.
Screenshot of the JSLinux effort, which runs a Linux system on top of an x86 emulator implemented entirely in JavaScript.
Instead of treating browsers like dumb terminals that wait for HTML to pour down from a powerful server, we can leverage the power of the user’s multiprocessor device with hardware-accelerated graphics if we run code inside the browser’s virtual machine. Some pretty cool demos with the D3 visualization and graphics library really drove the point home.
Scott couldn’t not mention the VanillaJS effort, which you should definitely check out. This is a 0 bytes download, giving you exceptional performance and a slew of great features ;-)
Some notable quotes from the talk:
“Furiously Googling with Bing”
“JavaScript is the assembly language of the web”
“JavaScript is an operating system”
Anders Hejlsberg – TypeScript
This talk was very educational although I already had a chance to fool around with TypeScript a little bit. The big problem TypeScript is designed to address is that of developing large-scale JavaScript applications. With the lack of modules, classes, clean inheritance and strong static typing, JavaScript makes large-scale development exceptionally hard, and tools can’t help with auto-completion, refactoring, compile-time errors and other goodies.
Unlike other efforts (such as Google’s Dart), TypeScript starts from pure JavaScript and adds support for arrow functions (lambdas), static typing, interfaces, classes, and modules. The whole thing is compiled down to readable JavaScript code that you can debug and inspect using the standard developer tools in any browser.
TypeScript source:
class Player {
constructor(public name: string, public id: number = 0) {
}
play(game: string) {
document.write(this.name + " is playing " + game);
}
}
var p = new Player("Sasha");
p.play("Diablo");
JavaScript produced by the TypeScript compiler:
var Player = (function () {
function Player(name, id) {
if (typeof id === "undefined") { id = 0; }
this.name = name;
this.id = id;
}
Player.prototype.play = function (game) {
document.write(this.name + " is playing " + game);
};
return Player;
})();
var p = new Player("Sasha");
p.play("Diablo");
With this static typing comes the power of the tools – in-browser playground or Visual Studio, you get auto-completion, error squigglies at compile-time, and refactoring support letting you keep your sanity when working on large JavaScript code bases (to quote: “at some point, JavaScript turns into write-only code”). Anders drove this point home by demonstrating a refactoring of the TypeScript compiler, which is itself written in TypeScript!
Anders then showed a bunch of TypeScript features, including classes, modules, interfaces, type inference, optional properties, accessor functions, static methods, default parameter values, inheritance, and many others. This is really not the same JavaScript you learned to hate!
import connect = module('connect');
import express = module('express');
var app = express.createServer();
app.get('/echo/:message', (req, res) => {
res.end('Your message: ' + req.params.message);
});
var port = process.env.port || 8080;
app.listen(port);
Using modules and arrow functions in Node Finally, you don’t have to write TypeScript declarations for standard libraries. The TypeScript installation ships with type declarations for libraries like jQuery and jQuery UI, whereas WinRT type declarations are automatically generated from WinMD files.
If you haven’t tried TypeScript yet, there’s nothing to download to get started: head on to www.typescriptlang.org and start fooling around in the online playground.
Nathan Totten – JavaScript from Client to Cloud
This was a Node talk, and I didn’t expect to learn anything new but mostly to see how Microsoft perceives Node and its accompanying Azure support. Nathan provided a quick overview of Node, the single-threaded event loop model, and discussed some scenarios in which to use Node, such as lightweight server-side applications and web services. Perhaps more importantly, Nathan discussed when to not use Node – for CRUD solutions over data (Rails or ASP.NET are better), and for CPU-bound processing (because of the single-threaded model). To conclude the overview part of the talk, Nathan mentioned iisnode, which is what the Azure Web Sites infrastructure uses to host Node.js apps on Azure. iisnode can scale node.exe processors to multicore machines, and now has also WebSockets support with Windows Server 2012.
Then, Nathan turned to the demo. In the demo, he started with a local website that uses socket.io for WebSockets, mentioned some developer tools like node-supervisor and node-inspector, and then uploaded the Node app to Azure and developed a simple Windows 8 JavaScript app connected to the same server. At the very end of the talk, he even managed to squeeze in a quick demo of the azure Node module, which he used to access Azure Storage from the on-premises Node app.
All in all, this has been a nice overview of Node and Windows Azure in just sixty minutes. I will definitely be applying some lessons from this talk to my own presentations ;-)
I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
Satya Nadella and others – Day 2 Keynote
Day two started with Satya Nadella orchestrating an amazing keynote with Scott Guthrie, Scott Hanselman, Josh Twist, Jason Zander, and David Campbell. This amazing lineup of speakers showed multiple demos, including live coding, covering the most recent developments – mostly around the Windows Azure Platform.
Satya’s point of view on the current state of Windows Azure is that it is a complementary offering to Windows 8 and Windows Phone 8, in that it provides a great backend for mobile apps. This great backend is not just words – it’s Windows Azure Mobile Services, announced just a few weeks ago and progressing at an incredible rate. (I have taken a great interest in WAMS, and even managed to squeeze it into my Node.js talk at the local Azure UG last week.)
For example, here’s what you need to do to authenticate a user using Facebook – yes, seriously:
var user = await MobileService.LoginAsync(MobileServiceAuthenticationProvider.Facebook);
Josh Twist from the WAMS team announced support for Windows Phone 8 – alongside the existing support for Windows 8 and iOS. I bet Android support is coming as well, which will make WAMS an ideal backend offering for your mobile apps on every platform. In his demo, Josh showed how easy it is to integrate data tables in your mobile app, to send push notifications from server-side scripts between devices, and how to perform user authentication using identity providers such as Twitter and Facebook.
Alongside with that great backend, Windows Azure powers more than 200 Microsoft services, including Bing, Office 365, and Xbox LIVE. Scott Hanselman hopped to the stage to build an ASP.NET web site with third party identity providers and a Web API service, and continued to use it from a Windows 8 app.
At this point Satya said that Windows Azure “takes the grudge out of building cloud-scale services”. Indeed, Scott Guthrie demonstrated how Windows Azure Media Services – which you can try online without downloading the SDK – streamline the process of uploading, encoding, publishing, and streaming media to a huge variety of devices. Scott also demoed SignalR and announced the Windows Azure Store with third-party add-ons, including New Relic’s free monitoring and analytics offer for all existing Windows Azure customers.
Towards the end of the keynote, Jason Zander announced that the Team Foundation Service is coming out of preview and is available for free to teams of up to five people. This is great news for small teams looking to bootstrap their source control and continuous delivery environment.
Finally, Dave Campbell announced HDInsight (Hadoop on Windows Azure) and showed the JavaScript library and .NET SDK for MapReduce. As someone who had some experience with the Java SDK for Hadoop, I can tell you that the .NET SDK looks stellar in comparison.
Don McBrady, Jim Radigan – It’s All About Performance: Using Visual C++ 2012 to Make the Best Use of Your Hardware
This was one long title, but a very interesting and informative session for C++ developers. The room was almost full; Jim and Don quickly went through the concepts of superscalar execution and vectorization in modern processors, and explained how the C++ compiler in Visual Studio 2012 determines which loops are safe to vectorize and emits parallelized and vectorized loops when possible.
This is the kind of hints you can give the compiler to automatically vectorize and parallelize your loops – and there’s a math library optimized for vector operations at your disposal:
#pragma loop(hint_parallel(4))
for (int i = 0; i < _countof(arr); ++i)
{
double s = sin(arr[i]);
double c = cos(arr[i]);
arr[i] = sqrt(s*s + c*c);
}
Many more parallelization and vectorization opportunities are available, and Visual Studio 2012 is just a first glimpse at what’s possible. Because the same compiler is used to build Windows, Office, and SQL Server, the compiler team has to be very conservative and cautious with optimization opportunities – so some things simply didn’t make it into the first release. I will definitely watch the space of further optimization improvements in the Microsoft compiler.
I covered some of these topics in the past on my blog [SIMD, auto-parallelization], and some tips on vectorization, instruction-level parallelism, and even C++ AMP have made it to Chapters 6 and 10 of Pro .NET Performance. If you’re more of a .NET guy, it might be easier to digest than this C++-oriented talk :-)
Towards the end of the talk, Don demonstrated C++ AMP with a live “cartoonizer” demo, in which the performance was improved by two orders of magnitude as a result of offloading CPU-bound image-processing code to the GPU. This and other impressive demos are something you can do today: just pick up C++ AMP by Kate Gregory and Ade Miller, and start getting your feet wet with C++ AMP.
Josh Twist – Azure Mobile Services
Josh, a Senior Program Manager on the Mobile Services team, introduced the capabilities of Mobile Services using a long demo in which he built a voting app for categorized events. In just under 50 minutes, he was able to cram the service creation, data tables, authentication using multiple providers, push notifications for both Windows 8 and Windows Phone 8, and most importantly: server-side code with insert and read scripts.
Brandon Bray – Evolution of .NET
In this historical perspective talk, Brandon recalled the design goal of the .NET framework – having developers fall into the pit of success. Indeed, .NET has increased its platform reach (with the embarrassing exception of Silverlight’s untimely demise), and maintained a consistent API surface that you can use across Xbox, Windows Phone, Windows 8, and desktop apps.
Brandon showed some funny videos and PDC slides from the last decade, including Bill Gates elaborating on how every application will become a website and how it will become increasingly important to sync and transition experiences between multiple devices. It’s interesting to see how these trends are even more relevant today, as the line between apps and websites is becoming even more blurry.
At the end of the talk, Brandon discussed some of the performance improvements in the recent CLR versions, including background server GC, multicore JIT, and automatic NGEN. He also briefly mentioned the CoreCLR’s resurrection in Windows Phone 8, where it powers the next generation of Windows Phone applications. Compile in the Cloud, the subject of a talk I plan to attend on Friday, brings machine-dependent IL to the phone, which is much faster to JIT and has a much smaller footprint, resulting in better performance due to better optimization and faster startup times.
Finally, Brandon mentioned the improvements in Compact Framework 3.9, coming to Windows Embedded Compact 2013 that targets less powerful devices, with 256MB of RAM or less – generational GC, code sharing server (like in WP7), and improved code quality for ARM processors.
I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
After nerve-wrecking last minute flight changes, I made it to BUILD on Monday and heroically spent almost 2 hours in the registration queue. After a good night’s sleep, we regrouped for the first day’s keynote. Another long line, and we were sitting in the third row!
Ishai Ram, Alon Levi, Noam Kfir – 3rd row of the keynote hall.
Steve Ballmer and others – Day 1 Keynote
Surprisingly, Steve Ballmer himself kicked off the keynote with an incredible amount of energy. Although he didn’t jump up and down screaming “Developers, Developers, Developers”, there was a great sense of accomplishment in his talk, showing off the great Windows 8 devices already released and many more that are yet to come, including an “82 inch slate” by PixelSense.
Steve Ballmer with an 82-inch slate.
Notably, the tablets Steve showed included the Surface RT, Microsoft’s flagship offering with Windows RT. The Surface is shaping up to be a great tablet, with the innovative Touch and Type covers. In fact, since every BUILD attendee got a Surface RT, I can attest myself for the keyboard quality on the Touch Cover. After a couple of minutes getting used to it, I was able to type pretty quickly with fairly little typos, without looking at the keyboard.
BUILD giveaways: Surface RT with 32GB and Touch Cover, Nokia Lumia 920, and 100GB of SkyDrive storage.
Steve shared some statistics on Windows 8 upgrades: four million upgrades over the weekend since the October 26 release. Some other notable insights included the focus on sharing code between Windows 8 and Windows Phone 8, syncing everything through SkyDrive, Xbox Music free streaming and Xbox Smart Glass.
Towards the end of the keynote, Kevin Gallo talked about the Windows Phone 8 SDK, focusing on tap-to-send, and concluded by telling us that we all get a Nokia Lumia 920 with “maps that work” :-)
“Maps that work”, screenshot from Lumia 920.
Shai Hinitz – Designing Games for Windows 8
This talk had a lot of potential, but unfortunately there were considerable problems with the demos exacerbated by a 20 minute projector blackout, that could only be restored by rebooting the projector… Shai showed several examples of Windows 8 games, and outlined some guidance for great Windows games, including creating a great splash screen, supporting multiple screen orientations and Snap view, and using the richness of the Windows sensor platform, all the more relevant to interactive games.
Will Tschumy – The Microsoft Design Language
This was an excellent session by a great speaker with a strong design background. Will explained in detail the level of commitment at Microsoft – all across the board – to design on every screen, including Windows 8, Windows Phone 8, and Xbox.
The core principles of the Microsoft design guidelines, then, are the following:
- Get the OS out of the way. Content before chrome, in every way.
- Screen edges are important when there is no chrome. This is why they were chosen for charms and app bars.
- Craftsmanship and quality are of paramount importance in apps. All apps are fast, responsive, grid-aligned, familiar, offering a consistent user experience.
- Focus is important. An app should do one thing, and do it great.
- Contracts with other apps and with the OS bring productivity and user confidence to the next level. A small set of charms addresses a huge variety of user needs, in a consistent way across the system.
Will had a great number of examples and explained these principles very vividly. Even if you’ve already written a Windows 8 app (or ten), I still recommend that you watch the session recording.
I am posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
More Posts
« Previous page -
Next page »