The Refined Geek

I heap a lot of praise on Windows Azure here, enough for me to start thinking about how that’s making me sound like a Microsoft shill, but honestly I think it’s well deserved. As someone who’s spent the better part of a decade setting up infrastructure for applications to run on and then began developing said applications in its spare time I really do appreciate not having to maintain another set of infrastructure. Couple that with the fact that I’m a full Microsoft stack kind of guy it’s really hard to beat the tight integration between all of the products in the cloud stack, from the development tools to the back end infrastructure. So like many of my weekends recently I spent the previous coding away on the Azure platform and it was filled with some interesting highs and rather devastating lows.

I’ll start off with the good as it was really the highlight of my development weekend. I had promised to work on a site for a long time friend’s upcoming wedding and whilst I had figured out the majority of it I hadn’t gotten around to cleaning it up for a first shot to show off to him. I spent the majority of my time on the project getting the layout right, wrangling JavaScript/jQuery into behaving properly and spending an inordinate amount of time trying to get the HTML to behave the way I wanted it to. Once I had gotten it into an acceptable state I turned my eyes to deploying it and that’s where Azure Web Sites comes into play.

For the uninitiated Azure Web Sites are essentially a cut down version of the Azure Web Role allowing you to run pretty much full scale web apps for a fraction of the cost. Of course this comes with limitations and unless you’re running on at the Reserved tier you’re essentially sharing a server with a bunch of people (I.E. a common multi-tenant scenario). For this site, which isn’t going to receive a lot of traffic, it’s perfect and I wanted to deploy the first run app onto this platform. Like any good admin I simply dove in head first without reading any documentation on the process and to my surprise I was up and running in a matter of minutes. It was pretty much create web site, download publish profile, click Publish in Visual Studio, import profile and wait for the upload to finish.

Deploying a web site on my own infrastructure would be a lot more complicated as I can’t tell you how many times I’ve had to chase down dependency issues or missing libraries that I have installed on my PC but not on the end server. The publishing profile coupled with the smarts in Visual Studio was able to resolve everything (the deployment console shows the whole process, it was actually quite cool to watch) and have it up and running at my chosen URL in about 10 minutes total. It’s very impressive considering this is still considered preview level technology, although I’m more inclined to classify it as a release candidate.

Other Azure users can probably guess what I’m going to write about next. Yep, the horrific storage problems that Azure had for about 24 hours.

I noticed some issues on Friday afternoon when my current migration (yes that one, it’s still going as I write this) started behaving…weird. The migration is in its last throws and I expected the CPU usage to start ramping down as the multitude of threads finished their work and this lined up with what I was seeing. However I noticed the number of records migrated wasn’t climbing up at the rate it was previously (usually indicative of some error happening that I suppressed in order for the migration to run faster) but the logs showed that it was still going, just at a snail’s pace. Figuring it was just the instance dying I reimaged it and then the errors started flooding in.

Essentially I was disconnected from my NOSQL storage so whilst I could browse my migrated database I couldn’t keep pulling records out. This also had the horrible side effect of not allowing me to deploy anything as it would come back with SSL/TLS connection issues. Googling this led to all sorts of random posts as the error is also shared by the libraries that power the WebClient in .NET so it wasn’t until I stumbled across the ZDNet article that I knew I wasn’t in the wrong. Unfortunately you were really up the proverbial creek without a paddle if your Azure application was based on this as the temporary fixes for this issue, either disabling SSL for storage connections or usurping the certificate handler, left your application rather vulnerable to all sorts of nasty attacks. I’m one of the lucky few who could simply do without until it was fixed but it certainly highlighted the issues that can occur with PAAS architectures.

Honestly though that’s the only issue (that’s not been directly my fault) I’ve had with Azure since I started using it at the end of last year and comparing it to other cloud services it doesn’t fair too badly. It has made me think about what contingency strategy I’ll need to implement should any parts of the Azure infrastructure go away for a extended period of time though. For the moment I don’t think I’ll worry too much as I’m not going to be earning any income from the things I build on it but it will definitely be a consideration as I begin to unleash my products onto the world.

If you’re a developer like me you’ve likely got a set of expectations about the way you handle data. Most likely they all have their roots in the object-oriented/relational paradigm meaning that you’d expect to be able to get some insight into your data by simply running a few queries against it or simply looking at the table, possibly sorting it to find something out. The day you decide to try out something like Azure Table storage however you’ll find that these tools simply aren’t available to you any more due to the nature of the service. It’s at this point where, if you’re like me, you’ll get a little nervous as your data can end up feeling like something of a black box.

A while back I posted about how I was over-thinking the scalability of my Azure application and how I was about to make the move to Azure SQL. That’s been my task for the past 3 weeks or so and what started out as a relatively simple task of simply moving data from one storage mechanism to another has turned into this herculean task that has seen me dive deeper into both Azure Tables and SQL than I have ever done previously. Along the way I’ve found out a few things that, whilst not changing my mind about the migration away from Azure tables, certainly would have made my life a whole bunch easier had I known about them.

1. If you need to query all the records in an Azure table, do it partition by partition.

The not-so-fun thing about Azure Tables is that unless you’re keeping track of your data in your application there’s no real metrics you can dredge up in order to give you some idea of what you’ve actually got. For me this meant that I had one table that I knew the count of (due to some background processing I do using that table) however there are 2 others which I have absolutely 0 idea about how much data is actually contained in there. Estimates using my development database led me to believe there was an order of magnitude more data in there than I thought there was which in turn led me to the conclusion that using .AsTableServiceQuery() to return the whole table was doomed from the start.

However Azure Tables isn’t too bad at returning an entire partition’s worth of data, even if the records number in the 10s or 100s of thousands. Sure the query time goes up linearly depending on how many records you’ve got (as Azure Tables will only return a max of 1000 records at a time) but if they’re all within the same partition you avoid the troublesome table scan which dramatically affects the performance of the query, sometimes to the point of it getting cancelled which isn’t handled by the default RetryPolicy framework. If you need all the data in the entire table you can then do queries on each partition and then dump them all in a list inside your application and then continue to do your query.

2. Optimize your context for querying or updating/inserting records.

Unbeknownst to me the TableServiceContext class has quite a few configuration options available that will allow you to change the way the context behaves. The vast majority of errors I was experiencing came from my background processor which primarily dealt with reading data without making any modifications to the records. If you have applications where this is the case then it’s best to set the Context.MergeOption to MergeOption.NoTracking as this means the context won’t attempt to track the entities.

If you have multiple threads running or queries that return large amounts of records this can lead to a rather large improvement in performance as the context doesn’t have to track any changes to them and the garbage collector can free up these objects even if you use the context for another query. Of course this means that if you do need to make any changes you’ll have to change the context and then attach to the entity in question but you’re probably doing that already. Or at least you should be.

3. Modify your web.config or app.config file to dramatically improve performance and reliability.

For some unknown reason the default number of HTTP connections that a Windows Azure application can make (although I get the feeling this affects all applications making use of the .NET frameworks) is set to 2. Yes just 2. This then manifests itself as all sorts of crazy errors that don’t make a whole bunch of sense like “the underlying connection was closed” when you try to make more than 2 requests at any one time (which includes queries to Azure Tables). The max number of connections you can specify depends on the size of the instance you’re using but Microsoft has a helpful guide on how to set this and other settings in order to make the most out of it.

Additionally some of the guys at Microsoft have collected a bunch of tips for improving the performance of Azure Tables in various circumstances. I’ve cherry picked out the best ones which I’ve confirmed that have worked wonders for me however there’s a fair few more in there that might be of use to you, especially if you’re looking to get every performance edge you can. Many of them are circumstantial and some require you to plan out or storage architecture in advance (so something that can’t be easily retrofitted into an existing app) but since the others have worked I hazard a guess they would to.

I might not be making use of some of these tips now that my application is going to be SQL and TOPAZ but if I can save anyone the trouble I went through trying to sort through all those esoteric errors I can at least say it was worth it. Some of these tips are just good to know regardless of the platform you’re on (like the default HTTP connection limit) and should be incorporated into your application as soon as its feasible. I’ve yet to get all my data into production yet as its still migrating but I get the feeling I might go on another path of discovery with Azure SQL in the not too distant future and I’ll be sure to share my tips for it then.

One of the peeves I had with the official Twitter client on Windows Phone 7, something I didn’t mention in my review of the platform, was that among the other things that its sub-par at (it really is the poor bastard child of its iOS/Android cousins) it couldn’t display images in-line. In order to actually see any image you have to tap the tweet then the thumbnail in order to get a look at it, which usually loads the entire large image which isn’t required on smaller screens. The official apps on other platforms were quite capable of loading appropriate sized images in the feed which was a far better experience, especially considering it worked for pretty much all of the image sharing services.

Everyone knows there’s no love lost between Instagram and I but that doesn’t mean I don’t follow people who use it. As far back as I can remember their integration in the mobile apps has left something to be desired, especially if you want to view the full sized image which usually redirected you to their atrocious web view. Testing it for this post showed that they’ve vastly improved that experience which is great, especially considering I’m still on Windows Phone 7 which was never able to preview Instagram anyway, but it seems that this improvement may have come as part of a bigger play from Instagram trying to claw back their users from Twitter.

Reports are coming in far that Instagram has disabled their Twitter card integration which stops Twitter from being able to display the images directly in the feed like they have been doing since day 1. Whilst I don’t seem to be experiencing the issue that everyone is reporting (as you can see from the devastatingly cute picture above) there are many people complaining about this and Instagram has stated that disabling this integration is part of their larger strategy to provide a better experience through their platform. Part of that was improve the mobile web experience which I mentioned earlier.

It’s an interesting move because for those of us who’ve been following both Twitter and Instagram for a while the similarities are startling. Twitter has been around for some 6 years and it spent the vast majority of that being a company that was extraordinarily open with its platform, encouraging developers far and wide to come in and develop on their platform. Instagram, whilst not being as wide open as Twitter was, did similar things making their product integrate tightly with Twitter’s ecosystem whilst encouraging others to develop on it. Withdrawing from Twitter in favour of their own platform is akin to what Twitter did to potential client app developers, essentially signalling to everyone that it’s our way or the highway.

The cycle is eerily similar, both companies started out as small time players that had a pretty dedicated fan base (although Instagram grew like a weed in comparison to Twitter’s slow ride to the hockey stick) and then after getting big they start withdrawing all the things that made them great. Arguably much of Instagram’s growth  came from its easy integration with Twitter where many of the early adopters already had large followings and without that I don’t believe they would’ve experienced the massive growth they did. Disabling this functionality seems like they’re shooting themselves in the foot with the intention of attempting some form of monetization eventually (that’s the only reason I can think of for trying to drive users back to the native platform) but I said the same thing about Twitter when they pulled that developer stunt, and they seem to be doing fine.

It probably shouldn’t be surprising that this is what happens when start ups hit the big time because at that point they have to start thinking seriously about where they’re going. For giant sites like Instagram that are still yet to turn a profit from the service they provide it’s inevitable that they’d have to start fundamentally changing the way they do business and this is most likely just the first step in wider sweeping changes. I’m still wondering how Facebook is going to turn a profit from this investment as they’re $1 billion in the hole and there’s no signs of them making that back any time soon.

Windows Azure Tables are one of those newfangled NoSQL type databases that excels in storing giant swaths of structured data. For what they are they’re quite good as you can store very large amounts of data in there without having to pay through the nose like you would for a traditional SQL server or an Azure instance of SQL. However that advantage comes at a cost: querying the data on anything but the partition key (think of it as a partition of the data within a table) and the row key (the unique identifier within that partition) results in queries that take quite a while to run, especially when compared to its SQL counter parts. There are ways to get around this however no matter how well you structure your data eventually you’ll run up against this limitation and that’s where things start to get interesting.

By default whenever you do a large query against an Azure Table you’ll only get back 1000 records, even if the query will return more. However if your query did have more results than that you’ll be able to access them via a continuation token that you can add to your original query, telling Azure that you want the records past that point. For those of us coding on the native .NET platform we get the lovely benefit of having all of this handled for us directly by simply adding .AsTableServiceQuery() to the end of our LINQ statements (if that’s what you’re using) which will handle the continuation tokens for us. For most applications this is great as it means you don’t have to fiddle around with the rather annoying way of extracting those tokens out of the response headers.

Of course that leads you down the somewhat lazy path of not thinking about the kinds of queries you’re running against your Tables and this can lead to problems down the line. Since Azure is a shared service there are upper limits on how long queries can run and how much data they can return to you. These limits aren’t exactly set in stone and depending on how busy the particular server you’re querying is or the current network utilization at the time your query could either take an incredibly long time to return or could simply end up getting closed off. Anyone who’s developed for Azure in the past will know that this is pretty common, even for the more robust things like Azure SQL, but there’s one thing that I’ve noticed over the past couple weeks that I haven’t seen mentioned anywhere else.

As the above paragraphs might indicate I have a lot of queries that try and grab big chunks of data from Azure Tables and have, of course, coded in RetryPolicies so they’ll keep at it if they should fail. There’s one thing that all the policies in the world won’t protect you from however and that’s connections that are forcibly closed. I’ve had quite a few of these recently and I noticed that they appear to come in waves, rippling through all my threads causing unhandled exceptions and forcing them to restart themselves. I’ve done my best to optimize the queries since then and the errors have mostly subsided but it appears that should one long running query trigger Azure to force the connection closed all connections from that instance to the same Table storage will also be closed.

Depending on how your application is coded this might not be an issue however for mine, where the worker role has about 8 concurrent threads running at any one time all attempting to access the same Table Storage account, it means one long running query that gets terminated triggers a cascade of failures across the rest of threads. For the most part this was avoided by querying directly on row and partition keys however the larger queries had to be broken up using the continuation tokens and then the results concatenated in memory. This introduces another limit on particular queries (as storing large lists in memory isn’t particularly great) which you’ll have to architect your code around. It’s by no means an unsolvable problem however it was one that has forced me to rethink certain parts of my application which will probably need to be on Azure SQL rather than Azure Tables.

Like any cloud platform Azure is a great service which requires you to understand what its various services are good for and what they’re not. I initially set out to use Azure Tables for everything and have since found that it’s simply not appropriate for that, especially if you need to query on parameters that aren’t the row or partition keys. If you have connections being closed on you inexplicably be sure to check for any potentially long running queries on the same role as this post can attest they could very well be the source of what ales you.

If you were to believe what some games industry big wigs were saying you’d be lead to believe that Windows 8 was the beginning of the rapture for games on the Microsoft platform. At first it was just a couple developers, big ones in their own right (like Notch), but when someone like Gabe Newell chimes in you start to take notice as distributing games on the Windows platform is his bread and butter and he doesn’t say things like this lightly. However as someone who’s grown up on the Microsoft platform, from the old MS-DOS days until today where I’m running Windows 8 full time on my home PC, and has made his career on their products I still can’t help but feel that their concerns are misplaced as they seem to hinge on a fundamental miscalculation about Microsoft’s overall product strategy.

Those concerns are laid out in lengthy detail by Casey Muratori in his latest instalment of Critical Detail: The Next Twenty Years. In there he lays out the future of the Microsoft platform, drawing on the past few decades of Microsoft’s developments and using them to draw conclusions about what the Microsoft ecosystem will look like in 2032. In this world the future of games on Windows seems grim as all the current AAA titles don’t meet the requirements to be present on the Windows Store and the desktop interface is long gone, effectively destroying the games industry on any PC running their operating system.

It’s a grim future and the number of people worried about this coming to fruition seems to increase on a daily basis. However I believe that some of the assumptions made ignore critical facts that render all this doom and glooming moot, mostly because they ignore Microsoft’s larger strategies.

Before I dive into that however let me just acknowledge that yes the Windows Store doesn’t seem like it would be a great place for current games developers. Realistically it’s no different from Google Play or the iOS App Store as many of the requirements are identical. Indeed all of the platforms strive for the same “family friendly” environment that’s bereft of porn (or anything overtly sexual), violence and excessive profanity which does exclude a good number of games from making their debut on the platform. This hasn’t stopped countless numbers of companies from profiting on this platform but there is no denying that the traditional games industry, with its love of all those things these market places abhor, would struggle with these guidelines.

The fundamental misstep that many games developers appear to be making though is thinking that the Windows Store and the guidelines that come along with it will be the only platform available for them to release games onto the Windows operating system. Looking back to previous examples of Windows does show that Microsoft puts an end date on many technologies however I don’t believe that the desktop will be among them. Sure you might not be able to write a DOS game and have it run in Windows 8 but you can take a MFC app built in 1992 and run today (with the biggest challenge there possibly being recompiling it, but the same code will work).

The reason for the Metro (or Modern or whatever they’re calling it now) interface’s existence is not, as many believe, a direct reaction to the success of the iPad/Android devices and Microsoft’s failure to capitalize on it. The Metro interface, which is built upon the WinRT framework, exists primarily to provide a unified platform where applications can provide a unified experience across the three major screens which users interact with. The capabilities provided within that framework are a fairly comprehensive subset of the larger .NET framework but it’s not fully feature complete as the instruction set needed to be cut down in order for it to be usable on ARM based devices. Whilst it still has access to the goodies required to make games (you can get DirectX on it for example) it’s still not the default platform, is just another one which developers can target.

If the WinRT/Metro framework was Microsoft’s preferred direction for all application development then it wouldn’t be the bastard step-child of their main development technologies, it would become the new .NET. Whilst it is going to be the framework for cross platform applications it’s most definitely not going to be the platform for native development on Windows PCs. The argument can be made that Microsoft wants to transition everyone to WinRT as the default platform but I’ve seen no evidence to support that apart from the idea that because the Metro UI is front and centre that means it’s Microsoft’s main focus.

I find that hard to believe as whilst Metro is great on tablets and smart phones it unfortunately struggles in a mouse and keyboard environment as nearly every review of it has mentioned. Microsoft isn’t stupid, they’ve likely heard much of this feedback through other channels and will be integrating it into their future product strategies. To simply say that they’ll go “Nope, we know we’re going in the right direction and completely killing the desktop” is to be ignorant of the fact that Microsoft works extremely closely with their customers, especially the big organisations who have been the most vocal opponents of Metro-first design. They’re also a pretty big player in the games industry, what with that Xbox being so darn popular, so again I fail to see how they wouldn’t take the feedback on board, especially from such a dedicated audience like us PC gamers.

I’d lend some credence to the theory if the desktop environment hadn’t received much love in Windows 8 in lieu of all the work done on Metro but yet again I find myself coming up empty handed. The UI received a massive overhaul so that the styling would be in line with the rest of Microsoft’s products and there have been numerous improvements in functionality and usability. Why Microsoft would invest so heavily in something that will be slated to be removed within a couple generations of Windows releases is beyond me as most of their deprecated technologies receive no updates for decades prior to them being made obsolete.

And the applications, oh don’t get me started about Microsoft’s own applications.

Whilst Metro has some of the basic applications available in it (like Office and….yeah Office) all of Microsoft’s current catalogue received a revamp as desktop applications, not Metro apps. You’d think that if their future direction was going to be all Metro-esque that more of their staple application suites would have received that treatment, but they didn’t. In fact the amount of applications that are available on the desktop vs the ones available on Metro makes it look more like Metro was the afterthought of the desktop and not the other way around.

If Microsoft’s future is going to be all Windows Store and WinRT apps there’s really no evidence showing to show for it and this is the reason why I don’t feel sympathetic to those developers who are bellyaching about it. Sure if you take a really, really narrow view of the Microsoft ecosystem it looks like the end is nigh for the current utopia of game development that is Windows 7 but in doing so you’re ignoring the wealth of information that will prove you otherwise. The Windows Store might not be your distribution platform of choice (and it likely will never be) but don’t think that the traditional methods that you’ve been using are going anywhere because if Microsoft’s overall strategy is anything to go by they aren’t.

I talk a big game when it comes to cloud stuff and for quite a while it was just that: talk. I’ve had a lot of experience in enterprise IT with virtualization and the like, basically all the stuff that powers the cloud solutions we’re so familiar with today, but it wasn’t up until recently that I actually took the plunge and actually started using the cloud for what its good for. There were two factors at work here, the first being that cloud services usually require money to use them and I’m already spending enough on web hosting as it is (this has since been sorted by joining BizSpark) but mostly it was time constraints as learning to code for the cloud properly is no small feat.

My first foray into developing stuff for the cloud, specifically Windows Azure, was back sometime last year when I had an idea for a statistics website based around StarCraft 2 replays. After finding out that there was a library for parsing all the data I wanted (it’s PHP but thanks to Phalanger its only a few small modifications away from being .NET)  I thought it would be cool to see things like how your actions per minute changed over time and other stats that aren’t immediately visible through the various other sites that had similar ambitions. With that all in mind I set out to code myself up a web service and I actually got pretty far with it.

However due to the enormous amount of work required to get the site working the way I wanted to work it ultimately ended up falling flat long before I attempted to deploy it. Still I learnt all the valuable lessons of how to structure my data for cloud storage services, the different uses of worker and web roles and of course the introduction into ASP.NET MVC which is arguably the front end of choice for any new cloud application on the Windows Azure framework. I didn’t touch the cloud for a long time after that until just recently when I made the move to all things Windows 8 which comes hand in hand with Visual Studio 2012.

Visual Studio 2010 was a great IDE in its own right the cloud development experience on it wasn’t particularly great, requiring a fair bit of set up in order to get everything right. Visual Studio 2012 on the other hand is built with cloud development in mind and my most recent application, which I’m going to keep in stealth until it’s a bit more mature, was an absolute dream to build in comparison to my StarCraft stats application. The emulators remain largely the same but the SDK and tools available are far better than their previous incarnations. Best of all deploying the application can’t be much simpler.

In order to deploy my application onto the production fabric all I had to do was follow the bouncing ball after right clicking my solution and hitting “Publish”. I had already set up my Azure subscription (which Visual Studio picked up on and downloaded the profile file for me) but I hadn’t configured a single thing otherwise and the wizard did everything that was required to get my application running in the cloud. After that my storage accounts were available as a drop down option in the configuration settings for each of the cloud roles, no messing around with copying keys into service definition files or anything. After a few initial teething issues with a service that didn’t behave as expected when its table storage was empty I had the application up and running without incident and it’s been trucking along well ever since.

I really can’t overstate just how damn easy it was to go from idea to development to production using the full Microsoft suite. For all my other applications I’ve usually had to spend a good few days after I’ve reached a milestone configuring my production environment the same way as I had development and 90% of the time I won’t remember all the changes I made along the way. With Azure it’s pretty much a simple change to 2 settings files (via dropdowns), publishing and then waiting for the application to go live. Using WebDeploy I can also test code changes without the risk of breaking anything as a simple reboot to the instances will roll the code back to its previous version. It’s as fool proof as you can make it.

Now if Microsoft brought this kind of ease of development to traditional applications we’d start to see some real changes in the way developers build applications in the enterprise. Since the technology backing the Azure emulator is nothing more than a layer on top of SQL and general file storage I can’t envisage that wrapping that up to an enterprise level product would be too difficult and then you’d be able to develop real hybrid applications that were completely agnostic of their underlying platform. I won’t harp on about it again as I’ve done that enough already but suffice to say I really think that it needs to happen.

I’m really looking forward to developing more on the cloud as with the experience being so seamless it really reduces the friction I usually get when making something available to the public. I might be apprenhensive to release my application to the public right now but it’s no longer a case of whether it will work properly or not (I know it will since the emulator is pretty darn close to production) it’s now just a question of how many features I want to put in. I’m not denying that the latter could be a killer in its own right, as it has been in the past, but the less things I have to worry about the better and Windows Azure seems like a pretty good platform for alleviating a lot of my concerns.

I always tell people that on the surface VMware’s products are incredibly simple and easy to use and for the most part that’s true. Anyone who’s installed an operating system can easily get a vSphere server up and running in no time at all and have a couple virtual machines up not long after. Of course with any really easy to use product the surface usability comes from an underlying system that’s incredibly complex. Those daring readers who read my last post on modifying ESXi to grant shell access to non-root users got just a taste of how complicated things can be and as you dive deeper and deeper into VMware’s world the more complicated things become.

I had a rather peculiar issue come up with one of the tools that I had developed. This tool wasn’t anything horribly complicated, all it did was change the IP address of some Windows servers and their ESXi hosts whilst switching the network over from the build VLAN to their proper production one. For the most part the tool worked as advertised and never encountered any errors, on its side at least. However people were noticing something strange about the servers that were being configured using my tool, some were coming up with a “Local Area Network 2″ and “vmxnet3 Ethernet Adapter #2″ as their network connection. This was strange as I wasn’t adding in any new network cards anywhere and it wasn’t happening consistently. Frustrated I dove into my code looking for answers.

After a while I figured the only place that the error could be originating from was when I was changing the server over from the build VLAN to the production one. Here’s the code, which I got from performing the same action in the VIClient proxied through Onyx, that I used to make the change:

            NameValueCollection Filter = new NameValueCollection();
            Filter.Add("name", "^" + ServerName);
            VirtualMachine Guest = (VirtualMachine)Client.FindEntityView(typeof(VirtualMachine), null, Filter, null);
            VirtualMachineConfigInfo Info = Guest.Config;
            VirtualDevice NetworkCard = new VirtualDevice();
            int DeviceKey = 4000;
            foreach (VirtualDevice Device in Info.Hardware.Device)
            {
                String Identifier = Device.ToString();
                if (Identifier == "VMware.Vim.VirtualVmxnet3")
                {
                    DeviceKey = Device.Key;
                    NetworkCard = Device;
                    Console.WriteLine("INFO - Device key for network card found, ID: " + DeviceKey);
                }
            }
            VirtualVmxnet3 Card = (VirtualVmxnet3)NetworkCard;
            VirtualMachineConfigSpec Spec = new VirtualMachineConfigSpec();
            Spec.DeviceChange = new VirtualDeviceConfigSpec[1];
            Spec.DeviceChange[0] = new VirtualDeviceConfigSpec();
            Spec.DeviceChange[0].Operation = VirtualDeviceConfigSpecOperation.edit;
            Spec.DeviceChange[0].Device.Key = DeviceKey;
            Spec.DeviceChange[0].Device.DeviceInfo = new VMware.Vim.Description();
            Spec.DeviceChange[0].Device.DeviceInfo.Label = Card.DeviceInfo.Label;
            Spec.DeviceChange[0].Device.DeviceInfo.Summary = "Build";
            Spec.DeviceChange[0].Device.Backing = new VMware.Vim.VirtualEthernetCardNetworkBackingInfo();
            ((VirtualEthernetCardNetworkBackingInfo)Spec.DeviceChange[0].Device.Backing).DeviceName = "Production";
            ((VirtualEthernetCardNetworkBackingInfo)Spec.DeviceChange[0].Device.Backing).UseAutoDetect = false;
            ((VirtualEthernetCardNetworkBackingInfo)Spec.DeviceChange[0].Device.Backing).InPassthroughMode = false;
            Spec.DeviceChange[0].Device.Connectable = new VMware.Vim.VirtualDeviceConnectInfo();
            Spec.DeviceChange[0].Device.Connectable.StartConnected = Card.Connectable.StartConnected;
            Spec.DeviceChange[0].Device.Connectable.AllowGuestControl = Card.Connectable.AllowGuestControl;
            Spec.DeviceChange[0].Device.Connectable.Connected = Card.Connectable.Connected;
            Spec.DeviceChange[0].Device.Connectable.Status = Card.Connectable.Status;
            Spec.DeviceChange[0].Device.ControllerKey = NetworkCard.ControllerKey;
            Spec.DeviceChange[0].Device.UnitNumber = NetworkCard.UnitNumber;
            ((VirtualVmxnet3)Spec.DeviceChange[0].Device).AddressType = Card.AddressType;
            ((VirtualVmxnet3)Spec.DeviceChange[0].Device).MacAddress = Card.MacAddress;
            ((VirtualVmxnet3)Spec.DeviceChange[0].Device).WakeOnLanEnabled = Card.WakeOnLanEnabled;
            Guest.ReconfigVM_Task(Spec);

My first inclination was that I was getting the DeviceKey wrong which is why you see me iterating through all the devices to try and find it. After running this tool many times over though it seems that my initial idea of just using 4000 would work since they all had that same device key anyway (thanks to all being built in the same way). Now according to the VMware API documentation on this function nearly all of those parameters you see up there are optional and earlier revisions of the code included only enough to change the DeviceName to Production without the API throwing an error at me. Frustrated I added in all the required parameters only to be greeted by the dreaded #2 NIC upon reboot.

It wasn’t going well for me, I can tell you that.

After digging around in the API documentation for hours and fruitlessly searching the forums for someone who had had the same issue as me I went back to tweaking the code to see what I could come up with. I was basically passing all the information that I could back to it but the problem still persisted with certain virtual machines. It then occurred to me that I could in fact pass the network card back as a parameter and then only change the parts I wanted to. Additionally I found out where to get the current ChangeVersion of the VM’s configuration and when both of these combined I was able to change the network VLAN successfully without generating another NIC. The resultant code is below.

            VirtualVmxnet3 Card = (VirtualVmxnet3)NetworkCard;
            VirtualMachineConfigSpec Spec = new VirtualMachineConfigSpec();
            Spec.DeviceChange = new VirtualDeviceConfigSpec[1];
            Spec.ChangeVersion = Guest.Config.ChangeVersion;
            Spec.DeviceChange[0] = new VirtualDeviceConfigSpec();
            Spec.DeviceChange[0].Operation = VirtualDeviceConfigSpecOperation.edit;
            Spec.DeviceChange[0].Device = Card;
            ((VirtualEthernetCardNetworkBackingInfo)Spec.DeviceChange[0].Device.Backing).DeviceName = "Production";
            Guest.ReconfigVM_Task(Spec);

What gets me about this whole thing is that the VMware API says that all the other parameters are optional when its clear that there’s some unexpected behavior when they’re not supplied. Strange thing is if you check the network cards right after making this change they will appear to be fine, its only after reboot (and only on Windows hosts, I haven’t tested Linux) that these issues occur. Whether this is a fault of VMware, Microsoft or somewhere between the keyboard and chair is an exercise I’ll leave up to the reader but it does feel like there’s an issue with the VIM API. I’ll be bringing this up with our Technical Account Manager at our next meeting and I’ll post an update should I find anything out.

I’ve gone on record saying that whilst the cloud won’t kill the IT admin there is a very real (and highly likely) possibility that the skills required to be a general IT administrator will change significantly over the next decade. Realistically this is no different from any other 10 year span in technology as you’d struggle to find many skills that were as relevant today as they were 10 years ago. Still the cloud does represent some fairly unique paradigm shifts and challenges to regular IT admins, some of which will require significant investment in re-skilling in order to stay relevant in a cloud augmented future.

The most important skill that IT admins will need to develop is their skills in programming. Now most IT admins have some level of experience with this already, usually with automation scripts based in VBScript, PowerShell or even (shudder) batch. Whilst these provide some of the necessary foundations for working in a cloud future they’re not the greatest for developing (or customizing) production level programs that will be used on a daily basis. The best option then is to learn some kind of formal programming language, preferably one that has reference libraries for all cloud platforms. My personal bias would be towards C# (and should be yours if your platform is Microsoft) as it’s a great language and you get the world’s best development environment to work in: Visual Studio.

IT admins should also look to gaining a deep understanding of virtualization concepts, principles and implementations as these are what underpins nearly all cloud services today. Failing to understand these concepts means that you won’t be able to take advantage of many of the benefits that a cloud platform can provide as they function very differently to the traditional 3 tier application model.

The best way to explain this is to use Microsoft’s Azure platform as an example. Whilst you can still get the 3 tier paradigm working in the Azure environment (using a Web Role, Worker Role and SQL Azure) this negates the benefits of using things like Azure Table Storage, Blob Storage and Azure Cache. The difference comes down to having to manually scale an application like you would do normally instead of enabling the application to scale itself in response to demand. In essence there’s another level of autonomy you take advantage of, one that makes capacity planning a thing of the past¹.

It’s also worth your time to develop a lot of product knowledge in the area of cloud services. As I mentioned in my previous blog cloud services are extremely good at some things and wildly inappropriate for others. However in my experience most cloud initiatives attempt to be too ambitious, looking to migrate as many services into the cloud as possible whether there are benefits to be had or not. It’s your job then to advise management as to where cloud services will be most appropriate and you can’t do this without a deep knowledge of the products on offer. A good rule of thumb is that cloud services are great at replacing commodity services (email, ERP, CRM etc.) but aren’t so great at replacing custom systems or commodity systems that have had heavy modifications to them. Still it’s worth researching the options out there to ensure you know how the cloud provider’s capabilities match up with your requirements, hopefully prior to attempting to implement them.

This is by no means an exhaustive list and realistically your strategy will have to be custom made to your company and your potential career path. However I do believe that investing in the skills I mentioned above will give you a good footing for transition from just a regular IT admin to a cloud admin. For me I find it exciting as whilst I don’t believe the cloud will overtake anything and everything in the corporate IT environment it will provide us with some amazing new capabilities.

¹Well technically it just moves the problem from you to the cloud service provider. There’s still some capacity planning to be done on your end although it comes down financial rather than computational, so that’s usually left to the finance department of your organisation. They’re traditionally much better at financial planning than IT admins are at capacity planning.

Many thanks to Derek Singleton of Software Advice for inspiring this post with his blog on Cloud Career Plans.

It’s no secret that I’m not a big fan of DLC. Whilst there are many games that I enjoy going back to it’s not usually because there’s a sliver more of content available for them, it’s because the games themselves warranted it. The trend now however is to continue to release bite sized chunks of additional game play after it’s been released rather than the more traditional model of expansion packs which delivered what amounted to a game in its own right. Still there have been some notable exceptions like the recent Deus Ex: Human Revolution Missing Link DLC which I’ve heard is quite lengthy and well worth the play through (I’ve still yet to play it, though). What irks me, and most gamers, is when a company releases DLC on the same day that they release the full game and an upcoming release has brought this issue to the table once again.

My first encounter with day one DLC wasn’t that long ago, it was with Dragon Age: Origins. I was a fair way through the game, not completely understanding the camp mechanic, when I saw a new character appear. Starting the conversation with them led to a quest (like it almost always does) but before he would accept it I was told that I’d need to pony up the cash to play it. Since the quest didn’t appear necessary and I had little interest in paying another $10 for a game I had just bought I left the optional DLC by the wayside and never looked back. Since then I’ve had several encounters with games that have had day one or close to it DLC and every time my reaction has been the same.

There is one exception though. Since my tendency is to buy the collector’s edition of games I’m usually treated to a free ride for most early DLC. This hasn’t changed my opinion on it though and in fact my experience with such DLC has reinforced my original stance that of if the game developers have time to develop early DLC then it should probably be included as part of the game. One of my all time favourite games will soon be releasing a sequel however and the outrage from the day one DLC has revealed that my current position might be somewhat ill informed.

The game in question is Mass Effect 3. Long time readers will know that my fanboyism for this game approaches near ridiculous levels: I bought a Xbox360 just to play it (I’ve bought other games for it, but make no mistake that Xbox360 was there for one reason only), I’ve got multiple characters and each time I’ve bought the collector’s edition. Had I done a Game of the Year post for 2010 it is quite likely that Mass Effect 2 would have come out on top. What I didn’t mention at the time was that there was some day one DLC included and whilst I did play it I didn’t feel like it added anything (nor distracted from) to the main core of the game. Indeed it could have been left out entirely and I wouldn’t have noticed a difference.

It has been revealed that Mass Effect 3 will have day one DLC, free to collectors and charged to everyone else. This put the community up in arms with many (myself included) wondering why this wasn’t part of the core game. Bioware came out and defended it fervently and revealed a point that I hadn’t really considered. The certification process for consoles is a long one, filled with all sorts of radical testing like clicking buttons thousands of times to ensure most of the bugs have been stamped out. This takes approximately 3 months and during that time many publishers elect to have the developers work on DLC rather than move them onto other projects (or do nothing at all). Since there’s less certification required to release DLC you then end up with a finished DLC product right on release day, much to the dismay of the fans.

That’s changed my view on day one DLC significantly, but it probably won’t change my purchasing patterns. Indeed I can understand why people are particularly frustrated about this particular DLC, it seems like a particular character (who’s previously appeared in the series) will only be available through it. That’s enough to put some people off it and I wouldn’t be too happy with somewhat plot critical elements being thrown into paid for DLC either. If it wasn’t included in the collector’s edition I certainly wouldn’t be bothered with it and my review later would reflect that.

For this case at least it looks like day one DLC didn’t come at the cost of the game itself but the gaming community is going to have a hard time swallowing that line from every publisher. It might then be worth delaying DLC to some time after the initial release in order to avoid this kind of negative publicity. Still I don’t have the numbers on this and if day one DLC works financially then you can bet on seeing more games with it in the future. I may not support it financially but so long as the core game isn’t affected by it I won’t say anything bad about it, but if  said DLC does impact on the game you can rest assured I’ll give them a thorough panning on here.

The idea behind Kickstarter is a great one: you’ve got an idea and you’ve got the fixins of a potential business going but the financial barrier of bringing it to market are keeping you from seeing it through. So you whip up a project on there, promise people rewards or (more commonly) the actual product you’re intending to sell and then wait for backers to pledge some cash to you. For the backers as well its great as if the project doesn’t get fully funded then no one has to donate any money, so your potential risk exposure is limited. Of course Kickstarter take their slice of the action, to the tune of 5% (plus another 3~5% for the payment processing) so everyone comes out a winner.

It’s a disruptive service, there’s no denying that. There are many products that wouldn’t have made it through a traditional venture capital process that have become wild successes thanks to Kickstarter. This of course gets people thinking about how those traditional systems are no longer needed, I mean who needs venture capitalists when I can get my customers to fund my project? Well whilst I’d love to believe that all we need for funding is crowdsourcing tools like Kickstarter I can’t help but notice the pattern of most of the successful endeavours on there.

They’re all done by people who were already successful in the traditional business world.

Take for instance the latest poster child for the success of Kickstarter: The Double Fine Adventure. For gamers the Double Fine name (and the man behind it, Tim Schafer) is a recognizable one, having worked on such cult classics as The Secret of Monkey Island, Grim Fandango and releasing others such as Psychonauts and Brutal Legend. Needless to say he’s quite well known and made his name in the traditional game developer/publisher world. Kickstarter has allowed him to cut the publishers out of this particular project, putting more cash in his pocket and allowing him total control of it, but could someone without that kind of brand recognition pull off the same level of success?

The answer is no.

For all the successes that are seen through Kickstarter only 44 percent of them will ever actually get the funding they require. Indeed in the Video Games category the highest funded game (there are a lot of projects in there that aren’t exactly games) before the Double Fine Adventure managed about $72,000. Sure it’s nothing to sneeze at, it was almost 6 times what they needed, but it does show the disparity between relative nobodies attempting a to crowdfund a project and when a well known person attempts the same thing. Sure there are the few breakout successes, but for the majority of large funding successes you’ll usually see someone who’s already known in that area involved somehow.

Now I don’t believe this is a bad thing, it’s just the way the process works. Nothing has really changed here, except the judgement call is shifted from the venture capitalists to the wider public, and as such many of the same factors influence if, when and how you get funded. Name recognition is a massive part of that, I mean just take a look at things like Color that managed to pull in a massive $41 million in funding before it had even got a viable product off the ground just because of the team of people that were behind the idea. Kickstarter doesn’t change this process at all, it’s just made it more visible to everyone.

Does this mean I think you should keep away from Kickstarter? Hell no, if you’ve got a potential product idea and want to see if there’s some kind of market for it Kickstarter projects, even if they’re not successful, are a great way of seeing just how much demand is out there. If your idea resonates with the wider market then you’re guaranteed a whole bunch of free publicity, much more than what you’d get if you just approached a bank for a business loan. Just be aware of what Kickstarter does and does not do differently to traditional ways of doing business and don’t get caught up in the hype that so often surrounds it.