I believe everyone is familiar with the concept of being “in the zone”, I.E. that state you attain when you’re so intensely focused on something that time becomes irrelevant and all you’re focused on is achieving some certain goal. I personally find myself in this state quite often usually when I’m writing here, gaming or programming. Whilst I knew it was a common phenomenon I only learnt recently that its also recognised as a part of psychology, where they’ve termed it Flow. The concept itself is interesting an most recently I’ve started to grapple with one of the more subtle aspects, defined as point number 8 of conditions of Flow or “The activity is intrinsically rewarding, so there is an effortlessness of action”.
Now this weekend just gone past saw me back, as I almost always am, coding away on my PC. Now since I’m somewhat of a challenge junkie I’ll always seek out the novel parts of an application first rather than the rudimentary and the first day saw me implementing some new features. This always goes well and I’ll be firmly in Flow for hours at a time, effortlessly jumping through reams of documentation and masses of Google searches as I start to nail down my problem. Once the new feature is done of course then I’ll have to choose another to start work on, thereby maintaining my Flow and project progress.
There’s a great quote by Donald Knuth (of The Art of Computer Programming fame) that says “Premature optimization is the root of all evil” which is basically a warning to avoid over optimizing your code when its still in the early stages. I’m a firm believer in the idea that you shouldn’t act like you have problems of scale until you have them but there are some fundamental differences between regular and scalable code that could prove to be incompatible with your codebase should you not make the decision early on in the piece. Of course optimization comes at the cost of progress on other pieces of work thus a balancing act between the two is required if your code is ever to see the light of day.
I guess I find it strange that optimizing my own code was so detrimental to achieving that state of coding nirvana. It’s quite possible that it was just the problem that I was working on as a previous optimization I had done, developing a cache system for a web service I was querying, seemed to have no ill effects. However that particular challenge was quite novel as I hadn’t created anything like it previously and the feedback was quite clear when I had finally achieved my goal. Unfortunately I have the feeling that most of the optimization problems will be more like the former example than this one, but so long as I write half decent code in the first place I hopefully won’t have to deal with them as much.
There’s an old saying that goes something along the lines of “The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man“. As someone who had lived much of his life trying to learn the rules of the world so I could work within them the notion that being unreasonable about something would be the catalyst for progress was initially met with harsh scepticism. However I began to notice that the ones who actually managed to enact change were in fact those who were making unreasonable demands, not just of others but also of themselves. They also seemed to flourish within boundaries, seeming to be far more capable working with some kind artificial constraint than they were with completely open ended problems.
I really started to believe in this whole progress comes from being unreasonable idea when I started working on my own projects and I started running up against things that people had never come across before. Now for us .NET developers it’s pretty much guaranteed that you’re not the first one to run up against a certain problem since there are so many people out there developing with it. However following on from that idea you’ll tend to find then that if people can’t find a solution to particular issue they’ll instead find some other way of achieving the same goal that’s been done before. This is the double edged sword of Microsoft’s black magic and it definitely traps the wise programmers in the loop of adapting themselves instead of trying to make changes to the world they’re operating in.
I had this recently when I was working on my latest project. I was working with an ASP.NET MVC 3 site and I had set up the web site to make multiple calls back to the server in order to retrieve the data it needed. Now this worked well to get it off the ground but anyone looking to optimize a website will tell you reducing the number of calls to your server will lead to much better performance, for both the client and your server. Eliminating all these calls and wrapping them up into the @Model of the view for this part of the website would do that, but I had no idea of how to get the same results as I had done with the multiple requests. After searching, hacking and testing several different things I eventually found myself with a very workable solution and I was left for many more ideas for improvements to the site.
Now had I been more reasonable with my expectations I would’ve instead just kept on doing what I had been doing (since it was functional) and wouldn’t have dared to consider changing. Indeed I sat on the whole idea for a day before pulling the trigger on it, precisely because of the amount of rework that was involved in doing so. However the changes I made will make it far more easier going forward since it allows me to work in the areas I’m much more comfortable with rather than fooling around with technologies I’m still in the process of understanding. Sure going the other way might have been a better learning experience but I’ve learnt quite a lot in the process of overcoming the goal I set myself, perhaps more than I would have should I have continued down the same path.
Having unreasonable expectations frees you from being constrained by perceived limitations, allowing you to push to the very limits of what is possible. Arbitrary boundaries help to limit the problem space considerably enabling you to focus more clearly on the ultimate goal rather than getting stuck in the multitude of minutia. Combining these two ideas has helped me push past my own limitations in many aspects of my life, from coding to fitness and even to unlocking my hidden creative self. So I put it to you to start being more unreasonable in your expectations and using arbitrary limitations as enablers rather than blockers to progress.
The multiple years of experience that came prior to it.
It’s no secret that whilst I’ve been developing for a long time I’m no rockstar when it comes to the world of web programming. Indeed my first foray into this world was a bastard of a page that was lucky not to fall on its face constantly and the experience had me running to find better solutions, eventually falling to Silverlight. The reason for this was obvious, it allowed me to leverage my desktop development experience into a new platform. Sure I struggled with the ideas that just couldn’t be boiled down into the desktop world (like that whole REST thing) but it was a quick way to get myself into this world and expand from there.
So of course when I saw people saying they built this incredible website in only a weekend when it took me several months worth of weekends just to get mine working I was intrigued. I even made the foolish mistake of reading up on some of their “how I did it” posts on Hacker News and saw all these wonderful frameworks that they had been using, assuming this would make me a master overnight. Stepping through some of the tutorials and looking at the tools available started to raise some eyebrows since they were unlike anything I had seen before, and this is where I got suspicious.
You see I could whip up a simple desktop app or PowerShell script in minutes that would do some function using the tools I have in front of me, but that doesn’t mean you should be using those tools to create your site. Neither does that mean you would be able to whip up the same thing using the same tools in the same amount of time, no matter how skilled you were in other languages. The simple reason for this is that whilst you might be a rockstar in ruby or an expert in PHP your experience is confined to the environment to which you’re most accustomed and should you need to retool and reskill for a new language it’s going to be several months before you’re at your maximum competency again.
Sure good developers are able to adapt much faster than so-so developers but there’s a significant opportunity cost in switching away from your current knowledge comfort zone in order to try and emulate those who you idolize. I came to this realization a couple months back after staring at so many Ruby/Python/SomeDynamicLanguage web sites, wondering at how the heck they got them looking and functioning so well. In truth the platform they were using had little to do with it, these guys had just been in the game for so much longer than me that they knew how to get these things done. With me still in the grok stage of my first really truly web framework I really shouldn’t be comparing myself to them just yet, not at least until I can get my new application functioning the way it should.
It’s so easy to get disillusioned with what you’re doing when you see others progressing so much faster than you ever thought you could. My new application was supposed to be a testament to my coming of age as a web developer, having giving myself only a short time to get it off the ground before actually launching it. Since my deadline for that has come and past I’ve been forced to change the way I view myself as a developer and have come to realize that unless I’m working in something I’ve developed with before I shouldn’t expect myself to be a rockstar from day one, instead recognizing that I’m still learning and pushing through the pain barrier until I become the rockstar I thought I was.
¹If you’re interested, what’s hot right now is photo sharing apps. What’s not? Location apps, go figure.
After reaching 1.0 of Lobaco I’ve taken a breather from developing it, mostly so I could catch up on my backlog of games and give my brain a well deserved break from working on that problem space. It’s not that I’m tired of the idea, I still think it has merit, but the last 6 months of little free time on the nights and weekends were starting to catch up with me and a break is always a good way to kick start my motivation. It didn’t take long for numerous new ideas to start popping into my head afterwards and instead of jumping back into Lobaco development I thought I’d cut my teeth on another, simple project that would give me the experience I needed to migrate Lobaco into the cloud.
The weekend before last I started experimenting with ASP.NET MVC, Microsoft’s web framework that based on the model-view-controller pattern that I had become familiar with after deep diving into Objective-C. I could have easily done this project in Silverlight but I thought that I’d have to man up sooner or later and learn a proper web language otherwise I’d be stuck in my desktop developer paradigm for good. The results weren’t spectacular and I could only bring myself to spend about half the time I usually do coding on the new site, but there was progress made there none the less.
The slow progress really frustrated me. After finally gaining competence with Objective-C I felt like learning yet another new framework would be easy, even if it meant learning another language. Somehow I managed to forget that frustrating first month where progress was almost nil and I convinced myself I wasn’t procrastinating when looking for other solutions to my problems. Eventually I came to the realization that I was still grokking the new framework I had chosen for my application and that I shouldn’t be expecting myself to be blazing trails when I was still establishing my base of fundamental knowledge.
I see lots of people go through the same struggle when trying out new things and can see how easy it is to give up when you’re not making the kinds of progress other people are. Believe me its even worse in the tech/start-up area where every other day I’m reading about someone who hacked together a fully usable service in a weekend whilst I struggle to get my page to look like it wasn’t written in notepad. The realization that you’re still in the grok stage of learning something new I find to be quite a powerful motivator as past experience has shown that it’s only a matter of time and persistence between floundering around and becoming quite capable.
I’m usually the first one to tell people to stick with what they know as re-skilling is extremely expensive time wise (and can be $$$ wise too, Objective-C set me back a few large) but the pay-offs of diversifying you skills can be quite large. Whilst I’ve yet to make any semblance of a dollar from all my adventures in iPhone development I still count it as a valuable experience, if for the mere fact it’s given me a lot of perspective and oodles of delicious blog fodder. Time will tell if this current foray into yet another web framework will be worth my time but I wouldn’t be doing it if I thought there was no chance of it ever paying off.
Take a good look at any big IT system and you can usually trace its roots to one of two places. The first is the one that all of us like to work with: the Greenfield project. In essence this is brand new work that has been born out of a requirement that didn’t exist before or a complete rethink of a current implementation. Talk to any consultant who’s trying to sell you some new tech and you can be guaranteed that they’ll be looking to sell you a greenfields solution, mostly because it’s cheaper and much easier for them to implement.
Sadly, and especially for those of us employed by the government, the majority of the projects that us IT guys will work on will never be greenfield situations and will usually be encumbered by some form of legacy system. This poses greater risks and constraints on what work can be done and ultimately you’re probably working to fix problems that someone else created. It’s been rare that I’ve been given the privilege of working on a project that was aimed at fixing my own mistakes, but I could put that down to my insatiable appetite for job hopping.
My own projects are a different beast as they are all my own work and thus all my own mistakes. Take for instance my intial foray into the world of web programming, Geon. Initially I decided that I’d code the whole thing in ASP.NET mostly because I could do it in C# (something I’m very familiar with) and there appeared to be a whole lot of resources available for doing the things I needed to do. For the most part that worked quite well and I was able to get the majority of the components up and running within a few weeks. Sure some of the subtleties of the design I had in my head didn’t quite work but for the most part I was able to get what I needed done, and even launch a few improvements along the way.
The transition was in fact a greenfields approach to the application. The initial iteration of Geon in Silverlight was, for the most part, a like for like system built upon a completely new code base. Whilst they share a common language the frameworks available and the UI design are wildly different. Still with a little effort I was able to replicate Geon into Silverlight in less than a weekend and everything seemed right with the world.
Then one day I had a fit of inspiration about a new layout for Geon. I quickly fired up Visio and started playing around with visual elements and cobbled together a better design. Everything seemed to be falling into place and I could see how it would be so much easier with this new design. Unfortunately this meant that the current code I had written for Geon in Silverlight was effectively unusable as the visual elements drove the underlying objects and thus couldn’t be used. The internal logic of some parts remained though and the new design took considerably less time to develop.
You might be noticing a couple patterns here. The first is (I’m going to start with the good here) that for the most part a lot of what I’ve created is reusable which is a classic example of modular programming at work. There was a bit of massaging between ASP.NET and Silverlight but thanks to Microsoft’s libraries this was fairly minimal. The second is I’m getting into a bit of a habit with starting a fresh each time I think of a new and better way of doing something, despite the amount of work that that entails.
I put this down to a form of analysis paralysis. In essence every time I’ve taken a long hard look at my code after a break from it the first thing I start to notice is how difficult it will be to get everything working just the way I have it in my head if I want to keep the current code. It stems from the way that I work on problems, by intensely focusing on a single problem until I have the solution. Whilst this usually ends up with an adequate solution to the problem I’ve often found myself spending a good 10 minutes on a function figuring out how exactly it does something. Repeat this for every function and in the end it becomes easier to just start over again instead of trying to rewrite everything so it fits together perfectly.
This all came to head when I started looking at the layout of Geon and realised that it had some inherit problems with viewing large amounts of information. Subsequently I’ve drawn up yet another design that is, you guessed it, almost wholly incompatible with the way I’m doing things now. I’ve since dedicated my weekend to developing the design and seeing how it works out but as you can imagine when I’m looking at dropping the code base for the 3rd time I start to question whether I’m really making any progress. Or maybe I’m just avoiding coding the real meat of Geon because it’s, you know, hard.
The good news is that the project manager in me isn’t going to be happy with feature creep and deadlines falling by the wayside so I firmly believe that this iteration of Geon will be the last major UI redesign before its final release on the world. This time I’ve made sure to include those “little things” like a user control panel which were strangely absent from my last 2 designs and hopefully I’ll achieve my goal of making the information much more visible than it currently is.
It really doesn’t help that my to-play list of games is getting longer every day either 😉