Posts Tagged‘json’

Necessity is the Mother of Invention.

I’ve been developing computer programs on and off for a good 7 years and in that time I’ve come across my share of challenges. The last year or so has been probably the most challenging of my entire development career as I’ve struggled to come to grips with the Internet’s way of doing things and how to enable disparate systems to talk to each other. Along the way I’ve often hit various problems that on the surface appear to be next to impossible to do or I come to a point where a new requirement makes an old solution no longer viable. Time has shown however that whilst I might not be able to find an applicable solution through hours of intense Googling or RTFM there are always clues that lead to an eventual solution. I’ve found though that such solutions have to be necessary parts of the larger solution otherwise I’ll just simply ignore them.

Take for instance my past weekend’s work gone by with Lobaco. Things had been going well, the last week’s work had seen me enable user sign ups in the iPhone application and had the beginnings of an enhanced post screen that allowed users to post pictures along with their posts. Initial testing of the features seemed to work well and I started testing the build on my iPhone. Quickly however I discovered that both the new features I had put in struggled to upload images to my web server, crashing whenever a picture was over 800 by 600 in size. Since my web client seemed to be able to handle this without an issue I wondered what the problem would be, so I started digging deeper.

You see way back when I had resigned myself to doing everything in JavaScript Object Notation, or JSON for short. The reason behind this was that thanks to it being an open standard nearly every platform out there has a native or third party library for serialising and deserialising objects, making my life a whole lot easier when it comes to cross platform communication (I.E. my server talking to an iPhone). Trouble with this format is that whilst it’s quite portable everything done in it must be text. This causes a problem for large files like images as they have to be changed into text before they can be sent over the Internet. The process I used for this is called Base64 and it has the unfortunate side effect of increasing the size of the file to be transferred by roughly 37%. It also generates an absolutely massive string that brings any debugger to its knees if you try to display it, making troubleshooting issues hard.

The image uploading I had designed and successfully used up until this point was now untenable as the iPhone client simply refused to play nice with ~300KB strings. I set about trying to find a solution to my problem hoping to find a quick solution to my problem. Whilst I didn’t find a good drag and drop solution I did come across this post which detailed a way in which to program a Windows web service that could receive arbitrary data. Implementing their solution as it is detailed there still didn’t actually work as advertised but after wrangling the code and over coming the inbuilt message size limits in WCF I was successfully able to upload images without having to muck around with enormous strings. This of course did mean changing a great deal of how the API and clients worked but in the end it was worth it for something that solved so many problems.

The thing is before I went on this whole adventure had you asked me if such a thing was possible I would’ve probably told you no, at least not within the whole WCF world. In fact much of my early research into this problem was centred around possibly implementing a small PHP script to accomplish the same thing (as there are numerous examples of that already), however the lack of integration with my all Microsoft solution means I’d be left with a standalone piece of code that I wouldn’t have much interest in improving or maintaining. By the simple virtue that I had to come up with a solution to this problem meant I tried my darnedest to find it, and lo I ended up creating something I couldn’t find anywhere else.

It’s like that old saying that necessity is the mother of all invention and that’s true for both this problem and Lobaco as an idea in itself. Indeed many of the current great Internet giants and start ups were founded on the idea of solving a problem that the founders themselves were experiencing and felt that things could be better. I guess I just find it fascinating how granular a saying like that can be, with necessity driving me to invent solutions at all levels. It goes to show that embarking into the unknown is a great learning experience and there’s really no substitute for diving in head first and trying your hardest to solve an as of yet unsolvable problem.

ProcrastinationOn: Apply Directly to the Forehead!

It was almost 9 months and 200 posts ago that I thrust my pre-alpha version of Geon into the world for everyone to see. Thanks to my innate shyness I didn’t go the whole hog and release it into the wild for the whole world to see and I’m still glad for that as the first version was, to put it lightly, a smoking pile of crap. Had anymore than about 5 users got on it at once (the record stood at 2) my server would have fallen on its face trying to deliver all the content over my poor little 1Mbps connection. The saving grace of Silverlight taught me that I could use my client side programming skills to do what I wanted on the web without having to completely relearn everything and the next few versions of Geon came along that much faster.

Right now I’m comfortable enough to let every reader of this blog know that there’s a new version of Geon up (those adventurous amongst you would’ve noticed a link to the new version in a previous post) and it comes along with a UI change that I had been alluding to a while back. In essence the change was done in order to increase the readability of the information streams you’ve selected as prior to this you just had the one bar that would scroll along madly if you dared to look at multiple locations at once or just so happened to add Twitter from anywhere that was mildly populated. In addition to the UI changes I have also made the switch to Silverlight 4 which added in things like native scroll wheel support (I can’t tell you how happy that made me) and a slight performance improvement over Silverlight 3. Thankfully none of the breaking changes they made in the transition affected Geon so the upgrade was only a few clicks and a restart of Visual Studio away.

The new UI works similarly to the old one as you select your location first by clicking the location button on the left hand side and then clicking the location on the map you want to see. Then you can add in information feeds from the same bar in a similar way and they’ll automatically add themselves to the closet location circle on the map. As of right now all the feeds available work apart from Facebook (you’ll get a pop up asking it to connect with your Facebook account but no information will appear) because their geolocation is still not fully implemented and I’m not keen to do a whole lot of mangling to get results that are more than likely irrelevant anyway¹. Once you’re done adding the streams hit the button up in the left hand corner to see your streams in all their glory. Rows are locations and the columns are the feeds, all titled properly so you can tell what’s what.

Having all that done means however I’m now out of options for procrastinating. You see whilst this version included some new streams (videos and Wikipedia), a much better UI and a cleaner back end (mmmm JSON) most of the heavy lifting had already been done in previous versions. After getting the initial hard parts out of the way with the UI most of it could have been done inside of a week, although I casually programmed it over the course of a month or so. The next thing on the list is the real meat of Geon: the request system.

That pretty much means I have to start diving into something I’ve never coded before: webservices. Whilst I can’t really say I’ve been avoiding this I haven’t been actively looking to do anything about it either, apart from the casual search for tutorials on how to build user authentication systems. I know I’m just being a big baby about this and I should just suck it up and do it but it’s just been so darn easy up until this point I’ve been wondering why no one has done it before. As it turns out the rudimentary parts that most netziens have come to expect are the most complex and tiresome parts which is why it hasn’t been done (and also explains why some services don’t have logins at all).

I’ve decided to suck it up and just start hammering away at it until I get the thing going. It’s much like when I first started out coding Geon and I was using RSS feeds for everything, it was just the first way I found to do things. After fiddling around for a while and getting some advice from a real developer mate I found that had I just taken the time to research it the whole idea of using other formats was so much easier. I’m sure with an afternoon of searching under my belt I’ll be ready to tackle the big bad demon that is the client/server architecture of Geon.

¹I thought I should elaborate on this a little bit. There’s been rumours of a geo-api from Facebook for a while now but with their developer conference f8 over and done with I haven’t heard anything solid about its actual implementation. They’ve tweaked their privacy policy to allow the storage of geo information in Facebook however the API as of right now remains unchanged. There are a lot of apps out there making use of geo data and Facebook but there’s no way to extract that out of Facebook currently. You can kind of figure stuff out by finding out a user’s hometown location, reverse geo-coding the location, figuring out if that’s within your bounding box and then displaying messages from them if it’s within your area HOWEVER that’s an incredibly messy way of doing it and honestly isn’ the kind of thing I was looking for. I’ll be integrating Facebook information when they finalize their geo-api but until then it won’t work.