My Notes From the Movie Startup.com

2011-10-29T14:50:00-04:00

This weekend I watched Startup.com, and here's some notes I jotted down while watching it.

We should all trust that any one of us will represent a vision of the business that will be seconded and thirded by anyone in the team.
We should never be in a confrontation / debate stage in front of anybody.
There's a sadness about watching ideas grow until they're beyond the point where you can do anything for them.
As soon as someone says "There's no need for this to be ugly" it's going to get ugly.
Meet your competitors, befriend them. Force them into making plays earlier than they want to. But only if your product tangibly is better.
Don't make your company about your personality as a CEO who 'doesn't lose'.
It doesn't matter if the market is big, it's a question of who can make money in it.
Always have a third party who can arbitrate.
Have a 'Days to Live' countdown board with focussed tasks to achieve in that time
Everyone should use the product, CEO down to the person who answers the phones.
Be critical on the status of your product's functionality. If it doesn't work, say it doesn't work.
Keep momentum on Testing.
Security of your product, your office, everything is paramount.
Be concerned about revenue. Don't be concerned about the Board. It's their prerogative to fire you or re*shuffle as they see fit, and they will do so.
Goals need to be specific and actionable.
Any company with two CEOs is in trouble.
Even in 2001, mobile phones had some horrific extendable antennas.
Sales cycles are always longer than anticipated.
Things change. That's the best part.
Decline happens quickly and noticeably.
The scariest part of this movie is Kaleil Tuzman driving along a wet highway, chanting to a CD and reading from a book held against the wheel.

Announcing: The Beast Master

2011-10-28T18:32:00-04:00

I needed a specific tool that would help record when a piece of software was deployed to a particular environment, keeping track of the who, the when and the SHA1 from Github that was deployed. So I wrote The Beast Master in a day.

It's written in Ruby 1.9.2 and Sinatra, and currently has an RSpec test suite coverage of 94%. It provides a simple HTTP API with two endpoints, one providing a dashboard showing the latest releases, the other a POST endpoint to mark a release with. There's the ability to enable Basic HTTP authentication on both endpoints.

All in all, it took about a day to drive out the functionality TDD style, following a strict Red-Green-Refactor approach.

It's located up on http://github.com/johnmcdowall/the-beast-master and I welcome you to try it out, send feedback or make improvements.

Bon appetit.

Announcing delimitedfileformats.info (I Need Help)

2011-09-24T13:00:00-04:00

I've come to understand the horror that is working with Text based delimited file formats, such as CSV, TSV or what have you. There are some published specs for both file formats that can be used as guidance when it comes to implementing write support for these formats, but mostly every file I see is broken in some way that expects the parser to be omniscient about the original context of the file.

So, in an effort to increase awareness around Delimited file formats, I've started a project on github, and a published site at http://delimitedfileformats.info that will attempt to give people useful information about how to read and write these particular file formats in a way that is consistent and productive.

But I need help.

I need help with the design, the content, and feedback on the overall usefulness of the site.

I've made an initial stab at it, but I'm hoping for some lovely team work on this...

Software and Old Houses

2011-09-24T11:43:00-04:00

Old Houses are products of accretion. Over the course of many years they gather new extensions, tweaked facades, upgraded central or under-floor heating systems, alarm systems, downstairs washrooms and nursereys. And usually, once one project is completled, you'll bask in the glow of it for a while until it's forgotten about and the next one embarked upon. Just like Software Design.

The house I live in is big and old: over 80 years. It's been split up so that a couple live in the back portion, my wife and I live in the front half, and there's a basement apartment. Sound travels in funny ways in old houses.

You see, there are remnants of what were once ducts for the purposes of a forced air central heating system, all through the house. The air vents are now sealed, but the ornamentation on the baseboard that gives away their purpose is still there. The ducts lie embedded within the walls still. Long forgotten about, these heating ducts were probably installed at great expense at the time, for what was felt to be the ultimate in home comfort: hot air blown into the rooms from a central heater!

At some point, the old air heater was pulled out, radiators were found and installed, and a fancy Swedish boiler system now silently and efficiently pumps hot water around the piping and heats the house. And the ducts remain, in the walls. Now sealed up and silent. Except for when they carry sound in funny ways. Old houses are like that.

Occasionally, we are treated to the sounds of someone somewhere else in the house enjoying vigorous, short lived coitus, the echoes of it carried inside the bones of the house, inside these long forgoten about ducts that serve no purpose. Should the ducts be ripped out? That would be very costly, and damaging to the house. Ripping them out would deliver no real value to anyone. So they stay. Silently embedded within the walls.

Those ducts are much like the infastructure portions of almost every software design that's ever been written. Look at your own current project’s codebase. Is there something in there, some remnant of infrastructure that was built with the greatest of hopes and expectations, only to be sidestepped when the problem was more fully understood?

Removing that code might be too hard now, because nobody writes perfect software. Some brittle tests somewhere might flare up, or a method call somewhere might explode, in a certain circumstance. Deadlines are looming and everything's on fire. Maybe you work in a job where your boss says things like 'You don't get paid to remove code, you get paid to write it!'.

So that code sits there, inside your digital walls.

Software Design is like building houses. The construction continues long after the thing is 'finished', and you end up living with the choices you make for a long time, usually forever. Sometimes it's ok to paper over some minor cracks and live your life.

And sometimes, in the night, you hear odd noises carried by those old ducts.

Continuum Movie

2011-08-27T12:14:00-04:00

While stumbling about the flavors.me site, I found a landing page for a new Indie movie called CONTINUUM. The premise is probably more interesting from a technical standpoint than plot wise:

Synopsis:
Osmond speeds down an open highway in retreat of his mysterious past. When he stops the car and steps off the path, he finds himself locked in a fractal time loop, and is doomed to realize his fate over and over... and over.
Production:
I'm making a movie about a time loop, which usually entails the main character meeting or interacting with their self in some way. That's the whole point. I wanted to achieve this for my film, but as one continuous shot, and without the use of hidden cuts, visual effects, or twin actors. After countless hours of brainstorming and lots of advice from really smart people, we found a practical way to pull it off. I'm not going to give away the secrets, so you'll have to wait and see for yourselves!

I'm intrigued to see how this could be done without the use of any of the usual staples of time travel movies, using only practical means.

You can check out the Movie's micro-site here

Excellent Newsletter Signup Example

2011-08-18T22:21:00-04:00

Getting people to sign up for your newsletter can be a tricky task, and everyone's trying to get their customers to do the same. I found an excellent example of how to make it clear what people are signing up for.

This image is from the Bvckup blog. The header is clear and succinct: 'To receive News and Promotions'. The email box has clear text that instructs the user how to complete the signup, with an arrow on the right hand side giving the affordance that this action will complete something significant.

And then the piste de résistance: 'Low volume, one-click unsubscribe'. This statement alone is genius. It's the very equivalent of a social proof substitute in that it tells the user that this newsletter won't deluge them with emails and if they want to unsubscribe, then they only have one click to make. Skilfully, both of the primary objections a user could have for signing up to this newsletter are tackled.

Bravo.

Intent

2011-08-12T17:21:00-04:00

The Mayor: Intent? How did you establish that?
Harry Callahan: When a naked man is chasing a woman through an alley with a butcher's knife and a hard-on, I figure he isn't out collecting for the Red Cross!
[walks out of the room]
The Mayor: He's got a point.

A quote from the excellent Clint Eastwood movie Dirty Harry. How do we establish someone's intent? By observing their actions, and the context of those actions. True intent is often disguised in a veil of language and ego-based posturing, but actions in their proper context aways reveal the nature of someone's intent.

Wallpaper: Freedom

2011-08-09T22:58:00-04:00

Sometimes I get the feelin' in my bones to make a wallpaper. So here's one. It's licensed with Attribution-NonCommercial, so you can do whatever you want with it as long as you attribute me, and don't use it for commercial purposes. Have a blast.

Click through for the download link.

Download it here. Drop me a line at john at mcdowall dot info if you love it, or hate it.

Data Is

2011-08-03T19:18:00-04:00

A great image about what Data is, sent to me from my friend @Daniellaanne:

From the Flowing Data blog.

Long Term Goal One

2011-08-01T20:37:00-04:00

‘I am striving not to become the grand old man of the profession, but to close my design doors when I reach forty-five’
Louis SwartLouis Swart in an Interview on His Thirtieth Birthday

Ruby Is Slow? She's Fast Enough for You

2011-07-31T12:37:00-04:00

For the past while, Twitter have been moving gradually from their legacy Ruby on Rails codebase to Java and the JVM. This has been greeted with the usual cries of 'Rails is slow!', 'Ruby is slow!' and hand-wringing in the startup circles that accompanies such announcements. At OSCON 2011, Raffi Kirkorian gave a presentation rounding up the reasons why Twitter has moved to the JVM.

Sadly, Raffi is sloppy with his usage of the terminology. When he speaks of 'Ruby' he actually means the MRI 1.8.x runtime.

Looking Deeper

It's a well understood issue in the Ruby community that MRI 1.8.x doesn't have a very good threading model. In fact, it's pathetic.

MRI 1.8.x

Green Threads are used, which is a scheme where the underlying Virtual Machine schedules the threads instead of the Operating System's Kernel. As such, they are managed in User Space with all of the accompanying drawbacks that doing so entails, particularly in multi-core environments. It is worth noting, however, that Green Thread were the only option in Java 1.1. In Ruby, this means that all of these 'fake' threads map to one single actual process thread.

MRI 1.9.x

Native threading was introduced, but the runtime still has a Global Interpreter Lock (GIL) that prevents a lot of concurrency. It seems that each revision of MRI 1.9.x makes this lock more granular, which improves the concurrency situation. The decision for this seems to be in order to make an easy path for C extension developers to migrate to truly concurrent threads.

JRuby 1.5.x and later

Native threading is provided through the JVM and there is no Global Interpreter Lock. Ruby threads are Java threads. You would have exactly the same level of concurrency here as you would writing Java apps on the JVM.

Rubinius

Since Rubinius aims to have 1.8.x / 1.9.x level compatibility, it also has a GIL, although there is a branch called Hydra which aims to reduce or remove the GIL as much as possible.

THE MYSTERY IS REVEALED

Raffi points to the fact that when Twitter acquired Summize in 2008, they inherited an infrastructure entirely written in Java.

Moreover, the engineering blog recently announced they made Search three times faster by moving to a Java server called Blender.

Additionally, Twitter have also spent a lot of time and money improving the MRI 1.8.7 garbage collector

WHAT'S REALLY HAPPENING

Twitter say that JRuby's ecosystem was immature, and a lot of the clients they would need were simply too slow. That article in particular has a headline claiming 'Twitter Shifting More Code to JVM, Citing Performance and Encapsulation As Primary Drivers'. Claiming moving to any language because it gives you better encapsulation is just bullshit, especially when Rails developers do have a tendency to write monolithic applications: this is nothing to do with Rails and everything to do with the developers who come to Rails.

Then there's Raffi himself, a former co-founder of WattzOn which was written using Scala.

In short, Twitter can't see past MRI 1.8.7, inherited a lot of Java code (and undoubtedly a bunch of Java developers along with it), and instead of exploring JRuby to exploit native threads made the decision to consolidate on a Java based technology stack. Blender became three times faster not because it was moved to Java, but because Twitter finally took advantage of native threads via the JVM.

CONCLUSION

Twitter do have a tough time scaling largely because they locked themselves into MRI 1.8.7. Java isn't giving them anything they didn't already have as a language, in fact quite the opposite is true. The JVM is providing features to them that they arguably could have leveraged using JRuby.

It seems that Twitter's move to the JVM and Java is motivated because of the acquisition of some Java codebases and engineering talent, and a lack of genuine understanding of what Ruby and Rails has to offer in these modern days we live in.

Twitter have a history of making really poor engineering decisions. Like not monitoring their internal network, not placing indices on tables and choosing obvious passwords for their admin systems. Could the move to Java and the JVM be based purely on not understanding what was built and how to manage it, and a desire to move to more familiar grounds for the new engineers?

That said, Twitter is also an unique case in terms of scale and real time performance requirements. The reality is that you will never be in the position of having to deal with as much traffic and real time requirements as Twitter is when you are starting up or trying to prove an idea. Chances are you'll get very far being able to iterate more quickly on Rails and still handle scaling up all you need to. Most shops aren't building real time trading platforms. Shopify, for example, are serving millions of requests are 100% Ruby on Rails.

So cease the hand-wringing. You can make bad engineering decisions in any language. Go build something awesome, fast.

FOOTNOTE

I don't mention C# and ASP.NET. In this case you'll be so bogged down in being caught up on the Microsoft treadmill and then swallowed alive by the costs of your Bizspark licenses when they come a-knockin' that you'll be overtaken by almost anybody else. You will have some fast-rendering webpages though. Nice.

Don't use TempFile in RSpec specs

2011-07-28T17:33:00-04:00

Problem

I had inherited some tests that used Ruby's TempFile to create a temporary file which then had some data written to it, and the was path passed along to the rest of the test. This worked fairly well until suddenly one of the tests just started failing randomly, and as it turns out TempFile would fail to create the temporary file it was asked to and proceed without any kind of warning. Later on in the test the file didn't exist, so the test failed.

So I created a simple little spec to test out TempFile on its own:

test_tempfile_spec.rb

require "tempfile"
describe "Testing Tempfile" do
  it "should always create the temp file" do
    paths = []
    (1..200).each {
      tempfile = Tempfile.new(['testthetempfile', '.tmp'])
      paths << tempfile.path
      tempfile.write "SOME DATA"
      tempfile.close
    }
    paths.each { |tempfile|
      p tempfile
      File.size?( tempfile ).should == 9
      File.exists?( tempfile ).should == true
    }
  end
end

Run the test, get a green dot. Hmm.

However, running the test in concert with the rest of the test suite causes the error to reveal itself:

Failures:

  1) Testing Tempfile should always create the temp file
     Failure/Error: File.size?( tempfile ).should == 9
       expected: 9
            got: nil (using ==)

The temporary file hadn't been created.

I finally isolated one spec in the suite that when run with all of the other specs, and the temp file spec, caused any Tempfile not to be created, and the above failure to appear. Testing that suspected spec with the tempfile spec only... did not cause the error to happen.

So I'm left with an error on creating a Tempfile that only happens when the entire test suite is run, and only when a particular spec is included in the suite. Logging the Tempfile paths when they are created shows that the are all unique, so there don't seem to be any naming conflicts. The error happened under both 1.9.2-p180 and 1.9.2-p290.

Solution

Don't use Tempfile in RSpec specs. RSpec may or may not be the culprit here, but the code in the specs is simple and straightforward and trying to fully isolate the issue would take up more time than I have at the moment. Maybe it's an OSX issue. It could be a John issue. But out of all the file access code in the specs, only Tempfile is failing. Don't use it. It introduces weird system level dependancies into your tests. Don't use it. It makes your specs run slower. Don't use it.

The Ten Principles of Good Data

2011-06-29T19:36:00-04:00

Taking a cue from my old pal Dieter Rams, I've been thinking a lot about the principles and properties of "Good Data", which to me has properties above what any old run of the mill data you might find lying in the street has. The list is by no means definitive or even completely coherent, but it represents a brain dump of my thoughts on the topic.

Update: Added 'accurate' from a comment on Hacker News, and an interesting post on crawl-ability here which pushes us to 12 principles!

Here is my list:

Good Data is accurate. It is factually correct.
Good Data is contextual. All interpretation of Data requires a context within which the interpretation takes place.
Good Data is useful. The Data bears some use in a context outside that of its own production.
Good Data is temporally well defined. The Data captures some specific time period.
Good Data is semantically correct. The Data doesn't need to be scrubbed or cleaned.
Good Data is long lasting. The Data can always be retrieved.
Good Data is maleable. The Data can be re-processed into new Data formats.
Good Data is valuable. The Data carries some financial or social value to someone in the correct context.
Good Data is accessible. The Data can be accessed on any device and any medium.
Good Data is social. The Data can be shared, referenced and a community built around it.
Good Data is irrefutable. The Data's provenance and history is verifiable.
Good Data is crawl-able. The spiders of the web are able to find the data and consume it.

Why Rails 3.1 streaming means you still have to think about your head

2011-06-04T23:03:00-04:00

Rails 3.1 will introduce Streaming Responses. The blog entry is a bit disingenuous as it says "In particular, if you flush the head of an HTML document CSS and JavaScript files are going to be fetched in parallel, while the server works on generating content." The implication here is that the CSS and JavaScript files themselves are going to be fetched in parallel, and the blog post goes on to say that "Modern clients do not even block on JavaScript files as old ones did".

Let's look at this in more detail.

CSS and JavaScript files are going to be fetched in parallel

Watch this RailsCast on the new upcoming Streaming, specifically at 2m58s. You'll see that when streaming is enabled, the HEAD is rendered back form the server immediately, while the main BODY response is built up in memory, server-side. What will the browser do here? It will start processing those script tags serially, with a maximum of two requests at a time per hostname. Because of this, Steve Souders suggested that as many blocking script tags as possible be put after the body, or other techniques used to load the scripts asynchronously. Mozilla have a great, still relevant post about this.

What happens if you don't do this? Your users will sit at a white, blank page needlessly while those external blocking scripts are loaded and processed. Even if your modern browser supports issuing more than two requests per hostname, chances are there are Proxies or Gateways in between you and the target server that support no more than two requests. So the main BODY content could still arrive from Rails 3.1 before those exernal SCRIPT and asset links have been fully loaded.

This statement should have read 'CSS and JavaScript files are going to be fetched in parallel to the main BODY, and those files potentially loaded faster by modern browsers and infrastructure"

Modern clients do not even block on JavaScript files as old ones did

This may be true for the latest Webkit, but how important an issue this is for you depends on what range of Browsers your application should support. I can't find any conclusive reference material about this and it's beyond my current patience to scour development forums and checkins for each browser out there. I personally find the idea of just ramping up pipelined connections somewhat worrying, after all the HTTP/1.1 spec is clear:

These guidelines are intended to improve HTTP response times and avoid congestion.

This statement should have read 'there are some modern clients that attempt to further improve the SCRIPT loading and evaluation process'.

Incidentally, Streaming responses were part of ASP.NET 2.0, and the same problems with too many SCRIPT and external asset tags existed then as now. It's therefore not crazy to say that all web applications will benefit from improvements in browser and internet infrastructure technology.

Conclusion

You still need to think carefully about how your SCRIPT tags will affect the end user experience. There is no magic solution, and Rails 3.1 will not magically alleviate you of these concerns. Even if all modern browsers can look at a document and begin intelligently prefetching SCRIPT referenced assets, there is no guarantee about how quickly those assets will load, and the user should not be penalized with a white screen while your application stalls.

Specifically with respect to Rails 3.1, streaming will bring it's ability to stream content in line with many other web frameworks out there. It's long overdue.

Until you hear definitively otherwise, from people who really know the fabric of the Internet and HTTP, you'd be wise to keep to the roads and at the very least A/B test the shit out of any new approach you try.

If anyone has any clarifications they'd like to make to my understanding, I welcome them!

Updates (Comments on original article)

My guess is the Rails team is expecting that by baking in an easy way to do concatenation and minification, each page should really require only one css and one js file from inside the application anyhow, and these can safely be put in a streamed head response. Things loaded from Facebook or elsewhere outside of your control obviously should still be placed at the end of the body. But your points are a good reminder that this is not a simple issue.
Adamwww.electriceloquence.net/…

The data about connections per host and whether loading SCRIPT tags blocks comes from this project http://www.browserscope.org/?category=network (it is linked at the bottom of the blog post).
The streaming response accounts one connection, so even in old browsers that only support two connections (eg IE7) depending on how heavy is content generation, they may still have time to fetch everything before the second chunk arrives. You’ve parallelized something for sure, one parallel connection at a time.
So yes, if you need to support old browsers this is not black and white. It depends on your particular application.
Note that streaming has no benefit for fast responses, in a similar way to not having any benefit for static files. Also little benefit if the client has the assets in cache. So you will typically enable it only for heavy and upfront views, and have normal responses for the rest of the website.
I think streaming will be exceptional rather than the norm, and it has also issues with compression we are working on. Point is you have the feature implemented if you need it.
Xavier Noriahashref.com/…

When You Buy Apple You Pay for Design

2011-05-26T13:00:00-04:00

That’s right. Design with a capital D. If you don’t care about that or don’t understand Design then move along – I want you off my lawn.

It’s perennially fashionable to complain about how much Apple products cost. The most visible attack point for Apple is their range of notebooks, which range from $999 for the current MacBook, up to $2,499 for the top of the range MacBook Pro. A lot of people feel that these are exorbitant prices for such machinery, and point to Dell or anyone else who makes ‘comparable’ units for less cost. “I can buy a Dell for $400 less and it will work just as well!” they cry. Well, yes, you can. And quite frankly, you, should.

Aside: I’m not going to touch on ecological concerns or working practices, as quite frankly child slaves in China is a secret shame of many companies and a byproduct of our Capitalist way of life and if we were to incorporate that argument we would find ourselves saying we do not need any of this stuff and we should cease all exploitative production.

Dieter Rams, former Head of Design at Braun A.G. – one of the most historically significant consumer product companies yet, and during his time there he cooked up the following ten rules of Design:

Is innovative – Rams states that possibilities for innovation in design are unlikely to be exhausted since technological development is always offering new opportunities for innovative design. He also highlights that innovative design always develops in tandem with innovative technology and can never be an end in and of itself.
Makes a product useful – A product is bought to be used. It has to satisfy certain criteria, not only functional, but also psychological and aesthetic. Good design emphasises the usefulness of a product whilst disregarding anything that could possibly detract from it.
Is aesthetic – Only well-executed objects can be beautiful. The aesthetic quality of a product is integral to its usefulness because products used every day have an effect on people and their well-being.
Makes a product understandable – It clarifies the product’s structure. Better still, it can make the product clearly express its function by making use of the user’s intuition. At best, it is self-explanatory.
Is unobtrusive – Products and their design should be both neutral and restrained, to leave room for the user’s self-expression. Products fulfilling a purpose are like tools and are neither decorative objects nor works of art.
Is honest – Honest design should not attempt to make a product seem more innovative, powerful or valuable than it really is. It should not attempt to manipulate the consumer with promises that cannot be kept.
Is long-lasting – It should avoid being fashionable and therefore never appears antiquated. Unlike fashionable design, it lasts many years – even when the trend may be in favor for disposable products.
Is thorough down to the last detail – Nothing must be arbitrary or left to chance in the design of a product since care and accuracy in the design process show respect towards the consumer.
Is environmentally friendly – Good design should make an important contribution to the preservation of the environment by conserving resources and minimizing physical and visual pollution throughout the lifecycle of the product.
Is as little design as possible – Dieter Rams makes the distinction between the common “Less is more” and his strongly advised “Less, but better” highlighting the fact that this approach focuses on the essential aspects thus, the products are not burdened with non-essentials. The desirable result would then be purer and simpler.

Here is an example of Ram’s work:

Look at that thing. Look at it! When was it designed? This year? Maybe the latter half of the first decade of this century? No! 1971.

Does it bear a resemblance to Apple products? If so, it’s due to Apple’s chief designer, Jonathon Ive, being heavily influenced by Ram’s principles, rather than Ram himself. To get a better understanding of this you can watch Gary Hustwit’s excellent Objectified documentary on Industrial Design.

Look at this fan. Look at it!

Again, designed in the early 1970′s, this desk fan by Braun is an absolute joy and pleasure to look at on your desk, use, touch and listen to. It could be a desk fan for the office or the home, such that its aesthetics don’t constrain it to either domain as its design honest, understandable, speaks to its function, unobtrusive, innovative… I’ve been lucky enough to see one on Pete Forde’s desk and it follows from Ram’s principles of design perfectly. I would love to own one. But I could also own this:

Sigh.

At Apple, Jonathon Ive and his team works to unbelievable lengths to design every Apple product, making meticulous designs, iterating, refining and polishing. Apple hire chemical engineers to figure out ways to make the chemistry in the battery different and better, so that it lasts many hours longer than their competitors (this secret chemistry is the reason MacBook batteries became locked into the unit – second market companies would make inferior batteries that would damage the brand).

Even the Headphones are meticulously designed at Apple:

And yes, they cost $29, but if by now you don’t understand the labour and aesthetics that has been brought to you, I can’t help you. This is great design, made affordable to the masses, at the cheapest cost a Capitalist business is able to do so.

Apple consistently produce products that change the world, and their design is key to that (see Ram’s principles above), and they do so at cost margins that absolutely no one else in the industry, not Dell, not RIM and their laughable Playbook, are able to even touch.

Why do they do this? To skin us consumers for money?

No. They do it because they care about the product and its design.

And they believe that should be worth something.

But if you don’t believe design has any value, enjoy reading this review of Dell’s latest morally bankrupt design void that does everything worse than the very thing they are aping.

Introducing the Deskbook

2011-04-25T12:52:00-04:00

I have a fascination with looking at pictures of people’s desks. It’s weird.

A while back, it was all the rage for people to post pictures on their Flickr accounts of their desks and show what they worked on: what type of computer, notebook, desk toys and what not. But not so much anymore.

So I tweeted one night asking if there was such a site in existence where you could just go and browse pictures of people’s desks, partly out of curiosity and partly out of inspiration. John Lepp of Agents of Good replied with ‘desk book?’ and the rest, as they say, is history.

I took the chance of a day off for the Easter weekend break to literally just throw something together and get it up as fast as humanly possible. I bought the domain name the night I had the conversation with John Lepp, and had been hosting a partial taunt to him at that domain as a single page site, so it was purely a matter of building something quick and deploying it to my server.

I naturally chose Rails and S3 to build the site. HTML and CSS3 were literally thrown together without any concern for semantics or best practice. The first version was ready in six hours. There were and are no tests. So at 2am on Friday 22nd April I pushed the thing live and tweeted some folk I know and sent them the URL, packed my bags and got up at the crack of dawn the next day and headed to Barrie, where I’d been invited for dinner by my friend Kimberley MacKenzie of Ontario Nature.

Checking the site briefly on the Saturday morning revealed that I’d forgotten about and oft neglected client – the Google Spider. Fixing this, I re-deployed and got on the Greyhound to Barrie.

Later that night I checked my Twitter to see that Mr Lepp had busted the site open and it had a bunch of new uploads and was all still working! With one flaw – the Like functionality was busted. Oh well. Much beer was consumed in celebration.

On the Sunday morning, I cracked open my laptop, made the fixes and pushed the changes, fixing the issue. Spending a lovely day up at Lake Huron, I packed my stuff and headed back to Toronto on the Greyhound.

On the bus, I started thinking: automated tweets when new uploads are made, Disqus for comments, some Ads to maybe try to cover hosting costs, and a little bit of spit and polish. Cracking open the laptop I quickly started sketching out the changes courtesy of Greyhound WiFi, got home and finished up the rough edges and pushed the changes live. Done. All in I’d say I spent about 8-10hrs on the site.

So check it out – http://thedeskbook.com – and upload your desk!

Parsing the UK National Rail Enquiries Feed at the Command Line

2011-03-15T12:56:00-04:00

The National Rails Enquiry site really annoys me. For some reason, even though I’m not logging in, the developers implemented a session time out across the whole site. I can understand why they might want to do that, but in this case it seems particularly unnecessary.

So, after getting fed up having my initial search query bombed out by the session time out and having to go through the whole station selection process again, I spent some time figuring out how to consume their JSON data feed using CURL, AWK and the deadly SED.

I used the Chrome developer tools to see what XHR calls were being made to populate the Live Departure board which comes across in the HTML as a table with an empty body (SIGH). Then, after discovering the call, I basically treat the returned JSON as a giant string and throw away the useless parts; I could have used jsawk, but I didn’t want to introduce another dependency that might not be on all machines.

If you run the following at mostly any UNIX / Linux command line:

echo "Trains from HYN to GLQ" && curl --silent \ --user-agent "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7" \ --referer "http://ojp.nationalrail.co.uk/en/s/ldbboard/dep/HYN/GLQ/To" \ "http://ojp.nationalrail.co.uk/en/s/ldb/liveTrainsJson?departing=true&liveTrainsFrom=Hyndland&liveTrainsTo=Glasgow+Queen+Street&serviceId=" \ | sed -e 's/[{"&;/"}]/''/g' | sed -e 's/ltbrgt/-''/g' | \ awk '{n=split($0,a,","); for (i=6; i<=n-2; i++) { printf("%s %-25s %-20s\n",a[i+0],a[i+1],a[i+2]); i+=5}}'

You’ll get a nicely formatted display of a live departure board on the command line. A simple lookup of the station codes on the National Rail Enquiries website and a substitution of the values in the above script and you’ll have a custom version for your preferred route. You would need to make sure to update the Title, the referer URL string and the liveTainsJson call.

As a passing note, it’s interesting to note that they went to the bother of having Airport style codes for all of the train stations, and then for the data feed just reverted back to doing a full station name lookup. Also, I’m by no means a bash guru, so if you can improve the script let me know!

Now for extra geek points, if you’re on OSX, grab yourself GeekTool 3, and use the above as a Shell script to have the output permanently displayed on your desktop!

Voila!