Last night was the 2nd Kiandra hackathon & attended by most our software team including BA’s, PM’s and QA’s.
Our hackathons are fun competitive events (with prizes!) and also give everyone a chance to work on a challenging problem that’s probably different to the type of work they would do normally & with people they might not normally work with.
Prior to each hackathon we have been agreeing upon a theme to give direction & get people working in an area they might not normally. Once we have a theme we create a user voice site where people can put up ideas for projects which are then voted on. People can then choose the teams they want to work on with the ideas that interest them.
At the start of each hackathon each participant gets a hackathon pack containing snacks, caffeine & a few silly items that help create a fun atmosphere.
The first hackathon’s theme was touch and gesture. My team worked with the Kinnect api’s to produce an interactive story wall kinda like Minority report – well that was the idea anyway..
Last night’s theme was big data/mashup’s and we had a number of different projects:
- “What is the most manly movie?” (some random title no one had ever heard of apparently!)
- A movie recommendation engine “Netflox” (sorry Cal but the film “free willy” should never be recommended to anyone)
- An analysis of traffic data
- A dashboard showing company information
- A look at the stock market & correlation between stocks rising & falling
These projects had varying degrees of success!
I worked on the traffic data team. Although some of our team members had set up Hadoop prior to the hackathon we decided that as no one on the team had used it before & we only had 4-5 hours our time was better invested concentrating on the actual problem rather than wrestling with Hadoop. We thus decided to import the data into something we were familiar with – SQL server.
In hindsight we should have given this more thought as the dataset was very large (hundreds of gigs) and it took forever to import. We should have setup indexes prior to import as they took forever to create afterwards and without them querying was far too slow.
Luckily some of the other members of our team were more successful in their tasks and managed to divide up the data into grid squares and show visualization based on fake data.
One of the things the hackathon really hammered home to me was all the cool stuff that can & will be done with large amounts of data and parallel/grid computation. A science fiction book I read recently discussed how a city’s traffic system was kept flowing by a super computer monitoring traffic flow – it doesn’t seem so farfetched & we are starting to see some of this type of thing already e.g. see Audi’s traffic light detection system.
I look forward to Kiandra hackathon 3!