XSS (Cross Site Scripting) Part 1 – What is XSS?

I’ve been interviewing web developers recently and was surprised that the majority did not know what XSS (Cross Site Scripting) is.

Perhaps part of this is that many web developers will begin their development career using a framework such as React or Angular that abstracts what’s actually going on.

Many frameworks will now also provide sensible defaults that can help prevent many XSS issues and you have to go out of your way to get round these for example React warns you that you are probably about to do something potentially silly with its wonderfully named dangerouslySetInnerHTML method.

However its still very possible despite some of these protections and defaults to have XSS issues so if you are developing for the web it’s important you understand what XSS is and how to prevent it.

XSS is a surprisingly big topic (and I certainly learnt stuff putting this series of blog posts together) and we’ll be going deep into this topic so I’m going to split this up into what’s hopefully a series of more digestible articles rather than a 30,000 word mega post.

In this series of posts, I will:

  • Introduce XSS and talk about its different types
  • Talk about what an attacker could do with XSS
  • Show an example using my demo app of how XSS could lead to compromised accounts and even RCE (Remote Code Execution) when combined with other issues
  • Look at ways to mitigate or prevent XSS
  • Review some of the thousands of ways XSS can occur and how attackers circumvent various protective measures and what to do about it


As with any offensive security topics its important you do not use these methods and techniques on sites you do not own or have permission to test on.

Trying some of the techniques we’ll discuss on sites you don’t have permission to is almost certainly illegal and you could end up in a lot of trouble or even jail.

There’s plenty of great labs to explore security topics such as hackthebox, tryhackme or bug bounty programs where you can learn and help make stuff safer which makes the world a better place.

These are all much better options than a criminal record/jail time..

Hack The Cat (vulnerable Node/Express application for learning/teaching appsec)

I wanted an application to play, learn and teach some of these issues so I’ve been developing a deliberately vulnerable Node/Express application called Hack The Cat that contains many different security issues (below):

Whilst early in its development I’ve added various common security issues including:

  • Various XSS vulnerabilities
  • SQL Injection
  • Serialization issue
  • File upload with naïve filter

My plan is to expand upon this and also add the following:

  • SSTI (Server Side Template Injection)
  • More advanced XSS filters
  • LFI & RFI (Local/remote File Inclusion)
  • OS command injection
  • JWT issues

Anyway i’ll release this shortly with a MIT license so its free for all but let’s talk about XSS.

What exactly is XSS?

XSS or Cross Site Scripting is one of the most common web security issues and occurs when a website accepts malicious client-side code or HTML and then executes it in the context of the user who is visiting the site.

A simple example of XSS would be a shopping site that allows users to post reviews of products and then renders the reviews submitted.

If an attacker can submit a comment containing a script block like say below..

<script>alert('hello you have XSS issue you'll need to fix up')</script>

..and the browser renders it without encoding this then you’ll be greeted with an annoying smug alert box.

In fact if right click/view page source on this blog post and search for “hello” in the source code you’ll see something like below:

Here, WordPress has thankfully properly encoded this code block – if they didn’t you’d all be seeing an alert box as it would run as if the original site developer had added it.

If you want to see just how common XSS issues are and how they can be used go to https://www.exploit-db.com/ (a site that contains a massive index of security issues and code for utilizing them) and search for “xss” – wow quite a few issues..

XSS was apparently “discovered” by the Microsoft Security Response Centre and the IE Security team who heard about attacks where script and image tags were injected into web pages. In 2000 they published a report in conjunction with CERT where the term Cross Site Scripting was first used.

Whilst popping up an alert box and taking your users back to early 2000’s web/common current debugging techniques is certainly annoying an attacker could do much worse and this can be used to devastating affect.

For this we need to get into cookies and sessions.

Cookies and Sessions

XSS can become especially destructive due to how web site sessions & cookies work.

Cookies are pretty important and one way of holding state about a visitor. Cookies are one of the main mechanisms websites use to identify individual user on a website and understand if a user is logged in.

When I log into many sites the site will issue me a cookie to identify me and that I should be able to access the site.

Now the thing about cookies is that they are sent with any request made from a site or “origin” (there are a few exceptions such as using fetch which doesnt send cookies by default and certain configurations we’ll talk about later).

Now this can be a big issue as if an attacker can get code to run on a site as we know that cookies are being used to identify users then the site may think the request is from a legitimate user!

Same-origin policy

Before we move on I just wanted to mention same-origin policy as this relates to cookies and will also prevent or make harder some types of attack.

This can get quite complex so here’s an overly simplified version.

Same-origin policy places restrictions on what a document or script can do when a page is loaded from one origin and interacting with another.

A site is considered as having the same origin if the following match:

  • Its protocol (http/https)
  • Port (if specified – note weirdly IE ignores the port bit because er IE)
  • host (the address bit – it’s more complex than this but anyway)

The following for example would be considered to have the same origin:



Now this is important as if an attacker is running a script on another server (origin) then they wont be able to do certain things (at least directly) like steal cookies. This is a good thing.

Ok enough theory lets talk about different types of XSS.

Different Types of XSS

XSS comes in several different and overlapping ugly flavours you should be aware of:

  • Stored XSS
  • Blind XSS
  • Reflected XSS
  • DOM Based XSS

Persistent/Stored XSS

In Stored XSS the attacker manages to save their horrible XSS payload somewhere (most likely a database as they are good for this kind of thing unlike er blockchain which really isnt very good at anything).

The bad news with Persistent/Stored XSS is that once malicious code is stored it could potentially be executed for thousands of users who don’t have to do anything but visit your site where the code is rendered which can give it a massive reach.

I guess you’d also have to clean up the stored malicious content at some point in your database which could be really annoying if an attacker automated the creation of thousands of comments.

Blind XSS

Blind XSS is a form of Persistent/Stored XSS but has the difference that the attacker is not sure when the code will actually execute and if at all.

Er what?

Imagine an application that allows the sending of a contact form message to admin users.

When an admin user views this message then the malicious code will then execute in the context of the admin user. This malicious code could then be used potentially to perform administrative functions, steal the admin’s session cookie etc as its executing in the context of the admin user.

In Blind XSS the attacker may not have access to the code base/application so they cant be sure their code will ever actually execute.

An attacker might send code that will “ping” a remote server or endpoint by perhaps requested an image and then monitor to see if a blind xss attack was successful. They could then perhaps send more information about the page its executing on.

Reflected XSS

Reflected XSS occurs when the XSS payload is not actually stored but is “reflected” back to the user.

A site may have a search function that allows users to search for products and then sends the search values unencoded in the query string e.g. simpleisbest.co.uk?search=<script>something bad</script>

Attackers could potentially send users links like the above that are pointing at a legitimate site that then executes their malicious code.

An attacker would likely take care to obscure their payload more carefully than the above example and this could be obscured with various methods e.g. JavaScript’s String.fromCharCode method and a hellish number of different encoding schemes that most browsers support.

It’s important to note that reflected XSS requires a user to perform an action (e.g. click the link) and may be avoided by more knowledgeable users or automated systems that scan emails for malicious content.

Reflected XSS attacks are likely to have limited reach compared to persistent/stored XSS.

DOM (Document Object Model) based XSS

In DOM based XSS the attacker finds a way to inject a payload into the pages DOM without modifying the HTML directly.

A site may after login put the users name in a query string to display a welcome message that is then output on a page e.g. simpleisbest.co.uk?username=Alex

If the receiving code doesn’t encode the output properly then an attacker could supply a malicious bit of code in the username parameter that would then be output and executed on a page.

DOM based XSS can be harder to find/scan for as the attacks never reach the server itself.

So what’s the big deal about XSS?

Whilst using XSS to pop up an alert box or maliciously redirecting users to Michael Buble’s site could be very annoying XSS can be devastating when chained with other attacks.

Depending on other protections & configuration in place we’ll talk about in the future XSS can be used to do pretty much anything a user on that page could do as code will execute in the context of that users browser.

This means of course that the code can:

  • Transverse and manipulate the pages DOM
  • Read users cookies, local and session storage (depending on other protections in place)
  • Send requests to backend APIs in the context of the current user even if not used on the current page
  • Redirect the user to other sites/applications
  • Modern APIs have access to the camera, geolocation, microphone and certain files (these will require user to grant permission if they haven’t already)

And any other endless possibilities – basically anything you could do with client side code in the context of the current user

This means an attacker could:

  • Harvest and steal session cookies (depending on other protections in place) allowing them to act as if they are the user logged in until the cookie/token expires or is revoked
  • Perform actions in the context of the current user as the browser will send cookies with any requests made e.g. change their password to something known
  • Transverse & manipulate the DOM to pull out sensitive information
  • Manipulate the DOM and trick users into performing certain actions or reading wrong information
  • Redirect the user to other sites e.g. a site hosting malware or Michael Buble’s music
  • Defacing a site and replacing it with other content (hopefully not Mr Buble’s music)

Ok so hopefully I’ve convinced you XSS can be a pretty big deal.

In the next post i’ll show an example with my demo application of how it could be used to gain control of an admin account and even an entire server and then we’ll look at how we can fix this up.

Further reading




What is Project Spartan and is IE dead?

It is naughty & generally a bad idea to isten to rumours but around September 2014 & again early January 2015 you may have heard that Microsoft was developing a new browser with the code name Project Spartan.

However it wasnt until 21st of January 2015 at a Windows 10 preview event that Microsoft confirmed the existence of this browser and gave us our first peek.

Oooo but most importantly what does it look like?

Well we can’t get our hands on it yet so we are reliant on demos and Microsoft blog articles. Here is a picture I have taken from the IE blog of Project Spartan’s UI to give you some idea:


A nice clean simple UI that bears very little resembalnce to Internet Explorer.

Tell me about the new features in Project Spartan..

Well it is still very much in development but here is a summary of some of the new features we know about so far compiled from various blogs, the Window 10 video and tweets from IEDevChat:

  • A nice new clean UI that bears little resemblance to IE’s
  • Ability to annotate & comment on web pages (and PDF’s!) to highlight issues/impress your co workers with your cat drawing abilities
  • Spartan can save content for later viewing offline. This feature is also integrated into Windows reading list application & I assume can sync between devices
  • Reading view – a distraction free version of a page optimized for the reading of content. This feature was actually first introduced in the Metro version of Internet Explorer however you probably never noticed unless you had a Surface device as Windows Metro mode was er pretty horrible
  • Integration with Microsoft’s digital assistant Cortana. In the launch event a demo was shown where the user started searching for weather in the address bar & the browser showed a weather forecast specific for that users location.
    A more complex example was also shown where Cortana was aware the user was tracking a particular flight & automatically pulled flight status details when the user searched for the flight/airline
  • Updated F12 developer tools – you can see some of these great new features in the latest version of IE11
  • Fixes for over 3000 interoperability issues
  • Possibility of an extensibility model (or perhaps more accurately one you’d actually want to use!). @IEDevChat tweeted “We’re working on a plan for extensions in Project Spartan. Stayed tuned!”  and “Still building our plan for extensions, but we’ll share more once we have details”. Cant wait to see what they have planned & I hope it will be similar to Chromes model
  • Microsoft say they are working on implementing over 40 new web standards including support for accessing the dom using XPath syntax, Web Audio, Media Capture API, WebRTC 1.1, Touch Events, Content Security Policy, HTTP/2
  • Apparently IE preview has the best support for ES6 features at the time of writing
  • A new User Agent String containing various other browser identifiers aimed at making sites work that would previously have excluded IE by user agent sniffing (never a good practice)


Above Project Spartan’s annotation features shown at Windows 10 event

Spartan sounds interesting how can I play with it now?

Not quite. You can’t quite yet play with the new UI/annotation features as they are still cooking but you can see the new rendering engine in action by downloading a preview of Windows 10 at http://blogs.windows.com/bloggingwindows/2015/01/23/january-build-now-available-to-the-windows-insider-program/ & enabling Edge mode by opening up the browser and going to about:flags.

So is this the end for IE?

Jacob Rossi says “Spartan is the browser we expect people will be using on Windows 10”.

So for the majority of consumers in the future er kinda yes..

However this is not the full story.

Before I answer this we need to understand that browsers are actually made up of many individual components and subsystems. In fact it’s probably better if you head over to the IE blog here and you can learn all about what makes up Internet Explorer from the people that actually create this stuff – go on i’ll wait.

Internet Explorer in its rendering related system(s) contains a huge amount of code dedicated to providing support for legacy versions of IE all the way right back to IE 5.5.

As you can imagine maintaining this became a bit painful (and does anyone really need support for weird IE5.5 rendering issues – no you don’t) & can slow progress. With Spartan Microsoft decided to draw a line in the sand and based Spartan’s rendering engine on IE11’s standards support and went to work throwing out the older legacy rendering stuff (and a few other items such as vbscript that no one not even its mother will miss).

So in short Spartan is built upon the rendering & JavaScript engine of IE11 (and I guess some other components) but without the old legacy stuff.

The result is a great standards compliant platform with an awesome JavaScript engine to build upon going forwards. Microsoft are referring to this new rendering engine as EdgeHtml.dll.

But what about backwards compatibility? glad you asked let me tell you about Spartan’s dual engine mode..

I work for a bank/government agency and am worried about backwards compatibility!

To support legacy applications Spartan operates in dual engine mode and transparently switches (it is unclear what triggers this) between using the new engine & the old Trident/MSHTML engine for legacy sites.

Internet Explorer will also be made available on Windows 10 to support those guys that have built applications on top of IE’s extensibility models toolbars/ActiveX etc (you poor brave fools!).

It looks like it is still to be decided if IE will be hidden away in order to encourage Win 10 users to move to Spartan. For example in a reply to a tweet regarding if IE will be part of Windows 10 @IEDevChat replies that “We’re evaluating different approaches, but our intent is for ppl to use Spartan unless they have specific compat needs”.

Can you force EdgeHTML rendering on intranet sites?

Yes. “You can get EdgeHTML on intranet using x-ua-compatible if intranet sites are set to use compat view.”

Why can’t Spartan/IE just use WebKit/Blink as its rendering engine – my life would be much easier?

Well maybe but having no completion/a mono culture is a very bad thing for a number of reasons.

Christian Heilmann (probably best known for his work at Mozilla but now working for Microsoft) answered a comment on Jacobs article similar to this with:

“The fact of the matter is that in order to have a standards based web, we have to have different browser engines to test standard proposals against. Having a WebKit only web would be as disastrous as having an IE6 only web or a Gecko-only web. Any engine that has no competition will deteriorate over time – no matter if it is open or closed.”

And I would certainly agree – competition and different is good and drives the web forward & stops one organisation having control over the webs direction.

Will Spartan auto update?

Yes Spartan is “Evergreen”.

Which operating systems can I use Spartan on?

Initially Windows 10 & Windows Phone 10. Remember that Win 7 updwards users will be able to upgrade to Windows 10 for free so there is really no excuse unless perhaps you are in Corporate world. Microsoft said regarding potential cross platform support in the future: “No current plans, we’re focused on making it great for Win10.”  and “Spartan is currently targeted at Win10. We’re focused on getting ppl upgraded (free) but will watch Win7 demand”

Will Spartan use vendor prefixes?

It will likely still contain some but these will be discouraged. “EdgeHTML engine will have a lot of ms prefixes removed; we want to use prefixes very sparingly if ever”.

Were there perf gains from removing the old rendering stuff?

“Yes. The disk size of the engine itself is decreasing and general web page performance has improved by various metrics, including JS perf. Preview builds are notoriously not our best foot forward with perf though, but you should see good things already and we’re not done yet.” Jacob Rossi comment http://www.smashingmagazine.com/2015/01/26/inside-microsofts-new-rendering-engine-project-spartan/

Which IE engine will web driver use?

The new Edge engine. John Jansen in a comment at http://www.smashingmagazine.com/2015/01/26/inside-microsofts-new-rendering-engine-project-spartan/ said:

“We are definitely committed to the Edge engine with WebDriver. We are still working out details about how it must work, but it should never randomly switch between them for no apparent reason. I expect that you will be able to test your internet sites or intranet sites in ways the very closely mimic the actions a real user would see and do”.

Upon release will the browser still be called Project Spartan?

If history is anything to go by Microsoft Marketing or whoever names MS products has a consistent ability to remove cool names so probably not.

I am a site developer what does this mean for me?

You should continue to develop your sites and applications using current standards and everything should be marvellous.

Will Spartan be integrated as closely integrated with Windows as previous versions?

Personally I hope not & in an ideal world I would like to see it possible to run Spartan entirely off a USB stick. I have my suspicions that Spartan may have some deep links given it integrates with Cortana, Sharing of comments, Reading list etc but guess we will have to wait and see how this will work.

Additionally in previous versions of IE many products took dependancies on IE components e.g. Outlook utilized it for rendering HTML email. Having these dependancies may have slowed development as the IE team would have had to be careful not to break these applications reliant on these components.

It is also worth noting that as Spartan updates automatically updates should not be quite as big a deal as they were previously so maybe if there is a close Windows integration it wouldnt be such a problem.

What new HTML/CSS/JavaScript features will Spartan have?

Head over to staus.modern.ie for the most up to date view.

When will it be available to use?

No idea.

Further reading/watching

You can watch the full preview event at http://news.microsoft.com/windows10story/. Note if you just want the Spartan related stuff fast forward to an hour in 🙂



This is a series of tutorials on WebGL using the three.js library – you may also be interested in Part 1 

In my previous post we discussed the history of WebGL and looked at a number of libraries to make working with WebGL as easy as possible.

We will be working with the three.js library so the first thing you will want to do is head over to http://threejs.org/ and download the latest version of three.js.

There is a little download link on the left hand side of the page you want to click on to which will give you a zip file containing the latest three.js library and a heap of examples.

Let me start by saying that Three.js is an awesome project, it has a team of very active developers, a big community and some great examples – there is a big but coming however..


Three.js’s documentation sucks. Really sucks – in fact it sucks more than White house down which is the worst movie I have seen this year. I’d love to help the three.js guys fix up the documentation but just don’t have enough knowledge to assist in this area yet so will just whinge (and warn you) about it right now..

It is worth mentioning that three.js does have an enormous number of examples available which is arguably better than documentation anyway. However I found that its not that obvious how some of them work as there is little in the way of explanation of some of the maths involved. This coupled with bad documentation can make for a frustrating process of trial and error to work out how to use the various features. I found a collection of some more readable examples at http://stemkoski.github.io/Three.js that you may want to check out.

So before we get any further I want to save you a lot of time & effort & recommend a book to you – Chris Storm’s 3D programming for kids.

Don’t worry I wont tell anyone you downloaded a programming book aimed at kids (I do have my doubts how many 10 year old’s will cope with matrix projections but I digress..).

Don’t let the title put you off – this is by far the best introduction I have read of three.js and contains a number of very useful examples that will save you heaps of time working out how to do common tasks such as animating a group of objects, move a character around & collision detection. This book also has a very readable explanation of some of the maths involved in 3D programming – its also apparently used as a university text in an introduction to 3D programming course so don’t feel too bad..

In our first example we will draw a wire frame sphere on the screen.

Why? – well why not and we’re not going to be coding GTA V in our first example.

The end product will look like this:


Start off by creating a new html page – note you don’t need jQuery but it will save some typing later on:

<!DOCTYPE html>
<title>Hello three.js</title>

<script src="three.js"></script>
<script src="jquery-2.0.2.js"></script>
<script src="helloExample.js"></script>



Now create a JavaScript file called helloExample.js and add the following code:

(function() {

"use strict";

var camera,
scene = new THREE.Scene(),
renderer = new THREE.WebGLRenderer(),
aspect = window.innerWidth / window.innerHeight,
camera = new THREE.PerspectiveCamera(75, aspect, 1, 1000);

camera.position.z = 300;

light = new THREE.PointLight(0xFFFFFF, 1);
light.position.set(0, 0, 200);

sphereMaterial = new THREE.MeshBasicMaterial({
color: 0x000000,
wireframe: true,
wireframeLinewidth: 2


sphere = new THREE.Mesh(new THREE.SphereGeometry(100, 16, 16), sphereMaterial);

container = document.createElement('div');
container.id = "container";

renderer.setSize(window.innerWidth, window.innerHeight);
renderer.render(scene, camera);


So what’s going on here?

Well before we run through the code let’s recap on how coordinates work. We will need a way of telling three.js where we want objects on the screen.

Three.js’s coordinate system looks like this:


So there are 3 axis’s that allow you to position objects on the scene. Three.js also contains some scale methods to make working with different size objects easy that we will use later on.

I think this is fairly straight forward & familiar to most programmers but dont worry this will quickly become complex later on when we want to point the camera at a specific object for example 🙂

OK now we have this foundation stuff out the way let’s look at the code.

First up we create and initialize a number of objects we will make use of later:

var camera,
scene = new THREE.Scene(),
renderer = new THREE.WebGLRenderer(),
aspect = window.innerWidth / window.innerHeight,

We then create a new PerspectiveCamera (there is also another type of camera called OrphographicCamera which doesn’t render perspective but don’t worry about that for now) that tells three.js where we are looking at:

camera = new THREE.PerspectiveCamera(75, aspect, 1, 1000);

The first parameter (75) refers to the vertical field of view of the camera (FOV), the second represents the aspect ratio of the screen and the 3rd and 4th parameter tell three.js we want to see objects between 1 and 1000 z position – this can be important for performance (why render stuff really far away?) and certain effects. Hmm wtf does all that mean? Well Jeff Atwood (coding horror) has a good diagram & explanation of FOV & aspect ratios that you should check out.

Next up we move our camera nearer to us (e.g. back) by changing the z position. We then add it to our scene:

camera.position.z = 300;

It’s important with three.js to remember to add a light to a scene otherwise we won’t see anything so lets do that:

light = new THREE.PointLight(0xFFFFFF, 1);
light.position.set(0, 0, 200);

There are a number of other types of lights and you can also play with the colors to create some great effects.

Next up we create a material (kind of what we will paint our object with).

In this example we will indicate that we  want a wire frame version of the object as we are hipster like that:

sphereMaterial = new THREE.MeshBasicMaterial({
color: 0x000000,
wireframe: true,
wireframeLinewidth: 2

Now up lets create our sphere by creating a mesh (think of meshes as a scaffold for the object based on a collection of x,y & z points – called Vertices in 3D speak) based on one of three.js’s primitives – SphereGeometry:

sphere = new THREE.Mesh(new THREE.SphereGeometry(100, 16, 16), sphereMaterial);

The 100 specifies the size of the object and the 16’s how many parts or segments make up the sphere – more parts = smoother sphere but potentially more memory to store object & slower to render.

Finally we need a way of getting this onto the page itself so lets create a div & append our renderer we created earlier to it:

container = document.createElement('div');
container.id = "container";
renderer.setSize(window.innerWidth, window.innerHeight);

This will all get rendered in a canvas element that three.js will create for us inside the container div.

Finally we need to tell three.js we want to render our content:

renderer.render(scene, camera);

And we are done!

Although there is a fair bit of code most of its around setup and you will need something similar each time so you will quickly get used to it.

Next time we will look at animation!

Intro to RxJS

RxJS is the JavaScript version of a .net library that amongst other things makes it easy to handle, combine and filter events.

I gave a talk about RxJS last week – my slides and code examples are available here.

On a recent project a colleague (Hendry Luk) suggested we utilize RxJS to solve a tricky issue we were having. We were developing a web based calculator to be used by financial consultants. The calculator contained a number of fields the consultants could modify to then see the effect they would have on their clients income. The client wanted the output to be automatically updated when any changes were made. Our initial implementation felt a bit clunky as we bound to every key press, select box change etc. Thus when a user was typing into a textbox a request was made on each key press and the output rebound.

There is obviously a few ways to solve this but RxJS made it very easy. RxJS allowed us to throttle this process and ignore out of date values e.g. if a value was changed before a pending response had returned.

For me I think getting started with Rx is probably the hardest part as there are currently few very basic examples around and it can be a bit confusing whats actually going on.

Let’s create a simple hello world example to demonstrate some of the main concepts.

We will create a simple web page containing a single button that does the following:

  • When the button is clicked we will ignore the first 2 clicks
  • We will display an alert for clicks 3 and 4
  • After clicks 3 and 4 an alert box saying “done” will appear

Let’s get started, first create a new html page called helloRx.htm.

RxJS has many different libraries (see above section) that offer rich functionality such as extensions for working with jQuery, testing & combining events but for our example we just want the core library (rx.js) so let’s import this:

<script src="rx.js"></script>

Now we want something that will trigger an event (an observable in RX terminology) so add a button to the page:

<button id="button">Click me!</button>

Now we need to turn the buttons click event into something RX can work with so we will create a function that wraps the Rx.Observable.create method (note the rx.jquery.js library allows you to do this in one line with methods such as clickAsObservable):

function CreateObservable(element, eventType) {
return Rx.Observable.create(function(observer) {
function eventHandler (eventObj) {
//keep simple for this example and ignore addEventListener/attachEvent browser differences
element.addEventListener(eventType, eventHandler);
return function() {
element.removeEventListener(eventType, eventHandler);

Now we will use this function to create an observable from our buttons click event. We will also use the methods skip and take on our observable to skip the first 2 clicks and tell rx we are only interested in the next 2 clicks.

var observable= CreateObservable(document.getElementById('button'), 'click')
return "button was clicked"

Next we need to create an observer to monitor our observable:

var observer = Rx.Observer.create(
function(evt) {
function(err) {
function() {

Note how you can (optionally) pass 3 functions into Rx.Observer.create method. The first (onNext) handles an event occurring, the second errors and the third completion of an observable sequence.

Finally we need to link (or subscribe) the observer to our observable with a call to the observables subscribe method:


Now run this in the browser and congratulate yourself for getting started with RxJS!

Whilst this wouldn’t be too hard to accomplish with traditional methods you can start to get an idea of the power and readability offered by Rx.

Thoughts on developing resilient Ajax code (Part 2 – ASP.net MVC)

In my last post we looked at different methods to indicate a call to a service that would return JSON was unsuccessful.

The conclusion was that we really want to be setting the HTTP status code of the response to indicate a request was unsuccessful.

In this post we will look at some possibilities for wrapping this logic up in ASP.net MVC controller methods (and there are many).

We want a way of:

  • Setting the status code of the response
  • Including in the response our errors formatted as JSON
  • Something that’s easy to reuse and maintain

The most obvious (and least reusable) option is probably something like:

Response.StatusCode = (int) HttpStatusCode.InternalServerError;
return Json(new { data = "bad error" });

We could also use the HttpStatusCodeResult class however this only allows us to set the description of the error status code rather than the body of the response:

return new HttpStatusCodeResult(HttpStatusCode.InternalServerError, "bad error");

We could remedy this with something like this:

Response.Write("bad error");
return new HttpStatusCodeResult(HttpStatusCode.InternalServerError, "bad error");

But we also want to format our response as JSON and set the return content-type to application/json so would then have to do something like this:

var errors = new List<string>() {"Bad error", "Another bad error"};
var serializer = new JavaScriptSerializer();
Response.ContentType = "application/json";
return new HttpStatusCodeResult(HttpStatusCode.InternalServerError, "bad error");


This is getting a bit messy & we want something easy to use, maintain & test.

ASP.net MVC is very flexible and we have a number of different ways to do this but I reckon inheriting from JsonResult works pretty well for our purposes:

public class JsonErrorResult : JsonResult

private readonly HttpStatusCode _statusCode;

public JsonErrorResult(HttpStatusCode statusCode)

_statusCode = statusCode;


public override void ExecuteResult(ControllerContext context)
context.HttpContext.Response.StatusCode = (int)_statusCode;


With this new type of ActionResult we get all the functionality of JsonResult and can also set the status code of the response easily:

var errors = new List<string>() {"Bad error", "Another bad error"};
return new JsonErrorResult(HttpStatusCode.InternalServerError) {Data = errors};

If we use browser tools we can see the Http status code was set correctly to 500, the content type is application/json and the response body contains our json encoded messages for later display:

1-12-2012 1-53-19 PM

Another option is to create a custom Action filter (this can be shortened by returning a JsonErrorResult as above):

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, Inherited = true, AllowMultiple = false)]
public class HandleErrorAsJsonResponseAttribute : ActionFilterAttribute, IExceptionFilter

public void OnException(ExceptionContext filterContext)

if (filterContext.HttpContext.Request.IsAjaxRequest())

filterContext.HttpContext.Response.StatusCode = (int) HttpStatusCode.InternalServerError;
filterContext.ExceptionHandled = true;
filterContext.Result = new JsonResult
Data = new { errorMessage = filterContext.Exception.Message}



This is very easy to apply (and can be done so at method, controller and even application level):

public ActionResult ThrowError()
throw new Exception("bad exception");

I think these serve my purposes pretty well but if anyone’s got better ways of doing this I would love to hear them 🙂

Next up what’s jQuery actually doing in $.ajax

Thoughts on developing resilient Ajax code (Part 1)

This sunny afternoon I was writing an ASP.net controller method to return JSON data from a calculation service.

As we all know things can always go wrong in code (and not necessarily just because I wrote it!) and this got me thinking about best practice when making and handling Ajax calls.

There are a number of issues we need to think about if we want to avoid the terrible situation of our users not being able to buy 1000’s of those new silver cat widgets we have been flogging because a vital Ajax shopping basket related request failed such as:

  • Errors encoding our request & its body before we have even sent it – maybe the users entered something dumb (did you let them? or are they a hacker and up to no good)
  • Timeout’s & a user’s session ending – e.g. the user felt the need for coffee and got caught up in a discussion about how would win in a chainsaw fight between the neighbours and home & away cast (dodgey Australian tv series)
  • Server side errors (you probably screwed something up)
  • Issues reading/decoding the response (see above)
  • Transient network conditions – those times things fail and no one’s really er sure why as it works if you try again

Mobile web applications in particular are very vulnerable to transient network conditions. I started (and forgot about) building a wrapper to retry requests which ill get round to looking at again sooner or later (https://github.com/alexmackey/retryajax).

Ok so what do we mean by resiliency?

Well to me in this context it means:

  • Having a service that works almost always
  • Something that can recovers from temporary errors such as timeouts without crashing the whole application
  • ..or failing the ability to recover provides sensible information about what might have gone wrong (and if there’s anything a user can do to remedy/help)
  • Handles a decent number of requests
  • Ideally it would also be really really nice if we had some info about what went wrong and if something went wrong so we could fix it

As you can see there are a decent number of issues here and some of these aren’t as straightforward to solve as they  may first appear so I am going to first look at how to return a valid/invalid response at a high level.

I guess there are three main (and combination of) ways to handle this:

  • Set a relevant HTTP status code on the response and potentially populate the body with some details of the error
  • Return a standard HTTP 200 everything’s peachy response and set a status code in the responses (true/false & success/fail seem in pretty common usage)
  • Don’t do anything

A lot of websites actually do the third option and assume that every AJAX call is going to work out fine and dandy (have a look in your browsers network tools & JavaScript console next time you are bored). I think we can all agree isn’t a good idea unless it’s a particularly trivial request.

Anyway since there were a variety of methods in use I wondered if people had a preference for how this was done so put this out to Twitter:

“If you were calling service to get JSON formatted data & error occurred would you prefer http error code to be set or 200 & error in json?”

Of course this being Twitter and the fact that it’s always dangerous to ask for peoples opinion I got a mix of useful and not so useful replies including:

The general consensus was that appropriate Http status code’s should be set before returning the response.


Well this approach has a number of advantages such as:

  • Http status code integrate nicely with various JavaScript libraries error handling routines like jQuery’s global .ajaxError and error methods
  • Its potentially easier for people to integrate with our service as its nice, descriptive & potentially RESTful in nature
  • Its easy to spot error status codes in Fiddler/browser tools/logs which helps if something goes wrong

And I would agree setting a status code is probably the best approach.

Can this lead to some ambiguity?

Let’s say we had a method that allows the retrieval of an individual entity by Id. We call it and get a  404 Not found – is my requested entity missing or do I have the wrong URI?

Or how about  we get a 401 (Unauthorized) were we unauthorized to call the Api or have we screwed up our security configuration?

Well I don’t think these are going to be too tricky to deal with as we can always include additional information in our response, status codes are a pretty accepted way of doing things and I think the benefits outweigh any confusion so let’s stick with using Http Status codes.

Now let’s look at how to set these codes in ASP.net & ASP.net MVC.

ASP.net MVC & ASP.net offer numerous ways (ASP.net Web API has more) to set the HttpStatusCode of a response (note there is a nice enum of the various codes called HttpStatusCode) including:

Response.StatusCode = 404;

throw new HttpException(404, “Um where’s it gone”);

return HttpNotFound();

return new HttpStatusCodeResult(404);

There are however a couple of potential pitfalls to watch out for when using with ASP.net & ASP.net MVC as certain error codes will trigger behaviour in IIS/ASP.net MVC framework which might not give you what you would expect:

  • Returning 404’s (not found) will kick in IIS’s 404 page without disabling this Response.TrySkipIisCustomErrors = true;
  • Http 401’s (unauthorized) depending on your authentication setup will send you off to the login page (aspnet.suppressformsredirect NuGet package apparently will prevent this)

Well none of these are too tricky to deal with so I am not overly concerned by these and they shouldn’t stop you from using HttpStatusCode to indicate the status of your request.

So in conclusion:

  • Use Http status codes they work well with libaries, help debugging and are an accepted way of doing things
  • Be consistent – a mix of approaches is really bad news

Next up – implementing this in ASP.net MVC..

How accurate are IE developer tools for testing previous versions?

Most web developers will hopefully test their sites and applications in a variety of browsers. It’s probably fair to say that the older versions of Internet Explorer in particular can cause even the most experienced dev to tear their hair out with their differences in rendering behavior and implementation of various DOM APIs so it’s always been particularly important to test a site in older versions of the big blue E.

I think IE 9 & 10 have done a pretty good job of keeping to standards so I don’t think this will be such a big issue in the future (that’s not to say you shouldn’t test these versions) but there are still a heck of a lot number of people using IE & and IE 7 (yes that’s your parents and big corporate’s with huge hulking Windows XP based networks) which means testing these previous browsers is still important – I don’t think my clients would be too happy if a significant portion of their customers couldn’t access their site or application.. 🙂

IE dev tools and testing previous versions

Since version 8 IE has contained an integrated set of development tools (available as a separate download for earlier versions) which amongst other features offer the ability to simulate older versions of the IE. This can give you a nice quick way to test how your site will function all the way back to good old IE 7.

To access this functionality hit F12 or right click inspect element on a page.

The below screenshot shows the development tools in IE 10:


But wait what is the difference between Browser and Document mode and more importantly how accurate a simulation is this of earlier versions of IE?

First let’s discuss browser & document mode..

Browser mode and Document mode

Document mode is important as changes how a browser renders a page, processes CSS & JavaScript and (in Browser modes case) how a browser identifies itself to the web server.

Determining the mode used to render a page is quite a complex process – see http://blogs.msdn.com/b/ie/archive/2010/03/02/how-ie8-determines-document-mode.aspx.

The following can influence the document mode a page is rendered in:

  • Browser & Document mode setting in the dev tools
  • Conditional comments and HTTP headers
  • Compatibility view list
  • Doctype (or lack of)

You can force IE to render a page in a particular document mode by using special meta tags or setting certain HTTP response headers. This might be useful for example in telling IE how to render content in a legacy intranet application where you don’t have control over the source and it displays better in IE7 quirks mode.

It can seem a bit un-intuitive & confusing that IE dev tools has two separate modes (which do overlap in functionality) but this split allows you finer control over how a page is displayed, the ability to override browser defaults and to test how a page would display in scenarios such as compatibility mode.

Compatibility mode is a way of emulating earlier versions of IE so a page displays properly – Microsoft maintain a list of sites that should be rendered in this mode and you can add sites to your own local version of this list in compatibility view settings.

OK back to our testing question let’s look at the differences between document and browser mode:

Document mode

  • Does not modify the user agent string that is sent to a web server
  • Determines how mark-up will be rendered
  • Will re-render a page if changed in dev tools but won’t make another request to a web server

Browser mode on the other hand:

  • Modifies the user agent string that is sent to a web server
  • Changes how conditional comments in html will be evaluated
  • Determines the Document mode that will be used by default (see above)

Probably the main difference as far as we are concerned is the user agent string that is sent and the defaulting of document mode. With default browser settings the user agent string that is sent to a web server from my Windows 8 machine and IE10 is:

Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)

If I set just the document mode to IE 7 then the user agent string will not change although a different rendering mode will be used.

If however I set the browser mode to IE7 then the following UA string will be sent:

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; .NET4.0E; .NET4.0C; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729; InfoPath.3)

You can try this by using the network tab and examining the user agent header.

User agent

Some frameworks (ASP.net for example) and some (cough.. badly written) JavaScript use the user agent string to present a different experience for users so for the closest simulation you probably will want to ensure you use the Browser mode setting so the correct user agent string is sent and then check the document mode is set correctly for the scenario you are testing.

For example if you want to test a page with a doctype in IE7 standards mode you can set the browser mode to IE7 (the IE7 user agent string will now be sent) and ensure the document mode is set to standards mode.

If you want to test how an IE7 user viewing it in compatibility mode would view the page you can set the browser mode to IE7 and the document mode to Quirks.

OK so we understand the importance of the browser & document mode but we haven’t answered how IE simulates previous versions yet so let’s look at browser sub systems now.

 Browser sub systems

Your average browser is a complex beast and is made up of a number of subsystems such as:

Rendering engine

  • JavaScript engine
  • Network engine
  • Security systems
  • And many others..

It will come as no surprise to learn that different versions of a browser have different versions of these components.

So does IE contain previous versions of individual subsystems to enable you to simulate earlier versions?


..however depending on the component in some cases kind of – it depends on the subsystem (and no I sadly don’t have a list).

One of the big things that is not simulated when you use the IE dev tools to simulate previous versions is security related changes. This makes sense because it would be a really bad thing if you could force the browser into an earlier security mode that contained various potential system compromising flaws. This means you will never get a 100% accurate simulation of how an earlier browser would process a page although this probably won’t affect the majority of scenarios users will be testing.

There are some components that are completely different in newer versions of Internet Explorer such as the JavaScript engine (the latest is called “Chakra” to its friends) that try and replicate older bugs & hide unavailable functionality when earlier versions are simulated which of course may not be a 100% accurate simulation.

However there are also some other components such as the DOM selection engine that although a later version still also contain the older code so it can be simulated meaning you are probably getting a fairly accurate simulation of how something might have worked previously.

Of course if every version of every component was shipped with each version of IE it would be a massive install, the maintenance would be a nightmare and there could be memory implications for loading previous versions of components.

All this however points to the conclusion that for a 100% accurate simulation of earlier versions of Internet Explorer you are going to have to use an earlier version of internet explorer L

So what to do…

Well I guess you could set up a number of virtualized instances allowing you to test every possible version of a browser but few teams are going to have the resources to do this so and hey that sounds like a heck of a lot of work!

My recommendation would be to instead use a service like browserstack.com or spoon.net that contains virtualized versions of earlier browsers all nicely setup for you. These services are inexpensive and allow you to simulate many different browser versions (and even offer other browser options such as mobile devices).

Secondly it’s a good idea to review the compatibility cookbooks that Microsoft release when they ship a new version of Internet Explorer as these can assist in giving you a heads up of breaking changes: http://msdn.microsoft.com/en-us/library/ie/hh771831(v=vs.85).aspx

Thanks to IE team for assistance in this article.

Websockets with ASP.net 4.5 and Visual Studio 2012

Web applications are becoming increasingly sophisticated and it is common to need to communicate with various services.

There are a number of options to accomplish this task with probably the most popular being to continually poll a server with XHR requests. Other alternatives exist that delay disconnections. These can be tricky to implement and don’t scale well (sometimes worse than polling as they keep a connection open) so aren’t used as much.

HTTP isn’t really an ideal protocol for performing frequent requests as:

  • It’s not optimized for speed
  • It utilizes a lot of bandwidth for every request with various headers etc sent with every request
  • To keep an application up to date many requests must be sent
  • Provides limited cross domain support (relying on workarounds such as JSONP http://remysharp.com/2007/10/08/what-is-jsonp/)
  • Firewalls & proxys sometimes buffer streaming/long polling solutions increasing latency
  • Long polling & streaming solutions are not very scalable

WebSockets are a new technology that attempts to resolve some of these limitations by:

  • Sending the minimum amount of data necessary
  • Making more efficient usage of bandwidth
  • Providing cross domain support
  • Still operating over HTTP so it can transverse firewalls and proxies
  • Works with some load balancers (TCP l4)
  • Provides support for binary data (note some JavaScript implementations don’t currently support this)

When would web sockets be a suitable protocol for your application?

You might want to consider using web sockets in the following scenarios:

  • Games
  • Real time data
  • Chat applications
  • News tickers

There is a nice set of demos at: http://www.html5rocks.com/en/tutorials/websockets/basics/ and an interesting article that compares a Web Sockets and polling solution in terms of latency & throughput at http://websocket.org/quantum.html.

Websockets pitfalls

Websockets is a relatively new protocol that has already undergone a number of versions as various issues are addressed. This is important as support across browsers varies.

At the time of writing Websockets (in some form) can be used by the following browsers (check caniuse.com for the most up to date info):

  • IE10
  • Chrome 13+
  • Firefox 7
  • Safari 5+
  • Opera 11+

Earlier implementations of websockets had some security issues so your connections may work but are not secure (Firefox disabled support in Firefox 4 & 5 for this reason).

The other issue that you may encounter is that some older proxy servers don’t support the http upgrade system that websockets uses to connect so some clients may be unable to connect.

.net 4.5 Web Socket Support

.net 4.5 introduces a number of APIs for working with web sockets. If you find you need more control than the ASP.net API’s offers then look into WCF as that has also been updated.

Before we begin there are a couple of requirements for using ASP.net web sockets API:

  • Application must be hosted on IIS 8 (available only with some version of Windows 8 – please note currently IIS Express currently does not work)
  • Web Sockets protocol feature installed (IIS option)
  • .net 4.5
  • A compatible browser on the client (IE10 or Chrome will 18 work fine at time of writing)
  • It would help if your Chinese birth animal was the horse

Currently Microsoft have no plans to release Websockets support for earlier versions of IIS so if you plan to run it on Windows Server 2008 then you are going to have to look at other options such as http://superwebsocket.codeplex.com/.

You could also look at the SignalR library from Microsoft which is designed for developing async applications and provides WebSockets (and fallback) support: https://github.com/SignalR/SignalR/wiki/WebSockets.

Hello Web Sockets Example!

Ok I am going to assume that you are already working with some version of Windows 8 that has IIS & ASP.net 4.5 installed. The other thing we are going to need to do is make sure IIS has the Web Sockets Protocol feature installed (this is in the add/remove programs bit):

First create a new empty ASP.net project called WebSockets

Add the Nuget package Microsoft.Websockets

Pull down the latest jQuery library and put it in a scripts directory (I am using 1.7.2) – note jQuery isn’t necessary it just saves a bit of tedious event and manipulation code.

Now add a file called index.htm and enter the following code:

<!doctype html>


<script src="Scripts/jquery-1.7.2.min.js" type="text/javascript"></script>

<script type="text/javascript">

$(document).ready(function () {

var name = prompt('what is your name?:');

var url = 'ws://' + window.location.hostname + window.location.pathname.replace('index.htm', 'ws.ashx') + '?name=' + name;

alert('Connecting to: ' + url);

ws = new WebSocket(url);

ws.onopen = function () {

$('#messages').prepend('Connected <br/>');

$('#cmdSend').click(function () {





ws.onmessage = function (e) {

$('#chatMessages').prepend(e.data + '<br/>');


$('#cmdLeave').click(function () {



ws.onclose = function () {

$('#chatMessages').prepend('Closed <br/>');


ws.onerror = function (e) {

$('#chatMessages').prepend('Oops something went wront <br/>');






<input id="txtMessage" />

<input id="cmdSend" type="button" value="Send" />

<input id="cmdLeave" type="button" value="Leave" />

<br />

<div id="chatMessages" />



We need to create an http handler so add a new generic handler to the project called ws.ashx and enter the following code:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Web;

using Microsoft.Web.WebSockets;

namespace WebSockets


public class WSHttpHandler : IHttpHandler


public void ProcessRequest(HttpContext context)


if (context.IsWebSocketRequest)

context.AcceptWebSocketRequest(new TestWebSocketHandler());


public bool IsReusable




return false;





Finally we need to create something to handle the websocket connection (TestWebSocketHandler that is created in the AcceptWebSocketRequest method).

Create a new class called TestWebSocketHandler and enter the following code:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Threading;

using System.Web;

using Microsoft.Web.WebSockets;

namespace WebSockets


public class TestWebSocketHandler : WebSocketHandler


private static WebSocketCollection clients = new WebSocketCollection();

private string name;

public override void OnOpen()


this.name = this.WebSocketContext.QueryString["name"];


clients.Broadcast(name + " has connected.");


public override void OnMessage(string message)


clients.Broadcast(string.Format("{0} said: {1}", name, message));


public override void OnClose()



clients.Broadcast(string.Format("{0} has gone away.", name));




That’s all you need so now compile the project and run it in a compatible browser (IE10 or the latest Chrome will do fine) making sure you are hosting your project from IIS (project properties if you are not).

Once you have run it up you will be prompted to provide a name, then an alert box will indicate the end point of your application (ws://localhost/.. – note the secure https version is wss://).

Now open up a different browser and you should find you can via websockets!

Tables are bad – or are they?

This week I was putting together some general web development guidelines for our team (ill publish these at some point when they are a bit more mature – we based some of them on these great css guidelines.

We are developing a new version of an existing site and one of the items that came up was the use of tables in html for layout.

One of my colleagues forwarded me a link about why tables for layout are bad (http://www.hotdesign.com/seybold/index.html). It’s a bit dated but most of the advice is still relevant.

Unfortunately what most developers have heard (and took from that article judging by the emails I received) was that tables are universally bad.

Of course this isn’t the case and few things are universally bad (apart from maybe the marquee tag and Microsoft SharePoint).

Tables are entirely appropriate for tabular data.

In fact lets go to the source and refer to the HTML5 spec – crazy eh? and see what it says about the table element

“The table element represents data with more than one dimension, in the form of a table”

And it goes on to say:

“Tables should not be used as layout aids”

The spec even goes into great detail about how to add further information to your tables (and when not to).

The reason tables are bad when used for layout are:

  • CSS is a better and more flexible way to lay out a webpage
  • Tables result in verbose html resulting in bigger slower loading pages
  • Tables are arguably harder to maintain consistent layout
  • Tables probably lock you into a particular layout
  • Accessibility issues – screen readers dont cope so well with tables
  • Can prevent incremental rendering and encourage reflow
  • You are abusing the elements purpose and they can encourage layout mark-up on the page itself
  • It can be tricky to get tables looking right across all browsers
  • SEO – I am not sure how much of an issue this is but we will go with it for now..

Some examples of where tables might be appropriate:

  • Comparison grids
  • Lists of results*
  • Agendas

So if you have some tabular data like then go ahead and use a table.

One  of my colleague’s pointed out all some big sites such as Google & Amazon using Tables to lay out the page. I suspect the Google example was to do with avoiding the need for an additional request but its also important to note that these sites have pretty simple designs and you’d quickly get into a mess with something more complex.

In conclusion:

  • Tables bad for layout
  • Tables are appropriate for tabular data

*Interestingly if you take a look at most large sites search results e.g. Amazon, Expedia you will find the results laid out using a divs rather than tables. I think it’s probably the way I would go too as I suspect it would be easier to style, maintain and less verbose.

Further reading: