Processing boomerang beacons in ASP.NET

A prerequisite to enabling boomerang web beacons for your web pages, as discussed in the previous post, is providing a web server component that expects incoming GET Requests for the boomerang.gif and understands how to respond to those Requests. Let’s see how to handle the boomerang.gif GET Requests in IIS and ASP.NET.

An excellent way to add support for boomerang web beacons is to provide a class that supports the IHttpModule interface, which works so long as you are running IIS 7.0 or later in integrated mode. In integrated mode, which is also known as integrated pipeline mode, IIS raises a pre-defined sequence of events for each and every web request. The IIS feature was called the integrated pipeline mode because it extended the original ASP.NET event pipeline, introduced in ASP.NET version 1, to apply to the processing of all Http Requests, beginning with IIS version 7. The IIS integrated pipeline provides “a single place to implement, configure, monitor and support server features such as single module and handler mapping configuration, single custom errors configuration, single url authorization configuration,” in the words of Mike Voldarsky, who was a Program Manager for the Microsoft Web development team at the time. (In addition, Mike published an excellent technical article discussing the capabilities of the on the subject in MSDN Magazine subsequent to the product’s release.)

So, to process boomerang beacons, you could instantiate a BoomerangBeacon class that supports the IHttpModule interface that adds an event handler for the IIS pipeline BeginRequest event:

public class BoomerangBeacon : IHttpModule

{

 public BoomerangBeacon()

{            }

 public void Dispose()

{             }

public void Init (HttpApplication application)

{

application.BeginRequest += (new EventHandler(this.Application_BeginRequest));

}

}

LISTING 5. DEFINING A BOOMERANGBEACON CLASS THAT SUPPORTS THE IHTTPMODULE INTERFACE TO INTERCEPT THE BOOMERANG WEB BEACON AS SOON AS IT IS PROCESSED IN THE IIS INTEGRATED PIPELINE.

IIS raises the BeginRequest event as soon as a GET Request is received and is queued for processing in the pipeline. Next, provide an Application_BeginRequest method to intercept the boomerang beacon GET Request as soon as the, something along the lines of the following:

private void Application_BeginRequest(Object source, EventArgs e)

{

// Create HttpApplication and HttpContext objects to access

         // request and response properties.

HttpApplication application = (HttpApplication)source;

HttpContext context = application.Context;

 

string beaconUrl = GetBeaconUrl();  // Helper routine to set beacon url

string filePath = context.Request.FilePath;

string fileExtension = VirtualPathUtility.GetExtension(filePath);

if (fileExtension.Equals(“.gif”))

{

if (filePath.Equals(“/” + beaconUrl) || filePath.Contains(beaconUrl))

{

  …   // Process the beacon parms

application.Response.End();

}

}

}

LISTING 6. A BEGINREQUEST EVENT HANDLER THAT INTERCEPTS THE BOOMERANG WEB BEACON GET REQUEST IN ORDER TO PROCESS ITS QUERYSTRING PARMS.

This example calls a Helper routine, GetBeaconUrl(), to retrieve the name of the beacon Url from a web.config application setting. After processing the parms attached to the boomerang beacon GET Request, the BeginRequest event handler calls the End() method on the ASP.NET Response object that flushes the web beacon GET Request from the pipeline and terminates any further processing of the boomerang beacon. Calling Response.End() returns an empty HTTP Response message to the requestor with an HTTP status code of 200 – which is the “OK” return code – to the web browser that issued the Request.

NameValueCollection parms = context.Request.QueryString;

LISTING 7. ASP.NET PROVIDES ACCESS TO THE REQUEST QUERYSTRING VIA THE BUILT-IN HTTPCONTEXT OBJECT.
The key, of course, is processing the web beacon Query string parms that contain the measurement data from the web client. ASP.NET automatically generates a QueryString property that parses the GET Request query string and returns a NameValueCollection

HttpRequest request = context.Request;
string Browser = request.Browser.Type;
string Platform = request.Browser.Platform;
string HttpMethod = request.HttpMethod;

LISTING 8. IDENTIFYING THE HTTP METHOD, THE BROWSER TYPE (I.E., IE, CHROME), AND THE OS PLATFORM WHERE THE REQUEST ORIGINATED.

making it a relatively simple matter of looping through the parms collection and pulling out the measurements, exactly how Yahoo’s boomerang howto.js script I referenced earlier does. Your ASP.NET code that processes the web beacon can also pull the HttpMethod, Browser.Type and Browser.Platform directly from the Request object, as illustrated in Listing 8.

In addition, your IHttpModule class can also process the IIS ServerVariables collection to pull the identifying web client IP address and TCP Port assignment. These IIS web server variables are returned as strings, but it is not too much trouble to transform them into binary fields, although you do need to be able to handle both IP 4-byte v4 and 16-byte v6 addresses.

In my version of the IHttpModule that processes the boomerang beacons, I decided to generate an ETW event from the NavTiming API measurements with an event payload that contains the web client Page Load time measurements. This approach allows me to integrate RUM measurements from the web beacons with other web server-side measurements that can be calculated from other ETW events. In the next section, I will discuss what web server-side measurements can be gathered using ETW, and how I integrated boomerang beacon RUM measurements into IIS web server performance management reporting using those trace events..

Boomerang web beacons

The DOM’s performance navigation and timing API makes it easy to gather Page Load time measurements in a consistent manner across the major browsers. But it does not address the problem of getting the measurement data from the visitor’s browser back safely into your hands for analysis and reporting. Tools like Google Analytics tool solve that problem using a web beacon that issues a GET Request that is directed to a Google web site. At the web site where the beacon GET Request is received and processed, timing data is gathered from the Request parms, which, in the case of GA, are then massaged and collated for you by Google. If, however, your IT organization is not enamored of the idea of giving Google access to all of your web site access and performance data, you want an alternative solution.

An attractive alternative is a robust open source JavaScript library called boomerang.js that was developed at Yahoo. The Yahoo boomerang script is functionally quite similar to the Google SiteSpeed script, with one crucial difference. Instead of sending browser timing data to a 3rd party, boomerang allows you to direct the web beacon to one of your own web sites. Sending boomerang beacon performance measurements from your company web site to another one of your web sites requires a web server-side component that is expecting the boomerang beacon and understands how to process it. I will show you how to gather the navigation timing data from the boomerang beacon and process it in IIS and ASP.NET in the next section.

Enabling boomerang web beacons

A good way to get started with the Yahoo boomerang scripts that measure web application Page Load times is to follow this boomerang.js link, copy the scripts to a folder (beginning with a test web application, of course) and then include the scripts required in your web page:

<script src=”javascript\Boomerang\boomerang.js” type=”text/javascript”></script>

<script src=”javascript\Boomerang\plug-ins\navtiming.js” type=”text/javascript”></script>

<script src=”javascript\Boomerang\plug-ins\rt.js” type=”text/javascript”></script>

LISTING 2. ADDING BOOMERANG.JS REFERENCES TO A WEB PAGE.

In this example showing how to add a reference to the boomerang JavaScript library to a web page, I created and referenced a folder named “boomerang” that is a subfolder of my \javascript folder, which is where in-house scripts my web application references reside. Creating a local folder to hold the boomerang scripts eliminates any of the security concerns associated with cross-site scripting or XSS. (Yahoo is not actively changing the boomerang library, which has remained stable since the NavTiming API was adopted.)

The first script referenced in Listing 2 is boomerang.js, which functions as a scaffolding for the rest of the application. The navtiming.js script from the boomerang plug-in library accesses the windows.performance object and simply maps its properties to the timing fields boomerang.js has previously defined. The NavTiming spec makes this code quite simple:

p = w.performance || w.msPerformance || w.webkitPerformance || w.mozPerformance;
if(p && p.timing && p.navigation) {
BOOMR.info(“This user agent supports NavigationTiming.”, “nt”);
pn = p.navigation;
pt = p.timing;
data = {
nt_red_cnt: pn.redirectCount,
nt_nav_type: pn.type,
nt_nav_st: pt.navigationStart,
nt_red_st: pt.redirectStart,
nt_red_end: pt.redirectEnd,
nt_fet_st: pt.fetchStart,
nt_dns_st: pt.domainLookupStart,
nt_dns_end: pt.domainLookupEnd,
nt_con_st: pt.connectStart,
nt_con_end: pt.connectEnd,
nt_req_st: pt.requestStart,
nt_res_st: pt.responseStart,
nt_res_end: pt.responseEnd,
nt_domloading: pt.domLoading,
nt_domint: pt.domInteractive,
nt_domcontloaded_st: pt.domContentLoadedEventStart,
nt_domcontloaded_end: pt.domContentLoadedEventEnd,
nt_domcomp: pt.domComplete,
nt_load_st: pt.loadEventStart,
nt_load_end: pt.loadEventEnd,
nt_unload_st: pt.unloadEventStart,
nt_unload_end: pt.unloadEventEnd
};

LISTING 3. THE BODY OF THE NAVTIMING.JS SCRIPT ACCESSES THE WINDOWS.PERFORMANCE OBJECT AND MAPS ITS PROPERTIES TO TIMING FIELDS BOOMERANG.JS HAS PREVIOUSLY ALLOCATED.

After first verifying that the performance object exists, the navtiming.js script assigns the performance object to variable called p. Then the script copies the navigation object from p to pn and the timing object to pt. Then the data object is allocated and its properties assigned from the respective pt and pn fields.

In case the NavTiming API is not available, Listing 2 includes the boomerang rt.js script. This is the script that the Yahoo performance team originally developed to gather page load time measurements prior to the NavTiming spec being adopted. It works by attaching event handlers to the appropriate window events. The script first checks to see if the NavTiming performance object is available and stops processing immediately if it is, so there is not a huge performance hit by including it. On older, non-compliant web browsers, executing the rt.js script allows you to still gather what page load time measurements can be collected, subject to the capabilities of the specific version of the web client, of course, and whether cookies are allowed.

I should also mention the boomerang howto.js script which is useful to verify that the boomerang scripts are working properly and to get a first look at the measurements. If you add the boomerang howto.js script to the html, it will print out the contents of the boomerang measurement data object at the bottom of your test web page.

The boomerang.js does require one bit of initialization – you need to specify where you want the web beacon URL sent:

<script lang=”javascript”>

BOOMR.init({beacon_url: “boomerang.gif”});

</script>

LISTING 4. SPECIFYING THE BOOMERANG BEACON URL AT INITIALIZATION.

Based on the script shown in Listing 4, the Page Load time measurement data will be appended as parameters in a Query string added to a GET Request that the script issues for a local resource called boomerang.gif. In addition to all the timing fields from the performance object, the boomerang beacon data includes the complete Request string for the Request the measurements apply to, including the parms for that Request..

Navigation Timing API

The W3C-sanctioned Navigation Timing API provides access to a built-in DOM object that contains timing data for a sequence of about twenty events, encompassing nine stages in the processing of a new web page request from the point of view of the web client. The nine processing stages that take place inside the web browser to load a new page are:

  • Prompt for unload
  • Redirect
  • App cache
  • DNS lookup
  • TCP connection handshaking
  • Request
  • Response
  • Processing
  • Load event

The DOM’s built-in Timing object contains a set of Properties marking the beginning and end of each processing stage, plus a few additional timers. The complete set of timing events from the NavTiming spec, and the processing stage they are associated with, is depicted in Figure 11.

NavTiming Drawing

The NavTiming API establishes a performance object associated with the window object of the DOM that has two sets of properties, one for timing the window load events and one detailing the navigation sequence. The timing object contains a timer value for each of the properties shown in Figure 11 corresponding to the sequence of events associated with page composition. The timer values are standard JavaScript timestamps, calibrated in milliseconds from the zero date, which is January 1, 1970. Subtracting the navigationStart value from loadEventEnd time gives the Page Load time in milliseconds.

Other useful metrics that can be obtaining from the window.performance object’s properties include the network latency or network delay time, which is interpreted as the time between the fetchStart and requestEnd timers. In addition, the time to first byte can be calculated by subtracting the responseStart timestamp from navigationStart. Note that performance analysts associated with web page caching and TCP/IP networking hardware and software sometimes prefer to focus on the Time to First Byte measurement because that is the portion of the page composition process that web page caching and network optimization can influence directly. The overall Page Load time measurement is more meaningful to the web application developers that are in a position to determine the size and complexity of the entire web page composition process.

Finally, the browser page composition and render time can be calculated as the time between the responseEnd and the loadEventEnd timer value. Note that not all requests have delays due to redirects, or for DNS look-ups or TCP connection handshaking, so it is not unusual to see timer duplicate values of some of the timers. The most commonly used web application response time measurements that are derived from the NavTiming’s window.performance object are summarized in Table 1.

Measurement

Derivation

Page Load time loadEventEnd navigationStart
Network delay time requestEnd fetchStart
Time to First Byte responseStart navigationStart
Render time responseEnd – loadEventEnd

TABLE 1. WEB APPLICATION PERFORMANCE MEASUREMENTS COMMON DERIVED FROM THE WINDOW.PERFORMANCE OBJECT’S PROPERTIES.

The JavaScript code to access the timing values for Page Load events is trivial, as illustrated in Listing 1, and all recent releases of the major browsers support the standard.

Prior to the navigation timing API, it was possible to gather basic Page Load time measurements by subscribing to the DOM window.unload and window.load events and gathering timer values in JavaScript code when those events were raised. The fact that the window.unload event that occurs when a new GET Request is issued fires in the context of processing the previous web page presents some difficulty, however. Since the HTTP protocol is stateless and connectionless by design, there was no reliable mechanism built into the browser to feed a start timer value gathered for the previous Request forward to the subsequent Request.

A solution to this difficulty was to store the timer value for the window.unload event in a cookie so that it is passed in the subsequent GET Request, where it can then be retrieved by a window.load event handler. However, cookies are not an entirely foolproof method that a JavaScript-based timing script can use to pass the timer value from the window.unload event since there are visitors that choose to disable them out of privacy concerns. In the interest of helping Google Analytics gather measurement data reliably, Chrome supported a proprietary extension to the DOM called window.chrome.loadTimes, a precursor of the NavTiming API. In addition, whenever the Google Toolbar was installed inside web client program other than Chrome, the Google toolbar also enabled the window.chrome.loadTimes object.

Prior to the NavTiming API, it was also difficult to negotiate the discrepancies among different browsers, some of which were due to underlying architectural differences. However, since the navigation timing API was adopted by the developers of the main browsers, all the major discrepancies seem to have been resolved. While the specific browser implementations are largely consistent, you will still notice some minor differences in the order in which certain events fire in different browsers.

The NavTiming API also established a standard DOMHighResTimeStamp type that is consistent with 100 nanosecond timer ticks in Microsoft Windows and a performance.now() method to retrieve the current time. (If the underlying OS is not Windows, however, DOMHighResTimeStamp values revert to milliseconds.)

 .