Modern Web applications are a conglomerate of many individual moving parts. Web server software and hardware, application servers, database systems, custom developed applications, a variety of network devices, the network transport and, of course, the ever-present Web browser all present opportunity for failure. Unfortunately, when a Web application fails, finding out exactly what caused the bug can be a peskier task than fixing the bug itself.
In our test, Xaffire xFire 3.0 platform attempts to provide a holistic view of a Web application’s health and usage to assist in bug isolation. The product mostly succeeds in this task by tying together a range of data, although a few rough edges in its interface and overall approach hold it back at times.
To properly debug any software application — Web-based or not — the problem must be reproduced. Yet when it comes to Web applications, simple log files often do not provide enough information, as you need to know what the user did and the context in which they did it. You need to know not only the series of requests that might have led up to a problem, but also any data a user might have entered, the type of browser they used, and the states of the server and the network at the time of the request.
XFire attempts to provide an instant replay of a Web application problem by capturing and correlating data at the network level and the Web server level. Some data is fairly easy to collect from various logs and network-monitoring interfaces, but to collect actual user session data, xFire needs an interceptor in the form of an Internet Server API filter for an Internet Information Server (IIS) or an Apache module for an Apache-based system. We ran our IIS testing on an IIS 5.0-based system, as the collection filter has not yet been ported to IIS 6.0. Xaffire officials indicated the IIS port is well underway.
One caveat with the filter/module approach for collecting user data is that it will induce some minor performance degradation on a server, particularly if it is under significant load. That seems to be a minor penalty given the payoff of being able to actively debug a live application.
XFire also collects other infrastructure-related data, such as the health of a Web server, application server, database system or an SNMP-capable network device. All the data from monitored sessions and systems is collected at an xFire archiver appliance for monitoring and later analysis. The archiver appliance has a management console that provides access to the three major aspects of the xFire approach: infrastructure monitoring, synthetic transaction deployment and monitoring, and live session capture and playback.
The infrastructure monitor provides a simple graphical overview of the various resources that might be found in a Web farm. Some monitors are very basic ping or SNMP monitors, while in other cases with databases such as MySQL or application servers such as Tomcat, more detailed measurements are taken.
Discovery and configuration of these monitors is generally straightforward, but be forewarned given the range of versions available — particularly for open source platforms — you might have to tinker with settings.
While the infrastructure-monitoring aspect of Xaffire’s offering certainly is not going to compete with a more network-operations-focused platform such as Computer Associates’ Unicenter, it provides reasonable system availability indicators for those interested in application-level activity. These indicators are helpful, because in most companies, the person debugging a Web application is often not always the person who can access an enterprise network management platform.
The synthetic-monitoring feature lets network managers create test scripts to be played at various intervals to exercise a site’s functionality. A service can be employed in the offering so that recorded transactions can be played not only from the local environment, but also from remote beacons on the Internet.
While the synthetic- and remote-monitoring aspect of the product does not represent actual user traffic, it is very appropriate. The health of the synthetic transactions provides insight into whether the network might be part of the problem the user is encountering. Like the infrastructure component, the synthetic transactions with local and remote monitoring xFire provided easily can be bested by specific services such as Keynote or Gomez, but xFire’s direct integration with its other reporting services makes the feature very useful. The product shines in its user session capture and playback capabilities. XFire provides a wealth of useful information showing full request and response headers and data.
Once a particular session is found by searching by time or hit — which can be difficult if you don’t have an exact time or a unique indication such as a logon value — you can replay the session to observe very much like the user did.
While the playback obviously is not an exact duplicate of what a user might see, it is close enough to be useful for a majority of Web applications.
During testing we used the product in a live scenario and determined that a reported instability of a monitored e-commerce site was not related to user activity, but instead unstable software. The problem wasn’t solved immediately, but the culprit was determined primarily by using xFire in conjunction with some log file pattern analysis.
While xFire proved itself in real use, a significant downside of the xFire system is that the interface is just plain unintuitive in places and often too cluttered. While it might be good enough to solve a problem, the console would benefit significantly from usability testing and user interface re-factoring.
On the bright side, the TiVo-like interface that let us see user data at a particular time was appealing.
While the xFire platform has rough edges, its integration of many disparate pieces of website user, system and application data shows that the whole is often greater than the sum of its parts. Together, this information could be used to diagnose serious Web application problems that users encounter.
Powell is the founder of PINT, a San Diego Web development and consulting firm. He can be reached at firstname.lastname@example.org. n
How we did it
We tested the xFire platform in a mixed IIS and Apache environment with a variety of middleware environments, including Tomcat, ColdFusion, Active Server Pages and ASP.NET and databases including mySQL and Microsoft SQL Server. The bulk of the testing was performed on an internal IIS5 platform running numerous websites and an externally hosted IIS5 served e-commerce site developed in class Active Server Pages using numerous Internet Server API filters.
For local pricing and a list of distributors visit www.xaffire.com
How do alerts work?
xFire alerts you when a performance attribute falls below acceptable levels. You can configure xFire to alert you via email, visually, via sound, or you can have a programmatic script run when a threshold is crossed.
Alerts can be triggered by thresholds that are either set by you to a raw number, or by threshold sensitivity to an automatic baseline that runs automatically once you install xFire.
xFire alerts in the following instances:
* When the performance of an entire synthetic Web transaction falls below a threshold that is either set by xFire auto-baselining or by you specifically.
* When the value of a system attribute exceeds a set baseline or threshold.
* When the value of a system or application resource on a system is “unavailable”.
* When an HTTP error code that is returned to an end user exceeds the automatic baseline (Live Transaction Capture required).
* When specified content is sent to an end user (Live Transaction Capture required). This allows you to alert on error messages or other text that may indicate a problem.
You may define the action to take place when a baseline is crossed. Actions include receiving an email, executing a program or playing a sound.
If no action is set, xFire will display visual status changes for the metrics that cross auto-baselined thresholds.