JMeter Tips and Tricks - Better (More Pretty) Graphs

This plugin has been around for a while and it’s definitely worth including in your JMeter library if you want to go for a glossy finish! The StatAggVisualizer will transform your graphs
from this to this

If you roll-up your transactions into separate transaction controllers as I demonstrated in a previous post, you will get useful summary views for all your transactions:

Alternatively, drop a stat agg visualizer listener under a single controller to get a graph for a single transaction, useful if clients are only interested in certain transactions.

Posted in JMeter | No Comments »

HttpWatch, a Free Alternative Using Ruby

When performance testing I often need to spy on TCP (http) traffic between the client and server on a frequent basis. This is pretty much a staple of any performance test script development effort.

In the past I’ve relied heavily on trace options within LoadRunner and the like, but often find this a tad cumbersome. LiveHTTPHeaders is a great plugin for Firefox but it only captures header information. WireShark is great if you need to ‘go deep’, but often equates to opening a walnut with a sledgehammer. FireBug and IE Developer Toolbar are great to see what’s happening on a page-by-page basis. TCPdump is great if I’m on a Mac or Linux box, but it can be tricky parsing binary data. In any case most client sites are windows based for me.

So that’s a great list of tools to help you correlate and sniff out dynamic data, but the closest I’ve found for IE to doing what I want, that is, capturing all HTTP data going back and forth with the ability to filter for specific information from a console/command line is HttpWatch. A $295USD price tag / crippled basic (free) edition really forced me down this path…to roll my own HttpWatch using Ruby and the WEBBrick proxy.

Read on for a description of the code.
Read more »

Posted in OSX, Ruby, Tools, Productivity, Windows | 4 Comments »

NumbrCrunchr - Performance Monitoring Using Sparklines

I’ve been working on a light weight performance monitoring solution which can be hosted on a LAMP (or WAMP, or OSX =) installation and uses the Google chart API to present data. Data is populated by custom Ruby scripts that either use ssh or perfmon (typeperf) to collect performance metrics from remote Unix and Windows machines.

The intent is to keep the data collection process detailed enough but simple. Presentation of the collected data then uses the API to present charts in a sparkline format.

Sparkline is a type of information graphics, that presents in a simple and condensed way trends and variation, associated with a measurement such as average temperature or stock market activity. These are often used as elements of a small multiple with several lines used together.

The Sparkline is proposed by Edward Tufte for “small, high resolution graphics embedded in a context of words, numbers, images” Tufte describes sparklines as “data-intense, design-simple, word-sized graphics”. Sparklines are intended to be succinct, memorable, and located where they are discussed.

The finished product looks like this:

Other functionality includes the ability to automatically ‘band’ expected data ranges for each metric collected, using a 30 day (customizable) sample that shows the 20 - 80th percentile. This should offer some insight into trends, and leverage off the sparkline format in being able to quickly assess a large range of metrics or performance data.

There’s also a SiteScope plugin, or in fact an emulator, so that if you’re still using LoadRunner you will be able to dip into the same metrics collected by NumbrCrunchr in a similar fashion to SiteScope with minimal configuration changes.

I’ve decided to go open source with this project in the interests of improving and continually updating its capabilities. If you’d like to contribute to the source please go to the project for more information. If you’d just like to see a quick demo, please go to NumbrCrunchr.com.

Posted in Statistics, Analysis, Tuning, SiteScope | No Comments »

Job Interview Question #11 - Summing Numbers

I’ve recently started following a blog which sets weekly programming challenges. I’m getting sick of Nintendo DS brain training and thought I’d try something more relevant to my line of work. Also, this gives me a good opportunity to learn more about my favoured scripting language at present, which is Ruby. So this week’s challenge states:

Given a list of n integers and another integer called m, determine (true / false) if there exist 2 numbers in that list which sum up to m.
Example: 2,6,4,9,1,12,7 and m=14 -> 2 and 12 sum up to 14, so the answer is true.
Provide the best algorithm in both manners: performance and memory to solve this puzzle. Don’t forget to mention the complexity of your solution!

Read on for my thoughts in solving this challenge.
Read more »

Posted in Ruby, Programming | No Comments »

JMeter Tips and Tricks - Application Simulation Models

I’ve been working with JMeter of late and after speaking with some people at conferences and the like, I thought it would be good to share what I’ve learned along the way in building, structuring and executing JMeter test plans.

One of the first things I like to do when performance testing is establish in reasonable detail what the Application Simulation Model (ASM) will entail. You can create an ASM visually using something like UCML. This is a good starting point for performance testers looking to structure their test planning efforts.

UCML™ is a set of symbols that can be used to create visual system usage models and depict associated parameters. When applied to Performance Testing these symbols can serve to represent the workload distributions, operational profiles, pivot tables, matrixes, and Markov chains that performance testers often employ to determine what activities are to be included in a test and with what frequency they’ll occur.

More often than not though, I create an ASM using a spreadsheet format as I find this easiest to communicate with people from the business teams. If people are interested I will share some of the spreadsheet templates. The following screenshot gives you an idea of what I’m talking about:

The pertinent points for an ASM, is that it should:

e.g. “we have 3 user groups with a total of 150 users, 80% of them are helpdesk support, 10% are team leads and 10% are management”

e.g. “we’re aiming for around 240 trans/hour, of which 50% are searches, 25% are updates and 25% are reports”

Read on to discover how to turn ASMs into working JMeter test plans.
Read more »

Posted in Testing, JMeter | No Comments »

Just Another Framework - for developing watir test cases

Recently I attended Railscamp 08 in Sydney Australia for a weekend of code, beer and bzflag. For my ‘project’ I decided on brushing up my my Ruby skills and constructing a simple framework from which I could structure watir test cases.

Specifically I wanted to achieve the following:

To that end I was reasonably successful. The only limitation was that I only had my macbookpro with me and probably would have been shunned even if I attempted to load a windows XP virtual machine at a rails camp full of mac and linux geeks. So I chose FireWatir as my test platform running on my Mac, but am reasonably comfortable you can easily port this framework for use with Watir.

Now on with the framework contents …
Read more »

Posted in Ruby, Automation, watir | No Comments »

Establishing Mysql DB Connectivity from Ruby

If you are a rails fan you will probably recommend using active record instead, but if you just want to hack away at your mysql db from within your Ruby code, you may find this handy…

I’m currently using a wamp box to serve up content for performance metrics. What I needed was a Ruby script which could interrogate data on this installation.

You can download the binary required for this from here. After you install the gem (I do this off-line as my command prompt doesn’t allow me internet access here at work) you may find that you get an error similar to the following:
"NameError: uninitialized constant Mysql"

See what’s happening?
Read more »

Posted in Ruby, Windows | 2 Comments »

CITCON 2008 - Melbourne

I recently attended the 2008 conference for Continuous Integration in Melbourne, and mixed with like minded professionals involved with all aspects of CI and testing in general.

In short this is a great (free) opportunity to attend a conference using an open session format. By that I mean the conference is run/organized like a wiki, where the attendees nominate topics they’d like to discuss or facilitate and then a user vote system organizes and makes it happen.

I avoided the performance testing oriented sessions and went for sessions that spoke about exploratory testing, defect management, benefits of test driven development, skills required for test automation and the last one titled ‘do we still need testers?’ …

They all generated a healthy amount of discussion and it was great to hear other people’s experienced opinions on the subject matter. It was also a great opportunity to network, find out who’s working on similar problems in the local industry and from afar. There was a healthy interest in using JMeter as a performance test tool to support CI, so I’ll be posting some more targeted blogs about this in the near future, as well as some consolidated info on performance metrics collection (without SiteScope) which a few people were also interested in.

Anyway, if you haven’t heard about the conference, keep an eye out for next years version. It’s well worth the zero dollar commitment, free beer and t-shirt. All you have to do is ‘use your feet’.
=)

Posted in General | No Comments »

Diff’ing Made Easier

If diff -u or wdiff isn’t cutting it for your correlating requirements when scripting load harnesses, you might want to try daisydiff as an alternative …

* Works with badly formed HTML that can be found “in the wild”.
* The diffing is more specialized in HTML than XML tree differs. Changing part of a text node will not cause the entire node to be changed.
* In addition to the default visual diff, HTML source can be diffed coherently.
* Provides easy to understand descriptions of the changes.
* Allow easy browsing of the modifications through keyboard shortcuts.

Of late have been working with pesky AJAX controls inside a web-app, and one of my gripes with more traditional tools such as wdiff or diff, is that they don’t tell you *where* in the line the change has occurred. Read on for some hacky ways I’ve implemented daisydiff with LoadRunner and other open source tools in general …
Read more »

Posted in Tools, Productivity, Testing | No Comments »

RailsCamp 08

Went to the third RailsCamp held in a scout hall near Gosford NSW last weekend; my brain has just about got back to normal after 3 solid days of drinking and coding =)

RailsCamp is a get together of mostly Rails developers, but also a fairly eclectic mix of Ruby hackers and geeks in general. 3 nights of drinking, camping and coding with NO internet. Possibly the hardest habit to kick is to stop checking for emails every 5 minutes.

For those with harder to kick habits such as Twittering, there was a ‘Twetter’ clone amongst other cool apps such as the ‘Duke’. The Duke was a user vote playlist driver for winamp (I think) and I got to observe just how close the geek community is when it comes to music tastes. Daft Punk was on fairly high rotation. Of mention was the first time we got Rick Rolled, which then led to a flurry of hack attempts at the guy hosting it on his Mac.

Guitar Hero was in force, as was Urban Terror and a touch of BZFlag. Nice.

Accommodation, setup and food was awesome, the organisers really did do a good job in getting this together. Big thanks to those guys! Perhaps the best part was the ability to network with like-minded people and discuss all things related to Ruby or Rails in a collaborative manner. I managed to get a little work done relating to a test framework for use with FireWatir which I’ll post later.

IF you didn’t or couldn’t make it, keep an eye out for the next one to occur. Definitely worth the $100 to nerd on for a full weekend …

Posted in General, Ruby, Programming | No Comments »

Installing Firewatir on Mac OSX

By now you’ve probably heard of watir or Web Application Testing In Ruby, a great automation framework which supports Microsoft Internet Explorer, and recently came in at number 3 as a popular oss tool on this blog.

I’ve been using this tool mostly from a windows box, and knew of the firefox equivalent, aptly named ‘Firewatir’ which is currently a Google code project. The beauty of this version, is that it allows you as a tester to explore functionality on your favourite platform for development, as I like to leave windows at work, and take OSX with me on the train.

Read on for some simple instructions in getting this running on your Macbook Pro …
Read more »

Posted in Ruby, Automation, watir | 1 Comment »

Using SFTP with LoadRunner Java Vusers

Recently had a requirement to construct a LoadRunner harness that could sftp files (over ssh) to and from remote servers. As some of the harnesses were already written in Java (for loading of JMS queues) it made sense to use a Java Vuser to achieve the result required.

A work colleague found a knowledge base reference that recommended using an RTE Vuser for secure FTP recording and replay. Document ID 41820 refers. I don’t know, but that seemed like a kludge to me, and I wasn’t keen on a multi-protocol harness, so we went searching on Google for a Java alternative.
Read more »

Posted in LoadRunner, Java, Security | No Comments »

Capturing Screenshots in Watir

Recently I noticed some discussion in the watir user group about trying to capture screenshots when running automated tests.

For such a simple requirement, it’s frustrating that solutions available either cost money (such as the purchase of SnagIt) or are a tad complicated in implementation requiring the installation of some out of date gems and dependencies. I’ve also come across this requirement in other automation tools such as QTP, so I thought it’s about time I rolled up my sleeves and hack together a simple solution with C#.NET 2.0.
Read more »

Posted in Ruby, Tools, Programming, watir | 2 Comments »

Using JDBC with LoadRunner Java Vusers

This is a short post on how to establish JDBC connectivity to a MySQL database when using the Java Vuser type within LoadRunner. Stuart describes a situation in which you might want to do this here. I’m not sure why but the example code is missing from his site, so I’ve supplemented with an example of how you’d implement this within LoadRunner.
Read more »

Posted in LoadRunner, Java | No Comments »

Performance Testing Flex Remoting (AMF) with JMeter

[UPDATE] this has since been resolved! JMeter support rocks
URL: http://svn.apache.org/viewvc?rev=661141&view=rev
Log: Bugs 44808 & 39641 - Proxy support for binary requests
—
You may still find the ruby proxy idea interesting …
—
The Action Message Format (AMF) is a binary protocol that Flex applications use to remotely communicate with the server. I’ve tested recording of this using LoadRunner which picks up the binary format nicely as part of the HTTP POST.

It will look a little like this:

  web_custom_request("amf_3",
    "URL=http://host.under.test.com/messagebroker/amf",
    "Method=POST",
    "Resource=0",
    "RecContentType=application/x-amf",
    "Referer=http://ost.under.test.com/messagebroker/client/main.swf",
    "Snapshot=t13.inf",
    "Mode=HTML",
    "EncType=application/x-amf",
  "BodyBinary=\\x00\\x03\\x00\\x00\\x00\\x02\\x00\\x04null\\x00\\x02/3\\x00\\x00\\x01\\x0F\n\\x00\\x00\\x00\\x01\\x11\n\\x81\\x13Mflex.messaging.messages.CommandMessage\\x13operation\\x1BcorrelationId\tbody\\x13messageId\\x11clientId\\x15timeToLive\\x13timestamp\\x0Fheaders\\x17destination\\x04\\x08\\x06\\x01×06\r"
    "my-amf\\x01\\x06#labelPrintService",
    LAST);


I understand that Segue Silk Performer also has this capability to natively record binary protocols arbitrarily attached to the POST. Unfortunately the native JMeter proxy (when used for recording) doesn’t pick up this data reliably.

This defect is current being tracked at the ASF Bugzilla

After looking around for some proxying software that would assist, products such as Charles Proxy looked tempting. But to be honest, if you’re reading this and using JMeter, you’re probably not interested in paying any license fees.

Read on for a free alternative when using JMeter.
Read more »

Posted in LoadRunner, Tools, Testing, JMeter | 5 Comments »

Installing GD Libraries for Leopard Apache PHP 5.2.5

Another post for my fading memory, how to install the GD libraries for your Leopard Apache installation running PHP 5.2.5. I will be using these libraries in a forthcoming blog about the use of ‘sparklines‘ to present complex performance monitoring data.

Now on with installation. Thanks to topicdesk for leading the way with detailed OSX 10.5.x server installation instructions. Here’s my own shorthand version …
Read more »

Posted in General, OSX, PHP, Tools | 16 Comments »

A Growl Hack to Remind You to Unplug Your Charger

If you’ve been using a Macbook Pro or similar for some time, you’ll find that their batteries need tender love and care, if they are to go the distance. According to Apple, I fit into the ‘ideal use’ category as I regularly use my notebook on the 1+ hour daily train commute.

However as the recent purchase of a new $199AUD battery has taught me, I often neglect to ‘unplug’ from the charger when weaseling away at work.

ioreg is part of Apple’s I/O Kit Registry, which you can use to determine programatically what the remaining charge of your battery is based on current and max capacity. Using this tool and Growl notifications, I’ve written a quick hack to remind me to unplug whenever my battery charge is greater than 99%.

Read more »

Posted in General, OSX, Hardware | 5 Comments »

Roll Your Own SiteScope, a Simple Alternative

In working with SiteScope of late, I’ve found that it doesn’t always collect performance metrics the way I want to. More importantly, it can often turn a simple monitoring activity into a complex disaster. Take monitoring via JMX for example. In SiteScope, it has a rather complicated (and sometimes broken) interface when trying to communicate with a busy MBean Server. One can quite easily roll your own JMX monitor using open source tools in about 65 lines of code as I demonstrated here.

But we still all use tools like LoadRunner in these commercial 9-5 contracts right? Wouldn’t it be nice, if you could roll your own custom monitors in Ruby, Perl or whatever language you like, store that data in a simple repository, let’s say a MySQL database, and still be able to hook into those metrics from a LoadRunner Controller during test execution!?

It is possible, with one PHP file and a simple WAMP (or LAMP) installation all wrapped up in a SiteScope-like alternative.
Read more »

Posted in LoadRunner, PHP, Perl, Solaris, Ruby, SiteScope | No Comments »

A New Contractor Toy: USB to VGA

Ever been on a client site and stuck with 1024 x 768?

I remember when even that was a dream, but after you get used to widescreen dual VGA in your own environment, weaning yourself off this can be a challenge. If you believe Microsoft research, dual screen users “never go back to a mono display” …

A friend and I bought a couple of these for $145AUD, to use in our day to day contracting work environment, where monitors are a plenty, but security (or warranty) forbids you from cracking open the case and installing a new video card.

The performance is a little sluggish, and sometimes the refresh goes a little awry, but all in all, it’s pretty handy to have around when debugging code. I wouldn’t recommend trying to play games on it, but definitely good to have in your toolkit.

Posted in Hardware, Productivity | No Comments »

Monitoring Weblogic 9.2 with JMX and JRuby

After getting nowhere with lack luster HP support, I turned to the power of the Open Source community and got a very simple script up and running to remotely monitor Weblogic JVM Performance and JMS queues using JMX and JRuby.

Despite having some initial issues with the code, the author of the jmx4r module turned around a fix inside 24 hours so that I could connect to a Weblogic 9.2 server and look at its domainruntime MBeans. SiteScope can’t handle domainruntimes of these size (short of being told to ‘add more memory’ to a 3GB+ machine!). So exit stage left Mercury, enter stage right JRuby…
Read more »

Posted in LoadRunner, Ruby, Analysis, Weblogic, SiteScope | No Comments »