There's a growing number of people in the testing world doing something I've been calling Method Testing, although that's probably not the right term. Named after Method Acting (Wikipedia : Method actors are often characterized as immersing themselves in their characters to the extent that they continue to portray them even offstage or off-camera for the duration of a project.) I'm using this term to describe the testers who emerse themselves in their end users environment to learn how their customers use their product and in what way and on what devices. They then bring this back to the lab and test against it, often in situ, often with real end users.
In my opinion this is taking testing to a higher level than many can realistically go, but it all comes down to your goals, budgets and aims. Unfortunately, those who do Method Testing don't seem to write much about it..... In the examples I hear about it's not uncommon for the test team to prepare a "real life" environment (not just a real life computer configuration) in which to test, often using real end users, sometimes using actors and often using props.
This differs from UAT in that they are doing this as part of their day to day testing, not just at the end, when to be honest..it's often far too late.
A friend of mine tests medical equipment and it's associated software. He has a fake emergency room kitted out with dummies (in various medical emergency positions), medical equipment and costumes, he even uses fake blood. Proving the software works (or doesn't) is one thing but making sure it works under pressure, in situ and is usable is something completely different. Usability testing at it's best. He even employs actors once a week to act as patients adding extra drama and realism. They often bring in real surgical teams to run through a typical procedure. The whole development team observe the behaviour and adjust accordingly. He films the sessions to learn more about the actions performed in surgery and potential flaws with design. Sure it's expensive...very expensive, but it's invaluable for him. The software industry he is in demands working kit..all the time. I'm trying desperately to get him to write about it.
At the almost completely free end of the spectrum, another friend of mine tests call centre software and he has a small call centre environment set up with a seating plan in the corner of the office, so he and his team can role play a typical call centre scenario. He's not only testing the software, but testing in sudo-situ, with terminals, seats, wallboards and background chatter.
He suggests that it "tricks the mind" in to thinking differently about the software. I think he's on to something.
I know of several testers who test their software whilst out and about on trams, trains, at the beach, from a cafe etc on a variety of mobile devices using free wi-fi or shared connections. That's because they know roughly how and where their software is to be used. They are testing in situ. They are exploring, learning, feeding back. They are immersing themselves in the environments their customers will use their product in. They are doing method testing. They are also extremely lucky :)
But what ties all of the above is that they are all reporting remarkable results in customer satisfaction <-- the ultimate feedback on product acceptability?
They are making an effort to emulate the end users real environment, to make sure the softwares original purpose is right, for the right people and under the right contexts.
But it's too tricky to find out who uses our software and how they use it..isn't it?
Not always. For a small few this might be true but in reality the majority of us just don't bother to seek out this information or we look in the wrong place. Why not try one or all of the following:
1. Speak to support desk or customer services. They are on the front line of complaints and feedback and can give you invaluable insight. They often maintain lots of records on system configurations and usage patterns. They don't stash it away from you to be mean, they probably just don't know you want it.
2. Speak to sales or accounts teams. They sell the stuff and maintain relationships with the customers. They will have more information than you would imagine.
3. Attend a sales pitch. You will learn loads, trust me. You'll also see the types of questions your customers ask about the product.
4. Attend (or give) a training session on the softwared to real users. You can observe them first hand and see which bits trip them up or confuse them.
5. For webapps or websites use Google Analytics or other website traffic monitoring tools or build in some performance/usage measuring. The amount of information Google Analytics gives you is insane. Browsers, screen resolutions, locales, referal sites, devices. You can find out bounce rates, pages navigated to/from, time on page, links hit etc etc etc etc Mind blowing stuff that really will make a difference to your test approach.
6. Talk to the customers....(might be worth checking with key people in your company first though :) )The majority of customers are more than happy to show you around their offices and show you the software in use, especially if it's not working too well!
7. Give customers easy and simple ways of reporting back problems from your software. Encourage them to be clear about the issues and log pertinent information.
8. Actively encourage customer involvement by hosting forums and chat rooms where they can discuss your product. Encourage them to be truthful and honest. Engage with them. You'll be amazed at how receptive customers are to honest and upfront conversations. Ask for feedback on what environments your product gets used in. Ask for the truth about your product. You can't blindly push on releasing if people are hating on your product. Or you can, but it won't be for long :)
The information is out there..everywhere. It's a case of widening your awareness fields and grabbing it. Analysing it. Using it. Reacting to it. Learn how your users use your product and you are gaining invaluable feedback on how to test it. It's never a replacement for real end users but it's a great addition to testing your software in a test lab with controlled environments and no idea about how your users use your software.
It might be time to step outside of this test lab comfort zone and do some in situ testing. Or should that be Method testing? Customer emulation testing? End user situ testing? No doubt someone will have a good name for it. I personally like method testing. And although your managers may think your mad for wanting to test in the same environments as your users, they'll appreciate it in the long run. There is, after all, some method in your madness.
