
Employing professional users of Office Software, SirValUse conducted a project for Sun Microsystems GmbH that covered several months and comprised various parts. One goal of the project consisted of indicating to the design teams the new StarOffice 8 version's actual potential of improvement while at the same time developing suggestions for its improvement based on the results of User Experience Tests.
Moreover, the usability tests were intended to provide an insight into the actual use of Microsoft Office in order to lower specifically the re-training costs for potential switchers to StarOffice. In the course of the project the usability of both, StarOffice 7 as well as StarOffice 8, was to be measured quantitatively in order to verify usability improvements by comparing the predecessor with the current version.
The diverse demands of the project and the comparison of both software versions required
Especially the conflicting demands on the test procedure presented a challenge: every test had to produce valid quantitative indices without the interviewer's exercising any influence; at the same time there was supposed to be room for the exploration of user behaviour that occurred during the test.
Thus, in close co-operation with Sun Microsystem's user experience team, SirValUse developed a test methodology, a usability index for quantitative measuring, and a detailed key to the recruitment of participants.
The team worked out a hybrid test procedure: a series of typical tasks and standard tests for word processing, spread sheet analysis and presentation tasks. During the test sessions (20 respondents per wave) measuring the behaviour (e.g. step by step solution of tasks, numbers of mistakes, duration) alternated with interview segments under simultaneous observation.
Conclusion:
Usability method Hybrid Testing