|
UW Card Sort Analyzer |
Card Sorting is a knowledge elicitation technique designed to explore how people organize and structure information. Subjects repeatedly categorize a stimulus set (e.g. index cards with different vocabulary terms on them) into separate piles based upon a self-chosen criterion. After each sort is formed, the structure of its piles and the criterion used are recorded. The subject is then encouraged to resort the cards using a newly self-chosen criterion.
Because the cards are constrained to a small set, quantitative analysis of card sort results is readily straightforward. For example, one can look at the co-occurence frequencies of pairs of cards being in the same pile. However, analyzing qualitative criterion descriptions is far more complex. Typically, content analysis is perform to group the sorts into superordinate groups in which each group reflects a human-decided central theme. Aside from the time expenses of performing content analysis, this stage of analysis is both subject to human biases and limits the size of card sort studies
As part of our analysis of a card sort study, my advisor, Richard Anderson, and I developed a series of tools that computationally look for similar card sorts. At the heart of this approach is the edit distance metric. This metric is a measure of how many cards need to be moved to transform one card sort into another. Using this metric, we have developed a series of tools (see the paper below for details) for finding and exploring small sets of closely related card sorts.
Currently, I am investigating the use of the distance metric to perform more large-scale clustering of card sort results. Though the experiments are still in their early stages, the preliminary results suggest that it is possible to "automate" the super-ordinate clustering process described above without having to use natural language programming. This approach will not be a complete replacement of the human-based strategy, though. The differences between the human and computer-based approaches will provide plenty of information to ponder.
Richard Anderson and Ruth Anderson and Katherine Deibel, "Analyzing Concept Groupings of Introductory Computer Programming Students," November, 2004. (pdf).
As part of my ongoing study of the edit distance metric, I have begun building a computer application for analyzing card sorts using this method. The UW Card Sort Analyzer is a C# Windows application that is freely available for use. Be certain to run the program locally since C# will not permit it to be run over a network drive.
The analyzer is currently still in production, so bugs are a possibility. If you encounter a problem or have a suggestion for improving the application, please contact me at the e-mail address below.
| Version | Program | Documentation |
|---|---|---|
| 1.1 | Analyzer (exe) | User's Manual (pdf) |
| 1.0 | Analyzer (exe) | User's Manual (pdf) |
Katherine Deibel's homepage
The Computer Science and Educational Technology Group Home Page