Practice Quiz Development

By Emily Chan, Summer 2019 NSF REU

One of my main projects this summer was designing a practice game that leveraged our large collection of high resolution images for volunteers to practice identifying macroinvertebrates.

BACKGROUND 

A tool for practicing macroinvertebrate ID has been one of the most requested features by volunteers and educators alike. Before getting started developing a game, we took a look at the current resources available online. While there are an abundance of identification-based practice games on the internet (though hardly any for macroinvertebrates specifically), many of these games have similar designs that use low resolution images or generic illustrations, have little possibility for user visual interaction, and use a handful of example specimens that never change, meaning users can usually only play the game once or twice before being able to memorize the answers. In addition, usually only four choices of answers are provided, multiple choice style. Examples below:

In contrast, when conducting ID activities in authentic citizen science practice, volunteers need to be able to identify a large number of macroinvertebrates that may be sampled from a stream, to work around variations in size, shape, and color, and to be able to choose the right identifier from a large list of choices.

Our goal in creating this game was to simulate authentic practice, so our design for this practice game was informed by the identification process for water quality monitoring, and the biotic indices used by volunteers to classify macroinvertebrates. ​

PAPER PROTOTYPE

One of my goals was to keep the game as similar to the actual website as possible, so that it’d be easy for Chris to actually implement. Therefore, my first prototype mimicked the panel design developed by Alice. Every mystery specimen would be a genus-level page, with the text in the panel replaced by a question panel. The key ideas in the paper prototype were: the game goes on indefinitely until you choose to stop (in order to take advantage of our large collection); after you finish, it compares the water quality rating you would have calculated based on your identifications with the actual water quality rating; it uses the diagnostic characters as hints; when you get an answer wrong, it provides information about the differences between the answer you chose and the correct answer; and it has all the same zooming and changing view functionality as the regular site.

Though every single practice game I looked at provided four choices you could choose from, it seemed that this might not make sense for our game--if the choices were randomly selected, we might end up in a situation where the user is asked to identify a mayfly, and the choices are clam, leech, beetle, and mayfly. Having choices that don't actually look similar to the specimen isn't particularly helpful for the user's learning, but if we wanted to only show visually similar choices, someone would have to manually create the choices available for each specimen, which seemed excessive. In addition, in the actual scenario, volunteers need to choose the right identifier from a very large number of choices. Therefore, I sketched a multiple choice version as well as a version with a dropdown menu.

XD PROTOTYPE 1

We collected feedback on this first XD prototype at the Macroinvertebrates.org site launch event, allowing educators and trainers to play around with the mockup and asking them about their thoughts. Their main concern was whether the game would be leveled--suggestions included student, volunteer, and expert levels. Who the target audience is turned out to be a central question that the game still struggles to fully address. We chose to focus on volunteers, and get a  working version of the game for that audience first. I also considered whether it'd be useful to separate by region or ecosystem (like the Cornell Bird Quiz does), but I didn't have enough information about who'd be using the game and what they'd want from it to make a judgement about that. We made some additional minor changes to the design based on suggestions, such as bringing the diagnostic characters back on the correct/incorrect page, including an illustration of the wrong answer as well (i.e. a small illustration of stoneflies next to "This is not a stonefly"), and trying alternatives to "incorrect" (e.g. "try again", "sorry", etc.).

XD PROTOTYPE 2

After getting feedback on the first XD prototype at the Macroinvertebrates.org site launch event, we made the following changes:
​

XD PROTOTYPE 4

For the next prototype, we decided to make use of the annotated illustrations that we had available (with the text enlarged) to replace or complement the informational text. 

XD PROTOTYPE 5

After consulting with Chris, our developer, we shifted towards a tab-panel design, to maintain consistency with the genus level pages and to make developing the game easier. The hints were shifted to the second tab, similar to the genus-level page, and the third tab would either be an "About this Game" tab, or a score-keeping tab. 

SPLASH SCREEN

Chris also informed us that the instructions and results would have to be shown on a splash screen. Below are examples of splash screens we considered:

XD PROTOTYPE 7

After getting more feedback from Tara, we decided to add a list of the ones you'd gotten incorrect in the third tab, with pictures of the specimen and the correct identifier. She also advised us to shift away from having the game be a simulated stream dump (i.e. calculating a water quality rating at the end), and rather focus simply on identification, since there were so many different systems and standards used to measure water quality. 

ICONS

 Examples of different icons we considered for the tabs:

---

FINAL PROTOTYPE

Minor changes for the final prototype included simplified icons and adjustment of the correct/incorrect screen.

FEEDBACK + RECOMMENDATIONS FOR FUTURE CHANGES

As it stands, the current game live on the site still needs to have the list and images of the incorrect specimens added, as well as small changes such as adjusting the category names (i.e. "Net-Spinning Caddisfly Larva" to "Caddisfly Larva (Net Spinning) "), swapping in illustrations with larger text, etc. Another major challenge is finding a more elegant way of dealing with the "None of the Above" category. User said that they enjoyed the challenge of identifying uncommon specimens, so I don't think that omitting specimens that would fall into that category is the solution. Since most of the "None of the Above" specimens were either true bugs or fly larva, one user suggested a "True Bugs" category. If future changes are to be made, I would recommend adding a "True Bugs" and a "Fly Larva (Other)"  category. Other user suggestions included adding scientific names, since many who used the game were more familiar with scientific names over common names.

However, the majority of user feedback centered around the lack of levelling in the practice game. Since the users were recruited from a broad base of experience levels, many wished that there was an expert version for identifing to family or genus; while others noted that they hadn’t been taught the distinction between particular taxa levels of macroinvertebrates, and felt such categories could be confusing. Future work includes producing a “student” (order-level) and an “expert" (genus) level game.

Users praised the custom-written informational text and illustrations on the correct/incorrect screen as well as the zoom and hint functionality. In our sample, no users expressed issues with the tab structure, start screen instructions, or the indefinite nature of the game, which were design choices we sought to validate. 
​

EXAMPLE EXPERT-LEVEL PROTOTYPE