By Emily Chan, Summer 2019 NSF REU
One of my main projects this summer was designing a practice game that leveraged our large collection of high resolution images for volunteers to practice identifying macroinvertebrates.
A tool for practicing macroinvertebrate ID has been one of the most requested features by volunteers and educators alike. Before getting started developing a game, we took a look at the current resources available online. While there are an abundance of identification-based practice games on the internet (though hardly any for macroinvertebrates specifically), many of these games have similar designs that use low resolution images or generic illustrations, have little possibility for user visual interaction, and use a handful of example specimens that never change, meaning users can usually only play the game once or twice before being able to memorize the answers. In addition, usually only four choices of answers are provided, multiple choice style. Examples below:
In contrast, when conducting ID activities in authentic citizen science practice, volunteers need to be able to identify a large number of macroinvertebrates that may be sampled from a stream, to work around variations in size, shape, and color, and to be able to choose the right identifier from a large list of choices.
Our goal in creating this game was to simulate authentic practice, so our design for this practice game was informed by the identification process for water quality monitoring, and the biotic indices used by volunteers to classify macroinvertebrates.
One of my goals was to keep the game as similar to the actual website as possible, so that it’d be easy for Chris to actually implement. Therefore, my first prototype mimicked the panel design developed by Alice. Every mystery specimen would be a genus-level page, with the text in the panel replaced by a question panel. The key ideas in the paper prototype were: the game goes on indefinitely until you choose to stop (in order to take advantage of our large collection); after you finish, it compares the water quality rating you would have calculated based on your identifications with the actual water quality rating; it uses the diagnostic characters as hints; when you get an answer wrong, it provides information about the differences between the answer you chose and the correct answer; and it has all the same zooming and changing view functionality as the regular site.
Though every single practice game I looked at provided four choices you could choose from, it seemed that this might not make sense for our game--if the choices were randomly selected, we might end up in a situation where the user is asked to identify a mayfly, and the choices are clam, leech, beetle, and mayfly. Having choices that don't actually look similar to the specimen isn't particularly helpful for the user's learning, but if we wanted to only show visually similar choices, someone would have to manually create the choices available for each specimen, which seemed excessive. In addition, in the actual scenario, volunteers need to choose the right identifier from a very large number of choices. Therefore, I sketched a multiple choice version as well as a version with a dropdown menu.
XD PROTOTYPE 1
We collected feedback on this first XD prototype at the Macroinvertebrates.org site launch event, allowing educators and trainers to play around with the mockup and asking them about their thoughts. Their main concern was whether the game would be leveled--suggestions included student, volunteer, and expert levels. Who the target audience is turned out to be a central question that the game still struggles to fully address. We chose to focus on volunteers, and get a working version of the game for that audience first. I also considered whether it'd be useful to separate by region or ecosystem (like the Cornell Bird Quiz does), but I didn't have enough information about who'd be using the game and what they'd want from it to make a judgement about that. We made some additional minor changes to the design based on suggestions, such as bringing the diagnostic characters back on the correct/incorrect page, including an illustration of the wrong answer as well (i.e. a small illustration of stoneflies next to "This is not a stonefly"), and trying alternatives to "incorrect" (e.g. "try again", "sorry", etc.).
XD PROTOTYPE 2
After getting feedback on the first XD prototype at the Macroinvertebrates.org site launch event, we made the following changes:
XD PROTOTYPE 4
For the next prototype, we decided to make use of the annotated illustrations that we had available (with the text enlarged) to replace or complement the informational text.
XD PROTOTYPE 5
After consulting with Chris, our developer, we shifted towards a tab-panel design, to maintain consistency with the genus level pages and to make developing the game easier. The hints were shifted to the second tab, similar to the genus-level page, and the third tab would either be an "About this Game" tab, or a score-keeping tab.
Chris also informed us that the instructions and results would have to be shown on a splash screen. Below are examples of splash screens we considered:
XD PROTOTYPE 7
After getting more feedback from Tara, we decided to add a list of the ones you'd gotten incorrect in the third tab, with pictures of the specimen and the correct identifier. She also advised us to shift away from having the game be a simulated stream dump (i.e. calculating a water quality rating at the end), and rather focus simply on identification, since there were so many different systems and standards used to measure water quality.
Examples of different icons we considered for the tabs:
Minor changes for the final prototype included simplified icons and adjustment of the correct/incorrect screen.
FEEDBACK + RECOMMENDATIONS FOR FUTURE CHANGES
As it stands, the current game live on the site still needs to have the list and images of the incorrect specimens added, as well as small changes such as adjusting the category names (i.e. "Net-Spinning Caddisfly Larva" to "Caddisfly Larva (Net Spinning) "), swapping in illustrations with larger text, etc. Another major challenge is finding a more elegant way of dealing with the "None of the Above" category. User said that they enjoyed the challenge of identifying uncommon specimens, so I don't think that omitting specimens that would fall into that category is the solution. Since most of the "None of the Above" specimens were either true bugs or fly larva, one user suggested a "True Bugs" category. If future changes are to be made, I would recommend adding a "True Bugs" and a "Fly Larva (Other)" category. Other user suggestions included adding scientific names, since many who used the game were more familiar with scientific names over common names.
However, the majority of user feedback centered around the lack of levelling in the practice game. Since the users were recruited from a broad base of experience levels, many wished that there was an expert version for identifing to family or genus; while others noted that they hadn’t been taught the distinction between particular taxa levels of macroinvertebrates, and felt such categories could be confusing. Future work includes producing a “student” (order-level) and an “expert" (genus) level game.
Users praised the custom-written informational text and illustrations on the correct/incorrect screen as well as the zoom and hint functionality. In our sample, no users expressed issues with the tab structure, start screen instructions, or the indefinite nature of the game, which were design choices we sought to validate.
EXAMPLE EXPERT-LEVEL PROTOTYPE
Hi! I'm Alice, a rising junior at Carnegie Mellon University studying communication design with a minor in professional writing. I joined the macroinvertebrates team this summer as a design/research assistant, and I’ve been working on some site iterations for macroinvertebrates.org the past three weeks—namely, building stronger relationships between icon navigation and the actual informational content. I’ve learned a lot about prototyping quickly and keeping my work organized, in order to present solutions that have to rapidly be pushed to development. Having learned some basic web design and HTML/CSS, it was also a good learning opportunity in working with a developer to figure out what can or should be coded.
On the previous version of the site, navigational icons floated above the panels that held important and interesting content. We wanted to change the structure for organizing the content without creating a major overhaul of the previous system.
Some of the questions I had to figure out for creating a side panel:
One major issue was having a system where all of the content (order/family overview, media, iNaturalist photos, life history, characteristics) could be visible together but still consolidated. The original site had some strange floating/scrolling rules in order to fit all of the information. To address this, I explored different ‘drawer’, or side-panel, designs.
In the end, we took the ‘Media’ content and the iNaturalist photos and grouped them into their own separate category, which was then referred to as the Media Tab. The panel thus had three tabs at the top: Information, Diagnostic Characteristics, and Media.
In trying to incorporate the ‘clear all’ function, to show the macroinvertebrates without any identifying information, we took advantage of the structure of the panel; when the panel closed, it also cleared all information, leaving just the images of the macroinvertebrates. The default then became having the panel open to the diagnostic characteristics tab.
I also played around with different hover cues for the diagnostic characteristics. Clicking on a characteristic opened up a gallery that compared that characteristic for the selected macroinvertebrate with others that had the same feature.
by Chelsea Cui, undergraduate research assistant, CMU
Hi, it’s Chelsea again! This spring I have been working with Jaclyn, our new designer, on a revision of the Order level key. Based on our previous usability test results, this time we decided to focus on the improvement of interactivity and element affordance. We wondered if making the key clickable and having paths highlight would make the key easier to follow. Here’s Jaclyn’s rough prototype:
At this point, I have added all the popup windows and labeled most order pictures, but I will not be in CMU during the summer, so this project is on hold. For future developers, I would suggest to follow these steps:
by Jamie Dorst, REU, CMU Human-Computer Interaction Institute
Recently, I've been working on more order level sheets since the Trichoptera order level sheet was such a hit at the November Stroud training. I created two more: one for Plecoptera and one for Ephemeroptera. It was a fun challenge trying to create these layouts, since each order varied in terms of number and size of specimens and amount of copy text. I also revised the initial version of the Trichoptera sheet, and put the insects on a gray background so they look better. Here are the latest versions! They'll be available for download soon on the Resources page.
by Chelsea Cui, CMU undergraduate researcher
Hi, I'm Chelsea, a third-year undergraduate in Information Systems with a double major in Statistics and Machine Learning. I started on the macroinvertebrates.org project last fall because I was interested in building platform that assists people with online learning. Since I started, I have worked on a few different parts of the project, including log file analysis and development of the mobile app. Right now, I'm working with Jamie to build an interactive Order level key.
We want something to help new users who don't know the insects very well. I took the initial layout by Jamie and made clickable versions. With HTML and CSS, I added hyperlink sections to the couplets in the key, and also implemented pop-up windows on each of them to show the corresponding diagnostic characters. This version is just a quick prototype. I am doing usability tests right now to get feedback before making the final design.
In the first round of tests, I found out that our current key design doesn’t look interactive enough to many users. Users may not think the key is clickable because the buttons are kind of simple and plain. For the next step, I’m planning to focus on the interactive design and create several different versions of the key to do more usability tests.
The initial version of this key is live on the new site now. Check it out here! (http://macroinvertebrates.org/key).
by Jamie Dorst, REU, Carnegie Mellon
Hi! My name is Jamie Dorst, and I’m a new Research Assistant on the Learning to See, Seeing to Learn project. Most of my responsibilities so far have been content related, such as creating an updated landing page, annotating illustrations, and creating an expansive Trichoptera info sheet for the family level training led by Stroud Research Center last November. I worked with Tara to put together a handout for participants showcasing all the caddisflies in our collection. I’ve really enjoyed my time on the team so far!
My latest project has been to create an interactive digital dichotomous key, which has been really interesting. Analog keys are very commonly used, but we felt that a key utilizing our resources could be extremely useful. We wanted to make a key that was easy to navigate, and incorporated our images and information. I started by laying a key out based on the Stroud key, placed our images into it, and then created clickable pop ups that people can use to help move through the key. With this tool, people can identify their insects, and simultaneously learn about the characteristics they have. It’s been super interesting for me to work on this and learn about how to make it really well designed. The next step is to create an interaction for the actual website, which I’ll be working on soon.
In the meantime, here's a preview of drafts of a few of the images!
Technical Report Released - Aquatic Macroinvertebrate Identification Trainings for Volunteers: Results of a National Materials and Practices Inventory Survey
by Jessica Roberts, postdoctoral researcher, HCII
In the fall of 2016 we launched a survey to citizen science water quality trainers throughout the country. We wanted to get a better sense of the state-of-the-field: What do trainings look like? Who are the trainers, and who are the volunteers? What materials and resources do they use?
The primary goal of this survey was to provide information about potential users and their needs as we began the redesign efforts expanding macroinvertebrates.org to the full 150-specimen collection (to be released early in 2019!). In the process, we gained valuable insight about the large community of educators and organizations engaged in this work, which we have consolidated into a technical report.
This report, published through CMU, presents data on six key areas of focus:
You can access the report through the Carnegie Mellon School of Computer Science at this link.
Special thanks to all the team members who contributed to this project, particularly our partners at Stroud Water Research Center, former postdoc Lauren Allen, and undergraduate research assistants Grace Guo and Aiqi Cui.
by Jessy Ma, NSF Research Experience for Undergraduates intern, Brown University
During my time here at CMU, my main focus was to conduct website usability tests and take what I learned from them to design a mobile companion app. At the same time, however, I got abundant exposure to the research process and the intricacies involved in conducting successful research. Things I did outside of designing the app included:
by Jessy Ma, NSF Research Experience for Undergraduates intern, Brown University
One of my main projects this summer was to design and develop a mobile companion app for macroinvertebrates.org. Initially, I had to decide whether to make a reference app that would essentially give information like the website does or make a game that would allow people to practice identification. I got feedback from my mentors, and at this point I had also started conducting usability tests for the alpha version of the website, so I got a feel for what users would use the site for and what was most important to them. I decided to do a reference app that would be a simplified offline version of the site, since this would be most helpful to people doing identification out in the streams, where Wi-Fi and cellular service can be unreliable.
In order to scale the site down into a mobile app that can be used offline, I had to figure out which features on the site were most important to include and how to include them without compromising user experience. I iterated my design a few times based on feedback from my mentors and from website usability tests, and once I was happy enough with the design, I conducted usability tests with the Young Naturalists at Pittsburgh Parks Conservancy. Here is an example of how one screen (the full specimen view) changed throughout the process:
The changes I made after the Young Naturalists' feedback are the final iteration displayed on my poster, which I presented at a poster session with all the other HCII summer REUs.
After I had already submitted my poster, I had a meeting with the designers of the website and made further changes to the prototype, which can be found here: https://marvelapp.com/db804b9/.
An interdisciplinary team