|
By Emily Chan, Summer 2019 NSF REU One of my main projects this summer was designing a practice game that leveraged our large collection of high resolution images for volunteers to practice identifying macroinvertebrates. BACKGROUND A tool for practicing macroinvertebrate ID has been one of the most requested features by volunteers and educators alike. Before getting started developing a game, we took a look at the current resources available online. While there are an abundance of identification-based practice games on the internet (though hardly any for macroinvertebrates specifically), many of these games have similar designs that use low resolution images or generic illustrations, have little possibility for user visual interaction, and use a handful of example specimens that never change, meaning users can usually only play the game once or twice before being able to memorize the answers. In addition, usually only four choices of answers are provided, multiple choice style. Examples below: In contrast, when conducting ID activities in authentic citizen science practice, volunteers need to be able to identify a large number of macroinvertebrates that may be sampled from a stream, to work around variations in size, shape, and color, and to be able to choose the right identifier from a large list of choices. Our goal in creating this game was to simulate authentic practice, so our design for this practice game was informed by the identification process for water quality monitoring, and the biotic indices used by volunteers to classify macroinvertebrates. PAPER PROTOTYPE One of my goals was to keep the game as similar to the actual website as possible, so that it’d be easy for Chris to actually implement. Therefore, my first prototype mimicked the panel design developed by Alice. Every mystery specimen would be a genus-level page, with the text in the panel replaced by a question panel. The key ideas in the paper prototype were: the game goes on indefinitely until you choose to stop (in order to take advantage of our large collection); after you finish, it compares the water quality rating you would have calculated based on your identifications with the actual water quality rating; it uses the diagnostic characters as hints; when you get an answer wrong, it provides information about the differences between the answer you chose and the correct answer; and it has all the same zooming and changing view functionality as the regular site. Though every single practice game I looked at provided four choices you could choose from, it seemed that this might not make sense for our game--if the choices were randomly selected, we might end up in a situation where the user is asked to identify a mayfly, and the choices are clam, leech, beetle, and mayfly. Having choices that don't actually look similar to the specimen isn't particularly helpful for the user's learning, but if we wanted to only show visually similar choices, someone would have to manually create the choices available for each specimen, which seemed excessive. In addition, in the actual scenario, volunteers need to choose the right identifier from a very large number of choices. Therefore, I sketched a multiple choice version as well as a version with a dropdown menu. XD PROTOTYPE 1 We collected feedback on this first XD prototype at the Macroinvertebrates.org site launch event, allowing educators and trainers to play around with the mockup and asking them about their thoughts. Their main concern was whether the game would be leveled--suggestions included student, volunteer, and expert levels. Who the target audience is turned out to be a central question that the game still struggles to fully address. We chose to focus on volunteers, and get a working version of the game for that audience first. I also considered whether it'd be useful to separate by region or ecosystem (like the Cornell Bird Quiz does), but I didn't have enough information about who'd be using the game and what they'd want from it to make a judgement about that. We made some additional minor changes to the design based on suggestions, such as bringing the diagnostic characters back on the correct/incorrect page, including an illustration of the wrong answer as well (i.e. a small illustration of stoneflies next to "This is not a stonefly"), and trying alternatives to "incorrect" (e.g. "try again", "sorry", etc.). XD PROTOTYPE 2 After getting feedback on the first XD prototype at the Macroinvertebrates.org site launch event, we made the following changes: XD PROTOTYPE 4 For the next prototype, we decided to make use of the annotated illustrations that we had available (with the text enlarged) to replace or complement the informational text. XD PROTOTYPE 5 After consulting with Chris, our developer, we shifted towards a tab-panel design, to maintain consistency with the genus level pages and to make developing the game easier. The hints were shifted to the second tab, similar to the genus-level page, and the third tab would either be an "About this Game" tab, or a score-keeping tab. SPLASH SCREEN Chris also informed us that the instructions and results would have to be shown on a splash screen. Below are examples of splash screens we considered: XD PROTOTYPE 7 After getting more feedback from Tara, we decided to add a list of the ones you'd gotten incorrect in the third tab, with pictures of the specimen and the correct identifier. She also advised us to shift away from having the game be a simulated stream dump (i.e. calculating a water quality rating at the end), and rather focus simply on identification, since there were so many different systems and standards used to measure water quality. ICONS Examples of different icons we considered for the tabs:  FINAL PROTOTYPE Minor changes for the final prototype included simplified icons and adjustment of the correct/incorrect screen. FEEDBACK + RECOMMENDATIONS FOR FUTURE CHANGES As it stands, the current game live on the site still needs to have the list and images of the incorrect specimens added, as well as small changes such as adjusting the category names (i.e. "Net-Spinning Caddisfly Larva" to "Caddisfly Larva (Net Spinning) "), swapping in illustrations with larger text, etc. Another major challenge is finding a more elegant way of dealing with the "None of the Above" category. User said that they enjoyed the challenge of identifying uncommon specimens, so I don't think that omitting specimens that would fall into that category is the solution. Since most of the "None of the Above" specimens were either true bugs or fly larva, one user suggested a "True Bugs" category. If future changes are to be made, I would recommend adding a "True Bugs" and a "Fly Larva (Other)" category. Other user suggestions included adding scientific names, since many who used the game were more familiar with scientific names over common names. However, the majority of user feedback centered around the lack of levelling in the practice game. Since the users were recruited from a broad base of experience levels, many wished that there was an expert version for identifing to family or genus; while others noted that they hadn’t been taught the distinction between particular taxa levels of macroinvertebrates, and felt such categories could be confusing. Future work includes producing a “student” (order-level) and an “expert" (genus) level game. Users praised the custom-written informational text and illustrations on the correct/incorrect screen as well as the zoom and hint functionality. In our sample, no users expressed issues with the tab structure, start screen instructions, or the indefinite nature of the game, which were design choices we sought to validate. EXAMPLE EXPERT-LEVEL PROTOTYPE by Emily Chan, 2019 NSF REU Goals One of the projects me and the other students working on Macroinvertebrates.org had this summer was to leverage the resources we had available to us to create a pop-up/tabling kit for educators to use. Digital resources we had access to included the website, with its information and high resolution zoomable images, and physical resources we had included lucite specimen blocks from Powdermill, specimens in liquid, magnifying lenses, flashlights, various field guides, and Voshell cards. Interviews As part of our initial research, we interviewed educators Pat, Jessica, Lauren, Taiji, Steven, and Nayja. We learned that there are a variety of settings educators work in, and our kit would depend on what setting we wanted to design for. Possibilities included a classroom kit (used standalone or as part of a rotational activity, engagement time of 45 min max), a tabling kit (around 5min), a stream dump field kit (around 20 min), or a living room kit (indefinite). We choose to focus on designing for the living room (pictures below), a space in the Frick Environmental Center with benches, chairs, shelves, and a TV, since it seemed to offer the most interesting possibilities. The educators articulated two fundamental learning goals: 1) to generate awareness of the bugs existence, and appreciation of their intrinsic value, which includes learning about their life history and their structure/function; 2) to promote understanding of macroinvertebrates' role in the ecosystem, and how they reflect the condition of the environment. I was more interested in the first goal, while another student on our team was more interested in the second one. We ended up creating two kits, one inspired by each learning goal. Concept Development Concepts that we presented and got feedback on at the macro website launch event. Educators seemed most intrigued by the "Balance-the-Stream Game". However, before jumping into developing one concept, we realized we still didn't have much information on who our audience was and how they would engage with our kit (what kind of questions would they ask about the bugs? How would they interact with the blocks? Which aspects of the bugs would they be most interested in and want to know more about?), and so we decided to go to the Frick Environmental Center to do some observation. Weekend Observation at the Frick Environmental Center We observed two main demographics at the Environmental Center on the weekend: older folks quietly browsing books near the bookshelf on their own, and parents with one to three kids, most under the age of 6, taking a break inside before or after a hike to cool down on a hot day. The older people who were just browsing usually only staying for a few minutes, but the parents with kids often stayed for 10-30 minutes. The parents seemed to use the living room as a space where they could step back from paying attention to their kids and instead sit in a corner and relax or use their phones while their kids amused themselves with the various toys in the room. When it came to interacting with the kit, many of the kids were very enthusiastic, but not necessarily in the way we’d like. Since the kids were much younger than we expected, they completely ignored any instructions, spoken or written, and required constant direction from their parents to be able to focus on engaging in any of the activities we’d prepared. Their preferred ways of interacting with our materials included moving the blocks from one box to the other, pouring the blocks onto the table, shaking the blocks, stacking them to build towers, using them as props or characters in make-believe stories, organizing the cards, etc. A few of the kids did engage in looking activities: one kid, who was shown how to look at blocks using the macro lens and flashlight, ran around looking at things in the room with the flashlight, and tried looking at each of the blocks under the macrolens, but only for a second or two each; another child expressed wonder, saying “wow” and making the observations “They kinda look scary but then they’re actually not”, “This one is like a giant house centipede”, and “It’s a stick bug”, as well as telling other kids “If you wanna look at bugs, you can”. Some older kids also tried to help younger ones interact with the exhibit correctly (“Here, use this” and “No Maisy, I’ll show you how to do it--you put it under here, and then you can see it better”). However, even those that did seem to understand the exhibit or expressed excitement or wonder (“Ooh bug!” or “Woah”) lost interest very quickly. Questions asked by the children included: “What is this bug?” “What are these for?” “What happens if you take them out?” In addition, multiple kids from different families said that they weren’t allowed to use the computer due to shabbat, which may be a complicating factor. Parents occasionally showed interest in the exhibit--one child brought a block over to her mother, who asked “What kind of bug is this?” Another parent came over to facilitate the exhibit for his son, giving directions like “Can you show me the legs?”, “Try this one, try the black one”, “What is this”, “Can you match this block?“ “you can see it larger” “Find me something like this”, “Are they the same? Are you sure?”, “Let’s try something else. What letter is this?”. When interviewed about what motivated him to facilitate, he simply said: “I just want to see what they understand.” However, for the most part, parents were indifferent to the exhibit. Insights:
Weekday Observation We went back to the Environmental Center on a weekday around 2pm, and stayed until 5. There were far fewer people there than we'd anticipated--it seems that most of the students doing summer programs didn't wait for pickup inside the center. We talked to staff at the center, who recommended that if we wanted to do a living room kit, we should find some way to secure all the pieces of the kit, since everything that isn't nailed down would be easily lost and broken, and there's no one to supervise or chaperone. There were one or two older kids around who had naturalist training, and had the observational skills to engage properly with the matching activity. They seemed to enjoy the activity, and their only suggestion was making sure that all the blocks had matching cards. Insights:
Final Prototype Based on our observation at the Environmental Center, we realized that the living room was probably not the most suitable place for our kit. Therefore, we focused on creating something flexible, that could set up somewhere and used in a variety of settings. Since the kids seemed to like the matching and sorting activities, we decided to create a box that unfolded into a stream-themed board that could be used for either activity. Since the specimens we had were representative of what one would find at Powdermill, we decided to use a photo of Powdermill on the inside of the box, so that the box would be like an ecological vignette of that particular location. As an alternative to the clunky and fragile computer/projector setup, we built simple lettered stands for the blocks that had NFC tags linked to that specimen's webpage on the bottom, so that the tags wouldn't be directly on the blocks (which would prevent users from being able to look at the insects from any angle), and got a small NFC enabled tablet to go with them. In addition to lucite blocks, NFC tagged stands, and a tablet, the box also contains a macro lens and a flashlight. We presented our final ideas to museum educators Pat and Jen, who gave us feedback. They said that they could see themselves using the kit and didn't think there was much they would change about it. Next steps:
If work on the kit continues, I'd like to see one more iteration using a sturdier material, such as gatorboard, with the lining printed on vinyl or a similar waterproof material. It could also incorporate magnets along the edges, so the box snaps together. In addition, it would be nice to see some way of making the cards detachable/creating other activities for the board that could be swapped in and out. Most importantly, I think there would be a lot of educational value in creating an additional box for Frick Park using an image of a stream in Frick park as the background in order to highlight the way that the diversity of the insects found reflect the ecology and pollution of a site, to tie in the other learning goal that educators had. By Wei Gong
After our first round of user testing, we gained insights, refined our original concepts and came up with new ideas. We decided to project with zoomable images with the projector, and designed two activities with which the audience can play. With these ideas, we plan to do a second round user test. We will go on weekday this time to test the concepts with different audience. The testing plan is as follow: Goals
Materials
Activity Preparation
Observation Template
Interview Question to Audiences:
Interview Question to staff in the living room:
Next Step
By Wei Gong With some general concepts like guiding the visitors with some questions so that they can explore the exhibit on their own, and attracting the visitors with zoomable images on the screen, we did our first round user test in the living room of Frick Environmental Center. Actually, this is a combination of discovery research, generative research and user test, since we aimed to discover design opportunities, gain insights on design solutions, and test our concept at the same time. Before the test, we wrote a proposal for the test plan. User Test Proposal Goal
Materials
Observation Note Template
Interview Question to Audiences:
Interview Question to staff in the living room:
Findings We spent around 2 hours in the living room doing contextual inquiry and testing the concepts, from which we gained a lot of information. The key findings are shown below. Audience:
“I want to make a tower with these blocks” “They are so pretty” “How does it feel to taste bugs”
“So Cool” ---- After putting them down the magnifying glass.
“I’ll show you how to play with the blocks, put your magnifying lenses above the blocks.”
“You want to zoom in and out?” “Let’s try another one” “Can you match them(the block and the card) “Do you want to see the legs?” “I just noticed they are interested in it. I want to know whether they really understand it.”
“Don’t touch the computer” Length: Most children who are attracted to the exhibit can play with the kit as long as they can until their parents require them to leave, usually more than 20 minutes. Reflection After the test, I think we need to design some activities that the audience can really interact with, either by themselves or facilitated. In the user test, although the kids had fun with the materials, they didn't really interact with it and got information from it. Most of them simply played treated them as toys. Several ideas I have in mind:
By Wei Gong During this summer, our team aims to design an educator kit for the educators to engage kids in macroinvertebrate education. Our first step was to do research to understand the problem and identify design opportunities. Before doing field research, we generated driving questions that we wanted to answer by the end of the research phase. Driving questions:
With the driving questions in mind, we conducted semi-structured interviews for the following interviewees:
After the interviews, we consolidated the data into several categories and answered the driving questions as follow: Question1: What are the learning goals for the students in the water quality and macroinvertebrate education? Based on data we got from all the interviews, we concluded three major learning goals for the students in macroinvertebrate education as follow: 1. Learners will explore the connection between the insects and the ecosystem and have a basic awareness of
Quotes:
Quotes:
Question2: What elements form a successful and engaging macroinvertebrate education? Probing into the engaging elements in the macroinvertebarete education, we got 3 insights in how to create a successful exhibit kit as follow: 1. Create a live experience for the learners in which they can touch, feel, and observe. Quotes:
Quotes:
Quotes:
Question 3: What pain points do educators have for water quality and macroinvertebrate education? Although under different contexts, the educators similar challenges and painpoints in the macroinvertebrate education, which we concluded as follow: 1. Difficulty in creating a real experience where students can touch, feel and observe clearly. Quotes:
Quotes:
Question 4: What resources do educators use right now for the water quality biomonitoring activities and macroinvertebrate education? Among the resources, what works well and what doesn’t? Currently, educators use the resources below in macroinvertebrate education:
Question 5: How can technology play a role in macroinvertebrate education and what are the technological constraints in the context? We investigated how technology is applied in macroinvertebrate education now and the potential opportunities in the future. Key findings are as follow: 1. Educators have concerns on students using their individual phones in camps, field trips or other similar activities. Quotes:
Quotes:
By Wei Gong, Emily Chan, Alice Fang After the initial research, we decided to focus our design scope on an unfacilitated exhibit that can be placed in the living room, museum, classroom, or any other open space. Brainstorm Based on the research findings, we generated the initial HMWs and conducted a brainstorming session, after which we sorted our initial ideas into several categories.  HMW:
Concept Boards After the brainstorming session, we generated 6 design concepts in total and collected feedback at the macroinvertebrate.org launching event with the educators. Among our 6 ideas, Jigsaw and Balance the stream are the most popular and received some helpful feedback
Jigsaw:
Balance the Stream:
Design Challenges With the concepts and feedbacks from the educators, we consulted Jessica and Marti for feedbacks and suggestions on next steps. We consolidated their feedbacks into several design challenges:
On going... |
Project TeamAn interdisciplinary team Categories
All
Archives
January 2021
|
RSS Feed
Project INFO |
Partners |
|
This material is based upon work supported by the National Science Foundation under Grant #1623969.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NSF.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NSF.