|
BACKGROUND A tool for practicing macroinvertebrate ID has been one of the most requested features by volunteers and educators alike. Before getting started developing a game, we took a look at the current resources available online. While there are an abundance of identification-based practice games on the internet (though hardly any for macroinvertebrates specifically), many of these games have similar designs that use low resolution images or generic illustrations, have little possibility for user visual interaction, and use a handful of example specimens that never change, meaning users can usually only play the game once or twice before being able to memorize the answers. In addition, usually only four choices of answers are provided, multiple choice style. Examples below: In contrast, when conducting ID activities in authentic citizen science practice, volunteers need to be able to identify a large number of macroinvertebrates that may be sampled from a stream, to work around variations in size, shape, and color, and to be able to choose the right identifier from a large list of choices. Our goal in creating this game was to simulate authentic practice, so our design for this practice game was informed by the identification process for water quality monitoring, and the biotic indices used by volunteers to classify macroinvertebrates. PAPER PROTOTYPE One of my goals was to keep the game as similar to the actual website as possible, so that it’d be easy for Chris to actually implement. Therefore, my first prototype mimicked the panel design developed by Alice. Every mystery specimen would be a genus-level page, with the text in the panel replaced by a question panel. The key ideas in the paper prototype were: the game goes on indefinitely until you choose to stop (in order to take advantage of our large collection); after you finish, it compares the water quality rating you would have calculated based on your identifications with the actual water quality rating; it uses the diagnostic characters as hints; when you get an answer wrong, it provides information about the differences between the answer you chose and the correct answer; and it has all the same zooming and changing view functionality as the regular site. Though every single practice game I looked at provided four choices you could choose from, it seemed that this might not make sense for our game--if the choices were randomly selected, we might end up in a situation where the user is asked to identify a mayfly, and the choices are clam, leech, beetle, and mayfly. Having choices that don't actually look similar to the specimen isn't particularly helpful for the user's learning, but if we wanted to only show visually similar choices, someone would have to manually create the choices available for each specimen, which seemed excessive. In addition, in the actual scenario, volunteers need to choose the right identifier from a very large number of choices. Therefore, I sketched a multiple choice version as well as a version with a dropdown menu. XD PROTOTYPE 1 We collected feedback on this first XD prototype at the Macroinvertebrates.org site launch event, allowing educators and trainers to play around with the mockup and asking them about their thoughts. Their main concerns were: 1) Whether the game would be leveled--suggestions included student, volunteer, and expert levels. I also considered whether it'd be useful to separate by region or ecosystem (like the Cornell Bird Quiz does), but I didn't know if I had enough information about who'd be using the game and what they'd want from it to make a judgement about that. I think perhaps we could first choose a target audience, such as student or volunteer, and focus on getting a working version of the game for that audience, since who the user is might inform some of the choices about what kind of language we want to use. 2) Terms--this followed from the first concern, which is what kind of terminology we'd use ("two to three tails" vs "two cerci always sometimes accompanied by one median caudal filament"), and how users who are less familiar with the terminology might use the game. Suggestions included finding ways to connect the terms in the text to the image (ex: hovering over "two to three tails" would highlight the arrow that pointed to that diagnostic characteristic), which would work well with Marti's suggestion of bringing the diagnostic characteristics back on the correct/incorrect page. I agree about bringing the diagnostic characters back on the correct/incorrect page. The markers should also pop up on the image when you hit the Hint button -- on your prototype the markers don't appear until one is selected. As for the terminology, you don't need to worry about changing the wording. We went over and over what to call things with the various stakeholders (specifically tails v cerci, among others), and while not perfect for all users, it's what we have decided. Your game will be pulling fields from the CAT database and you shouldn't plan on re-writing any of the diagnostic character names or descriptions. That said, it is something that you'll want to pay attention to in user testing, if it becomes clear that the terms are too difficult (or simplistic) for users. 3) Including an illustration of the wrong answer as well (i.e. a small illustration of stoneflies next to "This is not a stonefly"), so users have some more context. Good idea 4) Having more context to the water quality rating at the end Yes. I could interpret that two ways: 1) More info about what the numbers mean, i.e. is 35.4 significantly different than 33.2. That would require setting up some logic where if the actual and calculated numbers fall within the same quality range you'd give a message like "You correctly calculated the water quality of this stream as Fair. Good job!" and if they are in different ranges, e.g. "Your calculations rate this stream as Good, but the actual calculation is Fair. Keep practicing!" That would help people make more sense of it. 2) More info about how you calculated the numbers, i.e. where did 35.4 come from. That would be good to include, but as a linked popup from the answer box, labeled "What do these numbers mean?" or "How were these numbers calculated?" or if space is an issue "More info." 5) Try again vs Incorrect Did you get feedback whether people preferred one over the other? Given that we are providing an IBI calculation at the end, I lean toward not letting them try again. They either correctly identify or not, just like in the stream (and they can of course always abandon and then restart the quiz if they need that perfect score). Changes I was going to make: 1) Underline and highlight glossary terms, like they are in the information panel on the actual website. Yes 2) Link the diagnostic characteristics at the correct/incorrect screen to the terms in the description. I don't exactly know what you mean here, but it seems like a good idea. 3) Add the illustration of the stonefly Yes 4) Possibly add pollution tolerance information, if you guys think it's a good idea? Yes XD PROTOTYPE 2 After getting feedback on the first XD prototype at the Macroinvertebrates.org site launch event, we made the following changes: XD PROTOTYPE 4 XD PROTOTYPE 5 SPLASH SCREEN XD PROTOTYPE 7 ICONS  CORRECT/INCORRECT SCREEN FINAL PROTOTYPE FEEDBACK + RECOMMENDATIONS FOR FUTURE CHANGES Small details including substituting illustrations with larger text more elegant way of dealing with None of the Above. I don't think getting rid of specimens is the solution, as users enjoyed the challenge Since most of the "None of the Above" specimens were either true bugs or fly larva, one user suggested a True Bugs category refining the categories Design Implications The majority of user feedback centered around the lack of levelling in the practice game. Since the users were recruited from a broad base of experience levels, many wished that there was an expert version for identifing to family or genus; while others noted that they hadn’t been taught the distinction between particular taxa levels of macroinvertebrates, and felt such categories could be confusing. Additional feedback concerned taxonomic details such as the use of scientific names vs. common names, the granularity of the categories, and small bugs in the implementation. Users praised the custom-written informational text and illustrations on the correct/incorrect screen as well as the zoom and hint functionality. In our sample, no users expressed issues with the tab structure, start screen instructions, or the indefinite nature of the game, which were design choices we sought to validate. Future work includes producing a “student” and an “expert level game. EXPERT-LEVEL PROTOTYPE
0 Comments
Goals Interviews Weekend Observation at the Frick Environmental Center We observed two main demographics at the Environmental Center on the weekend: older folks quietly browsing books near the bookshelf on their own, and parents with one to three kids, most under the age of 6, taking a break inside before or after a hike to cool down on a hot day. The older people who were just browsing usually only staying for a few minutes, but the parents with kids often stayed for 10-30 minutes. The parents seemed to use the living room as a space where they could step back from paying attention to their kids and instead sit in a corner and relax or use their phones while their kids amused themselves with the various toys in the room. When it came to interacting with the kit, many of the kids were very enthusiastic, but not necessarily in the way we’d like. Since the kids were much younger than we expected, they completely ignored any instructions, spoken or written, and required constant direction from their parents to be able to focus on engaging in any of the activities we’d prepared. Their preferred ways of interacting with our materials included moving the blocks from one box to the other, pouring the blocks onto the table, shaking the blocks, stacking them to build towers, using them as props or characters in make-believe stories, organizing the cards, etc. A few of the kids did engage in looking activities: one kid, who was shown how to look at blocks using the macro lens and flashlight, ran around looking at things in the room with the flashlight, and tried looking at each of the blocks under the macrolens, but only for a second or two each; another child expressed wonder, saying “wow” and making the observations “They kinda look scary but then they’re actually not”, “This one is like a giant house centipede”, and “It’s a stick bug”, as well as telling other kids “If you wanna look at bugs, you can”. Some older kids also tried to help younger ones interact with the exhibit correctly (“Here, use this” and “No Maisy, I’ll show you how to do it--you put it under here, and then you can see it better”). However, even those that did seem to understand the exhibit or expressed excitement or wonder (“Ooh bug!” or “Woah”) lost interest very quickly. Questions asked by the children included: “What is this bug?” “What are these for?” “What happens if you take them out?” In addition, multiple kids from different families said that they weren’t allowed to use the computer due to shabbat, which may be a complicating factor. Parents occasionally showed interest in the exhibit--one child brought a block over to her mother, who asked “What kind of bug is this?” Another parent came over to facilitate the exhibit for his son, giving directions like “Can you show me the legs?”, “Try this one, try the black one”, “What is this”, “Can you match this block?“ “you can see it larger” “Find me something like this”, “Are they the same? Are you sure?”, “Let’s try something else. What letter is this?”. When interviewed about what motivated him to facilitate, he simply said: “I just want to see what they understand.” However, for the most part, parents were indifferent to the exhibit. Insights:
Weekday Observation at the Frick Environmental Center Insights:
Concepts that we presented and got feedback on at the macro website launch event. By Wei Gong
After the first round user test, we gained insights, refined our concepts, and planned to do a second round user test. We will go on weekday this time to test the concepts with different audience. The testing plan is as follow: Goals
Materials
Activity Preparation
Observation Template
Interview Question to Audiences:
Interview Question to staff in the living room:
Next Step
By Wei Gong With some general concepts like guiding the visitors with some questions so that they can explore the exhibit on their own, and attracting the visitors with zoomable images on the screen, we did our first round user test in the living room of Frick Environmental Center. Actually, this is a combination of discovery research, generative research and user test, since we aimed to discover design opportunities, gain insights on design solutions, and test our concept at the same time. Before the test, we wrote a proposal for the test plan. User Test Proposal Goal
Materials
Observation Note Template
Interview Question to Audiences:
Interview Question to staff in the living room:
Findings We spent around 2 hours in the living room doing contextual inquiry and testing the concepts, from which we gained a lot of information. The key findings are shown below. Audience:
“I want to make a tower with these blocks” “They are so pretty” “How does it feel to taste bugs”
“So Cool” ---- After putting them down the magnifying glass.
“I’ll show you how to play with the blocks, put your magnifying lenses above the blocks.”
“You want to zoom in and out?” “Let’s try another one” “Can you match them(the block and the card) “Do you want to see the legs?” “I just noticed they are interested in it. I want to know whether they really understand it.”
“Don’t touch the computer” Length: Most children who are attracted to the exhibit can play with the kit as long as they can until their parents require them to leave, usually more than 20 minutes. Reflection After the test, I think we need to design some activities that the audience can really interact with, either by themselves or facilitated. In the user test, although the kids had fun with the materials, they didn't really interact with it and got information from it. Most of them simply played treated them as toys. Several ideas I have in mind:
By Wei Gong During this summer, our team aims to design an educator kit for the educators to engage kids in macroinvertebrate education. Our first step was to do research to understand the problem and identify design opportunities. Before doing field research, we generated driving questions that we wanted to answer by the end of the research phase. Driving questions:
With the driving questions in mind, we conducted semi-structured interviews for the following interviewees:
After the interviews, we consolidated the data into several categories and answered the driving questions as follow: Question1: What are the learning goals for the students in the water quality and macroinvertebrate education? Based on data we got from all the interviews, we concluded three major learning goals for the students in macroinvertebrate education as follow: 1. Learners will explore the connection between the insects and the ecosystem and have a basic awareness of
Quotes:
Quotes:
Question2: What elements form a successful and engaging macroinvertebrate education? Probing into the engaging elements in the macroinvertebarete education, we got 3 insights in how to create a successful exhibit kit as follow: 1. Create a live experience for the learners in which they can touch, feel, and observe. Quotes:
Quotes:
Quotes:
Question 3: What pain points do educators have for water quality and macroinvertebrate education? Although under different contexts, the educators similar challenges and painpoints in the macroinvertebrate education, which we concluded as follow: 1. Difficulty in creating a real experience where students can touch, feel and observe clearly. Quotes:
Quotes:
Question 4: What resources do educators use right now for the water quality biomonitoring activities and macroinvertebrate education? Among the resources, what works well and what doesn’t? Currently, educators use the resources below in macroinvertebrate education:
Question 5: How can technology play a role in macroinvertebrate education and what are the technological constraints in the context? We investigated how technology is applied in macroinvertebrate education now and the potential opportunities in the future. Key findings are as follow: 1. Educators have concerns on students using their individual phones in camps, field trips or other similar activities. Quotes:
Quotes:
By Wei Gong, Emily Chan, Alice Fang After the initial research, we decided to focus our design scope on an unfacilitated exhibit that can be placed in the living room, museum, classroom, or any other open space. Brainstorm Based on the research findings, we generated the initial HMWs and conducted a brainstorming session, after which we sorted our initial ideas into several categories.  HMW:
Concept Boards After the brainstorming session, we generated 6 design concepts in total and collected feedback at the macroinvertebrate.org launching event with the educators. Among our 6 ideas, Jigsaw and Balance the stream are the most popular and received some helpful feedback
Jigsaw:
Balance the Stream:
Design Challenges With the concepts and feedbacks from the educators, we consulted Jessica and Marti for feedbacks and suggestions on next steps. We consolidated their feedbacks into several design challenges:
On going... By Alice Fang, Summer 2019 NSF REU Another major Macroinvertebrates.org site feature that we iterated on was the dorsal/ventral/lateral view changer function, that existed as a rotating icon on the order and family pages, but also as a click-through menu on the genus pages. From prior usability tests, people failed to recognize the rotation icon was there, or that the function existed across the order/family level pages. Dezudio provided some solutions using a bar on the bottom left of the page, that had a blue triangle indicating which view was open. However, several issues arose with that solution:
 Looking at how fashion websites addressed multiple views  So many different thumbnail attempts...  Trying out thumbnails on the family level page, with limited success. Originally, I looked at different fashion and clothing sites, because those webpages had to find ways to display multiple views of items for customers. A common thread in every fashion site was the use of thumbnail images; that way someone could see all the possible images (and therefore all the views), but also know which image is in focus or zoomed in. I explored using thumbnail images of different views, but at a small scale, it became difficult to tell between dorsal or ventral views. There was also no way to implement this at the order/family level. Instead, I returned to the Dezudio design, and looked at how sites like Netflix presented numerous shows all at once, and the small visual cues they used for scrolling and clicking through the choices. I played around with different click-through, infinite scrolling of views; this method of displaying information failed to account for a ‘total view’, so someone would have no idea how many total views existed for a macroinvertebrate unless they actually clicked through every single option. To workaround this, I added what Marti likes to call a “cookie-crumb trail” where small icons show the total number of views, and as each view is scrolled through, the “crumbs” also scroll through.  A mockup of greying out a macroinvertebrate if it didn't have the specific view. The actual images don't match up, I was just prototyping based on the total number of views each macroinvertebrate had.  A family level page, using Caddisflies as the prototype because of the numerous views.  Translating the same function to a genus level page. A little redundant having infinite scroll with only three views, but it keeps the action consistent between pages. Chris implemented this differently on the live site. The triangle icon is grey when on hover, and blue when selected, and re-clicking the view returns it to its default; this reduces the copy/written information needed. If a view is selected, and a specific macroinvertebrate doesn't have that image, it is greyed out entirely, which I think looks really nice on the live site.  This eliminates the scrolling function; in hindsight, most macroinvertebrates didn't have so many views that the scroll was entirely necessary. Hi! I'm Alice, a rising junior at Carnegie Mellon University studying communication design with a minor in professional writing. I joined the macroinvertebrates team this summer as a design/research assistant, and I’ve been working on some site iterations for macroinvertebrates.org the past three weeks—namely, building stronger relationships between icon navigation and the actual informational content. I’ve learned a lot about prototyping quickly and keeping my work organized, in order to present solutions that have to rapidly be pushed to development. Having learned some basic web design and HTML/CSS, it was also a good learning opportunity in working with a developer to figure out what can or should be coded. On the previous version of the site, navigational icons floated above the panels that held important and interesting content. We wanted to change the structure for organizing the content without creating a major overhaul of the previous system.  The original structure for information, with four icons floating above. Some of the questions I had to figure out for creating a side panel:
One major issue was having a system where all of the content (order/family overview, media, iNaturalist photos, life history, characteristics) could be visible together but still consolidated. The original site had some strange floating/scrolling rules in order to fit all of the information. To address this, I explored different ‘drawer’, or side-panel, designs.  Sketches of tab functions, and existing navigation panels on other platforms. In the end, we took the ‘Media’ content and the iNaturalist photos and grouped them into their own separate category, which was then referred to as the Media Tab. The panel thus had three tabs at the top: Information, Diagnostic Characteristics, and Media. In trying to incorporate the ‘clear all’ function, to show the macroinvertebrates without any identifying information, we took advantage of the structure of the panel; when the panel closed, it also cleared all information, leaving just the images of the macroinvertebrates. The default then became having the panel open to the diagnostic characteristics tab.  Three tabs organizing the information in the panel.  Closing the panel clears the information. I also played around with different hover cues for the diagnostic characteristics. Clicking on a characteristic opened up a gallery that compared that characteristic for the selected macroinvertebrate with others that had the same feature.
 Playing with different hover states.  Exploring different icons and color indicators. Hi! My name is Wei Gong, a graduate student in HCI Institute, majored in Educational Technology and Applied Learning Science. I will explore the application of NFC technology in informal learning contexts this summer. Specifically for the Learning to See Project, I will design a popup kit for the Carnegie Museum of Natural History to engage kids in macroinvertebrate identification.
I will graduate in 2 months, so this will be a super busy summer for me, but I'm excited about it! My personal goal in doing this summer research is to explore the design opportunities in museum settings since I'm a museum fan. Identification Task To get started with the project and to gain an understanding of the design challenge, I did an macroinvertebrate identification task using various resources including field guides and key printouts. Identified four bugs in total. In order to experience different materials, I tried nearly all the materials at hand. Generally, the overall experience of the ID task is fun. It took much less time than I thought I would, which was a relief. Regarding the materials for the ID task, I personally found the flashcard most helpful and the big field guide book ( An introduction to the aquatic insects of North America) least helpful. The flashcard is quite straightforward with big images of the bugs and just right amount of text explanation. When I was using the flashcard to identify the bug, my process was to first visually match the specimen with the order level identification in the flashcard, and then observed the specimen in more detail, found its obvious traits, compared it with different families on the flashcard until finally distinguished it. I barely read the texts unless it's really necessary. I found the big guide book least useful because there's too much information which scared me away, and I think if only for the task of identification, it's not necessary to provide so much information. The key insights I got from this task are: 1. We can deliver to the kids the idea that the macroinvertevrate ID is not scary or time-consuming, but can be fun. 2. We should reduce the cognitive load for the volunteers/learners to identify the insects. One possible way is to utilize more media like comic/animation/image than text. By Emily Chan, Harvard University, Summer 2019 REU
Hi there! I'm Emily, a rising junior at Harvard studying computer science with a minor in statistics. I'm a week and a half into the HCII REU, and I'm working on two projects with Marti Louw, one of which is Macroinvertebrates.org. I came to the HCII REU because as a computer science major and an artist, I'm interested in learning more about design, and I also want to experience what doing research is like. Some of my personal goals for the summer are to understand how and why design choices are made, get hands on experience developing tools for users using design principles and user feedback, make something that I can show people/people will actually use, and learn about prototyping with smart devices. My tasks for Macroinvertebrates.org for the foreseeable future include doing usability testing for the new dichotomous key, glossary, and search functionality; creating an improved popup kit for museums to use to engage kids in macroinvertebrate identification, trying to leverage the high quality media that we have and potentially using smart technologies such as NFC tags, etc.; and possibly working on developing identification quiz practice functionality for the site. Identification Task To gain an understanding of the design challenges at hand, we did an identification task using the current resources available to volunteers--field guides and key printouts. The sheer amount of materials was slightly overwhelming, so as someone with no experience doing macroinvertebrate identification, I just chose two field guides that had pictures and stayed with those. I identified five bugs, four using the field guides and one using the website. What I noticed when using the field guides was that the detailed text descriptions of the bugs was essential for family level identification, but the volume of the text also gets in the way when it comes to order level identification. My process was to first do visual pattern-matching to the order level, and then read the descriptions of ones that looked similar to the specimen to find traits that would help distinguish between families. I was surprised by how identification wasn't as difficult or time-consuming as I anticipated, but I also didn't use most of the materials, and never used a key. When I switched to using the website, I tried a similar approach, but the website's organization was slightly different from a field guide (I couldn't just flip through quickly to scan for pictures and then selectively read descriptions), which took some adjusting to. The home page worked well for facilitating my initial pattern-matching approach, but I can see how if I were trying to identify a ambiguous looking larvae, I would want the home page to be more explorable (to be able to zoom in more), since some of the pictures were quite small. Once I clicked through to the order page, I definitely don't think I would have known how to get rid of the diagnostic characteristic boxes obstructing the actual pictures that I wanted to look at. However, since I was already familiar with how to do that, I was able to match the specimen to the common burrowing mayfly just by appearance, but I wasn't able to find the characteristics that might help distinguish between similar looking families in a convenient place. For me, this raised the question of whether the purpose of Macroinvertebrates.org is primarily to be a learning resource or an identification resource. |
Project TeamAn interdisciplinary team Archives
July 2019
Categories
All
|
RSS Feed
Project INFO |
Partners |
|
This material is based upon work supported by the National Science Foundation under Grant #1623969.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NSF.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NSF.