MACROINVERTEBRATES.ORG
  • PROJECT
  • PEOPLE
  • PRODUCTS
  • Blog
  • Resources
    • Help
    • FAQ

USGS releases North American Aquatic Macroinvertebrate Digital Reference Collection

5/31/2017

 
USGS scientists based at the Aquatic Experimental Lab recently released the North American Aquatic Macroinvertebrate Digital Reference Collection.  Our team is taking a close look at how the content and images and in this Reference Collection  are being handled to support taxonomic ID.
Picture
The Society for Freshwater Science reviews the site in this newsletter.

Advisory Meeting & Design Charrette Reflections

5/14/2017

 

by Nora Tane

Nora Tane is an MHCI student at Carnegie Mellon University. To learn more about her role in this project, visit noratane.design.


In late May, the full Macroinvertebrates.org project team and advisors gathered at Stroud Water Research Center for a day and a half-long advisory meeting and design workshop.  A brief summary of workshop activities and key discussion points follows. We'll publish a full workshop report by the end of the summer. 

Macroinvertebrates.org is an NSF-funded three-year interdisciplinary collaboration to develop an open educational resource to support insect identification in education and citizen science biomonitoring efforts. The workshop was held at the end of the first year of the project with Prototype 1 created, and was the first and only full team meeting for the project. Over the next two years, Prototype 2 will be developed and the final website/application will be released.

Project Objectives

  • Make learning aquatic macroinvertebrate identification easier for non-experts by increasing confidence, accuracy, and engagement.
  • ​Improve the quality of data contributing to citizen science and environmental decisionmaking.
  • Understand how networked platforms can be used to support the observational skills and practices of learners with diverse experience and motivations.

Project Team

Core Research Partners

Picture
Picture
Picture
Picture
Picture
Picture
Picture

NSF Advisors

Picture
Picture
Picture
University of Vermont Watershed Science Policy & Education
Picture
Picture
Picture
University of Maryland Environmental Science Studies

Co-Design Partners

Picture
Picture
Picture
Picture
Picture

Workshop Objectives

  • Review Year 1 research findings & design implications from core partners in the six inter-related strands of project: design, software, imaging, entomology, learning, and training & evaluation.
  • Conduct visioning and prioritizing activities for the redesign of macroinvertebrates.org.
​
At the beginning of the workshop, participants introduced themselves and quickly shared what they believe to be the hardest part of teaching/learning organism identification. Common themes included:
  • Vocabulary and terminology are difficult to learn and remember
  • ​Variability in organism appearance and real world difference from field guide images
  • Building accuracy, confidence, and preventing discouragement with limited training sessions​
Picture
Picture
Picture

Research Findings & Design Implications

The full meeting of project partners provided a unique opportunity to present findings from different project strands over the the first year and discuss design implications for the website.

National Macroinvertebrate Training Survey
Presented by Tara Muenz and Matthew Wilson of Stroud Water Research Center
​​

​Findings
  • Most trainers want their volunteers be proficient identifying to the order level, with some families. 50% of trainers would like to go further, identifying more families and some genuses.
  • Trainers use physical materials more often than websites, mobile applications, or videos, but those trainers that want to train to finer taxonomic levels use websites the most often.

Design Ideas
  • Regional field guide
  • Interactive dichotomous key
  • Ability to create your own tests
  • Reference collection
  • Life history and identification tutorial videos

Questions
  • What do we do for the 40% trainers that do not have Internet access during trainings?
  • Do biomonitoring groups try to use their volunteer data to request assistance from state regulatory agencies?

Baseline Identification Trainings Evaluation
Presented by Camellia Sanford of Rockman et al

Findings
  • Volunteers were most confident identifying to order and least confident identifying to family.
  • Volunteers were less confident in their abilities to teach identification to beginners than in their own abilities to identify macroinvertebrates.
  • Trainers were accurate estimators of what volunteers could actually do.
  • Suprisingly, order names were better retained by volunteers than common names.

Design Ideas
  • Use a collage of images to represent the order level.
  • Present multiple images of a grouping and ask users what characteristics group them together.

​Questions
  • When inaccurate, do volunteers go astray at the first couplet or the last couplet of the key?
  • Representing orders will be difficult in one image since an order has many different characters.
  • Should the objectives of each training be considered? For a given training, the goal may not be developing the ability to identify to family level or train other beginners.

Field Guides & Keys Analysis
Presented by Jen Liu of Carnegie Mellon University

Findings
  • Similar strategies are used in novice, intermediate, and expert field guides to move users through different taxonomic groups

Design Implications
  • Creating heuristics for “how to get started”
  • Inform the structure of how to move users through the website
  • How can different characteristics be communicated?​
Picture
Picture
Along with analyzing different field guides used by novices, intermediates, and experts, we set up simulations of volunteer and expert identifications.

Learning Research Implications
​Presented by Kevin Crowley of the University of Pittsburgh Learning Research & Development Center

Findings
  • Identifying to the order level is automated in experts.
  • Experts do a lot of ruling out based on background knowledge.
  • Experts know the key and the specimens well enough to jump around inside a guide and not go through it linearly.

Design Implications
  • Novices are more inclined to get stuck on an unimportant detail, while experts may see the gestalt of a specimen.
  • Tools should be robust enough to highlight eye catching detail, but be general enough so the novice feels comfortable relying on the tool going forward.

Aquatic Macros Imaging
Presented by Andrea Kautz & John Wenzel of CMNH & Powdermill Nature Reserve

Findings
  • Important parameters to consider: flash use, image background color, wet or dry specimen, spacing between stacks of image, zoom, specimen size, and balancing imaging time with quality

Design Implications & Ideas
  • Contrast between the high quality clean images on the website and the level of detail volunteers are able to see on dirty bugs found in the field.
  • Add a scale bar to the images
Picture
Picture
High resolution gigapan images

Entomology & Identification
Presented by Maddie Genco & John Morse of Clemson University

Findings
  • Biomonitoring is important because animals are inherently more interesting to the public than water quality on its own
  • Biomonitoring is less expensive than chemical tests

Design Implications
  • The website will be able to help with major challenges for training citizen scientists by providing:
    • ​Scientific Names
    • Vocabulary for macroinvertebrate structures
    • Access to reference specimens
    • Access to microscopes
    • Time to build experience​

Society for Freshwater Science Taxa Certification
Presented by Mike Broomall of the Stroud Water Research Center

Stroud’s Society for Freshwater Science offers Taxa Certifications at the family and genus levels. The family level test success rate is very low and standards are very high.

Design Ideas
  • Add an order level test since many VBMOs train to order level.
  • Use collages of insects.

Collective Access & CAT Tool Demo
Presented by Chris Bartley of Carnegie Mellon University

The gigapan is a hardware device that a camera is mounted on for taking high quality large images.

Design Ideas
  • Use crowdsourcing to pick the “best” example of a dorsal claw place at the “best” angle.
  • Allow a user to drag their cursor across the screen and have definitions pop up.
  • ​Link term in a bug description back to an image.
  • Option to turn annotations on and off.

Macroinvertebrates.org Design Research - Usability Testing
Presented by Marti Louw of Carnegie Mellon University

Findings
  • Homepage tutorial, welcoming text, or video resources for how to use the site would be helpful.
  • Users liked the images of different order level bugs and examples of cases.
  • 50% of users did not read insect images as clickable and the other 50% did not read the "+" sign as clickable.
  • Users often missed the “i” information page tab and often skipped family pages all together.
  • Users found order level content and breaking apart the Latin names helpful.
  • Users played audio descriptions in background as they moved throughout the website.
  • On the ZUI (Zoomable User Interface) few users discovered the dorsal/ventral toggle.
  • Guitar pick symbols were a little unclear.

Design Ideas
  • Tree of life homepage
  • People appreciate simplicity without being overwhelmed
  • Including functional feeding groups may be helpful
  • There is no way to appreciate the total content of the website - can we use phylogeny to depict the total website content?
  • Include a landing page with a tab of how to move through the site

Design Activities

Design Activity 1: ​ID Training Scenarios & Opportunity Matrix

Activity Goals
  1. Develop a detailed set of different macroinvertebrate training scenarios by listening closely to what trainers think, feel, and do as they plan and deliver training workshops.
  2. Identify opportunities for Macroinvertebrates.org to address challenges or augment existing tools and practices in training scenarios​.

Procedure
  1. Directed Storytelling & Focused Listening: each VBMO trainer described a typical training session to their small group. Scribers noted the training characteristics and narrative. Focused listeners wrote down pain points, resources/tools, and best practices on color-coded sticky notes.
  2. Training Scenario Opportunities Matrix: the group placed each sticky note containing a pain point, resource, or best practice on a matrix with sections for before training, during training, and after training. Other group members also added notes from their own experiences conducting volunteer trainings.
  3. Opportunity Points & Team Shareouts: small groups placed stickers on notes representing pain points that the website may be able to help, resources that the website could augment, and best practices that the website design should support. The top 3-5 items were shared with the full workshop group.  ​
Picture
Picture
Picture
Picture
How Macroinvertebrates.org Can Help
  • Quality Assessment: use for pre-training tests, double checking volunteer quizzes, and volunteers self-testing throughout their training to build confidence.
  • Engagement: can act as a hook to get kids engaged; is more interesting than PowerPoint.
  • Reference: volunteers can go back to as a reference post-training; reinforces vocabulary; trainers can use as a refresher resource for trainings that occur infrequently.
  • Context: use for examples of the potential biodiversity of a degraded stream; comparisons between different kinds of streams; context for why macroinvertebrates are important indicators of water quality.
  • Enhanced Imagery: great for high quality presentation images; could allow trainers to show a detailed characteristic on a projector while volunteers look through a microscope.

Design Ideas
  • Show gigapan insect images along with user-generated and professional images.
  • ​Collection customization: to region, remove genus level information, hide certain families, etc.
  • Put findings from data on the site since volunteers want to know how their data will be used.
  • Trainer backend for the website: “I want to look at these 10 taxa and make a test."
  • Automated tests: on a weekly basis during sampling season, send a text message of an image and have volunteers respond with the identification.
  • Update users when new images are added.
  • Dynamic videos to support trainers as they talk about images

Activity Limitations
Not a lot of time was spent discussing the wide variety of trainer experiences in small groups. Although all group members were encouraged to add to the opportunity matrix with their own experiences as trainers, a small group conversation around these variances should be facilitated better going forward.

Design Activity 2: Rose, Bud, Thorn & Affinity Clustering

Activity Goals
Identify the strengths, opportunities, and problems of Macroinvertebrates.org.

Procedure
  1. Workshop participants explored the website and wrote down items on sticky notes: roses (positives - blue notes), buds (have potential - green), and thorns (problems - pink).
  2. Sticky notes went up on butcher paper and were grouped into clusters of similar topics.​

Rose, Bud, and Thorn Clusters

Picture

Design Activity 3: Wireframe Mockups & Design Priorities

Activity Goals
Facilitate discussion about the redesign of Macroinvertebrates.org, with particular attention to site navigation through taxa level, and how users will interact with content and images in their process of identifying specimens.

Procedure
  1. Wireframe Markup and Sketching: small groups used printed wireframes to depict the flow of users through the site as they identify specimens. Additional elements were created on blank wireframe pages.
  2. Wireframe Presentations: Small groups created simple slideshows to share their design decisions with the full workshop.

Feature Ideas
  • ​Netflix-like tiles
  • Master list available either as a phylogenetic tree or a gallery
  • Expandable classification tree for navigation: indicate to the user “you are here”
  • Alert users that what they see first isn’t everything on the site
  • Landing page with bigger pictures and bigger boxes and a scroll bar across orders
  • Show all 18 orders on landing page or do not show all orders at once?
  • When presented with many orders, orders should be shown in a purposeful way (e.g., alphabetically or by most common)
  • Video tutorial page
  • Volunteer resources and trainer resources tabs (including quizzes)
  • Explore a gigapan of all the bugs in a tray
  • Engaging zooming in and out video as the first interaction to hook the user
  • Include insects at actual size and scale
  • Show variation in bugs and lifecycle stages
  • Include functional feeding group and video of movement for each bug
  • Tool tips for terminology 
  • Highlight commonly confused macroinvertebrates (“see also” or “commonly mistaken for”)
  • Guitar picks are helpful indicators
​
Teams' wireframe slideshows are included below.



Next Steps: Expert Roundtables & Discussion

At the end of the workshop, participants were grouped into roundtables according to their expertise: educational research, entomology, volunteer biomonitoring, and NSF advisement. Through in-depth expert discussions and presentations to the entire group, we gathered important insights, suggestions, and questions to consider as the project enters its second year.

  1. What is the purpose of the tool?
    • Is the tool for teaching characters or identification?
    • Is the website a key? If not, how might we help users recognize that it is not?
    • Can this tool build quality assured confidence?
    • How best to use the tool teach volunteers scientific vocabulary?
  2. How will the website handle robust facts (e.g., correct scientific name) and norms (e.g., insects facing a standard direction)?​
  3. How might we layer scaffolding and information without overwhelming to support different groups (scientists, trainers, volunteers, general public)?​
    1. ​Include genera annotation that could be toggled off and on
    2. Full organism images are most useful for for volunteers and enthusiasts.
    3. Characteristics are more useful for higher-level volunteers, trainers, formal students, and professionals.
    4. General public/enthusiasts could upload their own images
    5. Use to connect enthusiasts to bioblitzes
    6. Push volunteers to go further although they may perform at their trainers’ expectations
  4. How do trainers use the website and what problems does it solve?
    1. ​The system can work as a training environment for volunteers
    2. Quizzes and other “homework” may help volunteers go more in depth.
    3. Improve teaching and knowledge of trainers
    4. Allow for side by side comparisons
    5. Use as a tool for reminders to test volunteer proficiency
    6. Reference tool for master list of state species
    7. Reference tool for showing characteristics listed in keys
    8. Give trainers the ability to customize collections
  5. What makes the tool compelling?
    1. ​Emphasize beautiful images on the site as a hook and give more contextual information
    2. Highlight significance of macroinvertebrates to engage users in water quality monitoring
    3. Provide a list of all available resources
    4. Include a simple key in the website​

    Project Team

    An interdisciplinary team
    ​of entomologists, learning scientists, software engineers and designers collaborating to improve macroinvertebrate identification training and technologies with volunteer biomonitoring organizations.

    Categories

    All
    Broader Impacts
    By Clemson
    By CMU
    By Educators
    By Powdermill
    By Stroud
    Design Studies
    Educator Innovations
    Entomology
    Imaging
    Learning Research
    Mobile App
    NSF REU
    User Research

    Archives

    January 2021
    December 2020
    September 2020
    August 2020
    July 2020
    May 2020
    November 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    October 2018
    September 2018
    August 2018
    June 2018
    May 2018
    March 2018
    December 2017
    August 2017
    June 2017
    May 2017
    April 2017
    March 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    March 2016

    RSS Feed

Project INFO

ABOUT
www.ept.macroinvertebrates.org
NSF Award  Listing

Partners

Learning Media Design Center@CMU
CREATE Lab @CMU
​
Clemson University 
Stroud Water Research Center
Carnegie Museum of Natural History

Collaborators

ALLARM
MD Streamwaders
Senior Environmental Corp
Pittsburgh Parks Conservancy

Trout Unlimited
Picture
This material is based upon work supported by the National Science Foundation under Grant #1623969.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NSF. ​​

  • PROJECT
  • PEOPLE
  • PRODUCTS
  • Blog
  • Resources
    • Help
    • FAQ