|
Using Log Visualization to Interpret Online Interactions During Self-Quizzing Practice for Taxonomic Identification by Chelsea Cui, Jordan Hill, Marti Louw, Jessica Roberts. We were excited to present CMU alumna, and former REU student Chelsea Cui’s study at the 2021 annual meeting of the American Educational Researchers Association (AERA). Chelsea analyzed log data from participants in a 10-day study in which they used our quiz feature to practice aquatic macroinvertebrate ID. We analyzed interactions with the quiz tool along with accuracy in pre- and post-tests to determine how the quizzing platform was being utilized by learners at different experience levels. Through the custom visualization platform we were able to glean insights on how to improve our quizzing platform for future use. See the poster here: https://aera21-aera.ipostersessions.com/Default.aspx?s=D0-9D-A0-1B-C0-49-87-5B-8A-3F-56-A7-02-C5-ED-66# Mobile App: More details for field guide revisions, quiz feature design and team collaboration.1/28/2021
by Estelle Jiang Before sharing about the work the team and I got done, I want to quickly wrap up about the main functionalities of the application, the app condenses the following aspects in terms of learning and teaching freshwater insect identification after a range of use cases centered around exploration: 1. Field Guide 2. Identification (ID) Key 3. Quiz This is a blog that further elaborate on the work and tasks the design team finished over the past semester. Moving from the summer work where we finished a big round of testing with users for the field guide design and interaction, the design team mainly focuses on iterating the field guide through second round of user testings and also switched the gear towards design for Macroinvertebrates assessment - design a quiz functionality to allow users to review the learning contents from the field guide and also quiz themselves on the understanding. 01. Field guide iterations and finalization - 1. Brainstormed and explored the design interaction for the field guide: There are some small features and interactions, such as global navigation and the zoomable page interaction that are not fully considered over the summer. Therefore, Alice and I started exploring different interaction and design possibilities before going to testing. Here are some my explorations that inform our final field guide page design: A. Global Navigation  B. Zoomable Image Flipping  C. Onboarding Instruction and Design  2. Provided guidance and drafted out the design system for the field guide and entire app. Along the way, we decided to start consolidating the design guidelines and system for the team and product which can be useful for further development and also team collaboration. I started the finding good industry practices in terms of generating and designing the system, provided guidances and gave feedbacks while Alice consolidated the overall visual/layout and turned them into components in Figma to better speed up the design process. 3. Facilitated the second round of user testings with product evaluation and conducted synthesis workshops on Miro - Instead of getting insights about application flow, logic and concept aspects of the app, the major goals of the second round testing are identify the specific usability issues for overall navigation and zoomable pages new design and evaluate more on the overall engagement and usability. Based on the 8 testing results, I facilitated the synthesis workshops on Miro with the team to generate iteration insights and help the team finalize the field guide MVP. For the main insights: You can refer Alice's post: [Mobile App Pt. IV: Refining the Field Guide] 02. Quiz / game to assess the learning goals 1. Explored the market product interactions. Before designing the quiz, we firstly explored some predecessors that have quiz features, such as Quizlet, Duolingo, and Lumosity. It helped us generate several possible formats of the quiz as well, such as matching games, card flipping games and flashcard reviews. 2. Considered what we want to assess and the learning goals of the application before going deeper into the user experience and interaction design. We took a pause and realized that know the learning goals and purpose of making the quiz is the first priority comparing to brainstorm design solutions for quiz. Without understanding what we want the learners to learn, we cannot provide the suitable design to meet their needs. I summarized some potential learning goals before meeting the current trainers and experts:
3. Mapped out the design formats and interaction and finalized the flow and logics for each learning goal we defined. A. Initial flow explorations (without knowing the learning goals)  B. New flow explorations that better inform the design with rationales.  C. New quiz flow by following and considering the design goals  D. Designed for varied use cases and learning cases - knew limitations and considered tradeoffs  —Critiqued and iterated the design with engineers and trainers/experts to narrow down the scope. Review the feedback we gathered in Dominique's post: Macro App Tasks This Semester 03. Being a designer by wearing different hats and entering the development stageWhile our design team kept working on the ID key and quiz design and conducted further testings, we got in touch with Chris Bartley, an experienced engineer and involved him into our design process along the way. We are still figuring out the better and more efficient way to collaborate, here are some attempts we took so far:
By Dominique Aruede, CMU Cognitive Psychology When I joined, the quiz section had not been developed at all in favor of refining and refreshing the ID key and field guide which are more directly interpreted from the original website. Consequently, it was important to gather human-centered data again in order to assess the efficiency of the current design and determine if we could branch out to the quiz or keep working on the current designs. First Task: Audubon Field Guide Walkthrough & Modeling I spent a little time doing some exploration with a physical field guide and noting how it maps to real-world use and facilitation of citizen science. This step was particularly useful for getting my bearings on what the purpose of the Macroinvertabrates.com project is, how insect identification fits in, and how it works. Second Task: User Testing We conducted several user tests on insect ID experts and novices alike, with a focus on novice user experience from high schoolers and other young students. The purpose of these usability tests was to gauge the design direction of our second iteration and to gather more user input to launch into the next iteration. We reused the protocol from the first round of user testing and changed the questions to capture answers on our new inquiries:
 The results revealed to us that the field guide was indeed informative, but subsequently neglected some utilitarian features that would be helpful for novice users. This helped to inform the learning goals and first iterations of the quiz design. Below is an affinity diagram of the synthesized insights. It includes
 Third Task: Quiz Design The earliest version of the quiz was a rough sketch I drew up in Figma, but it did not have any learning goal or research claim behind it besides identifying some image as belonging to an order. These are the sketches below.
After further visualization, we wrote out the learning goals we thought would be the best to target in quiz mode based on the insights gleaned from the previous affinity diagram. The learning goals are summarized here:
After we gathered our quiz references, including Quizlet and Duolingo, we used our learning goals to consolidate designs for one flow of the quiz section. Users select their quiz type first, then their learning goal. An alternative flow that we are yet to explore involves switching those two options. Then we incorporated three quiz types: a flash card review, multiple choice quiz, and a matching quiz. We drew this up on Figma and discovered a few issues/limitations
Afterwards we consulted our collaborator, who is an expert in aquatic macroinvertebrate education to give her opinion on the content and direction. From her insights, we decided to scrap learning PTV as a learning goal due to the inaccuracies of generalizing at the order level.
The next steps we plan to take include developing all the content necessary for accurate quizzing, fleshing out the flashcard review, the between order quiz learning goal for all quiz types, the common features learning goal for all quiz types, the common mistakes learning goal for all quiz types, exploring a new learning goal: “learning life history and fun facts about insect orders” as a deck in the flashcard review, and testing the usability and flow of our current design. Personas are a common design research consolidation technique to make sense of user research data. Here findings from observations, interview transcripts, and organizational document analysis were synthesized and a set of representative user types were created to allow the design team an abstracted view of a range of user types, their needs, motivations and use cases to continually consider in decision-making as the design evolved . Below are a fictionalized set of personas we created based on our data to characterize typical users of Macroinvertebrates.org. Personas developed by CMU design student Adrian Galvin. By Dominique Aruede Last week Dr. Louw tasked the design team with exploring the material design of the Audobon Society Field Guide and Pocket Guide and possibly perform a few insect identifications using the guides. The latter book was a smaller, simplified, more utilitarian version of the pocket guide. In an effort to frame and understand the flow of a typical bug ID guide, we spent a week looking into the specific design choices of Audubon identification pocket guides and field guides. Alice, Estelle, and I each went though the Field Guide, noting down different features that added value or clarity or ease of use. We all agreed the field guide had general helpful features like annotated images and a glossary of all the common names. Since I'm a very new member to the team I received another book, the pocket guide, which seemed to be for more novice audiences and was easy to digest for context purposes. The field guide was more advanced in technical information than the pocket guide, and that took away some of it's practicality. Below are our individual syntheses and visualizations of the insights produced from the objective. by Chelsea Aci and Ziqi Dong This summer we began development of an innovative open educational resource with the aim of making the task of learning to identify macronvertebrates easier and more engaging by developing a new kind of tool for guiding scientific observation and inquiry. The goal is to release a mobile version of Macroinvertebreates.org optimized for field use in low-no wifi conditions, and which supports learning to identify aquatic insects and water quality assessments. Watch a video demo'ing our prototype with supporting documentation. By Estelle Jiang and Alice Fang 🐞📱Moving to a higher fidelity prototype by following the design system. After showing our low-fidelity prototype to the entire team and developers, we decided to move forward by applying a more detailed design and developing the visual + design system. We also figured out how to showcase the relationship between orders, family and genius on the mobile application, and the UI components for each type of 'page'. It was one of the biggest challenges we met previously. For the color theme, we followed the guidelines the project used on the website and applied the blue color to highlight actionable parts. To keep the app clean and concise, we used white for the major user interface design. As for the typography, the body font is Nunito Sans and the title font is Roboto Slab. Due to limitations with the database information, and in an effort to bring about the features of the specimen in photography, we also worked with the gigapan background color, creating a floating, borderless 'under a microscope' look [see image on the right].
Changes we made for the high fidelity prototype after discussion:  Homepage - We thought the card view can be bigger to attract people’s attention and intrigue their interest. Since we only have 10 orders, we did not have too many concerns about accessibility at the very beginning. The ID key button is also replaced on the home page. The dropdown menu was also changed to help user easily navigate and get back to main page.  We also added icons to explain the functionalities and applied color for the side menu to make it stand out more. As we mentioned on the previous post, we were struggling between a button to expand, and swiping up. Since we were worried about the experience of swiping up which is too hidden on the bottom of the screen. we iterated and created a button on the bottom for accessibility instead. Planning and preparing first round usability testing To conduct our first round of testing, we started with writing the testing protocol, thinking about the purpose of the testing and the goals we want to achieve. The purpose of our testing was to test the logic of the user flows, and to identify potential navigation and usability issues. We wanted to understand if the application is engaging to users, and is useful in identifying macroinvertebrates and learning their characteristics. We assigned a few small tasks for users to finish during the testing: Pre-task: Users will be given 30 seconds to get familiar with the application before doing task. First task: Users are asked to browse the different orders though different ways. This way, we can then tell whether the design makes sense, and take note of how users navigate through the different levels of information. Second task: This task was focused on the detailed Order & Family page designs, users are asked to find out more detailed order and family information, as well as specific diagnostic characters for a specific family. By asking the users how difficult the task is, we can evaluate the slider design idea we had, and how accessible / noticeable the actionable button is. We also asked additional questions at the end of testing to check whether they can have a clear understanding about our application throughout the testing process, including:
Synthesize the findings to guide our next iterations. Overall, we conducted five user interviews, with macroinvertebrate experts, people familiar with the site, and a novice user; I also got feedback from my friend who knew nothing about the concept or field in order to get additional novice learner’s insights. Rather than use the normal user research method - affinity diagramming to synthesize the testing findings, Alice made the excel sheet to list out the key points the interviewee made for each task. It helped us highlight the common suggestions and feedback.  Here are the findings that guided our next iterations:
Iterations we made. Since the first version of high fidelity prototypes are hard for novice learners to learn and understand the additional information, I quickly brainstormed two other versions to display the information and hierarchy between orders and families.  The first version allows users to swipe and learn along the way. The experience is more immersive and easy to follow if the users have no idea about the insect and the order. However, it doesn’t give enough freedom and choices for users to explore themselves, and quickly becomes repetitive for more experienced users.  The second version can cater the needs of both experienced users and novice learners since it allows them to quickly switch between various levels of information. The structure of the insects are also easy to tell and discover. The homepage was also iterated from showing only one order to display multiple orders at once.
The developers moved forward with this version, and worked to develop a beta version. It was interesting and difficult figuring out how to work in parallel; they were focused on setting up the database and structure, while we were iterating through the designs, but we couldn't progress too far or change too many things after they began developing the pages. By Alice Fang New 🐞 bandanas! I designed a bandana that highlights the insect illustrations done by Morgan Summerlin, inverted on black. The illustrations are placed on a diagonal for optimal bandana-folding-wearing! :-). Handy for working in wet settings, and promote Macroinvertebrates.org when you have to maskup,  By Alice Fang 🐞📱Re-framing with a new team! Chelsea and I were joined by Estelle (Yi Cheng) Jiang and Dakota (Zi Qi) Dong in late May. Moving forward, Chelsea and Dakota will be working on development, while Estelle and I will be working on the design and user-testing of the app. There was an adjustment period as we introduced the project to the new members and got everyone acquainted while figuring out how to work collaboratively in a remote wfh environment. Through this process, we've been utilizing google docs and spreadsheets, as well as Figma, but coordinating between design and development has been tricky. Accommodating and synching the design and development timelines was difficult, and it was a bit touch-and-go. Spreadsheets and Organizing Data Previously, I struggled with establishing a structure to the app that allowed for navigation in and out of orders/families. To make the taxonomy clear (as we are non-scientists and non-bug experts), and to organize the information for design purposes, I created a spreadsheet with the following:  1. Inventory of functions that exist on the website In order to figure out the minimum viable product that can be developed by students within a summer, and to compare what needs to change from Order to Family, I listed all of the functions for Order/Family/Genus.  2. List of insect orders, the families in each order, and the number of specimen (genera) in each family, and the number of diagnostic characters for each family and order While many of the families have similar numbers of specimen, there were a lot of outlier cases we needed to keep an eye on, and account for while designing. The length of names, both common and scientific, impacts the typographic system, and the number of diagnostic characters affects how we visually set up that information. Some of the cases are as follows:
*Dev team has to set up the database (I don’t know the exact details). We ran into a challenge late in the summer [early August] where the Gigapan database had to be moved, and there was no way to extract some of the family traits and text information, requiring manual copy and pasting Collecting References and Resources for Ideation Round II Getting the design team on the same page. I showed Estelle the previous mockups and ideas that I had for the mobile app, but in order to refresh and sort of create a mutual visual language, we took the time to research and look at other apps. We compared and discussed field guide apps, quiz apps, and other text-based apps like news/media.  Estelle's feedback + insights   *Referencing Headspace, AirBnB, Medium, WWF Together Alice’s Points:
 *Referencing Twitter, Google Maps, Medium Merlin ID, Audubon Lo-fi Version 2 New Ideas and Changes Estelle quickly mocked some basic page structures, and documented user stories. This allowed us to see possible entry points and user profiles for the app. What are possible ways people would use the app? What would they be looking for? And how do we prioritize that development at the same time?  User Stories  She also created this flow diagram of entry in the app and access points to different functions; however, it didn't include navigation and returning to previous pages, or other major functions we hope to implement in the future, like a quiz.  Home Page and Order Page Templates that Estelle created
 Scroll-up Popup for additional information, as well as an 'introduction' to the Order Page
*Comparing what a bottom pop-up that scrolls up, and a button on the bottom, would look like. Between a button to expand, and swiping up, we originally decided on a swiping up action, but were worried about the screen experience that it would be too hidden on the bottom of the screen. As we moved into user testing, we created a button on the bottom for accessibility instead. It stood out more, and for someone holding a phone, was located in a position that was easy to access.  Through this process, we also started to get into the look and feel of the application. As the focus and beauty of the collection is the high definition images and close-up thumbnails, I really wanted typography to play a role in the visual style while staying close to what the desktop site looks like.
|
Project TeamAn interdisciplinary team Categories
All
Archives
June 2023
|
RSS Feed
