IE 418 Final Project: Evaluating interfaces and designing alternatives
With this final project we will accomplish several tasks:
- Identify a meaningful context in which to evalute the interface between humans and computers
- Identify and evaluate a meaningful interface in that previously identified context
- Suggest and engineer an alternative interface for that context
You should have an opportunity to work on this project throughout the semester with some in-class activities and labs. The latter of those activities mean that all project groups will need to be in the same lab.
Phase 1: Selection of a context and interface
As a part of selecting an interface for evaluating and imagining anew, you will select a context in which to begin your search and selection. You project should fall within the specific context you end up selecting.
To keep the topics balanced and help ensure there is plenty of space within each context, students will be able to select from the following four contexts (with a maximum of three groups per context):
Health
What interfaces should we use to understand and plan for our health? Groups within this context will find an computing interface dealing with Health outcomes and information.
Potential places to start (besides your normal library resources and search engine)
Disaster/Emergency Management
How should we interface with computing systems (and each other) for a disaster or emergency? Groups within this context will find an computing interface dealing with Disaster or emergency management.
Potential places to start (besides your normal library resources and search engine)
Digital Archives
What interfaces should we use to archive and understand our past? Groups within this context will find an computing interface dealing with Digital archives.
Potential places to start (besides your normal library resources and search engine)
Play
How should we interface with digital games and other computing systems for play?
Potential places to start (besides your normal library resources and search engine)
Once you’ve selected a topic, the next step will be to find the interface (or set of interfaces) you want to analyze, evaluate, and imagine anew.
Phase 1 submission
You will have to submit the following to Canvas:
One person from the team should Submit (to Canvas) team information to include the following information:
• Name of your team
• Project chosen
• Project description — include 1–2 paragraphs stating the project description in your own words.
• Perceived strengths and weaknesses of each team member (i.e., how do you envision dividing up duties?) This should be determined from an open, collaborative discussion among the teammates.
Phase 2: Imaginging futures & alternatives (Lab 1)
Once you have an initial idea of the context and interface you want to deal with, we will begin not by directly evaluating and analyzing that interface, but instead doing an early imagining and design towards a new interface. The goal of this (lab) will be to speculate designs given the context selected before we get to what exists.
Phase 3: Current interface analysis and evaluations (Related to several labs)
Over the course of the semester, you will have several chances to practice analysis your selected existing interface through several labs and assignments. You should be able to use these analysis as a part of your overall project portfolio.
Phase 4: Bringing it all together
The final phase is bringing together your analysis, evaluations, prototyped alternatives into one packaged project deliverable
Final Deliverables
Ultimately you’ll be required to submit a final paper, prototype documentation and design document, evaluation portfolio, and presentation for your final grade:
Final paper
docs/ResearchReport.pdf
- This is the main research paper where you should go into the interface and scenarios you tackled, why you chose them, and how your new interface addresses shortcomings of the old interface. This should be written like a research paper and have sufficient background on the subject. It should provide some detail on your new interface and the old interface, but more on the background and the results of your evaluations as they apply to your interfaces. The conclusion should discuss not only what was correct about your approach, but also what was wrong. You should have a solid foundation of citations (and a references section) to back-up claims made in the paper.
- You should format your paper according to IEEE Transactions on Systems, Man, and Cybernetics: Systems template
- This is a “Transactions/Journal”, you should be able to download the word template from the link above by finding the correct transactions in the list presented at the “Select publication” stage
- A specific grading rubric is given in the grading section below
TOTAL: Approx. 10-12 pages.
Project portfolio
Prototype documentation and design document
You should provide the following:
- Prototypes of your new interface design, from more simple (e.g., figma) prototypes to the final interactive, digital prototype your develop and submit
- Documentation of the prototype to include the specific scenarios selected for your final prototype evaluation and screenshots/images of what the prototype looks like during steps of those scenarios
- A design document for your new prototype to include specific salient design decisions made and comparisons in design to the old interface(s).
Evaluation
evaluation/*
Your project portfolio will consist of three main sets of artifacts:
Task Analysis
You should do a task analysis of both your old interface and the new interface:
- Identify 5 task-based scenarios to be used for task analysis (for both interfaces)
- Conduct a cognitive task analysis of each of those 5 scenarios (for both interfaces)
- Use cogulator to develop GOMS models of those tasks based on your CTAs
The GOMS models All of your GOMS models must have the following characteristics:
- At least 5 separate goals (including sub-goals)
- At least 3 sub-goals
- Note that sub-goals are a goal, so if you have 3 sub-goals, you only need 2 more goals for the minimum!
- Must use memory (i.e., the related cognitive operators) in some way
- Must use perception and motor operators (i.e., looking, clicking, etc.)
Overall, you should make sure the following is in your portfolio:
- The results of your CTA (can be a visual representation or textual representation)
- Your GOMS models (
.gomsfile) - Your GOMS file should load in cogulator without errors (and thus be able to give us an expected/predicted task time without error!)
- You should have 10 total GOMS files
- A file of screenshots of the cogulator gant charts that is legible
- Note that it may be better to take a screenshot yourself
Usability evaluation
You should pick an evaluation method provided by FDUCS or Research Methods in Human Computer Interaction and apply this method to the old interface(s) and the new interface you’ve developed
Provide data and documentation for your evaluation. This documentation should provide the following information under section headers with the same name:
- What method did you pick and why
- Major differences between this method and task analysis
- Results from your evaluation
- Advantages of this method for your purposes
- Limitations of this method for your purposes
Deceptive pattern evaluation
For this article, you should go through each of the deceptive pattern categories presented in the book and provide a report that outlines if those patterns show up in the original interface(s) and if so, how. (Each of the types listed in Part 3: Types of deceptive pattern section )
Project presentation
docs/presentation.pptx
Given the size of our class (!) we will do final presentations a bit differently than the norm - We’ll focus on more lightening talk style presentations with a more robust Q&A Here are the details:
A presentation video
- Each team has to make a final presentation video (recorded for virtual interaction)
- The length should be 8-10 minutes (with 10 minutes being the max length)
- Videos due 3 school days before presentation date assigned to group
- Those who sign-up for the 03-Dec time slot will have until Monday (01-Dec) 3:20pm
- Those who sign-up for the 08-Dec time slot videos are due by Wednesday (03-Dec) 3:20pm
- Those who sign-up for the 10-Dec time slot videos are due by Friday (05-Dec) 3:20pm
- Every team must submit 2 question for three of the videos on any presentation day other than their own
- Questions should be unique (first come first serve)
- Every video can have a max of 4 questions for it (first come first serve)
- (in-person) 4 minutes will be spent answering those questions (at least 2 questions answered)
- Dancy/TAs will flag 3 questions for each group to be answered in-person
- Each team must respond to 25% of questions asked virtually (within the discussion threads for those questions)
In-class
- Each team will have 5 minutes total
- The first minute (which will be enforced rigidly) will be spent giving a quick “lightening” overview of your teams project
- Think of a one minute elevator pitch (what did you do and why should we care?)
- The next 4 minutes will be spent answering questions picked by Dr. Dancy or the TAs
Grading
In general, the break down used to grade the final project is as follows:
| Graded item | Number of points |
|---|---|
| Group Information | 3 pts |
| Project Update Presentation | 5 pts |
| Project Report Rough Draft | 5 pts [all or none] |
| All files are organized as instructed | 5 pts |
| Final Research Report | 24 pts |
| Project Portfolio - Current interface evaluation | 15 pts |
| Project Portfolio - New interface prototype and implementation | 15 pts |
| Project Portfolio - Evaluation of new prototype | 5 pts |
| Final Project Presentation | 20 pts |
| Final project Presentations Questions provided as instructed | 3 pts [all or none] |
A note on grading and requirements
Generally, I will regard items (especially portfolio-based items) in the following manner (I use a 5 point scale to help illustrate a quantification scheme):
- Is it below minimum requirements, but shows enough that I can see a vision for how it would have gotten to the minimum (e.g., just missing a detail or two)? - 2 points
- Does it meet minimum requirements (and not much else)? - 3.5 points
- Is it above average? Does it go above and beyond minimum requirements in ways that are above minimum expectation and show some small amount of creativity? - 4.25 points
- Is it exceptional? Does it go well above and beyond minimum requirements and show high creativity in application of theory/engineering of artifacts? - 5 points
An emphasis here is that just meeting minimum requirements does get you a passing grade! But it also does not get you to an exceptional grade.
Project introduction/update Presentation (5 pts)
| Graded item | Number of points |
|---|---|
| You introduce everyone in your group | 1 pts |
| Introduction your topic and interface for that topic | 2 pts |
| Description of some early planned scenarios that will be used to evaluate your interface | 1 pt |
| Potential issues presented | 1 pts |
Final Presentation Rubric (20 Total Points)
| Graded item | Number of points |
|---|---|
| You introduce everyone in your group | 1 pts |
| Re-introduce the situation(s)/question(s)/interfaces they’re approaching | 2 pts |
| Introduce & discuss the new interface (full prototype) | 2 pts |
| Talk explicitly about differences between new interface (prototype) and old interface(s) | 2 |
| Present high-level skteches of scenarios used for testing/understanding interfaces | 3 |
| Provide appropriate evidence (analysis) that your new interface is a reasonable proposed system for the situations/scenarios of concern | 3 pts |
| Discuss challenges in project and what you learned | 2 pts |
| Appropriately answer questions asked during the Q&A | 3 pts |
| Presentation does not go over time and is appropriately timed | 1 pt |
| Appropriately answered at least of 25% of discussion questions online | 1 pt |
Final Project Research Report (24 total pts)
| Graded item | Number of points |
|---|---|
| Introduction introduced situation(s)/question(s)/contexts, connections to literature, and foreshadows solution | 5 pts |
| Description of old interface/system for given situation/context | 2 pts |
| Description of new interface/system built to better handle situations, questions, and/or needs | 3 pts |
| Provide the background and the results of your evaluations/analysis as they apply to your (old and new) interfaces | 8 pts |
| Conclusion provides concluding thoughts, future analysis work, any potential analysis, discussion, and adeqautely concludes report | 3 pts |
| Page length is appropriate for meeting requirements and is no longer than 12 pages | 1 pts |
| Formatting is correct (according to requirements) and consistent | 2 pts |
| Date | Milestone |
|---|---|
| Week of 08-Sept | Project assigned — Select team members and project |
| 16-Sept, 11:00pm | One person from the team should Submit (to Canvas) team information to include the following information: • Name of your team • Project chosen • Project description — include 1–2 paragraphs stating the project description in your own words. • Perceived strengths and weaknesses of each team member (i.e., how do you envision dividing up duties?) This should be determined from an open, collaborative discussion among the teammates. |
| 15-Oct, 11:00pm | 3-5 min (5 minutes max) Project introduction/update video due for each group |
| 22-Oct, 11:00pm | Group peer feedback due |
| 27-Oct, 11:00pm | Project Rough Draft (outline) of your final paper, screenshots/images of previous interface upon which your are improving, and rough prototypes of your new design due |
| 17-Nov and 19-Nov, In class | In-class group check-in (all group members should be presen during check-in) |
| 03-Dec, 08-Dec, & 10-Dec, in class | Final in-class presentations |
| 12-Dec, 11:00pm | All final project materials due |