Group Project

Download complete project description including grading criteria

Grading Criteria

CS/PSYC 3750 Group Project: User Interface Evaluation and Design

Outline

* Project Overview
* Project Report Book
* Part 0 – Project/team Definition
* Part 1 – Understanding the Problem
* Part 2 – Design Alternatives
* Part 3 – System prototype and evaluation plan
* Part 4 – Evaluation
* Final Project Presentation

Project Overview

This term you will undertake a group project (usually 4 people, sometimes 3, never 5) to evaluate some computing-related task/problem, to develop interface design alternatives for the task/problem, to implement a prototype of your design, and to evaluate your design. This will provide you with hands-on experience with the user-computer interface design process. Ideally, the topic of the project will be a problem that matters to some “real-life” people. These people then will serve as your “clients”, with whom you will communicate with and from whom you will learn. Sometimes we may have some specific projects for you to consider; more often you will need to develop your own project idea. Your project idea must be approved prior to completing Project Part 0.

Each project group will be graded as a team, that is, each person receives the same grade. I will poll team members, however, to determine each members’ individual contribution. Lack of effort/cooperation  will result in a grade reduction. Within the team, you must negotiate on what each person will contribute. Think carefully about your team members: Where do people live and what hours do they work? Where will you meet? What skills do the different individuals bring to the group (computing, programming, design, evaluation, statistics, etc.)? I want you to form a heterogeneous team of individuals with varying skills. Working with friends who happen to share your skill set is not a formula for success.

For due dates, refer to the class syllabus.

Project Report Book

Each part of the project will include a deliverable report. This report will be placed on T-Square as either pdf generated from Microsoft Word, or a Microsoft Word file. The report must be a single file. Your team name and member names should be at the start of each report.

Each team should have a “home” page that includes:

1)      A brief (one paragraph) description of the problem/task;
2)      Names and email addresses of the team members;
3)      Links to the reports for project parts 1-4 (no report is needed for part 0);
4)      Link to your PowerPoint presentation on Requirements Gathering Techniques and Results (if assigned);
5)      Link to your PowerPoint presentation on Testing and Evaluation Plans (if assigned);
6)      Link to PowerPoint Final Project Presentation.

The format of the reports for the individual parts is up to you, but it should be professionally prepared, expressive, grammatically sound, illustrative of your efforts and process, and easy to read and understand. Teams sometimes use long series of short bullet items for their report; this is not satisfactory – bullets are fine, but they need to be explained and motivated and interconnected. A good design effort can easily be hampered by poor communication of what was done.

Note that the PowerPoint presentations you prepare are NOT the report; they are generally derived from the report but are not a substitute for a written report.

Part 0 – Topic Definition

This is your one-paragraph project description, list of team members and link to (future) project deliverables.  This will be on our T-Square Wiki.

Include a listing of who will (at least initially) fill each of the following roles:

1) Project leader – leads meeting discussions, summarizes meeting results and who has agreed to do what by when.
2) Meeting scheduler – communicates with everyone to establish meeting times/places.
3) Task schedule monitor – checks with team in advance of deliverable due dates to ensure that things are on track.
4) Report integrator – pull together into a coherent whole the parts of reports that are typically written by multiple people.
5) Presentation creator ­– the PowerPoint expert who creates the team’s presentations.

These roles can rotate amongst team members – but it is important to identify who has each role. Also, you can re-define roles. The most important criteria is that everyone knows who is filling what role!

You must also print, sign and turn in a team contract (hard copy in class plus unsigned contract linked to from the wiki project page). Here is a template.  If you would like to make additions, feel free, but you must have the minimum of the template, including peer evaluations.

Part 1 – Understanding the Problem

The key goal of this first part of the project is to deeply understand the problem you are addressing, its set of pertinent users, and the issues and constraints that are involved in the problem. If there is an existing system/interface for performing the task, you should review that system to help you learn more about it. Most important is to identify the important characteristics of the user needs that will influence your subsequent design.

In class we will discuss different techniques for acquiring this kind of information. You should utilize the techniques that you feel are most appropriate to the particular task you are examining. Your report and deliverable for this part should deeply examine the application domain. Who are the potential users? What tasks do they seek to perform? What functionality should the system provide to allow the users to carry out their tasks? What type of hardware platform is most appropriate (traditional windows systems, tablet such as iPad, smart phone such as iPhone). Basically, you are establishing a set of user needs – requirements and constraints – for your subsequent design. You will also address criteria that will be used to judge if your design is a success or not.

Whatever methods you use, they must include face to face interviews with people, either individually or in focus groups.

The Project Part 1 Grading Criteria toward the end of this document outlines how your report should be structured, and indicates how many points are assigned to each section.

Be sure to stress both the functional and non-functional (usability principles) requirements for your eventual design. Functional requirements are all about what the system should do. Non-functional requirements are the usability attributes that this particular design must stress, e.g., learnability, robustness, etc. A successful project report for this part would be one that could be handed to a different group of people not familiar with the topic area, and then that group could do an excellent design.

Part 2 – Design Alternatives

The key goal of part 2 of the project is to use the knowledge gained in part 1, as well as that from class, to develop a set of three design alternatives for your problem. These multiple design alternatives should explore the potential design space for the problem. When you do a design, you have lots of different decisions to make, starting with the basic conceptual model (objects, relations, actions) and then the basic interaction styles (command line, wimp , gui) and then the detailed design (names of commands, layout of menus, ..)  The set of all possible designs is very very large, and is often called a ‘design space.’ The more different your three designs are from one another, the better.

The idea is simply to make the three designs as different from one another as you can. If you can think of three different conceptual designs, that’s great. Or maybe three different metaphors, or three different interaction styles – or combinations of any of these things.  Again, the idea is simply to have the three designs be noticeably different from one another. A rearrangement of the screen layout for a WIMP-style UI would not lead to a noticeably different design.

In this part of the project you will develop mock-ups, storyboards, and sketches of your interface designs. Provide pencil-and-paper or electronic images of the interface at various stages of use; you do not need to build a working prototype. Your design sketches should be sufficiently detailed for a potential user to provide useful feedback about the design, however. Along with your design mock-ups, you should provide a brief narrative walk-through of how the system will work. Perhaps most importantly, you should also include your justifications for why design decisions were made, and what you consider to be the relative strengths and weaknesses of your different designs.

The design process you follow in this part of the project is important. Don’t do the following: The group splits up and everyone creates one design, then these become your alternatives to be turned in. This is not how a good, creative design process works. It should be more like a brainstorming session with all team members present. You should seek to create some fundamentally different design ideas, concepts all over the potential design space for the problem you have chosen.

Your project report should include all the explanatory material mentioned above as well as all the design sketches, drafts, storyboards, etc., that you generated for each of the three designs. If some of your sketches are on paper, scan them. The posters used in your poster session are not a substitute for any part of the report.

Make sure that your report adequately reflects the design process that your group undertook. The key in this part of the project is to come up with fundamentally different design ideas, not just a small set of variations from some basic design.

It is okay if things change from your first report, in fact it is most likely good! The design process is intended to be iterative. Remember, abandoning bad ideas early is a good thing.

We will utilize one full class day as a poster session near the end of this part of the project. Each group will show their three design ideas on a poster in class. Everyone will then circulate and interact with the designers. The idea here is that each group can use this opportunity to get feedback about their design ideas as they narrow their design space and head into part 3 of the project.

The report should follow the outline of the grading criteria found toward the end of this document.

Poster Preparation Suggestions

1)      Set of bullet items to define the problem.
2)      Three panels, one for each of three design alternatives.
3)      Meaningful name for each alternative.
4)      Use screen sketches; if an action on one screen leads to another, draw a line leading from the action (button selection, menu item, etc.) to the new screen). See these pictures as examples:

Part 3 – System Prototype and Evaluation Plan

In part 3 of the project, your group will implement a detailed prototype of your interface. Use whatever software tools that you know and that are good for prototyping, such as Visual Basic, Flash, Hypercard, Macromedia Director or a web page editor. You should be able to get much of the interface functionality working, but in most cases you will not be able to implement all the back-end application functionality.

Provide a set of usability specifications for your system and a plan for an evaluation of it. To develop usability specifications, consider the objectives of your design. For example, if you are working on a calendar manager, you might specify time limits in which you expect a user to be able to schedule or modify an appointment, or a maximum number of errors that you expect to occur. Basically, you should list a set of criteria by which your interface can be evaluated. This will be a refinement of the usability goals you developed in Part 1.

Describe your initial evaluation plan for the system. What kinds of benchmark tasks would you have users perform to help evaluate the interface? What kind of subjective questionnaire would you deploy to have a user critique the interface? You will need to perform this evaluation in project part 4, so you should do your best to set it up now. The key here is not to do some exhaustive description of a usability evaluation plan, but to motivate why the particular plan you propose is appropriate for this interface.

Note that developing an initial evaluation plan is also a good way to figure out how much of the interface you need to develop. You should be able to build and connect to enough of the application functionality to be able to conduct an initial usability evaluation with the benchmark tasks as you are proposing here.

Your write-up for this part should include a description of your system prototype. You can include screen dumps to help explain it and text to describe how a user would interact with it. Discuss the implementation challenges you faced. Were there aspects that you wanted to build but were unable to do so? The key component to include in your project report is a justification of why you settled on the design that you chose. What’s special about this particular design with respect your problem?

The report for this part also must include the usability specifications that you established and a description of the evaluation that you are planning. This needs not be too detailed here as the actual evaluation will occur in part 4. We will try to give you helpful feedback about your plan here to assist with the testing in part 4.

Specifically, the report should follow the outline of the grading criteria for Part 3.

Part 4 – Evaluation

In the final part of the project, your group will conduct an evaluation of the prototype developed in part 3. You should utilize the evaluation measures, usage scenarios and personas that you identified in that part and in part 1. We expect that your evaluation will involve sample users interacting with your system. These users will likely be your client(s) and maybe other students from class or other people who would fit your target user population. Give the users a few simple benchmark tasks and have them interact with your interface. Make sure your usage scenarios are included. Closely study what occurs. Use a questionnaire to get subjective feedback about the interface and interaction. Since you are studying humans, you will need to provide participants with a consent form. A sample consent form is at the end of this document.

In some semesters, your evaluation will also include a Cognitive Walkthrough (this is a separate assignment for which you prepare a PowerPoint presentation summarizing the results).

Your report should follow the outline of the grading criteria for Part 4.

The key to this part of the project is not to simply describe your evaluation methodology but to rise above that and describe what you learned from it. Explain why you chose the benchmark tasks that you did. Why did you ask users what you asked? What conclusions can you draw from the studies? What aspects of your design “worked” and what failed to meet your specifications? If you had more time to work on the design, what would you now change and improve? Remember, no designer ever gets a system “just right.” We will reward teams who honestly and carefully assess their design and who clearly provide a plan for its improvement.

Final Project Presentation

Your final project presentation should take no more than 10 minutes and allow for a 5-minute question period afterwards. It should be more than just a re-hash of your Project Part 4 document. It is important that you do a good job communicating all your efforts for the semester. You want to make sure that your objectives in the project are discussed, your system is clearly presented, and your evaluation results/conclusions are clear. It is important to describe what you learned from the project. What are the take-away lessons that you will use in the future? What would you have done differently? Why?

Practice your presentation several times. Ten minutes is not long, so plan a snappy, get-to-the-point talk. Remember that you do not have to read to the audience everything that is on each PPt.

You will be graded on the final presentation, on the following criteria:

  • Keeping to time limit. (10 minutes)
  • Covering all the elements in the suggested outline.
  • Professional quality of the PowerPoint presentation itself, and professional demeanor and dress of the presenters (in the summer, professional dress can be very relaxed).
  • How well you handle questions.

All four team members should give the presentation, with a smooth hand-off from one to the other. Everyone can be involved in answering questions.

Final PPt presentation outline:

Introduction: Group name, members, overall goal for project.

 

Initial Designs: Summarize your three initial designs.

Narrowing process: What final design did you choose, and why?

 

Final Prototype: An explanation of the prototype you built for evaluation (demo of the system including any changes you made during the evaluation process).

 

Evaluation results: What did you learn about your prototype? Give quantitative and qualitative results from your evaluation. If you have a lot of results, emphasize the most interesting  / useful.

Changes to Prototype:  What changes did you make to the prototype during testing, if any?  What changes would you make now, based on the testing results or observations?

 

Lessons Learned: What take-away lessons did you learn from the entire project, what would you do differently (and why) if you were to start over again – both in terms of the dynamics/structure of the group process itself, and in terms of the project?

 

…And then there will be 5 minutes for questions.