User Tutorials

Who this tutorial is for

This tutorial is for any user of RubricApp (ELIPSS SkillBuilder). This includes instructors, teaching assistants, and researchers.

Table of contents

• Device requirements
• Create a new account and log in
• Projects
• Evaluations
• Rating groups
• Downloading data on group ratings
• Emailing students feedback
• When you have multiple raters in a course
• Editing the roster file
• Editing the rubric file

Device requirements

RubricApp is designed for use with any device with an internet connection and a web browser. Use it on your computer, smartphone, or tablet!

Create a new account and log in

1. From the homepage, click Signup.
2. Enter an email address and a password. Your email address serves as your username, and for security the password should be unique for this website!
3. Click Create account.
4. You'll be taken back to the login page. Enter your email address and password again to login.

Projects

A "Project" in RubricApp consists of a single rubric and a single roster of student groups. The idea of a Project, then, is that you will rate groups on the same rubric multiple times throughout the semester. This lets you track growth over time.

What if I want to use multiple rubrics throughout the semester?

All you need to do is create a new Project for each rubric that you plan to use.

Create a new Project

In this tutorial you'll set up a new Project for evaluating students on the ELIPSS rubric for Teamwork.

1. Click Create New Project in the top menu bar.

2. Enter a project name, along with a description that will remind you what the Project is. For example...

• project name: Teamwork
• description: A sample project using an ELIPSS rubric for Teamwork

3. Click (Download a sample roster files). This will download an Excel spreadsheet file (.xlsx) that contains the students in the course. See Editing the roster file for more information on the format of this file.

4. Upload the roster by clicking Choose file and selecting the .xlsx file you downloaded.

5. Click (Browse sample rubric files). Navigate to the teamwork folder, click teamwork_scale3.json, then right-click the Raw button and choose Save link as... (if you cannot right-click, then click Raw, then on the next page use Ctrl-s on Windows or Cmd-s on MacOS). If prompted with a file explorer, choose a place to save the file to. This action will download a rubric file (.json) that contains the contents of the rubric. See Editing the rubric file for more information on the format of this file.

6. Upload the rubric file by clicking Choose file and selecting the .json file you downloaded.

7. Click submit

8. You'll now see your Project called Teamwork in the list on the Projects page.

Evaluations

Each Project in RubricApp can have any number of "Evaluations". An Evaluation represents a time in the semester (e.g., a class period) when you rate all (or just some) of your groups on the rubric's criteria. You can look at performance over multiple Evaluations to see how groups grow in the skill you are working on.

Create a new Evaluation

Ideally, create the new Evaluation before the class period begins.

1. If not already in Home > Projects, get there by clicking Projects in the top menu bar.
2. Click the Project whose rubric you wish to evaluate students on. In our example, we'll pick the Project called Teamwork.

3. Click the Create a New Evaluation tab. Enter an Evaluation name (and optionally, description) that will help you remember when you did the Evaluation. In our example, we'll call the first Evaluation "Week 1" and leave the description blank. Click the Create a New Evaluation button.

4. The website will take you directly into the rating page.

Rating your groups

You can rate groups either during class or outside of class, depending on what you are evaluating.

Perform an evaluation

In this tutorial, you will evaluate a number of groups on the Teamwork rubric. The specific example assumes you've already created the Project called Teamwork using the sample roster, as well as an Evaluation called Week1.

1. If you are not already in the rating page, click Projects in the top menu bar, then click on the Project you want to use, then click on the Existing Evaluations tab. Then click on the a button (meaning metagroup 'a').

2. Let's suppose we are the instructor/TA assigned to rate groups in metagroup 'a'. If you haven't already chosen metagroup 'a', then you can click Switch Meta-Group in the left side menu and click meta group a. Let's rate group 'B'. Under Switch Group, if 'B' isn't already selected, then click 'B'. The breadcrumb at the top should now read Home > Projects > Teamwork1 > Week1 > a > B. The 'a' here indicates 'metagroup a' and the 'B' here indicates group 'B'. You'll also see 'B' highlighted under Switch Group in the left side menu.

3. Let's rate group B. For the first category (Interacting), make your selections. Note that in the ELIPSS rubrics, the default level N/A means you did not rate the group, while the level No Evidence (0) means that you observed that the group members displayed no evidence that they "Communicated with each other and worked together". When you are done with the category, be sure to click Save.

4. Rate group B on the rest of the categories. Or, if you want you can choose not to rate them. Note that the Save button must be clicked each time you are done with a category. Note that saved ratings can be changed - you can always re-rate a category and press Save again.

5. Rate another group by clicking its button under Switch Group.

6. If you were assigned to rate another metagroup, you could choose that metagroup under Switch Meta-group. As a result, you'll see the subset of groups change under Switch Group.

Viewing overall progress

As the instructor, during or after class you might be curious if any groups were missed. There is a dashboard for getting this information.

1. Click Manage Projects in the top menu bar.

2. Find your Project in the list (e.g., Teamwork). Click its Manage button.

3. Find the Evaluation you want to see the progress for. You'll see two items listing the number of Complete and Incomplete groups. The definition of "Complete" is that at least one category has been saved for that group. If you click a group's button, it will take you to the rating page for that group.

4. If you do want to see which categories have been rated for a given group, you can click Display all details and look for checkmarks in every column.

Downloading data on group ratings

You might like to download the ratings data for the purposes of analysis.

1. Click Manage Projects in the top menu bar.
2. Find the Project whose data you want.
3. Click Download all evaluations. This will download an Excel spreadsheet file (.xlsx) containing the data.
4. The spreadsheet has multiple sub-sheets. The ratings data is in the sub-sheet called eva.

Sub-sheets explained

• eva - Likert scale data are given as the numeric score associated with the level. Checkbox data are given as a|b|c..., where the letters refer to the checkboxes in the order they appear in the rubric. Missing data are given as blank cells.

• students - One row per student.

• meta - The assignment of groups into metagroups.

• group - the student emails assigned to each group

Emailing students feedback

Although this feature appears in Manage Projects, it is currently (as of 2020-08-13) under repair. When it works again, the intended behavior is to send all students a PDF copy of the rubric with the ratings filled in. Students in the same group each receive an identical copy of the feedback. Please email [email protected] if you have questions.

When you have multiple raters in a course

It may be common for an instructor and one or more teaching assistants (TAs) to share the task of rating the groups. This is especially desirable for courses with a large number of groups. There are three recommended methods.

Tip: If you want to try out Method 1 or Method 3 easily on your own, use the following tip to log into multiple accounts on the same device simultaneously. Open the website in "incognito" or "private" tabs, one for each account.

Method 1: Use RubricApp's sharing feature

You should use this method if:

• You want to maintain separate accounts between users
• It is okay if all TAs are granted access to view and rate the whole class roster
• You want to use the metagroups feature to assign subsets of groups to TAs

This tutorial will involve two sample users: [email protected] and [email protected]. The instructor will own the Project, create evaluations, and may rate groups or may not. The TAs will only rate groups. You can repeat the sharing process for as many TAs as you need to.

1. Make sure both the instructor and TA have created an account on RubricApp.

Logged in as the instructor

2. If the Project the instructor intends to share doesn't exist yet, create it in the usual way.
3. Click Manage Projects in the top menu bar.
4. Click Manage for the Project the instructor wishes to share.
5. Scroll down to the area called Permission and click Create new Permission to Share your Rubric.
6. Enter the username of your TA. In our example, this is [email protected]. Then click Share!.
7. Make sure you create the Evaluations for your Project. TAs are not able to do so.

Logged in as the TA

7. On the Projects page, click the Shared project tab. You should now see the instructor's project listed.
8. Follow instructions for performing an evaluation.

Method 2: Share one account

You should use this method if:

• You are okay with sharing a single account and credentials between all staff members
• You want to use the metagroups feature to assign subsets of groups to TAs

You are allowed to simultaneously be logged in on multiple devices at one time and rate different groups at the same time. It just means you'll have to share one set of credentials among all your staff members.

Method 3: Separate Project in different accounts for each "metagroup"

You should use this method if:

• You want to maintain separate accounts between users
• You want TAs to only have access to a subset of groups

Each staff member creates their own Project with just their own subset of the class roster. Each staff member has to manage their Project separately. If you still want one person (e.g., the instructor) to be able to view all of these individual Projects, you can use the Sharing feature (Method 1) but in reverse. That is, have each TA share their project with the instructor.

Editing the roster file

The roster file is an excel spreadsheet with four required columns.

Student Email group meta

• Student - name of the student
• Email - email of the student
• group - name of the group student is assigned to. Must be exactly identical for members of the same group.
• meta - name of the metagroup that the student's group is assigned to. All students in the same group should be in the same metagroup.

What is a metagroup?

A metagroup is a group of groups. The purpose is to make it easier to view just a subset of groups while rating. For example, if you have four raters (e.g., instructor and 3 teaching assistants) or four different sections, you might create four metagroups.

Editing the rubric file

(coming soon)