By Alex Fisher, Instructor of Accounting

Redesigning for the Online Learning Environment

Transitioning to the remote learning environment from face-to-face has raised some challenges, but also offered a unique opportunity for overall course improvement. In my experience, I’ve found that not all assessments translate 1:1 as each environment carries its own set of pros, cons, and administrative considerations. However, I found that when an assessment is well designed to clearly align with its learning outcomes (LOs), it can work well in either environment regardless. Achieving this flexibility takes time and is the result of an iterative process as you continue tweaking and improving assessments after your initial design.

Below I share some personal examples and considerations for redesigning your assessments for the online environment. These have been inspired by my own personal experience as well as input from our Schulich community. They can help ensure learning outcomes are met, reduce the possibility of academic dishonesty, and build in variation opportunities to keep content current each term.

There is considerable room for changes to your assessments as long as they don’t impact your course’s learning outcomes, core content, or contradict any policies, procedures, or guidelines.

Start With Your Learning Outcomes

Your learning outcomes (LOs) are key guidelines when (re)designing your course content and assessments. Ask yourself, “what should students be able to do at the end of this course?” and “do my assessments align with the LOs?” Consider the following:

  • Does your course include a mix of quantitative and qualitative assessment types?
  • Do your assessments reveal and reward the process and the ability to analyze and think critically?
  • Do your assessments do more than require students to calculate the final answer?
  • Do your assessments look like the kind of activity students will be doing when they enter the workforce?
  • Are your assessments designed in a way that prioritizes the students’ learning outcomes?

Note that more qualitative assessments may increase workloads for both students and instructors, but they may also be necessary to achieve your LOs. Find an appropriate balance between achieving LOs and student and instructor workloads to ensure no one is overwhelmed.

Assessment Redesign Example – Before & After

When transforming a question, first agree on the key LOs which are your goals and guiding principles. For example, below are the principal thematic learning outcomes for ACTG 2010, Introduction to Financial Accounting as outlined in the Course Syllabus (which are further disaggregated into about 60 topic-specific learning objectives which are presented to students on a class-by-class basis). By the end of ACTG 2010, students will be expected to:

  1. Have developed a foundational understanding of the accounting processes that underlie financial statements.
  2. Be familiar with the standard forms of accounting reports, and able to prepare and interpret them.
  3. Develop critical thinking and problem-solving skills.
  4. Understand the needs of different users of accounting reports, and to choose and evaluate the financial information appropriate to these users.
  5. Be able to select appropriate accounting information from financial statements, and use it effectively for decision making.
  6. Have developed a critical understanding of the many roles of financial accounting information in organizations and society.

Keep these LOs in mind to see how they align with the following ‘before’ and ‘after’ example of a related midterm question on cash flows. Originally students were required to build a cash flow statement from provided data. The redesigned version instead provided a final cash flow statement / balance sheet initially, and required students to explain it, analyze it, and determine what is was for.

Example Question on Cash Flow – Before

Provided Information:

QUESTION 1: (40 MINUTES, 20 MARKS)
Toys & Tots Ltd, a Canadian private company, prepared the following financial statements (see ‘Before Image’) in accordance with International Financial Reporting Standards (IFRS). Note: Additional company details were also provided.

Student Task:

Prepare, in good format, using the indirect method, a statement of cash flows.

Before Image: Financial Statements
(select to enlarge)

Example Question on Cash Flow – After

Provided Information:

QUESTION 1: (30 MINUTES, 15 MARKS)
You have recently been hired as a new junior financial analyst for Investment Corp. Your manager has asked you to examine the Statement of Cash Flows (see ‘After Image’) for the following three separate companies which operate in the same industry. She comments that “despite identical net incomes and total cash, there are differences in the way that each company manages its cash flows. I have some questions that I am hoping you can answer.”

After Image: Statement of Cash Flows
(select to enlarge)
© Alex Fisher 2020

Student Task: 

Respond to each of following questions. Be specific and ensure to briefly support your answer.

  1. Which company appears to be financing its assets primarily through debt?
  2. Which company appears to be increasing sales by selling on account (i.e. selling on credit)?
  3. Which company appears to be slowing payment to suppliers?
  4. Which company is the most committed to growth?
  5. Which company had no net free cash flow in 2018?
  6. Which company purchased land by exchanging/issuing shares (i.e. a non-monetary transaction)?
  7. If you were a shareholder wanting to receive cash dividends but were not too interested in owning shares for a long time, which company’s shares might you consider buying?

Before & After Redesign Summary

To complete the original ‘before’ question, a student must understand the mechanics and numbers. However, the assessment is purely quantitative, aligning with the first and second course LOs previously listed. In order to complete the redesigned ‘after’ question successfully, a student must still understand the mechanics and numbers, but their response must go beyond that and provide some real professional input as well. Notice the redesigned question now aligns with several course LOs, better preparing our students to recognize and adapt their learned skills to similar scenarios as graduates. After all, we are creating business leaders not just bookkeepers.

Design Tip: Notice either question style allows for relatively easy variations in both current and future terms. You can randomly assign different companies or alternative numbers in the provided statements. Another variation may be to solve for missing data when provided other related information. This will help keep content current and reinforce academic integrity while maintaining the intended LO. If using Canvas Quizzes, you can also incorporate these variations within the ‘question bank’ feature, randomly selecting one of many similar questions within the same exam.

Keep Your Questions Current

Not only is it good practice to keep both your course and questions current, it also limits opportunities for academic honesty infractions. Consider the many resources students may have access to, including previous assessments and online resources (e.g., Course Hero, reused textbook Q/As etc.). The reality is that any course content can be copied or made available online, regardless of the learning environment. Rather than fighting an up-hill battle to prevent sharing, consider designing assessments in ways that will ensure academic integrity regardless of if they are shared.

Suggestions:

  1. Avoid the single final answer. Consider breaking down questions where the final answer is worth 3/10, but the process is worth 7/10. Have students describe their thought process and calculations.
    • Design questions that challenge students’ understanding of course content, and encourages connections between the course and real-world scenarios.
    • Minimize the use of multiple-choice questions. These limit opportunities for students to reveal their understanding of the processes you are assessing.
  2. Build in variation opportunities where questions can be easily revised or updated each term vs. duplicated (academic honesty concern) or limited to the 1 time use (increased instructor workload).
    • Example: Change a company name, dates, initial set of data and/or variables within calculations.
  3. Ensure the assessment difficulty will get you into the grade range you need.
    • Design challenging, yet reasonable expectations, student workload, and timelines.
    • Minimize assessments where students may get full marks without encouraging their critical thinking skills. Ex: Minimize multiple choice questions or reserve them for smaller assessments.
  4. Reduce assessment reliance on tech.
    • Consider if a student lost internet for 30 minutes during your assessment window. How would that impact the student, the assessment and the course?
    • If using a Canvas ‘Quiz’: Students require a consistent internet connection when completing the quiz.
    • If using a Canvas ‘Assignment’: Students often only need internet twice, at the start and end. For instance, students can download and upload a Word file within their allotted time window. Tip: Allow an extra 15 minutes (5 to download, 10 to upload) for students to resolve technical difficulties or contact you with any concerns.

Limit Academic Integrity Issues

  1. Favour critical thinking assessments with student’s own professional input. Go beyond the single response questions, encouraging students to analyze and explain their reasoning.
  2. Narrow the assessment window where appropriate. Example: If held during class time, there is less time to collaborate.
  3. Randomize questions. The New Quizzes feature in Canvas allows for question banks to randomly display a number of questions from a predetermined group. You can also randomize the order of questions or answers.
  4. Use Turnitin to check for potential similarities between student assignments.
  5. Incorporate a holistic approach. Communicate academic integrity expectations clearly and frequently:
    • In the syllabus under related policies.
    • In Canvas within an initial announcement or as an initial ‘Student Honour Statement’ assignment, intended to uphold academic integrity throughout the course.
    • During class time.
    • Within each assessment description. Example: Include the following ‘Sample Student Honour Statement Text’ (select to view / hide) and mention the use of Turnitin where appropriate.

Your submitted Assignment will be subject to automatic review via Turnitin, which will identify instances of plagiarism and any unusual similarities among student responses for further investigation.

By completing and submitting this Assignment on Canvas, you are expressly agreeing to the following:

Academic Integrity – Student Honour Statement

“As a member of this class and of York University, I commit myself to the values and practices set forth in the Policies of Academic Honesty of both the Schulich School of Business and York University. I understand that I have a responsibility to maintain these values and, in order to do so, I will not engage in any form of cheating or other breach of academic honesty as defined by the Policies of Academic Honesty at the Schulich School of Business and York University.

More specifically, I promise to complete any remote online assessment in this course individually and without help from others (including former or current students enrolled in this course, outside tutors, or any other third party). I hereby expressly acknowledge and agree that engaging the help of others, including collaborating to answer questions together, is not permitted and constitutes academic dishonesty (i.e., cheating).

I also hereby expressly acknowledge and understand that, if I commit any form of academic dishonesty as described above, I may be subject to disciplinary action under York University policy which may result in serious adverse academic consequences for me.”

Importance of Communication

Clear and timely communication is incredibly important for a positive course experience for both instructors and students alike. Communication is especially important in the remote learning environment where it can be harder to openly clarify expectations than the typical classroom setting.

Suggestions:

  1. Include buffer times before and after the assessment length:
    • Clearly communicate what the extra time is intended for, such as time to download and upload related files, submitting the assignment, or in case of technical delays.
    • State the extra time is not intended for working on the assignment, and to manage this time appropriately.
  2. Clarify expectations for late assignments:
    • Clearly state due dates and any penalties for not adhering to them.
    • Remind students it is their responsibility to prep their computers and have a backup strategy for internet concerns.
  3. Include a backup plan:
    • Require an email to the instructor in the event of a technical issue, if late, or unable to submit on Canvas. Use your discretion to appropriately resolve the concern.
  4. Communicate assessment expectations clearly and frequently:
    • Examples include in the syllabus, Canvas, during class, and within assignments themselves.
    • See the following ‘Communicating with Students About Online Evaluations’ image for more detail.
  5. Provide timely grading and feedback:
    • See the following ‘Grading and Feedback’ image for more detail.

Final Thoughts

After considering all of this you may wish to look at your mix of assessments across your course vs. redesigning each individually. For example, you may find that delivering frequent short multiple choice ‘knowledge check’ quizzes complement a more qualitative final. The challenges that come along with online assessment design, such as administration, academic integrity concerns, and communication have always been present. They’ve simply been highlighted through an environment that requires more explicit expectations. Redesigning your assessments may take some time and iterations, but will ultimately improve the quality and integrity of your course. Don’t lose sight of your LOs as the key goal. Good luck!