Conceptualising different amounts of student engagement and its evaluation within a course

Jutta Törnqvist, Lecturer in Online Media, Department of Culture and Media, Arcada UAS

This text discusses student-centered learning and experiments with a new approach to meet the needs of a broader type of students with varying educational and socio-economic backgrounds. As they must complete 30 credits of elective extension studies as part of their degree, students find themselves in situations where they identify and enrol for interesting subjects in other departments that nicely complement their own major studies. However, these subjects might not be in the core specialisation of the student’s main studies, so they require a more introductory level course based on common knowledge to give that subtle touch and varied profile for the student’s professional identity. At the same time, the course is also taken by students who are majoring in that very subject, and are looking to deepen their skills and competencies in the subject. The question becomes how a lecturer facilitate these very different kinds of learning needs in the same course? That’s what I wanted to experiment with, and I chose to work with the course UX & UI for my case study. In light of the need for new assessment techniques to be sustainable, this experiment presents themes using an environmentally-friendly motif.

WHY ON EARTH?

The course layout within Online Media, one of five specializations in the Degree Programme of Film & Media, is very broad and serves to provide the student with a wide range of competencies throughout their studies. The possibility to use the elective extension studies for more specific profiling results in the fact that professional identities of individual Online Media students can be very different to one another on the day they graduate.

The experiment described in this text just sprung out of a sudden rush of creative thinking when a course, UX & UI, surprisingly got a request to expand the number of enrolled to give Information Technology students a relevant elective course.  Overnight, the class size doubled and the learning needs were clearly quite diverse within the group. The thoughts this change inspired had been sprouting in my mind for a long time, and this circumstance made them spring to life.

I recognize and describe the most essential ingredients for this happening in the following way:
1) The well-watered soil: the never-ending struggle to plan a course with the right amount of workload for all students;
2) The fertiliser: the need to evaluate student work in a way that is detailed enough for the amount of work and range of skills demonstrated;
3) The seeds: the final ingredient that made my thoughts bear fruit was Bloom’s taxonomy, especially in a digital age (Churches, 2012). We use this model of classifying learning in our curricula and often discuss how to implement it better.

THE SOIL – WORKLOAD

Since students are such different kinds of learners with different kinds of abilities, and possible disabilities, quantifying performance through a measure of hours per credits is a very problematic starting point. Is it in fact fair at all? Already the lecturer makes assumptions about the ambitions the students will have, as well as about the amount of work these supposedly ambitious students will do. Someone might need more time to read and write, whereas someone else has an easy time with tools and techniques needed in the course and can achieve similar results faster. How is the amount of work best estimated then? Few lecturers have solved this problem, yet. Still my impression of many of my courses is that even if the student input is less than I have intended or expected, it may still offering great learning experiences, nevertheless. So how then should workload be ‘measured’ or demonstrated in such a way as to better illustrate student performance on the individual level?

THE FERTILIZER – EVALUATION

I teach in thematic blocks where each block handles one subject both in theory and practice. UX & UI contains five blocks, starting from a theory block taking that theory into practice in later blocks. More theories are provided in between blocks following the flipped classroom model. (Bergmann & Sams, 2012) This gives a somewhat solid framework for evaluating the tasks and learning. Still I see a problematic nuance of the grader’s subjectivity in the evaluation, no matter how hard one tries not to be subjective.

Let’s look at an example: designing a user interface and motivating the design decisions based on theories presented in the course. Naturally we first look at the quality as well as the quantity of the theories the student has used in the theoretical part of the task. This should be quite easy to evaluate, especially when the task helps the student by mentioning the minimum number of sources that should be used. Then there is the more creative part of the task, evaluating the user interface. Even if the theories steer the design in some parts, there’s still room for that personal touch in the design. And here personal references like taste and practical limitations like technical skills might affect the evaluation: “That gradient colour is really not working for me”, “No, not Comic Sans!” These are examples on things that make a difference even if you know they shouldn’t matter in the general aspect of the task. (Yes, I know, Comic Sans actually makes a difference concerning usability, but let’s not go there!) Sometimes also the evaluation might be influenced by a single students’ huge efforts and big progress in learning, even though the final product is not that impressive. When such a thing is obvious and measurable, isn’t it more important to applaud the progress than focus on the ugly interface the student might have designed? The point is that evaluation is a rocky road and rarely can be pinpointed as exactly and fairly as it implies.

PLANTING THE SEEDS – BLOOM’S TAXONOMY

When planning the Online Media curricula throughout the years we have always kept a narrative, a road map, in mind. The road map takes the students from a foundational and more shallow level to a higher level where they broaden as well as deepen their skills. They go from recognizing and understanding via applying and analysing to evaluating and creating (Churches, 2012 p. 5). I decided to take these steps also into the concept of packaging the course content.

A very familiar and international concept of sizing when it comes to clothing is S (small), M (medium) and L (large). I decided to package the course material into different study paths for students with different needs of knowledge based on those recognised norms. The Online Media students that are specialising in web design need path M or L whereas Information Technology students can manage with just the foundational knowledge in package S. They need to understand design processes and usability if working as coders, they don’t necessarily need to be masters of the design. The choice of path also might activate the student to think about the professional identity they want to have and forces them to plan using concrete aims: “Is this a nice-to-know topic or the core of my dreams/future career?” It makes them assess their own study needs and the level of performance they are willing to commit to the course in order to meet that goal. Here is an example of how it appears on the online learning site:

Figure 1. Illustrating the content of the different study paths in UX & UI, presented on Itslearning

Let’s look at the different customer journeys (to use marketing parlance) to open up the potential of this concept of packaging.

S, recognizing and understanding

When choosing path S, the student gets a foundational knowledge base within UX & UI, recognizes theories, problems and understands how to solve them due to the inclusion of several practical exercises. The quantity or amount of input is the least possible in the course – but without losing quality! Choosing S doesn’t mean there is no quality check, it just means less work, and more time to complete those tasks leading to more basic learning outcomes than in the other paths. Therefore, this path is graded with 1 or 2 (out of a maximum of 5).

M, applying and analyzing

A student choosing the middle path gets the exact same tasks and skills as in S, but also gets exercises in applying the theories and analysing the work accordingly. The workload is larger and more indepth, and its learning outcomes are more advanced compared to S, so this path is graded with 3 or 4.

L, evaluating and creating

The ambitious student, or the student knowing the subject is core knowledge in future plans, chooses path L and works to meet the grading criteria of a 5. Since this package adds one fifth to the grading, it just brings that little deepening analytical ‘extra’ to the student’s work, who must complete the same tasks that are in path S and M, but on a more critical analytical level. Here the learning experience is both broad and deep, evaluating and creating more than in the other paths – taking all the steps mentioned in Bloom’s taxonomy.

Figure 2. Matrix to illustrate the correlation of engagement, knowledge and grading

Figure 3. The ladder structure of course tasks, showing the increasing workload and knowledge in the different paths.

SO, WHAT BLOOMS?

Now as the course is redesigned, I’m ready to rock the actual experiment! With double the number of students from the previous year, I set off to test this setup. There are many challenges: I still feel that evaluating the actual quality of the workload is not completely solved with this method, but I feel that a certain package leading to certain grade (and knowing about it (and the related criteria) beforehand) at least makes workload and grade correlate better. It is also possible that I have made my own workload unbearable as I must create and manage all the different levels of assignments on Itslearning. The crucial thing during the course as well as afterwards is to gather course feedback and analyse it for further development – or scrapping.

While writing the last lines of this blog, the UX & UI course is halfway through and there are already many lessons learned. I look forward to the aftermath when I can reflect and analyse it all – published here as the sequel to this blog.

 

REFERENCES

Churches, A. 2012. Bloom’s Digital Taxonomy. Available:

https://www.academia.edu/30868755/Andrew_Churches_-_Blooms_Digital_Taxonomy.pdf (External link) [Accessed: 24.9.2019].

Bergmann, Jonathan. Sams, Aaron. Flip Your Classroom: Reach Every Student in Every Class Every Day, International Society for Technology in Education, 2012, Washington, DC. ISBN 9781564845603

Student-centered learning. [online]. Available at: https://www.edglossary.org/student-centered-learning (External link)

[Accessed: 24.9.2019]

 

jutta.tornqvist@arcada.fi