The Total Learner Experience

Don't Be Half-Assessed

Assessment is used to measure learning outcomes, but if you see it only as a testing tool you’re missing half its value. Assessment should not be punitive; it should uncover the learner’s strengths and help identify areas for improvement. Well planned and integrated assessment creates opportunities for learners to reflect on their learning, apply new skills and knowledge, and also enables the business to recognize ROI on the learning intervention. Adapting existing models can take you much further than re-creating the wheel. In this post, we examine some guiding principles for effective assessment.

An excellent example of assessment as a way to support learning goals is Adobe’s Certified Associate practice tests. Learners can take a certification exam without any structured practice, but there is also an unlimited certification prep test package. The practice test closely mimics the structure of the final exam. Both combine multiple choice questions and simulations of the Adobe environment. The multiple choice questions require the learner to demonstrate knowledge of design concepts and the production process. The simulations require the learner to complete a task within the simulated software application interface. Keyboard shortcuts are disabled, but otherwise, learners can use any correct method they choose to complete the task, using menus or panels as appropriate.

Another good example is the Sun (now Oracle) Java certification paths. Each path contains a test prep “kit” that includes preparation recommendations, additional resources, a practice test, and a re-take policy. Each path is designed to prepare the learner to achieve a specific level of certification, and used as a benchmark against industry standards. These certifications are recognized by employers and can advance a person in their career.

Accreditation or certification can be used to validate mastery of a topic, but this is not the only way that assessment can be useful. By seeing assessment an integral part of instruction, we can support the learner’s career development while measuring true performance for the business.  

Factors that make an ASSESSMENT tool also a useful INSTRUCTIONAL tool are:

• Authenticity
  • The learner performs the task in context, not recalling theory, but actually demonstrating competency.
• Open ended
  • Because the end product is assessed, not the method used to get there, learners are able to use whatever menus or panels they choose.
• Learning while doing
  • Learners use contextual clues and critical thinking to complete tasks. They may not know how to adjust alpha levels in Photoshop, but they may know to investigate the color panel to find them.
• Self-reporting
  • Learners can mark questions that they’d like to return to if they have time or opportunity.
• Cumulative time
  • The test is timed with one master clock, not with individual times for certain sections or items.
• Feedback
  • Learners receive feedback on each item, with notes about the correct answer.
• Tracking
  • Performance from one practice test to another is tracked.

Assessment is not something that should only occur in a testing situation, indicating pass/fail rates, but it should be integrated throughout instruction to allow the learner to know how they are doing, so they can learn more effectively. Don’t leave your learner half-assessed!

 

Create Losing Designs

We spend a great deal of time planning for learner success. Do we ever create the ability for learners to lose? Should we?

Our society puts a premium on “winning” -- nowadays it seems like every team gets a trophy -- not just the winner. Americans are culturally bound to the ideal of winning. However, in the real world people lose all of the time: athletes lose games and tournaments, politicians lose elections. In high stakes situations like these, losing has a real consequence. How does this affect learning?

Low stakes losing occurs every day. When we lose, we tend to either give up, or continue to practice to overcome the failure. Any casual gamer is familiar with losing and attempting to overcome the loss by quickly replaying the game. Losing, in this context, can be a big motivator, driving the will to practice, which can lead to increased skill. Learning to lose effectively is actually a skill in itself. Resiliency in the face of losing and using strategies to improve is an important life lesson and professional skill. If losing always equates to complete failure, then learners stop striving, stop attempting creative solutions, and see themselves as incapable.

In learning design, a primary goal should be to create an atmosphere where losing is acceptable and intrinsically motivates the learner to try again. The potential learning value from this type of experience is measured not by how often the player loses, but by how much they improve through repeated practice.

In your training design include opportunities for low stakes losing before learners are thrust into their high stakes, real-world situations -- be it a qualifying exam or on the job performance.

Recommendations for a “losing” design:

• Make content available on demand so that learners can review material as needed
  • A common mistake is being overly controlling with content. If the goal is for learners to gain knowledge and skills, shouldn’t they have free access to the tools that can make that happen?
• Communicate how the learner is performing relative to passing scores or via peer-to-peer review as appropriate
  • By communicating how comparatively well they are doing, learners can better sense where they can improve. This also gets their competitive juices flowing.
• Allow plenty of practice opportunities before the final assessment occurs
  • If the only time that learners can demonstrate knowledge is a final graded test, their opportunities to “lose” are reduced to one high stakes moment. Build in low stakes assessment opportunities that prepare learners for the one that counts.

On Winning

“Training” and “competition” are not usually words that are associated with each other. However, competition is an innate motivator and humans by nature enjoy winning.

Certain aspects of winning are universal to all competitive activities, including learning. A recent Newsweek article about winning provides insight into how instructional designers can create more engaging training. The author notes that winning by itself is not the most compelling impetus, but that winning while a competitor loses is more satisfying (this would seem obvious to anyone with siblings).

Rather than using  a “task completion” metaphor, instructional designers should use a gaming and winning metaphor when designing training. Rote tasks can be made more engaging if instead of simply reading and reacting in a safe environment, the learner triumphs over a tension-filled activity. Similarly, you can provide competitive opportunities with other learners virtually.

In many K-12 situations “safe” learning environments take out so much of the excitement of competition and rating. You have an obligation to make sure every beginner learner succeeds, but you also have an obligation to groom the special talents that individuals have as well.

How do you turn cognitive tasks into challenge?

Learning objectives can remain the same, but it’s the way that it is presented that changes. You don't have to completely redesign your training to make it more challenging. Consider these simple ideas:

Allow learners to be wrong

• Allow and penalize for incorrect answers. If learners can complete a course by merely clicking through content, they have little reason to engage with the content. Activities that allow for “failure” can create “good tension”.

Add variable scoring

• Reward learners for learning more difficult material by acknowledging that all content is not equal.

Add timed components for some activities

• While not appropriate for all activities, it does create a feeling of tension.

Allow for replay opportunities

• This reduces some of the negative aspects of the capability for “failure” allowed by other competitive components.

Create opportunities to share leaderboard scores outside of the learning environment

• Rather than learning taking place within a silo, authentic learning events can bleed into the larger community.

A simple multiple choice game can be either boring or competitive with a simple design tweak:

Boring

• The learner answers a question and then views the answers tagged with "correct" and "incorrect" feedback.

Competitive

• The learner answers a question. They receive variable points based upon the correctness of the answer, difficulty of the challenge and the speed with which they answered. They are then shown their score relative to other learners. The feedback is also contextual and continues the gameplay.

Take some lessons from this short quiz. How many times did you feel compelled to play?

Data Change Everything

 During the past several months there have been low background rumblings in the land of education and training. That is the sound of the learning world discovering what Internet professionals working in other vertical markets have known for years: The digital “breadcrumbs” that learners leave behind about their viewing, reading, engagement and assessment behaviors, interests and preferences provide massive amounts of data that can be mined to better personalize online experiences. 

via elearningroadtrip.typepad.com

I have always thought that "adaptive learning design" would finally give us the holy grail of "contextual relevance" for our learners. This post illuminates what to consider when analyzing learner behavior through their data trails. Combine that with observing learner behavior and you can potentially get to that holy grail of relevance.

The Potential of Augmented Reality for Education

This month I will be participating in a panel discussion about the Potential of Augmented Reality (AR) for Education at SXSW in Austin. I'm lucky to be joined on the panel with some leading experts in the field of AR: Tish Shute, Brendan Scully, Karen Hamilton and our fellow blogger here, Enzo Silva.

Tish Shute, founder of Ugotrade, a leading AR blog, explains that AR is in a critical development stage:

"Augmented Reality is in its infancy and we are just beginning to see AR learning apps emerge for young children on consumer devices, and museums are pioneering AR in situated educational experiences. AR has a long history in military and industrial training and simulation excercises because in these areas bulky high priced equipment is not an obstacle to adoption. We are in the middle of a data revolution. Soon just about everything in our world will have a data shadow. Sensors everywhere will allow us to communicate with the world around us in totally new ways. AR is the natural interface to bring relevancy to data. It will turn arcane spreadsheets, vast unstructured data sets - the data wakes of our cyborg lives - into visible, visceral, interactive, optimizable, and actionable parts of everyday life." 

Others on the panel include Brendan Scully from metaio, Karen Hamilton from George Brown University in Toronto, and Enzo Silva from Oracle. Enzo organized the panel, and will be moderating.

We're hoping to have a dynamic and interactive discussion about where AR is currently, where it's going, and how organizations can begin to leverage the technology to build more authentic and immersive educational experiences.

We hope you can join us in Austin! Until then, check out this cool video from metaio on multichannel AR: