Diff for "LTEM"

LTEM: The Learning-Transfer Evaluation Model

Performance-Focused Learner Surveys: Using Distinctive Questioning to Get Actionable Data and Guide Learning Effectiveness

저자의 블로그 글에 모델이 소개되어 있음.

Tier 8. Effects of Transfer

Effects of Transfer: Including outcomes affecting (a) learners, (b) coworkers/family/friends, (c) organization, (d) community, (e) society, and (f) the environs. CERTIFYING EFFECTS OF TRANSFER REQUIRES: Certification of transfer plus a rigorous method of assessing transfer's causal impact -- including positive and negative effects.

Tier 7. Transfer

When learner uses what was learned to perform work tasks successfully -- as clearly demonstrated through objective measures.

Assisted Transfer: when performance is substantially prompted/supported. ADEQUATE TO CERTIFY ASSISTED TRANSFER.

Full Transfer: when learner demonstrates full agency in applying the learning. ADEQUATE TO CERTIFY FULL TRANSFER.

Tier 6. Task Competence

Learner performs relevant realistic actions and decision making.

Task Competence -- during or right after learning event. Not a fully adequate metric because learner may forget their task competencies.

Remembered Task Competence -- after several days or more. ADEQUATE TO CERTIFY TASK COMPETENCE.

Tier 5. Decision Making Competence

Learner makes decisions given relevant realistic scenarios.

Decision Making Competence -- during or right after learning event. Not a fully adequate metric because learners may forget decision making competencies

Remembered Decision Making Competence -- after several days or more ADQUATE TO CERTIFY DECISION MAKING COMPETENCE.

Tier 4. Knowledge

Learner answers questions about facts/terminology.

Knowledge Recitation -- during or right after learning event. A metric inadequate because knowing terminology does not fully enable performance.

Knowledge Retention -- after several days or more. A metric inadequate because remembering terminology does not fully enable performance.

Tier 3. Learner Perceptions

A. Learner is queried in a way that reveals insights related to learning effectiveness. Such measures can hit at outcomes but should be augmented with objective outcome measures.

B. Learner is queried in a way that does NOT reveal insights on learning effectiveness. A metric inadequate to validate learning success -- because such perceptions are not always related to learning results.

Tier 2. Activity

Learner engages in activities related to learning.

Measures of Attention A metric inadequate to validate learning success -- because learners may pay attention but not learn.

Measures of Interest A metric inadequate to validate learning success -- because learners may show interest but not learn.

Measures of Participation A metric inadequate to validate learning success -- because learners may participate but not learn.

Tier 1. Attendance

Learner signs up, starts, attends, or completes a learning experience. A metric inadequate to validate learning success -- because learners may attend but not learn.

Documents

• The Learning-Transfer Evaluation Model

• The Accompanying Report

The Accompanying Report

Model은 숨은 메세지를 담고 있다. 메세지를 보낸다. 그 메세지가 올바른 것이라면 좋은 모델이고, 메세지가 나쁜 것이라면 나쁜 모델이다.

기존의 KirkpatrickModel은 어떤 메세지를 보내는가?

좋은 부분

• learning에만 초점을 맞추면 안되고, on-the-job 행동을 만들어내고 조직에서의 결과(organizational results)를 만들어내야 한다고 말한다.
• 학습자의 의견(설문결과?)만이 충분하지 않으며, 나머지 3개 수준이 더 중요하다고 말한다.
• 네 가지 수준은 서로 연결되어 있다. 우리의 목표가 학습이 되어서는 안되고, 학습은 더 높은 결과(행동의 변화와 실제 결과물 만들기)를 위한 수단이어야 한다고 말한다.

하지만 문제가 되는 부분

• 학습자의 피드백이 학습 결과, 업무 능률, 조직에서의 성과와 연관되어 있다는 메세지를 주지만, 리서치에 따르면 그것은 참이 아니다. 전통적인 smile sheet는 learning results와 work performance와 연관성이 없다.
• lv3 (job performance), lv4 (org results)를 측정하는 것이 lv2 (learning results)를 측정하는 것보다 더 중요하다는 암시를 준다. 학습 전문가는 job performance나 organizational outcome보다는 learning result에서 더 많은 힌트를 얻는데 말이다.
• 게다가, lv3, lv4만을 측정 (lv2는 간과하거나 건너뛰고) 하게 될 수도 있다. learning에서 시작해서 job performance, org results까지 확장되어 나가는 causal pathway를 무시하게 만들 수도 있다. lv3, 4만을 측정해서 poor results를 얻었다 치자. 그렇더라도 learning design이 이 실패에 어떻게 기여했는지 알 수 있는게 없게 된다.
• training에 초점이 맞춰져 있는 모델이다. prompting mechanisms (job aids, performance support, etc)의 역할은 숨기고, on-the-job learning이 학습과 성과에 중요한 요인이라고 말한다. 측정된 결과를 가지고 뭔가 개선하려 할 때, prompting mechanism이나 on-the-job learning은 우리의 개선 대상에 포함되지 않기 쉽다.
• learning -- remembering -- performance to results에 이어지는 causal chain을 무시한다. remembering의 critical importance에 대해서는 단 한 마디의 언급도 없다. 또한, 이해를 돕고 망각을 줄이는 learning intervention도 중요하다. 우리는 long-term remembering을 지원하는 learning design을 사용하도록 만드는 모델이 필요하다.
• outcome에 무게를 두도록 만든다. good learning outcome에 핵심적인

CategoryLearning CategoryTeaching