Aspects of Expertise and Knowledge Audit Probe
기존의 knowledge audit의 약점은, 각 probe들이 서로 연관지어지지 않았다는 것. 그들을 통합하는 큰 틀이 없었다.
What Experts Know
Perceptual skills/noticing
Mental models/big picture
- Sense of typicality and associations
Routines/job smarts
- (Declarative knowledge)
What Experts Can Do
Run mental simulations/past and future
- To diagnose
- To explain
- To form expectancies
Spot anomalies and detect problems/anomalies
Find leverage points/opportunities/improvising
- Perform workarounds
- Manage uncertainty
- Plan and replan
- Access complex situations
- Manage attention
Take their strengths and limitations into account/self-monitoring
Knowledge Audit Probes: Current version
과거 & 미래 (Past & Future)
- 진단과 예측, 멘탈 시뮬레이션
- 상황의 과거와 미래 인지 (ex. 안봐도 비디오다)
상황을 접할 때, 전문가들은 그 상황이 어떻게 진전될지 그릴 수 있는 능력이 있다. 그리고 미래로 가서 상황이 어디로 흘러가는지 생각할 수 있다. 이 능력은 전문가가 상황이 진전되기 전에 문제를 제거할 수 있게 한다. (de Groot 1946/1978; Endsley, 1995; Klein & Crandall, 1995; Klein & Hoffman, 1993)
- 예전에 어떤 문제상황 속으로 들어가서 무슨 일이 벌어졌고 어떻게 진행될지 정확히 알았던 적이 있나요?
큰 그림 (Big Picture)
- 상황 인식(Situation Awareness)
- 큰 그림 보기
- 초보자는 상황의 부분과 단편만 볼 수 있다. 전문가는 전체 상황에 대한 이해, "큰 그림"을 빠르게 구축할 수 있는 능력이 있다. 이 능력은 전문가가 서로 다른 요소들이 어떻게 서로 들어맞고 서로 영향을 주는지 생각할 수 있게 한다. (Endsley, 1995; Klein, 1997)
- 이 작업에서 큰 그림이라는 측면에서 무엇이 중요한지 예를 들어 설명해주실 수 있나요? 당신이 이 일을 할 때 반드시 알아야 하고 계속 지켜봐야 하는 중요한 요소가 무엇인가요?
알아채기 (Noticing)
- 인식 능력
- 남들은 못봤는데 내가 알아챈.
Experts are able to detect cues and see meaningful patterns that less-experienced personnel may miss altogether (de Groot, 1946/1978; Klein & Hoffman, 1993; Shanteau, 1985)
Job Smarts
- (Heuristics, Tricks of the Trade, Contextual Practices)
- 업무 요령. 노력을 덜 들이면서 큰 효과를 보는 방법?
Experts learn how to combine procedures and work the task in the most efficient way possible. They do not cut corners, but they do not waste time and resources either (Klein & Hoffman, 1993)
- When you do this task, are there ways of working smart or accomplishing more with less -- that you have found especially useful?
Opportunities / Improvising
- 더 잘 할 수 있는 기회 어떻게 포착했었는지?
Experts are comportable improvising -- seeing what will work in this particular situation. They are also able to shift directions to take advantage of opportunities (Dreyfus & Dreyfus, 1986; Shanteau, 1985)
- Can you think of an example when you have improvised in this task or noticed an opportunity to do something better?
Self-monitoring
- (Metacognition)
- 하던걸 변경해야 한다고 깨달았던
Experts are aware of their performance, they check how they are doing and make adjustments. Experts notice when their performance is not what it should be (due to stress or fatigue, for example), and they are able to adjust to get the job dong (Cohen, Freeman, & Wolf, 1996; Glaser & Chi, 1988)
- Can you think of a time when you realized that you would need to change the way you were performing in order to get the job done?
Anomalies
- (Perceptual Skills, Situation Awareness)
- 평소와 뭔가 다른. 잘못된 상황 인지. 직전에 예측
Novices don't know what is typical so they have a hard time identifying the unusual or atypical. Experts can quickly spot unusual events and detect deviations, and they are able to notice when something that should happen does not happen (Chi, Hutchinson, & Robin, 1988; Klein, 1989; Klein, 1997; Klein & Hoffman, 1993)
- Can you describe an instance when you spotted a deviation from the norm, or knew something was amiss?
Equipment difficulties
- 통상적 데이터와 본인 인지 달랐던 경우
Experts know that their equipment can sometimes mislead. Novices usually believe whatever the equipment tells them, they don't know to be skeptical (Cannon-Bowers, Salas, & Converse, 1992)
- Have there been times when the equipment pointed in one direction, but your own judgement told you to do something else? Or when you had to rely on experience to avoid led astray by the equipment?
Gary Klein, Laura Miltello, Chap 23. The Knowledge Audit as a Method for Cognitive Task Analysis, How Professionals Make Decisions, p335