Social Security Administration Call Order 0001: Job Analysis

Social Security Administration Call Order 0001: Job Analysis

Social Security Administration Call Order 0001: Job Analysis Methodology Briefing of Final Project Results OIDAP Quarterly Meeting September 21, 2011 Lance Anderson, Ph.D. Brian Cronin, Ph.D. icfi.com | 1 Agenda Introductions/Overview of Team SSA ICF Subcontractors Introduction to Project and Purpose

Project Methodology Recommendations Job Analysis Procedures Job Analysis Models Summary Overarching Recommendations Potential Next Steps Questions? icfi.com | 2 Introductions icfi.com | 3

Overview of Team SSA Debra Tidwell-Peters David Blitz Michael Dunn Elizabeth Kennedy Mark Trapani

ICF International Brian Cronin, Ph.D. Lance Anderson, Ph.D. Beth Heinen, Ph.D. Jessica Jenkins, MPhil Allison Cook, M.S. Daniel Fien-Helfman icfi.com | Subcontractors Paul Davis, Ph.D. Kelly Day, OTD Len Matheson, Ph.D.

4 Introduction to Project and Purpose icfi.com | 5 Introduction to Call Order 0001 SSA is developing new occupational information system (OIS) tailored specifically to SSAs disability programs and adjudication process. OIS will replace the Dictionary of Occupational Titles (DOT) and companion volumes, including the SCO and RHAJ. To develop OIS, SSA needs detailed methodology and strategy for analysts to perform job analysis throughout U.S. labor market. Purpose of Call 0001 was to perform the research needed to identify useful features of existing practices to support

development of SSAs job analysis methodology. icfi.com | 6 Project Methodology icfi.com | 7 Project Methodology icfi.com | 8 Call Order 0001 Terminology

Overarching Goal: Summarize wealth of information on job analytic procedures across disciplines to allow SSA decision makers to quickly understand and evaluate various job analysis practices. To ensure consistency, we use the following nomenclature: Project Method(ology) Refers to steps taken to conduct research for this call order Job Analysis Method(ology) Refers to ultimate SSA job data collection process developed to address OIS needs Practice Refers to all job analysis approaches, models and procedures, identified through this call order Model Refers to an established, off-the-shelf job analysis approach identified through this call order Procedure Refers to a data collection technique identified through this call order icfi.com | 9

Recommendations on Job Analysis Procedures icfi.com | 10 Procedure: Review of Written Materials Recommendation: Effective and cost-efficient starting point for almost all job analysis. Should be included in SSAs ultimate data collection methodology. Potential Usage for SSA: Analysts are able to become more familiar with the target job.

Potential Challenges: Dependent upon the usefulness and availability of source documents. Should only be used in conjunction with other data collection procedures. Outcome: Knowledge gained can be used to inform development of subsequent data collection techniques (e.g., interviewing). icfi.com | 11 Procedure: Job Observation Recommendation: Use for jobs that include more manual, less cognitive tasks. Should be included in SSAs final data collection methodology.

Potential Usage for SSA: Useful for collecting detailed information about job tasks, equipment/materials used, and work environment; does not rely solely on testimony of incumbents. Potential Challenges: Can be costly and time-consuming; may not be appropriate for all jobs (e.g., highly cognitive jobs). Outcome: Should be used in combination with other data collection procedures to determine prevalence of work activities observed. icfi.com | 12

Procedure: Survey Recommendation: Use when gathering data from geographically dispersed incumbents. Should be further considered for inclusion in SSAs ultimate methodology. Potential Usage for SSA: Can be effective and efficient means of collecting data from numerous incumbents; useful for assessing prevalence of work activities/; can make comparisons across jobs. Potential Challenges: May be costly to develop and administer; Some threats to validity, such as inflated ratings, lack of understanding rating elements, lack of effort in completing survey. Outcome: Other data collection procedures should be used to supplement the data collected especially when more detail is needed. icfi.com | 13 Procedure: Structured Interviews Recommendation: Use when job is more complex and needs additional

clarification or when comparison is needed (as long as highly structured). Should be incorporated into SSAs ultimate data collection methodology. Potential Usage for SSA: Allow analysts to collect detailed job information through the direct questioning of incumbents. Potential Challenges: Can be time-consuming and costly to prepare protocol, coordinate participants schedules, and arrange travel (for faceto-face interviews). Outcome: Analyst should undergo training on interviewing skills; analysts should conduct multiple interviews to gain full value of this procedure. icfi.com | 14 Procedure: Focus Groups Recommendation: May be more time-efficient and cost-efficient than interviews, but we recommend that interviews be used instead of focus groups whenever possible to allow for clarification questioning. Potential Usage for SSA: Allows for discussion with multiple incumbents at one time; useful for idea generation or other data collection that depends on interaction to elicit information.

Potential Challenges: Group dynamics can influence participant responses; schedule coordination can be difficult; specialized training needed for job analysts to be skilled facilitators. Outcome: Participants should be provided with an agenda, ground rules, and background information prior to the session. icfi.com | 15 Procedure: Physical Demands Measures Recommendation: May be valuable for jobs requiring manual labor given the disability determinations SSA must make. Depending on SSAs ultimate construct model and data collection instrument, should include instrument measurement of physical demands (i.e., tools that measure PDs) but with limited use. Potential Usage for SSA: Advantages include precise nature of the collected data, high reliability, high validity, and data that are typically easy to aggregate. Potential Challenges: Can be resource intensive to administer and require

specialized training; can be intrusive to incumbents. Outcome: Must have technical expertise in the use of specific measurement tools selected. icfi.com | 16 Recommendations on Job Analysis Models icfi.com | 17 Model: AET The AET involves conducting an observation and interview to complete an ergonomic questionnaire. Examples of Effective Features

Example Limitations for SSA Context Use of descriptors that isolate specific types of physical effort Significant time and labor commitment to develop career ladders Use of scales that focus on frequency, duration, and significance Long term commitment to upkeep of career ladders Use of examples to assist in coding level of demand icfi.com |

18 Model: Common Metric Questionnaire (CMQ) The CMQ collects data via a survey administered directly to incumbents and/or their immediate supervisors. Examples of Effective Features Example Limitations for SSA Context Matrix structure of the questionnaire Does not include a comprehensive set of descriptors at a broad level Computerized interface to allow for effective use of the questionnaire Use of behavioral and observable descriptors that are easy for incumbents and supervisors to rate

icfi.com | Some respondents might not have the access to a computer to use the computerized interface CMQ-like items on the OIS would need to be continually updated 19 Model: Cognitive Task Analysis (CTA) The CTA approach involves using a variety of data collection procedures to ultimately identify the cognitive processes underlying a job with a particular focus on the processes that distinguish an expert from a novice. Examples of Effective Features Example Limitations for SSA Context

Use of structured observation and Tends to lack the detailed interviews together with completion of information needed by SSA on a structured tool by the analyst various physical abilities Identification of the various types of knowledge needed to do the job icfi.com | Would require extensive training of analysts 20 Model: Fleishman Job Ability Requirement Scales (F-JAS) The primary data collection procedure for the F-JAS model involves the administration of the Ability Requirements Scales to collect data on 52 types of

abilities. Examples of Effective Features Example Limitations for SSA Context Generally well supported by research Lacks some generalizable physical abilities constructs that are important to SSA Use of level scales anchored with observable behaviors icfi.com | Tends to provide details on the variation of jobs at the high end of

many abilities, whereas SSA might be more interested in information about the variation in the jobs at the low end of abilities 21 Model: Functional Job Analysis (FJA) The FJA approach gathers a variety of different types of job analysis data typically collected via interview and observation but may also include other data collection procedures. Examples of Effective Features Example Limitations for SSA Context Inclusion of work context and worker environment variables Lacks standardization on important issues such as how jobs are

sampled, how interviews are conducted, and how many interviews are conducted Use of procedures that can be easily trained Structured framework and structured protocols build validity icfi.com | The DOT scales lack detail on cognitive abilities and interpersonal skills 22 Model: Job Element Model (JEM) JEM focuses on the human attributes required for superior performance on the job and collects data via focus groups, interviews, and surveys.

Examples of Effective Features Example Limitations for SSA Context Approach to conducting interviews and observations to gather job specific information provides data for understanding the job Experts previously rated this model low in terms of reliability and standardization Although a low cost approach, it involves a significant amount of time to administer icfi.com | 23

Model: O*NET O*NET was developed using a job analysis approach that focuses primarily on surveys, with supplementary use of interviews and reviews of written material. Examples of Effective Features Example Limitations for SSA Context The hierarchical arrangement and use of the content domain Some items will likely have low reliability relative to other instruments because constructs are not observable A nationwide database supported and maintained by an external entity

Tends to focus on differentiating high-tech jobs as opposed to low skill jobs that are usually the focus of disability claims icfi.com | 24 Model: Position Analysis Questionnaire (PAQ) The PAQ is an existing job analysis model that uses a standardized 195-item instrument to collect data, which is typically completed by a job analyst based upon data collected in job observations and interviews. Examples of Effective Features Example Limitations for SSA Context Use of an instrument that focuses on Constructs at a level of abstraction generalizable work activities provides that may not provide a clear picture

data for cross-job comparison of the job. Focus on observable behaviors ensures greater verifiability of the findings icfi.com | 25 Model: Task Inventory The Task Inventory approach involves collecting data through procedures such as review of written materials, job observation, interviews, and surveys to ultimately develop a list of task descriptions. Examples of Effective Features Example Limitations for SSA Context Method of conducting interviews and

observations to gather job specific information provides data for understanding the job No generalizable constructs or scales to allow for cross-job comparison icfi.com | 26 Model: Threshold Traits Analysis (TTA) The TTA collects worker trait, job demand, and job function data using data collection procedures such as review of written materials, job observations, interviews, and surveys. Examples of Effective Features Example Limitations for SSA Context

The 33 traits include a parsimonious and simply worded set of constructs that might provide an effective perspective for sorting and locating jobs SSA would need more detail than is provided via the 33 trait focused scales In general, the TTA is well supported by research icfi.com | In our judgment, the TTA rating tool is not appropriate for use as incumbent/supervisor survey. It should only be used by trained

analysts 27 Summary: Overarching Recommendations and Potential Next Steps icfi.com | 28 Recommendations and Potential Next Steps Identify Work Taxonomy and Constructs to be Measured Measuring different constructs may necessitate different data collection procedures Data Should be Collected and Stored Using a Computerized

System or Online Application/Tool Provides a centralized location for data collection and minimize potential security issues/concerns Need to Determine Factors that are Most Important and Consider Job Analysis Practices Accordingly Fully conceptualize multiple prototypes of integrated systems that SSA might use and compare the systems side-by-side Integrated system= Occupational analysis system icfi.com | 29 Comparing Prototype Integrated Systems

(i.e., Occupational Analysis Systems) Side-by-Side icfi.com | 30 Recommendations and Potential Next Steps Full Methodology Must Include a Comprehensive Set of Procedures that Include Guidelines for Maintaining Data Security and Confidentiality Features of Existing Job Analysis Models Should be Adapted for SSAs Specific Purposes and Data Needs Additional job analysis questions or more precise questions may need to be added to an existing questionnaire

Need Pilot Testing to Ensure that the Final Methodology Meets SSA Objectives Ensures the final set of combined data collection procedures and/or models appropriately measure the desired characteristics icfi.com | 31 Questions? For more information, please contact: Dr. Lance Anderson Vice President ICF International

(703) 934-3000 [email protected] icfi.com | Dr. Brian Cronin Senior Manager ICF International (512) 388-3389 [email protected] 32

Recently Viewed Presentations

  • ВВЕДЕНИЕ. ЧТО ТАКОЕ МЕДИАСОЦИОЛОГИЯ?

    ВВЕДЕНИЕ. ЧТО ТАКОЕ МЕДИАСОЦИОЛОГИЯ?

    Contemporary results of the Russian Internet audience research .RU - 4,8 mln .РФ - 0,8 mln .SU - 0,1 mln At September 7, 2013 in Russia there were around 5,7 mln registered second-level domains: A few figures on Russian Internet...
  • Speech Dynamics The Main Idea: At an abstract

    Speech Dynamics The Main Idea: At an abstract

    The phonetic structure of speech is an example of a discrete combinatorial system: morphemes and words are built by combining separate, independent, snap-together parts called phonemes or phonetic segments.. DNA works this way as well: Genes are built by combining...
  • Athletics and genetics Personal Genetics Education Project (pgEd)

    Athletics and genetics Personal Genetics Education Project (pgEd)

    More about Michael Phelps can be found in this Scientific American article, "What Makes Michael Phelps So Good?" by Adam Hadhazy. Olympic champion Usain Bolt is one of the greatest sprinters of all time. He hails from Jamaica, a small...
  • I am hungry as a horse. You run

    I am hungry as a horse. You run

    If you're a pretender, come sit by my fire, For we have some flax golden tales to spin. Come in! Rhyme Scheme Letters Rhyme schemes are often identified with letter patterns Pairs or rhyming word families are assigned the same...
  • Developing a Suicide Deep Dive Data Collection Tool

    Developing a Suicide Deep Dive Data Collection Tool

    Help desk personnel - 5. External data collection programs, program support (e.g. eDHA, DRSi, CarePoint) Management Assistants - 1. ... PRIMS. Accession . Medical Encounter includes Purchased Care. TAPS - Families have to opt in to share their information. We...
  • Forming Sector Partnerships One Door Many Paths June

    Forming Sector Partnerships One Door Many Paths June

    Presenters. Beth A. Brinly. Vice President, Workforce Innovation. Maher & Maher. ETHICS DISCLAIMER "This presentation reflects our (personal) views of the changes implied by WIOA and the actions a proactive workforce system professional may want to consider in preparing for...
  • Getting Ahead in a Just Getting' by World - AhaProcess® SSVdP ...

    Getting Ahead in a Just Getting' by World - AhaProcess® SSVdP ...

    Getting Ahead in a Just-Getting'-by- World Building your resources for a better life!Aha ProceSs®SSVdP: EMPOWERING THOSE WE SERVEPresentation for United WayJune 14, 2013Green Bay DioceseMarinette DistrictSt Joseph ConferenceGoodwill Industries of Northern WI and UP of MI. introduce our team -...
  • Organic Reactions - Marvelous Ms. M's Science Page

    Organic Reactions - Marvelous Ms. M's Science Page

    The 120° trigonal planar structure of Carbon in C=C makes it available for attack bonds are weaker & not on plane of bond, so easier to break bond is area of e- density so electrophiles are attracted. Electrophile: species that...