 When the Employment and Training Administration’s CareerOneStop team set out to redesign portions of the site (namely, career, training, and job resources), they didn’t immediately begin rewriting code. Instead, they embraced a user-centered approach that focused on the user experience (UX). In a general sense, focusing on UX means taking a step back to learn about users’ core needs and preferences before making changes to a product or service. Before the CareerOneStop team completed their site redesign, they asked users the following questions: “Who’s using CareerOneStop resources?” This was the first question the team asked the people they interviewed. The answer? Just about everybody. CareerOneStop users include job seekers, businesses, students, current workers, laid-off workers, veterans, workers with disabilities, workers with criminal records, career counselors and other workforce professionals, and just about every other member of the public. Researchers were pleased to learn that such diverse audiences use CareerOneStop, and they talked to a varied group of users to answer the next questions. “Can users find what they need at CareerOneStop?” CareerOneStop offers such a large volume of information that some users weren’t able to quickly find the best resources for their unique needs. The research team talked to users and watched them use the system to find out what each type of user needs most (and most quickly), where they expect to find it, and what language is most meaningful to them. The researchers’ goal was to identify the clearest labels and determine the best way to organize the site’s information to help users connect with the most relevant information. CareerOneStop now offers streamlined access to targeted resources for each audience. “Are CareerOneStop’s tools easy to use?” CareerOneStop’s resources can’t only be easy to find—they also need to be easy-to-use and effective at helping users meet their career, training, and employment goals. The CareerOneStop team conducted usability testing on key tools and websites, during which they watched users interact with the site and learned how to improve functionality, organization, and language in order to better meet users’ needs. “How are users accessing CareerOneStop resources?” While some people are smartphone-wired 24/7 (one recent survey found that 83 percent of people use smartphones or tablets to job search), others may lack dependable Internet service on a daily basis. CareerOneStop’s goal is to make its resources valuable for all users. That’s why both the redesigned CareerOneStop.org site and the newly launched Credentials Center are mobile-friendly—that is, they automatically adjust to a user’s smartphone, tablet, or desktop screen, providing on-the-go employment, training, and job search help. Six key CareerOneStop tools—including Job Search, Training Finder, and Salary Finder—are also available as mobile web apps. While making its resources accessible to mobile-equipped users, CareerOneStop didn’t want to leave behind those with limited Internet access or low computer literacy skills. For those who may access the Web at a public library or American Job Center, CareerOneStop provides printable guides, along with the ability to easily download and print key information and tool results. And for those who are less comfortable with technology, printed and video help materials provide step-by-step guidance through many tools.  What’s next for CareerOneStop? The recently redesigned CareerOneStop and the new Credentials Center offer a wealth of assistance to anyone with career, training, or employment needs. Read more about key features in the press release or watch What can CareerOneStop do for you? But user-centered development doesn’t end with the launch of new products. The CareerOneStop team will continue collecting and learning from user feedback to continually improve its resources. Send us your feedback at info@CareerOneStop.org. This is the first draft of this Department of Labor blog post
0 Comments
These are exciting times for tech in the Federal government. DJ Patil has joined the administration as the first ever US Chief Data Scientist, Megan Smith looking for more technologists to join the United States Digital Services, and the Presidential Innovation Fellowship program is recruiting for the next round of Fellows. The excitement is both top-down and bottom-up. There are exciting developments on the ground - percolating in the agencies. At a roundtable hosted at the John Kennedy School of Government in Boston, Massachusetts, the participants spent over three hours discussing the current state of the skills economy, its current shortcomings and the actions required to make progress in the field. The job postings of businesses, which want employees, identify the demand in the market. The resumes of job seekers represent the supply side, and the educational institutions that provide training facilitate mechanisms to meet the long-term system needs. Unfortunately, there has been at least two decades of activity in the workforce/skills field that has yielded a fragmented mesh of uninteroperable and disconnected systems and portals. To ensure that the American worker has a fighting chance in this (and the upcoming) century, there has to be several fundamental steps that must be taken; and be used as the bedrock of the industry.  The first step is defining a skill. The individual components of a skill or competency need to be defined and agreed upon.  The second step is articulating (and growing) the set of skill terms. This would enable a common language and reduce the possibility of a 'Tower of Babel' situation.  The final step is the creation of a platform that demonstrates the value of an open API (Application Programming Interface) over a set of skills data that covers all aspects of the skills triangle. This platform is a public-private partnership that leverages Federal and business stakeholders.
** Note: The opinions expressed within this post represent only the views of the author and not any person or organization As a Presidential Innovation Fellow, I’ve had the pleasure of working with some of the most brilliant, forward-thinking minds in the workforce. During the past few months, my work has focused more specifically on skills and training arenas,and one thing that struck me is that the skills ecosystem is the entire economy. Its success will lead to a thriving and solid economic base. It’s one thing to make this realization; translating this realization into action is quite another. The problem of understanding the skills space and iterating on it to produce the system that its users‚ i.e. job seekers, employers, colleges, workforce investment boards, etc. —need is a very important one. Related, it’s one that will require the participation of many more people. Before we delve too deeply into this issue, let’s cover a few basics. The de facto source of skills information, both inside and outside the Federal government, is O*NET - the Occupational Information Network. What Is O*NET? O*NET is a data collection program that populates and maintains a current database of the detailed characteristics of workers, occupations, and skills. Currently, O*NET contains information on 974 detailed occupations. This information is gathered from a sample of national surveys of businesses and workers. For each of these 974 occupations, O*NET collects data on 250 occupational descriptors. As you might guess, O*NET is intended to be a definitive source for persistent occupation data — in other words, data on stable occupations that have existed (and will continue to exist) in the medium term. Examples of some such occupations include firefighter, teacher, and lawyer, to name a few.  Fig. 1: O*NET Content Model O*NET supersedes the U.S. Department of Labor’s (DOL’s) Dictionary of Occupational Titles (DOT) and provides additional occupational requirements not available in the DOT. The DOT is no longer supported by DOL. O*NET uses an occupational taxonomy, the O*NET-SOC, which is based on the 2010 version of the Standard Occupational Classification (SOC) mandated by Office of Management and Budget (OMB) for use by all federal agencies collecting occupational and labor market information (LMI). What Is O*NET Good At? O*NET is the bedrock of occupational data — some users think of it as a one-stop shop for data on occupations that have existed (and will exist) for some time. In a phrase, it is the gold standard of data on occupational skills – data upon which laws are createdand an economic ecosystem can be built. How Often Does O*NET Data Get Updated? The O*NET data set gets updated once a year. Currently, this is the fastest that comprehensive, consistent, representative, user-centered data, which complies with OMB’s Information Quality Guidelines and data collection requirements, can be gathered and processed. How Does O*NET Collect Its Data Anyway? Data collection operations are conducted by RTI International at its Operations Center in Raleigh, North Carolina, and at its Survey Support Department, also located in Raleigh. O*NET uses a 2-stage sample — first businesses in industries that employ the type of worker are sampled and contacted — then when it is confirmed that they employ those workers and will participate — a random sample of their workers in the occupation receive the O*NET survey form and respond directly. When necessary, this method may be supplemented with a sample selected from additional sources, such as professional and trade association membership lists, resulting in a dual-frame approach. An alternative method, based on sampling from lists of identified occupation experts, is used for occupations for which the primary method is inefficient. This method is reserved for selected occupations, such as those with small employment scattered among many industries and those for which no employment data currently exist on which to base a sample, such as new and emerging occupations. At the current funding level $6.2 million for PY 14 the O*NET grantee updates slightly more than 100 occupations per year with new survey data. Why Use Sampling And Surveys? O*NET currently uses sampling and surveys to gather its massive amounts of data because these are the best techniques that also comply with the OMB Information Quality Guidelines (IQG). These guidelines identify procedures for ensuring and maximizing the quality, objectivity, utility, and integrity of Federal information before that information is distributed. OMB defines objectivity as a measure of whether disseminated information is accurate, reliable, and unbiased. Additionally, this information has to be presented in an accurate, complete manner, and it’s subject to quality control or other review measures. O*NET was designed specifically to address OMB IQG, including OMB information collection request standards, and to correct some fairly serious limitations in the Dictionary of Occupational Titles. What Are Other Foundational Principles of O*NET? In addition to serving as a comprehensive database of occupational data and promoting the integrity of said data, O*NET upholds several other principles. These include the following:
Using survey-based data enables O*NET to have comprehensive coverage, be very representative, enable the data to be validated and cleaned relatively easily, enable the statistical calculation of margin of variance or error, and to meet the OMB data collection requirements. Even though this type of data tends to be higher cost than other data types, survey-based data results in average or mean values (by design). What About Other Types of Data? There are two other primary types of data that could be included in the O*NET data set — transactional data and crowdsourced data. Transactional data tends to be privately owned and have a potential for response bias. Currently, occupational and industry coverage is limited, e.g. in certain industries, online job postings are the norm and in others, it is not. Also, the use of online job postings vary with the size of the firm — smaller employers are less likely to post job openings online. Finally, many postings don’t specify the information desired by O*NET curators. Crowdsourced data tends to the least representative of the set, has the greatest potential for response bias, can be very difficult to validate independently and can easily capture leading or emerging variations. Both transactional and crowdsourced data would require significant investments in curation and management for them to satisfy the OMB Information Quality Guidelines and the O*NET Foundational Principles, which are intended to fix the problems with the DOT. What Is The Biggest Misconception About O*NET? “Why isn’t Python an O*NET Skill?”, “Why is Active Listening an O*NET Skill?”, “Why are O*NET Skills so broad and generic?” Members of the O*NET team hear these and similar questions with relative frequency. The above questions echo one primary misconception about O*NET,, which is O*NET Skills are too general. What many users don’t realize is that O*NET skills — for example, active learning and programming — refer to cross-cutting competencies fundamental to a given occupation. More specific descriptors (such as systems, platforms, and other job-specific tools) are captured under the Tools & Technology section of an occupational profile. O*NET Skills, as perceived by most of the public, are actually a combination of what we describe in the Tools & Technology, Detailed Work Activity, Abilities, Tasks, Skills and Knowledge sections of an O*NET occupational profile. In short, Skills are just the tip of the occupational iceberg. What Are The Issues That You Hear Voiced Most About O*NET? The current set of concerns with the O*NET program include:
Official datasets from the Federal government are constructed to be a stable, steady base upon which more transitional elements can exist. This stability is designed to ensure that the American people see a consistent face when dealing with their government. For this reason, jobs that appear and disappear within the space of months or a few years, like Chief Fun Officer or Growth Hacker, are not immediately included in the O*NET database. Only occupations that have stood the test of time should be included in an authoritative source; upon which regulatory decisions are made. How Do We Solve These Issues? A public-private partnership that produces a Skills Market Platform (SMP) (Fig. 2), which is an open, dynamic, growing, standards-based, public-facing platform for skills data.  Fig. 2: Reference Architecture for a Skills Market Platform (SMP) The SMP should be run by a nonprofit that will handle the data curation, quality control, analytics and the public application programming interface (API). It is assumed that all contributions to the SMP will be tagged with skills from the Skills Data Standard (SDS) – a standard that is currently being developed. Employers can provide tagged job descriptions. Job seekers can tag their resumes using the SDS taxonomy or import their skills using OpenBadges (or similar) framework. Colleges can tag their courses with the imbued skills. Certification bodies can tag their programs with the delivered skills. The core idea behind the SMP is to conceptualize, design, and build a national skills platform — which would be populated with data over time. Federal government information and existing taxonomies would provide the foundational structure and bring data from O*NET, from the Bureau of Labor Statistics, Commerce data from the Census, the American Community Survey, and the Longitudinal Employer-Household Dynamics data sets. Other partners would be able to upload additional data—such as a college course catalog, or the competencies imparted by a specific certification—or data mined from tagged job postings or resumes. Local areas where community colleges and the workforce system are already collaborating closely with business — such that curriculum is closely tied to employer skills demand could upload information on the linkages between employer skill demand and specific competency-based education modules. Such early adopters would be able to demonstrate the utility of such a platform and build a community to further build out and participate in the SMP. The ultimate goal is for the Federal government to facilitate the creation of the innovation platform for skills; enabling any American to freely and openly leverage a common language and knowledge base. What Needs To Be Done The Skills Market Platform relies on an existing:
Ever wondered what skills a particular certification course gives you? Ever debated the true skills details of a job description? Are you passionate about skills, competencies, abilities and knowledge? Contribute to this effort. Spend some time helping us create the Skills Data Standard. Donate some programming and design cycles to implementing the Skills Network Protocol and or the Skills Innovation Layer. Help change the world as we know it.  The skills market encompasses the entire economy. However, I feel like there are so many people who are not aware of the Federal sources in the skills arena. So, this blog will highlight the primary source from the Federal government, O*NET - The Occupation Information Network.  Fig. 1: O*NET Content Model What Is O*NET? O*NET is a (data collection) program that populates and maintains a current database of the detailed characteristics of workers, occupations, and skills. Currently, O*NET contains information on 974 detailed occupations, which are gathered from a sample of national surveys of businesses and workers. For each occupation, O*NET collects data on 250 occupational descriptors. O*NET is intended to be a definitive source for persistent occupation data, i.e. stable occupations that have been (and will continue to be) in existence over the medium term. O*NET superseded the U.S. Department of Labor’s (DOL’s) Dictionary of Occupational Titles (DOT) and provides additional occupational requirements not available in the DOT. The DOT is no longer supported by DOL. O*NET uses an occupational taxonomy, the O*NET-SOC, which is based on the 2010 version of the Standard Occupational Classification (SOC) mandated by Office of Management and Budget (OMB) for use by all federal agencies collecting occupational and labor market information (LMI). O*NET is the bedrock of occupational data. It offers a stable foundation that contains information on jobs and skills that are not fleeting. In a phrase, it is the gold standard of data on occupational skills. Why Use Sampling and Surveys? The primary reason O*NET currently uses sampling and surveys is that these are the best techniques for maximizing the quality, objectivity, utility, and integrity of information prior to dissemination. O*NET uses the OMB Information Quality Guidelines for all Federal statistical information, which defines objectivity as a measure of whether disseminated information is: 1. Accurate 2. Reliable 3. Unbiased 4. Presented/disseminated in an accurate, clear, complete, and unbiased manner 5. Subject to extensive review and or quality control What Are Other Foundational Principles of O*NET? O*NET is designed to be:
Why Does O*NET Only Contain Survey-based Data? Using survey-based data enables O*NET to have comprehensive coverage, be very representative, enable the data to be validated and cleaned relatively easily, enable the statistical calculation of margin of variance or error, and to meet the OMB data collection requirements. What About Other Types of Data? There are two other primary types of data that could be included in the O*NET data set — transactional data and crowdsourced data. Transactional data tends to be privately owned and have a potential for response bias. Currently, occupational and industry coverage is limited, e.g. in certain industries, online job postings are the norm and in others, it is not. Also, the use of online job postings vary with the size of the firm — smaller employers are less likely to post job openings online. Finally, many postings don’t specify the information desired by O*NET curators. Crowdsourced data tends to the least representative of the set, has the greatest potential for response bias, can be very difficult to validate independently and can easily capture leading or emerging variations. Both transactional and crowdsourced data would require significant investments in curation and management for them to satisfy the OMB Information Quality Guidelines and the O*NET Foundational Principles, which are intended to fix the problems with the DOT. How Is O*NET Used? O*NET is the common language and framework that facilitates communication about industry skill needs among business, education, and the workforce investment system. The O*NET data is used to:
The O*NET database and companion O*NET Career Exploration Tools are also used by many private companies and public organizations to tailor applications to their needs and those of their customers. How Do I Access O*NET Data? The O*NET database is provided free of charge to the public through O*NET OnLine. O*NET Data is also available through the My Next Move, and My Next Move for Veterans websites; through the O*NET Web Services application programming interface (API); or by downloading the database, particularly by developers who provide applications targeted to specific communities or audiences. How Does O*NET Collect Its Data? The O*NET data set gets updated once a year. O*NET uses a 2-stage sample — first businesses in industries that employ the type of worker are sampled and contacted — then when it is confirmed that they employ those workers and will participate — a random sample of their workers in the occupation receive the O*NET survey form and respond directly. When necessary, this method may be supplemented with a sample selected from additional sources, such as professional and trade association membership lists, resulting in a dual-frame approach. An alternative method, based on sampling from lists of identified occupation experts, is used for occupations for which the primary method is inefficient. This method is reserved for selected occupations, such as those with small employment scattered among many industries and those for which no employment data currently exist on which to base a sample, such as new and emerging occupations. At its current funding level, O*NET updates slightly more than 100 occupations per year with new survey data. What Are The Survey Response Rates? Currently, more than 144,000 establishments and 182,000 employees have responded to the survey request, resulting in an establishment response rate of 76% and an employee response rate of 65%, which compare favorably to other establishment surveys. O*NET Usage Statistics Current O*NET Usage Statistics During 2014, the O*NET websites (O*NET OnLine, My Next Move, My Next Move for Veterans, Mi Proximo Paso) averaged over 4 million visits per month. Fig. 2 shows the total site visits by year from 2002 to 2013.  Fig. 2: Total Site Visits for O*NET web properties There is some pattern of seasonality that appears to follow the school calendar. Peak months with over 5 million visits in 2014 were: April, September, and October. The O*NET database has been downloaded 83,573 times from 2002 to 2014. Fig. 3 shows the database downloads in each year.  Fig. 3: O*NET Database Downloads, 2002 - 2014 O*NET Web Services and APIs were introduced in 2012 and their usage is steadily increasing through 2014. At the close of 2014, there were 343 total registrants for O*NET Web Services. Fig. 4 shows the progress since 2012.  Fig. 4: O*NET Web Services Registrants, 2012 - 2014 The breakdown of the users of O*NET Web Services is detailed in Fig 5.  Fig. 5: Registrant Composition for O*NET Web Services Without the help of the O*NET team, this would not be possible. Thank you. A lot of the content comes from a factsheet that the team worked on.
|
Dr Tyrone Grandison
Executive. Technologist. Change Agent. Computer Scientist. Data Nerd. Privacy and Security Geek. Archives
May 2018
Categories |
RSS Feed
|