Human Resources is one of the most complex, imperfect areas of business. Virtually every decision we make about people (who to hire, who to promote, how much to pay someone, how to develop someone) is based on judgment, experience, personal bias, and some amount of data. And since well over 50% of all corporate spending is on salaries (United States payroll is around $15 Trillion), these “judgmental decisions” cost companies a lot of money.
And in my world, where we deeply study every part of management, leadership, and HR, we often try to correlate various “HR practices” against outcomes to figure out what works. Much of our business is based on this work, and we regularly “re-run” most of our analysis every few years as culture, the labor market, and technology change.
Right now, for example, we know that workplace stress, pay equity, and career growth is among the most important drivers of employee satisfaction and workforce productivity. Only a few years ago it was all about fancy benefits, bonuses, and grandiose titles.
So what I’m essentially saying is that much of HR is based on organizational psychology, many forms of social science research, and never-ending effort to experiment, learn from others, and figure out what works. And it’s difficult, imperfect, and always subject to debate.
The Underlying Data Set In HR Is Textual
While this massive effort has been going on, most of the “hard science” in HR and management has been focused on numbers. We ask people to take tests, we look at people’s “performance ratings” and grade point averages (which are extremely subjective), and we ask people for surveys, feedback, and lots of data to make decisions. And then we correlate business results (sales, profit, market share) against various people metrics, and think “we have the answer.”
For recruiting and selection, we look at experience, job-related tests, and opinions and scores from interviewers. Theoretically, if we get enough of this data we can make better and better hiring decisions. And the precise same thing happens when we look at who to promote, who to demote, and who should make it to the very top ranks of the company.
The whole premise of promotion is based on old ideas of “promotability” or “potential” rated against “current job performance” (the 9-box grid). That approach, which sounds quantitative, is filled with bias, so we have to “infer” who has high potential from various assessments, observations, and inputs. Again, when we get lots of data (looking at the background and behaviors of many high performers), we can improve the science of promotion. But for the most part, this is based on judgment.
The core “science” of HR is often rooted in Industrial Psychology, which is a fascinating domain that studies attributes, behaviors, and psychology at work. And as much as I admire and follow much of this science, most companies don’t use it very much. There is a billion-dollar industry of “validated pre-hire assessments” and they are extremely useful. But for many jobs they are misleading and companies have to validate these tests so they don’t get sued for discrimination.
So if you want to really do a “big data” analysis of your workforce’s skills, experience, and suitability for different work, you’re dealing with mountains of “anecdotal data,” much of which is encoded in biographies, work output, company leadership frameworks, assessments, and lots of communications. And of course, there are performance appraisals, business results, and more.
Consider the two most common parts of HR: a job requisition (job posting) and a job description. Both these artifacts are “thrown together” by hiring managers or HR professionals, often based on what people think a job is like, a set of company standards, and some “technical skills” we know this person will use. As we all know, these artifacts don’t really predict who will succeed, because so much of “success” is based on ambition, learning agility, culture fit, and alignment with purpose.
In other words, this is one of the most complicated and fascinating “mixed data” problems in the world, and making decisions a few percent better can drive billions of dollars of business value.
