On the synecdochic fallacy, or why data science needs the liberal arts

My father was a lawyer who also had an MBA and an undergraduate degree in economics. He lamented one of the great paradoxes of legal education: he explained that he saw people who did spectacularly well in law school, who were the editors of the law review, who clerked for a Supreme Court justice, or maybe even got the top score on the Bar exam. But when they went to set up their legal practice, those same people could crash and burn on the most fundamental task: running their brand-new small business. Law schools – and most other professional schools – understandably focus on teaching the technical details of their respective practices, but often risk neglecting the day-to-day issues that make a small business effective.

Related problems show up in several other, unrelated fields:

• In automobile racing, starting positions are often determined by who drives the fastest alone on an empty racetrack, but the races are typically won by the drivers who are best at negotiating high speed traffic and driving strategically, among other factors.

• In dance, the ability to perform physically demanding moves like triple pirouettes or high kicks are impressive, but essentially unrelated to being a great artist on the stage.

• In family life, having a thorough understanding of the principles of psychology is not the same as being prepared for the challenges of raising actual children (as my wife and I, with our collective five degrees in psychology, can personally attest; we were as bewildered with and overwhelmed by our children as everybody else).

The synecdochic fallacy

These situations are examples of what can be called the "synecdochic fallacy," or the error that arises in mistaking part of something for the whole. Synecdoche is not inherently problematic; in fact, it's a common practice in communication, where, for example, "the crown" refers to a monarch and their government, "hired hands" refers to people employed as workers, and "the baton" refers to the conductor of an orchestra. But when people forget that the reference is only a small part of the whole, then the problematic fallacy arises.

In my earlier example, the technical skills of a lawyer are important, of course, but there are so many other, unrelated skills that go into running a successful practice, such as finding clients, organizing multiple projects, keeping track of billing, staffing and managing an office, and dealing with the inevitable repetition and even drudgery that can come with being a working professional. And strikingly similar skills are required to run a psychotherapy practice, a construction company, a radiology practice, a fleet of food trucks, a social media marketing firm, and even a dog-walking service.

In data work, the problem manifests itself when people focus too much on developing a wide and deep set of technical skills, and do so to the exclusion of the other skills that are critical to the success of a data project. Too often, in discussions of "how to be a data scientist," people offer lists of programming languages and applications ("You need to know Python, and R, and Julia, and SQL, and Perl, and C/C++, and Java, and Tableau, and SAS, and Hadoop, and Spark, and Hive, and Pig...") and data skills ("You need to know deep learning, and time series, and cloud computing, and DevOps, and ETL, and data structures and algorithms..."). In each case, people seem to mistake one element of a job – the technical skills – for the entirety of the responsibilities involved in that job.

Back to the real world

Doing effective data work requires more than knowing certain procedures. In fact, that may be only a small part of what success in the job requires. The required technical skills are important – they're necessary conditions for competent data work – but they are rarely enough for a successful project – they are not sufficient conditions.

Depending on exactly what your work involves, many of these skills may be important to your day-to-day tasks. Then again, I imagine that most people who work with data have a much smaller collection of applications, languages, and procedures that they use on a regular basis. (At least 80% of my own, daily data work takes place in spreadsheets, for example.) It makes these lengthy lists of recommendations more aspirational than practical, more like wish lists or some Platonic ideal of a data scientist floating around in the æther.

I have worked in the data world for over thirty years and I have taught people ranging from first-year college students to working data professionals in global corporations. Their technical abilities varied dramatically, but what I found made the biggest difference in a data project – by far – was the ability to interpret and present data meaningfully. In fact, I recently described the skills needed in data science, and how hard they each were to develop, in these terms:

• Not so hard: Technical skills

• Harder: Organizational culture

• Hardest of all: Critical thinking

I say that technical skills are "not so hard" because they can be developed on an as-needed basis. For example, I usually work in R, but when I had projects that required Python – or Julia or Knime or RapidMiner or BigML or Orange or Bash or SQL or Tableau, all of which I needed at one point or another – I was able to learn the necessary parts relatively quickly. (I talk about the importance of picking up new skills in my last newsletter on "autodidact swagger.") You may be familiar with the "70:20:10 model for learning and development," which asserts that 70% of learning is done on-the-job. My own experience in learning technical skills is consistent with that assertion.

As for the second step in my hierarchy, organizational culture, that is based on the truism that it is easier to teach technical skills to somebody who already knows and functions well in your organization than it is to take somebody with technical skills and help them understand your business model and fit within your organizational culture. Maybe a better version of this is the saying "Hire for Passion, Train for Skill." Or, better yet:

Upskilling > onboarding

That is, teaching new, relevant skills to your existing employees, who already understand your company and are committed to its mission, is often faster, easier, and more productive than starting from scratch with new employees. It's one of the factors behind the upskilling and reskilling initiatives at LinkedIn, such as:

• "How Upskilling Is a Win for Both Employees and Their Companies"

• "Benefits of Upskilling Employees vs. Hiring Externally"

• The LinkedIn Learning Resource Center on Upskilling & Reskilling

• The LinkedIn Talent Solutions Guide to Upskilling Your Team

But the final – and most difficult – part of my three-step hierarchy is "critical thinking." And this is where the liberal arts comes in.

Data work ♡ liberal arts

There is a long and rich history of people making specious claims that relied on complicated mechanisms when much simpler processed could explain the same outcomes. This is, after all, why the 14th-century monk and philosopher William of Ockham created the theory known as Occam's Razor (note that the two different spellings refer to the same person), which can be paraphrased three different ways:

Entia non sunt multiplicanda praeter necessitatem

"Entities must not be multiplied beyond necessity"

"The simplest explanation is usually the best one"

There are too many examples in data science of things going wrong and people providing explanations that, to put it mildly, strain credulity. (I won't enumerate them here, but I describe a variety of them in my LinkedIn Learning course "AI Accountability: Build Responsible and Transparent Systems.") These are failures of critical thinking, or not adequately thinking about what the data mean and about all the possible factors that could have contributed to the observed data.

I personally believe that much of this problem comes from not spending sufficient time trying to understand people in all their complexities and contradictions. This is important because data is fundamentally a human creation: data doesn't exist in the natural world like, say, the length and density of a tree. Instead, data is the result of a person deciding to collect data, and then developing a method for measuring and encoding data. Data is created by humans, for human purposes, and, often, is about humans. As such, if you want to think critically about data, then it helps to think critically about humans, too. And I believe that nobody does that better than people who have been immersed in the liberal arts.

In its modern version, the "liberal arts" generally refers to the social sciences, arts, and humanities (and, surprisingly, to the natural sciences, as well, although most people probably think of the other fields first). People who have studied history, literature, art, and society at great length would, I believe, be less prone to some of the questionable statements about humans that can come up in the data science world. People who have critically studied human behavior are less likely to give simplistic explanations for complex events (e.g., the "fundamental attribution error"). And I don't just mean people in the social sciences, although that is my field (I have a PhD in social psychology). I have learned great lessons about human nature from poetry, opera, and theatre:

• Jane Kenyon's poem "Having It out with Melancholy" gave me a first-person view of bipolar disorder

• The best view of PTSD I have seen came from the opera The Long Walk by composer Jeremy Howard Beck and librettist Stephanie Fleischmann, which is based on Brian Castner's memoir of the same name

• Tom Stoppard's play Arcadia contains an excellent illustration of chaos theory, chance, and causality.

People in the humanities, the arts, and social sciences, in particular, have dedicated their professional work to exploring and understanding the human condition; in other words, to thinking critically about people. I believe that this thinking is essential to adequately interpreting and applying the results of data analyses.

I need to be clear: critical thinking about humans does not take the place of the required technical skills for working with data. My point is, rather, that (a) the technical skills needed for working with data are often not as encyclopedic as some would suggest; (b) the necessary skills can often be learned on the job; and (c) that adequately interpreting the data requires a critical understanding of the people who designed the systems for gathering data, the people who provided the data, and the people who will use the results of the analysis.

As such, people who aspire to work with data – especially when they are doing analyses for use by human decision-makers and not computer algorithms – can benefit by enriching their understanding of – and critical thinking about – people. Reading literature, delving into memoirs and biographies, exploring analyses of social life can all give greater content and nuance to data work. And, in that way, the synecdochic fallacy – the error of mistaking technical skills for the ability to derive and communicate actionable insights from data – can be avoided and data work can be more insightful and useful.

Thanks for joining me here. And remember, sharing is caring! Follow me on LinkedIn and share this  newsletter with a friend who you think would benefit from it.