Lies, Damned Lies and Scopus

I do not claim expertise in economics, econometrics or finance. Instead, Scopus – one of several powerful databases that collect and analyze information about academics and academia – claims it for me.

Underneath my title, Scopus lists these subject areas, and more! Business, management and accounting. Environmental sciences.

About any of these subjects, I have written not a word. I am instead a published author on politics in China, development, rural poverty, and local government. You wouldn’t know that from my Scopus page. Can I at least get credit as a political scientist? Not from Scopus.

I didn’t post any of the misinformation associated with me. Scopus did. When I realized Scopus’s mistake, I did what any horrified mischaracterized academic would do: I clicked on “edit author profile.” There, I found I am empowered to change my preferred name (empowerment!), merge my profiles, add or remove documents, and (new feature!) update my affiliation.

I cannot, however, correct “subject area.”

A trip down the rabbit hole

There must be some mistake! Uncovering the source of these errors entailed a journey with which Kafka could commiserate. Clicking “contact us” (“We aim to respond within 24 hours”), I emailed Scopus to request that they correct their mistake, as well as to ask why they classified my subject areas in this way, and why they wouldn’t allow me to change them. To their credit, Scopus representatives did write back fairly quickly—about three weeks later.

“Please note, the subjects displayed in the subject area field are automatically captured by Scopus algorithm from the sources (Journals) in the author Profile,” a member of the Author Feedback Team wrote. After explaining that if a journal is classified with three subjects, Scopus’s algorithm automatically tags the author with those same subject area, the representative added,

“So I am sorry, subjects area are not editable field.”

“If a journal publishes papers on cats and dogs if I publish a paper on cats, I’m also an expert on dogs?” I asked at one point. (None of the email participants were especially grammatical, especially me.)

After three weeks of back and forth (believe it or not, this is short version), a representative from the Contact Services Desk repeated,

“I like to inform you that the subject area is uneditable field… Please note it all depends on the Journal covering respective subject areas. Therefore it is not possible to make algorithm to pick the topics as required.”

Although not an econometrician, I nevertheless imagine it should be possible to make the algorithm do this. Just not Scopus’s algorithm, apparently.

But even at face value, Scopus’s explanation doesn’t make sense. For instance, the journal World Development has published some of my research. Indeed, I believe it is from this journal that I inherited its tag, “Economics, Econometrics and Finance,” a subject area that applies to some of the articles in World Development, but by no means all of them. And certainly not to mine. At the same time, World Development is also classified in two subject areas that I do study: Development, and Sociology and Political Science. Somehow Scopus’s algorithm didn’t tag me with these.

After adding this point to my argument, a third representative, also from the Content Service Desk, explained, “I understand that your point is to mention the specific subject area under the profile instead of the major subject area.” (This too makes little sense, as most of the subject areas mentioned are among Scopus’s specific, not major, subject areas.) Scopus’s representative reassured me, “We have informed the development team to include the specific subject areas only instead of the major subject areas.”

So wrote Scopus’s representative on the last day of the year.

The year 2018, that is.

Am I alone?

Were other scholars similarly affected? None of my classmates or colleagues was correctly tagged with even their “major subject area,” and many were incorrectly associated with an imaginative array of subjects. Perhaps as few datapoints among many, we’re not worth correcting?

How about Albert Einstein's? Scopus correctly identifies the subject areas “Physics and Astronomy” and “Mathematics.” But was the inestimable Professor Einstein also an expert in “Engineering,” “Computer Science,” “Medicine,” “Decision Science”, and (added just to be safe, perhaps) “Multidisciplinary?”

How about former US president Barack Obama, who served (1992-2004) as a lecturer and senior lecturer with University of Chicago? President Trump might object that Obama’s affiliation is still listed as “White House, Washington DC, USA.” (Thankfully, Scopus now provides that “Update Affiliation” feature.)

Per Scopus, Former President Barack Obama is an expert on medicine, psychology, and agricultural and biological sciences.

There’s no mention of his expertise in law, let alone constitutional law – or (despite eight years in the White House) even politics. Join the club.

Perhaps Scopus’s misclassification of Obama’s subject areas derives from articles that he published recently, including “Repealing the ACA without a Replacement,” and “The Irreversible Momentum of Clean Energy.” Or perhaps the misclassification originates from articles that Obama didn’t write. Scopus claims that a chapter of Karin Williamson Pedrick and Sandra Arnold Scham’s monograph, Inside Affirmative Action, was not only about Obama, but authored by him. Similarly, Scopus erroneously credits Obama with articles published during his presidency in the Nation, the Economist and Time magazine.

Does this matter?

Along with Google Scholar, Researchgate, Kudos, CrossRef.org and Academia.com, Scopus is one of several powerful tools used to evaluate different facets of academia. “Powerful” not in the sense of “effective,” but more in the sense of the ability “to control people or events.” These tools have different specialization, but they all intend to evaluate academics. In this way, Scopus is only unique because it performs this function especially badly.

For its part, Scopus claims to be the “largest abstract and citation database of peer-reviewed literature…Delivering a comprehensive overview of the world’s research output … Scopus features smart tools to track, analyze and visualize research.”

Smart tools.

This from a company that misclassifies academics, pleads powerlessness over their own flawed algorithm, and remains unwilling to correct blatant errors when pointed out to them?

Scopus’s failures would be funnier if they did no harm. It is bad enough to endure a Kafkaesque experience to ask Scopus to do something simple—the right to change incorrect subject area labels. However, many institutions (including my university) use Scopus to track performance and make decisions. Thus, Scopus and its ilk have outsize influence, authority granted by an academia that is desperately trying to prove its worth by quantifying scholarly output.

Fair enough. The attempt to quantify our output is for our own good. Academics too should be held to account.

But most of these attempts are failing. And these failed attempts affect our behavior, our careers and our futures. Scopus’s shoddiness is just one extreme example of quantification. Powerful individuals and institutions use and perpetuate impact factors, citation indexes, aptitude tests and university ranking systems, all misleading and flawed, to affect vital decisions in higher education. Misinformation misleads decisions that misdirects investment that perverts our purpose.

Scopus is the symptom. The disease is the quantification of the unquantifiable.