Inside the Ideaplex
How do you decide which book to read next? (Assuming you read books.) I don’t mean books you must read—for work or school or for the book group. I’m talking about the book that—of the hundreds of millions of possible choices—you choose to read, want to read, can’t wait to read, will ignore even the latest episode of Homeland to devour.
The once simple act of choosing your next book has, for a number of reasons, become absurdly complicated now, as your personal path to a book has come under the influence of factors beyond your control. As a result, I believe, some of the joy and anticipation associated with cracking a cover for the first time has gone up in smoke.
One factor in this overcomplication is, of course, Amazon. In the first eleven months of 2012, roughly half (43.8%) of all books bought by consumers in the US were purchased online—that’s up from 25% in 2010. This growth in online purchasing has largely come at the expense of the large chains, Borders and Barnes & Noble, while the other outlets—including mass merchandisers—have not shifted terribly much. Surprisingly (to me anyway), sales at independent (non-chain) bookstores have increased over the past two years—from 2.4 to 3.7%. A tiny piece of the pie, yes—but a significant one.
How we purchase our books has clearly shifted toward the digital, but the question of how we choose those books remains. Any prospective book-buyer sets out with some motivation—to buy a specific book, a book on a specific topic, a type of book, or to browse for a book they (or a gift-recipient) might enjoy, based on personal preferences, recommendations of friends, reviews they’ve read, ads they’ve seen, and many other influences.
Amazon and the Algorithmic Paradox of Choice
Here is where the algorithm rears its peculiar, multivariate head. Amazon is the acknowledged trailblazer in gathering data, analyzing it, and repackaging it into processes designed to influence your user experience. But Amazon can’t replicate the experience of a bookstore—which combines so many tangibles and intangibles that are not (yet) available online. In a bookstore (or a library), you are subject to a much freer and random form of browsability. You can flip through and riffle the pages of a book without the rigid control of the back/forward arrows. You can catch a quip that makes you laugh or a sentence that inspires you to buy. Glance to your left and you might notice an attractive person (a potential date?) chortling at something in a book with a suddenly intriguing cover. You can talk with a clerk who has read every book in the store, will size up your tastes in a minute or two, and can make a match better than the cleverest, most data-rich algorithm. And then there is the feel of the book—its smell, its heft, its texture, and all of that.
Don’t get me wrong. I buy tons of books on Amazon and am not going to make a plea for the salvation of the independent bookstore. Amazon offers unbelievable selection, terrific prices, and super-convenient, speedy delivery. But it’s important to know how Amazon helps you choose a book, because it employs a number of data-driven helpers. It has the lists, hundreds (if not thousands) of lists—Recommended for You; New For You; Best-Sellers; Popular by Category, and so on—designed to put titles (and covers) that may appeal to you in front of your eyes. This is fine. You can’t step back and see the entire Amazon store, so Amazon holds your hand and does the online walking for you.
Take, for instance, the Recommended For You list, and the patent Amazon submitted for its creation—for a “recommendation service” driven by “content-based filtering” or by “collaborative filtering.” The list of recommendations displayed on your screen is a selection of books concocted from activity occurring all over Amazon, both at present and in the past, from books similar to ones you viewed, put in your cart (whether you bought them or not), or purchased outright. Or, even, based on what a defined community has viewed, carted, or bought.
As we all know from experience (and as Amazon readily admits in its patent application), there are real limitations to this kind of recommendation. Unlike the smart human bookstore clerk (or friend or colleague or teacher), the algorithm has nothing to say about the quality of the recommended book. New books may suffer from a “cold start,” which means there just isn’t enough data to go on, so they’re less likely to be recommended. And, of course, the algorithm cannot begin to imagine what books might interest you that are unlike anything you have read before. And that is a very serious limitation: the algorithm keeps pushing you down the same path. It can’t make weird connections or be visited by flashes of recommendational genius.
Then there are Amazon’s ‘Popular’ lists, which—at least for books—show up within specific categories (e.g., Science Fiction, Romance), and differ significantly from its Best-Sellers list. The Best-Sellers list is a rather blunt instrument that displays the top-selling books in the last 24 hours. The Popular lists, however, are much more finely tuned. First, these lists incorporate a longer time frame—not 24 hours, but 24 days (a month, more of less). Popular lists also take into account free sales or downloads, which the Best-Sellers lists do not. And finally, these Popular lists are more affected by Amazon reviews, though only by quantitative aspects of those reviews, e.g.,, how many reviews, how many stars.
Why Amazon’s Reviews Matter…And Don’t
Which brings me back to the question of quality. Amazon does not know much about its reviewers. Yes, it has its Hall of Fame reviewers and Top 100 reviewers, with short profiles of them available—but that information is supplied by the reviewer. They could be anyone. And they could be intentionally misleading you. Even, *gasp*, deceiving you.
Indeed, as with any system, Amazon’s is ripe for gaming. Last year, Amazon deleted a trove of reviews (the number was never disclosed) after it became obvious the system was being manipulated. In August 2012, the New York Times ran a story about Todd Rutherford and his website, GettingBookReviews.com, where authors could go—and did go, in hoards—to pay Rutherford to write reviews of their books and post them to Amazon. And there was the issue of ‘sock-puppeting’: authors using pseudonyms and aliases to write reviews of their own books and those of their competitors. RJ Ellory, the British best-selling author of thrillers, was at the fore of this scandal—after being found out by another author—for writing glowing reviews of his own work and denigrating reviews of his competitors’ books.
Such manipulations—paying for 5-star reviews, writing dozens of reviews of your own work—increase a book’s visibility, which results in more clicks, higher rankings on other lists, and, theoretically anyway, more sales.
In response, Amazon not only deleted reviews—many of which may have been genuine and innocent—it also created new reviewer guidelines that may preclude legitimate reviews from appearing on their site. For instance, authors may encounter an Amazon blockade when posting a review of someone else’s book simply because they are an author. But those fake reviews commissioned by Rutherford through GettingBookReviews.com? Many of them are still online.
And there’s more to make one question the value of Amazon’s reviewing system than foul play. Harriet Klausner, an Amazon Hall-of-Fame reviewer, has written nearly 30,000 reviews. And, according to the New York Times, more than 99 percent of them were either four or five star reviews. Mrs. Klausner told the Times, “If I can make it past the first 50 pages, that means I like it, and so I review it.” I wouldn’t call that a particularly discerning attitude.
What You See is…What You Get?
What does this all mean for us as book-buyers, as readers? In the transition away from physical bookstore to online retailer, we’ve subjected ourselves to the world of metrics, where numbers, leaderboards, and statistical analyses generate and infuse meaning into our interactions and our lives. Instead of a good friend, a trusted reviewer, a kindred-looking stranger, we’re introduced to books by a conglomeration of data tweaked and weeded to meet our exact specifications, which have been defined by another conglomeration of data. And while it may be true that “numbers never lie,” I actually think they can. Or, at least, they tell a short-sighted kind of story.
It’s concerning that online purchasing is rapidly replacing in-life purchasing, but not because of its effect on the superstores (which often had a lot of books, but not much selection) or on the small independents (which aren’t dying anyway), but on how we think about selecting books. How much do we want to be guided by the numbers—the metrics, the %$#@! algorithms—the most-populars, the best-sellers, the most reviews and highest-starred, the books that everybody else is buying and reading, the books that are most like the ones I’ve already read?
So many of my richest and most memorable reading experiences have come about in unexpected and unpredictable ways. I don’t even remember how I got on to reading 13th-century Icelandic sagas or Joseph Schumpeter or Tin-Tin, but I guarantee you no algorithms were involved.
What’s more, just as Netflix is producing television material based on known likes, the pervasiveness of data-driven book selection will inevitably inform and influence the creation of books. The demands of data will determine what books get written, marketed, and bought. Readers will become a collection of data points. Books will be created and shaped by data points. And our choices will be determined by the analysis of still more data points.
Amazon’s attempts to help us choose stem from a problem of its own making: scale. The selection is far too vast for any human being to manage. And the intimate book-to-human relationship has been overshadowed by the giddiness of unlimited choice, the bullying of the crowd, the siren-song of the popular, and the rut of our own habits.
In terms of numbers, then, we’re no match for the algorithm. Fortunately, however, when it comes to creativity, accommodation of randomness, synthetic abilities, intuition, insight, risk-taking, and adventurousness, humans have a considerable edge. And that will keep writers writing weird and wonderful stuff and readers stumbling onto books they never knew they wanted. Please.
 Statistic based on reviews HK had written at time of NYT article, which numbered 25,000, not 30,000.
I recently gave a lunchtime “author’s talk” at Children’s Hospital in Boston and, although I thought the talk went well, somebody in the audience didn’t like it at all. On the evaluation form, the person in question wrote a single word in the comment box: CONFUSING.
Thank you, whoever you are. While everybody else gave me good marks and said nice things, which I appreciated, my critic forced me into self-examination. Was he the only one forthright enough to speak up, or was he the only one not paying enough attention to get it? What was confusing? The ideas? The presentation?
This all got me thinking about feedback. Whenever you go public with an idea — in a book, a talk, a presentation, a video, a graphic — you will inevitably get many kinds of responses. This feedback generally falls into one of three categories: praise, silence, and backlash.
Praise seems quite easy to handle — we all love to be praised, especially when the praise is nonspecific, such as “fascinating!” Go ahead and bask in the praise: It is a reward for your work and a motivation to push forward. But such praise is not necessarily valuable feedback. In order to make use of this praise, you must probe it deeper: What, exactly, was fascinating?
Silence can be difficult to interpret. A few years ago, during a 90-day interim as blogger-in-residence for BzzAgent, a start-up social media marketing firm, I wrote a daily blog about company issues and stories. Some of these posts received zero comments. I assumed my readers were indifferent, disengaged, or actively did not like these particular bits of writing. But, in face-to-face discussions with my audience (there I was, surrounded by them), I discovered that often Internet silence corresponded to deep thinking and reflection done off-line. So, as with praise, the value of silence may require mining: Did I leave you speechless? Or did you just not care?
An idea that advocates any kind of change is likely to receive some amount of negative response. When you’ve invested time, energy, and passion into your idea, this rejection can hurt. Your first impulse may be to lash back, to rebut the rebuttal. But a better response is to let the backlash unfold a bit: It is likely that negative feedback will be the most useful in further developing your idea.
Backlash takes many forms and is unleashed for many reasons, so it’s important to first understand the nature of the criticism, as well as its source. A thoughtful review from a credible source is not the same as a mean-spirited comment online from an anonymous Internet troll. (The latter of which you can ignore.)
If, as with praise and silence, you take a moment (or a night’s sleep) to reflect on the backlash — what kind is it? why is it happening? — you may realize that backlash has its own unique advantages:
- It deepens the appreciation of advocates. In light of a contrary opinion, those who initially said your idea was simply “fantastic” may be forced to think about it more deeply, and respond with more detail. I thought X was fantastic, but in light of these comments, I had to reconsider and found that XX… Additionally, backlash can cause those who were silent at first to speak up as advocates of the idea. Only when an idea is challenged, and especially when it is attacked, do people realize just how much they care about it.
- It creates new contexts for the idea. Consider backlash against Michael Pollan, the best-selling food expert, whose books include The Omnivore’s Dilemma and, most recently, Cooked. Adam Merberg, in the Berkeley Science Review, suggests Pollan misrepresents and even vilifies science. Tyler Cowen, in Slate, writes that Pollan “neglects the macro perspective of the economist.” And Emily Matchar argues against Pollan’s historic view of women’s role in cooking in her Salon.com article, “Is Michael Pollan a Sexist Pig?” Did Pollan think his ideas through from the point of view of science, economics, and feminism? Maybe, maybe not, but thanks to the backlash he received, the debate about the value of home cooking now embraces those topics. Negative feedback from disparate domains empowers you to articulate your idea more clearly — to incorporate, avoid, or merge it with other areas of thought.
- It improves the quality of the argument. Recently in The New Yorker, Adam Gopnik explains how brain science has become an explanation for just about everything (why we eat what we eat, say what we say, etc.), and how people are beginning to push back against it. This backlash may be a case of idea fatigue: People are merely tired of hearing about the brain, to a point of heated annoyance.
Discussion, debate, and positive-negative tussling serve to put an idea through a public testing that makes it stronger and better or, sometimes, rejects it. As the one who has brought the idea forward, it brings you into the conversation in a new way, giving you more license to speak further, create new expressions of your idea, and seek to influence outcomes you care about.
It is no small feat to stimulate genuine conversation about any idea, and to generate criticism, rebuttal, debate — and even attack — suggests that you have touched a nerve, surfaced a tension, or put your finger on an issue that needs discussing.
While the particular comments matter, what matters more is how you use the feedback to gather advocates, interpret your idea in new contexts, and improve its quality for your now broader audience.
So, to the person at the Children’s Hospital talk, please be in touch with a bit more detail. I don’t mind being called confusing, but I need to know exactly how and where and why you think that.
This post was originally published on HBR.org.
The meta story of a mega-bestseller is always interesting to me and to any author whose books do not quite fit that description. Of all the books published in a year, a decade, why did that particular one break out so massively?
It is the American question, really, about everything: What makes a success? — in books, sports, business, education, or any other endeavor.
Stephen Hawking, author of the super-mega-bestseller A Brief History of Time, has now published his autobiography, My Brief History, an adapted excerpt of which appeared in the Wall Street Journal. In it, Hawking attempts to analyze why his book gained such attention and sold so many copies. (Millions.)
I found the article rich with intriguing details that track closely with my model of the idea entrepreneur — how they go public with an idea, break out from the surrounding noise, and gain influence.
First is the matter of personal narrative. Hawking writes that most reviews of A Brief History of Time began with mention that he was wheelchair-bound as the result of Lou Gehrig’s Disease, was unable to speak, and did not have complete command of his fingers.
“Undoubtedly, the human interest story,” Hawking writes in the WSJ article, “has helped sales of my book.” He adds that people who bought it for that reason “may have been disappointed” as the book was “intended as a history of the universe, not of me.”
Hawking misses the point. People often buy a book because of the author’s story, whether the story is in the book or not. Hawking does not mention the massive amount of publicity, coverage, or discussion that surrounded the book. I find it hard to believe that had A Brief History of Time been written by an unassuming, non-colorful, regular person it would have had a similar trajectory of success.
However, the point is that the book couldn’t have been written by such a person. Ordinary, unassuming people just don’t write powerful, paradigm-busting books. And there is something inherently compelling about a man with ALS, which in most cases has no known cause, coming up with a book that addresses, as Hawking writes, “the biggest question of all: Where did we come from and where are we going?”
(Not to mention the conclusion Hawking reaches about the universe — that it is “neither created or destroyed: It just is”. Shades of Eckhart Tolle, another super-mega-bestselling author.)
The personal narrative is essential, fundamental even, in the end, whether or not it is addressed directly in the book.
Of course, I found A Brief History to be a very strong and well-written book, a view supported by many reliable critics. It’s clear. Insightful. Witty at times. On a topic of enormous interest and importance. Being a wheelchaired physicist with a fascinating personal narrative will do you no good if you have written a god-awful book.
Here’s another factor: Hawking admits he went into the adventure of writing a “popular book about the universe” partly with the intention of making money, to help pay for his daughter’s school fees. It’s meaningful that Hawking mentions money. Every author wants his or her book to be financially successful, of course, but Hawking was especially willing to do things that were likely to increase its chances of doing so.
To wit: Hawking chose to work closely with an editor (at Bantam) who, by Hawking’s telling, worked him very hard. “Each time I sent him a rewritten chapter, he sent back a long list of objections and questions. At times I thought the process would never end. But he was right: It is a much better book as a result.”
This is not the reaction of all authors: some resist the editor, try to subvert, give up, or simply can’t pull it off. Hawking, even with the use of fewer fingers than a full complement (the press constantly got the number wrong, he says), persisted.
He also took advice about popularizing the title, which had originally been From the Big Bang to Black Holes: A Short History of Time. The editor switched the title and the subtitle, and substituted the word “brief” for “short.” The editor’s “stroke of genius,” Hawking writes, “must have contributed to the success of the book.” (New York Times bestseller for 147 weeks; 10 million copies sold.)
If not for: a) an initial financial motivation, and b) the critical collaboration of a sales-savvy editor, A Brief History may have had a much shorter stint on the best-sellers list, not becoming the mega success we know it as today.
Still, even with all that success, Hawking says he has long sensed there was a lot of crowd-inspired purchasing going on, people buying the book to “display on their bookcase or coffee table.” This is undoubtedly true. I did read the book — no, really I did — but I remember chatting with many people during the book’s first few years of life who knew all about it, but confessed to not having actually cracked the covers.
Which brings me — and brought Hawking — to a final point. As I have written about many other idea entrepreneurs, the ultimate metric of success is not to be found in sales data, number of scholarly references, hits on a website, or ginormity of speaking fees. For Hawking, the proof of the value of his book, and the power of his influence, is there in the pile of letters he gets every day, filled with questions or “detailed comments.” This kind of one-on-one connection is the most tangible, measurable, and usually the most meaningful evidence that you, your book, and your ideas have hit home.
We’re all fascinated by new ideas and how they can grab hold of us, influencing how we think and affecting how we take action. How does Atul Gawande (the checklist doctor) get inside my head, when others don’t? Why does Gwyneth Paltrow make me adjust my behaviors, when others can’t?
In business, especially, we’re inundated with new ideas—so many we can hardly process or evaluate them.
If you have tried introducing a new idea into your organization or community—especially if it’s an abstract idea like sustainability, diversity, or innovativeness—you know it’s tough. People may ignore your idea, pooh-pooh it, or just steal it. You put your energy, your reputation, and maybe your future, on the line.
Still, when you have an idea you think is valuable and could change things for the better, some inexplicable force may compel you to “go public” with it. You want to change a prevailing mindset and you’re willing to stick your neck out, at least a little, to do it.
The question is how, especially when you’re not Gawande or Paltrow, to change an organization from the inside when you don’t necessarily have a ton of formal authority on your side.
Take a look at what the “idea entrepreneur” does. This is the new type of cultural player—idea-driven people like Gawande and Paltrow, Michael Pollan (food), Cesar Millan (dogs), Blake Mycoskie (business = philanthropy), and many others—who reach large audiences and gain widespread influence.
These are independent operators. They’re wealthy and famous. They publish books and appear on TV. But do their methods of idea-spreading apply to the office-dweller, the organizational citizen, the manager or executive? The answer is yes, with some modifications.
To illustrate, let me introduce Samantha Joseph, who works for Iron Mountain Incorporated, a leader in records storage and information management services. Sam is young, ambitious, and idea-driven: she believes that for-profit companies can drive significant business value when they take a strategic approach to social and environmental sustainability. After earning her MBA from MIT in 2009, Sam could not find a position in sustainability, so she joined Iron Mountain as a manager in strategy, knowing the company had no sustainability function and hoping to build it.
Sam saw opportunity. She plunged into the strategy job, but also began to work on bringing the idea of sustainability to Iron Mountain—an exciting if daunting task, considering the number of employees, locations, facilities, trucks, and other real assets involved.
Here are some of the things that big-time idea entrepreneurs do that Sam adapted for her quest:
- Accumulate evidence. To gain influence for an idea, you need an awful lot of supporting material—data, references, cases, stories, and analysis—which can take decades to gather. Sam didn’t have that long, but she did accumulate enough material to make her the “resident expert.” It took her a year to develop her case—largely on her own time—before she started talking about sustainability in public. But she was talking privately with lots of people all the while: gathering opinions, refining the ideas and practices, making connections and gaining supporters.
- Develop practices. An idea is an abstraction that won’t produce change until you provide people with specific, practical ways to put it into everyday use. Cesar Millan’s ideas get their teeth from his training methods of “calm assertiveness.” Sam worked with colleagues from across Iron Mountain to develop a volunteer program, a solar energy pilot, and a strategic charitable partnership—which got people doing sustainability without having to pledge allegiance to a theory first.
- Create a sacred expression. Practices without theory are nothing more than tips and techniques. You need to find your best form and use it to create a “sacred” expression—a talk, a video, a written piece, a visual—the most complete, authoritative, and compelling articulation of your idea that you can manage. Sam put together a short video that made the case, showed how Iron Mountain was involved with the community, and got people energized and emotionally engaged.
- Encourage “respiration” around your idea. Sam did not expect to do a TED talk or appear on Colbert, but she needed to engage with the audiences who would be most affected by, and most able to implement, the sustainability idea. The only way to get an idea breathing on its own is to show up, in person. Sam put together a road-show and visited many of Iron Mountain’s corporate departments and facilities, conversing and responding to questions. After she left, people kept talking. The idea started to come to life.
- Include your personal narrative. Idea entrepreneurs always present their idea in the context of their own life story. Dr. Berry Brazelton, expert on babies, relates anecdotes about himself as a kid. Sam talked about how her fascination with sustainability arose from an early family relationship. She presented the business case first, but the personal story gave the idea personality.
- Align with a metric. Influence cannot be definitively measured in financial terms, but people need some way to calculate its the value. People associate Malcolm Gladwell’s idea of mastery through the “10,000 hours of practice” metric. As the sustainability practices took hold at Iron Mountain, Sam measured and reported on key performance indicators—particularly cost savings—along with non-financial measures, such as volunteer participation rates, that demonstrated its success.
- Expect backlash. When you propose a new idea, expect an intense response—from useful debate to useless sniping. Sam didn’t experience direct backlash, but she did come up against some challenges. She spent almost a year talking and debating, pushing and being pushed, before respiration began to take hold. She realized (as all idea entrepreneurs do) that an intense response—positive and negative—is a sign that people are taking the idea seriously. No challenges, no point.
Each path to influence, and each change of mindset, looks different. Within two years, Iron Mountain had reduced its carbon footprint, helped archive the blueprints of some of the world’s most historic sites, and donated 65,000 volunteer hours to community efforts, among other initiatives. And today, corporate responsibility activities are started and driven by employees all over the country.
Sam most directly knows the idea has gained influence when someone says to her: I’m so proud of what we’re doing. One-on-one validation is a powerful metric.
This post was originally published on HBR.org.