Posts in Creative
Site Optimization: Lipstick on a Pig?, Or, Bacon and the Theory of Local Maximum
By Ryan Turner | Dec 5, 2008 12:34:33 PM
Cross-posted from Web Social Architecture.
Jason Carmel is a colleague of mine I learn a ton from. His expertise is in web site optimization--running experiments where he tests versions of web pages against each other to see which performs best. (Not to be confused with search engine optimization, improving a site's visibillity in search engines.)
Jason is a fairly unflappable guy. Nonetheless I recently started making an effort to get his goat. He gives me just enough encouragement that I keep going. The gist of my teasing is that optimization is nothing but a mechanical exercise to determine whether a red button works better than a blue button. "Glad to hear that red button worked out better by 2.84 percent Jason. The sum of your creative energy has produced yet another quarter million in revenue. You must really love your life, man. Hey, have you thought of trying one of those animated GIFs instead of a regular button?"
Fortunately, Jason is twice as nice as me, as well as twice as smart. He takes my ribbing well--and responds thoughtfully to the serious question underlying my teasing: We know optimization can move big numbers in terms of revenue, but can it do more than simply tweak pages to bump up conversion? Can it vet creative concepts? Can it maximize the creation of mutual value between businesses and customers? Can it help create more engaging experiences?
(image credit, found via Lee)
The short answer here, according to Jason, is that it depends, partly on what you're trying to achieve. If all you're focused on is moving business value measures, you're probably putting lipstick on a pig. But testing against value creation has the potential to uncover game-changing opportunities.
Here's an email exchange between Jason and me, in which he explains in a little more detail:
RYAN:
That whole web site optimization thing—isn’t it really just putting lipstick on a pig?
JASON:
I think “I hate you so much” might be a succinct way of responding, but I'll provide a little more detail:
Web Site Optimization is exactly like putting lipstick on a pig, but only if you start out with a pig. And if you are starting out with a pig, your opportunities for improving things are limited, and you’d be using the wrong tool to fix the problem. We are talking here about the concept of a “local maximum” which is a fancy, math term applied to mean “the best something can be within a limited dynamic.” Consider the aforementioned pig’s ability to fly, which, metaphorically speaking, is not particularly developed. We could take a pig and genetically modify it to be more aerodynamic. We could investigate building pig hang-gliders and attempt to train the smartest pigs to use them. But even in the best case, with the most aerodynamic pig, benefiting from the best training, and using the best pig flying technology, it will never fly as well as a bird. The best case flying scenario for a pig (the pig’s local maximum as far as flight is concerned) is nowhere near as effective as a bird’s. In that scenario, you’d be better off exchanging the pig for a bird at the start, rather than waste any time or effort teaching a pig to fly better.
Applied to the web: if a site sucks so much- if the goals and purpose are unclear, if the information architecture looks like my desk (at the moment), if the navigation is counterintuitive and the messaging has absolutely no intersect with the audience, then no amount of optimization in the world will make it right. The local maximum of that crappy site is too low for any optimization to matter. Or (even worse) you’d need the infinite number of monkeys to stop typing Shakespeare and to start applying experiments to your site to get the right combination where testing would make a real difference. Neither is very efficient. If your site is the pixilated equivalent of a pig, you need much more elemental help from a user experience expert first (know any?). Until you fix the fatal flaw(s) in a site, anything else you do will be throwing good money after bad.
Site Side Optimization works well in circumstances where the local maximum is high, but for some reason, the site is not achieving it. This can be due to single points of failure on the site, like a specific conversion path or page underperforming, or because the audience needs to be targeted more specifically, or because the existing content is stale/irrelevant. In each of these cases, experimental testing can make a huge difference. Optimization also works exceptionally well (and this is far more interesting to me) when applied as a method of trying out a new (and potentially risky) idea that could radically change and significantly improve an experience. In both of these examples, the basic site is healthy, and the optimization program serves as a tool to reach its fullest potential.
RYAN:
But what I keep looking for is the way to test birds against pigs, not in the sense of which flies better, because as a user experience expert I do have the capability to predict the winner of that contest—but when I don’t have a clear sense of the best conceptual solution. For example, maybe I just can’t decide between eggs and bacon. Can optimization help design a better breakfast, or only decide between pulp and no-pulp in the OJ?
JASON:
Optimization can test more conceptual ideas, but it will be really hard to unpack the WHY after we determine which one wins. Most sites aren’t deciding between bacon and eggs, but rather between the bacon, eggs and hashbrowns with coffee or the granola, fruit and yogurt with yerba matte. If the former wins in a test, I don’t know whether it’s because of the bacon (which will usually win over everything) or the coffee, or because the person deciding had granola for the past three days, and would have taken ANYTHING other than more granola.
The other trick about testing high concepts in a website format is that you would have to build each solution to test them, which is usually more expensive than testing out wireframes or front-end prototypes in front of a more controlled audience.
RYAN:
First, you seem to be suggesting that a test win for the bacon breakfast might not imply extensibility for bacon breakfasts in general: That because, lacking control, the results might be idiosyncratic, they might not therefore apply broadly. Next week you might get a different result. My question is, why does that matter? And why does “knowing why” the bacon breakfast worked matter, as long as you know it worked.
JASON:
It is definitely a question I get from clients a lot. Why do I care about the individual elements of a variant- if the variant as a whole makes us more money, let’s just launch it and move on. I can’t fault the sentiment, but knowing why the bacon worked could lead to better tests, more focused messaging and (even more) cash money. I want to know that it’s the bacon by itself that is the motivating factor outside of all the other influences. Let’s say that we ran a test breakfast against a bowl of Total cereal and we tested bacon with powdered eggs as the experimental variant. Now let’s assume that bacon and powdered eggs lost to the control by 1%. You could take the position that we would do better to serve Total because we want to avoid losing that 1%, and you would be right. But what if you knew that the bacon by itself actually improved the breakfast by 15%, but the powdered eggs were so crappy that they hurt the breakfast by 16%, so you netted the 1% loss? If I controlled for all the variables in my breakfast, that knowledge would a) help me make a better breakfast overall (just serve bacon), and b) will also prevent me from throwing away a positive variable simply because it was paired with a really negative one.
I imagine you run into that a lot with prototyping and wonder how you deal with it in the UX world. If a subject totally fails at a task, are you ever afraid of overcorrecting a prototype to account for it? Do you ever throw the baby out with the bathwater? How do you control for that?
RYAN:
A great question. One of the answers is that in usability testing, you're looking for usability problems. So as long as your test participants are representative of your users as a whole, major failures are, practically speaking, never anomalous. If your user population is one million, and one of the eight people participating in your test has trouble understanding some aspect of the interface, what are the odds they're the only person who's going to have that problem?
The other thing I wonder about is what happens when what you’re trying to accomplish is harder to measure than conversion (e.g. brand lift) or if you want to measure it over time (e.g. engagement). Especially in social media, it’s quality that matters, not quantity. You want to know how valuable your user-generated videos are more than you want to know how many of them you have. Can web site optimization help you get to answers?
JASON:
Ah, you and your social media. When are you going to come to terms with the fact that this whole thing is a fad? The future is in email, Ryan, and lots of ‘em. Mark my words.
Absent a more qualitative tie-in with optimization (surveys, satisfaction scores, etc.) you will be hard-pressed to get good data about branding or the impact of social media. But I’m not saying you shouldn’t optimize for branding or social media. I’m saying you need to get that qualitative kicker. I’ve done a few branding tests, and I think they provide some interesting feedback. But I’ve never optimized where a KPI has to be judged on quality (e.g., good comments vs. troll comments) or off the site entirely (e.g., buzz in the blogosphere). Sounds fun.
Twitter: Trivial. Beautifully So.
By Ryan Turner | Nov 11, 2008 11:18:37 AM
I wrote a post a year and a half ago on Twitter in which I identified two types of use for it. This was at the height of the initial buzz about Twitter, and while some folks were really excited to finally see an HTML / SMS / IM social application, others worried aloud that Twitter, for the same reasons it offers such a low threshold of entry, would also tend to erode the quality of social relationships online. The thoughtful anti-Twitter point of view was that it encourages triviality, and the gist of my post was that while that may be true, there are times when trivialities are useful.
Since I wrote that post, I've become an avid Twitterer, and I have to say my perspective on Twitter, and indeed on triviality, has changed. I now see triviality as maybe the critical element of truly meaningful relationships, online and off, and Twitter has accordingly become one of the cornerstone services of my personal and professional social lives.
That sounds crazy, I know, and while it may be true that I am prone to crazy-sounding declaratives, I'm actually not kidding about this. I now use Twitter at work very frequently, mostly within my team, and it has improved our functioning and, dare I say it, made us closer, more personally connected.
Let me float an assertion: The deeper the relationship, the greater the proportion of it dedicated to triviality; and beyond, say, 90% triviality, the relationship isn't a relationship at all. And likewise with meaningful interactions--if everything is meaningful, it's not a relationship, it's therapy.
So here is my Bullseye Diagram of Love, illustrating the way I'm starting to envision online social systems supporting healthy relationships:
So I'm arguing for Twitter, or an analogous triviality service (ha!) as a supplement to existing relationships, not as a full-fledged social channel in and of itself. As a social network, I actually do think it's useless, or worse. And you can take that as a caveat.
But I am saying I think triviality in general and as supported by Twitter, in both personal and professional settings, is indispensable--that we can and should deliberately design it into social systems.
All About Eve.
By Barrie Arliss | Aug 26, 2008 4:08:20 PM
When Leslie LaRue mentioned that all the bloggers on this site were male, I was confused. Maybe it’s because every blog I read is by a woman. Or maybe because I’m normally confused. Either way, I wanted to contribute. And since I’m a woman/chick/hey you lady and a writer here at this fine company, I figured we can end this male-only Zaazathon once and for all.
So for my first blog, I’m introducing you to some of the amazing blogs I go to. I can’t promise that everyone will like them, and most likely the men won’t. But as a writer, these blogs are an inspiration. They provide me with different views and ideas. They help me when I’m bored or looking for a recipe. And sometimes they even give me fashion advice. Whatever they do, I’m just glad these women have something to share and are getting noticed for it.
Dooce
She pretty much started the whole don’t talk about work on your blog phenomenon.
Not Martha
Not only does she live in Seattle, but her site was called Coolest Websites of 2006 by Time magazine.
101 Cookbooks
My all-time favorite cook, Heidi makes me want to be vegetarian. And then I remember I love bacon. Mmmm bacon.
What I Wore
She reminds me that I should throw away my holey t-shirts and try to look halfway girly sometimes.
Freckle Girl
Jess only stopped blogging so she and her coding hubby could start up a knitting/crocheting community that has quickly become a wild success.
Next up: Diaries. What happened to them?
Justin Marshall on Social Media Marketing
By Ryan Turner | Jul 16, 2008 4:21:03 PM
Cross-posted from Web Social Architecture.
Here is the video of Justin Marshall's talk last week, titled Money, Media, and Your Mom's Peach Cobbler: Social Media Marketing Done Right. Justin is a colleague of mine at ZAAZ and a major contributor to our efforts in social media.
Justin's take on social media revolves around 3 critical points: Find your customer's shared passion, build value through community, and focus on strategic objectives. His talk includes some great stories and examples, including a sample concept for a social media campaign done right. (Are you listening, Whole Foods?)
Here's the 15-minute video:
And here are the slides, so you can follow along:
I'll be posting the other presentations from the event as I get them ready. Video processing, especially at my novice level, takes forever.
You're Invited: Social Media Event at ZAAZ Seattle July 8th
By Ryan Turner | Jun 30, 2008 10:58:45 AM
Cross-posted from Web Social Architecture.
People keep asking me when we're going to host another event at the ZAAZ office's notorious Z-Bar in Seattle. Well...
I'm super excited about the lineup for this event, which includes industry leaders from corporate, agency, and independent circles who share passion and expertise in the human dimensions of social computing. Knowing most of these folks pretty well, I think I can guarantee an evening of thought-provoking conversation.
The format for this event will be similar to the previous one I organized, which seemed to go pretty well. We'll have short talks from each speaker followed by a panel discussion. Plus, snacks and beer.
Here's the rundown on the speakers:
Brian Fling of Flingmedia is a full-fledged mobile design geek and human Swiss Army knife. His talk, "Mobile 2.0: Design and Develop for the iPhone and Beyond" will explore some of the social dimensions of the emerging mobile world.
My colleague Justin Marshall is behind some of the most exciting work we've done in social media. His take on social media for marketers, titled "Money, Media, and Your Mom's Peach Cobbler: Social Media Marketing Done Right," offers guidance for marketers looking to engage with customers online.
Samantha Starmer is a highly-respected thought leader in the local information architecture community. In addition to her work at REI, she co-teaches the UW Information School's Summer IA Institute. Her talk, "Single Athletic Female seeking Single Slender Male: The Marriage of Social Media and Metadata," promises to reveal the secrets of better online living through metadata.
Nancy White of Full Circle Associates has been doing online community since WAY before it was cool. Her broadly-ranging expertise includes online learning and facilitation, communities of practice, technology in the developing world, and social technology in general. Her talk is titled "Slow Community."
Wendy Chisholm is a former co-editor of the W3C's Web Content Accessibility Guidelines and a passionate advocate for universally-accessible design online and off. She's currently working on what promises to become "THE book" on accessibility for the emerging Web.
We're really lucky in the Seattle area to have access to such quality thinkers and experts. I hope you'll join us, and if you do plan to attend, drop me a note at ryant (at) zaaz dot com so I can get a rough headcount.
The Facebook event is here: http://www.facebook.com/event.php?eid=66791410200.
I hope to see you there!
Consulting Skills: Winning with Logic, Not Bullets
By Ryan Turner | May 19, 2008 7:40:21 PM
Cross-posted from Web Social Architecture.
Boy howdy, do I ever love the header on this post!
What I'm actually thinking of is an approach I use to prepare presentations, which I learned from the best presenter I've ever worked with. He said something to me along the lines of, "Don't present the work, present the rationale for the work." And from, I think, that moment on, I broke free of the much-critiqued built-in rhetoric of popular presentation tools, which fragments when it should elaborate, reducing arguments to discrete, disconnected points, fragments the points into sub-points, and so on. The typical presentation relies on overviews and summaries to hold its rhetoric together, when it should be building and refining a cohesive point of view you can read, start to finish, and feel like it makes sense.
The fact that Tufte's classic lambasting of PowerPoint has become a cliche makes perfectly clear how much pain and suffering presentations cause. And the proliferation of books on the topic also makes clear how difficult it is to actually make good presentations. But I don't think it should actually be that tough.
In the consulting world, you have to go fast, and you have to collaborate. What ends up happening a lot of the time is that you divide up the work, and everyone goes off and makes some slides, then the account director puts them all into a "deck." (And the very word "deck" hints at exactly what's wrong with this approach.) When you present, the meeting is full of uncomfortable moments where someone says, "Justin, I think this is yours. Did you want to talk this slide?" But the problem with the deck concept isn't that labor can't be divided, it's that decks can be shuffled, and logical arguments can't.
So, If your slides make just as much sense almost no matter how you order them, go back to the drawing board. Simply put, if you can easily reorder your slides, their rational threads have frayed.
So here's my presentation tip: Approach the outline for the presentation as you would a persuasive essay. I think of it as outlining the presentation's rhetoric, as opposed to outlining the presentation's slides. Like so:
1. Your brand sits at the convergence of high trust and personal passion. Your customers are most likely to participate online in venues they trust around topics of great personal interest. Your products serve your customers passions, and your brand is highly regarded.
2. So the opportunity to build a customer community is there. This isn't true of every brand. Your great relationships with your customers are hard-earned, and the next logical step is to extend them online.
3. But it's not enough to build it and hope they will come. The Internet is full of empty, static communities that make perfect sense on paper. In truth, the social web is a highly competitive arena, and just showing up won't be enough for to realize the full opportunity.
4. You therefore need to think beyond innovation and create uniquely relevant customer value. Innovation is important, but it's not enough. You need to create value not realized elsewhere on the web or off--value your customers will wonder how they ever did without.
5. We have the chops (and the rigorous methodological foundation) to help you create a community where your customers will rush to engage, then discover on arrival that they already feel at home.
Not bad for completely generic copy! Reading back through that outline it almost feels like it... uh, makes sense, even though it's not about anything. The reason for that is the rhetorical connections across the paragraphs. Words like "so," "but," and "therefore" are like signposts, guiding the audience through the logic of the argument and connecting each point to the previous.
Now, this only works when you can torpedo the presentation-by-committee approach and put one person in charge of the outline. In my experience, everyone appreciates the person who will step up and figure out how to tell the story. Done right, with input from everybody, this approach ends up making everyone look better.
It's important, once you've created the outline, to stick with it. The random informational slide someone wants to drop in at the last minute has to inform the presentation's rhetoric or go into an appendix.
In practice, I usually end up putting the bold text on slides and using the rest as talking points. The last thing you want to do is read your slides. I'm lucky to work with top-shelf designers who contribute layers of visual meaning to the text, and frequently the text ends up getting replaced with something visual. That's fine, because the outline ensures that the final presentation holds together.
So there's Ryan's Ten-Cent Presentation Tip. Try it out!
Getting It Right: Designing Community to Support Your Core Offering
By Ryan Turner | Mar 26, 2008 10:17:23 AM
(Cross-posted from Web Social Architecture.)
These days, there's a huge amount of interest among corporate marketers about how to "tap into," "harness," or "ride the wave of" online community, Web 2.0, and social technology. And, despite the metaphors, despite the herd-like hype-chasing, rightly so. ("Herd-like," by the way, isn't a metaphor; it's a simile. A world of difference, if you're asking.)
The truth is, the widespread adoption of social technologies, including on the Web, are absolutely changing the game and creating new markets. Companies getting left behind on the Web are getting left behind in market share. For that reason, one recent survey indicates 90% of businesses will add Web 2.0 features to their sites in the coming year. (See Burby's ClickZ article for some caveats about that survey.)
But social technology is tricky to get right for companies who aren't selling social technology. The tendency is to implement a community that's either trivial, fails to produce business value, or simply fails altogether because nobody wants to use it. And failure hurts: Just ask Wal-Mart.
No, I Don't Want to Be Your Friend
There's an assumption out there that people "want to connect" with each other online. After all, people are increasingly connecting online, discussing topics of shared interest, getting dates, keeping in touch with friends and family. But on sites offered by companies that aren't in the business of connecting people online, in highly branded spaces in particular, this couldn't be further from the truth.
We do a lot of user research at ZAAZ, an increasing amount of it around social technology. Participants, when asked whether they want to connect with their fellow customers online, collectively say something along the lines of "absolutely not, no way, what do you want from me, what a creepy idea."
But changing the question changes the answer. When asked whether information or content provided by other customers in the context of a specific need would be of interest, the answer is, increasingly, yes, I must have it, it's crucial.
So what's the difference? Simply put: People aren't on your web site to make friends. They're there to get something done, and using community to help them get that thing done is a huge value, because it promises responsiveness, detail, honesty, and affinity.
Don't Just Build Community for Community's Sake
If you're in, say, the sailing equipment business, selling boat parts, life jackets, apparel, accessories, navigational charts, electronics, and so on, it might be tempting to create, say, a discussion board for your customers. Don't do it. It will fail.
I'm not saying nobody will use it. Sailors, after all, probably have stuff to talk about with other sailors. They might show up, possibly even in numbers to sustain a thriving discussion setting. Properly managed, a nice little community could emerge.
More likely, however, usage will be sporadic, volume low. You won't have the level of value needed to encourage repeat and sustained use. Your logo in the upper left-hand corner of the page will reduce the sense of authentic passion behind the community site, and users will wonder what your motives are. You'll moderate and be called a fascist. Threads will veer off course, so to speak, and topics will list hard a-starboard. You'll spend money keeping the thing afloat for a while, then late some Super Bowl Sunday, when nobody's looking, you'll quietly jettison the whole thing.
At best, your thriving discussion boards will give you a little brand boost. At worst, you'll end up a cautionary tale on some consultant's blog.
To Win with Community, Support the Core Offering
So instead of simply throwing open the gates to a discussion forum, think carefully about your real offering. How might your customers create more value around that offering?
Step 1: What do you sell?
Sailors, all of them, are gearheads. They love their gear, and because boats need a lot of maintenance, they're constantly tinkering with their gear. They read your catalog in the evening after work. They talk gear with their sailor friends. Other than sailing, talking about sailing gear is their favorite thing to do.
And hey, you sell sailing gear. So creating a venue for sailors to talk gear has the potential to not only appeal to your customers' enjoyment of gear talk, it also can drive sales. So create discussion forums, yes, but structure them around a taxonomy of sailing gear, and most crucially, tie them into your product pages. Excerpt discussions about products on product pages, and link to products from pages where people are discussing them. Add ratings and reviews for your products. Invite outstanding community content contributors to write posts for a group customer blog.
If your products are complex or require special knowledge to use, provide space for customers to support each other.
Step 2: Is what you sell the real offering?
If you're in the sailing equipment business, your offering might not be gear alone. Do you sell, for example, gear of a certain quality, at a certain price? What's your real brand promise? It's probably not "We sell sailing gear." It's probably something more like "We take the hassle out of boat maintenance, so you can get out on the water."
Another way of saying this is, What differentiates you from your competitors?
Step 3: How do you improve the core offering?
Without losing sight of the thing you're actually selling, think about the core offering of the business, and architect community to do more of it, extend it, complement it, enhance it, improve it, or fix what's broken with it. Providing highly-responsive technical support through community is sometimes a no-brainer, but there's always more you can do.
Imagine, for example, your sailing community offering a space for sailors to share their efficiency tricks. If your brand is about making sailing less fussy, you can offer a community site that promises not just gear, but the collective knowledge of the sailing community about how to get out there more and fuss with gear less.
Step 4: Tie Your Community to Your Product or Service
Remember, the core offering isn't always the thing that's getting sold, but that doesn't mean you can ignore the need to sell. And you don't have to feel bad about it! It's really tempting to want to adopt a perspective along the lines of "It's not about selling, it's about people." The truth is, it can and should be both. People aren't going to hate you.
I've been surprised many times by community participants' willingness to accept. selling within the community, as long as it's done right, without hard sells, and the community is providing value. Your customers know you need to make a living, and they'll find it perfectly normal for you to see a direct business return on your community site.
Think of it this way: If you were hosting a speaker at your sailing gear store presenting a slide show about a solo voyage around the world, folks wouldn't resent the visible presence of your products at the event. In fact, they'd appreciate it. You could even announce an upcoming sale or special deal for attendees, if you did it with taste and sensitivity, respecting the fact that people came primarily to hear the speaker and only incidentally, and only in some cases, to shop. Doing something analogous in your online community space is perfectly acceptable.
Don't be afraid to be in business. And don't be afraid to measure your community effectiveness in dollars--after all, if you're making money, you must be offering something of value! Without a doubt, branded communities require a different kind of thinking. But it's worth the investment. When community is working for you, you're not only running a cool web site that brings you cachet--you're co-creating, in partnership with your customers, a stronger realization of your brand promise.
