The Power of Doing it RITE

By Nick Leggett | Jul 30, 2008 2:59:33 PM

Cross-posted from User Research Findings.

I was once again reminded of just how powerful user feedback can be. And this time it was a RITE Usability study that reopened my eyes.

If you're not familiar, RITE stands for Rapid Iterative Testing and Evaluation, a variation of traditional usability testing documented by researchers at Microsoft in 2002. In short, you test a design with five users on Day 1, improve the design based on feedback on Day 2, test again on Day 3, iterate on Day 4, and then test the final design on Day 5 with eight users. A RITE study is not always appropriate, such as when there are many tasks or if the design is quite fixed, but whenever possible I'd highly recommend it. Here's why. 

Some benefits of the RITE Method:

Team collaboration: The development and design teams REALLY get into it. With traditional usability testing it's sometimes hard to get dev and design to attend even one session, but here it's a requirement. The user feedback quickly initiates intense collaboration sessions which are just plain fun. And it's very rewarding when changes to the designs resolve problems found earlier.

Client satisfaction: If the client attends any of the sessions they quickly see the value of user feedback, as well as the team's problem-solving skills and creativity in action. Usually this voodoo is behind the curtain, but putting it in plain sight actually demonstrates the value of the work.

Time savings: There's a reason "Rapid" is part of this method's name. The changes between the first and final designs were absolutely dramatic in our study. (Unfortunately we can't show the screens due to client restrictions.) We tested a Flash-based tool for narrowing down TVs of interest from a large number of choices.

Though it's not practical to go through all the changes, there were dozens of improvements based directly on user feedback. Most importantly, the most severe usability issues were completely resolved in the final iteration.

While there are challenges associated with the RITE Method such as perceived higher-costs (I say "perceived" because arguably it's actually less expensive but it buckets the costs more up-front) and a demanding schedule, I think the benefits easily tip the scale.

Taking a 9-Year-Old Out for a Beer

By Jason Carmel | Jul 22, 2008 3:49:33 PM

In exactly three days, my nine and seven year-old nieces will be visiting me and my wife in Seattle, and I'm terrified.

I should explain further that I'm not terrified of the girls themselves, who are about as fun-loving, well-mannered and adorable as you can get. This isn't the problem.

The problem is that I am expected to entertain said nieces for a period of several days and, after considerable reflection, I really have no idea how to do that. As an interesting point of contrast, I hosted a bachelor party for a friend last weekend and was able to pull that off without hesitation. The secret for the bachelor party was good food and lots and lots of beer. Since these map to my interests inherently and I was similar to the other attendees, I had a good chance of overlapping with the wants and needs of my guests. This is patently not the case with my nieces for whom (I'm told) beer is out of the question, and "good food" means, in all likelihood, a hot-dog and go-gurt. What do girls in elementary school like to do? How do I talk to one for longer than six minutes? I'm starting to panic here.

The ironic thing is that I see websites confront this problem all the time, and one of the things in my job description is to help them deal with it. A website gets 10 visitors- half of them are attendees at a bachelor party and the other half are 9-year-old girls. Make sure each of them is happy...and...GO! There are many different ways to approach this challenge, but the one truism is that an experience that proposes to serve both simultaneously will almost certainly fail, unless you are basically a blank, purely transactional page.

My User Experience colleagues will (rightly) suggest that I do a bit of research to understand why my nieces are coming to Seattle in the first place before I attempt to design an experience for them. Based on feedback from their parents, I think my nieces are coming to hang out with us (primary goal) and to do cool things (as defined by young girls) in Seattle (secondary goal). This is decidedly different from my bachelor party colleagues who came to Seattle to do things that will embarrass and endanger the groom (primary goal) and to not get arrested (secondary goal).

So I am on the lookout for places in Seattle that are unique and kid-friendly, while allowing for the primary goal of spending quality time with us, and I am thankful that I don't have to cater to my bachelor party friends at the same time, as I fear it would have failed to meet their expectations. That being said, if you have any ideas about where to take my nieces, please comment below. Please. Seriously.

In the meantime, I'll leave you with an intellectual exercise: How many customer types do you think you have and who are they (then ask three of your co-workers to see if they agree)? And is the experience offered on your site targeted to each customer type? Or are you trying to take a 9-year-old out for beer?

Wish me luck this weekend.

Justin Marshall on Social Media Marketing

By Ryan Turner | Jul 16, 2008 4:21:03 PM

Cross-posted from Web Social Architecture.

Here is the video of Justin Marshall's talk last week, titled Money, Media, and Your Mom's Peach Cobbler: Social Media Marketing Done Right. Justin is a colleague of mine at ZAAZ and a major contributor to our efforts in social media.

Justin's take on social media revolves around 3 critical points: Find your customer's shared passion, build value through community, and focus on strategic objectives. His talk includes some great stories and examples, including a sample concept for a social media campaign done right. (Are you listening, Whole Foods?)

Here's the 15-minute video:

And here are the slides, so you can follow along:

I'll be posting the other presentations from the event as I get them ready. Video processing, especially at my novice level, takes forever.

Quantcast: Defining Audience Analytics (initially published on ClickZ)

By Shane Atchison | Jul 10, 2008 11:58:00 AM

The best things in life are free. Quantcast is a free media measurement service that enables advertisers to view audience reports for millions of sites and services. Recently I chatted with CMO Adam Gerber about the methodology behind his company's success.

Shane Atchison: Quantcast is doing a fantastic job building its product and brand. What do you attribute your success to?

Adam Gerber: Our success is a function of delivering a solution the marketplace lacks and which it craves. We're trying to help publishers who sell audiences and advertisers/agencies who buy them have improved and more sophisticated ways of understanding, segmenting, and establishing value based on the characteristics of those audiences. Search advertising, which is effectively a hyper-addressable marketplace, works because it has transparency and liquidity. But search is about consumers finding advertisers; they're already in an active mode. Brand advertising, the upper funnel, is about marketers finding their consumers. Better targeting and eventually addressability in the upper funnel require definition around segmentation and real-time decisions, the equivalent of the keyword. But for brands, it's about the "who" -- demographically, contextually, behaviorally, etcetera. Our focus is on enabling that evolution so that buyers and sellers can drive better business results.

SA: Your open syndicated ratings service appears to be one of a kind. How does it work?

AG: It is one of a kind. We're actually doing a number of things that are revolutionary. We believe in an addressable media economy and to get there, the market needs a new way of thinking about audience data and measurement. As an industry, sellers and buyers of media need transparency and depth tied to the data they work with. But more importantly, we need measurement approaches that are built for a digital world, that provide both census-level capabilities to drive real-time advertising integration and the audience-based measures that will be used to drive targeting and allocation decisions.

Unlike any other measurement service, we enable a couple of key things. Most importantly, participation and control. Any publisher can have directly measured audience data generated via our Quantcast Publisher service -- and it's free. Publishers participate in that service, they have control over how they organize their audiences, and they decide what data is offered for public view. And I'd point out, we've had terrific adoption of our Publisher service. Over 10 million Web properties are now directly measured: sites, blogs, video channels, etcetera. Many of the largest publishers on the Web are now participating in the service.

The result is a unique solution that marries the benefits of direct measurement (cookies) with those of panel-based reference points (people/audience). Our model is based on inference. We don't just intersect our pixel data with our panel, do a simple extrapolation, and call it a day (e.g., for a property, if we see 100 directly measured cookies and see 10 of them across our panel, we don't just ascribe the characteristics of the people we saw in the panel to the full, directly measured audience). We are literally collecting billions of new directly measured data points every day and building a model that continually improves.

SA: Can you share some examples of how ROI (define) is measured?

AG: We're helping publishers today create actionable inventory packages that have directly measured audience data tied to them. That's a pretty powerful thing for a seller, and a really unique offering for buyers. They simply haven't had real audience data tied to the actual allocation of media they are buying in the online space. And I want to re-enforce [that] I'm not talking about impression counts. Obviously, we've been able to predict and account for impressions for over a decade. I'm talking about demographics and affinities for actual traffic. That's something unique. Marketers are also taking advantage of our solution to better understand who is buying. This helps them optimize their media buys.

SA: The Web analytics space has been incredibly hot the past year. How do you differentiate from the other players in your space?

AG: It's pretty simple. We don't put ourselves in the Web analytics space. Web analytics has been about understanding your own site traffic characteristics. Notice I used the term "traffic," which to me is about the "what," not audience, which is much more about the "who." Web analytics solutions are terrific because they give you a directly measured view of traffic flow, metrics tied to abandonment, content trends, merchandizing performance, etcetera. Analytics services up until now have been internal solutions. They are generally silo implementations that don't allow for Internet-wide models to be built (in aggregate form). We are about audience measurement and activation.

Quantcast is focused on enabling publishers and marketers to understand the characteristics of audiences as they flow through digital media. The limitation of analytics solutions as they exist today is they tell you what is happening, but they don't give you any insight into why from an audience-dimension perspective. More importantly, they don't enable easy marketplace consideration and implementation of the data. They aren't consistent, scalable third-party data providers. That last point may seem confusing, so let me put it a different way.

If you're a publisher/salesperson, you know that if you walk into an agency-buyer meeting and present your internal Web analytics data, it will probably be rejected. Buyers today defer to audience services like comScore and Nielsen for analysis. Agency buyers, especially those focused on brand spending as opposed to direct marketing/performance, don't have the time or resources to evaluate every publisher's independent data. It's based on different methodology than they care about (cookies instead of people), and more importantly it's not easily digestible. Buyers need scalable, independent services that facilitate easy consumption of trustworthy and transparent audience intelligence. We believe that we're bringing a fresh perspective to the marketplace: the data that buyers need and the participation, control, and accuracy that publishers expect.

SA: Who are your direct and indirect competitors?

AG: This is always a tough question to answer, because honestly we don't really see any direct competitors right now. Obviously, there are folks out there trying to do pieces of what we are doing.

When we first launched, we were bucketed in with folks like Alexa and Compete, mainly because we were free and we provided data for millions of domains. As we've grown, expanded our services, and focused attention on demographics and audience insights, folks have started to talk about us in relation to comScore and Nielsen. While we offer some similar services, we are focused on completely differently visions. The key difference is one of direct measurement and scale. Panels are great for providing broad visibility and competitive analysis, but they don't deal well with the fragmentation and distribution of today's digital media economy. Perhaps most importantly, though, panels will never enable addressable advertising. Direct measurement allows us to capture media consumption data at arbitrary levels of precision, and while other audience measurement vendors are toying with direct measurement, no one else is close to operating at the scale we currently are.

Google is also looking to make inroads into this market. Everyone saw Google's Ad Planner announcement in June. Ad Planner theoretically competes with one of our offerings: Quantcast Media Planner. There are stark differences between the services, though. We, like many folks in the industry who reacted to the launch, view Ad Planner as a tool aimed more at helping small marketers buy ads on the Google ad network. Google, of course, has the scale and technical expertise to handle lots of data. The market will decide if it has the independence to work directly with all of the top online publishers.

SA: How do you make money?

AG: It's a great question, and one that we get asked all the time. All of our services to date are free...We plan to maintain these services as open and free offerings. We believe strongly that the marketplace -- everyone in the market -- needs transparent access to data and ways to actively participate in audience measurement.

Ultimately, we believe the future of advertising rests on better use of audience data to drive addressability. We'll be introducing services that will allow more effective segmentation and better returns for publishers and advertisers alike. Quantcast will charge for these services. But I want to reinforce [that] our base services will remain free.

SA: In an economic slowdown, why should an organization move forward with your solution as opposed to other marketing solutions?

AG: Our solution is free, and it provides both publishers and marketers with intelligence that can help them drive their bottom line. Publishers can generate much better audience data to empower their sales teams. And that increased audience understanding drives higher value to advertisers. At the end of the day, we are focused on helping buyer and seller transact in more sophisticated ways, a reality that should drive results for both sides of the marketplace. We believe that's a pretty compelling offering at a time when buyers and sellers are looking for enhanced ROI.

SA: What are the big opportunities for your business, your customers, and the market as a whole in 2008?

AG: Our biggest opportunity is to keep engaging the marketplace and delivering innovative services: educating people about addressability and helping them understand how our collective business is going to change over the next decade. Some of those changes will happen in the short term, some long term. We're moving from a unit-based media economy to an impression-based world. It's going to have dramatic impact on how we think about audience activation, segmentation, and ROI.

We have seen a terrific adoption curve by media companies over the past 18 months. Everyone from CBS, NBC, and Fox to major publishing houses like Time Inc. and IDG. And we've seen Web-only publishers adopt at very high rates, everyone from PerezHilton.com to Huffington Post, WordPress, Slide, and Digg.

And since we started talking to agencies and marketers earlier this year, [we've had] just a fantastic response. While I can't talk specifics about which agencies and clients, suffice it to say [that] many of the largest digital shops and marketers are using our marketer-focused services.

Our success is a direct function of focusing on a need and delivering a solution for the publishers' and marketers' ecosystem. We just have to keep doing that, continuing to drive value for our publisher and marketer user base and maintaining our leadership position.

Insights into the Emergence of Search Analytics (Initially Posted on ClickZ)

By Shane Atchison | Jul 2, 2008 2:54:15 PM

In grade school, I really wanted a pair of Velcro tennis shoes. My parents obliged. Shortly after, I decided that I needed a pair of new basketball shoes (before I realized I was too short to succeed at basketball). My parents couldn't understand why I didn't just wear my new velcro tennis shoes. I tried to explain that each served distinctly specific functions. No dice.

While site-side Web analytics platforms are an appropriate, central hub for understanding overall digital performance, many marketing professionals still find it necessary to use complementary analytics solutions for more specific needs. Media and ad trafficking platforms are necessary for measuring display ad and paid search performance. Panel-based analytics are necessary to understand industry and competitive trends and insight. While these tools are all of an analytics nature, they each serve distinctly specific functions.

My Velcro shoes belonged on the street; basketball shoes were necessary to play basketball (or at least look good on the court). I don't want to replace my Web analytics platform, but I need a supplement for specific marketing functions to make effective business decisions.

SEO (define) continues to grow as a legitimate area of focus for enterprise marketing professionals. It's unequivocally the most efficient method for traffic acquisition, compared to paid search or display advertising, where costs scale with increased traffic. This makes SEO especially attractive in a recessed economic environment.

But we need to understand SEO performance at more granular level. Typical, high-level metrics provided by site-side analytics tools, like referring traffic or keywords, don't provide enough depth or breadth to make deep, effective decisions for SEO.

I recently spent some time with Richard Zwicky, founder and president of Enquisite, a search analytics solutions provider. I'm impressed by Enquisite's unique analytics capabilities specific to SEO, and I asked Zwicky to explain how search analytics can be an appropriate complement to a traditional Web analytics tool.

Shane Atchison: Let's start with the obvious question: why are marketers demanding more insight from search analytics?

Richard Zwicky:The budgets, activities, and expectations that drive search engine marketing -- SEO as well as SEM [define] and paid inclusion -- are all on the rise and are increasingly competing with other digital and traditional marketing and advertising spends. Search-specific analytics help marketers be more responsible with their budgets by offering a closer look at specific search performance and ROI [define].

SA: Why run search analytics alongside your Web analytics solution?

RZ:Enquisite focuses on natural search referrals. We're able to provide a level of depth beyond what traditional Web analytics tools offer. But search analytics isn't intended to replace Web analytics; rather, it complements Web analytics by shedding detailed insight on how search efforts specifically contribute traffic to a site and which specific keywords perform on a site. Our data is made actionable only when viewed in the context of traditional Web analytics reports and insights.

SA: Can you share examples of how search analytics are able to provide more "detailed insight?"

RZ:A good example is geotargeting. This is commonly used to inform paid search strategy but isn't commonly achievable for SEO efforts. Enquisite's reporting allows for a highly refined and customizable look at geographic performance from natural search. We show performance by city (and, in some cases, by Zip Code) for each individual search referral. This is especially useful because your site can rank differently in search engine results pages, depending on a searcher's location.

Traditional Web analytics would simply show you that you're getting search traffic but wouldn't explain your site's positioning in the SERPs [define] -- or the fact that it differs in geos. It also isn't going to distinguish the fact that some traffic comes from New York and some from Seattle.

SA:We see companies consistently wanting to view SEO success beyond just rank. Can this be more achievable with search analytics tools like Enquisite?

RZ: This has always been a common challenge, and one of the reasons that SEO and SEM performance have always been an apples-to-oranges comparison. The central problem is conversion. SEM managers are able to view conversion data at a keyword level, but SEO reporting has never really been able to tie specific keywords to specific site-side conversions.

We are soon to release conversion functionality, and we believe that Enquisite will be the first solution to offer detailed conversion tracking and analytics for natural-search-referring keywords. We can also identify which search terms, not currently optimized, have the highest potential for driving future traffic and conversion value.

SA:SEO conversion tracking is a promising level of value and insight that marketers haven't been able to realize from traditional tracking or Web analytics solutions. But it also seems like an opportunity for alignment with traditional solutions. How can that be achieved?

RZ:You're right; the value of providing SEO conversion insight is a core reason that dedicated search analytics tools will emerge as a necessary complement for marketing professionals. But you're also correct in identifying the need for alignment. The sophisticated insights that Web analytics provide in terms of looking at path data, funnel analysis, identifying and defining success events, goals, or conversions are crucial. We simply want to focus on specific insight and performance from search. This should both inform Web analytics and be informed by Web analytics. Search analytics insights can also align with other efforts like behavioral targeting.

SA:What are the big opportunities for search analytics in the next 12 to 18 months?

RZ:There is still a wealth of opportunity for search analytics providers to deliver deeper reporting functionality. Firms will need to focus on reporting functions that really address the decision-making needs of search marketing professionals. Accordingly, we really try to base our primary product focus on user feedback. We try to pay attention both to what our customers tell us and what the tight-knit search marketing community discusses. As important as search marketing is, providers need to realize that it is only one aspect of a larger analytics and marketing effort. Marketers only have limited time and attention to dedicate to search analytics. Firms need to try and avoid reporting clutter that over-delivers on analytics that we think are cool or are just nice to know.

Blogs as Unimaginative Failures

By Jason Carmel | Jul 1, 2008 3:28:59 PM

I'll admit, when my fellow ZAAZ-blog auteur Chris Kerns sent me a WSJ article entitled "Most Corporate Blogs Are Unimaginative Failures," and suggested I write about it, I couldn't figure out whether he was trying to be thought-provoking or secretly implying that my blogging sucks and tempting me to stand in front of the blogosphere in the emperor's new clothes (as it were). If I thought about it too hard, it almost seemed like the blogger-equivalent of telling someone that "gullible" isn't in the dictionary, or muttering very softly under your breath "adillholesayswhat". I'll assume he is taking the high road on this (a n00b mistake at ZAAZ, perhaps).

The article references a Forrester report that accuses most B2B blogs of being little more than refried corporate marketing hyped up as "Web 2.0," with all of the personality and ingenuity of prison food. Barely a quarter of all entries results in any significant conversation, and as a result, far fewer Fortune 500 companies are getting into the game.

I appreciate the good people at Forrester quantifying this, but I can't say it comes as any surprise. I imagine that the decision to get into the blogging game initially for many companies went a little something like this:

Marketing Guy #1 - My niece told me that she reads about American Idol on a blog. Says all her friends do too. Thing gets more visits than Santa's lap in December.

Marketing Guy #2 - Sweet Mary Lou Retton, I love American Idol! And we've been getting slaughtered in the "young tech-savvy people I can't relate to" demographic. We need a blog ASAP.

And so our intrepid Marketing Heroes set out to start a blog, without discussing important things like purpose, measurement of success, voice/tone, sustainability (i.e., do we have enough content for regular, meaningful posts), and ownership/moderation. Three months into the blog, they realize that coming up with posts takes real thought and work, which they can't support, so they phone it in by posting press releases word for word. They get a few negative comments and decide it's easier just to turn off the ability to comment altogether rather than address any criticisms directly. And (perhaps most importantly) since they can't linearly connect the readership of the blog to any purchases or leads, the project quickly loses what little financial and political resources it ever had. As I type this, to be honest, I'm surprised there are any B2B blogs left at all for Forrester to study.

For me, the biggest take-away from this is how a well-timed conversation about goals would lead to a startlingly different result. Understanding why the conversation inherent in a blog is important, how much effort it will require, and how it will be measured is especially important for social web applications to avoid "unimaginative failures."