The Shelf Life of an Observation, Part 1: Users Change

I’m going to begin my first UX Sears blog post (hurray!) with a shocking confession: Way, way deep down, I have no desire to create wireframes, program anything, or make prototypes.  I can and have done these things, but they aren’t what gets me out of bed in the morning.*  At Sears I do the one thing I love more than anything: Finding Out.  Finding Out what people do when they encounter a particular interface.  Finding Out why.  Finding Out how it’s possible for someone to miss an icon that seems so obvious to everybody involved in its creation.  Yes, I am a Researcher, through and through.  As such, I spend a lot of time thinking about what makes this job challenging.  Over my first three blog posts I’m going to tell you all about one of the top things I ponder regularly: The Shelf Life of an Observation.

image of expiration date on a can

“Shelf Life?”

By the “shelf life” of an observation, I mean the length of time that I can continue using an observation to draw a relevant conclusion.  I’m essentially asking, “If I run this exact test with the same stimuli a year from now, will the results be the same (taking sampling error into account)?”  Given the blazing rate of change in the tech and internet worlds, the answer is often no (but not always, and that’s really important—more on that in Part 2).

Users Change

Why is the answer so often no? Users have expectations and a wide variety of past experiences that affect the way they approach the things we build.  And these change over time as people are exposed to and get used to new technologies, interaction paradigms and design patterns (By the by, check out Beth Lankin’s post on patternality).

For example…

The other day I was supposed to call my dad during the time he was planning to be in his office, so we could talk about spring break plans.  He told me he’d be there until around 1:45 his time, which means the time in Tucson, Arizona.  So at some point during the day I thought, “Shoot…they are two hours behind us, right? One? Did they spring forward?  Isn’t Arizona in two time zones? Ugh, why can I never remember this??!”

Once upon a time, I would have approached this problem thinking about time zones.  I would have wanted to know what time zone Tucson was in, so I could do the math.  I might have flipped to the little time zone map that was always in the back of my paper planner (remember those?) and checked it out.  Or, going the web route, I might have typed in “U.S time zone map” for a digital version, or searched “Arizona time zone.”  But the past few years of my experience with ever-evolving search engine technology led me to type, “what time is it in tucson arizona?”

image of google search results showing the time in arizona

I asked and Google answered.


Any one of those other queries would have led me to the information I needed.  What has changed is my expectation that if I type a question like that, in regular human language, the search engine will be smart enough to simply give me the answer.  I know, without any doubt, that I will not need to click any links to web pages, nor do any math.  Sure enough, Google returned the answer, called out prominently in a box at the top of the results page.  This could seem to be a subtle difference in my thinking, but it isn’t so subtle.

In the past I would have been trying to think of the best terms to use to get me to a web page that would tell me the time zone or the current time in Arizona. Nowadays, I don’t expect to go to someone’s website.  I don’t spend time deciding what the best search terms will be.  I ask a question the same way I would ask another human being in order to get the answer delivered to me. We might forget sometimes just how huge this is because we are so used to it now, but it’s a big deal.

The Point Being…

Someone doing research on folks using Google to search not too long ago would have different findings than someone doing that same research now.  Not only because the page literally has a different appearance now, but also because our queries are different.  Not just the types of things we’re looking for, but the patterns in how we construct our search strings.  They are different now because our expectations of search engines are different.

image of google search results showing the time zone for arizona

For me, the researcher, the expected shelf lives of certain findings I highlight in a report can be mysterious enigmas.  User behavior patterns I observe in a usability study, for instance, include facts about how the interface works, infused with a snapshot of what’s going on in people’s minds right now.  It’s hard to know a priori how long things will remain that way.  This is why market research is an ongoing process (the market changes), and why good design and ux research is as well (your users change).

Developing an intuition about which observations are likely to continue popping up for a while and which are likely to become less common in the next year or so begins with getting really clear on the why when it comes to explaining the observation.  It begins with separating “This interaction is new to me” from “This doesn’t help me do what I want to do right now” from “My human brain really just can’t handle this.”  The latter of these is the topic of Part 2. Stay tuned…

Go to Part 2 now.

*Click here to see the wonderful thing that does.

Tags: ,

3 Responses to “The Shelf Life of an Observation, Part 1: Users Change”

  1. Chad Cline →
    March 14, 2014 at 7:24 am #

    Great article, well written. Look forward to the next one. The *annotation was fun too :).
    The bit about Google syntax/pattern made me think of a bit from a sitcom where an older lady was new to the web and would start all her searches with “Hello Google, I would like to know…” This is amusing to those of us that have learned to “play by the rules” over the years, as there are a lot of “unneccessary” words in there. However, the new MotoX (“ok Google Now”), Siri, and Google’s search engine (as you highlight) actually really close to these more human syntax/patterns…

    • Devin Carter →
      March 14, 2014 at 1:57 pm #

      Thanks, Chad!
      It’s funny because even though I know I don’t need all those extra words, the experience feels better if I can type it out like that.
      As technology has gotten more advanced and complicated behind the scenes, it has become more inconspicuous to the user. Where being able to see lots of bells and whistles and knobs used to signify the cutting edge in features, now the ultra sleek and subtle means “high tech” to us. I think that particular collective shift in what “advanced” looks and feels like was a necessary one in order for flat design to become as big as it is.

  2. Jayne Spottswood
    March 17, 2014 at 5:01 pm #

    Great post, Devin! I’m curious to know how audio-based search factored into the experience you described. My theory? Because intelligent personal assistants (like Siri) allow for speech-to-text with natural language to translate into meaningful results, users may increasingly expect the same functionality from all search engines.

    To your latter comment, I agree it’s probably not the most efficient method to ask a question. But does being able to ask a search engine a question the same way you’d ask your friend also have the potential to make you perceive that search engine as “friendly”, and the search experience more positively overall?