Mid-way through my internship I was approached by one of my mentors and recruited to conduct a highly sensitive internal intercept study. The purpose of the study was to get feedback on proposed major design changes to the YouTube site. The request for feedback came from upper management, in order to understand how the design decisions that had been made to that point would be received by users. Due to the sensitivity of the changes non-Googlers could not be considered for this researcher and instead a population of non-YouTube Googlers was chosen.

Objective

• The overall objective of the study was to get feedback on the design changes from employees outside of the YouTube org.
• These changes were sufficiently advanced, and this study was viewed by management as a way to validate the design decisions which had already been made.

Process

• Working with designers I gathered several different versions of the branding, including the current version, the proposed change, and several alternate versions which served as 'red herrings'.
• Using these designs I devised a type of A/B test where I would show participants the various versions of the branding, in a randomized order, to get feedback on the new branding design.
• I traveled to a Google office in another state in order to recruit participants outside of the YouTube org structure.
• The study took the form of a series of informal intercepts with a dozen participants at cafes and kitchens across the Google campus.

Challenges

• The study was commissioned at short notice and so I had to design and implement the study with only 2 days notice before travel.
• Creating a fully randomized study in order to minimize bias was a challenge.
• Finding a dozen full-time Googlers without exposure to the YouTube org also proved to be a challenge.

Outcomes

• The interviews discovered a number of valuable insights into the design changes.
• Due to this feedback a number a significant recommendations concerning the re-design were proposed to management.

Objective

• Following the results from the initial intercept study, management requested a follow-up study.
• The objective of this follow-up study was to confirm the results from the initial study and gain further insights into some specific feedback those users provided.

Process

• I utilized the same techniques from the first round, including randomization to reduce user bias.
• This study was performed as an intercept study at the Mountain View campus, users were approached in cafes and kitchens but were screened to make sure they were outside the YouTube org.
• In addition to the same A/B test performed in the first round, specific follow-up questions were asked when users provided similar specific feedback to those users in the first round.

Challenges

• Finding a sufficient number of Googlers in Mountain View who had both the time to be interviewed and were completely outside of the YouTube org was very challenging.

Outcomes

• The follow up interviews confirmed the results of the initial study, with additional insights gathered from questions added to this set of interviews.
• The results of this follow-up along with the results of the initial study were communicated to back management. Changes to the final rebranding effort were made based on this feedback.

This study provided me with valuable insights into performing internal intercept studies effectively, as well as with conducting highly sensitive research with a large potential impact. It is very unusual for an intern to be trusted with research with both this level of potential impact and sensitivity (this research was internally confidential and I was not permitted to share details even with other YouTube employees - I can only share such details as this now as the re-design has gone live), which demonstrated the trust that was placed in me by my mentors, as well as their opinion of my competence. Although this research was commissioned in a short period of time, it was very effective in changing the course of certain design decisions in the re-branding effort, which had a lasting impact beyond my internship.