In order to better understand social aspects of the short-form video watching experience, we investigated the journey to co-watching, from searching and discovering content, to choosing and experiencing videos with others. After identifying, through a large-scale survey, some of the most typical situations that bring people to YouTube, we deployed a one week-long diary study with 12 participants in which they performed a set of frequent video tasks at their leisure, half by themselves, and half with someone else. Following the diary study, we had participants reenact the diary study tasks remotely with the experimenter. We observed that users face multiple challenges on the journey to co-watching a video. They must share a device designed for an individual, use different methods for selecting videos than when by themselves, negotiate or turn-take in order to make a decision, and potentially watch a video that they do not enjoy. Along this journey, users must engage in impression management to consider how their choices might make them appear to others. We present design recommendations for remote and collocated co-watching to improve the social video watching experience.