Sound Design For Visual Media - Tutorial Notes Ep03
Episode 03: Spotting a Video & Creating a Road Map for Video Projects
So, we're hired as sound editor in a visual media project, we have a video, we have a template, what's next? In most productions, what happens next is what is referred to as a Spotting Session. Spotting simply means watching a video and taking notes. This would the first conversation you will have with a filmmaker, producer or editor about the project. You will meet in person, or maybe on Zoom, and you will watch the film together. This is your chance to get inside the filmmaker's head, as well as into the nitty gritty details of the video. So far you may know the overall story, and maybe you're even familiar with the director's style and philosophy, but now you get your first glimpse of a cut of the video, and get to hear the director's perspective in depth.
In larger episodic projects, a spotting session will provide you with context. You may have been brought on as the sound editor on a long-running series with 400 episodes. You are not expected to watch all these (though do your best to watch as many as you can obviously) but in the spotting session you can ask questions about the project's past:
-What is the overall storyline?
-What past events are relevant to this episode?
-What is the backstory for different characters and locations?
-What kind of Sound Design techniques are part of the series' language?
In stand-alone projects, you are still provided with tons of context and subtext that may not be easily apparent from a viewing of the project. The directors can fill you in on the intention of scenes, what certain colour-grading treatments say about the vibe of the scene, what motivates the characters and so on.
Something you should NEVER forget is that, as a sound editor, you are still a filmmaker, and the aim of your work is to tell a story. The only difference between you and a video editor or DOP, is the medium with which you convey the story, aka sound.
Finally, a spotting session, especially as a composer, is crucial to understanding the tone of each scene, as well as hopefully provide you with some sources of inspiration. In the last film I worked on, the director gave me a Spotify Playlist with various pieces of film and stand-alone music, that helped guide my orchestration, instrumentation, overall palette, and gave me a reference for how my composition needed to sound and feel like.
If you want to watch the movie I spotted in the video this week, you can watch it here.
How I Spot a video
The Classic method of spotting is to simply put the video on Fullscreen, grab a pen and paper, and jot down timecode areas and other notes, to be discussed afterwards. The way I do it is a bit more high-tech. I'm still taking notes with a pen & paper, but if I'm behind my desk, I would take advantage of Reaper Markers and my Spotting Screenset:
My Spotting Screenset in Reaper. Video takes up most of the screen, but I have my region/marker Manager on the right and I see a bit of my timeline below.
If the spotting session is happening somewhere else, I would jot down notes and then go home and import these as markers. Otherwise, I would simply take most of the notes on Reaper, for future reference. Before I start the spotting session, I will copy my video onto track 3 of my film editing template. You can download my templates in last week's blog post. Once I do that, I select that item, and hit play.
These are my goals for the first Spotting Pass:
Play the video in its entirety, uninterrupted.
Mark All Scene Changes By splitting Reference Video
Mark the climactic scenes, and any scene where I have a question.
As I watch the video, my right hand is holding my pencil, and my left hand is on letters 'N' and 'M'. 'N' is assigned to the command 'Split selected item(s) at Play Cursor, and 'M' is a default Reaper Hotkey for 'Insert Marker at Play Cursor, without opening dialog window'. So wherever there's a scene change, I would press N, and wherever there's a climactic scene or any spot I'm foggy on, I would press M.
Note: Scene Change is not to be confused with an Angle Change. An Angle change can occur multiple times in the same scene, for example, cutting to close-ups of speaking characters, interspersed with a Medium shot showing both. We're not particularly interested in marking all of those at this time. We want to mark where the setting changes, like from an Interior House Scene to an Exterior Street Scene. You will see why in a second!
Bear in mind that our scene splits will not be perfect. As a scene change occurs, your brain notices it, sends a signal to your hand, which then presses N. This may take a few frames to happen, but for now it's good enough. We will later finesse those. Similarly when I press M, it's often a few frames after the spot I put it on, but should get us close enough to the overall area of the video where I have a question.
Now that you have placed your markers, you have to go through them one by one, and discuss them with the director. Some important questions to ask:
-I don't understand this scene. What's being shown here? What's the aim of this scene?
Now that we understand the aim of the project is to tell a story, sometimes it's unclear what the aim of a scene is. We may be seeing a close-up of an object, without knowing why it's important, or we may be seeing a conversation but are unclear as to what it's trying to convey to the audience. The director will be able to clear up these matters for you!
-Are there things we want to foreshadow with sound prior to this scene?
This is arguably the most important question of all. What you need to remember, is that by the end of your sound editing job, you will have seen the movie, frame by frame, hundreds of times! With time comes an understanding of the film that most audience members will never attain. This is good and crucial for a thorough editing job, but it has the downside of making you lose touch with that first viewing experience. Things you miss the first time you will have gotten by the 10th viewing, and there are things you understand that few audience members will catch.
It's important to know what type of information you need to give to, and what type of information you need to withdraw from, your audience as you edit. For example, let's say we are editing the movie 'Inglorious Basterds' by Quentin Tarantino. (SPOILER ALERT)
Now if you've seen the movie, you will know that the opening scene ends with Landa finding and killing the family hidden beneath the floors. As the sound editor, you will know this by the end of your edit, so you will be aware that there's a family hidden beneath the floor, and there are Nazi Soldiers with guns stationed outside the hut. The question becomes, do we want to hint any of this to the audience? Should we cut some sounds of floorboard creaking below? Should we add sounds of german soldiers conversing, maybe reloading their weapons and so on? (END OF SPOILER)
Opening scene of 'Inglorious Basterds' (2009)
This is a question only the director can answer. Realism doesn't dictate the sound of movies, the story does. So it's up to the director whether we want to provide these hints. If you don't know where the directors stands on scenes like these, you may edit the scene in a way that doesn't help the story along, and may even provide unwanted spoilers to what's coming up. The last thing you want to do is edit a whole sub-story in the background, only to be told by the director to lose it, time you could've spent on another scene...
-Will you want music for this scene?
If you are the composer, this is obviously important, but if you're not, this question still immensely affects how you will edit. If you know a certain scene will have loud epic music over it, you will pay less attention to thoroughly editing BGs for the scene. Conversely if you know a scene won't have music or dialogue, that scene is your chance to let the BGs shine!
-What type of sound design is needed for this scene/character?
Sound is a great way of signifying the presence of certain characters. A subtle growl-like sound and a creak in the second Harry Potter movie can indicate the presence of a Basilisk. All robots and mythical creatures, sometimes even specific guns and cars, come with their own signature sound. You need to discuss each of these with the director, to make sure you are realizing their vision of the character.
More to the point, sometimes by the spotting session, these characters haven't been fully visualized by the VFS team, so you may simply be watching a mo-cap human moving around the scene, and obviously the director needs to tell you what that will be replaced by.
-What can you tell me about the environment?
This plays a little bit into the stuff mentioned in the foreshadowing question. Say you are watching a scene with two characters conversing in a room. You need to ask the director where they are and what's going on outside. Are they out of town? Or in the middle of a warzone? Are they in New York City or a small village? These questions should inform the way you design the background tracks and specs.
These are some of the most common questions you shouldn't forget to ask, but otherwise, ask as many questions as you want! Directors never mind talking about their project, and they often provide you with so much insight and inspiration doing so. Directors are also busy, meaning that while you may be able to ask follow-up questions later, the director may have little time to spare after the spotting session to give you feedback, so use this time wisely!
As I go through my markers and ask questions, I can take notes on paper, but I also put a summary of my notes in the marker name for quick reference at a a glance.
Creating a Road-map of the Project
Markers & Regions in a Spotted Project
Other markers I may have created as a note to self. If I see a particular object like a vintage phone, or an animal, I will place a marker there to remind myself to make sure I have the appropriate sounds. Elsewhere, I may place markers on scenes I'd like to start composing/editing first. You almost never start cutting from the first frame and working linearly. You will start at the most climactic scenes, then work your way through them, and then fill in the gaps. This is great because you will spend the most time on the most important scenes, and then you will manage the remainder of your time and distribute your effort among the remaining scenes.
As a composer, scoring the main scene(s) may provide you with thematic materials that then inform all the rest of the film. Themes you can foreshadow, sound palettes you can use in other scenes, and so on.
Once my markers are in place, I will add these prefixes to their names:
For musical cues: MxQ
For Sfx cues: SfxQ
These prefixes will trigger my Auto-Coloring system to re-colour them accordingly. This is useful because while I can always see my markers on my timeline, I can also sort them by color so I can browse through them by type. I have made a video specifically on this subject for you to check out if you're unfamiliar with the SWS Auto-Color Feature.
Next, we will focus our attention to the splits we made in the reference video by pressing N (or however you'd like to split your video) and we're gonna finesse them in the correct spot. I'll go through them one by one, and place the split on the EXACT frame where the scene change has occurred. Simply set your grid value to Frame, and use ',' and '.' to nudge the edit cursor. When you find the right spot, nudge your edit to that frame.
Once I have all the splits on scene changes, I will select them all, and run the command 'Create regions (With tail) from Selected Items' and set the tail to 0.05sec. I will explain later why I do this. Running this command will now create a number of regions in the project, each the length of one scene. I will go through them one by one and name them in this format:
Sc. (number) (Description of Scene)
Sc. Stands for Scene
The number indicates scene number
The Description could be anything. I use 'Int.' for Interior shots, 'Ext' for Exterior shots, and otherwise just indicate if the scene is a montage, time-lapse, VFX sequence, or whatever. I will also write in any signifiers that help me recall the scene's location and time. An example is:
Sc. 3 Int. Vanessa's House, Night
Once I name and number all scenes, I will go through and I will colour the same scenes similarly. If we go to Vanessa's House multiple times in the film, say at scenes 3, 6, 12, 19 & 32, I will simply sort alphabetically, select all the scenes and give them the same colour. I will explain why in future videos.
Scene-Specific Guide Tracks
Now I can once more select all items in my Reference Track, name and colour them by their regions, which now have the scene name and colour on them, thus creating Guide Tracks for specific scenes. There are a ton of ways I use these, so once again, I'll cover some uses of this in future videos.
Useful Hotkeys and Commands to Navigate your Road-Map:
The first set of hotkeys very useful for navigating markers are the numbers 1-9 each corresponding to the marker of the same number. Press any of those and your edit cursor will jump to that marker. 0 is for marker 10. Additionally, I have assigned Control+1-0 to jump to Markers 11-20. From there you can create additional hotkeys, or you can open the Jump window (Default hotkey 'J') to open a dialog box where you can input sequential hotkeys to navigate to markers and regions:
Jump to time/marker/region Dialog Box
Here you can type m plus the number of your marker (so jm22 to go jump to marker 22) or r plus number of your region (so jr8 to jump to region 8) and quickly navigate to that marker or region. Regions by default don't have hotkeys assigned to them, and I like to keep it that way. Once you are fluid with these sequential hotkeys, navigating through your project should be easy enough, and arguably way easier than memorizing other hotkeys to jump to one specific region.
And there you have it! You have now spotted your video, and created a quick road-map to navigate through your project. Just imagine working on a 90-minute scene. You could have 30+ scenes, and may need to compose/edit upwards of 200 cues. Having a way of quickly navigating through them is crucial in maximizing the time you spend on the project, and not through scrolling through your timeline looking for that one spot where a gun goes off.
Timeline of a movie project with a road-map
Where to Find Videos to Edit
Finding appropriate videos for your first (or next) editing projects as you learn is important. Firstly, you want to find a video that's appropriate in length and difficulty for you, but also you wanna make sure you are working on videos that won't get flagged as copyright infringement if you want to publish them.
For your first project, while you may be tempted to start with an epic sci-fi animation with tons of action and lots of mythical creatures, this may not be the best strategy. Not only you will find doing it difficult, but also harder projects require bigger and more expensive libraries and more sound design prowess. Furthermore, as you begin your career in sound editing, chances are, you won't be offered too many gigs in a sci-fi/horror/period piece type of videos. Those are the sound editor's dream, but as a beginner, you will very likely be put in a simple, short, drama or comedy project with mostly dialogue and very simple environments. It's important to use these as a stepping, but also learn some ways of making them interesting through your sound design and editing. Anyone can make a spaceship battle marginally interesting, the real challenge is taking a short film about a man walking his dog and make that interesting! That's just my personal advice. My first project was a short film where the majority of the runtime was one therapy session. Literally the most exciting scene was a conversation in the car as it was raining.
So a few places to find videos are:
-YouTube: Go on Youtube, and search for a short film or movie. You can additionally run a search like this:
the ',creativecommons' (one word, no caps, no spaces, and the comma is crucial) suffix will limit your search result to videos with a creative commons license, meaning they can be redistributed and edited legally as long as proper credits are provided. You can find tons of videos you can use as your first project and you can then upload and publish them with no legal consequences.
-Vimeo: Vimeo also has a large library of smaller films and cartoons you can use. Make sure these videos have creative commons licensing. Alternatively, if you found a video you really love, simply find the author and write them an email, asking for permission to do a sound-redesign. Most creators won't have a problem with this (if their publisher isn't an issue) and will gladly give you permission to do this.
-Public Domain & Open Source Videos: Videos made 50 years or longer ago are in the public domain (this is different from country to country but 50-100 years is standard) meaning their copyright IP has expired and they now belong to the public. You can find a large database of these here. There are tons of older cartoons, NASA footage, and other public domain documentaries and movies you can use with no legal consequences and free of charge. Open source movies are made more recently, but still don't have any copyright protection. These movies are meant to be reproduced and edited to your heart's content. Blender, an animation software, has tons these here.
-Fuck the Police: This approach is possible too, though I won't go as far as personally condoning it, though I will confess that I have done this in the past! If you don't plan to monetize your video, and don't plan to publish it in a large-scale manner, basically do whatever you want. Many video hosting platforms (Like Vimeo) don't regulate their content unless a report is made and evaluated, but also, most content publishers are likely not to go after you, a starting sound editor, for using their video. My reel piece is full of stuff I didn't secure permission to use, and still it's been on my website for a year and no one has flagged it. You can watch my reel piece here. I also did a video redesigning the sound from a Rick & Morty sketch and even put it on Youtube. YouTube is more tricky as it has bots that automatically scan and flag videos, but once again, if you are not monetizing them, you won't have a problem!
So that's it for today! See you next week when we will finally begin editing! Watch the episode here: