Super Volume

If you’ve arrived here from the Super Volume page thank you for watching, I hope you enjoyed it (otherwise you can watch it here). You’ll find more information about the project in this post. An ongoing (hopefully growing) list of the films and television shows included in the supercut can be found at the bottom of this page.

Super Volume is a randomly generated supercut collecting instances of volume manipulation from film and television. Each time the web page is refreshed (or the Stop/Randomise button is pressed) the existing database of clips is reshuffled and a new version of the supercut is loaded into the video player. There is no rendered or final version of this supercut, rather it is designed to be added to as and when new clips are found and loaded into the database. This version of Super Volume is accompanied by (but not set to) ‘Gut Feeling’ by Devo. A ‘mute’ option is included on the page so that viewers can listen to music of their own choosing (or no music at all) whilst they watch.

Super Volume is the first part of a larger project to explore the specific relationship between the on screen action of volume manipulation, and the reciprocal manipulation which this might precipitate during the post-production sound mixing process, namely the turning up (or turning down) of volume. Functionally then, this supercut is as much about providing me (as a researcher) with easy access to the material I am working with, as it is about creating a new videographic work. But Super Volume is also an exploration of the process of creating a videographic work, of defining and designing an appropriate presentational mode, where publishing on Vimeo (where my previous videographic works all reside) might prove to be a limiting factor.

“Whether one thinks about the supercut as a database or a collection of images and sounds, it implies a process of aggregating and sorting that has no beginning and no end and that could continue indefinitely as long as there were new additional sources.”

Allison De Fren, 2020

There is no theoretical limit to the number of clips which can be added to the Super Volume database, but the intent here is not to be exhaustive, rather to be incremental. This particular mode of presentation means that the supercut is a perpetual ‘work in progress’, with no effective beginning or end, and no set length. My goal is that the project will grow through future collaboration as I develop other strands of research alongside it. The generative nature of this presentation mode also serves to curtail my creative impact on the final video. The process of careful editing and synchronisation which is often so intrinsic to the videographic form, and the supercut in particular, is stripped away here. The task of editing is reduced to ‘topping and tailing’ clips and nothing more (and as you will see, for the most part these clips have been quite tightly edited to the action). Whilst this does nothing to show off my editing skills it does make it considerably easier for me to watch (and re-watch) the piece. I have very little creative stake in how the work comes together, so I am inured against the agonising process of second guessing my editing decisions as I watch it back. Rather, I can adopt something of a duel position as both a cinephiliac, indulging my fetishisistic impulse to collect these clips, and as a dispassionate viewer, able to “maintain an objective distance” (De Fren, 2020) from this particular object of study, precisely because I have no control over the form it will take each time I hit play. Of course the lack of any agency on my part in the editing of the piece does increase the potential for it to seem ‘cold’ (Bordwell in Kiss, 2013). What labour might have gone into the sourcing of the clips, and the coding of the video player is not replicated in the careful editing, sorting, and syncing of the final videographic work. It remains to be seen whether other viewers take anything away from the viewing experience, or feel the need/desire to explore the potentially endless random variations available to them.

I have already watched Super Volume a lot, at first to refine the code for the video player (more on the technical aspects of this below), and then to explore just what the random nature of the supercut might reveal to me. I intend to write more on this in the future, but initially I find it interesting to note how differently each hand approaches this seemingly simple task, disconnected as they are here from any narrative context (and body) which might clue us into their motivations. Some are hesitant, questioning, whilst others are definite, casual, happy even? Most are white and male, which I could suggest is indicative of the nature of the films and television shows where I have harvested these clips from. But implicit in that suggestion is my own potential bias, directing me to specific sources in search of these clips. Either way I am in need of a much broader sample of clips before I can make any meaningful analysis, and I am hopeful that the open and ongoing curation of the project will help with this.

The Super Volume player is based on this code by Ben Moren. Ben was kind enough to help me out with a few tweaks to the code despite it being quite a few years since he wrote it. Getting a video to play back in a modern web browser is a reasonably straightforward process, but finessing the functionality for Super Volume took quite a while (largely because I am not a coder). The Super Volume player is actually 2 video players stacked on top of each other (thus it takes a little longer to load than a normal web page). The transition between videos clips is the 2 players swapping places, one moving to the ‘front’ (in Z space) whilst the other loads the next video. The supercut will always start with the same 2 video clips so to avoid obvious repetition, I have added two short, blank videos as clips 1 and 2. Ben’s original player code already handled the random shuffling of the clips into an array for playback, but it needed a small tweak to stop it playing the clips more than once in what was essentially an endless loop. Now the video player will stop after the final clip in the database is played (though the music will continue). The Stop/Randomise button re-loads the webpage, running the code again resulting in a newly shuffled array of video clips, and a new version of the supercut.

Special thanks to Alan O’Leary, Ariel Avissar, and Will DiGravio who provided invaluable usability testing and feedback on the first version of this project.

This is the first part of a larger project which I anticipate will run for at least 12 months and more on that will be published through this blog as it comes to fruition. If you have any thoughts, comments, suggestions, or questions about Super Volume please email me at cormac@deformativesoundlab.co.uk

FilmNumber of Clips
Upstream Color (2013)1
Talk Radio (1988)1
Spider-Man: Into the Spider-Verse (2018)2
Back to the Future (1985)4
Bill & Ted’s Excellent Adventure (1989)3
Studio 666 (2022)1
Guardians of the Galaxy (2014)1
Caddyshack (1980)1
Berberian Sound Studio (2012)8
Airheads (1994)2
Bill & Ted Face the Music (2020)1
Bohemian Rhapsody (2018)1
Blow Out (1981)1
Deadwax (2018)1
Spiderhead (2022)1
The Gray Man (2022)1
The Last Word (2017)1
Things Behind the Sun (2001)1

References

de Fren, A., 2020. The Critical Supercut: A Scholarly Approach to a Fannish Practice. The Cine-Files15.
http://www.thecine-files.com/the-critical-supercut-a-scholarly-approach-to-a-fannish-practice/#_edn10

Kiss, M., 2013. Creativity Beyond Originality: György Pálfi’s Final Cut as Narrative Supercut. Senses of Cinema, (67).
https://www.sensesofcinema.com/2013/feature-articles/creativity-beyond-originality-gyorgy-palfis-final-cut-as-narrative-supercut/

Other reading/watching

Meneghelli, D., 2017. Just Another Kiss: Narrative and Database in Fan Vidding 2.0. Global Media Journal: Australian Edition, 11(1), pp.1-14.
https://www.hca.westernsydney.edu.au/gmjau/wp-content/uploads/2017/04/GMJAU-Just-Another-Kiss-Narrative-and-Database-in-Fan-Vidding-2.pdf

Tohline, M., 2021. A Supercut of Supercuts: Aesthetics, Histories, Databases. Open Screens, 4(1), p.8. DOI: http://doi.org/10.16995/os.45

The State of the Lab

I presented a paper last month at the Screen Studies Conference 22 on sonic perspective and the moving camera in Panic Room. Part of the presentation included my re-imagining the film as a ghost story, and I couldn’t resist making a trailer for that version of the film.

It’s been quiet in the Lab over the last few weeks. Various external factors (such as the above conference) account for this, but also the fact that, at some point, the experimental process will (should? might?) reveal a productive way forward for a piece of research. As such I am currently working on a new video essay employing some of the ideas from the Sound Stack experiment which I will be presenting at the Theory & Practice of the Video-essay Conference taking place at UMass in September. And I’m also preparing to launch Super Volume, a larger project which will be the first ‘published’ output from the Lab, and will run for the next 12 months or so. More on that in the next few weeks (with any luck).

From the Next Room

A new video experiment listening to a film playing in another room.

This was a trick we used to use in the recording studio; listening to a music track we were mixing from another space, essentially overhearing the music as it played in another room. It was an easy way to ‘disconnect’ from the process of working critically with the material, whilst at the same time engaging a different mode of listening, encouraged by the physical distance from the material, and the manner in which that distance reshaped the sound.

I set out to re-create this experience for a film and I felt that this scene from The Shining worked quite well, given how rooms feature so prominently in the film. The image I have used for background is taken from For All Mankind (Season 2, Episode 10), one of my favourite things on TV right now.

I decided against recording this overheard conversation for real (though I have done soundtrack re-recording in the past) but instead chose to use an Impulse Response to simulate the space of the room. An Impulse Response is essentially a sample of the acoustic characteristics of a particular space. To capture a response, a burst of sound or a sweep tone is played into the space and re-recorded. This recording is then processed to leave only the sonic characteristics of the space. The resultant Impulse Response can then be applied to any sound to give the impression of it existing in the sampled space. For this video I used a response from the wonderful free library A Sonic Palimpsest made available by The University of Kent. The image above shows a picture of the 2nd floor landing in the Commissioner’s House at Chatham where this particular impulse response was recorded. IR Dust is a software plugin which allows me to apply the response to the sound in my Resolve editing session.

I spent quite a bit of time experimenting with various impulse responses from this pack to tune the sonic spatialisation to my visual perception of the room (something which film sound professionals do all the time). Whilst I am happy with the ‘fit’ (marriage?) between sonic space and visual space, I’d be interested to hear how it sounded to you.

Looking for Alternatives

No video this week, but rather some brief thoughts on my current investigations into alternative video platforms for publishing videographic work.

This is a snippet of script from a new idea I’m working on. A few posts back I mentioned the presentation of the deformative work, and I’ve continued thinking on that. In this particular instance, I’m looking at how I can create a bespoke video presentation format which is also an integral aspect of the work. It means creating a custom video player hosted on my own website (and learning some of the coding that goes with that), but the upside is a highly customisable solution which lends itself quite readily to the presentation of audio visual experimentation. I’m not intending to abandon Vimeo just yet, if offers a lot of advantages for publishing video essays, but I’m curious to see what other experimental outputs might be inspired by the flexibility that comes with not publishing on Vimeo.

Noisy Jaws

This video (mis)uses the rather excellent RX suite of audio restoration tools created by iZotope

The following description taken from the iZotope website just about covers it;

Deconstruct lets you adjust the independent levels of tone and noise in your audio. This module will analyze your audio selection and separate the signal into its tonal and noisy audio components. The individual gains of each component can then be cut or boosted. 

iZotope.com

For the soundtrack to this video I’ve used the aptly named Deconstruct module to cut (lower) the tonal elements in the soundtrack as much as the software would allow me, whilst boosting the noisy elements by a small amount (too much boost would risk introducing digital distortion). I find the results fascinating; hearing where the software finds tonality (voices are affected, though Roy Scheider’s less than the others) and where it finds noise (opening and pouring the wine). Robert Shaw’s singing is reduced to a buzz whilst the tender moment between the Brody’s is lost in the cacophony of the dock.

To select the clips for this video I used Nicholas Rombes 10/40/70 as my parameter, taking 2 minute clips from the 10, 40, and 70 minute marks in the films running time (see Jason Mittell’s Deformin’ in the Rain for more on this). In another piece (Videographic Criticism as a Digital Humanities Method) Jason refers to the film as viewed in the video editor as an “archive of sounds and moving images”. For this video I’ve extended that archive a little by using the original mono mix of the Jaws soundtrack, sourced from one of the Laserdisc releases of the film. and synced up with a more recent Blu-Ray image track. I suppose that in itself is a deformative practice of sorts…

Internal Logic

This experiment uses Python code adapted from a number of sources (see below) to create a film ‘trailer’ based on the sound energy present in the films soundtrack.

In her chapter on ‘Digital Humanities‘ in The Craft of Criticism (Kackman & Kearney, 2018), Miriam Posner discusses the layers of a digital humanities project; source, processing, & presentation. With this experiment/video I’m stuck on the last one. I get the sense that there is a way to present this work that might expose the “possibilities of meaning” (Samuels & McGann, 1999) in a way which is more sympathetic to the internal logic of the deformance itself. A presentation mode (videographic or other?) which is self-contained, rather than relying on any accompanying explanation. So, more to be done with this one. Very much a WIP.

A brief note on the process

The code for this experiment is adapted from a number of Python scripts which were written to automatically create highlight reels from sports broadcasts (the first one I found used a cricket match as an example). Links to the various sources I’ve used are below.

Become a video analysis expert using python

Video and Audio Highlight Extraction Using Python

Creating Video Clips in Python

The idea is that the highlights in any sporting event will be accompanied by a rise in the ‘energy’ in the soundtrack (read as volume here for simplicity) as the crowd and commentator get louder. The Python script analyses the soundtrack in small chunks, calculating the short term energy for each. The result is a plot like this, which shows a calculation of the sound energy present in the center channel of Dune’s sound mix.

I’m not entirely clear what scale the X axis is using here for the energy (none of the blogs get into any sort of detail on this) but as the numbers increase, so does the sound energy. The Y axis is the number of sound chunks in the films sountrack with that energy (in this case I set the size of the chunks to 2 seconds). To create the ‘trailer’ I picked a threshold number based on the plot (the red circle) and the code extracted any chunks from the film that had a sound energy above this figure. Choosing the threshold is not an exact science so I tried to pick a figure which gave me a manageable amount of video to work with. A higher threshold would mean less content, a lower threshold would result in more. Note – The video above features the films full soundtrack, but the clip selections were made on energy calculations from the center channel alone.

I am not a coder!

I’m not going to share the code here. It took an age to get working (largely as a result of my coding ignorance) and I can’t guarantee that it will work for anyone else the way I’ve cobbled it together. If anyone is interested in trying it out though, please get in touch, I’m more than happy to run through it on a Zoom or something.

Kackman, M. and Kearney, M.C. eds., 2018. The craft of criticism: Critical media studies in practice. Routledge.
Kackman, Mary Celeste Kearney

Samuels, L. and McGann, J., 1999. Deformance and interpretation. New Literary History, 30(1), pp.25-56.
https://www.jstor.org/stable/20057521

Dodge This

This experiment goes back to some of my earliest thinking about sound and the (still) image. This was only made possible thanks to a new version of the audio software PaulXStretch which turned up recently.

This frame scan from The Matrix I’m using here was posted on Twitter some time ago (many thanks to John Allegretti for the scan). I couldn’t resist comparing this full Super35 frame with the 2.39 : 1 aspect ratio that was extracted for the final film. (I also love seeing the waveform of the optical soundtrack on the left of the frame).

The smallest frame is a screen shot from a Youtube clip, the larger is a screen shot from the Blu Ray release. The center extraction is nice and clear, but it really highlights to me the significant amount of the frame which is unused. (See this post for some excellent input from David Mullen ASC on Super 35 and center extraction). This frame was still on my mind when I spotted that a new version of PaulXStretch was available.

PaulXStretch is designed for radical transformation of sounds. It is NOT suitable for subtle time or pitch correction. Ambient music and sound design are probably the most suitable use cases. It can turn any audio into hours or days of ambient soundscape, in an amazingly smooth and beautiful way.

https://sonosaurus.com/paulxstretch/

This got me thinking again about the still image; those that I’d been looking at for years in magazines, books, posters. Those that I’d fetishised over, collected, and archived. The images that meant so much to me and were central to my love of film, but that were also entirely silent. So with the help of PaulXStretch, I have taken the opportunity to bring sound back to this particular still image. The soundtrack to this video is the software iterating on the 41.66667 milliseconds of audio that accompanies this single frame from the film.

But wait, there’s more…

When I first had this idea, about 12 months ago, I wanted to try and accomplish it with an actual film frame. I found a piece of software (AEO-Light) which could extract the optical soundtrack information from a film frame scan, and render it to audio. So I went and bought myself some strips of film.

These are quite easy to come by on Ebay but there was a fatal flaw in my plan (which I didn’t realise until some time later). On a release print like the frames I have here, the physical layout of the projector, and specifically the projector gate, means that there is no space for the sound reader to exist in synchronous proximity to the frame as it is being projected. The optical sound reader actually lives below the projector gate, which means that the optical soundtrack is printed in advance of the picture by 21 frames (this is the SMPTE standard for the creation of release prints, but how that translates to the actual threading of a projector is a little more up in the air according to this thread). So in this material sense, where the optical soundtrack is concerned, sound and image only come into synchronisation at the very instant of projection.

If you’ve made it this far and want to know more about projector architecture then I highly recommend this video (I’ve embedded it to start at the ‘tour’ of the film path). Enjoy.

Se7en Payne’s Constraint

This experiment was inspired by this post by Alan O’Leary for The Video Essay Podcast where he writes about Matt Payne’s video essay ‘Who Ever Heard….?’

For me, ‘Who Ever Heard…?’ is an example of ‘potential videography’. It offers a form that can be put to many other uses even as its formal character—its use of repetition and its ‘Cubist’ faceting of space and time—will tend to influence the thrust of the analysis performed with it (but when is that not true of a methodology?). 

Alan O’Leary On Matt Payne’s ‘Who Ever Heard…?’ 2020

I wanted to see how this form might respond if I made my clip choices purely on sonic grounds. This was also another opportunity to explore the ‘malleability’ of the multi-channel soundtrack. Most video editing software will allow you to pull apart the 6 channels of a 5.1 mix and edit them separately. To me this is a fundamental sonic deformation, where the editing software allows access to the component parts of the films sonic archive (riffing extensively on Jason Mittell here).

For my ‘Payne’s Constraint’ I decided to top and tail my sequence with the same image of Somerset in bed, accompanied by a different channel selection from the soundtrack each time. The opening shot uses just the surround channels which are carrying rain at this point, but on the way back round I used the front left and right channels from the mix where the sound of the city invades Somerset’s bedroom (and consciousness). The other sound selections are less intentional than this, I picked sounds that were interesting, or odd, but that also contributed to the sonic whole that was coming together. It’s worth pointing out that after I added the 4th or 5th clip Resolve refused to play back any video so, much like with the Surround Grid, this ended up being a piece which was fashioned without really understanding how it would work visually until I could render it out.

A few sonic takeaways from this;

  • There is a lot of thunder in this film, but no lightning
  • There is very little sky (excepting the final desert scene) but there are helicopter and plane sounds throughout
  • Low oscillating sounds are prevalent in the soundtrack. Lorries/trucks and air conditioning contribute to this, but room tones also move within the soundtrack, rather than being static ambiences.
  • There is music in here that I was never consciously aware of before. Here’s ‘Love Plus One’ by Haircut 100, which I found burbling in the background of the cafe scene (clip 14 in my video).

Singin’ Will Wall

These experiments are inspired by Hollis Frampton’s 1971 film Critical Mass and were made possible using the software HF Critical Mass created by Barbara Lattanzi

I think I only watched Critical Mass because it auto-played on YT after I’d finished (nostalgia), another Hollis Frampton film, also from 1971. When I tried to find out more about Frampton’s process making Critical Mass I came across Barbara Lattanzi’s site, and the HF Critical Mass software she created “…as an interface for improvising digital video playback.” These 3 videos were made with Version 2 of the software.

I originally thought I might pick one of the musical numbers from Singin’ in the Rain’ (the film seemed like an obvious choice given it’s centrality to deformative videographic practice!) for this first experiment, but as I scrubbed through the film I hit on this scene, which not only has it’s own ‘built in’ loopability, but also appeals to my sonic self. The HF Critical Mass software gives you control over the length of the loop it will make, and the speed with which the loop with advance through the video (amongst many other controls) and I set these specifically for each video. In this case the loop length was defined by the door slam and the clapperboard, essentially bookending the loop. I’m not sure if this is the first time I noticed the sound engineer’s exaggerated movements, but the looping did highlight the synchronicity between Lina’s head turns, and his sympathetic manipulation of the recording controls.

I wanted to see how this would work on some quick fire dialogue, and I always share this scene with my research students, so it was an easy pick. Finding the loop length here was harder, and I’m a little surprised how consistent some of the rhythms in the delivery are, and how many lines actually get a complete delivery ‘in loop’ (Should I be surprised? Or is a rhythmic line delivery, with consistent pauses, inherent to this kind of monologue). The looping again highlights the movement within the scene, and the camera moves also get ‘revealed’ in the looping. Favourite moment is definitely the (total accident) at the end where ‘unoriginal’ becomes ‘original’.

This is a scene which I’ve always loved listening to but I think I’m listening to it differently with this video. The looping, in combination with the staggered progress of the video, seems to hold the sounds in my memory just long enough that I feel I can grasp them a little more clearly. Each loop seems to exist as its own short sonic motif, almost self-contained, even as it contributes to the whole.