What is DataMoshing?
DataMoshing is exploiting compression by altering frames to create mind melting glitch effects. Just play like you understand I will explain below. There are three types of frames that can be manipulated. The P-Frame, The B-Frame and the I-Frame. DataMoshing could not be possible without compression. In the real world you can see Moshing when you skip ahead a poorly compressed DVD rip, occasionally streaming a movie when the bandwidth fluctuates or bench pressing at a gym looking up at the T.V. and seeing Arnold Schwarzeneggers face morphing into hemorrhoid cream. Then excitedly pointing at the T.V. to gym goers as they look back and forth from the T.V. to my shit eating grin and concluded that I am insane.True story. Not the insanity part of course….Humm…
What is compression?
In order to have an understanding of DataMoshing we need to understand compression. Compression at its basics is making a file smaller. A movie, text document, encoded music can be compressed. That’s the amazing thing about ones and zeros. Limiting bandwidth also can compress size. Telephones and cellphones. Are a good example. There are cool exploitations you can do with these also but it is not DataMoshing. I digress.
With DataMoshing we exploit certain aspects of compression called frames to create wonderful kinds of glitches. The beauty about this style of glitch is the variation and also its otherworldly nature. How does compression work? Before we delve into frames let us jump into this example so we can begin to see the full picture. ( Yes I know I am annoying my brain is non linear. So for all of you that are linear thinkers I apologize in advance. )
Hold on to this right here in yer brain ——> Lets look at a 10 second compressed and uncompressed file. A 10 second, 30 fps uncompressed movie would be composed of essentially 300 still images. A 10 second compressed video would use only 30 still images (I-Frames) and the rest would be just the information (P-frames) that changes from still to still.
OK got it? It’s ok if you don’t. Let’s blow this taco stand….
So what are these frames thingies, motion compensation and how do they relate to compression?
P-Frames : The granddaddy of compression. Frames in succession have a high similarity. P-frames are small reference files that inform the video player of changes in the image since the previous frame. Only information about areas of motion or changes in lighting values are contained in a P-frame. The large frames that are redundant can be thrown out reducing the file size. The valuable information of the thrown out video frames are stored in the small P-frames. These essentially re-create the video but lower resolution. This is why a smaller file has more P-Frames. Essentially you can think of P-Frames as small bits of information tracking what pixels change.
Another example if that was confusing. Imagine rather than storing 24 frames per second you only need to store 2 frames because the rest of the information is redundant. Now multiply 2 frames by the movie length in seconds and you have a much smaller digital file. Rather than gigabytes you have megabytes depending on how intense the compression is. More intense compression equals shittier looking videos.
Incase you didn’t know A pixel is the smallest controllable element of a picture represented on the screen. The word pixel is a combination of pix and el for element. The word pix appeared in variety in 1932, as an abbreviation for the word pictures, in reference to movie!
I-Frames : The big bloat frame. I-frames are “inter” frames also referred to as keyframes. They are full representations of a single frame of the video. They are still images containing all the color and light information of that frame. They contain all the information. This frame is what makes a video file large in size. P-Frames rely on this for prediction of what will happen and what to throwout or store.
B-Frames : This frame predicts both directions or forward and backward. Basically this is the interrelation between P-Frames and I-Frames. Got it? Haha you may need to read these again. Kind of mind bending.
Motion Compensation : More compression. A good example is panning. Rather than storing all of the frames of the panning it only stores the motion positions further compressing and making the file smaller. So you can think of it this way. Someone gives you a big book of pictures how to get to your friends house as opposed to a paper with the street names and the rights and lefts associated with them. Or if you are old school someone hands you a Thomas guide as opposed to an iPhone using Waze. Yes I just dated myself. One makes your arm fall off and one doesn’t. Ah the good old days of being a dyslexic pizza delivery man. Fun times.
So what exactly is a frame rate?
Motion in video is created by changing of image frames in succession. This is called a frame rate. The frame rates are calculated per second! So in a typical movie there are 24 frames per second! Multiple 24 by the amount of seconds in a movie and you have a shitload of frames! The frames need to be plentiful and fast so you do not perceive the film as a 1950’s stop motion movie. You perceive the film as fluid and realistic movement. The more frames the more perceived fluidity. The more frames the longer the moshing will take.
Another way to visually understand frame rates is refresh rates. Refresh rates are to monitors as frame rates are to film. Wave your hand across your computer screen and you will see your hand stuttering. Those stutters are our eyes perceiving the gaps in the refresh rates or the gaps in the refresh frames (I just made that up) . The higher the refresh rate is the less stuttering you will see and you will have more perceived fluid motion. Eventually in the future refresh rates will be so fast there will be no stutters.
Here is the secret sauce. Compression happens when you get rid of redundant information. Predictive frames make that happen. When you rearrange the predictive frames you get the crazy effects that transcend space and time shifting forward, backward, morphing, and jumping in time. Video time travel. Woh…
There are lots of ways to mess with the frames. Duplicating them, deleting them, swapping them etc…Or adding an I-Frame back to stop the madness for a bit. Each yields a different result from blooming, painterly effects, freakish morphing and others. DataMoshing Deluxe has other tricks under the hood as you will see it has millions of possibilities of DataMoshing. DataMosh Reactive relies on your fingers to do the duplicating and rearranging. Stay tuned for more apps and updates….
Also if any of you Glitch artists want to respond to this have at it. I might have missed things or fucked up some information.
Further ramblings :
You are very lucky and unlucky. Is convenience the devil? Normally the process of DataMoshing is very very time consuming. Sometimes with an art form the tools themselves become the art form with some people. DataMoshing and other esoteric forms of glitch have been underground. Never quite fitting in I prided and pride myself as being a glitch and underground thinker. Manually DataMoshing works and sometimes it does not. You are lucky because DataMosh Deluxe does does this for you in seconds and also reacts to music which has never been done before. DataMosh Reactive is more hands on and you can manipulate the frames jamming to music with your fingers. In essence we are making your life easier which MAY eventually make DataMoshing commonplace and not counterculture anymore. Just like all those glitch apps out there did. Does this mean the art is going to suck now? Well there is certainly going to be sucky artists whatever the tools are. Jimi Hendrix manipulated feedback on his guitar when it was annoyance to most guitarists. Others copied him and his techniques. Did this make artists suck more? Not really. You just have more artists and therefore more chance of suckage. (Is that even a word?)
You are unlucky because sometimes the magic of unpredictability in the process makes it a lot more special. We as humans will gravitate to convenience most of the time. We want presets, we want real time, instant satisfaction and we want to be told it’s going to be alright just follow the map.
An example would be loops for music and presets with almost every app for instant recall so there will be no mistakes, predictability and a lack of searching into the unknown. Perfect replication every time. This is to create order out of chaos. This makes us warm and fuzzy. I think we loose something valuable in the process or non-process.
Another example is the fact that I can fry one of my keyboards Circuit Bending. Part of the reason why I love circuit bending is it is a way to explore without having to know too much about electronics. Poking and prodding at an instrument until it makes something cool. Sometimes though you may fry the keyboard getting too overzealous with more and more bend points rendering that keyboard at best as a really nifty drink coaster. Hello Etsy. The painful road makes a successful Circuit Bent keyboard very special. The artist also remembers the unique road getting there. It’s risky. It’s unpredictable. It’s chaos which humans always try to avoid. Because of this chaos each keyboard has an adventure associated with it, a personality and a story. It is chaos at its finest. It is the same with Moshing.
We thought long and hard about the philosophy when coming up with the concept of DataMosh Deluxe and I believe the pros outweigh the cons. We left in randomness and the ability for the output to suck sometimes. If the DataMoshing process is more of an interest to you do not use this app. I can respect that.
Another topic for a rainy day is why leaving room for Glitches in the mind, spirit and noticing them in the tangible world can change the world in a positive way and open up creative processes unmanageable. One has to leave room in their lives for the golden Moments to let glitches happen.
As soon as controls leash is taken off, the unintended beauty starts making itself known.
There is beauty in the broken
Over and out