Music visualization

Music visualization or music visualisation, a feature found in electronic music visualizers and media player software, generates animated imagery based on a piece of music. The imagery is usually generated and rendered in real time and in a way synchronized with the music as it is played.

Visualization techniques range from simple ones (e.g., a simulation of an oscilloscope display) to elaborate ones, which often include a plurality of composited effects. The changes in the music’s loudness and frequency spectrum are among the properties used as input to the visualization.

Effective music visualization aims to attain a high degree of visual correlation between a musical track’s spectral characteristics such as frequency and amplitude and the objects or components of the visual image being rendered and displayed.

“Music visualization” can be defined, in contrast to previous existing pre-generated music plus visualization combinations (as for example music videos), by its characteristic as being real-time generated. Another possible distinction is seen by some in the ability of some music visualization systems (such as Geiss’ MilkDrop) to create different visualizations for each song or audio every time the program is run, in contrast to other forms of music visualization (such as music videos or a laser lighting display) which always show the same visualization. Music visualization may be achieved in a 2D or a 3D coordinate system where up to 6 dimensions can be modified, the 4th, 5th and 6th dimensions being color, intensity and transparency./

The first electronic music visualizer was the Atari Video Music introduced by Atari Inc. in 1976, and designed by the initiator of the home version of Pong, Robert Brown. The idea was to create a visual exploration that could be implemented into a Hi-Fi stereo system. It is described in US 4081829. In Great Britain music visualization was first pioneered by Fred Judd.

Music and audio players were available on early home computers, Sound to Light Generator (1985, Infinite Software) used the ZX Spectrum’s cassette player for example. The 1984 movie Electric Dreams prominently made use of one, although as a pre-generated effect, rather than calculated in real-time.

For PC/DOS one of the first modern music visualization programs was the open-source, multi-platform Cthugha in 1993. In the 1990s the emerging demo and tracker music scene pioneered the real-time technics for music visualization on the PC platform; resulting examples are Cubic player (1994), Inertia Player (1995) or in general their real-time generated Demos.

Subsequently, PC computer music visualization became widespread in the mid to late 1990s as applications such as Winamp (1997), Audion (1999), and SoundJam (2000). By 1999, there were several dozen freeware non-trivial music visualizers in distribution. In particular, MilkDrop (2001) and its predecessor “geiss-plugin” (1998) by Ryan Geiss, G-Force by Andy O’Meara, and AVS (2000) by Nullsoft became popular music visualizations. AVS is part of Winamp and has been recently open-sourced, and G-Force was licensed for use in iTunes and Windows Media Center and is presently the flagship product for Andy O’Meara’s software startup company, SoundSpectrum. In 2008, iTunes added the “Magnetosphere” visualizer created by The Barbarian Group.

Some of the more recent applications such as Luminant Music (2017), produce visualization in real time and render to a full-3D environment.

With the increasing popularity of virtual reality, several start ups have begun working on music visualization although reception has been mixed with one informal poll finding that only 33% of respondents were interested in music visualization for VR.

Technical application
The simplest variant is the representations via oscilloscope. In the simplest case, the waveform of the audio signal is plotted over the time course of the audio signal. This means that the electron beam of an oscilloscope is run across the screen at a certain frequency and sees the waveform of the audio signal.

Panorama knife
Another variant is the panorama display (also called correlation display) between the left and right audio channels. In this case, the audio signal for the left side controls the horizontal deflection of the electron beam, while the right audio signal controls the vertical deflection (left and right can also be reversed). This type of audiovisualization is usually used in music studios or in the TV or radio to visually assess the stereo width of the heard signal. At the waveform display you can optically evaluate the noise component with quiet signals. In the panorama display you can also see how an audio signal is arranged in the room. The representation varies between a diagonal line in the first squares and such a line in the 4th quadrant, which means a complete decorrelation. In general, the display on stereo signals pulsing is circular. In the songThe robots of the Düsseldorfer electric band Kraftwerk show the panorama display at the beginning of the song for a short time a square.

Level meter
Another common indicator is the so-called peakmeter. It shows the volume in the form of a bar. The classic that is still used in radio studios today is the peak meter from RTW. Also very well known are the variants of the company NTP. Peak meters are used to judge whether you are within a certain volume range. Especially on the radio, it is important to keep the volume level, as large fluctuations would disturb the listener. A peak meter is used by the moderators to check whether the volume is set correctly and whether really audio signals are being transported to the transmitter. Otherwise, a controller on the mixer may not have been pulled up.

Combined level and panometer
Important for the assessment is the so-called correlation. If a channel is twisted (left and right audio signal oscillate exactly opposite each other), the signal during mixing together (switch to mono, radios that play only mono) can cancel completely. The result would be that you can not hear anything, even though there is a signal. The human ear can not perceive phase shifts. Such phase rotors can only be visually assessed. In older radio mixing consoles the signals were via operational amplifiersmixed. The operational amplifiers used therein tended to shift the phase of audio signals. So-called correlation meters make such phase shifters optically visible and are integrated in modern peak meters. Earlier, oscilloscope-driven phase rotators were used, or with an additional peak meter that showed the sum of both signals (mono). The bars for the left and right sides, but the bars for the mono signal did not or only barely moved, so was a phase rotator in the signal. The moderators in this case have a button available that copies the right audio signal to the left channel or vice versa. The audio signal is in such a case only “pseudo-stereo”, but can also be heard on mono radios. This effect is by modern high-quality operational amplifier (for example, the company BurrBrown) barely exist. Since audio signals digitally (viaDSP), the effect no longer occurs.

Spectrum Analyzer
Similarly, the individual spectral components of a sound can be graphed by frequency. In signal processing systems, an FFT is often used and the amplitudes of the signal components are plotted logarithmically.

Artistic application

Audiovisualization as entertainment medium
Audiovisualisation fascinates many people. In the 1970s, light organs became popular.

In 1976 ATARI sold a product called Atari Video Music System C-240 Mint. This device was connected to the TV or stereo and displayed colorful images that moved or changed colors simultaneously with the music. The device had to be connected first to the power outlet and then to the stereo, otherwise the stereo system could sustain a defect.

For the legendary Commodore 64, which was used in the mid to late 1980s as a game console, there were a large number of games, many with copy protection. Crackers cracked the copy protection and left as a business card a so-called Cracktro on the disks. Today these “Cracktros” are known under the name “Demo”. It has formed its own scene around these demos around (demo scene). Very early on, moving bars and visual representations of the background music were integral parts of these demos. Around 1985, this demo scene was on the Amigacontinued the company Commodore. The computers of the series Amiga had four audio channels, the corresponding formats were SID or Mod. In many demos, but also in computer games, the four bars (peak meters) for each of the audio channels were used as an optical effect. Around 1987 more floppy disks with small song collections appeared (MusicMags), such as the group cactus. These floppy disks were loaded within seconds by the AMIGA and displayed a colorful universe of small hectic graphics. In addition, from a list of hundreds of songs a piece could be selected. Depending on the music, various greetings were sent to other groups across the screen, distorted by the various elements of the music, or changing in speed. These song collections were the first programs of their kind that depicted the elements of the music on graphics and ultimately served only for this purpose. The presentation of the music was almost celebrated by the graphics.

Since the computers are fast enough, the fast Fourier transform can be used to calculate the frequency spectrum of an audio signal in real time. This started for the PC platform with MS-DOS in the early 1990s, software that accomplished this. As the IntertiaPlayer or CubicPlayer, which achieved sufficient computing power on the still weak PC hardware through assembler programming.

Soon after the development of the mp3 data format, an audio player called Winamp was released in May 1997 with a plugin based audiovisualization concept (see also Winamp # visualization plug-ins). Winamp has been one of the first free MP3 players for PCs, has achieved widespread use and has dominated the audio player genre in design, GUI and look and feel. A well-known audiovisualization plugin was the 1998 Geissplugin, which for the first time generated a fractal-like visualization based on music.

Since Winamp, virtually all audio and media players have been equipped with an audio visualization, eg. B. iTunes of the company Apple. However, the connection between music and video is arbitrary and not always comprehensible realized.

Audiovisualisation in Art
On Friday, November 30, 2001, the Viennese sound artists sha and GTT installed a steel construction of 14 giant sound monoliths on Dornerplatz in Vienna. It was the largest sound art work in Europe. These monoliths were equipped with so-called NXT flat speakers. These speakers vibrate records, transmitting the sound. The 14 giant sound monoliths should now act as an integration attempt for urban problem areas. The term “audiovisualization” in this context falls in the metaphorical sense. The problems of the city and the shadows that will fall on these audiovisualized ‘. Analogously, this term semantics refers to the effect of sound and music on people’s perceptions. The sound, the music visualized in the minds of people the problem.

Audiovisualization in Design
Especially in the design of company logos has done a lot in recent years. Fonts were developed and extended by logos. This combination is once committed to a company and then lives in the company’s history as a corporate identitycontinue. Many design offices also offer an “audio visualization” of the corporate identity. This is the design of signature tunes, or tone sequences in addition to the logo. Very well known are the five tones of the T-Com, the four tones that always occur in the Intel advertising or the startup melody of the Windows operating system Microsoft. The consumer is always confronted with the logo and the audio logo at the same time. Thus, the subconscious of the consumer connects the tone sequence (the Acoustic logo) with the company name and the logo. Nowadays, one also speaks of “identity mix” or “brand communication”, instead of just corporate identity.

Car radios today use volume bars (peak meters) or a small frequency spectrum as audio visualization as a small optical gimmick.
Some stereos show the frequency spectrum.
VCRs with adjustable recording volume indicate two peak meters.
Laser shows in discos can be understood as audio visualization.
Videojockeys (VJs) show pictures of music in discos.
Music videos can be understood as an extension of audiovisualization.

List of visualization programs
(Annual information refers to the introduction)
Music Animation Machine (1985, MAM)
ZMusic (1987, Stephen Nachmanovitch)
Virtual Light Machine (1990, Jeff Minter)
Cthugha (1993, Kevin “Zaph” Burfitt, PC (DOS))
Inertia Player 1995, (Stefan Danes, Ramon van Gorkom, et al / Inertia Productions) Real-time spectrum visualization
Visual Music Tone Painter (1992-2004, Stephen Nachmanovitch)
Geiss-plugin (1998, Ryan Geiss), fractal visualization
SoundSpectrum / G-Force (2000, Andy O’Meara, SoundSpectrum)
MilkDrop (2001, Ryan Geiss) and OpenGL implementation, now projectM open source project
R4 (2003, Gordon Williams)
Vsxu (2003, Vovoid)
Neon (2004, Jeff Minter and Ivan Zorzin)
TronMe (2006, 3D Solar)
iTunes (2006, Apple)
Advanced Visualization Studio (Justin Frankel)
NoiseCradle (NoiseCradle)
Windows Media Player (Microsoft)
fish (2001-2013, Marcel Ebmer), also as XBMC plug-in (fishBMC)

List of electronic music visualizers
Atari Video Music, designed by the initiator of the home version of Pong, Robert Brown, and introduced by Atari Inc. in 1976.
Pixelmusic 3000, open source music visualizer on a microcontroller, made by Uncommon Projects in 2008.

List of music visualization software
Psychedelia (1984, Jeff Minter), an early “light synthesizer”, did not use audio input but was designed to create visualizations in accompaniment to music.
Virtual Light Machine (1990, Jeff Minter) (Platform: Atari Jaguar)
Cthugha (1993, Kevin “Zaph” Burfitt) (Platform: DOS)
Advanced Visualization Studio (Justin Frankel) (Platform: Windows)
MilkDrop (2001–2012, Ryan Geiss) reimplemented as projectM (Platforms: Windows, Linux, Android)
Music Animation Machine (1985–2013, Stephen Malinowski) visualizes MIDI, rather than waveforms.
Neon (2004, Jeff Minter and Ivan Zorzin) (Platform: Xbox 360)
Visual Music Tone Painter (1992–2004, Stephen Nachmanovitch)
Vsxu (2003–2014, Vovoid) (Platforms: Windows, Linux)
Magic Music Visuals (2012–2019, Color & Music, LLC) (Platforms: Windows, macOS/OSX)

List of media players supporting visualization
AIMP (AIMP DevTeam) (Platform: Windows)
Foobar2000 (Platform: Windows)
MediaPortal (OpenSource, Team MediaPortal) (Platform: Windows)
iTunes (2001, Apple) (Platforms: Mac OS X, Windows)
Winamp (Nullsoft/Radionomy) (Platforms: Windows)
Windows Media Player (Microsoft) (Platforms: Windows)
MediaMonkey (Ventis Media Inc.) (Platform: Windows)
Kodi (formerly XBMC) (Team XBMC) (Platform: Cross-Platform)
MusicBee (Steven Mayall) (Platforms: Windows)
K-Multimedia Player (Pandora.TV) (Platform: Windows)
Amarok (Open Source, KDE) (Platform: Cross-platform)
Totem (Open Source, Gnome) (Platform: Linux)
Clementine (Open Source) (Platform: Cross-platform)
Audacious Media Player (Audacious Team) (Platforms: POSIX)
VLC media player (VideoLAN Project) (Platforms: Cross-platform)
Xine player

Source from Wikipedia