Project Reflection (YiDa)
GET1033
Final Project Reflection
In this
project that I collaborated with Liang Hsi, we aimed to create a tool for
guitar players and musicians. This includes a primitive tuning tool and a mutable
chord library. The motivation for this was to incorporate technology into the
music-making and learning process. Additionally, I saw this as an opportunity
to practice the skills I have learnt during the course of this module-
particularly the programming and coding aspects.
The choice
of using OpenProcessing as the medium for our artefact was because of two main
reasons: 1. The ability to conveniently
host our sound samples online and 2. The availability of already built-in p5.js
libraries.
There were several challenges that I faced while creating this piece of work. Firstly, I underestimated the difficulty and mathematical complexity of creating a tuner that reacts to microphone input accurately. Secondly, abstracting the idea of chords and frets, that one associates with real physical guitar playing, into a computer code was not an immediately intuitive process.
There were several challenges that I faced while creating this piece of work. Firstly, I underestimated the difficulty and mathematical complexity of creating a tuner that reacts to microphone input accurately. Secondly, abstracting the idea of chords and frets, that one associates with real physical guitar playing, into a computer code was not an immediately intuitive process.
Storing
chord patterns in an Array data structure
My approach to storing chords and shapes on the guitar was to identify them based on their frets. An important assumption was that this tool is intended for 6-string guitars in standard E-A-D-G-B-E tuning. The range of chords are limited to shapes that are within the first 8-9 frets on the guitar. Each chord formed on the guitar can be identified by the specific frets that one has to finger, as well as the strings that are to be strummed. Given that each fret can be identified by a fret number, my idea was to store these frets using an array data structure of size 6. Each element of the array from 0-5 represents the fret that is to be pressed on the respective string, start from the lowest-pitched string. Open strings are represented with the value 0, and strings that are not played are represented using a string element ‘x’. For example, [‘x’,0,2,2,2,0] means that the 2nd fret of the D, G, B strings are to be pressed, the A, and high E strings are to be left open, and the low E string is to be muted. Using this abstraction, Liang Hsi and I created a dictionary of chords based on their unique fretting patterns. The dictionary was treated as an object in JavaScript with the keys being the name of the chords, and the values being an array of size 6.
My approach to storing chords and shapes on the guitar was to identify them based on their frets. An important assumption was that this tool is intended for 6-string guitars in standard E-A-D-G-B-E tuning. The range of chords are limited to shapes that are within the first 8-9 frets on the guitar. Each chord formed on the guitar can be identified by the specific frets that one has to finger, as well as the strings that are to be strummed. Given that each fret can be identified by a fret number, my idea was to store these frets using an array data structure of size 6. Each element of the array from 0-5 represents the fret that is to be pressed on the respective string, start from the lowest-pitched string. Open strings are represented with the value 0, and strings that are not played are represented using a string element ‘x’. For example, [‘x’,0,2,2,2,0] means that the 2nd fret of the D, G, B strings are to be pressed, the A, and high E strings are to be left open, and the low E string is to be muted. Using this abstraction, Liang Hsi and I created a dictionary of chords based on their unique fretting patterns. The dictionary was treated as an object in JavaScript with the keys being the name of the chords, and the values being an array of size 6.
Visual representation of chords
The next
step was to create a visual representation of this information. Liang Hsi was
responsible for creating the main canvas of the project, which includes the
lines that represented the strings as well as the fretboard of the virtual
guitar. Black dots represent the
frets to be depressed, unfilled dots
represent open strings to be struck, and crosses
represented strings that are to be muted. Initially, we used a rather
brute-force method to generate these visuals by manually defining a function
for each chord. In essence we used the “ellipse()”
and “line()” primitive shapes
available in the p5.js library to create these circles and crosses. Initially
we tried to brute-force this process by defining a function for each chord.
These functions included the ellipse()
functions calls where the exact coordinates of the x-y positions (and size) of
the dots and crosses on the canvas were specified. It did not take us long to
realise the inefficiency of this process, and I created an abstraction for
drawing these shapes.
Let us
denote the array of size 6 that represents the fretting pattern of each chord
as frets_array. Working under the
assumption that the distance between each fret and each string on our virtual
guitar is constant, I could come up with a mathematical relationship between
the fret number and the x-y coordinates of its position on the canvas. Through
the help of for- and while-loops, I defined the functions “make_x”, “draw_fret_on_string”,
“make_chord” and “draw_chord”. The first two functions
basically take each element of frets_array,
which is an integer value, and calculates the x-y positions of the frets
and returns an ellipse or cross that will be drawn at that specific position.
Each index of the array represents the corresponding string on the guitar in
order. For example, the 0th index of the array represents the low E
string, while the 4th index of the array represents the B string. Make_chord handles this information as
it takes in the entire frets_array and
extracts the elements of the array sequentially using a loop. The final piece
of this abstraction is the draw_chord
function which simply takes in the name of the chord (i.e. “E7” or “Am”) and
retrieves the associated array. This method made the process of defining the
library of chords much more intuitive and straightforward.
Audio playback of chords and sound sampling
Other than
simply displaying chords on the canvas, there is also audio playback of the
chords. This is achieved with the help of the p5.sound
library. Initially I intended to use the oscillators and filters available in
the p5.sound library to synthesize an
acoustic guitar sound. After some research, I concluded that this was too
complicated a process for me to achieve within this time frame, as the details
involved to create a convincing guitar tone was simply too complex. As such I
decided to implement the sound library using .mp3 samples. I manually recorded
short audio samples of each string of an acoustic guitar using my Korg Krome
keyboard at home. Using these 6 samples as a basis, I transposed the files on
Cubase manually and created a range of samples that covered all the notes from
E0 to E5 on the guitar. This brute-force method was preferred in the interest
of time. Ideally, however, I should have executed the transposition on OpenProcessing itself using some kind
of pitch-shift algorithm as that would have saved much file space. These sound
samples are pre-loaded when the program runs using the preload() function. I chose to use standard international MIDI protocol
to refer to each note – 60 refers to C4, 61
refers to C#4 and 59 refers to B3 etc. This make things very convenient as there is a clear
relationship between the reference for each note. The sound samples are thus
stored inside an object mySound_dict
where the “keys” are the MIDI notes
and the “values” are the
corresponding samples. In a similar fashion to the abstraction in the previous
paragraph, I defined a function “frets_to_midi_notes”
to convert frets_array into an array
that stores the respective MIDI notes
that need to be sounded.
In order to simulate a guitar being
strummed, I had to trigger each sample with an interval in between them, such
that there is a sense of separation (just like how a real guitarist plays).
This was achieved within my “playback_chord”
function that takes the MIDI array as input. Using the JavaScript built-in setTimeout function, I could separate
each individual call of “mySound_dict[note].play()”
by specifying a fixed time-delay of 150 milliseconds between each function
call, thus achieving a pseudo-strumming effect.
Buttons and User
Interaction
Finally, Liang Hsi and I created several
buttons to allow for user-interaction with the program using the p5.dom library. The first is simply a
search button and box where the user specifies the chord to retrieve. The ‘Strum again!’ button simply strums the
chord that is currently being accessed, which defaults to ‘Cmaj7#11’ when the
program initially runs. At the right-hand side of the UI, there are also 6
buttons corresponding to the open strings of the guitar that playback
pitch-accurate audio of the respective open strings.
An important aspect of our program is
the mutability of our library of chords. Given that a desired chord is not
found within our pre-loaded library, the user can define a custom chord by
specifying the name and its corresponding frets. When the ‘Create!’ button is pressed, the program takes in the frets as a
value, and assigns the corresponding frets_array
to the new chord inside the Chord_dict2
object. This way, the user can now lookup the new chord using the ‘Search!’ box and button. As a good
measure to provide permanent loss of data in the Chord_dict2 object, I made it such that the program will first
check whether a specific chord name has already been pre-declared. If it has
indeed been pre-declared, the program will prevent the user from re-assigning a
value to that name.
Additionally, in place of our proposed Fast Fourier Transform (FFT) Tuner, Liang Hsi implemented 6 buttons on the right side of the interface that triggers samples of the opening strings for the user to tune by ear.
Additionally, in place of our proposed Fast Fourier Transform (FFT) Tuner, Liang Hsi implemented 6 buttons on the right side of the interface that triggers samples of the opening strings for the user to tune by ear.
Difficulties and
Challenges
One of the biggest issues we faced
was scoping. Due to the nature of JavaScript and its built-in setup(), draw() and preload()
functions, where we chose to place our functions and objects in the code was
very crucial. Instinctively, we created and call all our functions that created
the ellipses and crosses within the draw()
function’s block. This worked perfectly well until we implemented the sound
library and the sound function calls mySound.play().
Because the draw() function
repeatedly calls itself once the program has started, our sound samples were
constantly called and playback when we placed our sound functions within the
same block. This created an ugly wall of sound. I then moved the sound
functions outside the draw() block
into the global frame, thinking it will eliminate the problem of repeated sound
function calls. However, a bigger problem arose – the sound library object
declared in the preload() function
cannot be accessed from the global frame. As a last resort, I moved all the
sound functions and drawing functions into the setup() function’s block, completely removing the draw() function. This solved most of the
problems until I realised that the dots and crosses drawn onto the canvas will
accumulate. That is to say, if I were to use the Search button to call another chord while an existing chord is
displayed on the canvas, the new chord’s visual representation will overlay
onto the previous chord’s. In order to resolve this issue, I placed a recursive
call of setup() within the draw_chord function to re-draw the
entire canvas every time a new chord is looked-up.
While the
above measures have indeed resolved our issues, I am aware that they likely not
the best ways to have written the program. Our canvas and background are
relatively simple and basic, allowing the recursive calls of setup() to be rather smooth. This likely
will not be the case had our canvas been much more elaborate. Furthermore,
there was significant compression of the audio samples from .wav into .mp3 in order to not exceed the maximum file size capacity that
OpenProcessing allows one to upload.
Initially, we intended to implement a full-fledged FFT-based tuning algorithm that was able to analyse input audio via the microphone. Unfortunately, after much research and attempt, we were unable to implement this from scratch due to the significant mathematical complexity, and the need for external libraries that were difficult to import onto the OpenProcessing platform. Instead of resorting to copying large chunks of open-sourced code, we decided to dumb-down the tuning mechanism, and simply provide pitch-accurate playback of the open strings on a guitar for the user to tune manually, by ear.
Initially, we intended to implement a full-fledged FFT-based tuning algorithm that was able to analyse input audio via the microphone. Unfortunately, after much research and attempt, we were unable to implement this from scratch due to the significant mathematical complexity, and the need for external libraries that were difficult to import onto the OpenProcessing platform. Instead of resorting to copying large chunks of open-sourced code, we decided to dumb-down the tuning mechanism, and simply provide pitch-accurate playback of the open strings on a guitar for the user to tune manually, by ear.
Learning Points/Conclusion
In conclusion, creating this digital artefact has allowed
me to explore and practice the ideas that were taught in class. Our artefact is
highly procedural, as there are many
functions at each step to fully abstract the idea of representing and
recreating the guitar chords on a digital medium. It is this specific set of idea that gives the user an intuitive experience in using our program. Our artefact however is
limited in being spatial and encyclopaedic. Due to the fixed nature
of the canvas, there is a limited number of chords that can be displayed within
the fixed visual frame of the canvas. While there is a large library of chords that already exists in our code, it is certainly not exhaustive. Additionally, manually defining each chord is a highly tedious process. In terms of being participatory, our artefact gives the user some freedom in choosing
and accessing the chords. Hands-on experience has taught me the importance of
good programming skills and habits, such as proper indentation and debugging
techniques.Our project makes use of the idea of objects and data structures that are available in computing to store information and ideas efficiently. Data structures like arrays and objects allow us to easily organize and manipulate information about chords as opposed to traditional method like print or physical charts. In terms of representation, we are essentially trying to transform the physical idea of sound and shapes into a digital one - bits and numbers to represent sounds and shapes, which can be manipulated by powerful digital systems. Furthermore, an important aspect of computers and computing is the ability for the system to update and change itself according to specific rules and inputs. This is achieved through the fact that our program and library of chords is mutable based on each user's specific requirements and inputs.
Our current program sets a foundation for possible future improvements, such as higher quality of sounds, given enough space to host the audio files. Implementation of chord strumming can be extended to include more varied styles, such as fixed finger-picking patterns. Animations can also be included to visualize the idea of the individual strings being struck. Alternative tunings and string configurations can also be considered to provide for different styles and kinds of guitar players. Additionally, user interactivity can also be improved by allowing the user to directly construct chords on the fretboard itself, as opposed to manually specifying numbers that correspond to fret numbers. This can possibly make the program even more intuitive and easy to use.
(By Koh Yi Da A0187674L, for GET1033 AY18/19 Semester 1)
Comments
Post a Comment