About the Artist
Having received a Bachelor’s Degree in Psychology at the University of Malta, she pursued her studies with a Diploma in Foundation Studies (Art & Design) and is currently in her final year of the Bachelor of Fine Arts in Digital Arts Degree.
Jessica is very eager to use her background in psychology to help her create better and more immersive narratives within the areas of branding and user experience.
ABout the project
Childhood Narratives conceptually explores the past childhood experiences of twins- a girl and a boy- interpreted through a twofold narrative of illustrations and animations accompanied with soundtracks.
It was intriguing for me to be able to represent and compare the perspectives of two individuals of the same age who were exposed to a similar upbringing within the same family.
How I got started
Initially, the information was gathered through two in-depth semi-structured interviews conducted with the twins to explore nostalgia and identity formation by understanding their past childhood experiences and their respective evoked emotions. Here I started creating conceptual illustrations and animations according to the emerging themes. All the animation are created through rotoscoping, which is an animation technique were one traces over motion picture footage, frame by frame, to produce realistic action.
A set of illustrations, found in the section ‘The Constant Bond’, were created from reference photos owned by the participants. The twins were asked to bring photos which have a special meaning connected to their childhood.
The connected paths in these illustrations are generated from an eye tracking device that tracked the movement of the participants’ eyes along the original photo.
Brainwave Data into Sound
Ultimately I want to provide the viewer with an immersive experience by delving into aspects of the senses. That is why during the interview a Muse headband, which employs electroencephalography (EEG) technology, was worn by the participants to monitor their brain activity.
The resulting graphic brainwave images gathered from the Muse headband while the twins were being interviewed were used to create soundtracks for each illustration and animation, which were inspired by what was said in the interview itself. The emerging graphic brainwave images were then converted to wave audio files using Photosounder. Photosounder (Rouzic, 2018) scans the vertical columns of pixels in a photo or drawing from left to right and each column of pixels is then digitized and transformed into a representative audio tone which is then merged into one melody. Here, I scanned each of the four waves individually using Photosounder to create four wave audio files.
Creating a Harmonious Sound
These were then converted to midi notes using Logic Pro X (Apple, 2018), a digital audio workstation (DAW) and MIDI sequencer software application for the macOS platform. This enabled the tweaking of loaded multichannel sounds onto the four midi sequences to create a final harmonious sound. All the variables – keys, tempo, duration, note length, panning – were determined by the variables that were delivered through the headband. However, the chosen multichannel sounds were based on the feeling I wanted the viewer to experience with each respective artwork.
Having an auditory representation of the brain’s activity, while talking and thinking about the subject at hand, was very intriguing to me. My aim was to further immerse the viewer in each artwork by having the auditory accompany the respective illustration and animation.