Digital Culture & AudioVisual Challenges

Summary:
Technological advancements on different types of brain-tracking technologies and biofeedback sensors, allow real-time recording of response to sensory stimuli, providing new approaches of artistic creation and outcome interpretation (Nijholt, 2019). This paper examines low-cost measurement tools in the scope of visual digital arts education, analyzing the results of a case study of the utilization of those devices from student artists. It also presents a prototype neurofeedback framework that combines different types of measurement systems with a virtual reality space that can be used both as a tool for artistic experimentation and one for accumulating quantitative results on the impact of the experience.

Objective:
The aim of the paper is to survey the current state in the field of brain–tracking tools and biofeedback sensors that can be utilized in visual digital arts education, the research and art projects that have been done upon that field, whilst it also presents a case study scenario of utilization of those mechanisms from students artists as a medium of artistic experimentation.
To further enhance this objective, different tracking mechanisms (brain and eye tracking) are analysed along with different biofeedback sensors (heart-rate, dermal conductivity, electromyography) and an ongoing prototype experimentation framework setup based on Unity3D software, that utilizes and combines different systems of measurement, is analysed. This framework can be utilized either by artists for experimentation, but also as a tool for data recording of the impact of an art project on the subject.
Technological and scientific advancements may bring a new era, new perspectives into art practices. Specifically regarding technology, tracking tools that are now available to a greater public and immersive technology such as VR, with the possible transformations of the concepts of space, time, subjectivity, connectivity, give a wide range of potentials for applying new approaches on artistic experimentation. Regarding science, advances on the way human subjects experience visual responses, the way brain responds and interprets sensory stimulus, the affective outcome (i.e. engagement, anxiety) and the availability of tools and software for measurements analysis, all contribute as a framework that can be utilized by artists as well.
Combining low-cost tracking tools with sensory input creation and analysis software, provides a tool that can be used as a medium for creating interactive artistic projects and experiences. In those works, the participant interacts actively in the work itself and it’s his/her activity that may give form to the work (Kwastek, 2015). The participation of the subject may be essential for the experience and for the realization of the idea of the project, which a person should perceive, be involved in and ponder simultaneously (Kwastek, 2015). Biofeedback systems as tools for creation, measurement, impact analysis, or even for self-regulation (i.e. attempting to learn to regulate behaviour - intersection with art therapy) can be implemented into art practises.

Method:
A comprehensive and detailed literature review is conducted in order to trace the usage of EEG systems and technological tools such as biophysical sensors (i.e. electromyographic activity, respiration rate, dermal conductivity) in capturing and evaluating a person’s response in the scope of visual arts.
A survey on different artworks that utilized such technology will be presented, whilst through quantitative research and the resulting material, a case study on the utilization of those devices from artists as means of experimentation will be analyzed.
The efficacy of the utilization of low-cost equipment is studied by conducting a comparative study between different EEG devices and biofeedback sensors. Considering the aforementioned studies and their findings, a rough framework on the potential of those tools in the field of digital visual art has been formed and a prototype system will be analyzed that combines multiple biophysical sensors, while the challenges of implementing this technology, the framework and the synergy between tools and software will be presented.

Conclusion:
Low-cost tracking technology is evolving rapidly and new possibilities are likely to become available to researchers and artists in the near future. Cooperation between artists and researchers in the fields of computer science, neuroscience and psychology could give a better understanding of our comprehension of the impact of a visual artwork to a subject, responses and even perception, thus understanding more about brain processes to sensory stimuli, but also may provide new perspectives into art practices. However, conclusive results prove challenging due to inherent technical limitations of the current technology, due to difficulties in data acquisition and analysis, implementation across different devices and software, the erratic nature of these signals (Accoto et al., 2021) and the process should be thoroughly tested in order to not compromise the outcome.

References:
Accoto, F., Vourvopoulos, A., Gonçalves, A., Bucho, T., Caetano, G., Figueiredo, P., De Paolis, L., & Badia, S. B. i. (2021). The Effect of Neurofeedback Training in CAVE-VR for Enhancing Working Memory. In T. Dingler & E. Niforatos (Eds.), Technology-Augmented Perception and Cognition (pp. 11–45). Springer International Publishing. https://doi.org/10.1007/978-3-030-30457-7_2.

Kwastek, K. (2015). Aesthetics of Interaction in Digital Art: MIT Press.

Nijholt, A. (2019). Brain Art: Brain-Computer Interfaces for Artistic Expression: Springer International Publishing.