Digital Culture & AudioVisual Challenges

This paper was created as part of Kapetaniou's final project in the master's program "Audiovisual Art in the Digital Age" of the Department of Sound and Visual Arts of the Ionian University of Corfu, under the supervision of professors: Vallianatos Thomas, Honorato Dalila, Tiligadis Constantinos. The focus is on the loss of anonymity of citizens in public spaces due to the use of facial recognition software in various countries. It is highlighted that the use of such software is not only disruptive to public order but also discriminatory due to the lack of education. The concept of Machine Learning is defined and the importance of its application in various fields is examined. Emphasis is placed on open source software and the principles of open collaboration that govern them, as they evolve the capabilities of Machine Learning. It then analyzes Google's "TesorFlow" open source learning software that allows other machine learning applications to be created and provides an API. The artistic work of Kapetaniou Christina is presented based on the relevant findings as a commentary on the violation of citizens' rights, freedoms and privacy. It uses a browser facial recognition API running on top of the core tensorflow.js API to recognize emotions based on facial expressions/ quality characteristics. Digital data is converted into an audio landscape and audio into graphics in a hybrid public environment that exists in both digital and analog. It is an interactive installation and website that allows human-machine interaction but also expands the experience of human interaction through technology.

The proposed project "YOU AND AI" expands the boundaries of technology and self. It focuses on decoding people's emotional states through facial expressions, ignoring quality characteristics such as gender/age/color/race. The focus is on humans' facial expressions and emotions as these can be interpreted by a machine. Is a comment on anonymity violation as it exposes the private/individual experience to a public dialogue, giving back the choice to people on how and if they want to participate. The artwork has a dual existence. As an installation, in which a mirror film with slits like a vagina for enter-exit, divides the space in two. In the inside space, there is a mirror column with an LCD panel at face height to display digital participants. There happens the interaction between a physical participant and a virtual by AI interpretation of their expressions. The values of the interpreted emotional status every dt shape the soundscape. In outer space, there are reflections of physical participants on mirror film and prismatic visuals projected upon them, activated by the soundscape, reflect simultaneous and complex emotional human experiences. The website has two portals; green, black. I reserve the green for the physical participants and the black is for the digital ones to enter the installation. In both cases, the camera opens real-time, and the programme starts the AI emotion recognition on each portal, extracting data for seven emotions. Their average value for each dt controls seven sound libraries. Each emotion controls its own intensity and that of others, plays at a random point in the panoramic landscape or stereo, and the one that dominates every moment presses the others through compressors. Transmitter and receiver are in dialogue as the transmitter produces to activate the receiver, and the activated receiver affects the content that the transmitter will then produce. It is an endless communication with emotion-driven sounds and sound-driven visuals.