Steve Zafeiriou is a New Media Artist, Technologist and Founder of Saphire Labs. His work explores the boundary between perception and reality, researching the ways technology influences, shapes, and distorts our understanding of the world around us. Through a practice grounded in generative algorithms, electronic circuits, and interactive installations, Steve investigate human behavior and the illusory nature of reality, encouraging viewers to rethink their understanding of the world around them.

In search of IKIGAI
dark mode light mode Search Menu

ESP32 Microcontroller Interactive Art: How “Choice” Translates Movement into Generative Art

ESP32 Microcontroller Interactive Art: Lilygo T-Display S3s on the lab

Have you considered how your movements might generate a unique work of art? This is the concept I developed with Choice, an interactive sculpture designed to translate viewer movements into generative art through the use of the ESP32 microcontroller and sensor technology. I will outline the process behind creating Choice and explore how ESP32-based interactive art is advancing creative expression through technology.

choice v1 info
ESP32 Microcontroller Interactive Art: Choice Interactive Sculpture version 1.0.1

The Inspiration Behind Choice: An Interactive Sculpture Driven by the ESP32 Microcontroller

As a new media artist with a background in technology, I have focused on exploring the intersection of art and technological innovation. The idea for Choice emerged from my research into Darwinian evolutionary theory, particularly the relationship between behavior and its impact on the surrounding environment. My objective was to design an interactive sculpture that not only presents art but also transforms it in real time in response to human engagement.

Choice is not a static piece; rather, it is a dynamic installation that captures and reacts to viewer interaction. By integrating real-time sensor data with generative art, my intention was to create a continuously evolving canvas, where each movement influences the artwork. This project embodies my belief that art should be an interactive experience, establishing a dynamic connection between the creator and the observer.

ESP32 Microcontroller Interactive Art: Lilygo T-Display S3 - ESP32S3 R8 WiFI BLE
ESP32 Microcontroller Interactive Art: Lilygo T-Display S3 microcontrollers

How the ESP32 Microcontroller Powers Choice

Selecting the appropriate technology was critical to the development of Choice. I chose the ESP32 microcontroller for its robust performance and versatility, specifically utilizing the LilyGo T-Display S3, an ESP32-S3 development board with Wi-Fi, Bluetooth, and an integrated TFT display.

The ESP32’s processing power and connectivity features enabled me to implement real-time data processing and Over-the-Air (OTA) updates, allowing the sculpture to evolve without the need for physical hardware changes. This microcontroller serves as the core of the installation, overseeing everything from the collection of sensor data to the generation of the visual art that is displayed.

Digital Art at the British Art Fair 2024
Digital Art at the British Art Fair 2024
Presenting “Choice”, an ESP32 driven sculpture.
Presenting “Choice”, an ESP32 driven sculpture.
Exhibiting at British Art Fair an interactive sculpture based on ESP32
Exhibiting at British Art Fair an interactive sculpture based on ESP32

Sensors and Real-Time Data Processing

The interactive sculpture Choice captures and processes movement with precision through continuous data input. I incorporated the MPU6050 accelerometer and gyroscope sensor to monitor movement along the X, Y, and Z axes. This sensor provides 16-bit raw data for both accelerometer and gyroscope readings, offering detailed insights into the sculpture’s motion.

To process this data effectively, the microcontroller converts the raw sensor values into meaningful measurements. For the accelerometer, I configured a full-scale range of ±2g, resulting in a sensitivity scale factor of 16,384 LSB per g-force. The microcontroller then divides the raw data by this factor to calculate acceleration along each axis, enabling a dynamic interaction with the art by translating motion into visual output.

ESP32 Microcontroller Interactive Art:  MPU6050 Gyroscope & Accelerometer sensor
ESP32 Microcontroller Interactive Art: MPU6050 Gyroscope & Accelerometer sensor

This setup enables the ESP32 microcontroller to detect significant movements with high accuracy. Once motion is detected, sensor data is immediately captured and processed, allowing the microcontroller to activate generative art functions that convert each unique movement into dynamic visual patterns on the display.

Real-time data processing is critical for delivering an immediate and immersive experience. The ESP32 analyzes accelerometer data to identify specific gestures, such as upward or downward movements, which then trigger the generative art algorithms. This ensures that each visual result is unique, shaped directly by the viewer’s interaction with the sculpture.

// Read accelerometer data
sensors_event_t a, g, temp;
mpu.getEvent(&a, &g, &temp);

// Convert raw data to acceleration in g-force
float accelY = a.acceleration.y;

// Detect significant motion along Y-axis
if (abs(accelY) > threshold) {
// Record the current time
unsigned long currentTime = millis();

// Trigger the generative art function with the acceleration value
generatePixelArtWithAnimation(accelY);
}

Generative Art Creation Through Viewer Movements

In Choice, transforming movement into art involves a direct, responsive process. When the accelerometer detects a qualifying gesture, the microcontroller activates a generative algorithm that renders dynamic visual patterns on the TFT display. Each movement adjusts specific variables within the algorithm, resulting in a unique generative art piece.

For example, the strength and speed of a gesture can influence the color palette or the complexity of the displayed patterns. This ensures that each interaction results in a distinct visual outcome, positioning the viewer as an active co-creator. The immediate visual feedback enhances engagement, making the experience both interactive and personally meaningful.

ESP32 Microcontroller Interactive Art:  React JS Application / Choice Manager
ESP32 Microcontroller Interactive Art: React JS Application / Choice Manager

Open-Source Commitment and Community Building

A fundamental principle of Choice is its dedication to open-source development. Transparency and collaboration are essential to fostering sustainability and driving innovation in both art and technology. To support this vision, all code, including the firmware and web application, is made freely available for review, modification, and enhancement.

By making Choice open-source, my goal is to encourage contributions from artists, technologists, and enthusiasts. This collaborative approach helps cultivate a creative community where ideas can be shared and developed further. I hope this foundation inspires others to build on it, exploring new possibilities in interactive sculptures and data-driven art.

The Role of Firmware and Web Application in “Choice”

The custom firmware developed for Choice is crucial in managing the complex interactions between hardware components to ensure optimal performance. Written in C++, this firmware handles the processing of sensor data, controls the display, maintains Wi-Fi connectivity, and supports Over-the-Air (OTA) updates for easy remote updates and maintenance.

Additionally, I developed an interactive web application, the Extension Manager, using React JS and MySQL. This application enhances the viewer’s experience by adding additional layers of interaction, seamlessly integrating with the sculpture. The connection between the physical and digital worlds enhances the experience, making it both comprehensive and interactive.

Integration with External Installations

Choice is designed to extend beyond a standalone sculpture, integrating seamlessly with external installations for enhanced interaction. By connecting to the Choice Extension via USB-C or WiFi, the sculpture can transmit generated data, working collaboratively to initiate new “pixelverses” of decision-based imagery.

Each new generative image is temporarily stored in a database and made accessible through a QR code. Viewers can scan the code to access a validation interface, where they can influence the outcome of their creation. This process not only deepens viewer engagement but also contributes to a collective gallery of generative art, reflecting a community of shared interactions.

Digital Art at the British Art Fair: Choice v1.0 by Steve Zafeiriou
ESP32 Microcontroller Interactive Art: Choice v1.0 by Steve Zafeiriou exhibited at British Art Fair 2024, Saatchi Gallery London, UK

The Future of Interactive Art and Technology

The development of Choice has reinforced my belief that the integration of art and technology unlocks vast potential for the future. Interactive installations like this have the power to transform how audiences engage with art, creating experiences that are both immersive and participatory.

I envision a future where sensor data becomes a standard element in artistic expression, with microcontrollers like the ESP32 playing a central role in installations. As technology advances, the possibilities for creating more complex and responsive artworks will continue to expand.

In my research for Choice, I also explored the integration of blockchain technology through Physical Backed Tokens (PBT), which offer a unique connection between physical artworks and digital ownership. By incorporating PBT into the project, Choice could serve as a model for how embedded systems in art can go beyond traditional experiences, opening up possibilities for immersive, secure, and interactive digital ownership models.

Choice demonstrates the potential of embedded systems in art, and the incorporation of blockchain opens new avenues for innovation. Additionally, I designed several 3D-printed case versions for Choice, improving both its aesthetic versatility and functional protection, making the work more adaptable and durable for a range of installations.

ESP32 Microcontroller Interactive Art: 3D Printed Case of Choice, variations and materials
ESP32 Microcontroller Interactive Art: 3D Printed Case versions of Choice

Overcoming Challenges and Lessons Learned

Creating Choice has been an ongoing journey marked by both technical and creative challenges. Integrating hardware components like the MPU6050 accelerometer and the LilyGo T-Display S3 required planning and extensive testing to achieve seamless functionality. Ensuring real-time data processing without lag was essential to preserve the interactive experience.

Developing custom firmware in C++ expanded my knowledge in embedded programming and ESP32 development. Managing Wi-Fi connectivity and implementing Over-the-Air updates added additional complexity, yet these elements were vital for the sculpture’s responsive capabilities. These challenges highlighted the significance of precise design and the persistence required to overcome technical obstacles.

Community Collaboration and Future Projects

Although Choice is yet to be officially released, it exhibited at the British Art Fair 2024 in the PIVOTAL: DIGITALISM section at Saatchi Gallery.

Following the exhibition, I released the software as open-source, fostering collaboration and building a community around the project. I am eager to connect with creatives in interactive media, sensor-driven art, and microcontroller applications, with the hope of inspiring new advancements in generative algorithms and sensor integration to enhance interactions. Sharing Choice serves as an invitation to explore the expansive potential of this field collectively, and I look forward to seeing where this journey leads.

Data-Driven Art and Darwinian Theory Article Cover
ESP32 Microcontroller Interactive Art: our Behavior Shapes our Reality

Conclusion

Choice embodies the intersection of art, technology, and human engagement. By utilizing the ESP32 microcontroller and sensor integration, this installation transcends static art, evolving in real time based on viewer interaction, making art a dynamic and collaborative experience. This project demonstrates how technology can push the boundaries of creativity, creating a shared, interactive encounter.

With a focus on open-source collaboration and continuous innovation, Choice aims to inspire others to explore the potential of connecting art and technology. I invite you to explore the project’s repositories on GitHub, experiment with the code, and share your insights. Your contributions are crucial in shaping the future of interactive art projects like Choice and beyond.

Total
0
Shares
Digital Art at the British Art Fair 2024 Presenting “Choice”, an ESP32 driven sculpture. Exhibiting at British Art Fair an interactive sculpture based on ESP32
Digital Art at the British Art Fair 2024 Presenting “Choice”, an ESP32 driven sculpture. Exhibiting at British Art Fair an interactive sculpture based on ESP32