Introduction
Have you considered how your movements might generate a unique work of art? This is the concept I developed with Choice, an interactive sculpture designed to translate viewer movements into generative art through the use of the ESP32 microcontroller and sensor technology. I will outline the process behind creating Choice and explore how ESP32-based interactive art is advancing creative expression through technology.
The Inspiration Behind Choice: An Interactive Sculpture Driven by the ESP32 Microcontroller
As a new media artist with a background in technology, I’ve long been interested in the convergence of art and technological innovation. The concept for Choice emerged from my study of Darwinian evolutionary theory, particularly the impact of behavior on our environment and reality. My goal was to develop an interactive sculpture that not only displays art but actively transforms it in response to human engagement.
Choice is not a static work; it is a multidimensional piece that captures and responds to viewer interaction. By combining real-time sensor data with generative art, I aimed to create an evolving canvas where each movement shapes the artwork. This project reflects my view that art should be an interactive experience, establishing a dynamic link between creator and observer.
How the ESP32 Microcontroller Powers Choice
Choosing the right technology was essential to realize Choice. I opted for the ESP32 microcontroller for its flexibility and strong performance capabilities, specifically using the LilyGo T-Display S3, an ESP32-S3 development board with WiFi, Bluetooth, and a built-in display.
The ESP32’s powerful processing and connectivity features allowed me to implement real-time data processing and Over-the-Air (OTA) updates. This meant that the sculpture could evolve and improve without physical hardware changes. The microcontroller became the heart of the installation, managing everything from sensor data collection to generating the visual art displayed.
Sensors and Real-Time Data Processing
The interactive sculpture Choice captures and interprets movement with high precision through continuous data input. I integrated the MPU6050 accelerometer and gyroscope sensor to monitor movement across the X, Y, and Z axes. This sensor provides 16-bit raw data for both accelerometer and gyroscope readings, offering detailed insights into the sculpture’s motion.
To accurately process this data, the microcontroller converts the raw values into meaningful measurements. For the accelerometer, I set a full-scale range of ±2g, yielding a sensitivity scale factor of 16,384 LSB per g-force. The microcontroller divides the raw data by this factor to calculate acceleration along each axis, creating a dynamic interaction with the art by translating motion into visual output.
This setup enables the ESP32 microcontroller to accurately detect significant movements. Upon detecting such motion, sensor data is captured and processed instantly, allowing the microcontroller to activate generative art functions that translate each unique movement into dynamic visual patterns on the display.
Real-time data processing is critical for delivering an immediate and immersive experience. The ESP32 analyzes accelerometer data to identify specific gestures, such as upward or downward movements, which then trigger the generative art algorithms. This ensures that each visual outcome is unique, shaped directly by the viewer’s interaction.
// Read accelerometer data
sensors_event_t a, g, temp;
mpu.getEvent(&a, &g, &temp);
// Convert raw data to acceleration in g-force
float accelY = a.acceleration.y;
// Detect significant motion along Y-axis
if (abs(accelY) > threshold) {
// Record the current time
unsigned long currentTime = millis();
// Trigger the generative art function with the acceleration value
generatePixelArtWithAnimation(accelY);
}
Generative Art Creation Through Viewer Movements
In Choice, transforming movement into art involves a direct, responsive process. When the accelerometer detects a qualifying gesture, the microcontroller activates a generative algorithm that renders dynamic visual patterns on the TFT display. Each movement modifies specific variables within the algorithm, producing a unique generative art piece.
For instance, the strength and speed of a gesture might alter the color palette or complexity of the patterns displayed. This ensures that no two interactions yield the same outcome, positioning the viewer as an active co-creator. The immediate visual feedback fosters engagement, making the experience both interactive and personally resonant.
Open-Source Commitment and Community Building
A core principle of Choice is the commitment to open-source development. Transparency and collaboration are essential to fostering sustainability and innovation within art and technology. To support this, all code, including the firmware and web application, is freely available for review, modification, and enhancement.
By making Choice open-source, I aim to encourage contributions from artists, technologists, and enthusiasts alike. This collaborative model helps build a creative community where ideas are exchanged and evolved. I hope this foundation inspires others to expand upon it, exploring new directions in interactive sculptures and data-driven art.
Subscribe
Join Steve’s Newsletter & connect with a community of art enthousiasts, new media artists & collectors exploring the intersection of art and technology.
Join Konnekt, Steve’s exploration of human behavior, where artistic research intersects with technology and AI, digital media, and new media art. Read how technology-driven creative processes offer insights into our perceptions of self and society.
The Role of Firmware and Web Application in “Choice”
The custom firmware developed for Choice plays a key role in managing complex interactions across hardware components to ensure smooth functionality. Written in C++, this firmware is responsible for processing sensor data, controlling the display, maintaining WiFi connectivity, and supporting Over-the-Air (OTA) updates for easy remote updates and maintenance.
In addition, I created an interactive web application, the Extension Manager, using React JS and MySQL. This application adds depth to the viewer’s experience by providing additional interaction layers, seamlessly integrating with the sculpture. This connection of the physical and digital worlds enhances the experience, making it both comprehensive and interactive.
Integration with External Installations
Choice is designed to extend beyond a standalone sculpture, integrating seamlessly with external installations for enhanced interaction. By connecting to the Choice Extension via USB-C or WiFi, the sculpture can transmit generated data, working collaboratively to initiate new “pixelverses” of decision-based imagery.
Each new generative image is temporarily stored in a database and accessible via a QR code. Viewers can scan this code to reach a validation interface, where they decide the outcome of their creation. This process not only enriches viewer engagement but also builds a collective gallery of generative art, representing a community of shared interactions.
The Future of Interactive Art and Technology
The development of Choice has affirmed my conviction that merging art and technology opens vast potential for the future. Interactive installations like this can transform the way audiences engage with art, fostering experiences that are both immersive and participatory.
I foresee a future where sensor data becomes a standard element in artistic expression, and microcontrollers like the ESP32 play a central role in installations. As technology evolves, the possibilities for developing increasingly complex and responsive artworks will only continue to grow.
During my research for Choice, I also investigated the integration of blockchain technology through Physical Backed Tokens (PBT), offering a novel link between the physical artwork and digital ownership. By embedding PBT into the project, Choice could serve as a model for how embedded systems in art can extend beyond traditional experiences, opening doors to immersive, secure, and interactive digital ownership models.
Choice exemplifies the potential of embedded systems in art, and incorporating blockchain creates further opportunities for innovation. Additionally, I designed multiple 3D-printed case versions for Choice, enhancing both its aesthetic versatility and functional protection, which makes the work more adaptable and durable for various installations.
Overcoming Challenges and Lessons Learned
Creating Choice has been an ongoing journey marked by both technical and creative challenges. Integrating hardware components like the MPU6050 accelerometer and the LilyGo T-Display S3 required planning and extensive testing to achieve seamless functionality. Ensuring real-time data processing without lag was essential to preserve the interactive experience.
Developing custom firmware in C++ pushed me to deepen my expertise in embedded programming and ESP32 development. Managing WiFi connectivity and implementing Over-the-Air updates introduced further complexity, yet these elements were crucial for the sculpture’s responsive capabilities. These challenges reinforced the importance of precise design and the role of persistence in overcoming technical hurdles.
Community Collaboration and Future Projects
Although Choice is yet to be officially released, it exhibited at the British Art Fair 2024 in the PIVOTAL: DIGITALISM section at Saatchi Gallery.
After the exhibition, I released the software as open-source, fostering collaboration and building a community around the project. I’m eager to connect with creatives in interactive media, sensor-driven art, and microcontroller applications, hoping to inspire new developments in generative algorithms and sensor integration for enhanced interactions. Sharing Choice is an invitation to collectively explore the vast potential in this field, and I look forward to seeing where this journey leads.
Conclusion
Choice embodies the intersection of art, technology, and human engagement. Through the use of the ESP32 microcontroller and sensor integration, this installation goes beyond generating static art—it evolves in response to viewer participation, making art a collaborative and dynamic experience. This project illustrates how technology can expand creative boundaries, fostering a shared, interactive encounter.
With a commitment to open-source collaboration and ongoing innovation, Choice aims to inspire others to delve into the possibilities of merging art with technology. I invite you to explore the project’s repositories on GitHub, experiment with the code, and share your insights. Your contributions are welcome and essential in shaping the future of interactive art projects like Choice and beyond.
