A Punching Bag with Computer Vision Leverages Motion for Creative Output

General project description

Developed for and initially displayed at the Jerusalem Design Week (2021), the Punch.Generate.Freak interactive installation invited its forty-thousand visitors to workout with punching bags as part of a physical-digital creative process.

Spread throughout Hansen House, a century-old leper asylum turned cultural venue, the lively “Design Week” showcased exhibits, installations, performance, live music and events all relating to the chosen annual theme; The Circus¹.

This umbrella theme helped frame the interactive punching bag installation with reference to sports spectacles, extraordinary human capabilities, and achievements. Tucked into the work’s title is also a tribute to the 1932’s traveling-circus horror film Freaks² directed by Tod Browning, thus hinting to the unexpected forms generated in the workout process that have diverged from the familiar to the uncanny.

The Design Week’s Punch. Generate. Freak. interactive punching bag installation leveraged participants’ movement and striking interaction with a punching bag as a creative force generating additional novel bags, constituting “snapshots” of the interaction (fig. 1). Through computer vision, both boxer and punching bag were identified during a workout and then transformed in real-time into two identical digital cylinders (virtual punching bags). The two became equals; in form and symbolic worth. The computer monitored the duo’s “dance” and captured moments of their intersection, fusing them into a single body (fig. 2), then outputted volumetric 3D files. These frozen and transformed interactions were then realized in the physical world as a series of custom-fabricated Freak punching bags.

At the core of the project was designing a process and workflow, although eventually its realization was quite substantial in its own right, including interior design, digital display design, sound, and lighting design, and even merchandising (a boxing club-like logo representing the freak bag outcomes printed on T-shirts).

The project brought together four creatives from various fields pooling their skills; Sarit Youdelevich³, a code-fluent Designer-researcher working with digital technology as a raw material, collaborator and partner; Dr. Doron Altratz⁴, artist and researcher with a focus on media studies and digital heritage, computational photography, human-computer interactions, and a former kickboxer; David Oppenheim⁵ an artist and academic with background in music, and media design; Prof. Dov Ganchrow⁶, product designer and academic, with an interest in digital/physical making technologies, and former national TaeKwonDo champion.

Understanding Punching Bags

Punching bags are in essence training aids, manifesting in many variations (form, material, fill, mounting…), to improve and condition fighting skills (conditioning of limbs, stamina, strike precision, legwork, combinations, etc.). They have been used for centuries in one form or another in military, sport, and martial arts training, and under a wider definition would include both designed and improvised striking targets ranging from a sand filled bag for bayonetting to a young banana tree used for low kick shin conditioning.

A punching bag can be designed for more general striking interaction or be tailored for a very specific application. This can be exemplified within traditional Japanese martial arts training in the more general striking Makiwara board⁷, a rope-encased wood post or wall-mounted board with horizontal incisions that can be struck with both open and closed hand techniques as well as kicks. An example of an extremely specific striking target would be those common in display of Japanese sword cutting skills, a vertical cylindrical object⁸ constructed of goza, the top layer of the traditional tatami, wrapped around green bamboo mimicking the flesh and bone of a human limb.

Common contemporary use of the term “punching bag” conjures an image of a suspended filled cylindrical target, sewn from a robust textile such as vinyl, canvas, or leather. Suspending, adds to the shock absorbing properties of the target (reducing stress on the trainee’s body), may allow the bag sway thus increasing dynamic interaction, and removes mounting obstructions from the workout space that may hinder movement during a workout (such as limiting striking approach angels when the back is mounted with a bracket to a wall, or limiting a kick’s trajectory if the bag is standing on a weighted mount).

Since training in martial arts and combat sports is most often under the pretext of overcoming an adversary, training aids such as punching bags are representative of a human foe. Bag size and weight are often anchored around this thought as are the fighting techniques used to engage with it. While some striking targets are sculpturally minimalist, expressing a human representation (such as the wooden chinese Mu ren zhuang⁹ also referred to as a “Wing Chung dummy”), others are realistic reproductions (such as the molded polyurethane torso known as “BOB”- Body Opponent Bag), for the most part typical punching bags will be strictly cylindrical, in part the technological outcome of simple pattern sewing and a volume that will disperse the internal forces of the fill under pressure (blows) in a mostly homogeneous manner. The cylindrical form also conveniently allows landing blows on identical receiving surfaces irrespective of how much it has rotated on its axis with the aid of a swivel fixture.

Punching bags in contemporary culture and digital age

The typical cylindrical hanging bag will be a familiar icon to anyone today, regardless of training background (or lack thereof), through vehicles of popular culture such as film and graphic representations. An association that transcends geographic locality and could just as easily exist in a North American boxing gym, as a French Kickboxing club, or a Thai open-air training camp. At the core of the iconic image/effect, the bag conjures up the raw explosiveness of human bodily force, sweat, conditioning, and pushing of oneself physically and mentally. The bag image has the human embedded in it, and with it force.

It is this popular image that is channeled by Romanian artist and professional boxer Szilard Gaspar¹⁰ into his version of the modern tradition of action painting (and sculpting), and performance art, while retaining the artist (individual identity/biography) as an inseparable part of the artwork. Just as the previously mentioned punching bag image harbors the human in it, so do Gaspar’s bag artworks harbor the artist. Gaspar re-envisions the traditional framed painter’s canvas as a punching bag, applying “brushstrokes” through a flurry of boxing combos, after first dipping his training gloves in paint. In his 2017 work Full Contact Gaspar spars with a soft clay hanging bag, deforming – or rather giving it form through repeated blows.

In his work for Louis Vuitton (part of the “The Icon and the Iconoclasts” project¹¹, fashion designer Karl Lagerfeld springboards off the punching bag image with its uninhibited and rough association, to create contrasting, elegant, and perhaps humorous tailor-made haute couture “gentleman’s bag” out of exquisitely crafted leather and with matching gloves. The clash of contexts, of opposing behaviors associated with the original bag’s image with that of the fashion brand’s, is what creates new meaning and holds our attention as we try and comprehend this new hybrid, linking leather handbags with leather hanging bags.

In parallel to the punching bag’s reinterpretation in the cultural realm, the bag is today also increasingly being absorbed into/or infused with digital culture, as a consumer product. The era’s prevalent obsession with quantified self¹²; the self-tracking, measurement and quantification of aspects of daily life and bodily function using technologies such as apps, wearable smart devices, sensors or external sensing devices, has not spared the worlds of martial arts and combat sports; smartwatches measure the distance, speed, grade, and heart rates during a warm-up jog, while wired body protectors¹³ measure the force of blows in a sparring session.

Technological approaches to the bag-punching interaction setup can vary; at times sensors are embedded into workout gloves (e.g. Move It Swift Smart Boxing Gloves¹⁴, at times they are paired with the bag itself (e.g. the I-Perskin smart punching bag cover¹⁵, or the whole interaction deferred into the virtual world (e.g. VR Liteboxer¹⁶). Typical sensing data include: the amount, precision, and force of blows; workout time; calories burned; and could also include blow trajectory and reaction time. Often these devices are paired with an application for further data generation; performance progression over a longer period of time, averaged parameters, goals, feedback, and even community connectivity.

The need for integrating technological sensing into people’s life has led to ongoing research and development of both market-oriented and critical design approaches to Human-Machine Interaction (HMI)/Human-Computer Interaction (HCI), by now well-established fields, in both academia, the tech sector, and the arts. With the balance of the interaction swaying depending on the project, and motivations, with at times human, and at times machine, taking the upper hand, or mutually interacting in equilibrium. Input information from the human side can be obtained casually (such as monitoring of physiological factors) or actively initiated (such as voice-activation, eye movement, motor control etc.).

Examples of casual physiological measurements directing output can be found in the critical interactive video projection work Perception iO by UK artist Karen Palmer¹⁷ shown at the Smithsonian’s Cooper Hewitt Design Museum in New York. Her work relied on monitoring visitors’ facial expressions, detecting, and defining emotions unconsciously expressed in visitors’ reactions to her work. The narrative in the projection changed based on the visitors’ detected bodily responses to the first-person viewed content, in this case exposing ethnic bias towards suspects in an unfolding urban crime scenario.

An additional topic to consider in the physical-digital interplay is the interaction logic that may differ and may be of an alternative character in each of these realms. Explicit commands, or machine reaction to casually obtained data, can retain a “cause and effect” rationale, when for instance swinging a ping pong paddle and watching a virtual ping pong ball on a VR headset travel on an anticipated trajectory. The physical-digital linkage, however, does not dictate a linear or topical chain of events. Swinging a ping pong paddle, could through the computer vision registering the motion and the coding of a response to that detection, produce a specific unrelated sound for instance, or while wearing a VR/AR headset, produce a downpour of hundreds of thousands of tennis balls from the sky. The “translation” of input data into output, need not adhere to what we might expect based on our life experience or what physics of the material world regulates.

Punch. Generate. Freak. interactive punching bag installation

The herein described project was a site-specific work installed in an upper-floor gallery space in Jerusalem’s cultural hub the Hansen House, a historic stone building constructed in 1867 as an asylum for Lepers. The installation space was transformed, for the duration of the 2021 Jerusalem Design Week, into a Blade Runneresque boxing gym, complete with mirror-paneled wall, foam floor tiling, hanging bags, as well as live projection and warm neon-like tinted lighting (fig. 3).

The Punch.Generate.Freak project was designed to be a visitor-driven creative process that leveraged bag workout data for the designing of new, sculpturally expressive, conjoined punching bags. The installation’s components and spatial placement supported and emphasized the project’s Freak bag generating workflow.

Due to the character of the installation and anticipated visitor volume (tens of thousands of visitors), solid technical and technological solutions were called for. The project therefore made use of computer vision software “understanding” (fig. 4) what it was seeing as input data, judged to be more reliable than the use of failure-prone physical hardware such as impact sensors, wiring, or accelerometers.

A simplified chronological description of the process’s workflow is as follows:

• Visitor strikes black color-coded “generating” punching bag (fig. 5).

• Camera films the interaction, with the software detecting both bag and visitor based on motion, framing them individually with graphic squares.

• Software captures a “snapshot” of the interaction based on overlap between detected entities coupled with their motion.

• Documented “snapshot,” essentially two overlapping square forms, is transformed into cylindrical forms, shifting from digital 2D to digital 3D (fig. 6 et 8).

• Overlapping cylindrical forms are “unwrapped” back to flat shapes to obtain fabrication-ready textile patterns.

• Patterns are plotted and cut out of a durable reinforced vinyl sheet.

• Patterns are manually sewn by a craftsman, reconstructing the overlapping cylindrical forms as a new conjoined bag, complete with zipper closures, anchoring, and hanging straps.

• New Freak bag is hung in a lineup with other previously (captured interaction “snapshots”) prepared bags for visitors to continue to strike (fig. 9).

The inclusion of a real-time projection in the space was deemed a necessity so as to give visitors a glimpse of the unfolding design process, as well as feedback on their physical “generating” punching bag interaction (fig. 10). Most of the projection’s graphic space was devoted to real-time interaction video with all its dynamic presence; it allowed visitors to see what and how the computer was seeing.

The lower portion of the projection frame included a timeline with previously captured interaction “snapshots” (fig. 11) proceeding to the right in steps where they are finally calculated as volumetric forms and visualized as 3D objects.

This display design encouraged visitors to work out and strike at least up to the point where they were “captured” by the software (signaled by both visual and audible feedback), kicking off a chain of events leading to the 3D Freak bag form.

As actual fabrication of the newly generated life-size bags in real-time was not feasible for various reasons, previously generated bags were fabricated and included in the space for additional visitor engagement, closing the loop of striking-to-create-something-new-to-strike.

Over the course of a few months leading up to the Design Week event, along with the debugging of the Freak bag-generating (and object/user recognition) code, sculpturally relevant and physical-world-realizable interaction Freak bags were curated from a large set of output forms (fig. 12) and selected for fabrication.

The tedious fabrication process relied equally on the digitally produced patterns and the skillful craftsmanship of a leather fashion accessories tailor, trained in the Moroccan craft tradition. With over fifty years of experience, the craftsman worked in relative solitude with his heavy industrial sewing machines, cutting patterns by hand and snipping away at loose ends of sewn seams. This very traditional, personal, and human stage of the workflow contrasted strongly with the preceding pixel and computational interaction that had taken place in the digital realm. The designed Freak bag generation weaved from the very physical to the digital and back again into the physical world in a pronounced manner.

On Design and Conflict

While the Punch.Generate.Freak project is a very visually comprehendible tool for the creation, through documentation and processing, of novel conjoined punching bag forms, it can also be understood on a more representational level.

Observed from outside the field of the martial arts, the training in delivery of blows can be viewed as a pursuit of the destructive, while for practitioners the opposite is often true – it is a constructive undertaking, building stamina, skill, confidence and more. A practical application of this notion can also be found underlying in fabrication and form-giving processes making use of blunt force; industrial explosion/blast forming for metal sheets¹⁸, or simply the repeated striking of a blacksmith’s hammer on the orange glow of a metal rod.

Traditionally the punching bag is understood as representing an imaginary foe, though through the computer’s “eyes” they are equals. Once fabricated their former identities are indistinguishable, now both are punching bags, sewn together for eternity, with the new form an identity into itself. Due to the previously discussed embedding of the trainee in the punching bag icon, the striker and the struck, continue to exist in the work, they too indistinguishable from one another.

The distinction between the active agent and the passive insentient object becomes referenceless once both these entities are fused in the work into a unity, which in turn may be seen as a new entity whose trajectory, movement, and contours are on the one hand the mechanical causal outcome of its components, but on the other hand, via its “freakishness” – carrying not just the outcome but the image of agency – exceeds, the sum of its parts. Moving between and transcending both its sense as destructive or as constructive, the work becomes a reverie on the bitter dynamics of conflict – in which the aggressor and the retaliator, via the web of their entanglement and intersubjectivity, rather than acting against each other, are shown to be in fact bounded together, creating a new reality deformed by violence, in which the agent is less than human, and the often dehumanized foe is no longer just an object¹⁹.

By retaining identifiable partial punching bag geometry or features (such as hanging straps), as traces of the previous entities, the fussing is legible in the newly formed object, allowing for an understanding of what has occurred. The new entity would not exist if it were not for the forceful interaction that preceded. Conflict is its life force.