Mundane Game

Anyone whose work and recreation takes place primarily on a computer will find that their daily repertoire of gestures–or the motions made by their hand and body to communicate and accomplish tasks–has shrunk considerably from what it would have been in the pre-digital age. A near-uniform set of hand gestures–point, click, scroll, swipe, tap, and zoom–are used to accomplish virtually every task across every device with a screen in the 2D virtual world (Beugnet 2020). The acts of wandering, searching, finding, choosing, examining, making, communing, and spectating–actions that once required a variety of different motions, gestures, and touches–have been condensed to what is allowable by a keyboard, trackpad, mouse, and screen. 

An example of the flattening of gesture is the change that has taken place in film editing techniques. In the days of analog film, editing meant physically cutting, joining, winding and unwinding rolls of film stock. When watching footage of analog film editors at work, one can see that each editor has their own way of handling the medium and their tools, manifested in subtle ways: a unique bend to the body or a characteristic flick of the hands (Manufacturing Intellect 2017). In contrast, in the 21st century a digital film can be edited solely with a single computer. Within the span of a decade, everyone’s gestures on these apparatuses have become nearly identical, whether they are cutting, joining, exporting the footage, or working with audio. No one’s fingers will physically touch the image at any point in a digital project. 

Within Natural User Interface (NUI) design practice, the flattening and condensing of gesture into a handful of streamlined motions is intentional. NUI is intended to design a universal set of “natural” or “playful” interactions between a so-called standard human hand and a computer (Cooley 2014, 28). However, despite NUI design’s self-characterization as natural, many NUI gestures (tapping, scrolling, swiping, pinching, and prolonged sitting or looking down) are actually quite unnatural, even for a so-called standard (abled, young, financially stable) human hand, to the point where repetition of the gestures over time will cause musculoskeletal disorders including hand and arm strain (Griffith, Mackey, and Adamson 2007), computer vision problems (Blehm et al. 2005), and diseases related to a sedentary lifestyle.

So how natural is NUI anyway? Its gestures do not have any one-to-one relationship with any gesture in the real world. In the real (physical) world, one does not click on a stranger’s face to find out their personal history; one does not scroll through nature and cityscapes as if they were webpages; one does not make decisions by swiping through them. The relationship between digital gestures and their function is not concrete, but abstract. One may mentally “wander” (scroll, swipe) the digital world and then “choose” (point, click) to go down a particular rabbit hole of information or interactions. Inherent in the language of digital gestures are acts of servitude: the ringing of a bell that calls an attendant, or the rubbing of a lamp that summons a genie; Siri and Alexa’s (feminine-gendered) presences are the digital helpmeets, or mechanical brides, that facilitate our desires. Drawing on feminist science and technology scholarship, Wagman and Parks (2021) posit machines as “social machines” that replicate and perpetuate social dynamics, a problematic phenomenon when the underlying assumptions treat place machines in the categories of tools, human companions, animals or creatures, or slaves (10).

Cultivating a wide repertoire of gestures is often considered recreational, at least for those normally reliant on computers. A digital worker can learn, say, to rock climb, play the banjo, or throw pottery, but these gestures are wholly extraneous to the physical requirements of their work. It would be very inefficient to conduct their paid labor with extraneous gestures–for example, doing something as silly as waving their hands wildly or tap dancing every time they tap the keyboard. Those extraneous gestures would not fit within the system of capital that governs the nature of their work and dictates their physical workflow. NUI design, therefore, is not simply based on the pursuit of naturalism or comfort, but is also driven by efficiency–even at the expense of healthy bodies and minds. 

To be sure, many people who do not primarily rely on computers for their work still require a specialized and unique set of gestures for their craft. For example, an experienced construction worker still should know, on a physical level, the best way to excavate different types of terrain with a specialized set of tools, and they may share this knowledge with others of their profession. Although these gestures may become mundane to that particular person over time, they are still specialized motions used by a small number of people, to accomplish specific tasks–in other words, they remain gestures rooted in a craft. But anyone who uses a phone, computer, or any digital machine will experience, to varying degrees, the same gestures, and the same flattening of gestures. Even a construction worker likely owns a smartphone: according to the most recent U.S. census data from a 2018 survey, 92 percent of households in the United States had at least one type of computer; smartphones were present in 84 percent of households, while 78 percent of households owned a desktop or laptop (Martin 2021).

Undeniably, many NUI gestures that govern human-computer interaction can feel novel and playful–this is why technology companies, like Apple, invite so many consumers to touch, play with, and marvel at their machines in brick-and-mortar stores. It can feel miraculous to commune with the computer, like a puppeteer with invisible strings, and feel as though it is responding to you through gestures alone. It is pleasant to experience a machine executing what its designers intended. And over time, a user feels an affinity for their machine; although it is identical to millions of other devices, repeated interactions with the object engenders familiarity and affection (Frykman and Frykman 2016).

Through repetition, gestures become mundane and integrated into the everyday as if there were no other alternative, in the same way that the “thumbs up” sign, descended from obscure origins, now feels like it has always been a part of humanity’s gestural lexicon (Fabry 2017). Digital gestures are no different: before the “swipe to pass,” however did we move from one choice to the next? When did the concept of “pointing” become so important to making decisions and navigating in digital space? Instead of aimless wandering, moving from one virtual space to another requires a concrete decision-making process before the movement even happens.

Human evolutionary anthropologists speculate that the use of tools, particularly those that required the dexterous use of opposable thumbs (Kivell 2021), shaped the ability and behavior of early humans. How we interact with digital tools will shape human consciousness and development just as profoundly. A funny turn of events:  an ever more thumb-centric culture seems to be at hand. As ancient tools drove early humans to rely on the thumb’s maneuverability, so did the invention of the typewriter keyboard introduce humans to typing with fingers and thumb. Now, digital media interfaces require the thumb to execute (rather than simply support) tasks like typing and swiping and other functions (Ramati 2021).

NUI gestures are here to stay. They may evolve with the development of virtual reality (VR), which generally calls into action more appendages, for example, headsets and motion trackers that track movements of the arms, head, and eyes. On a surface level, it seems that VR will increase the repertoire of gestures for human-machine interaction, and this may be true. But what VR gestures currently share with commonplace screen-based gestures is that each gesture triggers one (or a handful of) prescribed responses. In a virtual world–say, on the popular virtual world platform VRChat–there are limited and standardized ways to pick up a pencil, hit a tennis ball, or shake hands with another person/avatar. In the non-VR world, humans have personalized handwriting, signature ways of spinning a tennis ball across the court, and unique handshakes–distinctions that are lost when they enter the virtual realm. 

Perhaps human-machine interfaces will soon become advanced enough to replicate the individual traces of each human body and its unique gestures. Until then, cost-efficiency and convention continue to point towards the generalization of gestures, especially in regards to touch-input based tools. In his essay “The Question Concerning Technology,” Heidegger writes that enframing technology reveals the mystery–and power and danger–of its essence. It is worth questioning whether the technology that governs contemporary gestures is natural or inevitable? Do we want it? Should we disrupt the path that has been determined by the limited set of gestures allowable by current NUI, and if so, how? Writing of this path, Mizuko Ito posits that handheld devices generate a new kind of consciousness, by encouraging new habits of “sharing,” a “persistent alertness,” or a “new kind of personal awareness” (76) that is  “[a]ctive always unfolding, in-relation assumes the rhythm and time of a present that is always becoming, never still” (43). This persistent personal awareness climaxes in small actions, i.e., “the impulse to text, image, or post an update” (76). From an embryonic quagmire of urges, emerges the pre-circumscribed gestures to carry them out. A swathe of humanity has come to be governed by these fragmented urges and gestures, acting on impulses to ceaselessly update, browse, and purchase. The ease and homogeneity of gestures gives rise to the ease and homogeneity of modern-day consumption of images, services, and goods.

It is difficult to swim against this stream. I am typing this essay on a computer, repeatedly scrolling up and down and zooming in and out to better see the text. On other tabs, I am tempted to browse social media and do online shopping. It is a distraction from the labor of sustained thought–it prevents the labor of sustained thought. And yet (beyond writing the manuscript by hand), there is no other way. These are the most efficient tools available to me. In an attempt to swim at least in a crosscurrent, there is a double task to be completed: one, to point out the homogenization of human gestures as human-machine interaction becomes commonplace; and two, to identify gestures that remain unique and/or uncommon, whether those are the dying gestures of a trade, or gestures that protest against oppression around the world.

Films and Protest in China’s Era of Digital Authoritarianism

The long drive to the village of Songzhuang on the southeastern edge of Beijing is a maze of highways leading past beehive-like apartment buildings and dusty side roads leading nowhere in particular. I would have gotten lost if I were alone. Fortunately, I was accompanied by the independent documentary filmmaker Zhao Liang, who has frequented Songzhuang since the 1990s.

In fact, one of his early films, Farewell to Yuanmingyuan (filmed in 1995), documented the birth of the new Songzhuang—if this neighborhood of disintegrating buildings and overgrown alleyways was ever “new.” After the famous Yuanmingyuan artists’ community in Beijing was raided and demolished by the government, Yuanmingyuan’s residents fled to other pastures, maybe not greener, but with cheap property and farther away from the prying eyes of the central government. Songzhuang was transformed from a village with no paved roads or traffic lights into a colony of thousands of painters, photographers, sculptors, and mixed media artists. In Beijing proper, Yuanmingyuan has been replaced by the touristy 798 Arts District (think chic galleries and gift shops), and Songzhuang continues to exist as it has for the past decade: under the radar and an hour’s drive from the city center.

Zhao Liang’s name translated directly into English is “to shine light”, which is an apt moniker for a documentarian who has spent his career exposing China’s underbelly, including stories of heroin addicted rock-and-rollers (Paper Airplane), peasant petitioners airing grievances against the government (Petition), and dystopian mining operations in Inner Mongolia (Behemoth). But under the blinding summer sun, even Zhao Liang hesitated as we navigated the rabbit’s warren of Songzhuang. Every brown mud wall looked the same, papered with fading advertisements, and there were virtually no street signs; this makes residents who move between studios in the courtyards enclosed by these walls hard to track down.

We entered a street flanked by statues of dolphins that looked as if they had been stolen from Disneyland and then sandblasted, passed a bag of old clothes that someone had dumped on the street (“That’s disgusting,” the fastidious Zhao Liang commented), turned into a narrow alley, and finally stood at a doorway hidden by a curtain. Behind the curtain was a cool and verdant courtyard, welcoming after the heat and dirt. The courtyard’s owner/caretaker, a bearded man with pockmarks and fluffy hair, led us first into an adjacent room that served as a bar (stocked only with warm Tsingtao beer thanks to a broken refrigerator) and then into the dark maw of a small cinema. On that day, no film was showing, and there was no audience. The space was dark and empty but for eight rows of plush velvet theater seats. 

A few weeks later, I heard that there was to be a film shown in the cinema. Zhao Liang had already returned to work in his native Liaoning Province. Alone, I returned to Songzhuang, but nearly missed the movie theater. The dolphin statues were still there as signposts, but the bag of clothing was long gone. Passerbys said, “Yes, I’ve heard of it, but I don’t know where it is.” I finally found it when I location-tracked one of the filmmakers through his WeChat account. This must be how the police stalk people, I thought.

The local police are well aware of the artists—some of whom are on the government’s watch list—who live and work in the area. Local officials tolerate artistic boundary-pushing, up to a point, because art brings economic vitality to an otherwise podunk suburb. Developers envision a second 798 Arts District—and the money that would come with it. But this fine balance is overwhelmed when either side oversteps.

Case in point: the film being shown that night was made by a painter-turned-filmmaker, and it featured documentary footage of an artist being hounded by the police in the aftermath of a protest-performance he’d done in Tiananmen Square: he had stood in the Square and repeatedly punched himself in the nose until it broke and bled. The police caught on when the blood flowed from his nose and covered his white shirt. They dragged him away. Back home, the artist was hounded to the point where plainclothes policemen were stalking him in Songzhuang’s narrow alleys at night, and black cars surrounded his home by day. He couldn’t leave his house. To get food, he had to climb onto his roof and have a friend deliver it to him fastened to a pole. 

The year 2003 saw the founding year of the Beijing Independent Film Festival (BIFF; not to be confused with the state-sponsored Beijing International Film Festival). Started by the art critic and curator Li Xianting, BIFF was not exactly smiled upon by government censors. Despite this, thanks to the festival’s remote location in Songzhuang, it managed to screen films from directors who did not have a “dragon seal” of approval by government censors and, in certain cases, were overtly critical of Chinese government and society. In Songzhuang’s shaded courtyards, directors and audience members mingled between screenings to drink and talk about films that would never be shown in mainstream Chinese theaters.

All was well, inasmuch as they can be for independent operators in an authoritarian state, until 2012, the Festival’s 9th year. An American filmmaker and academic, J.P. Sniadecki, was a witness. In his recounting of events, as the courtyard venue filled with attendees for the premiere screening of Egg and Stone by director Huang Ji, local police officials suddenly flooded into the space. The police only allowed those with official invitation badges into the screening room, thereby reducing the audience by half. Forty minutes into the film, the electricity was cut and the screen went black. In the ensuing days, festival organizers and attendees and government representatives entered a sequence of negotiations and evasions as the festival stumbled on, with its screenings taking place in clandestine hotel rooms. After the 2012 crackdown, BIFF went on for only two years more until 2014, when police arrested festival organizers, and a blockade of hired men impersonating villagers prevented attendees from entering and anyone from photographing or filming the events. After that, the festival was permanently canceled.

On the bright side, Sniadecki, an affiliate of Harvard University’s Sensory Ethnography Lab, emerged from his years in China with footage for several films, including Yumen, a work done in collaboration with a couple of Chinese artists who lived in Songzhuang. Sniadecki first met one of the artists, Xu Ruotao, in 2012 at BIFF as police swarmed all around them. These kinds of chance meetings drive the collaborations between independent Chinese artists and their foreign counterparts.

While a noose tightens around the independent film community, China’s commercial film industry (approved by government regulators, shown in mainstream theaters) might be entering a Golden Age. In 2015, the super-patriotic action film Wolf Warrior—starring a heroic People’s Liberation Army soldier against a villainous Navy SEAL—topped Chinese box offices in its opening week. The 3D computer animated film Nezha (2019), raised the standard of Chinese animation above some frankly mediocre predecessors. Chinese conglomerates have gobbled up American production companies: in 2016, the Chinese conglomerate Dalian Wanda bought Legendary Entertainment, which is responsible for Christopher Nolan’s Batman movies, Jurassic World, and the King Kong franchise, among others. Wanda also owns AMC Theatres, which makes it simultaneously the biggest movie-theater operator and highest-revenue generating film company in the world. Wanda’s founder, the entrepreneur Wang Jianlin, served for nearly two decades in the People’s Liberation Army, was a deputy to the 17th National Congress, and maintains close ties with the Communist Party. 

While film cameras flourish in Shanghai’s production studios, facial recognition and closed-circuit cameras are the new norm on every urban street in China. These cameras document life with unblinking patience. They are observational cinema in the purest form. Recognizing a trove of prime raw material, director Xu Bing made the 2017 film, Dragonfly Eyes, which is woven together completely from found surveillance footage. In Chongqing, where there is 1 security camera for every 6 people, cameras are used to track down lost purses and to detract “social credit” when individuals commit minor crimes like jaywalking or failing to pay a bill. In Xinjiang, cameras are used to monitor and punish Muslims when they commit misdemeanors like grow beards, visit mosques, leave their house too many times, or use forbidden applications like Whatsapp and Facebook. In Hong Kong, pro-democracy protesters have attempted to subvert cameras through acts of self transformation by wearing cosmetics and face masks that look uncanny to human eyes but pass muster under mechanical ones. 

The recent Hong Kong security law, passed June 2020, punishes these actions by essentially declaring any suspected sedition, i.e. any protest, a crime punishable under mainland Chinese jurisdiction. Meanwhile, Hollywood’s collaboration with China’s mainstream film industry continues, though not without criticism. Disney’s latest live-action adaptation, Mulan, stars Liu Yifei, a Chinese-American actress who has expressed support for the Hong Kong police. In response, the #BoycottMulan hashtag began trending on Twitter. Users slammed Liu and dubbed Agnes Chow, a young Hong Kong activist who has been arrested for her actions, the “Real Mulan.”

Censorship is old news to independent filmmakers in China, even if technologies of surveillance are more sophisticated. Being censored has been in the fabric of Chinese independent film since the first DV camera was picked up. As Brian Hu aptly writes, “Independent Chinese directors always emit a collective groan when asked the same question by international audiences: what is it like to deal with the censors? Beyond the question’s inherent othering of an exotic “forbidden” China, it also reduces independent filmmaking to an act of resistance with only one intention: to oppose the government.” Not that it’s the sole fault of the audience—indie filmmakers know well the commercial and fame cachet that comes with the label “banned in China.” Famous artists like Ai Weiwei are aspirational examples of censorship at home, success overseas. Artistic boundary pushing is not only a political act, but an economic calculation: a way for disobedient artists to get noticed outside of China.

In China, censorship is simply a fact of life, no matter how much foreign film festivals adulate the theme. In Songzhuang, the film that I saw about the artist at Tiananmen would be seditious under any interpretation of the Hong Kong security law. At the screening, no one seemed to care. During intermission, the audience took a break from the stuffy theater to drink Tsingtao beer in the courtyard. The refrigerator had been fixed, so the bottles were cold. People were relaxed and congenial under the influence of alcohol and cigarettes; there was no underlying tension. There were no policemen in sight that night, but plenty of people—aspiring photographers, filmmakers, students—had brought their own cameras.