Graphical User Interference

In the summer of 2020, Aarhus University hosted Critical Interfaces Analysis, a three-day summit intended to “critically analyze potential disjunctions between the desires of users and the maximization of profit that goes hand in hand with a cultural interface industry.”1 Four years prior, the first Interface Politics Conference was held in Barcelona in the fall of 2016. As a precursor to the conference, Hangar hosted Poetic Deconstruction of the Interface, a series of workshops led by artist Joana Moll that sought to “stimulate and re-appropriate subjectivity, an essential process in the generation of critical thought about the true nature of technology.”2 The workshops were organized into three sections: revealing the physical layer of the interface, deconstructing it, and re-articulating the interface’s ideology. These examples point to sustained academic interest in the cultural criticism of interfaces, but also an interest in their artistic subversion. What would it mean to think about the interface as a material that can be reworked and transformed into taking a critical stance about its own biases or context?At a foundational level, to think about the construction of an interface is to think about its temporality. When viewed at their intended speed, interface animations—such as the macOS Genie effect—shape time into a smooth, fluid experience. By design, each frame organically flows into the next to minimize disruption. As an experiment, I recorded this effect (the vacuum motion that happens upon minimizing a window) and slowed it down to a fraction of its speed. The experience became cinematic: I was no longer in control of the screen and its unfolding contents. Each frame and the gaps between them became hypervisible. At first glance, it seems evident enough that the interface is a planned sequence that users participate in, but that gives the illusion of control. Slowing down the interface disrupts its sense of seamless time, and to watch it play as a recording suggests the interface’s flexibility for narrative and storytelling.The process of creating a user interface is similar to creating a storyboard for a film. Each possibility of movement is planned out within a predetermined pathway. How might an interface inform the narratives that are experienced through it? At the very least, it seems evident that the seamlessness of the interface shapes the interpretation of information. An interface is designed to keep the user’s movement flowing, but there can be little room for spontaneity or criticality. What if the gaps between frames could be the starting place to pose alternative narratives and structures? What if users are in the optimal position to do so? I would like to posit a practice of repurposing software that a) critically challenges the quotidian interface and its users, and b) aims to re-imagine the user as an active shaper of their digital experiences.Graphical user interfaces (GUIs) may be moving images, but what would it mean to think about them as sequential narratives? An interface rarely creates narrative in the linear sense, but it can be operated akin to a camera or animation software by structuring and recording screen movement. This technical affordance opens up GUIs to a discourse of montage, which weaves into a history of software and user subjugation. Beyond their means to create videos, GUIs are also the foundations on which narratives are experienced. Johanna Drucker suggests that graphic devices are embedded with “ideological values” that inform the navigation of narrative content.3 In this sense, the narrative encompasses dominant cultural biases that influence user pathways within the interface.As a visual access point to computational power, GUIs accommodate choice within these pathways to a certain point. The user’s behavior is mostly encoded by the interface design. Visual signifiers borrowed from material things—like a magnifying glass—help explain a task but abstract the computer’s internal operations. As Wendy Hui Kyong Chun has observed, screens and software conceal the code and hardware needed to run them, facilitating a kind of conceptual dissonance.4 The user-friendly face of software also masks the brutal legacy of human labor exploitation and natural resource extraction deployed to make internal components such as batteries and processors. Traces of the supply chain and its violence invariably contextualize the social, informational, and artistic narratives that disseminate through our screens.What defines an interface narrative? A helpful analogy for thinking through this question could be the design of a museum exhibition. In a typical large-scale exhibition, a visual system provides guidance for navigating artifacts in a physical space. This may include signage, wall text, and interior architecture. The system is not only for navigation but also serves to contextualize the artifacts. This visual system can hold implicit assumptions about an ideal way to engage with an exhibition. For example, the formatting of the wall text, which artifacts are given the most visibility, and what is not included. These choices might not be explicitly addressed within the design of the exhibition but they would still express the priorities of the stakeholders involved. These tacit values can have the power to direct the museum experience in a pronounced way because when they are not stated, they are understood as commonplace and conventional.What is common about GUIs today? Autoplay, AI assistants, predictive typing, and never-ending scrolls are pervasive. One might reasonably think an interface values seamlessness, neutrality, and efficiency. GUIs are smooth and seductive, designed to be neither interrupted nor interrupting. Like a free fall, this sets the user in constant motion, which becomes harder to notice over time, allowing power to operate covertly. The more seamless an interface is, the more calcified the user’s pathways.Sometimes the seams of an interface are visible. One example is how the perspective in Google Street View transitions between ground view and aerial. As the gap between perspectives artificially fills in, the image becomes ambiguous and disorienting. This is an aesthetic counterpoint to the neutral legibility of Google Street View. What does it mean to switch effortlessly between these two perspectives? What is the perspective of the in-between, and what kind of subject does it create? How might the in-between be reimagined? The utilitarian aesthetics of these interfaces have blind spots where possibilities for play and inquiry emerge. Implementing counternarratives in these blind spots could be a way to engage with the interface’s covert power.I would like to name this counterpractice “Graphical User Interference.” This GUI is not concerned with creating new interfaces, but leveraging what already exists. Graphical User Interference is interested in digital experiences where the narration is unreliable, undefined, or pluralized. It uses techniques presented by the interface to reframe and reorient. It takes advantage of ambiguity to propose alternatives to the monetization of attention. Graphical User Interference aims to grapple with software’s biases and reimagine them through play. At its core, it is a call to problematize and radically transform the practices of users.Through my research, my understanding of critical interface practices has broadened. I’ve recognized that artists and designers have created many methods for subverting the medium. They make unusable websites, desktop films, and Google Street View photography. They make spreadsheet art, programs that obstruct the interface, and infographics that visualize a website’s carbon footprint. A budding list and by no means complete, this essay explores several areas where critical interface practice is ripe for investigation: default aesthetics, re/de-narrativizing geographical space, desktop cinema, and benevolent software.Artists have engaged with the formal qualities of the default operating system and browser for decades. Alexei Shulgin and Adam Cruces come to mind, both of whom frame the desktop arrangement as an encounter with subjectivity in their respective works Desktop Is (1998) and Desktop Views (2012). Emilio Gomariz is an artist who creates psychedelic animations out of banal windows and folders. Ryan Kuo, who technically creates software, makes use of default aesthetics to draw the user into a narrative experience that places into question their own subjectivity and sense of choice.Kuo’s Family Maker (2017) is an interactive macOS application that plays on the trope of the system dialog box. Family Maker confronts the user with a series of pop-ups that represent a family: a father, a mother, and a son. The user maintains these entities’ well-being by interacting with the pop-ups, which proves to be a balancing act. One pop-up titled “Time” has two buttons: “Save” or “Waste.” Another titled “Ego” has a drop-down menu with two options: “Object” or “Subject.” While the narrative of Family Maker is compelling, it also emphasizes the binary subjectivity that software perpetuates. The option to choose one or the other—“Ok” or “Cancel,” “Yes” or “No”—is not an empowering choice but an extension of the software’s control apparatus. Family Maker finds playfulness in this seemingly rigid form. In a somewhat humorous and tragic fashion, Kuo subverts the expected behavioral response to a pop-up. The familiarity of this experience pulls the user in but they are simultaneously pushed toward other meanings and associations, and are ultimately led to reflect on the efficacy of their choices.GPS software such as Google Maps has changed urban navigation and our logistical relationship with cities. Some artists have identified other uses for it that center around historical and embodied storytelling. Queering the Map (2017) by Lucas LaRochelle builds an archive of queer memory and experience using OpenStreetMap data, a community-built testament to the relationship between colonization, mapping, and counterstruggle. GeoGoo (2008) by JODI is an interactive work that explores the signs and symbols of Google Maps as free-floating, destabilizing the user’s orientation within the interface. In his project Vacated (2013), Justin Blinder developed a method of visualizing gentrification patterns in New York City using Google Maps data, a project that predates the official Google Maps time machine tool that now allows users to view backdated photographs of urban streets.Vacated was made using an algorithm that cross-referenced Google Street View photography and PLUTO (Primary Land Use Tax Lot Output) datasets to find images taken at the same intersection over different time periods.5 Blinder then used these images to illustrate the jarring contrasts between past and ongoing urban developments. These urban changes are not surprising and are of course documentable by other means, but the implication is that Vacated was possible as an indirect consequence of technological power and corporate surveillance. Vacated does not address how Google itself might influence urban displacement, and it is unclear whether it served as a catalyst for the time machine tool, but it places into question the social meaning of Street View images.It is worthwhile to reiterate here that Graphical User Interference is not necessarily about hacking or rewriting technology, but responding to entanglements of power that crystallize on the surface. Vacated illuminates the potential in hijacking data and reordering it to present a different perspective. The current time machine tool is relatively fixed, and thus may oversimplify and obscure other potential connections within the data, but it is a starting point for users of the Google Maps software (and inhabitants of the urban communities it surveys) to make their own interpretations. Interventions on the level of surface and meaning are already well suited to the expertise of the user.In their 2018 extended essay Digital Tarkovsky, Metahaven contemplates digital platforms as a form of cinema. For Metahaven, connecting cinema with interface is to partly say that the means of display have blurred with the means of production, creating a more accessible kind of filmmaking.6 Aside from having built-in high-definition cameras, anything shown on a screen is video and audio recordable: typing out words in a document, opening a file, playing a video in the browser, etc. For filmmakers such as Kevin B. Lee and Chloé Galibert-Laîné, GUIs constitute the means of constructing and presenting a film, where the organization of folders, files, and windows build up the mise-en-scène.Lee specializes in the desktop documentary, which “seeks to both depict and question the ways we explore the world through the computer screen.”7 Central to his practice is the computer desktop as a tool for research, argumentation, and presentation. Lee orchestrates user interactions to create linearity and transition. For example, in the documentary short Transformers: The Premake (2014), the desktop wallpaper image in one scene foreshadows a YouTube video that follows. Voice-overs, text documents, and internet comments also allow Lee to provide multiple forms of narration, with no clear central narrator. The ease of moving between desktop and browser transgresses the boundary between “local” and “live” content. This demarcation no longer feels significant when the interface is intended to be watched. Lee’s desktop films invert the personalized intimacy of the desktop and transform the anonymity of standardized UX gestures into a kind of narrative agent.This method of filmmaking acknowledges that there is an intentional seduction to GUIs, and filmmakers such as Lee exploit this quality to create films that are critical of the screen and its values. More importantly, this method is about access. Desktop cinema is emphatically about making films as a user, which would mean reframing the user’s assumed agency and identity to encompass a variety of motives and interests beyond the ideal behavioral course that is encoded into the interface design.Sometimes, third-party software is designed to interrupt the normal flow of other software. There are browser extensions that block ads and some that bypass paywalls. F.lux, which was first released in 2009 by Lorna and Michael Herf, gradually changes the color temperature of the screen to better suit our inner circadian rhythms. This category is a slight outlier from my initial definition of Graphical User Interference: these are applications made inside-out by developers, not interactions repurposed by users. However, this approach brings up an important question: if software is designed to manipulate the user’s behavior, would it be ethical to do so if it were for their own well-being?Flight Simulator is an iOS mobile application created by Laurel Schwulst and Soft in 2019. The objective is straightforward: the user chooses where they would like to travel, and their phone is set to airplane mode for the exact flight duration, providing an opportunity to experience that time any other way. Equal parts fantasy and reality, Flight Simulator playfully asserts the experience of restricted technological access with the intention of shaping time into something outside of itself, something that can be lived but not digitally tracked. It acknowledges the addictive properties of smartphones, implies the behavioral limitations of screens, and points toward the unstructured possibilities beyond them. It might seem redundant or even chastising to emphasize “non-screen time” but it is worthwhile to situate this within the context of 24/7 anywhere capitalist production, which mobile devices undeniably facilitate. While ad blockers and paywall removers have a literal function, Flight Simulator is more speculative. How can we reimagine and reclaim our own subjective experience of time outside of devices that were designed to monopolize and standardize it?Disrupting the flow of software poses other questions. In the case of ad blockers, locating the intervention is unstable. Ads are typically designed to distract the user from other tasks, but they have been somewhat normalized within the browser. Simply blocking the ad doesn’t interrupt the flow of the browser either, nor does it encourage the user to question it. This is the potential weakness of benevolent software. New software conventions become accepted and integrated into its seamlessness over time. In the case of F.lux, the gradual color shift of the screen becomes more of a slight gesture, rather than cultivating a belief that excessive late-night screen use is harmful. Disruptive software must take this into account and challenge the user on a perceptual level, not only a practical one. Flight Simulator seems to do this because it asks the user to adopt a temporal mindset that is outside of manufactured seamlessness.Digital tools are not neutral but carry histories and social values in their visual representations. Icons of folders, pens, and envelopes explain certain functions but also point to a history of white-collar office culture. On a screen, everything exists as a double. With no clear-cut way to represent a computer’s hidden functions, dominant cultural symbols are invoked to fill this gap. These representations are not fixed, however. There is versatility in how users might imagine other visual metaphors for their digital tools. The more soft and pliable a metaphor is, the greater the yield of diverse interpretations.I don’t want to suggest that repurposing GUIs will fundamentally change a user’s agency within them, nor do I intend to say that all GUIs should be repurposed. This is about recognizing that the intended use of software creates an idealized user, but as demonstrated by these practices, artistic appropriation can cultivate nascent desires and subjectivities that question whom such an idealization is serving. the use of GUIs can shift how users imagine and their relationship to their software is a and diverse practices are to understanding the values that from the graphic where the of many are and made over