2nd version

master
Michael Murtaugh 3 years ago
parent 2cf605bd41
commit f8e386f88c

Binary file not shown.

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 121 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 149 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 119 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 122 KiB

@ -0,0 +1,176 @@
---
title: "torn at the seams: considering digital vernacular"
author: "Michael Murtaugh"
papersize: a4
---
> Processing is a free, open source programming language and environment used by students, artists, designers, architects, researchers and hobbyists for learning, prototyping, and production. Processing is developed by artists and designers as an alternative to proprietary software tools in the same domain. The project integrates a programming language, development environment, and teaching methodology into a unified structure for learning and exploration. […]
>
> Most of the examples presented in this book have a minimal visual style. This represents not a limitation of the Processing software, but rather a conscious decision by the authors to make the code for each example as brief and clear as possible. We hope the stark quality of the examples gives additional incentive to the reader to extend the programs to her or his own visual language.[^p1]
[^p1]: Casey Reas and Ben Fry, *Processing: A Programming Handbook for Visual Designers and Artists* (Cambridge: MIT Press, 2007), xxi-xxii.
Teaching programming with free software to media design students for years, I've resisted Processing as it has always seemed to me to embody a particular kind of solipsism of digital interactivity and graphics that I want my students to avoid.
> Solipsism: a theory holding that the self can know nothing but its own modifications and that the self is the only existent thing; also: extreme egocentrism[^solip]
[^solip]: <https://www.merriam-webster.com/dictionary/solipsism>
In the summer of 1996, as I was finishing my Masters degree, I met John Maeda who was just joining the MIT Media Lab to replace the recently deceased Muriel Cooper. Cooper was the first art director of the MIT Press, producing influential designs such as a 1969 catalog of the Bauhaus and the iconic MIT Press logo, a Bauhaus-inspired stylized graphical rendering of the letters “mitp”. Cooper started the Visible Language Workshop, later one of the founding groups of the MIT Media Lab, to research the intersection of publishing, design, and computation.
My friend Robin was a student in the VLW; through her I learned things like how Muriel would say "Swiss" instead of Helvetica. I took one class in typography at the VLW; among the many constraints we were given, we could only use one font, named for swiss typographer Adrian Frutiger.
Maeda created the Aesthetics and Computation group in part to continue Coopers research. Ben Fry and Casey Reas were among Maeda's first students. Maeda published a software system called Design by Numbers. It had extreme constraints such as a fixed 100 by 100 pixel size and monochrome-only graphics. The accompanying print publication also had a square format. DBN would go on to inspire Reas + Fry to create Processing. When one starts the Processing application, the default starting code comprises two commands: createCanvas and background. The default canvas output is square, and a single numeric parameter to the background command produces a gray-scale value, both echoes of earlier DBN constraints. Both DBN and Processing were written in Java, a desktop application that exists outside of the web but which can be used to publish sketches as "applets" embedded in a web page and published online.
![The p5js editor default starting code and output](images/p5_js_web_editor.png)
In 2004, Reas co-developed an exhibition at the Whitney gallery called Software Structures. Inspired by Sol Lewitt's wall drawings, Software Structures presented a series of abstract rules for the production of an image, including some from Lewitt. The rules were then implemented using a variety of "materials": Processing, Flash MX, and C++.
> A benefit of working with software structures instead of programming languages is that it places the work outside the current technological framework, which is continually becoming obsolete. Because a software structure is independent from a specific technology, it is possible to continually create manifestations of any software structure with current technology to avoid retrograde associations.[^sst]
[^sst]: Casey Reas, "Paragraphs on Software Art", Software Structures, retrieved October 28 2021, <https://artport.whitney.org/commissions/softwarestructures/text.html#structure>.
## two books
In 2007 I attended a book launch of: Processing: A Programming Handbook for Visual Designers and Artists (MIT Press, Reas + Fry, with a foreword by Maeda).
![](images/pbook.png)
The book comprises over 700 pages and is organized by topics like: color, control, data, drawing, image, input, math, motion, structure, typography. The book also contains interviews with artists working with digital tools not limited to Processing. As indicated in the book's introduction, many examples are illustrated by "stark" small square images and concise code.
![](images/pbook_inside.png)
Processing commands typically address the canvas using x,y (Cartesian) coordinates. Shape commands, like circle, rectangle, and triangle allow for the drawing of graphical forms, while other functions control parameters like the color of fill or the width of the stroke (or outline) of each shape.
![](images/p5editor_blurryrect.png)
Input functions allow access to the mouse and keyboard to produce dynamic graphics that respond to the user. Processing sketches consists of (at least) two functions: *setup* which is invoked once and *draw* which is invoked continuously; the default frequency being the refresh rate of the computers display (typically 60 times per second). By using variables, and changing their values, graphics can be made dynamic. In addition, graphics are rendered using a technique known as “anti-aliasing” to appear “pixelated”. This default behavior can be modified by using the noSmooth command.
Earlier in the day I had bought another technical book "ImageMagick Tricks: Web Image Effects from the Command line and PHP" by Sohail Salehi[^salehi]. While waiting for the presentation to begin, I crossed paths with Casey Reas at the back of the room. He was curious about the book I had with me and looked briefly at it. He had never heard of ImageMagick.
[^salehi]: Sohail Salehi, *ImageMagick Tricks* (Birmingham: Packt Publishing, 2006)
> ImageMagick started with a request from my DuPont supervisor, Dr. David Pensak, to display computer-generated images on a monitor only capable of showing 256 unique colors simultaneously. In 1987, monitors that could display 24-bit true color images were rare and quite expensive. There were a plethora of chemists and biologists at DuPont, but very were few computer scientists to confer with. Instead, I turned to Usenet for help, and posted a request for an algorithm to reduce 24-bit images to 256 colors. Paul Raveling of the USC Information Sciences Institute responded, with not only a solution, but one that was already in source code and available from USC's FTP site. Over the course of the next few years, I had frequent opportunities to get help with other vexing computer science problems I encountered in the course of doing my job at DuPont. Eventually I felt compelled to give thanks for the help I received from the knowledgeable folks on Usenet. I decided to freely release the image processing tools I developed to the world so that others could benefit from my efforts.[^cristy]
![](images/imbook.png)
ImageMagick, first released in 1990, is a popular free software tool thats often referred to as a “swiss army knife” due to its ability to convert between hundreds of different image formats, and for the many built-in features to filter, manipulate and generate images. Thanks to the software being "not chemically or biologically based", Cristy was able to release the software under a Free license (echoing the way the UNIX operating system became free software due to its marginality to the interests of Bell Labs). The software is full of particularities. For instance, there are a number of built in images including a wizard (the mascot of the software) seated at a drawing table contemplating an image of the Mona Lisa.
ImageMagick is a command-line tool, designed to be used via textual commands. The typical usage of ImageMagick is to take one image as input, applying one or more transformations to it, and output a new image. In this way the tool can be used repeatedly in a so-called “pipelines”, or otherwise composed together in structures called (BASH) scripts. In these scripts ImageMagick commands can be mixed with other commands from any software installed on the user's computer that also provides a command-line interface.
[^cristy]: John Cristy, "History", ImageMagick website. Retrieved from the Internet archive Oct 28, 2021, <https://web.archive.org/web/20161029234747/http://www.imagemagick.org/script/history.php>.
Salehis book directly reflects the structure of ImageMagick, with chapter organized around various incorporated “tools”: convert, mogrify, composite, montage, identify, display, conjure. The examples are practical, creating logos or adding captions or a border to an image. One example renders the word “Candy” with colorful stripes. Another series of examples duplicates and inverts the image and text of classical Persian poet “Hafez” to create a kind of playing card. Another example uses ImageMagick in conjunction with PHP and HTML to produce an online “e-card maker”, a sequence of commands is demonstrated to render the text “No More War” (in a dripping paint font), deform it, and project it onto the side of a chess piece.
![](images/imbook_inside.png)
In another extended example, a British flag is constructed in steps. Rather than approaching the project as drawing geometric forms on a canvas, Salehi uses the diversity of ImageMagicks manipulations, performing a series of commands whose textual names invoke a sense of physical construction: blocks of color are skewed, sheared, cropped, flipped, flopped, and finally spliced (with "gravity" set to center). The approach creates a number of intermediate images, thus creating the digital equivalents of "cuttings" or leftover materials in the process. Below, the (intermediate) results from the same example slightly modified to take an image as input:
![background2: intermediate file](images/background2.png) ![another intermediate file (mixed)](images/mixed.png)
![Source image: <https://en.wikipedia.org/wiki/Boris_Johnson#/media/File:Boris_Johnson_official_portrait_(cropped).jpg>, Ben Shread / Cabinet Office, UK Open Government Licence v3.0 (OGL v.3)](images/flag.png)
```bash
convert -size 300x200 xc:'#002377' -fill white -draw 'skewX 128 image over 0,0 0,0 in/boris.jpg' background1.png
convert -size 300x40 xc:none -background transparent -fill '#ce201a' -draw ' rectangle 0,0, 300,20' -shear 32 band.png
convert background1.png -draw ' image over -25,0 0,0 band.png' background1.png
convert background1.png -draw ' image over -25,0 0,0 band.png' background1.png
convert background1.png -crop 150x94+0+0 area1.png
convert area1.png -flip -flop area3.png
convert background1.png -draw ' image over -25,-21 0,0 band.png' background2.png
convert background2.png -flop background2.png
convert background2.png -crop 150x94+150+0 area2.png
convert area2.png -flip -flop area4.png
convert -size 300x188 xc:none -draw ' image over 0,0 0,0 area1.png' -draw ' image over 150,0 0,0 area2.png' -draw ' image over 0,94 0,0 area4.png' -draw ' image over 150,94 0,0 area3.png' mixed.png
convert mixed.png -background white -gravity center -splice 20x20 -background '#ce201a' -splice 40x40 flag.png
```
## constructivism
In the 1920s Russian avant-gardist El Lissitsky would move to Berlin and produce work that was highly influential to the then nascent Bauhaus. In 1923, Lissitsky illustrated a publication of poems by friend Vladimir Majakovskij. In it he created graphical forms by mixing typographic elements with geometric forms created by (mis-) using spacing elements (the “blind” elements typically used to create (negative/unprinted) space between lines of type, as positives producing geometric forms. This style, sometimes called constructivism, was part of an effort to make a radical break from traditional styles of typographic layout and illustration using the means then available for print. The book is notable for its "interactivity" via iconic tabbed pages.
![*For the Voice*, El Lissitzky designer; Image from the archive of Guttorm Guttormsgaard. Used with permission. <https://arkiv.guttormsgaardsarkiv.no/node/19/item/39>](images/forthevoice.png)
In the 1970s Seymour Papert co-developed a pedagogy for teaching children mathematics and programming based on the LOGO programming language. Part of the system was a (virtual) robotic turtle that could be programmed to draw figures. The system, known as Turtle graphics, had commands like: forward, turn left, turn right, pen up, pen down, that directly addressed the “turtle” to draw shapes while moving.
> The process reminds one of tinkering: learning consists of building up a set of materials and tools that one can handle and manipulate. Perhaps most central of all, it is a process of working with what youve got. …. This is a science of the concrete, where the relationships between natural objects in all their combinations and recombinations provide a conceptual vocabulary for building scientific theories. Here I am suggesting that in the most fundamental sense, we, as learners, are all *bricoleurs*.[^mindstorms]
![Made with <http://logointerpreter.com/turtle-editor.php>](images/turtle_circle.png)
Papert's approach is part of a pedagogic project (also) called constructivism. In a key example, Papert describes how students can be taught about circles by imagining (or better yet themselves enacting) the turtle repeatedly performing the instruction sequence "go forward a little, turn a little". He contrasts this with the formal equation of a circle (x^2^ + y^2^ = r^2^) typically taught in an elementary school geometry class.
In Belgium, where I currently live "bricolage" is the French language equivalent to "DIY" and is often used in a derogatory sense to indicate that something is made in an amateurish way. Papert embraces the term, a central point being how informal/intuitive methods not only appeal to "common sense" but also engage more profoundly with the materiality of its subject than would a formal approach. In the case of the circle, the "turtle" method is not only a way for the student to imagine the problem physically, it also relates to methods of differential calculus, something the algebraic formulation misses completely.
[^mindstorms]: Seymour Papert, *Mindstorms* (Cambridge: MIT Press, 1980), 173.
> For most there is a total dissociation between these live activities and the dead school math. We have stressed the fact that using the Turtle as metaphorical carrier for the idea of angle connects it firmly to the body geometry. We have called this *body syntonicity*. Here we see a *cultural syntonicity*: The Turtle connects the idea of angle to navigation, activity firmly and positively rooted in the extraschool culture of many children.[^mindstorms2]
[^mindstorms2]: Seymour Paper, *Mindstorms*, 68.
## tearing at the seams
Initially I came to the writing of this essay as part of a proposed dialog with the artist Winnie Soon. Unfortunately because of time constraints it wasn't able to happen. Together with Geoff Cox, Soon has recently published "Aesthetic Programming: A Handbook of Software Studies"[^aestheticprogramming]. The book itself, designed by Open Source Publishing, makes promiscuous use of different tools and techniques, such as the diagramming tool graphviz, HTML, Pelican (a CMS), and paged.js, and showcases an expressive use of typography (including a reinterpretion of a font originally drawn by Swiss typographer Adrian Frutiger).
[^aestheticprogramming]: Winnie Soon and Geoff Cox, *Aesthetic Programming: A Handbook of Software Studies* (Open Humanities Press, 2020)
[^app5]: Winnie Soon and Geoff Cox, 31.
Where Reas and Fry's Handbook is demure, Soon and Cox's is generous, voluptuous even, richly mixing programming exercises with diagrams, flowcharts, and illustrations. The discussions are explicit about the sociality of code and intersections with software art and software studies. The text is densely annotated with theoretical references and links to related critical artistic projects. Directly addressing the "dark sides" of technology such as the extractive and aggregative corporate practices of big data, this handbook explicitly positions coding as an act of activism. Most of the code examples are written using p5.js. As they describe in the first section:
> This book will use p5.js, a Javascript library which was created by artist Lauren McCarthy in 2014 for the purpose of what we call "aethetic programming." ... Importantly, the core idea of p5.js is not just to deploy Processing as a web-based platform, but to address diversity and inclusivity explicitly, and take these issues seriously in software development and communication"[^app5]
In 2016, the Whitney published a “restored” version of Software Structures.[^ssreloaded] As technologies like Java and Flash had then for reasons both technical and commercial fallen out of popular use on the web, the new version featured many of the processing sketches adapted by Reas to use p5.js.
Despite the project's earlier stated interest in exploring diverse "materialities", its telling that rather than considering the older processing implementations as a different material and presenting screenshots of them as was done for the Flash and C++ examples, the "restoration" maintains the illusion of a certain "permanence" to the processing sketches, making them thus appear closer to those imagined "software structures" than to "retrograde" technologies like Java or an out-dated browser version. In addition this "adaptation" hides the quite significant work that has gone into (1) the development and subsequent implementation in different browsers of the newly standardized canvas element[^canvas], and (2) McCarthy's work creating the p5.js library to bridge from the legacy processing code to this new standard.[^javascript]
[^javascript]: Despite the seeming similarity of names, Java and Javascript are two completely independent and quite different programming languages. Adapting software from one to the other is thus not trivial.
[^canvas]: <https://html.spec.whatwg.org/multipage/canvas.html#the-canvas-element>
[^ssreloaded]: <https://whitney.org/exhibitions/software-structures>
Alfred North Whitehead, writing on the sciences, established an influential idea of a "fallacy of misplaced concreteness". The idea is that making abstractions, such as what happens when a particular phenomenon is named, is a simplification that works by suppressing "what appear to be irrelevant details".[^whitehead] In Media Ecologies, Matthew Fuller extends this thinking to consider technical standards as "a material instantiation"[^fuller_p127] of Whitehead's misplaced concreteness, and considers how technical devices through a process of *objectification* "expect in advance the results that they obtain".[^fuller_p104] Fuller cites the example of Laurence Lessig, who makes an argument for regulation of the Internet based on the assertion of standard objects (such as networking protocols) being layered and reconfigurable.
[^whitehead]: Alfred North Whitehead, *Science and the modern world* (New York: Free Press, 1967), retrieved from the Internet archive Oct 28, 2021 <https://archive.org/details/sciencemodernwor00alfr/page/52/mode/2up>
> [Lessig's argument] allows us to recognize another characteristic of the standard object: that, while it may be simultaneously embedded within multiple compositions wherein it may be involved in many, separate, disjunctive, contiguous, or contradictory processes, it does provide a threshold, either side of which is differentiated enough for significant political, technical, aesthetic, and social conjunctions or conflicts to occur.[^fuller_p129]
[^fuller_p127]: Matthew Fuller, *Media Ecologies* (Cambridge: MIT Press, 2005), 127.
[^fuller_p129]: Matthew Fuller, 129.
[^fuller_p104]: Matthew Fuller, 104.
Susan Leigh Star takes Whitehead's "misplaced concretism" and proposes a feminist methodology specific to information technology.[^star] Her essay develops the idea of "standards" as one type of "Boundary object", which she describes as:
> [...] those scientific objects which both inhabit several communities of practice and satisfy the information requirements of each of them. Boundary objects are thus objects which are both plastic enough to adapt to local need and common identity across sites.[^star_p157]
Star cites Donna Haraway, who wonders in *A Manifesto for Cyborgs*:
> How do I then act the bricoleur that we've all learned to be in various ways, without being a colonizer.... How do you keep foregrounded the ironic and iffy things you're doing and still do them seriously [...]
Star draws on a tradition of diverse feminist thinking through the "articulation of multiplicity, contradiction, and partiality, while standing in a politically situated, moral collective" to synthesize and propose what she calls the important attributes of a feminist method:
> 1. experiential and collective basis;
> 2. processual nature;
> 3. honoring contradiction and partialness;
> 4. situated historicity with great attention to detail and specificity; and
> 5. the simultaneous application of all of these points.[^star_method]
[^star]: Susan Leigh Star, "Misplaced Concretism and Concrete Situations: Feminism, Method, and Information Technology" (1994), *Boundary Objects and Beyond* (Cambridge: MIT Press, 2015)
[^star_p157]: Susan Leigh Star, 157.
[^star_method]: Susan Leigh Star, 148-149.
For me tools like ImageMagick embody *collectivity* from it's origins as a way to "give back" to a community sharing code over usenet, through to its continued development by multiple authors and relation to the larger free software community as an invaluable toolbox for extremely diverse pratices. I find the *experiential* in the highly flexible commandline interface, itself also an example of honoring *contradiction* and *partialness*, with often more than one way to express the same transformation. The *processual* is implicit in its construction as a tool of transformation, encouraging an exploratory iterative approach to composing transformations to arrive at a desired outcome, often leading to missteps and errors that can be happy accidents and lead one to reconsider one's goals. Finally, in its extreme support of hundreds of different formats, ImageMagick use often leads to the discovery and exploration of diverse image formats, each with related practices, and contexts.
In contrast, behind a seemingly "neutral" aesthetic, Processing I think embodies a very particular set of values and assumptions. The “visual minimalism” claimed in the Processing handbook, belies the project's expansive claims on representing a pedagogic approach to a broad intersection of programming and the visual arts. The project's "neutral" aesthetics while dimly echoing a once-radical Bauhaus aesthetic, ignores the larger pedagogic program of the historical Bauhaus' experimentation with the materials of their (contemporary) technical production. The framework valorizes smoothness and fluidity, that leads one to prioritize interactivity as that which happens on the surface of a sketch, rather than say in the network, or among collaborators. As a pedagogic project, Processing has historically seemed uninterested in its own underlying materiality, preferring its students to explore the "world at large" by adding additional layers of technology in the form of sensors, rather than considering all the ways the technologies they use are *already engaged* and impacting the world.
> A concatenation of operations of misplaced concreteness thus allow the gaps, overlaps, and voids in the interrelated capacities of such systems to construct a more "accurate" account of its own operations.[^fuller_p104]
Vernacular rejects the "false neutrality" of the seamless universal design solution, embracing instead the tips and tricks of specific tools, in specific contexts. A vernacular composition tears open its seams proudly displaying the glitches and gaps as a badge of honor of a world view of truth as multi-threaded, incomplete, and sometimes uncomfortable, and is engaged with the world and not afraid to be seen as a monster.

Binary file not shown.

Binary file not shown.

@ -0,0 +1,156 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
<meta name="author" content="Michael Murtaugh" />
<title>torn at the seams: considering digital vernacular</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
span.underline{text-decoration: underline;}
div.column{display: inline-block; vertical-align: top; width: 50%;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
</style>
<link rel="stylesheet" href="style.css" />
</head>
<body>
<header id="title-block-header">
<h1 class="title">torn at the seams: considering digital vernacular</h1>
<p class="author">Michael Murtaugh</p>
</header>
<blockquote>
<p>Processing is a free, open source programming language and environment used by students, artists, designers, architects, researchers and hobbyists for learning, prototyping, and production. Processing is developed by artists and designers as an alternative to proprietary software tools in the same domain. The project integrates a programming language, development environment, and teaching methodology into a unified structure for learning and exploration.<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a></p>
</blockquote>
<p>Teaching programming with free software to media design students for years, Ive resisted Processing as it has always seemed to me to embody a particular kind of solipsism of digital interactivity and graphics that I want my students to avoid.</p>
<h2 id="design-by-numbers">design by numbers</h2>
<p>In the fall of 1996, John Maeda joined the MIT Media Lab to replace the recently deceased Muriel Cooper. Cooper was the first art director of the MIT Press, producing influential designs such as a 1969 catalog of the Bauhaus and the iconic MIT Press logo, a Bauhaus-inspired stylized graphical rendering of the letters “mitp”. Cooper started the Visible Language Workshop, later one of the founding groups of the MIT Media Lab, to research the intersection of publishing, design, and computation.</p>
<p><img src="images/dbn01.png" /></p>
<blockquote>
<p>Our forefathers at the Bauhaus, Ulm, and many other key centers for design education around the world labored to create a sense of order and method to their teaching. Thanks to their trailblazing work, teaching at the university level gradually became accepted as a meaningful and constructive activity. A drawing board, small or large, became the stage for paper, pen, ink, and blade to interact in the disciplined activity that characterized the profession of visual design.<a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a></p>
</blockquote>
<p>Maeda created the Aesthetics and Computation group in part to continue Coopers research. Maeda developed (with students Tom White, Peter Cho, Ben Fry, and later Casey Reas among others) a software system called Design by Numbers<a href="#fn3" class="footnote-ref" id="fnref3" role="doc-noteref"><sup>3</sup></a>. It had extreme constraints such as a fixed 100 by 100 pixel size and monochrome-only graphics. The command set is similarly constrained with only two drawing commands for lines and points. Commands like “paper” and “pen”, controlling the gray value of background and foreground colors, invoke the materiality of a (pre-digital) print practice. The accompanying print publication also had a square format.</p>
<p><img src="images/dbn02.png" /></p>
<blockquote>
<p>When designing this system for learning basic computational media design, I intentionally limited the set of commands and constructs to a minimal number of possibilities. If I had given you drawing capability beyond a line or setting a dot, the examples could have been more exciting, but the point could not be made clearly because your attention would be drawn to the picture and not to the code.<a href="#fn4" class="footnote-ref" id="fnref4" role="doc-noteref"><sup>4</sup></a></p>
</blockquote>
<p><img src="images/dbn03.png" /></p>
<p>This valorization of “code” over picture is evident in the book spread where a digital image of a vase is presented along with its “code”: a sequence of “set” commands describing the image pixel by pixel. Though its possible that such an image might be produced in this way (discipline!) in practice such images are created by translating a digital photograph using one of a family of techniques known as “dithering” to produce a pixelated image with limited gray values. Here these tools and techniques are unmentioned, with expansive space given instead to the listing of numbers.</p>
<p>I remember sitting with a friend at his Commodore 64 in the 1980s typing lines and lines of “poke” commands with digital data from the pages of home computer magazines to reproduce simple games. These pages would typically use compact representations (like hexadecimal) and include things like “checksums” and bootstrap programs to help you correctly enter and verify the data you entered.</p>
<p>In the case of the DBNs digital vase, the purpose does not seem to be for someone to actually type the numbers in, but rather to fill the pages of a book that seems destined for the coffee table rather than a desktop. Similar to the fields of 1s and 0s still popular as backgrounds of book sleeves and PowerPoint slides to suggest “all things digital”, the presentation is gratuitous and misleading in terms of actual practice. Like an observer unfamiliar with deaf culture confusing the hand gestures of finger spelling with the expressivity (and ambiguity) of actual sign language, the displays of 1s and 0s, or in this case of numeric coordinates and gray values, is a shorthand that refers digital practices without actually participating in them.</p>
<p>The above is just one example of DBNs lack of historicity. The year is 1999, and for instance the Sony PlayStation and access to the World Wide Web are popular phenomena. Yet the text seems remarkably devoid of any reference to specific tools or practices. Even Maedas invocations of historical figures like the Bauhaus and Paul Rand are vague, evoking a sort of nostalgic <em>Mad Men</em> universe with the creative young men (those future forefathers) at their drawing boards and a sense of “timeless design values” like a devotion to discipline and order.</p>
<p>At the end of the book Maeda responds to a critique apparently made to him by one of the students doing some of the programming of the DBN system itself. Recounting how a visit to a “university-level Java class for designers […] teach[ing] the finer points of object-oriented programming and bit masking of 24-bit color values” made him feel “lost in all the gibberish”, he reasserts his pedagogical approach as an alignment with the “simplification” that is the “constant goal” of programming.<a href="#fn5" class="footnote-ref" id="fnref5" role="doc-noteref"><sup>5</sup></a> Rather than trying to bridge the gulf between diverse practices, Maeda dismisses that which he doesnt (care to) understand in the name of simplification.</p>
<h2 id="processing">processing</h2>
<p>Processing was born as kind of fork or remake of DBN. Like DBN, Reas and Fry built Processing in Java, a desktop application that exists outside of the web but which can be used to publish sketches as “applets” embedded in a web page and published online. Reas and Fry added commands to work with color and multiple output sizes, as well as commands to draw basic shapes like circles, rectangles, and triangles.</p>
<p>Processing sketches consists of (at least) two functions: <em>setup</em> which is invoked once and <em>draw</em> which is invoked continuously; the default frequency being the refresh rate of the computers display (typically 60 times per second). By using variables combined with input functions allowing access to the mouse and keyboard, graphics can be made dynamic and responsive to the user. In addition, graphics by default are rendered using a technique known as “anti-aliasing” to appear “smoother” and less pixelated.</p>
<p><img src="images/pbook.png" /></p>
<blockquote>
<p>Most of the examples presented in this book have a minimal visual style. This represents not a limitation of the Processing software, but rather a conscious decision by the authors to make the code for each example as brief and clear as possible. We hope the stark quality of the examples gives additional incentive to the reader to extend the programs to her or his own visual language.<a href="#fn6" class="footnote-ref" id="fnref6" role="doc-noteref"><sup>6</sup></a></p>
</blockquote>
<p>Processing: A Programming Handbook for Visual Designers and Artists is published in 2006 by the MIT Press, with Reas + Fry authors, and a foreword by Maeda. The book comprises over 700 pages and is organized by topics like: color, control, data, drawing, image, input, math, motion, structure, typography. The book follows the visual style of the DBN book, with small square (mostly) monochrome images accompanying concise listings of code.</p>
<p><img src="images/pbook_inside.png" /></p>
<p>Despite the claim of leaving space for others to bring their own “visual language”, and thus an implicit proposition of its own aesthetics as “neutral”, Processing embodies a very particular set of values and assumptions. The framework valorizes smoothness and fluidity, that leads one to imagine interactivity as that which happens on the surface of a sketch, rather than say in the network, or among collaborators. The mechanism of the “draw loop” assumes that code runs in a negligible amount of time, that is less than the refresh rate (and the default 1/60th of a second). This particular, again implicit, relationship with time places the programmer in an adversarial relation with the processor of the viewers computer and makes it all too easy (certainly for novices) to produce code that makes the viewers computer struggle and lag.</p>
<p><img src="images/psquare2.png" /></p>
<p>Unlike DBN, the processing book contains extensive interviews with artists working with digital tools, mostly using a variety of tools outside of processing like C++, PHP and Flash. Despite the “minimalism” of the examples, the books subtitle claims relevance to a broad audience of “visual designers and artists”. The link to Processing is often unclear.</p>
<p>Theres a strange disconnect where a diversity of visual and computation practices are shown, such as sequential images to make animations, or cellular automata, but the only real link to processing seems to be as a kind of universal “blank canvas” onto which all kinds of (digital) practices can be projected.</p>
<p>Another disconnect occurs when one crosses out of what the software makes simple. For example, while very easy to make a sketch respond to mouse clicks, responding to a mouse click on a specific shape, suddenly involves using Java classes and adding code “hooks” to your draw loop to make sure the pieces work together. While this kind of code is not itself so unusual, the transition is really abrupt because the core abstractions are so simplified, there is no bridge.</p>
<p>Still another disconnect involves processing sketches when published online. Web pages have structures for text layout and content flowing, and structural elements (like checkboxes and links) that inherently respond to mouse clicks. There is a a built-in standard mechanism (the so-called DOM or document object model), to further customize these behaviors. Processing sketches, even situated in a web page, dont participate or give access to this model, nor does the framework offer using alternative graphical elements like SVG<a href="#fn7" class="footnote-ref" id="fnref7" role="doc-noteref"><sup>7</sup></a>.</p>
<h3 id="software-structures">software structures</h3>
<p>In 2004, Reas co-developed an exhibition at the Whitney gallery called Software Structures. Invoking Sol Lewitts wall drawings as an inspiration, the project presents a series of abstract rules (or software structures) for the production of an image, including some from Lewitt. The rules were then implemented using a variety of “materials”: Processing, Flash MX, and C++.</p>
<blockquote>
<p>A benefit of working with software structures instead of programming languages is that it places the work outside the current technological framework, which is continually becoming obsolete. Because a software structure is independent from a specific technology, it is possible to continually create manifestations of any software structure with current technology to avoid retrograde associations.<a href="#fn8" class="footnote-ref" id="fnref8" role="doc-noteref"><sup>8</sup></a></p>
</blockquote>
<p>In 2016, the Whitney published a “restored” version of Software Structures.<a href="#fn9" class="footnote-ref" id="fnref9" role="doc-noteref"><sup>9</sup></a> As technologies like Java and Flash had then for reasons both technical and commercial fallen out of popular use on the web, the new version featured many of the processing sketches adapted by Reas to use p5.js, a recent port of processing to Javascript made by artist Lauren Lee McCarthy.<a href="#fn10" class="footnote-ref" id="fnref10" role="doc-noteref"><sup>10</sup></a></p>
<p>Despite the projects earlier stated interest in exploring diverse “materialities”, its telling that rather than considering the older processing implementations as a different material and presenting screenshots of them as was done for the Flash and C++ examples, the “restoration” maintains the illusion of a “permanence” to the processing sketches, placing them closer to those imagined “software structures” than to “retrograde” technologies like Java or an out-dated browser. In addition the “adaptation” elides the work of the development and subsequent implementation of the then recently standardized canvas element<a href="#fn11" class="footnote-ref" id="fnref11" role="doc-noteref"><sup>11</sup></a>, as well as that of the creation of the p5.js library.</p>
<h2 id="imagemagick">imagemagick</h2>
<p>In 2007 I attended a book launch of the Processing Handbook. Earlier in the day, I had bought another technical book “ImageMagick Tricks: Web Image Effects from the Command line and PHP” by Sohail Salehi<a href="#fn12" class="footnote-ref" id="fnref12" role="doc-noteref"><sup>12</sup></a>. While waiting for the presentation to begin, I met Casey Reas at the back of the room. He was curious about the book I had with me and looked briefly at it. He had never heard of ImageMagick.</p>
<p><img src="images/imbook.png" /></p>
<blockquote>
<p>ImageMagick started with a request from my DuPont supervisor, Dr. David Pensak, to display computer-generated images on a monitor only capable of showing 256 unique colors simultaneously. In 1987, monitors that could display 24-bit true color images were rare and quite expensive. There were a plethora of chemists and biologists at DuPont, but very were few computer scientists to confer with. Instead, I turned to Usenet for help, and posted a request for an algorithm to reduce 24-bit images to 256 colors. Paul Raveling of the USC Information Sciences Institute responded, with not only a solution, but one that was already in source code and available from USCs FTP site. Over the course of the next few years, I had frequent opportunities to get help with other vexing computer science problems I encountered in the course of doing my job at DuPont. Eventually I felt compelled to give thanks for the help I received from the knowledgeable folks on Usenet. I decided to freely release the image processing tools I developed to the world so that others could benefit from my efforts.<a href="#fn13" class="footnote-ref" id="fnref13" role="doc-noteref"><sup>13</sup></a></p>
</blockquote>
<p>ImageMagick, first released in 1990, is a popular free software tool thats often referred to as a “Swiss army knife” due to its ability to convert between hundreds of different image formats, and for the many built-in features to filter, manipulate and generate images. Thanks to the software being “not chemically or biologically based”, Cristy was able to release the software as free software (echoing the way the UNIX operating system became free software due to its marginality to the interests of Bell Labs). The software is full of particularities. For instance, there are a number of built in images including a wizard (the mascot of the software) seated at a drawing table contemplating an image of the Mona Lisa.</p>
<p>ImageMagick is a command-line tool, designed to be used via textual commands. The typical usage of ImageMagick is to take one image as input, applying one or more transformations to it, and output a new image. In this way the tool can be used repeatedly in so-called “pipelines”, or otherwise composed together in structures called (shell) scripts. In these scripts ImageMagick commands can be mixed with other commands from any software installed on the users computer that also provides a command-line interface.</p>
<p><img src="images/imbook_inside.png" /></p>
<p>Salehis book directly reflects the structure of ImageMagick, with chapter organized around various incorporated “tools”: convert, mogrify, composite, montage, identify, display, conjure. The examples are practical, creating logos or adding captions or a border to an image. One example renders the word “Candy” with colorful stripes. Another series of examples duplicates and inverts the image and text of classical Persian poet “Hafez” to create a kind of playing card. Another example uses ImageMagick in conjunction with PHP and HTML to produce an online “e-card maker”, a sequence of commands is demonstrated to render the text “No More War” (in a dripping paint font), deform it, and project it onto the side of a chess piece.</p>
<p><img src="images/imbook_flag.jpg" /></p>
<p>In another extended example, a flag is constructed in steps. Rather than approaching the project as drawing geometric forms on a canvas, Salehi uses the diversity of ImageMagicks manipulations, performing a series of commands whose textual names invoke a sense of physical construction: blocks of color are skewed, sheared, cropped, flipped, flopped, and finally spliced (with “gravity” set to center). The approach creates a number of intermediate images, thus creating the digital equivalents of “cuttings” or leftover materials in the process.</p>
<p>By modifying the first step to use an image<a href="#fn14" class="footnote-ref" id="fnref14" role="doc-noteref"><sup>14</sup></a>, I produced the following (intermediate) results:</p>
<p><img src="images/background2.png" /></p>
<p><img src="images/mixed.png" /></p>
<p><img src="images/flag.png" /></p>
<h2 id="constructivism-and-the-bricoleur">constructivism and the bricoleur</h2>
<p>In the 1920s Russian avant-gardist El Lissitsky would move to Berlin and produce work that was highly influential to the then nascent Bauhaus. In 1923, Lissitsky illustrated a publication of poems by friend Vladimir Majakovskij. In it he created graphical forms by mixing typographic elements with geometric forms created by (mis-) using spacing or “blind” elements typically used to create negative/unprinted space between lines of type, as positives producing geometric forms. This style, sometimes called constructivism, was part of an effort to make a radical break from traditional styles of typographic layout and illustration using the means then available for print. The book is notable for its “interactivity” via iconic tabbed pages.</p>
<figure>
<img src="images/forthevoice.png" alt="" /><figcaption><em>For the Voice</em>, El Lissitzky designer; Image from the archive of Guttorm Guttormsgaard. Used with permission. <a href="https://arkiv.guttormsgaardsarkiv.no/node/19/item/39" class="uri">https://arkiv.guttormsgaardsarkiv.no/node/19/item/39</a></figcaption>
</figure>
<p><img src="images/mindstorms00.jpg" /> <img src="images/mindstorms01.jpg" /></p>
<p>Constructivism is also the name given to the pedagogic project associated with Seymour Papert. In the 1970s Papert co-developed a pedagogy for teaching children mathematics and programming based on the LOGO programming language. Part of the system was a (virtual) robotic turtle that could be programmed to draw figures. The system, known as Turtle graphics, had commands like: forward, turn left, turn right, pen up, pen down, that directly addressed the “turtle” to draw shapes while moving.</p>
<blockquote>
<p>The process reminds one of tinkering: learning consists of building up a set of materials and tools that one can handle and manipulate. Perhaps most central of all, it is a process of working with what youve got. …. This is a science of the concrete, where the relationships between natural objects in all their combinations and recombinations provide a conceptual vocabulary for building scientific theories. Here I am suggesting that in the most fundamental sense, we, as learners, are all <em>bricoleurs</em>.<a href="#fn15" class="footnote-ref" id="fnref15" role="doc-noteref"><sup>15</sup></a></p>
</blockquote>
<p>Papert described the pedagogic project of LOGO in book titled Mindstorms. In a key example, Papert describes how students can be taught about circles by imagining (or better yet themselves enacting) the turtle repeatedly performing the sequence “go forward a little, turn a little”. He contrasts this with the formal equation of a circle (x<sup>2</sup> + y<sup>2</sup> = r<sup>2</sup>) typically taught in an elementary school geometry class.</p>
<blockquote>
<p>TO CIRCLE REPEAT [FORWARD 1 RIGHT 1]</p>
</blockquote>
<p><img src="images/mindstorms-seq01.jpg" /> <img src="images/mindstorms-seq02.jpg" /> <img src="images/mindstorms-seq03.jpg" /> <img src="images/mindstorms-seq04.jpg" /> <img src="images/mindstorms-seq05.jpg" /> <img src="images/mindstorms-seq06.jpg" /> <img src="images/mindstorms-seq07.jpg" /></p>
<p>In a powerful central visual sequence, Mindstorms presents a series of illustrations showing the screen output of code alongside a running dialog. The conversation starts with a proposition to draw a flower like one sketched on paper. First they consider what programs they might already have to make use of, in this case they have a procedure to draw a quarter circle. Through a series of steps, mistakes are made, plans are adjusted and retried, happy accidents lead to discoveries (its a bird!). In the process the “ends become means” and a new tool is put to use to create a garden, and then, incorporating the “bug”, a flock of flying birds.</p>
<p>In Belgium, where I live “brico” is the French language equivalent to “DIY” and is often used in a derogatory sense to indicate that something is made in an amateurish way. Papert, is borrowing the term from Claude Lévi-Strauss, who first used the term in an anthropological context hypothesizing how “universal” knowledge might form from myth and fragmentary cultural knowledge<a href="#fn16" class="footnote-ref" id="fnref16" role="doc-noteref"><sup>16</sup></a>.</p>
<p>For Papert, bricolage exhibits a quality whereby informal methods not only appeal to “common sense” but also engage more profoundly with the materiality of its subject than would a formal approach. In the case of the circle, the “turtle” method is not only a way for the student to imagine the problem physically, it also relates to methods of differential calculus, something the algebraic formulation misses completely. In hacker circles, bricolage is evident in an approach of embracing “glue code” and “duct tape” methods, like the pipeline, that allow different systems to be “hacked” together to do useful (new) things.</p>
<h2 id="misplaced-concretism-a-feminist-method">misplaced concretism &amp; a feminist method</h2>
<p>Alfred North Whitehead, writing on the sciences, established an influential idea of a “fallacy of misplaced concreteness”. The idea is that making abstractions, such as what happens when a particular phenomenon is named, is a simplification that works by suppressing “what appear to be irrelevant details”.<a href="#fn17" class="footnote-ref" id="fnref17" role="doc-noteref"><sup>17</sup></a> In <em>Media Ecologies</em>, Matthew Fuller extends this thinking to consider technical standards as “a material instantiation”<a href="#fn18" class="footnote-ref" id="fnref18" role="doc-noteref"><sup>18</sup></a> of Whiteheads misplaced concreteness, and considers how technical devices through a process of <em>objectification</em> “expect in advance the results that they obtain”.<a href="#fn19" class="footnote-ref" id="fnref19" role="doc-noteref"><sup>19</sup></a></p>
<p>Susan Leigh Star takes Whiteheads “misplaced concretism” and proposes a feminist methodology specific to information technology.<a href="#fn20" class="footnote-ref" id="fnref20" role="doc-noteref"><sup>20</sup></a> Her essay develops the idea of “standards” as one type of “Boundary object”, which she describes as:</p>
<blockquote>
<p>[…] those scientific objects which both inhabit several communities of practice and satisfy the information requirements of each of them. Boundary objects are thus objects which are both plastic enough to adapt to local need and common identity across sites.<a href="#fn21" class="footnote-ref" id="fnref21" role="doc-noteref"><sup>21</sup></a></p>
</blockquote>
<p>Star cites Donna Haraway, who wonders in <em>A Manifesto for Cyborgs</em>:</p>
<blockquote>
<p>How do I then act the bricoleur that weve all learned to be in various ways, without being a colonizer…. How do you keep foregrounded the ironic and iffy things youre doing and still do them seriously […]</p>
</blockquote>
<p>Star draws on a tradition of diverse feminist thinking through the “articulation of multiplicity, contradiction, and partiality, while standing in a politically situated, moral collective” to synthesize and propose what she calls the important attributes of a feminist method:</p>
<blockquote>
<ol type="1">
<li>experiential and collective basis;</li>
<li>processual nature;</li>
<li>honoring contradiction and partialness;</li>
<li>situated historicity with great attention to detail and specificity; and</li>
<li>the simultaneous application of all of these points.<a href="#fn22" class="footnote-ref" id="fnref22" role="doc-noteref"><sup>22</sup></a></li>
</ol>
</blockquote>
<p>As a teacher, I enjoy using tools like ImageMagick in my teaching as it embodies <em>collectivity</em> from its origins as a way to “give back” to a community sharing code over usenet, through to its continued development by multiple authors and relation to the larger free software community as an invaluable toolbox for extremely diverse practices. I find the <em>experiential</em> in the highly flexible commandline interface, itself also an example of honoring <em>contradiction</em> and <em>partialness</em>, with often more than one way to express the same transformation. The <em>processual</em> is implicit in its construction as a tool of transformation, encouraging an exploratory iterative approach to composing transformations to arrive at a desired outcome, often leading to misusage and errors that can be happy accidents and lead one to reconsider ones goals. Finally, in its extreme support of hundreds of different formats, ImageMagick use often leads to the discovery and exploration of diverse image formats, each with related practices and histories.</p>
<p>In contrast, I find as a pedagogic project, I find Processing actively uninterested in its own underlying materiality, aspiring instead to a kind of disembodied and bland universality. Students are encouraged to explore the “world at large” by adding additional layers of technology in the form of sensors, rather than considering all the ways the technologies they use are <em>already engaged with the world</em>. The projects “neutral” aesthetics while dimly echoing a once-radical Bauhaus aesthetic, ignores the larger pedagogic program of the historical Bauhaus engagement and experimentation with the materials of their (contemporary) technical production.</p>
<blockquote>
<p>A concatenation of operations of misplaced concreteness thus allow the gaps, overlaps, and voids in the interrelated capacities of such systems to construct a more “accurate” account of its own operations.<a href="#fn23" class="footnote-ref" id="fnref23" role="doc-noteref"><sup>23</sup></a></p>
</blockquote>
<p>Fullers recipe for critical media engagement feels like the basis of what might be termed a digital vernacular. A digital vernacular is about working withing the constraints of available resources, rather than living with a fantasy of negligible time or unlimited storage. A digital vernacular rejects the sense of disembodiment, rather if locates itself in interactions between many bodies. A digital vernacular finds itself in conversational forms that are open to happy accidents. A digital vernacular rejects the “false neutrality” of the seamless universal design solution, embracing instead the tips and tricks of specific tools, in specific contexts. A digital vernacular tears open its seams proudly displaying its glitches and gaps. A digital vernacular rejects the illusory construction of an isolated artist sitting at a blank canvas creating works from scratch. Instead the digital vernacular thrives on working with the contingencies of existing systems, and embraces working with boundary objects as a means of bridging diverse communities of practices.</p>
<section class="footnotes" role="doc-endnotes">
<hr />
<ol>
<li id="fn1" role="doc-endnote"><p>Casey Reas and Ben Fry, <em>Processing: A Programming Handbook for Visual Designers and Artists</em> (Cambridge: MIT Press, 2007), xxi-xxii.<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn2" role="doc-endnote"><p>John Maeda, Design By Numbers. (Cambridge: MIT Press, 1999), 19.<a href="#fnref2" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn3" role="doc-endnote"><p><a href="https://dbn.media.mit.edu/" class="uri">https://dbn.media.mit.edu/</a><a href="#fnref3" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn4" role="doc-endnote"><p>Maeda, Design By Numbers. 144.<a href="#fnref4" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn5" role="doc-endnote"><p>Maeda, Design By Numbers. 252.<a href="#fnref5" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn6" role="doc-endnote"><p>Casey Reas and Ben Fry, <em>Processing: A Programming Handbook for Visual Designers and Artists</em> (Cambridge: MIT Press, 2007), xxi-xxii.<a href="#fnref6" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn7" role="doc-endnote"><p>A web standard for “scalable vector graphics”, SVG is more integrated into the DOM than are java applets (in classic Processing) or the canvas element (see p5.js).<a href="#fnref7" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn8" role="doc-endnote"><p>Casey Reas, “Paragraphs on Software Art”, Software Structures, retrieved October 28 2021, <a href="https://artport.whitney.org/commissions/softwarestructures/text.html#structure" class="uri">https://artport.whitney.org/commissions/softwarestructures/text.html#structure</a>.<a href="#fnref8" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn9" role="doc-endnote"><p><a href="https://whitney.org/exhibitions/software-structures" class="uri">https://whitney.org/exhibitions/software-structures</a><a href="#fnref9" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn10" role="doc-endnote"><p>Despite the seeming similarity of names, Java and Javascript are two completely independent and quite different programming languages. Adapting software from one to the other is not trivial. In making the adaptation McCarthy claims “diversity and inclusion as core values upon which the software is built”. See: <a href="https://lauren-mccarthy.com/p5-js" class="uri">https://lauren-mccarthy.com/p5-js</a> and <a href="https://p5js.org/" class="uri">https://p5js.org/</a>. However, most of the limitations I talk about here still apply to sketches made with the current version of p5.js.<a href="#fnref10" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn11" role="doc-endnote"><p><a href="https://html.spec.whatwg.org/multipage/canvas.html#the-canvas-element" class="uri">https://html.spec.whatwg.org/multipage/canvas.html#the-canvas-element</a><a href="#fnref11" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn12" role="doc-endnote"><p>Sohail Salehi, <em>ImageMagick Tricks</em> (Birmingham: Packt Publishing, 2006)<a href="#fnref12" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn13" role="doc-endnote"><p>John Cristy, “History”, ImageMagick website. Retrieved from the Internet archive Oct 28, 2021, <a href="https://web.archive.org/web/20161029234747/http://www.imagemagick.org/script/history.php" class="uri">https://web.archive.org/web/20161029234747/http://www.imagemagick.org/script/history.php</a>.<a href="#fnref13" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn14" role="doc-endnote"><p>Image: <a href="https://commons.wikimedia.org/wiki/File:Boris_Johnson_official_portrait_(cropped).jpg" class="uri">https://commons.wikimedia.org/wiki/File:Boris_Johnson_official_portrait_(cropped).jpg</a>, Ben Shread / Cabinet Office, UK Open Government Licence v3.0 (OGL v.3)<a href="#fnref14" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn15" role="doc-endnote"><p>Seymour Papert, <em>Mindstorms</em> (Cambridge: MIT Press, 1980), 173.<a href="#fnref15" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn16" role="doc-endnote"><p>Claude Lévi-Strauss. <em>The Science of the Concrete</em>. The Savage Mind (University of Chicago Press: 1966), Chapter 1.<a href="#fnref16" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn17" role="doc-endnote"><p>Alfred North Whitehead, <em>Science and the modern world</em> (New York: Free Press, 1967), retrieved from the Internet archive Oct 28, 2021 <a href="https://archive.org/details/sciencemodernwor00alfr/page/52/mode/2up" class="uri">https://archive.org/details/sciencemodernwor00alfr/page/52/mode/2up</a><a href="#fnref17" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn18" role="doc-endnote"><p>Matthew Fuller, <em>Media Ecologies</em> (Cambridge: MIT Press, 2005), 127.<a href="#fnref18" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn19" role="doc-endnote"><p>Matthew Fuller, 104.<a href="#fnref19" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn20" role="doc-endnote"><p>Susan Leigh Star, “Misplaced Concretism and Concrete Situations: Feminism, Method, and Information Technology” (1994), <em>Boundary Objects and Beyond</em> (Cambridge: MIT Press, 2015)<a href="#fnref20" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn21" role="doc-endnote"><p>Susan Leigh Star, 157.<a href="#fnref21" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn22" role="doc-endnote"><p>Susan Leigh Star, 148-149.<a href="#fnref22" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
<li id="fn23" role="doc-endnote"><p>Matthew Fuller, 104.<a href="#fnref23" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
</ol>
</section>
</body>
</html>

@ -0,0 +1,350 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 33,
"id": "74b112d3-10ff-4c51-9eab-b185502e6104",
"metadata": {},
"outputs": [],
"source": [
"TEXT=\"I've never particularly liked Processing.\""
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "9e388fc6-2dc3-48c0-8c46-44e67853daf4",
"metadata": {},
"outputs": [],
"source": [
"%%bash -s \"$TEXT\"\n",
"convert -background NavajoWhite -fill gray10 -font letterng-medium -pointsize 24 \\\n",
" -size 300x caption:\"$1\" \\\n",
" -borderColor NavajoWhite -border 10x10 out/processing01.png"
]
},
{
"cell_type": "markdown",
"id": "09d09465-79f3-4f37-b1e4-9d8af4dfe994",
"metadata": {},
"source": [
"![](out/processing01.png)"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "d658ddd1-8e6a-4e02-84fb-8ab25c5271a9",
"metadata": {},
"outputs": [],
"source": [
"TEXT=\"Teaching programming to art and design students for years, I've avoided Processing as it has always seemed to represent a kind of solipsism of digital interactivity and graphics.\""
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "ce98dd21-078c-43df-b625-d993da241736",
"metadata": {},
"outputs": [],
"source": [
"%%bash -s \"$TEXT\"\n",
"convert -background Peru -fill gray10 -font letterng-medium -pointsize 24 \\\n",
" -size 620x caption:\"$1\" \\\n",
" -borderColor Peru -border 10x10 out/processing02.png"
]
},
{
"cell_type": "markdown",
"id": "1da2e405-b022-4f84-bbb2-33158fcc627b",
"metadata": {},
"source": [
"![](out/processing02.png)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "98746079-6ba6-4c31-b0f4-3bc02d2c8fb7",
"metadata": {},
"outputs": [],
"source": [
"# This cell should be rendered by Processing!"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "f925bef4-5b7f-4309-b260-d3dc6491ed48",
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"convert -background gray50 -fill white -font Helvetica -pointsize 24 \\\n",
" -size 300x300 caption:@in/solipsism.txt \\\n",
" -borderColor gray50 -border 10x10 out/solipsism.png"
]
},
{
"cell_type": "markdown",
"id": "06aa9f63-5fab-4ec4-9c5f-312581b7152e",
"metadata": {},
"source": [
"![](out/solipsism.png)"
]
},
{
"cell_type": "markdown",
"id": "6b233284-38d9-4af2-993f-d667e05d958f",
"metadata": {},
"source": [
"In the summer of 1996, I was finishing my Masters degree, and I met John Maeda who was just joining the MIT Media Lab to replace the recently deceased Muriel Cooper."
]
},
{
"cell_type": "markdown",
"id": "9735a442-c09b-4b18-89e1-b9c39c3efe15",
"metadata": {},
"source": [
"Cooper was the first art director of the MIT Press, producing influential designs such as a 1969 catalog of Bauhaus and the iconic MIT Press logo."
]
},
{
"cell_type": "markdown",
"id": "5709cc5f-9c55-4cc5-8e6e-82e225dbbc58",
"metadata": {},
"source": [
"When the Media Lab was founded, Cooper started the Visible Language Workshop to research the intersection of publishing, design, and computation."
]
},
{
"cell_type": "markdown",
"id": "22a1955e-a40a-4012-9384-539f2ed10cc0",
"metadata": {},
"source": [
"My friend Robin was a student in the VLW; through her I learned things about Muriel like how she would say \"Swiss\" instead of Helvetica."
]
},
{
"cell_type": "markdown",
"id": "4ed32438-92c2-40f0-83ab-b70fc924db06",
"metadata": {},
"source": [
"NEED: Citation of how Helvetica / sans serif fonts were at one time a radical break from traditional typography."
]
},
{
"cell_type": "markdown",
"id": "587768ba-61b8-4bf5-92d4-9e7a1f50d08b",
"metadata": {},
"source": [
"Maeda created the Physical Language Workshop, later Aesthetics and Computation group, to continue this research. Ben Fry and Casey Reas were among Maeda's first students."
]
},
{
"cell_type": "markdown",
"id": "e22fd3fd-76e8-4209-8088-1e4559531c9f",
"metadata": {},
"source": [
"Maeda published a software system (and book) called Design by Numbers. It had extreme constraints such as a fixed 100 by 100 pixel size and monochrome graphics."
]
},
{
"cell_type": "markdown",
"id": "236df74e-d5f4-4345-aa58-48f779a4c3a6",
"metadata": {},
"source": [
"DBN would go on to inspire Reas + Fry to create Processing."
]
},
{
"cell_type": "markdown",
"id": "d2cccfa8-af8e-453f-82ae-b45aa35fb131",
"metadata": {},
"source": [
"In 2004, Reas wrote a series of essays around an exhibition at the Whitney gallery called Software Structures. Inspired by Sol Lewitt's wall drawings, Software Structures presented a series of abstract 'structures' (including some of Lewitt's instructions) that are then implemented in a variety of 'materials': Processing, Flash MX, C++."
]
},
{
"cell_type": "markdown",
"id": "c542de03-5ea3-4203-92cf-4d18caefe46d",
"metadata": {},
"source": [
"> A benefit of working with software structures instead of programming languages is that it places the work outside the current technological framework, which is continually becoming obsolete. Because a software structure is independent from a specific technology, it is possible to continually create manifestations of any software structure with current technology to avoid retrograde associations. -- Casey Reas"
]
},
{
"cell_type": "markdown",
"id": "1593d04f-1466-43bd-af7f-abcbff9125cc",
"metadata": {},
"source": [
"As technologies like Java and Flash have both become deprecated in the contemporary web, the work has been \"restored\" in 2016 with processing implmentations rewritten to use p5.js."
]
},
{
"cell_type": "markdown",
"id": "1eced9d5-8274-4d21-b291-4a9984c83fcc",
"metadata": {},
"source": [
"In 2007 I attended a book launch of the MIT Press Processing handbook (authored by Reas + Fry, and with an introduction by Maeda)."
]
},
{
"cell_type": "markdown",
"id": "2a7d5f07-5ff0-46be-9abf-f33075d1057c",
"metadata": {},
"source": [
"Ealier in the day I had bought another technical book \"ImageMagick Tricks\" by Sohail Salehi."
]
},
{
"cell_type": "markdown",
"id": "f838a603-082e-4662-bc67-bab6ba3d59c6",
"metadata": {},
"source": [
"Briefly I met Casey Reas at the back of the room before his presentation. He was curious about the book I had with me and looked briefly at it. He had never heard of ImageMagick."
]
},
{
"cell_type": "markdown",
"id": "f5803ac2-1e9b-4bfd-ba63-6301ecd1988b",
"metadata": {},
"source": [
"ImageMagick is full of particularities. Like the default images built in, giving you \"test sources\" to try out the many built in filters and effects."
]
},
{
"cell_type": "markdown",
"id": "9ec3dbc6-ef55-48f3-8fc3-139355de88db",
"metadata": {},
"source": [
"ImageMagick is a commandline tool, designed to be used via textual commands written in the generic space of the terminal."
]
},
{
"cell_type": "markdown",
"id": "f81e7860-28d4-49d7-830c-f825820d5476",
"metadata": {},
"source": [
"ImageMagick is a porous tool, with many shortcuts (such as using the @ charater to read from a file)."
]
},
{
"cell_type": "markdown",
"id": "c9ae02fd-ea0e-4960-b7d0-50b2efda8020",
"metadata": {},
"source": [
"ImageMagick is often referred to as a \"swiss army knife\" due to its handling of many input and output formats."
]
},
{
"cell_type": "markdown",
"id": "a4e64620-500a-455f-91ef-544877ccba1e",
"metadata": {},
"source": [
"It's particularly useful as part of \"pipelines\" -- series of commands issued on the commandline where the input of one process feeds in the input to another. In this way diverse tools can be composed to make custom tools in an ad hoc way."
]
},
{
"cell_type": "markdown",
"id": "026dd9a2-04f0-4408-900a-5e5043e5d80c",
"metadata": {},
"source": [
"If you don't like the built in handling of text, you can also make use of other layout engines like pango, an independent free software project that IM embeds."
]
},
{
"cell_type": "markdown",
"id": "b6e1ba7b-ae75-4f6e-a87a-c1b37a064eca",
"metadata": {},
"source": [
"Bauhaus typographers mis-used metallic blocks used to space typography as visible graphic elements, thus creating one of the defining visual characteristics of bauhaus typography."
]
},
{
"cell_type": "markdown",
"id": "19c4c325-0c47-4620-8d3e-be5d56fb0a35",
"metadata": {},
"source": [
"rather than stating from a blank canvas, bauhaus designers were exploring (mis) uses of the materials they had at hand to create new forms and effects."
]
},
{
"cell_type": "markdown",
"id": "a7c5c1c6-5490-4870-9357-286fdd3fe402",
"metadata": {},
"source": [
"the processing canvas is a little island cut off from the rest of the structure of a webpage, and endlessly demaning processor attention through it's draw loop."
]
},
{
"cell_type": "markdown",
"id": "3cb4aaa7-b5ee-4e1e-8fa4-7fce4311c5c1",
"metadata": {},
"source": [
"Mayakovsky"
]
},
{
"cell_type": "markdown",
"id": "f01f6152-ef3e-4573-a91b-d0c747ab2d26",
"metadata": {},
"source": [
"Lori Emerson!"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "4148ce55-3f47-4652-b817-4a410afe0730",
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"# convert -size 300x200 xc:'#002377' -fill white -draw 'skewX 128 skewY 0 rectangle -60,0 40,200' out/flag_background1.png\n",
"convert -size 300x200 xc:'#002377' -fill white -draw 'skewX 128 image over 0,0 0,0 in/boris.jpg' background1.png\n",
"convert -size 300x40 xc:none -background transparent -fill '#ce201a' -draw ' rectangle 0,0, 300,20' -shear 32 band.png\n",
"# convert -size 300x40 xc:none -background transparent -fill '#ce201a' -draw ' image over 0,0, 300,20 in/boris.jpg' -shear 32 band.png\n",
"convert background1.png -draw ' image over -25,0 0,0 band.png' background1.png\n",
"convert background1.png -draw ' image over -25,0 0,0 band.png' background1.png\n",
"convert background1.png -crop 150x94+0+0 area1.png\n",
"convert area1.png -flip -flop area3.png\n",
"convert background1.png -draw ' image over -25,-21 0,0 band.png' background2.png\n",
"convert background2.png -flop background2.png\n",
"convert background2.png -crop 150x94+150+0 area2.png\n",
"convert area2.png -flip -flop area4.png\n",
"convert -size 300x188 xc:none -draw ' image over 0,0 0,0 area1.png' -draw ' image over 150,0 0,0 area2.png' -draw ' image over 0,94 0,0 area4.png' -draw ' image over 150,94 0,0 area3.png' mixed.png\n",
"convert mixed.png -background white -gravity center -splice 20x20 -background '#ce201a' -splice 40x40 flag.png"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "389f44ee-3094-4968-99fb-deac4fb9c1c4",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.2"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

@ -25,7 +25,7 @@ In the fall of 1996, John Maeda joined the MIT Media Lab to replace the recently
[^dbn_252]: Maeda, Design By Numbers. 252. [^dbn_252]: Maeda, Design By Numbers. 252.
Maeda created the Aesthetics and Computation group in part to continue Coopers research. Maeda developed (with students Tom White, Peter Cho, Ben Fry, and later Casey Reas among others) a software system called Design by Numbers[^dbnsite]. It had extreme constraints such as a fixed 100 by 100 pixel size and monochrome-only graphics. The command set is similarly constrained with for instance only two drawing commands for lines and points. Commands like "paper" and "pen", controlling the gray value of background and foreground colors, invoke the materialty of a (pre-digital) print practice. The accompanying print publication also had a square format. Maeda created the Aesthetics and Computation group in part to continue Coopers research. Maeda developed (with students Tom White, Peter Cho, Ben Fry, and later Casey Reas among others) a software system called Design by Numbers[^dbnsite]. It had extreme constraints such as a fixed 100 by 100 pixel size and monochrome-only graphics. The command set is similarly constrained with only two drawing commands for lines and points. Commands like "paper" and "pen", controlling the gray value of background and foreground colors, invoke the materiality of a (pre-digital) print practice. The accompanying print publication also had a square format.
[^dbnsite]: <https://dbn.media.mit.edu/> [^dbnsite]: <https://dbn.media.mit.edu/>
@ -39,16 +39,16 @@ This valorization of "code" over picture is evident in the book spread where a d
I remember sitting with a friend at his Commodore 64 in the 1980s typing lines and lines of "poke" commands with digital data from the pages of home computer magazines to reproduce simple games. These pages would typically use compact representations (like hexadecimal) and include things like "checksums" and bootstrap programs to help you correctly enter and verify the data you entered. I remember sitting with a friend at his Commodore 64 in the 1980s typing lines and lines of "poke" commands with digital data from the pages of home computer magazines to reproduce simple games. These pages would typically use compact representations (like hexadecimal) and include things like "checksums" and bootstrap programs to help you correctly enter and verify the data you entered.
In the case of the DBN's digital vase, the purpose does not seem to be for someone to actually type the numbers in, but rather to fill the pages of a book that seems destined for the coffee table rather than a desktop. Similar to the fields of 1s and 0s still popular as backgrounds of book sleeves and powerpoint slides to suggest "all things digital", the presentation is gratuitous and misleading in terms of actual practice. Like an observer unfamiliar with deaf culture confusing the hand gestures of finger spelling with the expressivity (and ambiguity) of actual sign language, the displays of 1s and 0s, or in this case of numeric coordinates and gray values, is a shorthand that refers digital practices without actually participating in them. In the case of the DBN's digital vase, the purpose does not seem to be for someone to actually type the numbers in, but rather to fill the pages of a book that seems destined for the coffee table rather than a desktop. Similar to the fields of 1s and 0s still popular as backgrounds of book sleeves and PowerPoint slides to suggest "all things digital", the presentation is gratuitous and misleading in terms of actual practice. Like an observer unfamiliar with deaf culture confusing the hand gestures of finger spelling with the expressivity (and ambiguity) of actual sign language, the displays of 1s and 0s, or in this case of numeric coordinates and gray values, is a shorthand that refers digital practices without actually participating in them.
The above is just one example of DBN's lack of historicity. The year is 1999, and for instance the Sony PlayStation and access to the World Wide Web are popular phenomea. Yet the text seems remarkably devoid of any reference to specific tools or practices. Even Maeda's invocations of historical figure like the Bauhaus and Paul Rand are vague, evoking a sort of nostalgic *Mad Men* universe with the creative young men (those future forefathers) at their drawing boards and a sense of "timeless design values" like a devotion to discipline and order. The above is just one example of DBN's lack of historicity. The year is 1999, and for instance the Sony PlayStation and access to the World Wide Web are popular phenomena. Yet the text seems remarkably devoid of any reference to specific tools or practices. Even Maeda's invocations of historical figures like the Bauhaus and Paul Rand are vague, evoking a sort of nostalgic *Mad Men* universe with the creative young men (those future forefathers) at their drawing boards and a sense of "timeless design values" like a devotion to discipline and order.
At the end of the book Maeda responds to a critique apparently made to him by one of the students doing some of the programming of the DBN system itself. Recounting how a visit to a "university-level Java class for designers [...] teach[ing] the finer points of object-oriented programming and bit masking of 24-bit color values" made him feel "lost in all the gibberish", he reasserts his pedagogical approach as an alignment with the "simplification" that is the "constant goal" of programming.[^dbn_252] Rather than trying to bridge the gulf between diverse practices, Maeda dismisses that which he doesn't (care to) understand in the name of simplification. At the end of the book Maeda responds to a critique apparently made to him by one of the students doing some of the programming of the DBN system itself. Recounting how a visit to a "university-level Java class for designers [...] teach[ing] the finer points of object-oriented programming and bit masking of 24-bit color values" made him feel "lost in all the gibberish", he reasserts his pedagogical approach as an alignment with the "simplification" that is the "constant goal" of programming.[^dbn_252] Rather than trying to bridge the gulf between diverse practices, Maeda dismisses that which he doesn't (care to) understand in the name of simplification.
## processing ## processing
Processing was born as kind of fork or remake of DBN. Like DBN, Reas and Fry built Processing in Java, a desktop application that exists outside of the web but which can be used to publish sketches as "applets" embedded in a web page and published online. Reas and Fry added commands to work with color and multiple output sizes, as well as commands to draw basic shapes like circle, rectangle, and triangle. Processing was born as kind of fork or remake of DBN. Like DBN, Reas and Fry built Processing in Java, a desktop application that exists outside of the web but which can be used to publish sketches as "applets" embedded in a web page and published online. Reas and Fry added commands to work with color and multiple output sizes, as well as commands to draw basic shapes like circles, rectangles, and triangles.
Processing sketches consists of (at least) two functions: *setup* which is invoked once and *draw* which is invoked continuously; the default frequency being the refresh rate of the computers display (typically 60 times per second). By using variables combined with input functions allowing access to the mouse and keyboard, graphics can be made dynamic and responsive to the user. In addition, graphics by default are rendered using a technique known as “anti-aliasing” to appear “smoother” and less pixelated. Processing sketches consists of (at least) two functions: *setup* which is invoked once and *draw* which is invoked continuously; the default frequency being the refresh rate of the computers display (typically 60 times per second). By using variables combined with input functions allowing access to the mouse and keyboard, graphics can be made dynamic and responsive to the user. In addition, graphics by default are rendered using a technique known as “anti-aliasing” to appear “smoother” and less pixelated.
@ -60,17 +60,17 @@ Processing: A Programming Handbook for Visual Designers and Artists is published
![](images/pbook_inside.png) ![](images/pbook_inside.png)
Despite the claim of leaving space for others to bring their own "visual language", and thus an implicit proposition of its own aesthetics as "neutral", Processing embodies a very particular set of values and assumptions. The framework valorizes smoothness and fluidity, that leads one to imagine interactivity as that which happens on the surface of a sketch, rather than say in the network, or among collaborators. The mechanism of the "draw loop" assumes that code runs in a neglible amount of time, that is less than the refresh rate (and the default 1/60th of a second). This particular, again implicit, relationship with time places the programmer in an adversarial relation with the processor of the viewer's computer and makes it all too easy (certainly for novices) to produce code that makes the viewer's computer struggle and lag. Despite the claim of leaving space for others to bring their own "visual language", and thus an implicit proposition of its own aesthetics as "neutral", Processing embodies a very particular set of values and assumptions. The framework valorizes smoothness and fluidity, that leads one to imagine interactivity as that which happens on the surface of a sketch, rather than say in the network, or among collaborators. The mechanism of the "draw loop" assumes that code runs in a negligible amount of time, that is less than the refresh rate (and the default 1/60th of a second). This particular, again implicit, relationship with time places the programmer in an adversarial relation with the processor of the viewer's computer and makes it all too easy (certainly for novices) to produce code that makes the viewer's computer struggle and lag.
![](images/psquare2.png) ![](images/psquare2.png)
Unlike DBN, the processing book contains extensive interviews with artists working with digital tools, mostly using a variety of tools outside of processing like C++, PHP and Flash. Despite the "minimalism" of the examples, the book's subtitle claims relevance to a broad audience of "visual designers and artists". The link to processing is often unclear. Unlike DBN, the processing book contains extensive interviews with artists working with digital tools, mostly using a variety of tools outside of processing like C++, PHP and Flash. Despite the "minimalism" of the examples, the book's subtitle claims relevance to a broad audience of "visual designers and artists". The link to Processing is often unclear.
There's a strange disconnect where a diversity of visual and computation practices are shown, such as sequential images to make animations, cellular automata, the only real link to processing seems to be as a kind of universal "blank canvas" onto which all kinds of (digital) practices can be projected. There's a strange disconnect where a diversity of visual and computation practices are shown, such as sequential images to make animations, or cellular automata, but the only real link to processing seems to be as a kind of universal "blank canvas" onto which all kinds of (digital) practices can be projected.
Processing exhibits a huge disconnect when one crosses out of what the software makes simple. For example, while very easy to make a sketch respond to mouse clicks, responding to a mouse click on a specific shape, suddenly involves using Java classes and adding code "hooks" to your draw loop to make sure the pieces work together. While this kind of code is not itself so unusual, the transition is really abrupt because the core abstractions are so simplified, there is no bridge. Another disconnect occurs when one crosses out of what the software makes simple. For example, while very easy to make a sketch respond to mouse clicks, responding to a mouse click on a specific shape, suddenly involves using Java classes and adding code "hooks" to your draw loop to make sure the pieces work together. While this kind of code is not itself so unusual, the transition is really abrupt because the core abstractions are so simplified, there is no bridge.
Another disconnect involves processing sketches when published online. Web pages have structures for text layout and content flowing, and structual elements (like checkboxes and links) that inherently respond to mouse clicks. There is a a built-in standard mechanism (the so-called DOM or document object model), to further customize these behaviors. Processing sketches, even situated in a web page, don't participate or give access to this model, nor does the framework offer using alternative graphical elements like SVG[^svg]. Still another disconnect involves processing sketches when published online. Web pages have structures for text layout and content flowing, and structural elements (like checkboxes and links) that inherently respond to mouse clicks. There is a a built-in standard mechanism (the so-called DOM or document object model), to further customize these behaviors. Processing sketches, even situated in a web page, don't participate or give access to this model, nor does the framework offer using alternative graphical elements like SVG[^svg].
[^svg]: A web standard for "scalable vector graphics", SVG is more integrated into the DOM than are java applets (in classic Processing) or the canvas element (see p5.js). [^svg]: A web standard for "scalable vector graphics", SVG is more integrated into the DOM than are java applets (in classic Processing) or the canvas element (see p5.js).
@ -86,7 +86,7 @@ In 2016, the Whitney published a “restored” version of Software Structures.[
Despite the project's earlier stated interest in exploring diverse "materialities", its telling that rather than considering the older processing implementations as a different material and presenting screenshots of them as was done for the Flash and C++ examples, the "restoration" maintains the illusion of a "permanence" to the processing sketches, placing them closer to those imagined "software structures" than to "retrograde" technologies like Java or an out-dated browser. In addition the "adaptation" elides the work of the development and subsequent implementation of the then recently standardized canvas element[^canvas], as well as that of the creation of the p5.js library. Despite the project's earlier stated interest in exploring diverse "materialities", its telling that rather than considering the older processing implementations as a different material and presenting screenshots of them as was done for the Flash and C++ examples, the "restoration" maintains the illusion of a "permanence" to the processing sketches, placing them closer to those imagined "software structures" than to "retrograde" technologies like Java or an out-dated browser. In addition the "adaptation" elides the work of the development and subsequent implementation of the then recently standardized canvas element[^canvas], as well as that of the creation of the p5.js library.
[^p5]: Despite the seeming similarity of names, Java and Javascript are two completely independent and quite different programming languages. Adapting software from one to the other is not trivial. In making the adaptation McCarthy has explicitly stated that "diversity and inclusion as core values upon which the software is built". See: <https://lauren-mccarthy.com/p5-js> and <https://p5js.org/>. However, most of the limitations I talk about here still apply to sketches made with the current version of p5.js. [^p5]: Despite the seeming similarity of names, Java and Javascript are two completely independent and quite different programming languages. Adapting software from one to the other is not trivial. In making the adaptation McCarthy claims "diversity and inclusion as core values upon which the software is built". See: <https://lauren-mccarthy.com/p5-js> and <https://p5js.org/>. However, most of the limitations I talk about here still apply to sketches made with the current version of p5.js.
[^canvas]: <https://html.spec.whatwg.org/multipage/canvas.html#the-canvas-element> [^canvas]: <https://html.spec.whatwg.org/multipage/canvas.html#the-canvas-element>
@ -101,9 +101,9 @@ In 2007 I attended a book launch of the Processing Handbook. Earlier in the day,
> ImageMagick started with a request from my DuPont supervisor, Dr. David Pensak, to display computer-generated images on a monitor only capable of showing 256 unique colors simultaneously. In 1987, monitors that could display 24-bit true color images were rare and quite expensive. There were a plethora of chemists and biologists at DuPont, but very were few computer scientists to confer with. Instead, I turned to Usenet for help, and posted a request for an algorithm to reduce 24-bit images to 256 colors. Paul Raveling of the USC Information Sciences Institute responded, with not only a solution, but one that was already in source code and available from USC's FTP site. Over the course of the next few years, I had frequent opportunities to get help with other vexing computer science problems I encountered in the course of doing my job at DuPont. Eventually I felt compelled to give thanks for the help I received from the knowledgeable folks on Usenet. I decided to freely release the image processing tools I developed to the world so that others could benefit from my efforts.[^cristy] > ImageMagick started with a request from my DuPont supervisor, Dr. David Pensak, to display computer-generated images on a monitor only capable of showing 256 unique colors simultaneously. In 1987, monitors that could display 24-bit true color images were rare and quite expensive. There were a plethora of chemists and biologists at DuPont, but very were few computer scientists to confer with. Instead, I turned to Usenet for help, and posted a request for an algorithm to reduce 24-bit images to 256 colors. Paul Raveling of the USC Information Sciences Institute responded, with not only a solution, but one that was already in source code and available from USC's FTP site. Over the course of the next few years, I had frequent opportunities to get help with other vexing computer science problems I encountered in the course of doing my job at DuPont. Eventually I felt compelled to give thanks for the help I received from the knowledgeable folks on Usenet. I decided to freely release the image processing tools I developed to the world so that others could benefit from my efforts.[^cristy]
ImageMagick, first released in 1990, is a popular free software tool thats often referred to as a “swiss army knife” due to its ability to convert between hundreds of different image formats, and for the many built-in features to filter, manipulate and generate images. Thanks to the software being "not chemically or biologically based", Cristy was able to release the software as free software (echoing the way the UNIX operating system became free software due to its marginality to the interests of Bell Labs). The software is full of particularities. For instance, there are a number of built in images including a wizard (the mascot of the software) seated at a drawing table contemplating an image of the Mona Lisa. ImageMagick, first released in 1990, is a popular free software tool thats often referred to as a “Swiss army knife” due to its ability to convert between hundreds of different image formats, and for the many built-in features to filter, manipulate and generate images. Thanks to the software being "not chemically or biologically based", Cristy was able to release the software as free software (echoing the way the UNIX operating system became free software due to its marginality to the interests of Bell Labs). The software is full of particularities. For instance, there are a number of built in images including a wizard (the mascot of the software) seated at a drawing table contemplating an image of the Mona Lisa.
ImageMagick is a command-line tool, designed to be used via textual commands. The typical usage of ImageMagick is to take one image as input, applying one or more transformations to it, and output a new image. In this way the tool can be used repeatedly in a so-called “pipelines”, or otherwise composed together in structures called (shell) scripts. In these scripts ImageMagick commands can be mixed with other commands from any software installed on the user's computer that also provides a command-line interface. ImageMagick is a command-line tool, designed to be used via textual commands. The typical usage of ImageMagick is to take one image as input, applying one or more transformations to it, and output a new image. In this way the tool can be used repeatedly in so-called “pipelines”, or otherwise composed together in structures called (shell) scripts. In these scripts ImageMagick commands can be mixed with other commands from any software installed on the user's computer that also provides a command-line interface.
[^cristy]: John Cristy, "History", ImageMagick website. Retrieved from the Internet archive Oct 28, 2021, <https://web.archive.org/web/20161029234747/http://www.imagemagick.org/script/history.php>. [^cristy]: John Cristy, "History", ImageMagick website. Retrieved from the Internet archive Oct 28, 2021, <https://web.archive.org/web/20161029234747/http://www.imagemagick.org/script/history.php>.
@ -113,7 +113,7 @@ Salehis book directly reflects the structure of ImageMagick, with chapter org
![](images/imbook_flag.jpg) ![](images/imbook_flag.jpg)
In another extended example, a British flag is constructed in steps. Rather than approaching the project as drawing geometric forms on a canvas, Salehi uses the diversity of ImageMagicks manipulations, performing a series of commands whose textual names invoke a sense of physical construction: blocks of color are skewed, sheared, cropped, flipped, flopped, and finally spliced (with "gravity" set to center). The approach creates a number of intermediate images, thus creating the digital equivalents of "cuttings" or leftover materials in the process. In another extended example, a flag is constructed in steps. Rather than approaching the project as drawing geometric forms on a canvas, Salehi uses the diversity of ImageMagicks manipulations, performing a series of commands whose textual names invoke a sense of physical construction: blocks of color are skewed, sheared, cropped, flipped, flopped, and finally spliced (with "gravity" set to center). The approach creates a number of intermediate images, thus creating the digital equivalents of "cuttings" or leftover materials in the process.
By modifying the first step to use an image[^boris], I produced the following (intermediate) results: By modifying the first step to use an image[^boris], I produced the following (intermediate) results:
@ -138,16 +138,21 @@ Constructivism is also the name given to the pedagogic project associated with S
> The process reminds one of tinkering: learning consists of building up a set of materials and tools that one can handle and manipulate. Perhaps most central of all, it is a process of working with what youve got. …. This is a science of the concrete, where the relationships between natural objects in all their combinations and recombinations provide a conceptual vocabulary for building scientific theories. Here I am suggesting that in the most fundamental sense, we, as learners, are all *bricoleurs*.[^mindstorms] > The process reminds one of tinkering: learning consists of building up a set of materials and tools that one can handle and manipulate. Perhaps most central of all, it is a process of working with what youve got. …. This is a science of the concrete, where the relationships between natural objects in all their combinations and recombinations provide a conceptual vocabulary for building scientific theories. Here I am suggesting that in the most fundamental sense, we, as learners, are all *bricoleurs*.[^mindstorms]
Papert described the pedagogic project of LOGO in Mindstorms[^ref]. In a key example, Papert describes how students can be taught about circles by imagining (or better yet themselves enacting) the turtle repeatedly performing the "go forward a little, turn a little". He contrasts this with the formal equation of a circle (x^2^ + y^2^ = r^2^) typically taught in an elementary school geometry class. Papert described the pedagogic project of LOGO in book titled Mindstorms. In a key example, Papert describes how students can be taught about circles by imagining (or better yet themselves enacting) the turtle repeatedly performing the sequence "go forward a little, turn a little". He contrasts this with the formal equation of a circle (x^2^ + y^2^ = r^2^) typically taught in an elementary school geometry class.
> TO CIRCLE REPEAT [FORWARD 1 RIGHT 1] > TO CIRCLE REPEAT [FORWARD 1 RIGHT 1]
![](images/mindstorms04.jpg) ![](images/mindstorms-seq01.jpg)
![](images/mindstorms05.jpg) ![](images/mindstorms-seq02.jpg)
![](images/mindstorms-seq03.jpg)
![](images/mindstorms-seq04.jpg)
![](images/mindstorms-seq05.jpg)
![](images/mindstorms-seq06.jpg)
![](images/mindstorms-seq07.jpg)
In a powerful central visual sequence, Mindstorms presents a series of illustrations showing the screen output of code alongside a running dialog. The conversation starts with a proposition to draw a flower like one sketched on paper. They considers what programs they might already have to make use of, in this case a procedure to draw a quarter circle. Through a series of steps, mistakes are made, plans are adjusted and retried, happy accidents lead to discoveries (it's a bird!). In the process the "ends become means" and a new tool is put to use to create a garden, and then, incorporating the "bug", a flock of flying birds. In a powerful central visual sequence, Mindstorms presents a series of illustrations showing the screen output of code alongside a running dialog. The conversation starts with a proposition to draw a flower like one sketched on paper. First they consider what programs they might already have to make use of, in this case they have a procedure to draw a quarter circle. Through a series of steps, mistakes are made, plans are adjusted and retried, happy accidents lead to discoveries (it's a bird!). In the process the "ends become means" and a new tool is put to use to create a garden, and then, incorporating the "bug", a flock of flying birds.
In Belgium, where I currently live "brico" is the French language equivalent to "DIY" and is often used in a derogatory sense to indicate that something is made in an amateurish way. Papert, is borrowing the term from Claude Levi-Strauss, who first used the term in an anthropological context hypothesizing how "universal" knowledge might form from myth and fragmentary cultural knowledge[^levistrauss]. In Belgium, where I live "brico" is the French language equivalent to "DIY" and is often used in a derogatory sense to indicate that something is made in an amateurish way. Papert, is borrowing the term from Claude Lévi-Strauss, who first used the term in an anthropological context hypothesizing how "universal" knowledge might form from myth and fragmentary cultural knowledge[^levistrauss].
[^levistrauss]: Claude Lévi-Strauss. *The Science of the Concrete*. The Savage Mind (University of Chicago Press: 1966), Chapter 1. [^levistrauss]: Claude Lévi-Strauss. *The Science of the Concrete*. The Savage Mind (University of Chicago Press: 1966), Chapter 1.
@ -190,19 +195,13 @@ Star draws on a tradition of diverse feminist thinking through the "articulation
[^star_method]: Susan Leigh Star, 148-149. [^star_method]: Susan Leigh Star, 148-149.
For me tools like ImageMagick embody *collectivity* from it's origins as a way to "give back" to a community sharing code over usenet, through to its continued development by multiple authors and relation to the larger free software community as an invaluable toolbox for extremely diverse pratices. I find the *experiential* in the highly flexible commandline interface, itself also an example of honoring *contradiction* and *partialness*, with often more than one way to express the same transformation. The *processual* is implicit in its construction as a tool of transformation, encouraging an exploratory iterative approach to composing transformations to arrive at a desired outcome, often leading to misusage and errors that can be happy accidents and lead one to reconsider one's goals. Finally, in its extreme support of hundreds of different formats, ImageMagick use often leads to the discovery and exploration of diverse image formats, each with related practices and histories. As a teacher, I enjoy using tools like ImageMagick in my teaching as it embodies *collectivity* from its origins as a way to "give back" to a community sharing code over usenet, through to its continued development by multiple authors and relation to the larger free software community as an invaluable toolbox for extremely diverse practices. I find the *experiential* in the highly flexible commandline interface, itself also an example of honoring *contradiction* and *partialness*, with often more than one way to express the same transformation. The *processual* is implicit in its construction as a tool of transformation, encouraging an exploratory iterative approach to composing transformations to arrive at a desired outcome, often leading to misusage and errors that can be happy accidents and lead one to reconsider one's goals. Finally, in its extreme support of hundreds of different formats, ImageMagick use often leads to the discovery and exploration of diverse image formats, each with related practices and histories.
As a pedagogic project, Processing has seemed actively uninterested in its own underlying materiality, aspiring instead to a kind of disembodied universality. Students are encouraged to explore the "world at large" by adding additional layers of technology in the form of sensors, rather than considering all the ways the technologies they use are *already engaged with the world*. The project's "neutral" aesthetics while dimly echoing a once-radical Bauhaus aesthetic, ignores the larger pedagogic program of the historical Bauhaus' engagment and experimentation with the materials of their (contemporary) technical production. In contrast, I find as a pedagogic project, I find Processing actively uninterested in its own underlying materiality, aspiring instead to a kind of disembodied and bland universality. Students are encouraged to explore the "world at large" by adding additional layers of technology in the form of sensors, rather than considering all the ways the technologies they use are *already engaged with the world*. The project's "neutral" aesthetics while dimly echoing a once-radical Bauhaus aesthetic, ignores the larger pedagogic program of the historical Bauhaus' engagement and experimentation with the materials of their (contemporary) technical production.
> A concatenation of operations of misplaced concreteness thus allow the gaps, overlaps, and voids in the interrelated capacities of such systems to construct a more "accurate" account of its own operations.[^fuller_p104] > A concatenation of operations of misplaced concreteness thus allow the gaps, overlaps, and voids in the interrelated capacities of such systems to construct a more "accurate" account of its own operations.[^fuller_p104]
Fuller's recipe for critical media engagement feels like the basis of what might be termed a digital vernacular. A digital vernacular is about working withing the constraints of available resources, rather than living with a fantasy of negligible time or unlimited storage. A digital vernacular rejects the sense of disembodiment, rather if locates itself in interactions between many bodies. A digital vernacular finds itself in conversational forms that are open to happy accidents. A digital vernacular rejects the "false neutrality" of the seamless universal design solution, embracing instead the tips and tricks of specific tools, in specific contexts. A digital vernacular tears open its seams proudly displaying its glitches and gaps. A digital vernacular rejects the illusory construction of an isolated artist sitting at a blank canvas creating works from scratch. Instead the digital vernacular thrives on working with the contingencies of existing systems, and embraces working with boundary objects as a means of bridging diverse communities of practices.
Vernacular rejects the "false neutrality" of the seamless universal design solution, embracing instead the tips and tricks of specific tools, in specific contexts. A vernacular composition tears open its seams proudly displaying the glitches and gaps as a badge of honor of a world view of truth as multi-threaded, incomplete, and sometimes uncomfortable, and is engaged with the world and not afraid to be seen as a monster.
A vernacular approach rejects the illusory construction of an isolated artist sitting at blank canvas creating works from scratch. Instead the digital vernacular thrives on working the contingencies of existing systems, and embraces working with boundary objects that connect to diverse practices and practitioners without erasing their specificity, even when it involves contradictions or incomprehension.
The digital vernacular rejects the sense of disembodiment, rather if locates itself at the intersection between many bodies.
The digital vernacular is about working withing the constraints of available resources, rather than living with a fantasy of neglible time or unlimited storage.

@ -0,0 +1,207 @@
---
title: "torn at the seams: considering digital vernacular"
author: "Michael Murtaugh"
papersize: a4
---
> Processing is a free, open source programming language and environment used by students, artists, designers, architects, researchers and hobbyists for learning, prototyping, and production. Processing is developed by artists and designers as an alternative to proprietary software tools in the same domain. The project integrates a programming language, development environment, and teaching methodology into a unified structure for learning and exploration.[^p1]
Teaching programming with free software to media design students for years, I've resisted Processing as it has always seemed to me to embody a particular kind of solipsism of digital interactivity and graphics that I want my students to avoid.
## design by numbers
[^p1]: Casey Reas and Ben Fry, *Processing: A Programming Handbook for Visual Designers and Artists* (Cambridge: MIT Press, 2007), xxi-xxii.
In the fall of 1996, John Maeda joined the MIT Media Lab to replace the recently deceased Muriel Cooper. Cooper was the first art director of the MIT Press, producing influential designs such as a 1969 catalog of the Bauhaus and the iconic MIT Press logo, a Bauhaus-inspired stylized graphical rendering of the letters “mitp”. Cooper started the Visible Language Workshop, later one of the founding groups of the MIT Media Lab, to research the intersection of publishing, design, and computation.
![](images/dbn01.png)
> Our forefathers at the Bauhaus, Ulm, and many other key centers for design education around the world labored to create a sense of order and method to their teaching. Thanks to their trailblazing work, teaching at the university level gradually became accepted as a meaningful and constructive activity. A drawing board, small or large, became the stage for paper, pen, ink, and blade to interact in the disciplined activity that characterized the profession of visual design.[^dbn]
[^dbn]: John Maeda, Design By Numbers. (Cambridge: MIT Press, 1999), 19.
[^dbn_144]: Maeda, Design By Numbers. 144.
[^dbn_252]: Maeda, Design By Numbers. 252.
Maeda created the Aesthetics and Computation group in part to continue Coopers research. Maeda developed (with students Tom White, Peter Cho, Ben Fry, and later Casey Reas among others) a software system called Design by Numbers[^dbnsite]. It had extreme constraints such as a fixed 100 by 100 pixel size and monochrome-only graphics. The command set is similarly constrained with only two drawing commands for lines and points. Commands like "paper" and "pen", controlling the gray value of background and foreground colors, invoke the materiality of a (pre-digital) print practice. The accompanying print publication also had a square format.
[^dbnsite]: <https://dbn.media.mit.edu/>
![](images/dbn02.png)
> When designing this system for learning basic computational media design, I intentionally limited the set of commands and constructs to a minimal number of possibilities. If I had given you drawing capability beyond a line or setting a dot, the examples could have been more exciting, but the point could not be made clearly because your attention would be drawn to the picture and not to the code.[^dbn_144]
![](images/dbn03.png)
This valorization of "code" over picture is evident in the book spread where a digital image of a vase is presented along with its "code": a sequence of "set" commands describing the image pixel by pixel. Though it's possible that such an image might be produced in this way (discipline!) in practice such images are created by translating a digital photograph using one of a family of techniques known as "dithering" to produce a pixelated image with limited gray values. Here these tools and techniques are unmentioned, with expansive space given instead to the listing of numbers.
I remember sitting with a friend at his Commodore 64 in the 1980s typing lines and lines of "poke" commands with digital data from the pages of home computer magazines to reproduce simple games. These pages would typically use compact representations (like hexadecimal) and include things like "checksums" and bootstrap programs to help you correctly enter and verify the data you entered.
In the case of the DBN's digital vase, the purpose does not seem to be for someone to actually type the numbers in, but rather to fill the pages of a book that seems destined for the coffee table rather than a desktop. Similar to the fields of 1s and 0s still popular as backgrounds of book sleeves and PowerPoint slides to suggest "all things digital", the presentation is gratuitous and misleading in terms of actual practice. Like an observer unfamiliar with deaf culture confusing the hand gestures of finger spelling with the expressivity (and ambiguity) of actual sign language, the displays of 1s and 0s, or in this case of numeric coordinates and gray values, is a shorthand that refers digital practices without actually participating in them.
The above is just one example of DBN's lack of historicity. The year is 1999, and for instance the Sony PlayStation and access to the World Wide Web are popular phenomena. Yet the text seems remarkably devoid of any reference to specific tools or practices. Even Maeda's invocations of historical figures like the Bauhaus and Paul Rand are vague, evoking a sort of nostalgic *Mad Men* universe with the creative young men (those future forefathers) at their drawing boards and a sense of "timeless design values" like a devotion to discipline and order.
At the end of the book Maeda responds to a critique apparently made to him by one of the students doing some of the programming of the DBN system itself. Recounting how a visit to a "university-level Java class for designers [...] teach[ing] the finer points of object-oriented programming and bit masking of 24-bit color values" made him feel "lost in all the gibberish", he reasserts his pedagogical approach as an alignment with the "simplification" that is the "constant goal" of programming.[^dbn_252] Rather than trying to bridge the gulf between diverse practices, Maeda dismisses that which he doesn't (care to) understand in the name of simplification.
## processing
Processing was born as kind of fork or remake of DBN. Like DBN, Reas and Fry built Processing in Java, a desktop application that exists outside of the web but which can be used to publish sketches as "applets" embedded in a web page and published online. Reas and Fry added commands to work with color and multiple output sizes, as well as commands to draw basic shapes like circles, rectangles, and triangles.
Processing sketches consists of (at least) two functions: *setup* which is invoked once and *draw* which is invoked continuously; the default frequency being the refresh rate of the computers display (typically 60 times per second). By using variables combined with input functions allowing access to the mouse and keyboard, graphics can be made dynamic and responsive to the user. In addition, graphics by default are rendered using a technique known as “anti-aliasing” to appear “smoother” and less pixelated.
![](images/pbook.png)
> Most of the examples presented in this book have a minimal visual style. This represents not a limitation of the Processing software, but rather a conscious decision by the authors to make the code for each example as brief and clear as possible. We hope the stark quality of the examples gives additional incentive to the reader to extend the programs to her or his own visual language.[^p1]
Processing: A Programming Handbook for Visual Designers and Artists is published in 2006 by the MIT Press, with Reas + Fry authors, and a foreword by Maeda. The book comprises over 700 pages and is organized by topics like: color, control, data, drawing, image, input, math, motion, structure, typography. The book follows the visual style of the DBN book, with small square (mostly) monochrome images accompanying concise listings of code.
![](images/pbook_inside.png)
Despite the claim of leaving space for others to bring their own "visual language", and thus an implicit proposition of its own aesthetics as "neutral", Processing embodies a very particular set of values and assumptions. The framework valorizes smoothness and fluidity, that leads one to imagine interactivity as that which happens on the surface of a sketch, rather than say in the network, or among collaborators. The mechanism of the "draw loop" assumes that code runs in a negligible amount of time, that is less than the refresh rate (and the default 1/60th of a second). This particular, again implicit, relationship with time places the programmer in an adversarial relation with the processor of the viewer's computer and makes it all too easy (certainly for novices) to produce code that makes the viewer's computer struggle and lag.
![](images/psquare2.png)
Unlike DBN, the processing book contains extensive interviews with artists working with digital tools, mostly using a variety of tools outside of processing like C++, PHP and Flash. Despite the "minimalism" of the examples, the book's subtitle claims relevance to a broad audience of "visual designers and artists". The link to Processing is often unclear.
There's a strange disconnect where a diversity of visual and computation practices are shown, such as sequential images to make animations, or cellular automata, but the only real link to processing seems to be as a kind of universal "blank canvas" onto which all kinds of (digital) practices can be projected.
Another disconnect occurs when one crosses out of what the software makes simple. For example, while very easy to make a sketch respond to mouse clicks, responding to a mouse click on a specific shape, suddenly involves using Java classes and adding code "hooks" to your draw loop to make sure the pieces work together. While this kind of code is not itself so unusual, the transition is really abrupt because the core abstractions are so simplified, there is no bridge.
Still another disconnect involves processing sketches when published online. Web pages have structures for text layout and content flowing, and structural elements (like checkboxes and links) that inherently respond to mouse clicks. There is a a built-in standard mechanism (the so-called DOM or document object model), to further customize these behaviors. Processing sketches, even situated in a web page, don't participate or give access to this model, nor does the framework offer using alternative graphical elements like SVG[^svg].
[^svg]: A web standard for "scalable vector graphics", SVG is more integrated into the DOM than are java applets (in classic Processing) or the canvas element (see p5.js).
### software structures
In 2004, Reas co-developed an exhibition at the Whitney gallery called Software Structures. Invoking Sol Lewitt's wall drawings as an inspiration, the project presents a series of abstract rules (or software structures) for the production of an image, including some from Lewitt. The rules were then implemented using a variety of "materials": Processing, Flash MX, and C++.
> A benefit of working with software structures instead of programming languages is that it places the work outside the current technological framework, which is continually becoming obsolete. Because a software structure is independent from a specific technology, it is possible to continually create manifestations of any software structure with current technology to avoid retrograde associations.[^sst]
[^sst]: Casey Reas, "Paragraphs on Software Art", Software Structures, retrieved October 28 2021, <https://artport.whitney.org/commissions/softwarestructures/text.html#structure>.
In 2016, the Whitney published a “restored” version of Software Structures.[^ssreloaded] As technologies like Java and Flash had then for reasons both technical and commercial fallen out of popular use on the web, the new version featured many of the processing sketches adapted by Reas to use p5.js, a recent port of processing to Javascript made by artist Lauren Lee McCarthy.[^p5]
Despite the project's earlier stated interest in exploring diverse "materialities", its telling that rather than considering the older processing implementations as a different material and presenting screenshots of them as was done for the Flash and C++ examples, the "restoration" maintains the illusion of a "permanence" to the processing sketches, placing them closer to those imagined "software structures" than to "retrograde" technologies like Java or an out-dated browser. In addition the "adaptation" elides the work of the development and subsequent implementation of the then recently standardized canvas element[^canvas], as well as that of the creation of the p5.js library.
[^p5]: Despite the seeming similarity of names, Java and Javascript are two completely independent and quite different programming languages. Adapting software from one to the other is not trivial. In making the adaptation McCarthy claims "diversity and inclusion as core values upon which the software is built". See: <https://lauren-mccarthy.com/p5-js> and <https://p5js.org/>. However, most of the limitations I talk about here still apply to sketches made with the current version of p5.js.
[^canvas]: <https://html.spec.whatwg.org/multipage/canvas.html#the-canvas-element>
## imagemagick
In 2007 I attended a book launch of the Processing Handbook. Earlier in the day, I had bought another technical book "ImageMagick Tricks: Web Image Effects from the Command line and PHP" by Sohail Salehi[^salehi]. While waiting for the presentation to begin, I met Casey Reas at the back of the room. He was curious about the book I had with me and looked briefly at it. He had never heard of ImageMagick.
[^salehi]: Sohail Salehi, *ImageMagick Tricks* (Birmingham: Packt Publishing, 2006)
![](images/imbook.png)
> ImageMagick started with a request from my DuPont supervisor, Dr. David Pensak, to display computer-generated images on a monitor only capable of showing 256 unique colors simultaneously. In 1987, monitors that could display 24-bit true color images were rare and quite expensive. There were a plethora of chemists and biologists at DuPont, but very were few computer scientists to confer with. Instead, I turned to Usenet for help, and posted a request for an algorithm to reduce 24-bit images to 256 colors. Paul Raveling of the USC Information Sciences Institute responded, with not only a solution, but one that was already in source code and available from USC's FTP site. Over the course of the next few years, I had frequent opportunities to get help with other vexing computer science problems I encountered in the course of doing my job at DuPont. Eventually I felt compelled to give thanks for the help I received from the knowledgeable folks on Usenet. I decided to freely release the image processing tools I developed to the world so that others could benefit from my efforts.[^cristy]
ImageMagick, first released in 1990, is a popular free software tool thats often referred to as a “Swiss army knife” due to its ability to convert between hundreds of different image formats, and for the many built-in features to filter, manipulate and generate images. Thanks to the software being "not chemically or biologically based", Cristy was able to release the software as free software (echoing the way the UNIX operating system became free software due to its marginality to the interests of Bell Labs). The software is full of particularities. For instance, there are a number of built in images including a wizard (the mascot of the software) seated at a drawing table contemplating an image of the Mona Lisa.
ImageMagick is a command-line tool, designed to be used via textual commands. The typical usage of ImageMagick is to take one image as input, applying one or more transformations to it, and output a new image. In this way the tool can be used repeatedly in so-called “pipelines”, or otherwise composed together in structures called (shell) scripts. In these scripts ImageMagick commands can be mixed with other commands from any software installed on the user's computer that also provides a command-line interface.
[^cristy]: John Cristy, "History", ImageMagick website. Retrieved from the Internet archive Oct 28, 2021, <https://web.archive.org/web/20161029234747/http://www.imagemagick.org/script/history.php>.
![](images/imbook_inside.png)
Salehis book directly reflects the structure of ImageMagick, with chapter organized around various incorporated “tools”: convert, mogrify, composite, montage, identify, display, conjure. The examples are practical, creating logos or adding captions or a border to an image. One example renders the word “Candy” with colorful stripes. Another series of examples duplicates and inverts the image and text of classical Persian poet “Hafez” to create a kind of playing card. Another example uses ImageMagick in conjunction with PHP and HTML to produce an online “e-card maker”, a sequence of commands is demonstrated to render the text “No More War” (in a dripping paint font), deform it, and project it onto the side of a chess piece.
![](images/imbook_flag.jpg)
In another extended example, a flag is constructed in steps. Rather than approaching the project as drawing geometric forms on a canvas, Salehi uses the diversity of ImageMagicks manipulations, performing a series of commands whose textual names invoke a sense of physical construction: blocks of color are skewed, sheared, cropped, flipped, flopped, and finally spliced (with "gravity" set to center). The approach creates a number of intermediate images, thus creating the digital equivalents of "cuttings" or leftover materials in the process.
By modifying the first step to use an image[^boris], I produced the following (intermediate) results:
![](images/background2.png)
![](images/mixed.png)
![](images/flag.png)
[^boris]: Image: <https://commons.wikimedia.org/wiki/File:Boris_Johnson_official_portrait_(cropped).jpg>, Ben Shread / Cabinet Office, UK Open Government Licence v3.0 (OGL v.3)
## constructivism and the bricoleur
In the 1920s Russian avant-gardist El Lissitsky would move to Berlin and produce work that was highly influential to the then nascent Bauhaus. In 1923, Lissitsky illustrated a publication of poems by friend Vladimir Majakovskij. In it he created graphical forms by mixing typographic elements with geometric forms created by (mis-) using spacing or “blind” elements typically used to create negative/unprinted space between lines of type, as positives producing geometric forms. This style, sometimes called constructivism, was part of an effort to make a radical break from traditional styles of typographic layout and illustration using the means then available for print. The book is notable for its "interactivity" via iconic tabbed pages.
![*For the Voice*, El Lissitzky designer; Image from the archive of Guttorm Guttormsgaard. Used with permission. <https://arkiv.guttormsgaardsarkiv.no/node/19/item/39>](images/forthevoice.png)
![](images/mindstorms00.jpg)
![](images/mindstorms01.jpg)
Constructivism is also the name given to the pedagogic project associated with Seymour Papert. In the 1970s Papert co-developed a pedagogy for teaching children mathematics and programming based on the LOGO programming language. Part of the system was a (virtual) robotic turtle that could be programmed to draw figures. The system, known as Turtle graphics, had commands like: forward, turn left, turn right, pen up, pen down, that directly addressed the “turtle” to draw shapes while moving.
> The process reminds one of tinkering: learning consists of building up a set of materials and tools that one can handle and manipulate. Perhaps most central of all, it is a process of working with what youve got. …. This is a science of the concrete, where the relationships between natural objects in all their combinations and recombinations provide a conceptual vocabulary for building scientific theories. Here I am suggesting that in the most fundamental sense, we, as learners, are all *bricoleurs*.[^mindstorms]
Papert described the pedagogic project of LOGO in book titled Mindstorms. In a key example, Papert describes how students can be taught about circles by imagining (or better yet themselves enacting) the turtle repeatedly performing the sequence "go forward a little, turn a little". He contrasts this with the formal equation of a circle (x^2^ + y^2^ = r^2^) typically taught in an elementary school geometry class.
> TO CIRCLE REPEAT [FORWARD 1 RIGHT 1]
![](images/mindstorms-seq01.jpg)
![](images/mindstorms-seq02.jpg)
![](images/mindstorms-seq03.jpg)
![](images/mindstorms-seq04.jpg)
![](images/mindstorms-seq05.jpg)
![](images/mindstorms-seq06.jpg)
![](images/mindstorms-seq07.jpg)
In a powerful central visual sequence, Mindstorms presents a series of illustrations showing the screen output of code alongside a running dialog. The conversation starts with a proposition to draw a flower like one sketched on paper. First they consider what programs they might already have to make use of, in this case they have a procedure to draw a quarter circle. Through a series of steps, mistakes are made, plans are adjusted and retried, happy accidents lead to discoveries (it's a bird!). In the process the "ends become means" and a new tool is put to use to create a garden, and then, incorporating the "bug", a flock of flying birds.
In Belgium, where I live "brico" is the French language equivalent to "DIY" and is often used in a derogatory sense to indicate that something is made in an amateurish way. Papert, is borrowing the term from Claude Lévi-Strauss, who first used the term in an anthropological context hypothesizing how "universal" knowledge might form from myth and fragmentary cultural knowledge[^levistrauss].
[^levistrauss]: Claude Lévi-Strauss. *The Science of the Concrete*. The Savage Mind (University of Chicago Press: 1966), Chapter 1.
For Papert, bricolage exhibits a quality whereby informal methods not only appeal to "common sense" but also engage more profoundly with the materiality of its subject than would a formal approach. In the case of the circle, the "turtle" method is not only a way for the student to imagine the problem physically, it also relates to methods of differential calculus, something the algebraic formulation misses completely. In hacker circles, bricolage is evident in an approach of embracing "glue code" and "duct tape" methods, like the pipeline, that allow different systems to be "hacked" together to do useful (new) things.
[^mindstorms]: Seymour Papert, *Mindstorms* (Cambridge: MIT Press, 1980), 173.
## misplaced concretism & a feminist method
[^ssreloaded]: <https://whitney.org/exhibitions/software-structures>
Alfred North Whitehead, writing on the sciences, established an influential idea of a "fallacy of misplaced concreteness". The idea is that making abstractions, such as what happens when a particular phenomenon is named, is a simplification that works by suppressing "what appear to be irrelevant details".[^whitehead] In *Media Ecologies*, Matthew Fuller extends this thinking to consider technical standards as "a material instantiation"[^fuller_p127] of Whitehead's misplaced concreteness, and considers how technical devices through a process of *objectification* "expect in advance the results that they obtain".[^fuller_p104]
Susan Leigh Star takes Whitehead's "misplaced concretism" and proposes a feminist methodology specific to information technology.[^star] Her essay develops the idea of "standards" as one type of "Boundary object", which she describes as:
[^whitehead]: Alfred North Whitehead, *Science and the modern world* (New York: Free Press, 1967), retrieved from the Internet archive Oct 28, 2021 <https://archive.org/details/sciencemodernwor00alfr/page/52/mode/2up>
[^fuller_p127]: Matthew Fuller, *Media Ecologies* (Cambridge: MIT Press, 2005), 127.
[^fuller_p104]: Matthew Fuller, 104.
> [...] those scientific objects which both inhabit several communities of practice and satisfy the information requirements of each of them. Boundary objects are thus objects which are both plastic enough to adapt to local need and common identity across sites.[^star_p157]
Star cites Donna Haraway, who wonders in *A Manifesto for Cyborgs*:
> How do I then act the bricoleur that we've all learned to be in various ways, without being a colonizer.... How do you keep foregrounded the ironic and iffy things you're doing and still do them seriously [...]
Star draws on a tradition of diverse feminist thinking through the "articulation of multiplicity, contradiction, and partiality, while standing in a politically situated, moral collective" to synthesize and propose what she calls the important attributes of a feminist method:
> 1. experiential and collective basis;
> 2. processual nature;
> 3. honoring contradiction and partialness;
> 4. situated historicity with great attention to detail and specificity; and
> 5. the simultaneous application of all of these points.[^star_method]
[^star]: Susan Leigh Star, "Misplaced Concretism and Concrete Situations: Feminism, Method, and Information Technology" (1994), *Boundary Objects and Beyond* (Cambridge: MIT Press, 2015)
[^star_p157]: Susan Leigh Star, 157.
[^star_method]: Susan Leigh Star, 148-149.
As a teacher, I enjoy using tools like ImageMagick in my teaching as it embodies *collectivity* from its origins as a way to "give back" to a community sharing code over usenet, through to its continued development by multiple authors and relation to the larger free software community as an invaluable toolbox for extremely diverse practices. I find the *experiential* in the highly flexible commandline interface, itself also an example of honoring *contradiction* and *partialness*, with often more than one way to express the same transformation. The *processual* is implicit in its construction as a tool of transformation, encouraging an exploratory iterative approach to composing transformations to arrive at a desired outcome, often leading to misusage and errors that can be happy accidents and lead one to reconsider one's goals. Finally, in its extreme support of hundreds of different formats, ImageMagick use often leads to the discovery and exploration of diverse image formats, each with related practices and histories.
In contrast, I find as a pedagogic project, I find Processing actively uninterested in its own underlying materiality, aspiring instead to a kind of disembodied and bland universality. Students are encouraged to explore the "world at large" by adding additional layers of technology in the form of sensors, rather than considering all the ways the technologies they use are *already engaged with the world*. The project's "neutral" aesthetics while dimly echoing a once-radical Bauhaus aesthetic, ignores the larger pedagogic program of the historical Bauhaus' engagement and experimentation with the materials of their (contemporary) technical production.
> A concatenation of operations of misplaced concreteness thus allow the gaps, overlaps, and voids in the interrelated capacities of such systems to construct a more "accurate" account of its own operations.[^fuller_p104]
Fuller's recipe for critical media engagement feels like the basis of what might be termed a digtial vernacular. A digital vernacular is about working withing the constraints of available resources, rather than living with a fantasy of negligible time or unlimited storage. A digital vernacular rejects the sense of disembodiment, rather if locates itself in interactions between many bodies. A digital vernacular finds itself in conversational forms that are open to happy accidents. A digital vernacular rejects the "false neutrality" of the seamless universal design solution, embracing instead the tips and tricks of specific tools, in specific contexts. A digital vernacular tears open its seams proudly displaying its glitches and gaps. A digital vernacular rejects the illusory construction of an isolated artist sitting at a blank canvas creating works from scratch. Instead the digital vernacular thrives on working with the contingencies of existing systems, and embraces working with boundary objects as a means of bridging diverse communities of practices.

Binary file not shown.

Binary file not shown.
Loading…
Cancel
Save