Inside an abstract place, within a virtual reality, across a latent space, the territory of the NewNewNewWorld (NNNW) stretches out. Its history peaks centuries ahead of the New World as found by the XV century colonial Europe, and it does so after the Contemporary World as well. The worlding of it, the worldbuilding of it, seems to require a post-post-colonial politics as it is not only to be discovered but, just as the New World to some extent, to be also generated. Such politics, implying both an abducted and an abductive cognition, both a praxis of unlearning and one of designation and objectification, both an inside and an outside view, define an in-between kind of mediator, or as it were, a post-contemporary Malitzin.
At the moment, the NNNW only exists within the memory of the AI model that knows better than every other AI what the human world looks like: the BigGAN. To acquire such knowledge, BigGAN was trained on ImageNet –for years, the largest dataset for AI training, assembled to “map the entire world of objects”, according to its creators– which made it capable of generating accurate images of the more-than-20 thousand ImageNet object classes, comprehending around 14 million tagged images. Born from a “large scale GAN training for high fidelity natural image synthesis” (Andrew Brock + Jeff Donahue + Karen Simonyan, 2019), it is capable of “artificial” image synthesis –more precisely, synthesis of artificial objects. Within its Latent Space, a series of new new new objects –or Xenobjects, as I call them– all of them differentiable from ImageNet’s object classes in which it was trained, potentially exists. Even after having landed on the NNNW and having developed a basic cosmography of it already, I wonder if “we are still far from understanding the hidden world of the [BigGAN’s] latent space” (Zaid Alyafeaid, 2019).
Importantly, the virtual existence of all of these xenobjects is also non-correlated to our human capability of imagining possible worlds. To be clear: it is the BigGAN itself who imagines these xenobjects. It was not trained on them and they didn’t exist in the brains of humans nor in any other media, and they were not plannead neither anticipated –though they are now latent in my computer and they are latent in yours if you download it. This is actually the whole point with Machine Learning, with this specific type of AI and with everything called Software 2.0 (Andrej Karpathy, 2017); instead of following the “classical stack” in which the human programmer wites every line of code that, given some input, the computer will read and execute in order to complete a task and yield an output –in this case, the synthesis of a 512 x 512 pixels image–, the human developer moves to the sides, preparing a dataset that defines a desired goal for the behavior of the program –as well as a “skeleton” of the code–, and lets the computer learn how to come up with the appropriate program on the basis of massive iterations of trial and error. Millions of photographs of the Current World come in, a simulant worlding process runs, and images surprisingly resemblant of the Current World come out. Nevertheless, an Old World can go in, a non-expected worldbuilding can occur, and a New World can go out –which is exactly what happened in the XV and XVI centuries after europeans arrived to the Americas.
In fact, before geography revealed to europeans that they didn’t arrived to the unknown Indies, they have already projected onto the Americas what they speculated about the known Indies. But when encountering actual land, people, animals, and other objects, they mediated their own knowledge importing fantasy scripts in a generative fashion. In the early years of explorations…
“...no one had specific knowledge about the New World. Early sixteenth-century commercial contracts between the [Spanish] crown and entrepreneurs refer to common commodities such as precious metals, precious stones, plants, animals, fish, birds, medicines, and ‘any thing of any name and quality’ of value. Some of these contracts referred to ‘monsters’ and ‘serpents’ (Vas Mingo 1986 [Las Capitulaciones de Indias en el siglo XVI]).” (Antonio Barrera-Osorio, 2007)
The outputs of Spaniards and other “discoverers” also included other less indiferent creatures, like the Epaiwanoma, those headless people with their eyes and mouth in their chest. Jodocus Hondius depicted them in his chart 1598 chart of The Guianas (now French Guiana, Suriname and Guyana) and his map of North America –him among other cartographers who represented the Epaiwanoma in other parts of the Americas and beyond. The very long exploration and colonization process by the Spaniards was documented in very unregulated ways at the beginning and then increasingly organized by changing insitutions designated by the crown, ultimately being preserved mostly by the Archivo General de Indias, currently located in Seville, Spain. As Epaiwanoma were projected into maps, Quetzalcóatl and other local inventions were disappeared from the territory. If “a world in fact is the projection of meaningful patterns onto the surrounding space of lived experience” (Berardi, 2015), then the Current World for the pre-columbian Mesoamerican cultures ended as a World emerged anew for both the colonized and the colonizers. In the following centuries, for both, the monsters transformed but hardly disappeared.
What the BigGAN learned from ImageNet was, precisely, patterns in the space surrounding humans with digital cameras. It imported categories, such as “puppy”, “cross-country riding” and “globigerina”, but mostly it apprehended pixel arrangements at different scales: how deconstructed sets of color squares organize together across JPEG files previously separated in folders. Similarly to the New World, the NNNW is emerging form the gathering of fragmentary patterns of the Current World, taking bits from #snail and #skonk, and pieces from #strawberry and #chihuahua (and many more), giving shape to a #wildBot or an #UnidentifiedNavigatingObject. I’m not sure if I see the catastrophe on the interface –or “the past coming apart” (Land and Plant 1994)– but I do see the “anastrophe” –or “the future coming together”.
But… is the NNNW really coming together? Is the BigGAN actually printing a New New New World on my screen? Is the BigGAN constructing it for sure, or is it just me projecting patterns on your surrounding space? Is the BigGAN a NNNW_Constructor?
I will try to answer that question in terms of Constructor Theory, a theory in current development by David Deutsch and Chiara Marletto at the University of Oxford. This theory investigates which things (and transformations) can and cannot occur in the universe by means of a constructor (a machine) that performs (and can repeat) a task, proving a transformation and a resulting thing to be possible, which can finally be expressed in a “counterfactual statement”. In the case of this research, the counterfactual statement would assert that a New New New World (of objects) can be generated (i.e. can be imagined, can be printed as a digital image) by the BigGAN. Counterfactuals come relevant to this research on worldbuilding –and worldbuilding research–, as a counterfactual statement “describes how the world would have been if the antecedent [declared in the statement] had obtained” (ESSRM, 2004). “If P had obtained, then Q would have obtained”. If (not) P, then (not) Q. “If the BigGAN was only ably to reproduce the objects in the ImageNet dataset, then only objects in ImageNet would be its output”. According to Marletto, “a counterfactual is a statement about which transformations are possible and which are impossible in a physical system”, and “a transformation is possible when you have a ‘constructor’ that can perform a task and then retain the capacity to perform it again” (in Gefter, 2021), say a NNNW_Constructor that can generate a new xenobject many times. That would mean to generate multiple objects that are different from ImageNet’s –i.e. xenobjects–, but also different from the already generated xenobjects.
The assesment of the BigGAN as a NNNW_Constructor involves two technical devices. The first one, a NNNW_Archive, is a dataset comprised by hundreds of classified xenobjects. Thse second one, a NNNW_Explorer, is a Machine Learning model trained on the NNNW_Archive. The NNNW_Explorer, an image classifier, will compare the resemblance of new xenobjects to, both, ImageNet classes and NNNW’s classes, and that should provide a measure of how much the BigGAN is generating, not an ImageNet world nor noise, but a NNNW.
By the 1570’s, personal experience of merchants and explorers, beyond ecclesiastical and academic imaginary, “had become the indispensable tool for the study of natural things in the New World”, “a nature without referents in the classical traditions” according to Antonio Barrera-Osorio (2007), who argues that a strongly enhanced tendency towards empiricism within this process locates the beginning of the Scientific Revolution in the Iberic Peninsula. Nature, be it minerals or pineapples, were primarily conceived as commodities –i.e. “as a collection of isolated and extractible entities”. As merchants and explorers were the first intermediaries for the creation of knowledge about found nature (found culture not excluded), “empirical descriptions of these entities became the prevailing method for understanding them”. Personal observation, interviews, and expeditions, all translated to drawn and written reports, was the basis for collective and remote recognition; they circulated from hand to hand across networks that connected indigenous daily life, English academia, and the Spanish crown. Throughout these networks, representational patterns got enhanced, lessened, and nuanced to various degrees according to the accumulating experience of each involved agent.
Spain, says Barrera-Osorio, “was the first imperial and commercial power to create institutions for gathering, organizing, and disseminating empirical information about the New World”. In 1508, the crown established the position of the chief pilot at the Casa de Contratación, which “would become a veritable chamber of knowledge from this poin on” (Barrera-Osorio, 2007). The chief pilot would help “to correct personal bias by comparing several reports on the same topic, and by evaluating personal observation in the Casa’s social setting of expert evaluators”. Through such institutions, “they sought to train people in the new methods [of knowledge production and management] emerging from these activities”. I would argue that people got trained, not only in the protocols for processing the constituting patterns of the New World, but also in the very patterns they were processing. These New World-processing institutions served to gather and shape New World knowledge through the harvesting of information generated by large networks of empirically-driven agents and its evaluation by output discriminators. Were these information-processing systems the first Machine Learning systems –or as called in Spanish, Automated Learning systems?
Just as the Archivo General de Indias was the dataset in which the New World Constructors were (unevenly) trained, the NNNW_Archive is the memory in which I’m training human and non-human explorers –i.e. myself and an artificial co-pilot. The NNNW_Archive is the generation of a large diversity of Xenobjects that can be organized in differentiable categories which nevertheless contain differentiable instances (e.g., distinguishable #stemBot examples that can be all identified as #stemBot). Through this novel training set, through this post-contemporary General Archive of Indies, the NNNW will be able to infiltrate into the sentience of other AIs, to propagate its influence on the transformations of the present material world, and to occupy a place in the future.
Across the multidimensional territory of the NNNW, you can find xenobjects like a wildBot –undisclosedly carbon- or silicon-based, most likely who knows– wandering around in all kinds of environments, doing whatever thing of any name and utility. Another kind of xenobject you can stumble upon is arRock, which is not only a rock but something arising from it as if being synthesized directly from mineral stratum. There’s also stemBots, xenobjects of seemingly very small dimentions and a persisting latent state. I wonder if perhaps they evolve into wildBots or maybe stillArchitecture, which is still something that looks like the objects in which humans use to inhabit but which could turn into an UnidentifiedNavigatingObject at any moment. Do they actually evolve? do they merge and separate under changing conditions? Do they form from local matter and dissolve into it? I really wonder.
Those are just some of the xenobjects I’ve come across, but there may be plenty of them which I cannot recognize. I wish I could. I reality, there’s only a few characteristics that allow me to become aware of them, to extract them and save them on my drive, to search for them and print them, either temporarily on my screen or permanently in other substrates that would give them width, height and depth. I’m here to augment their reality, and for that to have the broadest set of possible outcomes, they have to be segmentable and photographically realistic –i.e. I have to be able to distinguish them from the background and I must feel I could 3D-sculpt them. Width, height, depth, and time are less dimensions than the > 21’000 dimensions of the BigGAN’s latent space, but those are the dimensions I can help them to inhabit as those are the dimensions (or the elemental technical abstractions) of the world that I inhabit myself.
The xenobjects’ on-screen and off-screen existence has required, up to date, a human mediation –specifically human sensing– that can prompt BigGAN’s generation of xenobjects as 2D images. Given that there were no AI models trained on these objects—because they didn’t previously exist—this task could have been very hardly automated through current AI techniques. So far, human vision that entails symbolic cognition and complex perceptual training is the best to identify imagined (imaged) novel objects that could be mapped and printed on- and off-screen.
But in fact, the automation of NNNW’s exploration has already started, not only as my own visual training but also as the training of an AI co-pilot that helps me to refine the classification –to consider possible misclassifying and possible classes not yet identified. The NNNW_Explorer is the first Machine Learning model trained on the NNNW_Archive dataset, and shares with me the speculative-realist function of providing evidence of the BigGAN’s objective capacity for creative, post-human, object-oriented worldbuilding, beyond any fabulation I could narratively convince you into –i.e. the function of arguing in favor of the existence of the NNNW_Constructor.
An “objective” fact, or “objective” knowledge, following Pablo Fernández Christlieb (2004), here means knowledge that has been subjected to the objection of other objects. If culture is a sort of inter-subjective knowledge, I’m looking for it’s un-severing from science as inter-objective knowledge. My counterfactual statement being “if the BigGAN wasn’t a NNNW_Constructor, it would not be able to generate xenobjects”, the NNNW_Explorer is here to make an objection about it. The wanted following assertion –to which, if this research is succesful, neither myself, neither the NNNW_Explorer, nor yourself nor many other objects will object– is of course that the xenobjects exist and that the NNNW can be printed.
The way the NNNW_Explorer works is, it takes an image of a test set corresponding to a NNNW_Archive class or to an ImageNet class, and assess its resemblance to the classes of both datasets. In doing so, it evaluates both the differentiability of xenobjects (from ImageNet objects) and the differentiability between xenobjects (one class from one another). The difference between the evaluated xenobject and the ImageNet objects will be given by how much it’s 100% resemblance is shared by ImageNet and by NNNW_Archive classes. The difference between the evaluated xenobject and other xenobjects will be given by how much it’s 100% resemblance is shared by other NNNW_Archive classes.
Nevertheless, that’s just how the NNNW_Explorer would ideally work. For the moment, it has not yet be trained on ImageNet as it is a huge dataset that will demand more time and resources, so it’s only capable of evaluating the existence of different types of xenobjects.
Indeed I’m engaging with politics of designation and politics of objectification. I feel like playing the role of a post-contemporary Malitzin, or Maliche (or both), who mediated the end of the world for pre-columbian cultures and the rising of the New Spain as the personal translator (and lover) of Hernán Cortés, head of the conquest of Tenochtitlan (the capital of the Aztec empire) and other Mesoamerican cities and cultures.
“A world in fact is the projection of meaningful patterns onto the surrounding space of lived experience”, says Berardi (2015), and I’m in a way creating a world by projecting meaningful patterns into your surroundings. Yes, my practice is about projecting the bias in my own neural net to some extent. But it is also to do as much as I can to see beyond the categories in which I’ve been trained. In fact, it is to become able to see other categories. It is to say “that’s a stemBot” where you would say “that’s a bug”. It is to say “that’s a xenobject” where you would say “that doesn’t look like anything to me”. It is not to be a Dutch cartographer at the middle of the past millennium, and it is in fact not to be a TERF at the beginning of the current.
It’s me and the BigGAN who are projecting patterns onto your surroundings. Just as I don’t want anyone to “lose their ability to feel like they are part of a common evolving reality” (Berardi, 2’015), I want every agent in my shared reality to have the feeling that they are part of a common evolving reality. This is an excersise on doing so with an agent developed to reproduce the meaningful patterns of others.
Computational culture seem to have the potential to supersede every non-computational culture. How can we make the computaional worldbuilding a symbiosis instead of a conquest? Perhaps suscribing to a Malitzin() function and to the practice of “opening to the incomprehensible other […] and reinventing language”, and culture, in order to transform the probabilities of catastrophe into the probabilities of anastrophe.
Alyafeai, Zaid (2018), BigGanEx: A Dive into the Latent Space of BigGan, available from https://thegradient.pub/bigganex-a-dive-into-the-latent-space-of-biggan/ [5 May 2022].
Avanessian, Armen and Malik, Suhail eds (2016). The Time Complex: Post-Contemporary. [NAME].
Barrera-Osorio, Antonio (2007), Nature and Experience in the New World: Spain and England in the Making of the New Science, (in Beyond the Black Legend: Spain and the Scientific Revolution, Víctor Navarro Brotóns (coord.), William Eamon (coord.), Universitat de València, 2007).
Berardi, Franco ‘Bifo’ (2015). ‘Malinche and the End of the World’, in Julieta Aranda, Brian Kuan Wood, Anton Vidokle eds., The Internet Does Not Exist. Sternberg Press / e-flux journal.
Brock, Andrew + Donahue, Jeff + Simonyan, Karen (2019), Large Scale GAN Training for High Fidelity Natural Image Synthesis, ICLR 2019, available from https://arxiv.org/abs/1809.11096 [5 May 2022].
Crawford, Kate and Paglen, Trevor (2019), Excavating AI: The Politics of Training Sets for Machine Learning, [online] Available from https://excavating.ai [5 May 2022].
Fernández Christlieb, Pablo (2004), La Sociedad Mental. Anthropos.
Gefter, Amanda (2021), How to Rewrite the Laws of Physics in the Language of Impossibility (interview with Chiara Marletto), Quanta Magazine.
Gershgorn, Dave (2017), The Data That Transformed AI Research—and Possibly the World, Quartz, available from https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/ [5 May 2022].
Karpathy, Andrej (2017), Software 2.0, available from https://karpathy.medium.com/software-2-0-a64152b37c35 [5 May 2022].
Land, Nick and Plant, Sadie (1994), Cyberpositive, available from http://www.sterneck.net/cyber/plant-land-cyber/ [5 May 2022].
Lewis-Beck, Michael (University of Iowa) + Bryman, Alan (Loughborough University) + Futing Liao, Tim eds. (2004), Encyclopedia of Social Science Research Methods. Sage Publications, available from http://www.worldcat.org/oclc/52706518 [5 May 2022].
Massumi, Brian (1998), ‘Sensing the Virtual, Building the Insensible’, in Stephen Perrella ed., Hypersurface Architecture, Architectural Design (Profile no. 133), vol. 68, no. 5/6, May-June 1998.
Rabasa, José (1993), Inventing America: Spanish Historiography and the Formation of Eurocentrism, University of Oklahoma Press.