design domain
this year's design domain's theme is Process, which offers a wide range of matters to explore. it is such an open-ended question that it could, literally, mean ontologically everything. Alfred North Whitehead's philosophy of process describes how reality itself is held together by various processes, perhaps this is why it's been applied to so many disciplines ranging from finance to ecology.
Process and Reality (1913) develops a philosophy of organism, called process philosophy. it's book of cosmology, as it proposes a definition for how reality comes to be. to be fair, i don't think i can understand that book as is without some formal education in academic philosphy. Whitehead's wording is very abstract and it's generally a very hard mental gymnastics session, but I perused online for takes about it that would help me understand, and I found an interesting book by Gilles Deleuze, A Thousand Plateaus (co-authored with Felix Guattari. not specifically because this book references Whitehead in any way but many pointed some similarity between Deleuze's speculative realism and Whitehead's metaphysics. this is probably due to the fluidity that both philosophers associate with reality itself. to put it simply Whitehead delineates existence as the derivative of a continuum of processes, and because of that there are no static entities - reality is formed by how each process is interconnected with one another. similarly, the rhizome concept expressed by Deleuze and Guattari proposes a non-linear networking system where each point is connected with one another with no hierarchical importance - called "multiplicity", and derived from the biological concept of mutualism where 2 different individuals act as one for the benefit of both. rhizome derives from Ancient Greek, and refers to a mass of roots. this "planar" concept of existence wants to show how history and culture are a bi-dimensional array of sorts more than a linear narrative, hence a one dimensional array.
these reads, alongside "Difference and Repetition" (Deleuze, 1968) are all quite heavy and abstract and complex, but they illustrate some views of reality, science and art that i really resonate with. especially Deleuze, it's my first time reading him and i enjoyed a lot the passages about repetition, difference and individuation. what these philosophers have in common is the creation of a plexus of sorts, a network of processes guides existence into being. some processes adhere to strict laws and are out of our control - Whitehead's Actual Entities and Prehensions, the first ones being spontaneous events and the latter being how these different events interact with each other.
mapping is a concept (and a process) that comes to mind when reading these theories, and it's a huge one. it can go from biology to computation, and here i try to explore some of it by simulating a plexus. the patch consists of a particle system driven by a 2d texture, in this case it's a 3d perlin noise (it's 3d because when its format is set to 31-bit float RGBA we get vales that range in all 3 directions). in this case the connections of the plexus are given by color amounts in the 2d noise itself - not all amounts, just the more prominent pixels or either R, G and B. i am in fact dictating the system to map where these values are, and connect them together. a plexus is normally a bit more complex than that, usually the connections are lactice-type (triangles), in my case i can only get it to connect as a single line.
this other version below is made with the aid of a tutorial on youtube by touchdesigner Guru Paketa12. in this case the plexus works perfectly. the dimension of the sphere is given by the amount of connections, and the amount of connections is given by the distance of each sphere. his technique is very clever and following this tutorial gave me great insight on working with 2d textures for 3d space.
despite my version is less efficient and clumsy, both these experiments describe a mapping of sorts, of defining some measurements and applying those definitions to influence how elements interact with each other. i would say that mapping is the pre-process that aids the beginning of an interaction by aligning the dots (or in my case, the pixels). i like to think that my experiments emulates how one would connect the emerging rocks in a river to jump across it safely, while Paketa's mapping is more involved and complex and could surely be a visualization of biology, anthropology, sociology and so much more. it was very useful to learn his technique.
so mapping is the process of defining a space, and some points in such space, and draw relations between those points. perhaps my network, being uglier, better describes the concept of mapping, or at least, it's more realistic - because we can only absorb a certain amount of information at intervals of time. everything happens all around us, just as Paketa describes, but we have to apply thresholds as our number of sensors, as living beings, is limited. not to mention our processing speed and power. according to Dr Pragya Agarwal (in Sway, 2020), our brain can process 11 million bits per second, but our conscious mind can only manage 40-50. those thresholds are probably the most primordial instances of mapping.
mapping as means of navigation
chemotaxis is how most eukaryote organisms move towards nutrition. simply put, the outer membrane of their body contains receptors able to detect and map the direction and distance of specific chemical signals. in this case, and in the case of a nomad tribe moving through the desert, mapping is the aid of motion. it defines targets, or points of interest. for humans it's a bit more complex than that. navigation is managed in the entorhinal cortex, the hippocampus instead is able to remember those places. locomotion is even more complex as it involves a whole number of different processes (including the mentioned ones) in relation to the usage of motor neurons to... move muscles to move the body somewhere. the hippocampus' episodic memory itself is THE example of innate mapping present in some organisms par excellence. the book/research "The Hippocampus as a Cognitive Map" (1978) by John O'Keefe and Lynn Nadel - which wone them the Nobel Prize - shows how certain region show complex interactions when an organism is placed in a specific location, like their nest. a recent paper i read proposes more research now that the tools are even more advanced, the researchers use VR headsets to analyse the activity of how the hippocampi work together to help us orientate through space.
located within the temporal lobe, the 2 hippocampi serve different functions depending on the organism. in animals both sides are dedicated to memory, location recognition and orientation, but in humans that function is exercised by the right hippocampus only, the left one is responsible for creating a "narrative" of sorts, connecting the dots not only by memory but by real-time processing. the neocortex is then responsible for the spatial frameworks, which are almost like internal motion sensors placed in those parts of the body that are both responsible of and translated in space by motion: the retina has spatial frameworks alongside our neck and hands - although the mapping exercised by the hippocampi is indeed important, the ability to recognise where to go is not enough, we need to be able to calculate our position in that space, and the position of our organs. in short, movement is complex and mapping makes it a little bit easier.
but mapping is not solely a means of movement. i would suggest that the notion of mapping changed dramatically for humans once the first settlements began to occur in Moravia and the East European Plain between 25000 and 17000 BC. we could envision the process of mapping in a tribe almost as the motion of the amoeba: driven by where the resources are, instead of using chemical signals, the tribe could be a bit more sophisticated and rely on seasonal and geographical knowledge alongside notions of trail-tracking to follow bovine herds et similia. to simplify things even further we can plot how the nomadic tribe moves and note that it would be a 2 dimensional graph. what i mean by the change brought by the conversion into sedentary life-style is that the graph is now tri-dimensional, if before the path was decided on a horizontal level, now the path becomes dictated from above, on a vertical level of hierarchy.
i absolutely don't want to start implying things like "it was better before" and "this is/isn't natural", but there are clear examples of how mapping influences populations and individuals on both micro and macro scale, and the usage of mapping (to better say, imposition) in sectors such as infrastructure, urban planning all the way up to diplomacy can be and has been instrumentalised as a means to subjugation, surveillance and control.
Foucault, space and power
Foucault proposes the image of a Panopticon, a prison that develops in a cylindrical shape, with the cell's windows pointed towards the centre, where a vertical structure comprised of outward facing windows sits menacingly. the panopticon is a conceptual space (although it was also implement in reality) that implies a watcher and a number of people that need to be watched. it's important for the watcher to be in a stable location, or at least a location that gives the advantage to always look at the subject, while the subject instead is unable to look at the watcher, hence unable to determine whether the watcher is or isn't effectively watching. in a way, the subject is also unable to properly state whether the watcher exists or doesn't exist, but the doubt and the established conditions (as it's normally a prison) usually lead the subject to a state of constant alertness. ironically, if we can't see whether we are watched or not, it's much easier for our brain to assume that we are and act accordingly.
the panopticon is one of many concepts centred around a dystopian reality of all-rounded surveillance driven by architectural form and fear of such surveillance -it is a metaphor designed to describe how the arrangement of elements in space embeds power dynamics between those elements.
I want to visualise these concepts using blob-tracking (sort of), creating a toy that allow a user to impersonate an observer.
blob-tracking is a tool often associated with computer vision. specifically, object detection, motion detection, but also color detection. in other words, bloc-tracking allows me to map a group of pixels according to their values. pixels don't really have embedded special values such as motion, but there are some solutions (more like hacks) that we can apply to simulate motion detection using pixels. generally known as pixel differencing, consists on comparing a selected frame with the previous frame and use the differential value as motion value. combining blob-tracking and pixel differencing can sort of emulate a very rudimental human recognition model: "does it more at a certain speed and it's colour tones have a certain range? then it's a human".
here i'm using some footage of time's square just to test things. this newtork could as easily be attached to a webcam.
to achieve pixel differencing in real time i store a number of pixels in my cache, in this way i can subract the frames stored in the cache with the current frame continuously. below is a comparison between the differential image and the original video.
blobtracking is then applied to the differential image, to further capture the blobs with some amount of a specific colour value (in my case i'm using red).
blob-tracking works by reading each frame as a long list and detecting at what point in the list the pixels of a certain colour are. the screen is "read" from left to right in rows, so it's possible to define aggregations of colours, which are called blobs.
the blob-trackerTOP gives me useful data such as the location and the size of the blobs. i can then turn this data into instancing data, and build something graphical to represent the blobs more clearly. I wanted to dive in the theme of surveillance and analytics and such, so I used white rectangles to represent the blobs and lines to connect the blobs together, as to simulate the interconnectedness of people moving in the same space.
I've also added a few more HUD things just for aesthetics: some randomly appearing beziers, a 3D Cartesian graph to emulate a visualization of how people are arranged in the space and a couple of other data-visualisers to indicate the general amount of activity on screen.
the UI is very rushed, it only consists of a few sliders for now. to explain it briefly, "tiMa_offset" modulates the number of frames that are stored in cache, hence making the system more or less sensitive to motion. the "thresh" sliders modulate the general sensitivity of the blob-trackingTOP, while "size" affects how big of a blob on average should be detected. note, i added another set of blob-tracking data, this time in blue, in the hope of isolating cars too
eventually more functions should be added, like a webcam support and generally it needs a more appealing UI, but that takes some time that i don't really have now. this toy could then go on a touchscreen, and allow people to interact with this "life-inspired" plexus.