expressive data
Mira is my partner's monstera. Mira in italian (and spanish as well) means 'look' -- she's called so because of her way of always turning to look towards the sun. she was found dumped in a backgarden, and lived with Nasim since then, usually in their bedroom, travelling between flats with the other many houseplants they own.
Nasim is now doing their master's degree in london, at Goldsmith's, so i'm here happily taking care of their many plants. in total, it's about 40 plants (without including the numerous cuttings i have lying around in random jars), and although i'm not as disciplined as Nas, i think i'm managing to stay on top of it. i was born a farmer, but i'm not that well trained in houseplants, they require a lot more precision with the watering cycles and placement, some of them even require some frequent spraying instead of single, timely-spaced waterings.
of all the plants, Mira is the biggest one, and probably the oldest one, and the one that requires the less case and space (relative to size), so long as she can find a way to the light she needs a small amount of water every 5 days. she's around 1.5m tall, with a 1m diameter, and a couple of bamboo sticks planted through the soil of the large pot help her standing straight. this because the monstera deliciosa is a hemiepiphyte, which means she is partly epiphyte, which means that a time of its life is spent growing around another tree, crawling its roots firstly downwards and then sideways to look for a host. when she finds a strong enough trunk, she then develops upwards, following the rules of phototropism -- hence growing towards the sunlight. that said, the monstera is not a parasitic plant, it gathers resources from around the chosen host, and it's also capable of growing on its own, like Mira does, although she still maintains the desire of growing those aerial roots that help her grapple onto other trees. as mentioned before, it's a fairly resistant plant, able to grow when the temperature is above 10-15 degrees but, although i'd really love to see it, Mira will never produce flowers in your average glasgow flat. to be fair, the temperature and sun levels here are just enough for her to thrive, but she's doing well nonetheless.
Mira is also the plant i personally feel most attached to. of course because of her dimensions and complex form, but also because of her reactivity and motility. she always eventually gets in the way of things, stuck in between things, tangled into things. definitely difficult and heavy to handle, but shows the most beautiful hollow leaves that only adult exemplars are able to display.
in a way, taking care of these plants is a method for cherishing Nasim's absence in the flat. before this separation, we lived here as a couple for 1 year and a half. this couple-in-the-same-flat thing was a first time for both of us, but we managed just fine, and so did Nas' plants. i'm not going to lie that her being in london caused some abruption in my mental health, so looking for other ways to feel Nasim's presence around me into the care of their plants has been therapeutic so far. in a way, these plants are the extension of Nasim, not solely as fetish objects, but also as a representation of how they care for other beings. Nas gives an enormous amount of love and care to other beings, they have a beautiful and genuine empathy, so i'm just trying to mirror such love despite me usually being quite dry and cynical.
project overview
this project requires 2 processes: data acquisition through a physical computing interface, and data-expression through another physical computing-based piece. i've not enquired on whether it's possible for the 2 to be 1 thing running in real-time, and I'm not sure I'd be able to do that, but i think it would be quite interesting to see and might try if there's the possibility. nonetheless, the dual nature of the project calls for a piece that is divided in 2, interconnected parts. this because the collection of data can be viewed as a piece itself, or at least i think it's worth exploring as a concept.
data collection has to be achieved through some kind of sensor, and the yield needs stored inside of an SD card, for later use. the process calls for a conscious use of time, both in length and in "steps". how long is the collection going on for? should it be a single continuous stream or some briefer intervals positioned equally throughout a day/week? it depends on how fast and frequent the changes in the data stream are. the "steps" system is valuable to keep in mind because it aims for efficiency and robustness over a certain period of time. if the data is slow-changing over a large period of time i should aim for a system that activates for smaller intervals, to avoid strain and redundancy. nevertheless, the requirement of an SD card means that the yield needs to be relatively large.
when we choose to collect data, we are in fact trying to get a more or less long exposure shot of a "subject", that shows the state of such subject over a period of time. the camera metaphor is even more interconnected if we look at the camera itself as a (considerably large nowadays) array of sensors, each pixel able to display not solely a 0-1 value - useful for, say, motion detection - but also RGB values as well, offering more uses of how the collected data could be parsed and observed. of course, the number of processes that take place once the data is collected as raw is larger and more various than that, and offers fascinating mathematical (and ethical) questions - because how the data is processed is how it will be visualised and shown to ie a public in a conference, a group of investors, voters. in other words the whole process, from the collection to the visualization, must be held together by precision, transparency and equality. the age old argument of garbage in-garbage out, except it's not solely resolved in the choice of quality data, but also in the choice of the electrodes being used to capture the data, how we choose to sort such yield, and what aspects need removing because of redundancy and, still, is that fair? like a photographer that chooses which camera, which film, which acids to process the film with and how, data collection is an art of conscious choice.
expressive data sounds to me like a sort of performative act, where the data is the performer. it calls for a certain amount of, yes, honesty but also something that allows for interpretation. in a way data is no longer necessarily made clear for a viewer, but retains its features while generating a different output. a metaphor for this could be a film projector. film by itself looks like a darkened strip of plastic, the negative is imprinted, well, in negative, and that's our data, collected. a projector then is used to interpret such data, shining a light through it and, in fact, projecting it onto a screen, in a newly formed surface made of photons. the features of the film are in this case amplified in the right way to create a readable image. we can add even more data-types to the situation: some projectors present a "reading light" that shines through a fine slit to only analyse a small portion of the film. this portion is illuminated, but not projected, or at least, not projected to us - its features are instead sent over to a phototransistor, which vibrates some magnets in order to generate sound. the act of data-interpretation works again: what was just a strip on the side of the film showing a wave imprinted via magnetism, is now a full fledged audio generator. to achieve comprehension a certain amount of interpretation is to be expected. actually, there's no widely comprehensive data structure that is shown without it being interpreted in some way - before i was mentioning how the term "expressive data" calls for preformism, but to be correct all types of data visualization require a bastardization of the data itself in order to be read by any group of people who's outside of the experts' category in any given field.
data collection as eyes and ears of capitalism
for this project i chose to explore plants, one plant in particular, Mira. i chose Mira because she is the best specimen i have in my house in terms of robustness and, i hope, voltage. i chose to work with plants because i would like the work to be about biology in some way, and plants are the closest and safest thing i could work with.
data collection is, by definition, a capitalist AND humanist practice. it's about monitoring a system in order to predict things like energy efficiency, product quality, robustness over long term and so on.. in other words, data collection is how people try and predict the future of a system, mainly for commercial benefit. it relies on reliability itself to ensure an output that is both consistent and long-lived. it survives on predictive models to estimate market fluctuations, to propose the most advantageous investments and resource allocations. the book "Philosophy of Prediction and Capitalism" (1987) by Manfred S. Fring talks about how the predictive practices associated with capitalism are, yes, sort of beneficial in the short term, but that also represent an epistemological and ethical challenge. this because prediction comes at the cost of precision. predicting how an event goes in relation to a system means to reduce both the system and the event into minimal terms, we are in fact extracting them from the real world they live in, making them quantifiable in numbers or in recognisable and legible strings of text. most of the times the sentence goes "if THIS shows THESE numbers, then it means THAT", the simplification is necessary to ensure ease of readability, and "guesstimation". needless to say, Fring points out how reality is an infinite number of complex system, many of them interconnected to each other, making the research of simplicity quite difficult. actually, there's no such thing as a simple thing, says Brian Ridley in the book "Time, Space and Things" (1976) - absolute complexity is ever present at all dimensions: from societies down to the life in a skyscraper down to the life in a single family home, down to the organisms living in their kitchen sink. matter itself, despite us trying to categorise it into a reliable table of a finite number of elements, is infinitely complex both at surface level and in depth level. brownian motion pervades every single object, things vibrate and oscillate continuously, following impossibly hard to predict vectorial forces. even the single atom, what for many years was considered to be the building block of all matter, it's not just a sphere with some fancy orbiting spheres. just the effort of looking at one atom is abysmal. and don't get me started with waves and light and sound and electromagnetism.
so capitalism, with its predictive models, tries to achieve a Sisyphus-esque task. to simplify reality, to homogenise it into legible factors and functions, a task that will always be impossible and will always fail, because nothing is easy. the main problem arises when capitalist systems are applied. as i said, it is fundamental for such system to be simple rules, but it is also important mentioning that simplification is not solely present in the translation of reality, but also in the construction of the reality per se. cities are built in grids, modular designs are implemented in contruction to ensure ease of maintenance. reality is fabricated arounds simple systems, hence it TRIES to resemble a simple system. obviously, the age old argument, which is very specific to coding but can be applied to so much more is: garbage in, garbage out - and the problem is, the garbage out is for everyone in society, because again, capitalism craves simplicity, it needs to englobe different economies under its own rules to avoid unexpected fluctuations. you like it or not, you'll have to experience capitalism, and you'll have to experience it in its own terms.
this is mainly why i want to work with something organic and inhuman. it's more of a challenge to both myself and the principles of data collection. my work is far from accurate: my sensors are cheap because i'm just a student trying to survive in very low wage, my equipment is made of scraps i found in random charity shops. everything in my setup screams "this is not accurate!", but then again, that's not the point. the point is to implement capitalistic practices in a complex system. the work is about translation but the translation itself isn't the goal. the final output will be almost illegible, as an artist i'm just a descriptor and my job is to highlight such inaccuracy.
process
in very short terms i want the work to be divided into 2 distinct processes: data collection, and data visualization. one part is the camera that records the fluctuations of resistance between the plant's branches and their relation to soil moisture. the other part is a projector, aimed at translating the numerical data into visible data.
the "camera" works as follows: 2 EMG sensors and a soil moisture level sensor are placed on the subject. the collected data consists of 3 channels named EMG_1, EMG_2 and SML. the data is collected on an SD card and then transferred to a PC for parsing. from then, data is modified into 3 different audio channels. the 2 channels of EMG data are then sent through a laser projector, which will "write" onto a thin roll of paper by stimulating a photosensitive coating. the SML channel will be used in a later stage.
the projector itself is a simple device. it consists of a light source and a series of lenses aimed at directing and concentrating the light. my plan is to use multiple lights as "pixels" - only one light is depicted above - and to make a very low quality projector. i have access to a 4 channel dimmer, so it'll be a 2 by 2 pixels projector. the function of the dimmer is to work as a "shutter" mechanism - although the SML will also control the general brightness of the output. the reason of a shutter is to represent time. my data logger works by collecting 10 minutes of data every hour. i want those interruptions to be present in the final output as they tell the story of how the recording is timed. the "film" is then rolled so that a layer of it goes between the final light-focusing lens and the image magnification lens. it'll be a relatively correctly built, very low quality and super slow shutter speed projector. i've had the chance to repair a couple super 8 film projectors (they only needed some new contacts and a change of grease) and i've grown fascinated with those machines. something about their purely mechanical work able to produce an image of light, and the sound of the film whirring through the motor tells a story of transparency and compromises.
studying plant electricity is far from a new thing, but there's lots of very interesting papers that started popping out from the 2010s onwards. might be because of general technological availability. artists have tried to make use of such signals for years, some receiving more or less impressive results. nevertheless the research continues, and what seemed to be a branch of pseudoscience has now become an academically sought after challenge.
obviously in my case money makes all the difference. my sensors are eEMG sensors, the small "e" stands for "external", as in "external electrodes", as in non-invasive measument probes. usually these probes consist of a pad with some electrically conductive gel that is aimed at maximising the contact between the electrode and the porous human skin. now, most of the interesting scientific experiments regarding plants and electricity involve some kind of invasive probe. a needle, or a small tungsten rod are inserted in a plant's tissues.
after various attempts at getting the non-invasive electrodes to work (which produced very little results) i opted for removing the pad electrodes from the wires, and then soldered the wires to a tiny needle made of lead. i'm aware that the EMG modules are probably set to read electricity from a different type of electrode, but that's what i have, and it produced interesting results.
the first, most impressive thing is that when the electrodes are placed correctly i do get a signal. I was fearing that the presence of liquid would ruin my system, but everything seems to work fine.
in this video you can see a few things. first of all, the interface i was using to record the data on PC - i was still testing things before having it running autonomously - is CoolTerm, i'm just using it to record the data quickly so that i can at least tell what's going on. secondly, the level i'm receiveing seems to surpass the EMG module's sensor by quite a lot (1023 is the max number), thirdly, the data resembles a pulsation. it goes up and then comes down at regular intervals. worth noting that the length of this was varied through the day.
does this mean i'm onto something. to be honest i have no idea. i read online that plants do communicate using waves and pulses that can be picked up, but i'm very far from declaring that i'm reading a plant, a lot more money should be dumped on this experiment to define what's normal and what's not. but there's both a regularity and irregularity in these waves -sometimes they disappear and come back minutes later, sometimes they are very erratic - so i'm thinking that there should definitely be something organic going on. and honestly, i've seen the CSM arts and science graduation show and it was such a pseudo-science fair despite them even having built in labs in the building. so if they can fake it, i can too.
this is the code i wrote for the data-logger. it's set to record data on the SD card using the appropriate libraries. i define my recording intervals so that the arduino records and writes for 10 minutes, stops for 1 hour, then starts recording again. there are a few useless bits in this code: most of the Serial.print statements are not visible when the data logger works on its own, they're just there to help me see things during testing
this video shows the data logger in action. the system will run for about a week, i should stop it on Sunday!
light theory and material choice
a few words about lasers and light theory:
following my tutor's advice, i decided to abandon the idea of using a laser. of course it's a safety matter, and it would make things uncomfortable to proceed because a tutor should be with me at all times in order to use such a device.
i opted for using a projector instead. it is less conceptually interesting in my opinion, but gets the job done.
the reason i wanted to use a laser is because, fundamentally, this work is more about usage of light more than the data itself. sure the data is important, but i want to use this work as a means to observe other phenomena, and i'm very much interested in the realm of photons and how they work.
this because a laser diode emits a particular type of light called "coherent" light, meaning a type of beam that doesn't really de-focus as it travels in space. the phenomenon of spatial coherence is a property of laser diodes that makes them unique to other light sources. for example, a projector yes will emit with some coherence, but because light is both vector and wave at the same time, splashing will occur, with relative de-focus. of course, i'll place my projector very close to the screen, so light won't have enough space to de-focus in a way that is relevant.
I was interested in the idea of using a laser exactly because i'm keen to work with the light's duality. a coherent light in a way is a "controlled" reaction, it either "goes there" or it doesn't, there's little to no noise to deal with. which is also the reason why lasers are used in precision works, as analytical tools, communication and "modelling" tools (laser cutting).
as for the medium used to store data, i chose an emulsion produced by Jacquard, and there's a few reason for this. firstly, i needed a photo-emulsion that would respond to light relatively quickly - to avoid strain in the system - and i wanted something that would utilize light in a specific way. let's consider a different photo-emulsion, IE cyanotype. cyanotype changes colour according to where some photo-radiation hits. it's more a work of oxidation in this case, the surface is not really changing morphologically, it is just a chromatic change.
as explained above, though, my idea was to utilize a home-made film in order to project some image. this means that i don't solely want an image constituted by colour, but i need the image to work as a physical barrier in order to project its shadow. in a way, i need 2 layers: the supporting layer, which is the paper, and the image layer, which is constituted by the Jacquard's photo-emulsion.
some research online led me to believe that the photo-emulsion i chose should be the perfect candidate for this. this photo-emulsion is normally used in the fabric's screen printing process. it is usually applied to a screen, a super fine grid usually made of silk, in order to create screens that would then be used to print images on fabric (using pressure). the way this photo-emulsion works (very simplified) is by magnetically binding the exposed material to the supporting material, effectively creating those 2 layers i talked about. this because the photo-emulsion, when dried, acts like acrylic paint. this process is fascinating to me, because the magnetism that holds the paint to the paper is a reaction given by light's electro-magnetic force itself. again, the duality of light, able to being both a wave and a electrically charged particle, shows up.
about the data
some considerations regarding the data i extracted and what it looks like.
the main thing to note is that i don't have enough money to make this project as it should be done. my sensors tend to stop reading, sometimes the batteries powering the sensor's boards just dies, my environment is far from being sterile and controlled. in general, it was more of a challenge of trying to get something with tools that anyone can find and buy!
of course that's not the point, the point is to explore some things while maintaining an eye out for notions of science and, more importantly, method.
as mentioned before, a lot of new literature about plants, voltage and resistance has emerged in the last decade, and this helped me focus on what exactly i should be looking for when probing a plant. according to science, plants have a range of different electrical signals they use to "express" their status. sometimes it's simple bursts as a reaction to sudden pain, most of the times it seems like they produce fluctuations that are perceived as waves, in a wide variety of amplitudes and frequencies. these waves are in fact voltage variations due to the plant's internal activity - in fact, leaves act almost as generators, and the stems are more like wires that transport energy around the system. it's a lot more complicated than that, but in short it should be possible to detect these waves as variations in the plant's resistance. this is why i decided to use these EMG sensors: these sensors work by sending a constant and low stream of current, and by comparing this stream from the short bursts emitted by muscular activity they convey a differential number which describes the subdermal electric impulses produced by muscles. of course my sensors are external so it's definitely different amplitudes of voltages, but they can surely detect something.
this is one of my first yields, SML stands for soil moisture sensors. the reason the SML was introduced was to add dimensionality to my data. of course it should have been done in a more holistic way: a phototransistor should have been used to determine light-cycles, a temperature sensor should have been present too. multi-dimensionality is good practice in data-extraction, it helps us understand what we're looking at by putting different things in comparison. so in my case, SML is more of a symbolic string of data, although it can still mean something. please note, the SML sensor will give a value between 0 and 1023 - 0 basically means 100% soil hydration, while 1023 means that the soil is completely dry. funnily enough, the principle works similarly to the one used by the EMGs.
this illustrated above is the same dataset. picture on the left is the dataset as a whole, while a zoomed in section is shown in the picture on the right.
this shows me that i am indeed receiving some kind of pulses. given my equipment it's impossible for me to say whether these are effectively emitted by the plant, but keep in mind that when the sensors are in idle state they'll emit a constant value of 1023, while if they are low on battery they'll send a constant-ish value that goes from 50-100. these pulses instead last for about 1-2 seconds, and go from 0 to 1023, and then gradually lower back to 0. the reason they appear as square waves is probably because the sensors are designed to have a good level of sensitivity, given that they normally have to be contact-sensors, so the electrical impulses seem to go out of that threshold by quite a lot. this obviously doesn't help me as i can't really define how potent these pulses are.
the whole dataset is shown above. this dataset was collected over 5 days. the 2 channels at the top are respectively the EMGs, while the purple, slowly rising line is the SML. some considerations:
i think it's worth noting how the plant's signal gradually stop as the soil moisture lowers. I don't know if it's related at all, but apparently plants do generate some activity when transporting fluids around. OR it could be that my batteries died - only the EMGs were connected by battery - and i think this is the most likely option.
writing process
starting in touchdesigner (TD), i wrote a simple patch that looks at the dataset, parses it to make it easier to use (eliminating letters, punctuations etc), and then arranges some particles and lines on screen accordingly to visualise said dataset. the patch also listens to a trigger (given by the arduino in the second part of this experiment) that tells it to read the next 50 samples of data - this because reading the whole dataset in one go wouldn't make it very legible (as you can see in the above images). while building my patch i took form into consideration, because the form of my output will be what stores information on the "film". this is why i opted for the most minimal approach in terms of form. peaks are represented by a small square, while they are connected with each other by a single line. simplicity is valuable in this case because i'm printing this data in a quite unconventional way.
this is how the patch looks like from the outer level. the box named "data_vis" contains the actual patch, the other 2 boxes are just for outputting the actual image to a projector.
this is the parsing side of the patch. it reads from left to right. data is imported as textDAT (data operator, TXT). a convertDAT (CVT) separates the text into columns, then a trio of selectDAT divides the columns, another convertDAT is inserted in the chain, this time to get rid of any non-numerical value. that's pretty much the gist of it. the channel operators (green) are simply normalizing everything to prepare it for instancing.
the visualization happens in another section of the network, and it's basically only an instancing operation. instancing is almost similar to a for-loop for 3d geometry: instead of rendering one particle at a time and individually, the particles are instead rendered as a group according to a varying serie of coordinates. in my case, an ascending ramp wave dictates the X axis, putting all of the particles in a row, while the data given by each channel dictates the Y axis position. in this case these are all the coordinates i need for each string of data, as it's visualised as a bi-dimensional array.
above is one of the channels singled out just to show how the final form will be. below is a wee screen visualization i made, just to allow me to put a window in a separate monitor and keep looking at how things progress. on the top-left corner i added a percentage monitor to show at what point the writing is, while the channels are color coded, some bloom is added just to make it a bit more pleasant to look at. note how the SML is so slow that, when viewed at 50 samples-per-time, it just looks like a flat line.
a series of experiments with colours and materials was made to determine what worked best for this technique. i already knew that white light would leave the strongest sign, but i tried and yes, white light is the best! i tried different fabrics and papers: rice paper, organza, thin cardboard, cling film, other transparent plastics. nothing was as clear as the simple printing paper. i forgot to take pictures of the other materials, but to be honest nothing at all would show up, or it was incredibly faint. i think that because of how this material works, a certain type of fibre is needed for it to really grip. organza and other fabrics had too much space in between the fibers, while cling film and plastics offered no fibre at all.
the downside of using paper is that the screen need to be washed afterwards, and this paper is very frail. but using a chopping board and some hot water did the job anyways
i also had a go with the dataset, just to calibrate line width and square size. i also experimenting adding some very slow filter and, although it looks cool, it makes things very unclear. basically i applied a 5 minutes long logarithmic filter. because a filter applied as such is non-linear, the motion should slow down a lot towards the end of the cycle, hence applying more light in the same point, hence adding a sort of gradient to the tone. and it worked, but i don't think i should go in this direction (yet).
(the data is quite mispositioned because i was basically holding the paper for 5 minutes then spinning it around to write the next 50 samples).
what i gathered from these experiments is that i need a white line width of 2 pixels, and a square size - for the peaks - of around 3pixels. i need a fibrous material, but with a very fine disposition of its fibres, paper is okay, but is maybe too fine. the exposition time is of 3-5 minutes in complete darkness.
i wanted to create a film of sorts, so i devised a couple of improvised rotary wheels to store the data on strips of A1 paper.
to do so i needed a couple of servos that would spin at the same speed and the same direction continuously. i received a couple of handy continuous servo motors from the studio and wrote these very simple lines. all it does is: print "0" in the serial monitor and spin the motors at a certain power for 1 second, then print "1" in the serial monitor and stop for 5 minutes. the servos are attached to the arduino in the fashion described by the picture above, which is not mine, and it has 3 motors instead of 2, but it basically works the exact same. a 5v 1a power supply sends energy to a breadboard that distributes that energy in parallel to the servos, ground is common and the signal goes from digital pins 2 and 9 only, in my case.
despite it being held together by tape, the system kinda works. basically the 1 and 0 in the signal monitor are captured in TD to trigger the rendering of the next 50 samples and also to modulate the brightness, turning off the image when the wheels are spinning and then turning it on again while they're still.
i tried my absolute best to get rid of the white rectangle, but apparently i really can't. must be a DLP chip thing - even at 0 brightness it shows up. fortunately it doesn't show up, but i'm sure it does have some kind of impact.
i know it's a mess. i've used bits of PVC piping as my rotary wheels, some metal wire to stop the paper from just sliding down. the servos are attached to a foamex panel to keep them from falling off.
the A1 paper i used seems to have even a finer fibre structure than the printing paper, so very little actually caught with it, but there's something nonetheless. each array of 50 samples is captured after the other, and where the emulsion stuck the lines are clearly visible. i think i also have a washing problem, because they were much more visible before washing.
final considerations, links and bibliography
i dacided to not proceed with the "projector" bit of the project. this because a lot more experimentation needs to be done with materials and type of paint and i've not even started looking into light focus and lenses and all that. i simply don't think i have the time to manage that section and work at the same time. but i consider those 3 strips of paper as a pseudo-satisfactory output. i don't really like them but they prove the point of what's doable. i really enjoyed the process. trying to finetune things, and setting up communication between things, trying (very badly) to simulate a scientific method. i want to work alongside scientists eventually in the future.
"Time, Space and Things" by Tom Ridley was a valuable book for the insights on how STEM people look at things and how, because of complexity, we can't really understand things fully, we need to apply filtering which normalizes reality into something digestible.
"Philosophy of Prediction and Capitalism" (1987) by Manfred S. Fring, the title is self explanatory.
this paper talks about the different signals emitted by certain plants.