Nick Stedman
From Collectivate Course Wikis
BluePrint
Untitled (after Deep Blue):
Untitled (after Deep Blue) is a prototype for a robotic partner for people. Its physical form is abstract though complimentary to the body. It attempts to engage a viewer in a sexualized physical relationship. It embodies a sense of being which is exhibited through its behavior. It uses sensors to map its contact and guide its interaction. In the final version these maps will be displayed for the audience as images. Neural Nets will be used to allow it to learn from interactions over time, to develop customized response patterns.
Project
This project relates sexuality to games of logic, games such as Chess or Checkers, but specifically Go. In this ancient Chinese game there is only one type of action, placing a stone on the board. Each player takes a turn placing a stone attempting to surround their opponent. Complexity arises because the game is so balanced. You place a stone, I respond, you respond to that, and so forth...as the game builds so do the spheres on which the dancing battles take place. Soon not only are my individual pieces being challenged, but also a small or large group of them. The key is to be able to flow in and out from these various levels of engagement and respond with just another single piece that represents your game. It's an interesting parallel with expressive sexual relationships. You move, and I respond, and you respond, and through this reciprocation and attention the engagement grows in complexity into a potentially brilliant exchange that involves the body, psyche and spirit.
In 1997, World Grandmaster Gary Kasparov faced Deep Blue, the Chess Playing SuperComputer built by IBM specifically to defeat him. Kasparov lost in 6 games. This was said to be a landmark in the social history of technology. It was previously thought that Chess was too complex for a computer system to navigate by calculation. It required the abstract reasoning and artful manouvering of a human. In fact Deep Blue's advantage turned out to be the most unfinessed resource at its disposal, "Brute Force": sheer speed and computing power. It could calculate millions of moves a second, and select the best. There was a minor public upset at Gary's loss. What did it mean for society? If we can be outcalculated at this the most human of endeavors, is there any realm in which computers can not outperform us. And in fact what does it matter if they can?
I'm interested in these questions, not in an apocalyptic sense, but rather in how it reflects upon people. Computers are a human construction. As such they represent what we have come to understand about the world, and its various materials, including ourselves. We have constructed an object which can perform this most artful of tasks, so what have we learned about ourselves through this creation?
Well, I enjoy Chess a lot, and Go, but there is an essential element missing from Deep Blue's victory, and therefore a huge piece of information missing about us. It is that games of logic are primarilly played out in the mind, but we as people have a physical presence which defines how we are able to interact with and perceive the world. So we need to incorporate this body into our games, and into our technological self-reflection.
I believe that sex marries these various elements that can elucidate the meaning of being human: the cognitive, the physical and the energetic. So I'm going to work on a robot that has sex. It should be able to perceive the ebbs and flows of the relationship and respond. It should learn over time. And hopefully it would help bring its partner to higher states of ecstacy. Now this is the goal, but I don't expect that it can be reached by a computer (this was once said of chess mind you), but through the process at least we can learn something about ourselves.
Instructor's Comment
Perhaps of interest:
Film:
Myths of the Organism by D. Makavejev
also Sweet Movie by the same director
Art:
Installation
Sabrina Raaf
MyDoki Doki
Institute for Infinitely Small Things
Wheel balancing robot
Educational Robots
Articifial Body, Real Body
Assistive Technology Project:
CNN on Paro
Paro Homepage
Paro Video in Nursing Home
This project relates to yours. The question raised here is that of affect and ethics.
How would you describe the problem space that emerges when the elderely people in the video
develop an emotional relationship to this robot and perhaps even feel love for paro?
Suggested Reading:
Social Aesthetics
Relational Aesthetics
Robotics Blog
Bibliography
Wiener, N. (1963). God & Golem, Inc. A Comment on Certain Points Where Cybernetics Impinges on Religion. MIT press, Massachusetts.
G. Di Marzo Serugendo, A. Karageorgos, O. F. Rana, and F. Zambonelli,
Engineering Self-Organizing Systems: Nature-Inspired Approaches to
Software Engineering. Berlin, Germany: Springer-Verlag, 2004
Simons, G.L. (1986). Is Man a Robot? Wiley & Sons, Chichester.
Johnson, S. (2001). Emergence. Scribner, New York.
Suzuki, S (1970). Zen Mind, Beginner's Mind. Weatherhill, New York/Tokyo.
"Morphologies, Motion and Cognition workshop at Alife X", February 14, 2006. Morphodynamics and Cognition Group, University of Sussex <http://www.informatics.sussex.ac.uk/morphodynamicsgroup/alifexwk/
Cont, A., Coduys, T., and Henry, C. "Real-time Gesture Mapping in Pd Environment using Neural Networks". In the proceedings of the International Conference on New Interfaces for Musical Expression. Hamamatsu 2004.
<http://www.la-kitchen.fr/download/kitchen_hardware/2_references/Nime_Article_2004.pdf>
Nissen, S (2005). "Neural Networks Made Simple". Fast Artificial Neural Network Library. January 8, 2006. <http://fann.sourceforge.net/fann_en.pdf>
Instructor:
Perhaps of interest:
http://www.nyls.edu/pages/1684.asp
The Body and Media Theory by Bernadette Wegenstein
My links
Body Politics
Research
Week 1: Simulation
Reading:
Can a selectional system be simulated? The answer must be split into two parts. If I take a particular animal that is the result of evolutionary and developmental seection, so that I already know its structure and the principles governing its selective processes, I can simulate the animal's structure in a computer. But a system undergoing selection has two parts: the animal or organ, and the environment or world. No instructions come from events of the world to the system on which selection occurs. Moreover, events occurring in an environment or a world are unpredictable. How then do we simulate events and their effects on selection? One way is as follows:
1. Simulate the organ or the animal as described above,
making provision for the fact that, as a selective system,
it contains a generator of diversity - mutations, alterations
in neural wiring, or synaptic changes that are unprdictable.
2. Independently simulate a world or environment constrained
by known physical principles, but allow for the occurrence of
unpredictable events.
3. Let the simulated organ or animal interact with the simulated
world or the real world without prior information transfer,
so that selection can take place.
4. See what happens.
-Gerald Edelman
[Johnson, 194]
Response:
Edelman presents a paradigm for modeling an agent capable of adapting to its environment, and the unpredictable interactions encountered within. Any such system should contain a mechanism for unpredictable mutation based on external stimulus, and then be released into the world in order to drive its evolution. It reflects the proposed installation of 'after Deep Blue' which will situate an artificial being in an installation interacting with a person. In the project, it is not the structure of the object which will adapt, but rather the object's movements will adapt. It will learn from its interactions with multiple partners in order to derive more sophisticated behavior. The research then involves figuring out what such a mechanism for unpredictable mutation is. It also involves figuring out what influences drive the agent's evolution. As stated external stimulus will cause it to adapt, but secondly, it must have a mechanism for comparing and measuring which adaptation is more suitable to present conditions.
Week 2: Shunryu Suzuki
Reading:
Response:
Shunryu Suzuki explains that most people live a life of distraction and illusion. Instead, let's lead a life of clarity, giving up our goals and simply exist. This principle is one which I would like to instill in myself and so in an effort to understand the meaning of Shunryu's statements better I will attempt to model the artificial being of 'after Deep Blue' after my interpretation of his writing. In this sense, 'after Deep Blue' is a personal project.
As a starting point, the creature's modus operandi is to maintain balance. It will have an essential posture or essential activity which it will try to maintain, but when interposed upon by others it will adapt, incorporating the new information into its activity.
How can zazen be simulated? Shall the object explore it's environment as an aspect of zazen? Yes. This being is an investigator of itself which includes its environment.
Week 3: Morphology, motion, and cognition
Reading:
Morphology, motion, and cognition are the three intertwined elements that describe how a moving organism engages with its environment. Theories such as autopoiesis and dynamical descriptions of cognition stress the physical, embodied organism acting in the world. They highlight how embodied, situated organisms must interact with the world through their bodies, and how their morphologies are an integral part of their cognition and behaviour. We now use robots, both real and simulated, to explore these issues because, like organisms, robots are embodied, situated agents that are constructed from closely coupled parts with their own physical dynamics. We acknowledge that the physical attributes and properties of each limb, joint, bone, tendon, muscle, nerve, and neuron contribute to the modulation of both the organism's motion and its cognitive behaviour.
In robotics, as in cognitive science, new approaches stress the close interaction between neural systems, body morphodynamics - i.e. the morphologies and motion of the body components - and elements of the environment. [1]
Response: Here embodiment and physical experience are emphasized in the formation of cognition. I strongly agree with this idea. It seems more rudimentary than social construction which would affect the higher levels of operation. (Need to research further into this area) The construction of my body, my sensory perception and how I function pre-lingually shape my basic experience of the world.
Additionally, the group regards robotics as an area of research that can elucidate this area. Robots are artificial creatures which can are crafted in a specific shape, set in motion, acquire information through sensors, and can be made to react to that information. All of these traits are crafted by the maker, even if the outcome is unforseeable. The result is that through robotics we can learn something about being an embodied creature in this world according to the framework that Edelman lays out.
Week 4: Ronald Arkin
Reading:
The possibility of intelligent behavior is indicated by its manifestations in biological systems. It seems logical then that a suitable starting point for the study of behavior-based robotics should begin with an overview of biological behavior. First, animal behavior defines intelligence. Where intelligence begins and ends is an open-ended question, but we will concede in this text intelligence can reside in subhuman animals. Our working definition will be that intelligence endows a system (biological or otherwise) with the ability to improve its likelihood of survival within the real world and where appopriate to compete or cooperate successfully with other agents to do so.
Arkin, 31
Response:
While Arkin is interested in a similar area of investigation, using robotics to investigate intelligence, his language suggests a different approach. First, he posits some animals as being "subhuman". This is valid if we consider "subhuman" to refer to organisms from which humans are composed, but there is an ambiguity here, and the term is equally suggestive of inferiority. Secondly, Arkin roots survival as the basic premise from which intelligence is derived, yet does not provide reasoning or evidence. Thirdly, he also suggests that competition drives the development of intelligence. Arkin's perspective is Darwinian, and there is ample reason to regard intelligence as a vector of this school of thought, most important of which is the ease by which we can situate intelligence into a Darwinian framework. Intelligence can be thought of as an occurance specific to each individual, thereby relating to their niche in the evolutionary chain. But it is not the only way of conceiving intelligence. It can also be considered a phenomenon that occurs outside of any individual, and is rather a phenomenon that individuals experience with more or less clarity. For 'after Deep Blue' I'll be using the latter definition of intelligence.
Week 5 Maturana & Varela
Reading:
...our experience is moored to our structure in a binding way. We do not see the "space" of the world; we live our field of vision. We do not see the "colors" of the world; we live our chromatic space. Doubtless...we are experiencing a world. But when we examine more closely how we get to know this world, we invariably find that we cannot separate our history of actions - biological and social - from how this world appears to us. It is so obvious and close that it is very hard to see. [Maturana & Varela, p. 23]
Reflection is a process of knowing how we know. It is an act of turning back upon ourselves. It is the only chance we have to discover our blindness and to recognize that the certainties and knowledge of others are, respctively, as overwhelming and tenuous as our own. [Maturana & Varela, p. 24]
Response:
It is difficult to seperate 'experience' of the world from the world itself. All we know is what we experience, and what we experience is based primordially on our structure, so to each of us this world is inseperable from our particular embodiment. In the second statement, Maturana and Varela show the value of self-reflection which seems to contradict some of the Buddhist teachings I have been reading. They point out that it allows us to recognize our own experience as well as those of others as objects...perhaps this is the problem. In the book of Genesis, humans partook of the fruit of the tree of knowledge and this is what caused them to become self-aware, and to fall out of favour of God. Can we or should we attempt to return to these roots as it seems Shunryu Suzuki would have when he talks of the frog as mentor for being, or is this the path of humans that we should embrace. Following the Judeo-Christian logic, Jesus redeemed us of this original sin with his sacrifice. So where does this leave us? (It leaves us with this exact question to answer).
Week 6: Learning Machines
Reading: (What is a learning machine?) To begin with learning machines: an organized system may be said to be one which transforms a certain incoming message into an outgoing message, according to some principle of transformation. If the principle of transformation is subject to a certain criterion of merit of performance, and if the method of transformation is adjusted so as to tend to improve the performance of the system according to this criterion, the system is said to learn. A very simple type of system with an easily interpreted criterion of performance is a game. (Wiener 14)
Through the process of recreation and representation we learn about the subject we are representing, as well as the medium through which we represent. In creating a learning machine we learn about learning, about how we grow as beings. Because of their hermetic nature, and reduced set of variables, games offer a simplified arena in which to study learning through (technological) representation. Yet because of their hermetic nature, play does not bear upon the world beyond the game. There are increasingly examples where the boundary between the game and the world beyond are obfuscated. But here it is my intention to start with something like a game which is integrated in the world.
(The art of the game) "we do not play them by making the best possible move, on the assumption that an opponent will make the best possible move". (Wiener 15) The Von Neumann strategy for playing games posits that games can be won by calculating the outcomes of all possible moves and selecting the best one for any given situation. Wiener suggests that the Von Neumann strategy may be undermined by employing tactics that use the opponents rigidity against them, to cause them to make the best move at the moment, which is the wrong move over all. But this approach only works if the opponent is unable to calculate the entire scope of outcomes, or even a critical mass. Wiener suggests that a game which is wholly calculated loses its interest as a human endeavor. As a game, sex still contains too many variables that make it beyond the scope of calculation, perhaps incalculable. So the point then, is to learn something about sex, considering its beautiful rule set.
(How to build a machine that learns) "The new use of the regulating machine is to examine games already played and, in view of the outcome of these, to give a figure of merit; not to the plays already made, but to the weighting chosen for the evaluation of these plays". (Wiener 21)
(The life-like qualities of a learning machine) "In playing against such a machine, which absorbs part of its playing personality from its opponent, this playing personality will not be absolutely rigid. The opponent may find that strategems which have worked in the past, will fail to work in the future. The machine may develop an uncanny canniness". (Wiener 21)
In general, a game-playing machine may be used to secure the automatic performance of any function if the performance of this function is subject to a clear-cut, objective criterion of merit. (Wiener 25)
In sex, there is not a clear-cut, objective criterion of merit. The criteria is dependant on the players. If there is any objective to the game at all, it is for each player to achieve heightened states of being, or to reproduce. Oddly, in order to acheive these heightened states we must lose our attachment to them. We must relax, be open and adapt. I see most similarity between Go and sex because of Go's fluid nature, it's infinite outcomes, yet it's entirely simple premise.
Retrieved from "http://wiki.critical-netcultures.net/wiki/index.php/StedmanResponse6"
Week 7: God and Golem, Inc.
Reading:
God & Golem, Inc.
(ch.5 to ch.8)
A related problem requiring the joint consideration of mechanical and human elements is the operational problem of invention not only with regard to what we can invent but also as to how the invention can be used and will be used in a human context. The second part of the problem is often more difficult than the first and has a less closed methodology. Thus we are confronted with a problem of develoment which is essentially a learning problem, not purely in the mechanical system but in the mechanical system conjoined with society. This is definitely a case requiring a consideration of the problem of the best joint use of machine and man.
Weiner 81
Response:
hat is meaningful? On a basic level, anything is meaningful if it elicits consideration. But that same thing is much more meaningful if it discernably impacts upon a person's behavior. It is difficult to invent something that is meaningful. Sometimes, a few people accumulate a certain knowledge and enterprise that gives birth to a particularly meaningful contribution to humanity. It is not something which can be presupposed before the idea is born. My frustration with the art industry is that it is predicated on the deliberate efforts to create meaning, which is a contradiction. So let's point our scope away from these self-referential attempts, and instead towards areas that we know need our attention. Ecology is one such area. Ideally I would like to make something which works towards bettering the environment. This to me would be meaningful. If I could learn practical skills which when implemented help transform some aspect of the environment into a healthier state I would be more pleased than creating an abstract form of representation which takes as its premise being outside of people's lives. Electronics and robotics certainly have much to offer in this area, but regardless, it's more a matter of me applying the knowledge I have to the subject matter, and by reverse-pollinated by it so that I can adjust my practice.
Week 8: Engineering Self-Organizing Systems
Reading: Engineering Self-Organizing Systems
Week 9: Emergent behavior in systems
Reading:
Local turns out to be the key term in understanding the power of swarm logic. We see emergent behavior in systems like ant colonies when the individual agents in the system pay attention to their immediate neighbors rather than wait for orders from above. They think locally and act locally. Take the relationship between foraging and colony size. Harvestor ant colonies constantly adjust the number of ants actively foraging for food, based on a number of variables: overall colony size (and thus mouths needed to be fed); amount of food stored in the nest; amount of food available in the surrounding area; even the presence of other colonies in the near vicinity. No individual ant can assess any of these variables on her own. The perceptual world of an ant, in other words, is limited to the street level. There are no bird's-eye views of the colony, no ways to perceive the overall system - and indeed, no cognitive apparatus that could make sense of such a view (my italics). "Seeing the whole" is both a perceptual and conceptual impossibility for any member of the ant species. [Johnson 74]
If you're building a system designed to learn from the ground level, a system where macrointelligence and adaptability derive from local knowledge, there are five fundamental principles you need to follow... [Johnson 77]
More is different...
Ignorance is useful...
Encourage random encounters... [Johnson 78]
Look for patterns in the signs...
Pay attention to your neighbors... [Johnson 79]
Response:
Steven Johnson presents numerous examples to demonstrate that complex behavior including intelligence arises from the interaction of a multitude of small "dumb" agents following simple rules of interaction, in this quote we are presented with the case of ant colonies.
Week 10
Reading:
Response:
Week 11: The human mind
Reading:
The human mind is poorly equipped to deal with problems that need to be solved serially - one calculation after another - given that neurons require a "reset time" of about five milliseconds, meaning that neurons are capable of only two hundred calculations per second. (A modern OC can do millions of calculations per second, which is why we let them do the heavy lifting for anything that requires math skills.) But unlike most computers, the brain is a massively parallel system, with 100 billion neurons all working away at the same time. That parallelism allows the brain to perform amazing feats of pattern recognition, geats that continue to confound digital computers - such as remembering faces or creating metaphors. [Johnson, 127]
Response:
Week 12: Rule-governed systems
Reading:
No one wants to play with a toy that's going to be fun after a few decades of tinkering - toys have to be engaging now, or kids will find other toys. And one of the things that make all games so engaging to us is that they have rules. In traditional games like Monopoly or go or chess, the fun of the game - the play - is what happens when you explore the space of possibilities defined by the rules. Without rules, you have something closer to pure improv theater, where anything can happen at any time. Tules give games their structure, and without that structure , there's no game: every move is a checkmate, and every toss of the dice lands you on Park Place.
...Emergent systems too are rule-governed systems: their capacity for learning and growth and experimentation derives from their adherence to low-level rules...If any of these systems - or, to put it more precisely, the agents that make up these systems - suddenly started following their own rules, or doing away with rules altogether, the system would sop working: there'd be no global intelligence, just a teeming anarchy of isolated agents, a swarm without logic. Emergent behaviors, like games, are all about living within the boundaries defined by rules, but also using that space to create something greater than the sum of its parts. [Johnson, 181]
Response: There is an inverse relationship here between the rules that give games their structure and an individual's embodied experience of the world as discussed in Tree of Knowledge. The construction of my sensors, nervous system, body, and the rules that I obey or disregard, the skills I learn affect the choices I make, equally compose the world, as the elements of the world itself. Likewise, if we invert the idea of games such that the game is the actor and the players are the field, we see that the games rules act as a nervous system which define how it engages the field, and the interactions it has. Even if this comparison over-reaches there is a connection between games and a beings engagement which can be useful in researching artificial life.
Week 13: If we could create intelligent robots, what should they be like?
Reading:
If we could create intelligent robots, what should they be like, and what should they be able to do? Answering the first part of this question - "What should they be like" - requires a description of both the robot's physical strucutre (appearance) and its performance (behavior). However, the second part of the question - "What should they be able to do?" - frames the answer for the first part. Robots that need to move objects must be able to grasp them; robots that have to traverse rugged outdoor terrain need locomotion systems capable of moving in adverse conditions. A guiding principle in robotic design, whether structural or behavioral, involves understanding the environment within which the robot operates and the task(s) it is required to undertake. This (is an) ecological approach, in which the robot's goals and surroundings heavily influence its design...
[Arkin, 1]
Response:
In creating a system which is made to participate in the world, the engineering of the system itself should reflect the principles which guide the system's choices. The guiding principle being 'balance' I must be conscientious of the impact of the materials used within the environment the system occupies, in this case that stated environment is the physical world. In other words I will attempt to make ecogological choices in the construction of the object.
Week 14: Memory
Reading:
Time-series prediction can also be used to introduce memory in controllers for robots etc. This could e.g. be done by giving the direction and speed from the last two time steps as input to the third time step, in addition to other inputs from sensors or cameras. The major problem of this approach is, however, that training data can be very difficult to produce since each training pattern must also include history.
Response
I would like to use self-organizing algorithms to generate the movements of the robot. But how can the robot figure out how to pattern itself to best propogate itself along or move so that it accomplishes its task. Neural Nets requiring training, but I'm not sure if I can provide the training it needs. This passage points to using previous movements to train the machine for the next one. In an informal discussion, David Rokeby similarly discussed his desire to figure out trajectories based on prior information. Still, as the text suggests there would need to be additional sensory input in order to let the machine know if and when it has continued as far as need be in a particular vein. Alternatively, I would like to try to use no training, but rather have a sensor (such as an accelerometer) which provides feedback to robot, and ask the robot to try to elicit a certain measurement from the sensor. It would try random things, and 'climb hills' that seem to bring it closer to the goal. But does this programming structure contradict the kind of overall balance I want to acheive?
Week 15: Neural network
Reading:
A neural network is made up of layers, each of which contains several neurons. With the above model, problems arise when constructing more complicated networks that demand a large number of neurons. To solve this sophistication, we consider each layer as a matrix operation as shown in Figure 5. In this approach, the number of neurons used depends on the size of the input vector and weight matrix and all operations including the transfer function would be on matrices instead of numbers. Figure 5. Matrix representation of a layer in a neural network Figure 7 shows a Pd realization of a layer in neural network. In this manner, each layer, despite the number of neurons will be represented by one abstraction and reduces the complexity of network representation. The left inlet accepts the input arranged as a matrix vector and other inlets would be the trained network parameters for the layer loaded once into the patch using the values obtained from training. These parameters are: layer’s weight matrix (equivalent of w in Figure 5), layer’s bias vector (equivalent of b in Figure 5) and the transfer function. The transfer function is selected by sending a symbol to the inlet which can be either a log-sigmoid or tan-sigmoid function. So far, we have not encountered any needs for other transfer functions and therefore these two are implemented in the layer abstraction. Response
Realization
April 23, 2006
For this class, I focussed on enhancing an existing machine <http://nickstedman.banff.org/tribot.html> by adding sensor feedback and software for self-organization. This is a stepping-stone towards building Untitled (after Deep Blue). I successfully tested two types of sensing. One was to measure the current draw from an individual servo and to have it adjust its behavior according to the amount of torque exerted upon its rotor. It is able to respond to direction of the torque by using a trial and error adjustment scheme, where it randomly turns in one direction to reduce the torque, and if the adjustment fails to reduce torque then it reverses direction. This scheme has proved successful in limited tests. The second variety of sensing is touch sensing using capitance sensors. These sensors output a electrical field that when disturbed by a physical object triggers an output change in the sensor which can be used to control other events. These sensors were successfully tested, and look as though they would be useful. Because they put out a field as opposed to requring direct touch they are more forgiving of structural variations, which is helpful since I am unable to produce something of such precision.
In addition to the sensors I looked at self-organization as an approach to creating efficient locomotion in the robot(s). So far I have only researched the principle of self-organization, and its occurrences in various systems, such as ant-colonies, urban centers, biology and computing. I have also started looking at
- This is a schematic for current sensing of a servo.
- This is the PIC code written in C for sensing the directionality and measuring the torque.
- This is the datasheet of the capacitance sensor. It shows the circuit which I tested.
- Here is a Puredata patch that I made to try to flush out how a Neural Net worked. (Interesting experiment, but inaccurate, could build on it somehow, useful perhaps for biasing successful).
- Here are some images of the robot as it was exhibited at the gallery without implementing any of the aforementioned technical research.
By the end of the class, I did not directly implement these sensors or the software enhancements in the exhibited machine. But the research seems to be on the right track for developing after Deep Blue, as well as the skill set I desire.
Future Resources
Games of No Chance
Winning Ways for Your Mathematical Plays
Munster, Materializing New Media
Maturana, Tree of Knowledge
Deleuze+Guattari, 1000 Plateaus
Future Topics
Willhelm Reich
Autopoeisis
Future Technologies
accelerometers