Embodied code. Guest lecture. 

We had several guests’ lectures planned for this term. All of them were online and due to the time difference between our location, here in the UK, and our guests being all over the world, these meetings were usually held on Friday evenings, so that was the one I am going to write about. 

We met a team of people fascinated by the data and the code being presented in an unusual way, when compared to traditional scripting.  

Embodied code, presented by Robert Twomey and his team, was the subject of the lecture, with the theoretical part but also with a little demo and us testing out the presented solution. 

Bodily experience, embodied schema, embodied metaphors were a starting point of the lecture, but also for the concept. 

Evidence of embodied cognition in language processing – visualising the content determines the description of what we are asked to describe – science. 

Evidence for embodied cognition in brain function. Our body-based experience is a base for how we understand different representations and concepts. 

Embodied learning in Science and Maths – an example of learning physics through play in an augmented reality environment. 

Embodied learning in Computer Science for example variables modelled as plastic bags with sticky notes, abacus, a stack of dishes. 

Embodied Code – Tim (has a background in dance and computer science) introduced us to the concept of their project as well as general lenses of embodiment. In embodied design, phenomenology and somatic: embodiment is the participatory process of existing as a living body, awareness of the body, and awareness through the body, engagement with meaning, and agency in relationship to self and other” (Wood T. R., 2021). 

Embodied design – designing with and for the whole body, diverse and unique bodies with unique experiences. First-person experience is prioritised. Body driven vs. brain driven designs.

Somatics – a field that studies the soma or body as perceived from within (Eddy M. 2009). 

Somaesthetics – learning and playing means engaging with moves (whole body is involved, muscles, nerves, senses etc., we extend and improve our own experience of being in the world (Höök K, 2016). 

This leads to somaesthetic designs with the principles of subtle guidance, making space, intimate correspondence, articulating experiences.  

How do we harness embodied cognition for coding? How can we harness the above ideas into VR?  

Thinking about the code – typically lines of code, but more often the visual representation of the code (colour-coded blocks, stacks etc. – still on the screen) – visual programming language (Pure data, Bolt, BluePrint). Not text-based representation of the code: Logo (1967), Boxer (1996), Jupyter/IPython (2001). 

Developing other ways to code- the core of this project. New opportunities for code, 3d structures, physical metaphors, spatial representation of the code, using body directly – hand gestures.  

Code comments on the project. The human comments for the code. Drawing graphs, adding arrows. Traditionally, people must stretch how they think when they code to fit in the way computers think. The way to create the code thoroughly is the expression of their body – walking, gestures etc. Not to make the person go to the code but to let the person express themselves via their body and then translate it into the code.  

An example from their app: a gesture as a form of communication, gesture also as an input, and affordances of spatial gesture.  

Prototyping human-centred design and placing the creative process of designing and problem solving in the actual places where the solution is going to be delivered instead of using just an abstract plan. Taking that to the coding level, it means that we have the problem space, the design space, and the solutions space in the same place, working together.

Embodied code in the presented example means using body gestures as a data controlling tool, specifying location or structure. In our guests’ demo, we could see a sketching tool as well as visual and gestural code – adding commands within the virtual space, to the objects in that virtual space.  

The Embodied code we have seen during that session is a funded research project that aims to use VR and AR in STEM and coding to boost interest and confidence in coding. It was a form of Virtual Reality learning tutorial for teaching us (people) how to code and how the code works through visuals (and traditional coding commands connected to these visuals). Brilliant solution! 

It was a remarkably interesting meeting with an innovative approach to coding and the possibility of coding being more user-friendly and more accessible to people who are not confident with the code (such as me). What is great, however, was not that straightforward for me to use that app in the way our guests did. At the same time, we were told that the team is still prototyping the product and we were one of the very first groups to see it, which indeed was a privilege.