Pierre - Shaping vectors

A vector is a mathematical entity which consists in a series of numbers grouped together to represent another entity. Often, vectors are associated with spatial operations: the entities they represent can be either a point, or a direction. In computer science, vectors are used to represent entities known as features, measurable properties of an object (for instance, a human can be said to have features such as age, height, skin pigmentation, credit score and political leaning). Today, such representations are at the core of contemporary machine learning models, allowing a new kind of translation between the world and the computer.

This essay sketches out some of the implications of using vectors as a way to represent non-computational entities in computational terms. As such, we extend the work of Jack Goody on the list, and of Bruno Latour on the perspective drawing, and suggesting epistemological consequences in choosing a particular syntactic system over another. While binary encoding allows a translation between physical phenomena and concepts, between electricity and numbers, and while Boolean logic facilitates the implementation of symbolic logic in a formal and mechanical way, vectors open up a new perspective on at least two levels: their relativity in storing (encoding) content and their locality in retrieving (decoding) content.

In machine learning, a vector represents the current values of a given object, such that a human would have a value of 0 for the property "melting point", while water would have a value of non-0 for the property "melting point". Conversely, water would have a value of 0 for the property "gender", while a human would have a non-0 value for that same property. However, this implies that each feature in this space is aware of all the other dimensions of the space: a human could potentially have a non-0 value for the property "melting point". Vectors are thus always containing the potential features of the whole space in which they exist, and are more or less relatively tightly defined in terms of each other.

As we retrieve information stored in vectors, we therefore navigate semantic spaces. However, such a retrieval of information is only useful if it is meaningful to us; and in order to be meaningful, it navigates across vectors that are in close proximity to each other, focusing on re-configurable, (hyper-)local coherence to suggest meaningful structuring of content. The proximity, or distance, of vectors to each other is therefore essential to how we can use them to make sense. The meaning is therefore no longer created through logical combinations, but by spatial proximity in a specific semantic space.

Given those the existence of features in relation to one another, and the construction of meaning through the local similarity of feature vectors, we can see how semantic space is both malleable in storing meaning, and structural in retrieving meaning. A next interesting inquiry is to think about the process through which this semantic space is being shaped, through the process of training.