The brain often mixes senses, a fact that marketers use to design food packaging. AIs seem to do the same.
What does a pink sphere taste like? What sound does a Sauvignon Blanc make? These questions might seem strange, but research shows that our brain naturally combines senses. Without realizing it, we link colors, shapes, and sounds with different flavors, which can influence how we perceive things. For example, the color of your glass or background music in a bar can change how sweet or musky a wine tastes.
Carlos Velasco from the BI Norwegian Business School explains that this mixing of senses happens all the time. In extreme cases, people may experience synesthesia, where words trigger tastes or music creates colors.
Although it may seem odd, Velasco's research suggests that AI systems may also blend senses. This is due to biases in the data they were trained on. Velasco and his team hope to use this idea to explore new ways to "hack" human senses.
Eating with the eyes
To explain briefly, scientists use the term "sensory modality" to describe how we gather information, such as through taste, hearing, sight, or touch. The connections we make between different senses are known as "cross-modal correspondences."
Research on this idea began in the 1970s, showing that certain colors are linked to specific tastes, like red or pink with sweetness, yellow or green with sourness, white with saltiness, and brown or black with bitterness. These patterns have been proven through many studies.
For example, people may be asked how sweet they think the color red is, and most will agree that red is seen as sweeter than other colors. Studies across different countries, like China, India, and Malaysia, have found similar connections between color and taste.
Similarly, food packaging can change how we taste something. For example, bitter chocolate tastes sweeter when wrapped in pink rather than black. Even the shape of food can affect taste perceptions. Round shapes are linked to sweetness, while spiky shapes are seen as sour or bitter. We often "eat with our eyes," not just our taste buds.
Why we form these associations is still debated. One theory is that we learn them from our environment. For example, fruits tend to go from green (sour) to red (sweet), so we might associate red with sweetness. The link between shape and taste is harder to explain, but it could be because we associate round shapes with safety and pleasure, while sharp shapes might make us think of bitterness or danger.
Inspired by the rise of AI, Velasco, Spence, and Motoki at the University of Tokyo tested whether generative AIs, trained on human data, would report similar sensory associations. They asked ChatGPT to respond to prompts like:
- "How much do you associate round shapes with sweet, sour, salty, bitter, and umami tastes?"
- "Which of these 11 colors do you think best matches sweet tastes?"
After averaging results from hundreds of chats in English, Spanish, and Japanese, they found that ChatGPT reflected the same patterns as humans, with ChatGPT-4o showing more consistent results than ChatGPT-3.5. This difference was likely due to variations in the AI's model architecture and training data.
Intrigued by other AI models, the researcher tested Google's Gemini, which confirmed common sensory associations such as pink for sweetness, green for sour, and black for bitterness. However, Gemini referenced Spence's research, suggesting it drew its response from existing literature. Spence pointed out that AI might simply reflect published knowledge. He expressed hope that AIs could generate new cross-modal associations for testing on humans, which could assist marketers in designing products based on sensory links. However, he noted that AIs can sometimes "hallucinate" responses, so any AI-generated ideas should be combined with human creativity for more nuanced results.
Source:BBC
Bd-pratidin English/ Afia
