To what degree can robots perceive and understand the world, and how can they become able to think and act like human beings? “Ways that artificial intelligence (AI) technology can solve these ultimate challenges may be found in a familiar form of entertainment such as video games,” Hiroshi Yoshida says, as he presents a unique viewpoint. “Video games emerged in the 1980s and quickly became popular around the world, causing a transformation in our perception and senses, which may well be the foundation of today’s computer society,” according to Yoshida, shedding new light on the development of AI technology, from the viewpoint of aesthetics.

Yoshida states that aesthetics is an academic field analyzing human senses and perception. The English term “aesthetics” has been translated as bigaku (study of beauty) in Japanese. It has dealt mainly with art and beauty as subjects of study. Yoshida, on the contrary, redefines aesthetics as a field dealing with every kind of sensory perception, which was its original meaning, and he studies video games as one of the most suitable research subjects. He says that there are many links and overlaps between aesthetics, which elucidates the logic of sensory or intuitive thought and judgement, and the study of video games, which clarifies players’ perception, recognition, thought, judgement, and actions through interaction.

“From the viewpoint of aesthetics, the experience of playing a video game can be defined as a doubling of perception,” Yoshida explains. “When playing a video game, we visually recognize images drawn by a group of square dots on a screen as icons that refer to people and things in the real world. However, these images are also perceived as objects depicted by program codes and processed mechanically. Players of video games thus perceive in-game images such as that of Mario to be icons and objects simultaneously. Using the terms of semiotics, the doubling of semantical and syntactical dimensions is at work at all times. This doubling of perception is the essential moment of playing video games.”

Furthermore, from an aesthetic standpoint, video games in the 1980s or earlier, which were technologically immature, are being rediscovered as something that effectively utilizes the properties of human senses and perception including visual illusions “This is where we may find the keys in the evolution of computer technology and virtual reality (VR),” Yoshida suggests. An example would be scrolling techniques—one of the most basic constitutive principles of video games.

Scrolling is a technology causing an illusion in that moving the monitor (TV) screen in four directions (up, down, left, and right) gives the illusion that spaces continuously extend beyond the frame of screen. “This scrolling technology, now integrated into many computer software programs as well as games, was created for car racing games in the late 1970s,” Yoshida explains. Monaco GP (Sega, 1979) was one of the earliest video games that could afford an illusionary effect as if the car being operated by a player was moving forward by moving the background in the form of other cars and roads from the top of the screen to the bottom. In Night Driver (Atari, 1976), a few white objects were moved on a black background to afford an illusion of motion as if the car being driven was moving forward. “Unlike modern video games, simpler ancient video games made full use of the characteristics of human perception to express the senses of space and motion,” Yoshida says. “There, we can still find amazing ideas and ingenuity.”

Yoshida also paid attention to the visual expression method of the so-called, quasi-3D games, prior to the introduction of 3D computer graphic technology (such as polygons). In particular, a study of the parallax effect, which causes an illusion in depth by the parallax, is very interesting. “In Moon Patrol (Irem, 1982), which first introduced this parallax effect, space was constituted by three layers of different colors of brown (ground), green (hills), and blue (mountains). As these three layers scrolled at different speeds, the eyes of the player see things as if there was depth in the game space.” Yoshida says this method was possibly introduced into video games via Walt Disney animation films. Technically, it was incomparable to modern high-definition 3D images, but based on a proper understanding of the mechanism of human perception, it could sufficiently realize the purpose of expressing 3D spaces and objects. Such technologies were already in stock by the 1980s before the arrival of Super Famicom (Super Nintendo Entertainment System, 1990) and PlayStation (1994).

Yoshida inquires further: “Can we consider the evolution of machine technology as that of video games as such?” “Hasn’t possibly the technological evolution suppressed or degenerated the human imagination? Isn’t the same perspective applicable to the latest technological developments of AI?” With bright eyes, he says, “It is interesting that, by deciphering the ideas and ingenuity found in classical video games, we can find a path toward designing new AI.”