Hypermedia, Memory, and Cognition: The Challenges of Information Retention in the Digital Age

Introduction 

A multitude of human endeavors have been benefited by the use of hypermedia. With hypermedia, we have the ability to explore, learn, amuse, and connect, but it is important that we are not so dazzled by its vast capabilities that we fail to question its growing role in our lives. 

By definition, hypermedia is a nonlinear medium of information that can include graphics, video, audio, plain text and hyperlinks. This now ubiquitous and interconnected form of media is relatively new, and as it continues to impact all areas of our everyday lives, the remarkable advantages and undeniable challenges that it presents are becoming more evident. While the Internet is an undoubtedly impressive and important resource in today’s world, its many applications can function much like a slot machine, constantly shifting and inundating our attention, often to our detriment. With people now spending an average of 3 hours and 15 minutes per day on their phones, it has become more urgent that we explore the effects that hypermedia is having on memory and cognition. How can we understand and enhance the benefits of hypermedia, while mitigating undesirable outcomes? 

How do memories form? 

The process of memory formation is an incredibly complex and vital function of the human brain. It involves consolidating data from previous experiences, and the ability to recall useful information at will. As Jeff Johnson explains, “memory formation consists of changes in neurons which make neural activity patterns easier to reactivate in the future. Some of these changes result from chemicals released near neural endings that boost or inhibit their sensitivity to stimulation. While these changes only last until the chemicals dissipate or are neutralized, more permanent changes occur when neurons grow and branch, forming new connections with others” (Designing with the Mind in Mind, p. 88-89). Any disruptions during this process will likely result in difficulty recalling information pertaining to the experience. Research suggests that the comprehension and retention of information involved in memory formation can be impeded by an overwhelmed visual working memory. The components of working memory are perceptions and retrieved memories that can include goals, numbers, words, names, sounds, images, and odors. (Designing with the Mind in Mind, p. 93). With the experience of reading online, the mental focus necessary to retain information is regularly interrupted by visual data that, in the case of promotional ads, is often unrelated to the material. 

#ReadingRainbow (Ads Included) 

The use of tablets, e-readers and online articles has increased as technology continues to improve, but over time research has continued to highlight the distinct advantages of reading on paper. Yes, these digital devices offer a mind-boggling range of content, and the ability to take dozens of books on the go is certainly invaluable, but these tools change our brains as we use them. What are we sacrificing on the altar of consumer choice? Results from polls and consumer reports indicate that the different haptics of a physical book allow readers to navigate long texts in a way that e-readers do not. These navigational difficulties may inhibit reading comprehension and therefore memory retention (Scientific American, 2013). There is inherent physicality involved in reading and when compared with paper, screens tend to drain more of our mental resources. Depending on the platform whether it be an e-reader or online article, pixilation, glare and certain glitches can also tire the eyes. (Computers in Human Behavior, pg. 377-394). This has resulted in the familiar complaints of eyestrain, headaches, blurred vision from people who read on screens and spend long hours on the computer. These symptoms are common enough that the American Optometric Association officially recognizes Computer Vision Syndrome (CVS) also known as Digital Eye Strain, which is characterized by the five major symptoms of eyestrain, headache, blurred vision, dry eyes, and back/neck pain. 

Given that the internet is oversaturated with advertising content, it becomes less and less likely that we will be able to read content online without these types of intrusions. In addition, the American Psychological Association published a study on people’s attitudes towards different types of media. They found that consciously or unconsciously, “people approach computers and tablets with a state of mind less conducive to learning than the one they bring to paper” (Ackerman, R., & Goldsmith, M. 2011). The same redirection of mental resources that explains why so many people find it difficult to remember what they have read on a screen, is also responsible for disrupting intrusive trauma memories, a core feature of PTSD. 

Games as a PTSD Intervention? 

Researchers are now validating what many trauma sufferers have discovered by chance. In some cases, intrusive memories can be mitigated by playing highly engaging visuospatial activities like Tetris, Candy Crush, and Word games (Alter, Adam. Irresistible, p. 304). Intrusive memories of a traumatic event involve mental imagery – based impressions that intrude into our psyche involuntarily and are emotionally disruptive (National Library of Medicine, National Institutes of Health, 2017). By focusing on visuospatial activities after experiencing trauma, a person can disrupt memory consolidation, and therefore impede the intrusive memories that are a core feature of PTSD. In a study conducted in 2017, study participants were shown an eleven-and-a-half-minute trauma film containing road traffic accidents, emergency service personnel working on injured patients and dead bodies being moved (National Library of Medicine, National Institutes of Health, 2017).

Background: Intrusive trauma memories are a key symptom of posttraumatic stress disorder (PTSD), so disrupting their recurrence is highly important. Intrusion development was hindered by visuospatial interventions administered up to 24 hours after analogue trauma. It is unknown whether interventions can be applied later, and whether modality or working-memory load are crucial factors.

Results: Reactivation + Tetris and Reactivation + Word games resulted in relatively fewer intrusions from the last day of diary A to the first day of diary B than reactivation-only (objective 1 and 2). Thus, both tasks were effective even when applied days after analogue trauma. Reactivation-only was not effective. Reactivation + Word games appeared to result in fewer intrusions than Reactivation + Tetris (objective 3; modality effect), but this evidence was weak. Explorative analyses showed that Word games were more difficult than Tetris. (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5678449/).

The study results seem to imply that the more mental fatigue the subjects experienced due to the Word games, the less recall they had of the analogue trauma. The same fracturing of attention that interrupts the formation of memories while someone is playing a game, is the same reason why we find it difficult to read on digital devices. In one case, this is a positive development that could allow sufferers of PTSD respite from intrusive memories. On the other hand, this research suggests that if we want to read deeply and retain information, we’d better trade in our screens for paperbacks.

Attention Overload 

We live in a time when screens in public spaces are likely to be tuned to the news. Whether it is MSNBC, CNN, Bloomberg News, or Fox News, the familiar drawl of “breaking news” updates feel omnipresent and hard to opt out of. Even when we are on the move, it’s not rare to get into a car share and find an iPad in a headrest playing a never-ending loop of CNN Breaking News alerts, and infinitely rolling chyrons. This is all the more ironic because research shows that due to the visually overloaded format of news broadcasts, we are less likely to absorb the material, and as a result we are retaining much less information.

As Nicholas Carr explains in The Shallows: What the Internet is Doing to Our Brains, “Kansas State University scholars conducted a study in which a group of college students watch a typical news broadcast with the anchor reporting four news stories while various infographics flashed on the screen while a textual news crawl ran at the bottom. They had a second group watch the same program but with the graphics and news crawl stripped out. Subsequent tests found that students who had watched the multimedia version remembered significantly fewer facts from the stories than those who had watched the simplified version. The researchers wrote, ‘It appears that this multimessage format exceeded viewers' attentional capacity (The Shallows, pg. 131).’ It seems that the more information they try to add, in the form of text crawl, graphs, and video, and audio, the harder our brain must work to keep up. Educational psychologist John Sweller notes, ‘auditory and visual working memory are separate to some extent, and effective working memory may be increased by using both processors rather than one…the negative effects of split attention might be ameliorated by using both auditory and visual modalities (The Shallows, pg. 131).”

What does this mean for user experience? 

The target audiences for books, games, and news are universal and non-specific. Yes, there are niches for these products and content is tailored to a desired audience, but most people play games, watch the news (intentionally or not), and read some type of media, whether it be books or online articles. For consumers looking for a more minimalist experience in media, most options range from downloading an online article and reading it in PDF form, to avoiding the news altogether. Games are optional, and the designers of addictive games like Candy Crush and Farmville seem to strike a perfect balance between completely immersive and relatively harmless (unless the game has in-app purchases, then you really see the slot machine analogy at work). Until web designers can find a way to strike a balance between obligations to advertisers and providing a genuinely enjoyable experience, we are likely to see increasingly overloaded screens. And this increasing interconnectedness that hypermedia facilitates between real-life experiences and the need for online advertisements to offset costs can become a hindrance if not executed properly.

A Better Approach: Product Enhancements for User Experience

How can we use this research to create a better product experience for users? E-readers provide a unique experience for consumers, but many of the haptics involved in physical books are all but impossible to recreate on a single screen. This product does not easily lend itself to redesign simply because the components of both objects are so distinct. My recommended product enhancements for both online articles and broadcast news are to provide a more integrated format that does not force your senses to compete against one another. Although this is counterintuitive to the capitalist hegemony, removing advertisements would go a long way to address the problem of attention overload. The scenario where companies stop using every single platform to sell us more services and products is unlikely. However, upgrading products to create a better experience for consumers is the most important goal. So, let’s get started!

Product upgrades for online articles: 

1. Set a thirty second timer for all screen ads, after which the ad content will disappear. 2. Replace the screens on current laptops, tablets and phones with the matte anti-glare coating as seen on Kindles and other E-readers.

3. Remove banner ads that allow text crawl on your web browser. 

4. Enable an app that will prevent videos embedded in the article from automatically playing when you open the page. 

5. Use E-ink that mimics chemical ink across laptops, tablets, and phones. 

Whether you are playing a game, reading an online article, or watching a standard news broadcast, your ability to understand, process, and store information may be decreased due to limits on human attentional capacity. I believe these small adjustments can enhance the benefits of technology and prevent more damaging conditions like Computer Vision Syndrome. Screens offer many things that paper cannot, and while you may want to read a dense text like Parable of the Sower in paperback, there are many visually engaging articles that provide a satisfying online reading experience. This research evolved from my interest in the cultural and intellectual consequences of the internet. In The Shallows, the author convincingly describes how the tools we use to find, store, understand and share information, change our neural pathways. He looks at the historical and scientific evidence of human’s use of the alphabet, maps, the printing press, the clock and computer to underscore the melding of media and mind. With Artificial Intelligence and Augmented Reality gaining commercial accessibility, the challenges I highlighted in my research will look quaint in comparison to the profound existential questions these new capabilities will bring. The onus is on us to continue to pave ethical pathways for the ever-evolving intersection of technology and the human experience.

References:

• Ackerman, R., & Goldsmith, M. Metacognitive Regulation of Text Learning: On Screen versus on Paper. Journal of Experimental Psychology: Applied, 2011. https://psycnet.apa.org/record/2011-05714-002 
• Alter, Adam. Irresistible: The Rise of Addictive Technology and the Business of Keeping Us Hooked. Penguin Press. Pg. 303-305, 2018.
• American Optometric Association. Computer Vision Syndrome Aka Digital Eye Strain. https://www.aoa.org/patients-and-public/caring-for-your-vision/protecting-yourvision/computer-vision-syndrome
• Carr, Nicholas. The Shallows: What the Internet is doing to our Brains. W.W. Norton Company. Pg. 131, 2010.
• Computers in Human Behavior. Volume 21, Issue 2. Effects of VDT and paper presentation on consumption and production of information: Psychological and physiological factors. Pg. 377-394, 2005. https://www.sciencedirect.com/science/article/pii/S0747563204000202
• Jabr, Ferris. "The Reading Brain in The Digital Age: The Science of Paper Versus Screens". Scientific American, 2020. https://www.scientificamerican.com/article/reading-paper-screens/. 
• Johnson, Jeff. Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Guideline. Morgan Kaufmann/Elsevier. Pg. 88-93, 2014.
• National Library of Medicine, National Institutes of Health. Tetris and Word games lead to fewer intrusive memories when applied several days after analogue trauma, 2017. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5678449/