On Mixed Reality and Speculative Fiction Forecasting

Exploring Emerging Tech, Speculative Fiction as a Foresight Method and My Personal Take On It To Bring It All Together

Emerging Technology: 

In a world where technological innovation is becoming increasingly prevalent in functions of society, Mixed Reality(MR) is a major force behind this convergence. Under the umbrella term of Extended Reality(XR), MR is an emerging technology that acts as a bridge between the virtual and real worlds to ‘create hybrid environments where physical and digital objects coexist and interact in real-time’(Tamer, 2023)(El-Arifi, 2022). MR is comprised of both Augmented Reality(AR) and Virtual Reality(VR) components in the form of holographic and immersive devices. Such devices generate semi-immersive user experiences with ‘interactive virtual holograms’(Mattoo, 2022) e.g. Microsoft HoloLens, which provides a ‘fully self-contained, holographic computer’(Tamer, 2023) that allows designers and engineers in the product manufacturing industry to create and manipulate 3D digital prototypes in MR by ‘literally pulling and stretching their designs into the desired form’(Tamer, 2023). This not only simplifies the process, but also gives designers a level of freedom and flexibility through real-time input with remote collaborators and ability to intuitively finetune creations from different angles. Six degrees of freedom (6DoF) are another feature of immersive technologies that allow users to see and move their body-parts in any direction without disrupting the immersion, while at the same time detecting physical objects (and recognising them) via spatial mapping and creating computer-generated replicas for a more realistic feel(Mattoo, 2022). Furthermore, through built-in cameras and motion controllers, ambisonics/virtual lights are added for supplemental realism. More mainstream uses of MR can also be seen in the fashion retail industry, such as Zara implementing virtual fitting rooms- thus ‘reducing the need for physical fittings and providing a convenient, personalized shopping experience’(Baba, 2023). 

Foresight Method:

As elucidated by (Hauptman and Steinmüller, 2018), “foresight needs neither puredaydreaming, nor incoherent fantasies, but – paradoxically – stringent, methodologically controlled kinds of imagination.”(Hauptman and Steinmüller, 2018). The distinctions and similarities between science fiction and foresight are examined regarding such a background. Futures studies frequently lack imagination, but science fiction writers often revel in it- so choosing speculative fiction as my foresight method in conjunction to mixed reality and future affects was most interesting to me. Due to its ontological potential, the speculative fiction genre indicates the possibility to shape and influence future events by depicting contingency, which refers to the display of a potential plausible future- albeit without certainty(Erol, 2018).

Regarding genre definitions, it encompasses any fiction with supernatural, fantastical, or futuristic elements- including dystopian and utopian narratives. Its legitimacy varies according to choice of texts, however its use as a tool for critical pedagogy can prove powerful, as ‘it constructs models of reality and can serve as a critique and response to oppression’(Thorne, 2021). Defining speculative fiction is a difficult task as its constantly expanding term based on societal demands and experiences- lived through writers. Marek Oziewicz, in his 2017 article for the Oxford Research Encyclopedia of literature- to me, describes it best by stating it as a “fuzzy super-set category that houses all non-mimetic-genres—genres that in one way or another depart from imitating conscious reality”(Thorne, 2021). 

A classic example of speculative fiction- from a dystopian narrative- that reflects on our world could be George Orwell’s ‘1985’ and Margaret Attwood’s ‘The Handmaids Tale’ which are adaptations set in the not-so-distant-future, and target topics from the likes of surveillance, lack of human rights/free will, and perpetual/aftermath of war- from an essentially horror perspective. Modern readers will notice that much of Orwell’s ‘future’ predictions have already come true, as “In 1984 it is a TV screen that watches you – today social media is an omniscient presence”(Seaton, 2018). As we move further into technological processes such as web3, users are noticing the amalgamation of physical and virtual reality- and with it the not-so-gradual increase of surveillance and lax of online privacy. 

Despite the broadness of its scope, speculative fiction can be very useful for present researchers and contingency planners as demonstrated by the University of Queensland for ‘The Ursula Project’(University of Queensland, 2022). The project was launched in 2022 and serves as a collaboration between the University and the Australian Department of Defence, Science, and Technology. The objective is to convert speculative fiction storytelling into workshop material that will help Defence personnel prepare for the disruptive consequences of emerging technologies. Setting, character, plot, and style are transferrable talents that are critical to technology foresight and are the focus of this project. It will address the ‘requirement to ascertain how technologies and tactics may converge in future military and security situations by enabling internal capacity inside Defence’ to conceive future technological scenarios for themselves(University of Queensland, 2022). The study will serve as the foundation for a brand-new methodology that envisions the ways in which science fiction and fantasy literature might assist groups in resolving challenging issues. 

The goal of this initiative is to incorporate creative arts—especially speculative fiction—into studies of global concerns.

Future Scenario:

Case Study: The Stolen Essence

Case Number: 2075-063

Date: September 12, 2075

Location: City of NeoLondis

Detective: Detective Smith

Incident Description

On September 12, 2075, an artist named Dimona Matthews was involved in a scandal that shook the peaceful city of NeoLondis. Dimona had been using mixed reality technology to spread propaganda, claiming that artificial intelligence was stealing the essence of creativity. The scandal sent shockwaves through the city, stirring up debates about the role of AI in the creative process.

Investigation Details

Upon receiving reports of the scandal, the Digital Crimes Division launched an investigation into the activities of Dimona Matthews. Preliminary findings revealed that Dimona had been utilizing advanced MR tools to create immersive experiences that portrayed a bleak future where AI-controlled systems dominated all creative endeavors. Through these experiences, she aimed to provoke a sense of fear and loss among citizens, making them question the authenticity of their own creativity.

Based on the investigation, Dimona Matthews utilized a variety of MR tools to create immersive experiences and spread propaganda. These tools included:

1. Holographic Displays: Dimona used holographic displays to project her artwork and propaganda in three-dimensional forms, creating an immersive visual experience for the audience.

2. Neural Network Interface (NNI): The Neural Network Interface (NNI) is a groundbreaking technology that has transformed the way humans interact with machines. It allows direct communication between the human brain and interconnected systems, enabling seamless integration with AR, VR, and smart devices. As a form of MR tech, its capabilities accredit users to not only transfer information from devices, but also transmit the visual into real-world environments in the form of holograms. By hacking the interface (suspicioned accomplice), Dimona overlayed virtual elements onto the real world, allowing her to blur the boundaries between reality and fiction. By being connected to NNI, viewers could see her dystopian scenarios where AI-controlled systems dominated creative endeavors.

3. Motion Tracking Technology: Dimona incorporated motion tracking technology to enable interactive experiences. This allowed viewers to physically engage with her installations and feel a greater sense of immersion and presence within the virtual world she created. Installations were triggered by passerby in particular areas such as NeoFalgar Square and Londis Orb. 

Through the combination of these tools, Dimona Matthews was able to create engaging experiences that conveyed her message about AI stealing the essence of creativity. These immersive installations served as a medium for her propaganda, leaving a lasting impact on the viewers and sparking debates about the role of AI in the creative process.

Through interviews with witnesses and analysis of Dimona's digital footprint, it was discovered that she had gained a significant following among disillusioned artists and intellectuals who felt threatened by AI-generated creativity. In early 2075, her works caused a crash in the digital economy, as decentralised blockchain due to rapid fluctuations in art value caused high market volatility. Dimona's virtual exhibits, blending reality and fiction, showcased artwork and stories that depicted AI as a ruthless thief, stealing the very essence of human creativity. Her provocative and thought-provoking installations resonated with many, igniting a movement against AI's potential dominance in the realm of artistic expression. The technocratic council of NeoLondis has made reprimanding allocations to CCU resources.

Evidence Collected

1. MRArt installations: Several immersive exhibits were found, displaying dystopian scenarios where AI was depicted as a malevolent force draining the creativity of humanity. These installations were meticulously designed to immerse visitors in a world where AI-controlled systems dictated every aspect of artistic production, leaving human artists devoid of their creative essence.

2. Communication records: Chat logs and encrypted messages suggested that Dimona had connections with an underground community of anti-AI activists and artists. These records indicated discussions about strategies to amplify her message and disseminate her propaganda to a wider audience, highlighting the existence of a network of like-minded individuals supporting her cause.

Suspects

1. Dimona Matthews: The primary suspect, an artist known for her controversial MR creations. Dimona's motive seems to stem from her belief that AI poses a threat to human creativity and the uniqueness of artistic expression.

2. Unknown accomplices: Investigation is ongoing to identify potential collaborators or individuals assisting Dimona in disseminating the propaganda. 

Official Suspect Statement: 

“What is the point of being human if we cannot create the world around us? We have collectively forgotten the feeling of self-expression, and the love that emanates through that. We have resorted to electing officials who rule from their shining ethrones and look down upon us with cold, calculated precision. I raised this topic in March when the council repurposed 3.6Million Ecrowns to fund the release of the NNI from the Creative Cultural Union, deeming CCU unnecessary. The NNI, while a remarkable advancement, raises critical questions about the preservation of human creativity and our authentic selves. I was outraged that in such a ‘free-thinking’ community, I would have to be subjected to questioning my own value as a being. What happened next is the same as now- a revolution to take back our own creative essence! I decided to fight back with their own fancy tech, showing the world our future should we continue down this path. We deserve to be seen! We deserve to be heard!”

Conclusion

The investigation into the scandal involving Dimona and her use of mixed reality to spread propaganda is still ongoing. The DCD is working closely with AI experts and legal authorities to determine the full extent of the impact of Dimona's actions. The case has sparked a larger debate within NeoLondis about the future of creativity and the role of AI in artistic endeavors.

Given the sensitive nature of the case and the potential consequences for the public perception of AI, it is imperative to bring this investigation to a swift and fair conclusion. The DCD remains committed to upholding the law and ensuring that the boundaries between art, technology, and ethics are respected in the ever-evolving digital landscape of NeoLondis.

Reflection:

Speculative fiction is a foresight method that explores future realities on fantastical and supernatural premises, but proves to be a valuable resource for contingency planning in important societal sectors- such as The Ursula Project. Its useful because it can help understand the digital future by creating stories and narratives that explore possibilities and implications of emerging technologies like Mixed Reality. However, although it allows for generation of new ideas it can be difficult to use due to its subjective storytelling nature, as well as the evident complexity in predicting the future. 

I learned that thinking critically with the aid of technology allows for a more immersive experience that could be more thought-provoking. I found it really interesting how big companies like DHL use MR to teach trainees while also being consulted by advisors through technological means. The way that its used in Art and Design is another compelling approach due to its flexibility in 3D holographic spheres, because it urges users to explore pieces in a more efficient and engaging manner. The idea that one can tweak designs with physical hand movements instead of working on a physical screen or paper makes for an easier way to imagine their work in tangible environments- while blending the digital and physical calls for an exciting prospect as a new art medium. 

By combining speculative fiction and MR, I found great pleasure as a writer/artist to develop my take on advanced futuristic tech and coining it as ‘Neural Network Interface (NNI)’. While the name may be a little on-the-nose as neural networks work as computational models inspired by the human brain (down to their interconnected ‘neurons’ that process data to recognise patterns in datasets), I thought the story would be strong when such groundbreaking technology would be used in an assumed ‘terrorist’ attack for a cause posing existential questions. Since the story is a police report case from NeoLondis, I had to make the wording glamorize the city as Utopian, while the character not only questions the authority of the technocratic council- but outright recognises the wrongness of valuing science over artistic creativity. So, rather than settling on a narrative stamped in bold as utopian/dystopian, I introduced views of both sides while posing questions of morality and individualistic authenticity, ultimately showcasing to the reader a world with plausible technology thats far from black-and-white. 

Reference list

Baba, Y. (2023). 15 Examples of the Use of Mixed Reality in Retail. [online] capsulesight.com. Available at: https://capsulesight.com/mixedreality/15-examples-of-the-use-of-mixed-reality-in-retail/ [Accessed 10 Nov. 2023].

El-Arifi, N. (2022). Disruptive Technologies. [online] Wunderman Thompson. Available at: https://www.wundermanthompson.com/insight/disruptive-technologies [Accessed 9 Nov. 2023].

Erol, A. (2018). Future Foresight through Speculative Visual Fiction: Technology and Individual Freedom via ‘Nosedive’. In: Believe in Technology: Mediatization of the Future and the Future of Mediatization. [online] pp.1–12. Available at: https://www.researchgate.net/publication/329059939_Future_Foresight_through_Speculative_Visual_Fiction_Technology_and_Individual_Freedom_via_'Nosedive' [Accessed 11 Nov. 2023].

Hauptman, A. and Steinmüller, K. (2018). Surprising Scenarios. Imagination as a Dimension of Foresight. Envisioning Uncertain Futures, (978-3-658-25073-7), pp.49–68. doi:https://doi.org/10.1007/978-3-658-25074-4_4.

Mattoo, S. (2022). What Is Mixed Reality? The Better Side of Technology. [online] learn.g2.com. Available at: https://learn.g2.com/mixed-reality#:~:text=Interaction%20Design%20Foundation- [Accessed 9 Nov. 2023].

Pearson, W.G. (2022). Speculative Fiction and Queer Theory. Oxford Research Encyclopedia of Communication. [online] doi:https://doi.org/10.1093/acrefore/9780190228613.013.1214.

Seaton, J. (2018). Why Orwell’s 1984 could be about now. [online] BBC. Available at: https://www.bbc.com/culture/article/20180507-why-orwells-1984-could-be-about-now.

Simard, Y. (2023). Strategic Value Driving Mainstream Adoption of Mixed Reality. [online] TechNative. Available at: https://technative.io/strategic-value-driving-mainstream-adoption-of-mixed-reality/ [Accessed 14 Nov. 2023].

Tamer, T. (2023). Real Examples and Use Cases of Mixed Reality in Design. [online] capsulesight.com. Available at: https://capsulesight.com/mixedreality/real-examples-and-use-cases-of-mixed-reality-in-design/ [Accessed 9 Nov. 2023].

Thorne, S. (2021). Through Critique and Be ough Critique and Beyond: Speculativ ond: Speculative Fiction as a T e Fiction as a Tool of Critical Pedagogy. [Thesis/Dissertation] pp.1–54. Available at: https://repository.usfca.edu/cgi/viewcontent.cgi?article=2582&context=capstone [Accessed 11 Nov. 2023].

University of Queensland (2022). WhatIF Lab: The Ursula Project - Speculative Fiction Storytelling for Technology Foresight. [online] hass.uq.edu.au. Available at: https://hass.uq.edu.au/ursula-project-speculative-fiction-storytelling-technology-foresight [Accessed 11 Nov. 2023].