Assignment 4 Report - Port Arthur AR Application

Port Arthur AR Application » Devlog
1 year ago by megans
Share this post:

Port Arthur

Corey Barr, Ruoxuan Liu, Henry Pockett, Megan Supplitt – KIT208 


Overview

Elevator pitch:

This application will allow visitors of Port Arthur to see the site as it once was and learn in more depth about the people who were there during its use as a convict settlement. It will leverage AR’s strengths to enhance the interactions already present at Port Arthur.  

  


Recording of early build: 

[Note: the final application will have proper voice acting instead of text to speech]



Application Description

The goal of this application is to teach users about Port Arthur’s rich history as well as improving the interactions visitors currently have with the site by using technology. As of 2021, Port Arthur staff give visitors a playing card that represents a convict, where the visitor can then discover a room full of drawers and find one with the matching playing card. This drawer contains information about the visitor’s convict. This application aims to use technology to extend this interaction by allowing users to bring their card to different landmarks and learn about their convict’s story relating to each site. The goal of this is to develop a deeper connection between the user and Port Arthur’s history and provide a memorable experience that sticks with the user. This application would further extend this concept by including different people relevant to Port Arthur outside of just convicts, such as workers and family members of convicts. This would provide the user with a broader range of experiences that people had with the historical site. As such, the cards that users have will be referred to as ‘Person of Interest’ (POI) cards. In addition to this, users can learn about each historical landmark with a unique interface and interactions for each site. Through the aid of technology users can view what the landmarks looked like in their prime and click on parts of the model to learn more about it, such as clicking on the bed in the penitentiary model to learn about the convicts’ sleeping conditions. 

The scenario interface problem is finding an engaging way to educate users on Port Arthur’s history and provide an experience not possible without the aid of technology. Current tours at Port Arthur involve listening to tour guides and reading signs. This is very passive and can result in a lot of the information not sticking in the user’s mind. This application is significant because it aims to engage the full bandwidth of the user’s brain by including interesting interactions in 3D space that provide visual and audio feedback. This is also stated in Cranmer’s (et al 2020) paper that AR can improve the engagement and entertainment experience for the tourist significantly. 

The final application would be free to download for users on mobile and the only other required object for them will be the POI card provided at the start of the tour. Each landmark would have an image target fixed in place at an appropriate location. Users could interact with the image target by aiming their phone towards it and exploring the unique model for that landmark. There would also be space to place a POI card next to the landmark image target to trigger their story for that location. To get the full experience of the application, users will need to physically visit every landmark in Port Arthur and place their POI card near each landmark image target. Whilst the prototype will be Android exclusive, the final application would have IOS and Android builds to accommodate for all users. 




Interface Solution Description

The use of AR for this application was chosen as it solves a variety of interface issues. Firstly, this application takes advantage of perceptual organisation as the user experiences the environment actively rather than as an outsider looking in. This will in turn make the experience more engaging for users and leverages the strength that “AR interfaces offer seamless interaction between the real and virtual worlds” (Billinghurst, 2002). Additionally, the 3D interaction aspect of AR takes advantage of users being 3D spatial beings in comparison to alternative interfaces such as traditional monitors that do not match human characteristics. Cranmer (et al 2020) suggested in the article that “AR can increase tourists' understanding and appreciation of their surroundings”. 

Additionally, the use of AR would allow this experience to be more accessible to those with disabilities by containing audio, visual, and spatial elements, meaning that almost anyone can find value in the experience. The inclusion of audio alongside visual prompts has also been proven to improve memory retention and enhance user engagement (Metatla et al., 2016).  

The setup cost would also be very low, as the only physical materials required to make are the image targets, which can be cheaply printed and replaced if damaged. The technology required is already embedded in the users' phones, meaning that this application takes advantage of the technology that the users own. An alternative solution we considered was having embedded technology at each landmark with the application downloaded onto it, so that users don’t need to use their phone and only require their POI card. This would make the application accessible to those without a smartphone, however this option would be very expensive to implement and maintain, and could result in people needing to wait in line to have their turn with the application due to the limited devices at each landmark. For these reasons, having the application operate on the users' mobile phones was chosen. 




Interaction Design

The main interaction in this application is placing the POI card near a landmark image target to uncover their story. This takes advantage of AR’s strength of operating in 3D space and offers excellent mapping / affordance, as it makes sense that physically bringing the POI card to a landmark would trigger unique dialogue related to that location. The final application would include many different POIs that each have a unique story at every landmark. While users will be given a specific POI card when they enter Port Arthur, they will have the opportunity to get other POI cards if they wish to fully explore the application. There would also be unique interactions if related POI cards are near a landmark image target at the same time. For example, if two brother cards are present at the same time, they could have a conversation reminiscing about an experience they shared at the landmark. This would make the app feel responsive to user’s actions and encourage them to fully explore the peoples’ stories and try to find unique hidden interactions.  

Another interaction is exploring the landmarks themselves. Each landmark will have a 3D model projected above its image target. This model will represent what the site looked like in its ‘prime’. This leverages AR’s strength of merging physical and digital space, as the user can interact with the model in the same augmented space as the real object, which creates a stronger connection between the landmark’s past and present. The user will be able to click on certain parts of the 3D model, which will then display a more in-depth model specific to that element, which the user can then explore in more depth. For example, when exploring the model of the penitentiary, the user could tap on a prison cell to bring up a zoomed in model of the cell, where they could then explore its contents and learn more about the convict's living conditions.

Users can also move their POI card to different parts of the model to trigger unique dialogue related to that area, such as moving the POI card near the entrance of the coal mine model to trigger dialogue about their experience entering / leaving the mine. In a finalised version of the application, audio guides that are higher in quantity and complexity could be used in this interaction to enhance the user experience if they wish to learn more in depth about certain aspects of the 3D model.  



Application Storyboard




Initial Technical Development

  • This application is developed in Unity 3D with Vuforia for both Android and IOS aiming at any mobile devices.  
  • It is designed to identify target images to present desired scenes as additional information to the physical environment. 

Our plan for the prototype is to include two landmarks and three POI cards, with unique dialogue for POIs who have connections to the relevant landmarks. Each POI card will also have a world-space UI element to display the person’s bio when their card is not near a landmark image target, and a dialogue box when the POI is near a landmark image target. The UI will also keep track of what landmarks the user has visited and display this to them.   

We also aim to develop unique interactions with the landmark 3D models and have a world-space UI element to display relevant information to the user. 


Current Development: 

We currently have the following c# scripts: 

Diaglogue: A scriptable object that holds a list of strings and a corresponding list of audio clips. Each Dialogue object represents a specific dialogue sequence. 

DialogueManager: This script manages the dialogue box for a POI. It can take in a Dialogue object to start a new dialogue sequence and has functions for starting the next line and finishing the dialogue. 

DialogueTrigger: This script ‘talks’ to the DialogueManager and uses it to start new dialogue sequences when certain conditions are met (such as colliding with a landmark collider). This script holds the different Dialogue objects for the convict to send to the DialogueManager. 

Bio: This script holds a string of the POI's bio details, and displays this as well as hides the dialogue button if the POI is not currently talking. 


To create the POI cards / dialogue, we used a mix of primary and secondary historical sources. For example, the James Boyd dialogue for the coal mine was based on a letter that he himself wrote.  



Initial 3D Models


This is a model set for the coal mine. Textures are referenced below. 


This is a  model set of the Penitentiary (still in progress).

(Still in the planning stage)

Penitentiary (interior) 




AR card Designs

For the current prototype three different designs were created; William Thompson, Margaret Dalziel, and James Boyd, with the former two being convicts and the latter being a commandant of the Port Arthur colony. The images used for Boyd and Thompson’s faces used real photos of them, whilst Dalziel had no photos of her available. As of such we used the website "This Person Does Not Exist" to generate a face for the application. 




Conclusion 

The goal of this application is to create a deeper connection between users and Port Arthur’s rich history. It will use AR's strengths to educate and tell stories about Port Arthur’s famous locations and elevate the people who experienced them in their prime.  

 


References 

Academic References:

  • Cranmer, EE, tom Dieck, MC & Fountoulaki, P 2020, 'Exploring the value of augmented reality for tourism', Tourism Management Perspectives, vol. 35. 
  • METATLA, O., CORREIA, N. N., MARTIN, F., BRYAN-KINNS, N. & STOCKMAN, T. 2016. Tap the ShapeTones: Exploring the Effects of Crossmodal Congruence in an Audio-Visual Interface. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. San Jose, California, USA: Association for Computing Machinery. 



Textures: 

 


Historical Information References


Technical Development References

  • Brackeys 2017, 'How to make a Dialogue System in Unity' Youtube, available at: 

Get Port Arthur AR Application

Port Arthur AR Application

Add Game To Collection

More posts