Enhancing Pedestrian Safety with Drones

Drones are now going to assist busy urban dwellers cross streets safely. In a new paper titled ‘Enhancing Pedestrian Safety through In-Situ Projections: A Hyperreal Design Approach’, the researchers Marius Hoggenmueller and Martin Tomitsch, from The University of Sydney, Australia present a conceptual application of drone-based in-situ projections to support people when crossing busy roads that lack dedicated pedestrian crossings.

To overcome technical, legal and risk challenges of evaluating these kinds of speculative pervasive display applications, researchers introduce the concept of hyperreal prototypes and how such prototypes allow the evaluation of HCI solutions in high-risk scenarios in a virtual-reality environment.

Researchers have recently started exploring the use of drones to project contextual information directly into the environment with the aim to create mobile, situated displays. Technical limitations e.g.- battery life, visibility of projected content and legal restrictions abound yet, such in-situ projections offer a number of conceptual advantages, which warrant an investigation of their use in urban scenarios. In-situ projections allow for the deployment of public displays without making costly changes to the physical infrastructure. Still researchers prefer falling back on concept videos and virtual reality (VR) simulations. However, these simulated environments suffer from a lack of realism in terms of interaction and representational fidelity.

Addressing these limitations this project introduces the concept of hyperreal prototypes, using 3D 360-degree video recordings of a real urban situation that is augmented with dynamic rendered overlays. We first describe a conceptual pedestrian safety application, before outlining the conceptual layers of hyperreal prototypes.

Hyperreal prototype setup to evaluate drone-based pedestrian crossing projections in virtual reality

Conceptual application

To investigate the conceptual application of drone projections for improving pedestrian safety in urban environments researchers chose an application area where pedestrians are particularly at risk of being hit by a car in dark environments-using a projection-based solution- a viable option. Compared to smart phone-based solutions, in-situ projections provide barrier-free access to information (i.e. not being limited to Smartphone users) while also reducing potential risks associated with pedestrians interacting with a Smartphone while crossing a road. Drones with their ability to sense the environment from a birds-eye view, could make precise calculations when it is safe to cross a road, and provide this information via real-time projected visualisations to pedestrians on the ground.

Hyperreal prototype

The notion of “hyperreality” refers to the fusion of physical and virtual realities, and the inability to distinguish them. Using the proposed conceptual pedestrian safety application the conceptual layers of hyperreal VR prototypes elevate the representational fidelity of purely 3D model-based VR simulations. First, hyperreal prototypes use 3D 360-degree video recordings of real urban situations to increase a sense of realism in users. For this application a busy, multi-lane street that lacked a pedestrian crossing and had a popular nearby park was video recorded. Importing the recorded videos in Unity1, the scene can be augmented through dynamic overlays: this includes 3D-modeled objects (i.e. drones that carry a projector and sense the environment) and 2D visualizations created in Processing2 (i.e. dynamic visual crossing patterns).

Creating and streaming the visualisation via Processing, allows to

1. Quickly explore various design concepts, and
2. Visualizations can be dynamically adapted to the traffic situation in the real-world video.

Navigating in the 360-degree video is supported through the VR system (e.g. HTC Vive3), allowing the user to rotate around their own axes. To overcome the limitation that comes with the inability to walk in 360-degree videos, the video is zoomed into to simulate a feeling of moving forward or backward. More sophisticated solutions could include taking shots at various positions, e.g. through a camera slider or a mobile-robot.

The final layer of hyperreal prototypes refers to interaction modalities: while interacting with objects in the real-world video is not feasible, controller- and gesture based interactions can be used to let users directly interact with the dynamic overlays. This allows, for example, to explore gesture-based interactions with the 3D-modeled drones, e.g. to request a nearby guidance drone. The evaluation of social interactions, i.e. having multiple users interacting within the same simulation environment is also in the pipeline. This could be achieved by developing multiple VR instances based on simultaneous video-recordings at different positions.

Conclusion and future work

Although still limited by technological and legal constraints, drone based projections provide a potential alternative to smart phones and fixed public displays for communicating information in an urban environment, in particular in low-light situations. In their future work the team plans to run a user study to evaluate the efficacy of hyperreal prototypes for simulating speculative interactive applications in urban environments.

Citation: Marius Hoggenmueller and Martin Tomitsch. 2019. Enhancing pedestrian safety through in-situ projections: a hyperreal design approach. In Proceedings of the 8th ACM International Symposium on Pervasive Displays (PerDis ’19). ACM, New York, NY, USA, Article 29, 2 pages. DOI: https://doi.org/10.1145/3321335.3329682