Virtual collaboration within and across industries – promises and challenges

Construction’s vanguard role in virtual collaboration

Construction is perhaps one of the best-positioned industries for leveraging virtual environments. Computer-aided design (CAD) and building information modelling (BIM) have been used for decades, and digital twins are becoming increasingly common and sophisticated in this industry.

Andrew Anagnost, CEO of design software leader Autodesk, believes that in future, people are going to collaborate in virtual spaces on building designs and solving building problems. “In the architecture, engineering and construction industry, XR’s potential is both massive and relatively untapped. Businesses, especially those that rely on cross-departmental collaboration, stand to gain huge benefits from embracing extended reality solutions.”

Experimental efforts in merging real and virtual environments hint at the flexibility and range of uses of future XR tools. In one example, Meta Reality Labs Research and the University of Duisburg-Essen are attempting to merge real and virtual worlds in new ways.

In the architecture, engineering and construction industry, XR’s potential is both massive and relatively untapped. Businesses stand to gain huge benefits from embracing extended reality solutions

Their researchers have developed a system in which objects in the real world can be manipulated virtually. Not only can these objects be virtually picked up in the real-world video stream, but the object also vanishes at the location in the feed as a result of the manipulation. This way, interior designers can change existing environments in the live stream. They can move, remove, or replace objects, for instance, to show clients or collaborators how such an adjustment would affect a place or room.

Business collaboration as the new frontier

Leslie Shannon, head of trend and innovation scouting at communications tech giant Nokia, notes that BMW has worked on collaboration in virtual reality for some time. In 2015, the carmaker started working with Epic Games to make VR usable for car and component design, and since 2020, it has been working with Hololight to shorten development cycles by leveraging XR.

Yet Shannon warns that collaboration in XR can raise security concerns. Rendering of images on wearable devices has setup and comfort advantages. Problematically, data on devices can create security issues if the device is lost or stolen. Instead, rendering visualisations on servers can not only improve the XR experience, but can also ensure data stays with the company that owns the intellectual property. Here, the visuals and data are streamed to the headset, for instance.

Retaining the information in virtual environments will be crucial in many collaborative tasks to ensure data security. For military use cases, such a configuration will be a requirement.

Lockheed Martin Skunk Works worked with operator Verizon and applications by Hololight to demonstrate “streaming for real-time visualisation content on edge computing devices to advance [US] Department of Defense sustainment missions”. The system enabled “complex visualisation applications such as augmented, virtual or extended reality experiences” and “streaming of real-time, complex, 3D visualisation content … to edge compute devices, including tablets, mobiles, head-mounted displays and more”.

In many cases, different layers of digital twins, for instance, might require separate security levels. In urban management applications, for example, streets and transportation maps could be freely available to the public, whereas representations of infrastructure networks such as sewage pipes and telecommunications lines represent sensitive information that requires data protection.

Monitoring and management in virtual space

Geoffrey Bund is CEO of Headwall, a provider of AR- and VR-enhanced command-and-control centre (CCC) applications. Headwall is creating CCCs that replace currently dominant video walls with headsets. Operators can view and control 32 virtual displays that can include dashboards, live video feeds, and other data to be monitored on headsets. The operators can occupy spaces with a small footprint, negating the need for substantial facilities that can house full-blown video walls.

The headsets can also be used for collaboration efforts that require access to many types of visual information, including live streams, virtual environments, and visualised sensor and environmental data. Bund maintains, “command and control is relevant to collaboration in virtual reality as command centres are intrinsically built for collaboration”, but notes, “early VR headsets were intrinsically isolating”.

Headwall’s solution is facilitating collaboration by synchronising the video wall in a physical security operations centre with the virtual one in Headwall’s software environment. In cases of emergencies, for instance, when experienced disaster handlers are not present in the physical command centre, they can access the video wall from the headset remotely. The expert can then coordinate efforts using VR. 

Bund highlights the flexibility of virtual applications and notes the need to look at new design options. “If you didn’t need to pay rent, hire contractors or source materials, what kind of collaborative space would you build? It probably wouldn’t look like an 800ft² room with fake windows and a whiteboard,” he says.

Adding information layers to enhance collaboration

As virtual collaboration environments become increasingly common, adding layers of information and finer degrees of data granularity to landscapes, objects and phenomena will support teamwork between experts from various fields. Motion and heat sensors, accelerometers and lidar, cameras and computer vision, GPS and spatial mapping, haptic and other devices will facilitate exchange of information, guide understanding and support decision-making.

Jani Jokitalo, senior advisor of information and communication technology at Business Finland, highlights specialised camera-based information that can support task-focused collaboration. For instance, Kopin Corporation is creating thermal imaging systems that can integrate into firefighters’ masks, and Agate Sensor is focusing on hyperspectral imaging sensors that can transfer a wide range of data to the visual spectrum to provide additional information such as medical conditions, surface quality and environmental measurements.

Janne Järvinen, mission lead for digitisation at Business Finland, points to 6G technology’s potential role beyond being the future standard for next-level wireless communications. A network build on this standard can provide new layers of information by itself. Nokia Bell Labs is working on such a network as a sensor.

“6G networks could sense their surroundings, allowing [users] to generate highly realised digital versions of the physical world. 6G will give our networks the ability to sense. By bouncing signals off objects, the network will determine what’s there, how things are moving – and potentially even what they’re made of,” says Järvinen.

The increasing use of artificial intelligence (AI) can render information and personalised applications in real time. Laura Olin, chief of staff at Younite-AI, notes that advanced eye-and-hand tracking, spatial audio, multi-platform support, AI assistance and cloud sharing are improving a wide range of features that will support collaborative applications.

Younite-AI is focusing on the convergence of XR and AI to enhance immersive experiences. In business-to-consumer applications, AI will help with the creation of personalised experiences at the moment consumers interact with the application. In business-to-business applications, AI will support simulations and training applications as digital twins are changing the industrial world. Olin believes that in the short term, AI assistance is “providing personalised guidance and contextual understanding that improves the efficiency and productivity of collaborative efforts”.

Olin highlights that AI-driven features will transform headsets into powerful tools that can perform complex tasks – such as spatial and facial recognition – in real time. She believes AI is pushing the boundaries of what mixed reality devices can achieve, transforming them into artificial companions that can enhance safety, optimise productivity and improve overall experiences – all supporting increasingly interactive collaborations in the virtual world.

Nokia’s Shannon also sees value in combining AI with digital twins of projects and cities. Collaborating teams would be able to render ideas and thoughts quickly and conveniently to be presented in virtual environments. For instance, planned street networks could be visualised in ways that participants can walk through, and possible pedestrian zones with plants and trees could be highlighted to illustrate the effect on neighbouring businesses. AI could then offer an understanding of the effects such changes would have on automotive and foot traffic.

In addition, AI could be employed to transfer unwieldy formats of information, such as spreadsheets, into intuitively accessible visualisations. Such visualisations also could facilitate widening discussions with stakeholders such as decision-makers and community representatives who might have a hard time understanding columns of numbers.

Considerations beyond technology

Finally, technological discussions can miss the main aspect of collaboration efforts – the human interaction. Karoliina Salminen, lead of smart manufacturing at Finnish research and technology organisation VTT, points to the need to transform user interfaces into “inclusive systems that consider different cultural aspects and individual skills and capabilities”,

Also, the potential for cognitive overload exists among users of advanced systems that use and convey an increasing amount of information at increasing speeds.

Salminen says we need to consider how people will cope with the vo information available, how to ensure mental well-being, and how to ensure comfortable and smooth communication and interaction with the unnatural merger of virtual and physical worlds. “An easily available and constantly increasing amount of information can be very tough and difficult to tolerate for many of us,” she cautions.

Martin Schwirn is the author of “Small data, big disruptions: How to spot signals of change and manage uncertainty” (ISBN 9781632651921). He is also senior advisor for strategic foresight at Business Finland, helping startups and incumbents to find their position in tomorrow’s marketplace.