Augmented and Mixed Reality in Construction

Augmented reality (AR) and mixed reality (MR) are in the headlines, thanks to the recent mobile gaming boom. How are these emerging technologies applicable to construction? In this blog post, I present six application areas to consider.

In AR—like Google Glass or Pokémon GO on a mobile device—the visible natural world is overlaid with a layer of digital content. In MR technologies, like Microsoft’s HoloLens or Magic Leap, virtual objects are integrated into and responsive to the natural world. In my earlier post, I wrote about virtual reality (VR), where the real world is replaced by a computer-generated environment.

All the virtual technologies are still in relatively early stages of development. However, they already demonstrate the potential to change how we design, build, commercialize, and use the built environment. I brainstormed six application areas for AR and MR in construction.

Architectural and engineering design

Designers use photo or video montages to show how, e.g., a new building, material, or a piece of furniture will sit in an existing environment. With AR, a designer can overlay an interactive design model on the real, live environment.

You don’t need special equipment to use AR in construction. Augment, for example, uses tablets and smartphones as AR devices. Using their app, a printed drawing of a house can be overlaid with an interactive 3D-model.

Microsoft’s HoloLens is a headset that supports applications that mix holographic images with the real environment that the user is in. It scans the user’s environment and adds computer-generated surfaces, graphics, and objects into the real scene. Depending on the application, a designer can, e.g., manipulate objects, use voice commands, and move inside the designed environment.

Seeing things at scale is helpful, even for a trained designer. I think that MR will finally free the designers from their desks or offices and possibly bring them to the site of a new or renovation construction. I’m convinced that these opportunities will also lead to innovations in design and construction.

Construction sites

We’ve seen tablets at construction sites, but the possibility of integrating digital content with the real world opens up new possibilities for planning work, quality management, safety, and inspection.

Almost two years ago, I interviewed Mallorie Brodie, co-founder of Bridgit. At the time, the company was prototyping an interesting app that combined Google Glass and Thalmic Lab’s Myo armband. The app allowed a worker to browse through specs and design documents with Google Glass using hand movements, even with gloves on.

Another AR app that I’ve written about is DAQRI Smart Helmet. It has a display visor that connects the user to the work environment and provides relevant information instantaneously. Since everybody on a construction site is already wearing a helmet, adding AR or MR capabilities to the helmet is a very natural choice.

Safety is a big issue in construction. At the AEC Hackaton 3.2 Helsinki, one of the hack teams, Team Safety, used BIM, HoloLens, and Trimble Connect to plan, analyze, adjust, and verify safety on a construction site (see the video).

Project collaboration

Construction project teams are often dispersed. There are already many digital ways to collaborate over a distance. AR and MR will add a new dimension to digital collaboration.

Software developer Object Theory has shown how MR can be used to gather a project team in one space around a 3D model of a building, as avatars. A construction firm, CDM Smith, has already used this technology successfully.

Commissioning and handover

There are already mobile punch list apps and cloud services that have made the closeout process considerably more efficient than the paper-based equivalent. Likewise, commissioning requires a lot of documents and information on building systems. AR and MR apps could bring the specifications, along with the actual performance metrics, right to where they are needed.

Handing over a project to the client is another phase where AR and MR could be used. An interactive building owner’s manual and building user’s guide could add considerable value over the life cycle of a building. However, there is a serious challenge with property-related apps. Since technology develops very fast, apps become obsolete in a relatively short time.


AR and MR can speed up learning processes considerably. Instead of sitting in a classroom, trainees could design, build, and test ideas in a real environment. Distant education would also benefit from AR and MR applications.

Maintenance training is also a promising application area, as the following video demonstrates.

Marketing and sales

In 2013, IKEA created an AR catalog app to help customers visualize how certain pieces of furniture would fit and look in their own home. Similarly, AR and MR can be used for visualizing prefabricated homes, renovation projects, building and interior products, and so on.

Caterpillar is developing AR and VR for maintenance and safety checks, but they also use AR for marketing and sales as this video shows.

The most feasible way to offer AR and MR services, so far, is with smartphones and tablets. Consumers are getting used to gaming, entertainment, and sales apps that use AR and MR. This interest and demand will lead to feasible and affordable technologies that the construction industry can benefit from. Cross-industry collaboration and imaginative thinking in this field could result in killer innovations.