How Augmented Reality and a Robot Arm Could Give Construction a Leg Up

by Drew Turney
- Aug 8 2018 - 6 min read
The Collaborative Robotic Workbench (CRoW) combines an LBR iiwa robotic arm with an augmented reality (AR) headset for the user.
The Collaborative Robotic Workbench (CRoW) combines a KUKA LBR iiwa robotic arm with an augmented reality (AR) headset for the user. Courtesy KUKA Aktiengesellschaft.

The immense construction industry—with annual expenditures of more than a trillion dollars in the US alone (projected to be $1.5 trillion by 2022)—has bounced back from the fallow years of the Global Financial Crisis and now represents 4.3 percent of the US GDP.

The industry has also found itself in a brave new world of possibilities thanks to digital and robotic advances unimaginable even a decade ago. With the value of US private construction reaching almost $900 billion in 2016, you’d think construction would be first in line to leverage the time and cost savings of the newest technologies, but engineer and architect Lauren Vasey is well-acquainted with the roadblocks.

“Everyone wants the construction industry to be more digital, but there are so many barriers,” she says. “There’s this compartmentalization of processes and of the entities that go into making a building, like the contractors, designers, and fabricators. They don’t have a digital workflow between them, and there’s no feedback between the various components.”

Research associate Ondrej Kyjanek, Vasey’s colleague at the Institute for Computational Design and Construction (ICD) at the University of Stuttgart, says the field is waiting for one thing that will make everyone say, “Wow, this is perfect.”

A team from the Institute for Computational Design and Construction (ICD) at the University of Stuttgart demonstrated CRoW at this year's KUKA Innovation Awards.
An ICD, University of Stuttgart team demonstrated CRoW at this year’s KUKA Innovation Awards—from left to right: Samuel Leder, Lauren Vasey, Bahar Al Bahar, Ondrej Kyjanek, and Benedikt Wannemacher. Courtesy KUKA Aktiengesellschaft.

Project CRoW is an attempt by Vasey and Kyjanek—along with fellow ICD team members Bahar Al Bahar and Benedikt Wannemacher—to deliver the construction industry’s new “wow.” The Collaborative Robotic Workbench (CRoW) combines a generative-design workflow with a KUKA LBR iiwa robotic arm, which has a gripper claw to pick up, move, and place components. The project was developed for the 2018 KUKA Innovation Awards, a yearly robotics competition with the theme of real-world interaction.

But the secret sauce is the augmented reality (AR) user headset. With a view of the materials and construction, AR lets the user digitally plan the next piece’s placement, trying alternatives and assessing data beforehand. The robotic arm’s precision movements position the next component exactly; the user simply fastens it with a nail gun.

The CRoW appeared at this year’s Hannover Messe, where ongoing production enabled the building of an eye-catching wooden sculpture. The device takes on tasks traditionally done with human hands—such as measuring, carrying, or holding—while constantly scanning and evaluating the structure with sensors for comparison to the 3D model. Through its knowledge of the model, the software can even suggest the best position for the next piece in the AR headset’s point of view. “As a designer, you can influence the final shape of the structure, but you have the choice to influence the fabrication sequence, as well,” Kyjanek says.

Informed by the the 3D model’s big picture of a project, the AR system can collect endless data. Each axis of the robot’s articulated joints contains torque-measurement sensors, which show the force exerted on each joint and an on-the-fly picture of whether the intended movement is feasible. Previewing the path the robot will use to get from A to B allows the user to plan for possible collisions or interference. It’s also possible to preview a project from where it is at a given point and compare it to pieces yet to be placed or even compare it to the final overall structure.

Using an AR headset, the CRoW user can digitally place the next piece of a construction, assessing data and trying alternative positions digitally.
Using an AR headset, the CRoW user can digitally place the next piece of a construction, assessing data and trying alternative positions digitally. Courtesy ICD, University of Stuttgart.

Watching the CRoW in action, it’s easy to imagine sharing and scaling this workflow. If an industrial-scale version of the system builds a house, apartment building, or skyscraper, the data file generated by the robotic arm’s movements could be recorded and refined and then applied again and again on different sites, with builders guaranteed a perfectly replicated result every time.

With the right know-how, architects or builders could even delve into the data to make tweaks to movements, fastenings, or pressure points and adjust them according to the conditions of the final product.

During the Innovation Awards demonstration, the CRoW team recorded the LBR iiwa’s movements for diagnostic purposes, which Vasey says could be a selling point. “If people just need to buy the robot and download designs and can manipulate them with a generative or computational workflow, then the intelligence would be in the software running in the background,” she says.

But the challenge remains that construction, no matter how advanced, has deeply entrenched practices that are hard to disrupt because the industry has invested a lot in getting them right. What could the CRoW potentially offer a construction company at the right scale, besides being cool new tech?

CRoW's robotic arm positions a piece exactly, constantly evaluating the physical structure against the 3D model, and the user simply fastens it with a nail gun.
The CRoW’s robotic arm positions a piece exactly, constantly evaluating the physical structure against the 3D model, and the user simply fastens it with a nail gun. Courtesy ICD, University of Stuttgart.

“It was proven it’s actually much faster to use and not just by our research but also other research institutions working with AR,” Kyjanek says. “It’s radically different, but it’s much more intuitive when you can see the components and construction in front of you rather than just on plans. It’s easier to train construction workers to work with the AR than to teach someone how to read plans.”

The CRoW is another example of the potentially limited lifecycle for a new technology’s buzz. Like 3D printing, AR enjoyed a white-hot 15 minutes of fame among consumers running around public parks and city streets looking for Pokemon Go creatures. But just as the promise of low-cost 3D desktop printers never quite materialized, it feels like those days are over.

Yet history has shown that early consumer buzz can be just a marketing proving ground; 3D printing is quietly transforming the industrial sector, and Kyjanek and Vasey believe AR’s time is coming, too.

Through its AR system, CRoW software can collect endless data and even suggest the best position for the next piece to the user through the AR headset.
Through its AR system, the CRoW software can collect endless data and even suggest the best position for the next piece to the user through the AR headset. Courtesy ICD, University of Stuttgart.

The CRoW is essentially a proof of concept, and Kyjanek says there are no current plans to commercialize it; any next steps will concentrate on further development. “So many new problems and challenges came up during the process that have be solved first,” he says.

Vasey adds that ICD’s role is to get the fundamental research in place to rethink how new technologies challenge existing construction processes and protocols. The Institute regularly finds interdisciplinary partners within a promising larger research network that enables extensive collaboration between academia and industry.

So it may be early days for a real-world AR-in-construction revolution, but when you construct a house or office building in the future, don’t be too surprised if the builder shows up with a hard hat, a tool belt—and a headset for an articulated robotic arm.

More Like This

Success!

You’re in.

Get smart on the future of making things.

Subscribe to our newsletter.