mixed reality solutions for industrial automation

Case: Visual Measurement Guide

October 30th 2019permalink

We were discovering the new case for aircraft parts manufacturing for the past two months. Quality assurance is one of the most important stages in the process. The thickness inspection of a wing panel which could be five meters long and two meters wide needs to be performed in about a thousand spots. One of the commonly used approaches is to mark these spots by the mylar, a plastic stencil covering the panel. It has a number of problems though: Preparing the mylar Placing it in the correct position Progressing through each end every spot without missing any No control over the range of spots out of range A Mixed Reality solution overlays a virtual layer over the wing surface with the spots to be measured. The marker is used for recognition:

However, in the first trials the grig was unstable while the procedure required high accuracy. We have found a way to stabilize it ultimately:

To assist in performing the inspection further we added an indication of the active point and the output of measurements:

Whether the measurement fits the tolerance range is indicated by color. This prototype was successfully tested and will be used as a base for a pilot product.

Reactor Updates

July 11th 2019permalink

Scaling up content

We entered an active phase of development focusing on adding new parts and components of an engine as well as expanding the interface. For example, we’ve created the functionality to attach component callout with the description and pictures of a selected part of the engine, and applied it to the engine Oil Tank:

Each callout can also store the video tutorial and allow video and photo capturing. This toolkit is aimed at enhancing the decision-making power of the engineers performing maintenance and supercharging technical training.

Adding new features

1. Click without focusing on anything

There is a special mode in which the user can click without pointing the cursor on anything clickable. It is designed for advancing through the instruction list and for inspections - where it adds the maximum convenience.

2. Click on a real object

Once the real object recognized any its part is becoming an active UI element. The user can zoom on it or change its state depending on the context.

3. Photo catalog and picture taking

Now any of the engine parts can have its own image gallery with the full functionality to manage it: delete, add new or take an actual picture. It would be most useful for all kinds of inspections and quality assurance procedures. Next step would be to add the voice to text notes.

4. Work order instructions for manufacturing

Often the manufacturing instructions are extensive and differentiated in nature containing text descriptions, photos, tables, quality checks, etc.

We created the layout and the functionality for such cases. The instruction layers could be expanded almost infinitely to the right-hand side and collapsed after being processed.

Thanks for reading and see you next month!

Use case: documentation use in aviation maintenance

December 1st 2018permalink

During of various inspections the maintenance engineers are using instructions with a step-by-step guides. There're also a lot of manuals for the inspection process, as well as reference materials with information about the equipment (images, diagrams, drawings), instructions for assembling and disassembling engine and its components. Every action the engineer is taking should be strictly corresponded within the docs and checklists.

All of that leads to a massive amount of every day printed papers. Manuals and references book are constantly in use by technicians and also presented near by every working place.

The instant access, usability and especially search the data provided by paper based docs are extremely weak. Some of them already are the e-docs but the access still is very delayed due to distance separation of technical part and place where laptops are used.

Thus common disadvantages of paper documentation are following:

  • The hands need
  • Delayed updating and actualization of the data
  • The search of the information
  • Remoteness from the workplace

Transferring the all kind of documentation to mixed reality solves these problems as well as gives number of advantages which can be provided neither paper docs nor the standard computer based materials. Some of these advantages are revealed in following cases:

  • The occurrence of non-standard damage.
  • The spare parts order.
  • The deeper understanding of assembly and disassembly of the engine by providing multimedia sources of information.
  • Ability of instant contact with the manufacturer

Aviation maintenance is still apart of the active use of mixed reality. But there are several examples of the introduction of advanced electronic documentation in other sectors.

One of them is the BMW entrusted the quality control of the assembled cars to Google smart glasses. Employees of the concern now remain in the car and see plans and inspection instructions in their glasses. When testing a vehicle, workers should conduct 10-25 tests and document them. Previously, for this, the staff needed to move from the car to the terminal. Now, workers remain near the car and document the inspection process with smart glasses, and in the event of a defect, transmit information to competent professionals.