At I/ITSEC 2019, BISim fielded a five-man team for the inaugural Iron Dev competition, an event similar to a competitive cooking show where teams were given a challenge (to develop a training solution to improve warfighter readiness) and a ‘secret ingredient’ (Iron Man). The teams had three days to develop their solutions using the software and tools provided by event sponsors and presented their prototype on the final day of the show.
The Bohemian Knights took home the People’s Choice award for their demonstration of J.A.R.V.I.S., which stands for Jointly Assisted Remote Visual Imaging System, which enables a qualified maintainer to work remotely with an outside technician.
Andy Rossetter, the team’s leader and BISim software engineer manager, says that J.A.R.V.I.S. was developed based on a problem they identified by talking with U.S. Air Force subject matter experts at the show.
“Let’s say we have an AC-130 that was diverted from its mission to a base because it requires maintenance but the closest qualified technician is located at another base far away,” Andy explains. “This creates an expensive logistical problem for getting that AC-130 back into the air.”
BISim’s version of J.A.R.V.I.S., adapted from Tony Stark’s sophisticated AI system in the Marvel Studios movie Iron Man, was developed using Unity, a VR-based desktop application, and a Magic Leap 1 for the onsite tech to view a 3D schematic overlay of the equipment to be repaired and hear guidance from the qualified remote maintainer.
“In the prototype, we targeted a PC for demonstration purposes but in the real world this could have been an aircraft panel or landing gear housing,” Andy says.
Here’s how it works: After a connection is established between the qualified remote maintainer and the onsite tech staff, they can communicate via audio and a video screen using the Magic Leap, which also provides a low-bandwidth option instead of video. The onsite tech staff first registers the equipment to be repaired by looking at a marker point on the component to be repaired. A wireframe appears once the registration is complete. Users can click on a part which will bring up a dialogue of information about the part, relevant warnings, and hints about tools needed for repairs. The tool also allows the remote maintainer to send ordered callouts to guide the onsite tech with repair tasks.
“This application has potential for the logistics of maintaining just about anything you can line up a hologram overlay to,” says Tim Turcich, BISim’s Sr. AR/VR Engineer.
The ‘Bohemian Knights’ competed against the Air Force Research Labs, Army Game Studio and Naval Postgraduate School.