Kjetil Nordby about OpenRemote, OpenBridge, AR and VR
Today Kjetil Nordby, research manager for Ocean industries Concept Lab, shares with Rhea Cabral, CEO of MASSworld.news 🌐The Guide to Autonomous Ships, insights on the development of a unified approach to Remote Operation Centres together with opinion on applicable technologies.
👨💼 Kjetil Nordby is a professor of interaction design at the Oslo School of Architecture and Design. He manages the Ocean Industries Concept Lab (OICL) and the OpenBridge Design System. His work primarily focuses on designing advanced maritime workplaces, emphasizing disciplines such as interaction, graphic, and industrial design. For over a decade, he has led numerous research projects in close collaboration with leading maritime industry players.
The OpenBridge Design System is an open-source design resource that supports the creation of advanced maritime workplaces. It began as a framework enabling vendor-independent, consistent design across systems for ship bridges. In later iterations, it has evolved to support any maritime workplace, both at sea and on land.
❓ OpenBridge has recently announced OpenRemote project aiming to unify the user interface of Remote Operation Centres (ROC), could you please elaborate on the details of the project?
🎤 OpenRemote aims to develop an open-source user interface architecture supporting user-centered design of advanced ROC workplaces. There are currently tremendous innovations in remote operations on, over, and below the ocean surface. These new systems are often made in collaboration between multiple stakeholders. OpenRemote aims to help coordinate the design of such systems by offering a user interface framework specifically for the design of ROC workplaces.
We aim to offer a reference design for the physical workplace, screen integration system, and central applications supporting remote and autonomous operations. The framework will be based on the already proven OpenBridge design system that supports ship bridge design. OpenRemote will deliver a remote operation center appendix to OpenBridge.
❓ Who are the main contributors to the OpenRemote project?
🎤 OpenRemote has 21 partners✳ (see the full list below) from industry, academia, and government. The partners span the entire value chain related to the design and operation of ROCs across several vessel and operation types.
❓ What are the specifics of ROCs and Maritime Autonomous Surface Ships (MASS) user interfaces?
🎤 OpenRemote will adapt the existing OpenBridge framework for the design of ship systems. These components are likely directly usable for ROC interfaces. However, there are some changes to be made to adapt it for land use, and there is a need for components that are missing from the current OpenBridge system. The screen control system will need to be adapted for use in a land workplace. There needs to be an extensive user interface framework supporting advanced communication and remote collaboration. Furthermore, there is a need to develop interface elements for human-autonomy interaction and graphical data overlays for video streams. Some of these elements are already in development in other OpenBridge related research projects.
❓ Why is UI standartization for MASS important?
🎤 There are two perspectives here. The first perspective is to ensure consistent design across systems used in the same workplace. Since advanced ROCs are often made in collaboration between several vendors and it is likely they will be upgraded faster than ship systems as innovation continues in this early phase of development, it is important to have a framework allowing all actors to realize consistent and harmonious design across all involved parties. An open-source framework makes this possible.
Furthermore, as operators for ROC centers are likely to change workplaces and people will be trained in different facilities, it is useful for the industry to have a neutral design framework making it possible for workers to transfer skills from one workplace to another. This is important at sea, but perhaps especially important for ROCs where operators are fully dependent on the interface presented to them.
Especially in multi-vendor build systems, developers need to realize that it is not enough to have user-centered design of individual systems. There is also a necessity for user-centered integration with the other systems that are in use.
❓ What impact do you think open user interface frameworks will have for future ROCs?
🎤 The potential impact of open innovation in safety-critical workplace designs is significant. To get a glimpse of where the industry is heading, one only needs to examine the list of partners collaborating with my lab. Significant innovations and developments are currently happening in this field. As companies collaborate, it’s crucial to ensure that the systems work optimally for end users. No system can be considered safe if the people controlling them are not adequately supported. Given that these systems will continue to be mashups of multiple systems, which will likely rapidly change and evolve in the coming years, it’s essential to focus on user-centered integration. I argue open-source design frameworks are poised to play a significant role in this field as we move forward.
❓ What do you think about Augmented Reality technologies (AR) for ROCs/MASS?
🎤 AR in the form of video overlays is useful for ROCs and it is likely that it will be an integrated part of future workplaces. We already have AR components in the existing OpenBridge design system. However, they need to be designed with care and should align with the visual symbols used on other equipment. It is important to realize AR interfaces that do not clutter the video image. It is also important to adapt AR to low frame rates and possible data lag. AR graphics can create an illusion of accurate real-time data since it might display for instance points of interest over video that look precise even if this might not really the case. It is important to design it so that people understand the quality of the data presented over the real world.
❓ Some may say that unified interfaces could make the systems uncompetitive and impact the unique appearance of products, do your views oppose such opinions?
🎤 First of all, the user interface framework does not replace UX design. Each company must still design the layout and content of their application. However, user interface components and generic structures will be similar across vendors. This is a trend that has already taken place in other industries, where most companies adhere closely to design frameworks from Google, Apple, or Microsoft to ensure users are familiar with their products before using them. Ease of use and familiarity increase the likelihood that users will be able to engage with your unique functional product offering.
There were some resistances when we introduced OpenBridge some years ago regarding companies that were using their user interface to differentiate their product for sales and branding purposes. However, this concern does not seem to be an issue any more for our current partners that realize that the benefits of an open standard far outweigh the challenges. Bear in mind that these are safety critical interfaces where safety and usability must be prioritized. Specific visual appearance must be weighed up against how it works together with other systems. Most of our partners deliver to multivendor builds where the benefits of a common user interface philosophy fast become obvious.
Instead, there is also an argument that not following a common standard in itself might make your product uncompetitive. The benefit for end users and buyers of an open user interface standard are so significant that it is very likely that when enough products use the standards, buyers might start to prioritize buying systems adhering to frameworks their existing users already are familiar with. We already see interest in that direction in the market.
❓ Some products may follow OB partly, but could advertise its compatibility with your approach. Do you have any plans for verifications of OB compatibility?
🎤 We do have plans for offering verification. As OpenBridge matures, it is natural that we, in collaboration with other organizations, offer voluntary OpenBridge approvals. We think it is likely that OpenBridge will increasingly be mandated in public and private tenders for new maritime systems. If this practice evolves, it will be necessary to have a mechanism for assessing whether a product is using OpenBridge properly. We think this will be a natural development for OpenBridge.
❓ Could you please advise the most interesting applications of OpenBridge?
🎤 JRC/Alphatron Marine’s Lynx conning module intended to show the usual ships navigation, engine and propulsion data, such as heading indicator, rudder angle indicator, rate-of-turn indicator, speed indicator, RPM and much more.
Brunvol’s BruCon DP & Joystick Control Systems intended to automatically maintains a vessel’s position and heading by using its propellers and thrusters
SEAM AS’s e-SEA® integrated bridge system
CDP Studio’s, no-code and full-code development tool for modern control systems and HMI including OpenBridge library by default
❓ There have been many discussions on VR in the past, but there seems to be only a few real applications. What is your opinion on VR?
🎤 We use VR in all our design processes. For designing workplaces like ROCs, I propose the technology is already very useful. In fact, some of the images we have submitted for this interview are drawn directly from the VR tools we use in design. We also think VR might be useful in operations. Now that the resolution is starting to increase significantly, it is possible to open up a ROC workplace in your living room just by having a set of VR goggles. This could make it possible to allow critical personnel to participate remotely in special cases. With increasing satellite bandwidth, it is also possible to use VR goggles to get an immersive experience of the conditions at sea. As the technology matures, we should explore these possibilities and certainly not rule them out based on the current generation of hardware.
❓ The Projects (and their results!) of Ocean industries Concept Lab are really fascinating, could you please advise who are some of the supporters of OICL?
🎤 There are two main groups of supporters of OICL. The first group is the 40 companies, institutions and government agencies that are participating in our current research projects. Among the supporters are companies such as Kongsberg, General Electric, Fugro and DNV to mention some. Then we have over 1000 maritime companies that have registered to access OpenBridge.
✅ Don’t hesitate to follow Ocean Industries Concept Lab page, as well as use OpenBridge in your developments (the design library is free to use!).
📷 Photos: Ocean Industries Concept Lab
Note:
✳ Full list of OpenRemote project participants:
🔹 Brunvoll
🔹 Skipper Jontron electronics
🔹 Norwegian Electric Systems
🔹 SEAM
🔹 ioStudioTech
🔹 R. Stahl Tranberg
🔹 Kongsberg Maritime
🔹 Massterly
🔹 Maritime Robotics
🔹 Vard Electro
🔹 Fugro
🔹 Norwegian Coastal Administration
🔹 Norwegian Maritime Authority
🔹 DNV Maritime
🔹 Lloyd’s Register
🔹 The Oslo School of Architecture and Design
🔹 Marine Institute – Memorial University of Newfoundland
🔹 Norwegian University of Science and Technology (NTNU)
🔹 Sintef Ocean
🔹 Korea Research Institute of Ships and Ocean engineering (KRISO)
🔹 Halogen