Are you facing the challenge of combining powerful native image processing with a flexible web-based application framework? Are you wondering whether hybrid apps are really performant enough? And how do you integrate specialised devices instead of just standard hardware? These questions fascinated me, and I found a solution that is reusable.

Image specialists: Industrial cameras

The use of industrial cameras has become increasingly important in various areas of application. These cameras are specially designed for industrial image processing tasks. They are used in areas such as quality control, robotics and medical technology.

Cameras such as the Baumer VLU-02M offer a number of advantages, for example

• High image quality: Thanks to high-quality sensors, detailed and precise images can be captured.
• High frame rates: At up to 160 fps, they are ideal for applications that need to capture fast movements or processes.
• Flexible customisation: With the option of setting exposure times from 0.004 to 60,000 ms, they can be adapted to different lighting conditions and applications.

Camera manufacturers such as Baumer usually provide an SDK (Software Development Kit) that enables the integration of their cameras into C/C++ programmes. This is particularly advantageous for fast and system-oriented data processing. Nevertheless, effective, powerful and user-friendly application software is essential to control the camera and display its images. This software must be customised depending on the application purpose, target group and context. However, modern and flexible software frameworks for platform-independent applications often use programming languages other than C++.

Electron: an all-rounder framework for platform-independent apps

Electron is a popular open source framework based on Node.js, which enables developers to create cross-platform desktop applications using web technologies such as HTML, CSS and JavaScript or TypeScript. This offers a number of advantages for app development:

• Cross-platform: with Electron, developers can use a single codebase to create applications for different operating systems such as Windows or MacOS, in conjunction with Ionic’s Capacitor, the code can also be used for mobile apps.
• Familiar standards: UI/UX designers and software developers can use their existing web development skills and tools without having to learn a new programming language or technology.
• Customisable user interface: Web technologies make it easier to create modern and engaging user interfaces that can be easily customised and scaled.
• Large community and resources: Access to an active community, numerous libraries and resources for developing Electron apps.
• apid development and prototyping: Fast iteration and the live reload feature allow developers to see changes in real time, speeding up development and prototyping accelerated.
• Integration with backend services: Electron applications can be easily integrated with various back-end services and APIs to enable advanced functionality and data processing.
• Security: Electron offers security functions such as content security policy to prevent cross-site scripting attacks and improve application security.

Despite all the advantages, the higher resource consumption compared to native applications is a challenge, especially in image processing. And while Electron offers access to standard system resources, in the case of a special industrial camera, it is of course necessary to create your own extensions outside of the features provided in order to exchange control and image data with the manufacturer’s SDK.

Advantages of C++ integration with Electron

The aim of my project was to create a bridge here to combine the advantages of two worlds: powerful native image processing and a flexible application framework.

• Access to hardware resources: Native integration allows you to directly access and control the functions and properties of the camera.
• Highly performant image processing: C++ enables efficient and fast processing of image data, which is particularly important for real-time applications in order to achieve a high frame rate.
• Scalability and flexibility: Electron enables the development of cross-platform desktop apps with web technologies, while C++ offers performance and efficiency.

Challenges with Electron+Native projects

The development of Electron apps with native integration brings with it a number of challenges. I was able to familiarise myself with and master these in the sample project:

• Concurrency: While the camera data is read natively into the buffer, the application should not wait or block.
• Complexity of the configuration: The integration of external C++ libraries and SDKs into an Electron project can be complex and time-consuming. Errors can often only be localised by means of tests.
• Debugging: Debugging C++ code in combination with JavaScript can be challenging. In order to at least be able to use the console for output, the only option is to access the log function via NAPI.
• Sandboxing and IPC: The separation of the main and renderer processes in Electron requires the use of inter-process communication mechanisms (IPC) such as the context bridge in order to exchange data between the processes securely and efficiently.

My approach and solution

First test app with C++

Before the integration into Electron, I built a first, purely native C++ application with OpenGL output to test the Baumer GAPI SDK and the image output. This allowed me to understand the basics and check the camera’s functions. I implemented a separate thread (std::thread) that took care of the regular reading of the camera image. I also implemented a test image function to test the function of my data interfaces even without a connected camera or if the SDK libraries could not be found, so that configuration problems could be localised.

Configuration and implementation as a NAPI add-on

My solution with Electron then consisted of developing a C++ NAPI add-on from this, which allocates memory, writes the camera’s image data buffer and also controls the exposure time, for example. NAPI (Native Application Programming Interface) is a node.js technology for accessing native APIs and libraries in Node.js modules.

To ensure that the C++ NAPI addon has correct access to the static and dynamic Baumer SDK libraries and C++ header files, the configuration had to be set up carefully. Electron uses node-gyp to compile Node.js modules, my native/binding.gyp file looks like this (excerpt):

I was then able to implement the actual functions in my native.cpp, which I would like to make available for Electron. In the example (excerpt), the current image from the camera and the set exposure time are read out – the functions for this are exported via NAPI. The transferred data objects (e.g. Napi::Uint8Array ) are provided directly to Electron when called from the JavaScript side, which means the fastest possible application-side access.

Integration into the Electron app and context bridge for the renderer sandbox

Seitens Electron/Javascript können die exportierten Funktion aufgerufen werden. Ich habe den eigentlichen NAPI-Aufruf in der Hauptroutine (main.js) jedoch wiederum über IPC der (in Electron aus Sicherheitsgründen entkoppelten) Render-

The exported function can be called from Electron/Javascript. However, I have provided the actual NAPI call in the main routine (main.js) again via IPC of the render sandbox (decoupled in Electron for security reasons). The calls and function returns are therefore passed through.

Sandbox bereitgestellt. Die Aufrufe und Funktionsrückgaben werden demnach durchgereicht.

Here is my main.js (extract):

In my preload.js I have prepared the ContextBridge as follows:

Renderer-side call

The actual Electron user interface is then defined in index.html, which integrates my RendererScript:

This renderer (excerpt from my renderer.js ) ultimately retrieves the data:

Summary

By carefully configuring and implementing the IPC mechanisms, I was able to realise a highly performant and stable solution for the live output of the camera image in my Electron app. The additional effort

through the NAPI export and the IPC mediation between the main and renderer sandbox is rewarded by a maximally fast display of the images.

Overall, the development of the project with the Baumer VLU-02M camera and Electron was an exciting but rewarding experience. The combination of industrial image processing, C++ programming and Electron development offers great potential for innovative applications and solutions. The approach can also be applied to other native data sources and functions, be it for efficient image or signal processing, integration of new hardware resources, complex algorithms or analysing large amounts of data.

We will certainly return to this solution in the future – for products that users love.

Frequently ask questions: