In the rapidly evolving world, the demand for industrial process automation is on the rise. To achieve efficient production environments, there's a need for automation, especially in the context of Industry 4.0. This article explores the development of Human-Machine Interface (HMI) applications for controlling specific hardware through a Raspberry Pi, emphasizing its potential in industrial settings.
Unlocking the Raspberry Pi's full potential for developing HMI applications for IoT involves a symphony of components, from the nimble Buildroot OS to the versatile OPC UA Server, TCP/IP Server, Serial Interface, RDP, SFTP, Camera, and seamless OS updates. Each element plays a pivotal role in enabling efficient communication, remote access, and adaptability, making it an ideal solution for the challenges of demanding industrial scenarios.
In this example, we start with the following requirements or functionalities that are desired to be achieved
- Integration with External Devices: Efficient communication with external devices for data control and measurement through serial ports.
- Flexible User Interface Design: User-friendly interface allowing customization for diverse production programs, catering to the production of various products.
- Comprehensive Analytics and Testing Tools: Diverse tools available for analytics, calculations, calibration, and testing processes.
- Manual Control Capabilities: Manual control options for each element, providing a high degree of flexibility.
- API Integration for Optimization: Seamless communication with the application via API, optimizing overall system performance.
- Expansion Capabilities: Integration with new measurement equipment facilitated through an API, ensuring scalability.
- Secure Remote Access: Robust security measures, including SFTP, enabling secure remote access for file management.
- Visual Monitoring with Camera Integration: Integration of a camera for remote monitoring of setups and progress, enhancing visibility.
- Remote System Connectivity for Support: Capability to connect remotely for providing support, ensuring efficient troubleshooting and assistance.
HMI Interface
The touch panel interface employs the Raspberry Pi microcontroller, specifically the Compute Module, designed for efficient use in demanding industrial environments. Its SODIMM DDR2 connection type facilitates integration and customization, making it adaptable to various applications.
Customized Operating System
In addressing the need for a multiplatform application with the potential for evolving requirements and enhanced functionalities, one approach is to craft a customized operating system. This strategic decision aims to meet the specific criteria for running the application on the Raspberry Pi Compute module and effectively controlling devices in the plant through the HMI.
Utilizing Buildroot, a collection of Makefiles and patches, serves as a valuable solution. It simplifies and automates the process of creating a complete, bootable Linux environment for embedded systems. Through cross-compilation, it allows the generation of multiple target platforms on a single Linux-based development system. Buildroot facilitates the automatic building of the required cross-compile toolchain, the creation of a root file system, compilation of a Linux kernel image, and the generation of a bootloader for the target embedded system.
This utilization of Buildroot as an image generator for running the operating system in the Compute Module enables the creation of a customized distribution tailored to the project's needs. It includes only the necessary packages and dependencies for running the application, resulting in minimal overhead and a short learning curve
The steps to follow to generate the distro are:
- Download buildroot from https://buildroot.org/
- Access the configuration menu using the make menuconfig command.
When you run the command, a distro configuration menu will be displayed. This menu allows the image to be generated according to the needs and requirements of the system.
To generate the image you must:
- Configure buildroot, including build options and toolchain, bootloader, kernel, package and filesystem type image.
- Configure other components such as the Linux kernel and BusyBox.
On the other hand, it is possible:
- Customize the generated target file system:
- Adding or overwriting files in the target filesystem.
- Modifying or deleting files in the target filesystem.
- Running arbitrary commands before generating the file system image.
- Setting file ownership permissions.
- Adding custom device nodes.
- Add custom user accounts.
- Run arbitrary commands after generating the image file system.
- Add project-specific patches to some package
- Add project-specific packages.
- Once the image parameters are established, execute the make command. By doing this, buildroot will automatically download the packages, compile them and generate the bootable image in the output directory.
- Once the compilation is complete, it is now possible to have a bootable image, customized to your needs, capable of running the desired application.
Window manager
For this application, a bespoke composer is essential, featuring exclusive full-screen window management based on Weston. This composer is responsible for controlling the content displayed on the screen and orchestrating various programs to render on it.
Weston, serving as the composer and display manager, implements the Wayland protocol. This protocol facilitates graphics acceleration using the Raspberry Pi GPU while maintaining minimal resource consumption. Weston is an alternative to X11, the predominant window manager on UNIX systems, known for its higher resource consumption and the complexity involved in GPU implementation.
Concurrently, Wayland serves as the communication protocol enabling direct interaction between window layout managers, applications, and video hardware.
This custom composer is oriented towards isolating the application from its environment. Any application connecting to the composer is presented in full-screen mode, overlaying other elements. This design ensures the uninterrupted functioning of the system, even in the face of potential issues with the application, thereby establishing independence in the management of the display.
Tools and languages
The application is programmable using QtCreator in an object-oriented paradigm. Business logic, databases, and system interfaces can be developed in C++, while the user interface (UI), responsible for on-screen display and user interaction, can be created using QML. Leveraging advantages such as modularity, polymorphism, and inheritance is crucial for effective problem-solving and ensuring future scalability. QML offers several advantages in UI development, owing to its straightforward syntax and compatibility with JavaScript.
As the demand for devices increased, along with the complexity of interconnection and usage within the serial layer of the application, the need for robust software design strategies became apparent. Diversification in device types and communication methods necessitated the implementation of strategies for fluent and efficient use of available serial ports. Hardware design restrictions, often unmodifiable, led to scenarios where communication with 12 devices connected in cascade—each with up to 5 actuators and sensors—had to occur through a single serial port.
To address these challenges, specific design strategies can be employed. A robust message dispatch and reception architecture can be developed, involving an arbitrator determining which device would receive a message. This arbitrator awaited responses from the designated device, translating and dispatching them through a dispatcher to the software block or object managing device-related information.
Given the possibility of multiple devices attempting simultaneous communication through the same serial port, buffers can be implemented to stack commands using the FIFO methodology. This ensured that the first message to arrive was sent to the device, and upon receiving and dispatching its response, the next message in the buffer was processed.
The implementation can involve a thread management strategy to isolate software portions into smaller, independent processes. This led to the establishment of a main thread containing the UI, along with smaller threads distributing Business Logic and data processing.
OPCUA server
To address the imperative for remote system access and control, a standard interface can be implemented to enable users to manage instruments remotely through the application. Choosing from various communication protocols for Industry 4.0 and IoT automation, OPC UA emerged as the preferred tool for this example.
OPC UA, is a robust communication standard, facilitates effective M2M communication over a TCP network in industrial process control and supervision. Operating on Microsoft technology, it provides a common interface for communication, allowing individual software components to interact and share data. OPCUA communication follows a client-server architecture, where the OPCUA server serves as the data source (akin to a plant-level hardware device). Any OPC-based application can access the server to read, write, or subscribe to variables offered by the server, offering an open and flexible solution to proprietary driver issues. Virtually all major manufacturers of control, instrumentation, and process systems have incorporated OPCUA into their products.
Generating the server and embedding it in the application required modeling the system to generate a set of variables and methods. This enabled any OPC client connecting to access and remotely control the devices. Users gain access to available methods and variables, executing specific routines in the application through scripting using an OPC UA client in Python. Additionally, the server can be accessed from any generic software acting as an OPC UA client.
You can see the blog Integrate an OPCUA server into an embedded or PC application using the QT platform
It's worth noting the complexity in communicating with devices due to their varied connection forms. However, connecting the device to an OPC UA client allows direct communication with the application, enabling remote control of the devices through the server. OPC UA signifies a future potential for standardization in device communication. As the project progressed, additional functionalities were incorporated based on emerging client needs.
TCP-IP server
A TCP IP can be enabled or disabled, and via Ethernet it connects to the client's own application that receives data from a specific detection instrument and sends data back to the application to analyze and graph them and , thus being able to analyze the quality of the product obtained and generate algorithms that allow correcting variables and optimizing processes.
SFTP server
These types of human-machine interfaces are sometimes located in places that are difficult to access or where, for security reasons, it is not possible to access the data contained in the panel using a pendrive. In other words, the client needed to be able to securely access system files. For this reason, two methods were implemented to access panel files securely.
The first consisted of support for a defined encrypted pen drive type, which can be added from the operating system to accept file transfers from a specific encrypted memory device.
On the other hand, an SFTP server can be added to the system, running as a separate process in the OS. SFTP (Secure File Transfer Protocol) is a file transfer protocol that is performed encrypted through Secure Shell (SSH), which takes advantage of a set of utilities and provides access to a remote computer to provide secure communications.
SFTP also allows you to view directories, change their name or limit access, which is extremely important, given that the aim was not to give users complete access to all system directories, only those related to the use of the application.
From the application, a username and password must be established that will be added to the system to guarantee remote access and secure file transfer.
RDP
Another functionality that can be added to the system is the capability to access what the panel screen shows via RDP (Remote Desktop Protocol). Therefore, any user can observe, from a remote PC, what is displayed on the HMI screen and control the devices by directly managing what happens on it, as long as they have access to do so.
The ability to access what the panel shows on the screen through the remote desktop generated the possibility of installing multimedia equipment in the HMI, which would allow observing the plant status through the application.
OS update system
When a project of this type is developed and begins to be distributed, the possibilities that begin to open up in the market are broad, giving rise to new needs for functionalities that add value to the product.
Since it is a new product customized to the client's needs, as time progresses, new requirements began to emerge that required adding new features to the project. This implied the need to add dependencies and/or packages to the image created according to previous specifications, already distributed among users. Which represented a challenge that posed the need to add new packages to the previous distro, without having to completely change the image on each user's panel.
This is why an update system has to be created within the application, that would allow the user, when installing a new version of the application on the panel, to update the operating system by adding, overwriting or removing the packages and dependencies that are necessary for the distro.
In order to implement the update system, it is initially necessary to generate a new buildroot image with the packages that you wish to add to the distro. Once this new image is obtained, the files contained in the target of the new distro are compared with those contained in the previous version. In this way, it is possible to get the necessary files, which are compressed and added to the application, along with a script that applies the required changes, such as a new update version of the operating system.
On the other hand, it is necessary to implement a versioning logic and management of operating system updates.
When you start the application, it checks if there is an operating system versioning file present on the panel. If this is not the case, the script corresponding to the first update will be applied and a file will be generated with the name of the recently installed version, and then proceed to reboot the operating system.
If the operating system already contains a file that indicates the current version, the application will compare said version with its own and apply, if it exists, the version immediately higher than the current version of the system, then rebooting the OS. This way, it will check the current (recently installed) version with the ones it contains in its own file tree, installing and rebooting the system until, finally, the latest available version is installed.
This way of updating allows that, despite the OS version a user is working on, any other version of the application to be installable regardless of whether or not it is an application several versions higher than the current one.
Conclusions
All the functionalities mentioned above were applicable thanks to the way in which the update system is generated, allowing new packages or dependencies to be added to the operating system according to needs, making it possible for the system to grow continuously.
Finally, it is important to mention that, in the development of a project of these characteristics, where it is necessary to continually offer value to clients, it is necessary to generate a scalable architecture that allows, through interfaces, to interact with the logic and control units that the system has. So that, ensure that the growth of the project to allow the addition of new functionalities, don’t generate major inconveniences. The use of strategies such as efficient memory management, processes, objects, libraries and architectures result in an indispensable way of working when resources are not excessive.
Daniel Avale
Embedded System Engineer at Emtech S.A
Any Comments or questions, please feel free to contact us: info@emtech.com.ar