The following projects are currently active:

M4-SKi – Multi-Modal Man-Machine Interface with Artificial Intelligence

-Technology transfer to SMEs funded by the European Regional Development Fund
-speech large language models, visual question answering, question answering with image responses

Customized implementation of production-related IT systems in enterprise networks (MOONRISE)

In order to successfully meet the challenges of digitization, new types of participative collaboration in existing networks are needed as a response to changing work and production processes. Currently, IT service providers and process consultants for manufacturing SMEs work predominantly according to the project principle, which often leads to high costs, the omission of basic functions and thus to low user acceptance.

The MOONRISE project aims to design a cross-industry, participatory collaboration of manufacturing SMEs, IT service providers and process consultants in corporate networks and to significantly increase the efficiency of the digital transformation with the help of an SME-specific process model.

In the two networks IT-Cluster Upper Franconia and SüdWestSachsen Digital as well as in the participating IT service providers, process consultants and manufacturing SMEs, Open Collaboration is a novel interaction and communication concept that combines the approach of Open Innovation (OI) with the participative instruments of Collaboration Engineering (CE). OI uses new information and communication technologies for intensive collaboration within a network. CE describes a systemic approach to designing efficient collaboration processes in open innovation networks. In this context, an SME-specific process model for the introduction of IT solutions in SMEs and, within the framework of six model projects, small project networks consisting of IT service providers, process consultants and producing SMEs are formed, which introduce various prototype IT solutions in the producing SMEs. 

Due to the various pilot projects and the broadly positioned, manufacturing SMEs in the project, cross-industry use by other SMEs and networks is possible. The prototypical results of the pilot projects lead to actual application benefits for the manufacturing SMEs and sustainable optimizations for the IT service providers and process consultants.

Digital transformation of medium-sized businesses with artificial intelligence (DAMMIT)

In the completed project “Wirtschaft 4.0 im Mittelstand” (WiMit), iisys has already helped numerous companies to take the first steps towards “Industry 4.0”, or even to selectively introduce the first advanced “Industry 4.0” solutions. In the recently approved follow-up project DAMMIT, the focus is on advanced data analyses and evaluations that are made possible with new quality machine learning methods. This includes in particular image and speech processing with deep neural networks, which can be applied in many ways in the industrial context, e.g., in the optical detection of quality defects or speech input for tasks that require free hands. The proven trio of university, application partner (industrial company) and development partner (IT company), who work together on these solutions, will be taken over by WiMit. Also remaining the same are the short durations for projects of 6-12 months and the advantage that there are no additional costs for the partner companies beyond their own personnel deployment. The project runs from April 2020 to September 2022, and we look forward to receiving numerous interested parties from the regional economy.

DAMMIT subproject speech recognition with transformers

Speech recognition has been significantly improved using deep neural networks over the last 3 years. There are now several freely available models that recognize English with good accuracy if sufficient training data is used.

The recognition process is two-step. First, the neural network generates letters from the audio signal, which are then processed by a language model to form the most likely words. Up to now, this second part has often been solved using n-grams. This means that for each letter the 2-3 immediate predecessors and successors are considered. This may be followed by an assignment to words from a dictionary, so that only words in the dictionary are recognized. This process is well suited for a limited vocabulary, but sometimes leads to meaning-distorting changes for unknown words or even names.

In this project, a further current trend in the field of deep learning will be addressed: An improved understanding of texts and sentence contexts by means of so-called transformer models with attention mechanism in general and BERT (Bidirectional Encoder Representations for Transformers) or derived models.

In the use case of the corporate partner Grundig Business Systems (Grundig BS), the focus is on improving speech recognition specifically for non-native speakers (so-called L2 speakers). Since more and more people with a migration background live in Germany and more and more non-native speakers are also represented in the healthcare sector, which is of particular importance to Grundig BS, an improvement in speech recognition would be of great benefit. In addition, existing ASR (Automatic Speech Recognition) systems have great difficulty in meeting this challenge, so a better solution would be welcome at Grundig BS. Moreover, it seems little researched so far, unlike e.g., dialects.

DAMMIT subproject speech synthesis

Reading text aloud is an important feature for modern computer applications. It not only facilitates access to information for visually impaired people but is also a pleasant convenience for non-impaired users. In some situations, such as driving a car, such a function also fulfills safety-related requirements.

The quality of speech synthesis (TTS, Text to Speech) has improved considerably in recent years due to the use of deep neural networks. The tinny sound of earlier TTS systems has given way to a quite natural sound, as international publications show, especially for the English language. The DAMMIT project “Speech Synthesis” will therefore investigate the acceptance of computer-generated voices that read out text content of medium length, e.g. messages with half a screen page length.

The company “ahearo” offers a service where users can listen to content that is otherwise only available as text as an audio podcast. Until now, these have been read aloud by human speakers. However, such production is expensive. However, due to the limited availability of professional speakers, this type of production reaches its limits.

For ahearo and other companies looking for a cost-effective way to integrate high-quality speech synthesis into their products, iisys develops and evaluates a solution for generating German speech from text, based on freely available TTS models. For this purpose, freely available, self-generated audio data provided by ahearo are used to train the TTS models. The evaluation is based on objective values, which can be measured physically, as well as on subjective assessments.

DAMMIT subproject 6D object tracking for virtual reality

Virtual Reality (VR) experiences have been experiencing great popularity for a few years now, after the hype died down in the 1990s and little was heard about VR in public for almost 20 years. This is due to much improved hardware with high-resolution displays that have nevertheless become relatively compact, lightweight and affordable. Studies show that the visual and auditory levels are already very well covered. What is still problematic, however, is the natural interaction with the VR environment and the haptic feedback.

In the DAMMIT project “VR object tracking”, we will therefore investigate whether camera-based tracking of substitute objects from the real world can compensate for the lack of haptic stimuli in the virtual world. Existing studies show that the substitute objects only need to resemble their virtual counterparts in order to appear believable. In the DAMMIT project, a solution with low-cost hardware, i.e., commercially available cameras and computers, will be tested. This should allow the recognition of objects and their position and rotation in 3D space by means of deep neural networks. In order to generate enough data for training the neural networks, VR will also be used. This process serves the purpose of generating photorealistic images of the object in different environments and storing the 3D coordinates to match. These are then used to train the mesh. Subsequently, it will be supplemented with a few otherwise collected data from the real world to further increase the recognition performance.

A high-end installation provided by the company Advanced Realtime Tracking (AR Tracking for short) serves as a comparison for the quality of the own solution, which can be used as a test position in the iisys. In addition, the accuracy is compared with a commercially available Vive tracker using larger objects as examples, since this is not well suited for smaller objects such as a wrench or a caliper due to its own size. However, it can be used for chairs and tables without any problems.

DAMMIT subproject Smart Production Planning and Scheduling (SPPS)

The system developed in the WiMit topic HiCuMES (Highly Customizable Manufacturing Execution System) forms the basis for SPPS. HiCuMES is a flexibly expandable solution for supporting manufacturing with the help of process modeling. However, the planning of the machine allocation still must be done manually and is taken for granted by the system. In the SPPS project, therefore, a planning module is to be developed that takes over the detailed planning of production based on criteria set by the administrator and generates an automatic proposal. This can be adapted by the user via a graphical web-based editor.

Planning covers the entire chain from converting sales orders into production orders, breaking down the ordered products into their bill of materials, determining necessary supplier orders for purchased parts, warehouse orders for stock goods and production orders for components or products to be produced. The production orders are then appropriately combined and distributed to the machines.

Producing companies from the region provide the requirements for the necessary criteria for planning and give feedback on the demonstrator as to whether it is considered fundamentally suitable to cover their previously ascertained requirements.

Ontec Automation GmbH from Naila serves as a development partner and beneficiary of the technology transfer. The system is available to them for further use by their existing and future customers. They can develop it further themselves and have a great knowledge advantage over others, since they are involved in important design decisions already during the development period. They provide feedback on the software architecture, technologies and algorithms to be used during development, so that the further development of the software is easy for them afterwards and the design decisions are made in a practical way.

Cooperation with the Bavarian State Tax Office

The new cooperation between Hof University of Applied Sciences and the State Office for Taxes is initially planned for two years from 1 October 2020. Head of department Michael Glauer and professors Valentin Plenk and René Peinl have planned this initiative. The core of the project is the development of an innovative web interface for sifting through large amounts of data and interactive query options, as well as support in setting up a data lake with Apache Spark.

The processing of the order with the support of the professors Sebastian Leuoth and Richard Göbel is carried out by the research group System Integration at iisys.

Selected student projects:

 Human Pose Estimation for the Control of a VR Avatar (2020).

Virtual reality experiences are perceived as good when they create presence, i.e. the user has the feeling of really being in the virtual place instead of in the real physical environment. There are many aspects that help create this illusion, and even more that easily destroy it. While visual and audio aspects are already simulated quite well with today’s VR headsets, both the interaction with the system and the user’s self-representation through an avatar are still immature. Hand and finger tracking allows for a much more natural way of interacting than pressing a button on the controller. In this student project, research is being conducted to evaluate existing deep neural networks for human pose estimation (accuracy and latency) and to transfer the detected poses to an avatar in a virtual reality experience, so that the user can see not only his hands, but his entire body.


Formula Student – Driverless (2019 & 2020)

Surprisingly for the layman, the students who are involved in the Formula Student and build an autonomous vehicle for the class “Driverless” have to struggle with the same challenges as their fellow students with the AutoMowBot. A LIDAR, stereo camera, (differential) GPS and other sensors are used to determine the position on the track, detect the track boundaries and move forward (Simultaneos Location and Mapping, SLAM). The difficulties lie in the faster speed and the associated higher demands on the computing power for sensor evaluation, especially image analysis. Furthermore, the routing is more demanding and not comparable to the straight-line navigation in the garden. On the other hand, the detection of the boundary is even easier, since different colored cones are used as boundaries.

Detection of free parking spaces via RGB camera (2019)

In many countries, there has long been a parking problem in the larger cities. One of the main reasons for this is that the old infrastructures cannot cope with the rapid increase in car ownership over the last decades. Universities like the one in Hof also face this problem, with many students and staff parking their cars there. A parking guidance system would help ease the flow of cars by showing users the locations of available parking spaces.
Sensor-based parking systems are not suitable for open parking lots because of the high cost.

Instead, an intelligent parking guidance system can be used to determine the number of available parking spaces using object recognition on RGB camera images and display parking information to the minute.

Cost-effectively create compelling VR experiences (2019).

Virtual reality has been a big hype since the Oculus Rift. In contrast to a similar enthusiasm in the 1990s, this time there is more behind it, because in the meantime the technology is much more mature and affordable. However, creating your own content for it, is still very cost-intensive. Therefore, the question arises which aspects of the VR experience are subjectively the most important for the users and which combination of factors and their qualitative expression represents the best compromise between low cost and high quality of the user experience. We explore this trade-off in a large-scale empirical study using the example of bathroom and furniture exhibitions.


Smart Speaker without Cloud (2018)

Amazon Echo or Alexa and Google Home occupy the largest share of the market for voice input in the home or office. All of the users’ voice commands end up in the manufacturer’s cloud. In this way, all users help to further improve the solution, thereby strengthen the technological lead of major US corporations in important fields of technology. We’d rather not know what else is done with the data. The iisys is therefore developing a smart speaker that is based on open hardware and open-source components and operates completely without an Internet connection. The basis is a Raspberry Pi, which is extended with the Matrix Voice Module with Far-Field microphones. The speech recognition (Speech-To-Text) is done by Mozilla Deep Speech. The speech output is done by MARY, the Text-To-Speech System of the DFKI. For faster processing of speech recognition, which is based on a deep neural network with the library Tensorflow, the Intel Movidius Computestick will be used. Various skills give the digital assistant in the smart speaker its capabilities. Planned are playback of locally available MP3 files, answers to everyday questions about people, movies, songs, cities, countries and landmarks extracted periodically from Freebase or DBpedia, control of the smart home via OpenHAB2, geographic information via OpenStreetMap, management of purchases and other lists, as well as reminders of events and calendar entries.

Energy data decomposition (2018)

Based on the energy consumption of the entire household, the consumption of individual typical large consumers such as refrigerators, stoves or televisions is to be calculated. For this purpose, the load profiles of various typical household appliances are trained in order to subsequently recognize these patterns in the overall consumption by means of deep neural networks. The challenge is to obtain the necessary training data, since only few data are publicly available, the household appliances can also be country-specific and the inexpensive measuring devices for private use do not have the necessary temporal resolution of 1 sec.

Completed Grant Projects:

Activity Mining in a Process-Driven Social Intranet (AMiProSI)

In this project, applied for at the BMBF in the funding line VIP+ (Validation of the technological and societal innovation potential of scientific research), the results of SCHub, in particular the activity mining algorithm, are to be examined for their practicality and commercial applicability in different scenarios.

The goal of the project is to create an integrated platform for the collaboration of regionally distributed employees in knowledge-intensive activities, which supports them in the best possible way. Typical knowledge processes are mapped directly, process-oriented and cross-system in software, so that the end user gets the impression of a unified intranet. The system is available as software-as-as-service but can also be operated in the company’s own data center. The core of the innovation is an activity mining algorithm, which makes it possible to generate partially structured processes for document-centered collaboration in the case management standard CMMN by collecting activities in typical information systems (IS) for collaboration (groupware, portal, document management) and to refine these further in sub-processes through users and system suggestions.

The integrated overall system ensures that sufficient context information can be captured to assist in structuring workflows. In AMiProSI, the focus is on document creation processes in which multiple users collaborate to produce a large common result over a long period of time. Examples include detailed bid documents for complex service or development or construction projects, as well as project proposals for internal projects.

Economy in SMEs 4.0 (WiMit), sub-project Digital Manufacturing Workplace (DFAP)

In many companies’ manufacturing operations, information is exchanged on paper between the machine shop and upstream or downstream units. Production orders are printed out and collected by the machine operator. Parameters for tool-specific machine settings and inspection instructions are also often available on paper. Feedback on produced pieces, as well as rejects and quality, is also provided in this way.

In this EFRE technology transfer project, the goal is to develop a highly modular MES or SCADA system that can be quickly adapted to the requirements of the deploying companies and integrated into the existing IT infrastructure in the best possible way. For this purpose, proven digital technologies from the office environment are used to support the processes in production completely electronically and, in addition, to relieve the user cognitively through context-sensitive provision of information. Production orders come digitally to the machine on a robust tablet. The necessary machine settings are proactively supplied. Feedback can be provided digitally. Adapted user interfaces are intended to achieve a high level of usability, especially for production employees who do not work at a PC all day and have the corresponding prior knowledge.

The company Vogler Engineering from Hof is our closest partner as an IT service provider and SME and benefits from the know-how and technology transfer through the university, as the company itself is currently making a turn from development with .NET desktop technologies to web technologies. Conversely, the university benefits from the practical experience of the company Vogler, which specializes in machine connectivity and has already mastered many pitfalls there. Gealan Fenstersysteme, based in Tanna (production) and Oberkotzau (administration), is available as an application partner to test the technologies in tough practical use. The security specialist Software Symbiose from Bayreuth sheds light on network security and performs security audits. Finally, the company Duramentum from Plauen is providing benevolent support for the project. In parallel to the DFAP subproject, the specialist for process improvement in manufacturing companies examines the existing processes for inefficiencies and coaches the production management with regard to change management, so that the affected employees are also taken along on the path to digital production. This relieves the university so that it can concentrate on technology transfer.


WiMit subproject Blockboat – Software License Management for IoT Apps in the Blockchain

The blockchain can be considered as its massively distributed NoSQL database with some interesting features. It is the technology that underlies well-known cryptocurrencies like Bitcoin and Ethereum. However, it can also be used for completely different applications such as cadastre, logistics or identity management. Open source blockchain implementations like Hyperledger Fabric or Hyperledger Burrow allow to compose application-specific features like guaranteed anonymity, transaction security and speed and to eliminate undesired features like high computational complexity and thus energy costs. The goal of the Blockboat project is to develop a solution for software license management that can be used by software producers and software licensees alike to manage their software licenses. To this end, one of the above implementations will be selected and a demonstrator will be developed based on it, showing how software licenses can be created, sold, resold and tracked with it. This will be achieved through the development of smart contracts, but also through adaptations to the basic system itself.

WiMit subproject HiCuMES – Highly Customizable Manufacturing Execution System

The system developed in the WiMit topic DFAP forms the basis for the HiCuMES project. DFAP is relatively rigid and can only be adapted via code changes. For HiCuMES, the system is made more flexible and easily adaptable to the different needs of various project partners via graphical editors. The data schema can be extended by company-specific attributes via an editor. A schema mapper ensures that machine data read in via OPC-UA can be mapped to the application’s internal data schema without programming. In the same way, data from an ERP system (e.g. SAP or ABAS) that is retrieved via XML-based web services can also be integrated into the system.

For each production step, the system offers the possibility of viewing the machine-specific production orders and selecting them for processing, displaying the settings suitable for the product and the machine/tool combination, and either manually entering the results of quality inspection and production progress or having them automatically recorded by machine sensors. By using a workflow management system, any number of production steps can be lined up and graphically modeled on a company-specific basis. With this method, it is also possible to define the process steps after which products ready for sale can be created.

Producing companies from the region provide the requirements for the flexibility of the data and give feedback on the demonstrator as to whether it is considered fundamentally suitable to cover their previously ascertained requirements. Novere GmbH from Hof serves as a development partner and beneficiary of the technology transfer. The system is available to them for further use by their existing and future customers. It can further develop it itself and has a great knowledge advantage over others, since it is involved in important design decisions already during the development period. She provides feedback on the software architecture, technologies to be used and algorithms during the development, so that the further development of the software is easy for her afterwards and the design decisions are made in a practical way. Open source software is used as a basis. The result of the HiCuMES project will be published as open source software and will be available to the general public. A license is chosen that allows commercial reuse (APL v2) and is not viral (in contrast to the GPL).

WiMit object tracking sub-project in the SMART City

Many companies rent out items of considerable value (several hundred euros) and find it difficult to keep track of where they are and whether the contract partners are handling them properly. Using the example of the rental of large sunshades, iisys has developed an IoT (Internet of Things) solution that enables rental companies to determine the location of their equipment at any time in compliance with the German Data Protection Act (DSGVO), as well as to monitor with additional sensors whether there are any disturbing influences, such as excessive wind on the sunshade. The solution consists of a compact and cost-efficient microcontroller unit with GPS sensor and internet connection via onward traffic radio. It is energy-efficient and, despite its compact power supply via a lithium-ion battery, should enable runtimes of one year. It sends its data at intervals of several minutes to a cloud platform, also developed by iisys, which the lender can operate itself or host with a cloud service provider. In the cloud platform, the data is collected, aggregated and visually processed. Simple device management functionality is also integrated. Open-source software serves as the basis. The result of the project will be published as open-source software and will be available to the general public. A license is chosen that allows commercial reuse (APL v2) and is not viral (in contrast to the GPL).

WiMit subproject Exhibition Tours in Virtual

As a technology transfer, iisys has developed a software system (demonstrator) with a procedure description, which is made available to the general public as open-source software. It allows companies with an exhibition, e.g. bathroom exhibition, kitchen studio or furniture store, to create a virtual version of the exhibition with little effort and without programming and to integrate it into their existing website. The basis for this is WebXR, is a newly standardized interface that mediates between browser and virtual reality headset (e.g. Oculus Rift, HTC Vive or Valve Index). However, the virtual exhibitions can also be viewed directly in the web browser without a VR headset. In this case, you have to use the mouse to select the section of the 360° view. The immersion suffers, but there is still an added value compared to normal photos. The experience is then similar to Google Streetview. The advantage of our solution over available commercial systems is that there are no restrictions on the resolutions and formats that can be used. Everything that is playable in the browser can be used. For example, the restriction to a 4k resolution for images is lifted, as is common with commercial offerings, although these are significantly worse in quality than 7k images, which can also be made with inexpensive 360° cameras such as the Insta360 One.

Building automation with energy-efficient control (GeneriS)

Smart Home, Industry 4.0 and Internet of Things are just three of the buzzwords that all describe the same phenomenon: Computers or microcontrollers and sensors are integrated into objects of our daily lives and ensure that comfort and safety are increased, and energy consumption is reduced through intelligent control. The hardware and software required for this is now available in great variety and at low cost. However, there is a lack of skilled workers to bring smart solutions into the homes of consumers, as well as small businesses and municipal facilities, although there is a high potential there due to the mass.

The aim of the ESF-funded knowledge transfer project GeneriS is to train electricians, electrical engineers and heating engineers according to their personal needs and to inform them about products, technologies and concepts in the field of building automation and especially energy-efficient control.

The aim is to provide neutral information about products, possibilities and limits, customer benefits and use cases. Background knowledge on networking and wireless protocols, as well as the functionality and configuration of selected solutions complement the offering. The focus is on open systems that allow a combination of sensors and actuators from different manufacturers. Closed ecosystems of individual manufacturers are only marginally considered. Furthermore, solutions for consumers and smaller company buildings or municipal facilities are predominantly highlighted and only supplementary information on solutions for large company buildings or industrial facilities is provided. The use cases of new building construction and retrofitting of existing buildings are equally considered.
Link to the project website

Social Collaboration Hub (SCHub)

In this BMBF project of the FHprofUnt funding line, an integrated infrastructure was created for the effective support of collaboration in companies, especially for knowledge-intensive activities with regionally distributed employees. This also includes the direct support of business processes, so that, for example, documents are automatically forwarded according to these processes.

In contrast to existing systems, however, the business processes are not to be modeled in an elaborate top-down manner, since this approach is too inflexible and sluggish for the knowledge-intensive work of experts and does not support the variability of the processes with their process variants and exceptions that is necessary here. This project therefore develops a methodology with which workflows can be structured interactively on the basis of automatically generated proposals step by step and realized via the intermediate step of case descriptions in Adaptive Case Management.

For this purpose, existing open-source software (OSS) systems from the areas of portal, document management (DMS), groupware and business process management (BPM) were integrated with each other in such a way that users are provided with continuous support without breaks in the workflow. Link to the project website

The following integration aspects were considered:

• Secure access to all systems after a single login (Single Sign-On, SSO)
• Central search in all systems under one interface (Enterprise Search)
• Cross-system workflows that can be modeled web-based by the user with a graphical editor
• Use of the DMS as a central document repository from the portal and groupware
• Collection of user activities from all systems and central display in the portal to generate awareness while observing data protection rules
• Tight linking of content elements (emails, documents) with workflows, as well as workflow tasks with personal tasks in groupware
• Central administration of users and contact data, as well as expert profiles

Your way to the cloud

In this ESF knowledge transfer project, the topic of cloud computing was processed and trained in several steps and a corresponding knowledge network was established. Guidelines and best practice approaches were made available in an information portal. The centrally collected and prepared information can be commented on, evaluated and supplemented by all participants.

This created a virtual community in which SMEs can exchange experiences with cloud services. In addition to the individual information, virtual courses are also offered, which convey knowledge about current cloud services to the participants with multimedia and didactically prepared learning units. A distinction is made between learning units for potential customers of cloud services and those for software manufacturers who want to make their own information systems cloud-capable. Finally, several cloud services are provided in a demo environment to test the learned knowledge in real-world environments. The focus is on open-source software-based cloud services that represent a typical SME scenario, allowing quick conclusions to be drawn about one’s own situation.

Green Technology Workshop (GTW) Hof – Subproject Energy Data Visualization (EnDaVi)

In the EnDaVi sub-project of the ERDF technology transfer project GTW Hof, various technologies are being developed that allow end consumers to better control and assess their energy consumption. On the one hand, data from the photovoltaic system, solar thermal system, air-source heat pump and hot water boiler are visualized and thus energy flows are presented in a vivid way. The use of web technologies creates a hybrid web app that can be distributed in app stores and feels like a native app on iOS and Android smartphones. The data comes from the web portal of Technische Alternative, whose controllers are compatible with many inverters, heat pumps and water heaters and deliver the data directly to the manufacturer’s cloud after customer approval.

On the other hand, detailed comparisons of the energy consumption of private households are made possible via energy data decomposition. This means that a machine learning procedure is used to identify typical load profiles of large electrical consumers in the household. For example, refrigerators typically start up every 30 -60 minutes, consuming a lot of electricity for a short time and then very little again. Especially at night, when the refrigerator is not open very often, such events can be easily detected. Analogously one learns the profiles of washing machine and dishwasher, television and standard lamp. In sum, instead of displaying the total annual consumption in comparison with average households, one can then provide detailed information on the consumption of the appliances mentioned and compare this with typical consumption of appliances of different sizes and quality classes.