The budget will be split in a balanced way between area Type A and Type B defined below. Proposals should clearly identify the area they are addressing.

Proposals aiming for Type A outcomes should adhere to the Type A scope, while proposals aiming for Type B outcomes should follow the Type B scope.

Type A Scope: While it is widely acknowledged that current use of generative AI has the potential to impact certain tasks in robotics such as improving user interaction or providing explanations about why a robot system made a particular decision, these are, in general, not within the critical operating flow of a robot. To reach next level of autonomy, generative AI must also enable robots to learn from their experiences, simulate realistic environments for training in challenging conditions, and enhance planning, decision making and control while considering the physical constraints imposed both by the environment and by the physical construction of the robot. This includes integrating ‘Human-in-the-loop’ mechanisms, where AI systems collaborate with human operators to enhance decision-making processes and adaptability, particularly in dynamic environments.

This represents a significant advancement in robotics, requiring the development of AI models that can effectively navigate the complexities of the physical world while ensuring safety. Generative AI is expecting to bring such a step-change in robots precision, adaptability, versatility and robustness, enabling them to efficiently achieve real world tasks such as complex moves (navigation, manipulations, etc.) with higher level of autonomy and precision.

In the context of advancing robotics capabilities, the use of generative AI stands as a transformative force, amplifying robots’ learning, interaction, and operational abilities. By enabling robots to learn from experiences, simulate diverse environments for training, and enhance human-robot interaction, it drives adaptability and efficiency. Additionally, generative AI facilitates the augmentation of robot situational awareness and planning capabilities, empowering them to predict outcomes of various actions, thereby elevating their autonomy and decision-making prowess.

Training current generative AI models, in particular Large AI models, requires high volumes of data to achieve effective levels of performance. The vast amount of data required present a significant challenge when it comes to robotics. Further research is necessary to find the appropriate balance between the quality, adequacy, and volume of data with regards to the performance of the AI model. Moreover, model distillation techniques may play a key role for the portability of the generative AI solution at the edge, in power-limited devices. The training data should come from the real world or from physical aware simulations of the real world. Where relevant, in particular in the context of human interaction, training data should encompass diverse individual characteristics, such as gender, age, racial and ethnical background, to mitigate potential bias and discriminations.

Proposals should detail strategies to leverage cutting-edge generative AI techniques to enhance the adaptability and reliability of these models across complex and dynamic scenarios, as well as how to ensure human-centricity and environmental considerations. The goal is to train and fine-tune generative AI models that meet the necessary standards for ensuring the safe operation of robotics hardware. These models should empower robots to autonomously plan and execute actions while maintaining high levels of performance and generalization capabilities.

Research activities should explore the training methodologies for these foundation models, emphasizing their ability to process multimodal data and derive actionable insights to inform robotic decision-making processes.

The proposals are also expected to include the validation of the trained models through applications. Proposals should detail methodologies for conducting rigorous testing procedures, incorporating both simulation-based evaluations and physical experiments. These tests aim to evaluate the performance and scalability of developed foundation models.

The research will be driven by impactful scenarios defined by major manufacturing industry players who should be well integrated in the consortium. They should be deeply involved in the proposed work in order to provide the use-case, the corresponding data and they will play an important role to accompany the validation process. They will define a number of representative real-world use-cases with gradually increased level of complexity to drive the technology development. They will provide existing relevant data and collect further data necessary to train and fine-tune the models, but also to validate the solutions. Given the sensitivity of sharing industrial data, manufacturers present in the consortium have to define upfront mechanisms to collectively provide and pool a sufficiently large dataset for training the models (this might involve a trusted third party as intermediary), ensuring sufficient quality and quantity of data needed to train the models. If necessary, they will have to put in place mechanisms to acquire data from sources outside the consortium.

Proposals are expected to enhance the accuracy and robustness of generative AI systems in robotics, ensuring that the solutions developed are trustworthy and reliable in their applications, hence in line with the AI Act requirements.

Proposals should address both the safety of robotic operations, ensuring protection against physical risks, and cybersecurity measures to safeguard against digital threats and ensure system integrity.

The emphasis lies in creating and disseminating general-purpose models and tools rather than being limited to narrowly focused solutions. Projects should also build on or seek collaboration with existing and upcoming projects and develop synergies and ensure complementarities with other relevant European (e.g. projects funded under HORIZON-CL4-2024-HUMAN-03-01: Advancing Large AI Models: Integration of New Data Modalities and Expansion of Capabilities), national or regional initiatives, funding programmes and platforms.

Type B Scope:

The objective is to enhance productivity and provide a competitive advantage to EU industry in the transition towards more sustainable, zero-carbon production, addressing the uncertainties and tensions on supply chains and the lack of highly-skilled workers. A new generation of digital technologies will integrate generative Artificial Intelligence, robotics, and advanced human interfaces in industry-grade applications with a high degree of autonomy. This will enable the development, production, and operation of complex and advanced high-tech products at lower cost while improving sustainability and flexibility, ultimately becoming a powerful tool for accelerating innovation in both processes and products.

The manufacturing sector should strongly benefit from increased levels of automation made possible by breakthroughs provided by AI, in particular by the family of technologies know as generative AI, including (e.g.) AI foundation models, large language models, transformers, multimodal generative AI. The main objective of this Type B is the development of Generative AI solutions dedicated to the manufacturing sector and making use of manufacturing data available in production lines.

Proposals should address at least one of the following use-cases:

1) Robustness and trustworthiness of digital technologies and data management at industry-grade quality, to raise the automation levels on production sites and across industry and supply chains;

2) Enhanced product and process qualification/certification and compliance assessment through higher levels of automation, digitalisation and data management, taking into account related requirements;

3) Automation of manufacturing processes to achieve higher reliability, efficiency and sustainability;

4) Automated tools for fast and large-scale deployment and reconfiguration of production assets and for rapid innovation cycles.

Proposals should accomplish these objectives exploiting the most suitable approach(es) among the ones described below:

• The integration of applications exhibiting advanced developments of generative AI model(s) specifically designed for manufacturing, providing measurable advantages in one of more of these key areas: manufacturing cost, increased productivity, quality, flexibility, resilience, sustainability, circularity, time to market and usability. Applications can target factory-floor operations and/or management of data, knowledge and documentation associated to products and production (for use-case 1 or 2);
• Development and integration of digital production systems capable of significantly increasing productivity and managing high-mix production with close to zero time needed for re-purposing and capability to manage different mixes of materials and components (for use-case 3);
• Development of deployment tools to automate the management of production lines, namely through automatic configuration, integration with legacy systems, placement of data translators and connectors, and deployment of machines and sensors on the shop floor (for use-case 4).

Proposals should indicate which approach they are targeting. Proposals may combine several approaches above, indicating which is the main approach, provided there is added value in such a combined approach; arbitrary combinations without integration are excluded.

The use of generative AI techniques is encouraged for all the approaches. The applicants will specifically describe how they will secure the acquisition of quality manufacturing data from real-world industrial use cases of industry partners or companies outside the consortium in the context of the data volume necessary to train and finetune the models used in the proposal.

Type A and Type B

For both Type A and Type B projects, proposal should allocate up to EUR 30 million towards the development of the foundation model. Each project is anticipated to focus on up to six use cases.

A minimum of EUR 10 million of the proposal budget must be allocated via FSTP for the fine-tuning phase. This phase aims to create Generative AI applications tailored to impactful industry-driven use cases.

• FSTP may be foreseen for up to EUR 2 million per use case, either for a single company (including SME/Start-up), user industry providing their data and use-case, or to a small consortium complementing such user industry company with one or two additional partners, such as AI developer/integrator. Such FSTP initiatives will develop mini-projects, working in close collaboration with the consortium partners, that will dedicate sufficient resources to support such FSTP projects, in order to develop advanced applications and demonstrate with quantitative KPIs the power of Generative AI solutions. These mini-projects will include data preparation, fine-tuning, validation of the Generative AI solution in the selected impactful use-cases.

Proposed projects should aim to develop models that align with European values and principles and regulation, including the AI Act. Research should build on existing standards or contribute to standardisation, particularly addressing the needs and requirements of the industry.

Where relevant, interoperability for data sharing should be addressed, focusing on open specifications and standards, enabling effective cross-domain data communities, and new data-driven markets.

If high computing resources are necessary, for both Type A and Type B proposals the primary source of computing resources for pretraining should be sought from external high-performance computing facilities such as EuroHPC or National centres. The proposal should describe convincingly the strategy to access these computing resources.