The PERKS Project has announced the second and final release of its Procedural Knowledge Elicitation tools, a suite of technologies that turn scattered, undocumented industrial know-how into structured, machine-understandable procedures. Building on the first release, the new version introduces multimodal AI extraction, a redesigned authoring environment, and a far more flexible human-in-the-loop validation process, all tested in real industrial settings with the
project’s use case partners.
The release covers three complementary solutions: tools for the automatic extraction of procedural knowledge, tools for manual and semi-automatic knowledge collection, and procedural knowledge validation with human-in-the-loop. Together they form a single elicitation ecosystem in which automation proposes and human experts decide, while every artefact remains interoperable with the Procedural Knowledge Ontology adopted across the platform.
project’s use case partners.
The release covers three complementary solutions: tools for the automatic extraction of procedural knowledge, tools for manual and semi-automatic knowledge collection, and procedural knowledge validation with human-in-the-loop. Together they form a single elicitation ecosystem in which automation proposes and human experts decide, while every artefact remains interoperable with the Procedural Knowledge Ontology adopted across the platform.
In industry, procedures live in PDFs, spreadsheets, plant-specific templates, operational data, and above all in the experience of the people who define and perform them. The second release addresses this fragmentation with three sets of advances:
- Multimodal, hybrid AI-based extraction: The end-to-end approach of the first release has been replaced by a modular framework combining document-structure analysis with two interchangeable pipelines — a heuristic one and one driven by large vision-language models. Extraction now covers PDF, Excel, plain text, Markdown and XML sources, supports multiple languages, and links procedure steps to the images and schematics that illustrate them. A dedicated pipeline additionally mines charging procedures directly from time-series sensor data. All models run on-premises, and names and signatures are removed immediately after pre-processing so that later stages only ever work on anonymised data.
- A more capable authoring environment: The procedure editor has been redesigned to provide a more effective interaction model with clearer validation feedback, and now supports deeply nested procedures and cyclic procedures defined through steps and transitions, and template procedures that can be instantiated automatically. Controlled vocabularies such as tools, locks and personal protective equipment are retrieved dynamically from the knowledge management system rather than hard-coded, so approved reference data is always up to date. New visualisation plugins offer table, flow and knowledge-graph views, with print preview and PDF export.
- Validation designed for real approval chains: The two-step workflow of the first release hasbecome a configurable multi-step procedure validation and process running on an on-premise human-in-the-loop platform, keeping confidential procedures inside the organisation. Reviewers assess procedures against generic validation criteria, leave suggestions for the next reviewer in a dedicated sidebar, and consult the full procedure lifecycle while deciding. Crucially, validation no longer stops at approval: feedback reported by operators during procedure execution automatically triggers a review task, which can suspend a procedure, open a revised version, or return an answer to the person who raised the issue.
A deliberate separation of responsibilities underpins the release. Editing now happens only in the authoring tool, while the validation environment is reserved for assessment and approval — a change made directly in response to pilot feedback, which showed that overlapping capabilities confused users and distracted validators from judging procedures as a whole.
“The second release shows what hybrid AI for procedural knowledge really looks like in practice: automation that adapts to the documents, data and languages industry actually uses, paired with validation workflows that respect how safety-critical procedures are approved. Human experts stay in control of every decision, while the tools remove the burden of producing structured, interoperable knowledge.” said Günter Neumann, DFKI, leader of the PERKS procedural knowledge elicitation activities.
Validated across lockout/tagout procedures, CNC commissioning guides and energy microgrid operations, the tools have reached demonstration-level maturity in their intended operational environments. By making elicitation faster, multilingual, auditable and genuinely human-centred, the PERKS Project demonstrates how Industry 5.0 organisations can capture the knowledge their operations depend on — and keep it trustworthy as it evolves.