Even the most advanced AI systems can introduce inaccuracies or misinterpret context. This makes human validation an essential step in the procedural knowledge management lifecycle. Experts must review automatically extracted knowledge, verify its correctness, and refine it where necessary.

Beyond technical correctness, there is another important dimension: organizational context and compliance. This is important not only for automatically extracted knowledge from Solution S1 (read more about S1 here), but also for procedural knowledge that is manually captures from scratch, as addressed in Solution S2 (read more about S2 here).

In many cases, extracted knowledge may be technically valid but not aligned with an organization’s internal policies, safety rules, or regulatory requirements. For example, a system may extract a procedure for operating a machine that is generally acceptable. However, within a specific company, that same action might be restricted due to safety regulations, certification requirements, or legal constraints.

Developed by WU Wien, Solution S3 introduces human-centric knowledge validation ensuring knowledge accuracy, compliance, relevance, and contextual appropriateness with review based on roles & authority.

How S3 Works

  • A key aspect of S3 is its role-based validation workflow. Knowledge is reviewed and approved by individuals with the appropriate expertise and authority within the organization. For example, technical experts may review and revise procedural correctness, while health & safety officers ensure adherence to regulatory constraints.
  • S3 supports an iterative validation workflow, where new tasks are automatically created for the individuals responsible for the next validation step. As knowledge moves through the workflow, each reviewer can inspect, comment on, request changes, or approve the content. Once a step is completed, the system assigns the next task to the appropriate role, ensuring that validation progresses in a structured and transparent manner until final approval is reached.
  • Another key functionality of S3 is the support of ongoing procedural knowledge management accounting for feedback collected during procedure execution. Authorized users, such as supervisors, can review collected feedback and take appropriate actions. e.g., suspending a procedure and triggering an update.

Why S3 Is a Key Building Block

S3 closes the human-in-the-loop cycle, linking knowledge elicitation with knowledge validation, and continuous improvement after execution.

  • S1 extracts draft procedures → stored in S5 → S3 assigns responsible user to review the procedure  → S2 handles procedure modifications if required.

  • S2 manages the manual procedure capturing  → stored in S5 → S3 triggers a multi-role approval process

  • S6 collects execution feedback → recorded via S4 →  S3 manages the review of feedback by authorised users

  • S5 handles access requests to procedures → S3 manages knowledge access approvals by the procedure owners.

Learn More on S3

Further details on the planning of task-based knowledge validation campaigns and the base reference architecture of Solution S3 can be found in the following publications:

  • Tsaneva, S., Käsznar, K. and Sabou, M., 2022, September. Human-centric ontology evaluation: Process and tool support. In International Conference on Knowledge Engineering and Knowledge Management (pp. 182-197). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-17105-5_14

  • Tsaneva, S. and Sabou, M., 2024. Enhancing human-in-the-loop ontology curation results through task design. ACM Journal of Data and Information Quality, 16(1), pp.1-25. https://doi.org/10.1145/3626960

Read more on semi-automatic knowledge validation patterns in:

  •  Tsaneva, S., Dessì, D., Osborne, F. and Sabou, M., 2025. Knowledge graph validation by integrating LLMs and human-in-the-loop. Information Processing & Management, 62(5), p.104145. https://doi.org/10.1016/j.ipm.2025.104145 

Read more about the vision of role-based knowledge validation enabled through S3 and S4 (knowledge auditing) in a knowledge engineering educational scenario:

Research Team

The PERKS Solution S3 has been designed and developed by the WU Wien SemSys research team: Stefani Tsaneva, Gregor Käfer and Marta Sabou.