PERKS S4 – Procedural Knowledge Auditing

Extracting and using procedural knowledge is crucial in the context of the PERKS project; however, it is also vital to understand where this knowledge comes from, how it was created, how it was validated, and how it is used. Procedural knowledge auditing addresses this need by systematically recording and examining the processes involved in knowledge creation, management, validation, and execution. Auditing supports to verify compliance with specific regulations, standards or objectives.

In PERKS, auditing includes the collection and analysis of metadata related to procedural knowledge workflows. This metadata helps verify the reliability of sources, the integrity of extraction and validation processes, and the traceability of procedural knowledge throughout its lifecycle.

Solution 4 (S4), addresses this need by offering a semantic-based approach collecting audit traces from different PERKS solutions, integrating them into a shared audit knowledge graph and making the data available through an auditing dashboard. This dashboard allows users to query, inspect and explore how procedural knowledge has been created, modified, validated and applied.

S4 captures, transforms and queries provenance information, leveraging Semantic Web technologies. Core functionalities include collecting raw provenance data, transforming it into an integrated graph format, such as RDF, validating the data, and storing it in an audit knowledge graph (KG) aligned with the Procedural Knowledge Ontology (PKO) and Procedural Knowledge Management System (PKMS).

The S4 auditing dashboard provides users with a structured overview of available procedure types, including version information, lifecycle status and access to execution or procedure details. This makes procedural knowledge easier to inspect, compare and trace across its lifecycle.

How S4 Works

A key function of S4 is the collection of audit traces related to procedural knowledge extraction, validation, storage, access and execution. For instance:

• S1, S2 and S3 provide extraction traces stemming from creation, editing and feedback loops.
• S5 provides metadata about stored procedures.
• S6 sends execution traces, including feedback and error reports.

S4 generates trace endpoints that allow other components to submit audit data. Since the incoming audit data may be in different formats, such as JSON or CSV, S4 transforms them into RDF to create an integrated view.

Auditors can analyse the collected audit traces via the auditing dashboard, allowing the execution of prepared queries and generating comprehensive overviews. This is enabled through the close adaption of the PKO ontology into S4, and the pko-audit extension developed in collaboration with Cefriel.

For each procedure execution, the dashboard displays key metadata in a compact card format. This allows auditors to quickly review execution context, status and related procedure details without having to inspect raw trace data manually.

Why S4 Is a Key Building Block

S4 provides essential auditing capabilities that allow auditors to inspect how procedural knowledge was created, edited, validated and applied across the PERKS system. This traceability supports assessing the quality and provenance of procedural knowledge while fostering continuous improvement throughout its lifecycle.

S1 extracts draft procedures → S5 stores them → So collects creation traces

S2 manages the manual procedure capturing → S5 stores them → S4 collects creation traces

S6 collects execution feedback →  S4 records execution, feedback and error repots

S5 handles access requests to procedures → S4 receives procedural metadata from S5

Learn More on S4

Further insights on Solution S4 are available in these publications:

Laura Waltersdorfer, Fajar J Ekaputra, Tomasz Miksa, and Marta Sabou. AuditMAI: Towards an Infrastructure for Continuous AI Auditing. AI-CERT Workshop on AI Certification, Fairness and Regulations at AIROV Conference, 2024. 10.15203/99106-150-2

Laura Waltersdorfer and Marta Sabou. “Leveraging Knowledge Graphs for AI System Auditing and Transparency.” Journal of Web Semantics, Volume 84, Article 100849, January 2025. DOI: 10.1016/j.websem.2024.100849.

Explore the role of provenance questions in defining transparency requirements and supporting accountable AI governance, which provides further conceptual grounding for S4’s auditing approach:

Waltersdorfer, L., Hausler, D., & Auge, T. 2025. Provenance Question-based AI Transparency and Accountable AI Governance. AAAI 2025 Workshop AIGOV. https://openreview.net/forum?id=EFsyy6DqCM&noteId=EFsyy6DqCM

Read more about the vision of combining role-based knowledge validation enabled through S3 and  knowledge auditing in S4 for a knowledge engineering educational scenario:

Tsaneva, S., Waltersdorfer, L., Llugiqi, M. and Sabou, M., 2025. A Transparent and Adaptive AI Assistant for Teaching Knowledge Engineering. https://ceur-ws.org/Vol-4093/Paper1hai.pdf

Research Team

The PERKS Solution S4 has been designed and developed by the WU Wien SemSys research team: Laura Waltersdorfer, Gregor Käfer and Fajar Ekaputra .