Process mining without magic: what event logs are needed for an honest picture
Process Mining algorithms have one dangerous trait: they are absolutely indifferent to the quality of input data. Even on the basis of completely distorted logs, the system will build a visualisation. If timestamps have offsets and parallel processes are recorded linearly, you will get a digital illusion rather than objective analytics. The first launch of the platform often ends in disappointment when, instead of a clear picture of business process optimisation, the system produces a chaotic tangle of lines known as a "spaghetti model".
The success of a Process Mining project depends not on the choice of analytical platform, but on a rigorous technical audit of timestamps and log structure before the implementation starts.
The illusion of accuracy and 27 categories of defects in event logs
When an enterprise initiates a Process Mining project, management expects to obtain an objective picture of its operations. However, analytical software works on the principle of mathematical modelling: it does not know how the process should have occurred according to the regulations, and is unable to independently detect logical anomalies in the input array. Research by the Eindhoven University of Technology proves that model-building algorithms are indifferent to data quality. They will process any provided set of events and generate a diagram, even if it is completely distorted.
To prevent the construction of false models, researchers classify log quality problems into 27 categories that must be checked before integration begins. These defects are grouped into four main classes: missing data (omitted events or mandatory attributes), incorrect formatting (syntax errors in logs), semantic anomalies (events that contradict physical reality, such as completing a task before it starts), and reference problems (incorrect or missing identifiers). Unprepared data from various IT systems always contains omissions, duplicates, and sequence violations. The algorithm will not filter them automatically, but will perceive them as real execution paths of the business process.
Timestamp vulnerability: granularity and time zone offsets
Timestamps are the most critical and, at the same time, the most vulnerable element of any event log. Any error in time automatically destroys the chronological chain on which process reconstruction is based. Data auditing practice reveals two main timestamp problems: insufficient granularity and time zone offsets between different systems.
The granularity problem arises when a system records events with an accuracy of only a day or a minute. If several operations are performed within one minute, the algorithm will not be able to determine their correct sequence. It will arrange them in a random order, creating artificial cycles. No less destructive is the time zone offset. When an ERP system records transactions in UTC, while a CRM system logs customer interaction in local time, attempting to merge these logs leads to the customer response chronologically preceding the receipt of the order.
- Verifying the uniqueness and end-to-end consistency of Case IDs in adjacent databases.
- Analysing timestamp granularity (minimum requirement is millisecond accuracy).
- Synchronising time zones (bringing all logs to a single UTC standard).
- Detecting and filtering automatic system events that are not user actions.
- Checking the completeness of event attributes (mandatory presence of data on performers).
- Analysing logs for duplicate transactions caused by technical network failures.
Limitations of Directly-Follows Graphs (DFG) in parallel tasks
Most commercial Process Mining tools use Directly-Follows Graphs (DFG) by default. This approach is simple to implement: if event B is recorded in the logs immediately after event A, the algorithm draws an arrow from A to B. However, in real corporate systems, this approach creates serious distortions due to the parallel execution of tasks.
When two process steps are performed by different specialists in parallel and independently of each other, they will still be recorded sequentially in the event log. The DFG algorithm interprets this as a rigid dependency and will generate a non-existent connection between them. On large datasets, this leads to the accumulation of hundreds of false transitions. To overcome this limitation, architects apply more complex algorithms, such as the Inductive Miner, which are capable of recognising concurrency, though their results are much harder for business users to interpret.
| Comparison criterion | Directly-Follows Graphs (DFG) | Inductive Miner algorithm |
|---|---|---|
| Sensitivity to parallel tasks | Low (creates false sequential connections) | High (correctly identifies concurrency) |
| Complexity of business interpretation | Low (intuitively understandable flowcharts) | High (mathematical models, process trees) |
| Propensity to form "spaghetti" | Very high on unprepared data | Low due to rigid structuring |
Data flattening: transition from case-centric analysis to OCPM
Classic Process Mining is based on a case-centric approach, where each event must have one unique process identifier (Case ID). However, real business processes are rarely linear. In a single procurement process, one order may contain ten different product items, for which three separate invoices are issued and five deliveries are arranged.
Attempting to "flatten" such multidimensional relationships into a single Case ID leads to artificial duplication of events or loss of context. If the order is chosen as the identifier, invoice payment events will be duplicated for each item. The object-centric approach (Object-Centric Process Mining) solves this problem by allowing a single event to be linked to multiple objects simultaneously. However, implementing OCPM requires significantly more complex data engineering and high maturity of the IT infrastructure.
- Data multidimensionality — the ability of IT systems to support "one-to-many" and "many-to-many" relationships without losing integrity.
- Unification of identifiers — the presence of end-to-end IDs for business objects across all systems.
- Repository performance — the readiness of databases for high loads when executing complex OCPM queries.
- Maturity of ETL processes — the availability of tools for extracting, transforming, and loading complex object models.
Architectural requirements for data sources and environment configuration
To build an honest picture of processes, the primary task is to create a reliable data source at the enterprise architecture level. Systems must have built-in audit mechanisms that record every change with absolute accuracy. For example, the UnityBase platform implements this approach through a built-in logging module. Thanks to its model-driven architecture, the system automatically generates structured and consistent event logs, which eliminates the problem of data "flattening" and provides a foundation for accurate analysis without the need for complex log post-processing.
In addition to the quality of the logs themselves, integration with Process Mining platforms requires precise configuration of authorisation and secure data access. According to the technical requirements of leading vendors, log extraction must occur via secure APIs using access tokens that limit data visibility only to the attributes necessary for analysis. This prevents the leakage of sensitive commercial information during the transfer of event arrays to the analytical environment.
Frequently Asked Questions
What should be done if a legacy ERP system does not support millisecond timestamp granularity?
In such cases, it is necessary to implement intermediate ETL procedures that artificially aggregate simultaneous events into logical macro-steps, or to use algorithms resilient to partial loss of order, although this will reduce the overall accuracy of the model.
Can DFG algorithms be used to analyse processes with a high degree of concurrency?
Using DFG for highly parallel processes is possible only after deep data preprocessing, which includes filtering out insignificant transactions. Without this, DFG will inevitably generate false sequential connections.
What are the technical obstacles when transitioning to OCPM in legacy systems?
The main obstacle is the lack of end-to-end identifiers for business objects and the inability of legacy databases to efficiently export multidimensional relationships. This requires the development of complex procedures for object mapping before loading them into the platform.
