Log data is essential for maintaining the health, security and performance of IT systems. It enables IT professionals to make informed decisions and optimize operations. Thus, log analyzer tools are crucial, as they provide the insights needed to make informed decisions, enhance security and ensure compliance, ultimately leading to more efficient and reliable operations.

The Intelligent Log Analyzer (iLA), an AI tool, leverages ML and Deep Learning (DL) algorithms to analyze log files, detect anomalies, identify root causes and recommend fixes. Despite its capabilities, the tool has limitations that require additional effort for deployment across different applications.

A significant challenge in the existing iLA tool is its inability to learn new data without input from SMEs. This limitation arises from iLA’s difficulty in capturing the semantic meaning and relational context from the available complex log data, which varies widely. Consequently, iLA’s prediction accuracy diminishes with new data, as its ML algorithms rely on text similarity to the learned data, lacking the depth of contextual and masked learning. This shortfall necessitates continuous SME feedback to improve the learning process incrementally, even for similar errors occurring in a different context.

This paper addresses the limitation of iLA in understanding the contextual relationship within the data by leveraging transfer learning using the Bidirectional Encoder Representations from Transformers (BERT) language model powered by the transformer architecture. Integrating transfer learning into our error analysis pipeline highlights our commitment to leveraging cutting-edge techniques and methodologies to address the evolving challenges of log analysis in today’s digital landscape.