Unified Monitoring and Telemetry Platform for Future Intelligent Optical Networks

With the advancement of machine learning (ML) in network automation, the importance of telemetry and monitoring across entire optical networks is paramount. Traditional, localized monitoring methods are not sufficient to provide the data required to support modern ML-driven network applications that are essential for planning, optimizing and managing large dynamic optical networks. In this paper, a unified monitoring and telemetry scheme is developed that leverages a Kafka-based telemetry pipeline, advanced ML applications and time-series databases implemented on InfluxDB with multiple functional plugins. The developed platform offers a robust and dynamic analytical platform, enabling proactive management of network operations and informed decision-making in real-time. It is shown that the proposed telemetry pipeline facilitates real-time data collection, processing and streaming with a total latency of less than 0.05s, ensuring high levels of performance, reliability, and operational efficiency through advanced telemetry and ML technologies.