A Framework for LLM Agent-Assisted Network Management with Human-in-the-Loop

1.1. Motivation

Traditional network automation systems often face challenges in handling unanticipated scenarios and managing complex multi-domain data dependencies. Large Language Models (LLMs) offer advanced multimodal data comprehension, adaptive reasoning, and generalization capabilities, making them a promising tool for network management and autonomous network[TM-IG1230]. However, full automation remains impractical due to risks such as model hallucination, operational errors, and a lack of accountability in decision-making[Huang25]. This document proposes a structured framework that integrates LLMs into network management through human-in-the-loop collaboration, leveraging their strengths while ensuring oversight, reliability, and operational safety.¶

1.2. Problem Statement

Network management faces significant challenges, including the complexity of multi-vendor configurations, the real-time correlation of heterogeneous telemetry data, and the need for rapid responses to dynamic security threats. LLMs offer a promising approach to addressing these challenges through their advanced multimodal data understanding and adaptive reasoning capabilities. However, the direct application of LLMs in network management introduces several technical considerations. These include the need for semantic enrichment of network telemetry to enhance LLM comprehension, a decision execution mechanism with confidence-based escalation, and auditability of LLM-generated decisions through provenance tracking. Addressing these requirements is critical to integrating LLM agents effectively into network management workflows while maintaining reliability, transparency, and interoperability.¶

2.1. Acronyms and Abbreviations

• LLM: Large Language Model¶

• RAG: Retrieve Augmented Generation¶

• MCP: Model Context Protocol¶

• A2A: Agent-to-Agent Protocol¶

3.1. Enhanced Telemetry Module

The Enhanced Telemetry Module improves the semantics of raw telemetry data, providing supplementary information to the LLM agent decision module for more informed decision-making. Telemetry data retrieved from network devices via NETCONF[RFC6241], e.g., in XML format, often lacks field descriptions, structured metadata, and vendor-specific details. Since this information is not included in the pre-trained knowledge of LLMs, it can lead to misinterpretation and erroneous reasoning. To address this limitation, an external knowledge base should be introduced to store YANG model schema, device manuals, and other relevant documentation. The Enhanced Telemetry Module functions as middleware between the network management system and the external knowledge base. Through its southbound interface, it retrieves NETCONF data from the NETCONF client of existing network management system. Through its northbound interface, the module queries the external knowledge base for the corresponding YANG model or device manual. To enhance semantic richness, the Enhanced Telemetry Module processes the retrieved data by simplifying formatted content (e.g., removing redundant or closing XML tags) and appending XML path and description information from the YANG tree to the relevant fields. This approach ensures that the LLM has access to structured, contextually enriched data, improving its ability to analyze and reason about network telemetry.¶

3.2. LLM Agent Decision Module

3.3. Config Verification Module

3.4. Operator Audit Module

The Operator Audit Module provides a structured mechanism for human oversight of LLM-generated configurations before deployment. The output from the LLM Decision Module should include both the generated configuration parameters and an associated confidence score. The configuration parameters are validated for compliance with the YANG model and are subject to Access Control Rules enforcement. The confidence score, e.g., ranging from 0 to 100, serves as a reference for operators to assess the reliability of the generated configuration. Each audit process must track the input context (e.g., input data, RAG query content, model selection, configuration files) and the corresponding output results. The auditing steps are as follows:¶

• Result Verification: The operator verifies the LLM-generated output to ensure alignment with business objectives and policy requirements.¶

• Compliance Check: The operator ensures that the LLM output adheres to regulatory standards and legal requirements.¶

• Security Verification: The operator examines the output for potential security risks, such as misconfigurations or vulnerabilities.¶

• Suggestions and Corrections: If issues are identified, the operator documents the findings and proposes corrective actions.¶

Upon completing the audit, the system maintains an audit decision record to ensure traceability of operator actions. The audit record includes the following information:¶

• Timestamp of the audit action¶

• LLM Task Agent ID associated with the action¶

• Operator decisions, including approval, rejection, modification, or pending status¶

• Executed command reflecting the final action taken¶

• Operation type (e.g., configuration update, deletion, or execution)¶

This structured approach ensures that all LLM-generated configurations undergo rigorous human review, maintaining operational accountability, and security.¶

The Operator Audit Module is enhanced with Explainability functionality to improve transparency and trust in LLM-assisted decision-making. This enhancement ensures that operators can understand the reasoning behind LLM-generated configurations, verify their reliability, and take corrective actions when necessary.¶

To achieve Explainability, the module provides a structured breakdown of the decision-making process by capturing key influencing factors, justification logs, and confidence scores. Each LLM-generated configuration must include a detailed rationale explaining why a particular decision was made. For example, if the system recommends increasing bandwidth allocation, the decision log should indicate whether this was due to high latency detected in telemetry, SLA thresholds being exceeded, or other contributing factors.¶

Additionally, the audit process incorporates counterfactual analysis, allowing operators to assess alternative outcomes. For instance, the system may indicate that if no action is taken, packet loss is expected to increase by 20% in the next ten minutes. This capability enhances operational decision-making by providing a comparative assessment of different actions.¶

To further ensure reliability, the module includes a mechanism to detect potential biases or hallucinations in LLM-generated outputs. If an LLM agent decision is based on incomplete or uncertain data, the system flags it accordingly, allowing operators to make informed judgments. For example, if real-time telemetry data is insufficient, the system may indicate that the confidence in a particular recommendation is low.¶

By integrating these Explainability features, the Operator Audit Module strengthens human oversight, ensuring that LLM-generated configurations align with business policies, security requirements, and regulatory standards. This structured approach enhances accountability and mitigates risks associated with automated decision-making.¶

4.1. DDoS Intelligent Defense

Distributed Denial of Service (DDoS) attacks remain a critical operational threat. While conventional systems use rate-limiting and signature matching, they often fail to adapt in real-time or generate fine-grained filtering rules based on multi-dimensional telemetry data.¶

This use case demonstrates how the LLM agent-assisted framework enables intelligent filtering rule generation and secure deployment with human oversight.¶

1. Telemetry Collection and Semantic Enrichment The Enhanced Telemetry Module retrieves real-time traffic statistics and interface metrics from network devices using NETCONF. These raw telemetry data are semantically enriched by referencing YANG models and device-specific documentation to generate a context-rich dataset suitable for LLM processing.¶

2. DDoS Filtering Task Instantiation The operator initiates a ddos-mitigation task. The Task Agent Module selects a security-specialized foundation model and task-specific prompt.¶

   It first analyzes the following conclusions: - Interface GigabitEthernet0/1 receiving sustained traffic > 100,000 pps - 95% of incoming packets are TCP SYN - Top source prefixes identified as: IP1/24, IP2/24¶

   The RAG Module supplements the analysis with: - Cisco ACL syntax documentation - Prior incident response templates¶

3. LLM-Generated Firewall Configuration Output The LLM agent reasons that a TCP SYN flood is underway and generates the following ACL-based filtering policy:¶

ip access-list extended BLOCK-DDOS
deny tcp IP1 0.0.0.255 any
deny tcp IP2 0.0.0.255 any
permit ip any any

interface GigabitEthernet0/1
ip access-group BLOCK-DDOS in

This configuration includes both source-based filtering and a global SYN rate-limiter. It is passed to the Config Verify Module for syntax validation and to the Access Control Module for permission checks.¶

1. Operator Audit and Decision The Operator Audit Module provides the following metadata:¶

• Task Agent ID: ddos-mitigation-task-01¶

• Confidence Score: 85/100¶

• RAG Context: Cisco IOS ACL Syntax, Internal Threat List v5¶

• Input Summary:¶

  • Affected Interface: GigabitEthernet0/1¶

  • Malicious Sources: IP1/24, IP2/24¶

The human operator performs the audit with the following steps:¶

• Result Verification: ACL syntax matches Cisco IOS 15.6 standards.¶

• Compliance Check: Confirmed that both prefixes appear in the enterprise blacklist.¶

• Security Review: Low false-positive risk for blanket deny of SYN traffic from identified prefixes.¶

• Audit Record:¶

  • Timestamp: 2025-07-16T17:03:00Z¶

  • Decision: Approved¶

  • Action: Apply via NMS interface¶

  • Operator Note: “Confirmed match with active threat list. Monitor for collateral traffic drops.”¶

The configuration is deployed through the network management system, completing the defense workflow with human oversight and traceability.¶

This simplified case shows how the LLM agent can assist in quickly generating actionable filtering rules in response to DDoS patterns, while adhering to operational safeguards.¶

4.2. Traffic Scheduling and Optimization

In large-scale networks, dynamic traffic scheduling is required to adapt to fluctuating loads, ensure QoS, and meet SLA requirements. Traditional methods may lack responsiveness or cross-domain visibility.¶

The proposed framework enables intelligent, context-aware traffic scheduling through LLM agent-based reasoning. The following illustrates the process:¶

1. Telemetry Data Acquisition The Enhanced Telemetry Module gathers link utilization, queue occupancy, and delay metrics from multiple routers. The semantic enrichment process tags each metric with human-readable labels from the YANG model, including path topology and policy tags (e.g., gold/silver/bronze service classes).¶

2. Optimization Task Execution An operator initiates a traffic scheduling task. The Task Agent Module selects a foundation model fine-tuned on traffic engineering datasets and uses a structured prompt to describe current constraints: high utilization on core links L1-L3, SLA violations for gold-class VoIP traffic.¶

3. Sample Configuration Output The LLM agent suggests adjusting RSVP-TE path metrics to reroute gold traffic via underutilized backup paths:¶

policy-options {
    policy-statement reroute-gold {
        term gold-traffic {
            from {
                community gold-voip;
            }
            then {
                metric 10;
                next-hop [IP];
            }
        }
    }
}
protocols {
    rsvp {
        interface ge-0/0/0 {
            bandwidth 500m;
        }
    }
}

This configuration is syntax-validated and checked against the Access Control Module to ensure that traffic engineering policy updates are permitted for this task agent.¶

1. Operator Audit and Decision The Operator Audit Module presents:¶

• Confidence Score: 91/100¶

• Audit Metadata:¶

  • Task Agent: traffic-opt-te-v2¶

  • Input Context: “Link L1 (95% utilization), gold-voip SLA latency breach”¶

  • Recommendation: “Increase TE path metric to reroute VoIP via L2 backup path”¶

The operator reviews:¶

• Result Verification: Simulates expected path shift via NMS.¶

• Compliance Check: Confirms SLA routing rules permit TE adjustment.¶

• Security Review: Confirms backup link L2 is secure and isolated.¶

• Final Action: Approved with modification: “Set bandwidth cap to 400m on backup to avoid overuse.”¶

The revised configuration is stored and forwarded to the network management system for application.¶

This use case illustrates how the framework enables LLM agents to propose adaptive, policy-compliant traffic engineering strategies while maintaining operator control, traceability, and auditability.¶

7.1. Normative References

[RFC6241]

Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., and A. Bierman, Ed., "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, <https://www.rfc-editor.org/rfc/rfc6241>.

[RFC8341]

Bierman, A. and M. Bjorklund, "Network Configuration Access Control Model", STD 91, RFC 8341, DOI 10.17487/RFC8341, March 2018, <https://www.rfc-editor.org/rfc/rfc8341>.

7.2. Informative References

[a2a]

"Announcing the Agent2Agent Protocol (A2A)", July 2025, <https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/>.

[Hu22]

Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and W. Chen, "LoRA Low-Rank Adaptation of Large Language Models", n.d..

[Huang25]

Huang,, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., and T. Liu, "A Survey on Hallucination in Large Language Models Principles, Taxonomy, Challenges, and Open Questions", n.d..

[Lewis20]

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., and S. Riede, "Retrieval-augmented generation for knowledge-intensive NLP tasks", n.d..

[mcp]

"Model Context Protocol", July 2025, <https://modelcontextprotocol.io/introduction>.

[TM-IG1230]

McDonnell, K., Machwe, A., Milham, D., O’Sullivan, J., Niemöller, J., Varvello, L. F., Devadatta, V., Lei, W., Xu, W., Yuan, X., and Y. Stein, "Autonomous Networks Technical Architecture", February 2023.

Appendix A.1 Data Model

This section defines the essential data models for LLM agent-assisted network management, including the LLM agent decision response and human audit records.¶

module: llm-response-module
  +--rw llm-response
     +--rw config?         string
     +--rw confidence?     uint64
     +--rw error-reason?   enumeration

module llm-response-module {
  namespace "urn:ietf:params:xml:ns:yang:ietf-nmrg-llmn4et";
  prefix llmresponse;
  container llm-response {
    leaf config {
      type string;
    }
    leaf confidence {
      type uint64;
    }
    leaf error-reason {
      type enumeration {
        enum unsupported-task;
        enum unsupported-vendor;
      }
    }
  }
}

module: human-audit-module
  +--rw human-audit
     +--rw task-id?      string
     +--rw generated-config?   string
     +--rw confidence?         int64
     +--rw human-actions
        +--rw operator?          string
        +--rw action?            enumeration
        +--rw modified-config?   string
        +--rw timestamp?         yang:date-and-time

module human-audit-module {
  namespace "urn:ietf:params:xml:ns:yang:ietf-nmrg-llmn4et";
  prefix llmaudit;
  import ietf-yang-types { prefix yang; }

  container human-audit {
    leaf task-id {
      type string;
      }
    leaf generated-config {
      type string;
      }
    leaf confidence {
      type int64;
      }
    container human-actions {
      leaf operator {
        type string;
        }
      leaf action {
        type enumeration {
          enum approve;
          enum modify;
          enum reject;
          }
        }
      leaf modified-config {
        type string;
        }
      leaf timestamp {
        type yang:date-and-time;
        }
    }
  }
}