Delay-Aware Reinforcement Coding for Multi-Hop Narrowband Wireless Mesh Networks: Joint RTT, Redundancy, and Forwarding Optimization

Practical narrowband telemetry networks exhibit a basic but difficult coupling: message size determines serialization time, hop count multiplies service and recovery exposure, packet loss drives retransmission, and queue pressure magnifies the upper RTT tail. Earlier practical and analytical studies quantified these effects in wireless mesh, narrowband, network-coding, and Internet-of-Things settings.^[1–7] The operational question addressed here is therefore not whether fast-forward switching or network coding can help in isolation, but when each mechanism should be activated as conditions change. Classical RLNC and practical wireless coding provide strong foundations for adaptive coded transmission,^[8–11] while Q-learning provides the basic learning mechanism.^[12] Contextual bandit methods provide a myopic adaptive comparator,^[13] and CVaR and retransmission-timer concepts motivate explicit treatment of tail latency and recovery timing.^[14,15] The broader routing, queueing, and recent reinforcement-learning literature is reviewed in Section 2. These foundations reinforce two requirements that are central here: the learning problem must be tied to an explicit system state transition, and the comparison set must include adaptive — not only fixed — baselines.

A contextual bandit is sufficient when an action influences only the immediate reward. DARC instead models queue/load evolution as action dependent: a coding/forwarding choice changes the useful service rate and hence the probability that the next load state rises, remains stable, or falls. This converts the controller into a finite MDP. The contextual UCB rule of Auer et al.^[13] is nevertheless retained as a strong myopic learning baseline, allowing the data to reveal whether long-horizon state coupling materially changes the decision.

Research question. Can a compact reinforcement-learning controller jointly adapt forwarding mode, RLNC generation size, and redundancy to reduce tail latency while maintaining reliability and useful goodput under changing narrowband multi-hop network conditions?

Methodological novelty boundary: DARC does not introduce a new RLNC coding theorem or a new Q-learning update. Its contribution is the joint, interpretable, state-aware control architecture: forwarding mode, RLNC generation size, and coded-packet redundancy are selected together from an 81-state context, the action affects the subsequent load/queue transition, and the objective explicitly combines CVaR95 tail latency, residual loss, coding overhead, and useful-goodput deficit. This differs from fixed coding or forwarding, RL-based coding that does not jointly adapt forwarding in the present comparison, and a contextual-bandit controller that omits next-state value.

The main contributions are as follows:

• An 81-state representation that includes packet loss probability, load/queue pressure, hop count and message size while retaining a compact 972-entry Q-table.
• An action-dependent transition model in which the useful service rate affects the next queue/load state, providing an explicit P(s_t+1|s_t,a_t) and a defensible MDP/Q-learning formulation.
• A CVaR95 tail-latency estimator using a final 256-packet window, 256-packet warm-up and 32-packet update stride; the choice is justified from the number of samples available in the 5% tail and is combined with residual loss, coding overhead and goodput deficit after every term is normalized to [0,1].
• An explicit RLNC model over GF(2⁸), including finite-field rank probability, coefficient-header overhead, source-only coding, generation-level cumulative ACK and rank-deficiency recovery.
• A stronger benchmark suite: store-and-forward, fast-forward, reliability-biased RLNC, adaptive heuristic, contextual UCB, greedy one-step control and an exact value-iteration MDP oracle.
• Multi-seed learning-stabilization diagnostics over 120,000 transitions, confidence intervals, a full learned-policy atlas, relative p95 gains, dual-axis-free ablation, message-size sensitivity, Pareto analysis and objective-weight sensitivity.
• A reproducibility package with code, transition and reward matrices, learned policy arrays, CSV data, and an empirical-calibration template.

For traceability, Table 1 positions DARC against representative alternatives. The closed-loop MDP architecture is defined in Fig. 1, and Algorithm 1 gives the online controller. Table 2 lists the state-bin boundaries, and Table 3 summarizes the state, action, learning, normalization, and transition parameters. Fig. 2 shows the results of the multi-seed learning diagnostics; Fig. 3 shows all 81 learned decisions; and Figs. 4–10 present the distributional, scaling, robustness, ablation, message-size, Pareto, and weight-sensitivity results. Tables 4–7 summarize representative baseline performance, exact discounted-policy performance, a dynamic-workload stress test, and novelty-to-evidence traceability.

Fig. 1: DARC closed-loop MDP architecture. The measured packet loss, queue/load pressure, route length and message size form the state; the selected coding/forwarding action determines the immediate performance and influences the next load/queue state.

Earlier narrowband experiments by the authors,^[1,2,5,6] together with independent studies of forwarding, network coding, capacity, and delay in multi-hop wireless networks,^[3,4,7] measured or analyzed how RTT/latency and capacity vary with message size, route length, forwarding behavior, and coding/redundancy. In the present study, the authors' prior findings are used only to choose physically meaningful state variables and operating ranges; the independent literature provides the broader comparison. Work on RLNC and wireless coding^[8–11] establishes the basic trade-off between coded redundancy and recovery. ETX, ExOR, and low-power routing standards^[16–21] address path quality, opportunistic forwarding, and scheduled communication. Queueing and stochastic network optimization^[22,23] motivate representing load evolution and action-dependent service. The 6TiSCH minimal profile^[24] and hard-real-time scheduling results^[25] further motivate reproducible scheduling assumptions and examination of deadlines and tail behavior rather than relying on mean throughput alone.

Recent studies provide several points of comparison. Learning-based network coding has been examined in drone-assisted secure links and for selective RLNC in tactile-Internet scenarios.^[26,27] Deep reinforcement learning has also been used for link adaptation,^[28] multi-hop offloading,^[29] delay-aware resource allocation in industrial IoT,^[30] energy-aware forwarding,^[31] and federated wireless control.^[32,33] Nuwa-RL focuses on application-sensitive congestion control in dynamic wireless networks.^[34] Other works have evaluated RL rate adaptation for time-critical Wi-Fi with a calibrated simulator^[35] and multi-timescale DRL for delay reduction in dynamic resource allocation.^[36] Belief-state RL has recently been proposed for latency-robust wireless control when observations are delayed or uncertain.^[37] These contributions support adaptive decision making, but they do not combine the narrowband variables considered here — message size, forwarding mode, RLNC generation size, and redundancy — under a CVaR tail-latency objective. The 2024–2026 literature is strengthened by recent studies on delay-aware industrial-IoT resource control, adaptive forwarding, federated and multi-timescale wireless reinforcement learning, and latency-robust digital-twin control.^{[30–32,36,37]} In addition, recent work on offline/distributional reinforcement learning for risk-sensitive wireless optimization^[38] and on digital-twin-driven reinforcement-learning reliability and resource-management transfer^[39,40] highlights the importance of uncertainty, tail behavior, and model-to-deployment mismatch.

The gap addressed by DARC is therefore specific. We ask whether a compact, auditable tabular controller can coordinate forwarding, generation size, and redundancy from four observable network descriptors while also representing the effect of the current action on subsequent queue conditions. The oracle, greedy controller, and contextual bandit are included as checks on that premise. If a myopic method performs as well as the exact MDP, the comparison should reveal it; no advantage from long-horizon reinforcement learning is assumed beforehand.

Table 1: Positioning of DARC relative to representative approaches

3.1 81-state context and 12-action controller

Consider a source sending traffic to a supervisory endpoint over H wireless hops. At control interval t, the gateway provides four quantities to DARC: the estimated uncoded end-to-end packet-loss probability p_t, the load/queue-pressure ratio ρ_t, the active route length H_t, and the pending application message size B_t. Each quantity is mapped to one of three bins. Operationally, ρ_t is a dimensionless load/queue-pressure indicator: smaller values denote lightly loaded service, whereas larger values indicate offered demand and backlog pressure approaching the available useful service capacity. It is a coarse gateway control observable rather than an exact queue-length measurement.

Each control action specifies three choices: RLNC generation size g∈{1,2,4}, whether to send one extra coded packet r∈{0,1}, and the forwarding mode f∈{S&F, FFS}. Throughout the manuscript, S&F denotes store-and-forward switching, and FFS denotes fast-forward switching.

Table 2: State-bin boundaries used in the 81-state context

3.2 RLNC, finite-field rank and recovery model

The simulator defines p as the raw end-to-end uncoded packet-loss probability before RLNC or ARQ. If per-hop packet-loss probabilities p_h are measured instead, the equivalent end-to-end uncoded packet-loss probability is obtained from Eq. (3). Unless stated otherwise, packet erasures are conditionally independent Bernoulli events at the packet level for a given state value p.

RLNC operates over GF(2⁸). For a generation of g source packets and r extra coded packets, a receiver can decode when at least g packets arrive and their coefficient matrix has full rank. The finite-field rank correction is included explicitly:

Relays do not recode in this simulator; RLNC is performed only at the source. The pair (g,r)=(1,0) is the uncoded case and bypasses the encoder; thus, it has no RLNC coefficient header. Whenever the coding is active (g>1 or r=1), each coded packet includes a 4-byte generation/rank field and g one-byte GF(2⁸) coefficients.

3.3 Action-dependent queue transition and Markov property

The key correction relative to a contextual-bandit formulation is that the selected action changes the useful service rate μ(s_t,a_t). Let λ_t denote the offered useful-bit demand and Q_t denote a normalized backlog indicator.

The discrete transition kernel uses the service-to-demand ratio μ/λ to update the load bin. When μ/λ>1.10, the probabilities of moving down, staying, or moving up are 0.70, 0.25, and 0.05, respectively. When μ/λ<0.82, they are 0.05, 0.25, and 0.70; the intermediate case is 0.15, 0.70, and 0.15. The loss, route length, and message size follow slower exogenous nearest-neighbor kernels: 0.80 probability of remaining and 0.10 for each adjacent move.

Because P_ρ depends on a through the useful service rate, the current coding/forwarding decision changes future congestion risk. This finding highlights the long-horizon coupling required for Q-learning. The contextual UCB baseline intentionally removes this next-state term and optimizes only the immediate reward.

3.4 Tail-latency estimator and CVaR95 objective

Tail latency is evaluated from a rolling RTT sample window. A 95% tail contains only about ⌈0.05W⌉ observations: approximately 4 samples for W=64, 7 for W=128, and 13 for W=256. To reduce tail-estimator sparsity, the final experiments use W=256, a 256-packet warm-up, and a 32-packet update stride.

CVaR is used because it responds to the magnitude of extreme delays, not only the 95th-percentile crossing.^[14] The sliding-window RTT estimator follows established timer and loss-detection principles in explicitly separating observation windows from retransmission decisions.^[15]

3.5 Fully normalized multi-objective cost and Q-learning

To prevent RTT from dominating the reward numerically, every component is clipped and normalized to [0,1]. The exact definitions are d_norm=clip[(CVaR95−0.50 s)/(6.00 s−0.50 s),0,1], e_norm=clip[p_res/0.25,0,1], o_norm=clip[O_c/1.50,0,1], and g_def=clip[1−G/R,0,1], where p_res is the residual delivery-loss probability, O_c=(transmitted coded bytes−source bytes)/source bytes, G is the useful goodput and R is the physical payload rate.

The reward is r_t=−J(s_t,a_t). DARC uses the standard tabular temporal-difference update:

For comparison, contextual UCB treats each context independently and selects an action from an optimistic immediate-reward estimate.^[13] In all reported UCB experiments, the exploration coefficient is β=0.10:

At each control interval, the gateway estimates the raw uncoded packet-loss probability from recent delivery/ACK statistics, calculates the load/queue pressure, obtains the active route length and reads the pending message size. The four values are quantized to the 81-state context. During offline training, ε-greedy exploration is used with uniform generative sampling of the 81 contexts to avoid poorly visited state bins; after training, the greedy action is applied. The RTT/CVaR reward is evaluated on the defined window, and the selected action remains active for one control interval to preserve causal attribution.

Algorithm 1: Delay-Aware Reinforcement Coding (DARC)

The algorithm's online complexity is O(12) for greedy action selection, and the stored table contains only 972 values.

5.1 Experimental envelope and statistical protocol

The simulator evaluates routes from two to seven hops, application messages from 128 to 1024 bytes, raw end-to-end uncoded packet-loss probability from 1% to 22%, and offered load/queue pressure from 0.25 to 0.90. The reference condition is five hops, 512 bytes, a 12% end-to-end packet-loss probability and a load of 0.75. The environment stores an explicit 81×12 immediate-reward matrix and an 81×12×81 transition tensor. Ten independent Q-learning seeds are trained for 120,000 transitions. Offline training samples the 81 contexts uniformly from the generative simulator, while the next states are drawn from Eq. (7); this improves coverage without changing the Bellman target. Static result figures use independent repeated samples with 95% confidence intervals; the exact discounted benchmark is computed directly from the transition matrix, avoiding Monte Carlo ambiguity in the oracle comparison. The uniform distribution is also used as d₀ for the exact discounted comparison, so the oracle gap is a state-space average rather than a result conditioned on one favorable start state.

Table 3: Model, learning, normalization, and transition parameters with rationale

5.2 Baselines

Seven comparison policies are used in addition to DARC: (i) uncoded store-and-forward; (ii) uncoded fast-forward; (iii) fixed reliability-biased RLNC with g=2, r=1 and FFS; (iv) a threshold heuristic that increases coding under loss/load; (v) per-context UCB, which is a true contextual bandit and ignores next-state value; (vi) a model-informed greedy one-step benchmark that selects the action minimizing the simulator's expected immediate cost matrix; and (vii) an exact value-iteration oracle over the same 81-state MDP. The oracle is not deployable because it assumes that the transition model is known, but it provides an upper bound on the achievable discounted performance within the stated model.

5.3 Calibration status and protocol

The present revision does not fabricate empirical calibration. The authors' historical narrowband publications^[1,2,5,6] motivate the radio rate, message-size and hop-count envelope, but the raw timestamped traces needed for a defensible RMSE/MAE/R² calibration were not supplied with this submission draft. The supplementary package therefore includes empirical_calibration_template.csv. Once measured mean/p95 RTT points are inserted, the simulator can be fitted on serialization, processing and error parameters and compared by RMSE, MAE and R². Until that step is completed, all new numerical results must be interpreted as a reproducible digital-model study rather than new over-the-air evidence.

6.1 Multi-seed learning behavior

The learning-stabilization diagnostics across ten independent seeds over an extended 120,000-transition horizon are shown in Fig. 2. The three panels show fixed-validation normalized cost, relative Q-table change and the fraction of contexts whose greedy action changes between checkpoints. At the final checkpoint, the mean relative Q-table change is approximately 0.0035, and the mean greedy-policy change fraction is approximately 0.026, both below the predeclared 1% and 5% stabilization thresholds. The figure is therefore described as stabilization evidence rather than a proof of asymptotic convergence; confidence bands expose seed-to-seed uncertainty.

Fig. 2: Ten-seed DARC learning-stabilization diagnostics over 120,000 transitions: fixed-validation normalized cost, relative Q-table change, and greedy-policy change fraction with 95% confidence intervals.

6.2 Learned-policy interpretability

The full 81-context policy is shown in Fig. 3 using three enlarged discrete panels. The generation size, redundancy and forwarding are separated rather than compressed into a single multicolor map. Rows are encoded as packet-loss-bin/hop-bin pairs (P1/H1 through P3/H3), while columns are load-bin/message-size pairs (L1/B1 through L3/B3); the bin values are printed below the atlas. This layout makes individual decisions readable at normal manuscript zoom.

Fig. 3: Enlarged DARC policy atlas over all 81 states: (a) RLNC generation size, (b) redundant coded-packet choice, and (c) forwarding mode. The compact P/H and L/B axis keys are expanded below the panels.

6.3 RTT distribution and hop-count scaling

The empirical RTT CDF at the representative operating point is shown in Fig. 4. The redesigned figure adds an upper-tail zoom so that the small differences among the uncoded/adaptive strategies remain visible near the 95th-percentile region. Reliability-biased RLNC shifts the distribution left because it reduces recovery exposure, but that benefit is obtained with additional coded airtime. DARC selects g=1, r=0 and FFS at this moderate-loss context, explicitly prioritizing useful throughput over heavy protection.

Fig. 4: Empirical RTT CDF at five hops, 12% raw end-to-end packet-loss probability and a load of 0.75: (a) full distribution and (b) enlarged upper-tail region.

The data in Fig. 5 strengthen the hop-scaling analysis by adding 95% confidence intervals, and the second panel shows the relative p95 gain against the store-and-forward process. The absolute curves can be visually close because serialization is dominant; the relative-gain panel reflects the smaller algorithmic contribution around that physical floor.

Fig. 5: p95 RTT versus hop count with 95% confidence intervals and relative p95 gain against store-and-forward.

6.4 Channel-loss robustness and ablation

The useful goodput as the raw end-to-end packet loss probability changes is shown in Fig. 6. The upper panel reports absolute goodput with 95% confidence intervals, while the lower panel reports the relative difference against store-and-forward. Store-and-forward and fast-forward overlap in goodput by construction because the forwarding mode changes relay processing delay but not radio serialization, coding overhead or residual delivery probability in the present abstraction.

Fig. 6: Goodput robustness versus raw end-to-end packet-loss probability: (a) useful goodput with 95% confidence intervals and (b) relative goodput difference with respect to store-and-forward.

The ablation in Fig. 7 avoids a dual y-axis. p95 RTT and goodput are normalized separately to the store-and-forward baseline and displayed in two aligned panels. Fast-forward switching removes processing time without changing payload goodput, whereas RLNC trades a useful rate for a lower recovery tail. Full DARC chooses among these mechanisms rather than assuming one permanent configuration.

Fig. 7: Ablation of forwarding and coding components using separate normalized p95-RTT and goodput panels.

6.5 Message size, Pareto behavior and representative comparison

The direct serialization effect is shown in Fig. 8. Increasing B increases the time required to place a message on a 19.2-kb/s narrowband link; thus, physical airtime accounts for a larger share of RTT as messages grow. By including the message size in the state, the controller can distinguish between operating regimes with different latency floors. DARC can influence queueing, relay processing, and recovery behavior, but it cannot eliminate the serialization time itself.

Fig. 8: Tail-RTT sensitivity to application message size with 95% confidence intervals.

Table 4 presents a compact all-policy comparison with p95 RTT, useful goodput, normalized cost, 95% confidence intervals, and the exact oracle gap. The same p95-goodput operating points are then visualized in Fig. 9.

Table 4: Representative baseline comparison at five hops and 12% raw packet loss (20 independent repetitions)

Fig. 9: Latency–goodput Pareto comparison at the representative condition; error bars show 95% confidence intervals.

6.6 Exact dynamic MDP benchmark and contextual-bandit comparison

For the exact policy comparison, Table 5 uses the normalized discounted criterion J̄^π=(1−γ) E_s₀~d₀[Σ_t=0^∞ γ^t J(s_t,π(s_t))], where d₀ is uniform over the 81 states. The prefactor (1−γ) keeps the reported quantity on the same normalized [0,1] cost scale as the one-step objective.

Table 5: Exact normalized discounted cost over the 81-state simulated MDP

With respect to the exact normalized discounted objective, DARC reaches 0.336350, which is within 0.29% of the value-iteration oracle. It improves the store-and-forward and fast-forward policies by 2.24% and 1.39%, respectively. Contextual UCB is 0.336435, which is only 0.025% above DARC, while the greedy one-step controller reaches 0.335396. The 0.025% deterministic difference between DARC and UCB is too small to interpret as a practically meaningful or statistically significant superiority claim; it is smaller than the learning-seed variability visible in Fig. 2. The present evidence therefore treats DARC and contextual UCB as practically comparable, while DARC retains an explicit long-horizon queue model.

6.7 Dynamic-workload stress test

Table 6 summarizes the dynamic-workload stress test over 20 independent seeds per policy. The benchmark reports the discounted normalized cost and a p95 window-cost statistic for each run.

Table 6: Dynamic-workload stress test over 20 independent seeds per policy

The dynamic stress test does not reveal a statistically separable long-horizon advantage for DARC over greedy or contextual UCB: the confidence intervals overlap substantially. DARC nevertheless remains close to both adaptive comparators and below the fixed S&F/FFS means. This negative/neutral result is scientifically important: it shows that the value of the action-dependent queue model is scenario-dependent and must be demonstrated with more bursty measured traffic rather than assumed from the RL formulation alone.

6.8 Objective-weight sensitivity

The default 0.45/0.25/0.15/0.15 weights are a tail-emphasized generic telemetry preference rather than weights fitted to a particular application or to measured RF data. Six normalized weight profiles are tested in Fig. 10 instead of relying on one arbitrary scalarization. Tail- and reliability-priority profiles tend to select stronger coding, whereas goodput- or overhead-priority profiles favor light coding. This confirms that the controller's tradeoff is governed by interpretable normalized weights rather than by the unit scale of RTT.

Fig. 10: Weight-sensitivity profiles for the fully normalized tail/loss/overhead/goodput-deficit objective.

Scalability and computational complexity: The corrected state space has 3⁴=81 contexts and 12 actions; thus, only 972 Q-values are stored. The online quantization is O(1), and the greedy action selection is O(12). The exact transition tensor is required only for the offline oracle and reproducibility analysis; a deployed DARC gateway needs only the learned Q-table, observation logic, tail window and action interface.

Robustness and generalization scope: The simulation envelope covers a 1–22% raw end-to-end packet loss probability, a load/queue pressure of 0.25–0.90, two to seven hops and 128–1024-byte messages. Confidence intervals, multi-seed learning stabilization, a full policy atlas, ablation, hop scaling, message-size sensitivity and objective-weight sensitivity test internal mechanism consistency. These results support interpolation inside the stated envelope but not extrapolation to burst-dominated channels, hidden terminals, rapidly changing routes, substantially lower bit rates, many competing flows or modem-specific firmware behavior.

Limitations: The greatest remaining scientific limitation is the lack of newly supplied over-the-air trace data. Interference maps, correlated burst errors, oscillator drift, exact MAC contention, codec CPU cost and modem buffering are not modeled. The RLNC abstraction is source-only and uses generation-level cumulative ACK rather than relay recoding. The load transition is deliberately low-dimensional. In addition, the contextual UCB and the model-informed greedy baseline perform very closely to DARC in the tested MDP; thus, the paper does not claim a large generic advantage of long-horizon learning.

Future work should use timestamped RTT/loss traces from the radios used in earlier practical studies, fit the digital model and report the RMSE, MAE and R² before empirical calibration is performed. A second stage should introduce correlated loss, link asymmetry, explicit queue occupancy and multiple flows and then evaluate constrained or distributional RL for probabilistic delay guarantees. Hardware-in-the-loop testing should compare inference and coding CPU overhead with saved retransmission time.

Table 7: Novelty-to-evidence traceability

In conclusion, DARC provides a technically distinct continuation of practical narrowband RTT/network-coding research by turning fixed coding and forwarding choices into an auditable 81-state MDP. The corrected formulation includes the message size, an action-dependent queue transition, a CVaR95 tail estimator, normalized objectives, explicit finite-field RLNC overhead, strong adaptive baselines, an exact oracle, multi-seed diagnostics and uncertainty-aware result figures. DARC reaches near-oracle discounted cost in the stated model while revealing that contextual-bandit performance can be very close. The appropriate interpretation is therefore a reproducible, falsifiable adaptive-control design ready for measurement-driven calibration rather than a claim of universal superiority.

El Miloud Ar-Reyouchi: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Supervision, Validation, Visualization, Writing – Original draft, Writing – Review & editing. Abdeljalil Alaoui Douiri: Investigation, Resources, Validation, Writing – Review & editing. Both authors have read and agreed to the published version of the manuscript.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

The synthetic data generated and analyzed during this study, including model-configuration details, expected-reward and transition matrices, learned policy arrays, seed-level validation results, and plot-data CSV files, are included in the Supporting Information and reproducibility package accompanying the manuscript. Additional reproducibility files or clarifications are available from the corresponding author upon reasonable request. Raw over-the-air RF measurement traces are not part of the present study because they were not supplied for this analysis; therefore, the digital model is not described as empirically calibrated.

There is no conflict of interest.

The authors declare that Generative AI was used to assist with language editing, organization, and drafting of selected revision passages. The authors reviewed and verified all revisions and remain responsible for the scientific content and interpretation. No generative AI was used to fabricate or alter the underlying data or images; numerical values reported in this revision are derived from the accompanying reproducibility files and remain independently reproducible.

The Supporting Information includes the reproducibility code, expected-reward matrix, full transition tensor, learned policy arrays, ensemble Q-table, seed-level validation outputs, plot-data CSV files, model card, and an empirical-calibration template for future measured RTT/RF traces. The complete Supporting Information is available at the link: Supporting Information.

1. [1] Y. Chatei, M. Hammouti, E. M. Ar-Reyouchi, K. Ghoumid, Downlink and uplink message size impact on round trip time metric in multi-hop wireless mesh networks, International Journal of Advanced Computer Science and Applications, 2017, 8, 223–229, doi: 10.14569/IJACSA.2017.080332.
2. [2] E. M. Ar-Reyouchi, M. Hammouti, I. Maslouhi, K. Ghoumid, The internet of things: Network delay improvement using network coding, Proceedings of the Second International Conference on Internet of Things, Data and Cloud Computing, ACM, 2017, 8, doi: 10.1145/3018896.3018902.
3. [3] M. Ploumidis, N. Pappas, V. A. Siris, A. Traganitis, On the performance of network coding and forwarding schemes with different degrees of redundancy for wireless mesh networks, Computer Communications, 2015, 72, 49–62, doi: 10.1016/j.comcom.2015.05.001.
4. [4] S. Kafaie, M. H. Ahmed, Y. Chen, O. A. Dobre, Performance Analysis of Network Coding with IEEE 802.11 DCF in Multi-Hop Wireless Networks, IEEE Transactions on Mobile Computing, 2018, 17(5), 1148–1161, doi: 10.1109/TMC.2017.2737422.
5. [5] S. Rattal, E. M. Ar-Reyouchi, An effective practical method for narrowband wireless mesh networks performance, SN Applied Sciences, 2019, 1, 1532, doi: 10.1007/s42452-019-1595-9.
6. [6] E. M. Ar-Reyouchi, K. Ameziane, O. El Mrabet, K. Ghoumid, The potentials of network coding for improvement of round trip time in wireless narrowband rf communications, 2014 International Conference on Multimedia Computing and Systems, IEEE, 2014, 130–134, doi: 10.1109/ICMCS.2014.6911418.
7. [7] J. Laarhuis, A. Chiumento, Capacity and delay analysis of multi-hop wireless networks, Ad Hoc Networks, 2025, 169, 103750, doi: 10.1016/j.adhoc.2024.103750.
8. [8] T. Ho, M. Medard, R. Koetter, D. R. Karger, M. Effros, J. Shi, B. Leong, A random linear network coding approach to multicast, IEEE Transactions on Information Theory, 2006, 52, 4413–4430, doi: 10.1109/TIT.2006.881746.
9. [9] S. Katti, H. Rahul, W. Hu, D. Katabi, M. Medard, J. Crowcroft, XORs in the Air: practical wireless network coding, IEEE/ACM Transactions on Networking, 2008, 16, 497–510, doi: 10.1109/TNET.2008.923722.
10. [10] S. Chachulski, M. Jennings, S. Katti, D. Katabi, Trading structure for randomness in wireless opportunistic routing, ACM SIGCOMM Computer Communication Review, 2007, 37, 169–180, doi: 10.1145/1282427.1282400.
11. [11] I. Chatzigeorgiou, A. Tassi, Decoding delay performance of random linear network coding for broadcast, IEEE Transactions on Vehicular Technology, 2017, 66, 7050–7060, doi: 10.1109/TVT.2017.2670178.
12. [12] C. J. C. H. Watkins, P. Dayan, Q-learning, Machine Learning, 1992, 8, 279–292, doi: 10.1007/BF00992698.
13. [13] P. Auer, N. Cesa-Bianchi, P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning, 2002, 47, 235–256, doi: 10.1023/A:1013689704352.
14. [14] R. T. Rockafellar, S. Uryasev, Optimization of conditional value-at-risk, Journal of Risk, 2000, 2, 21–41, doi: 10.21314/JOR.2000.038.
15. [15] V. Paxson, M. Allman, J. Chu, M. Sargent, Computing TCP's retransmission timer, RFC 6298, IETF, 2011, doi: 10.17487/RFC6298.
16. [16] D. S. J. De Couto, D. Aguayo, J. Bicket, R. Morris, A high-throughput path metric for multi-hop wireless routing, Proceedings of the 9th Annual International Conference on Mobile Computing and Networking, ACM, 2003, 134–146, doi: 10.1145/938985.939000.
17. [17] S. Biswas, R. Morris, ExOR: Opportunistic multi-hop routing for wireless networks, ACM SIGCOMM Computer Communication Review, 2005, 35, 133–144, doi: 10.1145/1080091.1080108.
18. [18] A. Brandt, J. Hui, R. Kelsey, P. Levis, K. Pister, R. Struik, J. P. Vasseur, R. Alexander, RPL: IPv6 routing protocol for low-power and lossy networks, RFC 6550, IETF, 2012, doi: 10.17487/RFC6550.
19. [19] P. Thubert, An architecture for IPv6 over the time-slotted channel hopping mode of IEEE 802.15.4, RFC 9030, IETF, 2021, doi: 10.17487/RFC9030.
20. [20] M. Vučinić, X. Vilajosana, S. Duquennoy, D. Dujovne, 6TiSCH Minimal Scheduling Function, RFC 9033, IETF, 2021, doi: 10.17487/RFC9033.
21. [21] IEEE, IEEE Standard for Low-Rate Wireless Networks, IEEE Std 802.15.4-2020, 2020, doi: 10.1109/IEEESTD.2020.9144691.
22. [22] J.-Y. Le Boudec, P. Thiran, Network Calculus: A theory of deterministic queuing systems for the internet, Springer, 2001, doi: 10.1007/3-540-45318-0.
23. [23] M. J. Neely, Stochastic network optimization with application to communication and queueing systems, Morgan & Claypool, 2010, doi: 10.2200/S00271ED1V01Y201006CNT007.
24. [24] K. Pister, T. Watteyne, Minimal IPv6 over the TSCH Mode of IEEE 802.15.4e, RFC 8180, IETF, 2017, doi: 10.17487/RFC8180.
25. [25] C. L. Liu, J. W. Layland, Scheduling algorithms for multiprogramming in a hard-real-time environment, Journal of the ACM, 1973, 20, 46–61, doi: 10.1145/321738.321743.
26. [26] L. Xiao, H. Li, S. Yu, Y. Zhang, L.-C. Wang, S. Ma, Reinforcement Learning Based Network Coding for Drone-Aided Secure Wireless Communications, IEEE Transactions on Communications, 2022, 70, 5975–5988, doi: 10.1109/TCOMM.2022.3194074.
27. [27] Shahzad, R. Ali, A. Haider, H. S. Kim, RS-RLNC: A reinforcement learning-based selective random linear network coding framework for tactile internet, IEEE Access, 2023, 11, 141277–141288, doi: 10.1109/ACCESS.2023.3340210.
28. [28] X. Ye, Y. Yu, L. Fu, Deep reinforcement learning based link adaptation technique for LTE/NR systems, IEEE Transactions on Vehicular Technology, 2023, 72, 7364–7379, doi: 10.1109/TVT.2023.3236791.
29. [29] T. H. Nguyen, V. D. Do, L. H. Lan, N. C. Luong, D. V. Le, D. Niyato, Deep reinforcement learning for multi-hop offloading in UAV-assisted edge computing, IEEE Transactions on Vehicular Technology, 2023, 72, 16917–16922, doi: 10.1109/TVT.2023.3292815.
30. [30] H. Wang, Y. Bai, X. Xie, Deep reinforcement learning based resource allocation in delay-tolerance-aware 5G industrial IoT systems, IEEE Transactions on Communications, 2024, 72, 209–221, doi: 10.1109/TCOMM.2023.3322736.
31. [31] N. A. Askar, A. Habbal, RLEAFS: Reinforcement learning-based energy aware forwarding strategy for NDN-based IoT networks, IEEE Access, 2024, 12, 177173–177188, doi: 10.1109/ACCESS.2024.3456669.
32. [32] S. K. Das, R. Mudi, M. S. Rahman, K. M. Rabie, X. Li, Federated reinforcement learning for wireless networks: fundamentals, challenges and future research trends, IEEE Open Journal of Vehicular Technology, 2024, 5, 1400–1440, doi: 10.1109/OJVT.2024.3466858.
33. [33] R. Ali, B. Bellalta, A federated reinforcement learning framework for link activation in multi-link Wi-Fi networks, 2023 IEEE International Black Sea Conference on Communications and Networking, 2023, 360–365, doi: 10.1109/BlackSeaCom58138.2023.10299778.
34. [34] G. Gong, X. Jiang, G. Jin, Y. Xie, H. Chen, Nuwa-RL: A reinforcement learning based receiver-side congestion control algorithm to meet applications demands over dynamic wireless networks, 2022 IEEE International Conference on High-Performance Computing and Communications, 2022, 508–515, doi: 10.1109/HPCC-DSS-SmartCity-DependSys57074.2022.00095.
35. [35] G. Peserico, T. Fedullo, A. Morato, F. Tramarin, L. Rovati, S. Vitturi, SNR-based reinforcement learning rate adaptation for time critical Wi-Fi networks: assessment through a calibrated simulator, 2021 IEEE International Instrumentation and Measurement Technology Conference, 2021, 1–6, doi: 10.1109/I2MTC50364.2021.9460075.
36. [36] L. Liang, P. Duan, G. Cui, W. Wang, Multiagent DRL for two-timescale bandwidth allocation in multibeam satellite networks, IEEE Internet of Things Journal, 2025, 12, 10613–10626, doi: 10.1109/JIOT.2024.3511672.
37. [37] H. Tiwari, B. Kar, P. Tiwari, Digital twin-assisted belief-state reinforcement learning for latency-robust ISAC in 6G Networks, IEEE INFOCOM 2026, 2026, doi: 10.1109/INFOCOM59046.2026.11571214.
38. [38] E. Eldeeb, H. Alves, Offline and distributional reinforcement learning for wireless communications, IEEE Communications Magazine, 2025, 63, 71–76, doi: 10.1109/MCOM.001.2400521.
39. [39] Z. Tao, W. Xu, X. You, Provable Performance bounds for digital twin-driven reinforcement learning in wireless networks, IEEE Transactions on Signal Processing, 2025, 73, 4430–4445, doi: 10.1109/TSP.2025.3624833.
40. [40] N. Cheng, X. Wang, Z. Li, Z. Yin, T. H. Luan, X. Shen, Toward enhanced reinforcement learning-based resource management via digital twin: opportunities, applications, and challenges, IEEE Network, 2025, 39, 189–196, doi: 10.1109/MNET.2024.3438543.