Telecommun Syst DOI 10.

1007/s11235-011-9658-2

On Multi Agent based modeling and control in large scale wireless communication systems for improved resource allocation performance
P.M. Papazoglou D.A. Karras R.C. Papademetriou

Springer Science+Business Media, LLC 2011

Abstract The goal of this paper is to investigate the application of the Multi Agent technology in modeling and simulating large scale wireless communication systems. There are very few studies using the multi agent concept as an alternative approach for designing and simulating resource allocation in cellular systems and especially in large scale ones. Thus, a novel modeling methodology of wireless network services exploiting multi agent technology and investigating in depth critical agent issues is proposed in this paper. It is shown that the multi agent concept proves to be a suitable solution for modeling and implementing cellular network services simulation even in the case of large scale wireless networks. The whole investigation is divided in two phases: (a) multi agent modeling analysis and (b) multi agent model development for controlling network performance. Simulation results show the effectiveness of the proposed multi agent model when is applied in combination with the state of the art event scheduling mechanism. Keywords Multi Agent systems Large scale cellular networks Simulation Concurrent events Agent control

1 Introduction 1.1 Simulating wireless communication systems Simulation tools and environments offer the opportunity to network engineers to develop and evaluate experimental networks towards the desired network system. A number of critical parameters such as network infrastructure specications, mobile user services, wireless environment models, etc., can be involved in the simulation model during the development phases using the available tools. On the other hand, there are critical issues regarding the simulation system itself which represents the hidden views behind the experimental network that are hardly known to network engineers. These issues include the event scheduling mechanism and the modeling methodology of the network components such as base stations, cells, network services, etc. The scheduling mechanism represents the model which denes the event servicing sequence by the network system. More specically, the scheduler classies the generated events by their time stamps. The servicing sequence is dened by the rule rst come rst serve. The state of the art event scheduling mechanism is called Calendar Queue (CQ) and was rst introduced by Brown [3]. Inside the computer, a CQ consists of an array of lists. Each list contains future events. According to CQ principle, the large list of N events is partitioned to M shorter lists called Buckets. Each bucket is associated with a specic range of time corresponding to future events. New events are enqueued inside buckets and the event with the lower time stamp (higher priority) is dequeued by the scheduler. A major drawback of this mechanism is that supports only sequential network events. In a real wireless network events can be happened sequentially or concurrently. Due to the great number of mobile users most events occur simultaneously. Based only on the CQ

P.M. Papazoglou Lamia Institute of Technology, Lamia, Greece e-mail: papaz@teilam.gr D.A. Karras ( ) Automation Dept., Chalkis Institute of Technology, Evoia, Psachna 34400, Greece e-mail: dakarras@teihal.gr D.A. Karras ( ) e-mail: dakarras@ieee.org P.M. Papazoglou R.C. Papademetriou ECE Department, University of Portsmouth, Anglesea Road, PO1 3DJ Portsmouth, UK

P.M. Papazoglou et al.

mechanism for the event scheduling, concurrent events are not supported and the network performance can not be improved regarding the mobile users requests. In this paper, a novel modeling methodology based on the multi agent technology is presented. This methodology involves a special treatment for time-close network events that can be faced as concurrent. 1.2 Multi Agent Systems in modeling and simulating wireless networks Multi Agent technology offers the opportunity for modeling efciently the behavior of real dynamic systems such as wireless communication networks. The agent characteristics such as adaptability (Splunter, Wijngaards, and Brazier [19], Russell, and Norvig [18]), autonomy (Huhns, and Singh [11], Norman and Long [15], Ekdahl [6]), collaboration and interactivity increase the adaptability of the model to the real system behavior. Agent technology must be applied not only where the agent characteristics have sense but also where these characteristics improve the whole system performance which is evaluated through selected metrics. Multi Agent technology has been used mainly for resource management in past studies (Iraqi and Boutaba [12], Bodanese [1]). Cellular systems modeling and especially simulation methodology has not been viewed so far in terms of multi agent systems. In our approach, the network services have been modeled as agents (Papazoglou, Karras, Papademetriou [16] and [17]) instead of base stations and other network nodes found in the literature. 1.3 Critical issues on Multi Agent System implementations for simulation models There is a great theoretical analysis of agent behavior in the literature (Wooldridge and Jennings [24], Wooldridge [23]) but the implementation of those theories remains a critical issue. In a Multi Agent environment, new requirements such as concurrency, synchronization, control and efcient execution must be met in order to implement correctly all the agent theoretical aspects. Thus the platform selection and the agent implementation methodology play a major role for the derived model. On the other hand, several difculties arise during implementation phase where synchronization and deadlock issues have to be faced effectively. All the above mean that the implementation methodology and technology is not obvious and a deeper study must be conducted towards the desired multi agent based simulation model. The above features play a more critical role when a large scale model is implemented due to the needed optimization for the whole system performance. The goal of this paper is to investigate as thoroughly as possible solutions to these problems for designing and simulating large scale cellular

Fig. 1 Cellular network layout

networks and propose suitable mechanisms and architectures for controlling Multi-Agent Systems, modeling such networks services, following the line of research presented by the authors in [16, 17].

2 Multi Agent based modeling and control in large scale wireless communication systems simulation 2.1 Cellular network layout and supported network services Figure 1 shows the cellular network layout for a cluster region which contains one central cell (A) and six neighbor cells (B to G). Each cell contains one base station in its center for servicing local mobile users. A base station allocates channels to mobile users. Based on the channel allocation strategy, a mobile user can occupy one or more channels that belong to local cell (where the mobile user request is occurred) or in neighbor cells. The cellular network operation can be divided in two levels: (a) user services and (b) Scheduling and synchronization. The cellular network offers to mobile users four individual services which are: New call admission (NC). The network allocates voice or multimedia channels for a new call. Reallocation (handoff) (RC). A number of channels are replaced by others due to not accepted CNIR (Carrier to Noise plus Interference Ratio). User Movement (MC). Channel reallocation due to mobile user movement (to another location within the network). Call termination (FC). A mobile user call is terminated when the call holding time is expired or the total amount of data has been transferred (for data services). Scheduling is referred to event (services) occurrences and future processing and synchronization is referred to implemented agents for the simulation procedure operation.

On Multi Agent based modeling and control in large scale wireless communication systems for improved

implementation. This analysis is necessary for the following reasons: (a) to ensure that the proposed network agents fulll the multi agents concepts (behavior, specications) (b) to ensure that the above concepts are fullled during the study faces (analysis, development, implementation) (c) for describe and develop formally with clarity the steps and procedures of the corresponding algorithms regarding the agents behavior, specications and attributes. 2.2.1 Agent denition A network agent interacts with its direct environment, takes decisions and performs actions that inuence the environment conditions (Papazoglou, Karras, Papademetriou [16]). Figures 4a and 4b show the interaction between network agents and cellular environment as well as interaction between clock agent and network agents. The network agents NCA and RCA are responsible for the new call admission and handoff services respectively and thus affect directly the blocking and dropping probability which are the basic statistical metrics for the performance evaluation of the cellular network. On the other hand, the clock agent synchronizes the agent activation and schedules the event processing by giving execution time to each of them. The above agents are characterized by reactivity (in the get/response manner), proactiveness (takes the initiative to exhibit its goal oriented behavior) and sociality (interaction with other agents). 2.2.2 Agent architecture and interaction with its environment Let the possible discrete states E1 of the cellular network environment: E1 = {LL, LH, HL, HH} (1)

Fig. 2 Multi Agent Interaction (AIN = Agent Interaction, CAI = Clock Agent Interaction)

Fig. 3 Multi agent layered architecture per cluster

2.2 Modeling network agent behavior The whole cellular network is divided in regions called clusters which contain seven cells. Mobile users within a cluster are served from the seven base stations through four network agents which are: NCANC Agent RCARC Agent MCAMC Agent FCAFC Agent

Additionally, an extra agent called Control Agent (CA) works in background for event scheduling and multi agent synchronization. Figure 2 shows the communication and interaction between the agents. Each network cluster is served by four network agents and one extra agent called clock agent which schedules and synchronizes the network agents. Figure 3 shows the layered structure of the agents for each cluster in the network. Sections 2.2.1 through 2.2.4 present a formal analysis of agents behavior and specication towards development and

where the members of E1 represent the network performance, L represents low level and H the high level. The pairs correspond to blocking and dropping probability respectively. Due to the fact that the network behavior is examined in terms of performance and this performance is affected by the channel allocation strategy, the members of E, L represents the low level and H the high level of this performance. The two basic statistical metrics for measuring the channel allocation effectiveness is the blocking and dropping probability and thus the pairs inside E represent that metrics respectively. On the other hand, it is assumed that each agent has a set of possible actions on this environment. For the simulation environment there is also a set E2 with discrete states: E2 = {NCTSlow , NCTShigh , RCTSlow , RCTShigh } (2)

P.M. Papazoglou et al. Fig. 4 (a) Network agent denition. (b) Clock agent denition

The members of E2 represent the agent execution time slices where TSlow is the lower time slice and the TShigh the highest time slice. The time slice denes the active execution period of each network agent and thus the corresponding priority for adapting the network behavior to current user needs. It is assumed also that the network agents (here the NCA and RCA) have a set of possible actions (ACnetagents ) on this environment: ACnetagents = {IPNC , IPRC , DPNC , DPRC , DN NC , DN RC } (3) where IP is the action Increase Priority, DP represents the action Decrease Priority and nally DN is the action Do Nothing. This set contains the possible agent actions in order to achieve their goals. For example, the NCA goal is the minimization of blocked calls (corresponding blocking probability). Moreover if the blocking probability is at acceptable level, the NCA can help the RCA for controlling also the dropping probability. The same principle is applied in the RCA. In the case of clock agent the corresponding sets of actions (for NCA/RCA) on the simulation environment are: ACclockagent = {ITSNC , ITSRC , DTSNC , DTSRC , DN NC , DN RC } (4)

In the same way, a run for the clock agent rclockagent within the simulation environment is expressed as: rclockagent : NCTShigh RCTSlow
DTSNC ,DN RC

NCTSlow RCTSlow

ITSNC ,DN RC

NCTShigh RCTSlow eusim

u1

DN NC ,DN RC

(6)

The corresponding time slice is dened by the clock agent actions. Let also the following sets (for network agent and clock agent respectively): Rnet as the Set of possible nite sequences (over E1 and ACnetagent ) and Rsim as the set of possible nite sequences (over E2 and ACclockagent . R ACnetagent as the subset of Rnet that ends with an action and R ACclockagent as the subset of Rsim that ends with an action. R E1 as the subset of Rnet that ends with an environment state and R E2 as the subset of Rsim that ends with an environment state. A state transformer function describes the effect of an agent (action) on the environment: netagent : R ACnetagent (E1 ), clockagent : R ACclockagent (E2 ) (7)

Where ITS represent the action Increase Time Slice, DTS represents the action Decrease Time Slice and DN represents the Do Nothing action. The agent priorities are controlled through time slices which dene the agent active execution time. The above actions constitute the response of clock agent to the corresponding agent actions regarding the needed priorities. The environment changes its state according to the above actions. A sequence of actions causes a sequence of state changes. Thus, a run, rnetagent , of a network agent within the environment can be expressed as rnetagent : HL
IPNC ,DN RC

The above function maps a run to a set of possible environment states. When no successor state exists to r, (r) becomes netagent (rnetagent ) = , clockagent (rclockagent ) = (8)

The whole environment is expressed (states, transformer function) as Envnet = E1 , e0net , netagent , Envsim = E2 , e0sim , clockagent (9)

LL

DN NC ,DN RC

LH

DN NC ,IPRC

LL (5)

DN NC ,DN RC

eunet

u1

The agent actions affect the network performance which is expressed with L and H indications.

E1 is the state set, e0net is the initial state and netagent the transformer function. For the simulation environment, E2 is the state set, e0sim is the initial state and clockagent the transformer function. For modeling agents, it is assumed that an agent represents a function for mapping runs to actions and so:

On Multi Agent based modeling and control in large scale wireless communication systems for improved

Agnetegent : R E1 ACnetagent , Agclockegent : R E2 ACclockagent (10)

e3net = {B, D},

e4net = {B, D}

(16)

In other words, an agent makes decisions about action (what action to perform) based on the history of the system. For representing now the whole system (agents, environment) a set is dened Rnet (Agnetagent , Envnet ), Rsim (Agclockagent , Envsim ) (11)

where B represents the statement Blocking probability is acceptable and D represents the statement Dropping probability is acceptable. Now, the set E contains four combinations of B and D. On the other hand, (15) and (16) for the clock agent can be expresses as follows: E2 = {{ANCA , ARCA }, {ANCAARCA }, , {ANCA , ARCA }, {ANCA , ARCA }} with e1net = {B, D}, e3net = {B, D}, e2net = {B, D} e4net = {B, D} (18) (17)

Finally, the sequences (e0net , 0 , e1net , 1 , e2net , . . .) and (e0sim , b0 , e1sim , b1 , e2sim , . . .) represent a run of an agents Agnetagent and Agclockagent (in the network and simulation environment respectively) Envnet = E1 , e0net , netagent and Envsim = E2 , e0sim , clockagent if: 0 = Agnetagent (e0net ), b0 = Agclockagent (e0sim ) (12)

A run of netagent or clockagent represented by the sequences (HL, IPNC , LL, DN, LH, . . .) and (NCTShigh RCTSlow , DTSNC , NCTSlow RCTSlow , ITSNC , . . .) if IPNC = Agnetagent (HL) and DTSNC = Agclockagent (NCTShigh RCTSlow ) respectively. For network and simulation environment and for unet > 0 and usim > 0 eunet netagent ((HL, IPNC , DN RC , . . . , (u1)net )) eusim clockagent ((NCTShigh RCTSlow , DTSNC , ITSNC , . . . , b(u1)sim )) unet = Agnetagent ((HL, IPNC , DN RC , . . . , eunet )) usim = Agclockagent ((NCTShigh RCTSlow , DTSNC , ITSNC , . . . , b(u1)sim )) The above formal analysis is used for developing the corresponding agent behavioral algorithms. 2.2.3 Network agents interaction Network and clock agents percept environment and act on it. These two distinct activities for these agents are represented by two functions respectively (Figs. 5a and 5b). The see function maps environment states to perception and action maps sequences of percepts to actions. Network behavior is evaluated via two basic statistical metrics which are the blocking and dropping probability and can be expressed as follows: E1 = {{B, D}, {B, D}, {B, D}, {B, D}} with e1net = {B, D}, e2net = {B, D} (15) (14) (13)

where ANCA represents the statement NCA request accepted, set new time slice and ARCA represents the statement RCA request accepted, set new time slice. The combinations of the above pairs are as follows: {B, D} = blocking not acceptable, dropping not accept able. {B, D} = blocking not acceptable, dropping accept able. {B, D} = blocking acceptable, dropping not acceptable. {B, D} = blocking acceptable, dropping acceptable. According to (17), (18), the see function of the NCA, will have two percepts in its range, P1 and P2 that indicate if the blocking probability is acceptable or not. The behavior of the see function can be described as follows: seeNC (enet ) = P1 P2 if enet = e1net if enet = e3net or or enet = e2net enet = e4net bad good (19)

(bad = performance metric (blocking) not acceptable, good = performance metric (blocking) acceptable and e1 = {B, D}, e2 = {B, D}, e3 = {B, D}, e4 = {B, D}. Similarly for RCA, the see (e) is formulated as follows: seeRC (enet ) = P1 P2 if enet = e1net if enet = e2net or or enet = e3net enet = e4net bad good (20)

(bad = performance metric (dropping) not acceptable, good = performance metric (dropping) acceptable, e1 = {B, D}, e2 = {B, D}, e3 = {B, D}, e4 = {B, D}. With two given environment states e E1 and e E1 , then e e can be written only if see(e) = see(e ). A network agent has perfect perception if the different environment states are equal to distinct perceives. In this case || = |E1 | and when the perception of an agent does not exist, || = 1 (22) (21)

P.M. Papazoglou et al. Fig. 5 (a) Agent interaction. (b) Network Agent interaction

2.2.4 MA encounters The following section describes Multi-Agent encounters for the network agents NCA and RCA which play the most critical role to the cellular network performance. Let two possible actions by two agents such as CR which represents the action Cooperation and CT which represents the action Competition and let the action set ACnetagent = {CR, CT}. With these assumptions, a function can be used to model the environment behavior: netagent : AcNCnet AcRCnet (23)

coding the corresponding results. In practice, any two different numbers can be assigned just to indicate an acceptable or not result and to distinguish between possible outcomes. A different utility function can be assigned to each agent (NCA, RCA) in order to measure how good the corresponding outcome is (blocking/dropping). This function, assigns a real number to each outcome, indicating how good is the outcome for the selected agent. In other words, this utility function denes the preference ordering over all the outcomes. The outcomes for NCA and RCA, can be expressed as: uNC ( (CT, CT)) = 1, uNC ( (CR, CT)) = 2, uRC ( (CT, CT)) = 1, uRC ( (CR, CT)) = 1, uNC ( (CT, CR)) = 1 uNC ( (CR, CR)) = 2 uRC ( (CT, CR)) = 2 uRC ( (CR, CR)) = 2 (26)

Cooperation (CR) and Competition (CT) are the two possible actions for NCA and RCA thus the action set becomes Acnet = {CR, CT}. For the two network agents, we assume the environment functions: net (CT, CT) = 1 , net (CR, CT) = 3 , net (CT, CR) = 2 , net (CR, CR) = 4 (24)

nally, the possible NCA, RCA actions (based on (26)) are CR, CR NCnet CR, CT >NCnet CT, CR NCnet CT, CT (27) It is obvious from (26) and (27) which action will be selected from each agent and thus each agent knows exactly what to do. 2.3 Large scale cellular network model In a large scale network, the network agents (NCA, RCA, MCA, FCA) are distributed in the corresponding clusters in the whole network. For a cellular network with N cells and i cells per cluster, the total number of clusters is N/i. Each set of the four network agents is duplicated in every cluster and thus, the total required network agents are 4 (N/i). Figure 6 shows the general concept of the network agent distribution in a large scale network. For a wireless network with 35 cells (Fig. 6), the total number of 7-cell clusters is 35/7 = 5. The total number of needed network agents is 4 5 = 20.

Based on (24), each action is mapped to a different outcome. For measuring how good the corresponding outcome is (blocking/dropping), a different utility function can be assigned to each agent (NCA, RCA). Based on this function, a real number is assigned to each outcome that indicates how good the outcome is for the selected network agent. The preference ordering over all the outcomes is dened by this utility function. Additionally, the utility functions (NCA, RCA) are uNCnet (1 ) = 1, uNCnet (3 ) = 2, uRCnet (1 ) = 1, uRCnet (3 ) = 1, uNCnet (2 ) = 1 uNCnet (4 ) = 2 uRCnet (2 ) = 2 uRCnet (4 ) = 2 (25)

The above numbers are indicative only and illustrate that the corresponding outcomes for an agent can be good or not (for NCA, a good result is the minimization of blocking probability). In other words, only two numbers are needed to

On Multi Agent based modeling and control in large scale wireless communication systems for improved Fig. 6 Network agent distribution in a large scale network

2.4 Multi Agent synchronization and event scheduling issues in large scale environments In a large scale network environment the agent synchronization and event scheduling becomes a very critical issue due to the fact the execution time (event processing) must be distributed among the requested services of the mobile users. Moreover, the network agent activation must be synchronized in order to control the whole cellular network operation. The clock agent gives time slices to each network agent to serve mobile user requests and controls also the corresponding activation. The cellular network performance is affected from the applied event scheduling scheme. Figure 7 shows the application of a global clock. In this approach, the global clock activates all the network agents by giving execution time slices. The global clock application creates cluster dependencies and decreases the system concurrency. For solving the above dependency problem, individual clocks can be distributed among the network clusters. Now, the global clock controls the cluster clocks. Figure 8 shows the distributed cluster clocks and the global clock that controls them. This approach guarantees the cells independency inside each network cluster. 2.5 Controlling the wireless network performance The most important issue in a wireless network is the corresponding performance regarding the offered services. In

such a network, the successfulness of call admissions and handoffs constitute the blocking and dropping probability respectively which are the most widely known and accepted performance metrics. Based on current network conditions, one or both of the above metrics can be at unacceptable level. Thus, the network must adapt its behavioral procedures to current conditions and needs. The above adaptation is achieved by giving priority to specic calls (network services) in order to improve the corresponding network performance. This treatment strategy can only be applied for concurrent mobile user requests, where the servicing sequence can be changed within a time period which is shorter as compared to the service initialization time. 2.5.1 Multi-Agent negotiation scheme The required adaptation for controlling the network performance is achieved through negotiation between network agents such as the NCA and RCA. NCA is responsible for new call admission in its cluster and thus it affects the corresponding blocking probability. On the other hand, RCA affects the dropping probability due to its involvement in call handoffs inside the cluster. The progressive network performance is measured after simulation time step completion. This measurement is derived through two moving windows (Fig. 9) which are overlapped for specic time steps (periods). A network agent (e.g. NCA, RCA) based on the corresponding performance,

P.M. Papazoglou et al. Fig. 7 Global clock application

Fig. 8 Distributed cluster clocks

Fig. 9 Network performance measurement

On Multi Agent based modeling and control in large scale wireless communication systems for improved Table 1 Network agent status NCA Status 1 (Good) 0 (Stable) 1 (Critical) Description Std Blocking Pr.(WB ) < Std Blocking Pr.(WA ) Std Blocking Pr.(WB ) = Std Blocking Pr.(WA ) Std Blocking Pr.(WB ) > Std Blocking Pr.(WA ) RCA Status 1 (Good) 0 (Stable) 1 (Critical) Description Std Dropping Pr.(WB ) < Std Dropping Pr.(WA ) Std Dropping Pr.(WB ) = Std Dropping Pr.(WA ) Std Dropping Pr.(WB ) > Std Dropping Pr.(WA )

Table 2 Network performance, agent decisions, actions and status relations 1 NCA status 0 0 0 1 1 1 1 1 1 2 RCA status 1 0 1 1 0 1 1 0 1 3 E1 NCA H H H L L L 4 E1 RCA H L H L H L DN DN DN DP DP DP IP IP IP DP DN IP DP DN IP DP DN IP 5 AcNCA 6 AcRCA 7 AcClock For NCA DN DN DN DTS DTS DTS ITS ITS ITS 8 AcClock For RCA DTS DN ITS DTS DN ITS DTS DN ITS 9 E2 NCA (Previous) (Previous) (Previous) TSlow TSlow TSlow TShigh TShigh TShigh 10 E2 RC1 TSlow TSlow () TShigh TSlow TSlow () TShigh TSlow TSlow () TShigh

adjusts its priority and requests from the other agent to decrease their priority if it is necessary. The negotiation between agents takes place when the agents status is critical (the corresponding performance metric is unacceptable). Table 1 shows the possible status for the above mentioned network agents. Developing the agent behavior algorithm For developing the agent behavior algorithm the corresponding formal analysis must be taken in consideration. The algorithm development must be based on clearly dened rules that arise from formal analysis. Towards the algorithm implementation four distinct levels must be described: Level 1: formal analysis (e.g. describe the environment). Level 2: status-action categorization (e.g. logical connection between status and actions). Level 3: algorithm development (agent behavior description). Level 4: code level implementation. Table 2 shows the relation between network performance and the necessary action that must be performed by a network agent. Moreover, the same table shows also the corresponding clock actions based on the agent decisions as well as the corresponding time slices assigned by the clock. Figures 10a and 10b show the algorithms that describes the NCA and RCA behavior respectively. In the following

examples its is assumed that the current network performance is High,Low regarding the successfulness of call admission and handoffs (low blocking probability, high dropping probability). The following algorithms development is based on the multi agent formal analysis which denes the corresponding steps for controlling the network behavior. According to Fig. 10a, the NCA decreases its priority to help the RCA to improve its performance regarding the handoff successfulness in order to minimize the dropping probability and hence to increase the corresponding network performance. On the other hand, Fig. 10b shows the algorithm for the RCA. In this gure, the RCA tries to improve its performance by increasing self priority and by sending a request to the NCA. In these gures we can see the actions and status for the network agents regarding Table 2 and formal analysis. More precisely, in Fig. 10a it is assumed that the corresponding network performance for call admission is high and the handoff performance is low (Table 2: columns 3 and 4, {HL}). That means that the agents status is 1 and 1 respectively (Table 2: columns 1 and 2, {1, 1}). Thus, the NCA decreases its priority in order to help the other network agent (Table 2: column 5, {DP}). According to this action, the clock decreases the corresponding time slice (Table 2: column 7, {DTS}). Finally, the time slice for the NCA is now limited (Table 2: column 9, {TSlow}).

P.M. Papazoglou et al.

Fig. 10 (a) NCA behavior algorithm. (b) RCA behavior algorithm

Figure 11 shows the design and development procedure from theoretical analysis to code implementation. Through formal analysis a set of clearly dened rules is created. The agent behavior is based on the above rules. The formal analysis is made at level 1. From this analysis the relation between environment status and agent action is also dened. The algorithm development takes place in level 3. The basis of the agent behavior is the corresponding status (e.g. blocking for NCA). The status is realized by the agent through the see function. The nal decisions of each agent are based on relations that are dened at previous levels. 2.5.2 Network performance dependability on agent behavior In a real wireless environment, the network events can be occurred sequentially or concurrently. The type of occurrence is based on the corresponding time that is between two network events. If this time is shorter as compared to the service initialization then the two events can be treated as concurrent. Figure 12 shows a set of occurred network events (requests for services).

The above network events are categorized in ve zones (A through E). Zones B and D contains concurrent events and zones A, C and E contains sequential events. If the CQ mechanism is applied, the mobile user requests will be served sequentially based on the unique generation times. On the other hand, if the multi agent solution is introduced, the concurrent events (zones B and D) can be faced differently. Based on current network performance, the network agents (e.g. NCA, RCA) have to decide and agree for the servicing sequence in order to fulll the corresponding constraints for network performance. Table 3 shows how a different treatment of concurrent requests leads to different levels of performance regarding blocking and dropping probability. In the scenario of Table 3, it is assumed that the cell congestion is 90% and only three channels are available. The challenge here is to nd the most suitable solution for channel management in order to control the network performance. In the rst case (CQ without Multi Agents), all the requests are served according to their time stamps. On the other hand, the CQ based on the Multi Agent approach faces the events of zones B and D as concurrent and thus a different treatment of existing requests can be made.

On Multi Agent based modeling and control in large scale wireless communication systems for improved Fig. 11 From formal analysis to algorithm implementation

Fig. 12 Occurred events within physical or simulated network time

P.M. Papazoglou et al. Table 3 Network performance dependability on servicing sequence for concurrent events (X = occupied channels at previous time, number of channels = 32, channel availability = 3, * = concurrent events) Calendar Queue (CQ) without Multi Agents CQ Srv N1 N2 R1 F1 N3 R2 M1 F2 N4 R3 R4 F3 F4 F5 R5 N5 M2 F6 R6 R7 N6 F7 F8 R8 F9 R9 Zone A1 A2 A3 A4 B1 B2 B3 B4 B5 B6 B7 B8 B9 B10 C1 C2 C3 D1 D2 D3 D4 D5 D6 E1 E2 E3 Channel status X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X R9 1/6 16% 5/9 55% R8 R6 R6 R6 R5 R5 R5 N5 N5 N5 N5 N5 N5 N5 M2 M2 M2 M2 M2 M2 M2 M2 M2 N6 R7 N4 N4 N4 N3 N3 N3 N1 N1 N1 N2 N2 N2 N2 N2 N2 N2 N2 N2 N2 N2 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R3 R4 R2 M1 Blocked Dropped Calendar Queue (CQ) with Multi Agents CQ MA Srv N1 N2 R1 F1 *N3 *F5 *R2 *F2 *M1 *N4 *F3 *R3 *F4 *R4 R5 N5 M2 *F6 *R6 *F7 *R7 *N6 *F8 R8 F9 R9 Zone A1 A2 A3 A4 B1 B2 B3 B4 B5 B6 B7 B8 B9 B10 C1 C2 C3 D1 D2 D3 D4 D5 D6 E1 E2 E3 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X R7 R7 R7 R7 R7 R7 R9 R8 R6 R3 R3 R3 R3 R3 R3 R4 R4 R4 R4 R4 R4 R4 R4 R4 R2 R2 R2 R2 M1 M1 M1 M1 N3 N1 N1 N1 N2 N2 N2 N2 N2 N2 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 R1 3/6 50% 2/9 22% 1, N6 N5 M2 R5 N4 Channel status Blocked Dropped

Blocking Probability (CQ only) Dropping Probability (CQ only)

Blocking Probability (CQ with Multi-Agents [call priorities]) Dropping Probability (CQ with Multi-Agents [call priorities])

Network behavior and performance based on the simple CQ approach Figure 13 shows the network behavior based on the CQ approach for zone B. In zone B, there are requests (events) that will be served by the network (N = NC, R = RC, M = MC, F = F C). The services treatment based on the simple CQ approach is as follows: Step 1: The R2 arrives but a free channel does not exist and thus the call is dropped. Step 2: The M1 arrives but still there in no free channel and thus this call is also dropped. Step 3: A call termination request is arrived (F2 ) and now one channel is released.

Step 4: A new call admission request arrives (N4 ) and the only one free channel is allocated. Step 5: A call reallocation (R3 ) arrives but the call is dropped due to channel unavailability. Step 6: Another call reallocation (R4 ) arrived and is also dropped. Step 7: A call termination request arrives (F3 ) and one channel is released. Step 8: A call termination request arrives (F4 ) and one channel is released. Step 9: A call termination request arrives (F5 ) and one channel is released. Total blocked calls: 0, Total dropped calls: 4, Total free channels after zone B: 3.

On Multi Agent based modeling and control in large scale wireless communication systems for improved Fig. 13 Network behavior in zone B based on CQ

Based on the above network behavior, four calls are dropped. At the end of zone B, three channels are available. There are many dropped calls due to the fact that the network did not exploit the existing requests for call termination and thus the nal free channels are almost useless. Network behavior and performance based on CQ and Multi Agent approach In the CQ with Multi agents approach all the events of zone B are faced as concurrent (Fig. 14). Thus,

the network is aware of the arrived requests and is able to nd the most suitable way for channel management. The existing concurrent requests are managed by the network as follows: Step 1: The call termination request (F5 ) is served rst in order to release a channel. Step 2: The R2 request is served successfully due to channel availability that is gained in step 1.

P.M. Papazoglou et al. Fig. 14 Network behavior in zone B based on CQ

Step 3: Another channel is released (F2 ) by giving high priority to call termination. Step 4: The M1 request is served by the new available channel due to step 3. Step 5: The new call request (N4 ) is blocked due to the fact that there is no free channel. Step 6: A channel is released by servicing the call termination request (F3 ).

Step 7: The call reallocation request is served successfully (R3 ). Step 8: Another channel is released (F4 ) Step 9: The last event in zone B (R4 ) is served due to channel availability. Total blocked calls: 1, Total dropped calls: 0, Total free channels after zone B: 0.

On Multi Agent based modeling and control in large scale wireless communication systems for improved

Based on the CQ with Multi Agents approach the channel exploitation is optimal for achieving better network performance. This approach constitutes a more intelligent methodology for managing efciently the available channels in the network. When the multi agent model is applied, the network agents adapt their behavior to the current network performance. Based on this principle, it is assumed that the desired performance is the minimization of dropping probability. According to this constraint the RCA priority is increased (translated in time slices) for supporting the corresponding services. Thus, the service sequence for zone B is re-arranged and the required channels can be now found by using the FC type events. With higher priority, the RCA serves more efciently the corresponding requests. As a result of the above agents actions, the dropping probability is decreasing to 22% as compared to 55% that is achieved in the case of no multi agent existence. Moreover, when additional services are offered such as data transfer the network (via the agents) can easily predict when the service will be completed in order to allocate efciently the available channels.

Fig. 15 Example calculation of periodicity

3 Simulation model evaluation The evaluation of the whole experimental system has been conducted in two distinct levels which represent the cellular network performance based on the tested scheduling mechanisms and the simulation model validation respectively. 3.1 Network performance evaluation Blocking and dropping probability represents the most widely known and applied statistical metrics for performance evaluation of wireless networks (Bigham and Du [2], Cheng, Li and Du [4], Cherriman, Romiti, Hanzo [5], Godara [7], Grace [8], Haas [9], Hollos, Karl and Wolisz [10], Katzela and Naghshineh [13], Lee, Lee and Huff [14], Salgado, Sirbu and Peha [20], Tripathi, Jeffrey, Reed and VanLandingham [21], Wong [22]). Blocking probability represents the percentage of blocked calls where the wireless network failed to allocate a proper communication channel to a mobile user. On the other hand, dropping probability refers to the dropped calls where the wireless network failed to give a new channel in a handoff situation. Equations (28) and (29) show how the corresponding probabilities are calculated. PBlocking = number of blocked calls (28) number of calls number of forced calls PDropping = number of calls number of blocked calls (29)

In the experimental network, simulation time is based on a daily model. Periodicity is a new proposed statistical metric found in Papazoglou, Karras, Papademetriou [17] which is used to investigate the simulation model behavior. Periodicity is calculated using all the possible pairs of days within simulation time. Assume that the blocking probability among the 24 hours of two days is represented by the sets setA and setB (Fig. 15). Each set has probability members 1 through n (A1 to An and B1 to Bn ). The elements of the two sets are subtracted one by one (|A1 B1 |, |A2 B2 |, . . . , |A(n1) B( n 1)|, |An Bn |). Thus, a set with the blocking probability differences BPDk for every hour within the pair of days is formulated: BPDk [i] = |setA[i] setB[i]| (30)

where k is the selected pair and i is the i-element of each set. The difference setA[i] setB[i] is calculated only if the setA[i] and setB[i] values are not outliers otherwise the data pairs are rejected. The mean Mk is calculated for all the possible pairs: Mk [] =
n i=1 BPDk [i]

(31)

Finally, the periodicity measurement by calculating the mean value among the mean Mk is performed. When the difference between elements within pairs of different days tends to zero, the pairs tend to be identical, therefore the periodicity factor tends to zero for the given pair. When more pairs have small periodicity factor, the mean factor value from more pairs is small too. Smaller periodicity factor means higher periodicity and the behavior of the simulation model is more stable with regard to daily model in simulation time.

P.M. Papazoglou et al.

Fig. 16 Simulation model validation

Fig. 18 Standard deviation of the Monte Carlo simulation execution for blocking probability

Fig. 17 Blocking probability of the tested approaches

Fig. 19 Dropping probability of the tested approaches

3.2 Simulation model validation Simulation model has been validated at the beginning of the experiments. This validation shows the experimental model performance versus the theoretical approach. Figure 16 shows the validation curve.

4 Simulation results The following simulation results show the corresponding performance in terms of blocking and dropping probability respectively as well as the model stability which is represented with standard deviation. Figure 17 shows clearly the performance improvement when the multi agent model is applied for efciently channel

allocation. The same gure shows also the stability of the specic statistical metric. Figure 18 shows the model stability. It is important to note that the standard deviation of the stability of the model (std in the above diagram) is also smaller as compared to the existing approach without the application of the multi agent model. More precisely, the standard deviation of stability is 6.178 104 and 2.368 103 for the proposed and existing approaches respectively. Figure 19 shows also the performance improvement when the multi agent model is applied for efciently channel allocation regarding the dropping probability metric. Figure 20 shows the model stability in terms of dropping probability. The corresponding standard deviation of the stability of the model (std in the above diagram) is smaller as compared to the existing approach without the application of

On Multi Agent based modeling and control in large scale wireless communication systems for improved

Fig. 22 Dropping periodicity Fig. 20 Standard deviation of the Monte Carlo simulation execution for dropping probability

events as they happen in a real concurrent environment. Towards the desired simulation model at least two stepsphases should be followed, (a) theoretical analysis for discovering the modeling aspects of the involved technologies (multi agents and wireless networks) in order to nd out the best matching, (b) simulation model design representing synthesis of the mentioned modeling aspects and involving agent control and implementation issues. Moreover, this paper shows exactly how a multi agent model affects the simulation model behavior towards the desired channel allocation improvement. The simulation results clearly shows that the proposed approaches improve signicantly the derived performance as compared to existing methodologies so far in the literature.

References
Fig. 21 Blocking periodicity 1. Bodanese, E. L. (2000). A distributed channel allocation scheme for cellular networks using intelligent software agents. Unpublished Doctoral dissertation, University of London, UK. 2. Bigham, J., & Du, L. (2003). Cooperative negotiation in a MultiAgent system for RealTime load balancing of a mobile cellular network. In AAMAS03, July (pp. 1418). 3. Brown, R. (1988). Calendar queues: a fast O (1) priority queue implementation for the simulation event set problem. Communications of the ACM, 31(10), 12201227. 4. Cheng, M., Li, Y., & Du, D. Z. (2005). Combinatorial optimization in communication networks. Berlin: Springer. 5. Cherriman, P., Romiti, F., & Hanzo, L. (1998). Channel allocation for third-generation mobile radio systems. In ACTS98 (Vol. 1, pp. 255261). 6. Ekdahl, B. (2001). How autonomous is an autonomous Agent. In Proceedings of the 5th conference on systemic, cybernetics and informatics (SCI 2001), Orlando, USA 7. Godara, L. C. (1997). Applications of antenna arrays To mobile communications. Part I. Performance improvement, feasibility, and system considerations. Proceedings of the IEEE, 85(7).

the proposed multi agent model. The standard deviation of stability is 4.388 105 and 7.382 105 for the proposed and existing approaches respectively. The proposed metric of periodicity shows also that the simulation model is stable between two or more successive days of network operation (Figs. 21 and 22).

5 Conclusions and future trends A novel model based on the multi agent technology for channel allocation performance improvement has been presented in this paper. This study clearly shows that the proper application of a multi agent model in simulating wireless communication systems can represent concurrent network

P.M. Papazoglou et al. 8. Grace, D. (1998). Distributed dynamic channel assignment for the wireless environment. Unpublished doctoral dissertation, University of York. 9. Haas, H. (2000). Interference analysis of and dynamic channel assignment algorithms in TDCDMA/TDD systems. Unpublished doctoral dissertation, University of Edinburg. 10. Hollos, D., Karl, H., & Wolisz, A. (2004). Regionalizing global optimization algorithms to improve the operation of large ad hoc networks. In Proceedings of the IEEE wireless communications and networking conference, Atlanta, Georgia, USA 11. Huhns, M., & Singh, M. (Eds.) (1998). Agents and multiagent systems: themes, approaches, and challenges. Readings in agents, Kaufmann, Los Altos (pp. 123). 12. Iraqi, Y., & Boutaba, R. (2000). A multi-agent system for resource management in wireless mobile multimedia networks. Lecture notes in computer science: Vol. 1960 (pp. 218229). Berlin: Springer. 13. Katzela, I., & Naghshineh, M. (1996). Channel assignment schemes for cellular mobile telecommunication systems: a comprehensive survey. IEEE Personal Communications, 3, 1031. 14. Lee, W. C., Lee, J., & Huff, K. (1999). On simulation modeling of information dissemination systems in mobile environments. In Lecture notes in computer science: Vol. 1748 (pp. 4557). 15. Norman, T., & Long, D. (1995). Goal creation in motivated agents. In M. Wooldridge & N. R. Jennings (Eds.), Lecture notes in articial intelligence: Vol. 890. Intelligent agents: theories, architectures, and languages. 16. Papazoglou, P. M., Karras, D. A., & Papademetriou, R. C. (2008). An improved multi-agent simulation methodology for modeling and evaluating wireless communication systems resource allocation algorithms. Journal of Universal Computer Science, 14(7), 10611079. 17. Papazoglou, P. M., Karras, D. A., & Papademetriou, R. C. (2008). On the multi-threading approach of efcient multi-agent methodology for modeling cellular communications bandwidth management. In Lecture notes in computer sciences: Vol. 4953. 2nd KES symposium on agent and multi-agent systems (pp. 431443). 18. Russell, S., & Norvig, P. (2002). Articial intelligence: a modern approach (2nd ed.). New York: Prentice Hall. 19. Splunter, S., Wijngaards, N., & Brazier, F. (2003). Structuring agents for adaptation. In E. Alonso et al. (Ed.), Lecture notes in articial intelligence: Vol. 2636. Adaptive agents and multi-agent Systems (pp. 174186). 20. Salgado, H., Sirbu, M., & Peha, J. (1995). Spectrum sharing through dynamic channel assignment for open access to personal communications services. In Proc. of IEEE intl. communications conference (ICC) (pp. 417422). 21. Tripathi, N. D., Jeffrey, N., Reed, H., & VanLandingham, H. F. (1998). Handoff in cellular systems. IEEE personal communications. 22. Wong, S. H. (2003). Channel allocation for broadband xed wireless access networks. Unpublished doctorate dissertation, University of Cambridge. 23. Wooldridge, M. (2004). An introduction to MultiAgent systems. New York: Wiley. 24. Wooldridge, M., & Jennings, N. R. (1995). Intelligent agents: theory and practice. Knowledge Engineering Review, 10(2), 115152. P.M. Papazoglou received the Ph.D. degree in cellular communication systems simulation from University of Portsmouth (2010-Department of Electronic and Computer Engineering). He also received the M.Sc. degree in Data Communication Systems from Brunel University (1999Department of Electrical and Electronics Engineering). Dr. Panayotis M. Papazoglou works as a Lecturer of informatics in the Department of Informatics and Computer Technology of Lamia Institute of Technology, Greece since 2002. Also, he received two best paper awards in two international conferences (Industrial Conference on Data Mining, Berlin, 2009 and 2nd KES Symposium on Agent and Multi-Agent Systems, Korea, 2008). Dr. Panayotis M. Papazoglou is the author of ve books (Greek) of informatics and more than 30 international publications so far. D.A. Karras received his Diploma and M.Sc. Degree in Electrical and Electronic Engineering from the National Technical University of Athens, Greece in 1985 and the Ph. Degree in Electrical Engineering, from the National Technical University of Athens, Greece in 1995, with honors. From 1990 and up to 2004 he has collaborated as visiting professor and researcher with several universities and research institutes in Greece. Since 2004, after his election, he has been with the Chalkis Institute of Technology, Automation Dept., Greece as associate professor in Digital Systems and Signal Processing as well as with the Hellenic Open University, Dept. Informatics as a visiting professor in Communication Systems (since 2002). He has published more than 50 research refereed journal papers in various areas of pattern recognition, image/signal processing and neural networks as well as in bioinformatics and more than 150 research papers in International refereed scientic Conferences. His research interests span the elds of pattern recognition and neural networks, image and signal processing, image and signal systems, biomedical systems, communications, networking and security. He has served as program committee member in many international conferences, as well as program chair and general chair in several international workshops and conferences in the elds of signal, image and automation systems. He is, also, editor in chief of the International Journal in Signal and Imaging Systems Engineering (IJSISE), topics editor in chief of the International Journal of Digital Content Technology and its Applications (JDCTA) as well as associate editor in various scientic journals. He has been cited in more than 350 research papers, his h-index is 8 and his Erdos number is 5.

On Multi Agent based modeling and control in large scale wireless communication systems for improved R.C. Papademetriou is a Principal Lecturer at the Electronic & Computer Engineering Department, Portsmouth University, Anglesea Building, Anglesea Road, Portsmouth, Hants, PO1 3DJ, UK. He is a lecturer in electronic systems and analytical management techniques. He is the admissions tutor for students from Greece and he is the University contact for M.Sc. Greek franchises. Rallis is a member of the Communications and Networks Engineering Group.