A Kind of Adaptive Negotiation Mechanism for Flexible Job Shop Scheduling*

Cixing Lv1,2, Zhongqi Sheng1, Sufen Li1, Chaowan Yin1, and Yunlong Zhu1
1

Shenyang Institute of Automation, Chinese Academy of Sciences 110016 Shenyang, China {smale, shengzhonqi, lisufen, ycw, ylzhu}@sia.cn 2 Graduate School of the Chinese Academy of Sciences 110089 Beijing, China

Abstract. Agent-based production scheduling is a promising approach to solve production scheduling problem, especially in a dynamic, uncertain environment. In the system, agents are connected through a network and negotiate with each other to fulfill scheduling. The negotiation mechanism specifies the way in which negotiation should take place. This paper proposes an adaptive negotiation framework and two kinds of negotiation policies to fulfill scheduling and rescheduling in the flexible job shop. The mechanism makes the system more adaptive in dynamic production environments. The computational experiments are given to demonstrate the feasibility and performance of the mechanism.

1 Introduction
In agent-based production scheduling system, agents are used to encapsulate physical, logic and/or function entities in the production system [1-4]. While making autonomous decision, agents in the system interacts and negotiates with each others to fulfill scheduling [4]. In the system, negotiation mechanisms play a key role in managing message passing, the behaviors of agents, reaching closure, and determining the final schedule. The negotiation policy can be classified into planning type and reactive type. The planning type, such as combinatorial auction [5,6], can get optimal or near-optimal schedule result in distributed framework. But this type of policy lacks of capacity to react to disturbances in dynamic environment. The reactive type allocates tasks or resources in real time manner, such as contract net [7], voting [8], etc. It behaves well in dynamic environment and balances the loads of agents. Therefore it is adopted in production scheduling system widely. However, the agents that adopt this type of negotiation policy make decision only according to local information and current state of the agents. It is hard to predict the performance of the result. The real world manufacturing problems are dynamic and tightly coupled. A slight change may make a previously ideal schedule totally unacceptable. Agents must have an efficient way to select right negotiation policies, related negotiation strategies and negotiation partners under various environment situations [9].
*

The research is supported by National Basic Research Program of China (No. 2002CB 312204 03) and National Nature Science Foundation of China (No. 70431003).

D.-S. Huang, K. Li, and G.W. Irwin (Eds.): ICIC 2006, LNAI 4114, pp. 324 333, 2006. Springer-Verlag Berlin Heidelberg 2006

A Kind of Adaptive Negotiation Mechanism for Flexible Job Shop Scheduling

325

In that case, such negotiation mechanisms, which achieve good performance and keep responsive capacity to handle disturbance at the same time, have not hitherto been considered, and so we consider them here. The adaptive negotiation mechanism utilizes an adaptive negotiation framework and two kinds of negotiation policy to fulfill scheduling and rescheduling problem of flexible job shop. The design and description of the negotiation mechanism are presented. In the last part of the paper, a prototype system and two experiments are presented to demonstrate and verify feasibility and performance of the mechanism.

2 The Adaptive Negotiation Mechanism

2.1 Problem Description The problem considered here is scheduling and rescheduling of flexible job shop in dynamic production environments. The scheduling problem is to schedule N tasks on M resources in order to minimize the total tardiness of the tasks. If there is disturbance in the system, such as emergency task, breakdown of resource or task cancelled, original schedule is not feasible, and rescheduling is needed to keep system running and keep performance of the system in certain extent. Adaptive negotiation mechanism is considered as an approach to increase the adaptability of the system. By adaptive negotiation more intelligence and rationality are integrated into negotiation mechanisms, thus makes the system adaptive in dynamic environments. The negotiation mechanism is presented below. The symbol used is presented as following: Agent types Part Agent PA; Resource Agent RA; Part Mediate Agent PMA; Resource Mediate Agent RMA; Job Manage Agent JMA; Mobile Submit Agent MSA. Variables O (i , j ) The j th subtask of the task i ; T The length of the planning horizon;

M The number of resources; t The index of time slot; h The index of resource; h , t The price for time slot t for resource h ; N The number of tasks;
ni Number of subtask of task i ; pi , j , h processing time of O (i , j ) if it is processed

on h H i , j , otherwise 0; i Weight of task i ; H i , j Resource set capable processing O (i , j ) ; I i , j Set of subtasks preceding immediately O (i , j ) ; TDi , t Cost for task i to complete in time slot t ; X i , j ,t = 1 if O (i , j ) is complete at time t , 0 otherwise; Yi , j , h =1 if O (i , j ) is processed on resource h , 0 otherwise. Logical symbols NOT ~; AND I ; OR U . Events and messages Resource breakdown Br; Emergency task E; Task cancelled C; Preparation for combinatorial auction negotiation mechanism Aauction; Adoption of disturbance

326

C. Lv et al.

handling negotiation mechanism Adynamic; All PAs task lists are empty Pempty; Start auction Sa; Stop auction Pa; MSAi is ready Mr. Bi Bid submitted by MSAi; PAis planning list Pli; Auction result Ar; Mlist MAjs planning list. State of scheduling policy: Salloc Static state; Dalloc Dynamic state. Value of auction state (AuctionState): Auction is initiated Sinit; MSA is dispatched Sdisp; A new round auction starts Snew_round; The prices is announced Sann; Bid is constructed Scon_bid; Bid is submit Ssub_bid; Market is cleared Scl_mar; Revenue is calculate Scal_rev; Prices is updated Supd_pri; Result is notified Sresult; Auction is end Send; MSA returns Sreturn; Auction is stopped Sstop. Basic behavior of agents SEND (message1<content1>, message2<content2> ) TO (agent1, agent2 ) SET_STATE (state1, state2 ) TO (state_new1, state_new2 ) SET_CONTEXT(context1, context2 ) DISPATCH (mobile_agent) TO (agent) (mobile_agent) RETURN TO (agent) CREATE (agent) DESTROY (agent) UNREGISTER (agent)
2.2 The Adaptive Negotiation Framework

To realize adaptive negotiation in the agent-based system, an appropriate framework is important [9]. The negotiation mechanism makes the system adaptive to the dynamic environments by applying different negotiation policy. The adaptive negotiation framework is proposed as a foundation for management of negotiation policies in the system. The framework implements adaptive characteristics of agent negotiation by system architecture, knowledge to select negotiation policy and different negotiation policy. The system architecture has some important characteristics which explicitly support adaptive negotiation among agents. The knowledge is integrated into the agent negotiation mechanism is used by agents in terms of rational decision making. Two kinds of negotiation policy and some selection heuristics have been presented as well. Furthermore, the preprocessing for switch negotiation policy is presented.
2.2.1 The Heuristics to Select Negotiation Protocols In the adaptive negotiation framework, negotiation policy selecting heuristics play an important role. They are used to select suitable negotiation policy under different situations. For the agent-based production scheduling, some domain specific heuristics are adopted to distinguish the important characteristics of different negotiation policies under different environment conditions. The selection process is shown in the Fig. 1. While in smooth progress, corresponding agents uses combinatorial auction negotiation policy to allocate tasks. The auction result is global near optimal. If disturbance happens, such as emergency task, task cancelled and resource breakdown, the combinatorial auction would be stopped. And after disturbance preprocessing is done, modified contract net would be initiated to allocate the tasks that are not processed. After that, according to the state of the system, combinatorial auction or modified contract net is used to schedule tasks. The corresponding behaviors of agents and its description are given in the table 1.

A Kind of Adaptive Negotiation Mechanism for Flexible Job Shop Scheduling

327

Fig. 1. The process of adaptive negotiation mechanism Table 1. The switch between different kinds of negotiation mechanism

Behavior agent Adopt combinatorial auc- JMA tion mechanism Adopt disturbance dling mechanism han- JMA

Behavior description Context: Pempty Behavior: SET_STATE (T') TO (Salloc); SEND (Aauction) TO (RMA) Context: Br U E U C Behavior: SET_STATE (T') TO (Dalloc); SEND (Adynamic) TO (RMA,MAs,PAs,PMA)

2.2.2 The Prepossessing for Switching Different Negotiation Protocols If disturbance happens, rescheduling is needed to keep system running and keep certain performance of the system. Rescheduling is done in two steps. The disturbance is handled and then tasks left in the Combinatorial Auction Time Window (CATW) are allocated through modified contract net. Preprocessing negotiation mechanism specifies corresponding agents to handle disturbance. The process of preprocessing is shown in Fig. 2. The preprocessing is performed according to the type of disturbance.

Fig. 2. The process of disturbance preprocessing

328

C. Lv et al.

The purpose of preprocessing is to recover from disturbance and prepare for task allocation through modified contract net. The preprocessing is done by corresponding agents behaviors (shown in table 2).
2.3 Description of the Negotiation Policies 2.3.1 The Combinatorial Auction Negotiation Policy Combinatorial auctions commonly refer to auction mechanisms in which agents are allowed to bid on combinations, or bundles f items. Being able to bid on bundles clearly mitigates the exposure problem, since it gives the agents the option to bid their precise valuations for any collection if items they desire. On the other hand, combinatorial auctions require the market maker, the agents, or both, to solve complex decision problems. For the scheduling problem, the time window considered for scheduling is divided into T time slots equably. All resources delegate their time slots to a mediator who acts as s-agent, jobs act as b-agents and auction items are the time slots of the resources and defined as a set of pairs (resource, timeslot). Each job competes for time slots with its precedence constraint, process requirement and local utility function. The s-agent adjusts the price of each time slot according to the demand from b-agents to reduce the resource conflicts among the b-agents until result of auction is obtained, and the resource and complete time for all operations in each job are determined. The schedule generated according to auction result is near-optimal [10]. Formally, we have: Definition 1 The descriptor of a CA is PCA = G, Buyer , S , p , Bid , tround , where:

B = {b1 , b2 ,L , bn } is finite set of identifiers of b-agents, where n is the number of b-agents, also, the number of jobs. S is the identifiers of seller-agent. Bid = { Bid1 , Bid 2 ,L , Bid n } is the bids constructed by the b-agents, where

G = { g h ,t } is the goods, time slots on resources, to be auctioned.

Bidi =

Here, pi is the payment for the bid, and triple Bidi , j ( mi , j , ti , j , li , j ) presents requirement of O ( i, j ) for timeslots, mi , j presents resource, ti , j presents start index of time slot, li , j presents the number of time slots which means the processing time on the resource. tround is used for defining the condition for terminating the CA, that is, the maximum number of iterations is reached or the result is acceptable, the CA terminates.
Definition 2 A round in a CA is the time period between two successive deals or the period from the beginning of the CA to the time when the first deal takes place. If a round is the

{( Bid

i ,1

, Bidi ,2 ,L , Bidi , ni , pi

) } is

the bid submitted by the ith b-agents.

A Kind of Adaptive Negotiation Mechanism for Flexible Job Shop Scheduling

329

rth ( r IN + ) round of the CA, then r is called the round number. A CA consists of multiple rounds. The following is the formal definition of the valid behaviors of agents during a CA. Definition 3 The CA protocol with descriptor PCA consists of the following steps: Each PA dispatches a MSA to the platform in which RMA lies in.

1. MSAs with routing, precedence constraints, utility function and construct bid method are dispatched into the platform which RMA lies in. 2. r = 1 , h,t = 0 .
3. A new round of the CA starts, r = r + 1 , RMA clears market, updates p and announces new prices to all MSAs. 4. MSAs construct bids to minimize the utility function under current prices and submit them to RMA. 5. If closure is reached goto 3, else goto 6. 6. MSAs bring result including the processing resource and starting time of each subtask to the corresponding PA. The reason for introducing MSA is to improve the communication efficiency, because the communication cost in same platform is 10 times lower than that in different platform [10]. For the design of combinatorial auction, Lagrangian Relaxation approach is adopted [11]. by In combinatorial auction, the communication cost is C = O ( N R ( M T + ni ) ) for each round auction, if platform factor (JADE [12], Java Agent Development Framework) is considered, the communication cost is adjusted to C = O ( N 2 R ( M T + ni ) ) [11].
2.3.2 Modified Contract Net Negotiation Policy After disturbance preprocessing is done, PAs compete for process time on resource through modified contract net until all tasks left in the CATW are allocated. According to the requirement to allocate tasks, several improvements are proposed to basic contract net protocol. The first improvement is sending offers to a limited number of agents instead of broadcasting them. And subtasks satisfying precedence constraints can send offers in parallel. At last the reactive time of agents is limited. There are three time windows: T1 for MA to receive offers, T2 for PA to select subtask and T3 for MA to confirm. In modified contract net, PAs select subtasks whose precedence subtasks have been allocated and sends offers to corresponding MAs. If MAs receive first requirement and the requirement list is empty, the T1 time window of the MA starts timing. When the T1 is out, the MA sorts the offers in the list according to its own objective and selects the first offer and notifies result to the concerned PA. And the T3 time window starts timing. When the PA receives first notification, the T2 time window starts timing. And the PA puts the notification into counteroffer list. When the T2 is out, the PA sorts the notification according to its own objective, selects the first notification and sends confirmation to the MA. If the MA receives confirmation message in the T3 time window, the MA would put the subtask into its planning list and send re-confirmation to the PA. The process of modified contract net is shown in Fig. 3.

330

C. Lv et al.

Fig. 3. Negotiation process of the modified contract net

3 Experiments
Based on prototype system, two experiments are used to demonstrate the feasibility and efficiency of negotiation mechanism. Experiment one compares combinatorial auction negotiation mechanism with dispatching rule for tardiness and flowing time. Experiment two shows a rescheduling example after disturbance happens. Parameters in the experiments are set as following. Because the value of affects the performance of the system, the value of is specified for 1.05, 1.1, 1.15, 1.2, 1.25. Also, considering of the dynamic character of the environment, the negotiation mechanism must has the character of anytime, which means that a feasible solution can be available whenever the negotiation is stopped. In real environment, the cycle times of the auction cant be guaranteed. For the reason, cycle times of 100, 200, 300, 400 and 500 are adopted respectively. Therefore, total 25 times for each experiment were performed and the average value of the experiment was obtained.
3.1 Experiment One

This experiment is designed to compare the performance of the negotiation mechanism with dispatching rules for tardiness and flowing time. Four groups data, including 66, 88, 1010, 1212, are generated randomly. The subtasks number is six in former two groups and eight in latter two groups. The experiment data is generated randomly in the manner as same as reference [13]. The parameters for generating are described as following. The process time of each subtask is an integer chosen randomly from [3,12]. The alternative resource number is chosen randomly from [0,2]. The process time in alternative resource is the process time in original resource multiply a coefficient chosen randomly from [0.5,2]. The weight of each production task is 1.0. The due date of each production task is calculated by

( min { p
N i =1

i, j,h

h Hi, j

}) .

The number of direct precedence of subtask is chosen randomly from [0,3]. Using above data, the performance of negotiation mechanism is compared to SPT and EDD dispatching rule.

A Kind of Adaptive Negotiation Mechanism for Flexible Job Shop Scheduling

Table 2. Simulation result of experiment one

331

problem 6 6

8 8

1010

1212

method SPT EDD Combinatorial auction SPT EDD Combinatorial auction SPT EDD Combinatorial auction SPT EDD Combinatorial auction

Total tardiness 749 855 126 1945 1706 593 3804 2963 818 4158 4267 1121

Flowing time 56 58 45 65 79 60 134 177 110 174 187 133

Using above data, average value of total tardiness and flow time of all tasks are got. As shown in table 2, combinatorial auction negotiation mechanism gets better performance in total tardiness comparing to SPT and EDD dispatching rules. In addition, the experiment shows that the negotiation mechanism does better in flowing time comparing to dispatching rules.
3.2 Experiment Two Example of Rescheduling

Take 66 experiment data in experiment one to describe rescheduling example after disturbance happens. The disturbances include resource breakdown, emergency task

Fig. 4. Example of rescheduling after disturbance

332

C. Lv et al.

and task cancelled. In the example, MA takes SPT dispatching rule, PA takes EDD dispatching rule, and we suppose the part, which is processing on the resource that is breakdown, need to be processed from beginning. The result of rescheduling is shown in Fig. 4. The example shows that the negotiation mechanism can fulfill rescheduling when disturbance happens in the system. And the negotiation mechanism is adaptive to dynamic environments, whereas the rescheduling is feasible but not optimal now.

4 Conclusions
Based on survey of negotiation mechanism in the agent-based production scheduling, an adaptive negotiation mechanism is proposed in the paper. The adaptive negotiation mechanism includes an adaptive negotiation framework, knowledge to select negotiation policy and two kinds of negotiation policies. The mechanism is designed to schedule production tasks with flexible routing and in-tree precedence constraints in dynamic environment in distributed manner. A prototype system is developed and two experiments are performed based on the mechanism to demonstrate the mechanism to be feasible and effective. The rescheduling heuristics in the paper is feasible but not optimal. According to specified production environment and production objective, appropriate selection of rules is needed to achieve optimal result. The selection of the rules hitherto mainly depends on experiences. How to transform the experiences into agents knowledge and help agent to make decision to improve the performance of rescheduling is our future work.

References
1. Rabelo, R.J., Camarinha-matos, L.M., Afsarmanesh, H.: Multi-Agent-Based Agile Scheduling. Robotic and Autonomous System 27 (1999) 15-28 2. Wang, Y.H., Yin, C.W., Zhang, Y.: A Multi-Agent and Distributed Ruler Based Approach to Production Scheduling of Agile Manufacturing Systems. International Journal Computer Integrated Manufacturing 16(2) (2003) 81-92 3. Wang, Y.H., Yin, C.W.: Multi-Agent and Distributed-Rules Based Agile Production Scheduling Approach. Control Theory & Application 21(4) (2004) 526-530(in Chinese) 4. Shen, W.: Distributed Manufacturing Scheduling Using Intelligent Agents. IEEE Intelligent Systems 17(1) (2002) 88-94 5. Kutanoglu, E., Wu, S.D.: On Combinatorial Auction and Lagrangean Relaxation for Distributed Resource Scheduling. IIE Transactions 31(9) (1999) 813 826 6. Dewan, P., Josh, S.: Auction-Based Distributed Scheduling in a Dynamic Job Shop Environment. International Journal of Production Research 40(5) (2002) 1173-1191 7. Smith, R.G.: The Contract Net Mechanism: High-Level Communication and Control in a Distributed Problem Solver. IEEE Trans. Computers 29(12) (1980) 11041113 8. Fordyce, K., Sullivan, G.: Logistics Management System (LMS): Integrating Decision Technologies for Dispatch Scheduling in Semiconductor Manufacturing, Chapter 17 in Intelligent Scheduling, edited by Mark Fox and Monte Zweben, Morgan Kaufman Publishers, (1994) 473-516

A Kind of Adaptive Negotiation Mechanism for Flexible Job Shop Scheduling

333

9. Wang, C., Shen, W.M., Ghenniwa, H.: An Adaptive Negotiation Framework for Agent Based Dynamic Manufacturing Scheduling. In: IEEE International Conference on Systems, Man and Cybernetics (2003) 1211-1216 10. Cortese, E.: Benchmark on JADE Message Transport System [EB/OL]. http:// jade.cselt.it/doc/tutorials/benchmark/JADERTTBenchmark.htm (2003) 11. Lv, C.X., Zhu, Y.L., Yin, C.W.: Resource Allocation Based on Combinatorial Auction in E-Manufacturing Environment. In: The Fifith International Conference on Computer and Information Technology (CIT 2005), Los Alamitos, California: IEEE Computer Society (2005) 983-989 12. Te, S.C., He, T.: Project. Java Agent Development Framework Is an Open Source Platform for Peer-To-Peer Agent Based Applications [EB/OL]. http://jade.cselt.it/ (2004) 13. Chen, H., Chu, C., Proth, J.M.: An Improvement of the Lagrangean Relaxation Approach for Job Shop Scheduling: A Dynamic Programming Method. IEEE transactions on robotics and automation 14(5) (1998) 786-795 14. Debra, J.H., Peter, B.L., Krishna, R.P.: A Practical Approach to Job-Shop Scheduling Problems. IEEE transactions on robotics and automation 9(1) (1993) 1-13 15. Chen, H.X., Peter, B.L.: An Alternative Framework to Lagrangian Relaxation Approach for Job Shop Scheduling. European Journal of Operational Research 149(3) (2003) 499-512