PRESENTED BY

LEARNING AGENTS LABORATORY GEORGE MASON UNIVERSITY

Disciple-RKF/COG: Agent Teaching by Subject Matter Experts

SPONSORED BY

The Summary of the DEMO

About Disciple-COG

Students use of Disciple-COG as a learning assistant

1. Specifying a scenario 2. Viewing the solutions generated by Disciple 3. Generating a center of gravity analysis report

Teaching Disciple-COG by a subject matter expert

1. 2. 3. 4. 5. 6. Specifying a training scenario Modeling experts reasoning Task and rule learning Rule refinement Exception handling Problem solving

Tools for the knowledge engineer

1. Ontology development tools 2. Ontology import tools

About Disciple-COG
Disciple is a theory, methodology, and learning agent shell for rapid development of knowledge bases and agents, by subject matter experts, with limited assistance from knowledge engineers

Disciple learns from the expert, building, verifying and improving its knowledge base The expert teaches Disciple in a way that resembles how the expert would teach a person.

DISCIPLE SYSTEM Problem Solving Learning

Interface

Ontology + Rules

Demo overview

First we will demonstrate how a developed Disciple agent helps the students at the US Army War College to learn about Center of Gravity analysis. Then, in the main part of the demo, we will show how this Disciple agent was developed by being taught the problem solving method of a military expert. Finally, we will demonstrate additional tools that are used by the knowledge engineer to perform knowledge base development tasks that are currently beyond the capabilities of a subject matter expert.

Students use of Disciple-COG as a learning assistant

In the Case Studies in Center of Gravity Analysis course students learn to identify the centers of gravity (COG) of the opposing forces in military conflicts. The COG of a force is its main source of strength, power, and resistance. Each student has to study a historic scenario (such as the World War II invasion of the island of Sicily by the Allied Forces) and has to develop a center of gravity analysis report. Disciple guides the students to specify the relevant aspects of the assigned war scenario. Then it identifies and tests the strategic center of gravity candidates for that scenario, and generates a center of gravity analysis report. The students study and critique the solutions generated by Disciple and finalize the report.

1. Scenario Elicitation
First the student is guided by the scenario elicitation tool to specify the relevant aspects of the war scenario.

1. Scenario Elicitation
The tool shows:

A table of contents that will be developed during scenario elicitation. A list of questions that have to be answered by the student.

1. Scenario Elicitation

The student is asked to provide a short English description of the scenario.

1. Scenario Elicitation

The student is asked to provide a short English description of the scenario.

1. Scenario Elicitation

Then the student has to name the opposing forces of the scenario.

Disciple assists by providing clarifications for each requested piece of information.

1. Scenario Elicitation
Each opposing force is automatically introduced into the table of contents

When the student selects one of these forces, Disciple asks more specific questions about it.

1. Scenario Elicitation

Because the student has characterized Allied_Forces_1943 as a multi state force, Disciple expands the table of contents with the aspects that are relevant for such a force.

1. Scenario Elicitation

When the student selects one of these aspects, Disciple asks more specific questions about it.

1. Scenario Elicitation

In this way the student is guided to specify the relevant aspects of the scenario.

2. Viewing the solutions generated by Disciple

After the scenario is specified Disciple can identify and test the corresponding strategic center of gravity candidates

2. Viewing the solutions generated by Disciple

The tool shows:

A description of the selected candidate and the rationale for its identification and testing

The strategic center of gravity candidates for each of the opposing forces

2. Viewing the solutions generated by Disciple

The solution includes a summary of the specific candidates solution.

2. Viewing the solutions generated by Disciple

A justification for the entities selection as a candidate.

2. Viewing the solutions generated by Disciple

And a summary of the candidates testing.

2. Viewing the solutions generated by Disciple

The justification may be presented in a more abstract form

2. Viewing the solutions generated by Disciple

Or it may be presented in a more detailed form

2. Viewing the solutions generated by Disciple

Or it may be presented in a more detailed form

3. Generating a center of gravity analysis report

Then Disciple generates a draft center of gravity analysis report.

3. Generating a center of gravity analysis report

The first part of the report is a description of the scenario which is based on the students specification. The student may further improve it by using a text editor.

3. Generating a center of gravity analysis report

The second part of the report is a list of COG candidates and their justifications. The student has to critique Disciples reasoning and specify his own justification when in disagreement with Disciple.

Teaching Disciple-COG by a subject matter expert

Now we will discuss how this Disciple agent was developed. First a knowledge engineer has used the tools of Disciple to build an object ontology that represents the concepts from the COG analysis domain. He has also defined elicitation scripts to guide the user in defining a specific scenario. After that an expert can teach Disciple his own reasoning in center of gravity analysis, with only limited assistance from the knowledge engineer. The teaching process consists of the sequence of displayed steps and results in a set of problem solving rules learned by Disciple. In the following we will demonstrate this teaching process.
<object> Scenario Force Force_goal Strategic_COG_relevant_factor Strategic_ Operational_ goal goal Other_relevant_factor Demographic_factor Civilization_factor Psychosocial_factor Historical_factor Geographical_factor International_factor Military_factor Political_factor resource_ or_ infrastructure_element

Economic_factor

Initial object ontology

Scenario elicitation Modeling Task learning Rule learning Rule refinement Problem solving
<object> Scenario Force Force_goal Strategic_COG_relevant_factor Strategic_ Operational_ goal goal Other_relevant_factor Demographic_factor Civilization_factor Psychosocial_factor Historical_factor
in sta n ce_o f

resource_ or_ infrastructure_element

IF the task is

IF thetas k is

Economic_factor

Scenarios (elicited)

Geographical_factor

International_factor
i nsta nce_ of Brita in _1 94 3 com pon en t_ st a te

Political_factor

S icily _1 94 3 ha s as o pp o sin g fo rce ins tan ce _o f ins tan ce _o f A ng lo _a lli es _1 94 3 h as _a s_ pr im ary_ fo rc e_ el em en t b ri ef_ descrip tion in sta nce_ of ty pe _o f_ op era ti on s W W II A lli ed i n vasi on o f S i ci ly in 19 43

Military_factor

insta nce_ of US _1 9 4 3 ha s_ as_ in dust ri al_ fa ct or ins ta nce_ of U S _7 th _A rmy _ (Fo rce_ 34 3) i nsta nce_o f in st a nce_ of B r_ 8 th_ A rm y_ (F orce _5 45 )

co mpo ne nt_ sta t e h as _a s_ su bg ro up

A ll i ed _f orce s_ op er ati on _Hu sk y

i n st an ce_o f ha s_a s_ su bg ro up h as_ as _sub gro up W e stern_ N a va l_ TF h as _a s_s ubg roup East ern_ N av al _ TF h as_a s_ su bg ro up ins ta nce_ of ha s_a s_ su bg ro up U S_ 9 t h_ Ai r_ Fo rce i nsta nce_ of N o rt hw est_A f ric a_ A ir_Fo rce

Identify t he strategic COG cand idat es withrespect to the Identif y the strate gic COGcandidateswith re spe ct to the industrial civilization of a statewhich isa m em ber of aforc e ind ust rial civilization of a state whichis amem ber of a force IF thetas k is IF the task is Thestate is ?O2 T he stateis ? O2 Id entify the strate gic COGca ndidates with re spe ct to th e Iden tifythestrateg ic C OGcandidateswith respec t tothe Theforce is ?O1 T he forceis ? O1 in dustrial civilization of a state whichisa mem ber of a force industrial c ivilization of a state wh ich is a m em berof a force IFstate thetas k is IF the task is expla nat ion explanatio n T he is ?O2 Th e state is ?O 2 Identify the strategic C OGcandidateswith respect to the Id t h e strategic COG cand idates with respect to th e ?O2 has_as _ind ustrial_factor ?O3 ?O 2 h as_as_industri l_factor T he force is ?O1 Th eentify force a is ?O 1 ?O3 ind ustrial civilization r of a fo rce in dustrial civilization e ?O3 is_a_m ajor_genera tor_of ?O4 ?O 3 is_a_m ajor_generator_o fk ?O4 IF th e task isof a statewhichis amembe IFstate thetas isof a state which isa member of aforc explanatio n exp lana tion Th e stateis ? O2 Thessential_go is ?O2 ?O4 IS strategic ally_essenti al_goods_or_m ateriel ?O 4 ISstrateg ically_e ods_or_m aterie l Iden tifythe strategic COG candidates with resp ect to the Identif y is the strate gic COG candidates with re spe ct to the ?O 2 h as_as_industri a l_factor O3 ?O2 has_a s_industrial_factor ?O 3 Th e force is ? O1 ? The force ?O1 industrial civiliz ation em berof aforce ind ust rIF ialthe civiltas ization ?O 3 is_a_m ajor_gen erator_o f ?O4 ?O3 is_a_m ajor_g enerator_of ?O4 Plausible Up per Bou ndn Con dition Plausible Upper Bound Condition IF the task isof a state which isa m k isof a state whichis amember of a force exp lanatio expla nation Thessential_go state is?O 2 T he stateis ? O2 ds_or_m ?O 4F IS strateg ically_e ods_or_m ateriel ?O4Force IS strategi cally _essential_goo ateriel ?O 1 IS ?O1 IS orce Id entify the strate gic COG ca ndidates with respect t o th e Identify thestrateg ic C OG can didateswith respec t tothe ?O 2 has_as_industrial_factor ?O2 has_as _ind ustrial_factor O3 Theforce is?O 1 ?O3 T he force is ?O1 ? 2 IS Force ?O2 IS F orce in dustrial civilization of a state which isa m em ber of aforc e ?O ind ustrial c ivilizati on of a state wh ich is a m em ber of a fo rce ?O 3 is _a_m ajor_g enerator_of ?O4 ?O3 is_a_m ajor_gen era tor_of ?O4 Plausible Upper Bound Condition Plausib le U pper Bound Co ndition IF thetas k is IF the task is explanatio n has_as_industrial _fac tor ? O3 has_as_ind ustrial_factor ?O3 T he state is ?O2 Th e state is ?O2 ?O 4 Force ISexplanation strateg icall y _essen tial_go ods_or_m aterie l strategically_essenti a l_goods_or_m ateriel ?O 1 IS ?O1 ?O4 IS IS Forc e Identif y the strategic C OGcandidateswith re spe ct to the ?O Iden tify the strategi c COG cand idates withrespect to the ?O2 has_as_industrial_factor ?O3 ?O 2 h as_as_industri l_factor ?O3 ?O3 IS Industrial_fa ctor 3 IS Ind ustrial_factor T he force is ?O1 Th e fo rcea is ?O1 ?O 2 IS r_generato Force IS Forc e ind ust rndi ial civilization r of a fo rce ?O2 industrial civiliz atio n of a statewhich is a m em ber of aforce ?O3 is_a_m ajor_generator_of ? O4 ?O 3 is_a_m ajor_generator_o f ?O4 is_a_majo r_of ?O4 is_a_m ajor_ge nerator_of ?O4 Pla usib le U pper Bound Co tion Plaus ible Up per Bou nd Con dition IF the task isof a state whichis amembe IF the task is nation exp lanatio n has_as_ind ustrial_factor ?O3 h as_as_i n du stria l_factor ?O3 Th e state is ? O2 The state is ?O2 ISexpla strategically _essent i al_g oo ds_o r_m ateriel ?O IS strateg ically_e ssential_go ods_or_m aterie l dates with respect t 4 IS Strategical l4 y_esse ntial_goods_o_materiel ?O4 IS Strategically ood s_o _m ateriel ?O 1 I?O4 S _essential_g Force ?O1 IS F orce Iden tify thestrategic C OG cand idateswithrespect ?O to the Id entify the strate gic3 COG ca ndi o th e ?O2 has_as_ind ustrial_factor O3 ?O 2 has_a l_factor ?O 3 IS Ind ustrial_factor ?O3 IS Indu strial_facto r s_industria Th e force is ? O1 ? Theforce is ?O1 ?O ?O 2 IS Force IS F orce industri a l civiliz atio no fa s tatewh ich is amem ber of a force ?O2 dustrial civilization of a state which isa m em ber of aforc e ?O3 is_a_m ajor_gen erator_of ?O4 ?O 3 is_a_ major_g enerato ?O4 is_ a_m ajor_ge nerator_of ?O4 is_a_m ajor_gen erator_o f in ?O4 Plausibl e Up per Condition Plausible Upper Bound Condition Plau sible Lo wer Bo und Con diti o n Bound IF the task is PlausibleLower Bound Condition IF ther_of tas k is explanatio n explanation h as_as_indu strial_factor ?O3 has_as_industrial _ fac tor ?O3 The state is ?O2 T he state isriel ?O2 ?O4 F IS strategically_essential_goods_or_m ateriel ?O 4Force ISsen strateg ically_essential_goods_or_mat eriel IS Strategically_esse ntial_goods_o_ma teriel IS Stra tegically_es tial_ goo ds_o_m ate IS orc e ?O 1 Industrial_factor IS ? O1 ?O IS 4 Anglo_allies ?O1 ?O4 IS An glo_allies_1943 Id entify t h strategic COG cand idates with respect to th e Identify strategic C OGcandidateswith re spect to the ?O 2 h as_as_industrial_factor ?O3 ?O2 has_as _in dustrial_fac tor ?O3 ?O 3 I?O1 S _1943 Indu strial_factor ?O3 IS The force i se ?O1 T he forcethe is ?O1 ?O2 IS F orc e 2 IS Force ? O2 IS US_1943 ?O2 ISaforc US _1943 in dustrial civilization e ?O ind ustrial civilization r of a fo rce ?O 3 is _a_m ajor_g enerator_o fk ?O4 ?O3 is_a_m ajo r_generator_of ?O4 is_a_ma jor_ generator_of ?O4 is_a_m ajo r_generato r_of ?O4 Plaus ible Upper Bou nd Condition Plausib le U pper Bound Co ndition Plausibledustrial_factor Lower Bou nd Condition IF the tas isof a state which isa member of Plausib le L ower Bound Co ndition IF the task isof a state whichis amembe exp lanation expla nation has_as_in dustria l_factor ? O3 has_as_ind ustrial_factor ?O3 has_as_in ? O3 has _as_industrial_ factor ?O3 The state is ?O2 Th e state is ? O2 ds_or_m ?O 4 ISstrateg ically_e ssential_go ods_or_m aterie l ates with respect IS strategic ally_essential_goo ateriel ?O IS Strategically_essential _ goods_o_m ateriel ?O4 IS lo_allie Strate gically_essential_g ood s_o _m ateriel ?O1 InIS Force ?O 1 I?O4 S Force Id entify the st r ategic COG candid to the Iden tify the strategi c COG cand idates withrespect to the ? O1Industrial_capacity_of_U IS 4 An glo _allies_1943 ?O1Ind IS Ang s_19 43 ?O2 has_as_in dus trial _ fac tor ?O3 ?O2 has_as_ind ustrial_factor O3 ?O3 IS du strial_factor ?O 3 IS Ind ustrial_factor ? O3 IS S_1943 ?O3 IS ust r ial_cap acity_o f_US_1943 The force ?O1 Th e force is ? O1 ? ?O2 IS Force ?O 2 IS Force IFis the isof astate which is a memb IF task is ? O2 maj IS or_g US _1943 ?O2 IS US_1943 in dustrial civitask lization er of a force industri a lthe civiliz n of a s tatewhich is a m em ber of aforce ?O3 is_a_m r_generator_of ?O4 ?O3 is_a_m ajor_gen erator_of ?O4 is _a_m ajor_g enerato r_o f ajo ? O4 is_ a_m ajor_ge nerator_of ?O4 is_a_ enerato r_o f Pla ?O4 is_a_m ajor_generator_of ?O4 usible U pper Bound Condition Plausible Up per Bo und Con diti on atio Plausible Lower Bound Condition Plau sible Lo wer Bou nd Con dition expla nation exp lanatio n has_as_ind ustrial_fac tor ?O3 h as_ as_indu strial_facto r ?O3 Iden tify the strategic C OG candidates with respect to the Id entify the strategic COG ca ndidates with respect to th e has _as_industrial_ factor ?O3 has_as_industri a l_factor ?O3 The state is ?O2 The state is ?O2 ?O4 IS strategically_essen tial_ goo ds_ or_m ateriel ?O4 I S strategically_essential_goods_or_m ateriel ?O4 IS S trategically_es sential_ goo ds_o_m ateriel ?O 4 IS Strategically_esse ntial_goods_o_materiel ? O4 IS War_ materiel_and_transports_of_US_1943 ?O4 IS Wa r_m ateriel_ and _transp orts _194 3 ?O 1 Industrial IS Force IS_of_US F orce ?O1 IS Ang lo_allies_1943 ? O1 IS Anglo_al lies _1943 ?O2 has_as _ind ustrial_factor ? O3 ?O 2 h as_as_industrial_factor ?O3 IS _factor ?O 3 I?O1 S Indu stri a l_facto r The The industrial force iscivili ?O1zatio no fa s tatewh ichis amem of a fo rce in dustrial force iscivilization ?O1 ?O3 of a state which isa m em ber of aforc e ? O3 IS In dustrial_cap aci ty_o f_U S_1943 ?O3 ISber Industrial_capacity _of_US_1943 ?O 2 IS Force ?O2 IS F orce ?O2 IS US_1943 ? O2_major_ IS US_1943 ?O3 is_a_m ajor_gen era tor_of ?O4 ?O 3 is _a_m ajor_g enerator_o ?O4 is_ a_m ajo r_generator_of ?O4 is_a_m ajor_gen erator_of ?O4 THEN TH EN Th e state is ?O2 The state isf?O2 is_a_m ajor_generator_of ? O4 is_a gen erator_o f Plaus ?O4 Plausib le U pper Bo und Co ndition ible Upper Bou nd Condition Plau sible Lower Bound Condition Plausible Lower Bou nd Condition n explanation has_as_ind ustrial_factor ?O3 has_a s_industrial_factor ? O3 tial_go has_as_ industrial_facto r IS ?O3 has_as_in dustrial_factor O3 ?O4 ISexplanatio strategically_essenti a l_goods_or_m ateriel ?O 4 IS strateg ically_e ssen ods_or_m aterie l ?O4 ISilo_allies_19 Strate gically_essenti al_g ood s_o _ma teri el ?O 4 IS Stra tegicall y _essen tial_goo ds_o_m ate Th e force i s ?O1 The force isriel ?O1 Conclud e that an econom ic factor s_transp a strategic COG nclude that an eco nom ic factor is_allies_1943 a strategic C OG ?O1 Force ?O1 IS? Force ? O4 IS War_m orts _of_US _1943 ?O4 IS War_materi e l_and_tran spo rts_of_U S_1943 ?O1 ateriel_and IS A ng 43 ?O1 IS An glo ?O 2 h as_as_indu strial_facto r ?O3 Co ?O2 has_as_in dus trial_fac tor ?O3 ?O 3 IS Ind ustrial_factor ?O3 IS Indu strial_fa ctor ?O3 IS Ind ustrial_capacity_of_US_1943 ? O3 IS In dustrial_capacity_of_U S_1943 candidate fo r a state wh ich is aU mem be r of a fo rceForce atefor a state whic h isa m em ber of aforc ?O2 IS ?O2 IS e Force ?O2 IS S_1943 ?O2 IS US _1943 exp lanation expla nation ?O 3 is _a_ major_gen erator_of ?O4 candid ?O3 is_a_m ajo r_generator_of is_a_m ajor_generator_of ?O4 is_a_major_generato r_o f ?O4 is_a_major_gen ?O4 is_a_ major_g enerato r_o f ? O4 TH EN is ?O2 Plaus ible Up per Bou nd Con dition THEN Pla usib le U pper Bound Co ndi tion ?O4 Plausibleerator_of Lo wer Bou nd Con dition Plausib le L ow er B oun d Co ndi tustrial_fac ion The state has_as_in dustrial_factor ? O3 has _as_industrial_ factor ?O3 T he stateis ? O2 h as_a s_indu strial_factor ? O3ntial_goods_or_materiel has_as_ind tor ?O3 ?O2 has_as_in dus trial_fac tor ?O3 ?O2 has_as_ind ustrial_factor ? O3 ateriel ?O 4 IS strateg ically_esse ?O4 IS strategic ally_essential_ goo ds_ or_m ?O 4 IS Strategical l y_esse ntial_goods_o_materiel ?O4 IS Strategically _essential_goo ds_o_m ateriel IS Wa r_m ateriel_and_tran rts_of_ US_194 3 ? O4 IS materiel_and_transports_of_US_1943 ?O1 ISCOG F orce ?O 1 IS COG Force Co nclu?O4 de t an eco nom ic isa strategic Con clude an War_ econ omic factor is aallie strategic ? O1 ISfactor Anglo _all ispo es_1943 ?O1 IS lo_ The force istha ?O1 he force isthat ? O1 ?O3 IS Industrial_capacity_of_US_1943 ?O3 IS Ind ust rAng ial_cap acity_o f_US_1943 ?O3 IS Indu strial_factor ?O3 IS s_1943 Industrial _factor ?O3 is_a_m ajo r_g enerator_of ?O4 T ?O3 is_a_m ajor_gen era tor_of ?O4 ?O2 IS e F orce ?O 2a fo IS Force candid atefor a state whic h isaUS_1943 m em ber of aforc can didate for astate w hich is aUS_1943 m emb er of rce ? O2 IS ?O2 IS The eco nom ic facto r is ?O3 T he economi c factor is ?O3 is_a_major_ generator_o f ?O4 is_ a_m ajo r_generator_of ?O4 is_a_maj o r_generato r_o f Pla ?O4 is_a_m ajor_generator_of ?O4 TH EN usible Upp er B oun dC ond ition THEN Plausible U pper Bo und Con dition Pla usi bledustrial_factor Lower Bound Condition Plau sible Lower Bo und Co ndition ?O4 ISstrategically_essen tial_ goo ds_ or_m ateriel ?O4 IS strategically_essential_goods_or_m ateriel has_as_industrial _ fac tor ?O3 h as_as_indu strial_factor ?O3 T he state is ?O2 has_as_in ?O3 Th e state is ?O 2 War_m has_as_industrial_factor ?O3 ?O4 IS W ar_materiel_and_transports_o f_U S_1943 ds_o_m ?O4 IS ateriel_ and _transp orts _of_US _1943 ?O4 IS Strategi c ally_essential_goo ateriel ?O4 IS Strate gically_essenti al_g ood s_o _ma teriel Co nclude an econ om ic factor is a strateg ic C OG Conclude an econom ic fact o r is COG ?O 1 Industrial_factor IS Fo rce ?O1 IS F orce ?O1 IS An glo_allies_1943 ?O1Ind IS Ang lo_allies T he force isthat ?O1 Th e force isthat ?O 1 ?O 3a strategic I S _1943 Ind ustrial_factor ?O3 IS ? O3 IS In dustrial _capacity_o f_U S_1943 ?O3 IS ustrial_capacity _of_US_1943 Plausib le U pper Bo undCo ndition Plaus ible Up perBou nd Condition can did atefor astate which isaUS m em ber o f2a force candidate fo r a state wh ich is aUS_1943 mem ber of a force ?O IS Fo rce ?O2 IS F orce ?O2 _1943 ?O2 IS is_a_ma jor_generator_of ?O4 is_a_m ajo r_generato r_of ?O4 T HEN TH EN is_a_m ajor_g enerato r_of ? O4 is_a _major_gen ?O 4 T he econom ic factor is ?O3IS Th e econom icfactor is ?O3 Plausible Lower Bo und Co ndition Plausibleerator_of Lo wer Bou nd Con dition ?O1 IS Force factor ?O3 ?O1 IS? Force has_as_industrial_ has_a s_industrial_factor ? O3 Th e stateis ? O2 Thestate is ?O2 has _as_ industrial_ factor ?O3 has_as_in du strial_factor O3 ?O4 IS Strate gically_essential_g ood s_o _m ateriel ?O 4 I S Strategical l y_essential _ goods_o_materiel C on clud e th at an ec ono mic factor i s a strategic COG Co nclude that an eco nom ic factor is a strategic C OG ? O4 IS War_m ateriel_and_transports_of_US _1943 ?O4 IS War_m ateriel_and_tran spo rts_of_US_ 1943 ?O1Ind IS Ang lo_allies_1943 ? O1Industrial_capacity_of_US_ IS Anglo ies_1943 ?O2 Industrial_ IS Force ?O2 Force Th e forceis ? O1 ?O 3 acity_of_US_1943 IS factor Theforce is ?O1 ?O3_all IS InIS du strial_fa ctor ?O3 IS ustrial_cap ?O3 IS 1943 cand idate fo rfactor astate which is aUS_1943 m emberis_a_m of a fo rce ndu candid atefor a state whic h isaUS_1943 m em beris_a_major_generato of aforc e ?O2 ? O2 IS h as_as_i strial_factor _ tor ajor_genera tor_of ?O4 ?O3 r_o ffac ?O4 ?O3 Th e econo mic is ?O3IS The eco nom ic facto r is ?O3 is_a_m ajor_generator_of ?O4 is_a_ maj or_generato r_o fhas_as_industrial ?O4 THEN THEN Plau sibleLo werBou nd Con dition Plausi ble dustrial_factor Lower Bound Condition The state is?O 2 T he stateis ? O2 has_as_indu strial_facto r ?O3 has_as_in ?O3 ?O3 IS Indu strial_factor ?O3 ISsports_of_US_1943 Industrial _factor ?O 4 Anglo_al IS Strateg ically_essential_go _o_ materiel ?O4 IS Strategically _essential_goo ds_o_m ateriel ?O4 IS Wa r_m ateriel_ and _transp orts _of_ US_194 3 ods ?O4 IS War_materiel_and_tran Conclude t an eco nom ic is a strategic COG Conclude an econ omi c fa ctor is a strategic C OG ? O1 ISfactor lies _1943 ?O1 IS An glo_allies_1943 The force istha ?O 1 T he force is that ?O1 ?O3 IS Industrial_capacity_of_US_ 1943 ? O3 IS In dustrial_capacity_o f_U S_1943 r_of ?O4 is_a_ma jor_ generator_o f ?O4 is_a_m ajo r_generato candi d atefor a state which is aUS_1943 m em ber of aforce can did atefor astate w hich isaUS m emb er of a force ? O2 IS ?O2 IS _1943 The eco nom icfactor is ?O3 he econom ic factor is ?O3 TH EN THEN is _a_major_gen erator_o f ?O4 is_a_m ajor_g enerato ? O4 Plausible Low er B ou nd C ond itio n Plau sible Lower Bo und Co ndition al_g ?O4 IS Strategi c ally_essential_goo ds_o_m ateriel T ?O4 ISr_of Strate gically_essenti ood s_o _ma teriel T hestate is ?O2 Th e state is ?O2 has_as_in dustrial_factor O3 has_as_ industrial_facto r ?O3 Co nclude that an econ om ic factor isa strateg ic? C OG Con clud e that a n econom ic factor is astrategic COG _1943 ?O4 IS War_materi e l_and_tran spo rts_of_US_1943 ? O4 IS War_m ateriel_and_transports_of_US ?O1 IS Anglo_allies_1943 ?O1 IS Ang lo_allies_1943 T heforce is ?O1 Th e fo rce is ?O1 ? O3 IS In dustrial_capacity_of_U S_1943 ?O3 IS Ind ustrial_cap acity_of_US_1943 candid atefor a state whic h Plausible isaUS_19 m em Lower ber f Bound aforc eCo ndition candidate fo r astate wh ich Plausible is a mem Lo be wer r of Bou a fo rce ndCon dition ?O2 IS 43 o ?O2 US_1943 T heeconom ic facto r is ?O3 Th e econom icfac tor is ?O3IS is_a_ major_g enerato r_o f ? O4 is_a_m ajor_generator_of ?O4 THEN THEN Th e stateis ? O2 ?O1 IS Ang lo_allies_1943 The state is ?O2 ? O1 IS Anglo _allies_1943 has_as _ind ustrial_factor ?O3 has_as_indu strial_factor ?O3 ? O4 IS War_ materiel_and_transports_of_US_1943 ?O4 IS r_m ateriel_ and _transpo rts_of_ US_194 3 Con clud e that a n econom ic factor is astrategic COG 43 Conclude t an Wa eco nom ic is a strategic COG Th e force is ? O1 The force istha ?O1 ?O2 Ind IS US_1943 ? O2 ISfactor US_1943 ?O3 IS ustrial_capacity_o f_US_19 ?O3 IS Industri al_capacity_of_US_ 1943 can didate for astate which is amem ber of a fo rcer ?O3 candi d atefor a state which is a m em ber of aforce ? stri al_facto has_as_in dustrial_factor Th e economi c factor is ?O3has_as_indu The eco nom icfactor is is_a_m ajor_generator_of ?O 4 is?O3 _a_ major_gen erator_o f ?O4 O3 THEN TH EN Thestate is ?O2 T hestate is ?O2 ?O3 IS Industrial_capacity_of_US_1943 ? O3 IS om In dustrial_capacity_of_U S_1943 ?O4 IS War_m ate riel_and _transports_of_US_1943 ?O4 IS War_materi el_and_tran spo rts_of_US_1943 Conclude that an econom ic fact o r is a strategic COG Co nclude that an econ ic factor is a strateg ic C OG Theforce is ?O1 T heforce is ?O1 is_a_major_gen erator_of ?O4 enerato r_of ? O4 candidate fo r a state wh ich is amem ber of a force can did atefor astateis_a_m whic h ajor_g isa m em ber o f aforc e THEN THEN Theecono micfactor is?O3 T heeconom ic facto r IS is?O3 materiel_and_transports_of_US_1943 ?O4 IS War_materiel_and_tran spo rts_of_ US_1943 ? O4 Thestate is ?O2 Th e state is ? O2 War_ Concl ude ic facto r is a st rategic C OG Con clud e that n econom icfactor is astrategic COG The force isthat ?O1an econom Th e force is ? O1a TH EN THEN candid ate for a state em berof aforce candidate fo rfactor astate ich is amem be r of a fo rce The eco nom ic facto r iswhich ?O3 isa m Th e econo mic iswh ?O3 Con clude om ic factor is astrateg ic C OG Conclude t an eco nom icfact or isa strategicCOG The state isthat ?O2an econ The state istha ?O2 can didate astate which is amem ber o f a force candidate a statewhich is amem ber of aforce The force isfor ?O1 The force is for ?O1 Th e state is ?O2 The state is ?O2 The econ om ic factor is ? O3 Theeco no mic factor is?O3 Th e force is ?O1 Theforce is ?O1 Th e econo micfactor is ?O3 Theeco nom ic facto r is?O3

Problem solving rules (learned)

c omb in ed an d jo int o pera tio ns

1. Specifying a training scenario

First the expert has to specify a training scenario, by using the Scenario Elicitation tool, as has been illustrated before.

In this illustration we will use the World War II invasion of the island of Okinawa by the US forces.

1. Specifying a training scenario

Based on the information elicited from the expert Disciple creates a formal representation of the scenario.

The internal representation may be viewed using the Association Browser

2. Modeling experts reasoning

Then the expert has to express his reasoning in center of gravity identification and testing for the Okinawa scenario, by using the task reduction paradigm.

I need to Identify and test a strategic COG candidate for the Okinawa_1945 scenario What kind of scenario is Okinawa_1945? Okinawa_1945 is a major theater of war scenario Therefore I need to Identify and test a strategic COG candidate for Okinawa_1945 which is a major theater of war scenario Which is an opposing force in the Okinawa_1945 scenario? Japan_1945 Therefore I need to Identify and test a strategic COG candidate for Japan_1945

2. Modeling experts reasoning

The expert expresses his reasoning process in English, using the Modeling tool of Disciple: The modeling tool shows:

The description of the node selected from the reasoning tree.

The entire reasoning tree.

2. Modeling experts reasoning

The first steps of the reasoning process show how the expert has identified Emperor Hirohito as a strategic center of gravity candidate for Japan. We will now show how the expert uses the Modeling tool to express his reasoning of testing whether Emperor Hirohito is a viable candidate.

To reduce the current task the expert needs some information that is identified by asking a question.

2. Modeling experts reasoning

Then the expert must provide an answer to this question

To reduce the current task the expert needs some information that is identified by asking a question.

2. Modeling experts reasoning

Then the expert must provide an answer to this question

2. Modeling experts reasoning

Then the expert must provide an answer to this question

2. Modeling experts reasoning

2. Modeling experts reasoning

The question and its answer lead to this reduction of the top task.

2. Modeling experts reasoning

The question and its answer lead to this reduction of the top task.

2. Modeling experts reasoning

2. Modeling experts reasoning

2. Modeling experts reasoning

2. Modeling experts reasoning

2. Modeling experts reasoning

2. Modeling experts reasoning

2. Modeling experts reasoning

2. Modeling experts reasoning

This process continues in the same way until the expert has enough information to either eliminate or not eliminate the COG candidate.

2. Modeling experts reasoning

3. Task and Rule Learning

We will now demonstrate how Disciple learns general tasks and rules from the experts reasoning.

3. Task and Rule Learning

First the expert and Disciple have to formalize the English statements of the tasks.

This is done in the Formalization mode.

3. Task and Rule Learning

In the formalization mode the tool shows:

The modeling in English

The formalized tasks

3. Task and Rule Learning

Disciple will propose a formalization of the task

3. Task and Rule Learning

Disciple will propose a formalization of the task

The expert may accept it or he may edit it

3. Task and Rule Learning

After the tasks are formalized the expert may explain the example to Disciple, which will learn a rule from it

3. Task and Rule Learning

The Rule Learning tool helps the expert to explain the example to Disciple

The tool shows the English form of the example

Disciple uses analogical reasoning and other heuristics to propose plausible explanations pieces that justify the task reduction step.

3. Task and Rule Learning

The expert selects those explanation pieces that correspond to the meaning of the question-answer pair from the task reduction example

3. Task and Rule Learning

The expert selects those explanation pieces that correspond to the meaning of the question-answer pair from the task reduction example

3. Task and Rule Learning

3. Task and Rule Learning

3. Task and Rule Learning

The expert may direct Disciple to generate explanation pieces related to certain objects from the example When the expert is satisfied with the identified explanation he can click on End learning. Disciple will then create a general rule corresponding to this example and its explanation.

3. Task and Rule Learning

This is the general task reduction rule learned by Disciple. Notice that it has a plausible upper bound condition and a plausible lower bound condition. During rule refinement the two conditions will converge toward one another, ultimately leading to a rule with a single condition.

3. Task and Rule Learning

This is the general task reduction rule learned by Disciple.

In addition to the formal structure of the rule, which is used in problem solving and learning, Disciple maintains also an informal structure of the rule.

The informal structure is used in the communication with the user.

3. Task and Rule Learning

After learning a rule from the current task reduction step Disciple returns to the formalization mode.

3. Task and Rule Learning

Following the same procedure, Disciple will learn another rule from the next task reduction step:

3. Task and Rule Learning

3. Task and Rule Learning

3. Task and Rule Learning

3. Task and Rule Learning

3. Task and Rule Learning

3. Task and Rule Learning

3. Task and Rule Learning

Following the same procedure, Disciple will learn another rule from the last task reduction step:

4. Rule Refinement
Disciple uses the partially learned rules in problem solving and refines them based on experts feedback.

4. Rule Refinement
Disciple uses the rules learned from the experts identification and testing of Emperor Hirohito to identify and test President Truman as a US strategic center of gravity candidate.

The ? preceding a reasoning step indicates that the step is less plausible, because it is based on the plausible upper bound condition of a learned rule.

4. Rule Refinement
The expert has to examine this step and has to indicate whether it is:

correct but incompletely explained

by selecting Explain Example

incorrect correct and completely explained

by selecting Correct Example by selecting Incorrect Example

4. Rule Refinement
The expert has indicated that the reasoning step is correct and Disciple has generalized the plausible lower bound condition of the corresponding rule, to cover this example.

4. Rule Refinement
In the case of a correct but incompletely explained example the expert may further explain the example, as has been illustrated before. For an incorrect example the expert has to help Disciple to identify an explanation of why the example is incorrect. In all the cases the rule is automatically refined by Disciple.

5. Exception Handling

Because the object ontology of Disciple is incomplete, it may not contain the knowledge pieces that explain why a certain rule example is incorrect. In such a case the expert has two options: End Rule Refinement (the incorrect example is kept as a negative exception of the rule; the knowledge base will be extended latter, with the help of the knowledge engineer); Invoke Exception Handling (the expert attempts to extend the object ontology by himself ).

We will demonstrate the handling of exceptions by the subject matter expert.

5. Exception Handling
Disciple generates a task reduction step which is rejected by the expert. The expert and Disciple attempt to find an explanation of why this reasoning step is incorrect.

However, there is no ontology piece that explains why this is an incorrect reasoning step.

The expert may end Rule Refining, and Disciple will keep the incorrect reasoning step as a negative exception to the rule that generated it.

Or, the expert may start Exception Handling

5. Exception Handling
The Exception Handling module shows: The incorrect reasoning step (the negative exception of the rule)

An object from the positive example of The the rule left and right buttons used to view the other objects from the rules examples The corresponding object from the negative exception of the rule

5. Exception Handling
Disciple generates an ordered list of candidate hypotheses that may distinguish between the positive example and the negative exception of the rule These candidate hypotheses are new ontology pieces (e.g. new feature values, new features, or new objects) This is a more detailed description of the candidate highlighted above The negative exception may be removed by providing a value for the property is_mission_oriented of Military_of_Japan_1945 which is different from the value for Military_of_US_1945

The expert selects this candidate to remove the exception

5. Exception Handling
Disciple guides the expert in the definition of new knowledge about the selected candidate

5. Exception Handling
The expert defines the value of the property is_mission_oriented for Military_of_Japan_1945

This property value differentiates between the object from the positive example (Military_of_US_1945) and the object from the negative exception (Military_of_Japan_1945)

in absolute terms and disregard of preserving the lives of its soldiers

5. Exception Handling

Disciple refines the object ontology and the rule with the elicited feature value

5. Exception Handling

6. Problem Solving
The automated problem solver presented at the beginning of this demonstration was customized for the Center of Gravity domain.

6. Problem Solving
The Disciple shell also contains a general automated problem solver.

Solution

Justification

Tools for the knowledge engineer

The Disciple shell contains also a suite of knowledge base development tools for the knowledge engineer: Ontology browsers and editors to develop the object ontology; Ontology import tools to import ontological knowledge for reusable knowledge repositories, such as CYC; Script editor to define elicitation scripts; More complex exception handling tools for the knowledge engineer. They will be briefly demonstrated in the following.

1. Ontology Development Tools

Object Viewer Object Browser

1. Ontology Development Tools

Object Editor

1. Ontology Development Tools

Script Editor

Object Browser

2. Ontology import tools

Ontology import tools perform a three step import process: Mixed-initiative retrieval of potentially relevant ontological knowledge from external knowledge repositories, such as CYC; Automatic translation of the retrieved ontological knowledge into an intermediate Disciple ontology; Mixed-initiative import from the intermediate Disciple ontology into the destination Disciple ontology. The following is a brief demonstration of ontology import from CYC.

2. Ontology import tools

The expert looks for relevant ontological knowledge in CYC

The expert specifies a searching string

2. Ontology import tools

The expert looks for relevant ontological knowledge in CYC

The expert specifies a search string

CYC returns all the matching names

The expert chooses to import NATO and the knowledge related to it

2. Ontology import tools

The knowledge retrieved from CYC is automatically translated into an intermediate Disciple ontology.

2. Ontology import tools

The expert and the knowledge engineer import knowledge from the intermediate Disciple ontology into the destination Disciple ontology The objects selected to be imported from the intermediate Disciple ontology into the destination Disciple ontology Fragment of the intermediate Disciple ontology translated from CYC

The expert and the knowledge engineer select the entities to be imported into the destination Disciple ontology

2. Ontology import tools

The expert and the knowledge engineer import knowledge from the intermediate Disciple ontology into the destination Disciple ontology NATO is being imported from the intermediate Disciple ontology into the destination Disciple ontology The user specifies the additional knowledge to be imported together with NATO

The selected superconcepts of NATO will also be imported

The selected features of NATO will also be imported

2. Ontology import tools

The ontology fragment represented by NATO, its super-concepts, and its features is copied into the destination Disciple ontology

A + denotes a term that does not yet exist in the ontology, while a + denotes an existing term The user specifies the position of the imported concepts in the destination Disciple ontology

2. Ontology import tools

The user updates the feature values

2. Ontology import tools

NATO is marked as imported in the intermediate Disciple ontology

The import of NATO has led to the partial import of these additional objects. Disciple suggests the user to complete their import.