Applying Semantic Technology to Early Stage Defense ...

Applying Semantic Technology to Early Stage Defense ...

Applying Semantic Technology to Early Stage Defense Capability Planning Analysis Based on JCIDS Artifacts Allen Moulton [email protected] Sociotechnical Systems Research Center 77 Massachusetts Ave, Cambridge, MA 02139 Dr. Donna Rhodes MAJ James Enos Prof. Stuart Madnick COL Douglas Matty MIT Sloan School of Management Chief, SE Branch, JRAD, J8 Chief, PDD, PAED, HQDA G8 Abstract ID 18026 18th NDIA Systems Engineering Conference 29 October 2015 Agenda Goals of JCIDS Semantic Architecture Framework Research Joint Capability Enterprise Architecture Exploratory Experiments Systematizing Method for Manual Use Leveraging Semantic Technology Next Steps 2 JCIDS (Joint Capabilities Integration and Development System) A Systematic Process for Warfighters to Develop, Validate, and Control Capability Requirements for Acquisition LIMITATIONS OF CURRENT JCIDS PROCESS Necessarily Document-Driven DODAF Architecture Not Fully Integrated Silos of Information by Capability/Program and Date of Writing Warfighters SMEs Docs DODAF Acquisition SMEs MIT Research Goals Unlock docs into data Apply inference to extend understanding Connect text info to architecture content Bridge

info silos Joint Capability Enterprise Architecture (JCEA) 3 System of Systems Complexity is Inherent in JCIDS Value Proposition for Capability-Based Planning (Aldrich Study, 2004) Strategy Desired Effects Capabilities Fielded Systems Capability-Based Planning Works Backwards from Goals to Factor Out Systems Needed Not as Simple and Linear as it Looks Investment decisions must be made years or decades in advance ... within limited and changing budget constraints ... to assure that Services will have the capabilities on hand ... to supply resources to combatant commanders ... to be dynamically integrated into joint task forces ... to achieve effects needed to accomplish future missions ... in support of national strategy Question: How to Manage the Inherent Complexity of the Problem? Combinatorics of the solution space vs. need to limit scope of each system Dynamic effects of decision lead times and necessity for integration Uncertainty on critical factors affecting the design e.g., strategy, threats, budgets, technology, related program outcomes 4 Joint Capability Enterprise Architecture (JCEA) JCEA content extracted from multiple views DODAF Views C-M-L Views JCEA used to generate other views Search Views SME Views Text Doc Views Other Capabilities, Systems and Time-Frames Views JCIDS Docs, DODAF and SMEs each

capture partial information on underlying reality as of a point in time Other Capabilities, Systems and Time-Frames Current State and Planned Future States Strategic Guidance Missions Threats Force Capabilities Functions and Tasks Materiel Systems Technology Other Capabilities, Systems and Time-Frames Other Capabilities, Systems and Time-Frames Other Capabilities, Systems and Time-Frames Other Capabilities, Systems and Time-Frames Decision Views Other Capabilities, Systems and Time-Frames JCEA holds content that can make connections across capabilities and time frames Other Capabilities, Systems and Time-Frames Underlying Fabric of Evolving Capabilities and Requirements over Time Ontology defines slots that structure data extracted from documents and DODAF Ontology also defines relationships among data elements in the JCEA model

5 Defining Semantics: Empirical Review of Documents Broad review of 88 unclassified sample JCIDS documents to build familiarity, recognize patterns, and discern ground truth Detailed deep-dive into three capability documents (ICD, CDD, CPD) 1) what SHOULD be in document? 2) what WAS in document? 3) what is ESSENTIAL in document? Documents selected for deep-dive experiment: 3 different stages of development (ICD, CDD, CPD) 3 different functional areas staffed by different FCBs All in Air domain with documents staffed in 2007-2009 ICD Logistics CDD Force Application CPD Battlespace Awareness Joint Future Theater Lift (JFTL) Move cavalry with armor Joint Air-to-Ground Missile (JAGM) Replace HELLFIRE,TOW and Maverick Extended Range UAS (MQ 1C) Dedicated support to Division Found implicit interdependencies across separately staffed capabilities. Framing a Joint Capability Enterprise Architecture: Capability Categories Joint Capability Areas To support needs definition, gap and excess analysis, major trade analyses, and capabilities planning, DoDs capabilities must be divided into manageable groups, or capability categories. Aldrich Study (2004) 2005 Original JCAs 4 top level categories (operational, functional, domain, institutional) 22 Tier 1 with 240 subordinate JCAs Too many overlaps and redundancies Unnecessary complexity for use as a taxonomy 2007 Revised JCAs 9 Tier 1 JCAs, 6 Tiers Functional only Aligned with FCBs Operational dimension removed Empirical Observations from Docs Conclusions

Most JCIDS docs use multiple Tier 1 JCAs JCAs are used as a framework for describing operational attributes of capabilities not just desired effects JCAs alone are insufficient to categorize capabilities A multidimensional category structure is preferable to a single taxonomy 7 Framing a Joint Capability Enterprise Architecture: Joint Staff Capability Mission Lattice (CML) Materiel and Non Capability-Mission Latti ce Logistics Maintain Nuclear Deterrence National Defense Strategy Defense of the Homeland National Military Strategy Quadrennial Defense Review National Security Interests Quadrennial Roles/Missions Defense Planning Guidance Guidance for Employment of the Force Unified Command Plan Joint Strategic Capabilities Plan Land Systems Planning Operations National Security Strategy Means Science & Technology / Research & Engineering Adapted by MIT from (RevJoint 0.8.1 /Staff 24 AprConcept

2014) Strategic Guidance - M a t e r i e l C a p a b i l i t y S o l u tio n s Materiel Acquisition /Investment, Including Legacy System Sustainment Sea Systems Air Systems Space Systems Cyber Systems SAP SAP SAP SAP SAP System Architecture(s) System Architecture(s) System Architecture(s) System Architecture(s) System Architecture(s) Threats / Near Peer Competitors Defeat Adversaries Operational Concepts Universal Joint Tasks Service Tasks Conditions Global Stabilizing Presence Combat Terrorism

Rogue Nations UJTs Counter Weapons of Mass Destruction Deny Adversary Objectives Non-State Actors (enable decomposition of missions against threats , and identification of associated capability requirements ) Crisis Response / Limited Contingency Operations Weapons of Mass Destruction Military Engagement / Security Cooperation Environmental / Natural Events Counterinsurgency and Stability Operations Assesment(s) Support to Civil Authorities Treaties /Alliances Humanitarian Assistance / Disaster Response Assesment(s) Operational Architecture(s) Operational Architecture(s) Op. Arch(s) SAP SAP Operational Architecture(s) Operational Architecture(s) Operational Architecture(s) Op.

Arch(s) SAP SAP SAP Op. Arch(s) Operational Architecture(s) Ends Functions Ways Force Support Battlespace Awareness Force App. Logistics Command and Control Prote Build Net-Centric ction Ptnr Corporate Management C a p a b i l i t y R e q u i r e m e n t P o r tfo l i o M a n a g e m e n t Basic ontology from Capability Mission Lattice has been expanded to include elements required in JCIDS Manual and taxonomies/frameworks in use 8 Using C-M-L Ontology to Find Interdependencies JFTL ICD JAGM CDD Phrases from JCIDS Docs attached to Ontology Slots Interdependencies C-1 30 JFTL ICD ER UAS CPD RE LFI HE L ER UAS CPD MVM

JAGM CDD Inferred /C-1 7 MVM: Mounted Vertical Maneuver The C-M-L based ontology can help identify interdependencies between systems that are not apparent in documents or with current taxonomies 9 Systematizing Semantic Architecture Framework JCIDS Ontology Design Task Central goal: Define a semantic knowledge base that captures the portfolio of capabilities & gaps early in development Ontology and architecture frame the knowledge base Ontology also captures and connects essential military and requirements process subject domain knowledge Requirements documents provide the content Text of documents (interpreted against ontology) Structured information in tables and DODAF artifacts attached in structured form suitable for machine use Images such as OV-1 (hard to extract info from) Additional content will come from SME annotations as an ontologybased knowledge base is put into use Data captured and organized in a semantic architecture framework will continue to be accessible and reusable as SMEs rotate in and out and as circumstances change 10 Overview of ICD Ontology Design based on 2015 JCIDS Manual and Capability-Mission-Lattice Metadata Operational Context Time Frame Strategic Guidance ROMO Operational Concepts Capability Reqts Define Capability Requirements in Lexicon of: o o o o o o o o Time Frame ROMO Org / Unit Type JCAs UJTL Tasks Service Tasks

Conditions Supported and supporting tasks Threats Threat context Expected operational environment Operational Current threats Attributes Anticipated threats o Metrics o Objective Values A. References B. Acronyms Cover Page Capability Gaps Match to Current Capabilities o Legacy fielded o In Development o Rapidly fielded o Predecessor system if recap or next gen Identify Gaps for each Operational Attribute (O/A): o Current capability O/A metric value o Gap from current to objective value Operational Impact of Gap C. Glossary Recommendations Materiel Solutions Suggested for AoA o Evolution of fielded system o Replacement or recap of fielded system o Transformational capability solution Technology Leverage to reduce Operational Risk o Functionality o Affordability DOTmLPF-P Recommendations D. DODAF 11

Example: JFTL ICD Extracted Capability Gaps Gap Functional Gap Description Ontology Concept Document Num Concept Inability to operate into austere, unimproved landing areas Data in Blue inshort, Yellow Inability to perform operational maneuver with medium weight armored vehicles and personnel or reposition medium weight armored vehicles and personnel by 1 IOM airlift Inability to reposition forces with combat configured medium weight armored vehicles via air Inability to operate into austere, short, unimproved landing areas Deliver cargo weights equivalent to the weight of combat configured medium 2 OMSD weight armored vehicles to austere, short, unimproved landing areas. Conduct precision air delivery of supplies, to the point of need/point of effect over strategic and operational distances with required velocity. DMSS Inability to operate into austere, short, unimproved landing areas Deliver cargo weights equivalent to the weight of combat configured medium 3 weight armored vehicles to austere, short, unimproved landing areas. DES Conduct precision air delivery of supplies, to the point of need/point of effect over strategic and operational distances with required velocity. Inability to transport forces over strategic and operational distances to points of need by passing traditional PODs, and to operate on austere, short, unimproved landing areas. 4 JFEO Inability to deploy and employ forces, with combat configured medium weight vehicles, via air across the global battle space from strategic, operational and tactical distances Reason for Gap Proficiency Proficiency Proficiency Proficiency Proficiency Proficiency Proficiency Proficiency Sufficiency Proficiency Proficiency 12 Example: Compare Gap Operational Attributes Operational attribute Cargo handling Combat Radius Cruise Speed Fuel efficiency In-flight Refuel Speed (as Receiver) Payload Weight &

Dimensions Precision Delivery Gaps by Functional Concept 1 2 3 4 IOM OMSD DMSS/ JFEO DES X X X X X X X X X X X X X X X X X X X X X X X X X Precision Landing Secure Communications Self Deploy Survivability X X X X X X X

X X X X X X Operational attribute values Ontology Concept in Yellow No MHE As determined in AoA As determined in AoA Fuel efficiency must be greater than that of the C-130J As required Combat configured medium weight armored vehicles (Army ground combat vehicles, Stryker) ~25 50 km of objective Point of need/point of effect Routine 0 ft takeoff & land (VTOL) to routine <1500 ft Document takeoff and land (STOL)1 over a 50 obstacle into austere, in Blue complex, urban or unprepared landing areas Data independent of external navigation aids Interoperable, secure, encrypted, voice and data, beyond line of sight/over the horizon 2,400 nm Ability to effectively integrate with future joint forces for threat suppression/mitigation in a low to medium threat environment 13 Semantics-Based Inference Can Help Fill in Missing Data and Inconsistencies in JCIDS Documents Capturing Implicit Information Documents reviewed often have inconsistent data Most have current JCAs; some have 2005 JCAs; some have JFCs JCAs often used for multiple purposes Some have UJTs; most do not SMEs can make sense of documents despite gaps & other inconsistencies Ontology-based data capture combined with inference rules can allow automation to follow same logic used by SMEs Connecting to other Knowledge Example of how can semantic inference can help: Joint Future Theater Lift (JFTL) ICD has no UJTs JFTL ICD references JP 3-17 (Air

Mobility Operations) and Joint Forcible Entry by name Joint Forcible Entry (JFEO) defined by JP 3-18 UJTL database ties UJTs to definitional docs JP 3-17 and JP 3-18 By combining these fragments of information, UJTs for JFTL can be inferred Semantic architecture provides the benefits of capturing the true capability provided by a system by interpreting text within a document. 14 Semantic Ontology Experiments Developed an ICD ontology containing 150 data slots based on draft 2015 JCIDS Manual, C-M-L, and other frameworks Manual text extraction experiments 6 ICDs as sources, 3 SMEs perform extraction Into Excel form structured by the ontology Reliability varied: some data were consistently extracted; other data inconsistent A parallel project showed potential for applying natural language processing to automate text extraction SMEs built a practical relational database by focusing on the more consistent areas and for wider sample of JCIDS documents Experiment showed that DODAF views can be generated from data extracted from JCIDS documents MIT continuing research is focused on formalizing and systematizing methods to extend the scope and value of the results 15 Research on Technologies and Methods for Storing and Accessing Semantic Knowledge 1) Documents repository (current as-is state) 2) Relational or spreadsheet data 3) DODAF architecture structured data New 2015 JCIDS Manual requires DODAF views to be submitted with requirements documents for validation Research is exploring how to connect text document content to DODAF data and artifacts 4) Semantic data store with inference rules Facts stored as RDF Triples (subject-predicate-value) Flexibility from capturing facts in small pieces Facts can be combined in multiple ways by inference rules and semantic query 16 Semantics Technology Proof-of-Concept Prototype Design Overview C-M-L M at e ri e l a n d N on Materiel Acq uisition /Inves tment, In cluding L egacy Sy stem Sus tainment Capability-Mission Latti ce Sc ience & T e chnolog y / Re se arc h & E ngine e ri ng L og istics Land

Syste ms (R ev 0.8.1 / 24 Apr 2014) Str a teg ic Gu idan ce Planning Oper atio ns N ationa l Se curity Stra te gy Ma intain Nuclea r De te rrence Na tional Defe nse Stra te gy De fe nse of the Homeland Nationa l M ilita ry Stra te gy Q uadre nni al D efe nse Re vie w N ationa l Se curity I nte res ts Q uadre nni al Role s/M issions De fense Planning G uidance Guida nc e for E mploym ent of the Force U nifie d Comma nd Pla n Joint Strate g ic Capa bilitie s Pla n - M ate r ie l C a pa b i l it y S o lu tio ns Sea Syste ms Air Syste ms Spa ce Syste ms Cyber Syste ms SAP SAP SAP SAP SAP Sys tem Ar chitect ur e(s)

Sys tem Ar chitect ur e(s) Sys tem A rchitectur e(s) Sys tem A rchitectur e(s) Sys tem A rchitectur e(s) / JCIDS Manual G lo b a l C ont ex t T h r ea t s Ne ar Pe e r Compe titors De fe at Adversarie s Rog ue Nations Global Stabilizing Pre sence Combat Terrori sm Counter Wea pons of Ma ss De struction De ny Adv ersa ry Obje ctive s Crisis Response / Limi ted Continge ncy Operations Mi lita ry E ngag ement / Security Cooperation Counterinsurge ncy a nd Stability Ope ra tio ns Support to Civi l Authorities Humanitarian Assista nce / Disaster Response UJTs N on-Sta te Ac tors (e nable de composition of m issions a ga inst thre ats , a nd ide ntific ation of ass oci ate d ca pa bility require me nts ) W ea pons of Ma ss De struction Env ironm enta l / Na tura l E ve nts As ses ment( s) T rea tie s /Alli ance s As ses ment( s)

Oper ational Ar chit ect ure(s ) Operatio nal Ar chit ect ure(s ) S AP SAP For ce Sup port B attles pace Awaren ess Force App . Op. Arch(s ) Oper atio nal Architectu re(s ) Op erational Ar chitectur e(s) SAP Logistics Command and Contr ol Op er ational A rchitectur e(s) Op. Ar ch(s ) S AP SAP Op. Arch( s) Prot e B uild Net-Cen tric ctio n Pt nr Oper ational Ar chit ect ure( s) Design / Ontology Design Other Sources Corp orate

Management Ca p ab il i ty Re q u ir em en t P o r tfo l io Ma n a gem en t JCIDS Docs DODAF Data Manual Extraction Automated Extraction Semantic Technology Tools RDF Graph Built on Semantic Web industry Store standards such as OWL, RDF, Semantic Technology Platform Other Sources Semantic Query Updates to ontology and methods Ontology design based on JCIDS Manual Capability-Mission-Lattice other terminology frameworks Dashboard Viewer Semantics Experiments Data Export DODAF Generation Evaluation of experimental results SPARQL & cyber-security Includes tools for working with ontology and data Highly flexible data store and semantic query/search Technology used allows research results to be ported to other COTS product sets DODAF Generation Tools COTS/GOTS tools, such as NoMagic/MagicDraw/CAMEO UPDM interface (probable) Python to convert data format 17 Connections in Capability Requirements Ontology

Category Frameworks Value of capability comes from effect produced Service & Universal Joint Tasks JCA Joint Capability Areas supports Strategic Guidance supports Mission Areas Universal Joint Task Mission Effects Operational Concept Performing Org/Unit Universal Joint Task Desired Effects Operational Activity Operational Activity Joint Capability Area Joint Capability Area specifies Operational Attributes Expected Operational Environment Mission Conditions Threats to Capability Threats to Mission

Capability Conditions Operational Attributes Time Frame Operational Attribute describes Threat Contex t Capability Requirement Mission Operational Context Generic Operationa l Attribute Capability Gap Required Initial O/A Objective Value Difference Metric for Operational Attribute Current Attribute Value Current Capability 18 JCIDS Semantic Architecture Framework Enables Capability Enterprise Architecture Multi-dimensional grouping of capabilities by category framework properties Logically deriving capability dimensions and similarities from operational attributes Capturing and retaining SME knowledge across silos and over time Identifies Capabilities Dependencies Tracing capabilities to assumptions, conditions, and threats Tracking interfaces and connections among capabilities Inferring dependencies based on effects produced and effects needed Supports Systems Engineering Trade space identification for capability requirements planning Trade space exploration at the capabilities portfolio level MIT Research is investigating and developing methods to apply semantic technology to Joint Capability Enterprise Architecture 19 Goals for Semantic Architecture (2016)

Unlocking Knowledge Decompose documents into conceptual elements independent of language, to enable translation of across terminology, frameworks, and taxonomies. Identify implicit interconnections and interdependencies across separately staffed capability requirements (including different time periods, different functional areas, and different services or components). Connect text to architecture to create a more complete picture in a form suitable for inference. Generate DODAF artifacts from ontology-based data extracted from text documents. Supporting Decisions Provenance: Maintain time-varying continuity of requirements across development stages and across separate branching threads. Drill down: Make conceptual connections across different levels of architecture (e.g. SoS vs. Systems, KPPs vs. DODAF) as designs evolve. Track changes to assumptions (e.g., strategic direction, mission profiles, threats, operational concepts, technology available). Support systems engineering methods such as Trade Space Exploration and Epoch-Era Analysis. 20 References Aldridge, Pete et al. (2004). Improving DOD Strategic Planning, Resourcing and Execution to Satisfy Joint Capabilities. Joint Defense Capabilities Studies, Jan 2004. Ahmed, Col. L. Najeeb (2014) Improving Trade Visibility and Fidelity in Defense Requirements Portfolio Management: A Formative Study of the Joint Capabilities Integration and Development System using Enterprise Strategic Analysis and Semantic Architecture Engineering. Unpublished MIT SDM Thesis. Allemang, Dean & Hendler, Jim (2011). Semantic Web for the Working Ontologist. Waltham, MA: Morgan Kaufman. U.S. Dept of Defense. JCIDS Manual (12 Febuary 2015) Acknowledgements The work presented here was supported, in part, by the MIT Lincoln Laboratories and the US Army under the "Study of JCIDS Semantic Architecture Framework" project. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not reflect the official policy or position of MIT Lincoln Laboratory, the US Army, the Department of Defense. All research and results reported are unclassified 21

Recently Viewed Presentations

  • R Data Analysis and Calculations

    R Data Analysis and Calculations

    data frames. Data frames are fundamental data type in R. A data frame is a generalization of a matrix. Different columns may have different types of data. All elements of any column must have the same data type. Age Weight...
  • Water Project for Chasnigua, Honduras

    Water Project for Chasnigua, Honduras

    Water Project for Chasnigua, Honduras. Graduate Students: John Jackson and Adrian Weaver. Undergrads: Daniel Howell, John Stein, Eric Strailyand Rebecca Walthall
  • Oral Antiplatelet Therapy in ACS - Medscape

    Oral Antiplatelet Therapy in ACS - Medscape

    Oral Antiplatelet Therapy in ACS. PLATO Primary Efficacy Outcome by Geography, ... (% of Patients) CHARISMABleeding End Points by Baseline Daily Aspirin Dose. CURRENT OASISEfficacy and Bleeding by Aspirin Dose at 30 Days. Clinical PearlsAspirin in Acute ACS. Clopidogrel in...
  • Alien Species  Aquaculture WFD  MSFD Keith Jeffery Centre

    Alien Species Aquaculture WFD MSFD Keith Jeffery Centre

    ) pending assessment. Pacific Oyster ( C. Gigas) ranked as moderate risk. Stocking out for Angling purposes or escapees, spat ? Could jeopardize good ecological status ? Subject to interpretation. Unclear if species listed in annex IV 708/2007 should be...


    Five-member advisory committee (2006) Provides advice and recommendations to the Secretary of Transportation and the FMCSA Administrator on the development and implementation of science-based physical qualification standards ensuring that drivers are physically qualified to operate commercial motor vehicles. Medical review...
  • Spreadsheet Modeling &amp; Decision Analysis:

    Spreadsheet Modeling & Decision Analysis:

    The Steps in Implementing an LP Model in a Spreadsheet. 1. Organize the data for the model on the spreadsheet. 2. Reserve separate cells in the spreadsheet to represent each decision variable in the model. 3. Create a formula in...
  • 2013 History w/ Mrs. Marsh

    2013 History w/ Mrs. Marsh

    Make-up Tests will be taken during a tutorial. For excused absences and extended absences due to illness or family emergencies, students will be given 3 days to make up tests and quizzes. Students who know in advance must inform me...
  • TEK 12A-Nature Relationships (5.RS) Relationships in Nature Mutualism-Both

    TEK 12A-Nature Relationships (5.RS) Relationships in Nature Mutualism-Both

    Starts with lichen and other pioneer species before moss and plants. Simple plants die adding nutrients to soil. ... Simple Molecule to life steps. Formation of simple molecules. ... Disruption or loss of control of the cell cycle can lead...