Framework for Early Detection of Unknown-Unknowns

Unknown-unknowns are operational scenarios in AI enabled autonomous systems, such as human-in-the-loop (HIL) human-in-the-plant (HIP) systems, which are not accounted for and may represent a major challenge in the design and validation phase. In these scenarios, the operational behavior of the systems is not guaranteed to meet requirements such as safety and efficacy in presence of out-of-distribution human inputs and interactions between un-modeled faults and validated behaviors. Early detection of unknown-unknowns is paramount to prevent safety violations with potentially fatal consequences.  
 
Researchers at Arizona State University have developed a novel framework for analyzing the operational output characteristics of safety-critical HIL-HIP systems to discover unknown-unknown scenarios and evaluate potential safety hazards. By utilizing a dynamics-induced hybrid recurrent neural networks (DiH-RNN), they are able to mine a physics-guided surrogate model (PGSM) which checks for deviation of the cyber-physical system (CPS) from safety-certified operational characteristics. This allows for early detection of unknown-unknowns based on the physical laws governing the system. This framework was validated by detecting the operational changes in an artificial pancreas due to unknown insulin cartridge errors.
 
The framework can discover unknown unknown scenarios and evaluate potential safety hazards before catastrophic events occur.  
 
Potential Applications
  • Design and validation of safety-critical systems for minimizing hazards due to unknown-unknowns:
    • AI-infused medical system – wearable, at home, or in-hospital settings
    • Semi-Autonomous Driving (SAD) systems
    • Active Machine learning models with human-in-loop and or human-in-plant
Benefits and Advantages
  • Enables early detection of unknown-unknowns
  • Instead of verification, statistical model conformance strategy is performed which enables the system to detect unknown errors
  • Physics guided surrogate models are used to ensure that surrogate models learn essential internal processes of the cyber physical systems
    • Allows the system to detect faults that are invisible in a black box model
  • Enables safety validated system for wider operating environments and scenarios
For more information about this opportunity, please see
 
For more information about the inventor(s) and their research, please see

 

Patent Information:
Direct Link:
https://canberra-ip.technologypublisher.com/tech/Framework_for_Early_Detectio n_of_Unknown-Unknowns
Keywords:
Bookmark this page
For Information, Contact:
Jovan Heusser
Director of Licensing and Business Development
Skysong Innovations
jovan.heusser@skysonginnovations.com