Human-Graphs Interface

Problem:

Medical decisions support in medicine are based on many criteria with their relative importance.
The human mind take decisions using a very large graph of about 10**12 neurons interconnected by an even larger number of synapses, about 10**15 Synapses. Computer can also manage information structured as large graphs.

Objectives:

Improvement of interactive exchanges between human graphs and computer graphs.

Approaches:

Eventually decisions require knowledge structured as graph. As well scientific knowledge as well knowledge about the patient.
Up to now the scientific knowledge was available as text. To some extend NLP, Natural Language Processing, can provide partial understanding, but this remain difficult and not very reliable due to the complexity of spoken languages.
In the initial fase of this experimental project the idea is here to bypass the NLP and to work directly on graphs. The project assume that adaptations will become possible from both sides:
- Computers:
  - Computers can make visual graphs easy to understand by people.
- Human:
  - At the other side people could manage graphs in an interactive way. Human can well read graphs with visual clues. Humans are excellent at visual pattern recognition
User motivation:
- The benefits of decision support using graphs are expected to be more important than the effort of people to adapt to the use of graphs.
Main intended uses:
- Knowledge base:
  - Up to now most medical knowledge was represented as free text. Medical experts should now translate their know-how into a more precise way suited for the processing of decision rules. This can best be achieved by means of graphs. The concept are represented by nodes and qualified relations between these nodes are essential.
  - Teachers of medicine have to explain to their students multiple relations between concepts. Some professors like already to use informal graphs in the preparation of their courses. Gaphs are also useful in training sessions.
  - Medical knowledge is always in evolution. Medical experts need easy tools for the maintenance of the details of he knowledge.
  - A knowledge base is intended to contain generic knowledge about symptoms, problems, actions, ... Typically persistent knowledge about a symptom, a disease, a treatment.
- Patient record:
  - During contacts with patients notes must always be registered immediately. Rather than doing that typing completely free text the idea is to try to do it in a more structured way as an interactive graph. Of course this option require a very ergonomic interface. Some NLP could help but the goal is anyhow to create graphs, for example converting MIMIC reports style into graphs.
  - The expected benefit of this effort is to get instantly "Assisted Intelligence".
  - A task oriented approach in order to try to solve the problems of the patient. Problems have a central role with links to symptoms and recommended actions. This according to the "Problem Oriented Medical Record" according of L. Weed philosophy.
  - Shared overview of patient's problems and related information will be also very useful for the coordination between the members of the "care team" in charge of a common patient.
  - A patient record is intended to contain instances of information related to a particular patient, as problems, symptoms, treatments, .... These instances will always be linked to the corresponding generic knowledge. For example many individual lab tests results should have links to the related knowledge about these tests.
Make an experimental prototype as proof of concept.
- Initially experimental decision support in a few limited use cases.
Essential requirements are:
- Real time interactivity and easy navigation
- Visual representation of logical attributes.
- Facility to create or modify element of the graph.
- Graphs must contain the complete information even if some fields are less well structured. Anyhow graphs are seen as backbone or as a "kind of table of content" at the top level, paying much attention to relations between main concepts represented by nodes. That said, if necessary, nodes may contain additional details in a less structured way like free text.
- As far as possible start from existing software freely available in Open Source. Some specific adaptations will be necessary in function of the healthcare context. The freedom to modify the source code is essential, e.i. Open Source.
- Event based architecture and non blocking workflow.
- Fuzzy logic approach, since available information is often incomplete.
Challenges:
- Currently graph can be visualized, but only with standard tools. Updates are possible, but very cumbersome. A much more flexible interface is necessary for non ITI people as medical experts.
Technical requirements for a Knowledge Graph Editor, Graph-Editor-Specs.html .
Needed resources:
- Expertise in ergonomics, software development, graph database, adaptation of existing software components like Neo4j "arrows.app" of Alister Jones, "neovis" of William Lyon, some aspects of Neo4j "bloom", the community of "vis-network", ...

Screen:

The full screen will be backbone of this man-machine interface.
This with freedom to zoom from the global picture to very specific details.
A temporary window in front or on the side of the main screen can present the details as traditional forms and text reports.

Main menu:

A persistent specific small group of nodes on fixed locations on top of the screen.
- The main commands the current user is allowed to ask. Only the few most important commands. For example:
  - Seek a patient, Predefined selections, like the problem list, the medications, User preferences, Administration
  - User preferences:
    - Selection of certain types of nodes and relationships, default selection at the opening of a patient record, preferred style of presentations.
- The main menu should always be available. At least one main node which can be expanded to a larger set of less frequent commands.
- Possibility to go back to the previous transactions.

Selections:

Selection of a subset of the large databases. For example selection in the knowledge base of one disease with related symptoms and recommended actions. In case of a patient record only one patient or even one problem of this patient.
By default the active node or relation is presented with most of his directly related nodes. However users may define preferences about types of relations and number of levels.

Current object:

Introduction:
- The current active object may be a node or a relation, presented as highlighted.
- Remark: access and updates of objects will depend on the current user profile and of the transaction type. The exact rules will be specified in a separate document according to "Role Based Access Control" and "Need to Know" principles. Typically the current "care team" of the patient need access to work. For example a nurse need to read the prescriptions but is normally not allowed to create new ones.
Flying over:
- Show temporarily the content of the node or relation in a pop up window. Intended to make very easy to see quickly the essential of the content. In front of the graph.
Actions on nodes:
- Navigation:
- Activation and opening:
  - ( in principle by means of a double click or shift click).
  - Presentation of the content as above, but in a more persistent way than in the case of the flyover. This frame will persist until explicit "Save" or "Exit".
  - Access to the full content of the node:
    - The content is a frame floating in front of the graph with if necessary the possibility to take 100 % of the screen.
    - The content of node begin at the top with a few standard information as type, date-time, responsible author, etc...
    - Contextual menu:
    - The full content may include any kind of documents as reports, discharge letters , images, XRays, etc...
    - If necessary a cursor in order to see more of a long screen page.
    - Maybe the possibility to jump to related pages, even on an external server. i.e. hypertext facilities, with later come back.
    - "Save" or "Exit".
  - Possibility to edit the content:
    - Some standard fields may be mandatory while additional fields are optional. A set of fields can be proposed
    - Some fields may accept natural language. Or even maybe voice converted immediately to text to be confirmed.
  - Authorizations:
  - "Save":
    - The identity of the current responsible user must always be recorded.
    - If a previous version exist of the same object exist, generate a new version.
    - Previous versions should remain available on request. Indeed it must be possible to retrieve the situation as it was at a given point in time in the past. Important in the context of a team of several actors who may sometimes have different opinions. Moreover a legal requirement in order to justify a decision based on available information at that time.
    - Changes in nodes could activate evaluation procedures.
    - Deprecation:
      - In principle delete should never be allowed immediately.
      - Relations to a nodes must have already been deprecated.
  - "Exit":
Actions on a Relations:
- Similar to actions on nodes.
- Also flyover for quick inspection.
- Attributes of relations:
  - Types of relations. A set of relations types will be defined. Only one type per relationship but more than one relationships may exist between the same 2 nodes.
  - Attribute of relations:
    - Very important in evaluation procedures; In patient care uncertainties are very common and many things are never completely "yes" or "no". In many situation doctors work with approximated best "likelihoods". The GrApH-AI project should try to quantify these relationships more precisely.
    - Weight representation:
      - Inside the database weights should be normalized in a scale from 0 to 1. Indeed decision procedures should be based on the best possible values.
      - On visual graph by means of the thickness of arrows.
      - In traditional reports the score should be rounded to adjectives, like "weak" or "strong", "not to exclude" or "certain", ...
    - Weight should take account of several factors having each their own weight.
      - For example "degree of belief" is different from "probability".
      - "Importance" can be seen from different point of view as life expectancy or patient quality of life.
      - Frequency of appearance of a symptom in relation to a diagnose should be quantified.
      - ...
- Relations are to be seen as independent objects. For example the author or a Relation may be different from the authors of the 2 related nodes.
- The same 2 nodes may have multiple relations. For example when 2 authors have different vision on a relation.
- Relations have a direction, but traversal is possible in both directions.
- As above changes in relations should be propagated to concerned evaluation procedures.
- Save or Exit as above.
Evaluation procedures:
- Nodes may contain rules, for example computation of the likelihood and the severity of a problem in function of the available symptoms. The extended specifications of these procedures will be discussed in an other chapter about decision support.
- Activation:
  - In case of change of some parameters. Could also be activated by "events" coming from other procedures, in a cascading sequence of procedures.
  - Manual activation in the edit window.
  - The result of a procedure could activate to evaluation of other procedures and recommendation procedures, specific new selections, to create new nodes and relations, to broadcast new events, .etc ...
- .........
Creation of a new Node:
- A node may be created in an empty space.
- Any new node must immediately be edited, at least with a type and the mandatory attributes.
- Authorization will depend on the profile of the current user and the type of node.
Creation of a new Relation:
- Drag an arrow from one node to an other one.
- Any new relation must immediately be edited, at least with the mandatory attributes.
- Dragging an arrow to an empty location will create a new node there, as above.
- Authorization will depend on the profile of the current user.
Visual presentation:
- Introduction:
  - In order to make a very intuitive visual interface the most important types and attributes should appear in a visual way, or style.
  - The choice of a particular style may depend on user profile and preferences.
- Logical options, as:
  - Type,
  - Weight, intensity,
  - Probability
  - Degree of belief,
  - Author, moreover possibility of endorsement by others,
  - Importance,
  - Importance decay over time.
  - Index of previous version,
  - New information, novelty:
    - In function of a predefined delay
    - In function of what is new for the current user.
  - ...
- Visual options, as:
  - In general:
    - Color
    - Intensity or darkness
    - Text color,
    - Background color,
    - Text size
    - Boder size and color,
    - Blinking:
    - Background,
    - Blinking,
    - Halo,
    - Movements or oscillations,
    - ...
  - For Nodes:
    - Size
    - Shape (round, oval, square, ..)
    - Border:
      - Color
      - Thickness
    - Position in the graph, attraction or repulsion to other types of nodes. The current active node could be moved to the center of the screen.
    - Inversed text and background colors,
    - Moving nodes and associated relations
    - Position in a virtual 3D space,
    - ...
  - For Relations:
    - Thickness
    - Direction,
    - Dashed style,
    - Attachment side,
    - ...
  - ...
..