Bayesian Network

TL;DR

Represents random variables as nodes and probabilistic dependencies as directed edges in a DAG.
Encodes a compact joint probability distribution via local conditional probabilities.
Enables answering conditional-probability queries (e.g., given some evidence, compute the probability of an outcome).

Definition

A Bayesian network is a probabilistic graphical model that represents a set of random variables and their conditional dependencies using a directed acyclic graph (DAG). Each node in the graph represents a random variable, and the directed edges represent the probabilistic dependencies between the variables. The Bayesian network provides a compact representation of the joint probability distribution of the variables and allows reasoning about their conditional dependencies.

Explanation

Nodes correspond to random variables; directed edges indicate how one variable probabilistically depends on another.
The model associates probabilities with each node, typically conditional probabilities given its parent nodes in the DAG.
Using these conditional probabilities and the rules of probability, the Bayesian network can be used to compute joint and conditional probabilities over the variables, allowing queries such as “what is the probability of X given evidence Y”.

Examples

Age, smoking, and lung cancer

A Bayesian network with three nodes — age, smoking, and lung cancer — where the smoking node has an edge pointing to the lung cancer node (indicating smoking can affect lung cancer risk). The network includes probabilities such as the probability of developing lung cancer given a person’s age and smoking habits. For instance, if we know a person is 50 years old and a smoker, we can use the probabilities in the network to calculate the probability of them developing lung cancer.

MYCIN medical diagnosis system

MYCIN, developed in the 1970s to diagnose infections and recommend antibiotic treatments, is an example of a Bayesian network applied to medical diagnosis. Its nodes represent various symptoms, lab test results, and antibiotic treatments, and edges represent the probabilistic dependencies between those variables. A doctor can input a patient’s symptoms and lab results and receive a treatment recommendation based on the network’s probabilities.

Financial analysis

Bayesian networks can model relationships among economic variables such as interest rates, inflation, and stock prices. The network can be used to forecast future values of these variables and to make investment decisions based on the probabilities of different scenarios.

Natural language processing

In NLP, Bayesian networks can model probabilities of different words and phrases occurring in a given context. For example, a network could predict the likelihood of a word appearing in a sentence based on the words that precede and follow it, which is useful for tasks like spelling correction and language translation.

Use cases

Medical diagnosis (example: MYCIN)
Financial analysis and forecasting
Natural language processing tasks such as spelling correction and translation

Directed acyclic graph (DAG)
Joint probability distribution
Conditional probability