Given a Markov chain G, we have the find the probability of reaching the state F at time t = T if we start from state S at time t = 0.
A Markov chain is a random process consisting of various states and the probabilities of moving from one state to another. We can represent it using a directed graph where the nodes represent the states and the edges represent the probability of going from one node to another. It takes unit time to move from one node to another. The sum of the associated probabilities of the outgoing edges is one for every node.
Consider the given Markov Chain( G ) as shown in below image:
Examples:
Input : S = 1, F = 2, T = 1 Output : 0.23 We start at state 1 at t = 0, so there is a probability of 0.23 that we reach state 2 at t = 1. Input : S = 4, F = 2, T = 100 Output : 0.284992
We can use dynamic programming and depth-first search (DFS) to solve this problem, by taking the state and the time as the two DP variables. We can easily observe that the probability of going from state A to state B at time t is equal to the product of the probability of being at A at time t – 1 and the probability associated with the edge connecting A and B. Therefore the probability of being at B at time t is the sum of this quantity for all A adjacent to B.
Below is the implementation of the above approach:
C++
// C++ implementation of the above approach#include <bits/stdc++.h>using namespace std;// Macro for vector of pair to store// each node with edge#define vp vector<pair<int, float> >// Function to calculate the// probability of reaching F// at time T after starting// from Sfloat findProbability(vector<vp>& G, int N, int F, int S, int T){ // Declaring the DP table vector<vector<float> > P(N + 1, vector<float>(T + 1, 0)); // Probability of being at S // at t = 0 is 1.0 P[S][0] = 1.0; // Filling the DP table // in bottom-up manner for (int i = 1; i <= T; ++i) for (int j = 1; j <= N; ++j) for (auto k : G[j]) P[j][i] += k.second * P[k.first][i - 1]; return P[F][T];}// Driver codeint main(){ // Adjacency list vector<vp> G(7); // Building the graph // The edges have been stored in the row // corresponding to their end-point G[1] = vp({ { 2, 0.09 } }); G[2] = vp({ { 1, 0.23 }, { 6, 0.62 } }); G[3] = vp({ { 2, 0.06 } }); G[4] = vp({ { 1, 0.77 }, { 3, 0.63 } }); G[5] = vp({ { 4, 0.65 }, { 6, 0.38 } }); G[6] = vp({ { 2, 0.85 }, { 3, 0.37 }, { 4, 0.35 }, { 5, 1.0 } }); // N is the number of states int N = 6; int S = 4, F = 2, T = 100; cout << "The probability of reaching " << F << " at time " << T << " \nafter starting from " << S << " is " << findProbability(G, N, F, S, T); return 0;} |
Java
// Java implementation of the above approachimport java.util.*;class GFG{ static class pair { int first; double second; public pair(int first, double second) { this.first = first; this.second = second; } } // Function to calculate the // probability of reaching F // at time T after starting // from S static float findProbability(Vector<pair>[] G, int N, int F, int S, int T) { // Declaring the DP table float[][] P = new float[N + 1][T + 1]; // Probability of being at S // at t = 0 is 1.0 P[S][0] = (float) 1.0; // Filling the DP table // in bottom-up manner for (int i = 1; i <= T; ++i) for (int j = 1; j <= N; ++j) for (pair k : G[j]) P[j][i] += k.second * P[k.first][i - 1]; return P[F][T]; } // Driver code public static void main(String[] args) { // Adjacency list Vector<pair>[] G = new Vector[7]; for (int i = 0; i < 7; i++) { G[i] = new Vector<pair>(); } // Building the graph // The edges have been stored in the row // corresponding to their end-point G[1].add(new pair(2, 0.09)); G[2].add(new pair(1, 0.23)); G[2].add(new pair(6, 0.62)); G[3].add(new pair(2, 0.06)); G[4].add(new pair(1, 0.77)); G[4].add(new pair(3, 0.63)); G[5].add(new pair(4, 0.65)); G[5].add(new pair(6, 0.38)); G[6].add(new pair(2, 0.85)); G[6].add(new pair(3, 0.37)); G[6].add(new pair(4, 0.35)); G[6].add(new pair(5, 1.0)); // N is the number of states int N = 6; int S = 4, F = 2, T = 100; System.out.print("The probability of reaching " + F + " at time " + T + " \nafter starting from " + S + " is " + findProbability(G, N, F, S, T)); }}// This code is contributed by Rajput-Ji |
Python3
# Python3 implementation of the above approach # Macro for vector of pair to store# each node with edge# define vp vector<pair<int, float> > # Function to calculate the# probability of reaching F# at time T after starting# from Sdef findProbability(G, N, F, S, T): # Declaring the DP table P = [[0 for i in range(T + 1)] for j in range(N + 1)] # Probability of being at S # at t = 0 is 1.0 P[S][0] = 1.0; # Filling the DP table # in bottom-up manner for i in range(1, T + 1): for j in range(1, N + 1): for k in G[j]: P[j][i] += k[1] * P[k[0]][i - 1]; return P[F][T]# Driver codeif __name__=='__main__': # Adjacency list G = [0 for i in range(7)] # Building the graph # The edges have been stored in the row # corresponding to their end-point G[1] = [ [ 2, 0.09 ] ] G[2] = [ [ 1, 0.23 ], [ 6, 0.62 ] ] G[3] = [ [ 2, 0.06 ] ] G[4] = [ [ 1, 0.77 ], [ 3, 0.63 ] ] G[5] = [ [ 4, 0.65 ], [ 6, 0.38 ] ] G[6] = [ [ 2, 0.85 ], [ 3, 0.37 ], [ 4, 0.35 ], [ 5, 1.0 ] ] # N is the number of states N = 6 S = 4 F = 2 T = 100 print("The probability of reaching {} at " "time {}\nafter starting from {} is {}".format( F, T, S, findProbability(G, N, F, S, T))) # This code is contributed by rutvik_56 |
C#
// C# implementation of the above approachusing System;using System.Collections.Generic;class GFG{ class pair { public int first; public double second; public pair(int first, double second) { this.first = first; this.second = second; } } // Function to calculate the // probability of reaching F // at time T after starting // from S static float findProbability(List<pair>[] G, int N, int F, int S, int T) { // Declaring the DP table float[,] P = new float[N + 1, T + 1]; // Probability of being at S // at t = 0 is 1.0 P[S, 0] = (float) 1.0; // Filling the DP table // in bottom-up manner for (int i = 1; i <= T; ++i) for (int j = 1; j <= N; ++j) foreach (pair k in G[j]) P[j, i] += (float)k.second * P[k.first, i - 1]; return P[F, T]; } // Driver code public static void Main(String[] args) { // Adjacency list List<pair>[] G = new List<pair>[7]; for (int i = 0; i < 7; i++) { G[i] = new List<pair>(); } // Building the graph // The edges have been stored in the row // corresponding to their end-point G[1].Add(new pair(2, 0.09)); G[2].Add(new pair(1, 0.23)); G[2].Add(new pair(6, 0.62)); G[3].Add(new pair(2, 0.06)); G[4].Add(new pair(1, 0.77)); G[4].Add(new pair(3, 0.63)); G[5].Add(new pair(4, 0.65)); G[5].Add(new pair(6, 0.38)); G[6].Add(new pair(2, 0.85)); G[6].Add(new pair(3, 0.37)); G[6].Add(new pair(4, 0.35)); G[6].Add(new pair(5, 1.0)); // N is the number of states int N = 6; int S = 4, F = 2, T = 100; Console.Write("The probability of reaching " + F + " at time " + T + " \nafter starting from " + S + " is " + findProbability(G, N, F, S, T)); }}// This code is contributed by 29AjayKumar |
Javascript
<script>// Javascript implementation of the above approach// Function to calculate the// probability of reaching F// at time T after starting// from Sfunction findProbability(G, N, F, S, T){ // Declaring the DP table var P = Array.from(Array(N+1), ()=> Array(T+1).fill(0)) // Probability of being at S // at t = 0 is 1.0 P[S][0] = 1.0; // Filling the DP table // in bottom-up manner for (var i = 1; i <= T; ++i) for (var j = 1; j <= N; ++j) G[j].forEach(k => { P[j][i] += k[1] * P[k[0]][i - 1]; }); return P[F][T];}// Driver code// Adjacency listvar G = Array(7);// Building the graph// The edges have been stored in the row// corresponding to their end-pointG[1] = [ [ 2, 0.09 ] ];G[2] = [ [ 1, 0.23 ], [ 6, 0.62 ] ];G[3] = [ [ 2, 0.06 ] ];G[4] = [ [ 1, 0.77 ], [ 3, 0.63 ] ];G[5] = [ [ 4, 0.65 ], [ 6, 0.38 ] ];G[6] = [ [ 2, 0.85 ], [ 3, 0.37 ], [ 4, 0.35 ], [ 5, 1.0 ] ];// N is the number of statesvar N = 6;var S = 4, F = 2, T = 100;document.write( "The probability of reaching " + F + " at time " + T + " <br>after starting from " + S + " is " + findProbability(G, N, F, S, T).toFixed(8));</script> |
Output:
The probability of reaching 2 at time 100 after starting from 4 is 0.284992
Complexity Analysis:
- Time complexity: O(N2 * T)
- Space complexity: O(N * T)
Feeling lost in the world of random DSA topics, wasting time without progress? It’s time for a change! Join our DSA course, where we’ll guide you on an exciting journey to master DSA efficiently and on schedule.
Ready to dive in? Explore our Free Demo Content and join our DSA course, trusted by over 100,000 neveropen!
Ready to dive in? Explore our Free Demo Content and join our DSA course, trusted by over 100,000 neveropen!
