Branch and bound is an algorithm design paradigm which is generally used for solving combinatorial optimization problems. These problems typically exponential in terms of time complexity and may require exploring all possible permutations in worst case. Branch and Bound solve these problems relatively quickly.
Let us consider below 0/1 Knapsack problem to understand Branch and Bound. Given two integer arrays val[0..n-1] and wt[0..n-1] that represent values and weights associated with n items respectively.
Find out the maximum value subset of val[] such that sum of the weights of this subset is smaller than or equal to Knapsack capacity W. Let us explore all approaches for this problem.
- A Greedy approach is to pick the items in decreasing order of value per unit weight. The Greedy approach works only for fractional knapsack problem and may not produce correct result for 0/1 knapsack.
- We can use Dynamic Programming (DP) for 0/1 Knapsack problem. In DP, we use a 2D table of size n x W. The DP Solution doesn’t work if item weights are not integers.
- Since DP solution doesn’t always work, a solution is to use Brute Force. With n items, there are 2n solutions to be generated, check each to see if they satisfy the constraint, save maximum solution that satisfies constraint. This solution can be expressed as tree.
- We can use Backtracking to optimize the Brute Force solution. In the tree representation, we can do DFS of tree. If we reach a point where a solution no longer is feasible, there is no need to continue exploring. In the given example, backtracking would be much more effective if we had even more items or a smaller knapsack capacity.
Branch and BoundThe backtracking based solution works better than brute force by ignoring infeasible solutions. We can do better (than backtracking) if we know a bound on best possible solution subtree rooted with every node. If the best in subtree is worse than current best, we can simply ignore this node and its subtrees. So we compute bound (best solution) for every node and compare the bound with current best solution before exploring the node. Example bounds used in below diagram are, A down can give $315, B down can $275, C down can $225, D down can $125 and E down can $30. In the next article, we have discussed the process to get these bounds.
Branch and bound is very useful technique for searching a solution but in worst case, we need to fully calculate the entire tree. At best, we only need to fully calculate one path through the tree and prune the rest of it.
Pseudo code
function knapsack(items, max_weight):
best_value = 0
queue = [{items: [], value: 0, weight: 0}]
while queue is not empty:
node = queue.pop()
if node is a leaf node:
update best_value if necessary
else:
for each remaining item:
child = create child node for item
if child is promising:
queue.append(child)
return best_valuefunction is_promising(node, max_weight, best_value):
if node.weight > max_weight:
return False
if node.value + bound(node.items) < best_value:
return False
return Truefunction bound(items):
# Calculate an upper bound on the value of the remaining items
# using some heuristic (e.g., the fractional knapsack algorithm)
…
Example:
Java
import java.util.*; class KnapsackNode { // The items that have been included in the knapsack so far List<Integer> items; // The total value of the items in the knapsack so far int value; // The total weight of the items in the knapsack so far int weight; public KnapsackNode(List<Integer> items, int value, int weight) { this.items = items; this.value = value; this.weight = weight; } } class Item { // The value of the item int value; // The weight of the item int weight; // The value-to-weight ratio of the item double ratio; public Item(int value, int weight) { this.value = value; this.weight = weight; this.ratio = (double) value / weight; } } public class Knapsack { // The maximum weight capacity of the knapsack int maxWeight; // The list of items Item[] items; public Knapsack(int maxWeight, Item[] items) { this.maxWeight = maxWeight; this.items = items; } // Solves the 0/1 knapsack problem using branch and bound public int solve() { // Sort the items in decreasing order of value per unit weight Arrays.sort(items, (a, b) -> -Double.compare(a.ratio, b.ratio)); // The best value found so far int bestValue = 0; // The queue of nodes to be explored Queue<KnapsackNode> queue = new LinkedList<>(); queue.add(new KnapsackNode(new ArrayList<>(), 0, 0)); while (!queue.isEmpty()) { KnapsackNode node = queue.poll(); int i = node.items.size(); if (i == items.length) { // This is a leaf node, so update the best value if necessary bestValue = Math.max(bestValue, node.value); } else { // Add the child nodes for the remaining items Item item = items[i]; KnapsackNode withItem = new KnapsackNode( new ArrayList<>(node.items), node.value + item.value, node.weight + item.weight); if (isPromising(withItem, maxWeight, bestValue)) { queue.add(withItem); } KnapsackNode withoutItem = new KnapsackNode( new ArrayList<>(node.items), node.value, node.weight); if (isPromising(withoutItem, maxWeight, bestValue)) { queue.add(withoutItem); } } } return bestValue; } // Returns true if the given node is promising (i.e., it may lead to a better solution than the current best) private boolean isPromising(KnapsackNode node, int maxWeight, int best |
Javascript
class KnapsackNode { constructor(items, value, weight) { this.items = items; this.value = value; this.weight = weight; } } class Item { constructor(value, weight) { this.value = value; this.weight = weight; this.ratio = value / weight; } } class Knapsack { constructor(maxWeight, items) { this.maxWeight = maxWeight; this.items = items; } solve() { this.items.sort((a, b) => b.ratio - a.ratio); let bestValue = 0; const queue = [new KnapsackNode([], 0, 0)]; while (queue.length > 0) { const node = queue.shift(); const i = node.items.length; if (i === this.items.length) { bestValue = Math.max(bestValue, node.value); } else { const item = this.items[i]; const withItem = new KnapsackNode( [...node.items, i], node.value + item.value, node.weight + item.weight ); if (this.isPromising(withItem, this.maxWeight, bestValue)) { queue.push(withItem); } const withoutItem = new KnapsackNode( [...node.items], node.value, node.weight ); if (this.isPromising(withoutItem, this.maxWeight, bestValue)) { queue.push(withoutItem); } } } return bestValue; } isPromising(node, maxWeight, bestValue) { return ( node.weight <= maxWeight && node.value + this.getBound(node) > bestValue ); } getBound(node) { let remainingWeight = this.maxWeight - node.weight; let bound = node.value; for (let i = node.items.length; i < this.items.length; i++) { const item = this.items[i]; if (remainingWeight >= item.weight) { bound += item.value; remainingWeight -= item.weight; } else { bound += remainingWeight * item.ratio; break; } } return bound; } } |
Python3
from queue import Queue from typing import List class KnapsackNode: def __init__(self, items: List[int], value: int, weight: int): self.items = items self.value = value self.weight = weight class Item: def __init__(self, value: int, weight: int): self.value = value self.weight = weight self.ratio = value / weight class Knapsack: def __init__(self, maxWeight: int, items: List[Item]): self.maxWeight = maxWeight self.items = items def solve(self) -> int: self.items.sort(key=lambda x: x.ratio, reverse=True) bestValue = 0 queue = [KnapsackNode([], 0, 0)] while queue: node = queue.pop(0) i = len(node.items) if i == len(self.items): bestValue = max(bestValue, node.value) else: item = self.items[i] withItem = KnapsackNode( node.items + [i], node.value + item.value, node.weight + item.weight ) if self.isPromising(withItem, self.maxWeight, bestValue): queue.append(withItem) withoutItem = KnapsackNode( node.items, node.value, node.weight ) if self.isPromising(withoutItem, self.maxWeight, bestValue): queue.append(withoutItem) return bestValue def isPromising(self, node: KnapsackNode, maxWeight: int, bestValue: int) -> bool: return node.weight <= maxWeight and node.value + self.getBound(node) > bestValue def getBound(self, node: KnapsackNode) -> float: remainingWeight = self.maxWeight - node.weight bound = node.value for i in range(len(node.items), len(self.items)): item = self.items[i] if remainingWeight >= item.weight: bound += item.value remainingWeight -= item.weight else: bound += remainingWeight * item.ratio break return bound |
C++
#include <iostream> #include <algorithm> #include <vector> #include <queue> using namespace std; class Item { public: int value; int weight; double ratio; Item(int value, int weight) { this->value = value; this->weight = weight; this->ratio = (double)value / weight; } }; class KnapsackNode { public: vector<int> items; int value; int weight; KnapsackNode(vector<int> items, int value, int weight) { this->items = items; this->value = value; this->weight = weight; } }; class Knapsack { public: int maxWeight; vector<Item> items; Knapsack(int maxWeight, vector<Item> items) { this->maxWeight = maxWeight; this->items = items; } int solve() { sort(this->items.begin(), this->items.end(), [](const Item& a, const Item& b) { return a.ratio > b.ratio; }); int bestValue = 0; queue<KnapsackNode> q; q.push(KnapsackNode({}, 0, 0)); while (!q.empty()) { KnapsackNode node = q.front(); q.pop(); int i = node.items.size(); if (i == this->items.size()) { bestValue = max(bestValue, node.value); } else { Item item = this->items[i]; KnapsackNode withItem(node.items, node.value + item.value, node.weight + item.weight); if (isPromising(withItem, this->maxWeight, bestValue)) { q.push(withItem); } KnapsackNode withoutItem(node.items, node.value, node.weight); if (isPromising(withoutItem, this->maxWeight, bestValue)) { q.push(withoutItem); } } } return bestValue; } bool isPromising(KnapsackNode node, int maxWeight, int bestValue) { return node.weight <= maxWeight && node.value + getBound(node) > bestValue; } int getBound(KnapsackNode node) { int remainingWeight = this->maxWeight - node.weight; int bound = node.value; for (int i = node.items.size(); i < this->items.size(); i++) { Item item = this->items[i]; if (remainingWeight >= item.weight) { bound += item.value; remainingWeight -= item.weight; } else { bound += remainingWeight * item.ratio; break; } } return bound; } }; |
C#
using System; using System.Collections.Generic; using System.Linq; class Item { public int value; public int weight; public double ratio; public Item(int value, int weight) { this.value = value; this.weight = weight; this.ratio = (double)value / weight; } } class KnapsackNode { public List<int> items; public int value; public int weight; public KnapsackNode(List<int> items, int value, int weight) { this.items = items; this.value = value; this.weight = weight; } } class Knapsack { public int maxWeight; public List<Item> items; public Knapsack(int maxWeight, List<Item> items) { this.maxWeight = maxWeight; this.items = items; } public int Solve() { items = items.OrderByDescending(i = > i.ratio) .ToList(); int bestValue = 0; Queue<KnapsackNode> q = new Queue<KnapsackNode>(); q.Enqueue(new KnapsackNode(new List<int>(), 0, 0)); while (q.Count > 0) { KnapsackNode node = q.Dequeue(); int i = node.items.Count; if (i == items.Count) { bestValue = Math.Max(bestValue, node.value); } else { Item item = items[i]; KnapsackNode withItem = new KnapsackNode( new List<int>(node.items), node.value + item.value, node.weight + item.weight); if (IsPromising(withItem, maxWeight, bestValue)) { q.Enqueue(withItem); } KnapsackNode withoutItem = new KnapsackNode( new List<int>(node.items), node.value, node.weight); if (IsPromising(withoutItem, maxWeight, bestValue)) { q.Enqueue(withoutItem); } } } return bestValue; } private bool IsPromising(KnapsackNode node, int maxWeight, int bestValue) { return node.weight <= maxWeight && node.value + GetBound(node) > bestValue; } private int GetBound(KnapsackNode node) { int remainingWeight = maxWeight - node.weight; int bound = node.value; for (int i = node.items.Count; i < items.Count; i++) { Item item = items[i]; if (remainingWeight >= item.weight) { bound += item.value; remainingWeight -= item.weight; } else { bound += (int)(remainingWeight * item.ratio); break; } } return bound; } } |
Java
Item[] items = { new Item(60, 10), new Item(100, 20), new Item(120, 30) }; Knapsack knapsack = new Knapsack(50, items); int bestValue = knapsack.solve(); System.out.println("Best value: " + bestValue); |
Javascript
const items = [ new Item(60, 10), new Item(100, 20), new Item(120, 30) ]; const knapsack = new Knapsack(50, items); const bestValue = knapsack.solve(); console.log("Best value: " + bestValue); |
Python3
items = [ Item(60, 10), Item(100, 20), Item(120, 30) ] knapsack = Knapsack(50, items) bestValue = knapsack.solve() print("Best value: " + str(bestValue)) |
C++
int main() { vector<Item> items = { Item(60, 10), Item(100, 20), Item(120, 30) }; Knapsack knapsack(50, items); int result = knapsack.solve(); cout << "Best value: " << result << endl; return 0; } |
C#
class Program { static void Main(string[] args) { List<Item> items = new List<Item> { new Item(60, 10), new Item(100, 20), new Item(120, 30) }; Knapsack knapsack = new Knapsack(50, items); int bestValue = knapsack.Solve(); Console.WriteLine("Best value: " + bestValue); } } |
output:
Best value: 220
Time Complexity: O(N), as only one path through the tree will have to be traversed in the best case and its worst time complexity is still given as O(2^N) .
Source:
Above images and content is adopted from following nice link. http://www.cse.msu.edu/~torng/Classes/Archives/cse830.03fall/Lectures/Lecture11.ppt Branch and Bound | Set 2 (Implementation of 0/1 Knapsack)
This article is contributed Utkarsh Trivedi and Abhay Mishra. If you like neveropen and would like to contribute, you can also write an article and mail your article to review-team@geeksforgeeks.org. See your article appearing on the neveropen main page and help other Geeks. Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.
Ready to dive in? Explore our Free Demo Content and join our DSA course, trusted by over 100,000 neveropen!
