In this article, we will create a News Web Application using Flask and NewsAPI. The web page will display top headlines and a search bar where the user can enter a query, after processing the query, the webpage will display all relevant articles (up to a max of 100 headlines). We will create a simple user interface using HTML and bootstrap. You can use any IDE of your choice for eg. VS code or sublime text etc.
Flask and NewsAPI introduction
Flask: Flask is a web framework written in python which allows us to create server-side endpoints. You can learn more about flask from this link: https://www.geeksforgeeks.org/python-introduction-to-web-development-using-flask/.
NewsAPI: News API is a simple Rest API for retrieving live articles from all over the web. Using News API we can fetch top headlines of a country from a particular source like the Times of India, Hindustan Times, BBC News, and many more. We can also fetch articles related to a particular topic. visit https://newsapi.org/ for more information.
API key generation
In order to use the News API in our application, we need to generate a unique API key from https://newsapi.org/. Visit this website and create your free account, on successful registration and email verification you will get your API key on the screen. While registering you may need to choose a developer plan (choose according to your requirements). Save this API key somewhere so that it can be used further.
Flask and NewsAPI installation
We will simply install Flask using the pip command :
pip install flask
After successful installation creates one folder for application name this folder flask_news or anything else of your choice. Inside this folder create a new file called app.py. We will write our entire back-end code in this file in the upcoming steps.
To install NewsAPI we will use a pip command like this :
pip install newsapi-python
You can learn more about the newsapi-python library from here: https://newsapi.org/docs/client-libraries/python
Creating backend endpoints
There are two things we want from our application :
- Display top headlines when the web application is opened.
- Show results based on a search query.
First of all, we will import flask and newsapi in app.py file.
Python3
# import libraries from flask import Flask, render_template, request from newsapi import NewsApiClient # init flask app app = Flask(__name__) # Init news api newsapi = NewsApiClient(api_key='YOUR_API_KEY') |
To get the top headlines we will use get_top_headlines function from newsapi library. We can pass parameters to this function like country, language, sources, etc. In this application, we will pass an additional parameter called page_size to get all the headlines. The newsapi library function only returns the top 20 articles but it also has a field called totalResults in the JSON response object. We will modify our function call to fetch all results using the page_size parameter. This part of the code looks like this :
Python3
# this function returns a JSON object top_headlines = newsapi.get_top_headlines(country="in", language="en") total_results = top_headlines['totalResults'] # the maximum value for page_size parameter is 100 # so we need to keep it at max 100 if total_results > 100: total_results = 100 # fetch articles where no. of articles=total_results # this returns a list of articles all_headlines = newsapi.get_top_headlines(country="in", language="en", page_size=total_results)['articles'] |
We will send this list of articles to the HTML code.
Get articles related to a search query
This will be related to a post request We are going to use the get_everything function with parameters like query sources, domains, language, sorting based on, and page_size. We will fetch all sources with the help of the get_sources function of newsapi and to fetch all domains we will use ‘URL’ property of each source. Here is the code which shows how we are going to handle it:
Python3
# this function will return sources and domains in str format # each of them separated by a comma def get_sources_and_domains(): all_sources = newsapi.get_sources()['sources'] sources = [] domains = [] for e in all_sources: id = e['id'] domain = domain.replace("www.", "") slash = domain.find('/') if slash != -1: domain = domain[:slash] sources.append(id) domains.append(domain) sources = ", ".join(sources) domains = ", ".join(domains) return sources, domains # in post request sources, domains = get_sources_and_domains() related_news = newsapi.get_everything(q=keyword, sources=sources, domains=domains, language='en', sort_by='relevancy') no_of_articles = related_news['totalResults'] # the max number of article can be fetched is 100 if no_of_articles > 100: no_of_articles = 100all_articles = newsapi.get_everything(q=keyword, sources=sources, domains=domains, language='en', sort_by='relevancy', page_size=no_of_articles)['articles'] # we will send this list of articles to the html web page |
Now we can model the entire program to get the required flask API end-points. Basically, we need a way to handle GET requests to fetch all the headlines and another to handle POST requests for a search query. This is what the entire app.py will look like after adding the necessary route.
Complete Code of app.py:
Python3
# import libraries from flask import Flask, render_template, request from newsapi import NewsApiClient # init flask app app = Flask(__name__) # Init news api newsapi = NewsApiClient(api_key='70fdb9ba81ba40b6bda148e672898bd9') # helper function def get_sources_and_domains(): all_sources = newsapi.get_sources()['sources'] sources = [] domains = [] for e in all_sources: id = e['id'] domain = domain.replace("www.", "") slash = domain.find('/') if slash != -1: domain = domain[:slash] sources.append(id) domains.append(domain) sources = ", ".join(sources) domains = ", ".join(domains) return sources, domains @app.route("/", methods=['GET', 'POST']) def home(): if request.method == "POST": sources, domains = get_sources_and_domains() keyword = request.form["keyword"] related_news = newsapi.get_everything(q=keyword, sources=sources, domains=domains, language='en', sort_by='relevancy') no_of_articles = related_news['totalResults'] if no_of_articles > 100: no_of_articles = 100 all_articles = newsapi.get_everything(q=keyword, sources=sources, domains=domains, language='en', sort_by='relevancy', page_size = no_of_articles)['articles'] return render_template("home.html", all_articles = all_articles, keyword=keyword) else: top_headlines = newsapi.get_top_headlines(country="in", language="en") total_results = top_headlines['totalResults'] if total_results > 100: total_results = 100 all_headlines = newsapi.get_top_headlines(country="in", language="en", page_size=total_results)['articles'] return render_template("home.html", all_headlines = all_headlines) return render_template("home.html") if __name__ == "__main__": app.run(debug = True) |
Creating User Interface
We will create one HTML page that will serve all user requests. Create a folder called templates in your root directory (flask_news folder). Create a file called “home.html” inside the templates folder. The folder structure should look like this :
Now add final code into home.html :
HTML
<!DOCTYPE html> <html> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>News application</title> <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-1BmE4kWBq78iYhFldvKuhfTAU6auU8tT94WrHftjDbrCEXSU1oBoqyl2QvZ6jIW3" crossorigin="anonymous"> </head> <body> <div class="container"> <form action="/" method="post"> <div class="input-group mb-3"> <input type="text" class="form-control" name="keyword" placeholder="Enter keyword ...." aria-label="Recipient's username" aria-describedby="button-addon2"> <button class="btn btn-outline-secondary" type="button" id="button-addon2">Search</button> </div> </form> </div> {% if all_headlines %} <center><h1>Headlines</h1></center> <div class="row row-cols-1 row-cols-md-2 g-4 mx-3 my-3"> {% for headline in all_headlines %} <div class="col"> <div class="card h-100"> <img src="{{headline['urlToImage']}}" class="card-img-top" alt="..."> <div class="card-body"> <h3 class="card-title">{{headline['title']}}</h3> <p class="card-text">{{headline['description']}} <a href="{{headline['url']}}" target="blank">Read More...</a></p> </div> <div class="card-footer"> <small class="text-muted">{{headline['source']['name']}}</small> </div> </div> </div> {% endfor %} </div> {% endif %} {% if all_articles %} <center><h1>Results for '{{keyword}}'</h1></center> <div class="row row-cols-1 row-cols-md-2 g-4 mx-3 my-3"> {% for article in all_articles %} <div class="col"> <div class="card h-100"> <img src="{{article['urlToImage']}}" class="card-img-top" alt="..."> <div class="card-body"> <h3 class="card-title">{{article['title']}}</h3> <p class="card-text">{{article['description']}} <a href="{{article['url']}}" target="blank">Read More...</a></p> </div> <div class="card-footer"> <small class="text-muted">{{article['source']['name']}}</small> </div> </div> </div> {% endfor %} </div> {% endif %} <script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.10.2/dist/umd/popper.min.js" integrity="sha384-7+zCNj/IqJ95wo16oMtfsKbZ9ccEh31eOz1HGyDuCQ6wgnyJNSYdrPa03rtR1zdB" crossorigin="anonymous"></script> <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.min.js" integrity="sha384-QJHtvGhmr9XOIpI6YVutG+2QOK9T+ZnN4kzFN1RtK3zEFEIsxhlmWl5/YESvpZ13" crossorigin="anonymous"></script> </body> </html> |
Testing the application
Let’s run the server and check if our application works or not. Go to the terminal and run the app.py file using this command :
python .\app.py
The output on the console should look like this :
Now go to http://127.0.0.1:5000 or localhost:5000 and the output should be like this:
