We’d like to welcome you to our thorough tutorial on creating a news aggregator with the help of the mediastack and Web page to PDF APIs. We will walk you through the steps of building a robust news aggregation platform in this tutorial that gathers real-time news stories from a variety of sources and converts them into PDF format. This lesson is for you, whether you’re a developer trying to improve your abilities or an entrepreneur interested in building your own news aggregator.
Table of Contents
What is a News Aggregator?
An online program or platform known as a news aggregator gathers news stories from numerous sources and displays them in one convenient spot. Thanks to it, users can access news articles from various providers without going to each website individually. Users may easily keep up with the most recent news across a variety of topics and categories by using news aggregators.
Why Build a News Aggregator?
The development of a news aggregator can have a number of advantages for both users and developers. Here are some justifications you could think about developing your own news aggregator:
- Centralized Access: Users can get news articles from various sources on a single platform thanks to news aggregators. By removing the need to visit each website independently, users can save time.
- Customized Content: You gain authority over the presented content by creating your own news aggregator. Users can receive tailored news updates by curating particular subjects or categories of interest.
- Real-time Updates: You can retrieve real-time news stories from various publications with the aid of APIs like mediastack. By doing this, customers are guaranteed to receive the most recent information as soon as it is made available.
- PDF Conversion: You can transform web pages into PDF format using the Web page to PDF API. Users can share articles with others in a more usable format or download them for offline reading with this function.
- Monetization Opportunities: With a huge user base, a well-designed news aggregator can generate revenue from advertising, subscriptions, or joint ventures with publishers.
Let’s explore mediastack and the Web page PDF API in more detail now that we are aware of the advantages of creating a news aggregator.
What is mediastack API?
A strong RESTful API called mediastack gives developers access to both recent and old news stories from thousands of sources around the world. You can access headlines, complete articles, and other news-related metadata using mediastack. You can use the API’s numerous filtering options to focus your search on factors like language, country, category, and more.
Here’s an illustration of how to retrieve the most recent headlines using the mediastack API:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
# Python 3 import http.client, urllib.parse conn = http.client.HTTPConnection('api.mediastack.com') params = urllib.parse.urlencode({ 'access_key': 'ACCESS_KEY', 'categories': '-general,-sports', 'sort': 'published_desc', 'limit': 10, }) conn.request('GET', '/v1/news?{}'.format(params)) res = conn.getresponse() data = res.read() print(data.decode('utf-8')) |
In this case, we send our API key as a query parameter along with a GET request made via HTTP to the mediastack API endpoint. To get the top 10 headlines, we set the limit parameter to 10. The response is given back in JSON format, which we can parse and use for collecting relevant information.
Overview of Web Page to PDF API
The Web page to PDF API is a user-friendly RESTful API that enables programmatic conversion of web pages into PDF files. With the help of this API, you may instantly create high-quality PDF files from any web page’s content. Page size, orientation, margins, headers, footers, and other customization options are all supported through the API.
Here’s an illustration of how the Web page to PDF API works to transform a website into a PDF file:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
import requests url = "https://api.apilayer.com/url_to_pdf?url=url" payload = {} headers= { "apikey": "QdSe87VnIPlTkpLrS5i46iVl8XJqlDlf" } response = requests.request("GET", url, headers=headers, data = payload) status_code = response.status_code result = response.text |
In the above example, we send our API key and URL as query parameters along with an HTTP GET request to the PDF API endpoint on the website. We are able to put directly into a file after the response, which is returned in binary format.
How to Use mediastack and Web page to PDF API to Create a News Aggregator
Now that we have covered the basics of mediastack and Web page to PDF API let’s see how we can use these APIs together to create a powerful news aggregator. In our demonstration we will be just integrating mediastack to our multi-platform Delphi FireMonkey application. When you get the news, you also get to select full page report where it fetches the URL of the news and fetches that page and converts into PDF file and saves it to any folder.
Step 1: Register for APIs
To get started with mediastack and Web page to PDF API, you need to register for an account on their respective websites:
Once you have registered for an account, will be able to play with the API and learn about its features.
Step 2: Set Up Your Environment
Before we start coding, let’s set up our development environment. We will be using Delphi for this tutorial, so make sure you have Delphi IDE installed on your system. You can download the latest version of Delphi Community Edition from the official website.
Step 3: Fetching News Articles with mediastack
After establishing our environment, let’s begin by obtaining news items via the mediastack API. We’ve already designed the user interface for you to demonstrate. You can see the used components here.
As you can see there is a button above which is the refresh button. So when you click that mediastack provides you news and if you want to get the complete copy of the news page in PDF format. You should click the bottom button.
Here is the function that fetches the news in our demonstration.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
procedure TFormMain.BtnGetNewsClick(Sender: TObject); begin // fetch news here RESTClient1.ResetToDefaults; RESTClient1.Accept := 'application/json'; RESTClient1.AcceptCharset := 'UTF-8, *;q=0.8'; RESTClient1.BaseURL := Format('http://api.mediastack.com/v1/news?access_key=%s&limit=1&countries=us', ['6a471d4d8e2df064e39580d6d25ec502']); // send API endpoint request RESTRequest1.Execute; // parse the country code from the response try var JSONValue := TJSONObject.ParseJSONValue(RESTResponse1.Content); var JSONArray := JSONValue.GetValue<TJSONArray>('data'); var ArrayElement: TJSONValue; for ArrayElement in JSONArray do begin Memo1.Lines.Append('Title: ' + ArrayElement.GetValue<String>('title')); Memo1.Lines.Append('Description: ' + ArrayElement.GetValue<String> ('description')); Memo1.Lines.Append('Source: ' + ArrayElement.GetValue<String> ('source')); Memo1.Lines.Append('Full News here: ' + ArrayElement.GetValue<String> ('url')); FUrl := ArrayElement.GetValue<String>('url'); // load images to the newly created TImage component var MemoryStream := TMemoryStream.Create; var HTTPClient := TNetHTTPClient.Create(nil); var HTTPRequest := TNetHTTPRequest.Create(nil); HTTPRequest.Client := HTTPClient; try var ImageURL := ArrayElement.GetValue<String>('image'); HTTPRequest.Get(ImageURL, MemoryStream); MemoryStream.Seek(0, soFromBeginning); Image1.Bitmap.LoadFromStream(MemoryStream); finally FreeAndNil(MemoryStream); FreeAndNil(HTTPClient); FreeAndNil(HTTPRequest); end; end; finally end; end; |
Step 4: Converting Web Pages to PDF with Web Page to PDF API
Now that we have fetched news articles using mediastack let’s convert them into PDF format using Web page to PDF API. Here’s an example of how you can convert a web page into PDF format. To do that, you should know the source URL of the news. In our above example, there is the FUrl property that we are saving the URL. So, we will be sending that URL to the Web Page to PDF API, and it gives us another URL. With that URL, we will be able to access the generated/converted PDF file.
Here is the code sample where it sends the URL, downloads the PDF, and finally saves it.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
procedure TFormMain.Button1Click(Sender: TObject); begin // get the full page into PDF RESTClient2.ResetToDefaults; RESTClient2.Accept := 'application/json'; RESTClient2.AcceptCharset := 'UTF-8, *;q=0.8'; RESTClient2.BaseURL := Format('https://api.apilayer.com/url_to_pdf?apikey=%s&url=%s&print=false', ['QdSe87VnIPlTkpLrS5i46iVl8XJqlDlf', FUrl]); // send API endpoint request RESTRequest2.Execute; var JSONValue := TJSONObject.ParseJSONValue(RESTResponse2.Content); try if JSONValue is TJSONObject then begin FUrl := JSONValue.GetValue<String>('pdf_url'); end; finally JSONValue.Free; end; // load images to the newly created TImage component var HTTPClientDownloadPDF := TNetHTTPClient.Create(nil); var HTTPRequestDownloadPDF := TNetHTTPRequest.Create(nil); var FileStream := TFileStream.Create('news.pdf', fmCreate); HTTPRequestDownloadPDF.Client := HTTPClientDownloadPDF; try HTTPRequestDownloadPDF.Get(FUrl, FileStream); finally FreeAndNil(FileStream); FreeAndNil(HTTPRequestDownloadPDF); FreeAndNil(HTTPRequestDownloadPDF); end; end; |
This is our application in action.
After you click the Complete Copy button you get this news.pdf file along with your executable file inside the folder.
You can find the complete source code in this repository. Moreover, check out our latest articles here on our blog:
- Automated Social Media Monitoring ChatGPT App With Keyword Extraction, Sentiment Analysis API
- How To Scrape Review From Amazon And Google
How to Grow the News Aggregator Project?
You can monetize your news aggregator in a number of ways:
- Advertising: You can run advertisements on your platform and get paid for clicks or impressions.
- Subscriptions: You can provide premium subscriptions with more features and content.
- Partnerships: By collaborating with publishers, you can generate income through referral or affiliate marketing schemes.
- Sponsorships: You can approach businesses or groups who share the goals and values of your platform about sponsoring you.
Conclusion
You can use the mediastack and Web page PDF APIs to build your news aggregator by following the instructions in this tutorial. Do not lose this opportunity to create your first news aggregator project with our best-in-class APIs powered by APILayer platform. Get your free APIs keys now!
