Gatsby Wikipedia Fetcher

Wikipedia logo
The encyclopedia we all build and use (wikipedia.org)

TL;DR

GatsbyJS plugin with the ability to retrieve various bits of Wikipedia data and reuse them in your site.

Raison d'être

Wikipedia is the most successful collaborative knowledge base ever achieved on this planet. Just as we all contribute to it with our bits of knowledge, it is only natural that we reach for it when we need to incorporate some of its information in other web projects.

Rather than trying to write the same again and again, we reuse what has already been written and reviewed at Wikipedia. Luckily, this is made simple thanks to the excellent MediaWiki API.

gatsby-wikipedia-fetcher is a plugin that makes it easy to pick larger or smaller pieces of various Wikipedia content (page extract, illustration image, etc.) and include them in your Gatsby site.

Further, it also empowers you to keep the information alive. That is to say that any future updates provided by fellow humans around the world will automatically find their way to your site as well. This plugin makes it easy for you to constantly refresh the fetched data with the new versions provided by the Wikipedia API.

Features

This plugin is currently able to fetch the following data for each of the specified Wikipedia pages:

  • title = Wikipedia page title (after redirects, if any)
  • url = Wikipedia page URL (after redirects, if any)
  • summary = One-line summary of the Wikipedia page
  • extract = Textual extract from the Wikipedia page; this is the same page delivered on the first page of Google search results when the search matches some Wikipedia entry.
  • extractHTML = Same as "extract" but includes the HTML from Wikipedia (links, etc.)
  • firstImage = The first image (if any) from the Wikipedia page; ideal for illustrations on theme pages etc.

If you regularly need to retrieve some other pieces of data, please make a request by creating a ticket at https://github.com/Vacilando/gatsby-wikipedia-fetcher/issues.

How to install

1. Install the package with npm or yarn

npm install gatsby-wikipedia-fetcher

yarn add gatsby-wikipedia-fetcher

2. Add to plugins in your gatsby-config.js

module.exports = {
  plugins: [
    {
      resolve: `gatsby-wikipedia-fetcher`,
      options: {
        // E-mail address to be included in the Wikipedia API calls to limit the risk of being blacklisted.
        email: `user@example.com`,
      },
    },
  ],
}

3. Create a file called "WikipediaFetcherList.js" in your "components" folder (./src/components/WikipediaFetcherList.js) and paste in the following

/**
 * Supply a list of Wikipedia articles and their languages to gatsby-wikipedia-fetcher.
 */
const WikipediaFetcherList = getNodes => {
  // Array of Wikipedia article titles (redirects are automatic) or full URLs and their language codes (may be empty strings).
  var articlesLanguages = [
    { article: 'Richard P. Feynman', language: 'en' },
    { article: 'https://en.wikipedia.org/wiki/Cosmology', language: 'en' },
    { article: 'Thor Heyerdahl', language: 'en' },
  ]

  return articlesLanguages
}

module.exports.WikipediaFetcherList = WikipediaFetcherList

You can generate the array any way you wish as long as its format is maintained (array of objects consisting of article and language properties). You can use function getNodes() as a source of data. The exact implementation will depend on your data sources and structure.

In the following example we retrieve the data from node fields field_wikipedia_article and field_wikipedia_language like this:

var articlesLanguages = []
getNodes().forEach(node => {
  if (node.field_wikipedia_article) {
    articlesLanguages.push({
      article: node.field_wikipedia_article,
      language: node.field_wikipedia_language,
    })
  }
})

Demo page

https://vacilando.org/article/cosmology is an example of a page that shows both an illustration image and a textual extract from Wikipedia's page on Cosmology.

We also welcome links to sites that make use of this plugin. Send us a representative URL via https://vacilando.org/contact and we will consider it for inclusion on the documentation page.

Contributing

Every contribution is very much appreciated. You are welcome to file bugs, feature- and pull-requests at https://github.com/Vacilando/gatsby-wikipedia-fetcher/issues. If you have other questions or collaboration ideas, feel free to contact the maintainer directly at https://vacilando.org/contact.

Maintenance and development

Developed and maintained by Vacilando since Friday, December 18, 2020.

This plugin stands on the shoulders of the excellent wtf_wikipedia parser by Spencer Kelly et al.

If this plugin is helpful for you, please star it on GitHub.


This plugin has been developed with support of the Encyclopedia of World Problems and Human Potential published by the Union of International Associations (UIA).

blog, project

Wikipedia API wtf_wikipedia gatsby-wikipedia-fetcher GatsbyJS

English

Webmention logo

Comments