A complementary package to the FAOSTAT database and the Statistical Yearbook of the Food and Agricultural Organization of the United Nations.

Description

Download Data from the FAOSTAT Database of the Food and Agricultural Organization (FAO) of the United Nations. A list of functions to download statistics from FAOSTAT (database of the FAO https://www.fao.org/faostat/) and WDI (database of the World Bank https://data.worldbank.org/), and to perform some harmonization operations.

See Also

Useful links:

• https://gitlab.com/paulrougieux/faostatpackage

• Report bugs at https://gitlab.com/paulrougieux/faostatpackage/-/issues

Compute Aggregates

Description

The function takes a relational data frame and computes the aggregation based on the relation specified.

Usage

Aggregation(
  data,
  aggVar,
  weightVar = rep(NA, length(aggVar)),
  year = "Year",
  relationDF = FAOcountryProfile[, c("FAOST_CODE", "M49_FAOST_CODE")],
  aggMethod = rep("sum", length(aggVar)),
  applyRules = TRUE,
  keepUnspecified = TRUE,
  unspecifiedCode = 0,
  thresholdProp = rep(0.65, length(aggVar))
)

Arguments

Details

The length of aggVar, aggMethod, weightVar, thresholdProp must be the same.

Aggregation should not be computed if insufficient countries have reported data. This corresponds to the argument thresholdProp which specifies the percentage which of country must report data (both for the variable to be aggregated and the weighting variable).

Examples

## example.df = data.frame(FAOST_CODE = rep(c(1, 2, 3), 2),
##                        Year = rep(c(2010, 2011), c(3, 3)),
##                        value = rep(c(1, 2, 3), 2),
##                        weight = rep(c(0.3, 0.7, 1), 2))

## Lets aggregate country 1 and 2 into one country and keep country
## 3 seperate.
## relation.df = data.frame(FAOST_CODE = 1:3, NEW_CODE = c(1, 1, 2))

This function avoids double counting of China.

Description

This function should only be used when performing aggregations.

Usage

CHMT(var, data, year = "Year")

Arguments

Details

We decide to use the smaller subsets in the regional level because weighting variable may not exist for other variables for the larger subsets.

The function only work for FAOST_CODE, if the country coding system is not in FAOST_CODE then use the translateCountryCode function to translate it.

Country profile

Description

The country profile containing the codes and names of countries.

The search tree for FAOSTAT3

Description

A table containing the relationship between the domain, element, item codes for downloading data from the FAOSTAT API.

Regional profile

Description

Region profile containing the codes, names and regional classifications of countries.

Cache files read in by the API

Description

Cache files read in by the API

Usage

cache_data(name, data, reset = FALSE, environment = .FAOSTATenv)

Arguments

Construct year to year change

Description

A function for constructing year to year change

Usage

chConstruct(
  data,
  origVar,
  country = "FAOST_CODE",
  year = "Year",
  newVarName = NA,
  n = 1
)

Arguments

Value

A data frame containing the computed year to year change rate.

Change case of column names

Description

Columns from FAOSTAT frequently have parentheses and other non-alphanumeric characters. This suite of functions seeks to give control over these names for easier data analysis

Usage

change_case(
  old_names,
  new_case = c("make.names", "unsanitised", "unsanitized", "snake_case"),
  ...
)

Arguments

This function perform some check on the data

Description

The function only works for FAOST_CODE. If the country coding system is not in FAOST_CODE then use the translateCountryCode function to translate it.

Usage

check_country_overlap(
  var,
  year = "Year",
  data,
  type = c("overlap", "multiChina"),
  take = c("simpleCheck", "takeNew", "takeOld", "complete")
)

Arguments

Examples

test.df = 
   data.frame(FAOST_CODE = rep(c(51,167,199), each = 3), 
  Year = rep(c(1990:1992), 3),
  Value = c(c(3,4,4), c(2,2,2), c(1,2,NA)))
check_country_overlap(var = "Value", data = test.df, type = "overlap", take = "simpleCheck")
check_country_overlap(var = "Value", data = test.df, type = "overlap", take = "takeNew")
check_country_overlap(var = "Value", data = test.df, type = "overlap", take = "takeOld")
check_country_overlap(var = "Value", data = test.df, type = "overlap", take = "complete")

Absolute change between the year

Description

Function for generating the n-period absolute change

Usage

chgr(x, n = 1)

Arguments

Details

In order to ensure the change calculated is reliable, the following rule are applied.

1. 50% of the data must be present.

2. The length of the time series must be greater than n

Otherwise the growth will not be computed.

Value

The n-period change of the time series.

Examples

test.ts = abs(rnorm(100))
chgr(test.ts, 1)
chgr(test.ts, 3)
chgr(test.ts, 10)

Construct/Creat new variable.

Description

A function used to construct new variables from existing variables.

Usage

constructSYB(
  data,
  origVar1,
  origVar2,
  newVarName = NA,
  constructType = c("share", "growth", "change", "index"),
  grFreq = 1,
  grType = c("ls", "geo"),
  baseYear = 2000
)

Arguments

Details

Currently two types of construction are supported, either share or growth rate computation.

Share can be a share of total or share of another variable depending on whether an additional variable is supplied or not.

Value

A data frame containing both the original data frame and the processed data and also a list indicating whether the construction passed or failed.

Download bulk data from the faostat website https://www.fao.org/faostat/en/#data

Description

• get_faostat_bulk() loads the given data set code and returns a data frame.

• download_faostat_bulk() loads data from the given url and saves it to a compressed zip file.

• read_faostat_bulk() Reads the compressed .csv .zip file into a data frame. More precisely it unzips the archive. Reads the main csv file within the archive. The main file has the same name as the name of the archive. Note: the zip archive might also contain metadata files about Flags and Symbols.

In general you should load the data with the function get_faostat_bulk() and a dataset code. The other functions are lower level functions that you can use as an alternative. You can also explore the datasets and find their download URLs on the FAOSTAT website. Explore the website to find out the data you are interested in https://www.fao.org/faostat/en/#data Copy a "bulk download" url, for example they are located in the right menu on the "crops" page https://www.fao.org/faostat/en/#data/QC Note that faostat bulk files with names ending with "normalized" are in long format with a year column instead of one column for each year. The long format is preferable for data analysis and this is the format returned by the get_faostat_bulk() function.

Usage

download_faostat_bulk(url_bulk, data_folder = ".")

read_faostat_bulk(zip_file_name, encoding = "latin1", rename_element = TRUE)

get_faostat_bulk(code, data_folder = tempdir(), subset = "All Data Normalized")

read_bulk_metadata(dataset_code)

Arguments

Value

data frame of FAOSTAT data

data frame of FAOSTAT data

data frame of FAOSTAT data

Author(s)

Paul Rougieux

Examples

## Not run: 

# Create a folder to store the data
data_folder <- "data_raw"
dir.create(data_folder)

# Load crop production data
crop_production <- get_faostat_bulk(code = "QCL", data_folder = data_folder)

# Cache the file i.e. save the data frame in the serialized RDS format for faster load time later.
saveRDS(crop_production, "data_raw/crop_production_e_all_data.rds")
# Now you can load your local version of the data from the RDS file
crop_production <- readRDS("data_raw/crop_production_e_all_data.rds")


# Use the lower level functions to download zip files, 
# then read the zip files in separate function calls.
# In this example, to avoid a warning about "examples lines wider than 100 characters"
# the url is split in two parts: a common part 'url_bulk_site' and a .zip file name part.
# In practice you can enter the full url directly as the `url_bulk`  argument.
# Notice also that I have choosen to load global data in long format (normalized).
url_bulk_site <- "https://fenixservices.fao.org/faostat/static/bulkdownloads"
url_crops <- file.path(url_bulk_site, "crop_production_E_All_Data_(Normalized).zip")
url_forestry <- file.path(url_bulk_site, "Forestry_E_All_Data_(Normalized).zip")
# Download the files
download_faostat_bulk(url_bulk = url_forestry, data_folder = data_folder)
download_faostat_bulk(url_bulk = url_crops, data_folder = data_folder)

# Read the files and assign them to data frames 
crop_production <- read_faostat_bulk("data_raw/crop_production_E_All_Data_(Normalized).zip")
forestry <- read_faostat_bulk("data_raw/Forestry_E_All_Data_(Normalized).zip")
 
# Save the data frame in the serialized RDS format for fast reuse later.
saveRDS(crop_production, "data_raw/crop_production_e_all_data.rds")
saveRDS(forestry,"data_raw/forestry_e_all_data.rds")

## End(Not run)

A function to bind the different entity level.

Description

A data frame is chosen over the list is solely for the purpose of transition to ggplot2.

Usage

ebind(territory = NULL, subregion = NULL, region = NULL, world = NULL)

Arguments

A function to get country code when not available in data.

Description

This function can be useful when a dataset provided does not have a country code available.

Usage

fillCountryCode(country, data, outCode = "FAOST_CODE")

Arguments

Geometric growth rate

Description

Function for generating the n-period rolling geometric growth rate.

Usage

geogr(x, n = 1)

Arguments

Details

In order to ensure the growth rate calculated is reliable, the following rule are applied.

1. 50% of the data must be present.

2. The length of the time series must be greater than n

Otherwise the growth will not be computed.

Value

The n-period geometric growth rate of the time series.

Examples

test.ts = abs(rnorm(100))
geogr(test.ts, 1)
geogr(test.ts, 3)
geogr(test.ts, 10)

Access to World Bank WDI API

Description

A function to extract data from the World Bank API

Please refer to https://data.worldbank.org/ for any difference between the country code system. Further details on World Bank classification and methodology are available on that website.

Usage

getWDI(
  indicator = "SP.POP.TOTL",
  name = NULL,
  startDate = 1960,
  endDate = format(Sys.Date(), "%Y"),
  printURL = FALSE,
  outputFormat = "wide"
)

Arguments

Details

Sometime after 2016, there was a change in the api according to https://datahelpdesk.worldbank.org/knowledgebase/articles/889392-about-the-indicators-api-documentation "Version 2 (V2) of the Indicators API has been released and replaces V1 of the API. V1 API calls will no longer be supported. To use the V2 API, you must place v2 in the call.

Original (2011) source by Markus Gesmann: https://lamages.blogspot.it/2011/09/setting-initial-view-of-motion-chart-in.html Also available at https://www.magesblog.com/post/2011-09-25-accessing-and-plotting-world-bank-data/

Value

A data frame containing the desired World Bank Indicator

See Also

getFAO, getWDItoSYB

and the WBI package https://cran.r-project.org/package=WDI for an implementation with many more features.

Examples

## pop.df = getWDI()

World Bank Indicator Metadata

Description

A function to extract the definition and the meta data from the World Bank API

Usage

getWDImetaData(
  indicator,
  printMetaData = FALSE,
  saveMetaData = FALSE,
  saveName = "worldBankMetaData"
)

Arguments

Examples

## pop.df = getWDImetaData("SP.POP.TOTL",
##                         printMetaData = TRUE, saveMetaData = TRUE)

Access to World Bank WDI API

Description

The function downloads data from the World Bank API.

Usage

getWDItoSYB(
  indicator = "SP.POP.0014.TO.ZS",
  name = NULL,
  startDate = 1960,
  endDate = format(Sys.Date(), "%Y"),
  printURL = FALSE,
  getMetaData = TRUE,
  printMetaData = FALSE,
  saveMetaData = FALSE,
  outputFormat = c("wide", "long")
)

Arguments

Value

A list containing the following elements

data

The country level data

aggregates

The aggregates provided by the World Bank

metaData

The metaData associated with the data

results

The status of the download, whether success/failed

See Also

getWDI, getFAO

Examples

## pop.df = getWDItoSYB(name = "total_population",
##                      indicator = "SP.POP.TOTL")

REST functions to connect to FAO

Description

REST functions to connect to FAO

Usage

get_fao(
  endpoint,
  params = list(),
  language = c("en", "fr", "es", "NA"),
  base_url = "https://faostatservices.fao.org/api/v1"
)

Arguments

Construct Growth rate

Description

A function for constructing growth rate variables.

Usage

grConstruct(data, origVar, newVarName = NA, type = c("geo", "ls", "ch"), n = 1)

Arguments

Value

A data frame containing the computed growth rate.

Examples

test.df2 = data.frame(FAOST_CODE = rep(c(1, 5000), each = 5),
                       Year = rep(1990:1994, 2),
                       a = rep(1:5, 2), b = rep(1:5, 2))
grConstruct(test.df2, origVar = "a", type = "geo", n = 1)
grConstruct(test.df2, origVar = "a", type = "geo", n = 3)
grConstruct(test.df2, origVar = "a", type = "geo", n = 5)

Construct indices

Description

A function for constructing indices

Usage

indConstruct(data, origVar, newVarName = NA, baseYear = 2000)

Arguments

Value

The indice

Examples

test.df = data.frame(FAOST_CODE = rep(1, 100), Year = 1901:2000,
                      test = 1:100)
indConstruct(test.df, origVar = "test", baseYear = 1950)

Least squares growth rate

Description

Function for generating the n-period rolling least squares growth rate.

Usage

lsgr(x, n = 1)

Arguments

Details

Missing values are ommited in the regression. (Will need to check this.)

WONTFIX (Michael): There is still some error associated with this function, will need to investigate further. Will need a rule for this, when the fluctuation is large and data are sufficient then take the lsgr, otherwise the geogr.

In order to ensure the growth rate calculated is reliable, the following rule are applied.

1. 50% of the data must be present.

2. The length of the time series must be greater than n.

Otherwise the growth will not be computed.

Value

The n-period least squares growth rate of the time series

Examples

test.ts = abs(rnorm(100))
lsgr(test.ts, 1)
lsgr(test.ts, 3)
lsgr(test.ts, 10)

Function for merging data from different source.

Description

This function searches for supported country system and translate the data to allow for join.

Usage

mergeSYB(x, y, outCode = "FAOST_CODE", all = TRUE, ...)

Arguments

Details

The names of the data to be merged has to be the same as the FAOcountryProfile code name.

This function checks whether there are overlapping between the transitional countries.

Description

This function checks whether there are overlapping between the transitional countries.

Usage

overlap(old, new, var, year = "Year", data, take)

Arguments

Print labels

Description

A function to print standardised formatted labels without having messy codes in the functions.

Usage

printLab(label, span = FALSE, width = getOption("width"))

Arguments

Value

The formatted print

Examine dimensions of a dataset

Description

Lists the dimensions of a dataset including ids and labels. These can be used to query dataset dimension names and the codes therein. They can also be used to access groups, flags, units and the glossary

Usage

read_dataset_dimension(dataset_code)

read_dimension_metadata(dataset_code, dimension_code)

Arguments

Access FAOSTAT API

Description

Uses the same functionality as the web interface to pull data from the FAOSTAT API. Contains most of its parameters. Currently only works for datasets that have area, item, element and year. Values for Chinese countries are not yet deduplicated.

Usage

read_fao(
  area_codes,
  element_codes,
  item_codes,
  year_codes,
  area_format = c("M49", "FAO", "ISO2", "ISO3"),
  item_format = c("CPC", "FAO"),
  dataset = "RL",
  metadata_cols = c("codes", "units", "flags", "notes"),
  clean_format = c("make.names", "unsanitised", "unsanitized", "snake_case"),
  include_na = FALSE,
  language = c("en", "fr", "es")
)

getFAO(
  area_codes,
  element_codes,
  item_codes,
  year_codes,
  area_format = c("M49", "FAO", "ISO2", "ISO3"),
  item_format = c("CPC", "FAO"),
  dataset = "RL",
  metadata_cols = c("codes", "units", "flags", "notes"),
  clean_format = c("make.names", "unsanitised", "unsanitized", "snake_case"),
  include_na = FALSE,
  language = c("en", "fr", "es")
)

Arguments

Value

data.frame in long format (wide not yet supported). Contains attributes for the URL and parameters used.

Examples

## Not run: 

# Get data for Cropland (6620) Area (5110) in Antigua and Barbuda (8) in 2017
df = read_fao(area_codes = "8", element_codes = "5110", item_codes = "6620", 
year_codes = "2017")
# Load cropland area for a range of year
df = read_fao(area_codes = "106", element_codes = "5110", item_codes = "6620", 
year_codes = 2010:2020)

# Find which country codes are available
metadata_area <- read_dimension_metadata("RL", "area")
# Find which items are available
metadata_item <- read_dimension_metadata("RL", "item")
# Find which elements are available
metadata_element <- read_dimension_metadata("RL", "element")


## End(Not run)

A function to standardize the unit

Description

The function standardize the data to the desirable unit when the multiplier vector is supplied. For example per 1000 people is scaled to per person by supplying a multiplier of 1000.

Usage

scaleUnit(df, multiplier)

Arguments

Examples

## Create the data frame
test.df = data.frame(FAOST_CODE = 1:5, Year = 1995:1999,
  var1 = 1:5, var2 = 5:1)

## Create the named vector for scaling
multiplier = c(1, 10)
names(multiplier) = c("var1", "var2")

## Scale the data
scaleUnit(test.df, multiplier = multiplier)

Search FAOSTAT dataset_codes, datasets, elements, indicators, and items

Description

Get full list of datasets from the FAOSTAT database with the Code, dataset name and updates.

Usage

search_dataset(dataset_code, dataset_label, latest = TRUE, reset_cache = FALSE)

FAOsearch(dataset_code, dataset_label, latest = TRUE, reset_cache = FALSE)

Arguments

Value

A data.frame with the columns: code, label, date_update, note_update, release_current, state_current, year_current, release_next, state_next, year_next

Examples

## Not run: 
# Find information about all datasets
fao_metadata <- search_dataset()
# Find information about forestry datasets
search_dataset(dataset_code="FO")
# Find information about datasets whose titles contain the word "Flows"
search_dataset(dataset_label="Flows")

## End(Not run)

Construct share variable

Description

A function for constructing the share of a variable of an aggregated variable.

Usage

shConstruct(data, totVar, shareVar, newVarName = NA)

Arguments

Details

The share of a variable can be share of the World (if additional variable were not supplied) or share of another variable (per Capita if population was supplied).

Value

A data frame with the new constructed variable

Examples

## Total variables provided, scale by totVar
test.df = data.frame(FAOST_CODE = 1, Year = 1990:1994, a = 1:5, b = 1:5)
shConstruct(data = test.df, totVar = "a", shareVar = "b")

## Total variables not provided, scale by world aggregate.
test.df2 = data.frame(FAOST_CODE = rep(c(1, 5000), each = 5),
                       Year = rep(1990:1994, 2),
                       a = rep(1:5, 2), b = rep(1:5, 2))
shConstruct(data = test.df2, totVar = NA, shareVar = "b")

Function to translate multipliers

Description

This function translates number to character name or vice versa

Usage

translateUnit(vec)

Arguments

Examples

## Create numeric vector
myUnit = c(1000, 1e6, 1000, 1e9, 1e9, 1e12)

## Translate numeric to character
myUnit2 = translateUnit(myUnit)
myUnit2

## Now translate back
translateUnit(myUnit2)

A function to translate between different country coding systems

Description

The function translate any country code scheme to another if both are in the are among the types present in the FAO API. If you require other codes or conversion of country names to codes, consider the countrycodes package.

Usage

translate_countrycodes(
  data,
  from = c("FAO", "M49", "ISO2", "ISO3"),
  to = c("M49", "FAO", "ISO2", "ISO3", "name"),
  oldCode,
  reset_cache = FALSE
)

translateCountryCode(
  data,
  from = c("FAO", "M49", "ISO2", "ISO3"),
  to = c("M49", "FAO", "ISO2", "ISO3", "name"),
  oldCode,
  reset_cache = FALSE
)

Arguments