GitLab

rewrite THE to not require google sheet for JSON URLs, include 2022 ranking

See merge request !10

rewrite THE to not require google sheet for JSON URLs, include 2022 ranking

See merge request monbusanalytics/unirank!1

Rewrite ARWU global and subject scrapers to use 2021 website structure

See merge request !9

Fix times scraping to use fromJSON

See merge request !8

Error in ur_scrape_times that appeared to
be due to data not converting to correct
format from GET request. Updated directly use
fromJSON instead to pull in JSON data.

Update QS data format

See merge request !7

QS rankings made minor changes to data format
which led to columns not being named correctly
in output. Update to fix column naming for new
QS data format.

Fix arwu subject

Closes #12

See merge request !6

Replace as_data_frame with as_tibble in ur_scrape_qs.
as_data_frame is now deprecated.

Minor updates to documentation. In particular,
fix examples in time scraping documentation to run
correctly. JSON URLs included in documentation
for times were out of date, updated to new URLs.

Fixes #12. ARWU subject ranking scraper had been
throwing error when naming columns due to ARWU
changing number of columns and metric names. Updated
to add function to check which year is being pulled
and returning appropriate column names for table.

Fix ARWU 2019 scraping

Closes #11

See merge request !5

Fixes #11. Errors were occuring when scraping
overall ARWU 2019 data. This was due to
ARWU website using 'null' instead of blanks for
N&S scores causing columns to be read in as
characters. Updated to replace 'null' with
NA and reparse columns.

Fixes #9. Errors were occuring when QS scraping
for some subjects if some indicators showed as '-'
on QS website. Error occured when attempting to
select columns with blank indictors as did not exist in
data frame. Update column selection approach so that
do not get errors if column does not exist."

Updated readme to mention that Times, ARWU and QS are included

Update documentation for QS scraping functions
and data reflecting change to getting JSON
URLs from Google Sheets document.

Change years in data/ur_data_qs.rda to reflect
years of data available in Google Sheets reference
document. This meant removal of some years that
had previously been included in data set that
are not listed in Google Sheets file.

Previously in QS scrape ran type conversion
on data frame to detect data types. As part of this
process also identified and converted values to NA.
This led to errors when pulling data/ur_data_qs.rda
data though as some years type conversion returned different
types preventing building of single data frame. Updated
to remove type conversion and instead just identify NA values.

Update tests for QS scrape to include
test for new rankings argument and tests for
ur_show_available_qs_data()

Previously URLs for QS scrape were hardcoded
into ur_scrape_qs. Update to read from external
Google Sheets document instead as per Times scrape.
Additionally add function ur_show_available_qs_data
to show available URLs.

Additional update to bug fixed in f0ec54.
f0ec54 fixed bug caused in QS scrape function
due to use of .data$ in unite and gather functions.
Replacing with tidy style variable names
caused warnings of global variables when running
package check. Replaced tidy style variables names
with strings to prevent warning.

Previously QS scrape parsed specific columns
converting character type into numeric. This
caused errors if pulling in data that did not contain
those specific columns (e.g., subject rankings).
Update to use type convert on full data frame instead
of parsing specific columns."

Errors were occuring in QS scraper due to use
of .data$ for new variable names in unite
and gather functions. Update QS scraper function
to remove unnecessary .data$ references.

Added mention of missing values = 9999 in qs data and function definitions

Corrected documentation for ur_data_qs

Converts to numeric, converts 9999 to missing.