---
title: "Searching catalogs"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Searching catalogs}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library(httptest2)
start_vignette("search")
set_redactor(function(x) {
  gsub_response(x, "index.php/api/catalog/", "")
})
```

`search_catalog()` is the main way to find studies. This guide goes through its
arguments one at a time. The examples use the World Bank catalog unless a
section says otherwise, and the output comes from saved responses.

```{r library}
library(nadaverse)
```

Every search returns one row per study. To keep the output short, this guide
prints only a few columns of the first five rows with a small helper:

```{r brief}
brief <- function(x) head(x[, c("idno", "nation", "year_start")], 5)
```

## Keywords

`keyword` searches titles, descriptions and keywords. `ps` is the number of
studies returned per page.

```{r keyword}
res <- search_catalog("wb", keyword = "poverty", ps = 5)
brief(res)
```

## Years

`from` and `to` take four-digit years and select studies by the period their
data cover.

```{r years}
res <- search_catalog("wb", from = 2015, to = 2020, ps = 5)
brief(res)
```

## Countries

`country` takes country names or ISO3 codes, and you can give several at once.
`country_codes()` lists the valid values for a catalog.

```{r country}
res <- search_catalog("wb", country = c("ETH", "UGA"), ps = 5)
brief(res)
```

## Collections and access types

Many catalogs group their studies into collections. `collections()` lists them,
and the `repositoryid` column is the value that `collection` expects.

```{r collections}
cl <- collections("wb")
head(cl[, c("repositoryid", "title")], 5)
```

```{r collection}
res <- search_catalog("wb", collection = "LSMS", ps = 5)
brief(res)
```

Studies also differ in how you can get the data. `access_codes()` lists the
access types, and `dtype` filters by them.

```{r access}
access <- access_codes("wb")
head(access[, c("type", "title")], 5)
```

```{r dtype}
res <- search_catalog("wb", dtype = access$type[1], ps = 5)
brief(res)
```

## Sorting and pages

`sort_by` accepts `"rank"`, `"title"`, `"nation"` and `"year"`. The word
`"country"` also works and is treated as `"nation"`. `sort_order` is `"asc"` or
`"desc"`, and `page` moves through the results.

```{r sort}
res <- search_catalog(
  "wb",
  country = "ETH",
  sort_by = "year",
  sort_order = "desc",
  ps = 5
)
brief(res)
```

```{r page}
res2 <- search_catalog(
  "wb",
  country = "ETH",
  sort_by = "year",
  sort_order = "desc",
  ps = 5,
  page = 2
)
brief(res2)
```

The `created` argument filters by the date a study was added to the catalog.
Give one date, or a range, in `YYYY/MM/DD` form:

```{r created, eval = FALSE}
search_catalog("wb", created = "2023/01/01-2023/12/31")
```

## Search information

By default you get only the table of studies. With `rows = FALSE` you get the
full result list, which also says how many studies matched.

```{r info}
info <- search_catalog("wb", keyword = "poverty", ps = 5, rows = FALSE)
info$found
info$total
names(info)
```

`found` is the number of studies that match your search and `total` is the
number of studies in the whole catalog. The table of studies is still there as
`info$rows`.

`inc_iso = TRUE` adds country codes to the results:

```{r iso}
res <- search_catalog("wb", country = "ETH", inc_iso = TRUE, ps = 5)
names(res)
```

## When nothing matches

A search with no matches gives a warning and returns `NULL`, not an empty
table. Check for it before you use the result.

```{r none}
none <- suppressWarnings(search_catalog("fao", keyword = "novalidsk"))
is.null(none)
```

## Comparing catalogs

The same arguments work on every catalog, so a short loop can compare them.
Here `ps = 1` keeps each download small, because only the count is needed.

```{r compare}
codes <- c("wb", "fao", "ihsn")
found <- vapply(
  codes,
  function(code) {
    info <- suppressWarnings(
      search_catalog(code, keyword = "household", ps = 1, rows = FALSE)
    )
    if (is.null(info)) 0 else as.numeric(info$found)
  },
  numeric(1)
)
found
```

Once you have a study, `vignette("studies", package = "nadaverse")` shows how to
read its description, files and variables.

```{r teardown, include = FALSE}
end_vignette()
```