---
title: "Building and exporting a reference set"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Building and exporting a reference set}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(collapse = FALSE, comment = "")
# Console colour carries no meaning on a rendered page. pkgdown turns it on for
# its own build, and the escape sequences then reach the reader as literal text,
# so colour is switched off here for a plain vignette render and a site build
# alike. The fixed width keeps tibbles inside the documentation column.
options(cli.num_colors = 1, cli.hyperlink = FALSE, crayon.enabled = FALSE,
        width = 80)
# Print data frames and tibbles as formatted tables.
local({
  kp <- function(x, ...) {
    if (any(vapply(x, is.list, logical(1)))) return(knitr::normal_print(x))
    knitr::knit_print(knitr::kable(x))
  }
  for (cls in c("data.frame", "tbl_df", "tbl")) {
    registerS3method("knit_print", cls, kp, envir = asNamespace("knitr"))
  }
})
```

```{r setup}
library(scopusflow)
```

A retrieval becomes useful once it leaves the package, as a reading list in a
reference manager or as input to a science-mapping tool. This article covers that
export end of the workflow. Every step runs without an API key, on the bundled
`example_records`, a corpus of 138 real journal articles that stands in for a
harvest of your own because Scopus records may not be redistributed. The
article `vignette("scopusflow")` describes where it comes from.

## Take stock first

Before exporting anything, it helps to see what the set contains. `summary()`
reports the size, the span of years, the number of distinct sources, the citation
spread and the most-cited record.

```{r}
summary(example_records)
```

Eleven of those 138 records arrived without a DOI, which is why the summary
counts 127 with one. That is ordinary in a real harvest, and the sections below
show how each export handles it.

A record set is an ordinary tibble underneath, so it drops straight into
tidyverse or base workflows. `as_tibble()` and `as.data.frame()` make that
explicit when a downstream tool expects a plain frame.

```{r}
head(tibble::as_tibble(example_records))
as.data.frame(example_records)[1:3, c("title", "year")]
```

## A clean, deduplicated DOI list

Reference managers such as Zotero import most reliably from DOIs.
`scopus_extract_dois()` pulls them out, normalises them and removes duplicates,
so the same article imports once even when its DOI was stored with a resolver
prefix or in a different case.

```{r}
dois <- scopus_extract_dois(example_records)
length(dois)
head(dois, 6)
```

The same cleaning applies to a plain vector of DOIs from any source, so the
first two entries below collapse to one because comparison ignores case and
resolver prefixes, while `dedupe = FALSE` keeps every occurrence, for instance
to count how often a DOI recurs across retrievals.

```{r}
scopus_extract_dois(c("https://doi.org/10.1/A", "doi: 10.1/a", "10.2/B"))
scopus_extract_dois(c("https://doi.org/10.1/A", "doi: 10.1/a", "10.2/B"),
                    dedupe = FALSE)
```

The list can be written to a single-column CSV at a path you choose. Nothing is
written unless a path is given. Here are the first few lines of the file, which
holds a header and one DOI per line.

```{r}
out <- file.path(tempdir(), "reference-set.csv")
scopus_extract_dois(example_records, file = out)
writeLines(head(readLines(out), 6))
```

## Into a reference manager

A DOI list is enough for an import-by-identifier, but a full record carries more.
`as_ris()` and `as_bibtex()` render the set in the two interchange formats that
reference managers read, so a search moves straight into Zotero, EndNote or a
LaTeX bibliography. Each record becomes one entry, with its authors split out.

```{r}
cat(substr(as_ris(example_records), 1, 470))
```

The first two entries of 138 are shown. The second has no DOI, so its `DO` line
is simply absent, with no empty field written in its place, and the entry still
imports on its title, author and year.

Pass a `file` to write the whole set. Nothing is written without one.

```{r}
bib <- file.path(tempdir(), "reference-set.bib")
as_bibtex(example_records, file = bib)
```

## Handing off to science mapping

`as_bibliometrix()` re-maps the records to the tagged column layout that the
[bibliometrix](https://www.bibliometrix.org) package and the wider ISI convention
expect.

```{r}
m <- as_bibliometrix(example_records)
head(m[, c("AU", "TI", "PY", "SO", "TC", "DB")])
```

From there the usual bibliometrix entry points apply. This step needs that
package, so it is shown but not run.

```{r eval = FALSE}
if (requireNamespace("bibliometrix", quietly = TRUE)) {
  results <- bibliometrix::biblioAnalysis(m)
  summary(results, k = 10)
}
```

As noted on the `as_bibliometrix()` help page, this reconstructs the core
descriptive fields only. Analyses that need full affiliations or cited references
still call for a complete 'Scopus' CSV or BibTeX export read with
`bibliometrix::convert2df()`.

## Saving the working set

To pick the work up in a later session, save the records and read them back. The
`.rds` form preserves the types and class exactly, while `.csv` is portable plain
text.

```{r}
path <- file.path(tempdir(), "records.rds")
write_scopus_records(example_records, path)
identical(read_scopus_records(path), example_records)
```