forked from paulcbauer/apis_for_social_scientists_a_review
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Chapter_Google_speech.Rmd
163 lines (101 loc) · 9.14 KB
/
Chapter_Google_speech.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
# Google Speech-to-Text API
<chauthors>Camille Landesvatter</chauthors>
<br><br>
```{r google-speech-1, include=FALSE}
knitr::opts_chunk$set(warning = FALSE, message = FALSE, cache=TRUE)
# setting cache to TRUE here allows that single API calls do not have to be run every time when knitting index or the single script, but only when something has been changed in index or single script
```
You will need to install the following packages for this chapter (run the code):
```{r google-speech-2, echo=FALSE, comment=NA}
.gen_pacman_chunk("Google_speech")
```
## Provided services/data
* *What data/service is provided by the API?*
Google's Speech-to-Text API allows you to convert audio files to text by applying powerful neural network models. Audio content can be transcribed in real time and of course (and possibly of higher relevance for social science research) from stored files.
The API currently recognizes more than 125 [languages]("https://cloud.google.com/speech-to-text/docs/languages"). It supports multiple audio formats, and audio files can either be transcribed directly (if the content does not exceed 60 seconds) or perform asynchronous requests for audio files longer than 60 seconds.
A demo of the API that allows you to record text via your microphone (or to upload an audio file) and explore the transcript can be found [here]("https://cloud.google.com/speech-to-text#section-2").
Also consider that there is a [Text-to-Speech API]("https://cloud.google.com/text-to-speech") - simply performing operations the other way around - offered by Google.
## Prerequisites
* *What are the prerequisites to access the API (authentication)? *
To access and to use the API the following steps are necessary:
- Create a [google account]("https://www.google.com/account/about/") (if you do not already have one).
- With this google account login to the [google cloud platform]("https://cloud.google.com/") and create a Google Cloud Project.
- Within this Google Cloud Project enable the Google Speech-to-text API.
- For authentication you will need to create an API key (which you additionally should restrict to the Translation API). If however, you are planning to request the Natural Language API from outside a Google Cloud environment (e.g., R) you will be required to use a private (service account) key. This can be achieved by creating a service account which in turn will allow you to download your private key as a JSON file (we show an example below).
## Simple API call
* *What does a simple API call look like?*
*Note.* For both Google's Translation API as well as Google's Natural-Language API, in this review we demonstrate an example for a simple API call via the Google Cloud Shell. In principle (and in a very similar procedure) this can be achieved for the Speech-to-Text API. However, your audio file will need some pre-processing. Audio data (such as our exemplary file in wav-format) is binary data. To make your REST request (via the Google Cloud Shell) however JSON is used. JSON eventually does not support binary data which is why you will have to transform your binary audio file into text using [Base64]("https://en.wikipedia.org/wiki/Base64") encoding (also refer to this [documentation]("https://cloud.google.com/speech-to-text/docs/base64-encoding#linux") from the Google Website for more information). If you enter audio data which is not Base64 encoded, the Google Cloud Shell will give you an error 400 stating that Base64 decoding failed for your (wav-)file. Nevertheless, in the box below we will provide the basic structure of the request.
* To activate your Cloud Shell, inspect the upper right-hand corner of your Google Cloud Platform Console and click the icon called “Activate Shell”. [Google Cloud Shell]("https://cloud.google.com/shell/#how_do_i_get_started") is a command line environment running in the cloud.
* Via the built-in Editor in Cloud Shell create a JSON file (call it for instance ‘request.json’). You can either upload your audio file directly via the Google Cloud Shell (search for the three-dotted “More” menu in the Shell and select “Upload file”), alternatively audio content can be integrated with Cloud Storage.
* The wav.file we uploaded for this example is an exemplary wav.file that comes along with the [‘googleLanguageR’ R package]("https://cran.r-project.org/web/packages/googleLanguageR/index.html").
```{r google-speech-3, echo=TRUE, eval=FALSE}
{
"audio": {
"content": "woman1_wb"
},
"config": {
"enableAutomaticPunctuation": true,
"encoding": "LINEAR16",
"languageCode": "en-US",
"model": "default"
}
}
```
* For sending your data, pass a curl command to your Cloud Shell command line where you refer (via @) to your request.json file from the previous step.
* Don’t forget to insert your individual API key (alternatively, you could define it beforehand via a variable in your environment -> see example in the API call for Google's NLP API later in this document).
```{r google-speech-4, echo=TRUE, eval=FALSE}
curl "https://speech.googleapis.com/v1p1beta1/speech:recognize?key=APIKEY" -s -X POST -H "Content-Type: application/json" --data-binary @request.json
```
## API access in R
* *How can we access the API from R (httr + other packages)?*
Example using R-Package [‘googleLanguageR’]("https://cran.r-project.org/web/packages/googleLanguageR/index.html")
In this small example we demonstrate how to..
*.. authenticate with your Google Cloud Account within R
*.. how to import an exemplary audiofile from the “GoogleLanguageR” package
*.. how to transcribe this audio file and calculate a confidence score
For the usage of further arguments, also read the `gl_speech()` [documentation]("https://cran.r-project.org/web/packages/googleLanguageR/googleLanguageR.pdf") and [this]("https://cran.r-project.org/web/packages/googleLanguageR/vignettes/speech.html") vignette.
*1. Load packages*
```{r google-speech-5, message=FALSE, warning=FALSE}
library(tidyverse)
library(googleLanguageR)
```
*Step 2: Authentication*
```{r google-speech-6_1, eval=FALSE, include=F, comment=NA}
gl_auth("api-reviews-gl_auth.json")
```
```{r google-speech-6_2, eval=FALSE, comment=NA, purl=F}
gl_auth("./your-key.json")
```
*Step 3: Analysis*
We will now get a sample source file which comes along with the `googleLanuageR` package. The transcript of this file is: “To administer medicine to animals is frequently a very difficult matter, and yet sometimes it’s necessary to do so” - which according to @Edmondson2017-pv (one of the authors of the [‘googleLanguageR’ R package]("https://cran.r-project.org/web/packages/googleLanguageR/index.html")) is a fairly difficult sentence for computers to parse.
<!-- A cache version of exemplary_audio is available in "data/google_speech-to-text_audio.RDS" -->
```{r google-speech-7, eval=F, comment=NA}
exemplary_audio <- system.file("woman1_wb.wav", package = "googleLanguageR")
```
```{r google-speech-8, echo = FALSE, message = FALSE, purl=F}
exemplary_audio <- readRDS("data/google_speech-to-text_audio.RDS")
```
We can now call the API via the function `gl_speech()`. Here you will have to specify the quantity of interest, namely the `audio_source` (this can either be a local file or a Google Cloud Storage URI) as well as the `languageCode` (language spoken in your audio file).
<!-- A cache version of exemplary_audio_analysis is available in "data/google_speech-to-text_analysis.RDS" -->
```{r google-speech-9, eval=F, comment=NA}
exemplary_audio_analysis <- gl_speech(audio_source=exemplary_audio, languageCode = "en-GB")
```
```{r google-speech-10, echo = FALSE, message = FALSE, purl=F}
exemplary_audio_analysis <- readRDS("data/google_speech-to-text_analysis.RDS")
```
The result is a list containing two dataframes: `transcript` and `timings`.
```{r google-speech-11, echo=TRUE, message=FALSE, warning=FALSE}
dimnames(exemplary_audio_analysis$transcript)
```
The `timings` dataframe stores timestamps telling us when each specific term was recognised. The `transcript` dataframe importantly provides the transcript as well as a confidence score. We can see that the transcript misses one term ("a") and indicates its confidence with a score close to 1.0.
```{r google-speech-12}
# Show transcript
exemplary_audio_analysis$transcript$transcript
```
```{r google-speech-13}
# Show confidence
exemplary_audio_analysis$transcript$confidence #0.92
```
## Social science examples
* *Are there social science research examples using the API?*
@Gavras2022-fn published research on how to use voice recording in smartphone surveys, i.e. they asked respondents to record their answer via microphone. For pre-processing and ultimately analyzing this data, they explicitly state to have used Google's Speech-to-Text API [@Gavras2022-fn, 9]. They also refer to another source, namely @Proksch2019-bw, who found the results of the API to be highly similar (e.g., r>0.9 between Google-transcribed and human-transcribed political speeches in German) to manual transcriptions carried out by humans. Importantly, collecting survey data via audio rather than text format has received increasing attention in the recent past [see e.g., @Schober2015-df; @Revilla2021-qb] as it is also part of a larger "text-as-data" movement.