Skip to content

Commit c284d07

Browse files
authored
Merge pull request #69 from embulk/README-20250217
Update README.md
2 parents 5be683d + d792262 commit c284d07

File tree

1 file changed

+95
-20
lines changed

1 file changed

+95
-20
lines changed

README.md

+95-20
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,22 @@
1-
# Google Cloud Storage file input plugin for Embulk
2-
[![Build Status](https://travis-ci.org/embulk/embulk-input-gcs.svg?branch=master)](https://travis-ci.org/embulk/embulk-input-gcs)
1+
Google Cloud Storage file input plugin for Embulk
2+
===================================================
33

4-
## Overview
4+
Overview
5+
---------
56

6-
embulk-input-gcs v0.3.0+ requires Embulk v0.9.12+
7+
embulk-input-gcs v0.5.0+ requires Embulk v0.11.0+.
78

89
* Plugin type: **file input**
910
* Resume supported: **yes**
1011
* Cleanup supported: **yes**
1112

12-
## Usage
13+
Usage
14+
------
1315

1416
### Install plugin
1517

1618
```
17-
embulk gem install embulk-input-gcs
19+
java -jar embulk-X.Y.Z.jar install "org.embulk:embulk-input-gcs:0.5.0"
1820
```
1921

2022
### Google Service Account Settings
@@ -34,7 +36,7 @@ If you chose "private_key" or "json_key" as [auth_method](#Authentication), you
3436
### run
3537

3638
```
37-
embulk run /path/to/config.yml
39+
java -jar embulk-X.Y.Z.jar run /path/to/config.yml
3840
```
3941

4042
## Configuration
@@ -50,11 +52,16 @@ embulk run /path/to/config.yml
5052
- **json_keyfile** fullpath of json_key (string, required when auth_method is json_key)
5153
- **application_name** application name anything you like (string, optional)
5254

53-
## Example
55+
Example
56+
--------
5457

5558
```yaml
5659
in:
57-
type: gcs
60+
type:
61+
source: maven
62+
group: org.embulk
63+
name: gcs
64+
verison: "0.5.0"
5865
bucket: my-gcs-bucket
5966
path_prefix: logs/csv-
6067
auth_method: private_key #default
@@ -67,7 +74,11 @@ Example for "sample_01.csv.gz" , generated by [embulk example](https://github.co
6774
6875
```yaml
6976
in:
70-
type: gcs
77+
type:
78+
source: maven
79+
group: org.embulk
80+
name: gcs
81+
verison: "0.5.0"
7182
bucket: my-gcs-bucket
7283
path_prefix: sample_
7384
auth_method: private_key #default
@@ -96,7 +107,11 @@ To skip files using regexp:
96107
97108
```yaml
98109
in:
99-
type: gcs
110+
type:
111+
source: maven
112+
group: org.embulk
113+
name: gcs
114+
verison: "0.5.0"
100115
bucket: my-gcs-bucket
101116
path_prefix: logs/csv-
102117
# ...
@@ -107,7 +122,8 @@ in:
107122
#path_match_pattern: .csv$|.csv.gz$ # match files whose suffix is .csv or .csv.gz
108123
```
109124

110-
## Authentication
125+
Authentication
126+
---------------
111127

112128
There are three methods supported to fetch access token for the service account.
113129

@@ -121,7 +137,11 @@ You first need to create a service account (client ID), download its private key
121137

122138
```yaml
123139
in:
124-
type: gcs
140+
type:
141+
source: maven
142+
group: org.embulk
143+
name: gcs
144+
verison: "0.5.0"
125145
auth_method: private_key
126146
service_account_email: ABCXYZ123ABCXYZ123.gserviceaccount.com
127147
p12_keyfile: /path/to/p12_keyfile.p12
@@ -133,7 +153,11 @@ You first need to create a service account (client ID), download its json key an
133153
134154
```yaml
135155
in:
136-
type: gcs
156+
type:
157+
source: maven
158+
group: org.embulk
159+
name: gcs
160+
verison: "0.5.0"
137161
auth_method: json_key
138162
json_keyfile: /path/to/json_keyfile.json
139163
```
@@ -142,7 +166,11 @@ You can also embed contents of json_keyfile at config.yml.
142166
143167
```yaml
144168
in:
145-
type: gcs
169+
type:
170+
source: maven
171+
group: org.embulk
172+
name: gcs
173+
verison: "0.5.0"
146174
auth_method: json_key
147175
json_keyfile:
148176
content: |
@@ -164,24 +192,32 @@ Compute Engine VM instance, then you can configure embulk like this.
164192
165193
```yaml
166194
in:
167-
type: gcs
195+
type:
196+
source: maven
197+
group: org.embulk
198+
name: gcs
199+
verison: "0.5.0"
168200
auth_method: compute_engine
169201
```
170202
171-
## Eventually Consistency
203+
Eventually Consistency
204+
-----------------------
172205
173206
An operation listing objects is eventually consistent although getting objects is strongly consistent, see https://cloud.google.com/storage/docs/consistency.
174207
175208
`path_prefix` uses the objects list API, therefore it would miss some of objects.
176209
If you want to avoid such situations, you should use `paths` option which directly specifies object paths without the objects list API.
177210

178-
## Build
211+
For Maintainers
212+
----------------
213+
214+
### Build
179215

180216
```
181-
./gradlew gem
217+
./gradlew jar
182218
```
183219
184-
## Test
220+
### Test
185221
186222
To run unit tests, we need to configure the following environment variables.
187223
@@ -230,3 +266,42 @@ $ launchctl getenv GCP_EMAIL //try to get value.
230266

231267
Then start your applications.
232268
```
269+
270+
### Release
271+
272+
Modify `version` in `build.gradle` at a detached commit, and then tag the commit with an annotation.
273+
274+
```
275+
git checkout --detach master
276+
277+
(Edit: Remove "-SNAPSHOT" in "version" in build.gradle.)
278+
279+
git add build.gradle
280+
281+
git commit -m "Release vX.Y.Z"
282+
283+
git tag -a vX.Y.Z
284+
285+
(Edit: Write a tag annotation in the changelog format.)
286+
```
287+
288+
See [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) for the changelog format. We adopt a part of it for Git's tag annotation like below.
289+
290+
```
291+
## [X.Y.Z] - YYYY-MM-DD
292+
293+
### Added
294+
- Added a feature.
295+
296+
### Changed
297+
- Changed something.
298+
299+
### Fixed
300+
- Fixed a bug.
301+
```
302+
303+
Push the annotated tag, then. It triggers a release operation on GitHub Actions after approval.
304+
305+
```
306+
git push -u origin vX.Y.Z
307+
```

0 commit comments

Comments
 (0)