Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updates to new client library and fixes broken link in README #427

Merged
merged 5 commits into from
Dec 1, 2016
Merged

Updates to new client library and fixes broken link in README #427

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
81d550f
Updates to new client library and fixes broken link in README
gguuss Nov 28, 2016
4baae60
Fixes error on test
gguuss Nov 28, 2016
a5f7fc0
Fixes type in test assertion
gguuss Nov 28, 2016
bd07872
Removes extraneous createService method, reduces line length in integ…
gguuss Nov 30, 2016
dad9f17
Merge branch 'master' into language-client
lesv Dec 1, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion language/analysis/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
This sample demonstrates the use of the [Google Cloud Natural Language API][NL-Docs]
for entity recognition.

[NL-Docs]: https://cloud.google.com/language/docs/
[NL-Docs]: https://cloud.google.com/natural-language/docs/

## Java Version

Expand Down
11 changes: 3 additions & 8 deletions language/analysis/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,9 @@ limitations under the License.
<dependencies>
<!-- [START dependencies] -->
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-language</artifactId>
<version>v1-rev1-1.22.0</version>
</dependency>
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.22.0</version>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-language</artifactId>
<version>0.7.0</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
Expand Down
126 changes: 43 additions & 83 deletions language/analysis/src/main/java/com/google/cloud/language/samples/Analyze.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,28 +16,21 @@

package com.google.cloud.language.samples;

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.language.v1.CloudNaturalLanguage;
import com.google.api.services.language.v1.CloudNaturalLanguageScopes;
import com.google.api.services.language.v1.model.AnalyzeEntitiesRequest;
import com.google.api.services.language.v1.model.AnalyzeEntitiesResponse;
import com.google.api.services.language.v1.model.AnalyzeSentimentRequest;
import com.google.api.services.language.v1.model.AnalyzeSentimentResponse;
import com.google.api.services.language.v1.model.AnalyzeSyntaxRequest;
import com.google.api.services.language.v1.model.AnalyzeSyntaxResponse;
import com.google.api.services.language.v1.model.AnnotateTextRequest;
import com.google.api.services.language.v1.model.AnnotateTextResponse;
import com.google.api.services.language.v1.model.Document;
import com.google.api.services.language.v1.model.Entity;
import com.google.api.services.language.v1.model.EntityMention;
import com.google.api.services.language.v1.model.Features;
import com.google.api.services.language.v1.model.Sentiment;
import com.google.api.services.language.v1.model.Token;
import com.google.cloud.language.spi.v1.LanguageServiceClient;

import com.google.cloud.language.v1.AnalyzeEntitiesRequest;
import com.google.cloud.language.v1.AnalyzeEntitiesResponse;
import com.google.cloud.language.v1.AnalyzeSentimentResponse;
import com.google.cloud.language.v1.AnalyzeSyntaxRequest;
import com.google.cloud.language.v1.AnalyzeSyntaxResponse;
import com.google.cloud.language.v1.Document;
import com.google.cloud.language.v1.Document.Type;
import com.google.cloud.language.v1.EncodingType;
import com.google.cloud.language.v1.Entity;
import com.google.cloud.language.v1.EntityMention;
import com.google.cloud.language.v1.Sentiment;
import com.google.cloud.language.v1.Token;
import com.google.protobuf.Descriptors;

import java.io.IOException;
import java.io.PrintStream;
Expand All @@ -49,16 +42,7 @@
* A sample application that uses the Natural Language API to perform
* entity, sentiment and syntax analysis.
*/
@SuppressWarnings("serial")
public class Analyze {
/**
* Be sure to specify the name of your application. If the application name is {@code null} or
* blank, the application will log a warning. Suggested format is "MyCompany-ProductName/1.0".
*/
private static final String APPLICATION_NAME = "Google-LanguagAPISample/1.0";

private static final int MAX_RESULTS = 4;

/**
* Detects entities,sentiment and syntax in a document using the Natural Language API.
*/
Expand All @@ -73,7 +57,7 @@ public static void main(String[] args) throws IOException, GeneralSecurityExcept
String command = args[0];
String text = args[1];

Analyze app = new Analyze(getLanguageService());
Analyze app = new Analyze(LanguageServiceClient.create());

if (command.equals("entities")) {
printEntities(System.out, app.analyzeEntities(text));
Expand All @@ -97,15 +81,17 @@ public static void printEntities(PrintStream out, List<Entity> entities) {
out.printf("%s\n", entity.getName());
out.printf("\tSalience: %.3f\n", entity.getSalience());
out.printf("\tType: %s\n", entity.getType());
if (entity.getMetadata() != null) {
for (Map.Entry<String, String> metadata : entity.getMetadata().entrySet()) {
if (entity.getMetadataMap() != null) {
for (Map.Entry<String, String> metadata : entity.getMetadataMap().entrySet()) {
out.printf("\tMetadata: %s = %s\n", metadata.getKey(), metadata.getValue());
}
}
if (entity.getMentions() != null) {
for (EntityMention mention : entity.getMentions()) {
for (Map.Entry<String, Object> mentionSetMember : mention.entrySet()) {
out.printf("\tMention: %s = %s\n", mentionSetMember.getKey(), mentionSetMember.getValue());
if (entity.getMentionsList() != null) {
for (EntityMention mention : entity.getMentionsList()) {
for (Map.Entry<Descriptors.FieldDescriptor, Object> mentionSetMember :
mention.getAllFields().entrySet()) {
out.printf("\tMention: %s = %s\n", mentionSetMember.getKey(),
mentionSetMember.getValue());
}
}
}
Expand Down Expand Up @@ -154,75 +140,49 @@ public static void printSyntax(PrintStream out, List<Token> tokens) {
}
}

/**
* Connects to the Natural Language API using Application Default Credentials.
*/
public static CloudNaturalLanguage getLanguageService()
throws IOException, GeneralSecurityException {
GoogleCredential credential =
GoogleCredential.getApplicationDefault().createScoped(CloudNaturalLanguageScopes.all());
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
return new CloudNaturalLanguage.Builder(
GoogleNetHttpTransport.newTrustedTransport(),
jsonFactory, new HttpRequestInitializer() {
@Override
public void initialize(HttpRequest request) throws IOException {
credential.initialize(request);
}
})
.setApplicationName(APPLICATION_NAME)
.build();
}

private final CloudNaturalLanguage languageApi;
private final LanguageServiceClient languageApi;

/**
* Constructs a {@link Analyze} which connects to the Cloud Natural Language API.
*/
public Analyze(CloudNaturalLanguage languageApi) {
public Analyze(LanguageServiceClient languageApi) {
this.languageApi = languageApi;
}

/**
* Gets {@link Entity}s from the string {@code text}.
*/
public List<Entity> analyzeEntities(String text) throws IOException {
AnalyzeEntitiesRequest request =
new AnalyzeEntitiesRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"))
.setEncodingType("UTF16");
CloudNaturalLanguage.Documents.AnalyzeEntities analyze =
languageApi.documents().analyzeEntities(request);

AnalyzeEntitiesResponse response = analyze.execute();
return response.getEntities();
Document doc = Document.newBuilder()
.setContent(text).setType(Type.PLAIN_TEXT).build();
AnalyzeEntitiesRequest request = AnalyzeEntitiesRequest.newBuilder()
.setDocument(doc)
.setEncodingType(EncodingType.UTF16).build();
AnalyzeEntitiesResponse response = languageApi.analyzeEntities(request);
return response.getEntitiesList();
}

/**
* Gets {@link Sentiment} from the string {@code text}.
*/
public Sentiment analyzeSentiment(String text) throws IOException {
AnalyzeSentimentRequest request =
new AnalyzeSentimentRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"));
CloudNaturalLanguage.Documents.AnalyzeSentiment analyze =
languageApi.documents().analyzeSentiment(request);

AnalyzeSentimentResponse response = analyze.execute();
Document doc = Document.newBuilder()
.setContent(text).setType(Type.PLAIN_TEXT).build();
AnalyzeSentimentResponse response = languageApi.analyzeSentiment(doc);
return response.getDocumentSentiment();
}

/**
* Gets {@link Token}s from the string {@code text}.
*/
public List<Token> analyzeSyntax(String text) throws IOException {
AnalyzeSyntaxRequest request =
new AnalyzeSyntaxRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"))
.setEncodingType("UTF16");
CloudNaturalLanguage.Documents.AnalyzeSyntax analyze =
languageApi.documents().analyzeSyntax(request);
AnalyzeSyntaxResponse response = analyze.execute();
return response.getTokens();
Document doc = Document.newBuilder()
.setContent(text).setType(Type.PLAIN_TEXT).build();
AnalyzeSyntaxRequest request = AnalyzeSyntaxRequest.newBuilder()
.setDocument(doc)
.setEncodingType(EncodingType.UTF16).build();
AnalyzeSyntaxResponse response = languageApi.analyzeSyntax(request);
return response.getTokensList();
}
}
16 changes: 9 additions & 7 deletions language/analysis/src/test/java/com/google/cloud/language/samples/AnalyzeIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@

import static com.google.common.truth.Truth.assertThat;

import com.google.api.services.language.v1.model.Entity;
import com.google.api.services.language.v1.model.Sentiment;
import com.google.api.services.language.v1.model.Token;
import com.google.cloud.language.spi.v1.LanguageServiceClient;
import com.google.cloud.language.v1.Entity;
import com.google.cloud.language.v1.PartOfSpeech.Tag;
import com.google.cloud.language.v1.Sentiment;
import com.google.cloud.language.v1.Token;

import org.junit.Before;
import org.junit.Test;
Expand All @@ -40,7 +42,7 @@ public class AnalyzeIT {
private Analyze analyzeApp;

@Before public void setup() throws Exception {
analyzeApp = new Analyze(Analyze.getLanguageService());
analyzeApp = new Analyze(LanguageServiceClient.create());
}

@Test public void analyzeEntities_withEntities_returnsLarryPage() throws Exception {
Expand Down Expand Up @@ -85,11 +87,11 @@ public class AnalyzeIT {
analyzeApp.analyzeSyntax(
"President Obama was elected for the second term");

List<String> got = token.stream().map(e -> e.getPartOfSpeech().getTag())
List<Tag> got = token.stream().map(e -> e.getPartOfSpeech().getTag())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea - Java 8. Good to see, but we don't typically use this in samples. Note - I'm Ok w/ this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack. Leaving as-is.

.collect(Collectors.toList());

// Assert
assertThat(got).containsExactly("NOUN", "NOUN", "VERB",
"VERB", "ADP", "DET", "ADJ", "NOUN").inOrder();
assertThat(got).containsExactly(Tag.NOUN, Tag.NOUN, Tag.VERB,
Tag.VERB, Tag.ADP, Tag.DET, Tag.ADJ, Tag.NOUN).inOrder();
}
}
35 changes: 24 additions & 11 deletions language/analysis/src/test/java/com/google/cloud/language/samples/AnalyzeTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@

import static com.google.common.truth.Truth.assertThat;

import com.google.api.services.language.v1.model.Entity;
import com.google.cloud.language.v1.Entity;
import com.google.cloud.language.v1.Entity.Builder;
import com.google.cloud.language.v1.Entity.Type;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;

Expand Down Expand Up @@ -67,20 +69,31 @@ public class AnalyzeTest {
// Arrange
ByteArrayOutputStream bout = new ByteArrayOutputStream();
PrintStream out = new PrintStream(bout);

// Mock natural-language entities based on actual data.
ImmutableList<Entity> entities =
ImmutableList.of(
new Entity().setName("Larry Page").setSalience(0.426f).setType("PERSON").setMetadata(
ImmutableMap.<String, String>builder()
.put("knowledge_graph_mid", "/m/0gjpq")
.put("wikipedia_url", "http://en.wikipedia.org/wiki/index.html?curid=60903")
.build()),
new Entity().setName("search engine").setSalience(0.188f).setType("CONSUMER_GOOD"),
new Entity().setName("something"));

// Act
Entity.newBuilder().setName("Larry Page")
.setSalience(0.426f)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems really specific, should you explain it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Added verbose comments calling out this is mock data based on actual data.

.setType(Type.PERSON)
.putAllMetadata(
ImmutableMap.<String, String>builder()
.put("knowledge_graph_mid", "/m/0gjpq")
.put("wikipedia_url",
"http://en.wikipedia.org/wiki/index.html?curid=60903")
.build())
.build(),
Entity.newBuilder()
.setName("search engine")
.setSalience(0.188f)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also seems very specific - does it deserve a comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Unless, you mean to comment on the inner members, e.g.

Entity.newBuilder()
    .setName("search engine") // The proper name for the mock entity
    .setSalience(0.188f) // The prominence of this entity relative to others in the document

I'm not sure how much detail is appropriate here in the mocks. For example, AFAICT the salience of entities sums up to 1. This means that if you sorted the entities by salience, descending, the most prominent entities in the target text will appear first: the actual salience values just tell you how important this particular entity is within the scope of the document.

.setType(Type.CONSUMER_GOOD)
.build(),
Entity.newBuilder().setName("something").build());

// Act on sample code with mock data.
Analyze.printEntities(out, entities);

// Assert
// Assert output from sample matches expected output.
String got = bout.toString();
assertThat(got).contains("Found 3 entities.");
assertThat(got).contains("Larry Page");
Expand Down