Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Narrative for NLP Solution #151

Merged
merged 3 commits into from
May 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion document-ai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ These are some examples of the use cases you can build on top of this architectu
The main components that we would be setting up are (to learn more about these products, click on the hyperlinks)

* [Cloud Storage (GCS) bucket](https://cloud.google.com/storage/) : for storing extracted data that must undergo some kind of transformation.
* [Big Query](https://cloud.google.com/bigquery) : Serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.
* [BigQuery](https://cloud.google.com/bigquery) : Serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.
* [Document AI ](https://cloud.google.com/document-ai) : Extract structured data from documents and analyze, search and store this data.
* [Cloud Function](https://cloud.google.com/functions) : Run your code in the cloud with no servers or containers to manage with our scalable, pay-as-you-go functions as a service (FaaS) product.

Expand Down
66 changes: 66 additions & 0 deletions nlp-sentiment-analysis/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
[![banner](../banner.png)](https://cloud.google.com/?utm_source=github&utm_medium=referral&utm_campaign=GCP&utm_content=packages_repository_banner)

# Perform Sentiment Analysis on your Text Data using Google Cloud

## Introduction
This architecture uses click-to-deploy to orchestrate a seamless sentiment analysis pipeline by leveraging Google Cloud's robust suite of services. It empowers users to glean valuable insights from text data through automated processes and intuitive visualizations.

At the heart of the pipeline is Google Cloud's Natural Language API which fers pre-trained models to developers for analyzing text and extracting insights like sentiment, entities, and syntax. Cloud Storage functions as a central repository for both input text files and analysis results. Upon the arrival of a new file, Cloud Functions is triggered, acting as the system's orchestrator. Cloud Functions subsequently invokes the NLP API. The API's output, in the form of structured JSON data, is then housed back to Cloud Storage.

To facilitate in-depth exploration of the analysis results, the architecture integrates seamlessly with BigQuery, Google Cloud's fully managed data warehouse. Users can leverage BigQuery's powerful querying capabilities to uncover patterns and trends within the sentiment data. Furthermore, Looker Studio offers an intuitive visual interface for creating interactive dashboards and reports, allowing users to communicate their findings effectively.

In essence, this architecture delivers a cohesive solution for sentiment analysis, streamlining the entire workflow from data ingestion to insightful visualization. By harnessing the power of Google Cloud's managed services, users can unlock the hidden value within their text data without the need for extensive infrastructure management.

## Use cases
* __Customer Feedback Analysis__: Analyze customer reviews, surveys, social media comments, and other feedback channels. By understanding sentiment trends, businesses can identify areas for improvement, gauge customer satisfaction, and make data-driven decisions to enhance their products or services.
* __Brand Reputation Monitoring__ : Monitor online conversations, news articles, and social media mentions related to their brand. Sentiment analysis helps track public perception, identify potential PR crises, and measure the effectiveness of marketing campaigns.
* __Market Research__ : Analyze large volumes of unstructured data, such as customer reviews, social media discussions, or industry reports. Sentiment analysis helps identify emerging trends, competitor strategies, and customer preferences, providing valuable insights for market research and product development.
* __Social Media Monitoring__ : Leverage sentiment analysis to track social media conversations around specific topics, events, or products. This helps identify trending topics, gauge public opinion, and uncover potential opportunities or risks.
* __Employee Engagement Analysis__ : Analyzing employee feedback through surveys, emails, or internal communication channels, sentiment analysis can help organizations gauge employee morale, identify areas for improvement, and foster a positive work environment.


## Architecture

<p align="center"><img src="/assets/architecture.png"></p>

The main components that we would be setting up are (to learn more about these products, click on the hyperlinks)

* [Cloud Storage](https://cloud.google.com/storage) : for storing extracted data that must undergo some kind of transformation.
* [Cloud Functions](https://cloud.google.com/functions) : Run your code in the cloud with no servers or containers to manage with our scalable, pay-as-you-go functions as a service (FaaS) product.
* [Natural Language API](https://cloud.google.com/natural-language) : Derive insights from unstructured text using Google machine learning.
* [BigQuery](https://cloud.google.com/bigquery) : Serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.


## Costs

Pricing Estimates - We have created a sample estimate based on some usage we see from new startups looking to scale. This estimate would give you an idea of how much this deployment would essentially cost per month at this scale and you extend it to the scale you further prefer. Here's the [link](https://cloud.google.com/products/calculator/#id=17907c9a-d2d9-43c2-901d-028fd324a1b2).


## Deploy the architecture

:clock1:


1. Click on Open in Google Cloud Shell button below.

<a href="https://ssh.cloud.google.com/cloudshell/editor?shellonly=true&cloudshell_git_repo=https://github.com/GoogleCloudPlatform/click-to-deploy-solutions&cloudshell_workspace=nlp-sentiment-analysis&cloudshell_open_in_editor=terraform/terraform.tfvars" target="_new">
<img alt="Open in Cloud Shell" src="https://gstatic.com/cloudssh/images/open-btn.svg">
</a>

2. Run the prerequisites script to enable APIs and set Cloud Build permissions.
```
sh prereq.sh
```

3. Run the Cloud Build Job
```
gcloud builds submit . --config cloudbuild.yaml
```

## Testing the architecture
Execute the command below on Cloud Shell to delete the resources.
```
gcloud builds submit . --config cloudbuild_destroy.yaml
```

This is not an official Google product.
Binary file modified nlp-sentiment-analysis/assets/architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed nlp-sentiment-analysis/assets/bq_results.png
Binary file not shown.
Binary file removed nlp-sentiment-analysis/assets/form.pdf
Binary file not shown.
Binary file removed nlp-sentiment-analysis/assets/gcs_results.png
Binary file not shown.
8 changes: 4 additions & 4 deletions nlp-sentiment-analysis/tutorial.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Extract data from your documents using Generative AI on Google Cloud
# Perform Sentiment Analysis on your Text Data using Google Cloud

## Let's get started

Expand Down Expand Up @@ -57,7 +57,7 @@ It happens because the Eventarc permissions take some time to propagate. First,

## Result

At this point you should have successfully deployed the foundations for a Three Tier Web Application!.
At this point you should have successfully deployed sentiment analysis for text in Google Cloud.

This process may take a while to deploy, please do not close the window when deploying.

Expand All @@ -67,10 +67,10 @@ Next we are going to test the architecture and finally clean up your environment
Once you deployed the solution successfully, upload the form.pdf to the input bucket using either Cloud Console or gsutil.

```bash
gsutil cp assets/form.pdf gs://<YOUR PROJECT NAME>-doc-ai-form-input
gsutil cp assets/form.pdf gs://<YOUR PROJECT NAME>-sentiment-analysis_input
```

Then, check the parsed results in the output bucket in text (OCR) and json (Key=value) formats
Then, check the parsed results in the output bucket in json (Key=value) format.

Finally, check the json results on BigQuery

Expand Down
Loading