Adding auto doc links for READMEs.

appengine/bigquery/README.md

@@ -2,6 +2,15 @@
 
 This sample demonstrates [authenticating to BigQuery in App Engine using OAuth2](https://cloud.google.com/bigquery/authentication).
 
+<!-- auto-doc-link -->
+These samples are used on the following documentation pages:
+
+>
+* https://cloud.google.com/bigquery/authentication
+* https://cloud.google.com/monitoring/api/authentication
+
+<!-- end-auto-doc-link -->
+
 Refer to the [App Engine Samples README](../README.md) for information on how to run and deploy this sample.
 
 ## Setup

appengine/blobstore/README.md

@@ -0,0 +1,8 @@
+# App Engine Blobstore Sample
+
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/appengine/docs/python/blobstore/index
+
+<!-- end-auto-doc-link -->

appengine/images/README.md

@@ -3,4 +3,11 @@
 This is a sample app for Google App Engine that demonstrates the [Images Python
 API](https://cloud.google.com/appengine/docs/python/images/usingimages).
 
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/appengine/docs/python/images/usingimages
+
+<!-- end-auto-doc-link -->
+
 Refer to the [App Engine Samples README](../README.md) for information on how to run and deploy this sample.

appengine/localtesting/README.md

@@ -0,0 +1,10 @@
+# App Engine Local Testing Samples
+
+These samples show how to do automated testing of App Engine applications.
+
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/appengine/docs/python/tools/localunittesting
+
+<!-- end-auto-doc-link -->

appengine/memcache/guestbook/README.md

@@ -1,5 +1,12 @@
 # Memcache Guestbook Sample
 
-This is a sample app for Google App Engine that demonstrates the [Memcache Python API](https://cloud.google.com/appengine/docs/python/memcache/usingmemcache).
+This is a sample app for Google App Engine that demonstrates the Memcache Python API.
+
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/appengine/docs/python/memcache/usingmemcache
+
+<!-- end-auto-doc-link -->
 
 Refer to the [App Engine Samples README](../../README.md) for information on how to run and deploy this sample.

appengine/multitenancy/README.md

@@ -2,4 +2,11 @@
 
 This sample demonstrates how to use Google App Engine's [Namespace Manager API](https://cloud.google.com/appengine/docs/python/multitenancy/multitenancy).
 
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/appengine/docs/python/multitenancy/namespaces
+
+<!-- end-auto-doc-link -->
+
 Refer to the [App Engine Samples README](../README.md) for information on how to run and deploy this sample.

appengine/ndb/overview/README.md

@@ -2,4 +2,11 @@
 
 This is a sample app for Google App Engine that demonstrates the [Datastore NDB Python API](https://cloud.google.com/appengine/docs/python/ndb/).
 
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/appengine/docs/python/ndb/
+
+<!-- end-auto-doc-link -->
+
 Refer to the [App Engine Samples README](../../README.md) for information on how to run and deploy this sample.

appengine/ndb/transactions/README.md

@@ -1,7 +1,14 @@
 ## App Engine Datastore NDB Transactions Sample
 
-This is a sample app for Google App Engine that exercises the [NDB Transactions Python API](https://cloud.google.com/appengine/docs/python/ndb/transactions)
+This is a sample app for Google App Engine that demonstrates the [NDB Transactions Python API](https://cloud.google.com/appengine/docs/python/ndb/transactions)
 
 This app presents a list of notes. After you submit a note with a particular title, you may not change that note or submit a new note with the same title. There are multiple note pages available.
 
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/appengine/docs/python/ndb/transactions
+
+<!-- end-auto-doc-link -->
+
 Refer to the [App Engine Samples README](../../README.md) for information on how to run and deploy this sample.

bigquery/api/README.md

@@ -0,0 +1,16 @@
+# BigQuery API Samples
+
+<!-- auto-doc-link -->
+These samples are used on the following documentation pages:
+
+>
+* https://cloud.google.com/bigquery/exporting-data-from-bigquery
+* https://cloud.google.com/bigquery/authentication
+* https://cloud.google.com/bigquery/bigquery-api-quickstart
+* https://cloud.google.com/bigquery/docs/managing_jobs_datasets_projects
+* https://cloud.google.com/bigquery/streaming-data-into-bigquery
+* https://cloud.google.com/bigquery/docs/data
+* https://cloud.google.com/bigquery/querying-data
+* https://cloud.google.com/bigquery/loading-data-into-bigquery
+
+<!-- end-auto-doc-link -->

compute/api/README.md

@@ -0,0 +1,13 @@
+# Compute Engine API Samples
+
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/compute/docs/tutorials/python-guide
+
+<!-- end-auto-doc-link -->
+
+
+
+
+

monitoring/api/README.md

@@ -0,0 +1,10 @@
+# Cloud Monitoring API Samples
+
+<!-- auto-doc-link -->
+These samples are used on the following documentation pages:
+
+>
+* https://cloud.google.com/monitoring/demos/
+* https://cloud.google.com/monitoring/api/authentication
+
+<!-- end-auto-doc-link -->

scripts/auto_link_to_docs.py

@@ -0,0 +1,103 @@
+"""
+Process docs-links.json and updates all READMEs and replaces
+
+<!-- auto-doc-link --><!-- end-auto-doc-link -->
+
+With a generated list of documentation backlinks
+"""
+
+from collections import defaultdict
+import json
+import os
+import re
+
+
+REPO_ROOT = os.path.abspath(os.path.join(
+    os.path.dirname(__file__),
+    '..'))
+DOC_SITE_ROOT = 'https://cloud.google.com'
+AUTO_DOC_LINK_EXP = re.compile(
+    r'<!-- auto-doc-link -->.*?<!-- end-auto-doc-link -->\n',
+    re.DOTALL)
+
+
+def invert_docs_link_map(docs_links):
+    files_to_docs = defaultdict(list)
+    for doc, files in docs_links.iteritems():
+        for file in files:
+            files_to_docs[file].append(doc)
+            files_to_docs[file] = list(set(files_to_docs[file]))
+
+    return files_to_docs
+
+
+def collect_docs_for_readmes(files_to_docs):
+    readmes_to_docs = defaultdict(list)
+    for file, docs in files_to_docs.iteritems():
+        readme = get_readme_path(file)
+        readmes_to_docs[readme].extend(docs)
+        readmes_to_docs[readme] = list(set(readmes_to_docs[readme]))
+    return readmes_to_docs
+
+
+def linkify(docs):
+    return [DOC_SITE_ROOT + x for x in docs]
+
+
+def replace_contents(file_path, regex, new_content):
+    with open(file_path, 'r+') as f:
+        content = f.read()
+        content = regex.sub(new_content, content)
+        f.seek(0)
+        f.write(content)
+
+
+def get_readme_path(file_path):
+    dir = os.path.dirname(file_path)
+    readme = os.path.join(
+        REPO_ROOT, dir, 'README.md')
+    return readme
+
+
+def generate_doc_link_statement(docs):
+    links = linkify(docs)
+    if len(links) == 1:
+        return """<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> {}
+
+<!-- end-auto-doc-link -->
+""".format(links.pop())
+    else:
+        return """<!-- auto-doc-link -->
+These samples are used on the following documentation pages:
+
+>
+{}
+
+<!-- end-auto-doc-link -->
+""".format('\n'.join(['* {}'.format(x) for x in links]))
+
+
+def update_readme(readme_path, docs):
+    if not os.path.exists(readme_path):
+        print('{} doesn\'t exist'.format(readme_path))
+        return
+    replace_contents(
+        readme_path,
+        AUTO_DOC_LINK_EXP,
+        generate_doc_link_statement(docs))
+    print('Updated {}'.format(readme_path))
+
+
+def main():
+    docs_links = json.load(open('docs-links.json', 'r'))
+    files_to_docs = invert_docs_link_map(docs_links)
+    readmes_to_docs = collect_docs_for_readmes(files_to_docs)
+
+    for readme, docs in readmes_to_docs.iteritems():
+        update_readme(readme, docs)
+
+if __name__ == '__main__':
+    main()

scripts/docs-links.json

@@ -0,0 +1,87 @@
+{
+  "/storage/docs/json_api/v1/json-api-python-samples": [
+    "storage/api/list_objects.py"
+  ],
+  "/bigquery/exporting-data-from-bigquery": [
+    "bigquery/api/export_data_to_cloud_storage.py"
+  ],
+  "/bigquery/authentication": [
+    "appengine/bigquery/main.py",
+    "bigquery/api/getting_started.py"
+  ],
+  "/compute/docs/tutorials/python-guide": [
+    "compute/api/create_instance.py",
+    "compute/api/startup-script.sh"
+  ],
+  "/appengine/docs/python/memcache/usingmemcache": [
+    "appengine/memcache/guestbook/main.py"
+  ],
+  "/appengine/docs/python/tools/localunittesting": [
+    "appengine/localtesting/test_mail.py",
+    "appengine/localtesting/runner.py",
+    "appengine/localtesting/test_login.py",
+    "appengine/localtesting/test_datastore.py",
+    "appengine/localtesting/test_task_queue.py",
+    "appengine/localtesting/test_env_vars.py",
+    "appengine/localtesting/queue.yaml"
+  ],
+  "/bigquery/streaming-data-into-bigquery": [
+    "bigquery/api/streaming.py"
+  ],
+  "/storage/docs/authentication": [
+    "storage/api/list_objects.py"
+  ],
+  "/monitoring/api/authentication": [
+    "appengine/bigquery/main.py",
+    "appengine/bigquery/app.yaml",
+    "monitoring/api/auth.py"
+  ],
+  "/storage/transfer/create-transfer": [
+    "storage/transfer_service/nearline_request.py",
+    "storage/transfer_service/aws_request.py",
+    "storage/transfer_service/transfer_check.py"
+  ],
+  "/monitoring/demos/": [
+    "monitoring/api/auth.py"
+  ],
+  "/appengine/docs/python/blobstore/index": [
+    "appengine/blobstore/main.py"
+  ],
+  "/bigquery/docs/data": [
+    "bigquery/api/sync_query.py"
+  ],
+  "/docs/authentication": [
+    "storage/api/list_objects.py"
+  ],
+  "/storage/transfer/create-client": [
+    "storage/transfer_service/create_client.py"
+  ],
+  "/appengine/docs/python/images/usingimages": [
+    "appengine/images/main.py"
+  ],
+  "/bigquery/docs/managing_jobs_datasets_projects": [
+    "bigquery/api/list_datasets_projects.py"
+  ],
+  "/appengine/docs/python/ndb/transactions": [
+    "appengine/ndb/transactions/main.py"
+  ],
+  "/appengine/docs/python/ndb/": [
+    "appengine/ndb/overview/main.py"
+  ],
+  "/appengine/docs/python/multitenancy/namespaces": [
+    "appengine/multitenancy/memcache.py",
+    "appengine/multitenancy/datastore.py",
+    "appengine/multitenancy/taskqueue.py"
+  ],
+  "/bigquery/loading-data-into-bigquery": [
+    "bigquery/api/load_data_from_csv.py",
+    "bigquery/api/load_data_by_post.py"
+  ],
+  "/bigquery/querying-data": [
+    "bigquery/api/async_query.py",
+    "bigquery/api/sync_query.py"
+  ],
+  "/bigquery/bigquery-api-quickstart": [
+    "bigquery/api/getting_started.py"
+  ]
+}

storage/api/README.md

@@ -0,0 +1,11 @@
+# Cloud Storage API Samples
+
+<!-- auto-doc-link -->
+These samples are used on the following documentation pages:
+
+>
+* https://cloud.google.com/storage/docs/json_api/v1/json-api-python-samples
+* https://cloud.google.com/storage/docs/authentication
+* https://cloud.google.com/docs/authentication
+
+<!-- end-auto-doc-link -->

storage/transfer_service/README.md

@@ -2,6 +2,15 @@
 
 This app creates two types of transfers using the Transfer Service tool.
 
+<!-- auto-doc-link -->
+These samples are used on the following documentation pages:
+
+>
+* https://cloud.google.com/storage/transfer/create-transfer
+* https://cloud.google.com/storage/transfer/create-client
+
+<!-- end-auto-doc-link -->
+
 ## Prerequisites
 
 1. Set up a project on Google Developers Console.