Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Search Implementation #9775

Merged
merged 48 commits into from
Mar 7, 2020
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
2c70b8e
Checkpoint
bryevdv Jan 29, 2020
eb46a03
build config updates
bryevdv Feb 14, 2020
9f4ad01
feedback
bryevdv Feb 14, 2020
e6a73ff
add credential unit tests
bryevdv Feb 14, 2020
88db804
add API to top level __init__
bryevdv Feb 14, 2020
97dbcb3
add IndexBatch unit tests
bryevdv Feb 14, 2020
fcaf9a3
add unit tests for query wrappers
bryevdv Feb 14, 2020
8833858
use standard service verbs
bryevdv Feb 18, 2020
b77ce0f
initial client tests
bryevdv Feb 18, 2020
44e4554
more tests
bryevdv Feb 18, 2020
55170cc
remove assert_called_once
bryevdv Feb 18, 2020
061f8b6
moreclient unit tests
bryevdv Feb 19, 2020
a9d0275
add async client impl
bryevdv Feb 20, 2020
48ab9fe
some docs work
bryevdv Feb 20, 2020
c7e4bc3
README edits
bryevdv Feb 24, 2020
37e0bb7
add tracing decorators
bryevdv Feb 25, 2020
2ac1276
mypy/pylint
bryevdv Feb 25, 2020
e7d0b87
add close, ctx manager methods
bryevdv Feb 27, 2020
976aca5
add index search/query examples
bryevdv Feb 27, 2020
5613cb9
update version
bryevdv Feb 27, 2020
bc0148b
add snippets to docstrings
bryevdv Feb 28, 2020
51d13e5
add a live test
bryevdv Mar 2, 2020
25a6b07
add basic crud samples
bryevdv Mar 3, 2020
9219ba1
add initial recording
bryevdv Mar 3, 2020
92f099d
add more live tests
bryevdv Mar 5, 2020
3a0717a
typos
bryevdv Mar 5, 2020
526a014
black formatting
bryevdv Mar 5, 2020
11359d3
typo
bryevdv Mar 5, 2020
3b41cd7
hide index subpackage
bryevdv Mar 5, 2020
3da6667
re-record
bryevdv Mar 6, 2020
676371f
move conftests to tests dir
bryevdv Mar 6, 2020
7a1c05f
don't use async list comp
bryevdv Mar 6, 2020
1e9124d
update tests and new swagger
bryevdv Mar 6, 2020
42d8eea
remove internal __all__
bryevdv Mar 6, 2020
bbbaacd
lint/black
bryevdv Mar 6, 2020
22858e8
IndexBatch -> IndexDocumentsBatch
bryevdv Mar 6, 2020
ec9d816
update version to 11
bryevdv Mar 6, 2020
93d2b04
formatting
bryevdv Mar 6, 2020
12d7f05
typo
bryevdv Mar 6, 2020
42e17dd
add odata function for escaping queries
bryevdv Mar 6, 2020
a52a84b
update README
bryevdv Mar 6, 2020
41c8145
update README
bryevdv Mar 6, 2020
1aa7c88
update some docs
bryevdv Mar 6, 2020
abdd724
more docs tweaks
bryevdv Mar 6, 2020
1d5449a
service_name -> endpoint
bryevdv Mar 6, 2020
9d1fa25
search_endpoint -> endpoint
bryevdv Mar 6, 2020
86f7b48
re-gen swagger
bryevdv Mar 6, 2020
63c77f9
Merge branch 'master' into bryanv/search_index_impl
bryevdv Mar 7, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions sdk/search/azure-search/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Release History

## 11.0.0b1 (Unreleased)
21 changes: 21 additions & 0 deletions sdk/search/azure-search/LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) 2017 Microsoft

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
5 changes: 5 additions & 0 deletions sdk/search/azure-search/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
include *.md
include azure/__init__.py
include LICENSE.txt
recursive-include tests *.py
recursive-include samples *.py *.md
230 changes: 230 additions & 0 deletions sdk/search/azure-search/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
# Azure Cognitive Search client library for Python

Azure Cognitive Search is a fully managed cloud search service that provides a rich search experience to custom applications.

[Source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search) |
[Package (PyPI)](https://pypi.org/project/azure-search/) |
[API reference documentation](https://aka.ms/azsdk-python-search-ref-docs) |
[Product documentation](https://docs.microsoft.com/en-us/azure/search/search-what-is-azure-search) |
[Samples](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples)

## Getting started

### Prerequisites

* Python 2.7, or 3.5 or later is required to use this package.
* You must have an [Azure subscription][azure_sub] and an existing.
[Azure Cognitive Search service][search_resource] to use this package.

If you need to create the resource, you can use the [Azure Portal][azure_portal] or [Azure CLI][azure_cli].

If you use the Azure CLI, replace `<your-resource-group-name>` and `<your-resource-name>` with your own unique names:

```PowerShell
az search service create --resource-group <your-resource-group-name> --name <your-resource-name> --sku S
```

The above creates a resource with the "Standard" pricing tier. See [choosing a pricing tier](https://docs.microsoft.com/en-us/azure/search/search-sku-tier) for more information.


### Install the package

Install the Azure Cognitive Search client library for Python with [pip](https://pypi.org/project/pip/):

```bash
pip install azure-search --pre
```

### Create an Azure Cognitive Search service

### Using an API Key

You can get the Query Keys or Admin Key from the resource information in the
[Azure Portal][azure_portal].

Alternatively, youcan se the [Azure CLI][azure_cli] snippet below to get the
Admin Key from the Cognitive Search resource.

```PowerShell
az search admin-key show --resource-group <your-resource-group-name> --service-name <your-resource-name>
```

### Authenticate the client

Interaction with this service begins with an instance of a [client](#client "search-client").
To create a client object, you will need the `search_endpoint` for your resource
bryevdv marked this conversation as resolved.
Show resolved Hide resolved
and a `credential` that allows you access:

```python
from azure.search import SearchApiKeyCredential, SearchIndexClient

credential = SearchApiKeyCredential("<api key>")

client = SearchIndexClient(endpoint="<service endpoint>",
index_name="<index name>",
credential=credential)
```

## Key concepts

### Client

The Cognitive Search client library provides a [SearchIndexClient](https://aka.ms/azsdk-python-search-searchindexclient) to perform search operations on [batches of documents](#Examples "examples").
It provides both synchronous and asynchronous operations to access a specific use of Cognitive Search indexes, such as querying, suggestions or autocompletion.


## Examples

### Retrieve a specific document from an index
Get a specific document from the index, e.f. obtain the document for hotel "23":
```python
from azure.search import SearchApiKeyCredential, SearchIndexClient
search_client = SearchIndexClient(service_endpoint, index_name, SearchApiKeyCredential(key))

result = search_client.get_document(key="23")

print("Details for hotel '23' are:")
print(" Name: {}".format(result["HotelName"]))
print(" Rating: {}".format(result["Rating"]))
print(" Category: {}".format(result["Category"]))
```
### Perform a simple text search on documents
Search the entire index or documents matching a simple search text, e.g. find
hotels with the text "spa":
```python
from azure.search import SearchApiKeyCredential, SearchIndexClient
search_client = SearchIndexClient(service_endpoint, index_name, SearchApiKeyCredential(key))

results = search_client.search(query="spa")

print("Hotels containing 'spa' in the name (or other fields):")
for result in results:
print(" Name: {} (rating {})".format(result["HotelName"], result["Rating"]))
```
### Get search suggestions
Get search suggestions for related terms, e.g. find search suggestions for
the term "coffee":
```python
from azure.search import SearchApiKeyCredential, SearchIndexClient, SuggestQuery
search_client = SearchIndexClient(service_endpoint, index_name, SearchApiKeyCredential(key))

query = SuggestQuery(search_text="coffee", suggester_name="sg")

results = search_client.suggest(query=query)

print("Search suggestions for 'coffee'")
for result in results:
hotel = search_client.get_document(key=result["HotelId"])
print(" Text: {} for Hotel: {}".format(repr(result["text"]), hotel["HotelName"]))
```
### Upload documents to an index
Add documents (or update existing ones), e.g add a new document for a new hotel:
```python
from azure.search import SearchApiKeyCredential, SearchIndexClient
search_client = SearchIndexClient(service_endpoint, index_name, SearchApiKeyCredential(key))

DOCUMENT = {
'Category': 'Hotel',
'HotelId': '1000',
'Rating': 4.0,
'Rooms': [],
'HotelName': 'Azure Inn',
}

result = search_client.upload_documents(documents=[DOCUMENT])

print("Upload of new document succeeded: {}".format(result[0].succeeded))
```

## Troubleshooting

### General

The Azure Cognitive Search client will raise exceptions defined in [Azure Core][azure_core].

### Logging

This library uses the standard [logging][python_logging] library for logging.
Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO
level.

etailed DEBUG level logging, including request/response bodies and unredacted
headers, can be enabled on a client with the `logging_enable` keyword argument:
```python
import sys
import logging
from azure.search import SearchApiKeyCredential, SearchIndexClient

# Create a logger for the 'azure' SDK
logger = logging.getLogger('azure')
logger.setLevel(logging.DEBUG)

# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)

# This client will log detailed information about its HTTP sessions, at DEBUG level
search_client = SearchIndexClient(service_endpoint, index_name, SearchApiKeyCredential(key), logging_enable=True)
```

Similarly, `logging_enable` can enable detailed logging for a single operation,
even when it isn't enabled for the client:
```python
result = search_client.search(query="spa", logging_enable=True)
```

## Next steps

### More sample code


Authenticate the client with a Azure Cognitive Search [API Key Credential](https://docs.microsoft.com/en-us/azure/search/search-security-api-keys):

[sample_authentication.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/sample_authentication.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/iasync_samples/sample_authentication_async.py))

Then for common search index operations:

* Get a document by key: [sample_get_document.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/sample_get_document.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/async_samples/sample_get_document_async.py))

* Perform a simple text query: [sample_simple_query.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/sample_simple_query.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/async_samples/sample_simple_query_async.py))

* Perform a filtered query: [sample_filter_query.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/sample_filter_query.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/async_samples/sample_filter_query_async.py))

* Get auto-completions: [sample_autocomplete.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/sample_autocomplete.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/async_samples/sample_autocomplete_async.py))

* Get search suggestions: [sample_suggestions.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/sample_suggestions.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/async_samples/sample_suggestions_async.py))

* Perform basic document updates: [sample_crud_operations.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/sample_crud_operations.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/search/azure-search/samples/async_samples/sample_crud_operations_async.py))

### Additional documentation

For more extensive documentation on Cognitive Search, see the [Azure Cognitive Search documentation](https://docs.microsoft.com/en-us/azure/search/) on docs.microsoft.com.

## Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [cla.microsoft.com][cla].

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the [Microsoft Open Source Code of Conduct][code_of_conduct]. For more information see the [Code of Conduct FAQ][coc_faq] or contact [opencode@microsoft.com][coc_contact] with any additional questions or comments.

## Related projects

* [Microsoft Azure SDK for Python](https://github.com/Azure/azure-sdk-for-python)

<!-- LINKS -->

![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-python%2Fsdk%2Fsearch%2Fazure-search%2FREADME.png)

[azure_cli]: https://docs.microsoft.com/cli/azure
[azure_core]: ../../core/azure-core/README.md
[azure_sub]: https://azure.microsoft.com/free/
[search_resource]: https://docs.microsoft.com/en-us/azure/search/search-create-service-portal
[azure_portal]: https://portal.azure.com

[python_logging]: https://docs.python.org/3.5/library/logging.html

[cla]: https://cla.microsoft.com
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
[coc_contact]: mailto:opencode@microsoft.com
6 changes: 6 additions & 0 deletions sdk/search/azure-search/azure/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# ------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------
# pylint:disable=missing-docstring
__path__ = __import__("pkgutil").extend_path(__path__, __name__) # type: str
53 changes: 53 additions & 0 deletions sdk/search/azure-search/azure/search/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# --------------------------------------------------------------------------
#
# Copyright (c) Microsoft Corporation. All rights reserved.
#
# The MIT License (MIT)
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the ""Software""), to
# deal in the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
# IN THE SOFTWARE.
#
# --------------------------------------------------------------------------

from ._version import VERSION

__version__ = VERSION

from ._index import (
AutocompleteQuery,
IndexAction,
IndexDocumentsBatch,
IndexingResult,
SearchApiKeyCredential,
SearchIndexClient,
SearchQuery,
SuggestQuery,
odata,
)

__all__ = (
"AutocompleteQuery",
"IndexAction",
"IndexDocumentsBatch",
"IndexingResult",
"SearchApiKeyCredential",
"SearchIndexClient",
"SearchQuery",
"SuggestQuery",
"odata",
)
19 changes: 19 additions & 0 deletions sdk/search/azure-search/azure/search/_index/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# ------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------
from ._credential import SearchApiKeyCredential # pylint: disable=unused-import
from ._index_documents_batch import IndexDocumentsBatch # pylint: disable=unused-import
from ._search_index_client import ( # pylint: disable=unused-import
odata,
SearchIndexClient,
)
from ._queries import ( # pylint: disable=unused-import
AutocompleteQuery,
SearchQuery,
SuggestQuery,
)
from ._generated.models import ( # pylint: disable=unused-import
IndexAction,
IndexingResult,
)
41 changes: 41 additions & 0 deletions sdk/search/azure-search/azure/search/_index/_credential.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------

import six


class SearchApiKeyCredential(object):
"""Credential type used for authenticating a SearchIndexClient
with an admin or query API key.

:param api_key: An admin or query key for your Azure Search index.
:type api_key: str

"""

def __init__(self, api_key):
# type: (str) -> None
if not isinstance(api_key, six.string_types):
raise TypeError("api_key must be a string.")
self._api_key = api_key # type: str

@property
def api_key(self):
"""The value of the configured API key.

:rtype: str
"""
# type () -> str
return self._api_key

def update_key(self, key):
"""Update the API key.
This can be used when you've regenerated your service API key and want
to update long-lived clients.

:param str key: The API key to your Azure search account.
"""
self._api_key = key
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

from ._search_index_client import SearchIndexClient
__all__ = ['SearchIndexClient']
Loading