Skip to content

Commit

Permalink
📚 docs: Enable ERDs for 10 top sources. (#45085)
Browse files Browse the repository at this point in the history
  • Loading branch information
letiescanciano authored Sep 9, 2024
1 parent a8e3837 commit f216a60
Show file tree
Hide file tree
Showing 14 changed files with 71 additions and 0 deletions.
6 changes: 6 additions & 0 deletions docs/integrations/sources/amazon-seller-partner.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,12 @@ The Amazon Seller Partner source connector supports the following [sync modes](h
- [Vendor Traffic Report](https://developer-docs.amazon.com/sp-api/docs/report-type-values-analytics#vendor-retail-analytics-reports) \(only available in OSS, incremental\)
<!-- /env:oss -->

<HideInUI>

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>
</HideInUI>

## Report options

Report options can be assigned on a per-stream basis that alter the behavior when generating a report.
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/bing-ads.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,9 @@ See more info about user-defined pk [here](https://docs.airbyte.com/understandin

:::

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

### Custom Reports

You can build your own report by providing:
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/facebook-marketing.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,9 @@ For more information, see the [Facebook Insights API documentation.](https://dev
Please be aware that some fields, such as `conversions` and `conversion_values`, may not be directly accessible when querying Ad Insights. For comprehensive access to all available fields, we recommend using a Custom Insight and specifying the necessary **breakdowns**.
::: -->

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

## Facebook Marketing Attribution Reporting

The Facebook Marketing connector uses the `lookback_window` parameter to repeatedly read data from the last `<lookback_window>` days during an Incremental sync. This means some data will be synced twice (or possibly more often) despite the cursor value being up to date, in order to capture updated ads conversion data from Facebook. You can change this date window by adjusting the `lookback_window` parameter when setting up the source, up to a maximum of 28 days. Smaller values will result in fewer duplicates, while larger values provide more accurate results. For a deeper understanding of the purpose and role of the attribution window, refer to this [Meta article](https://www.facebook.com/business/help/458681590974355?id=768381033531365).
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/github.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,9 @@ This connector outputs the following incremental streams:
- [WorkflowRuns](https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2022-11-28#list-workflow-runs-for-a-repository)
- [Workflows](https://docs.github.com/en/rest/actions/workflows?apiVersion=2022-11-28#list-repository-workflows)

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

### Notes

1. Only 4 streams \(`comments`, `commits`, `issues` and `review comments`\) from the listed above streams are pure incremental meaning that they:
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/google-search-console.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,9 @@ The granularity for the cursor is 1 day, so Incremental Sync in Append mode may
- [Analytics site report by site](https://developers.google.com/webmaster-tools/search-console-api-original/v3/searchanalytics/query)
- Analytics report by custom dimensions

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

## Connector-specific configurations

### Custom reports
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/hubspot.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,9 @@ The HubSpot source connector supports the following streams:
- [LineItemsWebAnalytics](https://developers.hubspot.com/docs/api/events/web-analytics) \(Incremental\)
- [ProductsWebAnalytics](https://developers.hubspot.com/docs/api/events/web-analytics) \(Incremental\)

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

### Notes on the `property_history` streams

`Property_history` streams can be synced using an `Incremental` sync mode, which uses a cursor timestamp to determine which records have been updated since the previous sync. Within these streams, some fields types (ex. `CALCULATED` type) will always have a cursor timstamp that mirrors the time of the latest sync. This results in each sync including many more records than were necessarily changed since the previous sync.
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/instagram.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,9 @@ the [Instagram Graph API](https://developers.facebook.com/docs/instagram-api/).
related to Instagram Ads, use the Facebook Marketing source.
:::

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

## Data type map

AirbyteRecords are required to conform to
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/jira.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,9 @@ This connector outputs the following incremental streams:

If there are more endpoints you'd like Airbyte to support, please [create an issue.](https://github.com/airbytehq/airbyte/issues/new/choose)

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

## Experimental Tables

The following tables depend on undocumented internal Jira API endpoints and are
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/shopify.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,9 @@ This source can sync data for the [Shopify REST API](https://shopify.dev/api/adm
- [Transactions (GraphQL)](https://shopify.dev/docs/api/admin-graphql/2024-04/objects/OrderTransaction)
- [Tender Transactions](https://shopify.dev/api/admin-rest/2024-04/resources/tendertransaction)

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

## Capturing deleted records

The connector captures deletions for records in the `Articles`, `Blogs`, `CustomCollections`, `Orders`, `Pages`, `PriceRules` and `Products` streams.
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/sources/stripe.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,9 @@ The Stripe source connector supports the following streams:
- [Transfer Reversals](https://stripe.com/docs/api/transfer_reversals/list)
- [Usage Records](https://stripe.com/docs/api/usage_records/subscription_item_summary_list)

### Entity-Relationship Diagram (ERD)
<EntityRelationshipDiagram></EntityRelationshipDiagram>

### Data type map

The [Stripe API](https://stripe.com/docs/api) uses the same [JSON Schema](https://json-schema.org/understanding-json-schema) types that Airbyte uses internally \(`string`, `date-time`, `object`, `array`, `boolean`, `integer`, and `number`\), so no type conversions are performed for the Stripe connector.
Expand Down
29 changes: 29 additions & 0 deletions docusaurus/src/components/EntityRelationshipDiagram.jsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import React, { useEffect, useState } from "react";
import erdBG from "../../static/img/erd-bg-cta.jpg";
import styles from "./EntityRelationshipDiagram.module.css";
const { getRegistryEntry } = require("../remark/utils");
const { getFromPaths } = require("../helpers/objects");

export const EntityRelationshipDiagram = ({}) => {
const [erdUrl, setErdUrl] = useState(null);

useEffect(() => {
async function getErdUrl() {
const entry = await getRegistryEntry({ path: window.location.pathname });
const url = getFromPaths(entry, "erdUrl_[oss|cloud]");
setErdUrl(url);
}

getErdUrl();
}, []);

if (!erdUrl) return null;

return (
<div className={styles.entityRelatonshipdDiagram}>
<a href={erdUrl} target="_blank">
<img src={erdBG} />
</a>
</div>
);
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.entityRelatonshipdDiagram {
width: 100%;
}
.entityRelatonshipdDiagram img {
width: 100%;
height: auto;
}
2 changes: 2 additions & 0 deletions docusaurus/src/theme/MDXComponents/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { PyAirbyteExample } from "@site/src/components/PyAirbyteExample";
import { SpecSchema } from "@site/src/components/SpecSchema";
import MDXComponents from "@theme-original/MDXComponents";
import { Details } from "../../components/Details";
import { EntityRelationshipDiagram } from "../../components/EntityRelationshipDiagram";

export default {
// Re-use the default mapping
Expand All @@ -22,4 +23,5 @@ export default {
PyAirbyteExample,
ProductInformation,
Details,
EntityRelationshipDiagram,
};
Binary file added docusaurus/static/img/erd-bg-cta.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit f216a60

Please sign in to comment.