Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Improve NFT fetch performance on rentals #126

Merged
merged 3 commits into from
Aug 24, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 17 additions & 12 deletions src/adapters/sources/rentals.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import {
Sortable,
} from '../../ports/merger/types'
import { INFTsComponent, NFTResult } from '../../ports/nfts/types'
import { getId } from '../../ports/nfts/utils'
import { IRentalsComponent } from '../../ports/rentals/types'

export function createRentalsNFTSource(
Expand All @@ -21,25 +22,29 @@ export function createRentalsNFTSource(
async function enhanceRentalListing(
rentals: RentalListing[]
): Promise<Sortable<NFTResult, NFTSortBy>[]> {
const nftResults: NFTResult[] = await Promise.all(
rentals.map(async (rental) => {
const nftResult = await nfts.fetchOne(
rental.contractAddress,
rental.tokenId
)
if (!nftResult) {
const tokenIdsOfRentals = rentals.map((rental) => rental.nftId)
const nftResultsOfRentals = await nfts.fetchByTokenIds(tokenIdsOfRentals)
const nftResultsOfRentalsById = Object.fromEntries(
nftResultsOfRentals.map((nft) => [nft.nft.id, nft])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a heads up, this is looping the nftResultsOfRentals twice (one for the map and another for the fromEntries) and creating N arrays + 1 object. Using a .reduce would loop only once and create just 1 object.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is meant just for a single page of 20 elements might not make a difference but for 1000 results it could.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to a reduce!

)
return rentals
.map((rental) => {
let nftResultForRental =
nftResultsOfRentalsById[getId(rental.contractAddress, rental.tokenId)]
if (!nftResultForRental) {
throw new Error('NFT for the rental listing was not found')
}

return {
...nftResult,
nft: { ...nftResult.nft, openRentalId: rental.id },
...nftResultForRental,
nft: {
...nftResultForRental.nft,
openRentalId: rental.id,
},
rental,
}
})
)

return nftResults.map(convertNFTResultToSortableResult)
.map(convertNFTResultToSortableResult)
}

async function fetch(
Expand Down
5 changes: 2 additions & 3 deletions src/logic/nfts/rentals.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,7 @@ export function shouldFetch(filters: NFTFilters): boolean {
filters.category &&
filters.category !== NFTCategory.ESTATE &&
filters.category !== NFTCategory.PARCEL
const landFilterIsSetAndIsNotLAND =
filters.isLand !== undefined && !filters.isLand
const categoriesIsNotSetAndIsNotLand = !filters.category && !filters.isLand
const prohibitedFilterIsSet = PROHIBITED_FILTERS.some(
(prohibitedFilter) =>
setFilters.includes(prohibitedFilter) &&
Expand All @@ -46,7 +45,7 @@ export function shouldFetch(filters: NFTFilters): boolean {
prohibitedSortingIsSet ||
prohibitedFilterIsSet ||
categoriesIsSetAndIsNotLAND ||
landFilterIsSetAndIsNotLAND
categoriesIsNotSetAndIsNotLand
) && Boolean(filters.isOnRent)
)
}
23 changes: 22 additions & 1 deletion src/ports/nfts/component.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
import { NFTFilters, NFTSortBy } from '@dcl/schemas'
import { ISubgraphComponent } from '@well-known-components/thegraph-component'
import { INFTsComponent, NFTResult } from './types'
import { getFetchOneQuery, getFetchQuery, getQueryVariables } from './utils'
import {
getByTokenIdQuery,
getFetchOneQuery,
getFetchQuery,
getQueryVariables,
} from './utils'

export function createNFTComponent<T extends { id: string }>(options: {
subgraph: ISubgraphComponent
Expand Down Expand Up @@ -81,9 +86,25 @@ export function createNFTComponent<T extends { id: string }>(options: {
}
}

/**
* Fetches up to 1000 NFTs by their token IDs.
*/
async function fetchByTokenIds(tokenIds: string[]): Promise<NFTResult[]> {
const query = getByTokenIdQuery(fragmentName, getFragment)
const variables = {
tokenIds,
}
const { nfts } = await subgraph.query<{
nfts: T[]
}>(query, variables)

return nfts.map(fromFragment)
}

return {
fetch,
fetchOne,
fetchByTokenIds,
count,
}
}
1 change: 1 addition & 0 deletions src/ports/nfts/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@ export type QueryVariables = Omit<NFTFilters, 'sortBy'> & {
export interface INFTsComponent {
fetch(filters: NFTFilters): Promise<NFTResult[]>
fetchOne(contractAddress: string, tokenId: string): Promise<NFTResult | null>
fetchByTokenIds(tokenIds: string[]): Promise<NFTResult[]>
count(filters: NFTFilters): Promise<number>
}
17 changes: 17 additions & 0 deletions src/ports/nfts/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,23 @@ export function getFetchOneQuery(
`
}

export function getByTokenIdQuery(
fragmentName: string,
getFragment: () => string
) {
return `
query NFTByTokenId($tokenIds: [String!]) {
nfts(
where: { id_in: $tokenIds }
first: 1000
) {
...${fragmentName}
}
}
${getFragment()}
`
}

export function getId(contractAddress: string, tokenId: string) {
return `${contractAddress}-${tokenId}`
}
3 changes: 3 additions & 0 deletions src/tests/adapters/sources/nft-source.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,15 @@ let options: {
}
let fetchMock: jest.Mock
let fetchOneMock: jest.Mock
let fetchByTokenIdsMock: jest.Mock
let countMock: jest.Mock
let shouldFetchMock: jest.Mock
let getRentalsListingsMock: jest.Mock
let getOpenRentalsListingsOfNFTsMock: jest.Mock

beforeEach(() => {
fetchMock = jest.fn()
fetchByTokenIdsMock = jest.fn()
countMock = jest.fn()
fetchOneMock = jest.fn()
shouldFetchMock = jest.fn()
Expand All @@ -39,6 +41,7 @@ beforeEach(() => {
nftComponentMock = {
fetch: fetchMock,
fetchOne: fetchOneMock,
fetchByTokenIds: fetchByTokenIdsMock,
count: countMock,
}

Expand Down
34 changes: 20 additions & 14 deletions src/tests/adapters/sources/rentals-source.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { SignaturesServerPaginatedResponse } from '../../../ports/rentals/types'
import { INFTsComponent, NFTResult } from '../../../ports/nfts/types'
import { IRentalsComponent } from '../../../ports/rentals/types'
import { convertNFTResultToSortableResult } from '../../../logic/nfts/utils'
import { getId } from '../../../ports/nfts/utils'

let rentalsNFTSource: IMergerComponent.Source<NFTResult, NFTFilters, NFTSortBy>
let rentalsComponentMock: IRentalsComponent
Expand All @@ -13,12 +14,14 @@ let getRentalsListingsMock: jest.Mock
let getOpenRentalsListingsOfNFTsMock: jest.Mock
let fetchMock: jest.Mock
let fetchOneMock: jest.Mock
let fetchByTokenIdsMock: jest.Mock
let countMock: jest.Mock
let rentalsResponse: SignaturesServerPaginatedResponse<RentalListing[]>
let nftResults: NFTResult[]

beforeEach(() => {
getRentalsListingsMock = jest.fn()
fetchByTokenIdsMock = jest.fn()
getOpenRentalsListingsOfNFTsMock = jest.fn()
fetchMock = jest.fn()
fetchOneMock = jest.fn()
Expand All @@ -30,6 +33,7 @@ beforeEach(() => {
nftsComponentsMock = {
fetch: fetchMock,
fetchOne: fetchOneMock,
fetchByTokenIds: fetchByTokenIdsMock,
count: countMock,
}
rentalsNFTSource = createRentalsNFTSource(
Expand All @@ -54,6 +58,7 @@ describe('when fetching rented nfts', () => {
data: {
results: Array.from({ length: 5 }, (_, i) => ({
id: i.toString(),
nftId: `parcel-${i}`,
contractAddress: `contractAddress-${i}`,
tokenId: `tokenId-${i}`,
startedAt: Date.now(),
Expand All @@ -67,24 +72,24 @@ describe('when fetching rented nfts', () => {
}
nftResults = rentalsResponse.data.results.map((rental) => ({
nft: {
id: getId(rental.contractAddress, rental.tokenId),
name: `nft-${rental.id}`,
contractAddress: `contractAddress-${rental.id}`,
tokenId: `tokenId-${rental.id}`,
openRentalId: rental.id,
createdAt: Date.now(),
soldAt: Date.now(),
} as NFT,
order: null,
rental: null,
}))
rentalsResponse.data.results.forEach((_, i) => {
fetchOneMock.mockResolvedValueOnce(nftResults[i])
})

fetchByTokenIdsMock.mockResolvedValueOnce(nftResults)
getRentalsListingsMock.mockResolvedValueOnce(rentalsResponse)
})

it('should return the fetched nfts', () => {
it('should return the fetched nfts', async () => {
return expect(
rentalsNFTSource.fetch!({ isOnRent: true })
rentalsNFTSource.fetch!({ isOnRent: true, isLand: true })
).resolves.toEqual(
nftResults
.map((nftResult, i) => ({
Expand All @@ -105,7 +110,7 @@ describe('when fetching rented nfts', () => {

it('should reject propagating the error', () => {
return expect(
rentalsNFTSource.fetch!({ isOnRent: true })
rentalsNFTSource.fetch!({ isOnRent: true, isLand: true })
).rejects.toThrowError('An error occurred')
})
})
Expand All @@ -127,6 +132,7 @@ describe('when counting rented nfts', () => {
data: {
results: Array.from({ length: 5 }, (_, i) => ({
id: i.toString(),
nftId: `nft-id-${i}`,
contractAddress: `contractAddress-${i}`,
tokenId: `tokenId-${i}`,
startedAt: Date.now(),
Expand All @@ -144,7 +150,7 @@ describe('when counting rented nfts', () => {

it('should return the number of fetched nfts', () => {
return expect(
rentalsNFTSource.count({ isOnRent: true })
rentalsNFTSource.count({ isOnRent: true, isLand: true })
).resolves.toEqual(rentalsResponse.data.total)
})
})
Expand All @@ -158,7 +164,7 @@ describe('when counting rented nfts', () => {

it('should reject propagating the error', () => {
return expect(
rentalsNFTSource.count!({ isOnRent: true })
rentalsNFTSource.count!({ isOnRent: true, isLand: true })
).rejects.toThrowError('An error occurred')
})
})
Expand All @@ -183,6 +189,7 @@ describe('when fetching and counting rented nfts', () => {
data: {
results: Array.from({ length: 5 }, (_, i) => ({
id: i.toString(),
nftId: `parcel-${i}`,
contractAddress: `contractAddress-${i}`,
tokenId: `tokenId-${i}`,
startedAt: Date.now(),
Expand All @@ -196,6 +203,7 @@ describe('when fetching and counting rented nfts', () => {
}
nftResults = rentalsResponse.data.results.map((rental) => ({
nft: {
id: getId(rental.contractAddress, rental.tokenId),
name: `nft-${rental.id}`,
openRentalId: rental.id,
createdAt: Date.now(),
Expand All @@ -204,16 +212,14 @@ describe('when fetching and counting rented nfts', () => {
order: null,
rental: null,
}))
rentalsResponse.data.results.forEach((_, i) => {
fetchOneMock.mockResolvedValueOnce(nftResults[i])
})

fetchByTokenIdsMock.mockResolvedValueOnce(nftResults)
getRentalsListingsMock.mockResolvedValueOnce(rentalsResponse)
})

it('should return the count and the fetched nfts', () => {
return expect(
rentalsNFTSource.fetchAndCount!({ isOnRent: true })
rentalsNFTSource.fetchAndCount!({ isOnRent: true, isLand: true })
).resolves.toEqual({
data: nftResults
.map((nftResult, i) => ({
Expand All @@ -235,7 +241,7 @@ describe('when fetching and counting rented nfts', () => {

it('should reject propagating the error', () => {
return expect(
rentalsNFTSource.fetchAndCount!({ isOnRent: true })
rentalsNFTSource.fetchAndCount!({ isOnRent: true, isLand: true })
).rejects.toThrowError('An error occurred')
})
})
Expand Down
28 changes: 20 additions & 8 deletions src/tests/logic/nft-rentals.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,23 @@ import {
shouldFetch,
} from '../../logic/nfts/rentals'

describe('when checking if the rental should be checked', () => {
describe('when checking if the rental should be fetched', () => {
PROHIBITED_FILTERS.forEach((filter) => {
describe(`and the filter ${filter} is set which is prohibited`, () => {
it('should return false', () => {
expect(shouldFetch({ [filter]: true, isOnRent: true })).toBe(false)
expect(
shouldFetch({ [filter]: true, isOnRent: true, isLand: true })
).toBe(false)
})
})
})

PROHIBITED_SORTING.forEach((sortBy) => {
describe(`and the sort by filter is set to ${sortBy} which is prohibited`, () => {
it('should return false', () => {
expect(shouldFetch({ sortBy, isOnRent: true })).toBe(false)
expect(shouldFetch({ sortBy, isOnRent: true, isLand: true })).toBe(
false
)
})
})
})
Expand All @@ -43,15 +47,21 @@ describe('when checking if the rental should be checked', () => {
})
})

describe('and the isOnRent filter is not true', () => {
describe('and the isOnRent filter is not true but the isLand one is', () => {
it('should return false', () => {
expect(shouldFetch({ isOnRent: false })).toBe(false)
expect(shouldFetch({ isOnRent: false, isLand: true })).toBe(false)
})
})

describe('and the isOnRent filter is set to true', () => {
describe('and the isOnRent and isLand filter are set to true', () => {
it('should return true', () => {
expect(shouldFetch({ isOnRent: true })).toBe(true)
expect(shouldFetch({ isOnRent: true, isLand: true })).toBe(true)
})
})

describe('and the isOnRent filter is set but isLand is set to false', () => {
it('should return false', () => {
expect(shouldFetch({ isOnRent: true, isLand: false })).toBe(false)
})
})
;[(NFTCategory.ESTATE, NFTCategory.PARCEL)].forEach((category) => {
Expand All @@ -73,7 +83,9 @@ describe('when checking if the rental should be checked', () => {
].forEach((filter) => {
describe(`and the filter ${filter} filter is set which is permitted`, () => {
it('should return true', () => {
expect(shouldFetch({ isOnRent: true, [filter]: true })).toBe(true)
expect(
shouldFetch({ isOnRent: true, isLand: true, [filter]: true })
).toBe(true)
})
})
})
Expand Down
Loading