refactor(datasource): migrate to class based datasource #6747

JamieMagee · 2020-07-14T08:59:26Z

A small experiment into what OOP/class based datasources might look like. Picked Cdnjs as it's the smallest & simplest.

With this approach we can share common logic, like error handling, rate limiting, etc. between different datasources, instead of having to reimplement it each time we write a new datasource. Currently there's nothing shared, as it's only 1 datasource, but the interesting stuff will come with the 2nd datasource!

viceice

Rate limiting should be on http layer, not on datasource layer.

lib/datasource/index.spec.ts

lib/datasource/cdnjs/index.ts

rarkins · 2020-07-15T06:19:27Z

#5557 - De-duplicate datasource error handling
#6757 - Add private/auth awareness to datasource index

Also, I'd been thinking the potentially the datasource index could instantiate a http instance for each datasource at startup (being sure to include the id) and then pass it as part of parameters to getReleases or getDigest.

viceice · 2020-08-30T15:16:51Z

? review or not review? 🤔

rarkins · 2020-08-30T16:03:38Z

? review or not review? 🤔

Sorry, requested review of wrong branch

JamieMagee · 2021-02-17T06:25:44Z

Rebased on top of current HEAD. Reverted crate changes as this branch has strayed too far from the existing implementation, and rewriting would be simpler than rebasing.

lib/datasource/api.ts

lib/datasource/cdnjs/index.ts

.eslintrc.js

lib/datasource/cdnjs/index.ts

lib/datasource/index.ts

lib/util/http/index.ts

lib/versioning/loose/generic.ts

package.json

tools/jest-gh-reporter.ts

viceice

I think we should only convert this two datasources with this pr and convert others in separate pr's, so it's easier to review

lib/datasource/datasource.ts

lib/manager/deps-edn/extract.ts

lib/datasource/index.spec.ts

.eslintrc.js

lib/datasource/datasource.ts

lib/datasource/index.spec.ts

viceice

needs lint fix 😉

lib/datasource/datasource.ts

A small experiment into what OOP/class based datasources might look like. Picked Cdnjs as it's the smallest & simplest. With this approach we can share common logic, like error handling, rate limiting, etc. between different datasources, instead of having to reimplement it each time we write a new datasource. Currently there's nothing shared, as it's only 1 datasource, but the interesting stuff will come with the 2nd datasource

JamieMagee · 2021-05-18T05:12:53Z

@viceice I think I've addressed all your comments.

@rarkins I think this is ready for your review.

JamieMagee · 2021-05-19T03:30:40Z

Pls no more conflicts 😨 🤞

viceice

Just some small stuff

lib/datasource/datasource.spec.ts

lib/datasource/clojure/index.spec.ts

lib/datasource/datasource.spec.ts

Co-authored-by: Michael Kriese <michael.kriese@visualon.de>

viceice

🥳🥳

renovate-release · 2021-05-21T10:50:28Z

🎉 This PR is included in version 25.26.4 🎉

The release is available on:

GitHub release
25.26.4

Your semantic-release bot 📦🚀

viceice reviewed Jul 14, 2020

View reviewed changes

lib/datasource/index.spec.ts Outdated Show resolved Hide resolved

lib/datasource/cdnjs/index.ts Outdated Show resolved Hide resolved

viceice mentioned this pull request Jul 14, 2020

feat(internal): make datasource loading syncronous #6748

Merged

JamieMagee force-pushed the refactor/class-based-datasource branch from 0b8463e to 68c6029 Compare August 19, 2020 07:55

JamieMagee force-pushed the refactor/class-based-datasource branch from 68c6029 to 857eecb Compare August 30, 2020 14:24

rarkins marked this pull request as ready for review August 30, 2020 14:53

rarkins requested a review from viceice August 30, 2020 14:53

rarkins marked this pull request as draft August 30, 2020 14:53

JamieMagee force-pushed the refactor/class-based-datasource branch from ce75f5b to 8615e66 Compare September 1, 2020 10:14

JamieMagee force-pushed the refactor/class-based-datasource branch 2 times, most recently from acc6cf5 to 385df5b Compare October 7, 2020 08:08

JamieMagee marked this pull request as ready for review October 7, 2020 08:08

JamieMagee requested a review from rarkins October 27, 2020 14:00

JamieMagee mentioned this pull request Oct 27, 2020

feat(nuget): handle http errors #6284

Closed

rarkins marked this pull request as draft December 4, 2020 06:23

JamieMagee mentioned this pull request Feb 11, 2021

Refactor: Datasource classes #8647

Closed

viceice mentioned this pull request Feb 13, 2021

feat: static modules #8679

Merged

6 tasks

JamieMagee force-pushed the refactor/class-based-datasource branch from a49935f to 0e5f281 Compare February 17, 2021 06:25

JamieMagee force-pushed the refactor/class-based-datasource branch from 0e5f281 to 9a09d00 Compare March 17, 2021 04:49

JamieMagee marked this pull request as ready for review March 17, 2021 04:50

viceice requested changes Apr 8, 2021

View reviewed changes

JamieMagee force-pushed the refactor/class-based-datasource branch from f4dc06d to 8fb734e Compare April 9, 2021 03:01

viceice reviewed Apr 9, 2021

View reviewed changes

lib/datasource/datasource.ts Outdated Show resolved Hide resolved

lib/manager/deps-edn/extract.ts Outdated Show resolved Hide resolved

lib/datasource/index.spec.ts Outdated Show resolved Hide resolved

JamieMagee force-pushed the refactor/class-based-datasource branch 4 times, most recently from ce19bb2 to e0ab61f Compare May 5, 2021 05:13

JamieMagee requested a review from viceice May 5, 2021 05:14

viceice reviewed May 5, 2021

View reviewed changes

JamieMagee force-pushed the refactor/class-based-datasource branch 3 times, most recently from 032ac52 to 56c72a0 Compare May 6, 2021 04:25

viceice reviewed May 6, 2021

View reviewed changes

JamieMagee force-pushed the refactor/class-based-datasource branch from 56c72a0 to 61ced89 Compare May 11, 2021 04:02

viceice reviewed May 11, 2021

View reviewed changes

lib/datasource/datasource.ts Show resolved Hide resolved

lib/datasource/datasource.ts Outdated Show resolved Hide resolved

JamieMagee force-pushed the refactor/class-based-datasource branch from 61ced89 to ff03e51 Compare May 14, 2021 05:11

JamieMagee requested a review from viceice May 14, 2021 15:21

JamieMagee force-pushed the refactor/class-based-datasource branch from ff03e51 to 3f7fc70 Compare May 18, 2021 01:01

rarkins and others added 2 commits May 18, 2021 10:32

Merge branch 'main' into refactor/class-based-datasource

ea93011

Merge branch 'main' into refactor/class-based-datasource

b32855d

viceice mentioned this pull request May 19, 2021

Maven datasource should use fs wrapper #10037

Closed

viceice reviewed May 19, 2021

View reviewed changes

lib/datasource/datasource.spec.ts Outdated Show resolved Hide resolved

lib/datasource/clojure/index.spec.ts Outdated Show resolved Hide resolved

lib/datasource/datasource.spec.ts Outdated Show resolved Hide resolved

JamieMagee and others added 4 commits May 20, 2021 14:03

Merge branch 'main' into refactor/class-based-datasource

0d8a39d

Apply suggestions from code review

89a77c0

Co-authored-by: Michael Kriese <michael.kriese@visualon.de>

remove renaming ClojureDatasource to datasource in tests

9f763fd

Merge branch 'main' into refactor/class-based-datasource

6803bca

JamieMagee requested a review from viceice May 21, 2021 05:11

viceice approved these changes May 21, 2021

View reviewed changes

viceice merged commit 621b77b into main May 21, 2021

viceice deleted the refactor/class-based-datasource branch May 21, 2021 05:40

github-actions bot locked as resolved and limited conversation to collaborators Jun 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(datasource): migrate to class based datasource #6747

refactor(datasource): migrate to class based datasource #6747

JamieMagee commented Jul 14, 2020

viceice left a comment

rarkins commented Jul 15, 2020

viceice commented Aug 30, 2020

rarkins commented Aug 30, 2020

JamieMagee commented Feb 17, 2021

viceice left a comment

viceice left a comment

JamieMagee commented May 18, 2021

JamieMagee commented May 19, 2021

viceice left a comment

viceice left a comment

renovate-release commented May 21, 2021

refactor(datasource): migrate to class based datasource #6747

refactor(datasource): migrate to class based datasource #6747

Conversation

JamieMagee commented Jul 14, 2020

viceice left a comment

Choose a reason for hiding this comment

rarkins commented Jul 15, 2020

viceice commented Aug 30, 2020

rarkins commented Aug 30, 2020

JamieMagee commented Feb 17, 2021

viceice left a comment

Choose a reason for hiding this comment

viceice left a comment

Choose a reason for hiding this comment

JamieMagee commented May 18, 2021

JamieMagee commented May 19, 2021

viceice left a comment

Choose a reason for hiding this comment

viceice left a comment

Choose a reason for hiding this comment

renovate-release commented May 21, 2021