diff --git a/README.md b/README.md index d6f1069..606d8ee 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ My Open Science presentation of 2024-02-28 * Duration of presentation (monologue): 30 minutes * Presentation as Quarto markdown: [presentation.qmd](presentation.qmd) * Presentation as PDF: [presentation.pdf](presentation.pdf). Latest version is also uploaded by the [build_pdfs](https://github.com/richelbilderbeek/open_science_presentation_enlight_arcus_20240228/actions/workflows/build_pdfs.yaml) continuous integration script - * YouTube video: [here](https://youtu.be/_RgoN9eGAgI?si=oJRAMbl2v36oipVX) + * YouTube video: [here](https://youtu.be/zHpiz2P4h-U) ## Description diff --git a/presentation.html b/presentation.html new file mode 100644 index 0000000..d7b1697 --- /dev/null +++ b/presentation.html @@ -0,0 +1,1098 @@ + + + + + + + + + + + + + + Open Science: pros, cons and community + + + + + + + + + + + + + + + + +
+
+ +
+

Open Science: pros, cons and community

+ +
+
+
+Richèl ‘Rea-shell’ Bilderbeek +
+
+
+ +
+
+

0 This presentation

+

https://github.com/richelbilderbeek/open_science_presentation_enlight_arcus_20240228

+
+ +
+
+

1 Goals

+
    +
  • To discuss Open Science
    • +
  • To share the usefulness of a community
    • +
+
+
+

1.1 What I do

+
+
+

+
+ +
+
+
+
+

2 This talk

+
    +
  • Experiment 1: Cons: more_literature
    • +
  • Experiment 2: Pros: literature
    • +
  • Experiment 3: Community: community
    • +
+

How much do you enjoy each part?

+
+ +
+
+

3 Experiment 1: more_literature

+
    +
  • Research question: what are the biggest drawbacks of Open Science for a researcher today?
    • +
  • Methods: search 5 hours, collect all results, rank these
    • +
+
+
+

3.1 Results: least relevant

+

Feature:

+
    +
  • Open Methods describe ‘how to do research’, not ‘what to research’ (1)
    • +
+

Dispelling myths:

+
    +
  • Open Science does not improve statistical power (1)
    • +
  • Open Infrastructure costs money too (2)
    • +
+

No idea what to think of this:

+
    +
  • Open Science may be neoliberal (3)
    • +
+
+
+

3.2 Results: maybe relevant

+
    +
  • Open Access may be less inclusive for the poor (4) (5)
    • +
  • There may be impactful cultural and institutional constraints on all facets of Open Science (2)
    • +
  • Training to do proper Open Data may be unaffordable for the poor (6) (5)
    • +
  • Open Methods may diminish qualitative methodologies for the sake of reproducibility (5)
    • +
  • Open Software may be hindered by reluctance to share code (2)
    • +
  • Open Peer Review may give bias and exclusion (5)
    • +
  • Open Evaluation may lead to perverse incentives (1)
    • +
  • Competition between scientists may discourage openness (2), researcher may be put in a social dilemma: (2)
    • +
  • Open Evaluation does not encourage to publish negative results (2)
    • +
+
+
+

3.3 Results: most relevant

+
    +
  • 🤓 Open Data excludes/inconveniences the industry (5)
    • +
  • 🤓 Lack of standards for sharing research materials (2)
    • +
  • 🤓 Open Infrastructure is not in place yet (1) (2)
    • +
  • 🤓 Open Data (a.o. FAIRification) takes more time (7) (2)
    • +
  • 🤔 … especially sharing sensitive data is more complex (2)
    • +
  • 🤔 Open Methods takes more time (7) (2)
    • +
  • 🤔 Open Data has no clear legal guidelines, nor sanctions (2)
    • +
+
+
+
+

4 Experiment 2: literature

+
    +
  • Research question: what are my favorite papers on Open Science that I know?
    • +
  • Methods: copy-paste these from earlier presentations with literature searches
    • +
+
+
+

4.1 Result: regular science is a problem (8)

+
+
+

+
+

⚠️ there exists a version with incorrect/inflated numbers!

+
+
+
+
+

4.2 Result: Open Science helps honest reporting (7)

+
+
+

+
+
    +
  • the file drawer problem
    • +
  • ‘being held hostage by your data’
    • +
+
+
+
+
+

4.3 Result: Open Science results in better papers 1/3 (9)

+ + +
+
+

4.4 Result: Open Science results in better papers 2/3 (9)

+ +
+
+

4.5 Result: Open Science results in better papers 3/3 (9)

+ +
+
+
+

5 Experiment 3: community

+
    +
  • Research question: what are the pros and cons of Open Science for a researcher today?
    • +
  • Methods: visit meetings, select favorites
    • +
+
+
+

5.1 The Open Science Uppsala community

+
    +
  • Who lives in a town with a local community?
    • +
  • Who has visited that local community at least once?
    • +
+
+
+

5.2 Open Science Uppsala goals

+
+
+

+
+
    +
  • teach
    • +
  • discuss
    • +
  • English
    • +
  • regularly
    • +
  • everyone
    • +
+
+
    +
  • Uppsala
    • +
  • free
    • +
  • publicly
    • +
  • scholarly
    • +
+
+
+
+
+

5.3 My statement

+
literature + community
+
+           >
+           
+literature + more_literature
+
+
+

5.4 It takes time to setup an infrastructure

+
+
+

+
+

Developing how to make data FAIR. Sometimes, the machines come to the researchers, but sometimes the researchers need to visit the facility.

+

Resulted in (10). 🤓 (1) (2).

+
+
+
+
+

5.5 Open may be unfair to companies

+
+
+

+
+

A company can be built around a non-sharable resource. It would be unfair to share that resource.

+

🤓 (5)

+
+
+
+
+

5.6 Open source alleviates constraints

+
+
+

+
+

Openly develop useful software and avoiding bureaucracy helps your software getting used.

+

🤓 see (2) on cultural and institutional constraints.

+
+
+
+
+

5.7 Qualitative research is different

+
+
+

+
+

Replication does not make sense for qualitative papers, (e.g. ‘What is democracy?’), so those fields need to be judged differently.

+

🤓 (1)

+
+
+
+
+

5.8 Citizen science helps

+
+
+

+
+

Achieving statistical power ignores the individual. Citizen science helps find strong effects in few individuals.

+

🤔

+
+
+
+
+

5.9 Preregistration with statistical power 1/3

+
+
+

+
+

One can do a specification curve analysis in a pre-registered study to keep statistical power

+

🤔

+
+
+
+
+

5.10 Preregistration with statistical power 2/3 (11)

+
+
+

+
+
    +
  • Data: Hurricane name, amount of deaths, year, wind speed, pressure
    • +
  • Hypothesis: the name of a hurricane is a predictor of the amount of deaths
    • +
+
+
+
+
+

5.11 Preregistration with statistical power 3/3 (11)

+ +
+
+

5.12 Science is more complex 1/4

+
+
+

+
+
    +
  • Gustav Nilsonne
    • +
  • Even if data is open, different teams can draw opposite conclusions with high confidence (12)
    • +
  • This makes interpreting results even harder!
    • +
+

🤔

+
+
+
+
+

5.13 Science is more complex 2/4 (12)

+ +

Hypotheses: ‘There is [an effect] in [a brain area] between treatment A and B’

+
+
+

5.14 Science is more complex 3/4 (12)

+
+
+

+
+

+
+
+
+
+

5.15 Science is more complex 4/4 (12)

+ +

And we are confident! Compare hypothesis 2 for team 2 and 4!

+
+
+
+

6 Another community

+ +
+
+

6.1 This community

+ +

Source: https://zenodo.org/records/10528857

+
+

7 Conclusion

+
    +
  • I hope to have convinced you that …
    • +
+
literature + community
+
+           >
+           
+literature + more_literature
+
    +
  • … is true, because people have broader ideas than you search the literature for
    • +
  • This is a community too!
    • +
+
+ +
+

8 Questions?

+

https://github.com/richelbilderbeek/open_science_presentation_enlight_arcus_20240228

+
+ +
+

9 References

+
+ +
+
+
+
1.
Banks GC, Field JG, Oswald FL, O’Boyle EH, Landis RS, Rupp DE, et al. Answers to 18 questions about open science practices. Journal of Business and Psychology. 2019;34:257–70.
+
+
+
2.
Scheliga K, Friesike S. Putting open science into practice: A social dilemma? First Monday. 2014;
+
+
+
3.
Uygun Tunç D, Tunç MN, Eper ZB. Is open science neoliberal? Perspectives on Psychological Science. 2023;18(5):1047–61.
+
+
+
4.
Bahlai C, Bartlett LJ, Burgio KR, Fournier AM, Keiser CN, Poisot T, et al. Open science isn’t always open to all scientists. American Scientist. 2019;107(2):78–82.
+
+
+
5.
Ross-Hellauer T, Reichmann S, Cole NL, Fessl A, Klebel T, Pontika N. Dynamics of cumulative advantage and threats to equity in open science: A scoping review. Royal Society Open Science. 2022;9(1):211032.
+
+
+
6.
Bezuidenhout LM, Leonelli S, Kelly AH, Rappert B. Beyond the digital divide: Towards a situated approach to open data. Science and Public Policy. 2017;44(4):464–75.
+
+
+
7.
Allen C, Mehler DM. Open science challenges, benefits and tips in early career and beyond. PLoS biology. 2019;17(5):e3000246.
+
+
+
8.
Munafò MR, Nosek BA, Bishop DV, Button KS, Chambers CD, Percie du Sert N, et al. A manifesto for reproducible science. Nature human behaviour. 2017;1(1):1–9.
+
+
+
9.
Soderberg CK, Errington TM, Schiavone SR, Bottesini J, Thorn FS, Vazire S, et al. Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour. 2021;5(8):990–7.
+
+
+
10.
Larsson ÅM, Bornsäter B. FAIR data för heritage science: Utvecklad praktik för öppna forskningsdata: Rapport från ett FoU-projekt. Riksantikvarieämbetet; 2023.
+
+
+
11.
Simonsohn U, Simmons JP, Nelson LD. Specification curve analysis. Nature Human Behaviour. 2020;4(11):1208–14.
+
+
+
12.
Botvinik-Nezer R, Holzmeister F, Camerer CF, Dreber A, Huber J, Johannesson M, et al. Variability in the analysis of a single neuroimaging dataset by many teams. Nature. 2020;582(7810):84–8.
+
+
+
+
+
+ + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/presentation.qmd b/presentation.qmd index ba309a1..fc55d89 100644 --- a/presentation.qmd +++ b/presentation.qmd @@ -42,6 +42,8 @@ number-offset: -1 - Experiment 2: Pros: `literature` - Experiment 3: Community: `community` +How much do you enjoy each part? + # Experiment 1: `more_literature` - Research question: what are the biggest drawbacks of Open Science for a researcher today?