Welcome to MetaResearch Journal Club!

[KarolaKirsanow]

📚 The PL Journal Club will focus on MetaResearch: the study of research itself - scientific methods, evaluation techniques, incentives, and more. We'll be reading and discussing ways of accelerating our research and research engineering, and learning about the successful R&D initiatives of the past and present. Tools, mental models, evaluation frameworks - the works!

Please use this board to post and discuss suggested readings!
We meet here: https://protocol.zoom.us/j/91732611849

👉 You can sign up to select a topic and lead a discussion here

[KarolaKirsanow]

Notes from an earlier discussion on the Journals and Conferences of the Future:

Meta-Journal_Club_the_knowledge_dissemination_models_of_the_future.pdf

[metasj]

• in development: Wikifunctions / Wikilambda, and a U.Mich function repository

[metasj]

Other sources to add to this list? HuggingFace models, Problem-generator-functions, Pipes (IFTTT, Import.io, ...) ?

^: Perhaps each session we could have a [lit review] section, w/ lightning-talks about areas that need fleshing out, as well as a [discussion topic] section w/ a more focused topic.

[KarolaKirsanow]

Do you want to start us off with a few pieces to focus on and a lightning talk ⚡ ?

[metasj]

I feel I don't have a sufficiently wide view of the space yet! But could talk about Wikifunctions and Pipes...

[KarolaKirsanow]

Would this working paper be a relevant starting point for the discussion?

[redransil]

As far as suggested reading, @davidad suggested Poston's Reason and Explanation - A Defense of Explanatory Coherentism a few months ago. I started it and am enjoying it - would be great to discuss!

[KarolaKirsanow]

We could do a session on the relationship of UX/UI design to the future of scientific publication and explanation.

Andy Matuschak and Michael Nielsen's 2019 essay would be a useful starting point for a discussion of tooling-for understanding-and-dissemination.

Cindy asked a great question in our session on Journals of the Future: how do you optimize for fun?

Related:

• Alan Kay on The Future of Reading Depends on the Future of Learning Difficult to Learn Things and the Dynabook

[joelchan]

Suggested readings (in no particular order):

• Bret Victor - Magic Ink: Information Software and the Graphical Interface: mind-expanding ideas around the medium itself for complex information
• Max Krieger - Chatting with Glue: Cognitive Tools for Augmented Conversation: mind-expanding ideas around the medium itself for discussion/conversation
• Allen Renear & Carole Palmer (2009) - Strategic Reading, Ontologies, and the Future of Scientific Publishing : bold predictions of semantic reading by 2019 (published in Science!), which... haven't yet come to pass, but the history and vision is worth a read. Last author is major player in library information science (LIS) literature on scholarly information behavior (scholarly workflows, primitives) and interdisciplinarity.
• Anna de Liddo, Agnes Sandor, and Simon Buckingham Shum (2012) - Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study: good entry point into another stream of work on discourse-centric models of scholarly collective sensemaking and publishing, with a special helpful twist emphasizing the contested nature of discourse (so a hard pushback against centralized "single source of truth" ontologies etc.). the last author (Simon Buckingham-Shum) is/was a major player in alternative models of scholarly communication and collective intelligence and deeply rooted in the HCI/CSCW literature on how hard this problem is (now mostly working in the learning analytics space, hopefully not because the problem is intractable!!)
• Anita de Waard (2010) - From Proteins to Fairytales: Directions in Semantic Publishing: similar to Renear, but with more details, focused on biomed, and makes a nice case for discourse-centric representations (not just RDF triples)
• Dave Randall et al (2011) - Distributed ontology building as practical work: deep dive into the use of a major tool (Protege) for building ontologies.
• Susan Halford et al (2013) Digital Futures? Sociological Challenges and Opportunities in the Emergent Semantic Web: Expands on Randall's themes of the fraught nature of semantic publishing in much more depth and breadth

[KarolaKirsanow]

These look great!

[KarolaKirsanow]

What we discussed on Monday:

How Can We Develop Transformative Tools for Thought?

the most powerful tools for thought express deep insights into the underlying subject matter

Hildegard von Bingen, 12thc, Columba aspexit - neumes notation

Ludwig van Beethoven 19thc, 9th Symphony

many tools for thought are public goods

Why do books work?

• Memory is far more important than people tend to think
• Take emotion seriously

The executable book

https://observablehq.com/embed/@bronna/parametric-seashell?cell=*

good tools for thought arise mostly as a byproduct of doing original work on serious problems

Necessary...

Python_scripts

...but not sufficient

[KarolaKirsanow]

Further notes on the discussion:

• what are the problems we want to solve with the tools we want to build?
• how does one go about developing a "research taste"
• what is the role of fun? lessons from music and sport?
• when can we consider two similar bits of evidence equivalent?
• can claims be equivalent if they originated from or relate to different contexts?

[KarolaKirsanow]

Suggestions for 12 July:

• McMahan, P., & Evans, J. (2018). Ambiguity and engagement. American Journal of Sociology, 124(3), 860-912 ✅
• Meadows, D. H. (1999). Leverage points: Places to intervene in a system.

[KarolaKirsanow]

Possibilities for 2 August:

• Swanson, D. - Undiscovered Public Knowledge and Rediscovering Don Swanson: the Past, Present and Future of Literature-Based Discovery
• Franzoni, Stephan, & Veugelers - Funding Risky Research
• Franzoni & Stephan - Uncertainty and Risk-Taking in Science: Meaning, Measurement and Management

[metasj]

• the Overedge Catalog (+ modes of risk-taking in each)

[cindywu]

+1 for Franzoni, Stephan, & Veugelers or Franzoni & Stephan

Having @maxkreiger present his own work and lead discussion was really fun. If the group wanted to do something similar, I could ask Chiara Franzoni if she wants to present and lead discussion on one of the papers above.

[cindywu]

Suggested reading:

• Any chapter from Gaming the Metrics: Misconduct and Manipulation in Academic Research

[KarolaKirsanow]

Cycles of Invention and Discovery
Questions from 4 October:

• is it fair to credit Vannevar Bush with creating the applied/basic science dichotomy?
• p.60: "the innovation culture of IBM emphasized producing results, even if they might not be commercially useful to the company" -- how important is the "slack" provided by monopoly position or rents (AT&T) or huge profit margins (IBM) in enabling this type of long-term vision? It sounds like AT&T considered itself as part of the basic national infrastructure

[KarolaKirsanow]

Science, the Endless Frontier by Vannevar Bush

[KarolaKirsanow]

Surprised to see that there are no public hypothes.is annotations on this piece

[KarolaKirsanow]

Mirror-Image Twins: The Communities of
Science and Technology in 19th-Century
America by Edwin Layton

[KarolaKirsanow]

Galileo’s stream: A framework for understanding
knowledge production by J.Y. Tsao et al.

[KarolaKirsanow]

Also of interest: Carlotta Perez's work on Technological Revolutions and Financial Capital