Community Health Analytics

[daniel-ospina]

Overview

DAOs depend on the health and vibez of their community—yet understanding and measuring Community Health is challenging. DAOs and Web3 projects at large are limited both by a lack of theoretical foundations and tools to advance the necessary research. A better understanding of the drivers and predictors of Community Health can:

• enable a wealth of insights that can benefit all web3 projects with a community to improve resilience and anti-fragility, effectiveness, attraction and retention, and more,
• provide investors (whether institutional, retail, or contributors investing their time) the tools to separate healthy, thriving communities from scams and vaporware,
• and create a more enjoyable experience and facilitate wellbeing for the thousands of community members and contributors.

Community is essential to Web3, understanding it and being able to provide reliable indicators is a fundamental enabler for a healthy ecosystem.

State of the Art

From a theory perspective, community managers intuitively know that simply measuring the number of participants is a flawed metric, and although significant research on communities has taken place in Web2 and before, we lack a conceptual framework to understand community in Web3 where stakeholder classes are blurry and organisational practices are being reinvented. As such, it's unclear whether previous findings (if collected and summarised) can be taken at face value and applied, or not.

And from a practice perspective, today DAOs are left to rely on the limited reporting provided by Discord or Discourse and a patchwork of "homemade" surveys to fill in the gaps. These solutions are time-consuming for community managers and contributors to use, and the results, hampered by poor indicators and/or poor sampling, are unreliable.
Perhaps even direr, the lack of real-time analytics leaves community leaders without established baselines to measure against to understand the impact of community-focused initiatives, monitor shocks to the system, or rapidly gauge the effects of system-wide changes (such as new deployments).

Current approaches for Community Analytics in Web3

Numerous tools exist to map community in Web2. However, these tools operate on Web2 stacks (Slack, Microsoft Teams, etc.).
For on-chain analytics, tools like Dune and the associated community of researchers have made inroads and analysed transactions data. However, there are few tools (and to our knowledge, none fit for purpose) to analyse discussions and collaboration interactions that preceded a transaction on-chain.

Different DAOs have started advancing the creation of custom analytics (e.g. Aragon, ENS, etc.). These efforts are currently scattered and limited in scope and resourcing and could benefit from refactoring (creation of a community of practice and a comprehensive research programme).

Current approaches within the broader research ecosystem

a range of techniques are currently in use to advance our understanding of communities and social networks, including Organisational Network Analysis and recurring pulse surveys and extensive surveys based on different theoretical constructs (employee engagement, team trust, virtual sense of community, etc.). The applicability of these approaches and theoretical constructs to Web3 is still largely unexplored.

Known shortcomings of existing solutions

Most tools and theoretical constructs are based on a command-and-control model of organisations and a model of community and social collectives based on passive consumption (or, at best, collaboration but no ownership or collective governance).
However, DAOs are arguably a new form of human coordination.
The unique properties of DAOs in particular (but also of the Web3 ecosystem in general) create both a conceptual and practical gap between existing solutions and the needs of Web3 community leaders to understand what drives community health, gauge progress, and find meaningful insights to improve and/or course correct.

Finally, existing solutions rely on Web2 data management and warehousing practices, which are incompatible with Web3 Ethos and hamper adoption.

Solving this Open Problem

Estimated impact

Gallup estimates that 87% of employees worldwide are not engaged at work - billions of people underperforming day after day and feeling like their work is meaningless, in turn enabling a mental health crisis. DAOs offer a potential solution to this problem, unlocking human potential and happiness.

Meanwhile, the number of scams in web3 risks setting the whole industry back, and the hyper financialization and usage of poor indicators to gauge the health of projects (e.g. Total Value Locked) hampers our ability to advance a humanistic. thriving web3. Understanding and being able to assess Community Health provides a countermeasure with broad applicability.

Proposed solution definition

The creation of a robust theoretical framework and reliable indicators for Community Health is a complex task, requiring review of previous research, development of data collection tools, experimentation and analysis. A solution can only be achieved iteratively when empirical results are replicable and have high validity. Even then, the continuous evolution of communities and organisations (including DAOs) will likely require ongoing research on this topic. As such, we're not proposing a final solution but rather a stream of work (including orchestration of efforts across communities), operating in cycles of review, experimentation, analysis, discussion and synthesis for knowledge sharing (see more on the Generative Action Research Methodology we draw inspiration from).

We propose a starting point with 1 iteration (1 cycle), including:

• open invitation for co-researchers and community leaders/community managers to join the research programme
• review of previous research (academic literature review, practitioner workshops to distil tacit knowledge, and review of previous research initiatives including Ecosystem Health research carried by Near, Governauts, et al. and DAO Health carried by TalentDAO).
• synthesis into a conceptual framework for DAO Community Health
• development of a data-gathering tool (Discord and Discourse interaction data and discord-native pulse survey functionality to maximise engagement)
• solving for data management and data warehousing using a decentralised solution (e.g. IPFS)
• data collection on communities supporting the project
• data analysis
• facilitated discussion with programme participants
• creation of a learnings and insights report to share with the broad ecosystem

Supplementary Material

Existing conversations/threads

We've already secured 1/3 of funding from Aragon to support this endeavour, and are looking for at least two other organisations to partner with us, RnDAO, in advancing this research programme.

RnDAO members leading this submission:

Danielo
Previously, Head of Governance at Aragon, 8 years experience in Organization Design consulting (clients include Google, BCG, Daymler, The UN, and multiple startups), and visiting lecturer at Oxford University.
Twitter: https://twitter.com/_Daniel_Ospina
LinkedIn: https://www.linkedin.com/in/conductal/

Katerinabc
Ph.D. in Team Dynamics using Social Network Analysis, Teaching Collaboration, and Organizational Performance at Northwestern University (since 2016).
Co-organized Learning in Networks sessions at the International Conference of Social Network Analysis (2018 - 2020), and previously advised a people analytics company on social network metrics.
Twitter: twitter.com/katerinabohlec
Linkedin: linkedin.com/in/katerinab
Github: https://github.com/katerinabc/

Thegadget.eth
Software Engineer. Previously, Product Manager at Neolyze (Business Intelligence Dashboard for Instagram).
Github: https://github.com/thegadget-eth/
Twitter: https://twitter.com/mr_gadget22

Team Advisor

Sam
Previously, Head of Technical Research at Aragon. Previously, Lead Developer of the official JavaScript API for the Ethereum blockchain.
Github: https://github.com/nivida
Twitter: https://twitter.com/furter_samuel