What Is an Online Laboratory?

When we call Futures Change Laboratory an online laboratory, we mean something specific. We do not mean a website with a research page or a course with a lab theme. We mean a working research organization with no shared building: one that designs and runs studies, protects the people and data it works with, competes for and manages grants, and publishes findings others can check. That ambition sits inside a long shift in how research is done. Knowledge production has moved steadily from individuals to teams (Wuchty et al., 2007), and from single sites to networks linked by digital infrastructure (Atkins et al., 2003; Dutton & Jeffreys, 2010; Hey et al., 2009). A substantial literature now studies how such distributed science is organized and why it succeeds or fails (Jirotka et al., 2013; National Research Council, 2015; Olson et al., 2008; Ribes & Lee, 2010; Sonnenwald, 2007). This post draws on that literature to answer two questions: what makes a laboratory a laboratory, and what it takes for a fully virtual one to be fundable.

1. What makes a laboratory a laboratory

Before asking how a laboratory can be virtual, we need to be clear about what a laboratory does. Historians and sociologists of science agree that it is far more than a room with equipment (Gieryn, 2006; Knorr Cetina, 1999; Kohler, 2002; Shapin, 1988).

Shapin (1988) showed that the seventeenth-century "house of experiment" was a social arrangement as much as a physical one. It controlled who could enter, who counted as a credible witness, and how private work became public knowledge. Laboratory ethnographies made a related point about modern science. Latour and Woolgar (1979/1986) described the laboratory as a system of "inscription devices" that turn messy material events into data, figures and papers that can travel and persuade. Knorr Cetina (1999) argued that the laboratory's power comes from reconfiguration. A lab does not have to study things as they are, where they are, or when they happen. It brings objects into its own space, works with traces and representations, and makes events happen often enough to study them. It also reconfigures researchers into instruments of inquiry (Knorr Cetina, 1999).

Two further lines of work complete the picture. Hacking (1992) argued that mature laboratory sciences become "self-vindicating," as theories, apparatus and data are adjusted to one another until they form a stable system. Kohler (2002) and Gieryn (2006) examined the border between laboratory and field, showing that researchers move between controlled and naturalistic settings and that some sites are treated as both. From this literature we can name four functions any laboratory must perform, physical or not (Hacking, 1992; Knorr Cetina, 1999; Latour & Woolgar, 1979/1986; Shapin, 1988):

1. Reconfiguration: bringing the object of study into a controlled space where it can be examined.
2. Inscription: turning observations into durable, shareable records.
3. Repetition: running and varying procedures under known conditions.
4. Accountable witnessing: governing who takes part, who judges, and how claims become public.

A laboratory is defined by what it does, not by its walls. An online laboratory has to perform the same functions with infrastructure instead of architecture.

A physical building does much of this work quietly. Locked doors control access, shared benches create common ground, and the host institution supplies ethics review, finance and legal standing. Infrastructure studies remind us that such supports are usually invisible until they break down (Star & Ruhleder, 1996) and that they develop through long, contested processes of design and adoption (Edwards et al., 2007). A fully virtual lab has to build each of these supports on purpose.

2. Labs without walls: what has already been tried

The idea of a laboratory without a building is older than the modern web. Five precedents show what has worked.

2.1 The collaboratory

In 1989 William Wulf proposed the "collaboratory," a center without walls where researchers could work without regard to physical location (Wulf, 1989, 1993). Finholt and Olson (1997) described collaboratories as a new organizational form for scientific collaboration, and two decades of projects followed across the sciences (Finholt, 2002; Olson et al., 2008). Bos et al. (2007) sorted these projects into seven types, from shared instruments and community data systems to virtual communities of practice. The type closest to what we mean here is the distributed research center, a unified research organization whose members work in different places (Bos et al., 2007). Across this body of work, shared tools and data proved easier to build than trust, coordination and incentives (Finholt, 2002; Olson et al., 2008; Sonnenwald, 2007).

2.2 The virtual institute

NASA's Astrobiology Institute, founded in 1998, was designed from the start as a virtual, distributed organization. Funded teams at universities and research centers were linked through NASA's Ames Research Center rather than housed together. By 2018 it involved about 600 researchers at roughly 100 institutions, and it ran for about two decades before its activities were suspended in 2019 (NASA Astrobiology Institute, n.d.). It shows that a major funder can build and sustain a research institute that exists mainly as a network, consistent with the cyberinfrastructure vision set out by Atkins et al. (2003).

2.3 Virtual research environments and science gateways

Virtual research environments give researchers web-based access to shared data, software and computing across institutions (Candela et al., 2013; Dutton & Jeffreys, 2010). Candela et al. (2013) describe them as web-based, community-oriented, flexible and secure working environments, and they identify three recurring challenges: integration and interoperability, sustainability, and adoption. Sociotechnical studies of cyberinfrastructure reach a similar conclusion. Long-term success depends as much on organizational work, funding cycles and community practice as on technology (Edwards et al., 2007; Ribes & Lee, 2010).

2.4 Cloud laboratories and online experiments

Even bench science is moving online. Commercial cloud labs such as Emerald Cloud Lab let scientists design experiments through software while automated instruments and staff run them in a remote facility, and Carnegie Mellon University partnered with the company to build an academic cloud lab (Lab Manager, n.d.). In the behavioral and social sciences, online data collection has been studied for more than two decades (Kraut et al., 2004; Reips, 2002). Horton et al. (2011) argued that online labor markets can serve as an "online laboratory," replicating classic experimental findings online and setting out validity threats specific to that setting. Later work on crowdsourced participant pools reached broadly similar conclusions about data quality while urging careful design (Buhrmester et al., 2011).

2.5 Living labs and independent distributed institutes

Two further models move research into everyday settings and outside universities. Living labs study and co-design innovations with users in real-life contexts (Følstad, 2008; Leminen et al., 2012), and systematic reviews show the term now covers a wide range of practices (Hossain et al., 2019). Independent research institutes such as the Institute for Globally Distributed Open Research and Education (IGDORE) affiliate researchers outside traditional universities and commit them to a shared code of research conduct and open science practices (IGDORE, n.d.). Both show that research credibility can rest on transparent governance and practice, not only on a campus (Nosek et al., 2015).

3. What the evidence says about distance

Research on distributed work gives a virtual lab both a warning and a design brief.

The warning comes from Cummings and Kiesler (2005). In their study of collaborative research projects, those involving more universities were significantly less well coordinated and reported fewer positive outcomes, while spanning more disciplines carried no comparable penalty. Organizational distance, not intellectual distance, was the problem. Reviews of virtual teams point the same way. Trust, shared understanding and coordination are recurring challenges in distributed teams (Gilson et al., 2015; Hinds & Kiesler, 2002), and early trust in global virtual teams depends heavily on how members communicate in the first weeks of working together (Jarvenpaa & Leidner, 1999).

The design brief comes from Olson and Olson (2000), who identified four conditions for successful distance work, later developed in their book on collaboration over the internet (Olson & Olson, 2014):

• Common ground: shared knowledge that people know they share.
• Loosely coupled work: tasks that do not need constant, ambiguous back-and-forth.
• Collaboration readiness: people who are motivated and rewarded to share.
• Technology readiness: teams already fluent in the tools they depend on.

A physical lab supplies much of this by default. A virtual lab has to produce it deliberately, through documentation, onboarding, clear roles and well-designed rhythms of work (National Research Council, 2015; Olson & Olson, 2014; Stokols et al., 2008). That is the central design problem of an online laboratory.

4. A reference architecture for a fully virtual, fundable lab

Bringing these strands together, we use the following working definition:

An online laboratory is a research organization without a shared physical site that performs the four functions of a laboratory through deliberately designed infrastructure, and that meets the legal, ethical and financial standards required to receive and manage research funding.

We organize that infrastructure as seven layers, building on models of virtual research environments (Candela et al., 2013), research infrastructure (Edwards et al., 2007; Star & Ruhleder, 1996) and distributed collaboration (Olson & Olson, 2000). People and roles run through all of them.

• Layer 7. Dissemination and accountability: preprints, open access publication, public research briefs, and reporting of what did not work.
• Layer 6. Methods and instruments: validated measures, study protocols, and intervention methods designed for distributed settings.
• Layer 5. Collaboration and coordination: asynchronous documentation, a lab handbook, decision logs, and a meeting rhythm that builds common ground.
• Layer 4. Compute and tooling: secure cloud workspaces, version control, reproducible analysis pipelines, and survey and interview platforms.
• Layer 3. Data architecture: storage, access control, encryption, retention, and FAIR-aligned sharing.
• Layer 2. Ethics and research integrity: ethics review, participant protections, conflict-of-interest and authorship policies.
• Layer 1. Legal, financial and funding foundation: legal entity, registrations, financial controls and cost policies.

4.1 Legal, financial and funding foundation

Funders award money to organizations, so a virtual lab needs a legal entity and the administrative systems a sponsor expects. Eligibility depends on entity type and program. The National Science Foundation, for example, accepts proposals from non-profit, non-academic organizations, including independent research laboratories, while for-profit organizations generally take part as subawardees (National Science Foundation, n.d.). The federal Uniform Guidance, revised in 2024, sets the cost and administrative rules for most federal awards (Office of Management and Budget, 2024). Among other changes, organizations without a negotiated indirect cost rate may now use a de minimis rate of 15% of modified total direct costs on awards made on or after October 1, 2024 (Cost Principles, 2024; National Institutes of Health, 2024). A fundable lab also needs budgeting, timekeeping, procurement and audit-ready records (Office of Management and Budget, 2024). Partnering with universities, as lead or subawardee, is often the fastest route to early funding, though multi-institution projects carry real coordination costs (Cummings & Kiesler, 2005).

4.2 Ethics and research integrity

A virtual lab that studies people needs the same protections as any university. U.S. federally supported human subjects research falls under the Common Rule, which requires review by an institutional review board for non-exempt research and sets standards for informed consent (Protection of Human Subjects, 2018). Institutions engaged in such research supported by HHS, and by agencies that use the same system, must hold a Federalwide Assurance from the Office for Human Research Protections (Office for Human Research Protections, n.d.). A virtual lab can meet the review requirement through a university reliance agreement or an independent IRB. Internet-mediated research raises its own questions about consent, privacy and the status of online traces, which the Association of Internet Researchers addresses in its ethical guidelines (franzke et al., 2020). In a virtual lab, consent also has to cover recordings, screen captures and digital records of participants' work. Around all of this sit written policies on conflicts of interest, authorship and research misconduct (Munafò et al., 2017; Nosek et al., 2015).

4.3 Data architecture

Data management is where many virtual projects fail quietly. Data are not natural objects; their value depends on the infrastructure, documentation and community practices around them (Borgman, 2015). Since January 25, 2023, NIH has required a Data Management and Sharing Plan for applications that generate scientific data, and the approved plan becomes a term of the award (National Institutes of Health, 2020). The FAIR principles (findable, accessible, interoperable and reusable) give a widely adopted design standard (Wilkinson et al., 2016). In practice this means tiered storage that separates identifiable from de-identified data, role-based access, encryption at rest and in transit, a retention schedule, and deposit of shareable data and code in repositories that issue persistent identifiers (Borgman, 2015; Wilkinson et al., 2016). Labs that handle controlled unclassified information under federal contracts must also meet NIST security requirements for nonfederal systems (Ross & Pillitteri, 2024).

4.4 Compute and tooling

The lab bench of a virtual lab is its software stack. That stack includes secure, institution-controlled workspaces; version control for code, protocols and analysis; reproducible analysis environments; and platforms for surveys, experiments, interviews and transcription that meet the lab's data and consent requirements. Practical guidance on reproducible computing recommends tracking every analysis step, versioning code and data, and automating repeated tasks (Sandve et al., 2013; Wilson et al., 2017). Candela et al. (2013) warn that integration and sustainability, not tool choice, decide whether such environments last, a point echoed in sociotechnical studies of cyberinfrastructure (Ribes & Lee, 2010). Fewer, well-integrated tools are better than many disconnected ones.

4.5 Collaboration and coordination

This layer answers the distance literature directly (Cummings & Kiesler, 2005; Olson & Olson, 2000). It includes a written lab handbook, onboarding that builds common ground quickly, clear ownership of each work stream, decision logs, and a predictable rhythm of synchronous and asynchronous work. Work is broken into loosely coupled pieces wherever possible (Olson & Olson, 2000), early interactions are designed to build trust (Jarvenpaa & Leidner, 1999), and team science practices such as shared goals, explicit roles and agreed authorship rules are written down rather than assumed (National Research Council, 2015; Stokols et al., 2008). The aim is to make coordination visible and cheap.

4.6 Methods and instruments

A virtual lab needs methods suited to studying distributed activity. The method FCL intends to draw on as its research develops is the Change Laboratory, a formative intervention developed within cultural-historical activity theory (Engeström, 1987; Engeström et al., 1996). Practitioners and researchers examine records of current work, often called mirror data, trace disturbances to deeper contradictions in the activity system, and design and test new ways of working (Engeström, 2001; Virkkunen & Newnham, 2013). In formative interventions, participants shape the design and the outcome is not fixed in advance (Sannino et al., 2016). Bligh and Flood (2015) show the method's promise in higher education and caution that its recommended form, including regular face-to-face sessions and dedicated space, can clash with institutional realities. For a virtual lab, that means adapting sessions, mirror data and follow-up for teams that never share a room, and studying those adaptations as research in their own right. Where the lab also runs surveys or online experiments, established standards for internet-based research apply (Kraut et al., 2004; Reips, 2002).

4.7 Dissemination and accountability

Shapin's (1988) point about witnessing still applies. A virtual lab earns credibility by making its work inspectable. Open science guidelines call for transparency in data, code, materials and design, and for preregistration where appropriate (Nosek et al., 2015). Broader reform proposals add independent checks, reporting of null results and incentives for reproducibility (Munafò et al., 2017). For FCL, that means preprints and open-access papers, data and code shared under each study's data plan, persistent identifiers for people and outputs (Wilkinson et al., 2016), and public reporting of failed interventions alongside successful ones. This is also the main safeguard against the self-vindication Hacking (1992) describes.

4.8 People and roles

Even a small virtual lab needs these roles filled, sometimes by the same person: a principal investigator, a research operations or grants manager, a data steward, an ethics and compliance lead, and technical support for the research stack. Team science research shows that unclear roles and authorship expectations are a common source of conflict, so naming the roles makes clear who carries each layer (National Research Council, 2015; Sonnenwald, 2007; Stokols et al., 2008).

5. A fundability checklist

Sponsors and reviewers rarely ask whether a lab has a building. They ask whether it can be trusted with money, data and people (National Science Foundation, n.d.; Office of Management and Budget, 2024). A fully virtual lab is ready to compete for funding when it can say yes to the following:

• The lab is a legal entity, registered in SAM.gov with a Unique Entity Identifier, and eligible under the target program's rules (National Science Foundation, n.d.).
• It has written financial controls, a cost policy and a plan for indirect costs (Cost Principles, 2024; Office of Management and Budget, 2024).
• It has an ethics review route and, where required, a Federalwide Assurance (Office for Human Research Protections, n.d.; Protection of Human Subjects, 2018).
• It has a data management and sharing plan template aligned with sponsor policy and FAIR principles (National Institutes of Health, 2020; Wilkinson et al., 2016).
• Its tools are secure, documented and reproducible (Ross & Pillitteri, 2024; Sandve et al., 2013; Wilson et al., 2017).
• It has a lab handbook covering roles, decisions, authorship and conduct (National Research Council, 2015; Stokols et al., 2008).
• It has a track record of outputs: briefs, preprints, data sets or pilot results (Nosek et al., 2015).
• It has partners, such as universities, community organizations or employers, who can co-apply or host participants (Cummings & Kiesler, 2005).

This checklist is for planning only and is not legal or financial advice. Requirements vary by sponsor and program, so confirm details in each funding announcement.

6. Open problems

A fully virtual lab solves some problems and creates others.

• Sustainability. Virtual infrastructures often outlive their first grant only if someone plans for maintenance and cost from the start (Candela et al., 2013; Edwards et al., 2007; Ribes & Lee, 2010).
• Coordination costs. Organizational distance carries a measurable penalty (Cummings & Kiesler, 2005), and building trust at a distance takes deliberate effort (Gilson et al., 2015; Jarvenpaa & Leidner, 1999).
• Presence and depth. In remote and virtual laboratories, the sense of contact with real phenomena shapes how people learn (de Jong et al., 2013; Ma & Nickerson, 2006). Intervention research that depends on shared attention may face the same limit.
• Compliance load. A small virtual lab carries the same ethics, data and financial obligations as a large university, without a university's support offices (Office of Management and Budget, 2024; Protection of Human Subjects, 2018).
• Self-vindication. A lab that builds its own instruments, data and theory risks fitting them together too neatly (Hacking, 1992). Publishing failures and inviting outside review are the main safeguards (Munafò et al., 2017; Nosek et al., 2015).

7. What this means for FCL

This architecture is the blueprint we are building Futures Change Laboratory toward. Our research subject and our working conditions are the same: we study distributed work and learning as a distributed organization. In Gieryn's (2006) terms, FCL is both laboratory and field. That is a reason to be strict about method, ethics and transparency (Bligh & Flood, 2015; Nosek et al., 2015; Sannino et al., 2016).

In the coming months we will publish notes on each layer as we build it, including the decisions we make, the tools we try and what does not work. If you work, lead or teach in distributed settings and would like to take part in our research, we would like to hear from you.

References

Atkins, D. E., Droegemeier, K. K., Feldman, S. I., Garcia-Molina, H., Klein, M. L., Messerschmitt, D. G., Messina, P., Ostriker, J. P., & Wright, M. H. (2003). Revolutionizing science and engineering through cyberinfrastructure: Report of the National Science Foundation Blue-Ribbon Advisory Panel on Cyberinfrastructure. National Science Foundation.

Bligh, B., & Flood, M. (2015). The Change Laboratory in higher education: Research-intervention using activity theory. In J. Huisman & M. Tight (Eds.), Theory and method in higher education research (pp. 141–168). Emerald Group Publishing. https://eprints.lancs.ac.uk/id/eprint/74672/

Borgman, C. L. (2015). Big data, little data, no data: Scholarship in the networked world. MIT Press.

Bos, N., Zimmerman, A., Olson, J., Yew, J., Yerkie, J., Dahl, E., & Olson, G. (2007). From shared databases to communities of practice: A taxonomy of collaboratories. Journal of Computer-Mediated Communication, 12(2), 652–672. https://doi.org/10.1111/j.1083-6101.2007.00343.x

Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5. https://doi.org/10.1177/1745691610393980

Candela, L., Castelli, D., & Pagano, P. (2013). Virtual research environments: An overview and a research agenda. Data Science Journal, 12. https://doi.org/10.2481/dsj.GRDI-013

Cost Principles, 2 C.F.R. § 200.414 (2024).

Cummings, J. N., & Kiesler, S. (2005). Collaborative research across disciplinary and organizational boundaries. Social Studies of Science, 35(5), 703–722. https://doi.org/10.1177/0306312705055535

de Jong, T., Linn, M. C., & Zacharia, Z. C. (2013). Physical and virtual laboratories in science and engineering education. Science, 340(6130), 305–308. https://doi.org/10.1126/science.1230579

Dutton, W. H., & Jeffreys, P. W. (Eds.). (2010). World wide research: Reshaping the sciences and humanities. MIT Press.

Edwards, P. N., Jackson, S. J., Bowker, G. C., & Knobel, C. P. (2007). Understanding infrastructure: Dynamics, tensions, and design [Report of a workshop on history and theory of infrastructure]. University of Michigan. https://hdl.handle.net/2027.42/49353

Engeström, Y. (2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work, 14(1), 133–156. https://doi.org/10.1080/13639080020028747

Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach to developmental research. Orienta-Konsultit.

Engeström, Y., Virkkunen, J., Helle, M., Pihlaja, J., & Poikela, R. (1996). The Change Laboratory as a tool for transforming work. Lifelong Learning in Europe, 1(2), 10–17.

Finholt, T. A. (2002). Collaboratories. Annual Review of Information Science and Technology, 36, 74–107.

Finholt, T. A., & Olson, G. M. (1997). From laboratories to collaboratories: A new organizational form for scientific collaboration. Psychological Science, 8(1), 28–36. https://doi.org/10.1111/j.1467-9280.1997.tb00540.x

Følstad, A. (2008). Living labs for innovation and development of information and communication technology: A literature review. The Electronic Journal for Virtual Organizations and Networks, 10, 99–131.

franzke, a. s., Bechmann, A., Zimmer, M., Ess, C., & the Association of Internet Researchers. (2020). Internet research: Ethical guidelines 3.0. Association of Internet Researchers. https://aoir.org/reports/ethics3.pdf

Gieryn, T. F. (2006). City as truth-spot: Laboratories and field-sites in urban studies. Social Studies of Science, 36(1), 5–38. https://doi.org/10.1177/0306312705054526

Gilson, L. L., Maynard, M. T., Jones Young, N. C., Vartiainen, M., & Hakonen, M. (2015). Virtual teams research: 10 years, 10 themes, and 10 opportunities. Journal of Management, 41(5), 1313–1337. https://doi.org/10.1177/0149206314559946

Hacking, I. (1992). The self-vindication of the laboratory sciences. In A. Pickering (Ed.), Science as practice and culture (pp. 29–64). University of Chicago Press.

Hey, T., Tansley, S., & Tolle, K. (Eds.). (2009). The fourth paradigm: Data-intensive scientific discovery. Microsoft Research.

Hinds, P. J., & Kiesler, S. (Eds.). (2002). Distributed work. MIT Press.

Horton, J. J., Rand, D. G., & Zeckhauser, R. J. (2011). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14(3), 399–425. https://doi.org/10.1007/s10683-011-9273-9

Hossain, M., Leminen, S., & Westerlund, M. (2019). A systematic review of living lab literature. Journal of Cleaner Production, 213, 976–988. https://doi.org/10.1016/j.jclepro.2018.12.257

IGDORE. (n.d.). Institute for Globally Distributed Open Research and Education [Zenodo community]. Retrieved October 21, 2026, from https://zenodo.org/communities/igdore/about

Jarvenpaa, S. L., & Leidner, D. E. (1999). Communication and trust in global virtual teams. Organization Science, 10(6), 791–815. https://doi.org/10.1287/orsc.10.6.791

Jirotka, M., Lee, C. P., & Olson, G. M. (2013). Supporting scientific collaboration: Methods, tools and concepts. Computer Supported Cooperative Work, 22(4–6), 667–715. https://doi.org/10.1007/s10606-012-9184-0

Knorr Cetina, K. (1999). What is a laboratory? In Epistemic cultures: How the sciences make knowledge (pp. 26–45). Harvard University Press.

Kohler, R. E. (2002). Landscapes and labscapes: Exploring the lab–field border in biology. University of Chicago Press.

Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological research online: Report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet. American Psychologist, 59(2), 105–117. https://doi.org/10.1037/0003-066X.59.2.105

Lab Manager. (n.d.). This lab allows scientists to conduct experiments remotely at any time. Retrieved October 21, 2026, from https://www.labmanager.com/labs-less-ordinary/this-lab-allows-scientists-to-conduct-experiments-remotely-at-any-time-27609

Latour, B., & Woolgar, S. (1986). Laboratory life: The construction of scientific facts (2nd ed.). Princeton University Press. (Original work published 1979)

Leminen, S., Westerlund, M., & Nyström, A.-G. (2012). Living labs as open-innovation networks. Technology Innovation Management Review, 2(9), 6–11.

Ma, J., & Nickerson, J. V. (2006). Hands-on, simulated, and remote laboratories: A comparative literature review. ACM Computing Surveys, 38(3). https://doi.org/10.1145/1132960.1132961

Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, Article 0021. https://doi.org/10.1038/s41562-016-0021

NASA Astrobiology Institute. (n.d.). About the NAI. National Aeronautics and Space Administration. Retrieved October 21, 2026, from https://astrobiology.nasa.gov/nai/about/

National Institutes of Health. (2020). Final NIH policy for data management and sharing (Notice No. NOT-OD-21-013). https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-013.html

National Institutes of Health. (2024). NIH implementation of uniform administrative requirements for federal financial assistance (Notice No. NOT-OD-25-059). https://grants.nih.gov/grants/guide/notice-files/NOT-OD-25-059.html

National Research Council. (2015). Enhancing the effectiveness of team science. The National Academies Press. https://doi.org/10.17226/19007

National Science Foundation. (n.d.). Proposal & award policies & procedures guide (PAPPG), Chapter I.E: Who may submit proposals. Retrieved October 21, 2026, from https://www.nsf.gov/policies/pappg

Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., … Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374

Office for Human Research Protections. (n.d.). Register IRBs and obtain FWAs. U.S. Department of Health and Human Services. Retrieved October 21, 2026, from https://www.hhs.gov/ohrp/register-irbs-and-obtain-fwas/index.html

Office of Management and Budget. (2024). Guidance for federal financial assistance. Federal Register, 89(78), 30046. https://www.federalregister.gov/documents/2024/04/22/2024-07496/guidance-for-federal-financial-assistance

Olson, G. M., & Olson, J. S. (2000). Distance matters. Human-Computer Interaction, 15(2), 139–178. https://doi.org/10.1207/S15327051HCI1523_4

Olson, G. M., Zimmerman, A., & Bos, N. (Eds.). (2008). Scientific collaboration on the Internet. MIT Press.

Olson, J. S., & Olson, G. M. (2014). Working together apart: Collaboration over the Internet. Morgan & Claypool.

Protection of Human Subjects, 45 C.F.R. pt. 46 (2018).

Reips, U.-D. (2002). Standards for Internet-based experimenting. Experimental Psychology, 49(4), 243–256. https://doi.org/10.1026//1618-3169.49.4.243

Ribes, D., & Lee, C. P. (2010). Sociotechnical studies of cyberinfrastructure and e-research: Current themes and future trajectories. Computer Supported Cooperative Work, 19(3–4), 231–244. https://doi.org/10.1007/s10606-010-9120-0

Ross, R., & Pillitteri, V. (2024). Protecting controlled unclassified information in nonfederal systems and organizations (NIST Special Publication 800-171r3). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-171r3

Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013). Ten simple rules for reproducible computational research. PLOS Computational Biology, 9(10), Article e1003285. https://doi.org/10.1371/journal.pcbi.1003285

Sannino, A., Engeström, Y., & Lemos, M. (2016). Formative interventions for expansive learning and transformative agency. Journal of the Learning Sciences, 25(4), 599–633. https://doi.org/10.1080/10508406.2016.1204547

Shapin, S. (1988). The house of experiment in seventeenth-century England. Isis, 79(3), 373–404. https://doi.org/10.1086/354773

Sonnenwald, D. H. (2007). Scientific collaboration. Annual Review of Information Science and Technology, 41(1), 643–681. https://doi.org/10.1002/aris.2007.1440410121

Star, S. L., & Ruhleder, K. (1996). Steps toward an ecology of infrastructure: Design and access for large information spaces. Information Systems Research, 7(1), 111–134. https://doi.org/10.1287/isre.7.1.111

Stokols, D., Hall, K. L., Taylor, B. K., & Moser, R. P. (2008). The science of team science: Overview of the field and introduction to the supplement. American Journal of Preventive Medicine, 35(2, Suppl.), S77–S89. https://doi.org/10.1016/j.amepre.2008.05.002

Virkkunen, J., & Newnham, D. S. (2013). The Change Laboratory: A tool for collaborative development of work and education. Sense Publishers. https://doi.org/10.1007/978-94-6209-326-3

Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, Article 160018. https://doi.org/10.1038/sdata.2016.18

Wilson, G., Bryan, J., Cranston, K., Kitzes, J., Nederbragt, L., & Teal, T. K. (2017). Good enough practices in scientific computing. PLOS Computational Biology, 13(6), Article e1005510. https://doi.org/10.1371/journal.pcbi.1005510

Wuchty, S., Jones, B. F., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science, 316(5827), 1036–1039. https://doi.org/10.1126/science.1136099

Wulf, W. A. (1993). The collaboratory opportunity. Science, 261(5123), 854–855.

Wulf, W. A. (1989). The national collaboratory: A white paper. In Towards a national collaboratory (Unpublished report of a National Science Foundation invitational workshop). Rockefeller University.