Observable Evidence

Canonical definition

Claim status: The distinction between observed indicators and latent constructs is grounded in construct-validity scholarship, while the use of digital traces and relational data draws on computational social science. The CSI evidence taxonomy and qualification requirements are research-program propositions informed by Cronbach and Meehl (1955), Messick (1995), and Lazer et al. (2009).

The role of evidence in CSI

CSI reasons from observable traces to a latent collective-level construct. The evidence may be directly recorded, computationally derived, or supplied through structured assessment, but its meaning depends on theory, boundary, time, and context. This separation between indicators and construct interpretation is central to construct validation (Cronbach & Meehl, 1955; Messick, 1995).

The same trace can support different interpretations. Reduced message volume might indicate disengagement, efficient coordination, task completion, or migration to another channel. CSI therefore requires evidence to be interpreted as part of a justified inferential chain rather than treated as a self-explanatory signal.

Requirements for defensible evidence

Major evidence categories

Computational social science demonstrates the analytic value of large-scale behavioral traces, while network scholarship shows that relational structure can carry information not reducible to isolated participant attributes (Lazer et al., 2009; Moody & White, 2003).

Evidence, feature, indicator, and criterion

Evidence is the broad observable basis for inference. A feature is a model-ready representation derived from that evidence. An indicator is a theoretically interpreted sign of the construct. A criterion is an external or independent basis used to evaluate the estimate. The distinctions reflect the broader validation principle that observed measures do not acquire construct meaning without an articulated interpretive argument (Messick, 1995).

Keeping these roles distinct reduces circular validation. A model should not claim success merely because it reproduces the same signal used to define its target.

Triangulation and multimodal evidence

No single channel should be assumed to provide a complete view of a collective state. Surveys may capture perceived experience, communication data may reveal interaction structure, and workflow traces may show coordination behavior. Convergence across independent sources can strengthen an inference; disagreement can reveal uncertainty, subgroup variation, or measurement failure. This logic is consistent with multimethod approaches to convergent and discriminant validation (Campbell & Fiske, 1959).

More data are not automatically better. Multimodal systems increase risks of misalignment, surveillance, missingness, and unequal representation. Each source should add distinct construct-relevant value.

Evidence quality and missingness

• Coverage: determine whose activity and which channels are represented or absent.
• Reliability: assess whether the trace is consistently recorded and measured.
• Validity: test whether the evidence represents the intended theoretical meaning.
• Comparability: identify changes in tools, norms, access, or logging that affect interpretation.
• Missingness: model absence explicitly when missing data may be systematic or meaningful.

Illustrative example

A decline in meeting participation could be treated as evidence of lower engagement. Yet the same period may show faster workflow completion, stable response reciprocity, and a planned shift to asynchronous work. The participation decline alone is ambiguous.

A defensible CSI analysis would examine multiple evidence categories, apply the chosen composition model, account for the contextual change, and preserve uncertainty before estimating engagement.

Ethical and governance requirements

Observable evidence about human interaction can be sensitive even when it is technically accessible. As a normative CSI requirement, the program proposes using the least intrusive evidence sufficient for the research or decision purpose, minimizing identifiable data, documenting consent or lawful authority, securing the data, and limiting downstream use.

Evidence collection should not convert ordinary collaboration into unrestricted surveillance. Where proportionality, transparency, contestability, and accountable oversight cannot be established, CSI should not be attempted. These requirements are CSI governance propositions informed by broader human–AI interaction principles emphasizing transparency, control, and appropriate reliance (Amershi et al., 2019).

Common evidence errors

• Treating an easily measured proxy as though it were the construct itself
• Using sentiment, volume, or a network metric without construct-specific justification
• Combining evidence from different levels or time windows without explicit alignment
• Ignoring participants or channels that are systematically missing
• Using the same signal as both model input and supposedly independent validation criterion
• Collecting intrusive data whose incremental value has not been demonstrated

Relationship to the CSI construct system

The collective establishes the unit, the collective state defines the latent target, emergence explains the higher-level condition, and the composition model specifies how observable evidence relates to that target. CSI integrates the evidence, estimates the state and uncertainty, and evaluates the result through independent collective-level validation.

Selected scholarly foundations

The principal foundations for this article are Cronbach and Meehl (1955) and Messick (1995) on construct meaning and validation, Campbell and Fiske (1959) on multimethod evidence, Lazer et al. (2009) on computational social data, and Moody and White (2003) on relational structure. The evidence taxonomy, adequacy requirements, and governance thresholds are CSI-specific synthesis and propositions.

Research status

RA-006 documents the current CSI position on observable evidence and is aligned with the pre-submission theoretical manuscript. The article remains a foundational draft. Future versions will add construct-specific evidence maps, data-quality tests, privacy-preserving methods, and findings from the planned operationalization program.