Outcomes¶

class sim_panel.outcomes.OutcomeModel(*args, **kwargs)[source]¶

Bases: Protocol

cfg: OutcomeConfig¶

evaluate(*, panelist, ctx)[source]¶

Return type:: OutcomeResult
Parameters:: ctx (EvaluationContext)

class sim_panel.outcomes.OutcomeResult(outcomes, traces, raw_text=None, errors=None)[source]¶

Bases: object

Parameters:

outcomes (Dict[str, Any] | None)
traces (Dict[str, Any] | None)
raw_text (str | None)
errors (list[str] | None)

outcomes: Dict[str, Any] | None¶

traces: Dict[str, Any] | None¶

raw_text: str | None = None¶

errors: list[str] | None = None¶

class sim_panel.outcomes.OutcomeConfig(name, questionnaire, temperature=0.2, max_tokens=None, include_raw_text=True, custom_few_shot_example=None)[source]¶

Bases: object

YAML-governed config for outcomes module.

Parameters:

name (str)
questionnaire (QuestionnaireSpec)
temperature (float)
max_tokens (int | None)
include_raw_text (bool)
custom_few_shot_example (Dict[str, Any] | None)

name: str¶

questionnaire: QuestionnaireSpec¶

temperature: float = 0.2¶

max_tokens: int | None = None¶

include_raw_text: bool = True¶

custom_few_shot_example: Dict[str, Any] | None = None¶

class sim_panel.outcomes.EvaluationContext(panelist_id, product_id, t, product_display, panelist_features=None, product_features=None, extra=None)[source]¶

Bases: object

Minimal context needed to render an evaluation prompt. Keep it JSON-serializable.

Parameters:

panelist_id (str)
product_id (str)
t (int)
product_display (str)
panelist_features (Dict[str, Any] | None)
product_features (Dict[str, Any] | None)
extra (Dict[str, Any] | None)

panelist_id: str¶

product_id: str¶

t: int¶

product_display: str¶

panelist_features: Dict[str, Any] | None = None¶

product_features: Dict[str, Any] | None = None¶

extra: Dict[str, Any] | None = None¶

class sim_panel.outcomes.QuestionnaireSpec(outcome_fields, trace_fields=())[source]¶

Bases: object

A questionnaire defines what the panelist must fill out after evaluation.

outcome_fields define the event[“outcomes”] object.
trace_fields define the event[“traces”] object (optional / free-form-ish).

The model output is expected to be JSON:: {“outcomes”: {…}, “traces”: {…}}

Parameters:

outcome_fields (Tuple[FieldSpec, ...])
trace_fields (Tuple[FieldSpec, ...])

outcome_fields: Tuple[FieldSpec, ...]¶

trace_fields: Tuple[FieldSpec, ...] = ()¶

outcome_names()[source]¶

Return type:: List[str]

trace_names()[source]¶

Return type:: List[str]

validate_payload(payload)[source]¶

Validate a parsed JSON payload against the questionnaire spec. Returns (outcomes, traces, errors).

Return type:: Tuple[Optional[Dict[str, Any]], Optional[Dict[str, Any]], List[str]]
Parameters:: payload (Dict[str, Any])

static from_config_dict(cfg)[source]¶

Build QuestionnaireSpec from a YAML-parsed dict.

Return type:: QuestionnaireSpec
Parameters:: cfg (Mapping[str, Any])

Expected shape:

outcomes:

fields:

<name>:: type: … question: … instruction: … choices: […] required: true/false min: … max: …

traces:

fields:: <name>: …

class sim_panel.outcomes.FieldSpec(name, type, question, instruction=None, choices=None, required=True, min_value=None, max_value=None, analysis_type=None, choice_order=None)[source]¶

Bases: object

A single questionnaire field. Used for both outcomes and traces.

name:

The JSON key that must appear in the model output under outcomes/traces.

type:

Controls prompt rendering and validation.

question:

The user-facing question / instruction to the panelist.

instruction:

Optional additional formatting guidance.

choices:

Optional list of allowed values (strongly recommended for categorical / int with discrete choices).

required:

If False, missing key is allowed. If True, missing is a validation error.

analysis_type:

Optional analysis-facing semantic type. This does not affect prompt rendering or payload validation used by existing outcome modules. It is intended for downstream analysis modules such as regression.

Allowed values: - “continuous” - “binary” - “nominal” - “ordinal”

choice_order:

Optional explicit ordering for ordinal outcomes. If omitted, downstream analysis may fall back to choices if appropriate.

min_value/max_value:

Optional numeric constraints for int/float.

Parameters:

name (str)
type (Literal['int', 'float', 'categorical', 'bool', 'text', 'json'])
question (str)
instruction (str | None)
choices (List[Any] | None)
required (bool)
min_value (float | None)
max_value (float | None)
analysis_type (Literal['continuous', 'binary', 'nominal', 'ordinal'] | None)
choice_order (Tuple[Any] | None)

name: str¶

type: Literal['int', 'float', 'categorical', 'bool', 'text', 'json']¶

question: str¶

instruction: str | None = None¶

choices: List[Any] | None = None¶

required: bool = True¶

min_value: float | None = None¶

max_value: float | None = None¶

analysis_type: Literal['continuous', 'binary', 'nominal', 'ordinal'] | None = None¶

choice_order: Tuple[Any] | None = None¶

validate_value(v)[source]¶

Return type:: Optional[str]
Parameters:: v (Any)

class sim_panel.outcomes.DeterministicOutcomeModel(cfg)[source]¶

Bases: object

Deterministic questionnaire filler for CI/tests.

Strategy:

For categorical / choices fields: pick the first choice deterministically (or hash-index if desired).
For int/float: hash into range if provided, else 0/0.0.
For bool: hash parity.
For text: short templated stub.

Parameters:: cfg (OutcomeConfig)

evaluate(*, panelist, ctx, prompting_strategy='persona')[source]¶

Return type:

OutcomeResult

Parameters:

ctx (EvaluationContext)
prompting_strategy (str)

class sim_panel.outcomes.LLMOutcomeModel(cfg)[source]¶

Bases: object

LLM-backed outcome model.

Renders a questionnaire prompt from YAML spec
Calls panelist.evaluate(…) (persona-endowed)
Extracts JSON and validates against the QuestionnaireSpec

Parameters:: cfg (OutcomeConfig)

evaluate(*, panelist, ctx, prompting_strategy='persona')[source]¶

Return type:

OutcomeResult

Parameters:

ctx (EvaluationContext)
prompting_strategy (str)

sim_panel.outcomes.build_outcome_model(cfg)[source]¶

Return type:: OutcomeModel
Parameters:: cfg (OutcomeConfig)

class sim_panel.outcomes.base.EvaluationContext(panelist_id, product_id, t, product_display, panelist_features=None, product_features=None, extra=None)[source]¶

Bases: object

Minimal context needed to render an evaluation prompt. Keep it JSON-serializable.

Parameters:

panelist_id (str)
product_id (str)
t (int)
product_display (str)
panelist_features (Dict[str, Any] | None)
product_features (Dict[str, Any] | None)
extra (Dict[str, Any] | None)

panelist_id: str¶

product_id: str¶

t: int¶

product_display: str¶

panelist_features: Dict[str, Any] | None = None¶

product_features: Dict[str, Any] | None = None¶

extra: Dict[str, Any] | None = None¶

class sim_panel.outcomes.base.OutcomeResult(outcomes, traces, raw_text=None, errors=None)[source]¶

Bases: object

Parameters:

outcomes (Dict[str, Any] | None)
traces (Dict[str, Any] | None)
raw_text (str | None)
errors (list[str] | None)

outcomes: Dict[str, Any] | None¶

traces: Dict[str, Any] | None¶

raw_text: str | None = None¶

errors: list[str] | None = None¶

class sim_panel.outcomes.base.OutcomeConfig(name, questionnaire, temperature=0.2, max_tokens=None, include_raw_text=True, custom_few_shot_example=None)[source]¶

Bases: object

YAML-governed config for outcomes module.

Parameters:

name (str)
questionnaire (QuestionnaireSpec)
temperature (float)
max_tokens (int | None)
include_raw_text (bool)
custom_few_shot_example (Dict[str, Any] | None)

name: str¶

questionnaire: QuestionnaireSpec¶

temperature: float = 0.2¶

max_tokens: int | None = None¶

include_raw_text: bool = True¶

custom_few_shot_example: Dict[str, Any] | None = None¶

class sim_panel.outcomes.base.OutcomeModel(*args, **kwargs)[source]¶

Bases: Protocol

cfg: OutcomeConfig¶

evaluate(*, panelist, ctx)[source]¶

Return type:: OutcomeResult
Parameters:: ctx (EvaluationContext)

sim_panel.outcomes.registry.build_outcome_model(cfg)[source]¶

Return type:: OutcomeModel
Parameters:: cfg (OutcomeConfig)

sim_panel.outcomes.registry.outcome_config_from_yaml_dict(d)[source]¶

Convenience helper: build OutcomeConfig from a YAML-parsed dict.

Return type:: OutcomeConfig
Parameters:: d (Mapping[str, Any])

Expected shape (suggested):

outcomes_model:

name: llm | deterministic temperature: 0.2 max_tokens: 512 include_raw_text: true

questionnaire:

outcomes:: fields: …
traces:: fields: …