oumi.core.configs.internal

Submodules

oumi.core.configs.internal.internal_model_config module

class oumi.core.configs.internal.internal_model_config.InternalFeatureFirstDimAction(value)

Bases: Enum

Enum representing how to handle the first feature dimension in datasets.

DROP_ALWAYS = 'drop_always'

1. and must be dropped.

In effect, this operation is applied: x = x[0, …], which reduces x’s rank by 1 (e.g., 3D->2D), and discards the following elements: x[1:, …].

Type:

The first dimension is commonly dummy (length

DROP_IF_DUMMY = 'drop_if_dummy'

1).

Type:

Drop the first dimension only if it’s dummy (length

KEEP = 'keep'

Always preserve the first dimension.

class oumi.core.configs.internal.internal_model_config.InternalFeatureSpec(name, required, variable_shape, first_dim_action, image_dependent)

Bases: NamedTuple

first_dim_action: InternalFeatureFirstDimAction

Action to apply to the first feature dimension.

image_dependent: bool

Whether the feature depends on image data.

For example, pixel_values, cross_attention_mask.

name: str

Feature name.

required: bool

Whether the feature must be always present (vs optional).

variable_shape: bool

Whether the feature can be of variable shape.

For example, input_ids is normally of variable length.

class oumi.core.configs.internal.internal_model_config.InternalModelConfig(model_type: str = '', chat_template: str = 'default', tokenizer_pad_token: Optional[str] = None, padding_side: Optional[oumi.core.configs.internal.internal_model_config.InternalPaddingSide] = None, model_input_features: dict[str, oumi.core.configs.internal.internal_model_config.InternalFeatureSpec] = <factory>, label_ignore_index: Optional[int] = -100, sanitize_negative_labels: bool = False, processor_kwargs: dict[str, typing.Any] = <factory>, ignore_features: list[str] = <factory>, visual_config: Optional[oumi.core.configs.internal.internal_model_config.InternalVisualModelConfig] = None)

Bases: BaseConfig

chat_template: str = 'default'

Default chat template.

ignore_features: list[str]

Features from processing the input to ignore in the model’s forward method.

label_ignore_index: int | None = -100

Special label value to be excluded from loss computation.

model_input_features: dict[str, InternalFeatureSpec]

Model input features specs.

model_type: str = ''

Model type.

padding_side: InternalPaddingSide | None = None

Padding side for the model.

processor_kwargs: dict[str, Any]

Extra params to pass to processor constructor.

sanitize_negative_labels: bool = False

Replace negative label values.

Some VLM processors can generate negative input_ids for image tokens, which can cause problems if a negative integer is used as a label to compute loss e.g., cross-entropy loss may expect [0, num_classes) range.

tokenizer_pad_token: str | None = None

The default padding token used by the tokenizer.

If specified in internal model type config and unspecified in ModelParams.tokenizer_pad_token, then this value will be used.

visual_config: InternalVisualModelConfig | None = None

Configuration specific to visual models.

class oumi.core.configs.internal.internal_model_config.InternalPaddingSide(value)

Bases: Enum

Enum representing how to do padding for the model.

PAD_LEFT = 'left'

Left padding.

PAD_RIGHT = 'right'

Right padding.

class oumi.core.configs.internal.internal_model_config.InternalVisualModelConfig(main_image_feature: str = 'pixel_values', variable_shape_image_features: bool = False, supports_multiple_images: bool = False)

Bases: BaseConfig

main_image_feature: str = 'pixel_values'

The key corresponding to the main image feature consumed by the model.

E.g., raw pixels, transformed image patches, etc. resulting from data preprocessing and consumed by the underlying model.

supports_multiple_images: bool = False

Whether the visual language model supports multiple images in one prompt.

variable_shape_image_features: bool = False

Whether image features can be of variable shape.

In this case, the image features can be difficult to collate (torch.stack() requires compatible shapes) and some workaround is needed: either require batch_size=1, or group examples so that each mini-batch only contains same-sized features.

oumi.core.configs.internal.supported_models module

Supported models configuration for Oumi framework.

This module defines the configuration for all non-standard models in the Oumi framework, including both language models (LLMs) and vision-language models (VLMs). It provides a centralized registry of model configurations that specify how different models should be handled during training, inference, and evaluation.

Note that most models should work without any special configuration, and therefore do not need to be added to this module.

Key Components:

• InternalModelConfig: Configuration parameters for model behavior

• Model-specific config creators: Functions that create configs for specific models

• Registry functions: For looking up and accessing model configurations

• _ModelTypeInfo: Metadata for each supported model type

How to Add a New Model:

1. Create a configuration function: ```python def _create_my_model_config() -> InternalModelConfig:

   config = InternalModelConfig() # Configure the model’s specific settings config.chat_template = “my_template” # Add any special features or parameters return config

   ```

2. Add the model to get_all_models_map(): ```python _ModelTypeInfo(

   model_type=”my_model”, # Must match HF config.model_type model_class=transformers.AutoModelForCausalLM, # Or appropriate class tested=False, # Set to True once tests are added config=_create_my_model_config(),

3. For VLMs, configure visual features: ```python vlm_config = _create_default_vlm_config(

   supports_multiple_images=True, # If model supports multiple images pixel_values_variable_shape=True, # If images can have different sizes

   ) # Add any model-specific image features vlm_config.model_input_features.update({…}) ```

oumi.core.configs.internal.supported_models.find_internal_model_config(model_params: ModelParams) → InternalModelConfig | None

Finds an internal model config for supported models using ModelParams.

Parameters:

model_params – The model parameters.

Returns:

Model config, or None if model is not recognized.

oumi.core.configs.internal.supported_models.find_internal_model_config_using_model_name(model_name: str, trust_remote_code: bool) → InternalModelConfig | None

Finds an internal model config for supported models using model name.

Parameters:

• model_name – The model name, either: - A HuggingFace model ID (e.g., “meta-llama/Llama-2-7b-hf”) - A local path to a model directory - A custom model name registered in Oumi

• trust_remote_code – Whether to trust external code associated with the model. Required for some models like Qwen2-VL that use custom code.

Returns:

• The model is a custom Oumi model (handled separately)

• The model type is not in the supported models registry

Return type:

InternalModelConfig for the model if it’s supported, or None if

oumi.core.configs.internal.supported_models.find_model_hf_config(model_name: str, *, trust_remote_code: bool, revision: str | None = None, **kwargs: dict[str, Any])

Finds HF model config by model name.

oumi.core.configs.internal.supported_models.get_all_models_map() → Mapping[str, _ModelTypeInfo]

Creates a map of all supported models with their configurations.

This is the central registry of the non-standard models supported by the Oumi framework. Each entry maps a model type (as defined in the HuggingFace model config) to its corresponding configuration and metadata.

Returns:

An immutable mapping from model_type strings to _ModelTypeInfo objects. The mapping includes both LLMs and VLMs with their specific configurations.

oumi.core.configs.internal.supported_models.get_default_vlm_model_config() → InternalModelConfig

Returns default VLM model config.

oumi.core.configs.internal.supported_models.is_custom_model(model_name: str) → bool

Determines whether the model is a custom model defined in oumi registry.