Skip to main content
Configuration for text generation

​
Interface Definition

interface TextGenerationConfig {
    frequency_penalty?: string | number;
    logit_bias?: {
        bias_value: string | number;
        token_id: string;
    }[];
    max_new_tokens?: string | number;
    max_prompt_length?: string | number;
    presence_penalty?: string | number;
    repetition_penalty?: string | number;
    seed?: string | number;
    stop_sequences?: string[];
    temperature?: string | number;
    top_p?: string | number;
}

​
Properties

​
frequency_penalty (optional)

frequency_penalty?: string | number
Frequency penalty

​
logit_bias (optional)

logit_bias?: {
    bias_value: string | number;
    token_id: string;
}[]
Logit bias to modify token likelihood Type declaration:
  • bias_value: string | number - Bias value from -100 to 100
  • token_id: string - ID of the token

​
max_new_tokens (optional)

max_new_tokens?: string | number
Maximum number of tokens to generate

​
max_prompt_length (optional)

max_prompt_length?: string | number
Maximum length of the prompt in tokens

​
presence_penalty (optional)

presence_penalty?: string | number
Presence penalty

​
repetition_penalty (optional)

repetition_penalty?: string | number
Repetition penalty

​
seed (optional)

seed?: string | number
Random seed for controlling the randomness of text generation

​
stop_sequences (optional)

stop_sequences?: string[]
List of sequences to stop generation

​
temperature (optional)

temperature?: string | number
Controls randomness of generated text

​
top_p (optional)

top_p?: string | number
Probability for most probable tokens sampling