What is ChatOptions

                              What is ChatOptions?


Q1. What is ChatOptions?

Answer

ChatOptions is a configuration object in Spring AI that controls how the underlying language model generates responses.

 

It allows developers to configure parameters such as model selection, temperature, maximum output tokens, sampling behavior, and other provider-supported settings.

 

 Instead of changing the prompt, it changes how the model generates the answer.


Architecture

User
   │
   ▼
ChatClient
   │
   ├── Prompt
   ├── Advisors
   └── ChatOptions
          │
          ▼
     OpenAI / Ollama
          │
          ▼
      AI Response

Notice:

  • Prompt → tells the model what to do.
  • ChatOptions → tells the model how to do it.

Example

String response = chatClient.prompt()
        .user("Explain Kafka")
        .options(OpenAiChatOptions.builder()
                .temperature(0.7)
                .maxTokens(300)
                .build())
        .call()
        .content();

Here:

  • Prompt = "Explain Kafka"
  • ChatOptions = temperature, maxTokens

Why do we need ChatOptions?

Without ChatOptions:

Explain Java

The model uses its default settings.

With ChatOptions:

Temperature = 0

Max Tokens = 100

TopP = 0.8

Now you control the response style.



Common ChatOptions

1. temperature ⭐⭐⭐⭐⭐

Controls creativity/randomness.

. temperature (0)

Output:

Very factual
Very consistent

Best for:

  • Banking
  • Healthcare
  • SQL generation
  • Code generation

. temperature (1.5)

Output:

Creative
Different every time

Best for:

  • Story writing
  • Poems
  • Marketing
  • Brainstorming

Example

Question

Write a birthday wish.

Temperature = 0

Happy Birthday.
I wish you good health and happiness.

Temperature = 1.5

May your birthday sparkle with joy,
laughter, and unforgettable memories...


2. maxTokens ⭐⭐⭐⭐⭐


Limits response length.


.maxTokens(100)


Means


AI cannot generate
more than 100 output tokens.


Useful when:
  • SMS
  • Tweet generation
  • Short answers

    • 





Example
Without
1000 words
With
.maxTokens(50)


2 paragraphs only.

3. frequencyPenalty

Prevents repetition.

Without

Kafka is...
Kafka is...
Kafka is...
Kafka is...

With

.frequencyPenalty(2)

Less repetition.

Useful for

  • Long articles
  • Blogs

4. presencePenalty

Encourages new topics.

Without

Java

Java

Java

Java

With

.presencePenalty(2)

The model is more likely to introduce related ideas like:

Java
Spring Boot
Microservices
JVM

5. stop


Stops generation when a specified sequence appears.


Example


.stop(List.of("END"))


If AI produces
Hello

END

Thank You


Output becomes
Hello
Generation stops at "END".



Top-P and Top-K


Suppose you ask the LLM:


Java is a ______ programming language.


The model predicts the next word with probabilities.


WordProbability
object-oriented40%
popular25%
powerful15%
compiled10%
interpreted5%
banana3%
elephant2%


The total probability is 100%.


The model now has to decide which word to pick.




6. Top K Sampling


Top-K means:
Only consider the top K most probable words.
Suppose
.topK(3)
Then only these 3 words are considered:


WordProbability
object-oriented40%
popular25%
powerful15%


The remaining words are discarded.


object-oriented ✔

popular ✔

powerful ✔

compiled ❌

interpreted ❌

banana ❌

elephant ❌


The model randomly picks one of these 3 according to their probabilities.




Example


If
.topK(1)
Only one word is available.
object-oriented
The answer becomes almost deterministic.




If
.topK(100)
Many words are available.
The response becomes more diverse.

2. Top-P (Nucleus Sampling)


Top-P doesn't look at a fixed number of words.


Instead, it keeps adding the most probable words until their cumulative probability reaches P.


Suppose
.topP(0.80)
Let's calculate:


WordProbabilityCumulative
object-oriented40%40%
popular25%65%
powerful15%80% ✅
compiled10%90%
interpreted5%95%


We stop at 80%.
Only these words remain:
object-oriented

popular

powerful


Everything else is discarded.




Suppose
.topP(0.95)
Now


WordProbabilityCumulative
object-oriented40%40%
popular25%65%
powerful15%80%
compiled10%90%
interpreted5%95% ✅


Now five words are available.
More variety.

Comments

Post a Comment

Popular posts from this blog

Async/await

  Async/await An async function is a function declared with the async keyword, and the await keyword is permitted within them. The async and await keywords enable asynchronous, promise-based behavior to be written in a cleaner style, avoiding the need to explicitly configure promise chains. "async and await make promises easier to write" async  makes a function return a Promise await  makes a function wait for a Promise  Async functions Let’s start with the async keyword. It can be placed before a function, like this: async function f() {   return 1; } The word “async” before a function means one simple thing: a function always returns a promise. Other values are wrapped in a resolved promise automatically. For instance, this function returns a resolved promise with the result of 1; let’s test it: async function f() {   return 1; } f().then(alert); // 1 …We could explicitly return a promise, which would be the same: async function f() {   return Promis...
Read more

Two Sum II - Input Array Is Sorted

 Given a 1-indexed array of integers numbers that is already sorted in non-decreasing order , find two numbers such that they add up to a specific target number. Let these two numbers be numbers[index1] and numbers[index2] where 1 <= index1 < index2 <= numbers.length. Return the indices of the two numbers, index1 and index2, added by one as an integer array [index1, index2] of length 2. The tests are generated such that there is exactly one solution. You may not use the same element twice. Your solution must use only constant extra space .   Example 1: Input: numbers = [2,7,11,15], target = 9 Output: [1,2] Explanation: The sum of 2 and 7 is 9. Therefore, index1 = 1, index2 = 2. We return [1, 2]. Example 2: Input: numbers = [2,3,4], target = 6 Output: [1,3] Explanation: The sum of 2 and 4 is 6. Therefore index1 = 1, index2 = 3. We return [1, 3]. Example 3: Input: numbers = [-1,0], target = -1 Output: [1,2] Explanation: The sum of -1 and 0 is -1. Therefore index1 = 1,...
Read more

Comparable Vs. Comparator in Java

  In Java , Comparable and Comparator both are interfaces used to sort objects. Comparable It defines  natural ordering  within a class. It is implemented  inside  the class being sorted. It uses the compareTo() method. It allows sorting  by one one . Comparable provides a  single sorting sequence . In other words, we can sort the collection on the basis of a single element such as id, name, and price. Comparable  affects the original class , i.e., the actual class is modified. Comparable is present in  java.lang  package. We can sort the list elements of Comparable type by  Collections.sort(List)  method. Only one compareTo() method can be implemented, restricting flexibility. Comparator It defines  custom sorting logic It is implemented  in a separate class  or using lambda expressions . It uses the compare() method. It allows sorting  by multiple criteria . The Comparator provides  multiple sorting...
Read more