You opened the chat window, picked a model, typed a quick “Hi”, and instead of a reply you got a red error. This post explains what that error means, why it happens, and how to fix it in under a minute. The short version: you sent a chat message to a model that does not do chat.
TL;DR
The model you selected is an embedding model. Embedding models turn text into numbers for search; they cannot answer questions. Switch the chat to a Completions model and send the message again.
The error
In the chat interface, sending a message to the model BAAI/bge-small-en-gpu returns this:
[{'error_code': 'INVALID_INFERENCE_REQUEST_FOR_MODEL_TYPE',
'loc': ['body', 'model'],
'msg': None,
'type': None,
'value': 'BAAI/bge-small-en-gpu'}]What it is telling you
Read the error from the inside out. The loc field points at body.model, which is the model you chose. The value is BAAI/bge-small-en-gpu. The error_code is INVALID_INFERENCE_REQUEST_FOR_MODEL_TYPE, which means: the request type you sent does not match the type of this model.
You sent a chat completion request. The selected model is an embedding model. Those are two different jobs, so the server refuses the request instead of returning a wrong answer.
Why it happens
Not every model in the runtime list does the same thing. A name like bge-small-en is a giveaway: BGE is a family of embedding models. An embedding model takes text and returns a vector, a long list of numbers that captures meaning so the system can compare and search documents. It has no concept of a reply, a prompt, or a conversation. When a chat front end asks it to generate a response, there is nothing for it to generate, and you get the error above.
This usually happens for one of three reasons:
- The model picker at the top of the chat was left on an embedding model from an earlier step.
- An embedding model was added to the chat interface by mistake instead of being attached to a knowledge base.
- The wrong model name was passed in an API call to the chat or completions endpoint.
How to fix it
- Open the model selector at the top-left of the chat window.
- Choose a model whose type is Completions (an instruct model such as a Llama or Qwen instruct build). These are the models built to answer.
- Send your message again. It will now go through.
If you actually need the embedding model, it does not belong in the chat box. Attach it to a knowledge base in the Data Indexing and Retrieval area. There it runs behind the scenes, turning your documents and questions into vectors so the chat model can retrieve the right context. You never talk to it directly.
Quick check: in the model runtime list, look at the Type column. If it says Embeddings, the model cannot be used for chat. If it says Completions, it can.
Which model type does which job
Use this table to match the model to the task and avoid the error next time.
| Model type | What it does | Can you chat with it? | Where it belongs |
|---|---|---|---|
| Completions / Chat | Generates text replies from a prompt. | Yes | The chat window and agents. |
| Vision-language | Generates text replies from an image plus a prompt. | Yes | The chat window, with image input. |
| Embeddings | Turns text into vectors for semantic search. | No | Attached to a knowledge base for retrieval. |
| Reranker | Scores how well a passage answers a query. | No | The retrieval pipeline, after the first search. |
| Speech (ASR) | Converts audio into text. | No | A transcription step before indexing. |
| OCR | Converts text in images into characters. | No | Document ingestion before indexing. |
How to prevent it
- Read the model name. If it contains
embed,bge,rerank,ocr, orasr, it is not a chat model. - Confirm the Type column before you select a model for chat.
- Keep embedding and rerank models wired to knowledge bases, not exposed in the chat picker.
- In API calls, send embedding models to the embeddings endpoint and chat models to the chat or completions endpoint.
The error looks alarming, but it is the platform protecting you from a meaningless result. Pick a Completions model for chatting, leave embedding models to power search behind the scenes, and the message goes through.
Troubleshooting note for the Private AI series. Vendor product names belong to their respective owners.
About The Author
DrJha