Styling

​

Basics

Use the inference endpoint to perform rewriting with your fine-tuned model. The inference endpoint has two modes: full and rewrite

​

Full Mode

Full mode performs inference from a standard AI model such as gpt-4o or Claude 3.7 Sonnet and then styles the output of the AI with your fine-tuned model. This provides a full end-to-end inference process that is similar to using the OpenAI or Anthropic API.

Specify the fine-tuned model ID in model.

Provide the list of messages in the converations in messages.

The full_mode_options parameter enables setting base_model and base_temperature to specify the model and temperature for the base AI model for inference.

​

Rewrite Mode

Rewrite mode uses your fine-tuned model to rewrite text that you provide it. In this mode, you will have generated text using an AI model separately, and you submit this AI-generated text to our endpoint to be rewritten.

Specify the fine-tuned model ID in model.

Provide the input text as a string in message.

No conversational turns or instructions to the AI need to be submitted.

If you have a large block of multi-paragraph text, include all of it in a single API call. You don’t need to segment the text into separate calls.

The styled content is returned at response["choices"][0]["message"]["content"]

​

Streaming

Set the stream option to True to enable streaming responses. Streaming can be used in full or rewrite mode.