Unlocking Transfer Learning with LLMs

We are building a support-ticket classifier that adapts a general-purpose LLM to a custom company taxonomy through in-context transfer learning. Instead of fine-tuning weights, we embed labeled examples directly into the system prompt and let the model generalize from them. For teams that handle hundreds of tickets daily, this approach ships in an afternoon without GPU training jobs.

What you'll need

Before we start, make sure you have Python 3.10 or newer installed. You will also need the OpenAI SDK, which acts as a drop-in client for Oxlo.ai because the platform is fully OpenAI compatible. Finally, grab an API key from the Oxlo.ai developer portal.

• Python 3.10+
• pip install openai
• An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Configure the Oxlo.ai client

The OpenAI SDK connects to Oxlo.ai with a single parameter change. Point the base URL at Oxlo.ai and load your key from an environment variable so you do not accidentally commit secrets.

# classifier.py
from openai import OpenAI
import os

# Oxlo.ai is fully OpenAI compatible, so we reuse the official SDK.
# Change nothing except the base_url and api_key.
client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY"),
)

This single client instance is all you need for the rest of the script. Because Oxlo.ai supports streaming, JSON mode, and function calling on this endpoint, you can extend the classifier later without swapping libraries.

Step 2: Define the labeled dataset

Our transfer learning material is a list of real support tickets mapped to categories and priorities. I collected twenty examples covering four categories and three priority levels so the model can pick up subtle linguistic patterns, like the difference between a polite suggestion and an urgent breakage report.

# Each dictionary acts as a labeled training example.
# We will inject these into the system prompt so the model learns the mapping
# without any weight updates or fine-tuning infrastructure.
EXAMPLES = [
    {"ticket": "My invoice shows a duplicate charge for March.", "category": "Billing", "priority": "High"},
    {"ticket": "The export to CSV button does nothing when I click it.", "category": "Technical Bug", "priority": "High"},
    {"ticket": "It would be great if you supported dark mode.", "category": "Feature Request", "priority": "Low"},
    {"ticket": "I forgot my password and the reset email never arrives.", "category": "Account Access", "priority": "High"},
    {"ticket": "Can I get a refund for last month's unused credits?", "category": "Billing", "priority": "Medium"},
    {"ticket": "The API returns a 500 error every morning around 9 AM.", "category": "Technical Bug", "priority": "High"},
    {"ticket": "Please add webhook support for completed tasks.", "category": "Feature Request", "priority": "Low"},
    {"ticket": "My account is locked after too many failed logins.", "category": "Account Access", "priority": "High"},
    {"ticket": "I was charged twice for the same subscription.", "category": "Billing", "priority": "High"},
    {"ticket": "Images uploaded via mobile are rotated 90 degrees.", "category": "Technical Bug", "priority": "Medium"},
    {"ticket": "A native iOS app would really improve my workflow.", "category": "Feature Request", "priority": "Low"},
    {"ticket": "I need to change the email associated with my account.", "category": "Account Access", "priority": "Medium"},
    {"ticket": "The receipt PDF is blank when I download it.", "category": "Billing", "priority": "Medium"},
    {"ticket": "Search results are missing recent entries.", "category": "Technical Bug", "priority": "Medium"},
    {"ticket": "Add two-factor authentication for extra security.", "category": "Feature Request", "priority": "Low"},
    {"ticket": "My teammate cannot join our organization invite.", "category": "Account Access", "priority": "Medium"},
    {"ticket": "Sales tax on my invoice is calculated incorrectly.", "category": "Billing", "priority": "Medium"},
    {"ticket": "The dashboard freezes in Safari after five minutes.", "category": "Technical Bug", "priority": "High"},
    {"ticket": "Let me pin projects to the top of the sidebar.", "category": "Feature Request", "priority": "Low"},
    {"ticket": "I accidentally deleted my account, can it be restored?", "category": "Account Access", "priority": "High"},
]

I intentionally picked twenty examples because that is the volume where token-based pricing starts to hurt on every request. Oxlo.ai uses flat per-request pricing, so you can optimize for accuracy instead of token economy. See the pricing page for details.

Step 3: Build the system prompt

This prompt transfers the model's general language understanding into our specific domain. It contains the taxonomy rules and the full set of examples, turning the context window into a lightweight adaptation layer.

# The system prompt embeds our entire labeled dataset.
# This is the core transfer learning step: we are not updating model weights.
# Instead, we are repurposing the model's pre-trained knowledge through
# few-shot in-context learning, which works best when long prompts are cheap.
SYSTEM_PROMPT = """You are a support ticket classifier. Your job is to read a user message and output exactly two lines:
Category: 
Priority: 

Use the following examples to learn the mapping. Be concise and follow the format exactly.

Examples:
""" + "\n".join([
    f"Ticket: {ex['ticket']}\nCategory: {ex['category']}\nPriority: {ex['priority']}"
    for ex in EXAMPLES
]) + "\n\nNow classify the new ticket provided by the user."

Notice that we are not updating model weights or running gradient descent. All the adaptation happens inside the context window at inference time, which is why flat per-request pricing is a strategic fit for this architecture.

Step 4: Create the classifier function

The function sends the system prompt plus an unseen ticket to Llama 3.3 70B. I keep temperature low so the output format stays consistent and predictable, which makes parsing trivial.

def classify_ticket(user_message: str) -> dict:
    # We use Llama 3.3 70B because it follows structured instructions reliably.
    # Oxlo.ai serves this model with no cold starts, so latency stays low.
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.2,
        max_tokens=60,
    )
    
    # Parse the rigid two-line format instead of forcing JSON.
    # This keeps token usage minimal and avoids schema overhead.
    raw = response.choices[0].message.content.strip()
    lines = [line.strip() for line in raw.splitlines() if line.strip()]
    
    result = {}
    for line in lines:
        if line.startswith("Category:"):
            result["category"] = line.split(":", 1)[1].strip()
        elif line.startswith("Priority:"):
            result["priority"] = line.split(":", 1)[1].strip()
    return result

Keeping the temperature at 0.2 strikes a balance between creativity and determinism. If you find the model occasionally hallucinates a fifth category, you can add a logit bias or switch to JSON mode, both supported by Oxlo.ai.

Step 5: Evaluate on a batch of unseen tickets

To verify that the transfer actually generalizes, we classify five new tickets that were not in the training set. These cover edge cases that often confuse zero-shot classifiers, such as a credit card form timeout that looks like billing but is actually an account access issue.

# Unseen tickets that test whether the model learned the taxonomy
# rather than simply memorizing the examples.
TEST_TICKETS = [
    "I tried to update my credit card but the form keeps timing out.",
    "Could you integrate with Slack so I get notifications there?",
    "My login session expires every ten minutes and it is frustrating.",
    "I see three identical charges on my statement this month.",
    "The mobile app crashes when I open the settings tab.",
]

if __name__ == "__main__":
    for ticket in TEST_TICKETS:
        prediction = classify_ticket(ticket)
        print(f"Ticket: {ticket}")
        cat = prediction.get("category", "Unknown")
        pri = prediction.get("priority", "Unknown")
        print(f"Predicted: {cat}, {pri}")
        print()

Watching the model generalize these distinctions from just twenty examples is the moment you realize in-context transfer learning is production ready.

Run it

Save the complete script as ticket_classifier.py, export your Oxlo.ai key, and execute it from your terminal. The batch completes in under a couple of seconds because Oxlo.ai keeps popular models warm, so there are no cold starts to spike your webhook latency.

export OXLO_API_KEY="sk-oxlo.ai-..."
python ticket_classifier.py

When I ran this against the live Oxlo.ai API, I got the following output. Your results should match closely because the examples and temperature are fixed.

Ticket: I tried to update my credit card but the form keeps timing out.
Predicted: Account Access, Medium

Ticket: Could you integrate with Slack so I get notifications there?
Predicted: Feature Request, Low

Ticket: My login session expires every ten minutes and it is frustrating.
Predicted: Account Access, Medium

Ticket: I see three identical charges on my statement this month.
Predicted: Billing, High

Ticket: The mobile app crashes when I open the settings tab.
Predicted: Technical Bug, High

Next steps

If you need to scale beyond the context window, swap the few-shot prompt for a retrieval step using Oxlo.ai's embedding models like BGE-Large. Embed your entire example library, search for the nearest neighbors at runtime, and inject only the top matches into the prompt. This keeps latency low while preserving the transfer effect, and it still benefits from flat per-request pricing because the final LLM call does not balloon in cost.

You can also experiment with Qwen 3 32B or DeepSeek V3.2 on Oxlo.ai if your tickets arrive in multiple languages or if you want to test reasoning-heavy classification rules. Both are available through the same client and base URL, so switching models is a one-line change.