Assign semantic IDs

POST /semantic-ids

Encode items into codes. Use this to backfill a catalog after fitting, to register new items as they go live, or to export semantic IDs into a ranker you already operate.

curl

curl https://api.jeantechnologies.com/v1/semantic-ids \
  -H "Authorization: Bearer $JEAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tokenizer_id": "tok_9k2m",
    "items": [
      {
        "item_id": "sku_771",
        "title": "Merino crew socks",
        "description": "Mid-weight, charcoal.",
        "image_url": "https://acme.com/771.jpg"
      }
    ]
  }'

Python

res = requests.post(
    "https://api.jeantechnologies.com/v1/semantic-ids",
    headers={"Authorization": f"Bearer {os.environ['JEAN_API_KEY']}"},
    json={
        "tokenizer_id": "tok_9k2m",
        "items": [
            {
                "item_id": "sku_771",
                "title": "Merino crew socks",
                "description": "Mid-weight, charcoal.",
            }
        ],
    },
).json()

TypeScript

const res = await fetch("https://api.jeantechnologies.com/v1/semantic-ids", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.JEAN_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    tokenizer_id: "tok_9k2m",
    items: [
      {
        item_id: "sku_771",
        title: "Merino crew socks",
        description: "Mid-weight, charcoal.",
      },
    ],
  }),
});
const { semantic_ids } = await res.json();

Body

Field	Type	Notes
tokenizer_id	string	required
items	object[]	required. Up to 1000 per call.

Each item takes item_id (required), plus optional title, description, image_url, and attributes (catalog attributes such as brand, category, or price, encoded alongside the text).

Response

{
  "semantic_ids": [
    {
      "item_id": "sku_771",
      "codes": [1487, 302, 91, 12],
      "tokens": "<sid_0_1487><sid_1_302><sid_2_91><sid_3_12>",
      "cold_start": false,
      "collision_suffix": null
    }
  ]
}

Cold start

An item that was not in the catalog at fit time is still assignable, because the code is a function of content rather than of interaction history. Send it the moment it goes live and it is recommendable on the next POST /generate call.

{
  "item_id": "sku_new",
  "codes": [1487, 88, 405, 3],
  "tokens": "<sid_0_1487><sid_1_88><sid_2_405><sid_3_3>",
  "cold_start": true
}

The cold_start flag is worth logging. It lets you measure quality on items the tokenizer never saw, which is the number that tells you whether the fit generalized or memorized.

Determinism and stability

The same item content against the same tokenizer_id always returns the same codes. This is the point of keeping behavioral signal out of the quantizer input: an item's identifier does not move as it gets popular.

Stability holds across continuous updates too. As new regions are absorbed into the codebooks, codes already assigned do not move, so stored semantic IDs stay valid and a model trained against the vocabulary stays trained. See Continuously updated.

warning

A full refit is the exception to that. It produces a new tokenizer_id and a new code space, so it does require a reindex. Keep the old tokenizer live until the cutover completes. Continuous updates are the routine path; a refit should follow a genuine change in what you sell, not ordinary catalog growth.

note

Batches are capped at 1000 items per call. For a full catalog backfill, the tokenizer fit already assigns every item in catalog.uri, so you should not need to page through it manually.