Light Logo
CLI (v0.55.0)Reference

wriftai model start

CLI reference for wriftai model start

wriftai model start

Pull and start a model locally

Synopsis

Pull and start serving a model locally. The model is pulled only if it's not already in the local cache.

For requirements and GPU setup, see https://wrift.ai/docs/clients/cli/running-models-locally.

wriftai model start <owner/name[:version-number]> [flags]

Examples

  Start the latest version of a model
  $ wriftai model start johndoe/sentiment-analyzer

  Start a specific version of a model
  $ wriftai model start johndoe/sentiment-analyzer:2

  Start a model on a specific port
  $ wriftai model start johndoe/sentiment-analyzer --port 1234

  Start a model with environment variables
  $ wriftai model start johndoe/sentiment-analyzer \
	--env ENV_VAR_1=VALUE_1 \
	--env ENV_VAR_2=VALUE_2

  Start a model with CPU and memory limits
  $ wriftai model start johndoe/sentiment-analyzer --cpus 2 --memory 4g

  Start a model with no GPU access
  $ wriftai model start johndoe/sentiment-analyzer --gpus 0

  Start a model with a specific number of GPUs
  $ wriftai model start johndoe/sentiment-analyzer --gpus 2

  Start a model on specific GPUs (find indices with nvidia-smi)
  $ wriftai model start johndoe/sentiment-analyzer --gpus device=0,1

Options

      --cpus string       Number of CPUs to allocate to the model (format: <number>, supports fractional values, e.g. "2", "1.5", "0.5"). If not set, there is no CPU limit.
  -e, --env stringArray   Set environment variables (format: KEY=value). Can be specified multiple times.
      --gpus string       GPUs to expose to the model. All available GPUs are exposed by default. Supported values: a GPU count (e.g. "0" to disable GPU access, "2" to request any two GPUs), or "device=<index,...>" to pin specific GPUs (e.g. "device=0,1", indices from nvidia-smi). If GPU support is unavailable, the model runs without GPUs.
  -h, --help              help for start
      --memory string     Maximum amount of memory to allocate to the model (format: <number>[<unit>], where unit = b, k, m, or g; e.g. "512m", "4g"). If not set, there is no memory limit.
  -p, --port string       The port on which the server will run. (default "8000")

Options inherited from parent commands

      --debug   Enable debug mode

SEE ALSO