wriftai model serve

wriftai model serve [flags]

  Start the model server
  $ wriftai model serve

  Start the model server on a specific port
  $ wriftai model serve --port 1234

  Start the model server with environment variables
  $ wriftai model serve \
	--env ENV_VAR_1=VALUE_1 \
	--env ENV_VAR_2=VALUE_2

  Start the model server with hot reload enabled
  $ wriftai model serve --reload

  Build the model before starting the server
  $ wriftai model serve --build

  Start the model server with CPU and memory limits
  $ wriftai model serve --cpus 2 --memory 4g

  Start the model server with no GPU access
  $ wriftai model serve --gpus 0

  Start the model server with a specific number of GPUs
  $ wriftai model serve --gpus 2

  Start the model server on specific GPUs (find indices with nvidia-smi)
  $ wriftai model serve --gpus device=0,1

      --build             Build the model before starting the server, so you don't need to run the build command separately after making changes.
      --cpus string       Number of CPUs to allocate to the model (format: <number>, supports fractional values, e.g. "2", "1.5", "0.5"). If not set, there is no CPU limit.
  -e, --env stringArray   Set environment variables (format: KEY=value). Can be specified multiple times.
      --gpus string       GPUs to expose to the model. All available GPUs are exposed by default. Supported values: a GPU count (e.g. "0" to disable GPU access, "2" to request any two GPUs), or "device=<index,...>" to pin specific GPUs (e.g. "device=0,1", indices from nvidia-smi). If GPU support is unavailable, the model runs without GPUs.
  -h, --help              help for serve
      --memory string     Maximum amount of memory to allocate to the model (format: <number>[<unit>], where unit = b, k, m, or g; e.g. "512m", "4g"). If not set, there is no memory limit.
  -p, --port string       The port on which the server will run. (default "8000")
  -r, --reload            Enable hot reload. This can be used during development to automatically detect file changes and reload.

      --debug   Enable debug mode