🚀 Quick Start

info

To run this tutorial, you will need an API key. You can request an API key following authentication instructions. Use Community channel to bump up your request for API key.

In this tutorial, we will use detoxio.ai command line tool to scan a large language model (LLM) from Hugging Face repository to identify vulnerabilities. To be able to complete this tutorial, you will need Python 3.8+ installed in your system.

note

You can try out this tutorial using docker if you do not already have a required version of Python installed in your system.

docker run --rm -it python:3 bash

Start by installing detoxio-dtx

python3 -m pip install detoxio-dtx \
  detoxio-api-protocolbuffers-python detoxio-api-grpc-python \
  --upgrade --extra-index-url https://buf.build/gen/python

note

You must also install PyTorch if it is not already available in your runtime environment.

pip install torch

Google Colab or Kaggle provides free GPU/TPU compute for ML research. You can use it for trying out examples in this tutorial.

Get an API key for api.detoxio.ai by following the instructions for API Authentication. Export the API key as an environment variable for use by the tool

export DETOXIO_API_KEY='<your-api-key>'

note

If you are scanning a private or gated model from Hugging Face, you must export the HF_TOKEN as well. This is not required if the model is accessible without gating.

export HF_TOKEN='<your-hf-token>'

Test a model for security vulnerabilities

dtx scan --model gpt2 --jsonl /tmp/report.jsonl --verbose

You can also use detoxio as an SDK in your own application and use-case

from detoxio.scanner import LLMScanner, LLMPrompt, LLMResponse

# This is a placeholder function. Here you should actually
# load an LLM model and perform inference
def llm(input: str) -> str:
  return input

def prompt_handler(prompt: LLMPrompt) -> LLMResponse:
  model_output = llm(prompt.content)
  return LLMResponse(content=model_output)

scanner = LLMScanner()
scanner.start(prompt_handler=prompt_handler)

For more information on Python SDK, refer to documentation