Deploying DeepSeek-R1 Distill Llama Models on Amazon Bedrock

Deploying the DeepSeek-R1 Distill Llama models on Amazon Bedrock is now easier than ever with the Custom Model Import feature. This allows seamless integration of externally fine-tuned models into the Bedrock environment, enabling you to utilize its serverless infrastructure and unified API for efficient model deployment.

Prerequisites

Model Compatibility

Ensure your DeepSeek-R1 Distill model is based on a supported architecture, such as:

• Llama 2
• Llama 3
• Llama 3.1
• Llama 3.2
• Llama 3.3

Amazon Bedrock supports these architectures for custom model imports.

Model Files Preparation

Prepare the necessary model files in the Hugging Face format, including:

• Model weights in .safetensors format (already available, so no additional conversion is needed)
• Configuration file (config.json)
• Tokenizer files (tokenizer_config.json, tokenizer.json, tokenizer.model)

These files should be stored in an Amazon S3 bucket that is accessible to your AWS account.

For an interactive guide, refer to the Jupyter Notebook: Deploy-DeepSeek-R1-On-Amazon-Bedrock.ipynb

---

Step-by-Step Deployment Guide

1. Install Required Dependencies

Install the necessary Python packages:

pip install huggingface_hub boto3

2. Download the DeepSeek-R1 Model

Use the Hugging Face Hub to download your DeepSeek R1 model:

from huggingface_hub import snapshot_download

model_id = "deepseek-ai/DeepSeek-R1-Distill-Llama-8B"
local_dir = snapshot_download(repo_id=model_id, local_dir="DeepSeek-R1-Distill-Llama-8B")

3. Upload Model Files to Amazon S3

Upload the model files to an S3 bucket in your AWS account:

import boto3
import os

s3_client = boto3.client('s3', region_name='us-east-1')
bucket_name = 'your-s3-bucket-name'
local_directory = 'DeepSeek-R1-Distill-Llama-8B'

for root, dirs, files in os.walk(local_directory):
    for file in files:
        local_path = os.path.join(root, file)
        s3_key = os.path.relpath(local_path, local_directory)
        s3_client.upload_file(local_path, bucket_name, s3_key)

Ensure that the S3 bucket is in a region supported by Amazon Bedrock, such as us-east-1 or us-west-2.

4. Import the Model into Amazon Bedrock

1. Navigate to the Amazon Bedrock console.
2. Select “Custom Models” and click “Import Model”.
3. Provide the S3 URI where your model files are stored (e.g., s3://your-s3-bucket-name/DeepSeek-R1-Distill-Llama-8B/).
4. Follow the prompts to complete the import process.

For detailed guidance, refer to AWS documentation on importing custom models.

5. Invoke the Model

You can now invoke the model using the Bedrock API:

import boto3
import json

client = boto3.client('bedrock-runtime', region_name='us-east-1')
model_id = 'arn:aws:bedrock:us-east-1:your-account-id:imported-model/your-model-id'
prompt = "Your input prompt here"

response = client.invoke_model(
    modelId=model_id,
    body=json.dumps({'prompt': prompt}),
    accept='application/json',
    contentType='application/json'
)

result = json.loads(response['body'].read().decode('utf-8'))
print(result)

Replace ‘your-account-id’ and ‘your-model-id’ with your specific AWS account ID and model ID.

---

Conclusion

By following these steps, you can efficiently deploy the DeepSeek-R1 Distill Llama model on Amazon Bedrock, leveraging its serverless capabilities for scalable AI inference.

For additional insights and a detailed walkthrough, check out the accompanying Git repository and video tutorial.

DeepSeekR1onAmazonBedrock

Import Custom Models Into Amazon Bedrock