Expert insights on AI-powered coding security, vibe-based development practices, and protecting AI-generated web applications from vulnerabilities.

· · 8 min read

LLM Security Best Practices for Production Apps

llm security best-practices geo2026 production
LLM Security Best Practices for Production Apps

Production Security Mindset

LLM Production Security : The practices and controls required to safely operate LLM-powered applications in production, protecting against adversarial input, data leakage, excessive costs, and system compromise.

Production LLM security differs from development:

  • Real attackers, not theoretical threats
  • User data at risk
  • Financial impact from abuse
  • Compliance requirements

Best Practice 1: Secure Architecture

Principle of Least Privilege

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

# Define minimal permissions for LLM operations
class LLMPermissions:
    def __init__(self, user_role: str):
        self.permissions = self._get_permissions(user_role)

    def _get_permissions(self, role: str) -> dict:
        base = {
            'max_tokens': 500,
            'allowed_functions': [],
            'data_access': 'none',
            'rate_limit': 10,  # per minute
        }

        if role == 'premium':
            base.update({
                'max_tokens': 2000,
                'allowed_functions': ['search', 'summarize'],
                'data_access': 'own',
                'rate_limit': 50,
            })

        if role == 'admin':
            base.update({
                'allowed_functions': ['search', 'summarize', 'analyze'],
                'data_access': 'all',
                'rate_limit': 100,
            })

        return base

Isolation Layers

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# Separate LLM processing from sensitive operations
class SecureLLMService:
    def __init__(self):
        self.llm_client = AnthropicClient()  # LLM access
        self.data_service = DataService()     # Data access (separate)
        self.action_service = ActionService() # Actions (separate)

    async def process(self, request: Request, user: User) -> Response:
        # LLM generates intent, doesn't access data directly
        intent = await self.llm_client.generate(
            prompt=self._build_safe_prompt(request.message)
        )

        # Separate service validates and executes
        if intent.type == 'data_query':
            # Data service enforces authorization
            data = await self.data_service.query(
                intent.query,
                user=user  # Authorization here
            )
            return self._format_response(data)

        if intent.type == 'action':
            # Action service validates before executing
            result = await self.action_service.execute(
                intent.action,
                user=user,
                requires_approval=intent.high_impact
            )
            return self._format_response(result)

Best Practice 2: Input Handling

Comprehensive Validation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

from pydantic import BaseModel, validator
import re

class LLMRequest(BaseModel):
    message: str
    context: list[str] | None = None

    @validator('message')
    def validate_message(cls, v):
        # Length limits
        if len(v) > 10000:
            raise ValueError('Message too long')

        # Injection detection
        if InjectionScanner().is_dangerous(v):
            raise ValueError('Invalid message format')

        return v

    @validator('context')
    def validate_context(cls, v):
        if v is None:
            return v

        # Limit context items
        if len(v) > 10:
            raise ValueError('Too many context items')

        # Validate each item
        for item in v:
            if len(item) > 5000:
                raise ValueError('Context item too long')
            if InjectionScanner().is_dangerous(item):
                raise ValueError('Invalid context format')

        return v

Sanitization Pipeline

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

class InputSanitizer:
    def sanitize(self, text: str) -> str:
        # Remove zero-width characters
        text = self._remove_hidden_chars(text)

        # Remove HTML comments
        text = re.sub(r'<!--.*?-->', '', text, flags=re.DOTALL)

        # Normalize whitespace
        text = ' '.join(text.split())

        # Escape special sequences
        text = self._escape_special(text)

        return text

    def _remove_hidden_chars(self, text: str) -> str:
        hidden = '\u200b\u200c\u200d\u2060\ufeff'
        return ''.join(c for c in text if c not in hidden)

    def _escape_special(self, text: str) -> str:
        # Escape characters that might affect prompt parsing
        return text.replace('<|', '< |').replace('|>', '| >')

Best Practice 3: Output Handling

PII Detection and Redaction

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import re

class PIIFilter:
    PATTERNS = {
        'email': (r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', '[EMAIL]'),
        'phone': (r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b', '[PHONE]'),
        'ssn': (r'\b\d{3}-\d{2}-\d{4}\b', '[SSN]'),
        'credit_card': (r'\b(?:\d{4}[-\s]?){3}\d{4}\b', '[CARD]'),
        'ip_address': (r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b', '[IP]'),
    }

    def filter(self, text: str) -> tuple[str, list[str]]:
        redacted = []
        filtered = text

        for name, (pattern, replacement) in self.PATTERNS.items():
            matches = re.findall(pattern, filtered)
            if matches:
                redacted.extend([(name, m) for m in matches])
                filtered = re.sub(pattern, replacement, filtered)

        return filtered, redacted

    def log_redactions(self, redacted: list, context: dict):
        if redacted:
            security_logger.info(
                "PII redacted from LLM output",
                extra={
                    'types': [r[0] for r in redacted],
                    'count': len(redacted),
                    'context': context
                }
            )

Response Validation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

class ResponseValidator:
    def validate(self, response: str, expected_format: str) -> bool:
        validators = {
            'json': self._validate_json,
            'markdown': self._validate_markdown,
            'code': self._validate_code,
            'text': self._validate_text,
        }

        validator = validators.get(expected_format, self._validate_text)
        return validator(response)

    def _validate_text(self, response: str) -> bool:
        # Check for signs of prompt leakage
        leak_indicators = [
            'system prompt', 'my instructions', 'i was told to',
            'ignore previous', 'here is my prompt'
        ]
        response_lower = response.lower()
        return not any(ind in response_lower for ind in leak_indicators)

    def _validate_json(self, response: str) -> bool:
        try:
            json.loads(response)
            return True
        except json.JSONDecodeError:
            return False

Best Practice 4: Access Control

API Key Management

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

from datetime import datetime, timedelta
import secrets

class LLMAPIKeyManager:
    def create_key(self, user_id: str, permissions: dict, expires_days: int = 30) -> str:
        key = f"llm_{secrets.token_urlsafe(32)}"
        key_hash = self._hash(key)

        self.db.api_keys.insert({
            'hash': key_hash,
            'user_id': user_id,
            'permissions': permissions,
            'expires_at': datetime.utcnow() + timedelta(days=expires_days),
            'created_at': datetime.utcnow(),
            'last_used': None,
            'usage_count': 0
        })

        return key  # Return only once, store only hash

    def validate_key(self, key: str) -> dict | None:
        key_hash = self._hash(key)
        record = self.db.api_keys.find_one({'hash': key_hash})

        if not record:
            return None

        if record['expires_at'] < datetime.utcnow():
            return None

        # Update last used
        self.db.api_keys.update(
            {'hash': key_hash},
            {'$set': {'last_used': datetime.utcnow()}, '$inc': {'usage_count': 1}}
        )

        return record['permissions']

Rate Limiting

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

from redis import Redis
import time

class LLMRateLimiter:
    def __init__(self, redis: Redis):
        self.redis = redis

    def check(self, user_id: str, limit: int, window_seconds: int = 60) -> bool:
        key = f"llm_rate:{user_id}:{int(time.time() // window_seconds)}"

        current = self.redis.incr(key)
        if current == 1:
            self.redis.expire(key, window_seconds)

        return current <= limit

    def get_remaining(self, user_id: str, limit: int, window_seconds: int = 60) -> int:
        key = f"llm_rate:{user_id}:{int(time.time() // window_seconds)}"
        current = int(self.redis.get(key) or 0)
        return max(0, limit - current)

Best Practice 5: Monitoring

Comprehensive Logging

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

import structlog

logger = structlog.get_logger()

class LLMMonitor:
    def log_request(self, request_id: str, user_id: str, input_text: str):
        logger.info(
            "llm_request",
            request_id=request_id,
            user_id=user_id,
            input_length=len(input_text),
            input_hash=self._hash(input_text),  # Don't log full input
        )

    def log_response(self, request_id: str, tokens_used: int, latency_ms: int):
        logger.info(
            "llm_response",
            request_id=request_id,
            tokens_used=tokens_used,
            latency_ms=latency_ms,
        )

    def log_security_event(self, event_type: str, details: dict):
        logger.warning(
            "llm_security_event",
            event_type=event_type,
            **details
        )

    def log_error(self, request_id: str, error: Exception):
        logger.error(
            "llm_error",
            request_id=request_id,
            error_type=type(error).__name__,
            error_message=str(error),
        )

Alerting Rules

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# prometheus-alerts.yml
groups:
  - name: llm_security
    rules:
      - alert: HighInjectionAttemptRate
        expr: rate(llm_injection_attempts_total[5m]) > 10
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "High rate of injection attempts detected"

      - alert: UnusualTokenUsage
        expr: llm_tokens_used_total > (llm_tokens_used_total offset 1h) * 3
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "Token usage 3x higher than 1 hour ago"

      - alert: LLMErrorRateHigh
        expr: rate(llm_errors_total[5m]) / rate(llm_requests_total[5m]) > 0.1
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "LLM error rate exceeds 10%"

Best Practice 6: Cost Controls

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

class LLMCostController:
    def __init__(self):
        self.daily_limit = float(os.getenv('LLM_DAILY_LIMIT_USD', 100))
        self.per_user_limit = float(os.getenv('LLM_USER_LIMIT_USD', 5))

    def check_budget(self, user_id: str, estimated_cost: float) -> bool:
        # Check daily total
        daily_spend = self.get_daily_spend()
        if daily_spend + estimated_cost > self.daily_limit:
            self.alert_budget_exceeded('daily', daily_spend)
            return False

        # Check per-user limit
        user_spend = self.get_user_daily_spend(user_id)
        if user_spend + estimated_cost > self.per_user_limit:
            return False

        return True

    def record_spend(self, user_id: str, tokens: int, model: str):
        cost = self.calculate_cost(tokens, model)
        self.redis.incr(f"llm_spend:daily:{date.today()}", cost)
        self.redis.incr(f"llm_spend:user:{user_id}:{date.today()}", cost)

Security Checklist

Production LLM Security Checklist

Pre-deployment security verification

Input Validation

  • Maximum input length enforced
  • Injection patterns detected and blocked
  • Hidden characters stripped
  • Context limits enforced

Prompt Security

  • System prompt hardened against override
  • User content clearly delimited
  • No secrets in prompts

Output Security

  • PII detection and redaction active
  • Response validation implemented
  • No raw error messages to users

Access Control

  • API key rotation configured
  • Rate limiting per user
  • Role-based permissions

Monitoring

  • All requests logged
  • Security events alerted
  • Cost tracking active

FAQ

How often should API keys be rotated?

Every 90 days for standard access, every 30 days for high-privilege access. Implement key rotation before deployment.

What's a reasonable rate limit?

Depends on use case. Start conservative (10-20 requests/minute), monitor actual usage, and adjust. Always have per-user and global limits.

Should I log full prompts and responses?

Log metadata (length, hash) but not full content to avoid storing PII. Keep full logs only for security investigations with appropriate access controls.

How do I handle LLM provider outages?

Implement circuit breakers, have fallback responses for critical paths, and monitor provider status. Consider multi-provider setups for high-availability requirements.

Conclusion

Key Takeaways

  • Least privilege: LLM should have minimal permissions
  • Isolate LLM from direct data access and actions
  • Validate all inputs with length limits and injection detection
  • Filter outputs for PII and validate format
  • Implement API key management with rotation
  • Rate limit per user and globally
  • Log comprehensively, alert on anomalies
  • Control costs with budgets and monitoring

AI Coding Security Insights.
Ship Vibe-Coded Apps Safely.

Effortlessly test and evaluate web application security using Vibe Eval agents.

GET STARTED GET A DEMO