​

Vision

​

Why Vision Matters

​

How Vision Works

​

1. Image Upload

​

2. Secure Processing

• Scanned for security threats and malicious content
• Validated for format and file size
• Stored securely with encryption at rest
• Processed using privacy-preserving techniques

​

3. Visual Understanding

• Text extraction: Reads error messages, labels, serial numbers, and any visible text
• Object recognition: Identifies products, components, UI elements, and physical objects
• Scene understanding: Comprehends context like settings, environments, and visual relationships
• Problem detection: Recognizes errors, defects, damage, or incorrect configurations

​

4. Contextual Response

• Conversation history and context
• Your knowledge base and documentation
• Customer information and past interactions
• Guidance instructions for handling specific scenarios

​

5. Intelligent Action

• Answer questions about what it sees
• Provide troubleshooting steps for visible errors
• Guide customers through processes using visual references
• Escalate to human agents when needed, with image context preserved

​

Supported Image Formats

​

Standard Formats

• JPEG (.jpg, .jpeg) - Most common photograph format
• PNG (.png) - Screenshots and graphics with transparency
• WebP (.webp) - Modern efficient format

​

Advanced Formats

• HEIC (.heic) - iPhone and modern iOS camera format
• HEIF (.heif) - High-efficiency image format
• GIF (.gif) - Animated or static images

​

File Size and Quality

• Maximum file size: Determined by your project settings (typically 10-25 MB)
• Automatic optimization: Large images are compressed to balance quality and processing speed
• Resolution: Higher resolution images provide more detail but take longer to process
• Best practice: 1920x1080 or lower for screenshots; original resolution for detailed product photos

​

Common Use Cases

​

Product Defect Identification

• Customer photographs the defect
• AI analyzes the image to assess severity and type of damage
• AI determines if it’s a manufacturing defect, shipping damage, or normal wear
• AI can initiate return, replacement, or refund workflows automatically

Customer: "The corner of my laptop is dented"
[Uploads photo showing dent]

AI: "I can see the dent on the bottom-left corner of your laptop. This
appears to be shipping damage rather than a manufacturing defect. I'll
process a replacement shipment right away. You can keep the current
device or return it - whichever you prefer. Your replacement will ship
within 24 hours."

​

Error Screenshot Analysis

• Customer shares screenshot of the error
• AI reads error codes, messages, and visible context
• AI matches error to known issues in documentation
• AI provides specific troubleshooting steps

Customer: "I'm getting an error when I try to log in"
[Uploads screenshot of error: "Connection timeout - Error 504"]

AI: "The error code 504 indicates a temporary server timeout. This
usually resolves within a few minutes. Try these steps:
1. Wait 2-3 minutes and try again
2. Clear your browser cache
3. Try a different browser
If the issue persists after 10 minutes, our servers may be experiencing
higher-than-normal load. I can notify you when service is restored."

​

Document and Receipt Processing

• Customer uploads receipt photo
• AI extracts purchase details, amounts, dates, and item information
• AI verifies against order history
• AI processes warranty claims, returns, or reimbursements

Customer: "I need to return this item but I lost the order number"
[Uploads photo of receipt]

AI: "I found your order from the receipt. You purchased the Premium
Widget on March 15, 2024 for $89.99 (Order #ORD-12345). You're within
the 30-day return window. I've created a prepaid return label and
emailed it to you. Once we receive the item, your refund will process
within 5-7 business days."

​

Installation Verification

• Customer shares photo of their setup
• AI verifies correct installation, configuration, or assembly
• AI identifies mistakes or missing steps
• AI provides corrective instructions with visual references

Customer: "I think I installed the router correctly but internet isn't working"
[Uploads photo of router with cables]

AI: "I can see your router setup. The issue is that the WAN cable (blue)
is plugged into LAN port 2 instead of the WAN port. Here's what to do:
1. Unplug the blue cable from LAN port 2
2. Plug it into the WAN port (the one separated from the others, usually yellow)
3. Wait 30 seconds for the router to restart
The WAN light should turn solid green when connected properly."

​

Visual Troubleshooting

• Customer screenshots the problem
• AI identifies UI elements, states, and potential causes
• AI guides customer through fixes or workarounds
• AI can escalate with full visual context if needed

Customer: "The upload button disappeared"
[Uploads screenshot showing file manager interface]

AI: "I can see the issue - you're in 'View Only' mode. The upload button
is hidden when you don't have edit permissions. To fix this:
1. Click on your username in the top-right corner
2. Select 'Switch to Edit Mode'
3. The upload button will appear in the top toolbar
If you don't see 'Switch to Edit Mode', you may need to request editor
access from your team administrator."

​

Serial Number and Model Identification

• Customer photographs serial number sticker or product label
• AI reads model number, serial number, and manufacturing information
• AI looks up specific documentation and support for that exact model
• AI provides personalized troubleshooting

Customer: "How do I update the firmware?"
[Uploads photo of product label showing model]

AI: "You have Model X-2000-B (serial: SN-12345). This model requires
the 2.4 firmware update process:
1. Download firmware v2.4.1 for the X-2000-B series: [link]
2. Connect via USB (not WiFi for this model)
3. Run the firmware installer
Note: The X-2000-B has different update steps than the X-2000-A, so
generic instructions won't work. Follow the model-specific guide here: [link]"

​

Enabling Vision

​

Enable Image Upload in Frames

1. Navigate to Frames in your botBrains project
2. Select the frame you want to configure
3. Go to the General Settings tab
4. Toggle Allow Image Upload to enabled
5. Save your frame configuration

​

Configure in Guidance

When a customer uploads an image of a product defect:
1. Analyze the image carefully to identify the type and severity of the defect
2. Determine if it's a manufacturing defect, shipping damage, or user damage
3. For manufacturing defects: offer immediate replacement
4. For shipping damage: apologize and process replacement with expedited shipping
5. For user damage: explain it's not covered by warranty but offer repair options
6. Always be empathetic and focused on resolving the customer's problem

​

Model Compatibility

• Claude 3.5 Sonnet (recommended for best vision performance)
• Claude 3 Opus
• Claude 3 Haiku (faster, good for simple image tasks)
• GPT-4o and GPT-4 Turbo with Vision

​

Best Practices

​

When to Use Vision

​

Guidance for Handling Images

If you're experiencing an error, please share a screenshot showing:
- The complete error message
- The URL or page you're on
- Any error codes or numbers visible

When photographing a product defect:
- Use good lighting
- Get close enough to show detail
- Include the full product in the frame
- Take multiple angles if helpful

"Looking at your screenshot, I can see the error code 404 on the checkout
page. This indicates..."

"In the photo you shared, I notice the connector cable is plugged into
the wrong port..."

Good: "The red error icon in the top-right corner indicates..."
Poor: "I see there's an error..."

​

Privacy and Sensitivity

• Credit cards or payment information
• Personal identification documents (unless absolutely necessary and encrypted)
• Medical conditions or private health information
• Passwords or security codes visible on screen

"If you're sharing a screenshot of your account page, please blur or
cover any personal information like your email address or account number
before uploading."

• Issue is resolved
• A defined retention period (30-90 days)
• Customer requests deletion

​

Performance Optimization

• Screenshots: PNG at screen resolution (typically 1-3 MB)
• Photos: JPEG compressed to 2-5 MB for sufficient quality
• Avoid unnecessarily large files that slow processing

• Maximum 5-10 images per conversation
• Maximum file size of 10-25 MB per image
• Automatic compression for oversized files

• Analyze images only when relevant to the conversation
• Don’t ask for images if text would suffice
• Cache common visual patterns (error screenshots, product photos)

​

Security and Privacy

​

Image Security

• Scanned on upload for malware, viruses, and malicious content
• Validated for format to prevent exploit attempts
• Encrypted at rest using AES-256 encryption
• Transmitted securely over HTTPS/TLS
• Access-controlled with role-based permissions

​

Data Privacy

• Images are stored in secure, encrypted cloud storage
• Access is logged and auditable
• Retention policies configurable per project
• Automatic deletion after retention period expires

• Images are processed by AI models without human review (unless escalated)
• No image data is used to train AI models
• Processing is ephemeral - images are analyzed in-memory and not cached
• Compliance with GDPR, CCPA, and other privacy regulations

• Request deletion of their uploaded images
• Access copies of images they’ve uploaded
• Opt out of image upload feature

​

Compliance Considerations

• Images are personal data and subject to GDPR
• Provide clear privacy notices about image processing
• Honor data subject rights (access, deletion, portability)
• Implement data retention limits

• Do not enable image upload for healthcare applications unless:
  • You have a BAA (Business Associate Agreement) with botBrains
  • Images are encrypted end-to-end
  • Access is strictly controlled and audited

• Never request or accept images containing:
  • Credit card numbers visible in screenshots
  • CVV codes or security codes
  • Full payment card images (front or back)

​

Troubleshooting Vision

​

Image Upload Not Available

1. Navigate to Frames > General Settings
2. Verify “Allow Image Upload” is enabled
3. Save and refresh the chat widget

• Ensure images are under maximum file size
• Try compressing large images before upload

• Image upload requires modern browsers (Chrome 90+, Safari 14+, Firefox 88+)
• Mobile browsers may have different file size limits

​

AI Not Understanding Images

• Use higher resolution for detailed analysis
• Ensure good lighting for photographs
• Avoid blurry or low-quality images

Instead of: [uploads image]
Better: "Here's a screenshot of the error I'm seeing" [uploads image]

• Screenshots for digital errors (PNG format)
• Photographs for physical products (JPEG format)
• Avoid highly compressed or low-resolution images

​

Vision Analysis Errors

• Some images may be too complex or ambiguous
• Very small text may be difficult to read
• Extreme lighting conditions can affect recognition

If you cannot confidently analyze the image or the issue requires
human judgment, say: "I'm having trouble analyzing this image clearly.
Let me connect you with a specialist who can review this in detail."
Then use `escalate_to_human` with the image attached.

​

Performance Issues

• Large images (>10 MB) take longer to process
• Complex images with many elements require more analysis time
• Multiple concurrent image uploads may be queued

• Encourage customers to compress screenshots
• Set reasonable file size limits (5-10 MB)
• Use faster models (Claude 3 Haiku) for simple image tasks

​

Examples of Vision in Action

​

Example 1: E-commerce Returns

Customer: "My order arrived damaged"
[Uploads photo showing crushed shipping box]

AI: "I'm sorry your order arrived in that condition. I can see significant
damage to the shipping box. Let's get this resolved:

Based on the damage visible in your photo, I'm processing:
✓ Full refund: $156.78 (credited within 3-5 business days)
✓ Replacement shipment: Expedited 2-day delivery at no charge
✓ Return label: No need to return the damaged item

Your replacement order is #ORD-67890. You'll receive tracking within
2 hours. Is there anything else I can help with?"

​

Example 2: Technical Support

Customer: "Getting this error when I export"
[Uploads screenshot: "Export failed: Invalid format specified (ERR-2045)"]

AI: "I can see error code ERR-2045 in your screenshot. This happens when
the export format doesn't match your file type. Looking at your screenshot,
you're trying to export a video file (.mp4) as an audio-only format (.mp3).

Here's how to fix it:
1. In the Export dialog, change 'Format' from 'MP3' to 'MP4'
2. Or, if you want just the audio, check 'Extract Audio' first
3. Then select MP3 format

The option you need is in the dropdown menu visible in your screenshot,
just to the right of where the error appeared. Try that and let me know
if you still have issues!"

​

Example 3: Assembly Verification

Customer: "Is this right? The instructions are confusing"
[Uploads photo of desk with parts laid out]

AI: "Looking at your photo, you're on the right track! You have all the
parts correctly identified and the frame pieces are in the right position.

However, I notice one issue: the support bracket (the L-shaped metal piece
on the left) needs to be rotated 90 degrees. The long side should face up,
not forward.

Once you rotate that bracket:
1. Attach it to the frame using the 4 long screws (visible in your photo)
2. Then attach the desktop (the large piece in the background)
3. Finally, add the drawer assembly

You're actually further along than the instructions show - good work!
Need any other guidance?"

​

Related Documentation

• Guidance - Configure how your AI responds to visual queries
• Actions - Enable tools for processing visual information
• Frames - Configure image upload in your chat widget
• Knowledge - Combine visual analysis with documentation
• Conversations - Review how Vision is used in customer interactions