How to Enable Image Understanding for Your AI Agent

Image Understanding allows YourGPT to process images alongside text. Your AI can interpret screenshots, documents, receipts, product photos, charts, and other visual information and use what it finds to respond to the user's request.

This is useful for businesses where users often need to share something they cannot easily explain with text.

For example, a customer can send a screenshot of an error, a photo of a damaged product, or a receipt for a refund request. Your AI can analyze the image and use the relevant information when handling the conversation.

What Your AI Can Do with Images

With Image Understanding enabled, your AI can work with different types of visual information during a conversation.

It can:

• Read documents such as receipts, invoices, forms, and scanned pages.

• Analyze screenshots to identify error messages, codes, labels, and visible interface elements.

• Understand product photos to help identify visible damage or other issues.

• Read charts and dashboards and summarize information shown in them.

• Extract information such as dates, amounts, reference numbers, and other visible details.

• Interpret diagrams and process visuals when they are relevant to the user's question.

• Analyze operational images such as inventory, equipment, or warehouse photos.

The information found in an image can then be used together with the user's message and your AI's business knowledge.

How Image Understanding Works

Image Understanding uses multimodal AI models that can process images and text together.

When a user uploads an image, YourGPT analyzes the information available in it and considers that information as part of the conversation.

For example, a customer might send a screenshot with:

"I'm getting this error. How do I fix it?"

Your AI can inspect the screenshot to identify the visible error message or error code. It can then use that information to understand the problem and provide relevant troubleshooting steps based on your business knowledge.

Depending on the image, the AI may identify:

• Text and numbers

• Error messages and error codes

• Product or reference information

• Labels and interface elements

• Dates and amounts

• Other relevant visual details

This allows the AI to use information from an image when answering questions or handling supported workflows.

How to Enable Image Understanding

Image Understanding is available through Agent Mode in the YourGPT Dashboard.

To enable it:

1. Log in to your YourGPT Dashboard with an account that has the required settings permissions.

2. Open Settings from the left-hand navigation.

3. Select General.

4. Find the Mode setting.

5. Switch from Chat Mode to Agent Mode.

6. Save the change if your workspace requires manual confirmation.

Once Agent Mode is enabled, users can upload images in supported conversations and your AI can process them alongside their messages.

Ways to Use Image Understanding

Image Understanding can support a range of customer-facing and internal business workflows.

Troubleshooting from Screenshots

Scenario:

A customer encounters an error and sends a screenshot with:

"This keeps happening. What should I do?"

Your AI can identify the error message or code visible in the screenshot and use your knowledge base to provide the relevant troubleshooting steps.

Result: Customers can show the problem instead of manually copying error messages or describing the screen.

Receipt and Invoice Analysis

Scenario:

A customer uploads a receipt and asks:

"Can I get a refund for this purchase?"

Your AI can identify relevant information such as the item, purchase date, amount, or reference number.

It can then use your refund policy and other business information to guide the customer.

Result: Customers can provide the document directly instead of entering its information manually.

Product Damage Review

Scenario:

A customer receives a damaged product and uploads a photo with:

"The item arrived like this. Can I get a replacement?"

Your AI can inspect the image for visible damage and consider it alongside your product and return policies.

Result: Customers can show the issue directly rather than trying to describe the damage in detail.

Document and Form Processing

Scenario:

A customer uploads a form and asks:

"What information is missing?"

Your AI can read the visible contents of the document and identify fields or information that appear to be missing.

Result: Customers can get help understanding documents without manually transcribing them.

Inventory and Operational Images

Scenario:

A team member uploads a warehouse photo and asks:

"How many units are visible here?"

Your AI can analyze the image and provide an answer based on what is visible.

This can support inventory checks, operational reviews, and other workflows where information is captured visually.

Result: Teams can work with operational images directly through their AI agent.

Charts and Dashboard Screenshots

Scenario:

A team member shares a dashboard screenshot and asks:

"What changed compared with the previous period?"

Your AI can interpret visible charts, labels, values, and other relevant information and summarize what the screenshot shows.

Result: Teams can ask questions about visual business information without manually copying the data into the conversation.

Configure Your AI for Image-Based Workflows

Image Understanding is most useful when it is connected to the information and workflows your business already uses.

Define Relevant Use Cases

Identify the types of images your users are likely to send.

For example:

• Support screenshots

• Receipts and invoices

• Product photos

• Forms and documents

• Inventory photos

• Dashboard screenshots

This helps you focus your AI experience on situations where visual information is genuinely useful.

Connect Your Business Knowledge

The image provides information about what the user has shared. Your AI also needs the relevant business knowledge to determine how to respond.

For example, if users upload receipts for refund requests, your AI should have access to the applicable refund policies and product information.

Decide What Happens Next

Understanding an image is only one part of the workflow.

Depending on the use case, your AI may need to:

• Answer the user's question

• Provide troubleshooting steps

• Explain a policy

• Collect additional information

• Continue a business process

• Escalate the conversation when required

Design the workflow around the outcome you want the user to reach.

Handle Missing Information

An image may not contain everything your AI needs.

If required information cannot be identified, the AI should ask the user for the missing details rather than assuming an answer.

This is particularly important for workflows where visual information affects a business decision.

Best Practices for Image Understanding

Start with Real User Scenarios

Test the types of images your users are actually likely to send.

For example, test different screenshots, document layouts, receipts, product photos, and dashboard images rather than relying on a single example.

Combine Images with Business Context

Image Understanding can identify information from an image, but your AI still needs your business knowledge to provide a useful answer.

Make sure relevant policies, product information, and support content are available to your AI.

Build Fallbacks

Decide what should happen when the AI cannot find the required information in an image.

A good fallback may be asking the user for the missing information or requesting a clearer image.

Verify Important Information

For workflows involving important business decisions, consider how information identified from an image should be verified before the AI takes a consequential action.

Test Before Deployment

Test the complete workflow from image upload to final response.

Check whether your AI:

• Understands the relevant information

• Uses the correct business knowledge

• Gives an appropriate response

• Handles missing information correctly

• Avoids making unsupported assumptions

Image Understanding vs. Image Generation

Image Understanding and Image Generation serve different purposes.

Image Understanding allows your AI to interpret images provided by users.

Image Generation creates new images.

For business AI agents, Image Understanding is useful when users already have visual information that needs to be interpreted as part of a conversation.

Frequently Asked Questions

Does Image Understanding work with screenshots?

Yes. Your AI can interpret information visible in screenshots and use it when responding to the user's request.

Can YourGPT read text from images?

Yes. Your AI can interpret visible text and other relevant information in supported images. Results depend on factors such as image quality, resolution, and how clearly the information is presented.

Can Image Understanding analyze receipts and documents?

Yes. Receipts, invoices, forms, scanned pages, and other document images can be used as visual information during a conversation.

Can my AI use information from an image to answer questions?

Yes. Information identified from an uploaded image can be considered alongside the user's message and your AI's available business knowledge.

Can Image Understanding analyze product photos?

Yes. Your AI can interpret visible information in product images, including visible damage or other details relevant to the user's request.

Does Image Understanding generate images?

No. Image Understanding is designed to interpret images provided to your AI. It does not generate new images.

Do I need to enable Agent Mode?

Yes. Image Understanding in YourGPT is enabled through Agent Mode.

Build an AI Agent That Understands More Than Text

Your users do not always have the right words to explain a problem.

Sometimes a screenshot explains the issue better than a description. A receipt contains information that would take several messages to type. A product photo can show damage that is difficult to describe.

With Image Understanding enabled through Agent Mode, your YourGPT AI can work with these visual inputs as part of the conversation.

Your users can show your AI the problem. Your AI can understand it and help move the conversation forward.