search_shop_policies_and_faqs Evaluations

Evaluate and rate the quality of responses to understand how well this tool performs

Rate content quality with 1-5 stars
Add notes to track what works
21
Total Evaluations
3
Average Quality
Content rating (1-5 stars)
50.0%
High Quality
1 with 4+ stars
2
Rated
Manual quality reviews
Avg execution time: 3046ms
13 of 21 calls completed successfully
Showing 21 of 21 evaluations

All Evaluations

error
11/19/2025, 12:33:02 PM0
Store:https://shopify.com/63984763134/account/customer/api/mcp
No notes added

Input Parameters

{
  "query": "search products"
}

Output

Unable to connect to MCP server at https://shopify.com/63984763134/account/customer/api/mcp: Error POSTing to endpoint (HTTP 403): error code: 1003
error
11/19/2025, 12:30:07 PM0
Store:https://ajayplayingaround.myshopify.com
User Prompt:
What is store's return policy
No notes added

Input Parameters

{
  "query": "returns exchanges how to return items return process",
  "context": "Customer asking about the store's return policy, looking for details on how returns work"
}

Output

Unable to connect to MCP server at https://ajayplayingaround.myshopify.com: Error POSTing to endpoint (HTTP 403): error code: 1003
success
6/24/2025, 11:14:38 PM
1285ms
Store:messaging-ai-shop.myshopify.com
User Prompt:
can I order online and pick up in store?
No notes added

Input Parameters

{
  "query": "can I order online and pick up in store"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"Question: Do you offer local pickup? Answer: No pickup locations available\""
    }
  ],
  "isError": false
}
success
6/24/2025, 11:14:38 PM
848ms
Store:messaging-ai-shop.myshopify.com
No notes added

Input Parameters

{
  "query": "search products"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"I'm sorry, I don't have an answer for that question.\""
    }
  ],
  "isError": false
}
success
6/24/2025, 11:14:36 PM
1395ms
Store:messaging-ai-shop.myshopify.com
User Prompt:
does the brand have a retail store/
No notes added

Input Parameters

{
  "query": "does the brand have a retail store"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"Question: Do you have retail store locations? Answer: Yes, our store is located at 495 Smith Street in Brooklyn, NY. Our opening hours are Monday to Saturday 9 AM to 6 PM EST.\""
    }
  ],
  "isError": false
}
success
6/24/2025, 11:14:35 PM
974ms
Store:messaging-ai-shop.myshopify.com
User Prompt:
do you have subscriptions
No notes added

Input Parameters

{
  "query": "do you have subscriptions"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"I'm sorry, I don't have an answer for that question.\""
    }
  ],
  "isError": false
}
success
6/24/2025, 11:14:16 PM
2269ms
Store:messaging-ai-shop.myshopify.com
User Prompt:
do you have subscriptions
No notes added

Input Parameters

{
  "query": "search products"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"I'm sorry, I don't have an answer for that question.\""
    }
  ],
  "isError": false
}
error
6/24/2025, 11:13:51 PM0
Store:messaging-ai-shop.myshopify.com
User Prompt:
do you have subscriptions
No notes added

Input Parameters

{
  "query": "do you have subscriptions"
}

Output

Invalid URL string.
error
6/24/2025, 11:13:51 PM0
Store:messaging-ai-shop.myshopify.com
User Prompt:
does the brand have a retail store/
No notes added

Input Parameters

{
  "query": "does the brand have a retail store"
}

Output

Invalid URL string.
error
6/24/2025, 11:13:51 PM0
Store:messaging-ai-shop.myshopify.com
User Prompt:
can I order online and pick up in store?
No notes added

Input Parameters

{
  "query": "can I order online and pick up in store"
}

Output

Invalid URL string.
error
6/24/2025, 11:13:07 PM0
Store:your-store.myshopify.com
User Prompt:
can I order online and pick up in store?
No notes added

Input Parameters

{
  "query": "can I order online and pick up in store"
}

Output

Unable to connect to MCP server at your-store.myshopify.com: Error POSTing to endpoint (HTTP 401):
error
6/24/2025, 11:13:06 PM0
Store:your-store.myshopify.com
User Prompt:
does the brand have a retail store/
No notes added

Input Parameters

{
  "query": "does the brand have a retail store"
}

Output

Unable to connect to MCP server at your-store.myshopify.com: Error POSTing to endpoint (HTTP 401):
error
6/24/2025, 11:13:05 PM0
Store:your-store.myshopify.com
User Prompt:
do you have subscriptions
No notes added

Input Parameters

{
  "query": "do you have subscriptions"
}

Output

Unable to connect to MCP server at your-store.myshopify.com: Error POSTing to endpoint (HTTP 401):
success
6/23/2025, 10:55:48 PM
1086ms
Store:nordicweb.myshopify.com
No notes added

Input Parameters

{
  "query": "Can I take these supplements with my existing medications or conditions?"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"I'm sorry, I don't have an answer for that question.\""
    }
  ],
  "isError": false
}
success
6/23/2025, 10:55:27 PM
1055ms
Store:nordicweb.myshopify.com
No notes added

Input Parameters

{
  "query": "Do you offer subscriptions?"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"I'm sorry, I don't have an answer for that question.\""
    }
  ],
  "isError": false
}
success
6/23/2025, 10:55:07 PM
812ms
Store:nordicweb.myshopify.com
No notes added

Input Parameters

{
  "query": "Which omega‑3 supplement is best for me?"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"I'm sorry, I don't have an answer for that question.\""
    }
  ],
  "isError": false
}
success
6/23/2025, 7:41:37 PM
8290ms
Store:dev-ai-playground.myshopify.com
No notes added

Input Parameters

{
  "query": "Who is Robin? "
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"Question: Who is Robin? Answer: The guy from Batman.\""
    }
  ],
  "isError": false
}
success
6/23/2025, 7:39:44 PM
8422ms
Store:dev-ai-playground.myshopify.com
No notes added

Input Parameters

{
  "query": "Who is Robin? "
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"I'm sorry, I don't have an answer for that question.\""
    }
  ],
  "isError": false
}
success
6/18/2025, 10:55:00 PM
4723ms
Store:dev-ai-playground.myshopify.com
No notes added

Input Parameters

{
  "query": "Who is Batman? "
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"Question: Who is Batman? Answer: Batman is the best super hero in history.\""
    }
  ],
  "isError": false
}
success
6/18/2025, 10:51:58 PM
5939ms
Store:dev-ai-playground.myshopify.com
No notes added

Input Parameters

{
  "query": "Who is Batman? "
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"I'm sorry, I don't have an answer for that question.\""
    }
  ],
  "isError": false
}
success
6/18/2025, 2:51:22 AM
2496ms
Store:dev-ai-playground.myshopify.com
No notes added

Input Parameters

{
  "query": "where is your gear made?"
}

Output

View Raw MCP Response
{
  "content": [
    {
      "type": "text",
      "text": "\"Question: where do you create your winter gear? Answer: Our winter gear is created in Colombia.\""
    }
  ],
  "isError": false
}