Optimize agents to minimize latency

This article shares a few key practices, and helps you optimize agents that depend on information from external sources, to reduce wait time for customers during conversations.

Set a maximum time-out delay for global variables set externally. For agents with global variables configured to receive context from external sources, set the optional time-out value so that your agents don't hang for longer.

Let the user know the agent needs a moment. The agent not responding for over a couple seconds can make the user wonder whether the system is working or not. Consider sending a brief message at the start of the applicable topic indicating that the agent is looking up information. This way, the user knows that the operation might take a moment, and things are working.

Minimize and optimize long off-agent operations. Calling external systems to look up information to personalize a request can make for a great experience, but it can also take an uncomfortably long time. If this personalization is necessary, ensure that the call is optimized and that the system being called is scalable.

Use HTTP Request nodes for quick requests. Using a Power Automate flow can make it possible to complete sophisticated operations easily, however, it can involve considerable overhead. If you only need a single off-box lookup call, the HTTP Request node can be a lighter-weight option for such simple operations.

For Power Automate calls, use the Power Automate dashboard to review request durations. Copilot Studio doesn't provide insights into how long off-agent operations such as Power Automate calls take. Therefore, it can be difficult to understand where time is being spent. In Copilot Studio, these off-box calls are synchronous, meaning no further activity happens in the conversation until a Power Automate flow is completed. Consider using the Power Automate dashboard for your flow. This dashboard might give you insights about the success and failure rates of operations, and how long they take.