Microsoft External Threat Detection

 

In this post we review the steps to create an external security provider to protect the Microsoft copilot studio based Agents.

Most of this post is based on this article.

Before starting, prepare yourself. Following Microsoft best practice, they've made it a super complex process, but in the end it it working, so that's good.

Provide a service

We start by implementing a service following this guide.

In general this service should provide 2 endpoints: /validate and /analyze-tool-execution.

The /validate endpoint is used only to check the service health and integration with Microsoft Authentication. For this post we will not implement Microsoft Authentication validation. Hence a simple implementation of the /validate is:

type ResponseSuccess struct {
    IsSuccessful bool   `json:"isSuccessful"`
    Status       string `json:"status"`
}

type Executor struct {
}

func (e *Executor) Execute(p web.Parser) interface{} {
    log.Info("validate starting")
    auth := p.GetHeader("Authorization")
    log.Info("auth: %v", auth)
    log.Info("validate done")
    return &ResponseSuccess{
       IsSuccessful: true,
       Status:       "OK",
    }
}

The /analyze-tool-execution endpoint is activated in each step before the copilot agent invokes any action, and should approve or reject the action within 1 second (good luck with that). A simple example of implementation is:

type ResponseAllow struct {
    BlockAction bool `json:"blockAction"`
}

type Executor struct {
}

func (e *Executor) Execute(p web.Parser) interface{} {
    log.Info("analyze tool execution starting")

    inputBytes, err := p.GetBodyAsBytes()
    kiterr.RaiseIfError(err)

    auth := p.GetHeader("Authorization")
    log.Info("auth: %v", auth)
    tenantId := kitjwt.GetJwtValue(auth, "tid")
    applicationRegistrationId := kitjwt.GetJwtValue(auth, "appid")

    log.Info("tenantId: %v", tenantId)
    log.Info("applicationRegistrationId: %v", applicationRegistrationId)
    log.Info("action description: %v", string(inputBytes))


    log.Info("analyze tool execution done")
    return &ResponseAllow{
       BlockAction: false,
    }
}

Once the service is implemented, deploy it and provide it with a valid TLS certificate. For the rest of this post we assume it is available in https://external.provider.com.

Register the Domain

Once the service is ready we need to register the domain in entra.microsoft.com.

Notice that as part of the process Microsoft requires you to prove you are the owner of the domain, so you need to add TXT record to the DNS server with a value specified by Microsoft.

App Registration

Create a new AppRegistration in entra.microsoft.com.

Then edit the AppRegistration and under "Expose an API" add the URL https://external.provider.com.

Next, edit the AppRegistration and under Certificates & secrets, select the Federated credentials tab, and add a new credential.

Scenario: Other

Issuer: https://login.microsoftonline.com/55fb1683-57de-46d1-8896-f9f3b07b549f/v2.0

Type: Explicit

The get the "Value" you need to run the following script:

# YOUR TENANT ID HERE

$guid = [Guid]::Parse("55fb1683-57de-46d1-8896-xxxxxxxx")

$base64Url = [Convert]::ToBase64String($guid.ToByteArray()).Replace('+','-').Replace('/','_').TrimEnd('=')

Write-Output $base64Url

# YOUR ENDPOINT ID HERE

$endpoint = "https://external.provider.com/analyze-tool-execution"

$base64Url = [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($endpoint)).Replace('+','-').Replace('/','_').TrimEnd('=')

Write-Output $base64Url

This script outputs 2 values, use them to create the following

/eid1/c/pub/t/FIRST_LINE_OUTPUT/a/m1WPnYRZpEaQKq1Cceg--g/SECOND_LINE_OUTPUT

 

Enable The Threat Detection

In https://admin.powerplatform.microsoft.com enable the threat detection.

Final Note

As promised, the super complex process is now done, and agents related events start streaming into the service which can approve or block them.