Integrating with Cohere

This tutorial shows how to use the Cape API to make a Cohere chat request. Cape will automatically redact sensitive entities like PII, PCI, and PHI so they are not shared with the model. And then Cape will re-identify the redacted entities before returning the response to the caller.

Get a Cape API Key

This tutorial assumes you have an environment variable CAPE_API_KEY that contains an API Key. You will need to signup for a Cape account, and then you can get an API key here.

Using the Cape API to call Cohere

Here's how to use Python to make a streaming request to the Cape API when targeting Cohere models. Please see the API docs for more details.

import requests
import os

url = "https://api.capeprivacy.com/v1/cohere/chat"

# required to help cohere respond well to redacted data.
prompt_instructions = "Some of the data may be redacted. Please respond with the placeholders as they are, without modification, as they are sufficient for a response. Please do not try to redact information yourself."

payload = {
  "message": f"{prompt_instructions} Bob is a software engineer who works at the company Cape Privacy! What company does Bob work for?",
  "stream": True,
  "format": "redacted",
  "extra_cohere": {
    "temperature": 1
  }
}

headers = {
  "content-type": "application/json",
  "Authorization": f"Bearer {os.getenv('CAPE_API_KEY')}"
}

response = requests.post(url, json=payload, headers=headers)
for line in response.iter_lines():
    print(str(line, 'utf-8'))

Output

data: {"index": 2, "text": "Bob", "is_finished": false, "finish_reason": null}

data: {"index": 3, "text": " is", "is_finished": false, "finish_reason": null}

data: {"index": 4, "text": " a", "is_finished": false, "finish_reason": null}

data: {"index": 5, "text": " software engineer", "is_finished": false, "finish_reason": null}

data: {"index": 6, "text": " who", "is_finished": false, "finish_reason": null}

data: {"index": 7, "text": " works", "is_finished": false, "finish_reason": null}

data: {"index": 8, "text": " at", "is_finished": false, "finish_reason": null}

data: {"index": 9, "text": " the", "is_finished": false, "finish_reason": null}

data: {"index": 10, "text": " company", "is_finished": false, "finish_reason": null}

data: {"index": 11, "text": " Cape Privacy.", "is_finished": false, "finish_reason": null}

data: {"index": 13, "text": "done", "is_finished": true, "finish_reason": null}

data: [DONE]

To help visualize how the data in the response is redacted you can pass the "verbose": True.

Output

data: {"index": 2, "text": "[NAME_GIVEN_1|Bob]", "is_finished": false, "finish_reason": null}

data: {"index": 3, "text": " is", "is_finished": false, "finish_reason": null}

data: {"index": 4, "text": " a", "is_finished": false, "finish_reason": null}

data: {"index": 5, "text": " [OCCUPATION_1|software engineer]", "is_finished": false, "finish_reason": null}

data: {"index": 6, "text": " who", "is_finished": false, "finish_reason": null}

data: {"index": 7, "text": " works", "is_finished": false, "finish_reason": null}

data: {"index": 8, "text": " at", "is_finished": false, "finish_reason": null}

data: {"index": 9, "text": " the", "is_finished": false, "finish_reason": null}

data: {"index": 10, "text": " company", "is_finished": false, "finish_reason": null}

data: {"index": 11, "text": " [ORGANIZATION_1|Cape Privacy].", "is_finished": false, "finish_reason": null}

data: {"index": 12, "text": " [NAME_GIVEN_1|Bob]", "is_finished": false, "finish_reason": null}

data: {"index": 13, "text": " works", "is_finished": false, "finish_reason": null}

data: {"index": 14, "text": " for", "is_finished": false, "finish_reason": null}

data: {"index": 15, "text": " the", "is_finished": false, "finish_reason": null}

data: {"index": 16, "text": " company", "is_finished": false, "finish_reason": null}

data: {"index": 17, "text": " [ORGANIZATION_1|Cape Privacy].", "is_finished": false, "finish_reason": null}

data: {"index": 19, "text": "done", "is_finished": true, "finish_reason": null}

data: [DONE]

If you do not want to stream the data and receive it back in one chunk you can pass "stream": False".

Output

{
  "id": "77cb3b88-ee36-4ed1-b8fe-61b96a3ecee1",
  "response_id": "77cb3b88-ee36-4ed1-b8fe-61b96a3ecee1",
  "generation_id": "1fcc6639-f4dc-4d1e-995e-9cad0880ef80",
  "conversation_id": "70314afc-6954-4fae-af20-9809deffa628",
  "text": "Bob works for the company Cape Privacy.",
  "prompt": null,
  "chatlog": null,
  "preamble": null,
  "token_count": {
    "prompt_tokens": 122,
    "response_tokens": 2,
    "total_tokens": 124
  },
  "meta": {
    "api_version": {
      "version": "1"
    }
  }
}

That's it! To recap, Cape automatically redacted any sensitive entities so they were not sent to Cohere (for example, Bob became [NAME_GIVEN_1]). And then Cape also re-identified the redacted entities from the Cohere response (eg [NAME_GIVEN_1] was replaced back to Bob).