Connectors

​

Overview

• Full ingestion and indexing. Data is ingested and indexed through our advanced pipeline, ensuring optimal performance and speed compared to live search.
• Entitlements Enforcement. Permissions from your upstream data sources are automatically synced and enforced within the Contextual platform. Users without access to a document will not see it in the Datastore or be able to query against it.
• Automatic and Manual Syncing. Data updates are synced every 3 hours, while permissions are refreshed every hour to maintain freshness. You can also trigger a manual sync at any time to capture changes.
• Folder-level selection. Only sync folders that you need.

• Google Drive: Google Docs, Google Slides, PDFs, Word Documents, Plain Text Files, PowerPoint Presentations
• Sharepoint: PDFs, Word Documents, Plain Text Files, PowerPoint Presentations

​

Setup Guide

1. First, navigate to the Datastores tab.
2. Click the Create button in the top right.
3. Fill up a name for your Datastore and select “Third-Party Connection”.
4. Select the data source you want to connect to. If you are using the “On Demand” mode, we recommend that you set up Billing Auto Top-up to ensure seamless ingestion (otherwise, your ingestion may fail halfway if you have insufficient balance). In this tutorial, we will use Google Drive as an example.
5. A pop-up will appear with the end user terms. Review it and click Next when done.
6. Select your Account Type. We recommend the following:
   1. If you are setting up a personal connection and you DO NOT HAVE admin access on your data source, select Read-only for non admins
   2. If you are setting up a connection for your organization and you HAVE admin access on your data source, select Read-only for admins . For a full list of permissions you need for admin authorization, see here.
   
7. Click Open Window to authorize your account.
8. Select which account you want to use to authorize the connection.
9. Authorize the connection by clicking Continue
10. You will see a spinner icon as your permissions get validated.
11. You have an option to sync specific folders or sync all your files. We recommend choosing the former, which will bring you to a folder-level picker.
12. To select a folder, hover over it and check the box. You can click into a folder to see the files inside. You can only select folders, not files. You can also select multiple folders.
13. When you are done with your folder selections, click Confirm in the bottom right corner.
14. You will see a spinner icon that your account is being set up. Click Next when the set up has completed.
15. You will be shown a confirmation page where you can edit your connection if needed (e.g. re-select a new folder). Click Finish when done.
16. You are ready for your connector to start syncing! Click Create for the Datastore to be created and for the connection to start.
17. You will be redirected to your Datastore where you will see the Initializing status. Depending on the size of data you’re syncing, give around 10 minutes for the sync to begin.
18. When the sync has started, you will see a progress bar with information on how many documents are left. The status will change to Processing. Depending on the size of data being synced, it may take up to a few hours. You can use this Datastore as the sync progresses, but you’ll only have access to the documents that’ve been synced.
19. When all documents have synced successfully, you will see a green checkmark for the status. Your Datastore is now fully ready for use!

​

Using your Connector with Entitlements Enforcement

1. The document will not be used to generate a response to the User’s query.
2. In the Datastore page, the document will be shown to the User as Private Document. They will only be able to view documents that they have access to.

​

Auto and Manual Syncing

• Initializing: The sync is setting up for the first time.
• Processing: The sync is underway.
• Synced / Green checkmark: The sync has completed successfully.
• Partially Synced: Some documents failed in the sync. To correct this issue, wait for 10-15 minutes and retrigger a manual sync. The failed documents will be retried.
• Failed: All documents failed in the sync. Retry with a manual sync after waiting for 10-15 minutes. If it still does not work, please contact us at connectors@contextual.ai