DataSource Management

A data source is a document repository that agents can reference. You can import data from various sources and automatically process them into your agent's knowledge base.

DataSource Types

THAKI Agent Platform supports the following data source types:

Type	Description	Use Case
File	Direct upload of PDFs, text files, etc.	Documents, manuals, reports
Google Cloud Storage	GCS bucket integration	Large files stored in the cloud
PostgreSQL	PostgreSQL database integration	Querying structured data

DataSource List

View List

Click the Data Sources tab in the left sidebar to see the list of data sources in the current project.

List Information

Column	Description
Name	Data source name
Documents	Number of uploaded file sources
Status	Completed/Error/Processing/Pending/Draft

Create DataSource

1. Click Create Button

Click the Create DataSource button.

2. Enter Information

• Name: A name to identify the data source (e.g., Security Policy Documents)
• Description: Description of the data source's purpose or content (optional)

3. Complete Creation

Click the Create button to create an empty data source.

File Upload

Supported File Formats

Format	Extension	Max Size	Notes
PDF	.pdf	100MB	Most recommended
Text	.txt	50MB	UTF-8 encoding
Markdown	.md	50MB	Markdown documents
Word	.docx	50MB	Microsoft Word
CSV	.csv	50MB	Table data

Upload Method

1. Go to the data source detail page
2. Click the Upload Files button
3. Select files to upload from the file explorer
   • You can select multiple files at once
4. Upload begins

Post-Upload Processing

After file upload, processing automatically begins:

1. Upload: Files are sent to the server
2. Parsing: Document content is extracted
3. Chunking: Content is split into appropriate sizes
4. Embedding: Each chunk is converted to vectors
5. Indexing: Becomes searchable

Processing Time

Processing time varies by file size and count:

• Small documents (< 1MB): seconds to 1 minute
• Medium documents (1-10MB): 1 to 5 minutes
• Large documents (10-100MB): 5 to 30 minutes

Check Processing Status

Status Types

Status	Icon	Description
Completed	✅	Processing complete, ready to use
Error	❌	Processing failed
Processing	🔄	Processing in progress
Pending	⏳	Waiting for processing
Draft	📝	Draft state, before file upload

When Processing Fails

1. Check if the file format is supported
2. Check if the file is not corrupted
3. Check if the file size doesn't exceed limits
4. If problems persist, delete and re-upload the file

File Management

File List

View the list of uploaded files on the data source detail page.

Delete File

1. Click the Delete button for the file to remove
2. Click Confirm in the confirmation popup

Reprocessing

After file deletion, the data source index is automatically updated.

Delete DataSource

Warning

Deleting a data source will delete all uploaded files and indexes. Agents using this data source will no longer be able to reference that knowledge.

How to Delete

1. Go to the data source detail page
2. Click the Delete button
3. Click Confirm in the confirmation popup

Connect to Agent

After creating a data source and uploading files, you can connect it to an agent to enable RAG (Retrieval-Augmented Generation) functionality.

Connect to Existing Agent

1. Go to the Agent tab
2. Click on the agent to connect
3. Click the Edit button
4. Select the data source to connect in the Data Sources section
5. Click the Save button

Connect During New Agent Creation

You can select data sources in the second step of agent creation.

Verify Completion

When the status becomes Completed, the agent can reference that document.

Next Steps

• Agent Management - Create and manage agents
• MCP Installation - Connect external tools to your agent
• Chat Guide - Chat with agents