Maximum performance at minimal cost. Start your server in seconds now!
    Vijona

    Cohere Toolkit Installation Guide for Ubuntu 24.04

    The Cohere Toolkit is a fully developed, open-source repository tailored for creating generative AI applications. It includes user interface components for knowledge-based assistants, supports AI models from Cohere Platform, Bedrock, and SageMaker, and seamlessly connects with enterprise data sources. With the Cohere Toolkit, you can develop and deploy Retrieval Augmented Generation (RAG) applications that are interactive, adaptable, and based on specialized knowledge domains.

    Prerequisites

    Before proceeding with the installation, ensure that you have:

    • Access to an Ubuntu 24.04 instance with a non-root sudo user.
    • Installed Docker and Docker Compose, which are required by Cohere Toolkit.
    • Acquired a Cohere Toolkit API Key.

    Obtain a Cohere Toolkit API Key

    1. Create a Cohere account.
    2. Log in to your Cohere account.
    3. Navigate to API Keys in the sidebar.
    4. Copy the default trial key from the Trial keys section and save it for later use.

    Installing Poetry

    Poetry is a Python dependency management tool that allows you to create an isolated environment for Cohere Toolkit. Follow these steps to install Poetry and Python 3.11 on Ubuntu 24.04.

    Add the Deadsnakes PPA Repository

    First, add the Deadsnakes PPA to access multiple Python versions:

    Copy Code 
    

    $ sudo add-apt-repository ppa:deadsnakes/ppa

    Update Package Index

    Apply the repository changes:

    Copy Code 
    

    $ sudo apt update

    Install Python 3.11

    Install the required Python version:

    Copy Code 
    

    $ sudo apt install python3.11

    Cohere Toolkit requires Python 3.11. If an unsupported version is detected, the build process will automatically use Python 3.11 if available.

    Install Poetry

    Next, install Poetry:

    Copy Code 
    

    $ sudo apt install python3-poetry

    Verify the Installation

    Confirm that Poetry is installed:

    Copy Code 
    

    $ poetry --version

    The expected output is similar to:

    Poetry (version 1.8.2)

    Initialize a New Project

    Create a new pyproject.toml configuration by running:

    Copy Code 
    

    $ poetry init

    During the setup, enter project details as prompted. Ensure to specify ^3.11 as the compatible Python version.

    Example Configuration

    Package name [linuxuser]:
Version [0.1.0]:
Description []:
Author [None, n to skip]: n
License []:
Compatible Python versions [^3.12]: ^3.11
Would you like to define your main dependencies interactively? (yes/no) [yes]
Package to add or search for (leave blank to skip):
Would you like to define your development dependencies interactively? (yes/no) [yes]
Package to add or search for (leave blank to skip):

Do you confirm generation? (yes/no) [yes]

    Activate the Virtual Environment

    Use Python 3.11 to activate a new virtual environment:

    Copy Code 
    

    $ poetry env use 3.11

    The expected output should resemble:

    Creating virtualenv linuxuser-W6smXhkx-py3.11 in /home/linuxuser/.cache/pypoetry/virtualenvs
Using virtualenv: /home/linuxuser/.cache/pypoetry/virtualenvs/linuxuser-W6smXhkx-py3.11

    Installing Cohere Toolkit on Ubuntu 24.04

    The Cohere Toolkit provides two configuration templates—configuration.template.yaml and secrets.template.yaml—that help set up new configurations. Follow these steps to clone the Cohere Toolkit repository and configure it accordingly.

    Step 1: Navigate to the Home Directory

    Ensure you are in your home directory before proceeding:

    Copy Code 
    

    $ cd

    Step 2: Clone the Repository

    Download the Cohere Toolkit repository from GitHub:

    Copy Code 
    

    $ git clone https://github.com/cohere-ai/cohere-toolkit.git

    Step 3: Move into the Project Directory

    Change into the cloned directory:

    Copy Code 
    

    $ cd cohere-toolkit

    Step 4: Configure Secrets

    Copy the secrets.template.yaml file to create a new secrets file:

    Copy Code 
    

    $ cp src/backend/config/secrets.template.yaml src/backend/config/secrets.yaml

    Open the secrets file with a text editor to add your API key:

    Copy Code 
    

    $ nano src/backend/config/secrets.yaml

    Locate the following section and replace the example key with your actual Cohere API key:

    deployments:
 cohere_platform:
  api_key: hMtMgJGNbtymq-example

    Step 5: Set Up the Configuration File

    Copy the configuration.template.yaml file to create a new configuration file:

    Copy Code 
    

    $ cp src/backend/config/configuration.template.yaml src/backend/config/configuration.yaml

     

    Configuring the Cohere Toolkit

    The Cohere Toolkit provides a Makefile that simplifies setup by automating essential tasks such as compilation, initialization, testing, and environment configuration. Follow these steps to properly configure both the backend and frontend components.

    Step 1: Navigate to the Toolkit Directory

    Before executing any setup commands, make sure you are in the correct directory:

    Copy Code 
    

    $ cd ~/cohere-toolkit

    Step 2: Run the Initial Setup

    Start the first-time setup process by executing:

    Copy Code 
    

    $ sudo make first-run

    If an error related to psycopg2 appears, such as:

    make[1]: *** [Makefile:104: setup] Error 1
make[1]: Leaving directory '/home/linuxuser/cohere-toolkit'
make: *** [Makefile:128: first-run] Error 2

    This happens because psycopg2 (2.9.10) does not support PEP 517 builds. The issue originates from the build backend and is unrelated to Poetry.

    Step 3: Update pyproject.toml

    To resolve this, open the pyproject.toml file in a text editor:

    Copy Code 
    

    $ nano pyproject.toml

    Locate the [tool.poetry.dependencies] section and comment out the existing psycopg2 entry, replacing it with psycopg2-binary:

    [tool.poetry.dependencies]
python = "~3.11"
#psycopg2 = "^2.9.9"
psycopg2-binary = "^2.9.9"

    Next, modify the [tool.poetry.group.setup.dependencies] section similarly:

    [tool.poetry.group.setup.dependencies]
python = "~3.11"
#psycopg2 = "^2.9.9"
psycopg2-binary = "^2.9.9"

    Once updated, save the file and close the editor.

    Step 4: Lock the Dependencies

    Run the following command to finalize dependency updates:

    Copy Code 
    

    $ poetry lock

    Step 5: Restart the Setup

    Now, rerun the setup process to continue with the installation:

    Copy Code 
    

    $ sudo make first-run

    Step 6: Finalizing the Configuration

    During the installation process, you will need to confirm several settings:

    • Press Enter to keep the default backend API hostname:
    Enter your public API Hostname or press enter for default [recommended]: http://localhost:8000
    • Select the development environment as the build target:
    Select the build target:
> dev
  prod
    • Press Enter to disable the Python interpreter (it requires additional configuration).
    • Press Enter to disable Tavily Internet Search unless you have a TAVILY_API_KEY.
    Enter the value for TAVILY_API_KEY:
    • Select Cohere Platform as the model deployment option and press Enter to save the configuration.
    Select the model deployments you want to set up:
> [ ] Azure
  [ ] Bedrock
  [X] Cohere Platform
  [ ] SageMaker

    Verifying the Configuration

    If the setup was successful, you should see the following output:

    backend-1    | INFO:     Started server process [9]
backend-1    | INFO:     Waiting for application startup.
backend-1    | INFO:     Application startup complete.

    Cohere Toolkit Endpoints

    The Cohere Toolkit is now running at the following addresses:

    Configuring Network Access

    To allow external access to the Cohere Toolkit backend, open the required port:

    Copy Code 
    

    $ sudo ufw allow 8000/tcp

    If the firewall utility UFW is not installed, install it and allow SSH access using:

    Copy Code 
    

    $ sudo apt install ufw -y && sudo ufw allow ssh

    Finally, reload UFW to apply the new firewall rules:

    Copy Code 
    

    $ sudo ufw reload

    Making API Calls with Cohere Toolkit

    The Cohere Platform model provides two API endpoints for interaction:

    • Non-streaming requests: http://localhost:8000/v1/chat
    • Streaming requests: http://localhost:8000/v1/chat-stream

    Follow the steps below to make an API request to the Cohere backend.

    Step 1: Sending a Request

    Use a prompt, instruction, or query to send a request to the non-streaming endpoint. The following example demonstrates how to request information:

    Copy Code 
    

    $ curl --location 'http://localhost:8000/v1/chat' \
--header 'User-Id: user-id' \
--header 'Content-Type: application/json' \
--data '{
  "message": "What is centron?"
}'

    Conclusion

    Congratulations! You have successfully installed the Cohere Toolkit on Ubuntu 24.04 and executed an API request to the Cohere backend. With this toolkit, you can leverage Cohere’s Command, Embed, and Rerank models. Additionally, you can integrate it with external frameworks such as LangChain and LlamaIndex to build applications powered by Large Language Models (LLMs).

    For further customization and detailed documentation, refer to the official Cohere Toolkit documentation.

    Source: vultr.com

    Create a Free Account

    Register now and get access to our Cloud Services.

    Try now

    Posts you might be interested in:

    Moderne Hosting Services mit Cloud Server, Managed Server und skalierbarem Cloud Hosting für professionelle IT-Infrastrukturen

    Apache Airflow on Ubuntu 24.04 with Nginx and SSL

    Apache, Tutorial

    This guide provides step-by-step instructions for installing and configuring the Cohere Toolkit on Ubuntu 24.04. It includes environment preparation, dependency setup, and key commands to run language models and implement Retrieval-Augmented Generation (RAG) workflows. Ideal for developers building AI applications or integrating large language models into their existing projects.

    Moderne Hosting Services mit Cloud Server, Managed Server und skalierbarem Cloud Hosting für professionelle IT-Infrastrukturen

    Install Ruby on Rails on Debian 12 – Complete Guide

    Linux Basics, Tutorial

    This guide provides step-by-step instructions for installing and configuring the Cohere Toolkit on Ubuntu 24.04. It includes environment preparation, dependency setup, and key commands to run language models and implement Retrieval-Augmented Generation (RAG) workflows. Ideal for developers building AI applications or integrating large language models into their existing projects.

    Moderne Hosting Services mit Cloud Server, Managed Server und skalierbarem Cloud Hosting für professionelle IT-Infrastrukturen

    Install VeraCrypt on Ubuntu 24.04 for Secure Encryption

    Security, Tutorial

    This guide provides step-by-step instructions for installing and configuring the Cohere Toolkit on Ubuntu 24.04. It includes environment preparation, dependency setup, and key commands to run language models and implement Retrieval-Augmented Generation (RAG) workflows. Ideal for developers building AI applications or integrating large language models into their existing projects.