OPEA ChatQnA on GKE

readme.md

OPEA ChatQnA on GKE

Create GKE cluster

• On https://console.cloud.google.com/kubernetes, click on Create on top left, choose Standard, Use Setup Guide, My first cluster, CUSTOMIZE
• Name: opea-demo
• c3-standard-8 (8 vCPU, 30 GB memory)
• Boot disk: 500 GB
• Change the number of nodes to 1
• Click on CREATE

Deploy Helm chart

• export HUGGINGFACEHUB_API_TOKEN="YourOwnToken"
• curl -fsSL https://raw.githubusercontent.com/opea-project/GenAIExamples/main/ChatQnA/kubernetes/intel/cpu/xeon/manifest/chatqna.yaml | sed -e 's/insert-your-huggingface-token-here/$HUGGINGFACEHUB_API_TOKEN/g' | envsubst | kubectl apply -f -
• Get the list of pods:

  arun_gupta@cloudshell:~ (opea-435401)$ kubectl get pods
  NAME                                      READY   STATUS             RESTARTS      AGE
  chatqna-64fd858bd7-qhf78                  1/1     Running            0             23m
  chatqna-chatqna-ui-6f59499f87-95m4b       1/1     Running            0             23m
  chatqna-data-prep-c47d67f89-jqkrs         1/1     Running            0             23m
  chatqna-embedding-usvc-8b4c7f748-qgphb    1/1     Running            0             23m
  chatqna-llm-uservice-5d5b6bb89b-jhfb9     1/1     Running            0             23m
  chatqna-nginx-656bc748d4-qqrbb            0/1     CrashLoopBackOff   9 (27s ago)   23m
  chatqna-redis-vector-db-65cc8d87b-x7g82   1/1     Running            0             23m
  chatqna-reranking-usvc-6bb5c68c9d-g2qgx   1/1     Running            0             23m
  chatqna-retriever-usvc-55f5676745-tc87h   1/1     Running            0             23m
  chatqna-tei-85d9484bf7-wrl7h              1/1     Running            0             23m
  chatqna-teirerank-589dd896d9-kjmxr        1/1     Running            0             23m
  chatqna-tgi-b95d94f55-45z7q               1/1     Running            0             23m  
  

Issue filed for CrashLoopBackoff opea-project/GenAIExamples#814

Verify service

• Do port forwarding:

  kubectl port-forward svc/chatqna 8888:8888
  

• In another console, check:

  arun_gupta@cloudshell:~ (opea-435401)$ curl http://localhost:8888/v1/chatqna \
    -H 'Content-Type: application/json' \
    -d '{"messages": "What is the revenue of Nike in 2023?"}'
  

  to get the response:

  data: b'\n'
  
  data: b'\n'
  
  data: b'N'
  
  data: b'ike'
  
  data: b"'"
  
  data: b's'
  

• Test without RAG:

  arun_gupta@cloudshell:~ (opea-435401)$ curl http://${host_ip}:8888/v1/chatqna -H "Content-Type: application/json" -d '{
     "messages": "What is OPEA?"
   }'
  data: b'\n'
  
  data: b'\n'
  
  data: b'The'
  
  data: b' Oklahoma'
  
  data: b' Public'
  
  data: b' Em'
  
  data: b'ploy'
  
  data: b'ees'
  
  data: b' Association'
  
  data: b' ('
  . . .
  

• Now apply RAG to the sample: