Understanding the Search Query — Part III

Deployment

Simply put, things always had to be in a production-ready state: if you wrote it, you darn well had to be there to get it running! — Mike Miller

We have containerized the application separately using docker, so as to keep it de-coupled. So, there are two images whose instances will communicate with each other — one for the Crocodile Model and another for the Service Orchestrator. Image for TF-Serving based Crocodile Model:

image: crocodile-model:latest
command: --model_config_file=/models/models.conf
environment:
  MODEL_NAME: crocodile_model
ports:
  - "8501:8501"
networks:
  qus_net:
    ipv4_address: 172.16.0.2

Image for Service Orchestrator:

image: service
ports:
  - "8080:80"
volumes:
  - ./log:/app/log
networks:
  qus_net:
    ipv4_address: 172.16.0.3

Here, we have created two containers and a bridge network on 172.16.0.0/16 subnet, in order to call another service from one container. The Crocodile model will expose REST API running on 8501 port, which will be consumed by the service orchestrator and after applying the logic to find the dimension and normalized query, return the response.

A docker-compose file has been created writing the above logic inside which can be run on any node having docker-compose and dockerd (docker-daemon) running.

Kubernetes

Kubernetes is going to set you free. But it is going to piss you off first.

We can deploy the containers on the kubernetes cluster. These two images can be written as a deployment kind where the number of replicas can be defined. This deployment kind will be the complete pod which will be running on the multiple nodes. These all pods are attached to a persistent volume to store the data which is needed during the spawning of another replica.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: qus-app
spec:
  replicas: 2
  selector:
    matchLabels:
      component: qus-app
  template:
    metadata:
      labels:
        component: qus-app
    spec:
      hostNetwork: true
      containers:
        - name: service
          image: service
          ports:
            - containerPort: 80
          resources:
            limits:
              memory: 2Gi
              cpu: "1"
            requests:
              memory: 512Mi
              cpu: "0.5"
        - name: crocodile-model
          image: crocodile-model
          args: ["--model_config_file=/models/models.conf"]
          env:
            - name: MODEL_NAME
              value: "crocodile_model"
          ports:
            - containerPort: 8501
          resources:
            limits:
              memory: 2Gi
              cpu: "1"
            requests:
              memory: 512Mi
              cpu: "0.5"

Now, We will expose the service using istio ingress virtualservice which manages the routing and also the load-balancing. All the HTTP request will go through the ingress to the service and finally reaches the pods running on each node. We can Configure Liveness and Readiness Probes as well for the health check. Orchestrator service will expose an endpoint “/healthz” for checking the health.

That’ all about the deployment part (this was a bit short)and query understanding as well. Keep watching this space for more updates. !!!