Host on JCloud#

Essentially clip_server is a Jina Flow. Any Jina Flow can be hosted on JCloud, hence clip_server can be hosted on JCloud as well. Learn more about JCloud here.

First, you need a Flow YAML file for deploy. A minimum YAML file is as follows:

torch-flow.yml

jtype: Flow
executors:
  - uses: jinahub+docker://CLIPTorchEncoder

onnx-flow.yml

jtype: Flow
executors:
  - uses: jinahub+docker://CLIPOnnxEncoder

Tip

port is unnecessary here as JCloud will assign a new hostname and port for any deployed service.

Executors must start with jinahub+docker:// as it is required by JCloud. We currently provide containerized executors jinahub+docker://CLIPTorchEncoder and jinahub+docker://CLIPOnnxEncoder on Jina Hub. They are automatically synced on the new release of clip_server module.

To enable GPU on JCloud, you need to configure it in the YAML file and use prebuilt docker GPU images. For example,

jtype: Flow
executors:
  - uses: jinahub+docker://CLIPTorchEncoder/latest-gpu
    jcloud:
      resources:
        gpu: shared

Please refer here for more details on using GPU in JCloud. Notice that you must specify a docker image GPU tag for your executor to utilize the GPU. For example latest-gpu. See the ‘Tag’ section in CLIPTorchEncoder and CLIPOnnxEncoder for docker image GPU tags.

To deploy,

PyTorch-backed

jc deploy torch-flow.yml

ONNX-backed

jc deploy onnx-flow.yml

If Flow is successfully deployed you will see:

You can now connect to it via client by setting server as the URL given by JCloud:

from clip_client import Client

c = Client(
    'grpcs://174eb69ba3.wolf.jina.ai'
)  # This is the URL you get from previous step
c.profile()