Deploying to Google Cloud Using Google Kubernetes Engine and Cloud Filestore

The following steps explain how to deploy an FME Server Kubernetes deployment consisting of multiple FME Server Core nodes to Google Cloud using Google Kubernetes Engine (GKE) and Cloud Filestore for storage.

Note  For a single-node deployment, use the quickstart script that corresponds to the FME Server version you want to deploy.
  1. Launch the Cluster
    1. Navigate to the Google Kubernetes Engine section of the Google Cloud Platform console.
    2. Select Create Cluster to deploy a new Kubernetes Cluster.
    3. From Cluster templates, select Standard cluster, and set the following parameters:
      • Name: Name of the cluster.
      • Location type: For test purposes, select Zonal.
      • Zone: Select the zone closest to you.
      • Master version: Keep the default value.
      • Node pool: For each FME Server Core you want to run, you need approximately 8GB of memory. For a single-Core deployment, we recommend 2 vCPUS, 8GB RAM, and 2 nodes. For a two-Core deployment, we recommend 4 vCPUs, 16GB RAM, and 3 nodes.
        • Note  More than one Core container requires a minimum of two nodes due to a constraint in the Helm chart that prevents Core containers residing on the same node. To ensure fault tolerance, we recommend three nodes because if a node is lost, two Cores can still run on the remaining two nodes.
    4. Click Create.
  2. Connect to the Cluster

    3. Run the following commands in Cloud Shell (recommended). Alternatively, configure your local machine to run kubectl and helm commands to deploy resources to the cluster.

    1. Click Connect beside the Kubernetes cluster you just launched.
    2. Copy the "Command-line access" command and run in Google Cloud Shell or execute it on your terminal.
    3. To check if you are connected to the cluster, run:

      4. kubectl get nodes

      You should see all nodes in the cluster.

    4. Install the NGINX ingress controller:

      5. kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-<version>/deploy/static/provider/cloud/deploy.yaml

      Replace <version> with the most recent version in the corresponding script in the NGINX Ingress Controller Installation Guide. As well, replace other syntax elements with any changes in the updated script.

  3. Set Up Shared Storage for the FME Server System Share

    4. Using Cloud Filestore, storage for the FME Server System Share is accessible across all nodes. This procedure creates a Cloud Filestore instance to host an NFS server for the Kubernetes cluster to use.

    1. Go to "Filestore" in the Google Cloud Platform console.
    2. Click Create Instance, and set the following:
      • Instance ID: Identifier for the instance.
      • Instance tier: Standard
      • Location: The same region and zone to which you deployed the Kubernetes cluster.
      • Fileshare name: Specify a name.
      • Fileshare capacity: 1TB (default)
    3. Click Create.
    4. To deploy a storage class in the cluster to use the new Filestore instance of the NFS server, Kubernetes provides the nfs-subdir-external-provisioner Helm chart. First, go to the details of your Filestore instance and note the IP address and file share name.
    5. Run:

      6. helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/

      helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \

      --set nfs.server=<ip_address> \

      --set nfs.path=<fileshare_name>

      Replace <ip_address> and <fileshare_name> with the values of your Fileshare. For example:

      helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \

      --set nfs.server=10.135.210.250 \

      --set nfs.path=/fmeserver-storage

      This command creates a storage class in the Kubernetes cluster called "fmeserver-storage" that uses this Filestore NFS server for volumes in Kubernetes.

      Note  For more information, see the NFS Subdirectory External Provisioner Helm Chart documentation.
  4. Deploy FME Server
    1. Add the Safe Software Helm repository:

      2. helm repo add safesoftware https://safesoftware.github.io/helm-charts/

    2. Get the IP address of the Ingress Controller:

      3. kubectl -n ingress-nginx get svc

      Note the "EXTERNAL-IP" of the "ingress-nginx-controller". This is the ingress to the cluster. In a production environment, set up a DNS that points to this IP address and use that as the hostname when deploying FME Server. In this procedure, we access FME Server directly using the IP.

    3. Create a namespace in which to deploy. Run:

      4. kubectl create namespace <namespace_name>

      For example:

      kubectl create namespace fmeserver

    4. Create a values.yaml file that will pass user-supplied parameter values into the Helm chart.

      5. To get the default Helm chart values file, run:

      helm show values <chart>

      For example:

      helm show values safesoftware/fmeserver-2021.2

      To write the values to a file:

      helm show values safesoftware/fmeserver-2021.2 >> values.yaml

      For more information on value files, see the Helm Docs.

      A current list of supported parameters for FME Server can be found on GitHub.

    5. (Optional) Update parameters in values.yaml.

      6. The following are parameters you may want to change in the values.yaml file:

      Parameter

      Example or Possible Value

      Description
      fmeserver.image.tag 2021.2 Set to the FME Server major release version you want to deploy, such as 2021.0 or 2021.2. You must also update the FME Server version when installing the chart; for example, safesoftware/fmeserver-2021.2
      fmeserver.engines[].name “standard-group” The name of the engine group. This can be changed, particularly if creating multiple engine groups.
      fmeserver.engines[].engines 2 Controls the number of engine pods to start.
      fmeserver.engines[].type STANDARD | DYNAMIC Controls the type of engine to start.
      deployment.hostname   Set to the IP address of the ingress controller noted previously.
      deployment.numCores 2 Starts two FME Server Core pods for fault tolerance and load balancing.
      deployment.useHostnameIngress true | false If DNS is set up for the ingress controller, specify the DNS name for deployment.hostname and leave this parameter to true. If not, specify false.
      storage.fmeserver.class nfs-client Uses the Cloud Filestore NFS Server set up previously.
    6. Install FME Server.

      7. helm install <name> <chart> -f values.yaml

      For example:

      helm install fmeserver safesoftware/fmeserver-2021.2 -f values.yaml

      If you make changes to the values.yaml file after FME Server is installed (such as to scale FME Server engines), use the helm upgrade command:

      helm upgrade <name> <chart> -f values.yaml

      For example:

      helm upgrade fmeserver safesoftware/fmeserver-2021.2 -f values.yaml

    7. To access FME Server after it deploys, use your browser to invoke the external IP in your browser.